Job Description:
- We are seeking a highly experienced Senior Data Engineer with 6 to 12 years of experience to join our team. In this role, you will be responsible for ETL development, data analysis, automation, and validation, ensuring the integrity and efficiency of our data processes.
- Your expertise in AWS, Talend, Python, data analysis, validation and automation will be vital in optimizing our data workflows.
Key Responsibilities:
- Utilize Python & AWS Services, including Glue, Lambda (Python), Glue Data Catalog, Kinesis, SNS, SQS, S3, Redshift, and Aurora, to design, develop, and maintain ETL processes.
- Work with Talend to create efficient ETL workflows and transformations.
- Possess a deep understanding of data and ETL concepts, ensuring data quality, integrity, and performance.
- Conduct data analysis to identify trends, patterns, and anomalies, and provide actionable insights.
- Develop and implement automation solutions to streamline data analysis, execution, and validation processes.
- Proactively identify and address data-related issues, ensuring data accuracy and consistency.
- Collaborate effectively with cross-functional teams and communicate technical solutions clearly.
- Drive the design and optimization of data pipelines for scalability and performance.
- Mentor junior team members and provide guidance on best practices.
- Stay updated on industry trends and emerging technologies in ETL and data engineering.
Requirements:
- Bachelor’s degree in Computer Science, Information Technology, or a related field.
- 6 to 12 years of experience in ETL development, data analysis, automation, and validation.
- Proficiency in AWS services, including Glue, Lambda, Kinesis, SNS, SQS, S3, Redshift, and Aurora.
- Hands-on Python experience.
- Strong exposure in Talend for ETL development.
- In-depth knowledge of SQL and data warehousing concepts.
- Proven experience in data analysis and visualization tools.
- Ability to design and implement automation solutions.
- Proactive attitude with a focus on problem-solving and continuous improvement.
- Excellent communication and interpersonal skills.