Lead Data Engineer

Job Description:
• We are seeking a highly experienced Lead Data Engineer with over 9 years of expertise in data engineering.
• As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.
• Your proficiency in Python, PySpark, Azure, Databricks/Snowflake, SQL, and leadership skills will be crucial for success.
Key Responsibilities:
• Lead and manage data engineering projects, overseeing the design and development of data pipelines.
• Architect scalable and efficient data solutions using Python and PySpark.
• Utilize cloud platforms such as Azure to build and maintain data infrastructure.
• Work with Databricks or Snowflake for advanced data processing and warehousing.
• Perform data modeling and design complex data structures.
• Write and optimize SQL queries and scripts for data transformation.
• Collaborate with cross-functional teams to define data strategies and deliver solutions.
• Proficiency in Azure ADF for ETL processes.
• Previous experience with ETL tools like Talend or Informatica is preferred.
• Troubleshoot and optimize data pipelines for maximum performance and scalability.
• Ensure data quality, accuracy, and reliability throughout the data lifecycle.
• Stay updated on industry trends and emerging technologies in data engineering.
Requirements:
• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Expertise in Python, PySpark, Azure, Databricks/Snowflake, and SQL.
• Strong data modeling skills and deep understanding of data warehousing.
• Proficiency in Azure ADF for ETL processes is preferred.
• Previous experience with ETL tools like Talend or Informatica is a plus.
• Proven leadership and project management skills.
• Excellent problem-solving, analytical, and communication skills.