Lead Data Engineer

We are seeking a seasoned Lead Data Engineer to design, develop, and implement scalable data pipelines. This role is perfect for a professional with 10+ years of experience in data engineering and architecture, with a strong focus on big data technologies. You'll play a critical role in strengthening our data infrastructure, mentoring junior engineers, and ensuring our data solutions are reliable, high-quality, and performant.

Key Responsibilities

• Data Pipeline Leadership: Lead the design, development, and implementation of scalable data pipelines and ELT processes using tools like Databricks, DLT, dbt, and Airflow. You'll optimize and maintain these pipelines to ensure data quality and reliability.
• Best Practices & Mentorship: Develop and enforce data engineering best practices, including coding standards, testing, and documentation. You will mentor junior data engineers, providing technical leadership and fostering a culture of continuous learning.
• Collaboration & Optimization: Work closely with stakeholders to understand data requirements and deliver high-quality data solutions. You will also proactively monitor and troubleshoot data pipeline issues to ensure minimal disruption to business operations.
• Continuous Improvement: Stay up to date with the latest industry trends and technologies to recommend improvements to our data engineering practices.

Required Qualifications

• Experience: A minimum of 10 years of experience in data engineering and/or architecture, with extensive production experience with Databricks and Apache Spark.
• Technical Skills:
• Expert knowledge of SQL.
• Proficiency in programming languages such as Python, Scala, or Java.
• Strong understanding of data warehousing concepts.
• Familiarity with orchestration and ELT tools like Airflow and dbt.
• Experience with cloud platforms like Azure, AWS, and Google Cloud.
• Knowledge of data governance and security best practices.
• Personal Attributes: Excellent problem-solving, communication, and leadership skills. You must be able to work in a fast-paced, collaborative environment and effectively mentor and guide team members.
• Certifications: A certification in Databricks, Azure, Google Cloud, or related technologies is a plus.