Senior Data Engineer - AWS Databricks

Responsibilities:

• Design, develop, and optimize scalable data pipelines using Databricks in AWS such as Glue, S3, Lambda, EMR, Databricks notebooks, workflows and jobs.
• Building data lake in WS Databricks.
• Build and maintain robust ETL/ELT workflows using Python and SQL to handle structured and semi-structured data.
• Develop distributed data processing solutions using Apache Spark or PySpark .
• Partner with data scientists and analysts to provide high-quality, accessible, and well-structured data.
• Ensure data quality, governance, security, and compliance across pipelines and data stores.
• Monitor, troubleshoot, and improve the performance of data systems and pipelines.
• Participate in code reviews and help establish engineering best practices.
• Mentor junior data engineers and support their technical development.

Requirements

• Bachelors or masters degree in computer science, Engineering, or a related field.
• 5+ years of hands-on experience in data engineering