EP

Lead Data Engineer

Epsilon Data Management
Mumbai8-13 LPA Posted 3 Jun 2025
FULL TIME
Sql
DataFlow
BigQuery
Aws
Python

Job Description

Key Responsibilities

  • Design & Implementation: Design and implement robust, scalable data engineering pipelines within GCP, specifically tailored for web analytics.
  • Data Integration: Integrate a variety of marketing & enterprise data sources, including AWS Aurora, SAP, and Salesforce, ensuring seamless data flow and accessibility.
  • End-to-End Project Execution: Execute at least 1-2 end-to-end data engineering projects, from conceptualization to deployment, demonstrating strong project management and technical prowess.
  • Scripting & Optimization: Employ advanced SQL, Python, and Bash scripting to optimize data processing, analysis, and automation tasks.
  • Cross-functional Collaboration: Collaborate with cross-functional teams to identify data needs, design effective solutions, and enhance data-driven decision-making capabilities.
  • Data Integrity & Security: Maintain and ensure the integrity and reliability of data pipelines, implementing best practices in data security and compliance.

Qualifications

  • Education: Bachelor's/Master's in computer/allied STEM engineering fields. (BS in Mathematics, Statistics, Computer Science, Engineering, Data Science, Analytics, or related field preferred).
  • Experience: Relevant 4-6 years of proven experience as a Data Engineer, with a specific focus on web analytics data engineering pipelines.
  • Cloud Proficiency (GCP & AWS): Hands-on experience with GCP & AWS, including their respective data processing and analytics services.
  • GCP Services: Pub/Sub, Dataflow, BigQuery, Cloud Functions, Cloud Run, Bigtable, etc.
  • AWS Services: Glue, Redshift, S3, SQS, Aurora, etc.
  • API Integration: Strong proficiency in integrating data from diverse sources using APIs.
  • Scripting Expertise: Advanced knowledge of SQL, Python, and Bash scripting is essential.
  • Project Execution: Demonstrated ability to execute end-to-end projects, showcasing effective project management and technical skills.
  • Problem-Solving: Excellent problem-solving abilities, with a keen eye for detail and a commitment to high-quality outcomes.
  • Communication: Strong communication skills, with the ability to convey complex technical concepts in a clear, concise manner.