Data Engineer

We are seeking a skilled Data Engineer to design, develop, and maintain robust data solutions that enable efficient data generation, collection, and processing. The role involves creating scalable data pipelines, ensuring data quality, and implementing ETL processes to support seamless data migration and deployment across systems. You will collaborate with cross-functional teams to understand business data needs and troubleshoot issues to maintain smooth data flow and processing.

Roles and Responsibilities:

• Design, develop, and maintain data pipelines and ETL processes
• Ensure data quality and reliability across various data sources and systems
• Collaborate with cross-functional teams to understand data requirements and deliver solutions
• Analyze and model client, market, and key performance data to generate business insights
• Work on data integration and migration projects ensuring efficient data flow
• Develop and optimize BigQuery SQL scripts for performance tuning
• Utilize cloud-native services such as Google Cloud Storage, BigQuery, Cloud Functions, Pub/Sub, Composer, and Kubernetes
• Automate CI/CD pipelines using tools like Azure DevOps, GitHub, JIRA, and Confluence
• Write clean, efficient Python code using libraries such as Pandas and NumPy
• Troubleshoot and resolve data pipeline issues independently or as part of a team
• Work effectively in an Agile development environment
• Mentor and guide junior team members when needed

Professional and Technical Skills:

• Expertise in SQL and Python programming (No Flex)
• Strong hands-on experience with Google BigQuery and BigQuery SQL performance tuning
• Proficiency with cloud-native data platform services on GCP (Google Cloud Platform)
• Experience with Shell scripting, Oracle, and various data structures (dictionary, array, list, tree)
• Knowledge of automated CI/CD pipelines and tools (Control-M, GitHub, Azure DevOps)
• Familiarity with Agile methodologies and collaborative teamwork
• GCP certification preferred

Professional Attributes:

• Good communication skills
• Strong analytical and problem-solving abilities
• Ability to work independently and in a team environment
• Capability to manage and handle teams