Job Description
As a Data Engineer, you will design, develop, and maintain data solutions that support the generation, collection, and processing of data. Your day-to-day responsibilities include building data pipelines, ensuring data quality, and executing ETL processes to facilitate data migration and deployment across various systems. You will work closely with cross-functional teams to understand data requirements and deliver efficient, scalable solutions, while also addressing and resolving issues in the data flow and processing stages. Your work will be critical in strengthening the data infrastructure and enabling reliable data for decision-making.
Roles & Responsibilities:
- Build and manage data pipelines using tools like Data Proc, Pub/Sub, and Dataflow
- Work with Kalka Streaming and Looker for real-time data processing and visualization
- Develop and maintain data integration and warehousing solutions
- Write and optimize complex SQL queries, especially in BigQuery (No FLEX)
- Implement cloud-native services like bucket storage, BigQuery, cloud functions, pub/sub, composer, and Kubernetes
- Collaborate with teams to understand data requirements and propose optimal solutions
- Conduct performance tuning of BigQuery SQL scripts
- Apply expertise in shell scripting, Python, and Oracle SQL to support data operations
- Automate CI/CD pipelines using tools like Control-M, GitHub, JIRA, and Confluence
- Operate within an Agile environment and demonstrate adaptability to new technologies
- Work independently as an individual contributor in Shift B
Professional & Technical Skills:
- Must to Have: Proficiency in Google BigQuery, strong SQL skills (No FLEX), Python (No FLEX), cloud-native services, data integration and migration
- Good to Have: GCP certifications, familiarity with tools like Looker, pytest, Pandas, NumPy, and CI/CD pipeline tools
Professional Attributes:
- Good communication and interpersonal skills
- Ability to collaborate across multiple teams and contribute effectively
- Strong analytical and problem-solving skills
- Capacity to work independently or in team environments
- Good team handling skills
Educational Qualification:
- 15 years of full-time education required
Additional Information:
- Candidate should be willing to work in Shift B
- Role requires working as an individual contributor