Senior Data Engineer

Job Overview

• Having 3-5 years of experience in implementing Big Data solution in Cloud environments.
• Hands-on in implenting Big data solution using Airflow
• DAG,Spark,Python,Hive,Trino,S3 Bucket,etc
• Having experience Cloud Data platform like SnowFlake , Databricks is a plus.
• Knowledge of Financial domain-Credit Card industry is a plus.

Responsibilites:

• Creating an ELT Data pipelines using python script and Airflow DAG.
• Optimizing the Data pipelines by configuring the Spark system resources
• Debug an existing Microsoft BI ETLs and understand business logic and convert in to big data pipeline using Airflow DAG.
• Able to interact with business partners to understand the requirements and provide technical solutions.
• Proficient with Agile Methodology and active team participation is required.
• Need to have overlap with Onsite team till 10-11 am Pacific time , for attending scrum ceremonies or to have onsite interaction