Azure Data factory

Job Responsibilities:

• Design, develop, and maintain robust data pipelines and ETL processes using Azure Data Factory (ADF) and Azure Databricks.
• Write proficient SQL queries to perform data mining and analysis, specifically focusing on Teradata as a primary RDBMS.
• Develop complex SQL queries, including the use of window functions and creation of stored procedures.
• Work independently and collaborate closely with key project stakeholders, including architects and business teams, to understand requirements and translate them into technical solutions.
• Gain hands-on experience with Azure Blob Storage and Azure Data Lake Storage (ADLS) for efficient data handling.
• Utilize PySpark / Scala for data transformation and processing within Azure Databricks.
• Apply proficient SQL scripting skills on cloud-based data warehousing tools.
• Utilize version control tools, preferably GitHub, for managing code repositories.
• Work with orchestration tools, preferably Airflow, for scheduling and managing data workflows.
• Conduct data analysis, understand ETL processes, and apply analytical abilities for problem-solving activities.

Required Skills:

• Strong experience in Azure Data Factory (ADF).
• Hands-on experience in Azure Databricks.
• Proficiency in PySpark / Scala.
• Strong knowledge of SQL (ANSI Standard), including writing proficient queries for data mining, window functions, and stored procedures.
• Experience in writing SQL queries on cloud-based data warehousing tools.
• Hands-on experience with Azure Blob storage and ADLS.
• Knowledge of version control tools, preferably GitHub.
• Good communication skills with strong analytical abilities for problem-solving.
• Understanding of Data Analysis and ETL processes.

Good to Have:

• Knowledge of orchestration tools, preferably Airflow.
• Domain experience in Health care.