Manager-Data/AI Engineering

Overall Purpose

• This position will manage a team of 10+ data engineers and interact on a consistent basis with other developers, architects, data product owners and source systems. This position requires multifaceted candidates who have experience in data engineering, data analysis, visualization, good hands-on experience PySpark Databricks, and in Azure Cloud Platform Services

Key Roles and Responsibilities

• Lead and manage a team of Data engineers to Develop, understand, and enhance code in traditional data warehouse environments, data lake, and cloud environments like Snowflake, Azure, Databricks
• Build end-to-end data business intelligence solutions. This includes data extraction, ETL processes applied on data to derive useful business insights, and best representing this data through dashboards.
• Write complex SQL queries used to transform data using Python/Unix shell scripting
• Understand business requirements and create visual reports and dashboards using Power BI or Tableau.
• Upskill to different technologies, understand existing products and programs in place
• Work with other development and operations teams.
• Flexible with shifts and occasional weekend support.

Key Competencies

• Full life-cycle experience on enterprise software development projects.
• Experience in relational databases/ data marts/data warehouses and complex SQL programming.
• Extensive experience in ETL, shell or python scripting, data modelling, analysis, and preparation
• Experience in Unix/Linux system, files systems, shell scripting.
• Extensive knowledge in Python, PySpark, Databricks, Azure Cloud Platform Services
• Good to have experience in BI Reporting tools Power BI or Tableau
• Good problem-solving and analytical skills used to resolve technical problems.
• Must possess a good understanding of business requirements and IT strategies.
• Experience in presentation design, development, delivery, and good communication skills to present analytical results and recommendations for action-oriented data driven decisions and associated operational and financial impacts.
• Experience in managing a team of 10 or more, is involved in appraisal and rating process

Required/Desired Skills

• Cloud Platforms - Azure, Databricks, Deltalake (Required 5-6 years)
• SQL Programming, Python/PySpark ETL (Required 8+ Years)
• Unix/Linux shell scripting (Required 6-8 years)
• RDBMS and Data Warehousing (Required 8+ Years)
• Iceberg enablement (Desired 2-3 years)
• Managed a team for 2+ years
• Snowflake Power BI / Tableau (Good to have)