Software Engineering Lead Analyst

Job Description & Responsibilities:

• Work with business and technical leadership to understand requirements
• Design to the requirements and document the designs
• Ability to write product-grade performant code for data extraction, transformations and loading using Spark, Py-Spark
• Do data modeling as needed for the requirements
• Write performant queries using Teradata SQL, and Spark SQL against Teradata and
• Implementing dev-ops pipelines to deploy code artifacts on to the designated platform/servers like AWS(pref) or
• Implement Hadoop job orchestration using Shell scripting, CA7 Enterprise Scheduler and Airflow
• Troubleshooting the issues, providing effective solutions and jobs monitoring in the production environment
• Participate in sprint planning sessions, refinement/story-grooming sessions, daily scrums, demos and retrospectives

Experience Required:

• Overall 5 - 8 Years
• Experience Desired:
• Experience in Jira and Confluence
• Health care domain knowledge is a plus
• Excellent work experience on Databricks OR Teradata as data warehouse
• Experience in Agile and working knowledge on DevOps tools
• Education and Training Required:

Primary Skills:

• Spark, Py-Spark, Shell scripting, Teradata
• SQLs (using Teradata SQLand Spark SQL) and Stored Procedures
• Git, Jenkins, Artifactory
• CA7 Enterprise Scheduler, Airflow
• AWS data services(S3, EC2, SQS)
• AWS Services- SNS Lambda, ECS, Glue, IAM
• Cloud Watch Monitoring tool
• Databricks (Delta lake, Notebooks, Pipelines, cluster management, Azure/AWS integration)

Good to have:

• Unix / Linux Shell scripting (KSH) and basic administration of Unix servers

Additional Skills:

• Exercises considerable creativity, foresight, and judgment in conceiving, planning, and delivering initiatives.