Data Engineer

• Build data systems and pipelines
• Standardize the data assets which maximize data reusability
• Evaluate business needs and objectives
• Implementing data pipeline to ingest and cleanse raw data from various source systems including SaaS, Cloud based and On-prep databases
• Conduct complex data analysis and report on results 
• Combine raw information from different sources
• Explore ways to enhance data quality and reliability
• Implementing functional requirements including incremental loads, data snapshots and identity resolution
• Optimizing data pipelines to improve performance and cost, while ensuring a high quality of data within the data lake:
• Monitoring services and jobs for cost and performance, ensuring continual operations of data pipelines, and fixing of defects.
• Constantly looking for opportunities to optimize data pipelines to improve performance

Must Have:

• Must have coding skills in Spark/Pyspark, Python and SQL
• Must have Knowledge of AWS tools, Glue, Athena ,step functions and S3