Data Engineer

Key Responsibilities

• Design and implement scalable data architectures on AWS cloud platform
• Develop and maintain data pipelines for batch and real-time processing
• Work extensively with Databricks (jobs, notebooks, clusters, Delta Lake)
• Develop advanced PySpark applications for large-scale data transformation
• Build and optimize ETL/ELT workflows
• Perform SQL query optimization and performance tuning
• Handle and process large datasets in distributed cloud environments
• Implement monitoring, logging, and alerting mechanisms
• Collaborate with cross-functional teams for requirement gathering and solution delivery
• Ensure data quality, governance, and security best practices