PySpark Developer

• Design, implement, and optimize ETL pipelines and data processing workflows using PySpark
• Work on distributed computing frameworks for large-scale data processing
• Collaborate with Databricks and other cloud platforms for data storage and transformation
• Perform data analysis, validation, and integration from multiple sources
• Troubleshoot and resolve data pipeline and processing issues
• Maintain proper documentation of data workflows, pipelines, and processes
• Ensure best practices for performance, scalability, and data governance

Key Performance Indicators

• Timely delivery of data pipelines and ETL workflows
• Accuracy, consistency, and integrity of processed data
• Performance and scalability of data processing solutions
• Effective collaboration with cross-functional teams