GCP Services (DataProc, Big Query, Composer) , Pyspark

Key Responsibilities

• Work on GCP-based Big Data deployments (Batch and Real-time) using components such as:
• BigQuery, Cloud Composer (Airflow), Google Cloud Storage, Data Fusion, Dataflow, Dataproc, etc.
• Develop efficient data pipelines using Python and PySpark.
• Manage and operate solutions on Linux-based environments.
• Design and implement CI/CD pipelines for Big Data release deployments.
• Build log monitoring and alerting mechanisms for data pipelines and cloud workloads.