OTA Update Management Professional

Key Responsibilities:

Cloud Data Platform & Architecture:

• Develop and maintain infrastructure-as-code using Terraform for Kubernetes clusters (AKS) and Databricks environments
• Architect data storage strategies considering processing, accessibility, availability, and cloud cost optimization
• Manage federated access to cloud resources based on organizational roles
• Implement shared access controls to support multi-tenancy and self-service tooling

Data Pipelines & Processing:

• Design, implement, and maintain data pipelines using Azure Data Factory, Amazon Data Pipeline, Apache Spark, Databricks
• Work with stream and batch processing systems like Kafka (Confluent/Open Source), Apache Storm, Spark Streaming, Apache Flink, and Kappa architecture
• Perform data transformation using KStreams App/KSQL/Processor Libraries
• Handle data ingestion and distribution using managed connectors such as Event Hubs, Kafka topics, ADLS2, REST APIs

Platform Automation & Orchestration:

• Set up and manage open-source stacks, including Airflow, Druid, Kafka (Open Source), OpenSearch, and Superset
• Utilize Python scripting for automation and integration tasks
• Build and deploy high-performance APIs using FastAPI
• Implement DevSecOps practices throughout the product lifecycle, including Day 2 operations with monitoring via Datadog

Collaboration & Innovation:

• Research, evaluate, and introduce new technologies to enhance platform capabilities
• Work closely with engineering teams under Agile/Scrum methodology
• Maintain and manage data catalog per topic/domain based on service use cases