LT
Job Description
Key Responsibilities:
Architectural Design & Strategy:
- Design end-to-end Kafka-based event streaming architectures for high-throughput, low-latency environments.
- Define architectural standards, best practices, reusable patterns, schemas, and governance models.
- Enable event-driven architecture (EDA) adoption across platforms and business units.
- Develop data modelling standards for events, Avro/JSON schemas, serialization formats, and schema evolution strategies.
- Implement enterprise-grade security: ACLs, RBAC, encryption, network segmentation, auditing, and compliance.
- Implement disaster recovery frameworks in seamless manner.
Platform Engineering & Implementation
- Architect and oversee deployment of self-managed Kafka clusters.
- Lead capacity planning, cluster sizing, replication, partitioning strategies, high availability, and disaster recovery.
- Implement Kafka Connect and ksqlDB Spark streaming pipelines.
Cross-Functional Leadership
- Collaborate with engineering, DevOps, cloud, data, and application teams to drive Kafka architecture adoption.
- Provide mentoring and technical leadership to engineering teams on Kafka patterns and anti-patterns.
- Review solution architectures, ensure alignment with enterprise standards, and participate in architecture governance boards.
Technical Expertise:
- 6+ years of hands-on experience with Apache Kafka, Confluent Platform, Schema Registry & Kafka Connect.
- Strong knowledge of event-driven architecture, pub/sub messaging, stream processing, and distributed systems.
- Experience with stream processing frameworks: Kafka Streams, Spark Structured Streaming, or ksqlDB.
- Strong understanding of DevOps and automation tools: Kubernetes, Terraform, Helm, Ansible, Jenkins, GitHub Actions.
- Expertise in cloud platforms (AWS/Azure/GCP) for deploying Kafka or managed services.
- Proficiency in languages such as Java, Scala, Python, or Go.
- Strong knowledge on Kubernetes and helm
- Monitoring and observability frameworks like Prometheus and Grafana