TE

Lakehouse Architect

Singapore ₹8K-10K LPA Posted 30 Mar 2026

FULL TIME

Kafka

Spark And Scala

Design, develop, and optimize end‑to‑end data pipelines within a Lakehouse Architecture, implementing Bronze/Silver/Gold (Medallion) layers for scalable batch and streaming workloads.
Design, develop, and optimize Spark (PySpark/Scala) jobs for ETL/ELT on large datasets across Lakehouse storage systems.
Build and operate Kafka producers/consumers; implement exactly‑once/at‑least‑once semantics where required.
Implement real‑time stream processing with Spark Structured Streaming / Kafka Streams / Flink (if available) and integrate with Lakehouse transactional layers.
Work with Hadoop ecosystem (HDFS, YARN, Hive/HCatalog, Oozie/Airflow, HBase/NoSQL) and modern Lakehouse table formats.
Manage Lakehouse schema evolution, data versioning, ACID transactions, and time‑travel using Delta Lake / Apache Iceberg / Apache Hudi.