Job Description
Inviting applications for the role of Vice President - Quality Engineering Leader
Genpact is leading the next frontier in AI with the transition to Agentic AI systems-autonomous, reasoning-driven agents capable of planning and executing complex workflows. To accelerate this transformation, we are seeking a visionary Head of Quality Engineering to dismantle legacy manual processes and build a world-class, fully automated validation ecosystem from the ground up.
In this role, you will not simply 'test' AI systems-you will define the global standard for how autonomous agents are governed, evaluated, safeguarded, and deployed at enterprise scale.
Responsibilities
1. Architect the Future of AI Quality
Design and implement a scalable, automation-first validation framework purpose-built for Agentic AI lifecycle management.
Develop methods to reliably evaluate non-deterministic model outputs at scale.
2. Drive the Shift to 'Quality as Code'
Lead the organizational and technical transition toward continuous, AI-driven testing pipelines integrated into modern CI/CD environments.
Replace manual QA processes with highthroughput automation systems.
3. Advance Agentic Evaluation Science
Build frameworks to measure:
Agentic reasoning and decision chains
Tool-calling accuracy
Multi-step goal achievement
Workflowlevel consistency and reliability
4. Lead Red Teaming & Safety Engineering
Establish adversarial testing protocols to protect systems against:
Bias and hallucinations
Unsafe or ungrounded decisions
Role drift and autonomy failures
Partner with Security and AI Governance teams to enforce high safety and compliance standards.
5. Build Scalable Evaluation Infrastructure
Implement automated evaluation harnesses using tools such as RAGAS, DeepEval, and synthetic data generators.
Develop custom frameworks for benchmarking LLM and agentic performance in realworld enterprise settings.
Qualifications we seek in you!
Minimum Qualifications
The AI Polymath
Deep understanding of the difference between testing deterministic software and evaluating stochastic AI systems.
Ability to translate qualitative heuristics ('vibe checks') into quantitative, automated evaluation metrics.
Automation Evangelist
Extensive experience replacing manual QA processes with fully automated, CI/CD-integrated validation suites.
Proven success modernizing large QA functions into engineering-driven Quality organizations.
Preferred Qualifications/ Skills
Technical Depth
Strong proficiency in Python.
Hands-on experience with LLM orchestration frameworks such as LangChain or LlamaIndex.
Familiarity with observability and evaluation systems:
LangSmith
Arize Phoenix
Other agent telemetry platforms
Leadership & Transformation Experience
Demonstrated ability to lead teams through high-impact transformation.
Skilled at upskilling traditional testers into AI-focused Quality Engineers or hiring specialized talent.
Able to influence senior stakeholders and advocate for a modern quality philosophy.