GEN AI DEVELOPER CUM TESTER

Job Title GenAI Tester

Role Overview

We are seeking a GenAI Tester who can design build and validate enterprisegrade RetrievalAugmented Generation RAG Agentic AI and Generative AI applications on Azure This hybrid role combines handson development with rigorous testing and evaluation to ensure solutions are reliable grounded and production ready The ideal candidate will have strong technical expertise in Azure AI services Python frameworks and automated testing methodologies

Experience Requirements

 7 years of overall experience in core IT and software development

 3 years of handson experience with Azure Cloud with AI technologies

 2 years of experience in Generative AI RAG LLMs Azure OpenAI LangChain and related technologies in building AI Agents

Key Responsibilities

 Build RAG bots Agentic AI bots and GenAI applications using

o Azure OpenAI Azure AI Search Python Flask Azure Functions HTTP Queue Blob Timer and APIM with secure configuration via environment variables

 Implement scalable data layers using Cosmos DB and Azure Storage Blob Queue Table optimize chunking embeddings and hybrid vector search pipelines

 Lead prompt engineering and tuning across system prompts retrieval prompts and tooluse prompts design guardrails to minimize hallucinations and improve grounding

 Establish automated testing with pytest and RAG evaluation using Ragas faithfulness precision recall answer correctness

 Ensure observability with Application Insights advanced logging and endtoend traceability

 Orchestrate workflows using LangChain and optionally LangGraph for stateful agent flows

 Deliver production SLAs by tuning rate limits scaling AI Search tiers and optimizing costperformance

MustHave Skills

 Strong proficiency in Python Flask FastAPI Azure Functions APIM Azure AI Search Azure OpenAI Cosmos DB and Azure Storage

 Proven experience with prompt engineering and tuning techniques

 Expertise in test automation using pytest and evaluation with Ragas knowledge of grounding techniques hybrid search semantic ranking and context filtering

NicetoHave Skills

 Experience with LangGraph for agentic flows

 Familiarity with CICD pipelines Azure DevOps GitHub Actions

 Exposure to structured output validation and finetuning

Preferred Certifications

 Microsoft Certified Azure AI Engineer AI102

Sent cv on - [HIDDEN TEXT]