Test Automation

Roles and Responsibilities

• Hands-on experience with test automation frameworks (Selenium, Cypress, JUnit, TestNG) within software or AI-focused teams.
• Hands-on experience with test automation frameworks and tools (Selenium, Cypress, JUnit, TestNG, Postman).
• Experience or and ability to automate testing of APIs and web interfaces.
• Familiarity with AI/ML systems and relevant testing methodologies, particularly familiarity with testing AI models, NLP systems, embeddings, and vector-based retrieval systems.
• Familiarity with common AI model evaluation around NLP, Retrieval-Augmented Generation (RAG), or similar model architectures.
• Experience conducting root cause analyses, CAPA metrics and methods.
• Experience with performance testing and load testing tools to validate scalability and reliability of AI services.
• Excellent analytical processes, and statistical analysis, troubleshooting, and root-cause analysis skills.
• Strong communication and collaboration
• Experience with performance and load testing tools (e.g., JMeter, Gatling) to ensure scalability and reliability of AI-driven applications.
• Knowledge of AI model evaluation metrics (precision, recall, F1-score, accuracy, BLEU, ROUGE, etc.) and techniques for validation.
• Familiarity with vector databases and embedding-based retrieval technologies (e.g., Pinecone, Weaviate, Chroma).
• Strong analytical, troubleshooting, and root-cause analysis skills with the ability to systematically address issues and propose effective corrective actions.
• Excellent communication skills and the ability to collaborate effectively across multidisciplinary teams (engineering, product management, data science).
• Experience working in agile software development environments (Scrum, Kanban)

Mandatory skills*

• Hands-on experience with test automation frameworks (Selenium, Cypress, JUnit, TestNG) within software or AI-focused teams.
• Experience or and ability to automate testing of APIs and web interfaces.
• Familiarity with AI/ML systems and relevant testing methodologies, particularly familiarity with testing AI models, NLP systems, embeddings, and vector-based retrieval systems.
• Familiarity with common AI model evaluation around NLP, Retrieval-Augmented Generation (RAG), or similar model architectures.
• Experience with performance testing and load testing tools to validate scalability and reliability of AI services.
• Desired skills*Knowledge of AI model evaluation metrics (precision, recall, F1-score, accuracy, BLEU, ROUGE, etc.) and techniques for validation.