AI QA Engineer in India, भारत

About Durapid

Durapid Technologies is a Microsoft Data & AI Solutions Partner and a Databricks Partner. We build AI-powered software, data platforms, and enterprise applications for clients in India, the USA, the UK, the Middle East, and Singapore. Our engineering teams work on Generative AI, LLM-based applications, data engineering, and enterprise software for global clients.

Role Overview

We are building a talent pipeline of AI QA Engineers for our clients’ current and upcoming AI and GenAI projects. Selected candidates will be deployed on client engagements. Their job will be to ensure the quality, reliability, and safety of AI/ML and GenAI-powered applications. Depending on the client, the role may be on Durapid’s payroll (full-time) or on a contract basis for the project duration.

The role involves testing traditional software and designing evaluation frameworks for LLMs, RAG pipelines, AI agents, and data workflows. It sits where conventional QA automation meets the newer discipline of AI evaluation.

Please note: This is a talent pipeline requirement. Durapid is building a pool of pre-screened AI QA Engineers for upcoming client projects in India, the USA, the UK, the Middle East, and Singapore. Shortlisted candidates will be engaged when a matching client requirement opens. The final role, location, engagement model, and joining date will depend on the specific client project.

Key Responsibilities

AI & LLM Testing

• Design and execute test strategies for LLM-based applications, chatbots, copilots, RAG systems, and AI agents.
• Build evaluation frameworks that measure accuracy, relevance, groundedness, hallucination rate, toxicity, and bias.
• Create and maintain golden datasets, prompt test suites, and regression benchmarks for AI features.
• Perform adversarial testing, including prompt injection, jailbreak attempts, edge cases, and guardrail validation.
• Validate ML model outputs, track model drift, and verify behaviour after model or prompt version changes.

Test Automation

• Develop and maintain automated test frameworks for UI, API, and backend services.
• Integrate automated tests and AI evals into CI/CD pipelines (Azure DevOps or GitHub Actions).
• Test the APIs of AI services, including latency, token usage, rate limits, and error handling.

Data Quality & Pipeline Testing

• Validate data pipelines, ETL processes, and data transformations, preferably on Databricks or Azure data platforms.
• Ensure that training and inference data is complete, consistent, and of good quality.

Performance & Reliability

• Conduct load and performance testing of AI endpoints and applications.
• Monitor how AI behaves in production and help define quality metrics and SLAs.

Collaboration

• Work closely with client teams, AI engineers, data scientists, and product owners from the requirement stage to release.
• Document test plans, test cases, defect reports, and evaluation results clearly for clients and internal teams.

Required Skills

• 3–5 years of experience in software QA, with at least 1 year testing AI/ML or GenAI applications.
• Strong Python skills for test automation and evaluation scripting.
• Hands-on experience with automation tools such as PyTest, Playwright, Selenium, or Cypress.
• Experience testing APIs with Postman, REST Assured, or Python requests.
• Understanding of LLM concepts: prompts, embeddings, vector databases, RAG, temperature, tokens, and agents.
• Exposure to LLM evaluation tools such as DeepEval, Ragas, Promptfoo, LangSmith, TruLens, or Azure AI Foundry evaluations.
• Working knowledge of SQL and data validation.
• Experience with CI/CD pipelines and Git.
• Strong analytical thinking and clear documentation skills.
• Comfortable working directly with client stakeholders.