Vacancy description

Quality Engineer - Test Automation and AI Assurance in India, भारत

Arohana Tech Solutions Private Limited

  • RemoteWork from anywhere
  • ContractFixed-term or freelance

Rs 14 - 20 lakhs p.a.

India, भारत

Quality Engineer, Test Automation and AI Assurance
Location: Remote
Company builds AI-powered insurance and financial advisory platforms for insurers and banks. Our SaaS microservices and no-code/low-code technology let clients design, launch and run products faster, cutting time-to-market by up to 90% while reducing budget overruns and delivery risk.
We are a profitable B2B enterprise software company with presence in Singapore, Malaysia, Thailand and the Middle East, and we are scaling to the next level.
THE WORK CULTURE
AI is how we work. Every role at Company uses AI agents and tools every day to research, write, build, test and deliver. We hire people who use AI to multiply their output, and who still own the judgement and the result.
We are a lean, high-output team. Everyone owns outcomes, not tasks. We hold a high bar, move fast and do not promise what we cannot deliver.
We are fully remote. Work from wherever you do your best work. We judge output, not hours online, and we expect clear, proactive written communication.
We hire for drive, hunger and real domain depth. Our hiring is selective, and we reward our people generously and fairly, including guaranteed bonuses.
WHAT YOU WILL DO
Two jobs in one. Conventional test automation for the platform, and evaluation for the parts that are not deterministic.
Our products combine calculation, where the answer is either right or wrong, with agents and LLMs, where the answer is a distribution. The second half of this discipline barely exists here yet. You would build it, and the standard that comes with it.
RESPONSIBILITIES
Test Automation
• Own the automation suite across UI and services, running in CI and trusted by the team.
• Build API and contract testing, test data management and environments that stay usable.
• Keep flake under control, and treat a flaky test as a defect rather than a nuisance.
• Build regression testing against actuarial benchmark outputs, where the number must be exact.
AI Assurance
• Build evaluation suites for LLM and agent behaviour: golden sets from real usage, scenario and persona generation, and regression on every prompt or model change.
• Build grader rubrics backed by deterministic detectors, and be honest about where a model-based judge is unreliable.
• Test grounding and citation behaviour for knowledge-backed answers, and catch hallucination before a client does.
• Run adversarial testing: jailbreaks, prompt injection, data leakage and unsafe tool use.
Release and Monitoring
• Gate releases on measurable thresholds for quality, cost and latency, and be willing to block one.
• Run canary or A/B before a model or prompt switch, and compare on evidence.
• Define post-deployment monitoring: drift, guardrail breaches and user feedback, with the action that follows a breach.
REQUIREMENTS
Who You Are
• You want to own quality as a discipline, not run a test queue.
• Comfortable saying a release is not ready, and able to show why.
• Curious about how models fail, and systematic about proving it.
• AI-native: you already use AI tools to work faster and better, and you want to push that further.
• Self-managed: you do your best work remotely, communicate proactively in writing and need no chasing.
What You Bring
• 5+ years in test automation or quality engineering, owning a suite the team actually relied on.
• Playwright, Cypress or similar for UI, and Pytest or equivalent for services, running in CI with flake under control.
• API and contract testing, test data management, and environment strategy on a real product.
• Evaluation suites for LLM or agent systems: golden sets, scenario generation, and regression on prompt or model change.
• Grader rubrics backed by deterministic detectors, and a view on where the judge breaks down.
• Hallucination, grounding and citation checks for RAG or knowledge-backed answers.
• Red teaming and adversarial testing: jailbreaks, prompt injection, data leakage, unsafe tool use.
• Release gating on measurable thresholds, with canary or A/B before a model switch.
• Post-deployment monitoring you defined: drift, guardrail breaches, and what happened when one tripped.
• Bring an evaluation suite to the interview: how you chose the thresholds, and a release you blocked.
WHAT GOOD LOOKS LIKE IN THE FIRST SIX MONTHS
• Nobody merges a prompt or model change without a regression result.
• The suite is fast and trusted, and a red build means something.
• We know how the system behaves in production, from measurement rather than from reports.

Similar vacancies

More like this role

  • Contract
  • QA
... https://jobeax.com/link/ev5LGayRhI4DJN88 Description :The Senior Quality and Test Automation Engineer leads quality strategy and automation for complex features, services, and end-to-end workflows. The role remains hands-on while shaping testability, framework design, nonfunctional coverage, release risk, and engineering practices across ...

India, भारत

  • Full-time
  • QA
... PET-CT systems across system and subsystem levels - Execute manual and automated testing for product verification, validation, ensuring compliance with functional and nonfunctional requirements - Design, develop, and maintain automation frameworks using Python for scalable testing What you need - Test automation - Selenium ...

Bangalore, भारत

Ultraviolette Carrers

Test Automation Engineer

  • Full-time
  • QA
... automotive applications - Responsible for understanding the system and writing Test Specifications from the requirements, Test automation - Responsible for understanding the existing HIL and test ing systems and fix them whenever the problem occurs What you need - Test automation - Python - Embedded C Job details - Location: ...

Bangalore, भारत

  • Full-time
  • QA
... tools to facilitate early detection of defects and quality issues - Root cause analysis for identified defects, working closely with developers to provide detailed information and support defect resolution What you need - Test automation - Test cases - Root cause analysis Job details - Location: Pune - Education: Any graduate ...

Pune, भारत

  • Full-time
  • QA
... implementation of automation and manual test strategies for various software and systems utilizing best practices and standards What you need - Selenium - Unit testing - Test automation - Java - Cucumber - Appium Good to have - Python - AWS - CI/CD - GCP - Data modelling - NoSQL Job details - Location: Gurugram - Experience: ...

Gurgaon, भारत

  • Full-time
  • QA
QA and testing role at Quest Global in Chennai. What you'll do - API Testing & Automation - Design, develop, and execute API test cases based on business and technical - Build, enhance, and maintain API automation frameworks using Karate Framework What you need - Test automation - API testing - Bug tracking - Test cases ...

Chennai, भारत

  • Hybrid
  • Full-time
  • QA
... chain. Important: This is a hands-on manufacturing engineering role focused on turbine and test-system verification. It is not a software test-automation role, and advanced PLC programming is not the primary responsibility. Responsibilities - Develop and maintain VTS test sequences by converting product, test, safety, and ...

Chennai, भारत

  • Hybrid
  • Full-time
  • QA
... chain. Important: This is a hands-on manufacturing engineering role focused on turbine and test-system verification. It is not a software test-automation role, and advanced PLC programming is not the primary responsibility. Responsibilities - Develop and maintain VTS test sequences by converting product, test, safety, and ...

Chennai, भारत

  • Full-time
... and Engineering leads. Automation Framework Development - Design, implement, and maintain scalable automation frameworks - Integrate tests into CI/CD pipelines and ensure fast feedback - Work closely with DevOps to optimize pipeline execution, parallelization of test execution, and environment provisioning. Quality Metrics ...

Hyderabad, भारत

  • Full-time
  • QA
... teams, developers, and business analysts to ensure test coverage and quality - Analyze requirements, define test strategy, and identify automation opportunities - Maintain CI/CD pipeline integration for automated test execution - Design framework as per standards. - Document processes, create test reports, and present results ...

Ernakulam, भारत