Design and maintain end-to-end business scenarios covering agent routing, tool and function calls, orchestration, handoffs, and multi-turn conversations.
Evaluate agent responses for correctness, relevance, groundedness, consistency, tone, and completeness using human and automated methods.
Build and maintain evaluation datasets, golden sets, edge cases, adversarial prompts, and automated regression suites.
Identify cross-agent, orchestration, context, integration, tool-call, looping, and runaway-behavior issues.
Test for hallucinations, prompt injection, jailbreaks, data leakage, and unsafe or out-of-scope responses.
Detect and quantify regressions caused by model, prompt, retrieval, or knowledge changes.
Triage issues with Engineering and Product, determine root causes, re-test fixes, and validate the end-user experience.
Document test strategy, evaluation results, and quality trends for global stakeholders.
Requirements
Bachelor's degree in software engineering, computer science, or a related field.
3-5 years of QA experience, including at least 1-2 years testing LLM-based applications, AI agents, or agentic workflows.
Hands-on experience evaluating non-deterministic outputs by defining pass/fail criteria, rubrics, and acceptance thresholds.
Experience building or maintaining AI evaluation or regression suites, including LLM-as-judge methods, golden datasets, or scoring rubrics.
Understanding of tool and function calling, multi-agent orchestration, planning, memory and context, and RAG.
Experience with test automation tools such as Selenium, Playwright, or TestComplete, and with API testing.
Benefits
Health and well-being programs supporting work-life balance.
Collaborative culture with growth and advancement opportunities.
Virtual-first work arrangement with primary work performed from home.
Occasional travel to a physical office may be required for collaboration, design, strategy, and alignment.
Compensation packages are based on experience and desired skill set.
Responsibilities - Work with product owners and developers to ensure requirements are effectively tested. - Coordinate with QA leads on regression coverage, risk assessment, and communication. - Prepare and execute test plans and test scripts. - Identify edge situations, user interface flaws, and potential vulnerabilities. ...
Senior QA Engineer At Sangoma, we're looking for a Senior QA Engineer who is passionate about delivering high-quality software and exceptional customer experiences. In this role, you'll play a critical part in ensuring the quality, reliability, scalability, security, and performance of our cloud communications platforms ...
... category-defining products, backed by an omni-channel network of manufacturing, retail, e-commerce and distribution. About the Role We are seeking a hardcore Senior QA Automation Engineer to drive quality and reliability across our entire technology stack. Your primary mission is to build, scale, and maintain robust test ...
... functional, integration, regression, exploratory and UAT support. - Own defect triage, regression scope and release quality evidence. - Partner with platform engineers and records/compliance stakeholders on test scenarios. About you Required experience and skills - 4-6 years in QA/QE with lead-level ownership. - Hands-on ...
... international team of professionals, working with the latest networking technologies and concepts, access to trainings. Responsibilities Work closely with functional engineering teams, test teams and customer solution engineering teams to design and develop feature functionality for data center networks, network security and routing ...
... responsible for designing and executing advanced testing strategies, building scalable automation frameworks, and ensuring software reliability across platforms. The Senior SQA Engineer collaborates closely with development teams, contributes to continuous integration efforts, and drives quality improvements through deep expertise ...
... invests in people and new advanced technologies to unlock the power of data. We have an amazing team of 25,200 people in 32 countries. Job Description As you are a QA Engineer, it is important to focus on quality control and assurance to give business confidence that the product meets the requirements. You will report to the ...
... BE, Electronics and Communication Engineering, Bachelor's of Technology, Computer Science and Engineering - Minimum of 2+ years of experience in an automation QA role within an agile setting, alongside a total of 3+ years of experience in software testing or related fields. - Passion for QA process, engineering, & tooling ...
We're looking for Senior QA Engineers with deep hands-on experience in Selenium, Appium - mobile testing , and GitLab, coupled with strong knowledge of UK products and regulatory environments (e.g., FCA/ PRA/PSD2/UK GDPR where applicable). You'll own end-to-end quality across complex, distributed systems and low-code workflows, ...
Dodge Construction Network (Dodge) is looking for a Senior QA Engineer to own quality across a product area of our platform, from the user-facing web application down to the data it produces. This is a hands-on senior role: you’ll set the test strategy for your area, write and maintain automation, and do the structured ...