Vacancy description

Staff Software Development Test Engineer in Bangalore, भारत

Tekion

Bangalore, भारत

About Tekion:

Positively disrupting an industry that has not seen any innovation in over 50 years, Tekion has challenged the paradigm with the first and fastest cloud-native automotive platform that includes the revolutionary Automotive Retail Cloud (ARC) for retailers, Automotive Enterprise Cloud (AEC) for manufacturers and other large automotive enterprises and Automotive Partner Cloud (APC) for technology and industry partners. Tekion connects the entire spectrum of the automotive retail ecosystem through one seamless platform. The transformative platform uses cutting-edge technology, big data, machine learning, and AI to seamlessly bring together OEMs, retailers/dealers and consumers. With its highly configurable integration and greater customer engagement capabilities, Tekion is enabling the best automotive retail experiences ever. Tekion employs close to 3,000 people across North America, Asia and Europe.

Roles & Responsibilities  

Generative AI & LLM Evaluation  

  • Build automated testing suites to detect hallucinations, bias, toxicity, and prompt injection vulnerabilities across LLM-powered products 

  • Implement automated evaluations for RAG systems measuring context relevance, groundedness, and answer faithfulness using frameworks like RAGAS or DeepEval 

  • Design test beds to validate multi-agent workflows — tool-calling accuracy, multi-step reasoning, memory, and autonomous decision loops 

  • Build and run automated conversation simulations — scripted and synthetic user journeys — to stress-test agent behaviour across intents, edge cases, and multi-turn dialog flows 

  • Create prompt regression frameworks to assess how changes in system prompts, temperature, and sampling parameters impact output consistency 

Data Quality Assurance  

  • Statistically validate AI data outputs — distributions, precision/recall, error pattern analysis — to catch silent data quality failures before production 

  • Programmatically audit data ingestion, transformation, and feature store pipelines for schema drift and data corruption 

  • Validate vector DB indexing, embedding semantic similarity accuracy, and retrieval latency 

  • Verify quality, diversity, and privacy compliance of synthetic datasets used for model training and evaluation 

Classical ML & Deep Learning Validation  

  • Maintain automated suites tracking ML metrics — Precision, Recall, F1, ROC-AUC — and deep learning loss curves across model versions 

  • Implement continuous monitoring scripts to detect data and concept drift on live inference endpoints 

Automation Engineering & CI/CD  

  • Build and maintain scalable test automation frameworks for APIs, backend services, and model endpoints 

  • Embed AI evaluation and data QA suites into MLOps and CI/CD pipelines so quality failures block releases automatically 

  • Define and track AI quality KPIs and communicate release readiness to engineering and product teams 

Experience of 8+ years SDET role 

Technical Skills & Frameworks  

Core Programming  

  • Python — expert level; test automation, eval pipelines, data analysis (Pandas, NumPy, Pytest) 

  • SQL — data output validation, ground truth querying, pipeline data quality checks 

GenAI & Evaluation  

  • RAGAS / TruLens / DeepEval / Promptflow etc — LLM evaluation frameworks for measuring faithfulness, hallucination rate, and task success 

  • LangChain / LangSmith / LlamaIndex — agent workflow testing, prompt tracing, and LLM response debugging 

  • OpenAI / Anthropic / Hugging Face APIs — direct LLM endpoint testing and output consistency validation 

  • Vector DBs — retrieval quality testing, embedding validation, and latency benchmarking 

  • Pandas / NumPy etc. — statistical analysis for output validation and error pattern investigation, data profiling, schema validation, and pipeline integrity checks 

 

API & Automation  

  • Pytest — modular, reusable test framework for AI eval and automation suites 

  • Postman / REST Assured / Requests — API contract validation and service-level integration testing 

MLOps & CI/CD  

  • MLflow — tracking model versions and eval runs to detect regressions across updates 

  • Docker / GitHub Actions / Jenkins — containerised test environments and deployment pipeline automation 

Observability  

  • Grafana / Kibana / OpenTelemetry — monitoring AI system health, output drift, and distributed tracing across agent pipelines 

 

Good to Have  

  • Cloud AI Services — AWS Bedrock, Azure OpenAI, or GCP Vertex AI for testing managed model endpoints and cloud-deployed agents 

  • MLOps Platforms — MLflow, Kubeflow, Weights & Biases, or Feast feature stores for experiment tracking and model governance 

  • ML Frameworks — Scikit-learn, TensorFlow, or PyTorch familiarity for understanding model internals and validating training pipelines 

  • Infrastructure as Code — Docker, Kubernetes, Terraform for managing containerised test environments at scale 

  • UI Automation — Playwright or Cypress for end-to-end conversational AI application testing 

  • Performance Engineering — Locust or JMeter for load testing heavy AI inference endpoints under peak traffic 

  • Statistical Hypothesis Testing — t-tests, confidence intervals, significance testing to distinguish real quality signal from noise 

  • Synthetic Data Generation — using LLMs to generate diverse test cases and evaluation datasets at scale 

 

Current Tekion Employees: Please apply via the Internal Job Board in Ashby.

Note: Tekion recently transitioned to a new recruiting tool and we appreciate your patience and feedback as we adjust to our new system!

Tekion is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, victim of violence or having a family member who is a victim of violence, the intersectionality of two or more protected categories, or other applicable legally protected characteristics.

For more information on our privacy practices, please refer to our Applicant Privacy Notice here .

Important : Tekion will only contact candidates from an @ https://jobeax.com/link/U6VLsepI2Q6oKh41 email address or through our applicant tracking system ([email protected]). We may also work with authorized recruiting partners from time to time. Tekion and its recruiting partners will never ask you to pay fees or send money for applications, interviews, training, or onboarding. If you receive a message that seems suspicious, please do not respond or share personal or financial information.

Similar vacancies

More like this role

  • Hybrid
  • On-site
  • Contract
  • QA
... AI era? Join us at Zscaler. Role We are looking for a Staff Engineer to join our team. This is a Hybrid role in Bangalore, reporting to the Senior Manager, Software Development Engineering in the Engineering department. This unique and challenging position bridges software design engineering and test engineering to design, ...

Bangalore, भारत

  • Full-time
... journey-level individual contributor role that does not lead teams. Responsibilities - Independently handles a wide range of DevSecOps, cloud and infrastructure engineering issues with limited day-to-day instruction. - Builds and maintains CI/CD pipelines and automation to support software development and deployment. - Works ...

Bangalore, भारत

  • Full-time
... regression test cases into automated coverage - Perform day-to-day manual testing, including feature and sprint testing, BVT execution, regression, and exploratory testing. Provide timely release feedback and sign-off What you need - Manual testing - Test automation - Java - Python - Test cases - CI/CD Good to have - Root cause ...

Bangalore, भारत

  • Full-time
Mid-level QA and testing role at DTCC in Hyderabad. Needs 4+ years. What you'll do - Apply software development and testing methodologies, tools, and technologies to validate business, functional - Design, execute, and maintain manual and automated tests across the delivery lifecycle, including functional, regression, integration, ...

Hyderabad, भारत

  • Full-time
... - Coordinate seamlessly across time zones, collaborating closely with our distributed teams and stakeholders in both India and the US What you need - Manual testing - Selenium - Test automation - Performance testing - JIRA - Java Good to have - Appium - Cucumber - iOS - Android - CI/CD - IoT Job details - Location: Chennai ...

Chennai, भारत

  • Full-time
Senior QA and testing role at Wipro in Bengaluru. Needs 5–8 years. What you'll do - Prepare user training documents and related frameworks to ensure smooth adoption new applications and minimize transition risks." What you need - Root cause analysis Job details - Location: Bengaluru - Experience: 5–8 years - Education: ...

Bangalore, भारत

  • Full-time
... API and Web - You would also be setting up automation infrastructure, which would need you to work on setting triggers in CI/CD pipeline What you need - Manual testing - Test cases - Selenium - Test automation - Python - Unit testing Good to have - GCP - SQL - Java - CI/CD Job details - Location: Bengaluru, Delhi - Experience: ...

Delhi, भारत

  • Full-time
... a degree. What you'll do - Apply different software development and testing methodologies, tools, and technologies - Develop, execute, and maintain automated tests across the delivery lifecycle - Maintain traceability from user stories through test cases to automated scripts What you need - API testing - Selenium - Test ...

Chennai, भारत

  • Full-time
... and cloud) based on infrastructure standards - Collaborate with Product DRIs, Automation Developers, and Infrastructure teams to translate requirements into testable automation scenarios What you need - Bug tracking - Selenium - Test automation - Unit testing - Java - JIRA Good to have - Regression testing - SQL - Git ...

Hyderabad, भारत

  • Full-time
Mid-level QA and testing role at Weekday AI in Bengaluru. Needs 3+ years. What you'll do - Design, develop, and maintain automated test frameworks and scripts to ensure high-quality software delivery, focusing on JavaScript and TypeScript stack - Lead the creation of robust API and load test suites using Rest Assured and ...

Bangalore, भारत