2,359 monitoring and evaluation jobs in India.

... translate their requirements into operational capabilities and controls. You will apply reproducible workflows, evaluation, controlled deployment, monitoring, and incident response to existing developer harnesses and coding tools, large language model (LLM) applications and agents, and machine learning training and inference. ...

... degradation in production environments. - Assess RAG-Based Solutions - Understand retrieval pipelines and be able to evaluate retrieval quality, citation accuracy, and response grounding. - Work with AI Observability Platforms - Experience with AI monitoring, tracing, evaluation logging, dashboards, and observability tools such ...

... execution, and documentation of audits. - Lead, supervise, and mentor Inspectors, serving as their direct superior within the Certification Division. - Conduct and document witness inspections of Inspectors as part of competence evaluation, authorisation, and ongoing performance monitoring. - Carry out lead audits of Inspectors ...

... Engagement & Partnership Building - Team Leadership & People Management - Budgeting, Planning & Resource Allocation - Risk Management & Problem Solving - Monitoring, Evaluation & Reporting - Compliance & Governance - Strong Communication & Interpersonal Skills - Proficiency in MS Office and project management/reporting ...

... solution accuracy, relevance, performance, and reliability.- Deploy and maintain AI applications on cloud platforms, ensuring scalability, security, observability, and operational stability.- Implement testing, evaluation, monitoring, and guardrails to assess AI outputs, manage failure scenarios, and minimise hallucinations ...

... data engineering experience with ETL, SQL/NoSQL, vector databases, and data pipelines.- Understanding of multi-agent architectures and MCP.- Experience with AWS and/or Azure.- Strong knowledge of Docker, CI/CD, and infrastructure-as-code.- Understanding of AI system monitoring, evaluation, debugging, and production https://jobeax.com/link/TkKbWHlgwW4jcLq2 ...

... required, lead audits and ad-hoc projects, including finding validations for CB and IB Operations and India Country audit team. - Contribute to the continuous monitoring and SME framework of the GA CB and IB environment, periodic update of auditable area risk assessments and contribute to the annual audit planning process as ...

... pings, check calls, carrier data) rather than clean, pre-processed datasets - Diagnose gaps between offline evaluation performance and live production accuracy, and drive fixes - Build and maintain automated training/retraining pipelines using orchestration tools such as Airflow - Set up and maintain model monitoring and ...

... pings, check calls, carrier data) rather than clean, pre-processed datasets - Diagnose gaps between offline evaluation performance and live production accuracy, and drive fixes - Build and maintain automated training/retraining pipelines using orchestration tools such as Airflow - Set up and maintain model monitoring and ...

... workflows. - Design and maintain a comprehensive observability platform encompassing metrics, logs, traces, dashboards, alerts, service-level agreements (SLAs), and AI-specific debugging and evaluation capabilities. - Build and operate OpenTelemetry collection infrastructure, including multi-tenant receivers, exporters, processing ...

... MCP-based tools and expose internal APIs/Databricks functions to agents.- Manage production AI systems including CI/CD, scaling, versioning, containerization, and Human-in-the-Loop (HITL) workflows.- Implement agent observability and evaluation, including monitoring reasoning, correctness, and tool https://jobeax.com/link/Bj9LbBTRYdOcEFtM ...

... troubleshooting and access-related queries. - Follow company policies regarding confidentiality, information security and access permissions. 2. HR Audit & Performance Monitoring - Review daily performance reports, task submissions and work records against defined evaluation criteria. - Verify the accuracy and completeness of performance ...

... Maintenance of rooftop and ground-mounted Solar PV power plants . The candidate should have strong electrical troubleshooting skills, knowledge of solar equipment, and a valid Wireman / Supervisor Licence . The role involves preventive and corrective maintenance, plant monitoring, troubleshooting, safety compliance, and travel ...

... embedding services, training/evaluation frameworks, SDKs, and model management APIs—to simplify the end‑to‑end AI/ML lifecycle. Your work will enable researchers and engineers to move rapidly and safely from early experimentation to production, with standardized workflows, governance, observability, repeatability and reproducibility, ...

... Understanding of multi-agent systems (MOE), reasoning frameworks, and agent orchestration.- Workflow Design: Hands-on expertise in Prompt Flow, automation pipelines, and human-in-the-loop systems.- Post-Training Techniques: Experience with fine-tuning, instruction tuning, RLHF, and domain adaptation.- Model Evaluation: Familiarity ...

... quality and reduce fraud losses while driving the cost of monitoring per member down, and scale agentic solutions where a wrong answer has consequences for a member and for Tide. As VP Engineering, Ongoing Monitoring you will: - Set the engineering strategy for Tide's fraud, financial crime and ongoing monitoring platforms - ...

... robust, compliant production systems - Define engineering standards, governance frameworks, and design patterns covering data models, APIs, security, deployment, and monitoring - Establish and operate AI/ML governance controls, GenAI production controls, and comprehensive evaluation frameworks for reliability and security ...

... implementation partners, or community-based organizations is preferred. Required Skills - Strong stakeholder and partner management skills. - Knowledge of Monitoring & Evaluation frameworks, indicators, and reporting. - Experience in data collection, validation, analysis, and documentation. - Excellent communication and interpersonal ...

... embeddings lifecycle, chunking strategies, re ranking, caching, and fallback logic that work at scale. - Model & Agent Evaluation: Define and operationalize agent evaluation frameworks (offline + online), including regression tests, task success metrics, hallucination detection, latency/cost controls, and MCP style orchestration ...

... training, evaluation, inference, and deployment. This is a hands-on leadership role for someone who wants to operate at the intersection of research, systems, and product. WHAT YOU’LL OWN – Own the end-to-end ML systems powering our company, from data and training to evaluation, inference, and deployment. – Build and evolve ...