2,346 monitoring and evaluation jobs in India.

... drift detection, and confidence-band calibration - Produce model explainability outputs (e.g., SHAP/LIME) suitable for regulatory audit review Ongoing - Build and maintain MLOps pipelines for evaluation, monitoring, and (if the classical modeling phase triggers) model deployment and retraining, within Azure environment ...

... practices, including source control, code review, automated testing, and technical documentation.- Implement MLOps and GenAIOps practices, including CI/CD, model and prompt versioning, experiment tracking, automated deployment, rollback, and lifecycle management.- Establish production monitoring and evaluation to track model ...

... resolve complex technical risks. - Drive complex production issue triage and root cause analysis, ensuring sustainable fixes and operational learning. - Design and deliver production patterns for AI/ML capabilities (batch and real-time inference, feature pipelines, evaluation, monitoring). - Partner with Data Science teams ...

... physical and operational constraints, validating across feedstock changes, campaigns, grades, and configurations. Model assurance and generalization - Establish standards for baselining, benchmarking, feature-ablation studies, calibration, sensitivity analysis, robustness, and out-of-distribution evaluation. - Challenge apparent ...

... conservation, waste management, livelihoods, and community infrastructure. Experience in working with NGOs, foundations, government agencies, multilateral organizations, and academic institutions. - Experience in project design, partner due diligence, grant management, budget oversight, monitoring and evaluation, and impact assessment. ...

... administration, configuration, and maintenance of enterprise monitoring platforms, with a primary focus on New Relic. The role supports the delivery of effective monitoring and observability across FNZ platforms by implementing agreed monitoring standards, maintaining tooling health, and ensuring monitoring configurations remain ...

... context compression middleware and the use of interrupt nodes to manage asynchronous human-in-the-loop (HITL) approval https://jobeax.com/link/YPGPn1ELH5zQ9EaH, Evaluation & Guardrails : - Evaluation Frameworks : Extensive experience with Ragas (Retrieval Augmented Generation Assessment) to programmatically benchmark faithfulness, ...

... AI Engineer, you will own clinical AI algorithms end to end: problem formulation, data and ground truth, modelling, clinical evaluation, production deployment and monitoring. This is a hands-on senior individual-contributor role for an engineer who has already shipped and operated production AI systems. Roles and Responsibilities: ...

... LangGraph, https://jobeax.com/link/Lhx8bXHKijn3cWof & DevOps : - Experiment tracking (MLflow, Weights & Biases), containerization/orchestration (Docker, Kubernetes), and CI/CD (GitHub Actions, Azure DevOps).Evaluation & Observability : - Evaluation metrics (precision, recall, F1) and monitoring tools (Prometheus, Grafana).Leadership ...

... engineering, training, deployment, and inference.- Experience building or integrating REST APIs and working with backend services.- Familiarity with structured and unstructured datasets.- Experience with cloud platforms, preferably GCP.- Understanding of model monitoring, evaluation, versioning, and performance https://jobeax.com/link/WrPpyfU0ZsWMfPbm ...

... applications, databases and external tools using tool/function calling.- Develop scalable, secure and reliable AI solutions for production environments.- Perform AI evaluation, testing, debugging, monitoring and optimization.- Implement appropriate AI guardrails, security, privacy and responsible AI practices.- Work with cloud platforms ...

... security, governance, and business value realization.- AI Governance & Responsible AI : Establish AI governance, model evaluation, prompt management, guardrails, monitoring, and responsible AI practices.- Cross-functional Leadership : Collaborate with business stakeholders, data engineers, analytics teams, and solution architects, ...

... Description :- Analyze structured and unstructured datasets to identify actionable insights, patterns, and trends.- Develop and deploy machine learning models and statistical solutions to address business challenges.- Apply statistical techniques including hypothesis testing, probability modeling, and model evaluation metrics ...

... align technical implementations with broader business goals. - Oversee system monitoring and performance optimization initiatives to ensure high availability and efficiency. - Provide expert-level application support and troubleshooting for critical cloud infrastructure and AI-driven tools. - Develop and refine prompt ...

... LangChain, LangGraph/LlamaIndex, Prompt Engineering, AI Agents- MLOps: Model Serving, CI/CD, Monitoring, Docker, Kubernetes- Cloud: AWS / Azure / GCP- Strong understanding of ML lifecycle, model evaluation, optimization, explainability, bias mitigation, AI governance, and responsible https://jobeax.com/link/j2wYDhLFedXkNTfL :- ...

... LangChain, LangGraph/LlamaIndex, Prompt Engineering, AI Agents- MLOps: Model Serving, CI/CD, Monitoring, Docker, Kubernetes- Cloud: AWS / Azure / GCP- Strong understanding of ML lifecycle, model evaluation, optimization, explainability, bias mitigation, AI governance, and responsible https://jobeax.com/link/j2wYDhLFedXkNTfL :- ...

... responsible AI implementation.- Oversee deployment and monitoring of AI models and agents across development, staging, and production environments.- Establish evaluation frameworks to assess AI accuracy, response quality, hallucinations, performance, and business outcomes.- Lead, mentor, and develop AI/ML engineers and technical ...

... use/function calling and enterprise AI integrations. - Experience with MCP (Model Context Protocol) or similar AI integration approaches. - Experience with model evaluation, observability, and AI application monitoring. - Knowledge of fine-tuning, model adaptation, embeddings, and NLP techniques. - Experience integrating AI solutions ...

... maintaining acceptable output quality.- Establish systematic evaluation and regression testing for model quality, retrieval relevance, groundedness, hallucinations, and application reliability.- Implement robust production practices, including logging, monitoring, retries, timeouts, caching, versioning, and failure recovery.- ...

... Skills: Multi-Agent System Design & Orchestration Full-stack AI Integration (Frontend to Backend) Advanced Prompt Engineering LLM Observability, Evaluation & Monitoring (LLMOps) Data Engineering Skills: Proficient in Pyspark coding and concepts. Example: Lazy evaluation, window functions, spark architecture and performance ...