2,359 monitoring and evaluation jobs in India.

... Build and own evaluation infrastructure: golden datasets per tenant, LLM-as-judge graders calibrated against human judgment, regression gates on every prompt and model change, and online quality monitoring. - Drive answer quality: grounding and attribution against ACL-visible sources, hallucination control, and confidence ...

... fine-tuning, adapters or distillation only where evidence shows the return justifies the operating burden — and be able to make that argument either way. - Build the evaluation harness: golden datasets, offline evaluation, LLM-as-judge with human calibration, regression suites running in CI, and online quality monitoring with structured ...

... tool permissions, retries and loop controls.- Drive productionisation. Move priority use cases from discovery to production with evaluation, MLOps/LLMOps, monitoring, fallbacks, cost controls, governance and adoption.- Lead the team. Set technical direction, review designs and model choices, mentor AI talent and create ...

... channels, paste sites, credential repositories, and other relevant sources for client-specific threats and material changes in the threat landscape. - Develop and refine collection plans, search strategies, source lists, personas, keywords, and monitoring logic aligned to intelligence requirements and investigative objectives. ...

... extraction, and reasoning. - Develop agentic AI solutions using LLMs, LangChain, LangGraph, and related frameworks. - Apply prompt engineering, orchestration, evaluation, monitoring, text processing, embeddings, and vector-based retrieval practices. - Integrate AI services with MSCI data science platforms and APIs and collaborate ...

... upgrades. - Validate platform capabilities such as ingestion, feature computation, model deployment/rollback, multi-tenancy, and governance. - Define quality metrics and SLO validation, with automated monitoring to detect platform anomalies before they impact ML teams or customers. - Contribute to evaluation frameworks for platform-level ...

... Understanding of web scraping and unstructured data processing. Soft. Engineering Experience with Git, testing frameworks, and CI/CD pipelines. Cloud Good understanding of Azure and cloud-native architectures. MLOps / LLMOps Familiarity with deployment, monitoring, and operational workflows for AI solutions. AI Evaluation ...

... methodologies, tools, and reporting processes. - Integrate ESG risk assessment and impact measurement into the complete lending and investment lifecycle, from pipeline evaluation to execution and portfolio monitoring. - Conduct and oversee ESG due diligence and environmental and social risk assessments for prospective and existing portfolio ...

... experimentation, benchmarking, model evaluation, and performance optimization with scientific rigor - Improve inference efficiency, latency optimization, cost management, and reliability of deployed AI systems - Establish guardrails, hallucination detection, monitoring, and responsible AI practices for production deployments - Contribute ...

... deployment of AI/ML models and solutions. - Define AI/ML strategies and technical roadmaps aligned with business objectives. - Build and optimize machine learning and deep learning models for real-world applications. - Oversee data preprocessing, feature engineering, model training, validation, and evaluation. - Work with algorithms ...

... memory and Human-in-the-Loop patterns.- Experience with one or more cloud AI ecosystems: Azure, AWS or GCP.- Strong knowledge of Python, REST APIs, microservices and distributed/cloud-native architectures.- Experience with LLM evaluation, tracing, monitoring, prompt/model versioning and LLMOps/MLOps.- Understanding of AI security, ...

... function/tool calling, and agent workflows.- Apply Machine Learning and NLP techniques to solve complex business problems.- Develop Python-based AI/ML applications and APIs.- Work with frameworks such as LangChain, LangGraph, LlamaIndex, Semantic Kernel, AutoGen, or similar.- Implement model evaluation, monitoring, guardrails, ...

... detection, face recognition, face quality assessment, and facial attribute analysis.- Train, fine-tune, and optimize deep learning models using PyTorch, TensorFlow, and/or JAX.- Build and maintain end-to-end ML pipelines covering data ingestion, preprocessing, training, deployment, and monitoring.- Design automated data pipelines ...

... agents that source online benchmarks, gather supplier contacts, and support automated negotiation. - Ship customer-facing features for real-time spend visibility and supplier benchmarking, and iterate quickly on customer feedback. - Build evaluation and monitoring for our AI systems so we can measure accuracy and catch regressions ...

... : Clean, prune, and collect datasets to analyze model performance, including hallucination rates, recall, and overall accuracy.- Advanced Research : Research and implement SOTA (state-of-the-art) approaches for knowledge graphs and Graph RAGs.- Evals & Monitoring : Run continuous evaluations and monitoring for production ...

... : Clean, prune, and collect datasets to analyze model performance, including hallucination rates, recall, and overall accuracy.- Advanced Research : Research and implement SOTA (state-of-the-art) approaches for knowledge graphs and Graph RAGs.- Evals & Monitoring : Run continuous evaluations and monitoring for production ...

... degradation, and clear incident management practices. - Evaluation & Data-Driven Decisions: Establish rigorous evaluation practices - offline evals, regression suites, and production monitoring - so agent behavior is measurable and comparable across model and prompt changes. Use that data to prioritize, evaluate trade-offs, and ...

... wise. 4. Manage productivity for fixing productivity parameters for Collection Officers Agencies/In-house team. 5. Monitoring performance against set parameters and weekly review with Collection executives. 6. Ensuring background check of all collection agencies hired/Identify and manage training needs of the Collection Executives/ ...

... model governance, responsible AI practices, and performance monitoring in production environments - Stay current with advancements in Generative AI, LLM research, and applied machine learning, incorporating relevant innovations into company solutions - Define evaluation metrics, build backtesting pipelines, and run controlled ...

... related, and gather logs and evidence.- Resolve known errors using runbooks: Socket reboot and HA failover checks, ISP link verification, Cato Client reinstall and version checks, basic policy hit verification.- Open and track carrier tickets for circuit outages at overseas sites and coordinate local smart hands for hardware ...