2,347 monitoring and evaluation jobs in India.

... access controls, guardrails, and secure-by-default features. - Educate developers, update security standards and procedures, and influence engineering roadmaps and architectural improvements. - Analyze monitoring and detection signals and incorporate Agentic AI and process automation into risk detection and evaluation. - ...

... - Analyze monitoring effectiveness and drive automation, standardization, and continuous improvement initiatives. - Provide technical expertise, consultation, and governance for infrastructure monitoring services. - Maintain monitoring standards, documentation, and compliance with organizational policies and processes. ...

... text-to-speech (TTS), conversational AI, and voice-enabled assistants.- Create and optimize prompt engineering strategies to improve LLM output quality, consistency, and reliability.- Work with open-source LLMs for inference, fine-tuning, evaluation, and deployment.- Implement model monitoring, evaluation, guardrails, and performance ...

... delivery of material. Developing alternate sources for cost saving & timely delivery. Discuss with various suppliers, Accounts/ Excise / Sales Tax Team. Supplier evaluation. Benchmarking of Technology and commodity prices in Industry. Ensuring process adherence in purchasing. Periodical MIS generation & monitoring. Preferred Industries ...

... deployment and monitoring. Technical skills - Practical Python and SQL skills for data preparation, model development, API integration, testing and implementation. - Hands-on experience with Azure Machine Learning and Microsoft Foundry : pipelines, compute, environments, model deployment, endpoints, agents and evaluation. - Strong ...

... Familiarity with API Gateway CloudWatch IAM Secrets Manager S3 queues databases and related AWS services • Understanding of cloud security permissions monitoring and production support AI agents and orchestration • Design build and deploy AI agents and AI-enabled workflows within the approved architecture • Use Amazon and ...

... retrieval, reranking, and context construction. - Develop AI applications using LLMs, structured outputs, function/tool calling, and agentic workflows. - Build and optimize semantic search and vector retrieval systems. - Develop frameworks for LLM evaluation, monitoring, tracing, quality measurement, latency, and cost optimization. ...

... across model versions - Implement continuous monitoring scripts to detect data and concept drift on live inference endpoints Automation Engineering & CI/CD - Build and maintain scalable test automation frameworks for APIs, backend services, and model endpoints - Embed AI evaluation and data QA suites into MLOps and CI/CD pipelines ...

... capabilities. - Integrate AI agents and applications with enterprise platforms, backend services, APIs, and external tools. - Support modern AI integration frameworks and agent communication patterns to enable reliable interactions between AI systems. - Implement AI observability, monitoring, telemetry, evaluation, and performance-tracking ...

... prompt engineering, and Retrieval-Augmented Generation (RAG) pipelines.- Build REST APIs and integrate AI capabilities into existing applications.- Process, clean, and analyze structured and unstructured data for AI applications.- Implement model evaluation, performance monitoring, and optimization techniques.- Collaborate with ...

... automated evaluation (RAGAS-style), and release gating for LLM-based features.- Build agentic workflows : tool-calling / MCP-style integrations, human-in-the-loop and AI-in-the-loop patterns.- Implement LLM observability : token usage, model failures, latency, and cost monitoring alongside existing Splunk/ELK/Grafana/New Relic ...

... clean after remediation and recurring threats can be identified. - Model Evaluation and Validation: Establish rigorous evaluation frameworks using time-based and campaign-based dataset splits, historical false-positive and false-negative cases, and regression testing to prevent data leakage and verify improvements before ...

... experimentation, model evaluation, software quality, and production deployment while fostering a culture of scientific rigor and continuous learning. - Responsible and Reliable AI: Improve agent reliability through systematic testing, guardrails, human review, and ongoing performance monitoring. Ensure AI systems meet established ...

... programming, Git- Deployment : Cloud platforms, Docker, CI/CD, MLOps, LLMOps, monitoringIdeal Candidate Profile :A hands-on AI professional with strong problem-solving and software engineering skills, capable of leading technical discussions, working directly with clients, and delivering secure, reliable, and scalable AI solutions ...

... Partner with technology, security and governance teams.- Maintain deployment documentation and https://jobeax.com/link/nldvKGrc9xboPeAF You Bring : - Strong Python and PySpark skills.- Hands-on experience deploying ML models and building production scoring pipelines.- Sound grasp of evaluation metrics, drift and model performance.- ...

... pings, check calls, carrier data) rather than clean, pre-processed datasets - Diagnose gaps between offline evaluation performance and live production accuracy, and drive fixes - Build and maintain automated training/retraining pipelines using orchestration tools such as Airflow - Set up and maintain model monitoring and ...

... red-teaming, and production quality monitoring in the AI ADLC model Technical Skills - Deep hands-on experience with LLM evaluation — familiarity with frameworks and libraries such as LangSmith, DeepEval,LLM as a Judge, Promptfoo, RAGAS, Hugging Face evaluation tooling, custom eval harnesses - Strong understanding of agentic ...

... training, evaluation, and performance optimization.- Deploy machine learning models into production environments using scalable APIs and cloud infrastructure.- Build and maintain end-to-end ML pipelines, including training, inference, monitoring, and model versioning.- Work closely with product, engineering, and business teams ...

... troubleshooting and access-related queries. - Follow company policies regarding confidentiality, information security and access permissions. 2. HR Audit & Performance Monitoring - Review daily performance reports, task submissions and work records against defined evaluation criteria. - Verify the accuracy and completeness of performance ...

... monitoring, drift detection and continuous https://jobeax.com/link/riKf6u8ilNIwkEvz Skills : - Strong foundation in statistics, probability, machine learning and data analysis.- Strong hands-on programming skills in Python and common data science libraries.- Experience with Pandas, NumPy, scikit-learn and relevant ML frameworks.- ...