QAD, Inc.
AI Operations Lead
- Full-time
- Contract
- AI / ML
Pune, भारत
Vacancy description
QAD, Inc.
Pune, भारत
The AI Operations Lead owns how AI runs at QAD after go-live. You will run AI FinOps (where the money goes whether it is worth it and how to keep spend inside budget) build the observability that tells us when an agent is drifting failing or getting expensive and stand up the operational support model that colleague across the business rely on when something breaks or they need help.
This is an operations and service ownership role not a reporting role. Dashboards are an input not the output. You will be measured on AI that stays reliable affordable and trusted in production. If spend runs ahead of value an agent quietly degrades after a model change or a function cannot get support that is your problem to solve not to report.
Where this role sits
Reports to: Head of the AI Centre of Excellence
Works with: AI Engagement Specialists Lead AI Architect and Senior AI Engineer daily; QAD Engineering teams External implementation partners and AI vendors; Finance Procurement IT Security GRC and Legal as required
Owns: AI consumption and cost governance production observability AI service and incident management orchestration agent registry and publish gate operations model refresh operations and partner and vendor service levels
Accountable for: Reliable affordable supported AI in production measured on spend against plan cost per transaction service levels and user satisfaction not ticket counts
What you will own
AI FinOps and consumption governance
Own AI spend end to end across licences usage-based model consumption cloud inference and partner run costs: forecast budget track and explain the variance every month with Finance.
Run the entitlement and consumption model: persona caps top-ups and tier upgrades kept in line with real usage. Reclaim idle licences and stop paying for tools nobody uses.
Make unit economics visible for every live workflow (cost per transaction per user and per outcome) and work with the architect on routing caching and model-tier choices. Most overspend is a frontier model doing routine work.
Bring usage evidence to commercial decisions: showback to functions and renewal commitment and tier reviews with Procurement and the vendors.
Observability and production performance
Build and run observability for the platform and every live agent: traces latency error and failure rates token usage and cost tool-call success guardrail triggers escalation and human-override rates.
Run evaluations in production with engineering: scheduled evaluation runs against curated datasets drift and regression detection after model or prompt changes and alert thresholds that someone acts on.
Own model and vendor change: plan model refreshes and deprecations regression-test before switching and keep a rollback path for every change.
Give leadership one operational view: service health spend and quality for the Head of AI and function leaders with a weekly operations review.
Operational support to the business
Stand up AI service management with IT: intake a tiered support model triage service levels and escalation to engineering and partners.
Own AI incident and problem management including AI-specific incidents: wrong or harmful output data exposure runaway cost and prompt injection. Run post-incident reviews and see corrective actions through.
Support colleagues across functions with access entitlements connectors and usage questions and keep the knowledge base and tool guidance current so most questions never need a ticket.
Feed operational insight to the Engagement Specialists: which workflows are struggling on quality adoption or cost with the evidence.
CoE operating model and controls
Run the agent registry and publish gate day to day: intake checks approval evidence named owners review dates and retirement.
Build operational requirements into every PRD: logging cost tagging evaluation hooks service levels and runbooks are acceptance criteria agreed before build not added after go-live.
Maintain operational controls with Security GRC and Legal: access reviews audit trails data-handling checks and evidence for audits.
Hold partners and vendors to their service levels: run cost availability quality and reporting with regular service reviews.
Run the CoE operating cadence and reporting to the Head of AI: a monthly AI operations and spend report and a quarterly business review.
Continuous improvement
Automate the operation: use scripting and AI to automate monitoring reporting and triage so the team scales with the number of live agents not with headcount.
Turn incidents into standards: reusable runbooks alert patterns and cost controls that every new workflow inherits.
Qualifications :
What we are looking for
Essential
15 years in cloud or platform operations site reliability engineering or IT service management including at least 3 years running ML data or Gen AI workloads in production.
Cloud FinOps in practice: forecasting cost allocation unit economics and optimisation working with Finance. FinOps certification is welcome; evidence of savings that held is what matters.
Observability: hands-on with monitoring and tracing tools (for example CloudWatch Datadog OpenTelemetry or LLM tracing tools such as Langfuse or LangSmith) and experience defining service level objectives and alerts that people act on.
Service management: incident problem change and request management in an enterprise. ITIL is welcome.
Working fluency in how LLM applications fail and cost money: tokens and context model tiers caching rate limits evaluation hallucination prompt injection and guardrails. Coding is not required; scripting in Python or SQL for analysis and automation is expected.
Data literacy: able to build and interrogate BI dashboards and define metrics that hold up to scrutiny.
Partner and vendor management: service levels commercial reviews and holding partners to account.
Clear communication with senior stakeholders: able to explain cost and risk plainly and to say no to spend that is not justified.
Calm ownership under pressure: you run the incident communicate clearly and close it properly.
Operational depth
This role needs credibility with Finance and IT from the first week. We are looking for someone who has run a production service and lived with its budget: who knows what a good on-call rota a useful alert and a defensible cost forecast look like and can tell when a number on a dashboard is wrong.
Strongly preferred
AWS / Azure / GCP AI operations and cost tooling including managed model services.
Administering enterprise AI tools such as Claude Enterprise Google Gemini or Microsoft 365 Copilot: seats usage analytics and policy controls.
Enterprise B2B SaaS; manufacturing supply chain or ERP domain knowledge. Not mandataory.
Having handled a serious AI or ML incident in production and what changed afterwards.
Experience operating across multiple regions and time zones and alongside system integrators.
What this role is not
Not a reporting role. Dashboards are an input; controlled spend and reliable agents are the output.
Not a help desk. You design the support model and own the hard problems; routine volume goes through the service desk and self-service.
Not a procurement role. You bring the usage evidence; Procurement negotiates with you.
Not a role that starts at go-live. Operational requirements go into the PRD and design before build begins.
What success looks like
First 3 months
One view of AI spend across all tools models and partners reconciled with Finance with an agreed forecast.
Persona caps top-ups and tier upgrades running as a process not as exceptions.
Baseline observability and an agreed support and incident model with IT for every live agent.
First 6 months
A monthly AI operations and spend report to the Head of AI that leadership uses to make decisions.
Cost per transaction tracked for every live workflow and production evaluations running on priority workflows.
Agent registry and publish gate operating with evidence; idle-licence reclamation delivering measured savings.
First 12 months
AI spend within budget with every variance explained including where value fell short of cost.
Service levels met for priority agents and AI incident resolution time falling quarter on quarter.
Model refreshes delivered without unplanned regressions and operational requirements built into every PRD by default.
Additional Information :
About QAD:
QAD is committed to ensuring that every employee feels they work in an environment that values their contributions respects their unique perspectives and provides opportunities for growth regardless of background. QADs DEI program is driving higher levels of diversity equity and inclusion so that employees can bring their whole self to work.
We are an Equal Opportunity Employer and do not discriminate against any employee or applicant for employment because of race color sex age national origin religion sexual orientation gender identity status as a veteran and basis of disability or any other federal state or local protected class.
Remote Work :
No
Employment Type :
Full-time
Similar vacancies
QAD, Inc.
Pune, भारत
Jobgether
भारत
Zeta Global
Hyderabad, भारत
Pune, भारत
Brillio
Bangalore, भारत
Cornerstone OnDemand
Hyderabad, भारत
Pune, भारत
Hyderabad, भारत
spotdraft
Bangalore, भारत
Chennai, भारत