What happens after launch? How to monitor and improve production AI

Launching an AI workflow is the start of an operating responsibility. Models change, source data changes, policies change and users find new ways to work around the system. Performance can deteriorate without a dramatic outage.

Production monitoring must answer two questions: is the system working as approved, and is the workflow still creating the intended outcome?

Monitor five layers

1. Technical health

Track availability, latency, timeouts, error rate, token or compute usage, dependency health and cost. These measures reveal whether the service is functioning, but not whether its outputs are good.

2. AI quality

Measure the task-specific criteria used before release. Examples include groundedness, required-fact coverage, extraction accuracy, policy adherence and severe failure rate.

Many quality measures require sampled human review. Automate stable checks, but preserve expert assessment where consequence and judgement matter.

3. Workflow performance

Track net handling time, queue movement, rework, end-to-end cycle time and service levels. Include the time people spend checking and correcting AI output.

4. User behaviour and adoption

Measure appropriate use on eligible cases, acceptance, material edits, rejection, override reasons and use outside scope. Watch for automation bias, rubber-stamping and unofficial workarounds.

5. Business and risk outcomes

Track the outcome that justified the investment, alongside complaints, incidents, control failures, customer harm indicators and relevant group-level performance.

The AWS guidance on production monitoring for generative AI describes monitoring, feedback and root-cause analysis as a loop back into development. That loop is more useful than a dashboard that nobody owns.

Connect every metric to an action

A metric without a threshold and owner creates observation, not control.

For each measure, record:

• Definition and data source.
• Expected range or target.
• Warning and critical thresholds.
• Review frequency.
• Named owner.
• Required response.
• Escalation and pause authority.

Use multiple time windows. A sudden severe error needs immediate action, while gradual drift may become visible only over weeks.

Build a production sampling plan

It is rarely practical or necessary for an expert to review every output. Use risk-based sampling.

Combine:

• A random sample of normal production work.
• Higher sampling for new users, models or case types.
• All cases that trigger defined risk indicators.
• Rejected, overridden or heavily edited outputs.
• Complaints, incidents and near misses.
• Segments with low volume but high consequence.

Do not sample only successful or easy cases. Preserve enough source context for a reviewer to judge the result, with appropriate privacy and access controls.

Calibrate reviewers and track disagreement. Changes in reviewer interpretation can look like model drift.

Capture feedback as structured evidence

A thumbs-up button is easy to add and hard to interpret. Users may express writing preference rather than task correctness, and non-response is ambiguous.

Capture feedback at the point of work:

• Accept, edit, reject or escalate.
• Reason for a material correction.
• Missing or unsupported fact.
• Incorrect classification or tool action.
• Case outside approved scope.
• Policy or source-data issue.
• User-interface or workflow problem.

Separate AI failures from process, data and integration failures. All matter, but they need different owners.

Make feedback lightweight enough to use. Sample deeper review rather than asking every employee to complete a long form for every case.

Detect drift broadly

Production drift is not only statistical model drift.

Input drift

Document types, customer language, product mix or case complexity changes.

Context drift

The knowledge base, policy documents or retrieved sources become outdated, incomplete or differently structured.

Behaviour drift

Users change how they prompt, review, override or route work.

Performance drift

Quality, latency, cost or failure rates move outside the approved range.

Outcome drift

Task measures remain stable but the business benefit falls because a bottleneck or operating process changes.

Governance drift

The real use expands beyond the approved purpose, permissions or user group.

Monitor leading indicators for each type. A rise in unknown document classes may appear before summary quality declines.

Control every production change

An AI system can change through the model, system instructions, prompts, retrieval logic, source documents, tool permissions, user interface or business policy.

Maintain a version record that ties each production output to the relevant configuration. Classify changes by risk and define the evaluation required before release.

For a material change:

1. State the reason and intended effect.
2. Identify affected failure modes and users.
3. Run the relevant regression and adversarial tests.
4. Compare against the current production version.
5. Approve through the defined route.
6. Release gradually where practical.
7. Monitor closely and retain a rollback path.

Do not allow an external model update to bypass internal release criteria. Where a supplier controls updates, understand notification, validation and version options before relying on the service for a critical workflow.

Run a clear incident process

Define what counts as an AI incident. Examples include sensitive-data disclosure, an unauthorised tool action, systematic unfairness, a severe unsupported conclusion, control bypass or widespread quality degradation.

The incident playbook should cover:

• Immediate containment and pause.
• Preservation of logs and affected versions.
• Identification of impacted cases and people.
• Required internal and external notification.
• Customer correction and redress where relevant.
• Root-cause analysis across model, data, system and process.
• Remediation and evidence for safe restart.
• Addition of the failure to the evaluation set.

Practise the pause and fallback before an incident. A control that has never been tested may not work under pressure.

Set an operating cadence

Use a cadence proportionate to risk and maturity.

Daily or automated

Service health, cost spikes, access anomalies, critical errors and pause conditions.

Weekly

Quality sample, adoption, corrections, exceptions, queue performance and open incidents during early operation.

Monthly

Workflow outcomes, segment performance, realised value, user behaviour, model or data changes and remediation progress.

Quarterly

Continued fitness for purpose, supplier review, risk assessment, autonomy level, evaluation-set coverage and strategic value.

High-risk or newly launched systems need more frequent attention. Stable low-impact tools may justify a lighter cycle, but not zero ownership.

Recalculate value after launch

Use real volume, adoption, review effort, exceptions and run cost. Confirm how released capacity is being used and whether the original bottleneck has moved.

A workflow can remain technically strong but lose its business case. It may also create new value not anticipated in the pilot, such as improved data quality or earlier risk identification. Record both, with evidence.

Our guide to measuring AI ROI explains how to connect these production measures to realised outcomes.

Decide when to improve, constrain or retire

Use explicit decision rules.

Improve when the outcome remains valuable and evidence points to a fix, such as better retrieval, clearer instructions, user-interface changes or additional training.

Constrain when risk rises or performance differs by case type. Reduce eligible scope, require extra approval or lower autonomy while the issue is investigated.

Retire when the system no longer creates enough value, cannot meet risk tolerance, has an unsupported dependency or has been replaced by a better process.

Retirement needs a plan for data, integrations, outstanding cases, user communication, access removal and retained audit evidence.

A minimum production control pack

For each live workflow, maintain:

• Approved purpose, users and exclusions.
• Current system and supplier versions.
• Evaluation set, thresholds and latest results.
• Monitoring measures and named owners.
• Sampling and feedback process.
• Change and release record.
• Incident, pause and fallback procedure.
• Open limitations and remediation.
• Value review and next decision date.

The NIST AI Risk Management Framework treats Manage as an ongoing response to measured risk, supported by governance and context. Production AI needs that same continuing connection between evidence and action.

The operating principle

Monitor the properties that justified release, in the workflow where the system is actually used. When evidence changes, change the system, the scope or the decision.

That is how AI becomes a managed operational capability rather than a model that happens to be live.

Sorsana helps teams build the evaluation and operating loop around production AI. Explore our AI evaluation and governance work or talk to us about a live system.