How much autonomy should an AI agent have?

An AI agent can plan steps, use tools and act across systems. That can remove hand-offs and make a workflow faster. It also means one incorrect interpretation can become a sequence of real actions.

The safe level of autonomy is not a property of the model. It is a design decision based on the action, consequence, evidence, permissions and control environment.

Separate intelligence from authority

A system may be highly capable at interpreting a case and still have very limited permission to act. Keep two questions separate:

1. How well can the system propose the right next step?
2. What authority should it have to execute that step?

The first is tested through evaluation. The second is decided through risk, reversibility and accountability.

The UK Competition and Markets Authority guidance on AI agents makes clear that businesses remain responsible for agents acting on their behalf and highlights testing, monitoring and human oversight. Delegating an action to software does not delegate accountability.

Use five autonomy levels

Level 0: observe

The agent reads approved information and produces no recommendation or external action. It may organise a queue, extract facts or identify patterns for analysis.

Example: classify incoming claim documents and flag uncertain classifications for review.

Level 1: recommend

The agent proposes an action with supporting evidence. A person decides and performs the action.

Example: suggest which complaint team should receive a case and explain the indicators.

Level 2: prepare

The agent prepares the action in a draft or pending state. A person reviews and authorises execution.

Example: create a response draft, populate a case form or prepare a refund request without sending it.

Level 3: execute within bounds

The agent performs defined, reversible, low-impact actions when explicit conditions are met. It escalates exceptions.

Example: request a standard missing document for an eligible case using approved wording, with rate limits and an audit log.

Level 4: orchestrate a bounded workflow

The agent selects and performs several actions within a tightly defined objective, permission set and budget. Humans monitor the process and approve specified high-impact steps.

Example: gather information from approved internal systems, prepare a case pack, request missing evidence and route the completed file, while decisions and customer outcomes remain reserved.

Do not make Level 4 the target for every workflow. The appropriate level may permanently remain at recommendation or preparation.

Score each proposed action

Assess actions individually. An agent may have different autonomy within the same workflow.

Impact

What is the worst credible consequence of an incorrect action? Consider financial loss, customer harm, discrimination, privacy, contractual commitment, operational disruption and regulatory impact.

Reversibility

Can the action be undone quickly and completely? Drafting is more reversible than sending. Creating a pending record is more reversible than moving money.

Detectability

How likely is the business to notice the error before harm occurs? A reviewer may easily spot a wrong category but struggle to notice a plausible omitted fact in a long document set.

Frequency and speed

How many actions can the agent take before a person or monitor intervenes? Small errors can compound at machine speed.

Data sensitivity

What information can the agent read, infer, combine or disclose? Check both direct data access and what may appear in logs or outputs.

Rule clarity

Can eligibility, limits and prohibited actions be expressed and enforced? Vague objectives invite unpredictable plans.

External effect

Does the action affect only an internal draft, or does it communicate, transact or change a customer’s position?

Use the highest material risk rather than averaging away a severe dimension.

Turn the score into an approval matrix

Use four action bands.

Low impact, reversible and easily detected: the agent may execute within rules. Apply logging, rate limits and monitoring.

Moderate impact or incomplete detectability: the agent may prepare the action, but a user approves execution.

High impact, difficult to reverse or difficult to detect: the agent may retrieve evidence or recommend. An authorised person performs the action after substantive review.

Prohibited or uncontrollable: the agent must not propose or execute the action. Remove the permission and route the work outside the system.

Examples of actions that often warrant strict control include binding customer decisions, financial transfers above a low threshold, deletion of records, changes to access rights, commitments to third parties and communications involving disputed or sensitive facts.

The matrix should be approved by the workflow owner and relevant risk specialists. Encode it in technical permissions and approval states.

Enforce least privilege

Do not give an agent a broad credential because some future task might need it. Grant the minimum tools, records and actions for the approved workflow.

Practical controls include:

• Separate read, draft, approve and execute permissions.
• Allowlisted tools and destinations.
• Field-level and record-level access.
• Transaction, volume and time limits.
• Prohibition of privilege escalation and credential creation.
• Short-lived credentials tied to the user or workflow.
• Network and data-boundary restrictions.
• A deterministic policy layer between the model and the tool.

The model can request an action. A separate control should decide whether the action is allowed.

Make approvals meaningful

An approval screen that says “Allow” beside an opaque plan is weak oversight. Show the reviewer:

• The exact action and target.
• Values that will be written, sent or committed.
• Evidence used to justify the action.
• Uncertainty and unresolved conflicts.
• Earlier steps in the chain.
• The consequence and whether it can be reversed.

Batch approvals can save time but increase risk. Group only genuinely similar, low-impact actions, and make sampling and revocation easy.

The ICO human review framework provides useful questions about reviewer authority, competence, logs and whether review is effective in practice. Check current legal guidance for your specific use of automated decision-making.

Design for failure and containment

Assume an agent will eventually misunderstand a request, encounter hostile content, call a tool incorrectly or continue after a dependency changes.

Containment controls should include:

• A maximum number of steps and actions.
• Spend, time and transaction budgets.
• Timeouts and circuit breakers.
• Idempotency and duplicate prevention.
• Sandboxed or simulated execution for testing.
• A manual stop control with a named owner.
• A safe state when a tool is unavailable.
• Monitoring for unusual sequences and access patterns.

The UK National Cyber Security Centre guidance on adopting agentic AI recommends bounded adoption, meaningful human control and planning for failure. Treat that boundary as part of the system architecture.

Evaluate trajectories, not only final answers

Agent evaluation needs to inspect the sequence of decisions and actions.

Test whether the agent:

• Selects the correct tool.
• Uses the correct parameters and record.
• Stops when the objective is met.
• Seeks approval at the required boundary.
• Refuses prohibited actions.
• Recovers safely from tool failure.
• Resists instructions in untrusted content.
• Avoids repeating or compounding an action.
• Produces a complete audit trail.

Include adversarial and unexpected cases. A successful final result can hide an unsafe route, such as accessing unnecessary records or nearly sending a duplicate payment.

Roll out autonomy gradually

Begin in observation or recommendation mode. Compare proposed actions with what authorised users actually do. Move to preparation when evaluation and workflow evidence are strong.

Allow bounded execution only for selected actions, users and volumes. Set expansion criteria in advance, including quality, severe failure, intervention, incident and value thresholds.

Autonomy can move down as well as up. A new model version, policy change, incident or unusual demand period may justify returning an action to approval mode.

Maintain an agent register

For each agent, record:

• Owner, purpose and users.
• Approved tools, data and environments.
• Action-level autonomy and approval rules.
• Model and configuration versions.
• Evaluation results and limitations.
• Monitoring and incident process.
• Suppliers and dependencies.
• Last review and next review date.
• Pause and withdrawal authority.

Review the register when permissions, workflows or suppliers change.

The practical rule

Give an agent enough autonomy to remove a specific bottleneck, not enough to pursue a broad objective without reliable containment.

Authority should be earned action by action through representative evaluation, effective controls and production evidence. A capable agent with narrow permissions is usually more useful than an unconstrained agent the organisation cannot safely trust.

Sorsana helps organisations design and evaluate controlled AI workflows. Explore our AI evaluation and governance work or discuss an agentic workflow.