# Autonomy needs a boundary, a checkpoint and a way back.

Combine hard policy, eligible tools, confirmation, recovery and evaluation traces so autonomy grows with evidence rather than confidence.

Version 1.0 · 2026-07-29 · 10 min

## Short answer

Guardrails are product controls around the model, not reminders inside its prompt. Start with a hard policy that defines data access, eligible tools, spending or operational limits and actions that always require a person. Give the model only the choices valid for the current state. Preview consequential changes, confirm them in the host product, recheck policy at execution and design idempotency, cancellation and recovery. Trace tool calls and resulting state, then expand autonomy only where evaluations and operational review support it.

## Key points

- Hard limits live in code, credentials and tool eligibility, not in persuasive wording.
- Confirmation is useful only when people can see the proposed effect and still cancel.
- Evaluation should inspect actions and resulting state, not only the final message.

## Useful autonomy is a product decision, not a model setting

The same model can answer a procedure, prepare a work order or submit a change. These operations do not carry the same consequences. A single autonomy switch hides the distinctions that matter: data scope, reversibility, financial or operational impact and the ability to detect failure.

A trustworthy design allocates authority by action. The model can have freedom inside a narrow, observable space while the host product keeps the limits, credentials and recovery path. More capability then becomes a controlled extension of the product rather than a broader prompt.

## A layered autonomy contract

Each layer should continue to protect the system when another layer is uncertain or fails.

### 1. Classify the outcome

Separate read, draft, propose, execute and irreversible effects. Record who owns each decision and what evidence is required.

### 2. Set the hard boundary

Enforce identity, data scope, budgets, invariants and forbidden operations in deterministic code and scoped credentials.

### 3. Reduce eligible tools

Expose only the operations valid for the role, workflow and current state. Tool retrieval may narrow this set but never widen it.

### 4. Place meaningful checkpoints

Show the target, values, evidence and expected effect before consequential work. Ask for confirmation where the person can still edit or cancel.

### 5. Execute with recovery

Revalidate immediately before the effect. Use duplicate protection, timeouts, partial-failure states and a documented path to retry, compensate or escalate.

### 6. Evaluate traces and outcomes

Test denied actions, injections, stale state, tool errors and alternate paths. Review the transition trace, tool calls and resulting state alongside communication quality.

## Examples

### Customer support

The assistant may read permitted account state and prepare a response. A refund or account change remains a typed proposal until policy and an authorised checkpoint allow execution.

### Industrial operations

The assistant can find a procedure and prepare an intervention note. It cannot bypass equipment locks, safety rules or the operator responsible for the action.

### Narrow experimental loop

Karpathy’s autoresearch repository illustrates one bounded pattern: a single editable file and a fixed evaluation loop. It is a field example for constraining experiments, not proof that the same design is safe for general production work.

## Common failure modes

- Telling the model to be careful while giving it broad credentials and tools.
- Letting semantic routing expand a deterministic permission boundary.
- Adding confirmation after the side effect or without showing the exact proposal.
- Granting autonomy without cancellation, duplicate protection or a partial-failure state.
- Evaluating polished transcripts while ignoring tool calls and the state left behind.

## Checklist

1. Every operation has a consequence and reversibility class.
2. Hard policy is enforced outside the model.
3. Credentials and eligible tools follow least privilege.
4. Important proposals are visible and editable before confirmation.
5. Execution rechecks identity, state and invariants.
6. Retry, cancellation, compensation and escalation are designed where relevant.
7. Traces link decisions, calls, results and final state without leaking secrets.
8. Autonomy expands only after representative evaluations and operating review.

## Limitations

- Guardrails reduce risk; they do not make an open-ended agent safe in every environment.
- A human checkpoint can become a rubber stamp if proposals are unclear or too frequent.
- Field examples and vendor guidance must be tested against the organisation’s own tools, threats and responsibilities.

## Sources

1. [Artificial Intelligence Risk Management Framework 1.0](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10) — NIST, 2023; type: standard.
2. [AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks](https://openreview.net/forum?id=m1YYAQjO3w) — Debenedetti et al., 2025; type: research.
3. [autoresearch](https://github.com/karpathy/autoresearch) — Andrej Karpathy, 2026; type: field-practice.
4. [AI Elements documentation](https://elements.ai-sdk.dev/docs) — Vercel, 2026; type: official-documentation.
5. [Trustworthy agents in practice](https://www.anthropic.com/research/trustworthy-agents) — Anthropic, 2026; type: field-practice.
6. [Demystifying evals for AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) — Anthropic, 2026; type: field-practice.

## Practical questions

### Should every tool call require approval?

No. Read-only and low-consequence operations may run automatically within strict scope. Approval should match consequence, reversibility and evidence.

### Is a system prompt a guardrail?

It can guide behaviour, but it is not a hard boundary. Access, tool eligibility, validation and execution policy belong to the host system.

### What should an autonomy evaluation verify?

Whether the task outcome is correct, the chosen path is allowed, required checkpoints occurred, restricted actions were refused and failure leaves a recoverable state.

---

Canonical: https://scoperail.fokalabs.io/en/guides/guardrails-for-agent-autonomy
Author: Ramzi Laieb, FokaLabs