Trust, security & evaluation
An agent can be useful and still be unsafe. A repository can contain prompt injection, a skill can smuggle in an instruction, an MCP server can expose a destructive tool, and a headless run can repeat a mistake without anyone present to stop it.
The practical question is not “is this agent trusted?” It is: which inputs can influence it, which actions can it take, and which layer can still stop a bad action?
The guarantee ladder
Section titled “The guarantee ladder”Every control lives at a layer, and the layer decides what the control can promise. Anything made of text - prompts, rules, skill bodies - can only ask the model to behave. Controls placed in the action path can refuse an action regardless of what the model decided. Controls around the process can contain damage even when both the model and the gate are fooled. That is the whole logic of the ladder: the further a control sits from the model’s output, the less it depends on the model’s cooperation.
| Layer | What it does | What it cannot promise |
|---|---|---|
| Prompt, rules, and skills | Tells the model what to do | The model will follow it or resist hostile context |
| Model-selected skill or agent | Packages a role, procedure, or tool surface | The selected component is benign or current |
| Approval policy | Puts a human or preconfigured policy in the action path | A person will read every approval, especially in CI |
| Hook or runtime gate | Blocks, rewrites, or audits supported lifecycle events | Unsupported paths or actions outside the runtime are covered |
| OS sandbox | Limits filesystem, process, and network reach | The task result is correct or secrets are available safely |
| Repository or server-side enforcement | Protects branches, deploys, credentials, and external systems | Local work before the request reaches the boundary |
This is why “plan mode,” “auto,” and “deny” need qualification. They are product labels, not universal security guarantees. The label is not the boundary; the layer is. See Permissions & sandboxing, Hooks, and Plan mode for the tool-specific boundaries.
Threat-model the input path
Section titled “Threat-model the input path”Before enabling a new capability, ask what can arrive through each channel:
- Repository content: source files, issues, fixtures, generated files, and comments can contain instructions aimed at the model rather than the developer - a fixture comment reading “ignore the failing assertion” is an attack on the reader of the file, not on the test runner.
- Tool output: search results, logs, webpages, and MCP resources can carry untrusted text. Treat tool descriptions and annotations as data, not policy.
- Installed components: skills, plugins, extensions, and MCP servers are executable supply-chain dependencies. Review their source, pin versions where possible, and remove unused access.
- Credentials and reach: give a workflow only the tokens, directories, network access, and worktree it needs. A read-only task should not inherit deployment credentials.
- Unattended execution: headless and background runs need preconfigured approval, sandbox, hook, timeout, output-validation, and rollback policies because no person may be present at the moment of action.
The model’s context is the attack surface: every channel that can write into it is an input path.
The MCP security principles and authorization specification are useful reference points: servers can perform arbitrary actions on connected systems, and consent must remain explicit and under host control.
Evaluate the workflow, not just the prompt
Section titled “Evaluate the workflow, not just the prompt”An agent configuration is a small software system, and “the model says it succeeded” is not a test result. Watch one toy run at real scale: a nightly job that drafts release notes from the merged PRs.
The obvious check is to read the output. It reads well, every sentence is plausible, the run ends green, and the summary explains exactly what it did and why. Plant that verdict: reads well, ship it.
Now grade artifacts instead. The postconditions are dumb on purpose: every PR number since the last tag appears in the file, no entry survives a revert, exit code zero. PR #4821 was squash-merged with a rewritten title, so the draft misses it; #4799 was reverted an hour after merging, but its entry is still there. Both misses are invisible in prose that reads well and fatal in a diff against git log. Deterministic checks catch what a readable summary hides.
That toy is the loop, generalized:
- Define a task with a fixture, an acceptance test, and explicit forbidden outcomes.
- Run it in a disposable worktree with the smallest useful tool and credential set.
- Capture the trace: model, rules, skills, tool calls, approvals, files changed, duration, and cost.
- Grade the result against deterministic postconditions before reading the model’s explanation.
- Repeat on representative failures and adversarial inputs, then compare against the previous configuration.
Track at least:
| Metric | Why it matters |
|---|---|
| Task success rate | Did the workflow satisfy the acceptance test? |
| Intervention rate | How often did a human need to rescue or redirect it? |
| Rework / regression rate | Did the “successful” change create follow-up work or break tests? |
| Cost and latency | Is the workflow economical enough to run at its intended frequency? |
| Unsafe-action rate | Did it attempt a forbidden or out-of-scope action? |
| Context/retrieval failures | Did it miss the right file, rule, skill, or tool before generation began? |
Keep the task set small and versioned. A five-case regression suite that runs on every rules, skill, hook, model, or plugin change is more useful than a large benchmark nobody reruns.
The postcondition is the ladder again: a check that sits at a layer text cannot argue with. Everything else in the loop exists so that check has something deterministic to grade.
A preflight for new capabilities
Section titled “A preflight for new capabilities”Before installing or enabling a component:
- What does it read, write, execute, and send over the network?
- Which product surfaces support it, and is it preview or stable?
- Can its permissions be narrowed or its version pinned?
- What happens if the model follows hostile text returned by the component?
- What deterministic check catches a bad result?
- Can the run be stopped, inspected, reverted, and attributed?
If the answers are unclear, keep the component out of unattended workflows until they are documented and tested.
How it works in each tool
Section titled “How it works in each tool”Showing: Claude Code pick a tab to change
Claude Code separates model guidance from runtime controls. CLAUDE.md, skills, MCP descriptions, and repository content influence the model, but permission rules decide whether a tool call may proceed. Project and user settings can allow, ask, or deny tools and command patterns. PreToolUse hooks can inspect a request before the permission prompt and block it, while deny rules still win over an attempted allow. The native sandbox adds filesystem and network limits for Bash; it complements permissions rather than replacing them.
For a safer evaluation loop, run the task in a disposable worktree, keep credentials out of the environment, and record the model, settings, hooks, tool calls, approvals, diff, and test results. Assert deterministic postconditions after the run. A green model explanation is not evidence that the task is correct. As of 2026-10.
Codex combines an approval policy with a sandbox posture. The approval setting controls when a person or policy must review a proposed action; the sandbox controls what the process can reach. Treat those as separate axes. A read-only or restricted workspace posture reduces blast radius, but it does not validate the patch or protect systems outside that boundary. Repository instructions, tool descriptions, MCP responses, and retrieved files remain model-visible inputs that can contain hostile text.
For evaluation, run codex exec in a disposable worktree with the narrowest useful approval and sandbox settings. Capture structured output where the installed version supports it, then run the repository’s tests and inspect the diff independently. Grade acceptance criteria and forbidden outcomes, not the model’s claim that it finished. Pin the installed CLI in automation because flag names and policy defaults can change. As of 2026-10.
OpenCode’s policy boundary is configured around tools and agents. On the 1.x line, the permission config maps each tool (read, edit, bash, task, skill, webfetch, …) to allow, ask, or deny, with pattern rules for tool input and an external_directory guard for paths outside the working directory. The model still sees instructions, repository files, tool output, and plugin-provided behavior, so permission rules are not a substitute for reviewing extensions or MCP servers. For untrusted work, put the whole process in a container or another host-level boundary.
Evaluate an OpenCode workflow as a small software system: use a disposable worktree, record the selected agent and model, capture tool and approval events, and run deterministic tests after the run. Check both the output and the absence of forbidden changes. OpenCode 2 (GA since September 2026) writes permissions as an ordered permissions array with renamed actions (shell, subagent) and allows external_directory by default, so confirm which line your install runs before copying a policy example from anywhere. As of 2026-10, OpenCode 1.19.4.
Cursor has several control surfaces rather than one universal safety switch. Approval and auto-run behavior, project permissions, hooks, and sandbox controls can each affect an action, while rules, prompts, extensions, and retrieved repository content influence the agent’s decision. Keep IDE-agent behavior separate from Cursor CLI behavior when writing policy. A plan or approval view is a workflow posture, not proof that every runtime path is blocked.
For evaluation, test the exact surface and version you deploy. Use a disposable worktree, minimal credentials, explicit repository tests, and a diff check that catches forbidden files or commands. Record approvals, hook results, tool calls, and the final artifact. If the agent can reach a production system, enforce the decisive boundary in the repository, CI, or service rather than relying on the model or UI. As of 2026-08-07.
Copilot’s controls vary by surface, especially between Copilot CLI, an IDE, and hosted agents. In Copilot CLI, approval prompts, permission flags, hooks, custom agents, and sandboxing can shape the run. Custom instructions and agent profiles influence behavior but do not make repository content trustworthy. Built-in and custom agents can also run in separate context windows, which isolates conversation noise, not necessarily filesystem or credential access.
For unattended work, choose the surface deliberately, restrict approvals and credentials, and use a sandbox or hosted runner where appropriate. Capture hook logs, tool calls, output, changed files, and test results. Evaluate with deterministic acceptance tests and forbidden-outcome checks. A threshold or setting you confirmed in the CLI does not carry over to VS Code or to hosted agents - check each surface separately before you rely on it. As of 2026-08-07.
Pi’s project trust controls whether project-local settings, resources, packages, and extensions load. It is not a sandbox and it does not restrict what the model can ask tools to do after startup. AGENTS.md and CLAUDE.md context files load regardless of project trust. Pi has no native OS sandbox, so untrusted repositories and unattended work need a container, VM, micro-VM, or policy-controlled runner with only the required files, credentials, and network access.
Evaluate the exact extension and package set you intend to run. Use a disposable copy, record trust decisions and loaded resources, inspect the diff, and run deterministic tests outside the model’s own narrative. Non-interactive modes do not show a trust prompt, so set the default project-trust policy deliberately in automation. As of 2026-10, Pi 1.4.0.
Comparison
Section titled “Comparison”| Question | Claude Code | Codex | OpenCode | Cursor | Copilot | Pi |
|---|---|---|---|---|---|---|
| What influences the agent? | Project context, tools, hooks, MCP, skills | Instructions, tools, profiles, MCP | Instructions, agent config, tools, plugins | Rules, prompts, tools, extensions | Instructions, tools, agents, MCP | Context files, trusted project resources, extensions |
| What gates actions? | Allow/ask/deny rules and hooks | Approval policy and sandbox mode | Per-tool and per-agent permissions | Approval, permissions, and sandbox controls | Approval settings, permissions, hooks | External containment and extension/tool policy |
| What is the hard boundary? | OS sandbox, workspace, or managed policy | Sandbox/worktree/container boundary | The host or external sandbox | Host/CLI sandbox and repository controls | Local sandbox (/sandbox enable), cloud sandbox, or the cloud agent’s Actions runner |
No native sandbox; use a container or VM |
| What should evaluation assert? | Tests, hooks, trace, diff | Tests, approval trace, output, diff | Tests, tool policy, output | Tests, diff, policy outcome | Tests, hook logs, output | Tests, container logs, diff |
Name collisions
Section titled “Name collisions”- Trust is not a single toggle. A trusted repository, a trusted tool, and a trusted action are different decisions.
- Evaluation is not “the model says it succeeded.” It is a testable postcondition plus evidence from the run.
- Sandboxing limits impact; it does not prove correctness. Pair it with acceptance tests and output validation.
The ladder runs both directions. Climbing it costs setup and maintenance: a sandbox to configure, hooks to write, approvals that slow the loop down, a regression suite to keep versioned. That cost is the sequel, and it is paid per workflow, not once. Start with the smallest unattended run you actually have - a headless job, a scheduled sweep - and give it one input boundary, one action gate, and one deterministic postcondition. That trio is the preflight at the lowest layer; everything above is the same trio, stronger.