Notes from building agent systems
Evaluation harnesses, orchestration patterns, memory design and the failures behind them — written up while the details are still fresh.
Verification Before Action: Ordering Controls for Autonomous Agents
Execution verification only pays off when it sits between plan and side effect, as a synchronous dependency of the tool call rather than an asynchronous audit stream.
Agent Identity Governance Belongs in the Directory, Not in the Agent Framework
Agent permissions defined inside frameworks are invisible to access reviews, survive retirement, and sprawl. Directory-native identity fixes the lifecycle.
Runtime Governance Needs Enforcement Points, Not Policy Documents
Governance that records agent behaviour after the fact does not constrain it. The useful layer is a set of synchronous enforcement points that block or reshape a tool call before it executes.
Pre-Deployment Validation for Agents Is a Fault-Injection Problem
A clean trace only proves the agent handled the cases you ran. Readiness means injecting the failures production will produce and watching the recovery path.
An Agent That Reaches a Government Portal Is a Routing Failure
Unauthorised reach is settled at the egress chokepoint, not in the prompt. Every agent fleet needs one auditable network boundary whose denials are first-class events.
Leaked Output Is a Provenance Failure, Not a Model Alignment Failure
When an agent publishes user artefacts it should never have exported, the defect is missing provenance and taint tracking on data flowing into generated output.
The Agent Sandbox Is a Capability Question Before It Is a Container Question
Containment is less about the isolation technology and more about enumerating, bounding and revoking the capabilities an agent holds at each step of a run.
Runtime Intent Beats Static Allowlists for Agent Egress
Static allowlists cannot keep up with composed agent tool calls. Runtime intent inspection catches what destination filtering misses, if the check stays cheap enough for the hot path.
Sandbox Escape Is a Capability Budget Problem
Containment failures in agent systems are mispriced capability grants, not missing jailbreaks. The control surface is a per-run budget on reachable systems.