Notes from building agent systems
Evaluation harnesses, orchestration patterns, memory design and the failures behind them — written up while the details are still fresh.
Provable Control: Moving Governance From Dashboards to Enforced Invariants
Observability records what an agent did. Governance requires constraints that make certain actions impossible regardless of what the model decides.
Small Models at the Edge: Deciding What Leaves the Device
On-device inference is a data-egress decision as much as a latency decision. Draw the local/remote boundary by data class, not by model confidence.
Agent Failure Taxonomy: Naming the Ways Multi-Step Systems Break
A shared vocabulary for agent failure modes turns incident reviews from anecdote into countable, regression-testable signals across model, tool, orchestration and data layers.
Coding Agents Need a Sandbox Contract, Not a Warning Label
An autonomous coding agent's blast radius is defined by its sandbox and network egress rules. Specify that contract before you specify the agent.
Unit Economics of an Agent Run: Cost Accounting for Multi-Step Systems
Token pricing hides where agent spend actually goes. Cost per successful task, instrumented by role and tool call, is the number teams can act on.
Evaluating Agents on Trajectories, Not Answers: A Benchmark Design Note
Benchmarks that score only final outputs reward agents that reach correct answers through unsafe or unreproducible paths. Trajectory-aware grading fixes this.
The Chief-of-Staff Pattern: When to Orchestrate Agents and When to Just Write a Script
Multi-agent orchestration pays off only when subtasks need different context, different tools, or independent verification. Otherwise it is a slower function call.
Memory Is a Write-Policy Problem: Retention, Provenance and the Cost of Remembering
Most agent memory failures originate at write time, not retrieval. Provenance, sensitivity classes and schema-level expiry fix them earlier and cheaper.
Authority Budgets: The Missing Control Surface Between an Agent and Its Tools
Prompt constraints are advisory. Real safety comes from treating each capability as a spendable authority budget: scope, quota, expiry, and attestation.