Notes from building agent systems
Evaluation harnesses, orchestration patterns, memory design and the failures behind them — written up while the details are still fresh.
Provable Control: Moving Governance From Dashboards to Enforced Invariants
Observability records what an agent did. Governance requires constraints that make certain actions impossible regardless of what the model decides.
Evaluating Agents on Trajectories, Not Answers: A Benchmark Design Note
Benchmarks that score only final outputs reward agents that reach correct answers through unsafe or unreproducible paths. Trajectory-aware grading fixes this.
The Chief-of-Staff Pattern: When to Orchestrate Agents and When to Just Write a Script
Multi-agent orchestration pays off only when subtasks need different context, different tools, or independent verification. Otherwise it is a slower function call.
Authority Budgets: The Missing Control Surface Between an Agent and Its Tools
Prompt constraints are advisory. Real safety comes from treating each capability as a spendable authority budget: scope, quota, expiry, and attestation.