Skip to main content
Blog

Notes from building agent systems

Evaluation harnesses, orchestration patterns, memory design and the failures behind them — written up while the details are still fresh.

All postsllm-agents 40agent-security 24multi-agent-systems 17evaluation 12agent-architecture 10agent-evaluation 10agent-safety 10authorisation 9policy-enforcement 9idempotency 8llm-evaluation 8provenance 8
Detailed image of microchip components viewed through a microscope lens.
22 Sept 2026·6 min read

Judging the Judge: When a Small Model Grades a Large One

Model-as-judge pipelines save cost, but the judge is a measurement instrument that needs calibration, drift monitoring and adversarial testing.

model-as-judgeevaluationcalibration
Minimalist image of a robotic hand reaching out on a white background.
20 Sept 2026·6 min read

The Chief-of-Staff Pattern: When to Orchestrate Agents and When to Just Write a Script

Multi-agent orchestration pays off only when subtasks need different context, different tools, or independent verification. Otherwise it is a slower function call.

multi-agent-systemsorchestrationllm-agents
Elegant neoclassical facade with columns and flags in downtown Chicago.
20 Sept 2026·5 min read

Memory Is a Write-Policy Problem: Retention, Provenance and the Cost of Remembering

Most agent memory failures originate at write time, not retrieval. Provenance, sensitivity classes and schema-level expiry fix them earlier and cheaper.

agent-memorywrite-policyprovenance