Notes from building agent systems
Evaluation harnesses, orchestration patterns, memory design and the failures behind them — written up while the details are still fresh.
Agent Commerce Is a Delegation Grammar Problem, Not a Storefront Problem
Walmart and Amazon are not arguing about bots. They are giving different answers to how a shopper's mandate is expressed, bounded and proved at checkout.
Decision Models Compress Policy You Already Wrote; They Do Not Write It
Small decision models are a latency and cost optimisation over an existing policy surface. Treating them as the place where policy is defined is a category error with review, composition and safety consequences.
The Judge Is a Component: Versioning LLM-as-Judge Pipelines Like Any Other Dependency
LLM-as-judge is trustworthy only when the judge model, prompt, rubric and decoding parameters are pinned and regression-tested as a first-class artefact.
Durable Execution Is the Real Infrastructure Requirement for Long-Running Agents
Funding and product moves around durable execution point at the actual hard problem in agent infrastructure: deterministic replay and idempotent effects for runs that span hours and external side effects.
Refusal Is a Routing Decision, Not a Model Property
Agent overreach is usually a missing authorisation decision upstream of the model. Safety belongs in the request path, not only in alignment training.
Agent Swarms Need a Discovery Contract, Not a Shared Database
At swarm scale, the hard problem is agreeing on how facts are named, versioned and retired. A discovery contract must precede any shared store.
Benchmark Contamination Is a Data Pipeline Bug: Versioning Eval Sets Like Code
Agent evaluation failures blamed on model weakness are often eval-set hygiene failures. Eval corpora need the same provenance, versioning and access controls as training data.
The Human Concierge Is a Fallback Path, Not a Feature
Routing agent failures to a human only works when the escalation contract is explicit: what state is handed over, what the operator may change, and how the correction returns to the agent.
Long-Horizon Processing Turns Context Management Into a Scheduling Problem
A larger context window does not remove the limit; it turns context assembly into a scheduling decision about what to load, when, and at what cost.