The Agent Stack — How I Build Agents
My opinionated answer to the question builders ask me most. This is a living document — I update it as the field moves. Every choice has a one-line opinion, and the opinions are the point.
Orchestration
LangGraph
Explicit graph-based control flow for agents that have real branching, loops, and states.
opinionExplicit graphs beat implicit chains; when your agent has a real control flow, model it as a graph, not a prompt.
Temporal
Durable, resumable workflow execution for long-running agent tasks.
opinionAgent work is distributed-systems work; if a step can crash, make the workflow durable and resumable.
Ray
Distributed compute and serving for parallel agent execution.
opinionSwarm-scale parallel execution is a compute problem first; reach for Ray before you reach for more cleverness.
K8s / Fly.io
Running agents where they can scale and die cleanly.
opinionRun agents where they can scale and die cleanly; orchestration that can't restart a worker isn't production.
Tool use & function calling
Function/tool calling + structured outputs (JSON schema)
Typed tool signatures with JSON-schema-enforced outputs.
opinionGive the model a contract, not prose; the whole agent is only as reliable as its tools' signatures.
First-class action API
A macro/actions API for the environment the agent acts in (the pattern from vision-based agent work).
opinionTo act in a world, you need a first-class action API — give the agent real handles and a verifier, in every domain.
MCP — Model Context Protocol
A standardized bus connecting models to tools and data sources.
opinionStandardize how the model reaches the world; a common tool bus beats twenty bespoke connectors.
Memory & context
Short-term context + vector retrieval (RAG)
Persisting what the agent needs to know across turns, retrieved from a knowledge base.
opinionPersist what the agent needs to know; a stateless agent is a replay-loop, not a worker.
Scratchpad & working memory
Structured intermediate state for multi-step planning and tool results.
opinionDon't cram everything into the window; keep a working memory the agent reads and writes deliberately.
Evaluation & observability
Environment-grounded verifiers
Sandboxes, simulators, and games that grade the agent in the world it acts in.
opinionGrade the agent in the world it acts in, not against vibes. Evaluation is an environment problem, not a rubric problem.
LLM-as-judge (sparingly, audited)
Model-based scoring, double-checked with deterministic checks.
opinionJudges are baselines, not truth; double-check them with deterministic checks.
Failure taxonomies + a benchmark you own
A classified catalog of failures (wrong command, missing step, logic error) and a benchmark you maintain.
opinionShip your failure modes in a taxonomy — that's how hard bugs become debuggable, and how you learn from every one of them. A reproducible failure is a research result.
Cost/latency budgets as first-class metrics
Track spend and latency per run like any other metric.
opinionA working run at a sane cost beats a prettier one at ten times the price.
Traces (OpenTelemetry / LangSmith-style)
Step-level traces and replay of full agent runs.
opinionYou cannot tune what you cannot replay; traces are the agent's flight recorder.
Dashboards for success-rate, cost, latency, failure distribution
SLO-style dashboards for agent health.
opinionRun it like an SLO, or you're running it like a demo. Confidence lives in the transcripts, not the summary.
Deployment
Containers + immutable deploys
Ship agents as containers with versioned, reproducible artifacts.
opinionIf you can't rebuild it from the same artifact twice, you don't have a deploy — you're guessing.
CI/CD with eval gates
Runs the agent's evaluation suite before promoting a new version to production.
opinionA model upgrade is a regression risk, not a feature; gate every release on the eval suite, not on intuition.
Managed serverless / container runtime
Scaling agents to zero and back without babysitting infrastructure.
opinionAgents are bursty by nature — deploy where they can scale to zero and come back fast.
Straight answers.
What is an AI swarm?+
An AI swarm is a coordinated system of multiple independent agents that share goals, decompose tasks, and negotiate solutions together. Unlike a single agent — one loop with one bottleneck and one point of failure — a swarm gets resilience and scale from redundancy and division of labor: parallel exploration, no single point of failure, and answers that emerge from collective reasoning rather than one model's guess. My thesis: intelligence emerges from structured interaction among many agents, not from a single bigger brain.
How do you evaluate multi-agent systems?+
Evaluation is the bottleneck, not the model. I grade agents in the environment they act in — sandboxes, simulators, games — not against intuition. I use LLM-as-judge sparingly and audited, backed by deterministic checks; I ship failure taxonomies plus a benchmark I own; and I treat cost and latency as first-class metrics. A feedback loop is only as smart as its verifier.
When should you use agents vs a single model call?+
Use a single model call for one-shot transformations with no state, no tools, and nothing to verify. Reach for an agent when the task needs multi-step planning, tool calls into a world, memory across turns, or verification that the work actually happened. And if you don't have a verifier, don't build an agent yet.