Provable Control: Moving Governance From Dashboards to Enforced Invariants

The limits of watching

Agent systems are usually governed by watching them. Traces are emitted, dashboards are built, thresholds are set, and an on-call rotation is established. This is useful, but it is a different property from control. A dashboard observes a completed action. By the time a panel turns red, the tool call has already executed, the write has already landed, and the external side effect has already occurred. The best a monitoring stack can do is shorten the interval between damage and discovery.

Watching also depends on the telemetry being complete. If a tool wrapper forgets to emit an event, if a retry path bypasses the instrumented client, or if a new integration ships without the tracing decorator, the dashboard is silent for exactly the actions that are least understood. Coverage gaps in telemetry are not evenly distributed; they cluster around the newest and least reviewed code paths, which is where incidents concentrate.

A monitored agent with broad permissions is still a broad-permission agent. Observability does not reduce the set of reachable states. It adds a camera to a room that remains unlocked. The permission model is the thing that determines what is possible; the dashboard only determines what is noticed.

Alerting thresholds introduce a further distortion. Thresholds are tuned to reduce noise, which means they are tuned to tolerate a certain rate of low-level anomalies. That is a reasonable operational compromise for detection, but it is the wrong instrument for prevention. A policy that fires when a sensitive operation occurs more than a configured number of times per interval is a policy that permits the first several occurrences. Prevention requires a decision before the first occurrence, not a rate limit after it.

The practical consequence is that governance expressed only as monitoring tends to converge on the same failure mode: the system is well understood in aggregate and unconstrained in the specific. The aggregate view is genuinely valuable for capacity, cost, and quality analysis. It is not a control surface.

Encoding invariants

The alternative is to express policy as preconditions that are checked at the boundary where the agent's intent becomes an action: the tool call. Every capability the agent has is reachable through some tool interface, and that interface is the narrowest point at which a decision can be enforced. If the check happens there, it happens for every caller, including the ones that were added after the policy was written.

A precondition is a predicate over the proposed call and the surrounding context. It can reference the arguments, the identity of the requesting agent, the session, the resource being touched, and the state of the environment. Examples of the shape, not the specific content, include: the target resource must be inside an allowlisted namespace; the operation must be a read when the session has not been elevated; the payload must not contain a field marked as restricted; the call must be accompanied by a valid approval token issued by a separate workflow.

Typed schemas make this enforceable rather than aspirational. If the tool's input is described by a schema with required fields, enumerated values, and bounded types, then a malformed call fails at parse time, before any handler runs. Fail-closed behaviour follows from the schema rather than from defensive code inside the handler. An agent that produces a plausible-looking but out-of-policy argument does not reach the side effect; it receives a structured rejection and must decide what to do next.

A concrete mechanism looks like this. The agent runtime does not call tools directly. It calls a policy gateway with a structured request:

{
  "agent": "research-worker-3",
  "session": "s-8841",
  "tool": "documents.write",
  "arguments": {"collection": "contracts", "document_id": "c-2210", "body": "..."},
  "context": {"approval": null, "elevated": false}
}

The gateway evaluates a policy bundle, which is a versioned set of rules with a deterministic evaluator. The evaluation returns a decision, a reason code, and the policy version. Only on an allow does the gateway invoke the underlying tool. Denials are returned to the agent as ordinary tool results, so the agent can adapt without the runtime needing to special-case error handling.

Two properties matter here. First, the policy engine sits outside the model's context. It is not a prompt, not a system message, and not a document the agent can retrieve. It cannot be argued with, summarised away, or superseded by a later instruction. The model's only influence over the decision is the content of the request it submits. Second, the decision is deterministic given the request and the policy version. The same call evaluated twice against the same bundle produces the same result, which is what makes the decision explainable after the fact.

The honest trade-off is that this moves complexity rather than removing it. Every new capability requires a policy entry, and every policy change is a change to production behaviour that needs review. A gateway in the request path adds latency and a new failure mode: if the gateway is unavailable, the correct default is to deny, which means an outage in the policy layer becomes an outage in the agent's capabilities. Teams that adopt this pattern generally accept that cost because the alternative is an outage that is silent and unbounded. The trade is availability of capability for boundedness of capability.

There is a second trade-off worth naming. Strict schemas and narrow allowlists reduce the space of actions an agent can take, which can reduce its usefulness on tasks that were not anticipated. The mitigation is not to loosen the boundary but to make policy changes fast and reviewable, so that the boundary can move deliberately rather than being bypassed.

Evidence for auditors

A control that cannot be evidenced is a control that will be questioned. Each policy decision, allow and deny alike, should emit a record that is sufficient to reconstruct the reasoning without access to the live system. That record includes the request, the decision, the reason code, the policy version, and a signature over the canonical serialisation of the decision.

Signing matters for a specific reason: it makes the record tamper-evident and attributable to the enforcement point rather than to the logging pipeline. A log line can be edited, dropped, or reordered. A signed decision record can be verified independently, and a gap in the sequence is detectable.

Retaining the policy version alongside the decision is what makes past behaviour re-explainable. Policies change. A decision that was correct under one bundle may be incorrect under the next. Without the version, an auditor reviewing a historical action has to guess which rules were in force. With the version, the same evaluator can be replayed against the recorded request and reproduce the decision exactly. This turns an audit from an interview into a verification.

Denials deserve the same retention as allows. A denial is evidence that the boundary worked. It is also the primary signal for policy tuning: a deny that fires constantly against legitimate work indicates a rule that is too broad, and a deny that never fires in a class of sessions may indicate a rule that is unreachable. Retaining both sides makes the policy surface observable without making it advisory.

The final piece is testing policy changes against recorded traces before deployment. If decisions are recorded with their requests, then a candidate policy bundle can be evaluated offline against a corpus of real historical calls. The output is a diff: which previously allowed calls would now be denied, and which previously denied calls would now be allowed. That diff is reviewable by the same people who review the policy, and it converts a policy change from an act of judgement into an act of measurement. It also surfaces the cases nobody anticipated, which is where policy regressions usually live.

This is the practical difference between observability and governance. Observability produces a record of what happened. Governance produces a boundary that constrains what can happen, plus a record of the boundary's decisions that can be independently verified. The first is necessary for operating a system. The second is necessary for delegating authority to it.

The direction of travel in the field reflects this. Recent coverage of agent governance and control frameworks continues to emphasise enforcement points and auditability over retrospective analysis, which is consistent with what the failure modes look like in practice. The reporting here is a useful entry point to that discussion.

What this implies for system design

Treat the tool boundary as the control plane. Put the policy evaluator outside the model's reach. Make the schema the first line of defence and let malformed calls fail closed. Emit signed decisions for both outcomes and keep the policy version attached. Replay candidate policies against recorded traffic before they reach production.

None of this replaces observability. Traces, metrics, and dashboards remain how a system is understood and improved. What changes is the ordering: the boundary decides, the record proves, and the dashboard explains. A system governed this way does not depend on the model choosing correctly. It depends on the model being unable to choose otherwise.

---

Cover photo: Raul Ling / Pexels.