The run governor

01 — The problem

Every piece already existed. The one that mattered didn’t.

The modern agent stack has runtimes that drive agents, orchestrators that queue work, and billing systems that meter usage. What existed nowhere was a component that could answer two questions:

• Did the agent actually do the work it claims it did?
• What did that cost, and was it allowed to spend it?

Most systems that come close are broken in the same direction — they report spend after the fact rather than bounding it beforehand. And they take the agent’s word for whether the work was done.

AOS is the component that answers both questions. It sits between you and an autonomous agent: it decides what the agent may run as, bounds what it may spend, proves whether it did the work, and undoes it if it didn’t. That is why we call it an operating system rather than a harness — a harness runs the agent; an OS decides what the agent may run as.

02 — How it all works

Nine steps, and the last one feeds the first.

This is the operating system. Everything else on this page is detail on one of these steps. A RunSpec goes in — a task type, a goal, a model, a budget — and what comes out is not just an answer but a record: what it cost, whether it was verified, and what the agent is therefore allowed to do next.

flowchart TB
  SPEC["RunSpec
taskType · goal · model · budget"]

  SPEC --> SEL["1 · SELECT
which model, and how hard should it think?
complexity score → tier, then FLOOR, BUDGET, TRUST override it"]
  SEL --> GUARD{"2 · GUARD
is this model even up?"}
  GUARD -->|"circuit open"| FREE["ABORTED — for free
don't reserve hope"]
  GUARD -->|"closed"| RES{"3 · RESERVE
take a HOLD, not a check"}
  RES -->|"balance cannot cover"| DEN["DENIED
nothing spent"]
  RES -->|"Lease"| CP["4 · CHECKPOINT
snapshot the workspace BEFORE the agent touches it
cannot be undone? say so NOW, not afterwards"]
  CP --> RUN["5 · RUN
the agent works
every call DRAWS · sub-agents SUB-LEASE
tool calls are WATCHED · a transient failure retries under the SAME lease"]
  RUN -->|"budget exhausted"| STOP["ABORTED
the backstop that should never fire"]
  RUN -->|"claim + observed artifacts"| VER{"6 · VERIFY
something that did not do the work
decides whether it is done"}
  VER -->|"refuted"| REJ["REJECTED"]
  REJ --> CON["7 · CONTAIN
roll the workspace back
recoverably: a tag, and a stash"]
  STOP --> CON
  VER -->|"survives"| OK["SETTLED"]

  CON --> SET["8 · SETTLE
charge the ACTUAL cost, release the rest
on every path — rejected and aborted too"]
  OK --> SET
  SET --> REC["9 · RECORD
append-only: cost, verdict, routing, rollback"]
  REC --> TRUST["the TRUST LEDGER"]
  TRUST -.->|"gates"| SCHED["the SCHEDULER
what may run unattended tomorrow"]
  TRUST -.->|"pins a proven model"| SEL

  class SEL,RUN,VER,SET,CON core
  class FREE,DEN,STOP,REJ money
        

Read it as one sentence

select a model the budget can actually fund → guard against a provider that is already down → reserve real money → run, drawing down the lease on every call and retrying transient failures inside it → verify against artifacts we observed rather than claims it made → settle at the true cost whatever the outcome → record it, so that twenty verified passes buy an agent the right to run at 4am without a human awake.

03 — The one rule

The core does not know where it is running.

The governor’s core does not know that Postgres, Stripe, or any tenant exists. It talks to its ports and nothing else. This is not stylistic — it is what lets the same governor run in three very different homes.

The rule held from the first commit to the last. The tenant identifier is an option on the platform budget adapter and appears nowhere else — which is why the same core still runs on a laptop with no network.

04 — Component map

One governor, six ports, many homes.

flowchart TB
  SCHED["Scheduler
cron + trust gate"] --> GOV
  HOST["Host application
work queue"] --> GR["GovernedRunner
opt-in decorator"] --> GOV
  GOV["Governor
select → guard → reserve → run → verify → settle"]

  GOV --> MP{{"ModelSelectorPort"}}
  GOV --> RTY["RetryPolicy
policy, not a port"]
  GOV --> BP{{"BudgetPort"}}
  GOV --> RP{{"RuntimePort"}}
  GOV --> VP{{"VerifierPort"}}
  GOV --> LP{{"LedgerPort"}}

  MP --> CX["ComplexityModelSelector"]
  CX -.injected.-> LR["complexity scorer
scores, never names a model"]

  RTY --> CB["CircuitBreaker
production-tuned params — overloads open it faster"]

  BP --> MEM["Memory"]
  BP --> SQL["SQLite"]
  BP --> BER["PlatformBudget"]
  BER -.HTTP.-> US["billing service
Postgres"]

  RP --> CC["agent runtime
the loop"]

  VP --> COMP["CompositeVerifier"]
  COMP --> S1["Sentinel
free"]
  COMP --> S2["Deterministic
free"]
  COMP --> S3["Model judge
costs money"]

  LP --> LSQL["SQLite / Postgres
append-only"]
  LSQL -.->|"earned trust"| CX

  class GOV core
  class BER,US money
        

05 — The runtime

AOS drives its own agents.

The runtime — the agent loop, the Anthropic and OpenAI-compatible providers, the cost table, the tools, the permission model — was adapted from a proven interactive coding agent and brought in-house, so AOS runs with no external checkout and no runtime dependency on its ancestors.

What was deliberately NOT taken

An interactive tool with a human watching can afford things an unattended, budgeted run cannot. Each omission below is a security or budget decision, not a missing feature.

Prompts have dependencies too

The original system prompt told the model to track progress with a todo tool, delegate with a sub-agent tool, and recall facts with a memory tool — three tools AOS does not register. Copied verbatim it would instruct the model to reach for tools that are not there: a wasted turn, an unknown tool error, and on a weaker model a derailed run.

06 — The ports

A hold, not a check.

The classic budget bug: a preflight that reads the balance and leaves it untouched — so N concurrent callers all pass at the same number and the cap is advisory. A reservation removes the money from the available balance immediately. It comes back only via settle (at the real cost) or expire.

flowchart LR
  A["reserve(amount)"] -->|"hold taken"| B["Lease"]
  A -->|"balance cannot cover"| N["null → DENIED"]
  B --> C["sublease(parent)"]
  B --> D["draw(amount)"]
  D -->|"exhausted"| E["abort the run"]
  B --> F["settle(actual)"]
  F --> G["released → back to the balance"]
  B --> H["expire() — the process died"]
  H --> G
        

Why a lease and not a per-call debit

A per-call check → call → debit cycle works for a single-shot gateway. An agent session is an unbounded loop of calls plus fan-out to sub-agents, and it does not know its own cost in advance.

The six

RetryPolicy and CircuitBreaker are concrete policy objects, not ports — there is nothing about “should I retry an overloaded provider” that varies by environment.

07 — Selection

The scorer scores. The governor decides.

The complexity scorer AOS consumes is pure — no database, no network — and it deliberately emits a tier while refusing to name a model. That is exactly the right seam to cut on. AOS consumes the tier and owns the ladder, because the three things that should be able to override a routing decision are three things a scorer has no concept of: a floor, a budget, and earned trust.

flowchart TB
  G["goal: “refactor the auth module…”"] --> SC["complexity scorer
pure function → 0.0–1.0"]
  SC --> T["tier
simple · standard · complex · expert"]

  T --> F{"FLOOR
default: standard"}
  F -->|"scored below it"| FU["raised
a terse ticket is not a small job"]
  F -->|"at or above"| OK1["kept"]

  FU --> B{"BUDGET"}
  OK1 --> B
  B -->|"can't fund a real run at this rung"| BD["stepped DOWN
$0.20 on a frontier model buys 8k tokens
— that is a truncated sentence and a bill"]
  B -->|"affordable"| OK2["kept"]

  BD --> TR{"EARNED TRUST"}
  OK2 --> TR
  TR -->|"this (taskType, model) has 20 verified passes"| PIN["PINNED
swapping the model would reset its record to zero"]
  TR -->|"unproven"| OK3["kept"]

  PIN --> OUT["model + effort
with a reason you can read in the ledger"]
  OK3 --> OUT

  class OUT core
  class BD,PIN money
        

The ladder moves effort, not just the model

This is the lever people forget. A better model at low effort often beats a worse one at max — so the cheap move is frequently lower effort on a better model, not a worse model. A ladder that only swaps models leaves most of the saving on the table, and usually the capability too.

The calibration finding

Measured against the real scorer, not a mock:

It is heavily length-biased. Shipping routing on it as-is would have sent real engineering to the cheapest model because somebody wrote a terse ticket.

Hence the floor. A borrowed scorer may route work up, never below the rung you already know the job needs. Characterization tests run against the real scorer and pin this behaviour — so the mismatch is documented by something that executes, and if the scorer is ever retuned they fail and somebody has to think.

Why trust pins the model

The subtlety, and it took connecting the two systems to see it. Trust is keyed on (taskType, model) — evidence about one model says nothing about another, so changing the model resets the record to zero.

So once a (taskType, model) pair has earned trust, the selector pins it. Explore on the unproven; exploit the proven.

08 — Resilience

A production circuit breaker. Bounded by the lease.

Before this layer, any runtime error aborted the run and charged for the tokens already burned. A transient provider overload meant: run dead, money spent, nothing delivered.

The circuit breaker

That last row is why this code was worth taking from a system that had shipped through real outages rather than writing fresh. It is the kind of parameter you only arrive at by having been on the wrong end of one.

stateDiagram-v2
  [*] --> closed
  closed --> open : 3 failures — OR JUST 2 OVERLOADS
  open --> open : available() is false · the run aborts FREE
  open --> half_open : 30s elapsed
  half_open --> closed : the probe succeeded — the provider is back
  half_open --> open : the probe failed — another full timeout
  half_open --> half_open : a second caller is refused — ONE probe, never a herd
        

What AOS adds: retries bounded by the lease

Retry logic built for single-shot API calls tends to retry generously — ten times, a minute apart, with no ceiling on cost. That is perfectly reasonable when each attempt costs a fraction of a cent.

flowchart LR
  L["Lease · $10 hold"] --> A1["attempt 1
burns $4, then an overload error"]
  A1 -->|"$4 already drawn"| C{"classify"}
  C -->|"transient"| A2["attempt 2
draws from the $6 that is LEFT"]
  A2 -->|"burns $6 → exhausted"| X["ABORT
the budget ran out before the retries did"]
  C -->|"permanent / refusal"| P["ABORT
it would fail identically, at the same price"]
  A2 -->|"succeeds"| V["→ VERIFY"]

  class L core
  class X,P money
        

Two independent bounds, and whichever binds first wins. A test found that on repeated overloads the circuit binds before the budget does — which is the better of the two outcomes: money left in the account, and no further load thrown at a provider that is already struggling.

Classification is the whole game

And a budget abort is not a provider failure. It must not be retried (there is nothing left to retry with) and must not trip the breaker (the model did nothing wrong).

09 — Run lifecycle

Every run that ends, settles.

stateDiagram-v2
  direction LR
  [*] --> REQUESTED
  REQUESTED --> SELECTING: pick model + effort
  SELECTING --> ABORTED: circuit open — FREE, nothing reserved
  SELECTING --> RESERVING: estimate cost
  RESERVING --> DENIED: balance cannot cover — nothing spent
  RESERVING --> RUNNING: lease acquired
  RUNNING --> RUNNING: transient error — retry UNDER THE SAME LEASE
  RUNNING --> ABORTED: budget exhausted
  RUNNING --> ABORTED: unpriceable model
  RUNNING --> ABORTED: permanent error / refusal
  RUNNING --> VERIFYING: agent claims done
  VERIFYING --> SETTLED: verifier could not refute
  VERIFYING --> REJECTED: verifier refuted
  DENIED --> [*]
  ABORTED --> [*]
  SETTLED --> [*]
  REJECTED --> [*]
        

ABORTED (budget) is a normal outcome, not an error. It is the system working — and it is recorded as passed: null, never as a failure, so a tight budget cannot quietly destroy an agent’s reputation.

Note the two free exits. DENIED spends nothing because the hold was never granted. ABORTED (circuit open) spends nothing because we did not start a run against a provider we already knew was down. Everything past RESERVING costs money, and therefore settles.

The self-transition on RUNNING is the whole retry story: an attempt that failed transiently does not get a new budget. It gets what is left of the old one.

10 — Budget

Three layers. Any one alone has a hole.

flowchart TB
  subgraph L1["① DESIGN — prompt caching"]
    C1["reads ~0.1× · writes 1.25–2×
catches: paying full price to re-read the same context"]
  end
  subgraph L2["② IN-MODEL — task_budget"]
    C2["a countdown the model sees WHILE generating
catches: overrunning because it didn't know it was near the limit"]
  end
  subgraph L3["③ HARD STOP — the lease"]
    C3["draw() → exhausted → abort
catches: everything else"]
  end
  L1 --> L2 --> L3
  L3 --> X["The backstop that should never fire"]
        

The middle layer is the one everyone skips, and it is the interesting one. A hard stop alone kills the run at a random point: you pay for the work and get nothing. The countdown lets the model prioritise and land the plane.

11 — The lease tree

One greedy sub-agent cannot starve its siblings.

A run that spawns sub-agents does not share one flat hold. Each child gets a carve-out from the parent’s unspent, un-subleased remainder.

flowchart TB
  R["root lease · $10
own_drawn $2 · subleased $6
headroom = 10 − 2 − 6 = $2"]
  R --> A["child A · $3
tried to draw $9 — got $3"]
  R --> B["child B · $3
drew $1, settled"]
  B -.->|"$2 returned to parent's headroom"| R
  A -.->|"cannot touch B's allowance"| A
        

With one flat hold, the first greedy sub-agent drains the whole budget and its siblings get nothing. The run then fails — not because the work was too expensive, but because one branch of it was.

Invariants — enforced by the database, not just the code

12 — Verification

Cheapest first. And it is an AND.

flowchart LR
  IN["claim + observed artifacts"] --> S["Sentinel
free · regexes"]
  S -->|"alert"| F["REJECTED"]
  S -->|"clean"| D["Deterministic
free · a script"]
  D -->|"red build"| F
  D -->|"green"| M["Model judge
costs money · asked to REFUTE"]
  M -->|"refuted, or unsure"| F
  M -->|"could not refute"| P["SETTLED"]
        

All must pass. A composite that passed when any member passed would be a machine for finding the one verifier that happened to be feeling generous. It refuses to be constructed empty, because an empty composite passes everything.

The judge is asked to refute, not to review

• Fresh context. It never sees the run’s conversation. A verifier that has read the agent’s reasoning is halfway to being persuaded by it.
• Refute, don’t review. “Check this work” invites agreement — it reads as a request for confirmation, and models are obliging. “Find the reason this is wrong” inverts the burden of proof.
• Uncertainty fails. "Looks good to me!" is a vibe, not a verdict, and a test asserts it does not pass. A verifier that fails open is a rubber stamp with extra steps and a bill attached.

Artifacts are observed, never self-reported

sequenceDiagram
  participant A as Agent
  participant H as Harness
  participant V as Verifier
  A->>H: Write(src/health.ts)
  H->>H: record artifact ✓
  A->>H: Bash(npm test)
  H->>H: record artifact ✓
  A->>H: "Added the endpoint. I also wrote
src/router.ts and migrations/007.sql."
  Note over H: those two were never written.
They are simply ABSENT from the evidence.
  H->>V: goal + [src/health.ts, npm test]
  V-->>H: verdict
        

Reads and greps are not artifacts. An agent that read forty files and wrote none has produced nothing to verify.

13 — Containment

A verdict with no containment is a logging feature.

The governor could establish, with a fresh-context judge and unbribable shell checks, that a run was wrong — and then leave the wrong work sitting in the repository. The money unwound perfectly. The damage did not. Everything upstream of the verdict was worth nothing, because the next thing to touch that repo — a human, a build, the next scheduled run — inherited the broken work.

flowchart TB
  V{"verdict"} -->|"settled"| KEEP["KEEP THE WORK
rolling back a passing run would make AOS
an elaborate way to do nothing"]
  V -->|"rejected / aborted"| T["1 · TAG
discarded COMMITS stay reachable"]
  T --> S["2 · STASH -u
a tag does NOT cover untracked files.
An agent that wrote a new file and never committed it
has produced work the tag cannot see."]
  S --> R["3 · RESET --hard
…and only now"]
  R --> OUT["workspace restored
+ exactly how to get the discarded work back"]

  class KEEP,OUT core
  class T,S money
        

A failed rollback is LOUD. Recording a run as contained while its changes are still on disk means the ledger is lying, and everything downstream inherits the lie. A rollback that throws does not swallow the verdict either — the verdict is the valuable thing.

The bug found while demonstrating it

The state directory is now git-ignored at creation: stash -u takes untracked files, not ignored ones, and reset --hard does not touch them. The category of bug — a tool operating on a workspace that contains the tool’s own state — is one you will meet again, and you will only ever catch it by running the thing for real.

14 — Using it

A governor you cannot invoke is a library with tests.

What matters architecturally is what both front doors refuse.

They refuse to guess a budget

They refuse to run something that cannot be verified

The verifier correctly fails a claim with no evidence. So a run with no check and no judge is decided before it starts: it would reserve the budget, drive the agent, spend the money, and only then be rejected — for a reason knowable before the first token. Both front doors stop it at zero cost.

Tier 1 is not allowed to be vacuous

Requiring a hand-written --check for every repository means, in practice, that most runs get no unbribable check at all — and verification collapses onto the model judge, which is the exact “talked into it” failure the design exists to avoid. So AOS detects the repo’s own commands, or reads a small toolchain config if you have one.

And one more rule: never emit a check that can fail for reasons unrelated to the agent’s work. npm test with no test script exits non-zero; pytest with no tests exits 5. Either would reject a good run — and one verified failure drops the trust tier to zero. A verifier that produces false failures does not make the system safer; it makes the ledger meaningless, and people learn to ignore it.

The console is a CLIENT of the governor, never a bypass

flowchart LR
  PAGE["the console
localhost only"] -->|"POST /api/run"| GUARD{"same-origin?
read-only?"}
  GUARD -->|"no"| NOPE["403
a site you are visiting must not
launch an agent on your machine"]
  GUARD -->|"yes"| GOV["Governor.execute
reserve → verify → contain → settle"]
  GOV -.->|"pass-through decorator"| LIVE["live feed
tool calls · spend"]
  LIVE -.-> PAGE
  GOV --> LEDGER["the ledger"]
  LEDGER --> TRUST["trust
NO endpoint grants it"]

  class GOV,TRUST core
  class NOPE money
        

The live view is fed by a pass-through decorator on the runtime: the governor sees the identical event stream it would have seen, so watching a run cannot change its outcome. That is the only way a console is safe to have at all.

Exit codes — the verdict IS the exit code

So aos run works in CI. rejected means the work was not done, however confident the agent’s summary sounded.

15 — Trust

Slow to earn. Instant to lose.

stateDiagram-v2
  direction LR
  T0: Tier 0 — every run needs approval
  T1: Tier 1 — approve destructive only
  T2: Tier 2 — unattended
  [*] --> T0
  T0 --> T1: 10 verified passes
  T1 --> T2: 20 runs @ ≥95% verified
  T2 --> T0: ONE verified failure
  T1 --> T0: ONE verified failure
        

• Budget aborts do not count. An agent that stops at its cap is the system working. Recorded passed: null and excluded from the rate entirely.
• Changing the model resets the record. The evidence was gathered about a different system.

16 — Scheduler

A schedule is a request. The ledger grants it.

flowchart TB
  T["tick(now)"] --> D["due firings
cron + IANA timezone"]
  D --> C{"claimFiring(id, fireAt)
PRIMARY KEY"}
  C -->|"already claimed"| SKIP["skip — restart-safe"]
  C -->|"claimed"| G{"THE TRUST GATE
trust(taskType, model)
≥ minAutonomyTier?"}
  G -->|"no"| BL["blocked_untrusted
recorded, with the exact reason"]
  G -->|"yes"| GOV["governor.execute(spec)
same lease, same verification — no bypass"]
        

Idempotency lives in the database. PRIMARY KEY (schedule_id, fire_at), keyed on the scheduled instant, not the actual one. A check-then-insert has a window in which two workers both decide to run the 3am job. A primary key does not.

17 — Multi-tenant mode

Real tenants. Real money. Same suite.

The same governor also runs inside a multi-tenant platform, where the budget adapter speaks HTTP to a billing service backed by Postgres and Stripe, and a lease is a row that holds a real customer’s money.

flowchart LR
  GOV["AOS Governor"] --> BB["PlatformBudget"]
  BB -->|"POST /internal/reservations"| US["billing service"]
  US --> PG[("Postgres")]
  PG --- T1["organization_credits"]
  PG --- T2["usage_records"]
  PG --- T3["credit_reservations
parent_id · own_drawn
child_spend · subleased"]
  US --> ST["Stripe
seats"]
  SW["Sweeper
every pod · 60s"] --> US
        

Compare the alternative: a prose document describing the billing behaviour. A doc drifts — we have watched one describe a column that did not exist and a cache-invalidation call that was never made. A conformance suite cannot drift, because it executes.

The sweeper — safe by construction, not by coordination

A killed process holds its money forever unless something releases it. Every pod sweeps every 60s. There is no leader election, no lock table, and nothing to go stale: each row is re-checked under SELECT … FOR UPDATE and only expired if this transaction is the one that settled it. Two pods racing the same stale hold means exactly one wins.

18 — What testing found

Six bugs no mock could see.

Unit tests that mock the database prove the code’s branches. They say nothing about whether the SQL is correct, whether the CHECK constraints bite, whether SELECT … FOR UPDATE actually serialises anything, or whether the settle cascade leaves the books balanced. Every one of these was found by wiring two real systems together — never by a test that stood alone.

19 — Design rules

Stated once, held throughout.

20 — Honest limits

What isn’t done.

An architecture document that lists only what works is marketing. These are the open edges, named on purpose.