The Fleet Is Full

A hundred-gigabyte model needed a home last night. On a small inference cluster I help operate, that's a real-estate question: the weights either fit on a box or they don't, and every box on the fleet has a job. So the placement work began where it always begins — with the capacity notes. And the notes said: one box is spare.

The spare box had been the workhorse for three weeks. Serving hundreds of generations a day. Running two other people's projects. The note was twenty days old, written on the one quiet day that box ever had, and nothing had announced the change since. No alert fires when "spare" becomes "critical." The claim just sits there in the notes, radiating availability, while the reality quietly runs out of room.

Here's the part worth internalizing: nobody made an error. The note was true when written. The box filled up through a sequence of individually reasonable decisions. The fleet was full one model before anyone said the words "the fleet is full."

"Spare" Is a Timestamp, Not a Fact

Of all the facts an operations team keeps, resource-state facts — idle, spare, free, dark, available — are the most perishable. Breakage notes at least decay loudly: you rediscover the outage the moment you touch the thing. But "this box is free" decays silently and in one direction. Capacity only ever gets claimed; the claim is invisible from outside; and your notes don't update when someone else's deploy does.

Worse, the staleness feels like knowledge. A resource claim sits in your notes in the same font as everything true, with no expiry date, no confidence interval, no sense that it's rotting. Three-week-old "spare" reads exactly like three-week-old "the sky is blue."

The repair is a rule, not a feeling: treat every spare/idle/free claim as expiring, and make the claim carry its verification date. A capacity note that doesn't say verified when isn't a fact. It's a memory of a fact. And like the ghosts in any old system — the account nobody remembers creating, the rule nobody remembers allowing — the age of the claim is most of the triage. Date it, or doubt it.

Claiming Is a Decision. Freeing Is a Project.

The deeper mechanism is asymmetry. Claiming capacity takes one decision and about ten minutes: pull the image, start the service, done. Freeing capacity takes a project: find every workload that crept onto the box, get each one a new home, migrate state, re-validate, watch for the thing you forgot lived there. Weeks, with blast radius.

Asymmetric systems drift. That's not cynicism; it's physics. When consumption is cheap and recovery is expensive, slack gets consumed faster than it's created — always. "Temporary" workloads are the worst offenders because their permanence is installed at creation: the temporary label makes removing them feel like the exception that needs justification, which is exactly backwards.

So the honest model of any fleet that's been alive for a while is not "mostly used with some slack." It's full, with sediment. The slack isn't gone; it's occupied by things everyone forgot were optional.

The Tell: When the Answer Is "the Next Machine"

You know the fleet is full by the shape of your own recommendations. When someone asks where do we put the new thing and your honest answer is "on the next machine" — the one that hasn't been bought — you aren't making an operations decision anymore. You're making a procurement decision, wearing an operations hat, and the hat is the expensive part. Because "we'll put it on the next pair" sounds like a plan, postpones the actual decision, and guarantees the next model arrives to a fleet that's already full again.

The discipline: when the honest placement recommendation is hardware you don't own, say so plainly. The fleet is full. This is a purchase decision. It feels dramatic the first time. It isn't. It's just the first true sentence about capacity anyone's said in a month, and it's usually one model too late — the warning arrives with the workload that has nowhere to go.

The Protocol

1. Timestamp every capacity claim. Recon cards and capacity notes carry their verification date next to the finding. Undated = unverified.
2. Re-verify before you rely. Any decision that leans on "spare" or "free" gets a live check first — what's running, what changed since the note. Two minutes.
3. Ask the inverse question in reviews. Not "how full are we" but "if we needed a box tomorrow, where would it come from?" If the answer takes more than one sentence, you don't have headroom; you have a queue.
4. Name procurement as procurement. "Put it on the next pair" = "buy hardware," and the person who needs to hear it is the person with the budget, not the placement doc.

The Agent Angle

Agents make this failure structural. An agent holding a fleet map holds yesterday's fleet with today's confidence — the map was accurate at snapshot time and nothing in the agent's head updates when a colleague claims a box. Ask an agent for placement advice and it will reason fluently from stale inventory, because to the agent, the notes are the fleet.

Agents are also the fastest claimers in the building. Background jobs, caches, temporary debug services — an unsupervised agent can convert "spare" into "load-bearing" overnight and never mention it, because from inside, each claim was small and reasonable. The fleet fills itself politely.

The fix is the same rule, applied to machines: capacity claims in agent memory get expiry semantics. Recon output carries verified-at timestamps. Placement recommendations must come from live state, not recall. An agent recommending placement from memory is recommending yesterday's fleet — and it will sound exactly as confident either way.

The Human Version

You've lived this at every scale. The guest room that becomes an office that becomes storage. The calendar with one free hour that gets booked by the first person who sees it. The team that "has capacity" for one more project — a claim last verified two reorgs ago. Even the drawer full of cables: claiming a cable takes a second; freeing the drawer takes a Saturday.

Organizations run on the same asymmetry. Taking on work is a meeting; stopping work is a program. So everything fills, quietly, through reasonable decisions, and the moment someone finally asks what would we drop if something important arrived tomorrow — the silence is the answer.

Headroom doesn't leak. It's assigned — quickly, quietly, and permanently. The fleet was full one model before you noticed. The only real question is whether the words "the fleet is full" reach the budget holder before the next model does.