01Budget caps at the tenant and session level
Hard limits enforced before a call goes out, not reported after — a session or a tenant simply can't exceed the cap it was configured with.
cap: $10.00 / sessioncap: $5,000 / tenant / month
Platform engineering
One bad prompt shouldn't bankrupt you
A recursive retrieval loop, a misconfigured integration, one tenant's bad session — any of these can turn into a bill nobody authorized before anyone notices. Budget caps and PII masking need to be enforced controls, not policies written down and hoped for.
A tenant's containment dashboard: budget spent against a hard cap, PII fields masked this month, and a circuit breaker that halted a recursive tool-call loop automatically at $6.40 against a $10 session cap.
Sound familiar?
The invoice is the incident report
Nobody notices a runaway agent loop until the monthly bill arrives with a number that doesn't match anything anyone remembers authorizing.
One tenant's bad session shouldn't be everyone's problem
A single misconfigured integration or a recursive retrieval loop from one customer can burn through shared budget meant for everyone else, with nothing stopping the blast radius at the tenant boundary.
PII masking is a policy document, not a control
There's a written rule that says sensitive fields should be masked before they reach the model. Whether that actually happens on every call depends on every engineer remembering to implement it correctly, every time.
Why the usual fixes don't move the number
A monthly spend alert email
By the time the alert lands, the money is already spent. A notification isn't a control — it's a postmortem with the incident already priced in.
Asking the model to be careful with tool calls
A prompt-level instruction to "avoid excessive retries" is exactly as reliable as any other prompt-level instruction — which is to say, not reliable enough to bet a budget on.
A secondary LLM call to double-check for PII
Running another model to catch what the first one might expose adds latency and cost to every single request, and it's still a probabilistic check on something that needs to be deterministic.
Manually reviewing usage dashboards weekly
A runaway loop can spend a month's budget in an afternoon. Weekly review catches it after the damage, at a cadence built for slow drift, not sudden spikes.
How we solve it
02Deterministic circuit breakers, not model judgment
A session that exceeds a call-rate or spend threshold gets halted automatically — a hard rule, not a soft suggestion the model can talk itself out of.
40 calls / 60s → haltdeterministic, not probabilistic
03PII masking runs as a hard rule, on every call
Regex-based masking for known sensitive patterns runs deterministically before content reaches the model — not a policy someone has to remember to apply.
masked before model sees itno secondary model call needed
What you get on every document
A dashboard that's a control, not a report
Spend, masking counts, and blocked calls are visible in real time against enforced caps — not a summary of what already happened uncontrolled.
Runaway sessions stop themselves
A recursive loop gets halted at the configured cap automatically — the mock above shows a session stopped at $6.40 against a $10 limit, not $600 against nothing.
Masking that doesn't depend on remembering
The same deterministic rule runs on every call for every tenant — there's no code path where an engineer forgot to wire it in for one integration.
Blast radius stops at the tenant boundary
One tenant's misbehaving session can't consume budget or trigger effects outside its own scope — the cap is per-tenant by construction.
