Voice AI

Building a Multi-Tenant AI Spend Ledger with Telnyx AI Gateway

If you operate AI features for other businesses, you eventually meet the same two 2 a.m. pages. The first: a tenant's AI assistant got stuck in a loop, burned through its monthly budget in a weekend, and now you're explaining an overage invoice to an unhappy customer. The second, worse: someone on your team pasted a customer's credit card number into a prompt to "quickly check something," and that number rode straight to a model.

ai-gateway-tenant-meter is a working TypeScript reference implementation that solves both problems at the infrastructure layer. It pairs the Telnyx AI Gateway — which meters spend and enforces budgets in the same plane as the model — with a Telnyx Edge Compute stateful actor that keeps a durable per-tenant ledger, sends budget alerts by SMS, and records guardrail findings without ever storing the sensitive text that triggered them.

This post walks through what it does, how it works, and how to run it. Full source: team-telnyx/telnyx-code-examples/ai-gateway-tenant-meter.

What the App Does

The running example is a managed IT support desk serving small clinics. Each clinic is a tenant with its own AI assistant — triaging inquiries, scheduling appointments, answering billing questions — and its own monthly AI budget. When a clinic is onboarded via POST /provision, the sample:

  • Creates a Telnyx AI Gateway token group for the tenant with a max_budget, a budget_duration of 30d, and guardrails that block secrets, flag financial DLP hits, and buffer streaming responses for inspection.
  • Creates a token key — the tenant's gateway credential — and hands back the one-time secret exactly once.
  • Runs an hourly durable rollup that pulls gateway usage and guardrail findings into actor SQL (spend_days, alerts, guardrail_events).
  • Sends exactly one admin SMS at 80% of budget (WARN_PCT, configurable).
  • At 100% (HARD_PCT), flips the tenant's spend view to read-only while the gateway itself starts denying requests with 403 budget_exceeded.
  • Survives platform restarts with all state intact: the SQL rollup resumes, the alerted80 flag prevents duplicate alerts, and the next budget period resets the guards automatically.

The enforcement point is the key design decision: the gateway denies over-budget requests itself. Your application code is never the thing standing between a tenant and an overage.

How It Works

The gateway meters and enforces

Each tenant gets a token group — a budgeted, guarded container for gateway spend:

const group = await this.gateway.createTokenGroup({
  max_budget: input.monthlyBudget,
  budget_duration: "30d",
  guardrails: {
    secrets: "block",      // card numbers, API keys, credentials
    dlp_financial: "flag", // financial data flagged for review
    streaming: "buffered", // inspect full responses, not chunks
  },
});

The tenant's assistant then calls the model through the gateway with its own key — an OpenAI-compatible endpoint, so existing client code works unchanged:

curl https://llm.telnyx.com/v1/chat/completions \
  -H "Authorization: Bearer ltg_sk_..." \
  -H "Content-Type: application/json" \
  -d '{"model":"Kimi-K2.6","messages":[{"role":"user","content":"..."}]}'

When the group's spend crosses max_budget, the gateway returns 403 budget_exceeded on the next call. No application code has to be awake for that to happen.

The actor remembers

The gateway meters and enforces; the actor remembers. Each tenant maps to one stateful actor whose durable state holds the alert history, the read-only flag, the findings, and the budget audit. Once an hour, the actor's rollup() pulls live numbers and findings into SQL:

async rollup() {
  const g = await this.gateway.getTokenGroup(this.tokenGroupId);
  // g.spend, g.resets_at, g.version (ETag)

  await this.storage.sql.exec(
    `INSERT INTO spend_days (day, spend) VALUES (?, ?)
     ON CONFLICT(day) DO UPDATE SET spend = excluded.spend`,
    dayKey(), g.spend
  );

  const events = await this.gateway.listGuardrailEvents();
  for (const e of events.data) {
    // Findings arrive as codes and counts only — never matched text.
    await this.storage.sql.exec(
      `INSERT OR IGNORE INTO guardrail_events (event_id, code, count) VALUES (?, ?, ?)`,
      e.id, e.code, e.count
    );
  }

  const pct = g.spend / this.budget;
  if (pct >= this.hardPct && !this.readOnly) {
    this.readOnly = true;
    await this.notify(`100% of budget used — tenant view is read-only`);
  } else if (pct >= this.warnPct && !this.alerted80) {
    this.alerted80 = true;
    await this.notify(`80% of budget used`);
  }
}

The SMS goes out through the [telnyx] binding — messaging credentials are injected at runtime, so there's no key in code:

await TELNYX.messages.send({
  to: this.env.ADMIN_SMS_TO,
  from: this.env.ADMIN_SMS_FROM,
  text: `Tenant budget at 80% (${spent} of ${budget}). Resets ${resetDate}.`,
});

The wire contract: idempotency and preconditions

Two details make the ledger safe to operate. Every gateway mutation — POST, PATCH, DELETE — requires an Idempotency-Key header, so a retried provision or budget change can't double-apply. Every read comes back wrapped in a data envelope. Both are handled once, in the GatewayClient in src/index.ts, so the rest of the code stays clean.

Budget changes use optimistic concurrency. The GET returns a version (ETag); the PATCH must carry it as If-Match:

@rpc()
async adjustBudget(newBudget: number) {
  const current = await this.gateway.getTokenGroup(this.tokenGroupId);
  await this.gateway.updateTokenGroup(
    this.tokenGroupId,
    { max_budget: newBudget },
    { ifMatch: current.version } // concurrent edit → 412, not a silent overwrite
  );
}

What happens on a restart

If the platform reboots mid-month, the actor restarts with durable state intact. The SQL rollup resumes where it left off, the alerted80 flag prevents a duplicate SMS, and the gateway's resets_at marks the 30-day budget period — when it rolls over, the actor resets its once-per-period alert guards automatically. Nothing depends on a process staying alive.

Setup

Requires Node.js 22+ and npm.

git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/ai-gateway-tenant-meter

npm install
cp .env.example .env

# Verify the wire contract offline (no API key needed)
npm run smoke

# Type-check against the real Agent SDK types
npm run typecheck

Required environment variables:

VariablePurpose
TELNYX_API_KEYAccount API key for the AI Gateway management API (Portal API keys)
ADMIN_SMS_TOOps phone number that receives budget alerts
ADMIN_SMS_FROMTelnyx SMS-capable number to send from (Numbers)

Optional: DEMO_MODE=true (the default) logs SMS to the console instead of sending; API_TOKEN requires Authorization: Bearer <token> on POST /provision, GET /spend, and POST /adjust; ALLOWED_MODELS restricts the model allowlist for new token groups; WARN_PCT and HARD_PCT default to 80 and 100.

Secrets deploy as Edge secrets (telnyx-edge secrets add <name> <value>), and telnyx numbers list plus telnyx messaging-profiles create --name spend-alerts get an SMS route ready. npm run smoke deserves a special callout: it verifies the full wire contract — idempotency keys, data envelopes, precondition handling — with no API key at all, so you can validate your integration before touching live credentials.

Where to Take It Next

The sample is deliberately small. Natural extensions: per-model spend caps surfaced from usage/summary's by_model totals, a tenant-facing dashboard over the same SQL tables, or alert routing beyond a single ops number.

Wrap-Up

The core idea is a division of labor: the gateway meters and enforces where the spend actually happens; a durable actor remembers what happened and tells a human when it matters. That's the shape of Telnyx's AI Communications Infrastructure — inference, messaging, and stateful compute in one API surface — and it's what makes per-tenant AI billing boring in the best possible way.

Grab the code, run npm run smoke, and you'll have a working ledger before your next budget cycle: ai-gateway-tenant-meter on GitHub.

=== END ===

Ready to build with low-latency voice AI?

Join developers building the future of real-time conversations