Voice AI

Replay Any AI Agent Conversation as a Live Stream — With LLM Commentary On Top

A customer tells your support agent it promised a refund. It didn't. Now what?

You have a transcript and a hunch. What you need is the session itself — replayable, scrubbable, and explained to you step by step. That is what the Agent Message Replay sample builds, and it works because of something the Telnyx Agent SDK already does for you: every conversation is persisted in a durable message log.

The Telnyx code example is here:

https://github.com/team-telnyx/telnyx-code-examples/tree/main/agent-message-replay

It is a TypeScript example for Telnyx Edge Compute that replays recorded agent conversations as live WebSocket streams — messages arrive in real time, the agent's state changes re-enact as they happened, and an LLM annotates each step using history straight from the message log.

What This Example Builds

The sample models a replay as a durable actor. One ReplayAgent extends Agent per conversation, keyed by conversation id — including phone numbers, so you can replay "the session with this customer."

Playback is not a loop in memory. Each step is played by a durable schedule() tick chain:

async tick(): Promise<void> {
  const state = await this.getState();
  if (state.status !== "playing") return;   // pause/finish = durable no-op

  const step = steps[index];
  await this.messages.add(step.role, step.content);  // live append → pushed to watchers
  this.desk.broadcastMessages([last]);

  if (step.stage && step.stage !== state.agentStage) {
    await this.events.emit("state_change", { stage: step.stage, stepIndex: index });
    await this.changeState({ agentStage: step.stage });  // re-enact + broadcast patch
  }

  await this.changeState({ playhead: nextIndex });
  const delaySeconds = Math.max(step.delayMs, 250) / 1000 / state.speed;
  await this.schedule(delaySeconds, "tick");  // durable next tick
}

Three things fall out of that shape for free:

  • Pausing is durable by construction. Pause sets a flag; the next scheduled tick wakes, sees it, and exits. No task-id bookkeeping.
  • Crash safety. If the actor restarts mid-replay, press play and it resumes from the persisted playhead.
  • Speed changes apply on the next tick, because pacing is recomputed from the current speed every time.

The WebSocket Half

The actor wires an AgentSocketServer to its state, message log, and event log:

private desk = new AgentSocketServer<ReplayState>(this, {
  getState: () => this.getState(),
  getMessages: () => this.messages.all(),
  getEvents: (after) => this.events.read(after),
  authorize: (token) =>
    token === (this.env.REPLAY_TOKEN ?? "replay-demo")
      ? ["read", "rpc"]
      : ["read"],
});

Clients attach with a token and get claims: the demo token grants read and rpc (play, pause, seed, speed, commentary); everyone else connects as a read-only watcher. On connect each client receives the state snapshot, the message history so far, and — with an event cursor — anything it missed while disconnected. After that, every append, state patch, and event is pushed live.

The demo UI, served by the same Edge function at /, is a dependency-free browser client speaking that protocol. It renders the conversation, a stage badge that re-enacts the original agent's state trail, a commentary feed, and a timeline scrubber.

Commentary From the Log You Already Have

When commentary is on, each played agent step triggers one LLM call. The context is the message log itself:

const history = toChatMessages(await this.messages.toOpenAI());
const completion = await this.env.TELNYX.ai.openai.chat.createCompletion({
  model: this.env.MODEL ?? "zai-org/GLM-5.2",
  messages: [{ role: "system", content: COMMENTARY_SYSTEM_PROMPT }, ...history],
  max_tokens: 120,
  temperature: 0.6,
});

Two details matter. First, env.TELNYX is pre-authenticated — no API key appears in code, config, or logs. Second, commentary rides the event stream, not the message log, so the replayed conversation stays a faithful recording. A 30-second timeout degrades a slow model call to a commentary_error event; the replay never depends on the model.

Bring Your Own Conversations

Two ways to load something other than the built-in demo: POST /ingest a recording (zod-validated), or — the real payoff — point the actor at your production agent's history. Because the Agent SDK persists every conversation as a durable message log, a replay is reading your production agent's history back. No instrumentation, no schema translation.

Load it, replay it, scrub it: pause at message five and see what the agent had concluded by then, watch the stage trail re-enact intake → verifying → investigating → resolving → resolved, and read the model's annotation of each step.

Run It

git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/agent-message-replay
npm install
npm run typecheck && npm test

Deploy with the Telnyx Edge CLI and open the function URL — full setup is in the README. The sample ships with a flow-conformance test suite that drives the real agent, socket server, and message log in memory, plus a live end-to-end script (npm run test:e2e:live) that verifies a deployed function in nine checks.

If you build agents that handle support, this is the debugging view you will wish you had the first time a customer says "your bot promised me a refund."

Ready to build with low-latency voice AI?

Join developers building the future of real-time conversations