Building an AI Meeting Facilitator on Telnyx Edge: Transcribe, Mediate, Summarize

Every developer has been in this meeting: three people carry the conversation, the person with the answer never gets a word in, and the "decisions" section of the notes is empty by Friday. Calendar software can't fix that. But a participant that's actually on the call can.

conference-agent-mediator is an open-source TypeScript sample that deploys an AI meeting facilitator to Telnyx Edge Compute. It joins a Telnyx conference bridge, transcribes every participant in real time, notices when the conversation needs a nudge, speaks that nudge into the room, and texts a summary when the call ends. Under the hood, it's a compact tour of Telnyx's AI Communications Infrastructure: Call Control, real-time inference, and programmable messaging orchestrated as one durable program — no servers, queues, or webhook receivers of your own.

This post walks through what it does, how it works, and how to deploy it yourself.

What the App Does

• Joins the bridge. The first dial-in creates the conference; the agent joins its own leg with join_conference.
• Listens to everyone. transcription_start on each participant leg streams finalized utterances to the agent as call.transcription events.
• Mediates turn-taking. A crash-safe timer fires every 30 seconds. The agent checks who has — and hasn't — spoken, asks the LLM for a facilitation prompt, and injects it live into the bridge with conference speak: "Before we move on, I'd love to hear from Dana, who hasn't weighed in yet."
• Summarizes and texts. On conference.ended, the same LLM condenses the transcript and Programmable SMS delivers the summary.
• Streams everything to observers. A built-in agent socket (wss://…/agents/conference/{id}) pushes a state snapshot on connect and an incremental merge-patch on every change — live transcript, mediator prompts, phase, and summary.
• Runs without a phone. A demo simulator exercises the full pipeline with no live calls (DEMO_MODE=true by default).

How It Works

One durable actor per conference

The entire backend is a single Edge Compute function (src/index.ts). Voice webhooks land at /webhooks/voice and are routed to a Stateful Actor — one ConferenceAgent per conference:

// src/index.ts (abridged)
if (url.pathname === "/webhooks/voice") {
  const event = await req.json();
  const id = event.data.conference_id ?? event.data.call_control_id;
  const agent = await ConferenceAgent.for(id); // 1 actor per conference
  await agent.handleWebhook(event);
  return Response.json({ status: "ok" });
}

The agent extends the Telnyx Agent SDK's Agent base class:

export class ConferenceAgent extends Agent {
  state = {
    phase: "waiting",     // waiting → live → summarizing → done
    conferenceId: null,
    participants: {},     // call_control_id → { joinedAt, utterances, spokeMs }
    transcript: [],
    prompts: [],
    summary: "",
  };
}

state is durable — SQL-backed and crash-safe. If the Edge runtime restarts mid-meeting, the agent comes back with the full transcript, participant talk-time, and phase intact. That's the difference between a demo and something you can leave on a 45-minute bridge.

The mediation loop

The heart of the sample is a timer that survives crashes. every(30s) re-registers on boot, so mediation keeps running even across restarts:

// Crash-safe timer: re-registered on boot, survives restarts
every("30s", () => this.mediate());

async mediate() {
  if (this.state.phase !== "live") return;

  const quiet = Object.entries(this.state.participants)
    .filter(([, p]) => p.spokeMs < QUIET_THRESHOLD)
    .map(([id]) => id);
  if (!quiet.length) return;

  // Zero-credential inference — no API key in code or env plumbing
  const completion = await ai.openai.chat.createCompletion({
    model: env.AI_MODEL ?? "zai-org/GLM-5.2",
    messages: [
      { role: "system", content: MEDIATOR_SYSTEM_PROMPT },
      { role: "user", content: JSON.stringify({
          recentTranscript: last(this.state.transcript, 12),
          quietParticipants: quiet,
      })},
    ],
  });

  const line = completion.choices[0].message.content.trim();
  this.state.prompts.push({ at: Date.now(), text: line });

  if (env.DEMO_MODE !== "true") {
    await telnyx.conferences.speak(this.state.conferenceId, { payload: line });
  }
}

Two details worth calling out. First, the [telnyx] binding in telnyx.toml pre-authenticates both ai.openai.chat.createCompletion and the Telnyx client — there are no credentials to rotate, leak, or wire through environment variables. Second, the mediation policy is just data: the system prompt and the quiet-participant heuristic are the only things deciding when to intervene, so you can tune the facilitator's personality without touching the plumbing.

Real-time transcription

When a participant joins, the agent calls transcription_start on their leg. Finalized utterances stream back as call.transcription webhooks:

onTranscription(e) {
  this.state.transcript.push({
    speaker: e.call_control_id,
    text: e.results.text,
    at: Date.now(),
  });
  // every state change emits a merge-patch to WebSocket observers
}

Because the agent runs on Telnyx Edge — the same infrastructure that terminates the media — the loop from spoken word to transcribed text to mediated response stays tight enough to feel conversational.

Summary, SMS, and observers

On conference.ended, the phase flips to summarizing. The LLM condenses the transcript, the summary is persisted to the agent's durable state, and messages.send texts it out — again over the zero-credential binding:

await telnyx.messages.send({ from: env.SMS_FROM, to: env.SMS_TO, text: summary });

Meanwhile, the built-in agent socket (AgentSocketServer) serves observers at wss://…/agents/conference/{id}: a full state snapshot on connect, then an incremental merge-patch on every state change. The repo ships a small observer client, so you can watch the transcript, mediator prompts, and summary stream in live.

Setup

1. Clone and install

git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/conference-agent-mediator
npm install

2. Deploy to Telnyx Edge

npm run deploy        # → telnyx-edge ship

The CLI prints your function URL (https://conference-agent-mediator-<id>.telnyxcompute.com). There's no separate local dev server — functions run on Telnyx infrastructure.

3. Wire up a number and Call Control application (live mode)

# Buy a number for the conference bridge / SMS sender
telnyx number-orders create --phone-number "<TELNYX_CONFERENCE_NUMBER>"

# Create a Call Control application pointing at your deployed webhook
telnyx call-control-applications create \
  --application-name "conference-agent-mediator" \
  --webhook-url "https://conference-agent-mediator-<id>.telnyxcompute.com/webhooks/voice"

# Attach the number to the application
telnyx numbers update <TELNYX_CONFERENCE_NUMBER> --connection-id <call_control_app_id>

For US SMS delivery, make sure the sending number has a messaging profile with a 10DLC campaign attached (telnyx messaging-profiles list).

4. Configure secrets

telnyx-edge secret set TELNYX_API_KEY your_telnyx_api_key
telnyx-edge secret set SMS_FROM <TELNYX_SMS_FROM>
telnyx-edge secret set SMS_TO <SUMMARY_RECIPIENT_NUMBER>

AI_MODEL, DEMO_MODE, and the [telnyx] inference/messaging binding are configured in telnyx.toml.

5. Verify, then run the demo