Voice AI

How to Build an Audio Transcription → AI Summary → SMS Pipeline with the Telnyx Agent SDK

Harpreet Seehra • Aug 19, 2026

Upload a voicemail audio file → transcribe it via speech-to-text → summarize with an LLM → text the summary to any phone number. An end-to-end pipeline on Telnyx Edge Compute with zero-credential inference and messaging.

The Voicemail Problem

Voicemails pile up. You get one after a missed call, listen to it once, and then it sits in your inbox. If you need the details later, you have to listen again. If you want to share the content, you have to paraphrase it or forward the audio. For teams that handle high volumes of voice messages, this friction compounds into real operational drag.

The standard approach to solving this involves stitching together multiple services. A cloud storage provider holds the audio. A transcription API converts speech to text. An LLM provider summarizes the text. An SMS gateway delivers the result. Four vendors, four API keys, four bills, four integration points, and four places where something can break.

The Audio Transcribe → Summarize → SMS example does all of it on one network. Telnyx Cloud Storage holds the audio. Telnyx AI Inference runs speech-to-text and summarization. Telnyx Messaging delivers the summary as an SMS. Telnyx Edge Compute orchestrates the entire pipeline. One API key for storage and STT. Zero credentials in code for inference and messaging. One deploy command.

What It Does

You upload a voicemail audio file to an Edge Compute endpoint. The pipeline runs three stages in sequence:

Transcribe — Download the audio from Cloud Storage and send it to the Telnyx speech-to-text API.
Summarize — Send the transcript to an LLM with a prompt that produces an SMS-friendly summary.
Notify — Send the summary as an SMS to the recipient phone number you provided at upload time.

Each stage is a queued job on a stateful actor. State survives across stages, so the transcript from stage 1 is available to stage 2, and the summary from stage 2 is available to stage 3. You can check the status of any pipeline run via a REST endpoint and get back the full transcript, summary, and completion status.

Stage	Action	API
Upload	Receive multipart form, upload to Cloud Storage (S3 PUT, SigV4)	`POST /upload`
Start	Create actor, store metadata, queue transcribe	`VoicemailAgent.start()`
1. Transcribe	S3 GET download → STT → store transcript	`POST /v2/ai/audio/transcriptions`
2. Summarize	LLM chat completion → store summary	`this.env.TELNYX.ai.openai.chat.createCompletion()`
3. Notify	SMS summary to recipient	`this.env.TELNYX.messages.send()`
Status	Check pipeline progress	`GET /status/:agentId`

The Architecture

The application is a TypeScript Agent SDK project deployed to Telnyx Edge Compute. It uses four Telnyx products: Edge Compute (hosting and actor runtime), AI Inference (STT and LLM), Messaging (SMS), and Cloud Storage (audio file persistence).

  Upload audio file (POST /upload)
        │
        ▼
  ┌──────────────────────────────────────────┐
  │ Upload to Cloud Storage (S3 PUT, SigV4)    │
  └────────┬─────────────────────────────────┘
           │
           ▼
  ┌──────────────────────────────────────────┐
  │ VoicemailAgent.start()                     │
  │  → this.queue("transcribe")                │
  │  → this.queue("summarize")                 │
  │  → this.queue("notify")                    │
  └────────┬─────────────────────────────────┘
           │
           ▼
  Stage 1: transcribe()
    → Download audio from Cloud Storage (S3 GET)
    → POST /v2/ai/audio/transcriptions (STT)
    → Store transcript in agent state
           │
           ▼
  Stage 2: summarize()
    → this.env.TELNYX.ai.openai.chat.createCompletion()
    → Store summary in agent state
           │
           ▼
  Stage 3: notify()
    → this.env.TELNYX.messages.send()
    → SMS summary delivered to recipient

The upload handler receives a multipart form with the audio file and a recipient phone number. It uploads the file to Telnyx Cloud Storage via S3 PUT with AWS SigV4 signing using the Web Crypto API. The S3 key includes a timestamp prefix so uploads are naturally ordered. Once the file is stored, the handler creates a VoicemailAgent actor instance, stores the audio key, bucket name, sender phone, and recipient phone in the actor's durable state, and queues the first pipeline stage.

The Agent SDK runtime handles the rest. Each queued stage runs as a separate invocation of the actor, with state automatically loaded from the previous invocation. If a stage fails, the error is captured in state and the pipeline stops. You can inspect the status at any time.

Zero-Credential Inference and Messaging

The most important design decision in this pipeline is the [telnyx] binding in telnyx.toml:

[telnyx]
binding = "TELNYX"

This single line gives the Agent SDK runtime a pre-authenticated Telnyx client. At runtime, the actor accesses it through this.env.TELNYX. Two critical operations use this binding with no API key in code:

LLM summarization:

const completion = await this.env.TELNYX.ai.openai.chat.createCompletion({
  model: this.env.AI_MODEL || "zai-org/GLM-5.2",
  messages: [
    { role: "system", content: "Summarize this voicemail transcript into a concise SMS-friendly message." },
    { role: "user", content: transcript }
  ],
  max_tokens: 200,
  temperature: 0.3
});
const summary = completion.choices[0].message.content;

SMS delivery:

await this.env.TELNYX.messages.send({
  from: this.env.SENDER_PHONE,
  to: this.state.recipientPhone,
  text: summary
});

No Authorization header. No TELNYX_API_KEY in the source. The binding carries the authentication context from the Edge Compute platform. This means the inference and messaging credentials are managed by the platform, not by the developer. Rotate keys in the portal and the binding picks up the new credentials automatically. No code changes. No redeploy.

Cloud Storage and the speech-to-text API still need an explicit API key because they use S3-compatible SigV4 signing and direct HTTP calls. But that is the same TELNYX_API_KEY you already have. One key for storage and STT. Zero keys for inference and SMS.

Durable State Across Pipeline Stages

The Agent SDK provides durable state through this.setState() and this.getState(). State is scoped to the actor instance and survives across queued invocations:

// In VoicemailAgent.start()
await this.setState({
  audioKey: "voicemails/1724359200000-voicemail.wav",
  bucket: this.env.STORAGE_BUCKET,
  recipientPhone: "+17177247292",
  senderPhone: this.env.SENDER_PHONE,
  status: "queued",
  transcript: "",
  summary: "",
  error: ""
});
this.queue("transcribe");

// In transcribe()
const state = await this.getState();
// ... download audio, run STT ...
state.transcript = transcriptText;
state.status = "transcribed";
await this.setState(state);
this.queue("summarize");

// In summarize()
const state = await this.getState();
// ... run LLM ...
state.summary = summaryText;
state.status = "summarized";
await this.setState(state);
this.queue("notify");

State is not held in memory between stages. It is persisted to the actor store and reloaded on the next invocation. This means the pipeline is resilient to runtime restarts, scaling events, or delays between stages. The state schema is a plain object, so you can store whatever the pipeline needs.

The Pipeline Queue Primitives

The Agent SDK uses this.queue() to schedule non-blocking work. Each call adds a stage to the actor's internal queue and returns immediately. The runtime invokes the queued method when capacity is available:

this.queue("transcribe");
this.queue("summarize");
this.queue("notify");

These three calls do not run in parallel. They run in the order queued, with state persisting between each. If transcribe() fails, summarize() never runs. If summarize() succeeds but notify() fails, the summary is still saved in state and you can retry the notify stage manually or inspect the error.

The queue is durable. If the Edge Compute worker restarts between stages, the queue is restored and processing resumes from the next pending stage.

Setup

Prerequisites

Telnyx Edge CLI v0.2.2+
Node.js 18+
A Telnyx Cloud Storage bucket (create in the Portal)
A Telnyx phone number with SMS capability

1. Install dependencies

npm install

2. Set secrets

telnyx-edge secret set TELNYX_API_KEY KEY0123456789ABCDEF
telnyx-edge secret set STORAGE_BUCKET my-voicemail-bucket
telnyx-edge secret set SENDER_PHONE +18005551234

TELNYX_API_KEY is your Telnyx API v2 key from the Portal. STORAGE_BUCKET is the name of the Cloud Storage bucket you created. SENDER_PHONE is a Telnyx number with SMS enabled.

Optional secrets:

telnyx-edge secret set STORAGE_REGION us-central-1
telnyx-edge secret set AI_MODEL zai-org/GLM-5.2

3. Deploy

telnyx-edge ship

The ship command builds the TypeScript project, uploads the bundle, and prints a URL like audio-transcribe-summarize-sms-<id>.telnyxcompute.com.

4. Test

Health check:

curl https://audio-transcribe-summarize-sms-<id>.telnyxcompute.com/health/liveness

Upload a voicemail and trigger the pipeline:

curl -X POST https://audio-transcribe-summarize-sms-<id>.telnyxcompute.com/upload \
  -F "file=@voicemail.wav" \
  -F "recipient_phone=+17177247292"

Check pipeline status:

curl https://audio-transcribe-summarize-sms-<id>.telnyxcompute.com/status/<agentId>

API Reference

`POST /upload`

Upload an audio file and trigger the transcribe → summarize → SMS pipeline.

curl -X POST https://audio-transcribe-summarize-sms-<id>.telnyxcompute.com/upload \
  -F "file=@voicemail.wav" \
  -F "recipient_phone=+17177247292"

Response:

{
  "action": "queued",
  "audioKey": "voicemails/1724359200000-voicemail.wav",
  "agentId": "voicemails-1724359200000-voicemail.wav",
  "recipientPhone": "+17177247292",
  "statusUrl": "/status/voicemails-1724359200000-voicemail.wav"
}

`GET /status/:agentId`

Check the pipeline status for a given upload.

curl https://audio-transcribe-summarize-sms-<id>.telnyxcompute.com/status/voicemails-1724359200000-voicemail.wav

Response:

{
  "audioKey": "voicemails/1724359200000-voicemail.wav",
  "bucket": "my-voicemail-bucket",
  "recipientPhone": "+17177247292",
  "senderPhone": "+18005551234",
  "transcript": "Hi, this is John. I'm calling about the invoice from last week...",
  "summary": "John called about an invoice from last week. He wants a callback to discuss it.",
  "status": "done",
  "error": "",
  "createdAt": 1724359200000,
  "completedAt": 1724359210000
}

`GET /health/{liveness,readiness}`

Health checks for load balancers and monitoring.

curl https://audio-transcribe-summarize-sms-<id>.telnyxcompute.com/health/liveness

Related Examples

Try It Yourself

git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/audio-transcribe-summarize-sms
npm install
telnyx-edge secret set TELNYX_API_KEY YOUR_API_KEY
telnyx-edge secret set STORAGE_BUCKET your-bucket
telnyx-edge secret set SENDER_PHONE +18005551234
telnyx-edge ship

Then upload a voicemail and watch the pipeline run.

Telnyx is AI Communications Infrastructure — voice, messaging, SIP, AI, IoT, and Cloud Storage on one private, global network. This sample uses four Telnyx products through a single API key and one deploy command. The Agent SDK's [telnyx] binding means LLM inference and SMS delivery need zero credentials in code, while Cloud Storage and STT use the same key for S3-compatible uploads and audio transcription.

Frequently Asked Questions

Q: Why does inference and messaging need zero credentials while Cloud Storage needs an API key?

The [telnyx] binding in telnyx.toml is a platform-managed authentication channel. Edge Compute injects a pre-authenticated client into this.env.TELNYX at runtime. This client can call Telnyx AI Inference and Messaging because those services are part of the same platform identity. Cloud Storage, however, uses S3-compatible SigV4 signing, which requires an explicit access key and secret. The Telnyx API key serves as that credential for S3 operations. One key covers both Cloud Storage and the speech-to-text API.

Q: What audio formats are supported for transcription?

The Telnyx Audio Transcriptions API accepts common formats including WAV, MP3, and OGG. The example uses WAV for simplicity. The API reference has the full list of supported codecs and sample rates.

Q: Can I change the LLM model for summarization?

Yes. Set the AI_MODEL secret to any model available on Telnyx AI Inference. The default is zai-org/GLM-5.2, a general-purpose model that works well for summarization. If you switch to a reasoning model, you may need to adjust the prompt because some reasoning models return chain-of-thought text before the final answer.

Q: What happens if a pipeline stage fails?

The error is caught, stored in the actor's state under the error field, and the pipeline stops. The status endpoint returns the error message and the stage that failed. You can inspect the state, fix the issue (for example, set a missing secret), and retry the stage manually if your agent implementation supports it.

Q: Is the pipeline synchronous or asynchronous?

The upload endpoint returns immediately with a queued status. The pipeline runs asynchronously in the background. For a typical 30-second voicemail, the full pipeline (transcribe + summarize + SMS) completes in a few seconds. You poll the status endpoint or build a webhook callback if you need real-time notification.

Q: Can I send the summary to multiple phone numbers?

The example sends to a single recipient per upload. To support multiple recipients, modify the upload handler to accept an array of phone numbers and loop over them in the notify stage. The this.env.TELNYX.messages.send() call is idempotent enough that you can invoke it once per recipient.