Route Phone Calls to an AI Agent With the Telnyx Voice API

Voice AI agents are transforming customer service, sales, and support. But before an AI can talk to a caller, you need the voice infrastructure layer — a way to receive inbound phone calls, control the call flow, and respond programmatically. Without this foundation, your AI model has no way to pick up the phone.

Telnyx Call Control gives you this through webhooks. When someone calls your Telnyx number, your application receives events in real time and issues commands back — answer, speak, gather input, transfer, or hang up. This is the foundation every voice AI agent runs on.

What Is Call Control?

Telnyx Call Control is a webhook-driven API for programmable voice. Instead of static IVR menus, your application receives HTTP events for every call state change and responds with commands.

The event loop works like this: Telnyx sends a webhook to your application, your app processes the event, your app issues a Call Control command, and Telnyx sends the next event when that command completes. This cycle repeats for the life of the call.

This event-driven pattern is what makes voice AI possible. Your app can insert AI inference — speech-to-text, LLM processing, text-to-speech — between receiving a caller's input and responding. The Call Control webhook loop gives you a hook into every moment of the conversation.

How It Works

The call flow has three steps:

1. A customer calls your Telnyx number. Telnyx sends a call.initiated webhook to your application with the caller's number, your number, and a call_control_id you'll use for all subsequent commands on this call.

1. Your application answers and responds. Your webhook handler calls the answer action to connect the call, then speak to play a text-to-speech message to the caller. Each command triggers the next webhook event when it completes — call.answered confirms the call connected, call.speak.ended confirms the TTS finished.

1. The call completes. When the TTS finishes, Telnyx sends a call.speak.ended event. Your app issues hangup to end the call. The call.hangup event confirms cleanup.

One webhook endpoint. A state machine driven by events.

The Architecture

┌──────────────┐         ┌──────────────────┐         ┌──────────────┐
│   Caller     │ ◄─────► │  Telnyx Voice    │ ──POST─►│   Your App   │
│   (phone)    │         │  Private Network │         │  (Flask)     │
└──────────────┘         └──────────────────┘         └──────────────┘
                                  ▲                          │
                                  │    Call Control commands  │
                                  └──────────────────────────┘

Your Python Flask application receives webhook events for every call state change and issues Call Control commands back to Telnyx. The call audio stays on Telnyx's private network — your application only handles the control plane.

Prerequisites

• Python 3.8 or higher (Python 3.11+ recommended)
• A Telnyx account — sign up at portal.telnyx.com for free trial credit
• A Telnyx phone number with voice capability
• A Call Control Application configured in the Telnyx Portal with a webhook URL
• ngrok or another tunnel for local webhook testing

Step 1: Clone and Configure

git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/route-phone-calls-to-ai-agent-python
pip install -r requirements.txt
cp .env.example .env

Edit .env with your credentials:

TELNYX_API_KEY=KEY_your_api_key_here
TELNYX_PUBLIC_KEY=your_public_key_here

Step 2: Initialize the Telnyx Client

The Telnyx Python SDK uses a single constructor with your API key and public key for webhook verification:

import telnyx
from dotenv import load_dotenv

load_dotenv()

client = telnyx.Telnyx(
    api_key=os.getenv("TELNYX_API_KEY"),
    public_key=os.getenv("TELNYX_PUBLIC_KEY"),
)

One client, one API key, one bill. Telnyx is an AI Communications Infrastructure platform — voice, messaging, verification, AI inference, and IoT all live behind the same authentication surface.

Step 3: Define the Call Control Functions

Each call control action maps to a single SDK call:

def answer_call(call_control_id: str) -> dict:
    response = client.calls.actions.answer(call_control_id)
    return {
        "call_control_id": response.data.call_control_id,
        "status": "answered",
    }

def speak_to_call(call_control_id: str, message: str) -> dict:
    response = client.calls.actions.speak(
        call_control_id,
        payload=message,
        voice="female",
        language_code="en-US",
    )
    return {
        "call_control_id": response.data.call_control_id,
        "message": message,
    }

def hangup_call(call_control_id: str) -> dict:
    response = client.calls.actions.hangup(call_control_id)
    return {
        "call_control_id": response.data.call_control_id,
        "status": "hangup_initiated",
    }

The speak action is the text-to-speech primitive. When you call it, Telnyx's media server converts the text to speech and plays it to the caller. The voice field accepts female or male. The language_code field controls the language and accent — en-US, en-GB, es-ES, and others are supported.

Step 4: Handle Webhooks as a State Machine

Every call event arrives as a POST to your webhook URL. The handler implements a state machine:

@app.route("/webhooks/call", methods=["POST"])
def handle_call_webhook():
    # Verify Ed25519 signature
    client.webhooks.unwrap(
        request.get_data(as_text=True),
        headers=dict(request.headers),
    )

    payload = request.get_json()
    event_type = payload.get("data", {}).get("event_type")
    call_control_id = payload.get("data", {}).get("call_control_id")

    if event_type == "call.initiated":
        # Inbound call received — answer it
        answer_call(call_control_id)
        # Play a greeting
        speak_to_call(call_control_id, "Thank you for calling. Goodbye.")
        return jsonify({"status": "call_answered"}), 200

    elif event_type == "call.answered":
        # Call connected
        return jsonify({"status": "call_answered"}), 200

    elif event_type == "call.speak.ended":
        # TTS finished — hang up
        hangup_call(call_control_id)
        return jsonify({"status": "message_finished"}), 200

    elif event_type == "call.hangup":
        # Call ended
        return jsonify({"status": "call_ended"}), 200

    else:
        return jsonify({"status": "event_received"}), 200

The state machine pattern is the key insight. Each webhook event triggers a specific action, and each action triggers the next event. call.initiated → answer → call.answered → speak → call.speak.ended → hangup → call.hangup. Your application follows this chain, responding to each event with the appropriate command.

Step 5: Handle Errors

The Telnyx SDK throws typed errors that map cleanly to HTTP status codes:

except telnyx.AuthenticationError:
    return jsonify({"error": "Invalid API key"}), 401
except telnyx.RateLimitError:
    return jsonify({"error": "Rate limit exceeded"}), 429
except telnyx.APIStatusError as e:
    return jsonify({"error": "API request failed"}), e.status_code
except telnyx.APIConnectionError:
    return jsonify({"error": "Network error connecting to Telnyx"}), 503

Step 6: Run It

python app.py

In another terminal, expose your local server:

ngrok http 5000

Set the ngrok URL as your Call Control Application's webhook URL in the Telnyx Portal. The webhook path should be your ngrok URL plus /webhooks/call. Now call your Telnyx number from your phone.

You'll hear the greeting message, and the call will end. Check your Flask console — you'll see the webhook events flowing through: call.initiated → call.answered → call.speak.ended → call.hangup.

Why This Pattern Works for Production

1. The state machine is explicit. Every event type has a handler. Every handler issues a specific command. There's no implicit state — the webhook events themselves are the state transitions. This makes debugging straightforward: log every event, and you have a complete trace of the call flow.

2. Webhook verification is mandatory. The Ed25519 signature verification runs before any call control logic. This prevents attackers from spoofing webhook events and hijacking your call flow. The SDK handles this with a single call.

3. The call control vocabulary scales. The example uses answer, speak, and hangup. The same API also supports gather (collect DTMF or speech input), transfer, conference, record, stream (real-time audio streaming), and play (pre-recorded audio). Each is another branch in your state machine.

Extending the Example: From Greeting to AI Agent

The base example plays a static greeting. A voice AI agent replaces the static message with dynamic inference:

• Speech-to-text — Use gather with type: "speech" to capture the caller's words as text.
• LLM processing — Send the transcript to Telnyx AI Inference or your own LLM to generate a response.
• Text-to-speech — Use speak to play the LLM's response back to the caller.
• Loop — Repeat the gather → process → speak cycle for multi-turn conversations.
• Transfer — When the AI can't help, use transfer to hand off to a human agent.

Each of these is a small addition on top of the same Call Control primitives. The webhook state machine stays the same — you're just adding more branches.

The Bigger Picture: Why Carrier-Layer Voice Matters

Voice AI agents need two things: intelligence (the AI model) and infrastructure (the voice layer). The intelligence is commoditized — every cloud provider offers speech-to-text, LLMs, and text-to-speech. The infrastructure is where the differentiation happens.

Telnyx is AI Communications Infrastructure. The voice traffic stays on Telnyx's private global network — no public internet hops between the caller and your application. When audio quality matters (and for voice AI, it always matters), carrier-layer control is the difference between a usable agent and a frustrating one.

The same API key that handles Call Control also handles messaging, verification, SIP trunking, and AI inference. One platform, one bill, one support team that owns the entire path from the caller's phone to your application.

Get the Code

The full example is in the telnyx-code-examples repository under route-phone-calls-to-ai-agent-python. It includes:

• app.py — the complete Flask application
• README.md — quickstart and API reference
• .env.example — environment variable template

Clone it, add your credentials, and you have a running voice webhook handler.

Related Examples

• Build an SMS Auto-Reply Bot with Python
• Send OTP Codes Over WhatsApp with Python
• Warm Transfer with Node.js
• Warm Transfer with AI Briefing (Python)