Build a Website AI Assistant That Can Control the Page with Client-Side Tools

Most AI assistant demos treat the website as a place to show a chat bubble. The assistant answers questions, maybe calls a backend webhook, and the page itself stays mostly passive.

Telnyx AI Assistant client-side tools change that pattern. With the Telnyx widget or @telnyx/ai-agent-lib, an assistant can invoke predefined JavaScript functions that run in the browser during a live voice or chat conversation.

The Telnyx code example is here:

https://github.com/team-telnyx/telnyx-code-examples/tree/main/ai-assistant-client-side-tools-nextjs

Clone it, configure a Telnyx AI Assistant with matching client-side tools, add the public assistant ID to .env.local, and run the Next.js app locally.

What This Example Builds

The sample is a Next.js, React, TypeScript, and Tailwind CSS dashboard called PolarForge AI. It is intentionally fictional, local-only, and built to show the mechanics of client-side tool execution.

The dashboard includes:

• A sidebar with Overview, AI Assistants, Analytics, and Settings
• A main content area that changes without a page reload
• Light and dark themes
• An AI Assistants page
• A Create Assistant modal
• A persistent tool activity panel
• A website voice agent call surface powered by @telnyx/ai-agent-lib
• Manual demo controls so every action can be tested without a live assistant

The assistant can call five browser functions:

set_theme
navigate_to_section
open_create_assistant_modal
get_form_state
update_assistant_form

Client-Side Tools vs Webhook Tools

Webhook tools call your backend over HTTP. They are useful when the assistant needs server-side data, privileged credentials, durable state, or integrations that should not run in the browser.

Client-side tools run in the current browser session. They are useful when the assistant needs to:

• Change UI state
• Read React state
• Open or update forms
• Navigate within a single-page app
• Call APIs the page is already authenticated to call
• Work with local browser-only context

That is the point of this demo. The assistant is not creating records in a database. It is showing that a Telnyx AI Assistant embedded on a website can call approved JavaScript functions in that website.

Architecture

visitor opens website
  -> Next.js renders PolarForge AI dashboard
  -> TelnyxAIAgentProvider connects to assistant by public agent ID
  -> React registers client-side tools with useClient().registerClientTool
  -> visitor clicks Call agent
  -> assistant decides a UI action is needed
  -> Telnyx sends client-side tool invocation over the live session
  -> browser handler validates input and updates React state
  -> browser returns result to the assistant
  -> activity panel records running/completed/failed status

There is no Telnyx API key in browser code. The browser only gets public NEXT_PUBLIC_* configuration such as the AI Assistant ID and version. The Telnyx API key belongs in server-side provisioning or Portal setup, not in the client application.

The Core Code

The React integration uses the current AI Agent Lib API:

<TelnyxAIAgentProvider
  agentId={telnyxConfig.agentId}
  versionId={telnyxConfig.versionId}
  environment={telnyxConfig.environment}
  clientToolTimeoutMs={telnyxConfig.clientToolTimeoutMs}
>
  <ToolRegistrar executeTool={executeTool} />
  <TelnyxWidget />
</TelnyxAIAgentProvider>

Each browser function is registered with the client:

useEffect(() => {
  client.registerClientTool("set_theme", (args, context) =>
    executeTool("set_theme", args, context),
  );

  return () => client.unregisterClientTool("set_theme");
}, [client, executeTool]);

The tool handler validates inputs before changing UI state. For example, navigate_to_section only accepts known dashboard sections, and update_assistant_form fails if the Create Assistant modal is closed.

Manual Testing

The sample includes a development-only Demo Controls panel. That matters because it lets someone prove the browser functions work before connecting a live assistant.

Run the app:

git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/ai-assistant-client-side-tools-nextjs
cp .env.example .env.local
npm install
npm run dev

Then use Demo Controls to:

• Switch light and dark mode
• Navigate between dashboard sections
• Open the Create Assistant modal
• Read form state
• Update form fields
• Trigger validation failures

When the assistant is configured, use the website call button instead of a phone number. This is a WebRTC browser session, not a PSTN call.

Production Considerations

Before adapting the pattern for production:

• Keep Telnyx API keys out of browser code
• Register only tools the assistant should be allowed to invoke
• Validate every tool input
• Return clear errors for invalid sections, fields, values, and modal state
• Log tool activity for debugging and demo proof
• Use server-side routes for privileged actions or durable persistence
• Review consent requirements before recording or analyzing voice sessions

Resources

• Code example: https://github.com/team-telnyx/telnyx-code-examples/tree/main/ai-assistant-client-side-tools-nextjs
• Client-side tools docs: https://developers.telnyx.com/docs/inference/ai-assistants/client-side-tools
• AI Agent Lib: https://www.npmjs.com/package/@telnyx/ai-agent-lib
• Release note: https://telnyx.com/release-notes/client-side-tools-ai-assistants
• Telnyx Developer Docs: https://developers.telnyx.com