OpenAI Realtime with WebRTC

Use this flow for browser-based OpenAI Realtime voice apps (React, etc.). TrueFoundry mints a short-lived ephemeral client secret so your API key stays server-side. The browser then connects directly to OpenAI for WebRTC. Live audio/media is not proxied through the gateway.

For server-side / WebSocket Realtime, use the Live / Realtime API (wss://{GATEWAY_HOST}/live/{providerAccountName}) instead.

Architecture

Step	Where	Auth
Mint ephemeral token (`ek_...`)	TrueFoundry Proxy API	TrueFoundry API key (backend only)
WebRTC SDP + audio	OpenAI `POST /v1/realtime/calls`	Ephemeral `ek_...` (browser)

TrueFoundry credentials never go to the browser. Because media does not pass through the gateway, live-session observability for the WebRTC media path will not appear on gateway traces the way WebSocket /live/... sessions do.

Prerequisites

An OpenAI (or Azure OpenAI / Azure AI Foundry) model account on TrueFoundry with a Realtime model (for example gpt-realtime or gpt-realtime-2.1).
Your Gateway Base URL and TrueFoundry API key.
Call this from your backend and return only the ephemeral key (ek_...) to the browser — do not put the TrueFoundry API key in client code.

1. Mint an ephemeral client secret (backend)

curl -sS -X POST \
  "${GATEWAY_BASE_URL}/proxy/realtime/client_secrets" \
  -H "Authorization: Bearer ${TFY_API_KEY}" \
  -H "Content-Type: application/json" \
  -H "x-tfy-provider-name: openai-main" \
  -d '{
    "session": {
      "type": "realtime",
      "model": "gpt-realtime"
    }
  }'

Replace ${GATEWAY_BASE_URL}, ${TFY_API_KEY}, openai-main, and gpt-realtime with your gateway URL, API key, provider account name, and realtime model id. The response includes a short-lived key (ek_...). Return only that value to the browser. Mint it immediately before connecting — ephemeral keys expire quickly.

2. Connect WebRTC from the browser (directly to OpenAI)

Follow OpenAI’s Realtime WebRTC guide. Do not send the SDP offer to TrueFoundry.

// EPHEMERAL_KEY comes from your backend (the ek_... value from step 1)
const EPHEMERAL_KEY = "...";

const pc = new RTCPeerConnection();

const audio = document.createElement("audio");
audio.autoplay = true;
pc.ontrack = (e) => (audio.srcObject = e.streams[0]);

const ms = await navigator.mediaDevices.getUserMedia({ audio: true });
pc.addTrack(ms.getTracks()[0]);

const dc = pc.createDataChannel("oai-events");
dc.onmessage = (e) => console.log(JSON.parse(e.data));

const offer = await pc.createOffer();
await pc.setLocalDescription(offer);

const sdpResponse = await fetch("https://api.openai.com/v1/realtime/calls", {
  method: "POST",
  body: offer.sdp,
  headers: {
    Authorization: `Bearer ${EPHEMERAL_KEY}`,
    "Content-Type": "application/sdp",
  },
});

await pc.setRemoteDescription({
  type: "answer",
  sdp: await sdpResponse.text(),
});

Serve this page over http://localhost or HTTPS so the browser can access the microphone.

What TrueFoundry does / does not do

Supported	Not supported
Minting ephemeral Realtime client secrets via `/proxy/realtime/client_secrets`	Proxying live WebRTC media through the gateway
Keeping TrueFoundry credentials server-side	Full gateway tracing/cost for the browser↔OpenAI media path
WebSocket Realtime via `/live/{providerAccountName}`	Using chat models (for example `gpt-4.1`) on this endpoint — only Realtime models work

Get Started

LLM Gateway

MCP Registry and Gateway

Skills Registry

Prompt Registry

Guardrails and Security

Observability

Deployment

Admin Guide

Chat

Messages

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Fine-tuning

Moderations

Models

Architecture

Prerequisites

1. Mint an ephemeral client secret (backend)

2. Connect WebRTC from the browser (directly to OpenAI)

What TrueFoundry does / does not do

​Architecture

​Prerequisites

​1. Mint an ephemeral client secret (backend)

​2. Connect WebRTC from the browser (directly to OpenAI)

​What TrueFoundry does / does not do

​Related

Architecture

Prerequisites

1. Mint an ephemeral client secret (backend)

2. Connect WebRTC from the browser (directly to OpenAI)

What TrueFoundry does / does not do

Related