> Agent-readable docs index: /llms.txt. Download /docs.zip to grep all markdown files locally.

---
title: Realtime WebSocket
description: Sentence-by-sentence streaming synthesis over a WebSocket.
---

# Realtime WebSocket

The WebSocket endpoint streams synthesis one sentence at a time. It is ideal
when text is produced incrementally — for example, sentences coming out of an
LLM — and you want each one spoken as soon as it is ready.

```text
wss://api.vakyam.ai/v1/tts/websocket
```

> **Note:**
> Send one **complete sentence or utterance** per `text` message. Input streaming
> (partial or word-by-word text) is not supported in v1.

## Connection lifecycle

1. Connect with your API key in the `Authorization` header. The server
   authenticates on the handshake; a bad key closes the socket with code `4001`.
2. The server sends `{"type":"connected"}`.
3. Send one `config` message to set `model_id`, `voice`, `language`,
   `output_format`, and `speed` for the session. The server replies with
   `{"type":"configured", ...}`.
4. For each sentence, send a `text` message. The server streams binary audio
   chunks, then a `end_of_utterance` message.
5. Wait for `end_of_utterance` before sending the next sentence.
6. Send `disconnect` (or close) when done.

```text
Client                                   Server
  |-- connect (Authorization) ----------->|
  |<-- {"type":"connected"} --------------|
  |-- {"type":"config", ...} ------------>|
  |<-- {"type":"configured", ...} --------|
  |-- {"type":"text","text":"..."} ------>|
  |<-- [binary audio chunk] --------------|
  |<-- [binary audio chunk] --------------|
  |<-- {"type":"end_of_utterance", ...} --|
  |-- {"type":"disconnect"} ------------->|
  |<-- closed 1000 -----------------------|
```

## Client message types

The server accepts five client message types:

| Type         | Purpose                                               |
| ------------ | ----------------------------------------------------- |
| `config`     | Set or replace the session synthesis config.          |
| `text`       | Synthesize one complete sentence with current config. |
| `cancel`     | Interrupt (barge-in) the current utterance.           |
| `ping`       | Receive a JSON `pong` and reset the idle timer.       |
| `disconnect` | Close the connection normally.                        |

### Messages you send

#### Config

Sets the session voice, language, model, output format, and speed. Must be the
first message sent after connecting.

```json
{
  "type": "config",
  "model_id": "raaga-v1",
  "voice": "Archana",
  "language": "ta-IN",
  "output_format": "pcm",
  "speed": 1.0
}
```

**Fields**

* `type` — always `"config"`.
* `model_id` — the TTS model. Use `raaga-v1`.
* `voice` — the preset voice name or a custom voice ID beginning with `vc_` (see [Voices](/concepts/voices-and-languages)).
* `language` — BCP 47 language code; must form a valid pair with a preset `voice`.
* `output_format` — `pcm` (default), `mp3`, `wav`, or `mulaw`.
* `sample_rate` — output sample rate in Hz: `8000`, `16000`, `24000` (default), or `48000`.
* `speed` — playback speed multiplier, `0.5`–`2.0` (default `1.0`).



#### Text

Synthesizes one complete sentence using the current session config. The server
starts processing the text immediately and streams audio back. Sending another
`text` message while one is still being processed may throw an error — always
wait for `end_of_utterance` before sending the next sentence.

```json
{ "type": "text", "text": "நான் சரியாக இருக்கிறேன்." }
```

**Fields**

* `type` — always `"text"`.
* `text` — one complete sentence to synthesize (max 3000 Unicode characters).



#### Cancel

Interrupts the current utterance (barge-in). Send it while audio is
streaming to stop the turn early — useful when a user starts speaking or
the upstream text changes. Cancelling clears the worker's internal text
buffer, so it immediately stops generating audio for the remaining text
instead of finishing the queued utterance.

```json
{ "type": "cancel" }
```

**Fields**

* `type` — always `"cancel"`.

After sending `cancel`, keep reading from the socket — drain any remaining
binary audio frames — until the server's `cancellation` message arrives.
Stopping reads early can stall on WebSocket backpressure. The connection
stays open, so you can send the next `text` afterward. Sending `cancel`
while no utterance is in flight is acknowledged with a `cancellation` that
reports `characters_used: 0`.



#### Ping

Keepalive. The server replies with a `pong`.

```json
{ "type": "ping" }
```

**Fields**

* `type` — always `"ping"`.



#### Disconnect

Closes the session cleanly.

```json
{ "type": "disconnect" }
```

**Fields**

* `type` — always `"disconnect"`.

### Messages you receive

#### Connected

Sent once after the connection is authenticated.

```json
{
  "type": "connected",
  "user_id": "f1e2d3c4-..."
}
```

**Fields**

* `type` — always `"connected"`.
* `user_id` — the authenticated account's ID.



#### Configured

Confirms the active session config after a `config` message.

```json
{
  "type": "configured",
  "model_id": "raaga-v1",
  "voice": "Archana",
  "language": "ta-IN",
  "output_format": "pcm",
  "speed": 1.0
}
```

**Fields**

* `type` — always `"configured"`.
* `model_id`, `voice`, `language`, `output_format`, `speed` — the config
  now active for the session, echoed back from your `config` message.



#### End of utterance

Sent after all binary audio chunks for a sentence have been streamed.

```json
{
  "type": "end_of_utterance",
  "characters_used": 22,
  "duration_seconds": 1.8,
  "truncated": false
}
```

**Fields**

* `type` — always `"end_of_utterance"`.
* `characters_used` — credits consumed for this sentence.
* `duration_seconds` — length of the audio just streamed.
* `truncated` — `false` for a normal completion. If synthesis stopped early
  at a safety limit, this is `true` with `characters_used: 0`
  and a `reason` (such as `step_budget_exceeded`); truncated utterances are
  not billed.



#### Cancellation

Sent to acknowledge a `cancel` (barge-in). Replaces `end_of_utterance` for
the interrupted turn; the connection stays open.

```json
{
  "type": "cancellation",
  "characters_used": 12,
  "duration_seconds": 0.86
}
```

**Fields**

* `type` — always `"cancellation"`.
* `characters_used` — the worker-reported billable count for audio already
  produced before the cancel (the portion you are charged for). `0` when the
  cancel arrived before any audio or while idle.
* `duration_seconds` — length of the audio produced before cancellation.

Cancellation is distinct from truncation: a cancelled turn bills the
worker's `characters_used`, while a truncated turn always bills `0`.



#### Pong

Reply to a `ping`.

```json
{
  "type": "pong"
}
```

**Fields**

* `type` — always `"pong"`.



#### Error

Per-message problems (rate limit, concurrency limit, credits, validation). The connection stays open.

```json
{
  "type": "error",
  "error": {
    "code": "rate_limit_exceeded",
    "message": "Too many requests. Retry after 12 seconds.",
    "retry_after_seconds": 12
  }
}
```

**Fields**

* `type` — always `"error"`.
* `error.code` — machine-readable error code.
* `error.message` — human-readable description.
* `error.retry_after_seconds` — seconds to wait; present on `rate_limit_exceeded`. Not sent for `concurrency_limit_exceeded`, which has no fixed reset.

## Binary audio chunks

Audio chunks arrive as raw binary WebSocket frames (not base64). The default
`output_format` for WebSocket is `pcm` to avoid encoding overhead. Reassemble
chunks in arrival order for playback.

## Interrupting playback (barge-in)

Send a `cancel` message to stop the current utterance before it finishes — for
example, when the user starts talking over the audio or the upstream text
changes:

```json
{ "type": "cancel" }
```

The benefit of interrupting this way is that `cancel` clears the worker's
internal text buffer: instead of finishing the queued utterance, the worker
immediately drops the remaining text and stops generating audio for it.

What happens next:

1. The server stops sending further audio for the current turn and forwards the
   cancel upstream, which clears the worker's internal text buffer so no more
   audio is generated for the discarded text.

2. Keep reading from the socket until the `cancellation` frame arrives. Any
   audio frames still in flight may be delivered first, and stopping reads early
   can stall on WebSocket backpressure.

3. The server sends a `cancellation` message instead of `end_of_utterance`:

   ```json
   { "type": "cancellation", "characters_used": 12, "duration_seconds": 0.86 }
   ```

4. The connection stays open — send the next `text` whenever you're ready.

You are billed only for the audio produced before the cancel
(`characters_used` on the `cancellation` frame), not for the full input text. A
`cancel` sent while nothing is being synthesized is acknowledged with
`characters_used: 0` and is not billed.

## Idle timeout

The server closes idle connections after **60 seconds** without an incoming
message. Any message you send — including a `ping` — resets the idle timer, so
send a periodic `ping` to keep a session open between sentences.

Both SDKs send a keepalive `ping` for you by default, so you rarely need to do
this manually:

#### Python

```python
# The Python SDK sends keepalive pings while the session is open.
# You can also ping explicitly between utterances:
ws.ping()
```



#### JavaScript

```ts
// The JS SDK sends a JSON ping every 30 seconds by default.
// Tune or disable it with keepAliveIntervalMs:
const socket = await client.tts.websocket(
  { model_id: "raaga-v1", voice: "Archana", language: "ta-IN", output_format: "pcm" },
  { keepAliveIntervalMs: 30000 }, // false to disable
);
```

## Using the SDKs

Both SDKs handle the handshake, config, chunk reassembly, and `end_of_utterance`
for you. Each call to send a sentence resolves with the full audio for that
utterance.

#### Python

```python
with client.tts.websocket(
    model_id="raaga-v1",
    voice="Archana",
    language="ta-IN",
    output_format="pcm",
) as ws:
    result = ws.synthesize("நான் சரியாக இருக்கிறேன்.")
    result.save("sentence.pcm")
    print(result.characters_used, result.duration_seconds)
```



#### JavaScript

```ts
import { writeFile } from "node:fs/promises";
import { VakyamAIAsync } from "@vakyam-ai/tts";

const client = new VakyamAIAsync({ apiKey: process.env.VAKYAM_API_KEY! });

const socket = await client.tts.websocket({
  model_id: "raaga-v1",
  voice: "Archana",
  language: "ta-IN",
  output_format: "pcm",
});

const utterance = await socket.sendText("நான் சரியாக இருக்கிறேன்.");
await writeFile("sentence.pcm", utterance.audioBytes);

socket.close();
```

> **Warning:**
> Send only one sentence at a time and wait for it to finish. The JavaScript SDK
> throws if you call `sendText` again before the current utterance resolves.

## Per-message errors

Rate limit, concurrency limit, credit, and validation problems are returned as
JSON `error` messages and the connection stays open:

```json
{
  "type": "error",
  "error": {
    "code": "rate_limit_exceeded",
    "message": "Too many requests. Retry after 12 seconds.",
    "retry_after_seconds": 12
  }
}
```

Possible codes include `rate_limit_exceeded`, `concurrency_limit_exceeded`,
`insufficient_credits`, `voice_language_not_found`, `text_too_long`,
`missing_websocket_config`, `validation_error`, and `internal_error`.

## Close codes

| Code | Reason                                                                        |
| ---- | ----------------------------------------------------------------------------- |
| 1000 | Normal closure                                                                |
| 1011 | Session lost (concurrency or worker capacity revoked mid-session) — reconnect |
| 4001 | Authentication failed                                                         |
| 4002 | Internal server error                                                         |

---

*Powered by [holocron.so](https://holocron.so)*