Browser SDK

Install

npm install @avaclone/sdk

Or, without a bundler, load the same ES module from the CDN:

<script type="module">
  import { AvaClone, MicCapture, CameraCapture } from "https://cdn.avaclone.ai/v1/sdk.js";
</script>

Mount

Your server creates a session with the API and hands the browser its ws_url, token and idle_loop_url. The browser mounts:

import { AvaClone, MicCapture } from "@avaclone/sdk";

const s = await fetch("/my-server/session").then((r) => r.json());   // your server called POST /api/v1/sessions

const avatar = AvaClone.mount(document.querySelector("#stage")!, {
  wsUrl: s.ws_url,
  token: s.token,
  idleLoopUrl: s.idle_loop_url,
  packed: s.packed,          // frames carry alpha (transparent background)
  lazy: true,                // dial only when the visitor begins
});

avatar.on("transcript", ({ role, text }) => console.log(role, text));
avatar.on("speaking", (on) => button.disabled = on);

const mic = new MicCapture();
await mic.start((pcm) => avatar.sendAudio(pcm));   // 24 kHz mono PCM, chunked

avatar.sendText("What are your opening hours?");   // typed questions work too

Mount options

Methods

Events

MicCapture and CameraCapture

MicCapture opens the microphone with echo cancellation and noise suppression, resamples to 24 kHz and calls you with PCM chunks. CameraCapture opens the camera (user or environment), optionally previews into an element, and keeps the last three seconds of frames in memory; flip() switches cameras on phones; sharpestRecent() is the best of those frames as a JPEG data URL. Neither sends anything by itself: audio goes only through sendAudio, and a camera still goes only when the agent asks for one or you call sendImage.

import { CameraCapture } from "@avaclone/sdk";
const cam = new CameraCapture();
await cam.start("environment", document.querySelector("#preview")!);
avatar.setCamera(cam);      // the agent asks for a still when the visitor refers to something in view

// or show it things yourself, from any camera, file or canvas
showButton.onclick = () => avatar.sendImage(cam);                        // the next question is answered seeing this
fileInput.onchange = () => avatar.sendImage(fileInput.files![0], "Do you stock this?");   // a photo, as a turn
avatar.on("image", ({ ok, reason }) => { if (!ok) console.warn(reason); });   // vision.off, image.rate, …
// later: avatar.setCamera(null); cam.stop();

Streaming your own speech (stream sessions)

A session of kind stream has no agent: your own model, knowledge and logic run on your side and AvaClone supplies the face and the voice. Two ways in:

• Text. sendText(line) each time your model has something to say. The line is read aloud in the session’s voice (the voice_id you gave POST /sessions, or the avatar’s own) and the face is lip-synced to it. Lines are spoken in the order sent; interrupt() cuts the current one and drops the rest. Symbols, money, units and markdown are spoken as words, and audio tags such as [chuckles] or [warmly] are performed on the Pro plan and above, never read aloud (the list).
• Audio. Send 24 kHz PCM of speech you generated yourself (your own TTS, a human operator, a recording) with sendAudio and the avatar lip-syncs to it. Call endUtterance() when a passage ends.

Tokens are drawn while the stream is live, 30 a minute on Standard.

Notes

• The element you mount into keeps its own positioning; the SDK only adds a canvas and video that fill it.
• Browsers require a user gesture before audio plays. Mount on load, but call sendText or start the microphone from a click.
• Frames arrive as JPEG at 25 fps with PCM audio in 0.96 s chunks; the SDK schedules both on one clock so lips and sound stay together.