Realtime emotion analysis

Analyse emotion from a live camera and microphone feed, as it happens. You stream media to Imentiv over WebRTC and receive facial emotion, voice-tone emotion, transcription and text emotion back over a WebSocket, typically within a second of it happening.

How the pieces fit together

Three connections, all tied together by one session_id.

                      ┌─────────────────────────────────┐
   your app  ────────▶│  REST — set up and tear down    │
                      │  create · offer · ice · end     │
                      └─────────────────────────────────┘
                                     │  session_id
                      ┌──────────────┴──────────────────┐
   camera + mic ─────▶│  WebRTC — your media goes up    │
                      └─────────────────────────────────┘
                      ┌─────────────────────────────────┐
   results  ◀─────────│  WebSocket — results come back  │
                      │  emotion · transcript · error   │
                      └─────────────────────────────────┘

REST creates the session and tears it down. WebRTC carries the audio and video — Imentiv never receives your media over the WebSocket. The WebSocket carries results back, server to client only; nothing you send up it is interpreted.

Before you start

• An API key. Send it as X-API-Key: <your-key> on every request — verbatim, with no prefix. See Authentication.
• A WebRTC client. In a browser that's built in; on a server, use a WebRTC library.
• Enough credits. Running out mid-session is a specific, handled condition — see Errors and retries.

The flow, step by step

1. Create the session

POST /v2/realtime/session → Create realtime emotion session

Returns a session_id and an ice_servers list. Pass ice_servers straight into your WebRTC connection — it includes short-lived credentials for getting through firewalls.

Set audio_only: true if there's no camera. If the service is at capacity you get a 503 with "code": "at_capacity" — that's a "try again shortly", not a failure in your request.

2. Connect the media

POST /v2/realtime/webrtc/offer → Forward WebRTC offer POST /v2/realtime/webrtc/ice → Forward ICE candidate

Create a WebRTC offer, send it, and apply the answer you get back. The server's own network candidates are already inside that answer, so there's nothing extra to fetch. Send your own candidates to the ice endpoint as your browser discovers them.

3. Open the results connection

wss://devapi.imentiv.ai/v2/ws/realtime?session_id=... → Realtime emotion results

This is where the analysis arrives. Every message has a type:

type	What it carries
emotion	Facial or voice emotion scores — check data.source for which
transcript	What was said
text_emotion	The emotional content of what was said
error	Something went wrong, including running out of credits

Open this connection promptly. Nothing is replayed, so whatever is emitted before you connect is lost.

4. End the session

POST /v2/realtime/session/{session_id}/end → End realtime emotion session

Closes the session and returns a summary of what was analysed. Always call it — it frees the analysis slot for the next customer. A session you never end keeps that slot until it hits the duration ceiling; see Limits and media.

If the connection drops instead, the session ends on its own. There is no reconnecting to a session: create a new one and start again from step 1.

Putting it together

// 1. create
const session = await fetch("https://devapi.imentiv.ai/v2/realtime/session", {
  method: "POST",
  headers: { "X-API-Key": apiKey, "Content-Type": "application/json" },
  body: JSON.stringify({ audio_only: false }),
}).then((r) => r.json());

// 2. connect the media
const pc = new RTCPeerConnection({ iceServers: session.ice_servers });
const media = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });
media.getTracks().forEach((track) => pc.addTrack(track, media));
const offer = await pc.createOffer();
await pc.setLocalDescription(offer);

const answer = await fetch("https://devapi.imentiv.ai/v2/realtime/webrtc/offer", {
  method: "POST",
  headers: { "X-API-Key": apiKey, "Content-Type": "application/json" },
  body: JSON.stringify({ session_id: session.session_id, sdp: offer.sdp, type: "offer" }),
}).then((r) => r.json());
await pc.setRemoteDescription({ type: "answer", sdp: answer.sdp });

// 3. collect the results
const ws = new WebSocket(
  `wss://devapi.imentiv.ai/v2/ws/realtime?session_id=${session.session_id}&token=${apiKey}`
);
ws.onmessage = (event) => {
  const frame = JSON.parse(event.data);
  if (frame.type === "emotion") console.log(frame.data.source, frame.data.dominant);
  if (frame.type === "transcript") console.log("said:", frame.data.transcript);
};

// 4. end it
await fetch(`https://devapi.imentiv.ai/v2/realtime/session/${session.session_id}/end`, {
  method: "POST",
  headers: { "X-API-Key": apiKey },
});

A browser cannot set headers when opening a WebSocket, which is why the example above falls back to ?token=. Anywhere you can set headers, prefer X-API-Key — a token in a URL ends up in server logs.

That is the shape, not a client you should ship: it skips the ordering rules around teardown, and it calls /end unconditionally, which 404s if the server already ended the session. The realtime session quickstart handles both.

When things go wrong

What you see	What it means
503 with "code": "at_capacity"	All analysis slots are busy. Retry shortly.
WebSocket closes with code 1008	Authentication, a missing session_id, or too few credits. The close reason says which.
error frame, code: insufficient_credits	Out of credits. The session has ended.
Media connects but no frames arrive	The WebRTC connection didn't establish — check that you passed ice_servers through from step 1.