Realtime emotion results
wss://devapi.imentiv.ai/v2/ws/realtime?session_id={session_id}
Query parameters
| Parameter | Required | Description |
|---|---|---|
session_id | Yes | From POST /v2/realtime/session. |
token | Only without the header | Fallback for clients that cannot set headers on a WebSocket upgrade — browsers, mainly. Prefer the X-API-Key header: a token in the URL ends up in access logs. |
Close codes
The connection is refused with 1008 and one of these reasons: Missing authentication token, Invalid or expired token, Missing session_id query parameter, or Insufficient credits for realtime session.
Imentiv sends — Emotion result stream
Frames arrive as they are ready, in no guaranteed order relative to each other. Video frames are frequent — up to about ten per second, sampled at a fixed minimum interval, so a 30 fps stream does not produce 30 results per second. Audio, transcript and text emotion arrive per speech segment. A text_emotion frame follows its transcript frame and matches on start_time_ms.
Every frame carries a type field — switch on it, then on data.source for emotion, which covers both facial and voice results.
emotion frames carry no session_id; the other three do. Parsing code that reads frame.session_id uniformly gets nothing on the frames you receive most often. You opened the socket with the session id, so key off your own value.
Ignore any type you do not recognise — more may be added.
Video emotion
Facial emotion scores for one analysed video frame.
| Field | Type | Required | Description |
|---|---|---|---|
type | "emotion" | — | Always emotion. Facial and voice results share one type — data.source is what tells them apart. |
timestamp | integer | Yes | Milliseconds since the session started. Not a Unix epoch: only timestamp_sent, on transcript and text-emotion frames, is a wall-clock time. |
frame_number | integer | Yes | Monotonic counter over the video frames analysed in this session. |
data | VideoEmotionData | Yes |
VideoEmotionData
| Field | Type | Required | Description |
|---|---|---|---|
source | "video" | — | Always video on this frame. Switch on type first, then on this. |
dominant | string | Yes | The highest-scoring label in scores. |
scores | object | Yes | The eight video labels, each 0…1: angry, contempt, disgust, fear, happy, neutral, sad, surprise. Read the keys rather than hardcoding the set — it may be extended. Note this is not the audio set: video has contempt where audio has calm, so the two are not interchangeable. |
valence | number or null | — | Circumplex valence in [-1, 1]. None if not computed. |
arousal | number or null | — | Circumplex arousal in [-1, 1]. None if not computed. |
intensity | number or null | — | Magnitude of (valence, arousal). Bounded by sqrt(2), the corner of the [-1,1]x[-1,1] valence/arousal box. None if not computed. |
expressiveness | number or null | — | 1 - p(neutral); how expressive the distribution is, distinct from intensity (see imentiv-face-analysis#144). None if not computed. |
emotion_entropy | number or null | — | Normalized Shannon entropy of the emotion distribution; 0 = certain, 1 = uniform/uncertain. None if not computed. |
circumplex_version | string or null | — | Which imentiv-face-analysis COORDINATES table produced valence/arousal/intensity/expressiveness/emotion_entropy (e.g. 'v1', 'v2'). None if not computed. |
Audio emotion
Voice-tone emotion scores for one analysed speech segment.
| Field | Type | Required | Description |
|---|---|---|---|
type | "emotion" | — | Always emotion — the same type a video frame uses. data.source distinguishes them. An audio frame carries no frame_number. |
timestamp | integer | Yes | Equal to data.start_time_ms: milliseconds since the session started, not a Unix epoch. |
data | AudioEmotionData | Yes |
AudioEmotionData
| Field | Type | Required | Description |
|---|---|---|---|
source | "audio" | — | Always audio on this frame. |
start_time_ms | integer | Yes | Segment start, in milliseconds from the start of the session. Also the join key: the matching text_emotion frame arrives carrying this same value. |
end_time_ms | integer | Yes | Segment end, in milliseconds from the start of the session. |
duration_ms | integer | Yes | Length of the speech segment in milliseconds. |
dominant | string | Yes | The highest-scoring label in scores. |
scores | object | Yes | The eight audio labels, each 0…1: angry, calm, disgust, fear, happy, neutral, sad, surprise. Audio has calm where video has contempt — do not merge the two sets into one chart without accounting for it. |
transcript | string | Yes | What was said during this segment. |
text_emotions | object | Yes | Zero-filled while the session is running. The real scores arrive on a separate text_emotion frame, joined on start_time_ms; rendering this field mid-session shows a wall of zeros. The exception is the final segment, finalised during POST /v2/realtime/session/{session_id}/end, which carries real values here and gets no follow-up frame. Code that joins on start_time_ms and falls back to the frame itself handles both. |
dominant_text_emotion | string or null | — | null while the session is running, for the same reason as text_emotions. Populated only on the final segment. |
valence | number or null | — | Circumplex valence in [-1, 1]. None if not computed. |
arousal | number or null | — | Circumplex arousal in [-1, 1]. None if not computed. |
intensity | number or null | — | Magnitude of (valence, arousal). Bounded by sqrt(2), the corner of the [-1,1]x[-1,1] valence/arousal box. None if not computed. |
expressiveness | number or null | — | 1 - p(neutral); how expressive the distribution is, distinct from intensity (see imentiv-face-analysis#144). None if not computed. |
emotion_entropy | number or null | — | Normalized Shannon entropy of the emotion distribution; 0 = certain, 1 = uniform/uncertain. None if not computed. |
circumplex_version | string or null | — | Which imentiv-face-analysis COORDINATES table produced valence/arousal/intensity/expressiveness/emotion_entropy (e.g. 'v1', 'v2'). None if not computed. |
Transcript
Speech transcribed from the audio track.
| Field | Type | Required | Description |
|---|---|---|---|
type | string | — | Always transcript. |
session_id | string | Yes | The session this frame belongs to. Present here and on text_emotion, but absent from emotion frames — see the operation description. |
data | TranscriptData | Yes |
TranscriptData
| Field | Type | Required | Description |
|---|---|---|---|
transcript | string | Yes | The text so far, or the complete text once is_final is true. Replace your stored value on each update; appending duplicates every word. |
language | string | — | Detected language code, e.g. en. |
start_time_ms | integer | Yes | Segment start, in milliseconds from the start of the session. |
end_time_ms | integer | Yes | Segment end, in milliseconds from the start of the session. |
transcript_id | string | Yes | Groups every frame belonging to one utterance. Key your display on this. |
seq | integer | Yes | Increments across the partial frames of one transcript_id. |
is_final | boolean | — | true once the utterance is closed. Only the final frame is punctuation-restored. |
timestamp_sent | integer | — | Server send time as a Unix epoch in milliseconds — a real wall-clock time, unlike timestamp on an emotion frame. |
Text emotion
Emotional content of the transcribed speech. Arrives after the matching transcript, as a separate follow-up frame. Best-effort: if analysis fails or times out for a segment, no frame arrives for it — treat absence as 'no update available', never as an error or a zero result.
| Field | Type | Required | Description |
|---|---|---|---|
type | "text_emotion" | — | Always text_emotion. This frame has no source field. |
session_id | string | Yes | The session this frame belongs to. |
data | TextEmotionData | Yes |
TextEmotionData
| Field | Type | Required | Description |
|---|---|---|---|
start_time_ms | integer | Yes | Join key — matches the start_time_ms of the audio emotion frame this analysis belongs to. |
end_time_ms | integer | Yes | Segment end, in milliseconds from the start of the session. |
text_emotions | object | Yes | The text labels, each 0…1. Twenty-eight today; read the keys rather than hardcoding the set — up to four more (sarcasm, boredom, contempt, envy) may appear. |
dominant_text_emotion | string | Yes | The highest-scoring label in text_emotions. |
timestamp_sent | integer | — | Server send time as a Unix epoch in milliseconds. |
Error
Something went wrong, or the session was cut off.
| Field | Type | Required | Description |
|---|---|---|---|
type | "error" | Yes | |
session_id | string | — | The session this error belongs to. |
error | string | Yes | Human-readable message. Display it, but never branch on it — the wording is free to change. |
code | string | — | Stable, machine-readable reason. Branch on this. insufficient_credits means the session was cut off mid-stream. |
credits | object | — | Spend summary for the session, present on the credit-cutoff frame. That path cannot use POST /v2/realtime/session/{id}/end — the session is torn down immediately, so a follow-up call would 404. |