What happens between speech and a caption

A live caption is a small miracle of scheduling: a sentence becomes a caption before the speaker has moved on. Behind the overlay there are four stages, and knowing them explains both the delays and the mistakes you see.

1. Capture and encode

Browsers hand audio to JavaScript as 32-bit floats at the device's sample rate, often 48 kHz. Speech models expect 16 kHz mono 16-bit PCM. TanLens runs that conversion in an AudioWorklet so it never blocks the interface, then sends roughly two seconds of audio per request.

Two seconds is a deliberate trade-off. Longer chunks give the recognizer more context; shorter chunks cut latency but starve it of words. For conversation, two seconds works; for a lecture with long sentences, slightly longer chunks tend to punctuate better.

2. Speech recognition

The recognizer turns audio into words with timings. Streaming providers such as Deepgram return interim hypotheses as you speak, which is what makes subtitles feel instant. Request-based providers such as Whisper return a complete utterance per chunk, which is slower but often more accurate on noisy audio.

Whatever provider you use, the model needs punctuation and sentence boundaries. Without them, the translation stage receives a wall of words and produces a wall of translated words.

3. Translation

A caption translator is not a document translator. It should see the previous few sentences, keep names unchanged unless a glossary says otherwise, and write short natural phrases that can be read at speaking speed.

TanLens sends a numbered batch of lines to the translation model with the recent context and the user's glossary, then maps the numbered output back to the original segments. If the model returns something wrong-shaped, the source text is shown instead of a broken caption — a missing translation is better than a misleading one.

4. Rendering

The overlay should update without reflowing the page. TanLens uses a Shadow DOM subtree, fixed positioning, and no layout-affecting animations. Partial lines fade in; final lines replace them. Screen readers get the same text through an aria-live region.

Where the latency actually goes

The current HTTP path first collects a two-second audio chunk. Upload, recognition, translation, and rendering are added after that, and all vary by network and provider. TanLens has not published production p50 or p95 measurements yet; any future latency claim should come from end-to-end telemetry rather than one provider stage.

What you can tune

• Choose a streaming recognizer when you need speed; a request-based one when you need accuracy on bad audio.
• Keep the target language fixed during a session so the glossary and context stay consistent.
• Use short sentences. Captions are read, not studied.
• Do not caption DRM-protected players and expect tab capture to work; test the specific title first.

The honest summary: live captions are a pipeline, not a model. The best results come from choosing per-stage trade-offs that match the content, not from chasing a single benchmark number.