Shipping
cadence
Provider-agnostic Go engine for low-latency STT→LLM→TTS voice pipelines — streaming engines behind uniform interfaces, swap vendors with a config change. Owns the hot path.
Measured
Latency budget
| Stage | Value | Note |
|---|---|---|
| LLM first token | 592 ms | vendor path, not cadence |
| Sentence buffer holding | 136 ms | cadence — near its floor for prose |
| TTS first audio byte | 289 ms | warm WebSocket, flat tax |
| VAD silence commit | 240 ms | cadence — the biggest knob I control |
Post-commit clock · 15 runs, one afternoon, one region · see the note for the full method
Shape
What it is
cadence sits between the telephony data plane and the model vendors. It owns the hot path: turn commitment, the sentence buffer that decides when accumulated text is worth speaking, and the warm WebSocket to the TTS engine.
Engines are behind uniform interfaces, so swapping a vendor is a config change rather than a rewrite. Nothing about a specific provider leaks into the pipeline.
Limitations
Streaming STT is server-endpointed; the batch path transcribes after the VAD commits, which adds finalization time the published benchmarks do not measure.
Explore