What is Speech-to-speech (S2S)?

Speech-to-speech is an approach to voice AI in which a single model takes the caller's audio as input and produces spoken audio as output directly, instead of passing through separate speech-to-text, language model and text-to-speech stages. The conventional cascaded pipeline converts speech to text, reasons over the text and synthesises a reply, and each hand-off adds delay and discards tone of voice. Speech-to-speech models can respond faster and carry emotion and emphasis through, but they are harder to control: it is more difficult to inspect what the model decided, enforce guardrails or call tools reliably mid-conversation. Indian languages and code-switched speech add further difficulty because training data is thinner.

Architecture matters to a business only through what the caller experiences: response speed, naturalness and whether the agent stays on script. The practical test is end-to-end latency and behaviour on real calls in the target language. Outpero runs at 0.8 to 1.3 seconds on average end-to-end conversational latency, from the caller stopping to the agent speaking, and keeps a full transcript of every call so its behaviour can be checked.