Introduction to Hydra
Hydra is Smallest AI's speech-native model for the most demanding real-time conversations. Built to reason continuously across long-running context, overlapping speech, interruptions, and tool-driven workflows while delivering fluid, low-latency responses.
Why voice agents stall
Most production voice agents use a cascaded pipeline. Every response passes through each stage. Latency accumulates across transcription, inference, tool execution, and speech generation. Turn detection adds another failure point around interruptions, pauses, and background speech.
Context decays over long calls
Summaries preserve the broad state of a conversation, but can lose details such as:
• Names and identifiers
• Dates and amounts
• User corrections
• Confirmation state
• Values returned by earlier tools
• Conditions introduced much earlier
System overview
Hydra replaces the conventional ASR → language model → text-to-speech cascade with a speech-native pipeline that remains in a continuous latent space. A unified conversational tokenizer converts incoming audio into ~50 ms latent frames that preserve semantics, prosody, timing, speaker characteristics, and emotional cues for the model to reason over.
As each causal audio frame arrives, Hydra continuously updates its internal conversational state and begins forming possible response trajectories before the speaker finishes. The latent conversational transformer refines those trajectories while listening, and a conditional decoder produces speech or text directly from the resulting state—supporting lower-latency turn-taking, natural backchannels, overlap, and fluid interruption handling without an intermediate transcript.
What Hydra handles
Hydra is optimized around three requirements of deployed voice systems: overlap-aware interaction, durable conversational state, and reliable tool execution.
In-session tool use
Hydra is evaluated on the complete tool-use loop:
• Selection: Choose the correct tool from a catalogue containing hard distractors.
• Arguments: Extract the expected values from speech.
• Deferred reuse: Reuse values returned by previous calls.
• Precision and recall: Avoid both unnecessary calls and missed required calls.
• Guardrails: Reject forbidden or premature operations.
• Clarification: Ask for missing information instead of guessing.
• Recovery: Handle tool errors and empty results without fabricating an answer.
• Spoken accuracy: State the retrieved facts correctly in the final response.
These capabilities allow Hydra to manage multi-step workflows without requiring the caller to repeat information between operations.
Tool use accuracy across 60 turns
ComplexFuncBenchmark (Single-turn tool calling)
Long-session state
Hydra tracks information across extended conversations and makes it available when required later. This includes:
• Names and identifiers
• Dates, times, and amounts
• Earlier corrections
• Accepted and rejected actions
• User preferences
• Values returned by tools
The goal is exact retention across turns. A value introduced early in a call should remain available for a decision or tool call much later.
Overlapping speech
Hydra is trained and evaluated on interaction states such as:
• Direct user interruptions
• Backchannels
• Speech directed at another person
• Background speech
• Mid-sentence corrections
• Input received during a response
The model must determine whether to stop, continue, yield, or resume without discarding the current state.