Speech is the oldest interface we have. Long before writing, long before screens, people learned the world by listening to one another, and we still judge a voice in the first fraction of a second: the pause before an answer tells us whether we are talking to someone who is listening or to something that is processing. A voice that answers at once feels alive. A voice that makes us wait feels like a machine reading from a script. Between the two lies a gap of a few seconds, and inside that gap lives most of the difference between a tool we tolerate and a companion we trust.
In most products today, the voice you hear from a device is borrowed. Your words go up to a server somewhere, the sound comes back down, and the machine in your home is only a speaker at the end of a very long wire. Meanwhile, open models have quietly crossed a threshold. Qwen3-TTS-12Hz-0.6B-CustomVoice, a 0.6-billion-parameter model with nine distinct voices and natural prosody, is released under Apache-2.0. The question is no longer whether a machine can speak well. It is whether the small, inexpensive machine already sitting in the corner — a storage box, a mini PC, an industrial gateway — can speak well by itself.
The Twenty-Second Pause
The machine we chose is deliberately ordinary: an Intel Core Ultra 5 225H, a laptop-class chip with four performance cores, eight efficiency cores and two low-power cores, an Arc integrated GPU, and no discrete graphics card at all. It has 32 GB of memory, runs Linux with a read-only system image, and was busy serving its owner’s other workloads the entire time we were on it. We installed nothing into the system, changed no driver or firmware, and used no containers. Everything we deployed lives in one folder on the data volume.
On that box, both the official PyTorch implementation and Intel’s OpenVINO reference notebook speak the same way: they compose the whole sentence first and only then begin to say it — like a person who insists on writing out an entire reply before reading it aloud. We measure speed as the real-time factor, the seconds of computation needed for each second of speech; anything above one cannot keep up with its own voice. On three sealed test sentences, the reference paths landed between 2.19 and 2.85, and the first sound arrived after fifteen to twenty seconds.
Think about the last time you asked a device something and waited. How long before the wait itself became the answer? Twenty seconds for a single sentence is not a product. It is a demonstration that a product is possible.
Speaking as You Think
People do not compose a paragraph before they open their mouths. They speak a few words, and while those words are still in the air they are already shaping the next. We rebuilt the inference path to work the same way. The model’s language component predicts sound frame by frame, each frame eighty milliseconds long. Every five frames — four tenths of a second of speech — are handed to a stateful vocoder running on the integrated GPU, which turns them into waveform while the CPU is already at work on the block that follows. The audio flows to the loudspeaker through a bounded queue as it is made, never waiting for the end of the sentence.
The numbers come from a thirty-minute resident run on the target machine: one process, 357 sentences, all nine voices in rotation, with ten seconds of idle between rounds so that it looks like a room rather than a benchmark. The first sound arrived 0.37 seconds after the text, every time within 0.35 to 0.54 seconds. The real-time factor settled at a median of 0.734 — the machine now generates speech 1.36 times faster than it can be played. Memory peaked at 2.4 GB and then stayed flat, growing by 756 kilobytes across the entire half hour. All 357 sentences completed, with no player errors and no mid-stream underruns, and every one of them produced acoustic tokens identical, bit for bit, to our reference run of the same text and voice.
Set against the batch INT8 reference on the same machine, that is three to four times less computation per second of speech, a first sound that arrives about forty times sooner, and half the peak memory: 2.4 GB in place of 4.7 GB. The model is ready 6.7 seconds after the engine starts and then stays in memory; it switches between all nine voices without reloading, stops a sentence mid-way at a keystroke, and if a sentence fails, the next one still plays.
“A machine that speaks in a third of a second is not doing the same thing faster. It is doing a different thing.”
A machine that speaks in a third of a second is not doing the same thing faster. It is doing a different thing. Twenty seconds is a request and a delivery; a third of a second is a reply. The first belongs to a workflow, the second to a conversation, and no amount of quality in the audio can move a voice from the first category into the second. Latency, at this scale, is not a performance metric. It is the difference between being read to and being spoken with.
What We Changed
None of this came from a faster chip. It came from six pieces of AtomGradient’s edge inference stack. Each of them has a name and a measured effect. None of them will be described in detail here.
The first is a stateful streaming vocoder. Our streaming decoder was born on Apple silicon; we carried the same algorithm to OpenVINO so that it runs on Intel’s integrated GPU. It decodes speech in 0.4-second blocks and carries its own state from one block to the next, so the seams disappear: block-by-block output matches whole-sentence output to 115.9–116.4 dB signal-to-noise. Along the way we fixed the reference decoder’s dynamic-shape export, which took a 31-frame decode from 4.56 seconds to 0.47.
The second is head-split inference for the code predictor. Qwen3-TTS predicts sixteen acoustic codebooks for every frame, fifteen of them through a small loop that runs hundreds of times per second of speech. We restructured that loop so that each step touches only the weights it actually needs. The fifteen-step loop fell from 40.1 to 32.3 milliseconds, three test sentences went from real-time factors of 0.93, 0.83 and 0.84 to 0.80, 0.74 and 0.72, and the output did not change by a single bit.
The third is an INT8 weight compression scheme: which components to compress to eight-bit weights, how, and how to prove that nothing was lost. The main language component shrank from 887 MB to 444 MB and the code predictor from 220 MB to 110 MB; the predictor’s per-frame loop sped up from about 79 to 45 milliseconds; the vocoder stays in FP16. In a blind listening test, two Chinese samples could not be told apart from the uncompressed version.
The fourth is an asynchronous pipeline across the CPU and the integrated GPU. Token generation runs on four performance cores, the vocoder runs on the Arc graphics, at most one decode is in flight at a time, and blocks leave in order. Decoding overlaps with generation instead of queuing behind it.
The fifth is performance-core affinity scheduling. A hybrid chip is a team of four sprinters and ten walkers, and for a relay made of thousands of tiny steps, the intuitive move — put everyone on the track — is the wrong one. Seven single-variable experiments took the resident real-time factor from 1.06 with eight unpinned threads to 0.86 with four threads pinned to the performance cores, measured before the head split. The walkers joining in made everyone slower.
The sixth is a resident service framework: one process, nine voices, no reloads; an audio queue that can never hold more than one sentence; cancel-and-continue; and isolation of any sentence that fails. After a deliberate cancellation, and after an injected failure of the audio player, the next sentence came out bit-identical to a normal run. This is the piece that turns a fast model into something a person can leave running.
Same Algorithms, Different Silicon
The part of this project we care about most is not the Intel box. It is that the streaming engine we built for Apple silicon — Metal on iPhone and iPad — moved to x86 with an Intel integrated GPU without changing what the algorithms do. The vocoder’s state handling, the block schedule, the bounded output, the cancel semantics: all of it came across. What changed was the execution backend.
“The intelligence lives in the algorithms. The silicon underneath is a deployment detail.”
The intelligence lives in the algorithms. The silicon underneath is a deployment detail. That is what an edge AI stack should look like, and it is why we can say with a straight face that the same voice engine belongs on a phone, on a storage box, in a car, and inside a robot. The hardware in each of those places is different. The thinking is the same.
A Voice That Stays Home
When speech is generated where it is heard, several things follow at once. The words you type and the voice you hear never leave the device; there is no network path through which they could. A third of a second to the first sound is the difference between a voice that responds and a voice that reads back. And it all runs on hardware people already own — no discrete GPU, 2.4 GB of memory, four of fourteen cores — while the rest of a 32 GB box keeps doing its day job. A resident service whose memory stays flat across 357 sentences has the shape of a product, not the shape of a demo.
The same shape fits a home robot, a kiosk, an assistant in a car, and any device that ought to be able to speak without asking a data centre for permission. A 0.6-billion-parameter model that speaks in a third of a second on a small x86 box is not a research curiosity. It is the point at which a voice interface stops needing the cloud.
Let the machines in our homes find their own voices — and keep them there.