Why architecture choice matters

Three different voice ML architectures solve three different problems, but they get mixed up constantly. Picking the wrong one means either overpaying on compute and storage (using full ASR where KWS would do) or underwhelming the UX (trying to scale a wake-word into a task that needed KWS).

This article is a short decision guide. Three architectures (wake-word, KWS, streaming ASR), a profile of each with real numbers, hybrid flows, a decision tree by use case, and common mistakes we see in projects.

Numbers below are VoxRT reference measurements (our shipping SDKs). Competitor profiles vary. The point of this article is architectural fit, not vendor benchmark.

The three architectures

1. Wake-word (always-on trigger detection)

  • Model size: 100 KB (verified voxrt-wake-word-linux)
  • CPU: 5.3% RTF on Raspberry Pi Zero 2 W (one A53 core at 1 GHz)
  • Latency: under 100 ms detection
  • Vocabulary: 1 fixed trigger phrase ("Hey Assistant" or similar)
  • Use case: entry trigger for voice interaction (24/7 listening at minimal load)
  • Trade-off: cannot do anything beyond detecting the one phrase

The wake-word model is the smallest and cheapest of the three. Its job is to listen 24/7 on a minimal CPU budget, decide whether the trigger phrase was said, and hand off to another model or wake up the app. Not used standalone for actual commands.

Real projects sometimes ship 2 or 3 wake-word models in parallel (an entry trigger plus a barge-in stop phrase, for example). Each adds its own CPU share, but at 5% per model this is affordable on all but the most constrained hardware.

2. KWS (keyword spotting)

  • Model size: 1.3 MB (verified voxrt-kws-linux)
  • CPU: 32.2% RTF on Pi Zero 2 W
  • Latency: 120 ms sustained detection (3 consecutive detection frames at 25 fps) plus output
  • Vocabulary: 14 fixed commands (yes/no/cancel/play/pause/next/previous/up/down/back/on/off plus 2 wake-like)
  • Use case: closed-set voice control (media controls, UI navigation, binary switches, confirmation flows)
  • Trade-off: vocabulary is fixed. Custom words require retraining, not runtime config

KWS sits in the middle ground between wake-word and ASR. It knows a limited command set (14 in our case), runs real-time on cheap hardware like Pi Zero 2 W, and is more accurate for its vocabulary than open-vocab ASR because the confusion space is smaller. Underrated for DIY voice-control projects. Most projects reach for ASR when KWS would do the job.

Beyond raw detection, real KWS deployments spend time tuning per-command thresholds, FAR (false accept rate) vs FRR (false reject rate), and sometimes applying downstream confidence filtering. Default threshold 0.9 is a starting point, not a final answer.

3. Streaming ASR (open-vocabulary transcription)

  • Model size: 60.4 MB (verified voxrt-asr-models v0.1.2)
  • CPU: does not fit Pi Zero 2 W (overload). Works on Pi 4 and up, plus mobile class hardware
  • Latency: 200 to 500 ms first partial transcript, more to end of utterance
  • Vocabulary: open English speech (any word)
  • Use case: dictation, voice assistant queries, free-form user input, transcription
  • Trade-off: large model, requires meaningful hardware (mobile or Pi 4 minimum)

Streaming ASR is the "say anything" architecture. It accepts arbitrary speech and emits text transcripts incrementally. Expensive on compute and storage compared to WW and KWS, but the only path when you need to handle arbitrary user speech.

Many ASR engines also support hot-word biasing: a runtime list of words to weight higher in decoding (useful for custom product names, people names, domain vocabulary). Sherpa-onnx and whisper.cpp both support this. It is a middle path between the fully open vocabulary and the fully fixed KWS set.

Hybrid flows

Real production voice systems rarely use just one architecture. Common combinations:

Wake-word plus ASR

The voice assistant classic (Alexa, Siri, Google Assistant pattern). Wake-word listens 24/7 cheaply and triggers streaming ASR for the complex query. Battery efficient because ASR only runs when needed, not as a sustained drain.

Wake-word plus KWS

For constrained UIs: smart display, kiosk, embedded controls, hands-free lab setup. Wake-word gates attention and KWS handles the limited command set. Faster than ASR for known actions because you do not need to load a large model. A good fit for Pi Zero or Pi 4 with media or IoT controls.

Wake-word plus KWS plus ASR (full hybrid)

Pi 4 class or mobile hardware. Wake-word triggers, KWS tries a fast path for common commands, ASR falls back for anything else. Best latency for predictable commands with graceful fallback for complex phrases.

Edge-cloud hybrid

Dominant pattern in shipping consumer products: on-device wake-word triggers cloud ASR. Cheap always-on listening with heavyweight transcription offloaded. Trade-offs on privacy, latency, and ongoing cost. Not an on-device-only architecture, but worth naming because readers often pick it without realizing they have.

Orchestration note: most SDKs do not ship a turnkey multi-stage orchestrator with VAD + audio buffer carryover + state machine for free. Picovoice ships Porcupine + Rhino + Cheetah with documented reference flows. Sherpa-onnx has KeywordSpotter plus streaming ASR with sample orchestration code in the repo. Everyone else leaves the state machine ("waiting for trigger", "listening for command", "transcribing") to the integrator. Expect to spend meaningful time on orchestration code even if the model integration is simple.

Decision tree by use case

You need a single trigger phrase ("Hey X"):
→ Wake-word only. 100 KB, lowest CPU, works on everything down to microcontrollers.

You need 5 to 14 fixed commands (UI control, media, IoT):
→ KWS. Faster and more accurate than ASR for this scope. Works on Pi Zero 2 W, Pi 4, mobile. 1.3 MB model.

You need open vocabulary (dictation, user queries, free-form input):
→ Streaming ASR. Mobile class hardware minimum (Pi 4 or above). 60.4 MB model. No way around it for truly open vocabulary.

You need both commands and dictation:
→ Wake-word plus KWS plus ASR hybrid. Pi 4 or modern mobile minimum. Plan to write orchestration code yourself.

You need custom commands not in the 14-list:
→ Three options: ASR with hot-word biasing for a few extra terms, custom KWS training for a few dozen fixed commands, or open-vocabulary ASR for anything beyond that.

You need multi-language support:
→ Streaming ASR with multilingual models. Whisper multilingual, sherpa-onnx multilingual Zipformer, Vosk 20+ language packs. Wake-word and KWS vocabularies are language-locked to the model you ship.

Target is a noisy environment (room-wide, car cabin, 2+ meters):
→ Architecture choice is secondary. Microphone and audio frontend matter more (beamforming array, AEC, noise suppression) than WW vs KWS vs ASR. Expect to spend most engineering time on the acoustic pipeline.

Hardware is a microcontroller (ESP32 class):
→ Wake-word only. KWS and ASR do not fit in MCU budgets today. Picovoice Porcupine runs on Cortex-M. TensorFlow Lite Micro has reference keyword-spotting examples. Expect to build the acoustic frontend yourself (PDM mic to MFCC), budget around 10 mW total, and plan for per-device calibration.

Browser (WASM):
→ Wake-word via WASM works (see our browser wake-word article). KWS has limited WASM support across vendors. Streaming ASR is possible: sherpa-onnx has WASM builds, whisper.cpp has a WASM port (non-streaming). VoxRT itself does not ship a browser streaming ASR today.

Always-on considerations

A reader on our recent KWS tutorial raised an important point: benchmark RTF may be measured over a short run, but always-on behavior can differ. Sustained 24/7 inference stacks heat, especially in small cases on Pi Zero 2 W (no stock heatsink, PCB with limited copper heatspreading, RAM PoP-stacked on top of the SoC).

Rule of thumb:

  • Wake-word at 5% RTF: comfortable 24/7 on the cheapest hardware
  • KWS at 32% RTF: works 24/7 but worth monitoring vcgencmd get_throttled and measure_temp in production, plus watching for thermal throttling that kicks in mid-inference and drops frames
  • Streaming ASR: usually NOT run always-on (triggered by WW). If sustained, hardware choice is critical (Pi 4 with heatsink minimum)

Duty-cycling (sleep between inference bursts), CPU governor settings, and memory fragmentation over multi-day runs are the next layer of concern once the model fits.

Common mistakes

1. Using ASR when KWS would do. Overpaying 60.4 MB and mobile hardware for a 10-command UI. KWS at 1.3 MB does the same job with equal or better accuracy on target vocabulary. We see this regularly. "We just want voice control for 10 things" often ends up in ASR territory when it should not.

2. Using only wake-word when multiple commands are needed. Single trigger plus "did it fire?" does not scale beyond entry. Do not bolt separate WW models per command. Use KWS.

3. Not budgeting always-on CPU and thermal. Wake-word at 5% sustained sounds free. 24/7 means real battery drain and heat buildup. Measure, do not assume.

4. No VAD pre-gate before KWS or ASR. Running inference on silence or non-speech wastes compute and makes false-positive rates look worse than they are. A cheap VAD (Silero, WebRTC VAD) in front of KWS and ASR is the single biggest wasted-compute fix we see.

5. Audio buffer sizing and format mismatches. Ring buffer too small drops audio under load. Device-native 48 kHz float versus the 16 kHz mono int16 that voice models want is the number-one integration bug on fresh projects. Capture format mismatches silently produce garbage inference. Test the audio path end-to-end before trusting any accuracy number.

Picking the right fit

A short honest check. What is the actual voice input need?

  • Specific closed command set → KWS
  • Open phrases or dictation → streaming ASR
  • Just an entry trigger → wake-word
  • Mix of commands plus dictation → hybrid (orchestration cost is real)

One caveat on "pick the smallest architecture that handles the use case". If the product roadmap is likely to grow from fixed commands to dictation within a year, shipping KWS first and then re-integrating ASR later means two SDK integrations, two sets of user retraining expectations, and sometimes two license agreements. Future-proofing has a real cost, but so does over-engineering for a hypothetical. Judgment call.

All three architectures ship as production SDKs in 2026: VoxRT (hybrid license), Picovoice (commercial), sherpa-onnx (Apache-2.0 open), Vosk (Apache-2.0), whisper.cpp (streaming via sliding window, MIT). A prior comparison article On-device streaming ASR in 2026 goes deeper on per-vendor trade-offs for ASR specifically.

If a project does not fit cleanly into one category, you probably need a hybrid. If it does fit, pick the smallest architecture that handles the use case, not the biggest.