Many embedded voice-control examples use one detector for one keyword:

  • turn on the light
  • turn off the light
  • increase volume

That works, but loading a separate model for every command is wasteful. A better pattern is a multi-keyword model: one model, shared feature extraction, one inference pass and multiple command IDs.

This is functionally similar to the role of MultiNet in ESP-SR, while using a portable TFLite model delivery path.

The model interface

The firmware does not need to treat every phrase as a separate application.

A multi-keyword model can expose results like:

0 -> no command 1 -> turn on the light 2 -> turn off the light 3 -> increase volume 4 -> decrease volume

The application maps the result to an action:

switch (command_id) { case 1: light_on(); break; case 2: light_off(); break; case 3: volume_up(); break; }

The exact API depends on the runtime integration, but the important design is stable: the model recognizes the fixed command set, and the firmware owns the device behavior.

Why one model is useful

A shared multi-keyword model can provide:

  • One feature-extraction path
  • One model invocation per audio window
  • A consistent thresholding strategy
  • A smaller integration surface
  • Better joint testing of similar commands

It is not simply a folder containing several unrelated single-keyword models. The commands are evaluated together, which is important when phrases sound similar.

How many commands?

For a small voice-controlled product, start with a small fixed vocabulary. Two to ten commands is a practical range for an embedded command interface:

  • lights on
  • lights off
  • brighter
  • darker
  • next mode
  • previous mode

More commands increase the confusion surface. Similar phrases should be tested as hard negatives, not added only after the model fails in production.

ESP32 deployment

The open-source ESP32 example is here:

https://github.com/voicute/onnx-wakeword/tree/main/esp32

The deployment shape is:

microphone -> 16 kHz mono PCM -> feature extraction -> INT8 TFLite model -> command ID -> ESP-IDF action

A wake word detector can start the interaction, after which the multi-keyword model listens for the fixed command set. In some products the command model can also run directly as the always-on detector, depending on the UX and power budget.

Do not confuse command recognition with ASR

This pattern is not general speech recognition.

It is a small-vocabulary classifier for known phrases. That is why it can be much smaller and cheaper to run locally than an ASR system. It is a good fit for:

  • Lights
  • Fans
  • Modes
  • Appliance controls
  • Simple robot actions
  • Fixed industrial commands

It is not a replacement for dictation or open-ended conversational speech.

Test commands as a group

For each command, measure:

  • Positive recall
  • Confusion with other commands
  • False triggers from unrelated speech
  • Detection delay
  • Performance at different distances
  • Performance with the device speaker playing

The most important test is often not “does command A work?” but “does command A remain distinct from commands B and C in the same room?”

Final takeaway

If your ESP32 product needs a handful of fixed voice commands, one multi-keyword TFLite model is often a cleaner architecture than one model per phrase.

The model handles recognition. The firmware handles command IDs and actions.

You can find the example runtime at voicute/onnx-wakeword, and the multi-keyword training workflow at Voicute.