asr_cmdline_tool

Overview

asr_cmdline_tool is a built-in command-line tool for running automatic speech recognition (ASR) inference on Genio Yocto platforms. It is a single entry point in front of the ASR runners, which are also pre-installed in the Yocto image:

Model	Runner	Description
Whisper	`main_whisper`	Encoder-decoder ASR. Reads a YAML configuration, runs the audio encoder DLA once, then decodes greedily with a forced start-of-transcript prefix.
Qwen3-ASR	`main_qwen3_asr`	Audio encoder DLA followed by an LLM. The encoder output is injected into the prompt as soft tokens, so the transcript comes out of the LLM’s generative loop.
Moonshine	`moonshine_asr`	Drives the Neuron Runtime API directly with three DLAs (encoder, decoder init, decoder). It has no YAML configuration; `run_moonshine_asr.sh` supplies the asset paths.

Every runner takes its audio as --wav <file>, and the input must be 16 kHz mono PCM16 WAV.

The tool does not hard-code model names. It resolves the model argument against a drop-in descriptor in /usr/share/asr-cmdline-tool/models/<model>.conf, which names the runner and, where applicable, the default YAML configuration. Listing the registered models therefore also reports whether each one is ready to run:

adb shell "asr_cmdline_tool --list"

Usage: asr_cmdline_tool <model> [--wav <file.wav>] [model args...]
Models:
  moonshine    run_moonshine_asr.sh     installed      assets: MISSING /usr/share/llm/moonshine-tiny
  qwen3-asr    main_qwen3_asr           installed      assets: MISSING /usr/share/llm/qwen3-asr-0.6b
  whisper      main_whisper             installed      assets: MISSING /usr/share/llm/whisper-base-8w16a

installed refers to the runner binary, which ships in the image. assets refers to the model package, which is downloaded and pushed separately as described below. A freshly flashed image shows every model as installed with assets: MISSING; that is expected.

Prerequisites

Before running inference:

Download the model package for your platform from Pre-built Model Packages in the Model Zoo.
Extract the package on your host PC:
```
unzip <ModelName>.zip
```

DLA assets are not part of the Yocto image. They are versioned by the model compiler and the APU configuration rather than by the image, so they are distributed as packages and pushed to the device.

Package Contents

Each package places its assets in directories grouped by role, plus a scripts/ folder with the push-and-run helpers. The three ASR packages differ only in which asset directories they carry.

whisper-base-8w16a/
├── encoder/
│   └── whisper-base_mtk_precision_8w16a_Overall_0.dla        # Audio encoder DLA
├── 256c/
│   └── whisper-base_mtk_precision_8w16a_Overall_1t256c_0.dla # Decoder DLA (1 token, 256 cache)
├── scripts/
│   ├── 00_detect_os.bat
│   ├── 01_push-models.bat
│   ├── 02_push-tokenizer.bat
│   ├── 03_set-performance.bat
│   ├── 04_run-inference.bat
│   ├── 00.wav                                  # Sample input, 16 kHz mono PCM16
│   ├── config-android_whisper-base-8w16a.yaml
│   ├── config-yocto_whisper-base-8w16a.yaml
│   ├── model_config.bat
│   ├── run_all_asr.bat
│   └── set_performance.sh
└── tokenizer/
    ├── added_tokens.yaml
    ├── embedding_int16.bin                     # Token embedding table
    ├── embedding_pos_int16.bin                 # Decoder positional embedding table
    ├── merges.txt
    └── vocab.txt

qwen3-asr-0.6b/
├── encoder/                  # Audio encoder DLA
├── llm_256t2048c_fms/        # Prompt-mode LLM DLA (FMS shell)
├── llm_1t2048c_fms/          # Generative-mode LLM DLA (FMS shell)
├── shared_weights_fms/       # LLM weights shared by both shells
├── scripts/                  # As above, plus 400.wav
└── tokenizer/

moonshine-tiny/
├── dla/
│   ├── encoder.dla
│   ├── decoder_init.dla
│   └── decoder.dla
├── scripts/                  # As above, minus the YAML files, plus test_audio_16k.wav
└── tokenizer/
    ├── vocab.bin
    └── embed_weights.bin

The Qwen3-ASR prompt and generative DLAs are thin FMS shells: the weights live in shared_weights_fms/ and are shared between them, which is why hot swapping between prompt and generative mode costs only tens of milliseconds.

Moonshine has no YAML configuration. Its CLI takes six positional paths, and run_moonshine_asr.sh in the image supplies them from the asset directory.

Run Inference

Quick Start

The scripts/ folder runs the whole flow. From the extracted package on a Windows host with ADB in the path:

cd <ModelName>\scripts
run_all_asr.bat

run_all_asr.bat calls the numbered scripts in order: push the model DLAs, push the tokenizer, apply performance mode, then push the sample WAV and run inference. 00_detect_os.bat detects the target OS and selects the device root, which is /usr/share/llm on Yocto.

The remaining sections describe the same steps manually.

Step 1: Push Model DLAs

Push the asset directories to the device. The relative layout must be preserved, because the YAML configuration and run_moonshine_asr.sh refer to these paths.

# Whisper
adb shell "mkdir -p /usr/share/llm/whisper-base-8w16a"
adb push whisper-base-8w16a/encoder /usr/share/llm/whisper-base-8w16a/encoder/
adb push whisper-base-8w16a/256c    /usr/share/llm/whisper-base-8w16a/256c/

# Qwen3-ASR
adb shell "mkdir -p /usr/share/llm/qwen3-asr-0.6b"
adb push qwen3-asr-0.6b/encoder            /usr/share/llm/qwen3-asr-0.6b/encoder/
adb push qwen3-asr-0.6b/llm_256t2048c_fms  /usr/share/llm/qwen3-asr-0.6b/llm_256t2048c_fms/
adb push qwen3-asr-0.6b/llm_1t2048c_fms    /usr/share/llm/qwen3-asr-0.6b/llm_1t2048c_fms/
adb push qwen3-asr-0.6b/shared_weights_fms /usr/share/llm/qwen3-asr-0.6b/shared_weights_fms/

# Moonshine
adb shell "mkdir -p /usr/share/llm/moonshine-tiny"
adb push moonshine-tiny/dla /usr/share/llm/moonshine-tiny/dla/

Step 2: Push Tokenizer

adb push <ModelName>/tokenizer /usr/share/llm/<ModelName>/

Step 3: Run asr_cmdline_tool

Push a WAV file and run inference. The tool injects the model’s default YAML configuration, so only the audio has to be supplied:

adb push whisper-base-8w16a/scripts/00.wav /usr/share/llm/whisper-base-8w16a/
adb shell "asr_cmdline_tool whisper --wav /usr/share/llm/whisper-base-8w16a/00.wav -m 128"

adb push qwen3-asr-0.6b/scripts/400.wav /usr/share/llm/qwen3-asr-0.6b/
adb shell "asr_cmdline_tool qwen3-asr --wav /usr/share/llm/qwen3-asr-0.6b/400.wav -m 200"

adb push moonshine-tiny/scripts/test_audio_16k.wav /usr/share/llm/moonshine-tiny/
adb shell "asr_cmdline_tool moonshine --wav /usr/share/llm/moonshine-tiny/test_audio_16k.wav"

Confirm that the assets were found:

adb shell "asr_cmdline_tool --list"

Models:
  moonshine    run_moonshine_asr.sh     installed      assets: ok (/usr/share/llm/moonshine-tiny)
  qwen3-asr    main_qwen3_asr           installed      assets: ok (/usr/share/llm/qwen3-asr-0.6b)
  whisper      main_whisper             installed      assets: ok (/usr/share/llm/whisper-base-8w16a)

To use a configuration other than the default, pass it after the model name. The runners take the last .yaml argument, so an explicitly supplied file wins over the injected default:

adb push whisper-base-8w16a/scripts/config-yocto_whisper-base-8w16a.yaml /usr/share/llm/whisper-base-8w16a/
adb shell "asr_cmdline_tool whisper \
    /usr/share/llm/whisper-base-8w16a/config-yocto_whisper-base-8w16a.yaml \
    --wav /usr/share/llm/whisper-base-8w16a/00.wav -m 128"

The runners can also be invoked directly, bypassing the dispatcher. In that case the configuration file is mandatory:

adb shell "main_whisper /usr/share/llm-cmdline-tool/config_whisper_base_8w16a.yaml \
    --wav /usr/share/llm/whisper-base-8w16a/00.wav -m 128"
adb shell "MODELS=/usr/share/llm/moonshine-tiny run_moonshine_asr.sh \
    --wav /usr/share/llm/moonshine-tiny/test_audio_16k.wav"

Parameters Reference

asr_cmdline_tool Parameters
Parameter	Required	Description
`<model>`	Yes	Registered model name: `whisper`, `qwen3-asr` or `moonshine`. Run `--list` to see which are present. An unknown name exits with status 2; a model whose runner is not installed exits with status 3.
`--list`	–	Print the registered models with the runner each resolves to, whether that runner is installed, and whether its assets are present on the device.
`--wav <file>`	Yes	Input audio, 16 kHz mono PCM16 WAV. Forwarded to the runner unchanged.
`[model args...]`	No	Everything after the model name is passed through to the runner. See the tables below.

main_whisper Parameters
Parameter	Required	Description
`<config_yaml>`	Yes	YAML configuration. Injected automatically when the model is launched through `asr_cmdline_tool`.
`--wav <file>`	Yes	Input audio.
`-m <max_tokens>`	No	Maximum number of tokens to generate. Defaults to 128.
`--tokens <ids>`	No	Forced start-of-transcript prefix, as a comma-separated list of token ids. Defaults to `50258,50259,50359,50363` which is start-of-transcript, English, transcribe, no-timestamps. The large-v3 family added a language token, which shifts the last two ids by one.

main_qwen3_asr Parameters
Parameter	Required	Description
`<config_yaml>`	Yes	YAML configuration. Injected automatically when launched through `asr_cmdline_tool`.
`--wav <file>`	Yes	Input audio.
`-m <max_tokens>`	No	Maximum number of tokens to generate. Defaults to 200.
`-p <text>`	No	Additional user text appended after the audio in the prompt, for example a language or formatting instruction.

run_moonshine_asr.sh Parameters
Parameter	Required	Description
`--wav <file>`	No	Input audio. Defaults to `test_audio_16k.wav` in the asset directory.
`MODELS=<dir>`	No	Environment variable overriding the asset directory. Defaults to `/usr/share/llm/moonshine-tiny`.

Configuration File

Whisper and Qwen3-ASR read a YAML configuration that describes the model architecture and the on-device asset paths. Each package ships a ready-to-use file in scripts/; the image also carries a copy under /usr/share/llm-cmdline-tool/, which is what the dispatcher injects. Update the paths under runtimeOptions if the assets are placed elsewhere.

# config-yocto_whisper-base-8w16a.yaml
modelOptions:
  promptTokenBatchSize: 1
  genTokenBatchSize: 1
  encTokenBatchSize: 1500     # Encoder sequence length (30 s of audio)
  cacheSize: 256              # Decoder self-attention cache
  hiddenSize: 512
  numHead: 8
  numLayer: 6
  maxTokenLength: 448
  vocabSize: 51865
  modelInputType: INT16       # Boundary types for the 8w16a build
  modelOutputType: INT16
  cacheType: INT16
  maskType: INT16

runtimeOptions:
  specialTokens:
    bosId: 50257
    eosId: 50257              # <|endoftext|>
    addBos: False             # The runner forces the SOT prefix itself
  tokenizerPath:
    - /usr/share/llm/whisper-base-8w16a/tokenizer/vocab.txt
    - /usr/share/llm/whisper-base-8w16a/tokenizer/merges.txt
    - /usr/share/llm/whisper-base-8w16a/tokenizer/added_tokens.yaml
  tokenEmbPath: /usr/share/llm/whisper-base-8w16a/tokenizer/embedding_int16.bin
  decPosEmbPath: /usr/share/llm/whisper-base-8w16a/tokenizer/embedding_pos_int16.bin
  dlaEncPath: /usr/share/llm/whisper-base-8w16a/encoder/whisper-base_mtk_precision_8w16a_Overall_0.dla
  dlaGenPaths:
    - /usr/share/llm/whisper-base-8w16a/256c/whisper-base_mtk_precision_8w16a_Overall_1t256c_0.dla

Qwen3-ASR uses the same structure with the LLM fields the other generative runners use (dlaPromptPaths, dlaGenPaths, sharedWeightsPaths) plus dlaEncPath for the audio encoder and specialTokens.audioPadTokenId for the placeholder the audio embeddings replace.

Expected Output

Whisper

The runner reports the encoder time and the generative token rate, then the transcript. With 00.wav from the package, decoding ends on the model’s own end-of-text token well inside the -m 128 limit:

INFO: Whisper IO check ok: numLayer=6 cache=262144B slot=1024B cross=1536000B
INFO: Init done in 0.0668729s
INFO: Vocab size: 51865
INFO: WAV: rate=16000 ch=1 samples=238720
INFO: Mel: n_mel=80 n_len=3000

[Forced tokens] 50258 50259 50359 50363

Response [Max = 128]:
 I wanted to share a few things, but I'm not going to not share as much as I wanted to share
 because we are starting late. I'd like to get this thing going so we all get home in a decent
 hour. This election is very important to us.</eos></end>

[Latency]
  Encoder (after WAV loaded -> encoder done): 160.644 ms
  Generative Mode: 174.766 token/s (5.72194 ms/token)

Qwen3-ASR

The transcript is emitted by the LLM, prefixed with the detected language. Prompt mode covers the audio soft tokens, so its token rate is reported alongside the generative rate:

INFO: Init done in 0.2982s
INFO: Vocab size: 151705
INFO: Mel: n_mel=128 n_len=3000 valid_frames=930
INFO: validFrames=930  audioTokens=121
INFO: Prompt tokens: 136
INFO: Audio encoder done in 0.404045s
INFO: Prompt done: 136 tokens in 1.06607s (127.571 tok/s)
INFO: Hot swap done in 0.0244652s

Response [Max = 200]:
language English<asr_text>Everywhere that slave morality gains the ascendancy, language shows a
tendency to approximate the significations of the word good and stupid.</eos></end>

[Latency]
      Prompt Mode: 127.571 tok/s
  Generative Mode: 7.1084 tok/s (140.679 ms/inference_pass)

Moonshine

Moonshine processes the audio in two-second chunks and prints each chunk’s transcript before the combined result. A trailing chunk shorter than the minimum length is skipped, so a 4 s clip announces three chunks and runs two:

[INFO] Loaded encoder.dla (1 in / 1 out), decoder_init.dla (4/25), decoder.dla (30/13)
[INFO] Audio: 4.08713 s (65394 samples @ 16000 Hz)
[INFO] Chunk 1/3  [0s - 2s] ... -> 'It was the best of time.'
[INFO] Chunk 2/3  [2.000000s - 4.000000s] ... -> 'Once it was the worst of times.'

============================================================
Transcription: It was the best of time. Once it was the worst of times.
============================================================

The reported rates should be within 5% of the reference performance data on the platform pages in the Model Zoo.

Troubleshooting

Symptom	Cause and resolution
`asr_cmdline_tool: unknown model '<name>'` (exit 2)	The name does not match a descriptor. Run `--list` for the registered names.
`'<model>' is registered but '<runner>' is not installed` (exit 3)	The runner binary is missing from the image. Confirm the image includes the ASR runners, for example with `command -v main_whisper`.
`--list` reports `assets: MISSING <dir>`	The model package has not been pushed, or it was pushed to a different directory. Push it to the directory named in the message, or point the runner at the actual location by passing your own YAML configuration.
`Missing asset: <dir>/dla/encoder.dla` from Moonshine	The package layout was not preserved. Push the `dla/` and `tokenizer/` directories as directories rather than copying individual files.
`adb push` to `/usr/share/llm` fails as read-only	Remount the root filesystem read-write with `adb shell "mount -o remount,rw /"`, or push the package elsewhere and pass a YAML configuration with matching paths. Moonshine also accepts `MODELS=<dir>`.
`WAV rate <n> != 16 kHz` warning, or a nonsensical transcript	The input is not 16 kHz mono PCM16. Convert it before pushing, for example `ffmpeg -i in.wav -ac 1 -ar 16000 -sample_fmt s16 out.wav`.