Skip to content

Mkulakow/update audio docs#4400

Open
michalkulakowski wants to merge 4 commits into
mainfrom
mkulakow/update_audio_docs
Open

Mkulakow/update audio docs#4400
michalkulakowski wants to merge 4 commits into
mainfrom
mkulakow/update_audio_docs

Conversation

@michalkulakowski

Copy link
Copy Markdown
Collaborator

🛠 Summary

JIRA/Issue if applicable.
Describe the changes.

🧪 Checklist

  • Unit tests added.
  • The documentation updated.
  • Change follows security best practices.
    ``

Copilot AI review requested due to automatic review settings July 24, 2026 11:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates OVMS audio-related documentation and demos to reflect current recommended text-to-speech (Kokoro) and expanded speech-to-text examples (Whisper + Qwen ASR) using the OpenAI-compatible /v3/audio/* endpoints.

Changes:

  • Switches TTS documentation/examples from SpeechT5 to Kokoro, including voice usage in request examples.
  • Expands the audio demo to include Qwen ASR alongside Whisper and reorganizes sections using tabbed layouts.
  • Updates OpenWebUI integration steps to use pre-exported Kokoro models pulled by OVMS.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
docs/parameters.md Updates CLI parameter documentation for audio (text2speech), including model type guidance.
docs/model_server_rest_api_text_to_speech.md Updates REST example request to Kokoro and adds voice.
docs/genai.md Updates GenAI speech generation blurb to reference Kokoro models.
docs/clients_genai.md Updates client examples (Python + curl) to use Kokoro + voice.
demos/integration_with_OpenWebUI/README.md Revises OpenWebUI demo steps to pull/add a pre-exported Kokoro model.
demos/audio/README.md Reworks audio demo instructions (TTS + STT) with tabs and adds Qwen ASR examples.
Comments suppressed due to low confidence (4)

demos/audio/README.md:149

  • This command is under the CPU heading but no longer sets --target_device CPU. Because OVMS auto-detects the target device by default, the command may not actually run on CPU on systems with an available GPU.
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 -v $(pwd)/models:/models:rw openvino/model_server:latest --rest_port 8000 --source_model dseditor/Qwen3-ASR-1.7B-INT8_OpenVINO --model_name Qwen3-ASR-1.7B-INT8_OpenVINO --model_repository_path /models

demos/audio/README.md:156

  • This command is under the GPU heading but no longer sets --target_device GPU. With the default auto-detection, the actual device selection may not match the heading.
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/models:rw openvino/model_server:latest-gpu --rest_port 8000 --source_model dseditor/Qwen3-ASR-1.7B-INT8_OpenVINO --model_name Qwen3-ASR-1.7B-INT8_OpenVINO --model_repository_path /models

demos/audio/README.md:572

  • This command is labeled CPU but does not set --target_device CPU. Since OVMS defaults to auto-detection, it may select a different device on systems with a GPU, making the CPU/GPU split misleading.
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 -v $(pwd)/models:/models:rw openvino/model_server:latest --rest_port 8000 --source_model OpenVINO/whisper-large-v3-fp16-ov --model_repository_path /models --model_name whisper-large-v3-fp16-ov

demos/audio/README.md:579

  • This command is labeled GPU but does not set --target_device GPU. With the default auto-detection, it may not run on GPU (or may fall back), so the example should be explicit.
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/models:rw openvino/model_server:latest-gpu --rest_port 8000 --source_model OpenVINO/whisper-large-v3-fp16-ov --model_repository_path /models --model_name whisper-large-v3-fp16-ov

Comment thread docs/parameters.md
Comment on lines 116 to 121
When pulling models outside of OpenVINO organization the optimum-cli api is used inside ovms. You can set additional parameters for this mode.
| Option | Value format | Description |
|------------------------------|--------------|---------------------------------------------------------------------------------------------------------------|
| `--extra_quantization_params`| `string` | Add advanced quantization parameters. Check [optimum-intel](https://github.com/huggingface/optimum-intel) documentation. Example: `--sym --group-size -1 --ratio 1.0 --awq --scale-estimation --dataset wikitext2` |
| `--weight-format` | `string` | Model precision used in optimum-cli export with conversion. Default `int8`. |
| `--vocoder` | `string` | The vocoder model to use for text2speech. For example `microsoft/speecht5_hifigan`. |

Comment thread docs/parameters.md
Comment on lines 188 to 193
### Text to speech
| option | Value format | Description |
|---------------------------|--------------|--------------------------------------------------------------------------------|
| `--num_streams` | `integer` | The number of parallel execution streams to use for the model. Use at least 2 on 2 socket CPU systems. Default: 1. |
| `--model_type` | `string` | Type of the source TTS model: `speecht5` (default) or `kokoro`. |
| `--vocoder` | `string` | The vocoder model to use for text2speech. For example `microsoft/speecht5_hifigan`. |
| `--model_type` | `string` | Type of the source TTS model. Supported values: `kokoro` (default, recommended) and `speecht5` (legacy). |

Comment thread demos/audio/README.md Outdated
```bash
mkdir -p models
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 -v $(pwd)/models:/models:rw openvino/model_server:latest --rest_port 8000 --task speech2text --source_model OpenVINO/whisper-large-v3-turbo-fp16-ov --model_name whisper-large-v3-turbo-fp16-ov --model_repository_path /models --target_device CPU
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 -v $(pwd)/models:/models:rw openvino/model_server:latest --rest_port 8000 --source_model OpenVINO/whisper-large-v3-turbo-fp16-ov --model_name whisper-large-v3-turbo-fp16-ov --model_repository_path /models
Comment thread demos/audio/README.md Outdated
```bash
mkdir -p models
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/models:rw openvino/model_server:latest-gpu --rest_port 8000 --task speech2text --source_model OpenVINO/whisper-large-v3-turbo-fp16-ov --model_name whisper-large-v3-turbo-fp16-ov --model_repository_path /models --target_device GPU
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/models:rw openvino/model_server:latest-gpu --rest_port 8000 --source_model OpenVINO/whisper-large-v3-turbo-fp16-ov --model_name whisper-large-v3-turbo-fp16-ov --model_repository_path /models
@michalkulakowski
michalkulakowski force-pushed the mkulakow/update_audio_docs branch from fa1228c to 1f73c03 Compare July 24, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants