Balalaika is an end-to-end speech data pipeline: download or ingest audio, diarize and chunk it, filter low-quality material, run multi-model ASR with ROVER, restore punctuation and stress marks, phonemize text, and export the result to Parquet / WebDataset.
The current runner is stage-based (--stage / --stop_stage), so long jobs can
be resumed from any point without editing shell scripts.
Install system tools and create the development environment:
sudo apt update && sudo apt install -y \
ffmpeg \ # video/audio toolkit
python3 \ # Python
python3-pip \ # Pip package manager
python3-venv \ # std-lib virtual-env support
python3-dev \ # headers for compiling native wheels
python-is-python3
wget -qO- https://astral.sh/uv/install.sh | sh
git clone https://github.com/mtuciru/balalaika
cd balalaika
bash create_dev_env.shCreate .env in the repo root:
HF_TOKEN=<your_huggingface_token>
YANDEX_KEY=<your_yandex_music_token>Edit configs/config.yaml:
- set
podcasts_pathin every stage you plan to run; - set model paths under
preprocess,separation, etc.; - tune
runtime:(venv_path,cpu_affinity,log_dir, TensorRT cache).
Run the default tail of the pipeline (stages 11..14):
bash base.sh --config_path configs/config.yamlRun only a range of stages:
# Preprocess only: chunking -> crest filter -> loudness normalization
bash base.sh --config_path configs/config.yaml --stage 1 --stop_stage 3
# Transcription only
bash base.sh --config_path configs/config.yaml --stage 7 --stop_stage 7
# Regenerate the filtering report only
bash base.sh --config_path configs/config.yaml --stage 14 --stop_stage 14| ID | Stage | Module |
|---|---|---|
| 0 | Download from Yandex Music | src.download.download |
| 1 | Preprocess: diarization + chunking | src.preprocess.preprocess |
| 2 | Preprocess: crest-factor filter | src.preprocess.crest_factor_remover |
| 3 | Preprocess: loudness normalization | src.preprocess.preprocess_audio |
| 4 | Separation: music detection | src.separation.music_detect |
| 5 | Separation: DistillMOS scoring | src.separation.distillmos_process |
| 5.5 | Separation: DistillMOS filter | src.separation.distillmos_filter |
| 6 | Separation: Spectra-0 raw scoring | src.separation.antispoofing |
| 6.5 | Separation: anti-spoofing filter | src.separation.antispoofing_filter |
| 7 | Transcription + ROVER | src.transcription.transcription |
| 8 | Punctuation | src.punctuation.punctuation |
| 9 | Stress marks / accents | src.accents.accents |
| 10 | Phonemization | src.phonemizer.phonemizer |
| 11 | Denoising / enhancement | src.denoising.denoising |
| 12 | Collate to Parquet | src.collate |
| 13 | Export to WebDataset | src.to_webdataset |
| 14 | Filtering report | src.report |
base.sh --help prints the same map.
Shell-level values live in configs/config.yaml under runtime: instead of
being hardcoded in base.sh:
runtime:
venv_path: .dev_venv
cpu_affinity: "0-24" # empty string disables taskset
log_dir: ./logs
audio_paths_source: csv # csv avoids repeated rglob after balalaika.csv exists
work_shard_size: 10000 # file paths per on-disk multiprocessing shard
trt_cache_path: ./cache/trt
trt_workspace_bytes: 4294967296
trt_fp16: Truebase.sh reads this block through src.utils.runtime_env and exports
BALALAIKA_* variables before running stages. Python modules use the same
values for logging, TensorRT provider setup, CSV-backed file discovery, and
on-disk work-shard sizing. Heavy stages write work queues under
<podcasts_path>/.balalaika_work/<stage>/ so multiprocessing workers claim
small shard files instead of receiving millions of paths through pickle.
Balalaika avoids silent quality degradation:
preprocess.chunk_format: autopreserves the source container when chunking (.flacinput produces.flacchunks,.wavstays.wav).- Set
preprocess.chunk_formattoflac,wav,mp3,ogg, oropusonly when you explicitly want a fixed output container. - Loudness normalization writes FLAC/WAV through
soundfileas lossless containers. Lossy formats are handled bytorchaudio.save. - Denoising uses a dynamic ONNX export of ClearerVoice-Studio
MossFormer2_SE_48K, converts clips to 48 kHz mono, and overwrites audio in place before export. - WebDataset export copies the produced audio bytes as-is.
Every stage writes a timestamped log file:
<runtime.log_dir>/<stage>_YYYYMMDD-HHMMSS.log
Filtering stages append audit rows to:
<podcasts_path>/filter_summary.csv
The final report stage reads this CSV and writes:
<podcasts_path>/filter_report.md
The report shows files and hours kept / removed at each filtering step.
Main dataset-level files:
| File | Purpose |
|---|---|
balalaika.csv |
Per-chunk metadata and quality scores |
filter_summary.csv |
Machine-readable filtering audit |
filter_report.md |
Human-readable filtering report in hours |
balalaika.parquet |
Aggregated metadata for downstream training |
| WebDataset shards | Audio + JSON samples for large-scale loading |
Per-chunk sidecars include model ASR text, optional timestamps, ROVER output, punctuated text, accent-marked text, and phonemes.
- Preparing your dataset — expected folder layout and setup.
- Usage Guide — detailed stage behavior and commands.
- example/README.md — loading exported WebDataset shards.
Per-module notes live under src/*/README.md.
| Piece | Role |
|---|---|
| Sortformer (ONNX) | streaming diarization and speaker turns |
Smart Turn (smart-turn-v3.0.onnx) |
end-of-turn refinement |
| WavLM music detector | removes music-heavy chunks |
| DistillMOS | predicts speech quality score |
| Spectra-0 anti-spoofing (ONNX) | removes generated / spoofed speech |
| onnx-asr | ASR models and optional TensorRT |
| RUPunct | punctuation restoration |
| ruAccent | lexical stress marks |
| TryIParu | grapheme-to-phoneme conversion |
| ClearerVoice-Studio MossFormer2_SE_48K (ONNX export in pipeline) | denoising / speech enhancement |
@article{borodin2025datacentric,
title={A Data-Centric Framework for Addressing Phonetic and Prosodic Challenges in Russian Speech Generative Models},
author={Borodin, Kirill and Vasiliev, Nikita and Kudryavtsev, Vasiliy and Maslov, Maxim and Gorodnichev, Mikhail and Rogov, Oleg and Mkrtchian, Grach},
journal={arXiv preprint arXiv:2507.13563},
year={2025}
}Paper: arXiv:2507.13563
DOI: 10.48550/arXiv.2507.13563
| Piece | Role |
|---|---|
| Sortformer (ONNX) | streaming diarization, single-speaker slices |
Smart Turn (smart-turn-v3.0.onnx) |
end-of-speech / turn boundaries |
Music detector (music_detection.safetensors) |
drop music-heavy chunks |
| DistillMOS | predicted MOS in balalaika.csv |
| onnx-asr | GigaAM v3 CTC/RNNT, Vosk, T-one, Parakeet, Canary, Whisper, … |
| RUPunct | punctuation |
| ruAccent | stress marks |
TryIParu (tryiparu) |
grapheme → IPA |
| Spectra-0 | generated / spoofed speech detection |
| MossFormer2_SE_48K | 48 kHz denoising / speech enhancement via ONNX Runtime |
See LICENSE.