⚠️ Important: This project is not production ready and is still in early development. Breaking changes may occur, and backwards compatibility between alpha versions is not guaranteed.

Bewer is an evaluation and analysis framework for automatic speech recognition in Python. It defines a transparent YAML-based approach for configuring evaluation pipelines and makes it easy to inspect and analyze individual examples through a web-based interface. The built-in preprocessing pipeline and metrics collection are designed to cover all conventional use cases and then some, while still being fully extensible.

Contents | Installation | Quickstart |

Installation

pip install bewer

Quickstart

Create a Dataset

from bewer import Dataset

dataset = Dataset()

Add data

From a file:

dataset.load_csv(
    "data.csv",
    ref_col="reference",
    hyp_col="hypothesis",
)

Or manually:

for ref, hyp in iterator:
    dataset.add(ref=ref, hyp=hyp)

List available metrics

dataset.metrics.list_metrics()

Compute metrics lazily

print(f"WER: {dataset.metrics.wer().value:.2%}")

Freezing

Requesting a metric freezes the dataset: its contents can no longer change, so further add()/load_*() calls raise DatasetFrozenError. Use clone() for a fresh, modifiable copy to keep building.

dataset.metrics.wer()       # freezes the dataset
dataset.add(ref, hyp)       # raises DatasetFrozenError

extended = dataset.clone()  # modifiable copy with the same data
extended.add(ref, hyp)      # works