fgmetric

Type-validated Python models for delimited data files.

Visit us at Fulcrum Genomics to learn more about how we can power your Bioinformatics with fgmetric and beyond.

Overview

fgmetric lets you define Python classes ("Metrics") that map directly to rows in CSV/TSV files. It handles parsing, type coercion (strings → int, float, bool), and validation automatically using Pydantic.

Installation

Requires Python 3.12 or later.

pip install fgmetric

Or with uv:

uv add fgmetric

Why fgmetric?

If you're a bioinformatician or data engineer processing delimited files in Python, you've probably written code like this:

import csv

with open("metrics.tsv") as f:
    reader = csv.DictReader(f, delimiter="\t")
    for row in reader:
        quality = int(row["mapping_quality"])
        is_duplicate = row["is_duplicate"].lower() in ("true", "1", "yes")
        if row["score"]:  # handle empty strings
            score = float(row["score"])
        # ... repeat for every field

fgmetric replaces this with:

for metric in AlignmentMetric.read(path):
    # metric.mapping_quality is already an int
    # metric.is_duplicate is already a bool
    # metric.score is already Optional[float]

How it compares:

• vs. csv + dataclasses — Automatic type coercion and validation without boilerplate. Built on Pydantic, so additional custom validators and serializers can be readily added.
• vs. pandas — Unlike pandas, fgmetric processes records lazily — you can handle files larger than memory. And Metrics are type-validated and can be made immutable, making them safe to pass between functions without defensive copying.
• vs. Pydantic alone — fgmetric handles CSV/TSV specifics (header parsing, delimiter configuration) and provides out-of-the box features like empty value handling and Counter field pivoting.

Quick Start

Define a class to represent each row:

from fgmetric import Metric, MetricReader, MetricWriter


class AlignmentMetric(Metric):
    read_name: str
    mapping_quality: int
    is_duplicate: bool = False

Then read or write:

# Reading
for metric in AlignmentMetric.read("alignments.tsv"):
    print(f"{metric.read_name}: MQ={metric.mapping_quality}")

# Writing
metrics = [
    AlignmentMetric(read_name="read1", mapping_quality=60),
    AlignmentMetric(read_name="read2", mapping_quality=30, is_duplicate=True),
]
with MetricWriter.open(AlignmentMetric, "output.tsv") as writer:
    writer.writeall(metrics)

Metric.read() reads the whole file into a list. To stream metrics one at a time without holding them all in memory, use MetricReader.open():

# Streaming from a path
with MetricReader.open(AlignmentMetric, "alignments.tsv") as reader:
    for metric in reader:
        print(f"{metric.read_name}: MQ={metric.mapping_quality}")

To read from an open file handle or any other text IO source (e.g. StringIO), use MetricReader directly:

# Reading from an IO source
with open("alignments.tsv") as handle:
    reader = MetricReader(AlignmentMetric, handle)
    for metric in reader:
        print(f"{metric.read_name}: MQ={metric.mapping_quality}")

Example input file (alignments.tsv):

read_name	mapping_quality	is_duplicate
read1	60	false
read2	30	true

Invalid data raises pydantic.ValidationError with details about which field failed.

Core Usage

Delimiters

The field delimiter is inferred from the file extension, so common formats need no configuration: .csv is comma-delimited, while .tsv, .txt, .tab, and Picard-style *metrics files (e.g. .insert_size_metrics) are tab-delimited. A trailing compression suffix (.gz, .bz2, .xz) is ignored, so data.csv.gz is still comma-delimited.

# Comma-delimited, inferred from the .csv extension
for metric in MyMetric.read("data.csv"):
    ...

Pass delimiter= to override inference, or to read or write a file whose extension isn't recognized. An unrecognized extension with no explicit delimiter raises ValueError.

# Pipe-delimited file with an unrecognized extension
with MetricWriter.open(MyMetric, "output.dat", delimiter="|") as writer:
    ...

Compression

Reading and writing transparently handle gzip, bzip2, and xz files based on the path extension (via xopen):

for metric in AlignmentMetric.read("alignments.tsv.gz"):
    ...

with MetricWriter.open(AlignmentMetric, "output.tsv.bz2") as writer:
    writer.writeall(metrics)

Overwriting existing files

MetricWriter.open() refuses to clobber an existing file, raising FileExistsError. Pass overwrite=True to truncate and overwrite it:

# Raises FileExistsError if output.tsv already exists
with MetricWriter.open(AlignmentMetric, "output.tsv") as writer:
    writer.writeall(metrics)

# Replaces output.tsv if it exists
with MetricWriter.open(AlignmentMetric, "output.tsv", overwrite=True) as writer:
    writer.writeall(metrics)

List Fields

Fields typed as list[T] are automatically parsed from and serialized to delimited strings:

class TaggedRead(Metric):
    read_id: str
    tags: list[str]           # "A,B,C" becomes ["A", "B", "C"]
    scores: list[int]         # "1,2,3" becomes [1, 2, 3]
    optional_tags: list[str] | None  # "" becomes None

The list delimiter defaults to , but can be customized per-metric:

class SemicolonMetric(Metric):
    collection_delimiter = ";"
    values: list[int]  # "1;2;3" becomes [1, 2, 3]

Counter Fields

When your file has categorical data with one column per category (e.g. base counts A, C, G, T), you can model them as a single Counter[StrEnum] field:

from collections import Counter
from enum import StrEnum
from fgmetric import Metric


class Base(StrEnum):
    A = "A"
    C = "C"
    G = "G"
    T = "T"


class BaseCountMetric(Metric):
    position: int
    counts: Counter[Base]


# Input TSV:
# position  A   C   G   T
# 1         10  5   3   2

# Parses to:
# BaseCountMetric(position=1, counts=Counter({Base.A: 10, Base.C: 5, ...}))

Contributing

See the contributing guide for development setup and testing instructions.