Skip to content

CLAUDE.md

Latest commit

 

History

History
101 lines (80 loc) · 4.92 KB

CLAUDE.md

File metadata and controls

101 lines (80 loc) · 4.92 KB
Raw

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project purpose

Build an Evo-2-based classifier that distinguishes fungal genomes from two ecological niches:

  • Subsurface (cave/deep-earth fungi) — 5 genomes in classify/Subsurface/
  • Terrestrial (surface-soil fungi) — 191 genomes in classify/Terresterial/

The classifier must support both whole-genome inputs and short metagenome reads. Trained parameters persist to models/ so future runs skip embedding extraction.

Environment setup

pixi install          # uses pixi.toml; install pixi from https://pixi.sh if needed
pixi shell            # or prefix commands with: pixi run python ...

Evo-2 model weights download automatically on first use (~7 GB for evo2_1b_base, GPU strongly recommended).

Common commands

Train — annotation features only (no GPU needed, fast baseline)

pixi run python train.py --mode annotation --data-dir classify --model-dir models/annotation

Train — Evo-2 embeddings from CDS transcripts

pixi run python train.py --mode embedding --seq-type cds --data-dir classify \
    --model-dir models/evo2_cds --embedding-cache models/embeddings

Train — hybrid (embeddings + functional annotations)

pixi run python train.py --mode hybrid --seq-type cds --data-dir classify \
    --model-dir models/hybrid

Classify a new genome (whole genome)

pixi run python predict.py --input unknown.scaffolds.fa --model-dir models/hybrid

Classify metagenome short reads

pixi run python predict.py --input bin.fa --model-dir models/hybrid --short-reads

Classify all FASTA files in a directory

pixi run python predict.py --input-dir /path/to/new_genomes/ --model-dir models/hybrid --out results.csv

Architecture

src/
  data_loader.py          Discovers genomes; GenomeRecord dataclass with lazy loaders
  embeddings.py           Evo-2 embedding extraction; tiles long scaffolds into windows;
                          caches per-genome .npy files in models/embeddings/
  annotation_features.py  CAZyme classes, secretome (SignalP), membrane (TMHMM),
                          protease (MEROPS), PFAM coverage → numeric feature vector
                          from annotation_summary TSV files
  classifier.py           sklearn Pipeline (StandardScaler + LogisticRegression/MLP);
                          save/load via pickle; handles class imbalance with balanced weights
  features.py             Feature importance: logistic coefficients, permutation importance,
                          SHAP values, UMAP visualization

train.py                  End-to-end training with cross-validation; saves model + plots
predict.py                Inference on new FASTA files (whole genome or short reads)
models/                   Saved pipelines (pipeline.pkl) and metadata (metadata.json)
results/                  Plots and CSVs produced by train.py
classify/
  Subsurface/             5 cave fungi
    cds/                  *.cds-transcripts.fa  (primary embedding source)
    dna/                  *.scaffolds.fa
    annotation_summary/   *.annotation_summary.tsv
  Terrestrial/            191 surface fungi
    cds/                  *.cds-transcripts.fa
    dna/                  *.scaffolds.fa
    annotation_summary/   *.annotation_summary.tsv

Key design decisions

  • Class imbalance (5 vs 191): all classifiers use class_weight='balanced'; CV folds are clamped to minority class size; evaluation uses balanced accuracy, F1, and ROC-AUC.
  • Three feature modes: annotation (no GPU, fast), embedding (Evo-2 only), hybrid (both concatenated).
  • Embedding caching: computed embeddings are stored as .npy files keyed by {genome_name}.{seq_type}.npy; reuse across training runs with --embedding-cache.
  • Genome tiling: scaffolds are chunked into overlapping chunk_size-bp windows (default 8192 bp, 50% overlap); per-window embeddings are mean-pooled to a single genome vector.
  • Annotation features: sourced from annotation_summary/ TSVs; CAZyme class counts (GH/GT/PL/CE/AA/CBM), secreted gene count (SignalP prob > 0.5), membrane proteins (TMHMM helices > 0), protease count (MEROPS hit), PFAM-annotated gene count — all normalized by gene count to produce per-genome rates.
  • Evo-2 API: package evo2 exposes Evo2(model_name) with a .model attribute and .tokenizer; forward pass with return_embeddings=True returns (logits, embeddings).

Data file types

Subdir Suffix Content
cds/ .cds-transcripts.fa CDS sequences — primary input for Evo-2 embedding
dna/ .scaffolds.fa Genome assembly scaffolds
annotation_summary/ .annotation_summary.tsv TSV: protein_id, pfam_domains, signalp_{start,end,prob}, merops_{id,pct_id,evalue}, tmhmm_{pred_hel,exp_aa,topology}, cazy_{family,EC,substrate}