Skip to content

Enable mixed precision without loss of accuracy #51

@r1cheu

Description

@r1cheu

The main aim of this feature is to optimize computational performance and reduce memory overhead for large-scale genomic studies by implementing mixed precision (e.g., using float for genotype decoding and intermediate matrix operations while retaining double for sensitive accumulations and final estimates).

Currently, Gelex extensively uses double precision across its core library (e.g., in decode_lut.h and math_utils.h). Implementing mixed precision would benefit memory-mapped I/O and BLAS/LAPACK accelerated computations.

Requirements:

  • Identify performance-critical sections (e.g., GRM calculation, MCMC sampling) where mixed precision can be applied.
  • Implement mixed precision strategies in gelex::core.
  • Validate results against double precision baselines to ensure no loss of accuracy in heritability and marker effect estimates.
  • Provide benchmarks for speed and memory usage.

Acceptance Criteria:

  • Functional mixed precision support in fit, assoc, and grm subcommands.
  • Accuracy metrics showing results remain within a tight tolerance of the full-precision baseline.
  • User documentation on performance benefits and how to enable it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions