Skip to content

to_f32/to_f64 doesn't reliably round to nearest #87

Description

@Shoeboxam

RBig::to_f64 appears to return the adjacent incorrectly-rounded f64 for at least one rational value. The docs for RBig::to_f64 say it converts with guaranteed correct rounding using nearest, ties-to-even semantics, but this example appears to round one ulp toward zero.

Minimal reproducer:

use dashu::rational::RBig;

#[test]
fn rbig_to_f64_double_rounding_case() {
    let input = RBig::from_parts(
        (-10534148920556696739i128).into(),
        73786976294838206464u128.into(),
    );

    let got = input.to_f64().value();

    // Current Dashu result:
    assert_eq!(got.to_bits(), 0xbfc2_461a_1430_9b16);

    // Expected nearest f64:
    assert_eq!(got.to_bits(), 0xbfc2_461a_1430_9b17);
}

Observed:

got      = f64::from_bits(0xbfc2461a14309b16)
expected = f64::from_bits(0xbfc2461a14309b17)

I think this is a double rounding issue. The rational is first approximated to an intermediate representation with more precision than f64, and then that intermediate value is rounded/encoded again to f64. I worked around this by rounding the rational directly to the target native-float precision before encoding.

For context, here is the OpenDP workaround/reference implementation:

https://github.com/opendp/opendp/blob/8c1a761a7f4ab2c5758ad99df4cbc03934b452fb/rust/src/traits/cast/to_float.rs

I found it while implementing a randomized sampling algorithm that depends on float rounding semantics to terminate.

Thanks as always! If you'd prefer a PR I can open one.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions