Skip to content

feat(core): add decimal type#1719

Open
benjamin-awd wants to merge 2 commits intovectordotdev:mainfrom
benjamin-awd:add-decimal
Open

feat(core): add decimal type#1719
benjamin-awd wants to merge 2 commits intovectordotdev:mainfrom
benjamin-awd:add-decimal

Conversation

@benjamin-awd
Copy link
Copy Markdown
Contributor

@benjamin-awd benjamin-awd commented Mar 26, 2026

Summary

This PR adds support for a new decimal type backed by the rust_decimal crate, providing 96-bit precision decimal arithmetic to solve long-standing floating-point precision loss issues (#1503, #755). Sibling Vector PR: vectordotdev/vector#25048.

Added a d'...' literal syntax for decimal values in VRL programs (e.g., d'0.12379999458789825'), consistent with VRL's existing quoted-literal conventions (r'...' for regex). Decimal literals are parsed at compile time into rust_decimal::Decimal values with no f64 intermediate.

parse_json(parse_float: "decimal") enables precision-preserving JSON ingress. Instead of deserializing through f64, this path uses serde_json::RawValue to capture each number's original string representation, then parses directly into Decimal. This avoids the two-step precision loss that causes issues like 0.123799994587898250.12379999458789824.

This is fully opt-in -- existing behavior is unchanged.

Note: moved away from the arbitrary_precision syntax since that's quite specific to serde_json and an implementation detail that some users may not understand, instead drew inspiration from Python's parse_float(decimal) syntax.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

We'll keep this opt-in. Within the VRL crate, unless the arbitrary_precision argument is passed in parse_json the default behavior will occur.

On the Vector side, it'll similarly be opt-in e.g.

sources:
  my_json_source:
    type: file
    include: ["/var/log/*.json"]
    decoding:
      codec: json
      json:
        parse_float: decimal  # Opt-in to parsing numeric data as decimal

How did you test this PR?

Tests within this PR and a sibling PR in Vector

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on
    our guidelines.
  • No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

Note: rust-decimal is already included in the LICENSE-3rdparty.csv file.

References

Closes #1692

@thomasqueirozb thomasqueirozb removed the request for review from a team March 30, 2026 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preserve numeric precision with decimal type

3 participants