Skip to content

feat! Mimalloc as the default allocator#434

Merged
anakrish merged 1 commit into
microsoft:mainfrom
anakrish:mimalloc
Aug 25, 2025
Merged

feat! Mimalloc as the default allocator#434
anakrish merged 1 commit into
microsoft:mainfrom
anakrish:mimalloc

Conversation

@anakrish

@anakrish anakrish commented Jul 11, 2025

Copy link
Copy Markdown
Collaborator

This change integrates mimalloc as the default memory allocator for Regorus,
delivering significant performance improvements across all evaluation modes
and language bindings.

Technical Implementation:

  • Build mimalloc in vendored mode from C sources (following QSharp approach)
  • Implement GlobalAlloc trait for seamless Rust integration
  • Add optional 'mimalloc' feature flag for conditional compilation
  • Add comprehensive ACI benchmarks to measure evaluation performance

Performance Impact:

Rust Engine Evaluation:

  • Single-threaded: ~29% improvement (423 vs 328 Kelem/s)
  • Multi-threaded: Better scaling with reduced thread contention
  • Fresh engines: ~24% improvement (56 vs 45 Kelem/s)

Rust Compiled Policy Evaluation:

  • Single-threaded: ~41% improvement (426 vs 303 Kelem/s)
  • Multi-threaded: Improved allocation efficiency under contention
  • Fresh compilation: ~26% improvement (53 vs 42 Kelem/s)

C# FFI Bindings:

  • Engine evaluation: ~27% improvement (279 vs 219 Kelem/s)
  • Compiled policies: ~29% improvement (273 vs 211 Kelem/s)
  • Better threading characteristics through improved underlying allocation

Key Benefits:

  • Reduced allocation-related contention in multi-threaded scenarios
  • More consistent performance across different thread counts
  • Improved memory allocation efficiency for both native Rust and FFI workloads
  • Better scaling characteristics for production deployments

The mimalloc integration provides substantial performance gains while
maintaining full compatibility with existing code through feature flags.

Reference: QSharp allocator implementation
(https://github.com/microsoft/qsharp/tree/main/source/allocator)

Fixes #297

@anakrish anakrish force-pushed the mimalloc branch 3 times, most recently from 1796833 to 2d21208 Compare July 11, 2025 16:43
@anakrish anakrish marked this pull request as ready for review July 11, 2025 16:43
@anakrish anakrish requested a review from Copilot July 11, 2025 16:43

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Enable the use of Mimalloc as the global allocator under a cargo feature and bundle the Mimalloc C library via a Rust FFI layer.

  • Add mimalloc::assign_global!() to src/lib.rs under the mimalloc feature
  • Introduce new mimalloc crate implementing GlobalAlloc with tests for alloc and alloc_zeroed
  • Define and expose the assign_global macro and FFI bindings in mimalloc-sys

Reviewed Changes

Copilot reviewed 49 out of 56 changed files in this pull request and generated no comments.

File Description
src/lib.rs Call mimalloc::assign_global!() when feature = "mimalloc"
mimalloc/src/mimalloc.rs Implement GlobalAlloc in Mimalloc and add basic allocation tests
mimalloc/src/lib.rs Export assign_global macro
mimalloc/mimalloc-sys/src/lib.rs Declare extern C bindings for allocation functions
Comments suppressed due to low confidence (2)

mimalloc/src/mimalloc.rs:30

  • Consider adding a test for the realloc method to verify that data is correctly preserved and the pointer is properly reallocated.
    unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {

mimalloc/src/lib.rs:5

  • [nitpick] The mimalloc module duplicates the crate name, which may be confusing; consider renaming the module to something more descriptive like allocator or global.
pub mod mimalloc;

This change integrates mimalloc as the default memory allocator for Regorus,
delivering significant performance improvements across all evaluation modes
and language bindings.

Technical Implementation:
- Build mimalloc in vendored mode from C sources (following QSharp approach)
- Implement GlobalAlloc trait for seamless Rust integration
- Add optional 'mimalloc' feature flag for conditional compilation
- Add comprehensive ACI benchmarks to measure evaluation performance

Performance Impact:

Rust Engine Evaluation:
- Single-threaded: ~29% improvement (423 vs 328 Kelem/s)
- Multi-threaded: Better scaling with reduced thread contention
- Fresh engines: ~24% improvement (56 vs 45 Kelem/s)

Rust Compiled Policy Evaluation:
- Single-threaded: ~41% improvement (426 vs 303 Kelem/s)
- Multi-threaded: Improved allocation efficiency under contention
- Fresh compilation: ~26% improvement (53 vs 42 Kelem/s)

C# FFI Bindings:
- Engine evaluation: ~27% improvement (279 vs 219 Kelem/s)
- Compiled policies: ~29% improvement (273 vs 211 Kelem/s)
- Better threading characteristics through improved underlying allocation

Key Benefits:
- Reduced allocation-related contention in multi-threaded scenarios
- More consistent performance across different thread counts
- Improved memory allocation efficiency for both native Rust and FFI workloads
- Better scaling characteristics for production deployments

The mimalloc integration provides substantial performance gains while
maintaining full compatibility with existing code through feature flags.

Reference: QSharp allocator implementation
(https://github.com/microsoft/qsharp/tree/main/source/allocator)

Fixes microsoft#297

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

@dpokluda dpokluda left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@anakrish anakrish merged commit 2a0b4ae into microsoft:main Aug 25, 2025
33 checks passed
@anakrish anakrish deleted the mimalloc branch August 25, 2025 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for custom allocators

3 participants