Skip to content

nlog: replace ConcurrentHashMap with phmap::parallel_flat_hash_map#1138

Open
SBALAVIGNESH123 wants to merge 1 commit intotimeplus-io:developfrom
SBALAVIGNESH123:fix/replace-concurrenthashmap-with-phmap
Open

nlog: replace ConcurrentHashMap with phmap::parallel_flat_hash_map#1138
SBALAVIGNESH123 wants to merge 1 commit intotimeplus-io:developfrom
SBALAVIGNESH123:fix/replace-concurrenthashmap-with-phmap

Conversation

@SBALAVIGNESH123
Copy link
Copy Markdown
Contributor

@SBALAVIGNESH123 SBALAVIGNESH123 commented Mar 20, 2026

Fixes #441

This PR replaces the current ConcurrentHashMap implementation in NativeLog, which uses a single std::shared_mutex around an absl::flat_hash_map, with phmap::parallel_flat_hash_map from the parallel-hashmap library. The old design meant every read and write operation competed for one global lock, which becomes a bottleneck under concurrent access.The new implementation uses phmap's internal submap-level locking — the map is split into 16 independent submaps, each with its own std::shared_mutex. This means operations on keys that hash to different submaps run fully in parallel with zero contention. Multiple readers on the same submap can also proceed concurrently thanks to the shared_mutex reader-writer separation.parallel-hashmap is a well-established header-only library built on top of Google's absl::flat_hash_map, widely used in production OLAP systems like Apache Doris. It was the library suggested in the original issue.All 15 public methods are preserved with identical signatures, so LogManager.cpp requires no changes. The only method removed is eraseIf, which was dead code — never called anywhere in the codebase. Template defaults (std::hash, std::equal_to) are kept the same to maintain backward compatibility with StreamIDShard's existing std::hash specialization.

Benchmark references from the original issue:

Note: The .gitmodules merge conflict is due to new submodules added to develop since my fork was last synced. Happy to rebase onto latest develop if needed — just let me know. The actual code change (ConcurrentHashMap.h) merges cleanly.

Fixes timeplus-io#441

Replace the single std::shared_mutex + absl::flat_hash_map design with
phmap::parallel_flat_hash_map using per-submap std::shared_mutex locking.
16 independent submaps enable reader-writer parallelism: operations on
keys in different submaps proceed with zero contention.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nlog: replace concurrenthashmap to phmap::parallel_hash_map

1 participant