MultiLibraryClassLoader not being thread-safe for the MCAP plugin

[Baranbologur]

Description

Attempting to reindex multiple ROS 2 bags in parallel threads using the MCAP storage backend fails intermittently. Only one reindex thread works reliably; additional concurrent reindex or recording threads often fail to load the mcap storage plugin, and in some cases this leads to runtime errors. Sometimes multiple of them passes. But multiple parallel reindexing or one reindex + one record in parallel is not reliable.

Expected Behavior

• Multiple threads should be able to reindex different bags simultaneously without errors.
• Reindexing should not interfere with ongoing recordings.

Actual Behavior

When running reindex_directory() concurrently in multiple threads, reindexing behaves inconsistently and is not thread-safe.

The typical behavior is:

• When starting multiple reindex operations in parallel, only one thread reliably succeeds.
• Sometimes two or more reindexes succeed, but this is inconsistent.
• At least one reindex usually completes successfully.
• In the “reindex only” case (no recording running), failures do not result in a runtime crash.

The following errors are logged during failed reindex attempts:

[manager-1] [INFO] [1771850012.113741178] [rosbag2_cpp]: Beginning reindexing bag in directory: /ros2_ws/rosbags/idle_recordings/idle_recording_2026-02-23_15-22-18 
[manager-1] [INFO] [1771850012.113760954] [rosbag2_cpp]: Beginning reindexing bag in directory: /ros2_ws/rosbags/idle_recordings/idle_recording_2026-02-23_15-21-39 
[manager-1] [INFO] [1771850012.114113859] [rosbag2_cpp]: Beginning reindexing bag in directory: /ros2_ws/rosbags/idle_recordings/idle_recording_2026-02-23_15-21-25 
[manager-1] [ERROR] [1771850012.115750823] [rosbag2_storage]: Unable to load plugin 'mcap': MultiLibraryClassLoader: Could not create class of type rosbag2_storage_plugins::MCAPStorage 
[manager-1] [ERROR] [1771850012.115765324] [rosbag2_storage]: Could not load/open plugin with storage id 'mcap' 
[manager-1] [INFO] [1771850012.116833776] [rosbag2_cpp]: Reindexing complete. 
[manager-1] [ERROR] [1771850012.132195918] [rosbag2_storage]: Unable to load plugin 'mcap': MultiLibraryClassLoader: Could not create class of type rosbag2_storage_plugins::MCAPStorage 
[manager-1] [ERROR] [1771850012.132219592] [rosbag2_storage]: Could not load/open plugin with storage id 'mcap'
[aura_auv_data_recording_manager-1] [ERROR] [1771908125.190763640] [rosbag2_storage]: Could not load/open plugin with storage id 'mcap'

Multithread reindexes + recording

When running multiple reindexes in parallel together with a recording (rosbag2_transport::Recorder::record()):

• Usually only one reindex succeeds.
• Recording startup sometimes fails (~20% of runs).
• In some cases, the entire process crashes.

Additional observed errors:

[aura_auv_data_recording_manager-1] tcache_thread_shutdown(): unaligned tcache chunk detected
[ERROR] [aura_auv_data_recording_manager-1]: process has died [pid 13269, exit code -6, cmd '/ros2_ws/install/aura_auv_data_recording_manager/lib/aura_auv_data_recording_manager/aura_auv_data_recording_manager --ros-args -r __node:=aura_auv_data_recording_manager -r __ns:=/GYB/GYY --params-file /ros2_ws/install/aura_auv_data_recording_manager/share/aura_auv_data_recording_manager/config/parameters.yaml --params-file /tmp/launch_params_uxhp28ca --params-file /tmp/launch_params_irpvl8i3'].

Multithread recording + convert

When running one recording thread and one conversion thread in parallel, no issues have been observed so far. This combination appears stable.

To Reproduce

void ReindexHelper::reindex_directory(const std::string & bag_dir)
{    
    try {
        rosbag2_storage::StorageOptions storage_options;
        storage_options.uri = bag_dir;
        storage_options.storage_id = "mcap";
        
        rosbag2_cpp::Reindexer reindexer;
        reindexer.reindex(storage_options);
    } catch (const std::exception & e) {
        (void) e;
    }
}
// Spawn multiple threads calling reindex_directory() on different bags

1. Start multiple threads calling reindex_directory() on different MCAP bags.
2. Optionally, start a recording with rosbag2_transport::Recorder::record() in another thread.
3. Observe intermittent failures loading the mcap plugin.

System (please complete the following information)

• OS: Ubuntu 22.04
• ROS 2 Distro: Humble Hawksbill
• Install Method: APT
• Version: 0.15.15-1jammy.20251228.122637

Additional context

This appears related to MultiLibraryClassLoader not being thread-safe for the MCAP plugin.

Reindexing works perfectly single-threaded.

Concurrent calls (either multiple reindexes or reindex + recording) trigger the plugin load error.

[Baranbologur]

Interestingly, according to my tests, using two threads, one for record and one for reindex, causes error. But using two threads, one for record and one for convert, does not cause any error. However, for all of these three operations, I am using mcap as the storage type.

[MichaelOrlov]

cc: @clalancette @fujitatomoya @mjcarroll Curious if it is a known issue with the MultiLibraryClassLoader?