Version 2.0

LaueMatching v2.0 Release Notes

We are proud to announce LaueMatching v2.0, a major release that introduces a GPU streaming pipeline for high-throughput multi-image processing and a complete modular decomposition of the Python codebase. This release transforms LaueMatching from a single-image tool into a production-ready system capable of processing thousands of diffraction images with minimal overhead.

Key Highlights

1. GPU Streaming Pipeline (LaueMatchingGPUStream)

A new persistent GPU daemon eliminates the dominant bottleneck in multi-image workflows — reloading the 6.7 GB orientation database for every image:

• LaueMatchingGPUStream.cu: A CUDA-based TCP server that loads the orientation database once into GPU memory and accepts preprocessed images over a socket, returning indexed orientations in real time.
• laue_orchestrator.py: A top-level Python orchestrator that launches the daemon, starts the image server, monitors progress with live rate and ETA, and runs post-processing automatically.
• laue_image_server.py: Reads HDF5 files from a folder, preprocesses each frame (background subtraction → enhancement → thresholding → filtering → Gaussian blur), and sends processed images to the daemon over TCP.
• Wire Protocol: Each frame is transmitted as a 2-byte little-endian image number followed by the full double[] pixel array, matching the daemon's handle_client receiver.

	Single-Image (RunImage.py)	Streaming (laue_orchestrator.py)
Orientation DB	Loaded per image (~10 s)	Loaded once, reused
GPU utilization	Idle between images	Continuous
Throughput	~1 image/min	Limited only by preprocessing
Progress tracking	Per-image logs	Live frame_mapping.json with rate + ETA

2. Modular Python Architecture

The monolithic RunImage.py (3,553 lines) has been decomposed into focused, reusable modules — a 53% reduction in its line count:

Module	Lines	Purpose
RunImage.py	1,673	Core single-image pipeline (load → preprocess → index → refine → export)
laue_config.py	782	Dataclass-based configuration (LaueConfig, VisualizationConfig, OptimizerConfig) and ConfigurationManager for parsing params.txt
laue_stream_utils.py	1,108	Shared utilities: image I/O, preprocessing pipeline, TCP wire protocol, HDF5 helpers, orientation sorting and filtering
laue_visualization.py	937	8 standalone visualization functions (interactive Plotly spot maps, simulation comparison, 3D orientation views, analysis reports)

All extracted functions accept explicit parameters and require no class instantiation, making them easy to import and reuse from any script.

3. Enhanced Post-Processing (laue_postprocess.py)

The streaming post-processor has been upgraded with two key improvements:

• Quality-Based Sorting: Filtered orientations are now automatically sorted by quality score (descending) using sort_orientations_by_quality, ensuring the best solutions appear first in output files.
• Per-Image Interactive Visualization: A new --per-image-viz flag generates individual Plotly HTML files for each processed image, enabling detailed inspection of indexing results on a per-frame basis.

4. Scripts Directory Reorganization

All Python scripts have been moved from the repository root into a dedicated scripts/ directory:

scripts/
├── RunImage.py                   # Single-image indexing pipeline
├── laue_orchestrator.py          # Streaming pipeline entry point
├── laue_image_server.py          # H5 → preprocess → TCP sender
├── laue_postprocess.py           # Results filtering + visualization
├── laue_config.py                # Configuration dataclasses & manager
├── laue_stream_utils.py          # Shared I/O, preprocessing, TCP utilities
├── laue_visualization.py         # 8 standalone visualization functions
├── GenerateHKLs.py               # Generate valid HKL list
├── GenerateSimulation.py         # Create synthetic Laue patterns
├── ImageCleanup.py               # Pre-process raw detector images
└── README.md                     # Comprehensive CLI reference

A new scripts/README.md provides full CLI documentation for every script, including argument tables, module architecture diagrams, and usage examples.

5. Comprehensive Documentation

• scripts/README.md: Full CLI reference for all 10 scripts, Mermaid architecture diagrams, wire protocol documentation, and shared module API summaries.
• simulation/README.md: Updated all script paths to reflect the new scripts/ directory, added streaming mode quick-start example.
• README.md: Expanded project structure, version history, and cross-references to the new documentation.

Quick Start — Streaming Pipeline

# Process an entire folder of H5 images through the persistent GPU daemon:
python scripts/laue_orchestrator.py \
    --config params.txt \
    --folder /path/to/h5_images/ \
    --h5-location /entry/data/data \
    --ncpus 8

This automatically launches the daemon, preprocesses and streams all images, monitors progress, and generates per-image HDF5 results with interactive Plotly HTML visualizations.

Upgrading from v1.0

• Script paths: All scripts have moved from the repository root to scripts/. Update any shell scripts or workflows that reference ./RunImage.py to use python scripts/RunImage.py.
• No parameter file changes: The params.txt format is fully backward compatible.
• Build: Run ./build.sh gpu to compile the new LaueMatchingGPUStream binary alongside the existing CPU and GPU executables.

Version 1.0

LaueMatching v1.0 Release Notes

We are proud to announce the release of LaueMatching v1.0, a major milestone in the development of robust, high-performance orientation indexing for Laue diffraction. This release represents a comprehensive architectural overhaul, eliminating code duplication while fixing critical bugs that affected orientation refinement accuracy.

Key Highlights

1. Unified Architecture & Code Consolidation

The codebase has undergone a significant refactor to ensure consistency and maintainability:

• Shared Core Logic: Over 700 lines of duplicated logic (symmetry operations, math utilities, overlap calculations) have been consolidated into a single LaueMatchingHeaders.h.
• Logic Parity: The CPU and GPU implementations now share the exact same mathematical definitions, eliminating subtle drift between the two backends (e.g., Trigonal symmetry definitions are now identical).

2. Critical Bug Fixes & Accuracy Improvements

Several long-standing issues affecting scientific accuracy and stability have been resolved:

• c/a Ratio Fitting: Fixed a critical integer division bug (1/3 vs 1.0/3.0) in the pow() function that previously caused lattice parameter refinement to fail for non-cubic systems.
• GPU Unique Solutions: Fixed an indexing error in the unique-solution merging step on the GPU (orients[l*9+m] → orients[rowNrs[l]*9+m]), ensuring correct orientation clustering.
• Bounds Safety: Fixed an underflow issue where negative pixel coordinates (stored as uint16_t) were not correctly discarded, preventing potential out-of-bounds memory access.

3. Stability & Resource Management

Dramatic improvements for long-running batch jobs:

• Memory Leaks Resolved: Fixed multiple memory leaks in the per-grain fitting loop (arrays were calloc'd but never freed).
• File Descriptor Cleanup: Implemented proper munmap and close calls to prevent resource exhaustion during high-throughput processing.
• Race Condition Fixes: Resolved OpenMP race conditions related to file opening/creation in parallel regions.

4. Modern Build System (CMake)

The build system has been modernized for better portability and developer experience:

• Strict Compilation: The project now compiles cleanly with -Wall (all warnings enabled), ensuring higher code quality.
• CMake Integration: Replaced legacy Makefiles with a robust CMakeLists.txt that automatically handles dependencies (NLopt) and conditionally compiles CUDA support.

Getting Started with v1.0

Installation

Clone the repository and build using the provided script (which now uses CMake):

git clone [https://github.com/AdvancedPhotonSource/LaueMatching.git](https://github.com/AdvancedPhotonSource/LaueMatching.git)
cd LaueMatching
./build.sh        # For CPU build
# ./build.sh gpu  # For GPU build (requires CUDA)

Running the workflow

LaueMatching is designed to be run via the Python wrapper:

./RunImage.py process -c params.txt -i image.h5 -n <nCPUs>

v1.0-data

100MilOrients file upload in 4 parts. The data download path will be corrected in build.sh.