This repository contains my implementations and performance analyses for the Software and Programming of High Performance Systems (HPC) course exercises. The projects focus on scaling applications through parallelization, vectorization, and GPU acceleration.
Focus: Shared memory, distributed memory, and hybrid environments.
- Objective: Parallelizing loops and sections using directive-based shared memory programming.
- Key Concepts: Thread safety, work-sharing constructs, and avoiding race conditions.
- Objective: Developing distributed memory programs in C.
- Key Concepts: Point-to-point communication (
MPI_Send/MPI_Recv), collective operations (MPI_Reduce,MPI_Bcast), and domain decomposition.
Both of those tools where used for implementing parallel versions of mds algorithm
- Concurrent Futures (ProcessPoolExecutor)
- MPI4py (mpi-futures and simple communication)
- Used for parallelizing an application of randomsearch.
Focus: Exploiting instruction-level parallelism and hardware accelerators.
- Objective: Manually optimizing code using Streaming SIMD Extensions (SSE) and Advanced Vector Extensions (AVX) intrinsics.
-
Key Concepts: Data alignment, register packing, and
$N$ -way speedup (e.g., 256-bit wide registers for 8 single-precision floats). - Used for parellism of the WENO5 algorithm
- Objective: Offloading computationally intensive tasks to NVIDIA GPUs.
- Key Concepts: Kernel functions, thread hierarchy (Grids/Blocks), shared memory optimization, and host-to-device data transfers.
- Used for simple implementation of complex matrix multiplication (with and without CuBLAS)
To run these exercises, you will typically need:
- Compilers:
gcc,nvcc(for CUDA) - Libraries: OpenMPI or MPICH, CUDA Toolkit
- Python:
mpi4pypackage
Each directory contains a short report regarding the observed speedup and efficiency (performance testing was doene by running multiple times with different parameters the programs using shell scripts also included in the repo)