LLM Report Analysis System

LLM-powered pipeline for analysing UN Human Development Reports using local Llama 3.2, with automated information extraction, evaluation, interactive dashboards, and HTML reporting.

Overview

This project implements an end-to-end Large Language Model (LLM) pipeline for analysing UN Human Development Reports. The system extracts text from PDF documents, performs chunk-based processing using a locally hosted LLM via Ollama, generates structured analytical outputs, evaluates extraction quality, and produces an interactive HTML dashboard with visualisations.

The project demonstrates the practical application of prompt engineering, local LLM inference, automated document analysis, data visualisation, and report generation.

---

Features

• PDF text extraction using PyMuPDF
• Automatic chapter detection
• Intelligent document chunking
• Local LLM inference using Ollama
• Executive report summarisation
• Chapter summarisation
• Theme extraction
• Strengths and challenges extraction
• Numerical indicator extraction
• Demographic trend extraction
• Automatic evaluation metrics
• Dashboard generation
• HTML report generation
• Data visualisations

---

Technologies Used

• Python 3
• Ollama
• Llama 3.2
• PyMuPDF
• Matplotlib
• JSON
• HTML & CSS

---

Project Structure

LLM Assignment/
│
├── config.py
├── main.py
├── run_outputs.py
├── requirements.txt
│
├── data/
│   └── thailand_2006_report.pdf
│
├── src/
│   ├── pdf_processor.py
│   ├── llm_extractor.py
│   ├── evaluator.py
│   ├── dashboard.py
│   ├── visualisation.py
│   ├── prompts.py
│   ├── pipeline.py
│   └── utils.py
│
├── outputs/
│   ├── cleaned_text/
│   ├── json/
│   ├── summaries/
│   ├── evaluation/
│   ├── dashboard_data/
│   ├── figures/
│   └── report/
│
├── test_dashboard.py
├── test_visualisation.py
└── test_report.py

---

Installation

Clone the repository:

git clone <repository-url>
cd LLM-Assignment

Create a virtual environment:

python -m venv .venv

Activate the environment:

Windows

.venv\Scripts\activate

macOS/Linux

source .venv/bin/activate

Install dependencies:

pip install -r requirements.txt

Install Ollama and pull the required model:

ollama pull llama3.2

---

Running the Pipeline

Run the full extraction pipeline:

python main.py

Generate dashboard outputs:

python run_outputs.py

---

Outputs

The system automatically generates:

• Cleaned document text
• Chapter information
• Text chunks
• Executive summary
• Chapter summaries
• Themes
• Strengths and challenges
• Numerical indicators
• Demographic trends
• Evaluation report
• Dashboard JSON
• Visualisations
• HTML report

---

Dashboard Visualisations

The dashboard includes:

• Document Statistics
• LLM Extraction Distribution
• Pipeline Quality Metrics
• Pipeline Completion
• Output Validation

---

Model

Model used:

Llama 3.2

Running locally through Ollama.

---

Assignment Learning Outcomes

This project demonstrates:

• Large Language Model applications
• Prompt engineering
• Document processing
• Information extraction
• Local LLM deployment
• Automated evaluation
• Data visualisation
• Report generation

---

Author

Maxwell Ororho

MSc Data Science

University of Hertfordshire