USGS Lidar Bridge Classification

An end-to-end pipeline for processing bridge lidar data organized by Hydrologic Unit Code (HUC) regions. This project downloads lidar point cloud data, applies weak supervision rules for labeling, normalizes coordinates, and prepares data for machine learning. It includes model training (sparse 3D U-Net) and inference for bridge point cloud classification with multiple output modes (masked, raw, or both); scaling with AWS Batch for parallel inference on SPOT instances is supported.

Documentation

For detailed design documentation, see the docs/ directory (or build the docs site with MkDocs):

Architecture - System design, classification schema, algorithm details.
Data Pipeline - Step-by-step data flow walkthrough with data shapes at each stage.
AWS Batch Inference - Terraform infrastructure, job submission, SPOT instance handling, inference modes, post-run audit, and configuration reference for scaling inference with AWS Batch array jobs.
Module Reference - Summary of every module's public API and CLI arguments.
Design Decisions - Rationale for key architectural choices.

Building the docs locally

The docs are readable as plain Markdown on GitHub. To build a searchable site with navigation and Mermaid diagrams:

pip install -r requirements-docs.txt
mkdocs serve

Open http://localhost:8000 to preview. Changes to docs/ are hot-reloaded.

Deploying to GitHub Pages

mkdocs gh-deploy

Pipeline Overview

Data flows from OSM bridge geometries and USGS LiDAR sources through download and weak supervision, then normalization, train/val/test split, optional class-weight computation, and finally model training. The pipeline scales to hundreds of thousands of bridges; ensure sufficient disk space for silver_training and normalized outputs.

flowchart LR;
  DataDownload[Download and Weak Supervision]
  Preprocess[Preprocess and Normalize]
  Split[Split Train/Val/Test]
  CalculateWeights[Calculate Weights]
  Train[Train Model]
  DataDownload --> Preprocess;
  Preprocess --> Split;
  Split --> CalculateWeights;
  CalculateWeights --> Train;
  Split --> Train;

Installation

Prerequisites

Python 3.11
Conda or Mamba package manager
GPU Requirement: An NVIDIA GPU is required for spconv and full model training.

Setup

Option 1: Docker (Recommended)

We use Docker to manage the complex geospatial (GDAL/PDAL) and GPU (CUDA) dependencies. This ensures the environment works consistently across different machines.

Build the image:

# If on Linux/Windows (Standard)
docker compose build

# If on Mac (Build only - cannot run GPU training locally)
docker build --platform linux/amd64 -t bridge-classifier .

Before running with Docker:

Environment file: Copy .env.example to .env and edit if needed. DATA_DIR is used by docker-compose to mount the ML data directory (e.g. /data/ml-data or an absolute path on the host).
```
cp .env.example .env
```
Experiments directory (before training): Create the experiments directory at repo root and make it writable so the container can write logs and checkpoints (default ./experiments). Required when using Docker; without it, Step 4 (Train Model) may fail with a permission error.
```
mkdir -p experiments && chmod 777 experiments
```

Run the Pipeline:

Tip: Use tee to log training output: ... 2>&1 | tee experiments/<name>/training_console.log

# Step 0 (Optional): Prepare Run from Flat GeoPackage
# If you have bridge data in a flat GeoPackage (e.g. from NOAA), split it into
# the per-HUC directory structure expected by Step 1:
docker compose run --rm bridge-classifier \
  python utils/prepare_run.py \
  --input ./data/noaa-provided/bridges.gpkg \
  --run-name my-run-name \
  --huc-column huc8 \
  --osmid-column osmid
# Then point Steps 1-2 at the run's directories:
#   --hucs-dir ./data/runs/my-run-name/hucs
#   --source-dir ./data/runs/my-run-name/source
#   --silver-dir ./data/runs/my-run-name/silver_training

# Step 1: Download & Weak Supervision
# --skip-existing skips already processed outputs, bridges with no lidar points (count==0), and bridges that timed out.
docker compose run --rm bridge-classifier \
  python src/download_and_weak_supervise_hucs.py \
  --source-dir ./data/ml-data/source \
  --silver-dir ./data/ml-data/silver_training \
  --hucs-dir ./data/osm/hucs \
  --lidar-resources ./data/usgs_entwine/lidar_resources.geojson \
  --workers 12 \
  --skip-existing


# Step 2: Preprocess & Normalization
# --skip-existing skips already processed outputs.
docker compose run --rm bridge-classifier \
  python src/preprocess_bridges.py \
  --input-dir ./data/ml-data/silver_training \
  --output-dir ./data/ml-data/silver_training_normalized

# Step 3: Split data (train/val/test)
docker compose run --rm bridge-classifier \
  python utils/split_data.py \
  --laz-dir ./data/ml-data/silver_training \
  --npy-dir ./data/ml-data/silver_training_normalized \
  --output-dir ./data/ml-data \
  --holdout-test-ids ./data/ml-data/holdout_test.txt \
  --train-ratio 0.7 \
  --val-ratio 0.15 \
  --test-ratio 0.15 \
  --symlink

# Step 3a: Compute class weights (optional). Use output in training with --class-weights ./data/ml-data/class_weights.json
docker compose run --rm bridge-classifier \
  python utils/calculate_weights.py \
  --data-dir ./data/ml-data/training \
  --output ./data/ml-data/class_weights.json

# Step 4: Train Model (Requires NVIDIA GPU)
# Batch size tuning: --batch-size 4 --accumulate-grad-batches 4 → effective 16

mkdir -p experiments/bridge-base-all-data-v0
docker compose run --rm \
-e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
bridge-classifier python src/train.py \
  --train --augment \
  --val-dir='/data/ml-data/validation' \
  --train-dir='/data/ml-data/training' \
  --epochs 10 \
  --voxel-size 0.1 \
  --batch-size 4 \
  --accumulate-grad-batches 4 \
  --exp-name bridge-base-all-data-v0 \
  --class-weights /data/ml-data/class_weights.json \
  --num-workers 4 \
  --early-stopping \
  --early-stopping-patience 6 \
  --max-voxels 100000 \
  2>&1 | tee experiments/bridge-base-all-data-v0/training_console.log

Batch size tuning: effective_batch = batch_size × accumulate_grad_batches. For example, --batch-size 16 --accumulate-grad-batches 1 or --batch-size 4 --accumulate-grad-batches 4 both give effective batch 16.

Resume training (continue from a saved checkpoint to more epochs, e.g. 10 → 25): use --ckpt-path and a new --exp-name so the resumed run writes to a separate experiment directory. Checkpoints are saved under experiments/<exp_name>/version_0/checkpoints/ (e.g. last.ckpt).

(Adjust --ckpt-path if your experiments dir is mounted elsewhere; e.g. if experiments is at /app/experiments, use /app/experiments/bridge-base-all-data-v0/version_0/checkpoints/last.ckpt.)

mkdir -p experiments/bridge-base-all-data-v1
docker compose run --rm \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  bridge-classifier python src/train.py \
  --train --augment \
  --val-dir=/data/ml-data/validation \
  --train-dir=/data/ml-data/training \
  --epochs 25 \
  --ckpt-path /app/experiments/bridge-base-all-data-v0/version_0/checkpoints/last.ckpt \
  --exp-name bridge-base-all-data-v1 \
  --voxel-size 0.1 \
  --batch-size 4 \
  --accumulate-grad-batches 4 \
  --class-weights /data/ml-data/class_weights.json \
  --num-workers 4 \
  --early-stopping \
  --early-stopping-patience 6 \
  --max-voxels 100000 \
  2>&1 | tee experiments/bridge-base-all-data-v1/training_console.log

Use the same --exp-name as in your train command so the log lives next to version_0/ (e.g. experiments/bridge-base-all-data-v1/training_console.log).

Fine-tune from a pretrained checkpoint (load weights only, fresh optimizer and epoch counter - unlike --ckpt-path which resumes full training state). Use --freeze-encoder to train only the decoder and classifier while keeping the encoder frozen:

mkdir -p experiments/ft-gold-optA-v0
docker compose run --rm \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  bridge-classifier python src/train.py \
  --train --augment --augment-extra \
  --finetune /app/experiments/bridge-base-all-data-v3/version_0/checkpoints/bridge-unet-epoch=35-val_deck_iou=83.3369.ckpt \
  --freeze-encoder \
  --train-dir=/data/ml-data/gold-split/training \
  --val-dir=/data/ml-data/gold-split/validation \
  --learning-rate 1e-4 \
  --epochs 30 \
  --batch-size 16 \
  --accumulate-grad-batches 1 \
  --dice-loss \
  --early-stopping --early-stopping-patience 10 \
  --monitor val_deck_iou \
  --exp-name ft-gold-optA-v0 \
  --voxel-size 0.1 \
  --max-voxels 100000 \
  2>&1 | tee experiments/ft-gold-optA-v0/training_console.log

Training options (for src/train.py):

--monitor: Metric used for best-model checkpointing and early stopping (default: val_deck_iou). Use val_deck_iou to optimize for deck IoU, or val_loss for validation loss. When no validation data is used, train_loss is used instead.
--early-stopping: Stop training when the monitored metric does not improve.
--early-stopping-patience: Number of epochs to wait with no improvement before stopping (default: 10). Used only when --early-stopping is set.
--dice-loss: Use combined Dice + CrossEntropy loss (0.5*CE + 0.5*Dice) instead of CE only. Dice loss directly optimizes IoU (the target metric) and handles class imbalance naturally. Recommended for fine-tuning on small gold datasets.
--augment-extra: Enable extra augmentation on top of --augment: random XY-flip, random scaling (0.9–1.1x), intensity jitter, and random point dropout (5–10%). Requires --augment. Recommended when training on small datasets to increase effective sample diversity.
--ckpt-path: Path to a checkpoint to resume training (e.g. .../checkpoints/last.ckpt). Use a new --exp-name for the resumed run so logs and checkpoints go to a separate experiment directory; the original run is left unchanged.
--finetune: Path to a checkpoint for fine-tuning. Loads weights only (fresh optimizer and epoch counter). Mutually exclusive with --ckpt-path.
--freeze-encoder: Freeze all encoder layers so only the decoder and classifier are trained. Useful for fine-tuning on small datasets (e.g. gold annotations) where you want to preserve learned encoder features.

See Troubleshooting for permission and other issues.

Development Mode: Run docker compose run --rm bridge-classifier for an interactive shell (or docker run --gpus all -v "$(pwd):/app" -it bridge-classifier without Compose). Then run scripts directly (e.g., python src/train.py --help).

Option 2: Local Conda Install

For local installs without Docker, environment.yaml handles all dependencies (geospatial and CUDA).

# 1. Create the environment from file
mamba env create -f environment.yaml

# 2. Activate the environment
mamba activate bridge-classify

# 3. Verify GPU availability
python -c "import torch; print(f'CUDA Available: {torch.cuda.is_available()}')"

See Troubleshooting.

Data-processing only (no GPU): For data-processing scripts (download, split, verify, visualize) on a CPU-only machine, use the lighter env that omits PyTorch/CUDA/spconv:

conda env create -f environment-data.yaml
conda activate bridge-classify-data

File	Env name	Use case
`environment.yaml`	`bridge-classify`	Full stack: training, inference, data processing. Requires NVIDIA GPU.
`environment-data.yaml`	`bridge-classify-data`	Data processing only. No GPU deps. Safe for CPU-only machines.

Option 3: Manual Installation

If the YAML installation fails or you need to build the environment step-by-step, follow these commands:

# Create mamba/ conda environment
mamba create -n bridge-classify python=3.11
mamba activate bridge-classify

# Install core dependencies
mamba install -c conda-forge python-pdal gdal entwine matplotlib geopandas tqdm seaborn
# if needed interactive shell
mamba install ipython

# Install PyTorch & Lightning
# Note: Using --index-url to find CUDA 12.6 specific wheels
# adjust cuda version as needed
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126
pip install lightning tensorboard

# Install Spconv (Sparse Convolution)
# for CPU (only for forward pass and network check); won't full train
# pip install spconv
# needs gpu for full training
# Adjust CUDA version as needed (https://github.com/traveller59/spconv)
pip install spconv-cu120

# Pin NumPy to avoid the Floating point exception (core dumped) error ([spconv #725](https://github.com/traveller59/spconv/issues/725))
mamba install numpy=1.26.4

# Optional: for saving graph
# pip install torchview graphviz
# on linux, you also need an OS-level graphviz package
# sudo apt-get install graphviz

# later when running script if getting an error
# Error: /lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.15' not found
#mamba install -c conda-forge libstdcxx-ng
# if above doesn't work; run this in terminal before starting the script
# export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH

See Troubleshooting for libstdc++ and other issues.

Model Registry

Trained models are tracked in an S3-based registry (registry.json) for experiment lineage and promotion to production.

Register a model (uploads best checkpoint + config to S3):

python utils/register_model.py \
  --exp-dir ./experiments/bridge-base-all-data-v3/version_0 \
  --name bridge-base-all-data-v3 \
  --description "Silver-only training, 477K bridges, base_channels=16, 0.1m voxel" \
  --bucket my-bucket --prefix bridge-classification/models

Register a fine-tuned model (with parent lineage):

python utils/register_model.py \
  --exp-dir ./experiments/ft-gold-optA-v0/version_0 \
  --name ft-gold-optA-v0 \
  --description "Fine-tuned from v3, frozen encoder, gold data" \
  --parent bridge-base-all-data-v3 \
  --bucket my-bucket --prefix bridge-classification/models

Register evaluation results (updates registry with metrics, changes stage to evaluated):

# With pre-computed inference (no GPU needed):
python utils/evaluate_model.py \
  --gold-dir ./data/ml-data/gold-data \
  --test-dir ./data/ml-data/testing \
  --inference-dir ./evaluation_results/v3/inference_output \
  --output-dir ./evaluation_results/v3 \
  --register --model-name bridge-base-all-data-v3 \
  --bucket my-bucket \
  --prefix bridge-classification/models \
  --profile my-profile

# With live inference (GPU):
python utils/evaluate_model.py \
  --gold-dir ./data/ml-data/gold-data \
  --test-dir ./data/ml-data/testing \
  --model ./experiments/.../checkpoints/best.ckpt \
  --output-dir ./evaluation_results/new-model \
  --register --model-name new-model \
  --bucket my-bucket \
  --prefix bridge-classification/models \
  --profile my-profile

Inspect the registry:

aws s3 cp s3://my-bucket/bridge-classification/models/registry.json - --profile my-profile | python -m json.tool

MLOps workflow: train → register_model.py → evaluate_model.py --register → promote_model.py

The registry stores model metadata (config, lineage, evaluation results) and follows a stage lifecycle: experimental → evaluated → staging → production.

Gold Annotation

Gold data consists of manually annotated point clouds using the same 4-class scheme (Background, Ground/Water, Bridge Deck, Obstacles). Silver labels from the weak supervision pipeline provide the starting point; annotators correct and refine them.

Gold data lives in gold-data/{huc_id}/ on S3, preprocessed into gold-data-normalized/, and split via gold-split/ (training/validation/testing).

To find new candidates for annotation:

python utils/find_new_source_candidates.py --proven-linear false --sample-size 0

See docs/data-pipeline.md (Extended Workflows) for the full discovery-to-annotation pipeline.

Testing

Install test dependencies and run the suite:

pip install -r requirements-test.txt
python -m pytest tests/ -v

Tests run in seconds and require only numpy + boto3 (no GPU, PDAL, or conda). CI runs automatically on push to main/dev and on PRs via GitHub Actions.

Troubleshooting

Permission denied: When running the pipeline (e.g. writing to data/), ensure permissions: chmod -R 777 <folder>. If training fails with permission errors on the experiments directory, ensure experiments exists and is writable: mkdir -p experiments && chmod 777 experiments.
libstdc++ / CXXABI_1.3.15: Common on Linux. Try mamba install -c conda-forge libstdcxx-ng. If that fails, run before scripts: export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH.
NumPy and spconv: Pin NumPy to avoid "Floating point exception (core dumped)" (spconv #725): mamba install numpy=1.26.4.
No JSON files / no class distribution: If calculate_weights.py reports no files or no distribution, run the split step (Step 3) first and use --data-dir ./data/ml-data/training.

Data Download

Make a folder named data/ in the same level as src/
Make a subfolder usgs_entwine/ and osm/hucs/ inside data/ folder.
Download the USGS lidar resources: wget https://raw.githubusercontent.com/hobuinc/usgs-lidar/refs/heads/master/boundaries/resources.geojson -O data/usgs_entwine/lidar_resources.geojson
HUC-level OSM data: s3://my-bucket/bridge-classification/osm/hucs/ (organized by huc_id folder level)
The pipeline creates data/ml-data/ and its subfolders when you run Steps 1–4; see Output directories for the full layout. All paths are configurable via CLI arguments.

Download and organize it to match this structure.

data/
├── usgs_entwine/
│   └── lidar_resources.geojson
├── osm/
│   └── hucs/
│       ├── 02050206/
│       │   └── osm_bridges_lidar_subset__02050206.gpkg
│       ├── 03070101/
│       │   └── osm_bridges_lidar_subset__03070101.gpkg
│       ├── 11010009/
│       │   └── osm_bridges_lidar_subset__11010009.gpkg
│       └── ... (other huc_id folders)
└── ml-data/                 # Created by pipeline (Steps 1–4); see Output Structure
    └── ...

Classification Labels

See Architecture - Classification Schema for the 4-class labeling scheme and ASPRS code mappings.

Output Structure

Output directories

All directory paths are CLI-configurable - the defaults shown below can be overridden with --source-dir, --silver-dir, --output-dir, etc. See each script's --help for options.

The pipeline supports two directory layouts:

Default paths (data/ml-data/) - standard training workflow (Steps 1–4)
Named runs (data/runs/<run-name>/) - isolated runs created by utils/prepare_run.py (e.g., from a flat NOAA GeoPackage)

data/
├── osm/hucs/                        # HUC bridge geometries (input)
├── usgs_entwine/                    # Lidar source index (input)
│
├── <data-dir>/                      # Training pipeline output (default: ml-data)
│   ├── source/                      # Step 1: raw LAZ per HUC
│   ├── silver_training/             # Step 1: weak-supervised LAZ
│   ├── silver_training_normalized/  # Step 2: .npy + .json
│   ├── training/                    # Step 3: train split
│   ├── validation/                  # Step 3: val split
│   ├── testing/                     # Step 3: test split
│   ├── gold-data/                   # Human-annotated bridges
│   ├── gold-split/                  # Gold train/val/test
│   ├── evaluation_results/          # Runtime: model evaluation outputs
│   ├── experiments/                 # Runtime: training experiment logs
│   ├── split_manifest.json          # Step 3: split metadata
│   ├── split_{train,val,test}_ids.txt  # Step 3: per-split bridge IDs
│   └── class_weights.json           # Step 3a (optional)
│
└── runs/<run-name>/                 # Named runs (via prepare_run.py)
    ├── input/                       # Original input GPKG (for reproducibility)
    ├── hucs/                        # Per-HUC bridge geometries
    ├── source/                      # Raw LAZ
    ├── silver_training/             # Weak-supervised LAZ
    ├── predictions/                 # Batch inference output
    └── run_config.json              # Run metadata

Batch inference outputs

When running inference at scale via AWS Batch, the pipeline writes prediction files and two observability files to the S3 output prefix:

_run_config.json - saved at job submission by scripts/submit_batch_job.py (job ID, manifest URI, array size, git commit, SPOT rate estimate)
_run_report.json - generated after completion by scripts/post_run_report.py (S3 audit results, per-child and per-bridge timing, computed cost estimate, failure reasons for missing bridges)
_missing.txt - manifest of bridges that failed or were skipped (for re-submission)
Per-bridge prediction files (one per bridge, naming depends on inference mode):
- masked: {huc}_bridge_{osmid}_{source}_bridge_masked.laz - original ASPRS classes preserved, bridge deck points reclassified to class 17
- raw: {huc}_bridge_{osmid}_{source}_predicted.laz - all model class labels (0–3) written directly
- both: writes both files per bridge

Notebooks

The notebooks/ folder contains reproducible Jupyter notebooks:

notebooks/dataset_overview.ipynb - Downloads ml-data artifacts from S3 (split ID files, class_weights.json, optional osm_bridge_counts.json), computes unique HUCs in the split, train/val/test line counts, total points from class weights, and OSM bridge counts (when the counts file is on S3). Produces a class distribution horizontal bar chart and, if SILVER_NORMALIZED_DIR is set to a local path, a per-bridge point count histogram. Downloads HUC8 boundaries from S3 and produces a map of which HUC8s appear in the dataset split.
notebooks/training_plots.ipynb - Plots training curves from experiment metrics (compare/merge runs, optional best-epoch annotation). Configure EXPERIMENTS_ROOT, EXPERIMENT_NAMES, and ANNOTATE_BEST_METRIC in the notebook. Can be extended later with validation/test metrics, confusion matrix, etc.

Run the dataset overview notebook after configuring the S3 bucket/prefix (and optional AWS profile) in the Config cell or via environment variables (BRIDGE_S3_BUCKET, BRIDGE_S3_ML_PREFIX, AWS_PROFILE). HUC8 boundaries are read from S3 (BRIDGE_S3_HUC8_KEY); see the notebook for details.

File Naming Conventions

Download & Weak Supervision:

Source files: bridge_{osmid}_{source_name}.laz
Silver files: bridge_{osmid}_{source_name}.laz

Normalization:

Data files: bridge_{osmid}_{source_name}.npy
Metadata files: bridge_{osmid}_{source_name}.json

Metadata JSON Structure

The normalization script generates JSON metadata files with the following structure:

{
    "original_file": "bridge_5069009_USGS_LPC_PA_South_Central_B2_2017_LAS_2019.laz",
    "original_path": "data/ml-data/silver_training/02050206/bridge_5069009_USGS_LPC_PA_South_Central_B2_2017_LAS_2019.laz",
    "offsets": {
        "x_center": -8548539.406538319,
        "y_center": 4995360.8107266445,
        "z_min": 128.93
    },
    "stats": {
        "point_count": 27166,
        "bridge_points": 13030,
        "ground_water_points": 5910,
        "obstacle_points": 6112,
        "background_points": 2114
    },
    "class_distribution": {
        "0": 2114,
        "1": 5910,
        "2": 13030,
        "3": 6112
    }
}

Visualizing training metrics

Viewing TensorBoard while training is running

docker compose run -p 6006:6006 --rm bridge-classifier \
  tensorboard --logdir=experiments/<exp_name>/version_0/ --bind_all

Open http://localhost:6006. The --logdir path should match your experiment name.

CSV metrics and static plots

Training (Step 4) writes metrics via Lightning CSVLogger to ./experiments/<exp_name>/version_<N>/metrics.csv. The script utils/visualize_metrics.py plots epoch-level train/validation curves (loss, deck IoU, precision, recall, overall accuracy) and saves training_curves.png in the same directory. Diagnostic metrics such as Num Voxels and Max Sample Voxels are excluded. In compare mode, plots use distinct colors for train vs validation and short labels suitable for presentation slides.

Examples:

Default (loads ./experiments/bridge_classify_base/version_0/metrics.csv):
```
python utils/visualize_metrics.py
```

Specific experiment and version:

python utils/visualize_metrics.py --exp bridge_classify_base --ver 0
# optionally: --root ./experiments

Direct path to a metrics file:

python utils/visualize_metrics.py --csv ./experiments/bridge_classify_base/version_0/metrics.csv

With Docker:

docker compose run --rm bridge-classifier python utils/visualize_metrics.py

Compare mode

Use --compare with comma-separated experiment names (under --root) to plot multiple experiments on the same axes. Optionally set --compare-versions (e.g. 0,0,0); if omitted, version 0 is used for all. Use --out to set the output path (default: {root}/compare_training_curves.png).

python utils/visualize_metrics.py --root experiments_copy --compare bridge-base-all-data-v0,bridge-base-all-data-v1 --out experiments_copy/compare_v0_v1.png

Merge-resumed (one continuous curve)

When comparing exactly two experiments, add --merge-resumed to merge them into one timeline (first run’s epochs, then second run’s later epochs) and plot a single train/val series with a short legend (e.g. "Train", "Val") and run info in the figure suptitle (e.g. "v0 → v1").

python utils/visualize_metrics.py --root experiments_copy --compare bridge-base-all-data-v0,bridge-base-all-data-v1 --merge-resumed --out experiments_copy/merged_v0_v1.png

Reference

For full CLI options: python utils/visualize_metrics.py --help

Point Cloud Visualization

utils/visualize_pointcloud.py renders side-by-side 3D point cloud figures comparing gold annotations vs model predictions, or raw elevation vs weak supervision labels.

# Gold annotations vs model predictions
python utils/visualize_pointcloud.py gold-vs-model \
    --gold data/ml-data/gold-data-normalized/03070101/bridge_40787878_GA_Central_1_2018.npy \
    --model data/ml-data/evaluation_results/v5-gold-134/inference_output/03070101/bridge_40787878_GA_Central_1_2018.laz \
    -o notebooks/outputs/gold_vs_model.png

# Raw elevation vs RANSAC weak supervision labels
python utils/visualize_pointcloud.py source-vs-silver \
    --source data/ml-data/source/01010005/bridge_1090522653_ME_Eastern_TL_2017.laz \
    --silver data/ml-data/silver_training/01010005/bridge_1090522653_ME_Eastern_TL_2017.laz \
    -o notebooks/outputs/source_vs_silver.png

For full CLI options: python utils/visualize_pointcloud.py --help

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

USGS Lidar Bridge Classification

Table of Contents

Documentation

Building the docs locally

Deploying to GitHub Pages

Pipeline Overview

Installation

Prerequisites

Setup

Option 1: Docker (Recommended)

Option 2: Local Conda Install

Option 3: Manual Installation

Model Registry

Gold Annotation

Testing

Troubleshooting

Data Download

Classification Labels

Output Structure

Output directories

Batch inference outputs

Notebooks

File Naming Conventions

Metadata JSON Structure

Visualizing training metrics

Viewing TensorBoard while training is running

CSV metrics and static plots

Point Cloud Visualization

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 113 Commits
.github/workflows		.github/workflows
docs		docs
notebooks		notebooks
scripts		scripts
src		src
terraform		terraform
tests		tests
utils		utils
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
docker-compose.yml		docker-compose.yml
environment-data.yaml		environment-data.yaml
environment.yaml		environment.yaml
mkdocs.yml		mkdocs.yml
pytest.ini		pytest.ini
requirements-docs.txt		requirements-docs.txt
requirements-test.txt		requirements-test.txt

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

USGS Lidar Bridge Classification

Table of Contents

Documentation

Building the docs locally

Deploying to GitHub Pages

Pipeline Overview

Installation

Prerequisites

Setup

Option 1: Docker (Recommended)

Option 2: Local Conda Install

Option 3: Manual Installation

Model Registry

Gold Annotation

Testing

Troubleshooting

Data Download

Classification Labels

Output Structure

Output directories

Batch inference outputs

Notebooks

File Naming Conventions

Metadata JSON Structure

Visualizing training metrics

Viewing TensorBoard while training is running

CSV metrics and static plots

Point Cloud Visualization

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages