Hotdog_NotHotdog

Hotdog_NotHotdog is a CLIP-based NSFW detector for images and videos. It uses OpenAI CLIP to score similarity against a fixed set of prompts and then decides whether content is Safe For Work (SFW) or Not Safe For Work (NSFW) using either a rules-based policy or a trained RandomForest classifier.

This README describes v0.1.1 of the script.

Features

• CLIP-powered: Uses OpenAI CLIP via open_clip_torch, defaulting to ViT-L-14 laion2b_s32b_b82k.
• GPU-accelerated: Runs CLIP on GPU when available (or on CPU otherwise), configurable via --device.
• Dual modes:
  • Rules mode: Deterministic policy based on CLIP scores and a similarity threshold.
  • Learned mode: RandomForest classifier on top of CLIP features (you train it yourself; no pre-trained classifier is shipped).
• Video support: Samples frames from videos at a configurable FPS (default 1.0), scores each frame, and aggregates with per-prompt max.
• Backward compatibility:
  • --classifier-mode, --auto-tune-thresholds, and --apply-auto-tune remain as legacy aliases.
  • --legacy-model restores the original v0.1.0 CLIP backbone (ViT-B-32 laion2b_s34b_b79k).
  • --legacy-csv emits the original v0.1.0 CSV schema in addition to the newer per-prompt fields.
• Local operation: After the initial CLIP weight download, everything runs locally.

Installation

Prerequisites

• Python 3.8+
• pip

Windows

1. Clone or download the repository.
2. Create a virtual environment:

   python -m venv .venv
   .\.venv\Scripts\activate

3. Install dependencies (or use pip install -r requirements.txt):

   pip install torch open_clip_torch pillow numpy tqdm scikit-learn joblib opencv-python

Linux / macOS

1. Clone or download the repository.
2. Create a virtual environment:

   python3 -m venv .venv
   source .venv/bin/activate

3. Install dependencies:

   pip install torch open_clip_torch pillow numpy tqdm scikit-learn joblib opencv-python

Usage

Basic scan (rules mode)

Scan a directory of images or videos with the simple rules policy:

python Hotdog_NotHotdog.py /path/to/images --out results.csv

Key flags:

• --threshold: similarity threshold for rules mode (default 0.20).
• --device: cuda or cpu (default: auto-detect).
• --batch-size: batch size for CLIP inference (default 32).
• --fps: video sampling FPS (default 1.0).

Learned mode overview

Learned mode uses a RandomForest classifier on top of CLIP features. You train the classifier yourself on CSVs generated by this script; no pre-trained classifier is included.

Two modes are exposed via --mode:

• --mode rules (default)
• --mode learned

Legacy flag --classifier-mode is still accepted and mapped to --mode behind the scenes.

Training workflow (v0.1.1)

1. Prepare a labeled dataset

   • Put SFW images in files whose names start with Faces (for example: Faces_0001.png).
   • Put NSFW images in files with any other name.

2. Generate a training CSV in rules mode

   From your project root:

   python Hotdog_NotHotdog.py /path/to/training_images --out training.csv

3. Auto-tune and save the learned classifier

   Use the built-in auto-tune path with an explicit training CSV:

   python Hotdog_NotHotdog.py /path/to/training_images \
     --auto-tune \
     --apply-tune \
     --training-csv training.csv \
     --classifier-model-path nsfw_classifier.pkl

   This will:

   • Train a RandomForestClassifier on features derived from the per-prompt CLIP scores.
   • Search for a probability threshold that heavily penalizes SFW false positives while keeping good NSFW recall.
   • Save:
     • the classifier to nsfw_classifier.pkl (or your chosen path),
     • thresholds/metadata to nsfw_classifier_thresholds.json.

   Legacy aliases --auto-tune-thresholds and --apply-auto-tune are still accepted and mapped to --auto-tune / --apply-tune.

4. Run learned mode

   Once you have a classifier and thresholds saved:

   python Hotdog_NotHotdog.py /path/to/images \
     --mode learned \
     --classifier-model-path nsfw_classifier.pkl \
     --out learned_results.csv

   The script will:

   • Load CLIP and run it on GPU if available.
   • Load your RandomForest from the given path.
   • Load the learned threshold from <classifier_stem>_thresholds.json if present.
   • Emit per-file CSV rows with both CLIP scores and the final decision (rules vs learned).

Sanity check mode

To validate that your trained classifier is behaving reasonably on a known set, you can use the built-in sanity check. It expects a mixture of SFW Faces* files and other NSFW samples.

python Hotdog_NotHotdog.py . \
  --sanity-check \
  --sanity-path test_images \
  --classifier-model-path nsfw_classifier.pkl

Behavior:

• Uses --sanity-path if provided; otherwise falls back to test_images/ if that folder exists.
• Requires a trained classifier at --classifier-model-path.
• Runs both rules and learned decisions over the sanity set.
• Fails (exit code 1) if:
  • any Faces* file is not SFW in both modes, or
  • no non-Faces* file is flagged NSFW in learned mode.
• Prints Sanity check PASSED and exits 0 on success.

This is useful for quick smoke tests and CI.

Labeling convention

• SFW: files starting with Faces (for example: Faces_001.png).
• NSFW: all other files.

This convention is used both when training the classifier and when running the sanity checker.

Output

By default, results are saved as CSV with one row per file. In v0.1.1 the schema includes explicit per-prompt scores:

• file
• safe_sim
• female_nipples_sim
• male_nipples_sim
• penis_sim
• vulva_sim
• anus_sim
• breast_sim
• chest_sim
• bikini_sim
• lingerie_sim
• cleavage_sim
• decision (sfw/nsfw)
• reason
• mode (rules/learned)

If you pass --legacy-csv, the script also emits the original v0.1.0-style aggregate columns (nipples_sim, clothing_sim, nsfw_youtube_guess) for easier comparison with older runs.

Learned-mode tuning philosophy

• High SFW precision: The auto-tuner heavily penalizes SFW false positives when choosing a probability threshold. In practice, the goal is that Faces* images remain SFW in both rules and learned modes on your sanity sets.
• Reasonable NSFW recall: Within that constraint, the RandomForest and threshold search are configured to recover borderline explicit samples that might be ambiguous in rules mode.
• Stable features: Feature engineering is versioned (FEATURE_VERSION = "v2") so that future changes can be introduced without silently invalidating older models.
• Legacy compatibility: --legacy-model and --legacy-csv let you compare against the original v0.1.0 behavior (CLIP backbone and CSV layout) while benefiting from the newer tooling around training and sanity checks.

Important caveats

• This tool is a filter, not a guarantee. No automated NSFW detector is perfect; there will always be some false negatives and false positives, especially on edge cases or out-of-distribution content.
• The built-in auto-tuning and sanity checks are designed to keep Faces* images SFW and catch clearly explicit content, but you remain responsible for reviewing and validating results in your own workflows, products, or moderation pipelines.

Release checklist (suggested)

Before tagging a new release or shipping a model:

1. Run sanity check

   python Hotdog_NotHotdog.py . \
     --sanity-check \
     --sanity-path test_images \
     --classifier-model-path nsfw_classifier.pkl

2. Generate a fresh training CSV (optional but recommended)

   python Hotdog_NotHotdog.py /path/to/training_images --out training.csv

3. Auto-tune and apply the classifier

   python Hotdog_NotHotdog.py /path/to/training_images \
     --auto-tune \
     --apply-tune \
     --training-csv training.csv \
     --classifier-model-path nsfw_classifier.pkl

4. Run learned mode on a test set

   python Hotdog_NotHotdog.py /path/to/test_images \
     --mode learned \
     --classifier-model-path nsfw_classifier.pkl \
     --out learned_results.csv

5. Spot-check results

   • Confirm Faces* samples are SFW.
   • Confirm clearly explicit samples are NSFW.
   • Optionally compare with earlier CSVs using --legacy-csv.

License

Licensed under the Apache License, Version 2.0. See LICENSE for the full text. Commercial use is permitted.

Copyright 2025 Creative Mayhem UG.

Contributing

Contributions welcome! Please follow PEP 8 and add tests for new features where practical. By contributing, you agree that your contributions will be licensed under the Apache License, Version 2.0.