Skip to content

Commit 14043b3

Browse files
djeshkovdfw0000
djeshkov
and
dfw0000
authored
docs: add real-world case study from first-hour production observation (#5)
Adds docs/CASE-STUDY.md showing concrete deployment results when the per-IP pass was enabled on a Laravel-fronted reference site: - 402 bot IPs banned in the first scan - 4 new scrapers caught autonomously in the next 50 minutes - 89 nginx 444 responses to bot retry attempts in the first hour - 0 false positives observed, 0 customer support tickets - ~10-15 sec cron run time Includes: - Top blocked IPs with score breakdown - Score signal distribution across the 402 first-pass entries - Per-IP pass vs subnet pass — what the subnet pass misses by design - Resource cost and recommended defaults from this data All site/customer identifiers anonymized; only public hostile-source data (scraper IPs, ASNs, user-agents) shown. README links to the case study from the per-IP section. Co-authored-by: Daniil Zheshkov <dan@cloudzen.it>
1 parent dc3c835 commit 14043b3

2 files changed

Lines changed: 133 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,7 @@ Enable the per-IP pass when you observe **either**:
112112
- Session-recording or analytics tools show short bot-like sessions (< 5s, 0 clicks) from many countries / IPs — but `--show-scores` (the subnet pass) finds nothing because no /24 is hot enough.
113113

114114
Backtest details and signal calibration: [docs/SCORING.md § Per-IP pass](docs/SCORING.md#per-ip-pass-opt-in).
115+
Real-world first-hour results from a Laravel-fronted reference site: [docs/CASE-STUDY.md](docs/CASE-STUDY.md).
115116

116117
## Quick install
117118

‎docs/CASE-STUDY.md‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# Case study — 1-hour production observation
2+
3+
Real-world data from the first hour after the per-IP pass was enabled on a Laravel-fronted reference site behind Cloudflare. All site/customer identifiers anonymized; only public hostile-source data (scraper IPs, ASNs, user-agents) is shown.
4+
5+
## TL;DR
6+
7+
| Metric | Value |
8+
|--------|-------|
9+
| **Bot IPs banned in the first run** | 402 |
10+
| **New IPs caught in the next 5 cron ticks** (50 min) | 4 |
11+
| **`nginx 444` responses** (actual blocks fired) | 89 |
12+
| **Subnet-pass candidates above threshold** during the hour | 0 |
13+
| **False positives observed** | 0 |
14+
| **Customer support tickets / regressions** | 0 |
15+
| **Server load impact** | ~10-15 sec per cron run, no nginx reload errors |
16+
17+
## Background
18+
19+
The site was already running the **subnet pass** (`/24` aggregation) with default settings, but had recently noticed a wave of distributed Chinese-cloud bots hitting `/<some-public-url>/<UUID>` pages — each IP making 1-2 requests over a 24h window. None of these triggered the subnet pass because no single `/24` accumulated enough volume.
20+
21+
The per-IP pass was enabled with the default threshold (`per_ip_threshold=9`).
22+
23+
## T+0 — first scan after enabling per-IP
24+
25+
Output of `autoblock --show-per-ip` on the 30-minute window of access logs immediately after enabling:
26+
27+
```
28+
Per-IP pass: threshold=9, candidates=402
29+
SCORE REQS IP ASN reasons
30+
----- ---- ------------------ ----------------------------------- ------------------------------------------------
31+
13 20 202.46.62.24 BAIDU Beijing Netcom noassets,noref,4xx,upath,cloud,ua:oldchrome
32+
13 15 202.46.62.81 BAIDU Beijing Netcom noassets,noref,4xx,upath,cloud,ua:oldchrome
33+
13 15 202.46.62.117 BAIDU Beijing Netcom noassets,noref,4xx,upath,cloud,ua:oldchrome
34+
13 15 202.46.62.20 BAIDU Beijing Netcom noassets,noref,4xx,upath,cloud,ua:oldchrome
35+
... (~30 more from 202.46.62.0/24 at score 11-13)
36+
10 6 104.210.56.225 MICROSOFT-CORP-MSN-AS-BLOCK noassets,noref,cloud,ua:oldchrome (HubSeedsBot/1.0)
37+
10 7 95.211.164.101 LEASEWEB-NL-AMS-01 Netherlands noassets,noref,upath,cloud (VacancyValidator/1.0)
38+
10 8 182.242.168.225 CHINANET-BACKBONE noref,4xx,upath,cloud,ua:oldchrome
39+
... (~370 more)
40+
```
41+
42+
All 402 entries written to `/etc/nginx/blocked-ips.conf`, nginx reloaded automatically by the script.
43+
44+
## T+10 to T+50 — five cron ticks
45+
46+
The script runs every 10 minutes. Each tick re-scans the 30-minute window and either adds new candidates or refreshes the TTL of existing ones.
47+
48+
```
49+
T+10: +1 new block | 402 existing IPs extended | 511 subnets scanned
50+
T+20: +1 new block | 403 existing IPs extended | 498 subnets scanned
51+
T+30: +2 new blocks | 404 existing IPs extended | 553 subnets scanned
52+
T+40: +0 new blocks | 406 existing IPs extended | 563 subnets scanned
53+
T+50: +0 new blocks | 406 existing IPs extended | 604 subnets scanned
54+
```
55+
56+
**Four new scrapers caught autonomously:**
57+
58+
```
59+
T+10 2a09:bac5:50ee:3032::4cd:b IPv6 cloud, score 9 noassets,extref,upath,cloud
60+
T+20 45.148.10.21 score 10 26 req, 4xx probing, old Chrome
61+
T+30 139.199.162.133 score 10 cloud + old Chrome
62+
T+30 100.24.210.54 AMAZON-AES, score 10 cloud + old Chrome
63+
```
64+
65+
Each was first observed in the access log between scan ticks, scored above the threshold once it had enough requests to evaluate, and added to the block list at the next tick.
66+
67+
## Actual nginx blocks fired (`HTTP 444`)
68+
69+
Counting `444` responses in the access log over the first hour:
70+
71+
```
72+
89 total 444 responses to bot retry attempts
73+
74+
Top hit IPs:
75+
5 45.148.10.21 (the new scraper caught at T+20 — its remaining attempts blocked)
76+
3 100.24.210.54
77+
2 54.193.6.55
78+
2 202.46.62.88
79+
2 202.46.62.79
80+
... (~15 more Baidu Netcom IPs at 2 hits each)
81+
```
82+
83+
Pattern: most scrapers came back 2-5 times within the hour, each time getting `444` (silent close, zero response body). The Baidu Netcom subnet is "polite" enough to keep some interval between attempts; the more aggressive ones (AWS-hosted, generic VPS) retry harder.
84+
85+
## Score signal distribution across the 402 first-pass IPs
86+
87+
| Signals present | Count | Typical source |
88+
|-----------------|-------|----------------|
89+
| cloud + ua:oldchrome + noref + noassets + upath + 4xx (score 13) | ~120 | Coordinated probing botnets (Baidu Netcom, parts of CHINANET) |
90+
| cloud + ua:oldchrome + noref + noassets (score 10) | ~180 | One-shot cloud scrapers (AWS, Azure, GCP, OVH, Hetzner) |
91+
| cloud + ua:oldchrome + upath (score 8-9) | ~70 | Low-volume scanning from cloud |
92+
| cloud + ua:headless (score 6, didn't pass) | not blocked | Edge cases — held under threshold for safety |
93+
94+
The signals composed naturally: every IP had a different combination, and the score reflected how confident the fingerprint was.
95+
96+
## What the subnet pass would NOT have caught
97+
98+
The Baidu Netcom subnet `202.46.62.0/24` — the largest single concentration in the data — **did** previously trigger the subnet pass after the operator manually added a one-off `/24` block (because the original cron was misconfigured and scanning the wrong access log). After fixing that, the subnet pass would have caught it eventually — but only as one block of the entire `/24`.
99+
100+
The per-IP pass produced:
101+
- **30+ separate `/32` bans** with individual evidence trails (score, signals, request count, ASN)
102+
- The remaining 370 unrelated scrapers from **other cloud sources** — none of which would have triggered the subnet pass because each one came from a different `/24` with only 1-5 requests.
103+
104+
That's the architectural difference between concentration-based and behavior-based detection.
105+
106+
## Resource cost
107+
108+
- **Cron run time**: 8-15 seconds per tick (down from ~30 sec when the script was hitting the larger main access log without any tuning; jumped up because parsing a 12 MB log on each scan and running ip-api batches takes time).
109+
- **nginx reload**: handled inside the script after each successful write; no errors, no observable downtime.
110+
- **`blocked-ips.conf` file size**: ~50 KB after first hour (~400 entries × ~120 bytes per line including comment).
111+
- **Cleanup cron**: at 03:30 UTC daily, removes expired entries (default 7-day TTL). Tested working.
112+
113+
## What this doesn't show
114+
115+
- **Long-term FP rate**. The 0% false-positive number is after manual review of the top 35 candidates against ip-api.com. A small fraction of edge-case real users (VPN exit + old browser + ad blocker + direct link, no referer) could theoretically score 8-10. Recommend monitoring `444` responses by country/device in Cloudflare analytics for the first week.
116+
- **Comparison against other tools**. We didn't run Cloudflare Bot Fight Mode or fail2ban in parallel for A/B benchmarking. Just internal validation against ip-api.com reputation flags.
117+
- **Effect on actual server load**. The site was already healthy before per-IP enable; nothing to "rescue" — this is preventive defense, not incident response.
118+
119+
## Recommended defaults from this data
120+
121+
For a similar setup (Laravel/PHP CMS behind Cloudflare, mostly residential ISP traffic + cloud-hosted scrapers):
122+
123+
```ini
124+
per_ip_enabled=true
125+
per_ip_threshold=9 # default; ~1.5% of IPs flagged
126+
per_ip_ttl_days=7
127+
per_ip_chrome_min_version=142 # ~6mo behind current Chrome stable
128+
internal_ref_hosts=your-domain.com # critical for noref/extref signals
129+
self_ips=YOUR.SERVER.IP # exclude your own IP for health-checks
130+
```
131+
132+
Threshold 9 was tested against this traffic profile. Sites with heavy power-user / VPN traffic might raise to 10 (cuts to ~80 IPs in first run, very conservative). Sites that want maximum coverage and have CF + Turnstile or similar as fallback might drop to 8 (extends to ~600 IPs but admits some borderline cases).

0 commit comments

Comments
 (0)