A black-box scanner sends its prayers into the dark.
Blackhole answers with pages, headers, flows, lies, half-truths, and—when needed—the unpleasant courtesy of ground truth.
Blackhole is a Python ASGI mock server for black-box scanner testing, education, and reproducible benchmarking. It serves vulnerable-looking behavior from replay profiles and explicit stateful mini-flows, while exposing a truth/scoring API to compare scanner findings against expected cases.
In other words: a scanner can hallucinate, overfit, panic, or boast. Blackhole keeps the receipts.
And every white hat should remember: all requests eventually fall into the black hole.
https://github.com/scadastrangelove/zhet-blackhole
https://github.com/scadastrangelove/zhet-blackhole
General idea
Most scanner demos are too clean, too shallow, or too theatrical. Real targets are noisy. Controls matter. False positives matter. Second-order flows matter. “Looks vulnerable” is not the same as “is vulnerable.”
Blackhole exists to make that difference explicit.
It gives you:
replayable vulnerable-looking behavior for black-box testing
safe/control branches so you can measure false-positive suppression, not just noisy detection
stateful multi-step flows for cases that do not fit in a single request
machine-readable ground truth so results can be scored instead of argued about on vibes alone
a compact educational lab for understanding how scanner logic behaves under realistic-but-controlled conditions
This is not a production honeypot. It is not a vulnerability zoo for chaos tourists. It is a ground-truth harness for people who want to test scanners, improve detection logic, compare builds, and teach others what signal and noise actually look like.
Main use cases
1. Scanner testing
Use Blackhole as a deterministic target for:
- regression testing
- FP/FN analysis
- matcher tuning
- crawler and payload evaluation
- back-to-back comparison between scanner versions
- comparison against other tools or north-star baselines
2. Education
Use Blackhole to demonstrate:
- why banner-based detection is fragile
- why multi-step workflows matter
- why “error present” is not enough to prove exploitability
- why safe/control branches are necessary for realistic evaluation
- how scanner output should be compared to truth, not mythology
3. Ground-truth source for benchmarks
Blackhole exposes a truth manifest and scoring endpoints so it can serve as a benchmark substrate for:
- internal QA
- CI regression packs
- engineering acceptance tests
- research experiments
- demo environments where you want repeatability instead of folklore
Security testing is full of heroic certainty built on flimsy evidence.
A scanner sees one stack trace and declares victory.
A benchmark shows one toy login form and calls itself realism.
A dashboard paints everything red and hopes nobody asks “compared to what?”
Blackhole is built for that uncomfortable follow-up question.
It is a small artificial world where appearances can deceive, controls can look tempting, and truth is still available if you are disciplined enough to ask for it.
Because that is the whole joke of black-box testing: you stare into the dark, send requests into the void, and hope meaning comes back. Sometimes it does. Sometimes it is only your own reflection, gravitationally bent.
No comments:
Post a Comment