Data Sources
Shield Base fetches data from 13 built-in sources across five categories: network routing, geographic location, anonymity and proxy detection, threat intelligence, and application-layer fingerprints. Each source produces one or more binary database files in either MMDB format (IP range lookups) or LMDB format (key-value lookups).
Sources at a Glance
| Source | Flag | Output file | Format |
|---|---|---|---|
| BGP / ASN | --bgp | asn.mmdb | MMDB |
| City Geolocation | --city | city.mmdb | MMDB |
| Country Geolocation | --geo | country.mmdb | MMDB |
| Proxy Detection | --proxy | proxy.mmdb | MMDB |
| Tor Nodes | --tor | tor.mmdb | MMDB |
| Verified Crawlers | --seo | goodBots.mmdb | MMDB |
| FireHOL Threats | --l1 --l2 --l3 --l4 --anonymous | firehol_*.mmdb | MMDB |
| Suspicious User-Agents | --useragent | useragent-db/useragent.mdb | LMDB |
| Disposable Emails | --email | email-db/disposable-emails.mdb | LMDB |
Database Formats
MMDB (MaxMind DB) is a binary format optimized for IP range lookups. Given an IP address, an MMDB reader returns the record associated with the matching CIDR prefix in microseconds. Every MMDB source requires the mmdbctl binary to compile.
LMDB (Lightning Memory-Mapped Database) is a key-value store with zero-copy reads and memory-mapped access. It is used for sources keyed by string identifiers rather than IP addresses: user-agent patterns, email domains, and TLS fingerprints. LMDB sources do not require any external binary.
Compiling All Sources
pnpm dlx @riavzon/shield-base --all --parallel --contact "Name https://example.com - [email protected]" --acceptFireholRisk
yarn dlx @riavzon/shield-base --all --parallel --contact "Name https://example.com - [email protected]" --acceptFireholRisk
npx @riavzon/shield-base --all --parallel --contact "Name https://example.com - [email protected]" --acceptFireholRisk
bunx @riavzon/shield-base --all --parallel --contact "Name https://example.com - [email protected]" --acceptFireholRisk
import { generateData as executeAll } from '@riavzon/shield-base';
await executeAll('./data/mmdb', 'Your Name https://example.com - [email protected]', true, 'mmdbctl');
