Bot Detection

Known Bad User-Agents

Matches the User-Agent string against a database of known scrapers, scanners, and malicious automation tools.

The known bad user-agents checker tests the User-Agent header against a continuously updated database of patterns associated with scrapers, vulnerability scanners, exploit frameworks, and other malicious tools. Each pattern carries a severity level that maps to a configurable penalty weight.

This checker runs in the heavy phase. It loads patterns from the useragent.mdb LMDB database built during bot-detector init.

How It Works

On first run, the checker loads up to 10,000 User-Agent patterns from the LMDB database and groups them by severity level: critical, high, medium, and low. It builds a combined regular expression for each group and caches the compiled patterns in memory.

For each incoming request, the checker tests the User-Agent string against the patterns in order from highest to lowest severity. On the first match, it applies the corresponding penalty and attaches BAD_UA_DETECTED to the reason list. Only one severity level fires per request: the highest matching level wins.

This checker only runs when enableUaAndHeaderChecks.penalties.badUaChecker is true. It is controlled by the enableUaAndHeaderChecks checker, not by its own top-level enable flag. See the UA & Header Analysis page for details.

Configuration

server.ts

await defineConfiguration({
  store: { main: { driver: 'sqlite', name: './bot-detector.db' } },
  checkers: {
    knownBadUserAgents: {
      enable: true,
      penalties: {
        criticalSeverity: 100,
        highSeverity: 80,
        mediumSeverity: 30,
        lowSeverity: 10,
      },
    },
    // Also requires badUaChecker: true in enableUaAndHeaderChecks
    enableUaAndHeaderChecks: {
      enable: true,
      penalties: {
        badUaChecker: true,
      },
    },
  },
})

All weights live inside the penalties: {} sub-object.

criticalSeverity

number

Penalty for User-Agents matching patterns at the critical severity level. These are known exploit tools and high-confidence malicious agents. Default: 100.

highSeverity

number

Penalty for high-severity patterns. Typically known scrapers, headless tool wrappers, and aggressive crawlers. Default: 80.

mediumSeverity

number

Penalty for medium-severity patterns. Tools that may be used legitimately in some contexts but are frequently abused. Default: 30.

lowSeverity

number

Penalty for low-severity patterns. Weak signals such as older or generic tool identifiers. Default: 10.

Reason Codes

Code	Trigger
`BAD_UA_DETECTED`	The User-Agent matched a pattern in the LMDB database at any severity level.

Severity Levels

Level	Description	Default penalty
`critical`	Known exploit tools, attack frameworks, confirmed malicious agents	`100`
`high`	Known scrapers, headless wrappers, aggressive crawlers	`80`
`medium`	Dual-use tools that appear in both legitimate and abusive contexts	`30`
`low`	Generic or outdated tool identifiers with lower confidence	`10`

The criticalSeverity penalty defaults to 100, matching banScore. A single critical match immediately bans the visitor. This is intentional: critical patterns correspond to known attack tools that should never reach your application.

Keep the user-agent database current by running bot-detector refresh periodically. The useragent.mdb database is compiled from curated community sources and grows over time as new tools are identified.

Geolocation

Security