Filed under

GEO

Every Vault entry filed under GEO, across every shelf.

ai.robots.txt: drop-in files to block the AI crawlers you do not want

A community-maintained, continuously updated list of AI crawlers, shipped as drop-in robots.txt, htaccess, Nginx and Caddy files so you can block or allow AI bots in minutes.

Starter · 10 minutes · via ai-robots-txt

Community pick

Cloudflare AI Crawl Control: see which AI bots read your site, and gate them

A Cloudflare feature, free on every plan, that shows exactly which AI crawlers hit your site and lets you allow or block them per bot with one click.

Starter · 10 minutes · via Cloudflare

The Tuesday Briefing

One editor-curated email. The updates that matter, the noise filtered out.

Free. Occasionally includes sponsored messages from our partners. Unsubscribe anytime.