New: $CRAWL holders can read sites that block bots
Let your agent read the whole web. Even the sites that block bots.
Search, scrape, map and crawl any site, free and local. Hold $CRAWL and stealth mode gets your agent past the sites that turn bots away.
Free version: no account, no key. Works with Claude, Cursor, Codex, Windsurf and Gemini.
Sample results. Install it free to run it on any site.
PRODUCT
Four tools. One for every way an agent reads.
Search
Find pages on the live web. Titles, links and snippets your agent can pick from.
Scrape
Any page to clean Markdown. Menus, ads and cookie banners left behind.
Map
Every page a site has, from its sitemap, or from its links when there isn't one.
Crawl
A whole docs site or section in one go, ready for your agent's context.
INSTALL
Thirty seconds. One command.
Run setup
Needs Node 18 or newer.
It finds your apps
Claude, Cursor, Codex, Windsurf and Gemini get it added. Your settings are backed up first.
Just ask
"Read the docs at this link and write me the integration." Your agent handles the rest.
Using another app? Add it by hand
Paste this into your app's MCP settings:
PRICING
Free to run. Better to hold.
Free
- ✓ Search, scrape, map, crawl
- ✓ No account or key
- ✓ Polite by default: robots.txt, paced requests
Holder
$CRAWL- ✓ Everything in Free
- ✓ Stealth mode for sites that block bots
- ✓ JavaScript pages, PDFs and Office files
Buy and burn
Fees buy $CRAWL on Uniswap and send it to the dead address. The contract can't send money anywhere else, not even to us.
View the contract ↗Questions
Is it really free?
Yes. The free version runs on your own computer, so it costs us nothing to give away. The pro cloud costs money to run, which is why it's for holders.
What does stealth mode do?
It reads pages that turn away bots, using a real browser in the cloud. It won't solve CAPTCHAs, log in, or get past paywalls.
How do I get holder access?
Hold $50 of $CRAWL for an hour, connect your wallet and sign a message. You get a key to paste into the app. No forms, no email. Your balance is checked every hour, and access holds as long as it stays above $40.
Is it polite to websites?
By default it follows each site's robots.txt, waits between requests to the same site, and caches pages so it doesn't fetch them twice. You can turn robots.txt off at your own discretion.
Where does the money go?
Into the burn contract. It buys $CRAWL and sends it to the dead address. Anyone can check it onchain.
Give your agent a library card.
Free forever on your computer. One command.