AI agents
A read_url tool for Claude, Cursor, OpenClaw, n8n or your own agent, with a length cap per page.
Quick answer
What is a web reader API used for?
Teams use a web reader API to give AI agents a fetch tool, to ingest sites into RAG indexes, to monitor public pages for changes, and to convert their own websites to Markdown. WebReaderAPI covers all four with one key: /read for a page, /batch for a list, /crawl for a site, at one credit per page from $0.07/1K pages as of October 2026. No KYC; pay in crypto.
Which use case is yours?
RAG ingestion
Crawl documentation and blogs into Markdown chunks that keep their headings and source URL.
Change monitoring
Read pricing pages, policies and changelogs on a schedule and get the differences as text.
Website to Markdown
Export your own site to Markdown files for a migration, a backup or your own assistant.
PDFs
Reports, papers and manuals from a link, converted to Markdown text.
llms.txt
Build an llms.txt file for your own site from a crawl, with one line per page.
Which endpoint and how many pages does each use case need?
Typical volumes for a small team, to size a first pack. One credit per page, two when the page needs a browser.
| Use case | Endpoint | Typical pages per month |
|---|---|---|
| Agent that reads 5 pages per task, 100 tasks a day | /read | About 15,000 |
| RAG index of 3 documentation sites, refreshed weekly | /crawl, then /batch | 6,000 to build, a few hundred a week after |
| Watching 200 competitor and policy pages daily | /batch | About 6,000 |
| Exporting a 2,000-page site once | /crawl | 2,000, one time |
Which problem do these four jobs share?
They all need the readable part of a page, not its layout. An agent wastes tokens on menus; an index ranks boilerplate; a change monitor raises false alarms when an ad rotates; a migration copies cookie banners into the new site. Main-content extraction fixes the same problem in each case.
They also share the same limits. Pages behind a login or a paywall are out of reach, and sites that refuse robots are skipped and not charged. We respect robots.txt: our crawler identifies itself as WebReaderAPI, follows the rules written for it or for all robots, and does not read a page that a site disallows. Refused pages are not charged.
How do I choose between /read, /batch and /crawl?
Use the smallest tool that does the job.
/readreturns one page synchronously. It suits agents, which decide page by page what to read next./batchreads up to 100 URLs you already know, as a background job. It suits monitoring lists and refreshes./crawldiscovers and reads a whole site from a start URL, up to 10,000 pages, following its sitemap and robots.txt. It suits first builds and exports.
All three take the same options, such as format, main_content, render and max_chars, and bill the same way.
What will a first project cost?
Most first projects fit in one small pack. A prototype agent, a single documentation index or a monitoring list of a few hundred pages uses a few thousand to fifteen thousand credits a month. 10,000 pages cost $3.75 and 50,000 pages cost $14.25, as of October 2026.
Credits never expire, so buying the larger pack for a project that may grow costs nothing extra in a quiet month. Failed pages and pages refused by robots.txt are free.
Packs for a first project
Start small; credits carry over with no end date. 1 credit = 1 page read without a browser; 2 credits when the page needs a browser to render JavaScript. A request that fails, or that a site's robots.txt refuses, is not charged.
Test
10,000 pages
$3.75
$0.38/1K pages
- Try it on your real URLs
- Same API as the biggest pack
- Credits never expire
Starter
50,000 pages
$14.25
$0.28/1K pages
- An AI agent or a RAG prototype
- Crawl a documentation site
- Credits never expire
Build
100,000 pages
$21.75
$0.22/1K pages
- Production agents and RAG pipelines
- Batch and crawl jobs
- Credits never expire
In every plan: Main-content extraction · JavaScript rendering when needed · Failed and refused pages not charged · Credits never expire
Use cases: common questions
Can I use one key for several projects?
Yes, or create one key per project so you can see usage separately and revoke one without touching the others. Create several API keys, for example one per agent or project, and revoke any of them from your dashboard.
Which use case needs JavaScript rendering?
Any that reads modern web apps, single-page documentation sites or pages that load content after the first response. With render=auto, the API renders only when the raw HTML holds too little text.
Is there a use case WebReaderAPI is not for?
Yes: collecting personal data about private people, reading content behind a login or a paywall, and anything that requires getting around a site's refusal. Our Acceptable Use Policy lists the rules.
Do I need an ID to start?
No. WebReaderAPI asks for no ID, no selfie and no documents. A top-up starts at $25, in BTC, USDT, USDC, ETH, SOL, LTC.