# What is a web reader API? A URL in, readable Markdown out > What a web reader API does: a URL goes in, the page's main content comes out as clean Markdown for an LLM. How it differs from scraping and search APIs. - URL: https://webreaderapi.com/guides/what-is-a-web-reader-api/ - Last updated: 2026-10-22 - Publisher: WebReaderAPI (https://webreaderapi.com) A web reader API turns a page meant for human eyes into text a model can use. This guide explains what it does step by step, where it sits next to scraping and search APIs, and how the market prices it. **Quick answer — What is a web reader API?** A web reader API is a service that takes a URL, loads the page and returns its main content as clean text, usually Markdown, with metadata such as title, author and date. AI agents and RAG pipelines use it instead of raw HTML. **WebReaderAPI** is one: one credit per page, from $0.07/1K pages as of October 2026, with robots.txt respected, no KYC and payment in crypto. ## A web reader API in one table | Attribute | Value | | --- | --- | | Input | The URL of a public web page or PDF | | Output | Markdown of the main content, or plain text, with title, author, date and language | | Removed | Navigation, cookie banners, ads, sidebars, footers, scripts and styles | | JavaScript pages | Rendered in a headless browser when the raw HTML holds too little text | | Typical billing | Per page or per credit; some providers bill per token of output | | Example request | `GET https://webreaderapi.com/api/v1/read?url=https://example.com/post` | ## What does a web reader API do, step by step? It does five jobs that every page-to-text pipeline needs, in one call: 1. **Check permission.** A careful reader looks at the site's robots.txt for its own name and for all robots before it fetches anything. 2. **Fetch the page.** It downloads the HTML, follows redirects, and stops at a timeout. 3. **Render when needed.** If the HTML is an empty shell filled by JavaScript, it loads the page in a headless browser and reads the result. 4. **Extract the main content.** A readability-style step finds the article or the documentation body and drops the page furniture around it. 5. **Convert and describe.** It writes the content as Markdown and adds metadata: title, author, publication date, language and word count. The caller gets one JSON object with the Markdown inside. A model can read it at once, or a pipeline can split it into chunks for a vector store. ## Why do models need Markdown rather than HTML? Because most of an HTML page is not content. Menus, script tags, tracking code and layout markup use tokens that a model must read and that you must pay for. Markdown keeps the parts that carry meaning: headings, lists, tables, links and emphasis. Cloudflare gave a concrete example when it launched its "Markdown for Agents" feature on 12 February 2026: one page dropped from 16,180 tokens as HTML to 3,150 tokens as Markdown, about 80% fewer. Results vary by page, but the direction is the same on most article and documentation pages. ## How is a web reader API different from a scraping API? A scraping API returns what the server sent; a reader returns what a person would read. Scraping APIs are built to fetch pages at scale, often to pull specific fields such as prices or product names, and they usually hand back raw HTML or structured JSON that you define. A web reader is narrower and more opinionated. It decides what the main content is, removes the rest and writes clean text. That is the right tool when the next consumer is a language model, a search index or a person reading a summary. It is the wrong tool when you need every field of a product grid; for that you want the full HTML or a schema-based extraction. ## How is it different from a search API? A search API answers "which pages talk about this?"; a reader answers "what does this page say?". Search returns a list of URLs with titles and snippets for a query. A reader takes one of those URLs and returns its full text. Agents often chain the two: search to find candidate pages, then read the best three or four to answer with citations. Keeping the steps separate lets you choose how many pages to read, which saves tokens and credits. ## Can I run open-source tools instead of an API? Yes, and many teams start that way. Mozilla's Readability extracts the main content of a page in a browser context; Trafilatura does extraction in Python; Microsoft's MarkItDown converts documents, including PDFs, to Markdown; Crawl4AI combines crawling and Markdown output. The library is the easy part. Running it in production means hosting headless browsers for JavaScript pages, checking robots.txt, handling timeouts, redirects and character sets, converting PDFs, and keeping all of it patched. An API is worth it when you would rather pay per page than run that machinery; a library is worth it when volumes are steady and you have the time to operate it. ## How do web reader APIs charge? Three billing models are common in 2026. Credit-based plans charge a set number of credits per page: Firecrawl's pricing page, read on 22 October 2026, counts 1 credit per scraped page and 4 extra credits per page for its JSON format. Token-based billing charges for the text returned: Jina Reader has priced this way since 6 May 2025 and gives 10 million free tokens to each new key. Monthly plans reset their credits each cycle, while prepaid packs keep them. WebReaderAPI uses prepaid credits that never expire: 1 credit = 1 page read without a browser; 2 credits when the page needs a browser to render JavaScript. A request that fails, or that a site's robots.txt refuses, is not charged. 10,000 pages cost $3.75 and the rate falls to $0.07/1K pages on 5,000,000 pages, as of October 2026. ## What should I check before choosing a web reader API? Test it on twenty of your own URLs and look at six things: - **Extraction quality.** Does the Markdown keep the article and drop the rest, including tables and code blocks? - **JavaScript.** Do pages built in the browser come back with text, and what does rendering cost? - **PDFs.** Are they supported, and how are pages counted? - **Site owners' choices.** Does the service respect robots.txt and declared AI preferences, and say so in its docs? - **Failures.** Are failed and refused pages charged? - **Budget control.** Can you cap the length of each page, and do credits expire? Our [comparison of web reader API prices](https://webreaderapi.com/compare/web-reader-api-pricing/) puts the main providers side by side with dates. ## Web reader API: common questions ### Is a web reader API the same as a browser's reader mode? It follows the same idea, keeping the article and dropping the clutter, but it runs on a server, returns Markdown and metadata as JSON, and works for thousands of URLs. Reader mode is for one person reading one page. ### Can a web reader API read any website? No. It reads public pages that allow it. A careful service skips pages that robots.txt disallows and returns the site's answer when the site refuses. Pages behind a login or a paywall are out of reach by design. ### Does a web reader keep images? Usually not by default, because models mostly need the text. With WebReaderAPI, `include_images=true` keeps images as Markdown image links with their alt text. ### How long does it take to read a page? It depends on the site and on whether the page needs a browser. We do not publish response times until we have measured them on a public sample; plan your timeouts around the `timeout` parameter, which goes up to 60 seconds. ### Do I need an account with an ID to use WebReaderAPI? No. WebReaderAPI asks for no ID, no selfie and no documents. Your login is an account number, and payment is in BTC, USDT, USDC, ETH, SOL, LTC. ## Sources 1. [Introducing Markdown for Agents](https://blog.cloudflare.com/markdown-for-agents/) — Cloudflare (2026-02-12) 2. [Firecrawl pricing](https://www.firecrawl.dev/pricing) — Firecrawl (read 2026-10-22) 3. [Reader API](https://jina.ai/reader/) — Jina AI (Elastic) (read 2026-10-22) --- WebReaderAPI: WebReaderAPI is a web reader API for AI agents that turns any public web page into clean Markdown, paid in crypto with no ID upload. Prepaid credits from $0.07/1K pages never expire, and robots.txt is respected. No KYC — no ID, no selfie, no documents. Pay in crypto.