Crawl4AI or a hosted API? When to run your own web reader

What does self-hosting a web reader really cost?

The software is free; running it is not. Before you compare a hosted price with zero, count these:

• Servers and browsers. Static pages need little CPU, but JavaScript pages need a headless browser per request, with memory to match. Peaks need spare capacity that sits idle the rest of the time.
• Maintenance. Browser versions, extraction rules and dependencies change. Crawl4AI shipped v0.9.5 on 14 October 2026; MarkItDown 0.1.9 on 21 October 2026. Someone has to test and deploy each update.
• Network. Outbound bandwidth, DNS, timeouts and retries are yours to tune. Sites can refuse traffic from your servers' addresses, and the right answer then is to respect the refusal.
• Policy. Crawl4AI leaves check_robots_txt off unless you turn it on, and lets you set or randomise the user agent. Deciding and enforcing your own robots.txt and identification rules is part of the job.
• Security. MarkItDown's README warns that it performs I/O with the privileges of the current process and that inputs should be sanitised in untrusted environments. A service that fetches arbitrary URLs must also refuse private and local addresses.

What do hosted readers charge in comparison?

Hosted APIs turn those costs into a price per page. Two examples from 22 October 2026:

Crawl4AI Cloud is run by the Crawl4AI maintainers and marks its prices as a "soft launch" that may change. Its credit is worth $0.001, so a static page costs $0.50 per 1,000 on Starter.

When should I self-host?

Self-hosting pays off when three things are true: your volume is high and steady, you already run servers and browsers, and you need control that an API cannot give.

• Custom extraction. Crawl4AI's CSS, XPath and LLM schemas, or your own rules on top of Readability or Trafilatura.
• Data residency. Pages never leave your infrastructure.
• Inputs beyond web pages. MarkItDown converts Word, Excel, PowerPoint, EPub and more, which a web reader API does not.
• Licence fit. Apache-2.0 and MIT tools allow closed products; Firecrawl's AGPL-3.0 core asks you to share changes if you offer it as a network service.

When is a hosted API the better choice?

Buy pages instead of running servers when your workload is uneven, your team is small, or the reader is a tool inside an agent rather than a product of its own.

• Spiky or seasonal volume. A crawl twice a year does not justify a browser fleet. With WebReaderAPI, Prepaid credits never expire: unused pages stay on your account until you use them.
• Agents. A hosted endpoint plugs into MCP, LangChain or OpenAI tools in minutes; see our MCP server.
• A policy you do not have to build. We respect robots.txt: our crawler identifies itself as WebReaderAPI, follows the rules written for it or for all robots, and does not read a page that a site disallows. Refused pages are not charged. If a site declares Content-Signal: ai-input=no in its robots.txt, we do not read its pages for you.
• Payment without a card. WebReaderAPI takes BTC, USDT, USDC, ETH, SOL, LTC with no KYC.

Can I combine both?

Yes, and many teams do. Run an open-source converter for files you already hold, such as PDFs and Office documents with MarkItDown, and call a hosted reader for live web pages, especially JavaScript pages that would otherwise need your own browsers.

Keep the output format the same on both sides, Markdown with a source URL and a title, so your chunking and indexing code does not care where a document came from. Our RAG guide shows that format in practice.