# Web page to Markdown in Node.js

> Convert web pages to Markdown in Node.js with native fetch: one page, a small concurrency pool and a crawl polling loop. From $0.07/1K pages, no KYC, crypto.

- URL: https://webreaderapi.com/docs/node/
- Last updated: 2026-10-22
- Publisher: WebReaderAPI (https://webreaderapi.com)

Node.js 18 and later ship `fetch`, so you need no dependency at all. Three recipes: read one page, read a list with a small concurrency pool, and follow a crawl job until it is done.

**Quick answer — How do I convert a web page to Markdown in Node.js?**

Use the built-in `fetch` of Node.js 18+: call `https://webreaderapi.com/api/v1/read?url=…` with your key in the `Authorization` header and read `markdown` from the JSON answer. **WebReaderAPI** returns the page's main content with its title, author and date. One page costs 1 credit, 2 with a browser, from $0.07/1K pages as of October 2026. No KYC; pay in crypto.

## What do I need?

Node.js 18 or later, for the global `fetch`, and an API key in the `WEBREADER_API_KEY` environment variable. The examples are ES modules: save them with the `.mjs` extension, or set `"type": "module"` in your `package.json`, so that top-level `await` works.

The JSDoc comments describe the response shape, so editors such as VS Code give you completion without a TypeScript build step.

*read.mjs: one page*

```javascript
/**
 * @typedef {Object} Page
 * @property {string} url
 * @property {string} final_url
 * @property {number} status_code
 * @property {boolean} rendered
 * @property {number} credits_used
 * @property {string} [title]
 * @property {string} [author]
 * @property {string} [published_at]
 * @property {string} [language]
 * @property {string} [markdown]
 * @property {number} word_count
 */

/** @returns {Promise<Page>} */
export async function readPage(url, options = {}) {
  const params = new URLSearchParams({ url, ...options });
  const res = await fetch(`https://webreaderapi.com/api/v1/read?${params}`, {
    headers: { Authorization: `Bearer ${process.env.WEBREADER_API_KEY}` },
    signal: AbortSignal.timeout(75_000),
  });
  const body = await res.json();
  if (!res.ok) throw Object.assign(new Error(body.error.message), { code: body.error.code });
  return body;
}

const page = await readPage('https://example.com/docs/getting-started', { max_chars: '20000' });
console.log(page.title, page.word_count, 'words,', page.credits_used, 'credit(s)');

```

## How do I read a list of URLs without hitting the limit?

Run a fixed number of workers that take URLs from a shared queue. It is simpler than `Promise.all` over the whole list, which would start every request at once and hit `429 concurrency_limit`. Every account has a limit on how many pages it reads at once; your dashboard shows it, and support can raise it.

The pool below keeps errors next to the results, so one refused page does not stop the run. Refusals such as `blocked_by_robots` are final and free; count them and move on.

*pool.mjs: a list of URLs, 8 at a time*

```javascript
import { readPage } from './read.mjs';

export async function readAll(urls, workers = 8) {
  const queue = [...urls];
  const results = [];
  async function worker() {
    while (queue.length) {
      const url = queue.shift();
      try {
        const page = await readPage(url, { max_chars: '50000' });
        results.push({ url, page });
      } catch (err) {
        results.push({ url, error: err.code ?? err.name });
      }
    }
  }
  await Promise.all(Array.from({ length: workers }, worker));
  return results;
}

const urls = Array.from({ length: 30 }, (_, i) => `https://example.com/blog/post-${i + 1}`);
for (const r of await readAll(urls)) console.log(r.url, '->', r.error ?? r.page.title);

```

## How do I follow a crawl job in Node.js?

Start the job with `POST https://webreaderapi.com/api/v1/crawl`, then read `https://webreaderapi.com/api/v1/jobs/{job_id}` in a loop. Pass `cursor` each time `next_cursor` comes back; when there is no cursor and the status is `done`, the job is finished. Wait between empty polls and double the wait up to 30 seconds, since a crawl that respects a site's Crawl-delay can be slow.

The [batch and crawl page](https://webreaderapi.com/docs/crawl/) describes how the crawl chooses its pages and how `limit` caps what it reads.

*crawl.mjs: start a crawl and collect every page*

```javascript
const headers = {
  Authorization: `Bearer ${process.env.WEBREADER_API_KEY}`,
  'Content-Type': 'application/json',
};
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

const start = await fetch('https://webreaderapi.com/api/v1/crawl', {
  method: 'POST',
  headers,
  body: JSON.stringify({ url: 'https://docs.example.com/', limit: 300, include_paths: ['/docs/*'] }),
});
const { job_id } = await start.json();

const pages = [];
let cursor = '';
let wait = 2000;
for (;;) {
  const url = `https://webreaderapi.com/api/v1/jobs/${job_id}${cursor ? `?cursor=${encodeURIComponent(cursor)}` : ''}`;
  const job = await (await fetch(url, { headers })).json();
  pages.push(...job.results.filter((r) => !r.error));
  if (job.next_cursor) {
    cursor = job.next_cursor;
    wait = 2000;
    continue;
  }
  if (job.status === 'done') break;
  await sleep(wait);
  wait = Math.min(wait * 2, 30000);
}
console.log(pages.length, 'pages read, job', job_id);

```

## How do I pass the result to a model?

Send the Markdown, not the whole JSON. The `markdown` field is what a model should read; the metadata is better kept for citations. Prefix it with the title and `final_url`, so the model can name its source, and keep `max_chars` at a size your model's context can hold, such as 20,000 characters per page.

The [OpenAI function-calling page](https://webreaderapi.com/docs/openai/) shows the full tool loop in Python; the same `readPage` function above can serve as the tool body in a Node.js agent.

## Node.js: common questions

### Does this work in Deno, Bun or the browser?

Deno and Bun support the same `fetch` and ES modules. Do not call the API from a browser page: the key would be visible to every visitor. Call it from your server instead.

### Is there an npm package?

No. The functions on this page are the whole integration. WebReaderAPI is plain HTTPS and JSON, so you keep control of retries, timeouts and logging.

### How do I handle 429 concurrency_limit?

Lower the number of workers, or retry after a short wait that doubles each time. A 429 costs nothing, and the [errors page](https://webreaderapi.com/docs/errors/) has a ready backoff function.

### What does reading 10,000 pages cost?

About 10,000 credits for plain pages, more if some need a browser. 10,000 pages cost $3.75; Prepaid credits never expire: unused pages stay on your account until you use them.

---

WebReaderAPI: WebReaderAPI is a web reader API for AI agents that turns any public web page into clean Markdown, paid in crypto with no ID upload. Prepaid credits from $0.07/1K pages never expire, and robots.txt is respected. No KYC — no ID, no selfie, no documents. Pay in crypto.