# Web page to Markdown with curl and jq

> Turn web pages into Markdown from the terminal with curl and jq: save a page, list its links, run a batch and check credits. From $0.07/1K pages, no KYC.

- URL: https://webreaderapi.com/docs/curl/
- Last updated: 2026-10-22
- Publisher: WebReaderAPI (https://webreaderapi.com)

Everything the API does fits in a shell one-liner. These recipes use curl to call the API and jq to pick fields out of the JSON, with a PowerShell version for Windows at the end.

**Quick answer — How do I convert a web page to Markdown from the command line?**

Run `curl -sG https://webreaderapi.com/api/v1/read --data-urlencode "url=…"` with your key in an `Authorization: Bearer` header, then pipe the answer to `jq -r .markdown`. **WebReaderAPI** returns the page's main content as Markdown with its metadata. One plain page uses 1 credit, from $0.07/1K pages as of October 2026. No KYC; pay in crypto.

## How do I set up the terminal?

Install curl and jq, which most Linux and macOS systems already have, and export your key once per session. Every recipe below reads it from `$WEBREADER_API_KEY`, so the key never appears in your shell history as part of a command.

Use `-G` with `--data-urlencode` for GET requests: curl then encodes the page address for you, which matters because addresses contain `?`, `&` and `=` of their own.

*Set the key and save one page as a .md file*

```bash
export WEBREADER_API_KEY="your-key"

curl -sG "https://webreaderapi.com/api/v1/read" \
  -H "Authorization: Bearer $WEBREADER_API_KEY" \
  --data-urlencode "url=https://example.com/guides/compost" \
  | jq -r '.markdown' > compost.md
```

## How do I see what a call returned and cost?

Ask jq for the metadata instead of the content. `final_url` shows where redirects led, `rendered` tells you whether a browser was used, and `credits_used` is what the call cost. If the answer is an error, the same filter prints `null` fields; print `.error` to read the code and message.

*Metadata only, or the error if there is one*

```bash
curl -sG "https://webreaderapi.com/api/v1/read" \
  -H "Authorization: Bearer $WEBREADER_API_KEY" \
  --data-urlencode "url=https://example.com/guides/compost" \
  | jq '.error // {title, final_url, rendered, credits_used, word_count}'
```

## How do I list a page's links?

Add `include_links=true` and turn the `links` list into tab-separated lines. That gives you a quick map of a page's outgoing links, which you can sort, filter with grep or feed to a batch job.

*Links as text and URL, one per line*

```bash
curl -sG "https://webreaderapi.com/api/v1/read" \
  -H "Authorization: Bearer $WEBREADER_API_KEY" \
  --data-urlencode "url=https://example.com/docs/" \
  -d include_links=true \
  | jq -r '.links[] | "\(.text)\t\(.url)"' | sort -u
```

## How do I read a JavaScript page or a PDF?

Add `-d render=true` for a page that builds its content with JavaScript; it then costs 2 credits. A PDF needs nothing special: send its URL and the text comes back in `markdown`. PDF files up to 20 MB and 100 pages are converted to Markdown, at 1 credit per started block of 10 pages.

*A rendered page, then a PDF*

```bash
curl -sG "https://webreaderapi.com/api/v1/read" -H "Authorization: Bearer $WEBREADER_API_KEY" \
  --data-urlencode "url=https://example.com/app/changelog" -d render=true \
  | jq -r '.markdown' > changelog.md

curl -sG "https://webreaderapi.com/api/v1/read" -H "Authorization: Bearer $WEBREADER_API_KEY" \
  --data-urlencode "url=https://example.com/files/annual-report.pdf" \
  | jq -r '.markdown' > annual-report.md
```

## How do I run a batch job from the shell?

Post a JSON body with your URLs to `/batch`, keep the `job_id`, then read the job. A small loop waits until the job is `done` and prints each page's title or error code. For long jobs, follow `next_cursor` as shown on the [batch and crawl page](https://webreaderapi.com/docs/crawl/).

*Batch of three URLs, then wait for the result*

```bash
JOB=$(curl -s "https://webreaderapi.com/api/v1/batch" \
  -H "Authorization: Bearer $WEBREADER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.com/a", "https://example.com/b", "https://example.com/c"]}' \
  | jq -r '.job_id')

until [ "$(curl -s "https://webreaderapi.com/api/v1/jobs/$JOB" -H "Authorization: Bearer $WEBREADER_API_KEY" | jq -r '.status')" = "done" ]; do
  sleep 5
done

curl -s "https://webreaderapi.com/api/v1/jobs/$JOB" -H "Authorization: Bearer $WEBREADER_API_KEY" \
  | jq -r '.results[] | "\(.url)\t\(.title // .error.code)"'
```

## How do I check my credits?

Call `/credits`; it is free. It returns `credits_remaining`, `credits_used_today` and your `concurrency_limit`. Prepaid credits never expire: unused pages stay on your account until you use them.

*Remaining credits*

```bash
curl -s "https://webreaderapi.com/api/v1/credits" -H "Authorization: Bearer $WEBREADER_API_KEY" | jq '.credits_remaining'
```

## What is the PowerShell version on Windows?

`Invoke-RestMethod` parses the JSON for you, so no jq is needed. Build the query with `[uri]::EscapeDataString` to encode the page address, and pass the key in a headers table.

*PowerShell: save one page*

```powershell
$headers = @{ Authorization = "Bearer $env:WEBREADER_API_KEY" }
$target = [uri]::EscapeDataString('https://example.com/guides/compost')
$page = Invoke-RestMethod -Uri "https://webreaderapi.com/api/v1/read?url=$target" -Headers $headers
$page.markdown | Out-File -Encoding utf8 compost.md
"$($page.title): $($page.word_count) words, $($page.credits_used) credit(s)"
```

## curl: common questions

### Why does my URL with & in it fail?

Because the shell or curl split it. Quote the whole `url=…` argument and pass it with `--data-urlencode`, as in the recipes above; curl then encodes `&`, `?` and `=` inside the page address.

### How do I see the HTTP status of an error?

Add `-w '\n%{http_code}\n'` to print it after the body, or `-i` to see all headers. The body's `error.code` is the stable value to test in scripts.

### Is a failed call billed?

No. 1 credit = 1 page read without a browser; 2 credits when the page needs a browser to render JavaScript. A request that fails, or that a site's robots.txt refuses, is not charged.

---

WebReaderAPI: WebReaderAPI is a web reader API for AI agents that turns any public web page into clean Markdown, paid in crypto with no ID upload. Prepaid credits from $0.07/1K pages never expire, and robots.txt is respected. No KYC — no ID, no selfie, no documents. Pay in crypto.