Read web pages in n8n AI agents with the WebReaderAPI

Why use the HTTP Request node as the tool?

Because it already does everything this API needs: one GET request, one header and two query parameters. n8n documents that the HTTP Request node can run as a regular node or be attached to an AI agent as a tool, and that some options exist only in the tool form.

There is nothing of ours to install or update, and the key sits in n8n's credential store rather than in the workflow. Give n8n its own key, so its usage is visible apart from your other scripts and you can revoke it alone. Create several API keys, for example one per agent or project, and revoke any of them from your dashboard.

How does the model choose the URL to read?

Let the model define the url parameter. n8n shows a button next to eligible parameter fields that hands the value to the model, and you can also write the expression yourself with n8n's $fromAI() function. It takes a key, a description, a type and an optional default, for example the key url, the description "The http or https address of the page to read" and the type string.

n8n documents that $fromAI() works only in tools connected to the AI Agent node. Write the tool description in plain words too, such as "Reads a public web page and returns its main content as Markdown", so the agent knows when to call it.

Why turn on Optimize Response?

Because everything a tool returns goes into the model's context. Optimize Response is a tool-only option that reduces the data passed to the model. With the JSON response type you can include only selected fields; keep title, final_url and markdown and drop the rest.

Cap the length on our side with max_chars, which cuts the Markdown before it leaves the API. The HTML and Text response types of the node have their own Truncate Response option, but with our JSON answer max_chars is the simpler control.

How do I use the API in a scheduled workflow?

Use the HTTP Request node as a regular node, not a tool, behind a Schedule Trigger. Give it a fixed url, such as a changelog or a pricing page you follow, and send the markdown field to the next node: a file, a database, a chat message or a comparison with the previous run.

Each run reads one page per URL. Watching 20 pages once a day uses about 600 credits a month for pages without a browser, so 10,000 pages ($3.75) lasts more than a year. Prepaid credits never expire: unused pages stay on your account until you use them.

What happens when a site refuses the read?

The node receives an error response with a code such as blocked_by_robots or target_refused, and nothing is billed. We respect robots.txt: our crawler identifies itself as WebReaderAPI, follows the rules written for it or for all robots, and does not read a page that a site disallows. Refused pages are not charged. In an agent, let the tool return the error so the model can explain it or try another source; in a scheduled workflow, route errors to a branch that logs them. The errors page lists every code.

Can I try the call before building the workflow?

Run the same call in a terminal with curl, as shown in the curl recipes. If it works there and fails in n8n, the cause is in the node settings, most often the credential value missing the Bearer prefix or the url parameter left empty.