n8n Web Scraping: Which Node, Which Mode, and What It Costs

Every n8n scraping tutorial starts with the same two nodes: HTTP Request, then HTML. That is the right place to start and it is genuinely all you need for a large share of the web. It is also why the second week of a scraping project feels so different from the first — you build something that works on a documentation site, point it at a real target, and get back a page with no content in it.

The ladder from there has three rungs, and there is one question that tells you which rung you are on. Not "is this site hard", which is unanswerable, but: when I fetch this URL with no JavaScript, is the data in the response?

Everything below follows from that.

Rung One: HTTP Request Plus the HTML Node

Add an HTTP Request node with the URL and nothing else. Run it. Look at the response.

If the text you want is in there, you are done, and everything past this section is overspending. Add an HTML node set to Extract HTML Content, with Source Data set to JSON and the JSON Property set to data, then define your extraction values:

Turn Return Array on where a selector matches many elements and you want all of them.

Two adjustments make this rung work considerably more often than the default does.

Send a real User-Agent. n8n's default identifies itself as n8n, and a surprising number of the "this site blocks scraping" reports come from sites that are simply answering an unfamiliar client differently. Under Send Headers, add a User-Agent that matches a current browser. It costs nothing and it is honest — you are a program either way, and the header is not the thing that establishes that.

Read the response, do not just check the status. A 200 that returns a challenge page is the normal failure here, and a workflow that branches on status code will happily process it as data for a month. An IF node checking that the extracted title is non-empty catches more real failures than any status check.

Rung Two: The Same Request, From a Different Address

If the response is a challenge page, a login wall, region-specific content you did not want, or empty after a while when it used to work, the problem is not the parsing. It is that every request from your n8n instance leaves from one IP address in a datacenter range, and the site has noticed.

This is one field — Options → Proxy on the same HTTP Request node — and it is covered in full, including country targeting, sticky sessions and mid-workflow rotation, in using proxies in the n8n HTTP Request node.

Rung two is worth trying before rung three whenever the page is static, because it is much cheaper. You are still paying for one HTTP fetch and some bandwidth, not for a browser.

Rung Three: A Node That Renders

You are on rung three when the response genuinely does not contain the data — a React or Vue application that renders client-side, an infinite-scroll list, a page whose prices arrive from a separate XHR call after load.

You have three options and they are not equivalent:

Read the underlying data instead of the rendered page. Before reaching for a browser, check whether the framework left the data in the document. Next.js applications embed it in __NEXT_DATA__, and a great many sites publish JSON-LD in a <script> tag for search engines. Both are cleaner, cheaper and more stable than any selector, and parsing them properly is often a twenty-minute job that removes the need for the rest of this section.

Run a browser yourself. Community Puppeteer and Playwright nodes exist. You get full control, and you take on Chromium in your n8n container, memory that scales with concurrency, and the whole fingerprinting problem as your own.

Call a scraping API from a node. Someone else runs the browser; you get the result. On a hosted n8n this is the only realistic version of rung three anyway, since you cannot install Chromium on it.

Configuring the Evomi Node

Evomi's node is verified by n8n, so it installs from inside the editor rather than from npm: open the nodes panel, search "Evomi", and look under More from the community. An instance owner or admin has to do the install once; everyone can use it afterwards.

The credential is an API key from Settings → API in the dashboard, plus a base URL that defaults to https://scrape.evomi.com and should be left alone. n8n verifies it against a health endpoint when you save, so a green tick means the key works.

Then there are exactly three decisions.

Mode

Leave it on Auto. The upgrade is decided per page, so a mixed list of URLs pays browser price only for the pages that earned it, and every JSON response carries a mode_used field telling you afterwards which path that call took — auto (request) or auto (browser). That field is the cheapest cost-monitoring you will ever set up: if everything reads auto (browser) you are scraping a set of pages that all need rendering, and it is worth knowing that before the invoice says so. (The API also returns it as an X-Mode-Used header, but the node hands you the response body and not the headers, so in n8n the body is where to look.)

Choose Request explicitly when you know the target is static and you want to guarantee you never pay for a browser. Choose Browser explicitly when Auto keeps guessing wrong.

Output

The Include Content toggle is the one people miss. JSON output omits the page body by default to save bandwidth, so a workflow that asks for JSON and then finds no content has not failed — it has been given exactly what it asked for.

Markdown is the right output when the next node is a model. Feeding raw HTML into an LLM spends most of the context window on div soup, and the model does worse work on it than it does on markdown. If the next node is a spreadsheet or a database, use JSON with Include Content on and parse it with the HTML node.

Additional Fields

Two options, and the node hides them under Add Field:

• Proxy Country — a two-letter ISO code. Uppercased and validated for you, so de works and Germany returns a clear error rather than silently doing something else.

• Wait Seconds — 0 to 30, applied after page load before capture, and only sent for Auto and Browser modes. The default of 5 covers most late-loading pages. Raise it for dashboards, lower it to 0 or 1 for static pages you are running Browser mode on for another reason, because every second here is wall-clock time in your workflow.

That is the entire node. One resource, one operation, five fields. It does one thing, and knowing that up front saves you looking for the crawl feature.

What It Costs

The Scraper API starts at $0.13 per 1,000 results, and each call consumes credits according to what it had to do:

One constraint follows from that table and it catches people out:

Datacenter proxies only work in Request mode. Browser and Auto both require residential, and asking for either over datacenter returns an error rather than falling back. So the moment a page needs JavaScript, you are on residential whether or not that was the plan — and note that Auto requires residential even for the requests it resolves without a browser, which is why the cheapest row in the table is the one you have to choose deliberately.

If you want the model or the workflow to see its own spend, use JSON output — credits_used and credits_remaining come back on every response. Wiring credits_remaining into an IF node with a floor, and a Slack message below it, is ten minutes of work that will eventually be the reason a runaway loop cost you five dollars instead of fifty.

Errors, and Not Silently Collecting Nothing

The Evomi node rewrites four API failures into readable messages — invalid key, insufficient credits, rate limit, unfetchable URL — so the Executions tab tells you what happened rather than showing a raw 4xx.

For batch work, turn on Continue On Fail in node settings. The node then emits { success: false, error, itemIndex } for the items that failed and passes the rest through, which means one dead URL in a list of five hundred does not end the run. Route the failures to a separate branch and retry them at the end.

The failure worth building for explicitly is the quiet one. A page that returns 200 with an empty body, or with a challenge page in it, will flow down your workflow as data. Two guards cost nothing:

• An IF node asserting the field you care about is non-empty before anything writes to a database.

• A run summary — items in, items with content, items failed — posted somewhere you will see it. A scrape that returns 500 rows one week and 40 the next is broken, and no individual execution will have told you.

Feeding It Into an AI Agent

The Evomi node declares itself usable as a tool, which means it can be attached directly to an AI Agent node as a tool sub-node. The agent supplies the URL; you fix the mode and output. Markdown output plus the description "Fetch a web page and return its readable content" is the whole configuration.

If the model lives outside n8n rather than on the canvas — Claude Desktop, Cursor, a coding agent — the same node goes behind an MCP Server Trigger instead, and your API key stays in the instance. And if the model needs to manage the connection rather than just fetch through it, that is a provider's own MCP server, which is a different surface with different tools. Those three arrangements get called "n8n MCP" interchangeably and they are not the same thing — which one you want depends on where the model sits.

What People Get Wrong

• Starting at rung three. Plenty of targets never needed a browser. Fetch the URL first and look at the response before deciding anything.

• Checking the status code instead of the content. A challenge page is a 200.

• Leaving Include Content off and concluding the API is broken. JSON output omits the body on purpose.

• Sending HTML to a language model. Markdown output exists precisely for this and produces better answers on fewer tokens.

• Forcing Browser mode everywhere. Auto already upgrades when it needs to; forcing it means paying browser price on pages that would have worked over plain HTTP.

• Expecting datacenter proxies to render JavaScript. Browser and Auto are residential only, and the API returns an error rather than downgrading for you.

• Scraping at 3am on a schedule and never looking again. Sites change. The quiet failure is the expensive one.

• Ignoring robots.txt because the node does not check it. It does not, and that is still your call to make.

Wrapping Up

The whole decision in n8n scraping is which rung you need, and the way to find out is to fetch the URL and read what comes back rather than to reason about it. Most projects end up mixed — half the targets are fine over plain HTTP with a sensible User-Agent, a quarter need a different exit IP, and the rest need something to render the page. Building all three straight away is how a two-day project becomes a two-week one.

What is worth building on day one, regardless of rung, is the check that tells you the data is still arriving. Scrapers do not usually break loudly. They start returning fewer rows, and then none, and the workflow keeps reporting success the entire time.