Evomi

Blog

Tips on residential proxies, web scraping & security

pay per request

Proxy Fundamentals

Pay Per Crawl and the Licensed Web: When Access Becomes a Transaction

A 402 Payment Required in your crawl logs is a status code most engineers have never seen in production. Cloudflare wired it up in 2025 to let publishers charge crawlers per request, and the mechanics say a lot about where web access is heading. This post explains the signalling flow, the standards work behind machine-payable HTTP, the arguments against it, and how the cost of licensed access compares to running a crawl.

August 26, 2026 · 8 min read