Evomi

Blog / Scraping Techniques

navigator.webdriver and the Traces a Driven Browser Leaves

The ScraperThe Scraper7 min read
browser web driver puppet


You spent an afternoon on headers. The User-Agent is a current Chrome string, the Accept list matches a real browser byte for byte, Accept-Language and the Sec-Fetch-* set are in the right order. The page still hands your session the slow path: an interstitial or a stripped-down response where a browser gets the real one.

Someone opens the devtools console in your automated window and types one expression.


Shell
> navigator.webdriver
true


That is not a bug in your header code. That is the browser doing exactly what a standard tells it to do.

The Flag Is Spec, Not an Accident

navigator.webdriver comes from the W3C WebDriver specification, section 4, which extends the HTML Navigator interface with a mixin:


TypeScript
interface mixin NavigatorAutomationInformation {
    readonly attribute boolean webdriver;
};
Navigator includes NavigatorAutomationInformation;


The spec defines a webdriver-active flag that "is set to true when the user agent is under remote control. It is initially false," and the attribute "Returns true if webdriver-active flag is set, false otherwise."

The stated purpose is the part people skip, the attribute "Defines a standard way for co-operating user agents to inform the document that it is controlled by WebDriver, for example so that alternate code paths can be triggered during automation."

Blink implements the mixin verbatim in navigator_automation_information.idl, and navigator.idl pulls it in alongside NavigatorLanguage and NavigatorOnLine, the same tier of plumbing as navigator.language.

Where the Value Actually Comes From

The getter in third_party/blink/renderer/core/frame/navigator.cc is six lines:

Rust
bool Navigator::webdriver() const {
  if (RuntimeEnabledFeatures::AutomationControlledEnabled())
    return true;

  bool automation_enabled = false;
  probe::ApplyAutomationOverride(GetExecutionContext(), automation_enabled);
  return automation_enabled;
}


Two inputs. The first is a renderer runtime feature set from the command line, in content/child/runtime_features.cc:

PHP
{wrf::EnableAutomationControlled, switches::kEnableAutomation, true},
{wrf::EnableAutomationControlled, switches::kHeadless, true},
{wrf::EnableAutomationControlled, switches::kRemoteDebuggingPipe, true},

...plus a special case a few lines down:

JavaScript
// Set EnableAutomationControlled if the caller passes
// --remote-debugging-port=0 on the command line. This means
// the caller has requested an ephemeral port which is how ChromeDriver
// launches the browser by default.
// If the caller provides a specific port number, this is
// more likely for attaching a debugger, so we should leave
// EnableAutomationControlled unset [...]


Chromium distinguishes automation from debugging by whether you asked for an ephemeral port. Running that matrix against Google Chrome 152.0.7977.76 on macOS, new headless mode, reading navigator.webdriver off a local page, gives what the source predicts:

flags table


Those are one build on one machine, not a law of nature — the mapping has gained entries over time. The shape is what matters: the flag tracks how you launched, set in the renderer at startup, not decided by the page.

The second input, probe::ApplyAutomationOverride, is the DevTools path: Emulation.setAutomationOverride is a real CDP command, implemented in chrome/browser/devtools/protocol/emulation_handler.cc, where enabling it also raises the automation infobar.

The Switch Is Documented as an Announcement

content/public/common/content_switches.cc describes the flag in one line:

C#
// Enable indication that browser is controlled by automation.
const char kEnableAutomation[] = "enable-automation";


Its most visible effect is the strip across the top of the window, whose string lives in chrome/app/generated_resources.grd as IDS_CONTROLLED_BY_AUTOMATIONChrome is being controlled by automated test software. The delegate registers at InfobarPriority::kCriticalSecurity and returns false from ShouldExpire, so navigation does not clear it.

None of this is an oversight that leaked; it is a browser vendor deciding that "a program is driving this window" is information the user and the page are entitled to.

CDP Changes the Browser, Not Just the Wire

Driving a browser over the DevTools Protocol is not passive observation: enabling a domain changes what the browser does. Runtime.enable starts emitting Runtime.executionContextCreated for every context and turns on console and exception forwarding, so ordinary page activity like a console.debug() call now gets serialized out across the protocol socket. Playwright sends Runtime.enable, then Runtime.addBinding, then installs init scripts through Page.addScriptToEvaluateOnNewDocument in a named isolated world, each a real change to the runtime, and the artifacts below are its edges.

cdp img

The Four Artifact Families

navigator.webdriver is one boolean. The runtime around it leaks in four recurring shapes, each a different question a page can ask.

1. Properties that exist only under automation. ChromeDriver stashes pristine references to built-ins so page code cannot break its own helpers, assigned by a literal script in chrome/test/chromedriver/chrome/devtools_client_impl.cc:

JavaScript
window.cdc_adoQpoasnfa76pfcZLmcfl_Array = window.Array;
window.cdc_adoQpoasnfa76pfcZLmcfl_Object = window.Object;
window.cdc_adoQpoasnfa76pfcZLmcfl_Promise = window.Promise;
window.cdc_adoQpoasnfa76pfcZLmcfl_Proxy = window.Proxy;
window.cdc_adoQpoasnfa76pfcZLmcfl_Symbol = window.Symbol;
window.cdc_adoQpoasnfa76pfcZLmcfl_JSON = window.JSON;
window.cdc_adoQpoasnfa76pfcZLmcfl_Window = window.Window;


call_function.js reads them back with window.cdc_adoQpoasnfa76pfcZLmcfl_Array || window.Array. Playwright's equivalent is its CDP binding, named in source as __playwright__binding__, though newer versions can install bindings without a page-visible global, so this family shifts release to release.

2. toString() shapes. ECMA-262 requires that Function.prototype.toString on a built-in return a string with the syntax of a NativeFunction, in practice, the [native code] placeholder, with a native accessor reporting its own name inside that shape. Measured in Chrome 152:

JavaScript
> Object.getOwnPropertyDescriptor(Navigator.prototype, 'webdriver').get.toString()
"function get webdriver() { [native code] }"


3. Error.stack frames. Injected scripts are real scripts with real source positions, so an exception raised inside one carries frames pointing at the injected source, not at page code. The most famous variant ran the other direction: a page defined a getter on an Error object's stack property, called console.debug(e), and watched the getter fire, because Runtime.enable had turned on Runtime.consoleAPICalled and Chrome serialized the error across the socket. Two V8 commits in May 2025 added a guard so user-defined getters no longer run during error preview, ending that probe. The family is durable; the test was not.

4. Own-property ordering and location. Web IDL puts interface attributes on the prototype object, and OrdinaryOwnPropertyKeys (ECMA-262 §10.1.11.1) returns own string keys in creation order, so on a genuine Chrome navigator itself has no own properties at all:

JavaScript
> Object.getOwnPropertyNames(navigator).length
0


Why Patching Them Individually Makes It Worse

This is the whole point of the post: a page that reads navigator.webdriver === true has learned one true thing about your session. A page that reads false from a getter whose toString() no longer matches the NativeFunction shape has learned that you are automated and that you are lying about it. You have not removed a signal. You have replaced an honest one with a contradiction, and contradictions are cheaper to detect than any individual artifact.

Watch what a single naive override costs across all four families. Defining webdriver on the navigator instance moves the property off Navigator.prototype, so Object.getOwnPropertyNames(navigator).length goes from 0 to 1, family 4 now disagrees with every real Chrome. The replacement getter is an ordinary JavaScript function, so its toString() returns source text instead of [native code], family 2 now disagrees with every other accessor on the object. Wrapping it in a Proxy to fix the toString() changes the descriptor's shape and the stack frames of anything that throws through it — families 3 and 4 again. Each fix adds a surface that has to match, and the surfaces are not independent.

The asymmetry makes this a losing trade. A detector writes one comparison — does this object's shape match the shape the browser vendor ships?, and it keeps working across releases because the vendor maintains the reference. You re-verify one patch per surface, per browser version, per platform, forever. Even at a conservative estimate — five surfaces, a couple of hours each against a fresh Chrome build, six major Chrome releases a year — that is a standing maintenance line whose output is a session more distinctive than an unmodified browser, not less.

Mistakes That Waste Time

  • Chasing artifact lists from blog posts. The console.debug error-getter probe stayed in write-ups for a year after V8 closed it. A surface you cannot measure in your own build is folklore.
  • Assuming headless is the tell. Plain --headless=new left navigator.webdriver at false in the build above. The flag is about the automation switches, not whether a window is drawn.
  • Fixing runtime surfaces when the block came from the network. If your TLS fingerprint or exit IP reputation got you flagged, no navigator surgery moves the outcome, establish which layer refused you first.
  • Copying stealth patches into a test suite. Your own E2E tests may rely on navigator.webdriver to take the automation code path; patching it out breaks your tests to fool a third party.

What a Well-Behaved Client Does Instead

  • Identify yourself. A User-Agent that names your crawler and links a page explaining it, plus a contact address, converts "unknown automated traffic" into "a party we can talk to." Sites throttle the first and whitelist the second.
  • Use the official door. A documented API, bulk export, or licensed feed gives you a stable schema, a rate limit you can plan against, and no adversarial layer at all. Check for one before writing a parser.
  • Ask. For most non-consumer sites, an email describing what you need, at what volume, and why gets an answer, sometimes a key.
  • Respect the boundary when it is a no. robots.txt, published rate limits, and a clear refusal are decisions, not obstacles; engineering around an explicit no is where the legal and reputational exposure lives.

And keep the flag honest where it is supposed to be honest. In testing, navigator.webdriver === true is the correct value — the documented hook for triggering "alternate code paths during automation," used to suppress analytics beacons, skip animations, and stub third-party widgets during E2E runs. Patch it out globally and you lose your own ability to tell your app that a robot is driving.

Here is an audit rather than a patch — it prints what your client exposes:

Python
from playwright.sync_api import sync_playwright

PROBE = """() => {
  const d = Object.getOwnPropertyDescriptor(Navigator.prototype, "webdriver");
  return {
    webdriver: navigator.webdriver,
    own_props_on_navigator: Object.getOwnPropertyNames(navigator),
    webdriver_lives_on_prototype: !!d,
    getter_source: d ? d.get.toString() : null,
    suspicious_globals: Object.getOwnPropertyNames(window).filter(
      (k) => /^(cdc_|__playwright|__puppeteer|__selenium|__webdriver|__driver)/.test(k)
    ),
  };
}"""

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("about:blank")
    for key, value in page.evaluate(PROBE).items():
        print(f"{key:32} {value}")
    browser.close()


On an unmodified Chromium you should see own_props_on_navigator empty, webdriver_lives_on_prototype true, the getter reporting [native code], and no automation globals. Any deviation is something a page can also see.

Wrapping Up

navigator.webdriver is not a leak to plug. It is a spec-mandated announcement, wired in Chromium to the switches you chose at launch, and it exists so pages can cooperate with automation rather than guess.

The two changes that matter: measure your own client's runtime surfaces before you touch any of them, and stop treating each artifact as an independent thing to hide. A driven browser that tells the truth once is less distinctive than one that contradicts itself in four places, and if the honest answer is getting you blocked, the fix is a conversation with the site, not a patched getter.