Non-Intrusive Web Recon: Techniques from Chrome DevTools Recorder

Posted on December 25, 2024  •  7 minutes  • 1475 words

Introduction: The Art of Non-Intrusive Web Recon

Hello, I’m pizzacat83 (@pizzacat83 ), a software engineer at Flatt Security Inc.

When hunting for bugs, understanding the behavior of a target application is invaluable. The more knowledge you gain about the application—where each functionality resides, how pages and APIs interconnect—the greater your ability to pinpoint weaknesses and unravel vulnerabilities.

Imagine a tool that could seamlessly assist with this process, extracting key insights from the browser in real-time as you navigate through the application. A tool that, while you click through the interface, captures the underlying DOM structure, traces interactions, and reveals valuable data connections—all without disrupting your flow. This would make it possible to map the application’s behavior as you go, without needing to slow down to document every detail manually.

However, building such a tool presents a unique technical challenge. Any tool that gathers data by interfacing with the DOM APIs, for example, risks interfering with the very application it’s meant to observe. What if the tool disrupts the app’s behavior, altering its normal functionality? Conversely, if the application monkey-patches the JavaScript environment, the modifications may obstruct the tool’s ability to gather data correctly. This is where the concept of non-intrusive web reconnaissance becomes essential.

For a tool to support bug hunters and security researchers investigating various web applications, it should not hinder them from observing the application’s natural behavior. Likewise, the tool must be resilient to the application’s modifications to the JavaScript environment.

In this article, we’ll take a closer look at Chrome DevTools Recorder, a tool designed not for security, but for recording user interactions such as clicks, inputs, and other actions within a web page. While it may seem unexpected, this developer-focused tool offers remarkable insights into non-intrusive web reconnaissance. By isolating the JavaScript environments, Chrome DevTools Recorder is able to capture user interactions without altering the application’s natural behavior, no matter how the application has modified its JavaScript environment. Through examining its approach, we uncover methods that could be invaluable in building non-intrusive security tools.

Understanding Chrome DevTools Recorder: Observing Without Interference

When it comes to non-intrusive reconnaissance, there’s much to learn from exploring the internals of Chrome DevTools. Globally used by developers to debug, analyze, and optimize applications, DevTools have been meticulously battle-tested across diverse web environments. Over years of evolution, Chrome DevTools have been carefully refined to operate seamlessly across different applications, avoiding interference with the page’s normal behavior. Thanks to this non-intrusiveness, we can inspect web pages using DevTools without disrupting its functionality, no matter what the application is.

In this article, we’ll focus on one feature within DevTools: the Recorder . The Recorder is a tool designed to capture user interactions—such as clicks, key presses, and other input events—allowing developers to record user workflows directly within the browser. Such recordings can be exported as scripts for automation tools like Puppeteer, enabling automated browsing and end-to-end (E2E) testing. To see the Recorder in action, check out the official demo video below for a closer look at its functionality.

To better understand the Recorder’s architecture, refer to the diagram below. At its core, the Recorder functions by registering event listeners—such as click, keydown, and input to detect user gestures. These event listeners then relay the captured event data back to the Recorder.

The source code for these injected scripts is located in the front_end/panels/recorder/injected directory of the ChromeDevTools/devtools-frontend repository. The core logic for listening to events and forwarding them to the Recorder is managed by the RecordingClient class, defined as follows:

class RecordingClient {
  // ... snip ...
  
  start = (): void => {
    this.#logger.log('Setting up recording listeners');

    window.addEventListener('keydown', this.#onKeyDown, true);
    window.addEventListener('beforeinput', this.#onBeforeInput, true);
    window.addEventListener('input', this.#onInput, true);
    window.addEventListener('keyup', this.#onKeyUp, true);

    window.addEventListener('pointerdown', this.#onPointerDown, true);
    window.addEventListener('click', this.#onClick, true);
    window.addEventListener('auxclick', this.#onClick, true);

    window.addEventListener('beforeunload', this.#onBeforeUnload, true);
  };
  
  // ... snip ...
  
  #onClick = (event: MouseEvent): void => {
    // ... snip ...

    // transmit the event to the Recorder
    this.#addStep({
      type: event.detail === 2 ? 'doubleClick' as StepType.DoubleClick : 'click' as StepType.Click,
      selectors: this.#initialPointerTarget.selectors,
      duration: duration > 350 ? duration : undefined,
      ...attributes,
    });
  };

Source: ChromeDevTools/devtools-frontend/front_end/panels/recorder/injected/RecordingClient.ts

This script captures user gestures by registering event listeners with window.addEventListener. The click event handler #onClick relays the event to the #addStep method, which transfers the event from the page to the Recorder.

Seemingly, this setup allows observation of user interactions. But what if the application overwrites window.addEventListener? Surprisingly, this does not affect the Recorder, thanks to the isolation described in the following section.

The Power of Isolation: Separate Worlds for Recorder and App

The key to avoiding interference lies in how the scripts are injected into the application. The scripts are injected by the method RecordingSession.#injectApplicationScript, called when the recording starts, defined as follows:

  async #injectApplicationScript(target: SDK.Target.Target): Promise<void> {
    const injectedScript = await Util.InjectedScript.get();
    const script = `
      ${injectedScript};DevToolsRecorder.startRecording(/* snip */);
    `;
    const [{identifier}] = await Promise.all([
      // evaluate the script when new frames are created
      target.pageAgent().invoke_addScriptToEvaluateOnNewDocument(
          {source: script, worldName: Util.DEVTOOLS_RECORDER_WORLD_NAME, includeCommandLineAPI: true}),
      // evaluate the script in existing frames
      evaluateInAllFrames(Util.DEVTOOLS_RECORDER_WORLD_NAME, target, script),
    ]);
    this.#scriptIdentifiers.set(target.id(), identifier);
  }

Source: ChromeDevTools/devtools-frontend/front_end/panels/recorder/models/RecordingSession.ts

The helper function evaluateInAllFrames, used for existing frames, is implemented as follows:

export async function evaluateInAllFrames(
    worldName: string,
    target: SDK.Target.Target,
    expression: string,
    ): Promise<void> {  
  // ... snip ...
  
  for (const frame of resourceTreeModel.frames()) {
    // ... snip ...

    const {executionContextId} = await target.pageAgent().invoke_createIsolatedWorld({frameId: frame.id, worldName});
    await target.runtimeAgent().invoke_evaluate({
      expression,
      includeCommandLineAPI: true,
      contextId: executionContextId,
    });
  }
}

Source: ChromeDevTools/devtools-frontend/front_end/panels/recorder/models/SDKUtils.ts

In short, the script is injected via the following Chrome DevTools Protocol (CDP) methods:

• Page.addScriptToEvaluateOnNewDocument (docs ) to run the script in newly created frames
• Runtime.evaluate (docs ): to execute the script in frames that already exist

The isolation mechanism hinges on the argument Util.DEVTOOLS_RECORDER_WORLD_NAME. This value is used for the parameter worldName of the CDP methods Page.addScriptToEvaluateOnNewDocument and Page.createIsolatedWorld. But what is a “world”?

The concept of “worlds” is explained in the Chromium document “Design of V8 bindings .” Different worlds have separate contexts, and thus do not share the global scopes or prototypes. While underlying C++ DOM objects are shared between worlds, in the JS realm, each world has its own set of DOM wrappers—the interfaces that allow access to C++ DOM objects from JavaScript.

The Recorder’s scripts are evaluated within a dedicated world, specified by the name Util.DEVTOOLS_RECORDER_WORLD_NAME. As a result, the Recorder’s scripts and the application’s scripts do not directly interfere with each other. Even if the application’s scripts overwrite global variables or prototypes, those of the Recorder’s world would remain unaffected. Additionally, we can be confident that the Recorder scripts won’t disrupt the application’s behavior, as long as the Recorder scripts only read from the DOM and do not write to. The isolation between worlds prevents unwanted interference between the Recorder and the application.

Applying Lessons from Chrome DevTools Recorder

At Flatt Security, we offer expert security assessments and penetration testing for software. Alongside these services, we also develop a range of tools to empower and streamline these processes, many of which are integrated into Shisho Cloud, the cloud AppSec platform we provide.

Among such tools is a browser-side sidecar that extracts insightful runtime information of the web application from the running browser. Additionally, we develop a crawler that automates the exploration of applications as well as the extraction of insights. Lessons learned from Chrome DevTools, including the techniques discussed in this article, have significantly influenced the design of these tools, enabling us to build robust and reliable solutions.

Conclusion

Designing a tool to extract runtime information from web applications comes with two critical challenges: ensuring that the tool does not interfere with the application’s normal behavior and enabling the tool to function reliably across a wide variety of applications. These principles are essential for building tools that can support security researchers in uncovering vulnerabilities in the background of their inspection process, without hindering the normal behavior of the application.

From Chrome DevTools Recorder, we’ve seen how the isolation between the application and the Recorder ensures the integrity of both the observer and the observed. By leveraging isolated worlds, Recorder effectively isolates its scripts from the application’s environment, allowing it to capture interactions without being affected by or affecting the application itself.

Exploring the internals of Chrome DevTools provides a wealth of insights into non-intrusive web reconnaissance. DevTools, a widely adopted and battle-tested tool, has evolved through extensive real-world use to function seamlessly across diverse web applications without disrupting their behavior. By studying its techniques, we gain valuable guidance for designing robust, reliable tools that can advance the field of security assessments.

About us

At Flatt Security, we specialize in providing top-notch security assessment and penetration testing services. We also offer a powerful security assessment tool called Shisho Cloud, which combines web application security testing with Cloud Security Posture Management (CSPM) and Cloud Infrastructure Entitlement Management (CIEM) capabilities.

If you’re interested in learning more, feel free to reach out to us at https://flatt.tech/en  .