hex

Posted on Jun 16

Building a Chrome Extension Request Interceptor: A Complete Development Guide

#webdev #programming #javascript #tutorial

Executive Summary

Understanding how websites communicate with backend services is crucial for developers, security researchers, and system integrators. While browser DevTools provide basic network monitoring, they miss dynamically generated requests and lack advanced filtering capabilities. This guide details the complete development process of a Chrome extension that intercepts all HTTP requests (XHR and Fetch) at the JavaScript level, providing deep visibility into website behavior through a custom DevTools panel.

Key Benefits:

Captures requests missed by standard DevTools
Provides stack traces for request origins
Offers detailed timing and performance data
Enables comprehensive website behavior analysis

Project Requirements and Architecture Planning

Initial Problem Statement

Standard browser DevTools have limitations when analyzing complex web applications:

Limited Request Context: No stack traces showing where requests originate
Timing Gaps: Missing JavaScript-level performance data
Dynamic Request Blindness: Some programmatically generated requests aren't captured
Integration Challenges: Difficult to export or process captured data

Technical Requirements

Core Functionality:

Intercept all XMLHttpRequest and Fetch API calls
Capture complete request/response data including headers and bodies
Provide stack traces for debugging request origins
Display data in an intuitive DevTools panel interface
Support real-time monitoring across all website tabs

Technical Constraints:

Must work with Manifest V3 (Chrome's latest extension format)
Cannot break existing website functionality
Must handle high-frequency request scenarios
Needs to work across all domains and protocols

Architecture Design

The extension uses a multi-layer architecture to overcome Chrome's security restrictions:

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐    ┌──────────────────┐
│   Web Page      │    │  Content Script  │    │ Background      │    │  DevTools Panel │
│                 │    │                  │    │ Service Worker  │    │                  │
│ ┌─────────────┐ │    │ ┌──────────────┐ │    │ ┌─────────────┐ │    │ ┌──────────────┐ │
│ │ Injected    │◄┼────┼►│ Message      │◄┼────┼►│ Message     │◄┼────┼►│ UI Display   │ │
│ │ Script      │ │    │ │ Forwarder    │ │    │ │ Router      │ │    │ │ & Controls   │ │
│ └─────────────┘ │    │ └──────────────┘ │    │ └─────────────┘ │    │ └──────────────┘ │
└─────────────────┘    └──────────────────┘    └─────────────────┘    └──────────────────┘
        │                        │                        │                        │
        ▼                        ▼                        ▼                        ▼
   Intercepts                Bridges                 Routes &                 Displays
   HTTP calls               contexts                Queues                  captured data

Phase 1: Project Setup and Manifest Configuration

Creating the Extension Structure

First, establish the project directory structure:

request-interceptor/
├── manifest.json
├── background.js
├── content_script.js
├── injected.js
├── devtools.html
├── devtools.js
├── panel.html
├── panel.css
├── panel.js
└── icons/
    └── icon16.png

Manifest V3 Configuration

The manifest.json file defines the extension's capabilities and architecture:

{
  "manifest_version": 3,
  "name": "Request Interceptor",
  "version": "1.0",
  "description": "Intercepts requests in-page and displays them in a DevTools panel.",
  "permissions": [
    "storage"
  ],
  "host_permissions": [
    "<all_urls>"
  ],
  "icons": {
    "128": "icon.png"
  },
  "background": {
    "service_worker": "background.js"
  },
  "content_scripts": [
    {
      "matches": ["<all_urls>"],
      "js": ["content_script.js"],
      "run_at": "document_start",
      "all_frames": true
    }
  ],
  "devtools_page": "devtools.html",
  "web_accessible_resources": [
    {
      "resources": ["injected.js"],
      "matches": ["<all_urls>"]
    }
  ]
}

Key Configuration Decisions:

"run_at": "document_start": Ensures injection before any website scripts execute
"all_frames": true: Captures requests from iframes and embedded content
"<all_urls>": Universal access for comprehensive website analysis
web_accessible_resources: Allows content script to inject our monitoring code

Phase 2: Core Interception Logic Development

The Injected Script Foundation

The injected.js file runs in the website's context and performs the actual HTTP interception:

(function () {
    // Prevent multiple injection
    if (window.__HybridInterceptorActive) return;
    window.__HybridInterceptorActive = true;

    const sendDataToDevtools = (data) => {
      window.postMessage({
        type: "FROM_PAGE",
        payload: data
      }, "*");
    };

    const getStackTrace = () => {
      try {
        throw new Error();
      } catch (e) {
        return e.stack ? e.stack.split("\n").slice(2).join("\n") : 'Stack trace not available';
      }
    };

Design Rationale:

Global Flag: Prevents multiple injections that could cause conflicts
PostMessage API: Safely communicates with content script across security boundaries
Stack Trace Capture: Provides debugging context for request origins

XMLHttpRequest Interception

XMLHttpRequest interception requires monkey-patching three prototype methods:

const originalXhrOpen = XMLHttpRequest.prototype.open;
const originalXhrSend = XMLHttpRequest.prototype.send;
const originalSetRequestHeader = XMLHttpRequest.prototype.setRequestHeader;

XMLHttpRequest.prototype.open = function (method, url) {
  this._interceptorData = {
    type: 'XHR',
    method,
    url,
    headers: {},
    start: performance.now(),
    trace: getStackTrace()
  };
  return originalXhrOpen.apply(this, arguments);
};

XMLHttpRequest.prototype.setRequestHeader = function (key, value) {
    if (this._interceptorData && this._interceptorData.headers) {
        this._interceptorData.headers[key] = value;
    }
    return originalSetRequestHeader.apply(this, arguments);
};

XMLHttpRequest.prototype.send = function (body) {
  if (this._interceptorData) {
    this._interceptorData.body = body || null;
  }

  const onStateChange = () => {
    if (this.readyState === 4 && this._interceptorData) {
      this._interceptorData.end = performance.now();
      this._interceptorData.duration = this._interceptorData.end - this._interceptorData.start;
      this._interceptorData.status = this.status;
      this._interceptorData.response = this.responseText;
      this._interceptorData.timestamp = new Date().toISOString();

      sendDataToDevtools(this._interceptorData);
    }
  };

  this.addEventListener("readystatechange", onStateChange);
  return originalXhrSend.apply(this, arguments);
};

Technical Challenges Solved:

State Management: Using instance properties to track request data across method calls
Header Capture: Intercepting setRequestHeader calls to build complete header map
Timing Precision: Using performance.now() for microsecond-accurate duration measurement
Non-intrusive Design: Preserving original functionality while adding monitoring

Fetch API Interception

Fetch interception is more complex due to its Promise-based nature and response streaming:

const originalFetch = window.fetch;
window.fetch = function(...args) {
  const [url, config] = args;
  const fetchData = {
    type: 'Fetch',
    method: config?.method || 'GET',
    url: url.toString(),
    headers: config?.headers ? new Headers(config.headers).toJSON() : {},
    body: config?.body || null,
    start: performance.now(),
    trace: getStackTrace(),
    timestamp: new Date().toISOString()
  };

  return originalFetch(...args)
    .then(response => {
      const responseClone = response.clone();
      fetchData.status = response.status;
      fetchData.end = performance.now();
      fetchData.duration = fetchData.end - fetchData.start;

      responseClone.text().then(text => {
        fetchData.response = text;
        sendDataToDevtools(fetchData);
      });

      return response;
    })
    .catch(error => {
      fetchData.status = 'Error';
      fetchData.response = error.message;
      fetchData.end = performance.now();
      fetchData.duration = fetchData.end - fetchData.start;
      sendDataToDevtools(fetchData);
      throw error;
    });
};

Key Implementation Details:

Response Cloning: Essential for reading response body without consuming the original stream
Headers Processing: Custom toJSON() method for Headers object serialization
Error Handling: Capturing both network errors and successful responses
Promise Chain Preservation: Maintaining original Fetch behavior for website compatibility

Phase 3: Inter-Context Communication System

Content Script Bridge

The content script (content_script.js) bridges the isolated website context with the extension's privileged context:

// 1. Inject the monitoring script into the page
const script = document.createElement('script');
script.src = chrome.runtime.getURL('injected.js');
(document.head || document.documentElement).appendChild(script);
script.onload = () => script.remove();

// 2. Listen for messages from the injected script
window.addEventListener("message", (event) => {
  // Security validation
  if (event.source !== window || !event.data.type || event.data.type !== "FROM_PAGE") {
    return;
  }

  // 3. Forward the data to the background script
  chrome.runtime.sendMessage({
    type: 'INTERCEPTED_REQUEST',
    data: event.data.payload
  });
}, false);

Security Considerations:

Source Validation: Ensures messages originate from the same window
Type Filtering: Only processes messages with expected structure
Script Cleanup: Removes injection script element after execution

Background Service Worker

The background script (background.js) manages connections and message routing:

const connections = {};
const messageQueue = {};

chrome.runtime.onConnect.addListener(port => {
  const extensionListener = (message) => {
    if (message.name === "init") {
      const tabId = message.tabId;
      connections[tabId] = port;

      // Process any queued messages for this tab
      if (messageQueue[tabId]) {
        messageQueue[tabId].forEach(msg => port.postMessage(msg));
        delete messageQueue[tabId];
      }

      port.onDisconnect.addListener(() => {
        delete connections[tabId];
      });
    }
  };
  port.onMessage.addListener(extensionListener);
});

chrome.runtime.onMessage.addListener((request, sender) => {
  if (sender.tab) {
    const tabId = sender.tab.id;
    if (tabId in connections) {
      connections[tabId].postMessage(request);
    } else {
      // Queue messages when DevTools panel isn't ready
      if (!messageQueue[tabId]) {
        messageQueue[tabId] = [];
      }
      messageQueue[tabId].push(request);
    }
  }
  return true;
});

Advanced Features:

Connection Management: Maintains persistent connections to DevTools panels
Message Queuing: Buffers messages when DevTools isn't open
Tab Isolation: Routes messages to correct panel instance
Cleanup Handling: Removes stale connections on disconnect

Phase 4: DevTools Panel Integration

Panel Registration

DevTools integration starts with panel registration (devtools.js):

chrome.devtools.panels.create(
    "Request Spy",
    "icons/icon16.png",
    "panel.html"
);

Panel HTML Structure

The panel UI (panel.html) uses semantic HTML for accessibility and maintainability:

<!DOCTYPE html>
<html>
  <head>
    <link rel="stylesheet" href="panel.css">
  </head>
  <body>
    <div class="toolbar">
      <button id="clear-button">Clear</button>
    </div>
    <div id="requests-container"></div>

    <template id="request-template">
      <div class="request-entry">
        <details>
          <summary class="request-summary">
            <span class="status-badge"></span>
            <span class="method-badge"></span>
            <span class="url"></span>
            <span class="duration"></span>
          </summary>
          <div class="details-grid">
            <div class="grid-header">URL:</div>
            <div class="grid-value full-url"></div>
            <div class="grid-header">Timestamp:</div>
            <div class="grid-value timestamp"></div>
            <div class="grid-header">Request Headers:</div>
            <div class="grid-value"><pre class="headers"></pre></div>
            <div class="grid-header">Request Body:</div>
            <div class="grid-value"><pre class="request-body"></pre></div>
            <div class="grid-header">Response Body:</div>
            <div class="grid-value"><pre class="response-body"></pre></div>
            <div class="grid-header">Trace:</div>
            <div class="grid-value"><pre class="trace"></pre></div>
          </div>
        </details>
      </div>
    </template>

    <script src="panel.js"></script>
  </body>
</html>

UI Design Decisions:

Details/Summary Elements: Native browser collapsible functionality
Template Element: Efficient DOM cloning for dynamic content
Grid Layout: Organized data presentation
Pre Elements: Preserve formatting for code and JSON

Panel Styling

CSS styling (panel.css) follows DevTools design patterns:

body {
  font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
  font-size: 14px;
  margin: 0;
  padding: 0;
  background-color: #f3f3f3;
}

.status-badge {
  padding: 2px 8px;
  border-radius: 10px;
  font-size: 12px;
  color: white;
  font-weight: bold;
  white-space: nowrap;
  flex-shrink: 0;
}

.status-badge.status-ok { background-color: #28a745; }
.status-badge.status-error { background-color: #dc3545; }
.status-badge.status-other { background-color: #6c757d; }

.details-grid {
  display: grid;
  grid-template-columns: 150px 1fr;
  gap: 8px;
  padding: 12px;
  border-top: 1px solid #eee;
  background-color: #fafafa;
}

Visual Design Principles:

Status Color Coding: Immediate visual feedback for response status
Responsive Grid: Adapts to different panel sizes
Developer-Friendly Typography: Monospace fonts for code content
Consistent Spacing: Professional appearance matching DevTools aesthetic

Panel JavaScript Logic

The panel logic (panel.js) handles data reception and UI updates:

document.addEventListener('DOMContentLoaded', () => {
  const requestsContainer = document.getElementById('requests-container');
  const requestTemplate = document.getElementById('request-template');
  const clearButton = document.getElementById('clear-button');

  // Establish connection to background script
  const backgroundPageConnection = chrome.runtime.connect({ name: "devtools-panel" });
  backgroundPageConnection.postMessage({ 
    name: 'init', 
    tabId: chrome.devtools.inspectedWindow.tabId 
  });

  // Handle incoming request data
  backgroundPageConnection.onMessage.addListener((message) => {
    if (message.type === 'INTERCEPTED_REQUEST' && message.data) {
      addRequestToView(message.data);
    }
  });

  // Clear functionality
  clearButton.addEventListener('click', () => {
    requestsContainer.innerHTML = '';
  });

  function addRequestToView(data) {
    const template = requestTemplate.content.cloneNode(true);

    // Status badge configuration
    const statusBadge = template.querySelector('.status-badge');
    const statusStr = data.status ? data.status.toString() : '???';
    statusBadge.textContent = `${data.type} ${statusStr}`;

    if (statusStr.startsWith('2')) {
      statusBadge.classList.add('status-ok');
    } else if (statusStr.startsWith('4') || statusStr.startsWith('5') || statusStr === 'Error') {
       statusBadge.classList.add('status-error');
    } else {
       statusBadge.classList.add('status-other');
    }

    // Populate template with request data
    template.querySelector('.method-badge').textContent = data.method;
    template.querySelector('.url').textContent = data.url;
    template.querySelector('.full-url').textContent = data.url;
    template.querySelector('.duration').textContent = data.duration ? `${data.duration.toFixed(0)}ms` : '';
    template.querySelector('.timestamp').textContent = data.timestamp;

    template.querySelector('.headers').textContent = JSON.stringify(data.headers || {}, null, 2);
    template.querySelector('.request-body').textContent = formatBody(data.body);
    template.querySelector('.response-body').textContent = formatBody(data.response);
    template.querySelector('.trace').textContent = data.trace || 'N/A';

    requestsContainer.appendChild(template);
  }

  function formatBody(body) {
    if (body === null || body === undefined) return 'N/A';

    if (typeof body === 'string') {
        try {
            // Pretty-print JSON if possible
            return JSON.stringify(JSON.parse(body), null, 2);
        } catch (e) {
            return body;
        }
    }

    if (typeof body === 'object') {
        return JSON.stringify(body, null, 2);
    }

    return body.toString();
  }
});

Advanced UI Features:

Template Cloning: Efficient DOM manipulation for high-frequency updates
Smart JSON Formatting: Automatic pretty-printing of JSON content
Real-time Updates: Immediate display of intercepted requests
Connection Management: Robust handling of DevTools panel lifecycle

Phase 5: Testing and Deployment

Local Testing Process

1. Extension Loading:

# Navigate to chrome://extensions/
# Enable "Developer mode"
# Click "Load unpacked" and select project directory

2. Functional Testing:

Open any website with AJAX functionality
Open DevTools and navigate to "Request Spy" tab
Trigger various HTTP requests (form submissions, API calls, etc.)
Verify all request data is captured and displayed correctly

3. Performance Testing:

Test on high-traffic websites (social media, news sites)
Monitor for performance degradation
Verify memory usage remains stable during extended sessions

Common Issues and Solutions

Issue: Requests Not Appearing

Diagnosis: Check if injected script loaded successfully
Solution: Verify web_accessible_resources configuration in manifest

Issue: DevTools Panel Not Loading

Diagnosis: Check background script connection handling
Solution: Ensure proper tab ID routing and connection management

Issue: Missing Request Data

Diagnosis: Timing issues with response body reading
Solution: Implement proper Promise handling in Fetch interception

Production Deployment

Chrome Web Store Preparation:

Icon Assets: Create required icon sizes (16x16, 48x48, 128x128)
Store Listing: Prepare description, screenshots, and promotional images
Privacy Policy: Document data handling practices
Permissions Justification: Explain necessity of broad permissions

Advanced Use Cases and Applications

Website Behavior Analysis

API Discovery:

Example: Analyzing a Social Media Site
┌─────────────────────────────────────────────────────────────┐
│ Captured Requests:                                          │
│ POST /api/v1/posts          → Content publishing           │
│ GET  /api/v1/feed           → Timeline loading             │
│ POST /api/v1/analytics      → User behavior tracking       │
│ GET  /api/v1/notifications  → Real-time updates            │
└─────────────────────────────────────────────────────────────┘

Authentication Flow Mapping:

Capture OAuth token exchanges
Monitor session refresh mechanisms
Identify security headers and CSRF protection

Performance Analysis:

Measure API response times across different endpoints
Identify slow or failing requests
Analyze request frequency patterns

Security Research Applications

AJAX Endpoint Discovery:

Identify hidden API endpoints not visible in source code
Map complete API surface area
Discover internal vs. public endpoints

Data Flow Analysis:

Track sensitive data transmission
Identify unencrypted communications
Monitor third-party integrations

Rate Limiting Detection:

Observe request throttling behavior
Identify API usage limits
Test error handling mechanisms

Integration Development Support

Third-Party API Integration:

Reverse-engineer API call patterns
Understand required headers and authentication
Capture complete request/response examples

Microservices Architecture Analysis:

Map service-to-service communication
Identify service dependencies
Document API contracts

Performance Optimization and Best Practices

Memory Management

Request Data Lifecycle:

// Implement request history limits
const MAX_REQUESTS = 1000;
if (requestHistory.length > MAX_REQUESTS) {
  requestHistory.splice(0, requestHistory.length - MAX_REQUESTS);
}

Cleanup Strategies:

Clear stored request data on tab close
Implement periodic cleanup of old entries
Optimize large response body handling

Scalability Considerations

High-Frequency Request Handling:

Implement request batching for performance
Add filtering options to reduce noise
Consider sampling for extremely high-traffic sites

Resource Usage Optimization:

Minimize memory footprint of stored request data
Implement lazy loading for response bodies
Optimize DOM updates for large request lists

Security Best Practices

Data Sanitization:

function sanitizeRequestData(data) {
  // Remove sensitive headers
  const sensitiveHeaders = ['authorization', 'cookie', 'x-api-key'];
  const cleanHeaders = { ...data.headers };

  sensitiveHeaders.forEach(header => {
    if (cleanHeaders[header]) {
      cleanHeaders[header] = '[REDACTED]';
    }
  });

  return { ...data, headers: cleanHeaders };
}

Permission Minimization:

Request only necessary permissions
Implement user consent for sensitive data capture
Provide opt-out mechanisms for privacy-conscious users

Future Enhancement Roadmap

Phase 1 Enhancements

Request Filtering: Add URL pattern and method filtering
Export Functionality: Generate HAR files for external analysis
Search Capabilities: Full-text search across request/response data

Phase 2 Advanced Features

Request Replay: Resend captured requests with modifications
Performance Metrics: Detailed timing breakdowns and statistics
Custom Scripting: JavaScript execution against captured data

Phase 3 Enterprise Features

Team Collaboration: Share captured sessions across team members
Integration APIs: Connect with external monitoring systems
Automated Testing: Generate test cases from captured requests

Conclusion

This Chrome extension demonstrates sophisticated browser extension development techniques while solving real-world web development challenges. The multi-context architecture, advanced JavaScript interception patterns, and custom DevTools integration showcase modern web platform capabilities.

Key Technical Achievements:

Successful implementation of non-intrusive HTTP monitoring
Robust cross-context communication system
Professional-grade developer tool interface
Comprehensive website behavior analysis capabilities

Applications for Technical Documentation:

Demonstrates complex system architecture documentation
Shows integration between multiple technical components
Provides real-world example of developer tool creation
Illustrates security considerations in extension development

The extension serves as both a practical development tool and an excellent example of technical writing that combines deep technical knowledge with clear, actionable documentation. This approach directly aligns with Google's technical writing requirements for roles involving developer tools, system integration, and infrastructure documentation.

Repository Link: https://github.com/hexcreator/hook-http-requests

DEV Community