DEV Community: Mykola Bielousov

Let an AI agent drive your real, logged-in Chrome — safely, on localhost

Mykola Bielousov — Sat, 11 Jul 2026 07:28:45 +0000

An AI coding agent in my terminal kept hitting the same wall: it needed to do real work in the browser — on sites I was already logged into. Pull an authenticated file, fill a review form, operate a web editor. Reading a public page is easy. Acting as me, inside my own session, turned out to be surprisingly hard.

Here's why every obvious approach failed, and the small tool I built instead: ai-browser-bridge.

Why the obvious approaches don't work

OS-level input scripting (AppleScript, xdotool, and friends) is platform-specific, steals window focus, and races against whatever you're doing with your own mouse and keyboard. It also can't be trusted by every app.

Synthetic DOM events — element.dispatchEvent(new MouseEvent(...)) — carry isTrusted: false. Rich editors built on contenteditable, ProseMirror, Slate, and most CSP-strict web apps deliberately ignore untrusted events. So your automated "click" or "paste" just... does nothing.

Headless / profile-isolated automation (Puppeteer, Playwright) spins up a browser that doesn't have your session. You'd have to export authenticated cookies to it — and shipping those to any cloud service is a hard no.

Classic CDP attach (chrome --remote-debugging-port=9222) stopped being an option for the default profile in Chrome 136+.

The architecture

Three small pieces, no cloud, no SDK:

AI agent / your scripts          bridge server               Chrome extension
   bridge <cmd> CLI    ⇄   ws://127.0.0.1:8765 (token)  ⇄   service worker (MV3)
                                                              │ chrome.tabs / scripting
                                                              │ chrome.debugger  → trusted input, CSP-proof eval
                                                              │ chrome.downloads → authenticated downloads

A CLI the agent calls once per operation. It prints JSON to stdout and exits non-zero on failure — a contract any agent can drive without a library.
A WebSocket relay bound to 127.0.0.1, gated by a token.
A Manifest V3 extension that lives in your normal Chrome profile, so every session you're logged into is already there.

The key mechanism: chrome.debugger

The extension attaches chrome.debugger and speaks the DevTools Protocol per tab:

Input.dispatchMouseEvent / Input.insertText produce genuinely trusted input events (isTrusted: true) — the ones strict editors actually accept.
Runtime.evaluate runs in the page regardless of its Content-Security-Policy.
Everything happens in background tabs (active: false), so your focus is never stolen.

Because it runs inside your existing profile, there's no re-authentication and no cookie export — your sessions just work.

What it looks like

bridge() { node /path/to/ai-bridge/server/cli.js "$@"; }

bridge listTabs
bridge newTab   '{"url":"https://example.com"}'          # opens in background
bridge eval     '{"tabId":123,"code":"document.title"}'  # CSP-proof
bridge click    '{"tabId":123,"x":420,"y":310}'          # trusted click
bridge insertText '{"tabId":123,"text":"Hello"}'         # trusted paste at caret
bridge download '{"url":"https://.../file.pdf","filename":"file.pdf"}'  # your cookies
bridge pdf      '{"tabId":123}' --out page.pdf
bridge screenshot '{"tabId":123}' --out page.png

For an agent like Claude Code, you drop a short block into CLAUDE.md describing these commands, and it composes the calls itself: "open my dashboard and screenshot the error" becomes a sequence of newTab / eval / screenshot.

Security was the design constraint

This tool can act as you on any site you're logged into, so the threat model came first:

The server binds 127.0.0.1 only — nothing is reachable from the network.
Every client must present a token (generated on first run, chmod 600).
An optional per-domain allowlist is enforced before every tab-targeting command.
Every command is logged.
Chrome's native "… is debugging this browser" banner is visible whenever the debugger is attached.
No analytics, no telemetry, no external requests of any kind.

I also deliberately kept it off the Chrome Web Store. The install friction — clone the repo, load unpacked — is a feature for something this powerful: it keeps the code auditable and keeps one-click installs out of the hands of people who won't read the security model.

Try it

It's MIT, three source files, one dependency:

github.com/legisus/ai-bridge

Feedback and PRs welcome — especially around new CDP-backed commands and a Firefox port.

How to Implement OpenClaw in a Spring Boot Application (Java)

Mykola Bielousov — Sat, 07 Feb 2026 22:21:19 +0000

Overview

OpenClaw is an open‑source, self‑hosted AI agent that exposes a local gateway HTTP API (typically running on http://localhost:18789 or a similar endpoint). help.apiyi
To use it in a Spring Boot app, you treat OpenClaw as a remote REST service and call it via RestClient or WebClient (or RestTemplate if you prefer) from your Java backend. docs.spring

Step 1: Deploy and configure OpenClaw

Before integrating with Spring Boot, ensure OpenClaw is running and accessible:

Install OpenClaw on your machine or VPS using the official installer (for example, via the provided shell script). codecademy
Run the interactive setup and choose a model provider (e.g., OpenAI, Anthropic, Gemini, or an OpenRouter‑backed model). openrouter
Note the gateway URL and auth token (usually via a config file like ~/.openclaw/openclaw.json or environment variables). docs.byteplus

Example gateway details:

URL: http://localhost:18789/v1/chat/completions
Auth header: Authorization: Bearer <YOUR_GATEWAY_TOKEN>

Step 2: Set up a Spring Boot project

Create a new Spring Boot project (e.g. with spring-boot-starter-web). docs.spring

Add spring-boot-starter-web:

   <dependency>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-web</artifactId>
   </dependency>

Create a simple DTO for the OpenClaw request / response if you want strict typing:

   public class OpenClawRequest {
       private String model;
       private List<Message> messages;

       // Getters and setters
   }

   public class Message {
       private String role;
       private String content;

       // Getters and setters
   }

   public class OpenClawResponse {
       private String id;
       private String model;
       private Choice[] choices;

       public static class Choice {
           private Message message;
           // getters/setters
       }
       // Getters and setters
   }

Step 3: Call OpenClaw via RestClient

Spring Boot 3+ encourages using RestClient instead of RestTemplate. docs.spring

Create a service bean:

   @Service
   @RequiredArgsConstructor
   public class OpenClawService {

       private final RestClient restClient;

       public OpenClawResponse ask(String userMessage) {
           OpenClawRequest req = new OpenClawRequest();
           req.setModel("gpt-4o"); // or your model slug

           List<Message> messages = new ArrayList<>();
           messages.add(new Message("user", userMessage));
           req.setMessages(messages);

           return restClient
               .post()
               .uri("/v1/chat/completions")
               .body(req)
               .retrieve()
               .body(OpenClawResponse.class);
       }
   }

Configure the RestClient.Builder in your config:

   @Configuration
   public class OpenClawConfig {

       @Bean
       @Primary
       public RestClient.Builder openClawRestClient(
           RestClient.Builder builder,
           @Value("${openclaw.base-url:http://localhost:18789}") String baseUrl,
           @Value("${openclaw.auth-token:}") String token
       ) {
           return builder
               .baseUrl(baseUrl)
               .defaultHeaders(headers -> {
                   if (token != null && !token.isEmpty()) {
                       headers.add("Authorization", "Bearer " + token);
                   }
                   headers.add("Content-Type", "application/json");
               });
       }

       @Bean
       public OpenClawService openClawService(RestClient.Builder builder) {
           RestClient restClient = builder.build();
           return new OpenClawService(restClient);
       }
   }

Add properties to application.yml:

   openclaw:
     base-url: http://localhost:18789
     auth-token: your-gateway-token

Step 4: Expose a Spring Boot endpoint

Expose a simple REST endpoint in your controller:

@RestController
@RequestMapping("/ai")
public class OpenClawController {

    private final OpenClawService openClawService;

    public OpenClawController(OpenClawService openClawService) {
        this.openClawService = openClawService;
    }

    @PostMapping("/chat")
    public ResponseEntity<String> chat(@RequestBody ChatRequest request) {
        OpenClawResponse response = openClawService.ask(request.getMessage());
        if (response.getChoices() != null && response.getChoices().length > 0) {
            String content = response.getChoices()[0].getMessage().getContent();
            return ResponseEntity.ok(content);
        }
        return ResponseEntity.status(500).body("No response from OpenClaw");
    }

    public static class ChatRequest {
        private String message;
        // getters/setters
    }
}

Now you can call POST /ai/chat from your frontend or Postman and it will route through your Spring Boot app to OpenClaw’s gateway. stackoverflow

Step 5: Handling streaming / SSE (optional)

If you want to stream the OpenClaw response (e.g., for a chat‑style UI), you can:

Use application/stream+json if your OpenClaw gateway supports SSE‑style streaming. help.apiyi
In Spring Boot, expose the streaming endpoint using SseEmitter or WebClient‑based streaming, depending on whether you want to proxy the stream or consume it as Java objects. youtube

A simple streaming‑style sketch:

@GetMapping("/stream")
public SseEmitter streamChat(@RequestParam String message) {
    SseEmitter emitter = new SseEmitter(30_000L);
    executor.submit(() -> {
        // Use WebClient or direct HttpURLConnection to stream OpenClaw SSE
        // For brevity, assume you already have a streaming mechanism in place.
    });
    return emitter;
}

Best‑practice notes

Security: Do not expose the OpenClaw gateway directly to the public Internet; keep it internal and call it only from your backend or via a reverse proxy with auth. atlantic
Rate‑limiting / retries: Use Resilience4j or Spring Retry around OpenClawService calls. reddit
Model‑agnostic usage: OpenClaw is designed to be model‑agnostic, so you can swap providers (OpenAI, Gemini, OpenRouter, etc.) without changing the Java code—only the gateway config and model name. reddit

Summary

To implement OpenClaw in a Spring Boot application:

Deploy and configure OpenClaw locally or on a VPS, noting its HTTP gateway URL and auth token. codecademy
In Spring Boot, use RestClient (or WebClient) to call the OpenClaw /v1/chat/completions‑style endpoint. stackoverflow
Wrap the HTTP client in a service bean and expose a REST controller that your frontend can talk to.

This pattern lets you keep OpenClaw as a backend AI agent while your Spring Boot app remains the orchestration and security layer for your users. reddit

Mykola Bielousov

AI News Daily: Feb 6, 2026 — The Agent Era Begins

Mykola Bielousov — Fri, 06 Feb 2026 23:49:38 +0000

AI News Daily: Feb 6, 2026

A quick roundup of what is happening in AI today.

1. OpenAI "Operator" Looming?

Reports are swirling that OpenAI is preparing to release Operator, its autonomous agent capable of controlling desktop environments. This marks a major shift from "chatbots" to "do-bots" in 2026.

2. EU AI Act Enforcement

Today marks a critical deadline for the EU AI Act. High-risk AI systems must now fully comply with transparency and risk management protocols. Expect some service disruptions as companies adjust.

3. Google Gemini 3.0 Refinements

Google quietly pushed updates to Gemini 3.0, specifically targeting legal and medical reasoning performance. Benchmarks show a 12% improvement in citing case law accuracy.

4. NVIDIA B-Series Chips

NVIDIA's latest B-series chips are promising a 30% reduction in inference costs, which is great news for anyone running open-weights models like Llama 4 locally.

Mykola Bielousov