DEV Community

Mateus Cavalcanti
Mateus Cavalcanti

Posted on • Originally published at mateusdcc.vercel.app

I Reverse-Engineered Codex's Web Search and Made It Available to Claude, Gemini, and Local Models

#ai #opensource #typescript #reverseengineering

I wanted Codex's web search. I did not necessarily want Codex to be the model doing the reasoning.

That distinction ended up being much more interesting than I expected.

Pi makes it easy to switch the active model between Gemini, Claude, OpenRouter models, local models, etc. But model capability and agent capability are not the same thing. Search, browsing, filesystem access, execution, memory and the rest of the harness can easily become coupled to whichever provider you happen to be using.

So I started looking at how Codex itself performs web search.

What I found was a standalone search path inside the Codex infrastructure that could be called independently from the normal model interaction flow.

I reverse-engineered its interface and turned it into pi-gpt-search, an open-source Pi extension that lets the active model use Codex's web retrieval layer while remaining the model responsible for reasoning.

The resulting architecture is basically this:

                   Pi
                    |
                    v
       Claude / Gemini / Local LLM
              reasoning model
                    |
                    | tool call
                    v
          codex-search / codex-research
                    |
                    v
      Codex standalone search backend
                    |
                    v
        structured web information
                    |
                    v
       Claude / Gemini / Local LLM
             continues reasoning
Enter fullscreen mode Exit fullscreen mode

The search backend and the reasoning model no longer have to be the same thing.

That is the part of the project I find most useful.

Finding the standalone search interface

I did not start by intercepting random network traffic and guessing requests.

Codex already exposes a surprisingly useful source of structural information through its app-server protocol generation.

I generated its TypeScript protocol bindings:

codex app-server generate-ts --out /tmp/codex-ts
Enter fullscreen mode Exit fullscreen mode

Searching the generated definitions exposed types such as WebSearchItem and WebSearchAction.

One Rust documentation comment was particularly useful:

"Structured search results returned out-of-band by standalone web search."

"Standalone" and "out-of-band" were exactly the words I wanted to see.

At this point there was evidence that web search was not necessarily just an opaque side effect buried inside a normal Codex model turn.

The next step was the binary.

Mining the Codex binary

I searched the installed Codex executable for strings related to web search:

strings /Applications/ChatGPT.app/Contents/Resources/codex \
  | grep -iE "(search|alpha/search|backend-api)"
Enter fullscreen mode Exit fullscreen mode

Among the results were identifiers like:

standalone_web_search
codex.web_search.results
alpha/search
https://chatgpt.com/backend-api/
Enter fullscreen mode Exit fullscreen mode

That gave me a very plausible endpoint:

https://chatgpt.com/backend-api/codex/alpha/search
Enter fullscreen mode Exit fullscreen mode

Finding the URL was only half the problem. I still did not know the actual request schema.

And this is where the backend itself became useful.

Reconstructing the request schema from errors

I reused the OAuth credentials generated by codex login and started probing the endpoint with deliberately incomplete requests.

For example, sending only a query led to an error about a missing id.

Adding an id then exposed the next required field, model.

Adding that produced another validation response. Sending command instead of commands produced a correction. Sending the wrong type for search_query exposed that it expected an array of objects.

So instead of blindly fuzzing the API, I could use its validation errors to progressively reconstruct the contract.

The resulting minimal request looked roughly like this:

{
  "id": "1",
  "model": "gpt-4o",
  "commands": {
    "search_query": [
      {
        "q": "OpenAI Codex GitHub repository"
      }
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

And that returned structured web results.

The model field deserves some clarification because it is easy to misunderstand what I am claiming here.

The endpoint requires it as part of its protocol. That does not mean pi-gpt-search sends the user's prompt to GPT and asks GPT to solve the task.

The active Pi model still decides what to search for, receives the resulting web information and performs the reasoning.

Turning the endpoint into an actual research tool

A raw undocumented endpoint is interesting, but not especially useful by itself.

So I built a provider around it.

The current project exposes two main tools.

codex-search handles simple single-query retrieval. It accepts the query itself plus optional domain filtering, recency filtering and response-length control.

For example, the active model can invoke something equivalent to:

codex-search({
  query: "latest Rust release",
  domains: ["rust-lang.org"],
  response_length: "short"
})
Enter fullscreen mode Exit fullscreen mode

The other interface is more interesting: codex-research.

Search systems used by agents need more than a query -> results function. The model frequently needs to search, inspect one result, find a specific section, follow another reference and only then construct an answer.

So codex-research exposes the richer command structure I recovered from the backend:

{
  "search_query": [
    {
      "q": "OpenAI Codex GitHub repository",
      "domains": ["github.com"]
    }
  ],
  "response_length": "medium"
}
Enter fullscreen mode Exit fullscreen mode

A later tool call in the same research session can then open a returned reference:

{
  "open": [
    {
      "ref_id": "turn0search0"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Or search inside that document:

{
  "find": [
    {
      "ref_id": "turn1view0",
      "pattern": "terminal"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

There is also support for click.

The provider keeps a stable session identity across these calls, so references produced during one operation remain useful during subsequent research operations.

At that point this stops being a search wrapper and becomes a small web research harness.

Search is not reasoning

This was the architectural motivation behind the project.

A lot of discussion around coding agents collapses everything into "the model".

But a coding agent is not just a model.

There is the model, and then there is the harness around it: tools, retrieval, context management, filesystem access, shell execution, browser/search infrastructure, memory, verification and the policies controlling when these things are used.

Those components do not inherently need to come from the same vendor.

With this extension I can run Gemini as the reasoning model:

Gemini
   |
   | decides that current information is required
   v
Codex search infrastructure
   |
   | returns web evidence
   v
Gemini
   |
   v
final reasoning
Enter fullscreen mode Exit fullscreen mode

The same applies to Claude or a local model.

I am still using OpenAI infrastructure for retrieval, so calling the project "provider-independent" without qualification would be wrong.

It is reasoning-model-independent, not infrastructure-independent.

That distinction matters.

What "zero GPT inference" means here

The repository describes this as zero-GPT inference, but I want to define that claim precisely.

I added a live test that wraps fetch() with an intercepting proxy.

The test watches outgoing requests for routes associated with model execution, including completion, response, conversation and turn-start paths. If the extension attempts to call one of those routes, the test fails.

It then performs a real search followed by a real open operation.

The expected network behavior is:

pi-gpt-search
     |
     +------> /backend-api/codex/alpha/search
     |
     +------> /backend-api/codex/alpha/search

separate model inference requests: 0
Enter fullscreen mode Exit fullscreen mode

The current test expects exactly two standalone-search HTTP requests and zero separate GPT/model-inference requests.

This verifies something specific: the extension itself does not perform an additional GPT/Codex model turn as part of the search workflow.

It does not prove what OpenAI internally executes behind the /codex/alpha/search service. I cannot inspect their backend, so claiming that no model of any kind exists anywhere internally would go beyond the evidence I have.

For my use case, the important property is that the active Pi model remains the reasoning agent and no additional model turn is initiated by the extension.

Keeping web retrieval out of the main context where possible

There is another problem with agentic browsing that is less visible than the API call itself: context pollution.

Dumping raw web responses, HTTP metadata and huge result structures directly into the active model context is a very easy way to waste tokens.

pi-gpt-search separates model-facing content from TUI metadata.

The cleaned output and citations go back to the active model.

The detailed structured results can remain attached as local details metadata used by Pi's UI.

The output layer also converts Codex's internal citation references into usable source references and terminal hyperlinks.

The model gets the information required to continue reasoning without blindly inheriting every byte returned by the search backend.

Auth is intentionally boring

The extension reuses an existing Codex authentication session.

If you already ran:

codex login
Enter fullscreen mode Exit fullscreen mode

it can read the access token and account ID from:

~/.codex/auth.json
Enter fullscreen mode Exit fullscreen mode

You can also provide the credentials through environment variables.

The actual search request contains the command being executed and the required session/protocol metadata. The extension does not need to forward the entire Pi conversation, project files or system prompt to the search endpoint just to execute a search.

This is one of the reasons I wanted search to remain a separate tool instead of silently delegating the entire problem to another agent.

Handling the less interesting failure modes

Once I decided to publish this instead of keeping it as a local hack, some boring things became necessary.

The provider has explicit handling for authentication failures, authorization failures, HTTP rate limits, timeouts and cancellation.

Transient 502, 503 and 504 responses are retried.

Pi's AbortSignal is connected to the request, so cancelling a tool operation cancels the underlying fetch rather than leaving an HTTP request running in the background.

There are unit tests for command validation and response normalization, integration tests for provider behavior, live endpoint tests, the zero-additional-inference interception test and an end-to-end research harness test.

I do not think an undocumented API becomes "stable" because you added tests around it, but tests at least make breakage observable instead of mysterious.

Trying it

The package is published on npm and can be installed directly through Pi:

pi install npm:pi-gpt-search
Enter fullscreen mode Exit fullscreen mode

Or executed temporarily:

pi -e npm:pi-gpt-search
Enter fullscreen mode Exit fullscreen mode

After installation, the model gets access to codex-search and codex-research.

There is also a direct command:

/gpt-search Rust 1.97 release notes
Enter fullscreen mode Exit fullscreen mode

The requirements are currently Pi, Node.js 18+ and an authenticated Codex session.

The project is MIT licensed.

GitHub logo mateusdcc / pi-gpt-search

pi-gpt-search

Native, Model-Independent Web Search for Pi using OpenAI Codex Standalone Search Engine.

pi-gpt-search gives any Pi model (Gemini, Claude, local models, OpenRouter) real-time web search capabilities by reusing OpenAI Codex's standalone web retrieval infrastructure - with ZERO GPT Model Inference Turns and ZERO GPT Tokens Consumed.

⚡ Quick Start: 1-Line Installation

Install via npm:

pi install npm:pi-gpt-search
Enter fullscreen mode Exit fullscreen mode

Or install via GitHub:

pi install https://github.com/mateusdcc/pi-gpt-search
Enter fullscreen mode Exit fullscreen mode

Or install project-locally for your current repository (-l flag):

pi install npm:pi-gpt-search -l
Enter fullscreen mode Exit fullscreen mode

Or try it temporarily in a single session without installing:

pi -e npm:pi-gpt-search
Enter fullscreen mode Exit fullscreen mode

⚡ Key Highlights: ZERO-GPT INFERENCE

  • 🚀 Zero GPT Tokens Spent: Pure web retrieval via OpenAI's backend endpoint. No GPT/Codex LLM turns are executed, meaning 0 input tokens, 0 output tokens, and 0 reasoning credits are billed.
  • 👑 Model Sovereign: Your active Pi model (e.g., Gemini 3.5 Flash / Gemini 3.1 Pro) remains the sole reasoning model.
  • …


View on GitHub


The obvious limitations

The biggest limitation is also obvious from the first paragraph: this is an undocumented backend interface.

OpenAI can change the request schema, authentication behavior, endpoint or availability at any point.

This is not a replacement for an official supported search API.

It also still depends on OpenAI's search infrastructure and on a valid Codex authentication session. The model doing the reasoning can be Claude, Gemini or something local, but retrieval itself is still vendor-dependent.

And this is not a headless browser implementation. The research interface exposes search and document-oriented operations such as open, find and click, but it does not give you a full DOM renderer with arbitrary browser automation.

Those are real constraints, not footnotes.

Why I published it anyway

The reverse engineering was fun, but the endpoint itself is not the main idea I took away from this.

The interesting part is how much of what we call an "AI model capability" is actually a property of the harness around the model.

Codex search can be useful without Codex being responsible for the reasoning.

A Gemini model can use retrieval infrastructure discovered in Codex. A local model can use the same interface. The tool can expose a stable contract even when the implementation behind that contract belongs to a completely different system.

Once model and harness are treated as separate components, agent architectures become much more composable.

That is what I wanted from pi-gpt-search: not another agent, but one useful subsystem extracted from an existing one and made reusable.

Top comments (0)