GitGuardian helps developers and security teams detect secrets (API keys, tokens, credentials) that have been accidentally committed to source code. At the core of our platform sits our secret detection engine: a component that takes raw bytes as input and outputs detected secrets, running against hundreds of gigabytes of code and data every day. Migrating this engine to Rust was driven by two goals: raw performance gains to reduce infrastructure costs, and improved portability, allowing us to eventually run the engine in environments where a Python runtime is not available.
Rewriting legacy software in Rust has been a trend in recent years. So much that it did not only spawn numerous rewrites but also its own meme. When we decided to migrate our secret detection engine to Rust, one of our main goals was performance: Our Python implementation was fast, but given that we scan hundreds of gigabytes on a daily basis, even small improvements can massively reduce our costs.
We set out on this journey with an audacious goal: Rewrite the engine from scratch and have nobody notice it. In retrospect, the decision to avoid breaking changes as much as possible ended up causing most of our headaches during the migration. The plan was to change the plane's engine, mid-flight, with all business class passengers on board.
A rewrite goes way beyond the code that needs to be rewritten
In order to limit in-flight turbulence, we decided to start our rewrite by migrating the public data types of our engine, so that issues in downstream usage would surface early on in the process, even though this required occasional acrobatics in our Python bindings.
We started by migrating our public API to identify potential blocking issues early on
Early awareness of issues also motivated our decision to use our new detection engine as soon as possible in production. Once we had migrated the first building blocks of our engine, we immediately started to scan with a subset of detectors running in Rust. In our engine, a detector is a self-contained unit responsible for identifying a specific type of secret (for example, an AWS access key or a GitHub token). Each detector encapsulates its own matching logic, making them natural units for an incremental migration: we could port them one by one and run Rust and Python detectors side by side. Incidentally, this progressive migration had the convenient side-effect that we reviewed and improved many of our detectors along the way.
We started scanning with Rust-based detectors as soon as possible
The beauty of Python is that it's malleable. If Rust with its elaborate type system is Lego, Python is Play-Doh. If you need to access an internal property of a library's Python class, the language generously lets you poke a hole into it so that you can access the property value and leave the library maintainer to work on more important things than exposing an internal value via a public API.
import numpy as np
a = np.array([1.0, 2.0, 3.0])
print(a.dtype.type.mro()[2].__subclasses__())
Want to dynamically learn about numpy's architecture? Go for it!
When migrating code with downstream users to Rust, this aspect of Python has an enormous impact. Throughout our migration, we invested a significant amount of time into reliable scaffolding for the actual rewrite. Fortunately, the pool of projects that depend on our detection engine was small and contained only internal codebases. Therefore we were able to immediately test every commit of Rust replacing Python against the test suites of our downstream consumers. It allowed us to survey the usage of every private property before we would enclose it in a private struct field in Rust.
A rewrite needs expertise in both the source and target language
Talking of bindings: We decided early on to use pyo3 for the Python bindings of our Rust rewrite. Initially, we had some concern that the binding layer might eat up our performance gains, but a first proof of concept quickly dispelled our concerns. pyo3 is a terrific project that does an excellent job of balancing performance and memory safety. Given that Rust and Python are built on fundamentally different core principles, it is remarkable how little pyo3 users are affected by the constant mediation between languages under the hood.
That said, our team's experience in both ecosystems proved invaluable: even with pyo3 smoothing out the boundary, the migration still surfaced a number of subtle, time-consuming bugs. About half-way through the rewrite, we noticed that our engine's performance was hitting an invisible ceiling. Despite notable improvements in local benchmarks, large-scale multiprocess scans with our detection engine ran at the same pace as before. At first, we were suspecting some sort of resource contention between processes related to Python's primary performance scapegoat, the GIL, but given that every process was running its own Python interpreter with its own lock, we quickly discarded this hypothesis.
As it turned out, the issue was not in the GIL itself, but how our Rust engine was treating it. Scanning data for secrets is a computationally expensive operation. Our Python engine may have taken longer than our Rust rewrite to scan, but during the scanning process it would regularly release the GIL for the interpreter to run its routines, for example garbage collection.
Our Rust implementation on the other hand held the GIL during the entire scan process. As a result, no other Python thread was able to run, garbage piled up and chaos ensued. Once we identified the issue, the solution was as easy as a single pyo3 call that ensured that the GIL would be released during long-running operations in Rust. The next large-scale scan showed that without holding the GIL our new engine ran four times as fast.
The numbers
Relative increase of scanning speed over the course of our rewrite
Once the GIL bottleneck was resolved, we finally saw the raw power of the Rust implementation, resulting in a 3x global speedup compared to the legacy Python engine. Throughout the rewrite, our performance increased steadily as more of our secret detectors were using the new components in Rust. There were few components of our engine that gained an extraordinary performance boost when we implemented them in Rust. Rather we saw consistent improvements across the entire engine as the abstraction costs of Python disappeared — a result that gave us confidence both in the quality of our existing codebase as well as our decision to rewrite it in Rust while retaining the general engine structure from Python.
A word about AI
When we began the migration project, coding was still down-to-earth manual labor. The full force of frontier models began to show when we were about half-way through the rewrite, and the agentic assistance significantly increased our ability to detect implementation disparities and to quickly prototype different feature representations in Rust. At the time of writing, the migration could have likely been completed in a fraction of the time we took, and in a largely automated fashion. However, besides the performance improvements, as a team we profited immensely from rewriting every feature of the engine. We reviewed every line of what most of us only knew as legacy code and every one of us now has an extensive, detailed understanding of the codebase, which is invaluable as we increasingly automate development.
There is a next step to this: we have achieved important performance goals with our Rust migration. However, we are scanning secrets in more sources than before, and with the advent of autonomous coding agents there's a need to catch secrets before they leave the user's machine. All of this requires even faster secret detection. We are already working on the next improvements.
Lessons Learned: Migrating to Rust
1. Start with the public API, not the implementation
Migrating public data types first forces downstream compatibility issues to surface early, before the bulk of the rewrite is done. It is much cheaper to fix interface problems at the start than mid-migration.
2. Ship to production incrementally
Don't wait until the rewrite is complete to deploy. Running Rust and Python side by side (via a dispatcher) allowed the team to catch real-world issues early and build confidence gradually, while keeping a safe fallback.
3. Build a kill switch
A runtime dispatcher that can route traffic between the old and new implementation is essential when rewriting a production system. It removes the risk of a hard cutover and makes rollbacks instant.
4. Understand the boundary between languages, not just each language
Even with a great bridging library like pyo3, subtle cross-language bugs will emerge. In this case, the Rust engine was holding the GIL throughout long scans, blocking Python's garbage collector and neutralising the performance gains entirely. You need expertise in both ecosystems to catch these issues.
5. Always release the GIL during long-running Rust operations
When calling Rust from Python, any long-running operation must explicitly release the GIL so the Python interpreter can continue its housekeeping. Failing to do so can make your rewrite significantly slower than the original.
6. Test against downstream consumers continuously
Running the full test suites of every downstream codebase on every commit keeps compatibility problems visible in real time and prevents surprises at the end of the migration.
7. Avoiding breaking changes costs more than you expect
The decision to make the migration invisible to users was the right call for business continuity, but it was also responsible for most of the headaches. Be prepared for significant scaffolding and binding work if you commit to backward compatibility.
8. The rewrite is also a code review
A full rewrite forces the team to read and understand every line of legacy code. The knowledge gained is a long-term asset, especially as more development gets automated.
Top comments (0)