DEV Community: cosmol-studio

Rust Cheminformatics Validation: From ChEMBL 37 to Billions of Exact RDKit Comparisons

95028 — Mon, 07 Sep 2026 00:00:00 +0000

“Validated on millions of molecules” sounds impressive, but by itself it says surprisingly little.

A cheminformatics implementation can process millions of molecules while comparing only one final output per molecule. It can also report 99.9% agreement while hiding a small but systematic semantic difference. For COSMolKit, neither is sufficient to define RDKit parity.

The validation model is instead built around an explicit question:

For this declared feature boundary, which observable behaviors must match the pinned reference?

That turns validation from a molecule-count benchmark into a behavioral contract.

The Current Validation Snapshot

COSMolKit currently uses RDKit 2026.03.1 as the pinned chemistry reference for its documented parity-covered surfaces. Its largest evidence layer uses the complete ChEMBL 37 structure table: 2,897,819 source records , of which 2,897,804 are mutually parseable under the relevant sanitized comparison boundary.

The current extended ChEMBL profile contains 31 phases and 3,968 ordered shard tasks. Across the consolidated accepted evidence, COSMolKit records:

2,931,581,192 matching checks
0 blocking mismatches

Distance-geometry validation additionally traverses 2,757,910,995 individual matrix entries.

The important word here is checks.

A check may be a fingerprint vector, an atom-state field, a bond-state field, an option branch, an error outcome, a serialized state, a coordinate result, or another explicitly declared observation. Molecule count tells us how broad the input set is; comparison count tells us much more about how deep the tested boundary goes.

Parity Is Boundary-Scoped

COSMolKit does not use “RDKit compatible” as a repository-wide binary label. A feature is parity-covered only for its documented boundary, and upstream APIs outside that boundary remain separate capabilities.

Inside a declared boundary, however, the rule is strict. A mismatching molecule or parameter branch cannot simply be reclassified as unsupported after the result is known. Likewise, 99%, 99.9%, or any other aggregate threshold does not convert remaining mismatches into parity.

The parity contract is conceptually simple:

same input
same operation
same options
same output schema

COSMolKit
    vs
pinned RDKit

If that comparison exposes a difference, the response must be to understand and resolve the difference, withdraw a genuinely separate capability boundary, or retain it as an executable development failure. Silently dropping the mismatching field, adding molecule-specific exclusions, or weakening the assertion is explicitly disallowed.

This matters because a clean percentage can otherwise hide exactly the edge cases that compatibility testing is supposed to discover.

Three Corpus Layers, Three Different Jobs

A single enormous corpus is not the entire validation strategy.

COSMolKit currently uses three complementary layers:

Layer	Size	Purpose
Project corpus	152 records	Fast daily checks and focused source-port regressions
Maintained strict corpus	5,000 records	Dense parameter matrices and detailed boundaries practical for regular testing
ChEMBL 37	2,897,819 source records	Large-scale stress, composition, batch and concurrency auditing

These corpora are intentionally not interchangeable. The small corpus preserves highly focused regressions cheaply. The 5,000-record layer can exercise parameter combinations that may be too expensive to run exhaustively over millions of structures. ChEMBL then asks whether those implementations survive a much broader chemical distribution.

The resulting model is closer to:

focused semantic cases
        +
dense option coverage
        +
large-scale stress

than simply “use the largest dataset available.”

One Molecule Can Produce Hundreds of Comparisons

SMILES writing makes the distinction especially clear.

The current ChEMBL validation expands the supported writer boundary into 768 parameter profiles , combining canonical/isomeric behavior, Kekulé output, stereo cleaning, explicit bonds and hydrogens, dative handling, atom-map behavior, and rooted output choices.

Across 2,854,362 eligible records, one complete matrix represents:

2,192,150,016 comparisons

and that matrix was executed three times to expose instability rather than merely deterministic disagreement.

So saying “2.85 million molecules passed SMILES validation” would substantially understate what was tested. The real boundary is closer to:

molecules
×
parameter branches
×
repeated execution

The same principle appears elsewhere. Fingerprint validation compares full vectors and, where relevant, provenance outputs. Molecular-state tests can compare atom and bond properties rather than only a canonical string. Modern CIPLabeler validation includes full and selected assignment paths together with _CIPCode, _CIPRank, _CIPNeighborOrder, stereo state, and exact success or failure outcomes.

Exact Does Not Always Mean String Equality

Different chemistry surfaces require different comparison boundaries.

For discrete behavior, COSMolKit generally requires exact equality: bytes, bits, return status, atom and bond state, stereochemistry, fingerprint outputs, or deterministic error behavior.

Numerical algorithms require explicit tolerances. The current project-level validation description reaches 1e-8 for matrix entries and 1e-6 for coordinates, energies, and gradients where those numerical surfaces are part of the covered contract. Stochastic workflows can additionally include seed handling, RNG state, random-draw behavior, and fixed-seed outcomes rather than merely asking whether a conformer “looks similar.”

The principle is not “everything must be bit-identical regardless of algorithm.”

It is:

Every parity surface must define what equivalence means before the result is inspected.

Validation Must Survive Composition

Scalar correctness is necessary, but production chemistry is rarely a sequence of isolated default calls.

A molecule may be parsed, sanitized, hydrogenated, edited, serialized, restored, embedded, optimized, fingerprinted, and processed through a batch pipeline. State left behind by one operation may only become observable several operations later.

The ChEMBL audit therefore includes more than isolated scalar results. Current evidence exercises operation-order composition, scalar-versus-batch behavior, shared-object concurrent reads, binary roundtrips, fixed-seed conformer outcomes, and force-field paths.

This connects directly to the earlier operation-contract work in COSMolKit. Internal contracts attempt to keep every molecular transition valid; differential validation then asks whether those transitions compose into the same observable behavior as the reference.

A Validation Run Has an Identity

Large validation is useful only if the result can be tied to exactly what was executed.

The ChEMBL runner therefore records much more than a log saying “passed.” Its identity includes the corpus manifest, every shard checksum, phase profile, audit scripts, Git state and tracked diff, installed COSMolKit extension, Python version, NumPy environment, and pinned RDKit version. Resume is allowed only when the identity remains compatible, and individual task outputs are themselves checksummed.

The ChEMBL source is deterministically partitioned into 128 shards. A run is not accepted merely because all completed jobs happened to match: missing tasks, failed tasks, a time-limited partial run, or a noninformational mismatch make the run incomplete or failing.

This is particularly important for multi-million-record testing. Without an execution identity, “we once ran ChEMBL” quickly becomes difficult to distinguish from reproducible release evidence.

The Most Useful Result Was a Failure

The strongest demonstration of the validation model is not one of the clean phases.

During the retained 2026-08-20 validation execution, two new phases exposed systemic problems. Unsanitized RemoveHs(sanitize=false) produced millions of mismatching state observations because surviving explicit-valence and implicit-hydrogen state differed from RDKit. Binary roundtrips preserved visible graph state and deterministic bytes while changing downstream hash and Morgan behavior after deserialization.

The original report did not reinterpret these as an acceptable error rate. It explicitly recorded that the overall gate was not accepted.

Both findings were subsequently traced to their underlying state semantics, corrected, retained as focused regressions, and rerun over the complete affected phase boundaries. The topology rerun now records 45,669,848 matching observations over 2,854,376 records with zero mismatch , while the binary-roundtrip rerun records 11,534,336 matching observations over 524,288 records with zero mismatch. The current VALIDATION.md incorporates those accepted reruns into the consolidated zero-blocking-mismatch evidence.

That episode captures the intended workflow better than any headline percentage:

large-scale audit
      ↓
systemic mismatch discovered
      ↓
source-level investigation
      ↓
implementation corrected
      ↓
focused regression retained
      ↓
complete affected boundary rerun
      ↓
evidence accepted

Validation is useful precisely because it is allowed to fail the project.

Why Billions of Comparisons Matter

The point of billions of comparisons is not to claim that software can be proven bug-free by testing enough molecules.

It cannot.

Their value is that they combine several dimensions that are easy to conflate:

chemical breadth
×
parameter breadth
×
state depth
×
execution modes
×
repetition

A million molecules compared only by final SMILES provide one kind of evidence. The same molecules compared across structured molecular state, complete option matrices, fingerprints, deterministic errors, serialization, operation composition, batch paths, stochastic behavior, coordinates, energies, and matrices provide a substantially different kind.

This is why COSMolKit counts comparison observations rather than presenting molecule count as the whole validation claim.

From Source Semantics to Validation Evidence

The earlier articles in this series described the other parts of the methodology.

COSMolKit first separates RDKit’s mature chemistry semantics from the architecture used to carry them. Compatibility-critical behavior is reproduced from pinned upstream source rather than reconstructed by repeatedly fitting to corpus outputs. Registered operations and strict development checks then constrain how those semantics interact with the redesigned Rust molecular-state model.

Validation is the final layer:

pinned upstream semantics
          ↓
source-backed Rust port
          ↓
operation contracts + strict CI
          ↓
focused regression matrices
          ↓
ChEMBL-scale differential audit

The corpus is therefore not asked to invent the implementation.

It is asked to break it.

And when it does, the mismatch remains blocking until the semantic difference is understood, corrected, preserved as a regression, and rerun over the declared boundary.

That is what the current 2.93 billion matching checks are intended to represent. Not “COSMolKit tried a lot of molecules,” but a much more specific claim:

Within the explicitly documented parity-covered boundaries, the current accepted evidence reproduces pinned RDKit behavior across the recorded chemical, parameter, state, and execution surfaces with zero blocking mismatch.

That is the standard COSMolKit uses when it says RDKit parity.

COSMolKit Resources

Source repository ·Documentation ·Web tools ·Rust crate ·Python package

Rust Cheminformatics: Porting RDKit Source Semantics to Rust

95028 — Tue, 01 Sep 2026 00:00:00 +0000

COSMolKit currently validates its parity-covered Rust cheminformatics surfaces against pinned RDKit 2026.03.1 using three complementary corpus tiers, culminating in the complete ChEMBL 37 structure table. The current extended profile covers 2,897,819 source records , 31 validation phases , and 3,968 shard tasks. Its consolidated accepted evidence records 2,931,581,192 matching checks with zero blocking mismatch , with distance-geometry validation additionally traversing more than 2.75 billion matrix entries.

These are not simply “molecules that produced the same final string.” Depending on the declared parity boundary, COSMolKit compares exact molecular state, fingerprints, parameter branches, serialization behavior, operation composition, batch and concurrent execution, stochastic outcomes, matrices, coordinates, energies, gradients, and errors. The exact scope is documented separately in VALIDATION.md.

The more interesting question for this article is not how large the validation corpus became.

It is how the implementation reached that result.

A straightforward way to build an RDKit-compatible Rust library is to treat RDKit as an executable oracle: implement a feature approximately, run a corpus through both implementations, inspect the mismatches, let an agent patch the failing cases, expand the corpus, and repeat. Modern coding agents make this workflow remarkably fast.

COSMolKit deliberately follows a different development model.

For compatibility-critical chemistry, the implementation is derived from the pinned upstream source first. Differential corpora are then used to audit that port rather than to iteratively teach a heuristic implementation how to imitate RDKit.

That distinction is the subject of this article.

Two Ways to Build an RDKit-Compatible Implementation

With an executable reference such as RDKit, the most obvious development strategy is differential output fitting. Start with an implementation of the general algorithm, run the same molecules through RDKit and the new implementation, compare the outputs, and repair whatever differs.

In the agent era, this approach is especially attractive. An agent can implement a feature from documentation or a high-level algorithm description, execute thousands of differential examples, cluster failures, infer additional rules, patch the implementation, and repeat the process with very little human intervention.

Conceptually:

implement an approximate feature
            ↓
   run reference corpus
            ↓
   observe mismatches
            ↓
  infer missing behavior
            ↓
          patch
            ↓
      expand corpus
            ↓
         repeat

This is a legitimate engineering strategy when approximate compatibility is sufficient. It can also produce very high empirical agreement.

The difficulty is that the corpus gradually becomes part of the implementation specification.

A feature may reach 99% on a small corpus, only to expose an entirely new class of failures when the corpus grows. Another increase may reveal interactions between aromaticity and stereochemistry; another may expose query behavior, unusual valence state, operation ordering, or some previously unseen combination of options.

The implementation therefore tends to grow together with the corpus:

larger corpus
      ↓
new mismatch family
      ↓
new heuristic branch
      ↓
larger implementation
      ↓
larger corpus
      ↓
     ...

At that point, development progress becomes difficult to estimate. “99.9% complete” does not tell us whether the remaining 0.1% consists of ten isolated cases or an entire semantic family that the implementation has not yet modeled.

More importantly, even a corpus with millions of molecules cannot prove that a real production workflow will not exercise an unobserved combination.

COSMolKit tries to invert this relationship.

Corpus Agreement and Semantic Reproduction Are Different Claims

Consider two implementations that both report 100% agreement on the same million-molecule corpus.

Implementation A was built through repeated differential correction. When the corpus exposed a mismatch, the implementation was modified until the output matched.

Implementation B was constructed by tracing the relevant upstream source behavior, reproducing those state transitions in Rust, and then using the million molecules to test whether the port was correct.

The headline result can be identical:

1,000,000 / 1,000,000 matched

but the evidentiary meaning is different.

The distinction is essentially this:

Corpus-driven output fitting	Source-backed reproduction
Reference output reveals missing behavior	Upstream source defines intended behavior
Failing example motivates implementation change	First source-state divergence motivates implementation change
Corpus gradually acts as specification	Corpus acts as auditor
New corpus scale may require new heuristic branches	New corpus scale tests an already-defined transition
Passing observed cases supports empirical agreement	Passing cases support a source-derived semantic claim
Progress can become difficult to estimate as edge cases accumulate	Remaining work can be tracked against source/call-graph closure

The first approach is not inherently illegitimate. It simply supports a different kind of confidence.

For COSMolKit’s parity-covered chemistry, we wanted the second.

The Corpus Should Audit the Implementation, Not Write It

The central development rule is therefore:

Validation should verify the port. It should not become the process by which the implementation discovers what the reference probably meant.

The intended direction is:

pinned upstream source
        ↓
identify source semantics
        ↓
reproduce them in Rust
        ↓
focused source regression
        ↓
large-scale differential validation

not:

large reference corpus
        ↓
observe wrong output
        ↓
invent a correction
        ↓
      rerun
        ↓
repeat until the corpus is green

That change in direction has consequences throughout the project.

It changes how source code is written.

It changes how a mismatch is debugged.

It changes what an agent is allowed to do when a test fails.

And it changes what “100% parity” means.

A Feature Name Is Not a Specification

Many cheminformatics features appear simple at the API level. Canonical SMILES, Morgan fingerprints, RemoveHs, CIP assignment, Kekulization, ETKDG, and MMFF optimization each have understandable high-level descriptions.

But mature behavior is not contained entirely in those descriptions.

RDKit’s observable semantics can depend on details such as ordering conventions, helper dispatch, property-cache transitions, stereo cleanup, query handling, aromatic edge cases, implicit hydrogen state, exception timing, random-number consumption, floating-point behavior, and legacy branches.

A paper can describe canonicalization.

It does not necessarily describe every ordering and cleanup decision that a mature implementation makes before producing a canonical string.

Documentation can describe hydrogen removal.

It may not expose every state transition that later operations observe when sanitize=false.

So COSMolKit distinguishes:

implements the same general feature

from:

reproduces the pinned reference semantics
within a declared boundary

The second claim requires more than an API with the same name.

What “Line-by-Line Port” Means

COSMolKit describes compatibility-critical work as line-by-line, source-backed porting. That phrase does not mean translating C++ syntax mechanically into Rust syntax.

Rust should still be Rust.

The project deliberately redesigns ownership, errors, molecular state, mutation APIs, container choices, batch execution, and other architectural layers. A C++ vector can become a more appropriate Rust set; an exception can become a typed Result; mutable object behavior can be exposed through a value-style public API.

What must remain traceable is the behavior.

The project’s source-reproduction protocol requires relevant original source lines to be retained verbatim beside their corresponding Rust implementation as comments. Those lines become inline review anchors rather than being replaced by vague comments such as “implements RDKit behavior.”

This matters even more with agent-driven development. A human or agent returning to the implementation later can see both the Rust code and the upstream logic it is supposed to reproduce without reconstructing the relationship from commit history or memory.

Behavior and Performance Are Tracked Separately

A source-backed port should also be allowed to improve implementation quality.

For that reason, COSMolKit’s source markers use two independent dimensions. The first records behavioral reproduction; the second records performance and algorithmic-complexity status.

A behaviorally equivalent Rust implementation can therefore still be marked as slower, equivalent, unresolved, or better in implementation complexity. Conversely, a faster implementation does not gain behavioral-completion status merely because it looks elegant.

This enables transformations such as replacing a linear membership structure with a more appropriate Rust container while retaining the same externally observable transition.

The principle is:

Compatibility should constrain semantics, not source syntax.

This is what allows COSMolKit to combine a Rust-native architecture with conservative reference behavior.

Follow the Source Until the Behavior Actually Lives

Another practical problem is that mature C++ libraries rarely keep the full semantics of an operation in one function.

A public function may dispatch into a helper, which calls another helper, which performs the branch that actually determines the observable result.

An agent instructed only to “port function X” can easily reproduce the visible dispatcher while approximating or simplifying the deeper dependency.

COSMolKit’s source protocol therefore treats the relevant upstream call graph as the real boundary. Cross-file helper functions must be followed when they contain behavior necessary to the supported operation.

The question is not:

Did we translate this entry point?

It is:

Did we reproduce the source-defined behavior reached from this entry point?

That difference becomes substantial for canonicalization, stereochemistry, sanitization, conformer generation, and other deeply layered cheminformatics workflows.

Even Language-Level Behavior May Matter

Some reference behavior does not look like chemistry at all. It can depend on unsigned arithmetic, stream behavior, iteration order, substring handling, object lifetime, exception timing, allocation behavior, or floating-point reduction.

When those details are observable inside the declared compatibility boundary, they have to be considered explicitly.

The project allows three honest outcomes: reproduce the defined behavior, establish a deliberate Rust behavior where the upstream path itself is undefined, or leave the capability outside the supported boundary.

The scalar InChI implementation illustrates this distinction. COSMolKit reproduces the source-defined official InChI and RDKit adapter behavior for its four documented scalar APIs, but an undefined official-C initial-allocation path becomes a deterministic structured Rust allocation error rather than an attempt to reproduce undefined memory behavior.

Source-backed does not mean blindly preserving every C accident.

It means being explicit about which behavior is actually being preserved.

When the Large Corpus Finds a Failure

The most important difference between the two development paradigms appears when validation finally finds a mismatch.

In a corpus-fitting workflow, the mismatch itself often becomes the starting point for a fix. An agent inspects the failing molecule, finds a plausible region of the Rust implementation, makes a change, and checks whether the output now agrees.

COSMolKit forbids that as sufficient justification for a source-port correction.

The project’s source-bisection debugging protocol requires the mismatch to be localized to the first divergent state boundary between upstream and Rust before the implementation is changed.

Suppose a canonical SMILES differs. The final string may depend on parsing, property-cache updates, stereo preparation, ring state, canonical ranking, start-atom selection, traversal, and serialization.

Instead of modifying the writer because the writer emitted the wrong string, both RDKit and Rust are instrumented at comparable checkpoints.

The result might look like this:

raw parse match
property-cache update match
stereo preparation match
double-bond stereo update DIFFER
canonical ranks DIFFER
final SMILES DIFFER

At that point, the final string is only a symptom. The first semantic divergence occurred earlier.

The correct debugging target is the double-bond stereo transition.

Patch the First Divergence, Not the Last Symptom

This gives the source-port methodology one of its most important rules:

Patch the first divergent state transition, not the final visible symptom.

A final-output patch can make one corpus case green while compensating for a different bug upstream.

A first-divergence patch instead answers three questions: what was the last state where RDKit and Rust still agreed, where did they first disagree, and which upstream source block defines that transition?

The source-bisection protocol explicitly rejects explanations such as “this looks likely,” “this branch seems related,” or simply “this makes the failing test pass.” A correction is expected to have a concrete upstream boundary explanation.

This is particularly important for agents.

Agents are very good at optimizing toward an executable objective.

If the objective is merely “make this differential test pass,” they can often find a local patch rapidly.

COSMolKit instead changes the objective to:

Find and reproduce the missing upstream transition.

That is a much harder objective to game accidentally.

Large-Corpus Failures Become Small Semantic Regressions

When a ChEMBL-scale run discovers an unusual failure, the failing molecule should not remain only as one row in a multi-million-record corpus.

After the source divergence has been understood and fixed, the project retains a focused regression at the smallest stable state boundary, together with a higher-level regression when appropriate.

The lifecycle is therefore:

large-corpus discovery
        ↓
source-state bisection
        ↓
first divergent source transition
        ↓
source-backed correction
        ↓
focused permanent regression
        ↓
large-corpus revalidation

This is one reason large validation does not have to produce ever-growing heuristic complexity.

A new failing molecule does not become “case 17,532 that needs special handling.”

It becomes evidence that some part of the source port was incomplete or incorrect.

Once that transition is understood, the implementation is corrected at the semantic level.

Why Development Progress Becomes More Predictable

This difference also affects project planning.

In a corpus-driven heuristic implementation, completion is difficult to estimate because the remaining work is partly unknown. Every larger corpus can reveal a new family of behavior that requires another local model or special case.

The feature may appear almost complete for a long time while repeatedly producing new edge cases:

99.0%
99.7%
99.93%
99.98%
...

The number moves toward 100%, but the amount of semantic work remaining is not necessarily proportional to the remaining percentage.

A source-backed port has a different progress model. The project can inspect the upstream call graph, track which blocks are behaviorally reproduced, which remain partial, and which dependencies are still unresolved.

COSMolKit’s source markers deliberately preserve that incompleteness instead of converting everything into one repository-wide “RDKit percentage.” Current feature areas therefore have different source-closure states, and the project inventory records those differences rather than pretending the entire toolkit advances as one scalar metric.

That does not make source porting trivial.

It makes the remaining work more legible.

Why This Matters in Real Applications

A large differential corpus is extremely valuable, but production workloads are not sampled uniformly from that corpus.

Real applications compose operations.

A user may parse an SDF record, remove hydrogens without sanitization, modify topology, assign stereo, generate a conformer, optimize it, calculate fingerprints, serialize the result, restore it later, and run the same molecule through batch processing.

A heuristic implementation may pass every isolated default-output test while failing on one of those state combinations.

That is why COSMolKit’s validation includes operation-order composition, scalar-versus-batch comparisons, shared-object concurrency, serialization state, option matrices, and intermediate molecular state—not only default input/output pairs.

But even this broader validation cannot enumerate every future workflow.

The deeper protection comes from combining the evidence with source-defined implementation semantics.

If a production input lies outside yesterday’s corpus but inside a correctly reproduced source transition, there is a reason to expect the implementation to generalize beyond the exact examples that taught nothing to the implementation in the first place.

That is the important distinction.

Where Operation Contracts Fit

The previous article described another part of COSMolKit’s agent-era design: registered operations and strict state-transition contracts.

Those contracts solve a different problem.

Source reproduction asks:

What chemistry and state transition does RDKit actually define?

The operation system asks:

How is that transition allowed to interact with COSMolKit’s redesigned Rust molecule architecture?

An upstream function may define a specific hydrogen-removal transition. The source port reproduces that logic. COSMolKit’s operation system then ensures that the Rust implementation declares the topology edit, mutation authority, mapping, and derived-state obligations required to carry that transition safely through its own value/COW architecture.

So the complete development path is not just source porting:

upstream semantics
        ↓
source-backed Rust implementation
        ↓
operation contract
        ↓
strict development checks
        ↓
focused regression
        ↓
large-scale parity validation

Each layer catches a different class of failure.

Source reproduction protects against inventing the wrong chemistry. Operation contracts protect against dropping state obligations while adapting that chemistry to a different architecture. Strict mode makes many of those obligations executable during agent development. Validation then tests whether the complete observable boundary actually agrees with the reference.

What the ChEMBL Numbers Mean

This brings us back to the numbers at the beginning.

The current COSMolKit ChEMBL 37 record is not meant to say merely:

We tried 2.9 million molecules and they all looked good.

The stronger claim is narrower and more precise:

For the explicitly documented parity-covered surfaces, a source-backed implementation was subjected to the declared exact or numerical comparison boundary over the recorded corpus and option profiles, and the accepted validation currently contains zero blocking mismatch.

The distinction matters.

2,897,804 molecules describes corpus breadth.

2,931,581,192 matching checks describes far more of the observable comparison surface.

The source-reproduction protocol describes where the implementation semantics came from.

The source-bisection protocol describes what happens when the validation finds a difference.

The operation system describes how those semantics are transferred safely into COSMolKit’s redesigned molecular-state architecture.

None of these numbers or mechanisms is sufficient alone.

Together, they form the evidence chain.

From Fast Agent Coding to Controlled Scientific Porting

Coding agents make a previously impractical class of work possible. A small team can now inspect, port, test, and debug enormous mature scientific codebases at a speed that would have been unrealistic only a few years ago.

But the easiest way to use that speed is not necessarily the safest.

If an agent repeatedly learns from final output mismatches, implementation speed can simply accelerate the accumulation of heuristic branches. Corpus size grows, the implementation keeps chasing it, and the remaining development effort becomes difficult to predict. Even extremely high observed agreement cannot guarantee that an unseen production composition does not exercise a rule the implementation never actually understood.

COSMolKit tries to use agents differently.

The agent is not asked to infer RDKit from outputs. It is asked to read the pinned source, preserve the source correspondence, reproduce the relevant transition, pass through explicit operation contracts, and use differential validation as an auditor. When a mismatch appears, the job is not to make the example green; it is to locate the first semantic divergence and finish the port.

That changes the role of the corpus from teacher to examiner.

And it changes the role of the agent from heuristic optimizer to source-constrained implementer.

Conclusion: The Result and the Method Are the Same Story

The current validation results are intentionally large: the complete ChEMBL 37 source, millions of molecular records, billions of compared states and outputs, option matrices, operation composition, batch execution, concurrency, serialization, stochastic paths, force fields, fingerprints, and state-sensitive chemistry. Within their explicitly documented covered boundaries, the current accepted evidence records zero blocking mismatch against pinned RDKit 2026.03.1.

But the point of this article is that the scale of the benchmark is not the core differentiator.

A large corpus can tell us that two implementations agree on a large corpus.

It cannot, by itself, tell us why.

COSMolKit’s answer is to connect the output evidence back to the implementation:

pinned upstream source
        ↓
traceable source reproduction
        ↓
Rust-native architectural adaptation
        ↓
operation contracts + strict checks
        ↓
first-divergence debugging
        ↓
focused regressions
        ↓
large-scale validation

This is why we do not treat 99% or 99.9% agreement as an almost-finished version of parity inside a covered boundary. A remaining mismatch is evidence that some semantic transition is still unexplained.

And it is why reaching 100% is not supposed to require an endless sequence of corpus-specific patches.

The goal is not to teach a Rust implementation to imitate RDKit’s outputs.

It is to port the semantics that produce those outputs, then use the largest practical validation surface to prove that the port survived contact with reality.

That is the distinction COSMolKit is trying to make in Rust cheminformatics.

COSMolKit Resources

Source repository ·Documentation ·Web tools ·Rust crate ·Python package

Rust Cheminformatics State Management and Molecular Mutation

95028 — Fri, 28 Aug 2026 00:00:00 +0000

AI agents have changed the economics of software porting.

A translation task that once required days of manual work can now be pushed forward at remarkable speed. Large call chains can be traced, C++ functions can be translated into Rust, tests can be generated, and mismatches can be investigated in parallel.

For Rust cheminformatics , this is exciting.

It is also dangerous.

The hardest part of porting a mature chemistry library is often not translating the visible algorithm. It is preserving the decades of implicit state semantics surrounding that algorithm.

RDKit has been exercised by real users, real datasets, and real combinations of operations for many years. Its state-management model is not always what we would design from scratch today, but its behavior has been shaped by enormous practical exposure.

An AI agent porting one function into Rust does not inherit that history automatically.

It may reproduce the visible control flow perfectly and still miss something like:

this operation changes atom identity
this property must be remapped
this cache survives
this one must be cleared
this error path leaves partial state
this stereochemical state is still observable
this non-sanitizing branch preserves explicit valence

That leads to one of the central engineering problems in COSMolKit:

How do we scale agent-driven source porting without scaling hidden molecular-state bugs at the same rate?

Our answer is not to assume that the agent will always remember every lifecycle rule.

Instead, we try to move as many of those obligations as possible into executable architecture.

That is the purpose of the COSMolKit operation system.

The Bottleneck Has Moved

Before modern coding agents, porting a mature cheminformatics routine was expensive largely because writing the code itself was expensive.

A human developer would spend substantial time reading the source, translating data structures, rewriting control flow, compiling, debugging, and adding tests.

That slowness had an accidental benefit: the developer often accumulated a large mental model of the surrounding implementation while working.

The situation is different now.

An agent can process source much faster than a human can deeply review every state transition.

The bottleneck moves from:

Can we write the port?

to:

Can we trust the port to preserve every hidden state obligation?

That distinction matters enormously for cheminformatics.

A molecular operation rarely changes only the thing named in its API.

Consider hydrogen removal.

At first glance:

RemoveHs
=
find hydrogen atoms
+
delete them

But deleting explicit hydrogen atoms changes the molecular topology.

That can affect:

atom indices
bond indices
coordinates
atom-indexed properties
bond-indexed properties
stereo references
explicit valence
implicit hydrogen state
rings
aromaticity
computed chemistry state
cached representations

An agent can translate the hydrogen-removal loop correctly and still get the resulting molecule wrong.

This is the difference between:

source control flow copied correctly

and:

molecule lifecycle copied correctly

The first is a local code problem.

The second is a system problem.

Mature Libraries Contain More Semantics Than Their APIs Reveal

RDKit is a mature C++ codebase.

Some of its behavior is explicit in algorithmic code. Some of it is encoded through object state, property caches, ordering conventions, mutation timing, helper functions, legacy compatibility, and the sequencing of multiple internal operations.

That accumulated behavior has survived years of production use.

A new Rust implementation does not get the same guarantee merely because its code looks cleaner.

In fact, a cleaner-looking rewrite can be wrong precisely because it simplifies behavior that turned out to matter.

For example, an agent might reason:

topology changed
→ valence cache is stale
→ invalidate valence

That sounds reasonable.

But if the reference implementation intentionally updates property state before the topology edit and preserves specific resulting values afterward, then generic invalidation is not equivalent behavior.

The problem is subtle:

A molecular state transition can be correct only relative to the semantics of the operation being reproduced.

There is no universal rule saying that every topology edit should clear every derived value.

There is also no safe universal rule saying that apparently unaffected state should be preserved.

The correct transition depends on the source-defined behavior.

This is why source-backed porting and state-management contracts have to work together.

Agent Speed Magnifies Hidden-State Risk

Agents are particularly good at translating explicit structure.

They can usually handle transformations such as:

std::vector → Vec
pointer traversal → indexed/reference access
exception → Result
for loop → Rust loop or iterator
enum integer → typed Rust enum

These are visible in the source.

The more dangerous obligations often live one level above the visible implementation:

after this edit, coordinates must follow a new atom mapping

ring state is still valid only under a particular structural condition

stereo state must be recomputed because ligand identity changed

this operation is allowed to modify topology but not properties

this source error occurs after some mutation has already happened

this cache is semantic state, while that cache is only a performance optimization

These are easy to lose when the unit of work is “port this function.”

A prompt such as:

Please consider all dependent molecular state.

is not a sufficient architecture.

The better approach is:

Make dependent-state obligations explicit enough that the implementation cannot quietly ignore them.

That is the direction of COSMolKit’s operation system.

Do Not Make Correctness Depend on the Agent Remembering Everything

The goal of the operation framework is not to make agents infallible.

It is to reduce the number of correctness properties that depend on trusting the agent.

Public mutation-capable molecule operations are registered through the COSMolKit operation system.

Each operation specification describes more than its function name.

The current registry model includes fields such as:

method
implementation function
domain
operation kind
topology edit
block access
mutation surface
automatic remapping
derived-state effects
semantic preconditions
mapping requirements
support policy
parity policy
I/O roundtrip metadata
invariant profile

The exact machinery will continue to evolve, but the architectural idea is stable:

Before an operation is implemented, its authority and obligations should be declared.

That changes the review model.

Instead of asking a reviewer to infer from hundreds of lines of code:

What does this operation think it is allowed to mutate?

the registry should answer directly.

And instead of trusting that the implementation remembered all affected derived state, the operation contract should describe what must happen to it.

The registry is therefore not just metadata.

In strict builds, important parts of it have execution owners and runtime checks.

Mutation Authority Is a Capability

One of the most important registry concepts is access.

For each molecular block that an operation may touch, access is classified as:

none
read
write

This is the authoritative block capability.

If an operation is declared read-only for coordinates, its implementation should not suddenly mutate coordinates because doing so was convenient.

If it has no access to properties, property mutation is a contract violation.

If the operation actually needs more authority, the correct fix is not to bypass the framework.

The contract must be updated and reviewed.

This creates a useful separation:

chemistry code:
what transition should happen?

operation contract:
what state may this operation touch?

This is especially valuable in an agent-driven codebase.

An agent can make an implementation mistake.

But a mistake that crosses a declared capability boundary can be detected structurally rather than waiting for a chemically visible failure.

`OpParts`: The Agent Does Not Receive the Whole Mutable Molecule

COSMolKit operation bodies do not receive unrestricted mutable access to all molecule internals.

They operate through an internal capability object called OpParts.

OpParts is responsible for infrastructure such as:

cheap working-value creation
copy-on-write block detachment
registry-derived access capabilities
mutation permission checks
topology mapping
registered remapping
cache transition tracing
derived-effect tracking
operation finalization

It is deliberately not responsible for chemistry.

OpParts should not decide:

which hydrogens are removable
how aromaticity is perceived
how CIP ranking works
how a ring algorithm behaves
how sanitization should proceed

Those rules belong to source-backed chemistry implementations.

The separation can be viewed as:

                Pinned Source
                     |
                     | defines chemistry
                     v
             Operation Body
                     |
                     | requests state changes
                     v
                 OpParts
          +----------+----------+
          |          |          |
       access     mapping    derived state
          |          |          |
          +----------+----------+
                     v
                  finish()
                     |
                     v
                 strict CI

The chemistry implementation says what should happen.

The operation framework controls how that transition is allowed to touch shared molecular state.

This distinction is one of the main safeguards against agent-generated architectural drift.

Rust Privacy Is Part of the Guardrail

A design rule is much stronger when violating it requires fighting the language.

COSMolKit therefore keeps the internal working Molecule inside the private operation runtime.

Operation bodies are expected to receive &mut OpParts, not raw mutable molecule internals.

Helpers called by operation bodies should consume narrowed representations such as:

MoleculeReadParts
atom slices
bond slices
coordinate blocks
typed assignment plans
typed update plans

rather than using a raw whole-molecule escape hatch.

That boundary exists for a reason.

If every operation body could simply obtain:

&mut Molecule

then the registry would describe one architecture while the implementation quietly used another.

For agent-generated code, such escape paths are especially dangerous because they provide the easiest local solution.

An agent trying to make a test pass will naturally prefer the shortest path available.

The architecture should make the shortest path the correct one.

Write-Owned State Should Have One Timeline

Another subtle rule prevents stale-state bugs inside one operation.

Suppose topology is write-owned.

A dangerous implementation shape would be:

read old topology
      |
      +---------------+
      |               |
mutate working        |
topology              |
      |               |
      v               |
continue reading <----+
old topology

Now the same operation is reasoning from two different molecular timelines.

COSMolKit instead expects reads and writes of a write-owned block to come from the same local owned working value.

Conceptually:

begin topology
      |
      +-- inspect
      +-- calculate
      +-- mutate
      +-- return/commit

Fallible mutation APIs are scoped so the current owned block returns to the working molecule on both success and error.

This matters because agent-generated code frequently uses ? aggressively.

Without a structured mutation scope, it is easy for an early return to leave state in an unexpected intermediate representation.

Strong and Weak Topology Operations

One of the most useful distinctions in the operation system is between strong topology edits and weak topology-state edits.

A strong topology operation changes one or more of:

atom count
bond count
atom ordering
bond ordering
atom identity mapping
bond identity mapping

Examples include:

add atom
remove atom
add bond
remove bond
add/remove hydrogens
renumber atoms
fragment
combine

These operations potentially change the index space used by other molecular state.

A weak topology-state operation keeps atom and bond identities stable but modifies graph state.

Examples include:

kekulize
sanitize
set aromaticity
change formal charge
change bond order
assign stereo from existing topology or coordinates

Weak does not mean harmless.

A weak operation can still invalidate valence, aromaticity, stereo, drawing state, or other derived information.

But it does not require the same identity migration as deleting or renumbering atoms.

This classification gives the system more information than:

"the molecule changed"

And again, this is useful specifically in an agent-driven environment.

An agent should not have to rediscover from scratch whether an operation needs topology remapping.

That fact should be part of the registered operation contract.

Topology Mapping Must Be Explicit

Consider removing atom 4 from a six-atom molecule.

Before the edit:

atom 0 → coordinate row 0
atom 1 → coordinate row 1
atom 2 → coordinate row 2
atom 3 → coordinate row 3
atom 4 → coordinate row 4
atom 5 → coordinate row 5

After compaction:

new atom 4
=
old atom 5

If coordinates are not remapped, the molecule may still be perfectly memory-safe.

The coordinate array may even still have the correct number of rows.

But the chemistry is wrong.

The same problem applies to:

atom property lists
bond property lists
stereo references
substance groups
drawing annotations
index-sensitive derived state

For strong operations, topology mapping is therefore treated as a first-class artifact where required.

Dependent state must be handled explicitly:

remap
recompute
invalidate
drop a separately unsupported capability
or fail

Leaving stale indices behind is not a valid result.

Rust’s borrow checker cannot detect this category of bug.

The operation contract can at least force the implementation to acknowledge that the problem exists.

Block Access and Derived-State Effects Are Different Questions

A particularly important design decision in the current operation system is that state access and derived-state obligations are separate axes.

access answers:

What may this operation read or write?

derived_effects answers:

What must happen to affected derived chemistry state?

These are deliberately not interchangeable.

For example:

access:
    read → topology, derived cache
    write → properties

derived effects:
    recompute → aromaticity
    preserve → rings
    invalidate → drawing

Declaring that ring state should be preserved does not automatically give an operation permission to read or write every cache entry.

Likewise, declaring recomputation does not itself grant unrelated block access.

This separation makes the contract harder to accidentally overinterpret.

Four Derived-State Outcomes

The current molecule operation model classifies affected derived state into four pairwise-disjoint categories:

recompute
preserve
invalidate
operation_defined

Recompute

The operation must produce fresh framework-visible state, or explicitly clear it when the reproduced source behavior leaves no materialized replacement.

Invalidate

The existing value is stale and must be cleared.

Preserve

The previous value remains valid.

Operation-defined

The source requires a state transition that cannot be truthfully described by the other three categories.

This final category is intentionally narrow.

Preservation Should Require Evidence

Imagine an agent ports hydrogen addition.

It reasons:

adding terminal H atoms cannot create a ring
therefore preserve ring state

That may be correct.

But in a large codebase, “probably still valid” is not a strong enough state-management policy.

COSMolKit’s strict operation system can require an approved preservation proof.

For example:

PreservationProof::LeafAtomAppend

can validate structural conditions such as:

old atom identities preserved
old bond identities preserved
new atoms only appended
new atoms are degree-one leaves

before ring information is accepted as preserved.

So the sequence becomes:

agent:
"rings remain valid"

framework:
"show the structural condition that makes that true"

This is precisely the kind of responsibility that should not depend only on an agent’s local reasoning.

Why an Escape Hatch Is Still Necessary

A contract system becomes dangerous if it forces real source semantics into an oversimplified model.

Sometimes the upstream implementation performs a state transition that cannot honestly be described as:

preserve
recompute
invalidate

COSMolKit therefore keeps operation_defined as a narrowly controlled escape hatch.

It delegates the transition mechanism.

It does not waive the correctness obligation.

The current contract permits exactly one use:

valence
in the hydrogen-removal operation family

The registry macro rejects other uses, strict runtime validation repeats that allow-list, and registry tests lock the current decision.

This is a good example of an important project principle:

The architecture must constrain the port, but it must not rewrite source semantics merely to make the architecture prettier.

If RDKit defines an awkward but observable transition, the Rust design should model it honestly.

`RemoveHs(sanitize=false)`: A Real Agent-Era Failure Mode

Hydrogen removal is an excellent example of why this system exists.

A superficially reasonable agent implementation could look like:

identify removable H atoms
delete atoms
delete bonds
remap coordinates
invalidate valence
return

That implementation might even be architecturally clean.

It can still be semantically wrong.

The current validated COSMolKit boundary for RemoveHs(sanitize=false) includes RDKit’s non-strict property-cache update before removal and the surviving explicit-valence and implicit-hydrogen fields afterward.

Those states have to be migrated through the declared topology mapping rather than generically erased.

The current ChEMBL 37 topology-operation phase exercises both value-style and in-place RemoveHs(sanitize=false) branches.

Its accepted result covers:

2,854,376 records
45,669,848 exact matches
0 blocking mismatches

against pinned RDKit 2026.03.1.

The lesson is larger than hydrogen removal.

An agent can produce something that is:

locally reasonable
memory-safe
cleanly written
well tested on simple examples

and still miss source-defined molecular state semantics.

That is exactly the category of mistake the operation system is trying to make harder.

Strict Mode Turns Architecture Into an Executable Guardrail

A design document is useful.

A failing build is stronger.

COSMolKit’s development and CI mode enables strict operation contracts and runtime invariants.

The current development rules require:

cargo check -p cosmolkit-core --features op-contracts-strict
cargo test -p cosmolkit-core --release --features op-contracts-strict

op-contracts-strict enables both operation-contract checks and molecule runtime invariants in development/CI.

That turns several architectural expectations into executable failures.

Examples include conceptually:

mutate an undeclared block
→ strict failure

perform a strong topology edit
without required mapping
→ strict failure

declare preserved derived state
without an approved proof
→ strict failure

declare invalidation
but never clear the affected state
→ strict failure

write derived cache state
without matching effect authority
→ strict failure

The precise enforcement varies by contract field, and the repository explicitly documents which fields are currently runtime-enforced, generated as evidence matrices, or still require operation-specific tests.

That honesty is important.

For example, io_roundtrip is currently registry metadata and operation-specific testing responsibility rather than a universal field-driven runtime runner, while invariant_profile is represented in generated matrices but does not yet select truly distinct profile-specific execution.

A contract system should not claim enforcement it does not actually have.

Strict Mode Is Especially Valuable for Agent-Generated Code

A human developer reading the operation-system design might remember:

don't access working Molecule directly
don't mix independent read and write views
record strong topology edits
return checked-out blocks on errors
clear invalidated state

An agent may remember all of that too.

But it may also forget one rule during a large refactor.

The point of strict mode is to make forgetting less survivable.

This changes the development model from:

prompt says:
"please respect architecture"

to:

architecture says:
"violate this and CI should fail"

That is a much better match for high-throughput agent development.

Agents are extremely useful when the feedback loop is strong.

They are much more dangerous when correctness depends on invisible conventions.

The operation system tries to convert conventions into feedback.

The Goal Is Not to Trust the Agent More

This distinction is worth stating clearly.

COSMolKit’s operation system is not based on the idea that agents can be made trustworthy through better prompts.

Its purpose is closer to the opposite:

Make fewer correctness properties depend on trusting the agent.

The agent can still write the wrong chemistry.

It can still misunderstand an upstream branch.

It can still port a helper incorrectly.

Operation contracts cannot prove that CIP ranking is chemically correct or that a ring-perception algorithm matches RDKit.

They solve another layer:

Did this operation stay inside its declared mutation surface?

Did it produce the required topology mapping?

Did it handle affected derived state?

Did it leave the molecule structurally coherent?

Did its error path respect the operation lifecycle?

That distinction gives COSMolKit a layered correctness model.

Four Different Correctness Layers

The current architecture can be understood as four separate questions.

1. Source reproduction

Did we identify and reproduce the intended upstream state transition?

This is handled by the source-reproduction discipline.

Relevant C/C++ source is retained beside the Rust implementation as review anchors, and behavioral reproduction is tracked explicitly.

2. Operation contracts

Did the implementation touch only the state it declared, and did it declare the necessary migration obligations?

This is the operation registry and OpParts layer.

3. Strict execution

Did the implementation actually fulfill those obligations during development and CI?

This is where contract traces, preservation proofs, access checks, mappings, and invariants become executable guardrails.

4. Differential validation

Does the resulting supported observable behavior match the pinned RDKit reference?

This is the parity layer.

Conceptually:

             RDKit Source
                 |
                 v
         Source-Backed Port
                 |
                 v
          Operation Contract
          +------+-------+
          |      |      |
       access  mapping  effects
          |      |      |
          +------+-------+
                 v
              strict
                 |
                 v
        RDKit parity validation

No single layer replaces the others.

Contracts Do Not Prove Chemistry

This is an important limitation.

A completely contract-valid operation can still implement the wrong chemical rule.

For example:

topology mapping correct
cache invalidation correct
all permissions respected
molecule structurally valid

but:

wrong aromaticity assignment

The operation system would not magically know.

That is why COSMolKit continues to require source-backed behavior and parity validation for supported RDKit-compatible surfaces.

The current project explicitly separates invariant tests from parity tests:

invariants:
is the COSMolKit state internally valid?

parity:
does the supported behavior match RDKit?

A molecule may pass all invariants and still fail parity.

This separation is critical.

Otherwise, a sophisticated architecture could create false confidence.

The Real Enemy Is Composition

Many molecular bugs do not appear when an operation is tested alone.

They appear after composition.

Consider:

parse
 ↓
sanitize
 ↓
remove hydrogens
 ↓
add hydrogens
 ↓
assign stereochemistry
 ↓
generate coordinates
 ↓
calculate fingerprint

Each individual function may produce a plausible result.

The failure may occur only because operation 3 left behind state that operation 6 later trusted.

This category of bug is particularly hard for a fast-moving agent port.

Single-function tests encourage local reasoning.

Production workflows exercise state transitions across long chains.

A mature library like RDKit has accumulated years of exposure to such combinations.

A new implementation cannot wait decades to discover every interaction through users.

Operation contracts are an attempt to move some of that integration discipline earlier.

Instead of:

operation A leaves whatever state it happens to leave

operation B assumes whatever state it happens to receive

the desired model is:

operation A
    ↓
must finish at a declared contract boundary
    ↓
operation B
    ↓
receives a state with explicit lifecycle semantics

This does not eliminate composition bugs.

But it reduces the amount of undocumented state that can leak from one operation into another.

Manufacturing Some of the Discipline That Time Normally Provides

Mature scientific libraries acquire robustness through multiple forces:

careful developers
large user bases
edge-case bug reports
production workflows
years of accidental stress testing

A new Rust toolkit does not have all of that history.

Agent development makes this more extreme because implementation breadth can grow much faster than production exposure.

That creates an imbalance:

code surface
grows quickly

real-world validation history
grows slowly

The operation system is one attempt to close part of that gap.

Not by pretending that contracts replace production experience.

They do not.

But by converting certain classes of integration assumption into explicit, testable structure before users discover them.

A useful way to think about it is:

Operation contracts try to manufacture some of the integration discipline that mature libraries normally acquire only after years of production use.

That is particularly valuable when implementation throughput is agent-amplified.

Release Builds Should Not Pay the Full Contract Cost

Strict architectural verification is useful in development.

Production chemistry should not necessarily pay for all of it.

COSMolKit therefore separates ordinary operation execution from contract-only checking.

Strict development and CI builds can include:

source snapshots needed for contracts
permission assertions
mutation traces
preservation proofs
mapping checks
full invariant scans
finish-time validation

The optimized release build still follows the same:

public wrapper
operation implementation
OpParts mutation route
copy-on-write/in-place path
topology migration
derived-state updates

but omits development-only checking where the contract permits it.

The important rule is:

Release optimization must not switch to a different chemistry algorithm.

Otherwise the project would validate one implementation and ship another.

Strict mode is therefore a development guardrail, not an alternate chemistry engine.

Agent Development Makes Fail-Closed Behavior More Important

There is another related principle.

When an agent encounters an unimplemented source branch, there is a temptation to return something plausible and move forward.

In scientific software, that can be worse than an explicit failure.

A chemically meaningful-looking result may propagate through a long pipeline before anyone notices it was produced by a fallback.

COSMolKit therefore prefers explicit unsupported errors at separately documented capability boundaries rather than:

silent fallback
best-effort approximation
placeholder chemistry

But the distinction must remain strict:

separately unsupported capability
≠
failing row inside a claimed parity boundary

Once a capability is claimed as parity-covered, individual mismatches cannot be carved out after the fact and renamed unsupported.

This is another place where architecture helps contain the tendency of high-throughput development to optimize for immediate green tests.

Source Porting and Operation Contracts Solve Different Problems

It is tempting to think that a line-by-line source port makes the operation system unnecessary.

It does not.

Source porting answers:

What did the upstream implementation do?

The operation system answers:

How is that behavior allowed to interact with COSMolKit’s redesigned molecular state model?

Remember that COSMolKit deliberately does not clone RDKit’s entire object architecture.

It uses:

value-style APIs
explicit in-place mutation
copy-on-write storage
typed state
registered operation boundaries
Rust ownership

So even when the source chemistry is reproduced correctly, the surrounding state lifecycle must be adapted to the new architecture.

That adaptation is exactly where semantic mistakes can appear.

Therefore:

source-backed port
without state contracts
→ chemistry may be correct locally
  but migration may drift

state contracts
without source-backed port
→ state may be coherent
  but chemistry may be wrong

The two disciplines are complementary.

Large-Scale Validation Is the Final Layer, Not the First Design Tool

COSMolKit then validates these implementations against pinned RDKit behavior.

The current ChEMBL 37 validation uses:

2,897,819 source records
2,897,804 mutually parseable records
31 configured phases
3,968 shard tasks

The consolidated evidence currently records:

2,931,581,192 matching checks
0 blocking mismatches

with distance-geometry validation additionally traversing billions of matrix entries.

The important point is not merely the size.

It is the direction of causality.

The intended workflow is:

upstream source
     ↓
source-backed Rust implementation
     ↓
operation-contract enforcement
     ↓
focused regression tests
     ↓
large-scale validation

not:

large corpus
     ↓
observe mismatch
     ↓
invent local patch
     ↓
rerun
     ↓
repeat until green

The corpus verifies the implementation.

It should not become the algorithm.

Agent Throughput Without Semantic Throughput Is Dangerous

AI agents make it possible to move faster than previous scientific software projects.

But implementation throughput is not the same as semantic throughput.

You can add:

more functions
more APIs
more branches
more file formats
more fingerprint families

faster than you can truly understand the interactions between them.

Without architectural constraints:

agent speed
→ implementation surface grows
→ hidden semantic assumptions grow
→ composition risk grows
→ semantic debt grows

The desired alternative is:

agent speed
      +
source-backed porting
      +
operation contracts
      +
strict CI
      +
large parity validation
      ↓
implementation throughput can grow
without semantic debt growing at the same rate

That does not make large-scale agent development automatically safe.

It makes safety a first-class engineering problem rather than an assumption.

A Different Role for Architecture in the Agent Era

Traditional software architecture is often discussed in terms of maintainability:

clean abstractions
modularity
separation of concerns

Those still matter.

But agent-driven development adds another purpose:

Architecture becomes a constraint system for code generation.

A useful architecture does not merely make correct code elegant.

It makes incorrect shortcuts difficult.

For COSMolKit, that means:

no unrestricted mutable molecule in operation bodies

no undeclared mutation authority

no strong topology edit without explicit migration semantics

no silent cache preservation without evidence

no easy unsupported fallback inside a claimed boundary

no contract-sensitive development signoff without strict mode

This is a different way to think about software design.

The architecture is not only for humans who read the code later.

It is also part of the feedback environment in which agents write the code now.

The Goal Is Fewer Trust Assumptions

AI agents are extraordinarily useful for source analysis, translation, testing, debugging, and large-scale engineering.

COSMolKit uses that capability aggressively.

But the correct response to faster code generation is not weaker engineering discipline.

It is stronger executable discipline.

For a mature domain such as cheminformatics, the difficult knowledge is already distributed across:

upstream source
state transitions
error behavior
operation ordering
cache semantics
real-world edge cases

An agent can help reproduce that knowledge.

It should not be expected to hold all of it implicitly at once.

So the goal of the COSMolKit operation system is not:

Make the agent trustworthy.

It is:

Make fewer correctness properties depend on trusting the agent.

Source-backed ports define the intended chemistry.

Operation contracts define mutation authority and state obligations.

Strict execution turns those rules into development-time failures.

Parity validation checks whether the observable result still matches the reference.

Together, these layers provide a path toward something that matters increasingly in the agent era:

high-throughput scientific software development without treating semantic correctness as an afterthought.

For Rust cheminformatics, that may be one of the most important architectural problems to solve.

COSMolKit Resources

Source repository ·Documentation ·Web tools ·Rust crate ·Python package

Rust Cheminformatics Beyond RDKit Bindings: Redesigning Molecular APIs for Rust

95028 — Sun, 23 Aug 2026 01:00:00 +0000

Most discussions about Rust cheminformatics begin with the same question:

How do we bring RDKit functionality into Rust?

The obvious answers are bindings, FFI, or a direct API clone. All three are useful approaches.

When we started building COSMolKit, however, we became interested in a different question:

If the chemistry must remain compatible with RDKit, does the software architecture have to remain compatible with RDKit too?

Our answer is no.

RDKit contains more than two decades of accumulated cheminformatics knowledge: edge cases, ordering rules, stereochemical behavior, file-format semantics, cache transitions, error paths, and production experience that would be extremely difficult to rediscover independently.

That is exactly the part worth preserving.

But preserving chemistry semantics does not require reproducing every ownership pattern, mutation convention, cache lifetime, or object-model decision inherited from a mature C++ codebase.

That distinction became the foundation of COSMolKit:

preserve difficult-to-rediscover chemistry semantics
                         ↓
                redesign ownership
                         ↓
               make mutation explicit
                         ↓
          control molecular state transitions
                         ↓
             scale naturally to batches

The goal is not “RDKit with Rust syntax.”

It is to ask what a modern molecular toolkit can look like if reference chemistry and software architecture are treated as separate design problems.

Molecular Software Has Two Correctness Problems

A cheminformatics toolkit has at least two different responsibilities.

The first is chemical correctness :

Is aromaticity correct?
Is stereochemistry correct?
Does canonicalization match?
Are fingerprints correct?
Does hydrogen handling reproduce the reference?

The second is software-state correctness :

Did this operation mutate the source unexpectedly?
Are coordinates still aligned with atom indices?
Is cached ring information still valid?
Did stereo state survive a topology edit correctly?
Does a failed operation leave a coherent molecule?

These problems are related, but they are not the same.

A molecule can be perfectly memory-safe and still contain stale chemistry state.

It can have a coordinate matrix with exactly the right dimensions whose rows now correspond to the wrong atoms.

It can have valid bond indices while retaining stereochemistry derived from an earlier topology.

Rust helps enormously with the first layer of software safety: ownership, lifetimes, aliasing, and memory safety.

But Rust’s borrow checker cannot answer:

Is this cached molecular state still semantically valid after this chemical operation?

That requires a higher-level model.

This distinction drives much of COSMolKit’s architecture:

chemical correctness
        ↓
source-backed reference semantics

software-state correctness
        ↓
value semantics
explicit mutation
controlled state transitions
mapping and invalidation

Rust Cheminformatics Should Not Hide Mutation

Molecules look deceptively simple as objects.

A molecule has atoms and bonds. Add hydrogens. Remove hydrogens. Kekulize it. Generate coordinates. Sanitize it.

But each of those operations can affect much more than its name suggests.

Removing an atom may affect:

atom and bond indices,
coordinates,
atom- and bond-indexed properties,
stereochemistry,
ring state,
valence state,
aromaticity state,
adjacency,
and downstream computed representations.

This makes hidden mutation particularly dangerous in cheminformatics.

COSMolKit therefore makes the default molecular workflow value-oriented:

let mol = Molecule::from_smiles("CCO")?;

let mol_h = mol.with_hydrogens()?;
let mol_2d = mol_h.with_2d_coordinates()?;

Conceptually:

mol
 │
 ├── with_hydrogens() ──────> mol_h
 │
 └── remains unchanged

mol_h
 │
 └── with_2d_coordinates() ─> mol_2d

A transformation produces another molecular value.

The source remains observable as the value it represented before the transformation.

This makes scientific workflows easier to reason about:

raw
 ↓
sanitized
 ↓
hydrogenated
 ↓
embedded
 ↓
optimized

Each stage has an explicit identity.

Intermediate states can be inspected, compared, cached, branched, or reused without asking which earlier object may have been silently modified.

The public promise is value semantics.

The implementation is free to optimize how those values share physical storage.

Value Semantics Do Not Require Deep Copies

A naïve value-oriented molecule API could be prohibitively expensive.

A molecule may contain topology, coordinates, conformers, properties, and derived state. Deep-copying all of that for every transformation would turn a cleaner interface into a performance penalty.

COSMolKit therefore separates semantic identity from storage identity.

Large pieces of molecular state can be shared internally and detached only when an operation actually needs to write them.

Conceptually:

Molecule A
 ├── topology ───────┐
 ├── coordinates ────┼──── shared
 └── properties ─────┘

        with_hydrogens()

Molecule B
 ├── topology' ← changed
 ├── coordinates' ← remapped when necessary
 └── properties ───── shared if unchanged

The important distinction is:

public contract:
    molecule values do not visibly alias

implementation strategy:
    unchanged storage may be shared

Copy-on-write is therefore an optimization, not a public semantic promise.

Users should not need to know whether an internal Arc detached during an operation.

They should be able to rely on something simpler:

Transforming one molecular value must not unexpectedly alter another one.

This is one place where Rust’s ownership model is more than an implementation detail. It provides a natural foundation for separating what a molecular value means from where its bytes happen to live.

In-Place Mutation Still Matters

Value semantics are a strong default, but there are workloads where preserving an old molecular value is unnecessary.

Rust cheminformatics also needs an efficient mutation path.

COSMolKit uses one deliberately simple rule:

Every public in-place Molecule operation ends in _, and _ has no other public Molecule meaning.

For example:

let next = mol.with_hydrogens()?;

versus:

mol.add_hydrogens_()?;

The convention extends naturally:

mol.remove_hydrogens_()?;
mol.kekulize_()?;
mol.sanitize_()?;
mol.compute_2d_coordinates_()?;

This is deliberately boring.

That is the point.

A reader should be able to identify mutation from the call site without remembering operation-specific conventions.

value style

mol2 = mol.with_hydrogens()

source preserved
new value returned

in-place style

mol.add_hydrogens_()

receiver may change
mutation obvious at call site

The two interfaces are not intended to become separate chemistry implementations.

They pass through the same operation machinery and chemistry logic.

The difference is ownership behavior.

A value-style operation preserves the source and detaches writable state when necessary.

An in-place operation can mutate uniquely owned storage directly.

So:

Choosing the in-place API changes ownership behavior, not the intended chemistry semantics.

The error contract is explicit as well. In-place operations are not generally transactional. If the underlying algorithm fails after partial mutation, the molecule remains internally complete, but it is not necessarily restored to its previous value.

When failure-preserving behavior matters, the value-style interface is the appropriate choice.

The Hard Problem Is Everything Mutation Invalidates

The real difficulty begins once a molecular operation changes state.

A conventional implementation can easily accumulate rules like:

remove atom
→ rebuild adjacency

change bond
→ clear rings

remove H
→ update valence somewhere

renumber atoms
→ remember coordinates

change coordinates
→ maybe update stereo

Every individual rule may be reasonable.

The problem is that molecular-state management becomes distributed across dozens of functions.

Eventually correctness depends on every operation remembering every consequence.

COSMolKit instead routes public mutation-capable operations through a registered operation system.

At a high level, an operation declares things such as:

what state it may access,
what state it may mutate,
what kind of topology edit it performs,
what dependent data requires migration,
what derived state is affected,
what support/parity boundary applies.

This changes the architecture from:

operation implementation
    decides chemistry
    decides mutation authority
    decides mapping
    decides invalidation
    decides preservation

toward:

operation contract
    declares authority and obligations

operation body
    implements chemistry

operation framework
    controls state transition

The difference matters because chemistry implementations should answer chemical questions.

They should not individually reinvent the lifecycle rules for the entire molecule.

From Memory Safety to Molecular-State Safety

This is the easiest way to understand the operation system.

Rust already asks:

Who is allowed to mutate this memory?

COSMolKit adds another question:

Which molecular operation is allowed to mutate this class of chemical state?

That higher-level capability is represented internally through the operation system and its OpParts execution boundary.

Operation bodies do not simply receive unrestricted mutable access to the entire molecule.

Instead, they work through state capabilities derived from the registered operation.

Conceptually:

          Operation Contract
                 |
                 v
              OpParts
         +-------+--------+
         |       |        |
     topology coordinates properties
         |       |        |
         +---- controlled +
                 |
                 v
          chemistry logic
                 |
                 v
              result

The chemistry implementation still decides how hydrogen handling, stereochemistry, ring perception, or sanitization behaves.

The framework decides which state that implementation is allowed to touch and how the surrounding molecular value must be migrated.

Rust protects memory safety.

The operation system is intended to protect molecular-state safety.

Topology Changes Need Explicit Consequences

Not every molecular operation changes topology in the same way.

Some operations change atom or bond identity:

add/remove atoms,
add/remove bonds,
add/remove explicit hydrogens,
renumber,
fragment,
combine.

Others keep atom and bond identity stable while changing graph state:

kekulize,
sanitize,
change bond order,
change formal charge,
assign aromaticity,
assign stereochemistry.

The distinction matters because identity-changing operations can invalidate the index space used by other data.

Consider coordinates.

Before removing an atom:

atom 0 → coordinate row 0
atom 1 → coordinate row 1
atom 2 → coordinate row 2
atom 3 → coordinate row 3
atom 4 → coordinate row 4
atom 5 → coordinate row 5

Remove atom 4, compact the table, and:

new atom 4 = old atom 5

The coordinate row must follow the same mapping.

The same problem applies to:

atom properties,
bond properties,
stereochemical references,
SDF property lists,
structural annotations,
and any other index-sensitive state.

After an identity-changing topology operation, dependent state must be:

remapped
recomputed
invalidated
explicitly dropped when outside support
or rejected

“Leave it there” is not a valid option.

Derived State Needs More Than `clear_cache()`

A simple molecular-state strategy would invalidate everything after every topology edit.

That is safe in some cases, inefficient in others, and sometimes semantically wrong relative to the reference behavior.

The opposite strategy—preserve everything that appears unaffected—is more dangerous.

COSMolKit therefore models different derived-state outcomes explicitly.

At the conceptual level, an affected state may be:

recomputed
preserved
invalidated

and rare source-defined transitions may require more specialized treatment.

One detail is especially important: preservation should not be a casual annotation.

Suppose adding explicit hydrogens should preserve previously computed ring information.

An implementation could simply declare:

rings remain valid

But a stronger design asks why.

For example, if the topology change can be structurally established as only appending degree-one leaf atoms while preserving the old graph identity, then keeping the old ring state can be justified by a concrete preservation condition.

Conceptually:

operation:
"rings remain valid"

framework:
"under what structural condition?"

This is a much stronger model than scattering clear_cache() calls throughout chemistry code.

The detailed derived-state permission model deserves its own article; the key point here is that state transitions become part of the operation contract instead of invisible local convention.

Development Contracts Without the Runtime Tax

At this point an obvious objection appears:

Doesn’t all of this checking make every molecular operation expensive?

It would, if the complete development contract were always part of production execution.

COSMolKit separates the chemistry/state-transition path from development-only verification.

In strict development and CI builds, the operation framework can check things such as:

mutation authority,
mapping obligations,
derived-state transitions,
preservation conditions,
molecule invariants,
operation finalization.

The default optimized release build follows the same chemistry implementation and the same state-migration path, but omits contract-only tracing and assertions where appropriate.

Conceptually:

development / CI

chemistry
   +
state migration
   +
contract verification
   +
invariant checking

release

chemistry
   +
same state migration

The important rule is:

Optimization mode must not select a different chemistry or state-transition algorithm.

Otherwise the project would validate one implementation and ship another.

The contract system is a development guardrail, not a second chemistry engine.

Redesign Does Not Mean Reinterpreting Chemistry

All of this architectural freedom raises the obvious question:

If COSMolKit redesigns ownership, mutation, and molecular state, how do we know it has not quietly changed the chemistry?

This is where we draw a hard boundary between architecture and semantics.

For compatibility-sensitive behavior, COSMolKit uses source-backed reproduction against pinned upstream implementations.

Relevant upstream logic is retained alongside corresponding Rust code as review anchors, and compatibility claims are attached to explicit behavioral boundaries rather than inferred from API similarity.

The architecture may change:

C++ object mutation
        ↓
Rust value-style operation

or:

source container
        ↓
different Rust data structure

but the declared observable semantics must remain compatible.

This leads to one of the core principles of the project:

Compatibility should constrain observable semantics, not prevent architectural improvement.

And this is not only a design intention.

For surfaces declared parity-covered, COSMolKit validates against pinned RDKit 2026.03.1 over explicit comparison boundaries. The current ChEMBL 37 validation uses 2,897,819 source records and records more than 2.9 billion matching checks with zero blocking mismatch across the consolidated covered surfaces. A mismatch inside a declared parity boundary remains unfinished compatibility rather than being absorbed into an aggregate success percentage.

The exact validation methodology deserves a separate article. The important point here is simpler:

source semantics
        ↓
redesigned Rust architecture
        ↓
independent differential evidence

The redesign is not asking readers to trust architectural intent alone.

Value Semantics Become Batch Semantics

The same design choices become especially useful once the unit of work is no longer one molecule.

Rust makes parallel iteration easy.

That does not automatically make a cheminformatics API batch-native.

Real dataset processing also needs:

stable input ordering,
per-record failures,
selection and filtering,
output correspondence,
parallel scheduling,
reproducible composition.

The value-oriented molecular model gives this abstraction a natural foundation.

If transformations already operate as:

molecule value
      ↓
new molecule value

then a batch can extend the same idea:

ordered molecular values
          ↓
ordered transformed values
          +
per-record failure state

COSMolKit exposes this through MoleculeBatch.

For example:

batch = MoleculeBatch.from_smiles_list(
    ["CCO", "c1ccccc1", "not-smiles"],
    errors="keep",
).with_parallel_jobs(8)

prepared = (
    batch
    .with_hydrogens(errors="keep")
    .with_2d_coordinates(errors="keep")
)

print(prepared.valid_mask())
print(prepared.errors())

The important abstraction is not merely multiple threads.

It is that:

10,000 molecules
+
37 failures

can remain a structured dataset state instead of becoming an exception-handling accident.

Record correspondence remains visible:

input[0] ───────────── output[0]
input[1] ───────────── output[1]
input[2] ─ failed ──── error[2]
input[3] ───────────── output[3]

This is why batch processing belongs in the architectural story.

Value semantics are not only about making scalar code aesthetically cleaner.

They provide a much better foundation for ordered, isolated, failure-aware transformations over large molecular collections.

The batch layer is still evolving, and not every batch surface is claimed as fully closed. But the direction follows naturally from the same state model rather than being added later as a collection of parallel wrappers.

What “Rust-Native” Means Here

A narrow definition of Rust-native cheminformatics would be:

no C++
+
builds with Cargo
=
Rust-native

That is useful, but incomplete.

For COSMolKit, Rust-native also means taking Rust’s software model seriously:

ownership should be visible;
molecular values should not alias unexpectedly;
mutation should be explicit;
writable state should have controlled authority;
index-changing edits should have migration semantics;
derived state should have explicit lifetimes;
failure behavior should be visible;
batch workflows should be structural rather than incidental.

None of this makes the chemistry “more Rust-like.”

That is not the objective.

The objective is to make chemical software easier to reason about without giving up the semantics that took mature libraries decades to establish.

That brings us back to the two correctness problems:

chemical correctness
+
software-state correctness

Reference semantics address the first.

Rust ownership and COSMolKit’s state architecture address the second.

A robust molecular toolkit needs both.

Beyond RDKit Bindings

FFI remains an excellent solution when an application simply needs mature RDKit functionality from Rust.

COSMolKit is exploring a different design space.

The aim is to preserve source-defined chemistry where compatibility matters while being willing to redesign the system around it:

      RDKit-compatible semantics
                |
                v
       source-backed chemistry
                |
                v
       Rust molecular values
                |
        +-------+--------+
        |       |        |
     explicit  state   copy-on-write
     mutation  rules
        |       |        |
        +-------+--------+
                v
      scalar + batch workflows

That is what “beyond bindings” means here.

Not abandoning RDKit.

Not pretending decades of chemical edge cases can be reconstructed from a feature list.

And not reproducing a historical object architecture merely because we want the chemistry implemented inside it.

The opportunity for Rust cheminformatics is to separate those concerns:

Keep the semantics that are difficult to rediscover. Redesign the architecture that we now know how to make safer.

The next articles in this series go deeper into the two systems that make this separation practical: executable operation contracts for agent-driven molecular-state changes, and source-backed porting discipline for reproducing RDKit semantics without accumulating heuristic semantic debt.

DEV Community: cosmol-studio

Rust Cheminformatics Validation: From ChEMBL 37 to Billions of Exact RDKit Comparisons

The Current Validation Snapshot

Parity Is Boundary-Scoped

Three Corpus Layers, Three Different Jobs

One Molecule Can Produce Hundreds of Comparisons

Exact Does Not Always Mean String Equality

Validation Must Survive Composition

A Validation Run Has an Identity

The Most Useful Result Was a Failure

Why Billions of Comparisons Matter

From Source Semantics to Validation Evidence

COSMolKit Resources

Rust Cheminformatics: Porting RDKit Source Semantics to Rust

Two Ways to Build an RDKit-Compatible Implementation

Corpus Agreement and Semantic Reproduction Are Different Claims

The Corpus Should Audit the Implementation, Not Write It

A Feature Name Is Not a Specification

What “Line-by-Line Port” Means

Behavior and Performance Are Tracked Separately

Follow the Source Until the Behavior Actually Lives

Even Language-Level Behavior May Matter

When the Large Corpus Finds a Failure

Patch the First Divergence, Not the Last Symptom

Large-Corpus Failures Become Small Semantic Regressions

Why Development Progress Becomes More Predictable

Why This Matters in Real Applications

Where Operation Contracts Fit

What the ChEMBL Numbers Mean

From Fast Agent Coding to Controlled Scientific Porting

Conclusion: The Result and the Method Are the Same Story

COSMolKit Resources

Rust Cheminformatics State Management and Molecular Mutation

The Bottleneck Has Moved

Mature Libraries Contain More Semantics Than Their APIs Reveal

Agent Speed Magnifies Hidden-State Risk

Do Not Make Correctness Depend on the Agent Remembering Everything

Mutation Authority Is a Capability

OpParts: The Agent Does Not Receive the Whole Mutable Molecule

Rust Privacy Is Part of the Guardrail

Write-Owned State Should Have One Timeline

Strong and Weak Topology Operations

Topology Mapping Must Be Explicit

Block Access and Derived-State Effects Are Different Questions

Four Derived-State Outcomes

Recompute

Invalidate

Preserve

Operation-defined

Preservation Should Require Evidence

Why an Escape Hatch Is Still Necessary

RemoveHs(sanitize=false): A Real Agent-Era Failure Mode

Strict Mode Turns Architecture Into an Executable Guardrail

Strict Mode Is Especially Valuable for Agent-Generated Code

The Goal Is Not to Trust the Agent More

Four Different Correctness Layers

1. Source reproduction

2. Operation contracts

3. Strict execution

4. Differential validation

Contracts Do Not Prove Chemistry

The Real Enemy Is Composition

Manufacturing Some of the Discipline That Time Normally Provides

Release Builds Should Not Pay the Full Contract Cost

Agent Development Makes Fail-Closed Behavior More Important

Source Porting and Operation Contracts Solve Different Problems

Large-Scale Validation Is the Final Layer, Not the First Design Tool

Agent Throughput Without Semantic Throughput Is Dangerous

A Different Role for Architecture in the Agent Era

The Goal Is Fewer Trust Assumptions

COSMolKit Resources

Rust Cheminformatics Beyond RDKit Bindings: Redesigning Molecular APIs for Rust

Molecular Software Has Two Correctness Problems

Rust Cheminformatics Should Not Hide Mutation

Value Semantics Do Not Require Deep Copies

In-Place Mutation Still Matters

The Hard Problem Is Everything Mutation Invalidates

From Memory Safety to Molecular-State Safety

Topology Changes Need Explicit Consequences

Derived State Needs More Than clear_cache()

`OpParts`: The Agent Does Not Receive the Whole Mutable Molecule

`RemoveHs(sanitize=false)`: A Real Agent-Era Failure Mode

Derived State Needs More Than `clear_cache()`