Scientific_fastapi: How I Organize Reproducible Scientific Computing Through an API

Author: Artem — Artem7898

A scientific algorithm usually starts as a research script or notebook. While one person runs the calculations, parameters can be changed directly in the code and results saved manually. Once other users become involved, additional requirements emerge: validating input, managing concurrent tasks, recovering from failures, and understanding exactly how a result was produced.

With scientific_fastapi, I am developing infrastructure for this transition. The project connects a Python computational core, an HTTP API, background task execution, and scientific artifact storage. In this article, I describe the architecture of v2.0.0, its capabilities, and practical applications.

The central idea is to make each computational run an identifiable, verifiable unit. Every run should have input parameters, an identifier, an execution state, a result, and enough information to investigate how that result was obtained.

This is particularly important for scientific services. A number returned by an API gains meaning through the model description, units, assumptions, and uncertainty estimate that accompany it.

From Request to Result

FastAPI provides the HTTP layer. It accepts requests and validates parameters before computation begins. Inputs follow a strict schema: unknown fields, unsupported values, and invalid numeric inputs are rejected. Additional limits control the computational workload and the number of accepted tasks.

Long-running calculations execute in the background. Once a request is accepted, the client receives HTTP 202 Accepted, a task identifier, and a URL for checking its status. The client can then poll for updates:

pending → running → completed
                  → failed
                  → cancelled

This contract works for notebooks, web interfaces, and other services. A client submits parameters and waits for the result without keeping an HTTP connection open throughout the calculation.

I separate the architecture into four responsibilities:

• The API validates requests and exposes task states.
• The computational core implements the scientific model.
• The background execution mechanism manages tasks and execution attempts.
• Artifact storage preserves results and run descriptions. This separation allows scientific functions to be tested independently of HTTP and reused in research scripts.

A Scientific Model for Testing the Infrastructure

The current reference computational core is a two-dimensional model of independent particle velocities governed by an Ornstein–Uhlenbeck thermostat. Its identifier is ideal-gas-ou-v1.

The model has analytical properties that make it useful for validating the numerical implementation. It supports investigating temperature relaxation, the statistics of random realizations, and the relationship between uncertainty and particle count.

Calculations use reduced units:

m = k_B = 1

At a constant thermostat temperature, the exact transition for a velocity component is:

a = exp(-dt)
v_next = a * v + sqrt(T_bath * (1 - a²)) * ξ

Here, ξ is an independent standard normal random variable. The relaxation rate in this expression is one. This transition corresponds to the solution of the Ornstein–Uhlenbeck process over the specified time interval.

The physical meaning of the model also defines its scope. It describes the velocities of independent particles. Full molecular dynamics would require positions, interactions, forces, and an appropriate integrator. Temperature and time are expressed in reduced units; conversion to kelvin or femtoseconds requires a separate physical justification.

What Reproducibility Means in This Project

Reproducibility in scientific_fastapi combines several mechanisms.

Random numbers come from a local PCG64 generator with an explicitly specified seed. A run does not depend on the process-wide random number generator state. This makes it possible to reproduce numerical results with identical parameters in a consistent computational environment.

Dependencies are fixed in uv.lock. Installation and build checks use the locked versions, and the build tools are pinned as well. Export to the PEP 751 format is available for sharing an environment specification.

Each scientific run saves a JSON description and its final state in NPZ format. The artifacts record parameters, numerical results, the model version, and information about the random number generator and environment. SHA256 checksums allow the consistency of the saved files to be verified.

As a result, an output can be connected to a specific set of conditions: which parameters were used, which model performed the calculation, and which files belong to that run.

An identical seed alone does not guarantee bitwise equality across different libraries, hardware, and computational backends. Verifying reproducibility requires considering the complete computational context.

I also tie uncertainty estimates to the specific model. In the reference calculation, the uncertainty concerns temperature estimated from a final snapshot of independent particles. Integrating a different algorithm requires its own justification of uncertainty.

Recovering from Failures During Computation

An in-memory backend is available for local use. Redis supports shared task state and task recovery.

The Redis backend stores task states and execution information. A worker acquires a task for a limited period and periodically confirms that it is still running. If those confirmations stop, the task can be recovered and assigned to another execution attempt.

This uses at-least-once delivery: a calculation may execute again after a failure. The project therefore distinguishes between two identifiers:

• task_id identifies the user’s task.
• attempt_id identifies a specific execution attempt. Artifacts from different attempts are stored separately. An additional ownership check protects result publication from an outdated worker.

This is particularly useful when an application restarts or loses its Redis connection. If an integrated algorithm performs external actions, such as modifying another database, the possibility of repeated execution must be handled separately.

Deployments with multiple API instances also require shared artifact storage. Redis coordinates tasks, while access to result files is addressed at the deployment level.

Observing the Service

For a computational system, it is useful to distinguish between time spent waiting and time spent calculating. A slow response may originate in the queue, the computational core, or the storage infrastructure.

scientific_fastapi therefore provides an optional OpenTelemetry integration. It connects the HTTP request, task enqueueing, and computation, including context propagation into worker threads. Metrics capture waiting time, execution duration, and task outcomes.

MLflow can be enabled for experiment tracking. It allows run parameters and artifacts to be recorded in a dedicated tracking system.

Redis, MLflow, observability tools, notebook tools, and research dependencies are provided through optional extras. Users can select the components their scenario requires while keeping the core dependency set small.

Where the Project Can Be Applied

The first application is teaching computational physics and statistical modeling. The reference model can be used to study relaxation, random realizations, and changes in statistical uncertainty. The same computational code can serve an educational interface, a notebook, and automated checks.

The second is preparing reproducible computational experiments. The research pipeline saves results, figures, and a manifest containing checksums. This helps connect a published figure to its input parameters and verify that the data can be regenerated in a consistent environment.

The third is a shared computational service for a research group. Multiple users submit tasks over HTTP, track their status, and retrieve results through a common contract. This is useful when people need to access the same computational code from different client tools.

The fourth is a foundation for an API around a custom scientific algorithm. The reference core can be replaced with a validated solver, experimental data processing, or a scientific machine learning model. Task management, observability, and artifact storage provide a foundation for this work. Each new core still needs an input contract, resource requirements, and scientific validation methods.

The fifth is controlled comparison of execution approaches. The project includes benchmarks for NumPy, threads, processes, Numba, and JAX. Comparisons use a fixed workload, check numerical consistency, and separate warmup from repeated measurements.

For me, such benchmarks are tools for choosing an implementation for a particular task. Small arrays may be dominated by overhead; other workloads may be limited by data transfer or compilation costs. Free-threaded Python likewise requires measurements on the selected computation: changing the GIL configuration alone does not determine overall performance.

What This Approach Provides

In v2.0.0, checks cover the scientific model, HTTP contract, Redis task recovery, artifact isolation, and optional integrations. CI validates Python 3.12, 3.13, and 3.14, installation of the built wheel, and container behavior after restart. Releases are published after successful checks for the corresponding commit, with a wheel, source distribution, and SHA256SUMS attached.

I see scientific_fastapi as a foundation for scientific services with clearly defined responsibilities between the computational model and its infrastructure. The model determines the physical and mathematical meaning of the result. The infrastructure handles request validation, controlled execution, data preservation, and the ability to investigate how that result was produced.

This approach becomes useful when other people begin using scientific code. It helps turn an individual algorithm into a tool for collaborative work while preserving the ability to validate and develop the computation itself.

Source code and setup instructions are available in the scientific_fastapi repository. The project is distributed under the MIT license.