Building open-source AI infrastructure with reproducible tests, explicit limitations, and safer defaults
RagLeap continues to expand its open-source AI infrastructure across voice, physical AI, security, evaluation, graph intelligence, tracing, and optimization.
This release cycle introduces new capabilities across several RagLeap libraries while focusing on something equally important: making it clear what has actually been tested, what remains experimental, and what should not yet be treated as production-ready.
The direction is straightforward:
Build useful AI infrastructure, verify it with reproducible tests, and document the boundaries of what it can do.
RagLeap is an open-source, self-hosted project. These releases are part of the broader effort to build AI employees and supporting infrastructure that can operate with explicit controls rather than unrestricted autonomy.
Here is a detailed look at the latest work.
1. RagLeap Physical: Simulation-Only AI-to-Physical Infrastructure
Release: ragleap-physical 0.0.5
Physical AI is an area where experimentation needs clear boundaries.
This release continues RagLeap Physical’s simulation-focused development. It does not control real hardware.
The project remains pre-alpha. It is not safety-certified, independently audited, or ready to be represented as a production industrial-control system.
What changed?
The release includes several improvements to the simulation and safety infrastructure.
Offline ROS 2 mock
A bug in the offline ROS 2 mock was fixed. Calling clear_stop() no longer raises an exception.
Invalid-value rejection
The one-joint and two-link stand-in arms, along with the simulated machine-tool adapter, now reject invalid values through direct apply() and dry_run() calls.
The rejected values include:
- NaN
- Infinity
- Text
- Booleans
- None
The tests check that invalid requests raise ValueError and do not modify the simulated state.
The bridge already rejected these values. This release extends that behavior to the direct adapter methods.
MTConnect clarification
The MTConnect documentation and adapter wording were corrected.
The adapters remain observe-only. The documentation now more clearly distinguishes observation from direct machine commands and describes the request/response state behavior.
Adapter conformance tests
The release adds conformance coverage for the adapters.
Approval-state testing
The approval flow now has an explicit state model.
The model contains 18 reachable states and five properties. Tests replay every one-proposal sequence up to five events through both the abstract model and the real bridge.
This is useful for checking that the implemented approval flow behaves consistently with the model.
However, the model is hand-written. It is not a formal proof of safety.
Verification
The published TestPyPI source archive was tested in Colab on November 7.
Environment:
- Python 3.13.18
- pytest 8.4.2
- MuJoCo 4.0.0
- MCP SDK 1.30.0
Result:
604 passed, 16 skipped
The skips were intentional.
CI also passed on Python 3.10, 3.11, and 3.12 for the release PRs.
The release was tagged:
ragleap-physical-v0.0.5
Commit:
635ea2d
What remains unverified?
The Windows refusal fix introduced in an earlier release has not been retested in this release.
Cross-machine repeatability has not been measured.
Most importantly, this release does not demonstrate that the system can safely operate real industrial equipment.
The current focus remains simulation, observation, validation, and explicit approval boundaries.
2. RagLeap Voice: Optional Model-Based Voice Activity Detection
Release: ragleap-voice 0.3.6
Voice infrastructure needs to handle streaming input, silence, short pauses, and noisy recordings without silently turning experimental models into default behavior.
This release introduces an optional SileroGate.
What changed?
SileroGate provides model-based voice activity detection through the same feed() and finish() interface and event structure used by StreamGate.
The implementation is optional.
The additional vad dependency installs ONNX Runtime and NumPy. The core package remains standard-library-only.
The model is:
- Not exported as a default public feature
- Not enabled by default
- Not wired into
transcribe(gate=...) - Not wired into
transcribe_stream
No model is bundled or downloaded automatically.
Users must provide the model file:
silero_vad_16k_op15.onnx
A model file without a pinned SHA-256 is refused.
There is also no models.toml entry for this model, and its weights license and training data have not been verified.
These restrictions are intentional. Introducing a model-based component does not automatically make it a supported default.
Benchmark observations
The release includes per-call timing measurements for transcription and the Transcriber on a Tesla T4, along with a GPU-use check.
Clean-speech WER was compared across:
- FFmpeg mu-law
- RagLeap's 8 kHz mu-law path
- RagLeap's 8 kHz A-law path
The sample included English, Spanish, and Russian, with 30 clips per language.
No difference was found in that sample.
This is a limited benchmark, not evidence that the paths are equivalent for every recording or environment.
Noise-only experiment
A noise-only experiment used recorded noise from the MUSAN free-sound folder.
Observed false accepts:
| Gate | False accepts |
|---|---|
| StreamGate | Approximately 99% |
| SileroGate | Approximately 4.4–5.4% |
This result is important, but it needs context.
It is an experiment on a specific noise sample. It is not a general speech-detection accuracy claim, and it does not establish performance in real rooms, telephone calls, or production audio.
The release also includes verification criteria and a smoke test.
Silero can split speech around short pauses. Whether that behavior provides better practical value remains unmeasured.
Release tag:
ragleap-voice-v0.3.6
Commit:
c12e4f1
3. RagLeap Security: Encryption, Key Management, Rate Limiting, and Integrity
Release: ragleap-security 0.2.0
Security infrastructure should make boundaries explicit and avoid pretending that a single library solves every threat.
This release adds new security modules while preserving the existing 0.1 API.
The package supports Python 3.10 and newer.
Key handling
The new shared key module provides:
KeyKeyRing
The standard-library Key object does not print its secret material and cannot be pickled or hashed.
KeyRing provides a structure for key rotation.
Optional cryptography
The optional crypto functionality includes:
- AES-256-GCM
- Versioned encrypted envelopes
- Key identifiers
- Authenticated context, such as a
Scopekey - Key rotation and re-encryption
- Passphrase encryption using scrypt
- Generic decryption failures
Authenticated context allows encrypted data to be associated with an expected scope rather than relying only on possession of the encryption key.
Rate limiting
The standard-library rate limiter uses a token bucket keyed by a Scope or string.
Storage options include:
- In-memory storage
- SQLite-backed storage
The implementation includes bounded tracking of keys and flood-resistant reset behavior.
The rate limiter fails closed when it cannot safely proceed.
However, the SQLite implementation is not a distributed rate limiter.
In-memory state resets on restart, and distributed coordination is not provided.
Integrity checks
The integrity module provides bounded SHA-256 file digests and directory manifests.
Optional HMAC support is available.
The implementation avoids following symlinks and special files.
Integrity checks can help detect unexpected changes. They do not prevent someone from copying data or modifying a system outside the protected workflow.
Verification
The release was tested through:
- CI
- VPS testing
- Sandbox testing
- Fresh PyPI installation checks
- Mutation testing
- Release artifact checks
Results included:
Python 3.10 with crypto: 328 tests
Python 3.10 without crypto: 295 passed, 1 skipped
Mutation testing: 135/135 mutations killed in both tested environments
The sandbox also tested Python 3.10.20 and 3.12.3 with cryptography 42.0.0 and 50.0.2.
The release's CI checks passed.
The package was published with trusted publishing and wheel provenance. The wheel digest was checked against the release.
Tag:
ragleap-security-v0.2.0
Commit:
6f40f7d
Security limitations
This release has not received an independent security review.
The tested environments do not establish behavior on Windows or macOS.
SQLite behavior on network filesystems and under heavy multiprocess workloads remains unverified.
The implementation cannot protect against attackers who already control the process or its keys.
4. RagLeap RAG: Evaluation 2 and Cheaper Inference
Release: ragleap-rag 0.15.0
Evaluation is essential when improving retrieval or generation.
This release adds Evaluation 2 functionality for ranking metrics, latency, usage, and result comparison.
Evaluation features
The new evaluation capabilities include:
- Ranking metrics
- Latency and usage measurements
- Tail-aware summaries
- A support-checker hook
- Run comparison
- Source ordering by first appearance
The package also adds a CheaperInference provider entry.
Python 3.10 is no longer supported by this release.
The minimum supported Python version is now 3.11.
What this enables
Evaluation 2 provides infrastructure for comparing runs and inspecting ranking behavior.
It does not automatically establish that a retrieval system produces better answers.
The support checker is an optional hook. No judge model is bundled.
Real database and model evaluation still require suitable fixtures and services.
5. RagLeap Graph: Deterministic Community Detection
Release: ragleap-graph 0.10.0
This release introduces deterministic community detection.
The new functionality includes:
GraphIndex.detectCommunitiesGraphIndex.refreshCommunitiesGraphIndex.findCommunities- Opt-in community lookup through
GraphRetriever
Community retrieval is disabled by default through:
include_communities
The implementation was compared with the Python version.
The release adds community detection as an optional retrieval capability rather than changing the default retrieval path.
Verification
The release was tested with:
- Deterministic community detection comparisons
- Graph index operations
- Retrieval behavior
- Existing graph tests
The Java graph release depends on the corresponding RagLeap RAG library.
The community refresh and lookup operations were checked once against a live Neo4j 5.26 environment.
The release recorded:
174 tests, 0 failures, 40 skipped
What is not claimed?
No improvement in answer quality has been demonstrated.
Community detection is now available as infrastructure, but whether it improves a particular application depends on the graph, retrieval strategy, and evaluation dataset.
6. RagLeap Java RAG: Evaluation 2
Release: ragleap-rag-java 0.14.0
The Java RAG library now includes Evaluation 2 ranking functionality.
The release adds:
- Hit rate
- Recall
- Precision
- Reciprocal rank
- nDCG
- Percentile and statistical summaries
- Run comparison
- Optional support-checker integration
RagLeap.evaluateV2- DemonRoute provider support
The ranking metrics were compared bitwise against the Python implementation.
Verification
The release build recorded:
533 tests, 0 failures, 53 skipped
Some live and parity tests require fixtures or external services.
The new evaluation API has not yet been verified against a real database or a real model.
This release expands the evaluation toolkit without claiming that evaluation infrastructure itself improves model quality.
7. RagLeap Trace: Open-Source Tracing Infrastructure
Release: ragleap-trace 0.1.0
Tracing helps developers understand how AI systems execute without requiring every prompt or response to be stored.
This is the first release of RagLeap Trace.
Core components
The package introduces:
TracerSpantracedInMemoryExporterJsonlExporter
The tracer records completed spans through:
Tracer.record()
RagLeap RAG adapter
The RagTraceAdapter provides hooks for:
on_ingeston_queryon_answer
The adapter uses allowlisted counts.
The core tracing design avoids recording prompt, answer, query, file, or credential content in the content keys.
This is a design boundary, not a guarantee that every possible integration is automatically safe.
Verification
Core tracing:
34 tests
24 deliberate-break tests
The adapter:
63 tests
42 deliberate-break tests
The tests were run on Python 3.10 and 3.12.
The integration tests used:
- RagLeap RAG
- PostgreSQL 16
- pgvector 0.8
- Fake embedder
- Fake generator
Integration results:
9 tests
12 deliberate-break tests
The integration tests passed in the sandbox and CI on Python 3.11.
The project also checked pinned GitHub Actions dependencies and fresh-install release artifacts.
Limitations
The release does not provide retention or rotation.
It does not guarantee that every integration will avoid sensitive data.
Thread-level spans have not been verified.
Windows permission behavior remains untested.
The integration has not been tested against other vector databases or real LLM providers.
8. RagLeap Optimizer: First Release
Release: ragleap-optimizer 0.1.0
This is the first release of RagLeap Optimizer.
The release adds optimization infrastructure without changing the behavior of existing RagLeap packages.
Request optimization
The optimizer runs a sequence of steps and returns:
- A new request
- An optimization report
- Before-and-after information for each step
- Loss information
The caller's original request remains unchanged.
A step that produces a longer request than its input is discarded.
Available transformations include:
- Exact duplicate-message removal
- Whitespace optimization
- Stable system-message prefix handling
- Stable tool JSON ordering
- Tool-output trimming
Tool-output trimming retains the beginning and end of the output and marks the result as lossy.
Exact cache
ExactCache provides scoped in-memory caching.
It uses the entire request as the cache key.
The cache supports:
- Expiration
- LRU behavior
- Scope-based erasure
It is not a shared distributed cache.
Retry and fallback
The package includes bounded retry and fallback behavior.
The implementation supports:
- Bounded attempts
- Exponential backoff
- Capped delays
- Deadlines
- Failure names without exposing failure text
Evaluation
The evaluation functionality compares the original and optimized requests using the package's own cases and judge.
It reports:
- Input-token savings
- Quality delta
- A 95% interval
- Cases where quality became worse
When fewer than 30 cases are available, the package does not report a savings figure.
The package also includes a nine-row threat model and mutation testing.
Verification
The release recorded:
156 tests
84/84 mutations killed
Testing covered Python 3.12 in the sandbox and Python 3.10.12 on the VPS.
CI tested Python 3.10, 3.11, and 3.12.
The release passed fresh PyPI installation checks and README examples.
Wheel and source archive hashes were checked against the tagged build.
Limitations
Real-world traffic savings have not been measured.
Multi-process cache behavior remains unverified.
The package has not been evaluated against every OpenAI-compatible message format.
Token estimates default to one token per three UTF-8 bytes unless a caller supplies a token counter.
No independent security review has been performed.
Tag:
ragleap-optimizer-v0.1.0
Commit:
8917b06
9. Additional Voice Improvements
Voice 0.3.5: Engine Concurrency
The previous voice release addressed races around shared engine loading, running, and closing.
The changes include:
- Reference counting and locking around offline environment variables
- Registry registration locking
- Per-engine serialization of shared engine calls
- Parallel operation across different engines
The defaults and API remain unchanged.
The implementation does not control environment variables changed externally, and it does not address every Faster-Whisper internal behavior.
An engine cannot call itself from inside its own run operation.
Voice 0.3.4: Japanese Number-Word Mode
The earlier release introduced an opt-in Japanese kanji number-word mode.
It is disabled by default and is not applied to published tables.
The implementation was contributed, but the Japanese text has not been fluent-reviewed.
The release also fixed malformed Chinese numeral handling while preserving valid forms.
These are helper-level changes, not claims of general multilingual transcription accuracy.
Voice 0.3.3: Mandarin and Streaming Tests
The earlier release introduced an opt-in Mandarin Chinese number-word helper.
It is disabled by default and is not applied to published tables.
The model-folder fingerprint now includes the device identifier.
The Transcriber rejects use after closing.
Tests cover hard-link writes and swapped folders.
Synthetic StreamGate and Silero VAD benchmarks were performed, but only one run each.
They do not establish performance in real rooms or telephone calls.
10. What This Release Cycle Represents
Across these packages, the latest work covers several layers of AI infrastructure:
| Area | New capability |
|---|---|
| Voice | Optional model-based VAD and concurrency improvements |
| Physical AI | Simulation validation, invalid-value rejection, approval-state testing |
| Security | Key management, encryption, rate limiting, integrity checks |
| RAG | Evaluation 2 and inference provider infrastructure |
| Graph | Deterministic community detection |
| Java RAG | Evaluation 2 metrics and comparison |
| Trace | Span recording and RagLeap RAG hooks |
| Optimizer | Request transformations, exact caching, retry, and evaluation |
The common theme is not that every feature is production-ready.
The common theme is that the project is expanding its infrastructure while making the testing evidence and limitations visible.
Some releases have extensive test coverage. Others remain experimental, optional, or dependent on external services.
Those distinctions matter.
A passing test suite does not replace independent security review.
A deterministic graph algorithm does not prove better retrieval quality.
A simulation does not establish safe real-world hardware control.
A benchmark on a small sample does not establish general performance.
And an optimizer that reduces estimated tokens does not automatically demonstrate lower production cost or better answer quality.
These are the boundaries we want to keep explicit as the project grows.
11. What's Next?
RagLeap continues to develop its open-source infrastructure around AI employees, memory, security, and physical systems.
The current direction is to expand capabilities while keeping the underlying design understandable, testable, and self-hostable.
For contributors, the most useful work is often small and specific:
- Improve reproducible tests
- Add missing adapter coverage
- Validate behavior across operating systems
- Review security boundaries
- Improve documentation
- Test integrations against real services
- Contribute focused fixes
The project welcomes engineering contributions that improve reliability without overstating what has been verified.
Explore RagLeap
GitHub: https://github.com/ragleap/ragleap-core
Website: https://ragleap.com
Documentation: https://docs.ragleap.com
RagLeap is open-source and self-hosted.
If you're interested in AI infrastructure, retrieval, voice, security, tracing, optimization, or simulation-based physical AI, explore the repository and contribute through focused issues and pull requests.
Open-source AI employees that ask before they act.
Top comments (0)