DEV Community: Thomas Virdis

DILIGENT: A Local Workspace for Structured Drug-Induced Liver Injury Review

Thomas Virdis — Sun, 18 Oct 2026 06:31:26 +0000

Drug-induced liver injury is a hard call, and the difficulty is rarely about spotting a signal. A clinician looking at a rising ALT already knows something is happening. The hard part is the reasoning that has to follow: which of the patient's medications is a plausible cause, how the injury pattern reads on the lab values and the timeline, whether a competing explanation fits better, and whether the label of "causality" is defensible enough to act on.

That reasoning is structured. Causality assessment frameworks such as RUCAM and the DILIN expert-consensus categories, the Hy's Law thresholds, and the LiverTox monographs all exist precisely because the argument has to be laid out in a consistent, inspectable way. A lot of the work of a DILI review is therefore assembly and consistency rather than insight: collecting the right inputs, resolving the liver chemistry pattern, checking the same boxes every time, and keeping the evidence visible next to whatever conclusion gets written.

DILIGENT is an application for that part of the job. It is a local, single-user decision-support tool for structured DILI review: it collects the case, runs a deterministic pipeline over it, assembles an evidence bundle, and produces a reviewable draft assessment that a clinician can edit, approve, or reject. The draft is not an answer. It is the assembled case, written down in a form that can be checked.

What a review looks like

The workflow is one screen at a time, and each step has a defined output.

Configure. The application needs several models, and it treats them as roles rather than as a single chatbot: a clinical model for the consultation step, a text extraction model for structured extraction, a revision model for editing passes, and a timeline model for chronological reconstruction. Each role can point at a local Ollama model or at a cloud provider. Where a provider needs a key, the key is stored by the backend and shown back only as a fingerprint, and it stays inactive until it is explicitly activated, so a rejected key cannot replace the one that works.

Enter the case. The DILI Agent form asks for the things the frameworks need: patient and case identifiers, demographics, the suspected drug with dose and timing, symptom onset and history, laboratory values with units and reference limits, concomitant medications, competing causes, dechallenge and rechallenge status, and the clinical question being asked. Where the deployment enables it, files can be attached.

Pre-flight checks. Before any model is called, rule-based checks run over the input. Blocking issues have to be fixed. Warnings can be acknowledged and carried forward. Section extraction confidence is enforced here too: an aggregate below 0.65 blocks the run, and between 0.65 and 0.85 the operator has to acknowledge the weakness. This is the first place where the design chooses to stop rather than guess.

The pipeline. The assessment runs as a background job, so a long consultation does not hold the browser. The stages are concrete: normalization, deterministic section extraction, structured LLM extraction of drugs, diseases and labs with bounded retries and a deterministic fallback, presentation hepatic pattern resolution, case qualification kept deliberately separate from drug attribution, conservative drug identity resolution, per-drug RUCAM evidence, an internal structured causality synthesis, Hy's Law status, an explicit severity grade, a mandatory competing-cause checklist, optional retrieval, a bounded clinical consultation, and finally the evidence bundle and the report with a faithfulness audit.

The R ratio is computed from paired ALT and ALP values at the presentation episode, each divided by its own laboratory upper limit of normal rather than by a shared reference: R at or above 5 reads hepatocellular, at or below 2 cholestatic, between 2 and 5 mixed, and the pattern stays indeterminate when the inputs cannot support the calculation. Case qualification is its own answer, too. Whether the labs meet the application's typical liver-injury detection criteria is recorded separately from which drug, if any, is implicated, and a qualifying episode is not treated as a diagnosis.

Review. The result lands in Clinical Sessions as two things side by side: the narrative report and the structured dossier behind it, including twelve acceptance questions answered with quotes and a fourteen-section dossier in the shape clinicians already expect from an FDA-style review. Drug matching is exposed rather than hidden, with per-drug flags distinguishing accepted matches from ambiguous ones that need a human decision and from missing ones. From there the report can be edited by hand, put through an LLM revision that becomes a new official version, compared against the previous version, saved, copied, or exported. A separate tab reconstructs the patient timeline, and Data Inspection shows the local catalogs the matching ran against.

The report is built from the evidence, not from the model

The design decision that shapes everything else is the order of operations. The final report is generated from a persisted evidence bundle that exists before any LLM summary is written. The narrative is then checked against that bundle by a faithfulness audit, and blocking faithfulness issues are finalization blockers: a session with an unresolved one is forced into "requires human review" and can never be marked successful.

Revisions get the same treatment rather than a lighter one. A revised report is audited again against the structured evidence before it can be persisted as a successful revision, so an editing pass cannot talk its way past a competing-cause, Hy's Law, causality or rechallenge safeguard that the deterministic layer already enforced.

The same instinct shows up in the extraction layer. When a model-based extraction fails, the pipeline falls back to deterministic rule-based parsers and records a fallback warning instead of failing the run or inventing content. Degradation is visible in the session rather than silent.

Causality stays conservative

Scoring a drug as a cause is where clinical software is most likely to overreach, so the causal semantics here are deliberately narrow. If the patient's own record already documents a RUCAM score, that score is preserved and mapped to its category. If it does not, the pipeline does not manufacture a numeric pseudo-score: it keeps a criterion-level evidence checklist, with its limitations stated, instead of a number nobody computed. RUCAM output is supportive rather than dispositive in either case.

The same line is drawn around drug-level knowledge. A LiverTox likelihood and an FDA DILIrank concern class are prior opinions about a drug, not patient-level causality, and neither is converted into a weight on this patient's case. A rechallenge is never recommended. A missing follow-up laboratory value is recorded as missing, not as negative, and a general laboratory improvement is not attributed to every medication in a polypharmacy case. Competing causes are checked against a mandatory list and carry four explicit states, excluded, not excluded, unknown and missing data, so an unresolved differential stays unresolved in writing.

None of this makes the tool cautious in a vague way. It makes each statement in the report traceable to either a computed value or an explicit absence, which is what an audit of the case would need.

Retrieval that runs on the machine

The RAG layer is optional per assessment, and its embedding stack is local by construction: the pinned IBM Granite multilingual embedding model, 384 dimensions, run through ONNX Runtime on CPU with a quantized artifact whose SHA-256 is checked against a pinned value. Retrieval itself is hybrid over LanceDB, combining vector similarity with text search, weighted so that vector evidence dominates while exact terms still matter.

The corpora come from updaters built into the application: LiverTox monographs retrieved through NCBI's machine services, the RxNorm-backed RxNav drug catalog, the FDA DILIrank workbook, and a documents folder you point the app at, which the desktop build selects through the native operating system picker. DILIrank is deliberately read-only and row-level: a record that cannot be linked deterministically to the canonical drug catalog stays inspectable but never feeds the clinical consultation. A retrieval run does not need a cloud embedding service, and it does not need to send the case anywhere to find the relevant monograph.

Models are a configuration choice

The four roles can run entirely on Ollama, drawing from a local catalog that includes general and medical models, or they can point at cloud providers. Switching is configuration rather than architecture. That matters for two audiences at once: a hospital-adjacent user who needs the case to stay on the workstation, and a researcher who wants to compare how different models handle the structured extraction and consultation steps over the same evidence bundle.

Running it, and where the data lives

On Windows the application ships as a packaged desktop build: a portable executable and an MSI installer, a Tauri 2 shell using the system WebView2 runtime, and the Python backend frozen with PyInstaller into its own binary. Release artifacts carry SHA-256 manifests, and the published binaries are unsigned under a policy the release notes state plainly rather than leaving you to discover it at launch. Application data lives under the user's local application data directory, beside a hash-addressed runtime directory, so installing a new version does not touch the data the previous one wrote. A source mode is also available for anyone who would rather build from the tree; it pins the exact Python, Node, uv and Rust versions it expects.

macOS and Linux are supported in source mode only, with no one-step launcher, and containers are explicitly not supported: there is no Dockerfile in the repository and no official container path. Storage is a relational database, either embedded SQLite or an external PostgreSQL instance, and the API surface is described by a checked-in OpenAPI document so the client and backend cannot drift apart silently.

One caveat belongs in the first paragraph of any deployment discussion rather than the last: if the model roles are pointed at a cloud provider, the clinical text of the case leaves the machine. The local configuration exists, and the choice is the user's.

What it deliberately is not

DILIGENT is not a diagnostic device and does not replace clinical judgment. Every input, generated statement, conclusion and piece of exported text is the responsibility of the qualified clinician using it. It is a single-user tool: there is no account system, no multi-user deployment, no mobile client and no hosted service, and network deployment behind a reverse proxy is explicitly unsupported. The packaged build does add a loopback guard, a bootstrap token exchanged for a session cookie plus host and origin checks, but that is protection for one user on one machine, not a login system. It does not train or fine-tune models, and it does not publish accuracy numbers, because none have been measured. The repository ships user documentation, architecture documentation and runtime documentation instead, which is the honest artifact for software at this stage.

The licence is the GNU General Public License, version 3 or any later version, so it is free software in the full sense: inspectable, modifiable, and redistributable under the same terms.

Who this is for

If you work with drug-induced liver injury and you have ever wanted the case assembled the way the frameworks actually specify it, with the scoring visible and the report traceable line by line to the evidence behind it, this is worth an afternoon. It runs on a workstation, it takes a fictional case in minutes, and it will show you exactly how it reached whatever it says.

The repository, documentation and packaged Windows builds are at github.com/CTCycle/DILIGENT-Clinical-Copilot.

Eight green rows and an empty string

Thomas Virdis — Tue, 15 Sep 2026 20:04:01 +0000

I wanted a narrow answer this morning: does the 2B model I keep on this laptop return a function call when I ask for one, or does it wander off mid-sentence?

I got a report with eight green rows, zero errors, and eight empty response strings. I noticed the first two facts and missed the third for a while.

The laptop

6 GB RTX 3060, laptop version. Ollama loads qwen3.5:2b at 5.0 GB of VRAM with a 131072-token context window, and reports the split as 25 percent CPU, 75 percent GPU. It does not fit, so it spills, and that is a lot of cache to be holding next to weights that already overflow.

Your machine will differ in every one of those numbers, which is the whole reason for measuring at home.

The run

The tool is LLMeter, one Rust binary that benchmarks whatever OpenAI-compatible endpoint you point it at. Standard suite, temperature 0.0, sixteen tokens per answer:

llmeter --provider ollama bench run --suite llm \
  --models qwen3.5:2b --benchmarks all \
  --runs 1 --max-tokens 16 --temperature 0.0 \
  --export both --report both

Benchmark	Prompt	Wall time	Output tok/s	Valid output
chat generation	short	103,621 ms	0.15	n/a
responses API	responses	834 ms	19.2	n/a
consistency	x2	844 ms	38.5	similarity 1.0000
prompt sizes	short (22 tok in)	389 ms	41.1	n/a
prompt sizes	medium (49 tok in)	723 ms	22.1	n/a
prompt sizes	long (124 tok in)	2,144 ms	7.5	n/a
structured JSON	schema required	812 ms	19.7	0% schema valid
tool call	function required	522 ms	30.7	0% tool call valid

The first row: 103.6 seconds for sixteen tokens. Everything after it ran between 389 ms and 2.1 s. Short prompts pushed 41 tokens per second, the long one managed 7.5 because the prefill had 124 input tokens to chew before generation could start.

Two of the eight requests came back flagged: structured JSON invalid, tool call invalid. At sixteen output tokens this model had no room to do either.

Then I stopped believing the first row

My first reading was that 103 seconds told me something about the model. To check that, I unloaded the weights and measured the load itself.

I unloaded the weights (ollama stop qwen3.5:2b, then ollama ps to confirm nothing was resident) and ran the same single request again: twelve point six seconds, then eight point seven, then nine point seven.

Same card, same model, same command, twenty minutes apart. The first measurement of the session was twelve times slower than the fastest reload, and nothing about the model changed in between. The first load pulled 2.7 GB of weights off a cold disk. By the third run the operating system had those files in the page cache and the card had less work to do.

So the biggest number in that chart mostly measures how fast this disk handed those files over. The model's own contribution to it was small.

The empty string

I turned on response previews to see what the model actually said, and the preview column came back empty. I re-ran the same request with the token budget raised to 32, 64 and 200: still empty, at every budget. The provider returned usage numbers the whole time, 22 tokens in and 16 out, and the CLI computed throughput from them correctly, 19.7 tokens per second.

Then the consistency row, the one with 1.0000 next to it. Two identical prompts, both answered with an empty string, compared pairwise and scored as a perfect match. That row reads like the healthiest number in the report, and it is the emptiest.

Nothing failed. Eight records, zero errors, successful records: eight. Had I stopped at the executive summary I would have written down that this model produces about twenty tokens per second and I would have been describing the speed of an empty response.

I still do not know why the content field was empty. Something about how this model's output arrives through the OpenAI-compatible endpoint, or the token budget. I did not chase it down. The part that mattered was already in the report, in a column I could read.

What the tool got right

The capability flags stayed visible. schema_valid: false, tool_call_valid: false, json_parse_success: false, each in its own column instead of folded into a pass rate. That is the difference between a report I can act on and a green tick I have to trust.

It attaches to the endpoint and measures what comes back. It does not start your servers or load your models. That sounds like a missing feature until you have had a load generator restart the thing it was supposed to be measuring, mid-run, and then report the result as a number.

Twelve provider presets, three of them first-class (ollama, lmstudio, llama-cpp), then vLLM, SGLang, LocalAI, LiteLLM, TGI, text-generation-webui, Jan, MLX-LM and a generic OpenAI-compatible target. Subcommands are scriptable and the exit codes are meaningful, so this can run in CI rather than in a screenshot.

Two smaller decisions that mattered more than they sound. Results are written as schema 3.0 with a mandatory run kind, and a result file without that version is refused on load rather than silently upgraded. That refusal is what keeps a series of results a series. Previews are off by default and credential-shaped values are redacted before anything reaches disk, so turning previews on was a deliberate act on my part rather than something I found later in a JSON file.

The export behind the chart near the top is in a scratch folder on my laptop. The repository ships the code and the manual, not the artifacts, and its QA directory is gitignored on purpose. The run ID and the command are in this post, so you can repeat the measurement on hardware you own, which is the only comparison worth making.

What I do differently now

I read the content field before any average. OK in the status column means the request completed, not that the answer was usable, and this run is the cleanest example of that gap I have produced.

The capability flags are the result, the tokens-per-second column is the footnote. My question was answered by two zeros, not by 41 tokens per second.

One run is a sample of one. The same request on the same laptop covered a twelvefold range inside twenty minutes, so three runs with the model unloaded in between, median reported, is the minimum I would put in front of another person.

And I stopped comparing across machines. The 25/75 CPU-GPU split is the number that explains everything above, and it exists only on this machine.

The model still is not good enough for the agent I had in mind. I found that out in one command, and I have a report I can keep and diff against the next run instead of a leaderboard entry for somebody else's hardware.

LLMeter is MIT licensed: github.com/CTCycle/LLMeter-local-benchmarks. Release archives cover Windows, Linux and both macOS architectures with checksums. There is no crates.io package yet, so the archives or a checkout are the two routes.

Benchmarking Tokenizers Without the Guesswork

Thomas Virdis — Mon, 31 Aug 2026 12:41:05 +0000

Benchmarking Tokenizers Without the Guesswork

What TKBEN measures, why tokenizer evaluation is deceptively hard, and how reproducible evidence beats a one-off script.

The quiet bottleneck under every model

Every language model has a tokenizer bolted to its front door. Before the model sees a single word, the tokenizer chops the text into units, maps them to numbers, and decides, in effect, what the model is allowed to think about. That choice is not cosmetic. Two tokenizers over the same corpus can differ in vocabulary size, in how many pieces a word gets split into, in how fast they run, in how much memory they use, and in whether arbitrary text can even be reconstructed at the end of the round trip. Each of those differences changes real-world behavior: inference cost, latency, how gracefully a model handles a language it was never trained on, and how trustworthy the decoded output is.

Yet tokenizers are usually the least-examined part of a model pipeline. Most benchmarks of language models treat the tokenizer as a fixed input, something to be accepted rather than evaluated, and most teams pick one the way they pick a default: it came with the model, so it will do.

TKBEN (TKBEN-tokenizers-benchmarker) is built on the opposite assumption. A tokenizer is a choice, and it should be examined the way you would examine any performance-critical component: on repeatable, inspectable data, not on a gut feeling.

Why the naive benchmark is not enough

The obvious way to compare tokenizers is to run a quick script: load one, feed it some text, print a number, move on. That approach has a weakness that shows up only later. The number is gone. The person who ran it cannot reopen the evidence, cannot re-run the comparison against a different dataset, cannot show how the figure was produced, and cannot tell you whether it reflects the test set that actually matters.

Tokenizers are sensitive to what you feed them. A tokenizer that looks efficient on English news text may fragment a multilingual dataset badly. A vocabulary that seems rich can still leave you with a high unknown-token rate on a domain you cared about. A fast encode time tells you nothing about latency at the tail, or memory under load. None of these come through in a single printed stat.

TKBEN treats this properly. It is a local web application that keeps downloaded tokenizer assets, validation reports, and benchmark results in its own workspace, so a later session can reopen and inspect the same evidence. A benchmark becomes a saved report with provenance, re-openable weeks later, rather than a printed number that is gone the moment the script exits.

The concept: validation before comparison

The heart of TKBEN is a separation that most ad-hoc benchmarking collapses: you cannot compare tokenizers until you first understand the text you are going to run over them.

Dataset validation comes first. You load a Hugging Face dataset, a custom identifier, or a local CSV, XLS, or XLSX file, and run it through the validation pipeline. The result is a saved report covering corpus scale and structure (document counts, lengths, and how spread out they are), lexical diversity and frequency (type-token ratio, moving-average TTR, word distributions), word and character signals, document quality signals, structural regularity, and compression and redundancy. In plain terms, TKBEN tells you what kind of text you are about to benchmark, so you are not comparing tokenizers on data you do not actually understand.

Then, and only then, do you bring in the tokenizers. You scan or add tokenizer identifiers, download the assets you need, upload a custom tokenizer.json, and inspect each report's vocabulary statistics and token preview before trusting it in a comparison. Once you have a validated dataset and a set of tokenizers you have actually looked at, you can build a cross-benchmark run.

What the comparison actually measures

The benchmark result is a dashboard of widgets, each one a separate view of a tokenizer's behavior on the validated dataset. The metrics fall into six groups, and each answers a different question:

Metadata. Vocabulary size, added-token count, special-token share. The basics of what a tokenizer carries with it.
Efficiency. Tokens per second, characters per second, bytes per second, and the time split across encoding, dataset streaming, post-processing, and the complete run from start to finish. This is the throughput story.
Latency. Median, p95, and p99 encode latency, plus the raw latency distribution. The tail, where interactive models actually feel slow, is not hidden behind an average.
Fidelity. Whether text survives the trip. Exact token-ID round trip, normalized text round trip, unknown-token rate, byte-fallback rate, and how much character coverage the vocabulary provides. A fast tokenizer that cannot faithfully round-trip its input is not fast, it is lossy.
Fragmentation. Tokens per character, characters per token, pieces per word. A measure of how aggressively text is split, and how that shifts with word length.
Resources. Peak RSS and memory delta. What the tokenizer costs in resident memory while it works.

You can switch between bar, horizontal-bar, box-plot, histogram, dot-whisker, and grouped-bar views, reorder the widgets to match how you think about the problem, and export the whole comparison as a PDF. The dashboard below compares a GPT-2 style tokenizer against RoBERTa-base over the same C4 sample, side by side, with the numbers that actually differ made visible.

The value of a dashboard like this is not the top-of-the-fold number. It is that a memory-hungry tokenizer, a slow tail, and a fracture-prone vocabulary can all live in the same view, so a team sees the whole trade-off instead of a single ranking.

A report you can trust a month later

TKBEN's core design decision is that a benchmark is a saved report, not an event. Each validation and each cross-benchmark run persists in the workspace, with a page to reopen it. Combined with the PDF export, that turns a fragile one-off script into an auditable workflow: you can show exactly which dataset, which tokenizer versions, and which metrics produced a given comparison, and you can reopen that evidence days or weeks later without re-running anything.

That matters whether you are choosing a tokenizer for a new model, auditing the efficiency of an existing pipeline, or documenting a decision for a team. The report becomes the artifact you argue from, and the artifact still exists the next time somebody asks how the decision was made.

Tokenizer assets you can really inspect

Under the hood, a tokenizer report shows what the download actually gave you: model metadata, a vocabulary count, a token-length distribution, and a paginated preview of the tokens themselves. This is a small thing that saves real confusion. Reported and actual vocabulary sizes can drift, especially with added tokens, and downloading a model you think is one thing and inspecting another is how wrong comparisons start. TKBEN surfaces the actual token list and lets you confirm that what was downloaded is what you intended to evaluate.

Local, and honest about it

Like the rest of the CTCycle portfolio, TKBEN runs locally. The browser is the interface; a local FastAPI service does the processing and stores the reports. Config stays in a machine-specific, never-committed settings file, and the embedded SQLite database is managed through explicit migrations. This keeps your datasets and analysis on your machine, which matters when the data you are benchmarking is proprietary or simply not something you want to send to a cloud service. The local setting is a completeness of control, not a limitation.

The evidence stays

Most tooling treats a tokenizer as a fixed input and a benchmark as a number you print and forget. TKBEN does the opposite. It validates the text before it benchmarks, measures the trade-offs that actually differ between tokenizers, and persists the whole thing as evidence you can reopen later.

If a team wants to pick a tokenizer with something better than intuition, TKBEN is the workspace to do it in. It runs on Windows through a single launcher, or on macOS and Linux with a short manual setup, and the source is open under an MIT licence. The repository is the place to start: github.com/CTCycle/TKBEN-tokenizers-benchmarker.

Part of the CTCycle open-source portfolio. See the full open-source portfolio at ctcycle.github.io/CTCycle.

XREPORT: A Local-First Lab for Draft Radiology Reports

Thomas Virdis — Mon, 31 Aug 2026 12:41:03 +0000

XREPORT: A Local-First Lab for Draft Radiology Reports

What it is, why it exists, and the design choices that make it a cautious tool instead of an ambitious one.

The job that hides inside the scan

If you have ever watched a radiologist work a stack of chest X-rays, the first thing you notice is how much of the job is not diagnosis. A lot of it is dictation. The lungs, the cardiac silhouette, the mediastinum, the bones. Normal, unremarkable, no acute findings. The kind of report that makes up most of a working day and takes very little of a radiologist's brain to produce correctly, but still has to be written, in the right format, with the right words, every single time.

That is the problem XREPORT is aimed at, and the honest way to state it is also the narrow way. Doing radiology reporting well is exhausting, and fatigue is the last thing you want anywhere near a diagnostic decision. Automation cannot fix that fatigue, but it can carry some of the routine load so the human review happens on a less tired brain.

This is not a tool that reads a scan and tells you what is wrong. It is a tool that produces a structured first draft of the report, which a qualified clinician then reads, corrects, and signs. The difference between those two jobs is the whole design. XREPORT is built to be a laboratory for making drafts, with the review step kept firmly human.

Local first, because the data decides

The most important sentence in the whole project is in the repository's opening note: generated drafts are not clinically approved and require qualified independent review. The second most important is the privacy boundary. XREPORT is local-first, and for medical imaging that is not a packaging preference, it is the entire point.

X-ray images are patient data under legal and ethical constraints. A tool that sends chest scans to a cloud service is a non-starter for most hospitals regardless of how accurate the model is, because the institution cannot control where the data goes or who can look at it. XREPORT never phones home. Models are downloaded once, cached in a project-local resource directory, verified, and reused offline. Training, validation, and generation all run on the machine the user controls. For clinical settings this is not a feature that needs to be argued for; it is the price of entry.

That single decision shapes everything downstream. Because there is no cloud backend, there is also no silent model update, no surprise behavior change between Tuesday and Wednesday. What the user validated last month is what runs next month, unless they choose to change it.

Not one report engine, but a choice with provenance

XREPORT avoids locking the user into a single black box. It ships with a fixed catalogue of five pinned public Hugging Face report-generation models, and it also accepts locally trained checkpoints through its own training workflow. The pinned models are chosen because each is a known, verifiable starting point. The custom checkpoints exist because a research lab or a hospital with its own data should not be forced onto a generic model when it has spent months building one on a domain-specific set.

Every generated report carries its provenance with it. The output comes back as editable Findings and Impression fields with the model, provider, and revision metadata attached, so anyone reading the draft can see where the text came from. That traceability matters in a domain where the reader needs to know whether the suggestion was produced by the model the team validated or by something else. It is the difference between a reproducible artifact and an output you cannot explain a month later.

What the custom model actually is

The in-house training path treats report generation as an image-to-text problem, the same framing you would use for image captioning, but with medical language and a strict output shape. It is not a chat model asked nicely to describe an X-ray. It is a purpose-built encoder-decoder that maps an image to a token-by-token sequence of report language.

The image side starts with a BEiT image encoder, a vision transformer pretrained on image patches, which turns a 224 by 224 chest X-ray into a set of visual feature vectors. Those features are projected into the shared embedding space. On the text side, a positional embedding encodes the report tokens as they are produced, and a stack of transformer encoder and decoder layers learns to attend from the image to that sequence. The default shape is four encoder layers and four decoder layers with eight attention heads and a 256-dimensional embedding, though the sizes are configuration options rather than hard constants. A softmax classifier over the vocabulary produces the next token at each step.

Training uses a masked cross-entropy loss that only scores the report tokens, with a masked-accuracy metric tracking how often the model predicts the right next token at the right position. The optimizer is AdamW with an optional warmup learning-rate schedule. The important practical detail is the checkpoint lifecycle. A run can be paused, resumed from any saved state, and evaluated against held-out examples, so a team is never forced to finish a model in one sitting or trust a single untested run. The evaluation view shows loss and accuracy for a saved checkpoint before that model is allowed anywhere near real generation.

The five public models, and what they trade off

Because not everyone wants to train their own model, XREPORT also ships a pinned catalogue of five public chest imaging models, each with its capabilities spelled out in the config rather than assumed. The catalogue is deliberately small and explicitly pinned to exact revisions, so a model that works today still works six months from now.

At the lightweight end is CXRMate Multi TF, a 0.1B parameter model that is the recommended default. It is designed for multi-view chest X-rays and its output is structured, with separate Findings and Impression sections. On modest hardware it is usable locally, which makes it the sensible first choice for most runs. Its sibling CXRMate-ED is a similar small model that additionally accepts a clinical indication or history from the user and maps it into the model's context input. Same family, one richer input.

Further up the scale sit two larger, more demanding models. CheXOne, a 4B parameter vision-language model from Stanford, is capable not only of drafting reports but of reasoning and of output that can be localized back to regions of the image, so it is the entry point if grounding matters. CXRMate-2 is a 3B parameter specialist with structured output and clinical-context support, at the cost of much heavier local storage and memory. Both are far more capable than the lightweight pair, and both are far more demanding on hardware, which is exactly the trade-off a local-first tool has to surface rather than hide.

Rounding out the set is MedGemma 1.5 4B, a broader medical-imaging model that, unlike the chest specialists, is not validated only on radiographs. It produces a single raw report rather than separate structured sections, and it is the one model that also supports prior images as context. It is also gated: downloading it requires accepting Google's model terms and configuring a local token, which XREPORT notes up front rather than pretending the download is automatic.

The picture that emerges is not "here is the one best model." It is a ladder of choices from a small fast default up to heavyweight research models, with their parameter sizes, licence restrictions, hardware demands, and structured-versus-raw output all visible in one place. A small radiology group on a workstation and a research lab with a GPU server look at the same screen and pick different entries on the same list.

The workflow is the concept

XREPORT is a loop, not a single inference button. The loop is the point:

Dataset preparation. Import an image folder plus CSV or XLSX report metadata, review how images and reports match, inspect paired records, then clean, tokenize, split, and build training-ready datasets. MIMIC-CXR is supported as a starting validation dataset, alongside custom image-report pair formats. The review step matters, because a training set built without checking that each scan is paired with the right report is quietly teaching the model wrong associations.

Training. Configure CPU or GPU training, watch live metrics and logs, save checkpoints, resume, and evaluate. Model readiness and provenance warnings stay visible throughout.
Generation. Add up to sixteen study images, pick a generation profile, and submit a background job against the chosen model or checkpoint.
Review. The result returns as editable Findings and Impression text with its metadata attached, ready to be copied or exported for qualified review. The screen is built around the edit, not the acceptance. Nothing is published straight from the model; the draft is handed to a person.

Two parts of this deserve emphasis. One is that the user can generate from up to sixteen images in a single case, which matters because a real radiological study is rarely a single view. The other is that validation is a first-class step, not an afterthought. A tool that shows a confident report with no indication of whether the model behind it was ever validated is a tool that has quietly handed the judgment to the machine. XREPORT keeps the validation state in front of the user instead of hiding it.

Why the tool deliberately does less

The temptation in a project like this is to keep pushing toward the interesting part: the idea that a model might one day make a clinical call. XREPORT is explicitly built to stop well short of that line. The value it claims is a coherent first draft that flags what a reader should look for, which is most useful on the routine cases where the findings are unremarkable. For rare or critical findings, an automated draft is only as useful as the clinician's judgment in catching what the model missed. That is why the human-in-the-loop is not a concession but the designed center of the system, and why the output is labeled a research-use draft rather than a diagnosis.

Getting from local promise to usable software took real work on the delivery side too. On Windows the app runs through a single launcher that manages the portable runtimes and dependencies. A Tauri 2 desktop shell packages it as CPU and CUDA installers, so a user who needs the version built for their graphics setup has a normal installer rather than a pile of manual steps. The binaries are currently unsigned, and MSI installs are per-machine, so this is a tool that assumes a technically literate operator, which is honest about the audience.

The restless part under the surface

Underneath, the application is a FastAPI backend with an Angular 22 frontend, with settings kept local and machine-specific, never committed, and the SQLite database managed through explicit migrations. None of that is the story a reader needs to remember. The reason to mention the stack at all is that it was chosen to stay boring and auditable: a report you can trace, a schema you can reason about, dependencies you can pin. In a tool that produces medical text, boring and auditable is a feature.

The design stance

Most report-generation tools are pitched on how much they can automate. XREPORT is more honest than that, and I think that is its best quality. Its automation effort is aimed squarely at the routine load, it keeps the data local, it attaches provenance to every output, and it never pretends to own the diagnostic step. Whether you train a checkpoint or pick a pinned public model, you can see what produced the draft and how validated it was, and the final sign-off stays human.

If a radiologist wants to spend less of their day on unremarkable dictations and more on the cases that actually need them, that is the version of XREPORT that exists. The first run is a research-use tool, not a clinical product, and the source is open under an MIT licence, so any team that disagrees with a design decision can take the code and change it.

If that sounds like your problem, the repository is the place to start: github.com/CTCycle/XREPORT-radiological-reports-generator.

Part of the CTCycle open-source portfolio. Models and generated drafts are not clinically approved and require qualified independent review.

A Deep Q-Network That Plays Roulette: What Happens When You Train RL on Pure Chance

Thomas Virdis — Wed, 26 Aug 2026 15:43:52 +0000

A Deep Q-Network That Plays Roulette: What Happens When You Train RL on Pure Chance

Why train a DQN on a mathematically hostile environment

European roulette carries a house edge of 2.7% on every bet. No amount of pattern recognition, trend following, or "system" playing changes the fact that the expected value of every spin is negative. It is, in the most literal sense, a stochastic process designed to drain capital over time.

So why build a deep Q-network to play it?

Because this is exactly what makes it a revealing reinforcement learning problem. When you train an agent in an environment where the optimal long-term strategy is simply "don't play," you get to watch a learning algorithm that desperately wants to find signal confront a data-generating process that contains none. The training dynamics reveal how DQNs behave at the boundary of detectability. That has implications far beyond the casino.

This article walks through FAIRS (Fabulous Automated Intelligent Roulette System), an open-source research application built to explore that tension. The full implementation is MIT-licensed on GitHub.

The problem: RL in a zero-signal environment

The classic DQN breakthrough (Mnih et al., 2015) demonstrated human-level performance on 49 Atari 2600 games. Those environments share a critical property: they contain genuine signal. Pixels correlate with game state, actions produce predictable outcomes, and a consistent policy can achieve superhuman scores.

Roulette shares none of these properties. Each spin is independent and identically distributed. The wheel has no memory. Past outcomes do not inform future ones. The state transition function is:

$$P(s_{t+1} \mid s_t, a_t) = P(s_{t+1})$$

That is, the next state is completely independent of both the current state and the agent's action. The Q-function, which in standard RL represents the expected return of taking action $a$ in state $s$, degenerates to its immediate-reward term plus an action-independent constant:

$$Q(s, a) = \mathbb{E}[R \mid a] + C$$

where $C$ is the discounted expected future reward, identical for every action because future spins are independent of both the current state and the current action. The constant does not change how actions rank, so $\arg\max_a Q(s, a) = \arg\max_a \mathbb{E}[R \mid a]$. The state $s$ still carries no predictive information; the only thing that distinguishes actions is the immediate expected value, which for any roulette bet on a European wheel is $-0.027$ per unit wagered.

This makes roulette a useful null-case benchmark. If an RL algorithm learns something, that something is either an artifact of variance, overfitting to noise, or a bug. Observing how it fails tells us about the algorithm's inductive biases.

The DQN algorithm: a refresher

The DQN (Deep Q-Network) algorithm approximates the optimal action-value function $Q^*(s, a)$ using a neural network. The standard temporal-difference update is:

$$Q(s, a) \leftarrow Q(s, a) + \alpha \left[ r + \gamma \max_{a'} Q(s', a') - Q(s, a) \right]$$

In practice, the network is trained to minimize the mean squared Bellman error:

$$\mathcal{L}(\theta) = \mathbb{E}{(s, a, r, s') \sim \mathcal{D}} \left[ \left( r + \gamma \max{a'} Q(s', a'; \theta^-) - Q(s, a; \theta) \right)^2 \right]$$

Where $\theta$ are the online network parameters and $\theta^-$ are target network parameters that are periodically copied from $\theta$ to stabilise training. The transitions $(s, a, r, s')$ are sampled uniformly from a replay buffer $\mathcal{D}$, breaking temporal correlations (Mnih et al., 2015).

FAIRS extends this with the Double DQN modification (Van Hasselt et al., 2016), which decouples action selection from action evaluation to reduce overestimation bias. The target becomes:

$$y = r + \gamma Q(s', \arg\max_{a'} Q(s', a'; \theta); \theta^-)$$

This is visible in the source code at app/server/learning/training/agents.py:

next_action_selection = model.predict(...)
best_next_actions = np.argmax(next_action_selection, axis=1)
q_futures_target = target_model.predict(...)
q_future_selected = q_futures_target[np.arange(batch_size), best_next_actions]

The online model selects the best action; the target model evaluates it.

FAIRS architecture: a dual-input Q-network

The FAIRS implementation diverges from the vanilla DQN in several design choices worth spelling out.

47-action space

Unlike the common 11-action mapping found in simplified roulette simulations, FAIRS exposes 47 actions: 37 straight number bets (0-36), 9 outside bets (Red, Black, Odd, Even, Low, High, and three Dozens), and a Pass action, so 37 + 9 + 1 = 47. This is a deliberately high-dimensional action space, and most of those 47 are strictly dominated by even-money bets. Watching which actions the agent settles on is therefore more informative than it would be with a small action set.

Dual-input network with learned embeddings

The Q-network (app/server/learning/models/qnet.py) takes two inputs:

Timeseries: the last 64 roulette outcomes as integer tokens, fed through a RouletteEmbedding layer that learns distributed representations of each number
Gain: the current capital-to-initial-capital ratio as a scalar context signal

These pathways are merged via an AddNorm layer (a residual connection with layer normalisation), then passed through a QScoreNet head that produces Q-values for all 47 actions. The network uses BatchNormDense layers with ReLU activations, 30% dropout for regularisation, and the AdamW optimiser (Loshchilov & Hutter, 2019).

The FAIRS training workspace. Dataset management on the left, checkpoint panel on the right, and the live training monitor charting rewards and loss in real time.

Reward scaling

Raw roulette rewards range from -10 (loss) to +350 (straight-up win). The environment scales these to $[-1, 1]$ using an asymmetric transform:

negative rewards: (reward + max_bet) / max_bet - 1   → maps to [-1, 0)
positive rewards: reward / (max_bet * 35)             → maps to (0, 1]

This prevents the rare +350 reward from dominating gradient updates while preserving the relative ranking of outcomes.

Experience replay with periodic target updates

The agent stores transitions in a deque buffer of configurable size (default 10,000). Training begins once the buffer exceeds the replay batch size (default 1,000). Every update_frequency steps (default 10), the target network weights are synced to the online network:

target_model.set_weights(model.get_weights())

Training dynamics: what the agent actually learns

Phase 1: Uniform exploration

At the start of training, epsilon is 0.75. The agent selects actions nearly at random, exploring all 47 actions roughly uniformly. The replay buffer fills with transitions from all bet types. Loss values are high and unstable because the network has no predictive structure to latch onto.

Phase 2: Convergence to even-money bets

As epsilon decays (default rate: 0.995 per step), the agent begins to follow its learned policy. The key finding: the Q-network converges to a strong preference for even-money bets (Red, Black, Odd, Even) regardless of the state input. This is purely statistical. The network has discovered that these actions minimise variance.

The action distribution shift is the clearest signal that learning has occurred. Early in training, the 47 actions are roughly equiprobable. By the time epsilon reaches 0.1, the even-money bets account for 60-70% of action selections, while straight number bets are nearly abandoned.

This is not the network "understanding" roulette. It is the network learning that the expected reward of a straight bet is -0.027 per unit but with catastrophic variance, while even-money bets deliver the same expected value with much smaller variance. The Q-network has independently discovered the variance-minimising policy within the constraints of its function approximation.

Phase 3: The loss never converges

Unlike supervised learning, where loss typically decreases toward zero, DQN loss in a stochastic environment oscillates persistently. The root-mean-square error hovers around 0.5-1.0 without trending downward.

This is the "moving target" problem in a purely stochastic setting. The target Q-values depend on the network's own predictions. As the network updates, the targets shift. In a non-stationary environment where the optimal Q-values are themselves noisy estimates of a negative expectation, the network never settles.

Persistent non-convergence of the TD error, together with well-behaved action distributions, is a diagnostic signature. It tells you the environment has no exploitable structure. Any apparent convergence in loss on a real-world RL problem should be examined critically: are you learning genuine structure, or just memorising variance?

Capital evolution

Over repeated episodes, the agent's capital follows a random walk with negative drift. Even with the "optimal" policy of exclusively playing even-money bets, expected loss is approximately 2.7% of total wager per spin. The variance can mask this edge for hundreds or even thousands of spins, creating the illusion of a winning strategy before the mathematics reasserts itself.

Here is the core point: in a zero-signal environment, the best any agent can do is minimise the rate at which it loses money. The agent that converges to even-money betting has "solved" roulette in the only way that is possible for it, by losing the least.

Implementation nuances: where the code diverges from the textbook

A few design choices in the FAIRS codebase illustrate the gap between textbook RL and a working implementation.

Gamma = 0.5 (myopic discounting). Standard DQN implementations use Gamma = 0.95–0.99, allowing the agent to reason about long-term consequences. FAIRS uses 0.5 by default, which gives the standard effective horizon of $1/(1-\gamma)=2$ steps (the discount weight $\gamma^t$ only falls below 1% after about 7 steps, a more generous and nonstandard reading). For roulette, where the data-generating process is i.i.d., this is defensible; there are no long-term consequences to reason about because each spin is independent. A higher gamma would simply accumulate more noise. But it also means the agent cannot learn temporal patterns even if they existed, which is a deliberate constraint that simplifies interpretation.

Reward scaling design. The asymmetric scaling function maps losses to $[-1, 0)$ and wins to $(0, 1]$, with straight-up wins (35:1) scaled to 1.0 and even-money wins scaled near 0.03. This creates an inherent bias: the network sees straight-up bets as having higher scaled expected value than their true mathematics warrants, because the rare +350 reward is compressed into the same $[0,1]$ range as smaller wins. This is a deliberate tradeoff to prevent the extreme variance of straight-up bets from dominating gradient updates, but it subtly warps the action-value landscape.

Frequent target network updates (every 10 steps). Vanilla DQN synchronises the target network every 5,000–10,000 steps (Mnih et al., 2015). FAIRS syncs every 10 steps. This is so frequent that the stabilising benefit of a separate target network is largely negated. The motivation is practical: the agent reaches its training horizon in ~2,000 steps per episode, so a 5,000-step sync interval would never fire during a single training run. The net effect is that FAIRS behaves more like standard Q-learning with a slowly-following target than true Double DQN.

Hierarchical dual-agent architecture. The dynamic betting mode introduces a second DQN agent (the StrategyAgent) that selects between 5 betting strategies (Martingale, Reverse Martingale, D'Alembert, Fibonacci, Keep). Both agents receive the same reward signal simultaneously, creating a non-stationary multi-agent learning problem: each agent's policy shifts the reward landscape the other observes. This is an advanced design pattern that goes beyond most DQN implementations, but its effectiveness in a zero-signal environment is questionable. Both agents are chasing noise.

These observations are not criticisms of FAIRS. It is a research tool, not a production trading system. They are documented here because understanding how an implementation diverges from the textbook is often where the real learning happens.

Implications for applied RL

The FAIRS experiment produces takeaways that transfer to real-world RL applications:

1. Low-signal environments require careful validation. If your agent's reward curve looks like the FAIRS training trace, with high variance, no trend, and action distributions converging to something that looks reasonable, you may be learning noise. Run a permutation test: shuffle the transitions and retrain. If the "policy" survives, it is an artifact.

2. Action distribution analysis is more informative than reward curves. In FAIRS, the reward curve is nearly unreadable. It bounces between positive and negative values for hundreds of episodes. The action distribution, by contrast, tells a clear story. When evaluating RL agents, watch what they choose, not just what they earn.

3. The Double DQN modification matters in stochastic environments. Van Hasselt et al. (2016) showed that DQN overestimates Q-values, particularly in noisy environments. FAIRS uses action selection from the online network with evaluation from the target network, which mitigates this. The difference is modest in a purely random environment (overestimation is bounded by the action space size) but becomes critical in environments with a real signal buried in noise.

Running the experiment yourself

FAIRS is designed to be accessible. The project runs as a local web application with a FastAPI backend and React frontend. The launcher script handles dependency installation:

.\start_on_windows.ps1

The training page lets you upload or generate roulette datasets, configure hyperparameters (episodes, learning rate, epsilon schedule, network architecture), and watch the training metrics live. The inference workspace lets you load trained checkpoints and evaluate policies.

What FAIRS is and is not

FAIRS is a research tool. It provides a clean, instrumented environment for studying how deep Q-networks behave in purely stochastic settings. The training monitors, checkpoint system, and inference workspace are designed to make it easy to run experiments, vary hyperparameters, and observe the results.

FAIRS is not a winning roulette system. The house edge is real. No deep learning architecture, no amount of training data, and no exploration strategy can overcome the mathematics of a negative-expectation game. The project's value lies in what the training process reveals about the algorithm.

FAIRS is open source (MIT). The entire codebase is at github.com/CTCycle/FAIRS-Roulette-Player. You can fork it, extend it with new agent architectures (PPO, SAC, curiosity-driven exploration), add roulette variants, or adapt the environment to other stochastic dynamics.

Closing

The roulette wheel is a useful adversary for RL research precisely because it is not trying to be beaten. It offers no pattern, adapts no strategy, and reveals no structure. Training a DQN against it is a reminder that not every problem is solvable by adding more layers. Recognising the absence of signal is itself a form of learning.

The full source code, including unit tests, end-to-end tests, and a pre-configured development environment, is available on GitHub. Contributions, experiments, and open issues are welcome.

References

Mnih, V., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529-533.
Van Hasselt, H. (2010). Double Q-learning. Advances in Neural Information Processing Systems 23 (NeurIPS), 2613-2621. The original Double Q-learning paper, which used a roulette game as an evaluation environment.
Van Hasselt, H., Guez, A., & Silver, D. (2016). Deep reinforcement learning with double Q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1).
Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. International Conference on Learning Representations (ICLR).
FAIRS source code: app/server/learning/ (MIT License).

Built by CTCycle. ML researcher and data scientist working on clinical AI, RL agents, and scientific computing tools. Find me at github.com/CTCycle and on Dev.to at @ctcycle.