DEV Community: jack

What Kind of Medical AI Can Ordinary Hospitals Actually Use?

jack — Thu, 29 Oct 2026 15:48:40 +0000

What kind of medical AI can an ordinary hospital actually use?

I mean a hospital that needs useful tools without assuming a large, fully paired research dataset, a specialist AI engineering team, or access to every advanced assay.

My clinical background shapes the question I bring to AI research:

When samples are scarce, modalities are incomplete, and platforms differ, should we pursue more complex models—or organise evidence more explicitly and report the boundaries of what the model can support?

I don't think these are mutually exclusive choices. More complex models can be useful. But each added component should earn its place through reproducible benefit under the conditions in which it will be used.

What I mean by constrained AI

Here, “constrained” refers to the conditions around the model: limited samples, incomplete modality pairing, compact representations, platform differences, and finite computing and maintenance resources. It does not refer to restrictions on what a chatbot is allowed to say.

These constraints can occur together. A model may have little labelled data, only some of the expected inputs, and measurements produced by a platform different from the one used during development.

That changes the engineering question. We need to ask what evidence remains usable, what changes when an input disappears, and when a model's earlier validation no longer applies.

A research example from our preprint

Our preprint, Restricted Multimodal Oncology AI: Role-Aware Evidence Organisation and Cross-Domain Transfer Boundaries, explores this problem in retrospective oncology cohorts through Role-Aware Evidence Engineering (RAEE) and its BLF-Net-MUL implementation.

The design starts by assigning evidence roles before evaluating fusion:

Input	Intended evidence role
Compact transcriptomic features	Molecular evidence
RNA-derived pathway features	Functional molecular context
Low-dimensional pathology-derived features	Phenotypic context
Clinical variables	Patient-level context

The pathway features are derived from RNA; they are not an independent omic measurement. The pathology inputs are compact features, not whole-slide foundation-model embeddings.

The question is whether combining these sources helps under a specified condition—not whether more modalities must always be better. Component analyses and external transfer tests are useful precisely because they can reveal where a mechanism stops helping.

This is a research example, not a demonstration that our system is ready for routine hospital use. The linked manuscript is a preprint and has not been peer reviewed. Retrospective classification and cross-repository evaluation do not establish prospective clinical benefit or reliable deployment at a new hospital.

What I would want before ordinary hospitals rely on a system

These are the requirements I would put on a development and evaluation plan:

Obtainable inputs. State exactly which measurements are required and whether the intended hospital can obtain them. Test missing-input scenarios explicitly; supporting incomplete data in a research protocol is not a guarantee of safe operation with any missing modality.
Affordable operation. Measure inference, storage, integration, maintenance, and staff time. Compact inputs alone do not establish a low total cost.
Evidence from the destination setting. Evaluate relevant equipment, laboratory platforms, patient groups, and workflows. A source-cohort score cannot stand in for local validation.
A defined response to uncertainty. Specify when the system should flag insufficient evidence, request review, or withhold an output. These behaviours need validation too.
Traceable outputs. Let a reviewer see which inputs were present, which model version produced the result, and which limitations apply.
A place in clinical work. Define who reviews the output, what action it is intended to inform, and how errors, drift, and overrides will be monitored. Any necessary institutional and regulatory review belongs in that deployment process.

These are proposed requirements for usable medical AI. They are not a list of capabilities already established for BLF-Net-MUL.

Complexity should answer a concrete need

I would like to see model comparisons answer a practical question: does this additional complexity improve a relevant outcome enough to justify its data, compute, and maintenance requirements?

A simpler model still needs rigorous validation. A larger model may be the better choice when the evidence supports it. Neither model size nor an interpretable-looking report establishes safety on its own.

In our research, the intended output stops at bounded evidence preparation for expert review. It is not an autonomous diagnosis or treatment recommendation. Explicit boundaries make a claim easier to examine; they do not eliminate clinical risk.

What would you require?

If you build medical AI, maintain hospital software, or work with clinical teams, I would value your perspective:

What would make an AI system usable in a hospital without a dedicated AI infrastructure team?
Which failure matters most in your setting: a missing input, platform shift, poor calibration, an opaque output, or maintenance burden?
What evidence would convince you that added model complexity is worth it?

What should a medical AI system have to demonstrate before an ordinary hospital can reasonably rely on it?

Please keep examples free of identifiable patient information.

Preprint: Hu et al. Restricted Multimodal Oncology AI: Role-Aware Evidence Organisation and Cross-Domain Transfer Boundaries. doi:10.21203/rs.3.rs-10194336/v1.

Disclosure: I am a co-author of the cited preprint. This post was prepared with AI writing assistance from our manuscript and the discussion above. It presents research questions and design priorities, not clinical advice or a claim of deployment readiness.

A Green Test Doesn't Mean an AI Coding Agent Finished the Job

jack — Thu, 29 Oct 2026 15:42:28 +0000

A coding agent says, “Done.” The test suite is green. The diff looks plausible.

But which requirement did the tests actually verify?

This is a question I keep returning to while working on AI-assisted coding workflows. The risk is not only that an AI assistant generates incorrect code. It is that we can mistakenly turn a narrow observation—one command passed—into a much broader claim—the requested behavior is correct.

A green test can be completely irrelevant

Imagine the requirement is:

Reject invalid email addresses during signup.

The application currently contains:

def signup(email: str) -> bool:
    return True

And the test suite contains:

def test_smoke():
    assert 1 + 1 == 2

Running pytest -q will report success. It should: that assertion is true. But the test has not exercised signup, let alone invalid input.

Add a test that actually checks the requirement:

def test_signup_rejects_invalid_email():
    assert signup("not-an-email") is False

That test fails against the current implementation. Nothing mysterious happened: the first green result was real, but the conclusion drawn from it was too strong.

This matters when a coding agent summarizes a run as “all tests pass” and we silently interpret that as “your requested feature works.” Those are different statements.

Five questions worth keeping separate

When reviewing AI-generated changes, I find it useful to separate at least five evidence roles:

Role	Question	What it does not prove
Intent	What behavior did the user request?	That it was implemented
Change	Which files and revision changed?	That the change was correct
Verification	Which checks actually ran?	That they covered every requirement
Environment	Where did those checks run?	That another environment behaves identically
Boundary	What remains untested, stale, or unknown?	That nothing else can fail

A coding-agent transcript can provide helpful context, but its statement that a task was completed is still a claim. A test result is an observation. A signed CI statement may authenticate provenance. None of these can replace the others.

Making an evidence gap visible

I've been experimenting with this idea in an open-source Python CLI called RAEE-Lite. Its narrow goal is to keep the requirement and its supporting evidence separate rather than automatically turning a passing test into a verified feature.

The current public source is a v0.10.0rc2 release candidate, available for review—not a stable or independently security-certified release. If you want to try it, inspect the source candidate and install it in a disposable Python environment.

Then, in a new temporary project directory:

raee-lite init
raee-lite add "Signup rejects invalid email addresses"
raee-lite expect --for R1 --kind unit
raee-lite status
raee-lite check

The intended result is that requirement R1 remains UNTESTED, with an evidence-gap report. raee-lite status intentionally exits with status 1 while the requirement lacks evidence; the default check operation doesn't execute your project's tests.

This is deliberately not an automatic proof of code correctness. RAEE-Lite does not semantically certify that a test covers an entire requirement. It makes the absence of documented evidence visible and leaves the judgment open to human review.

The second problem: stale evidence

Even a relevant test result should be tied to a particular source revision. If a file changes after a test run, the earlier result does not automatically apply to the new code.

The same distinction applies to environments. A check executed locally is not identical to a check executed on a protected CI branch, and a signed provenance statement authenticates who made a claim—not whether its test design was sufficient.

This is why I prefer reports that say “untested” or “stale” rather than projecting certainty they haven't earned.

What remains unverified

RAEE-Lite is still in early access. Although it has local test results and synthetic security regression cases, the release candidate has not yet completed the protected-main Linux/Windows/macOS acceptance matrix, independent GitHub Artifact Attestation verification, or third-party security review. It's also not a substitute for comprehensive test coverage, a security audit, or rigorous code review.

I'd value developer feedback on the evidence model itself: What do your AI coding tools claim after a green test? Which kinds of missing evidence would help you catch false confidence? A sanitized, minimal reproduction is more useful than a star or promotional repost.

The public project, including Chinese, English, German, and Spanish documentation, is at github.com/jinxinche/RAEE.

Research context: The idea of keeping evidence roles distinct was motivated by a separate oncology AI preprint: Hu et al. (2026), Restricted Multimodal Oncology AI: Role-Aware Evidence Organisation and Cross-Domain Transfer Boundaries, DOI:10.21203/rs.3.rs-10194336/v1. That preprint did not evaluate or validate this coding tool.

Writing disclosure: This article was drafted with AI writing assistance based on my project's documentation and test records. I am responsible for reviewing the technical claims before publication.

A Green Test Doesn’t Mean an AI Coding Agent

jack — Thu, 29 Oct 2026 15:34:20 +0000

A Green Test Doesn’t Mean an AI Coding Agent Finished the Job” description: “A reproducible example of why passing tests, verified requirements, and coding-agent claims are different kinds of evidence.” published: false tags: ai, testing, python, productivity