I’ve been working on a small open-source project called MEVA (Medical Evidence Verification Agent).
The idea came from a simple problem I kept thinking about: an AI agent can retrieve the correct information, but that doesn’t necessarily mean the answer it gives actually matches that information.
So I wanted to separate those two things.
MEVA works with synthetic FHIR medical records and lets an AI agent retrieve evidence through MCP tools. After the answer is generated, MEVA checks the claims against the retrieved evidence using normal deterministic Python logic.
The important part for me was that another LLM doesn't decide whether the answer is correct.
The verifier returns one of four results:
- SUPPORTED
- CONTRADICTED
- UNSUPPORTED
- UNVERIFIABLE
I also built a benchmark setup for comparing local models and a small Streamlit sandbox so the verification part can be tried directly in a browser.
The public sandbox doesn't run an AI model. You choose a synthetic patient, create a claim, and MEVA checks that claim against the patient's recorded FHIR evidence.
Everything uses synthetic Synthea patient data. There is no real patient data, and the project isn't meant for diagnosis or treatment decisions.
The project is still pretty new, so I’m mainly looking for feedback from people interested in AI evaluation, FHIR, MCP, local LLMs or Python.
There are also a few open issues for anyone who wants to contribute.
GitHub:
https://github.com/Tanz2024/meva-health-ai
Live sandbox:
https://meva-health-aigit-exbml8bjbokk28zs6amu3h.streamlit.app/
If you try it and find something confusing, broken, or something that could be designed better, feel free to let me know.
Top comments (0)