DEV Community: Meryeme ramdi

The day I asked three LLM agents to rewrite legacy Java for me — and what actually happened

Meryeme ramdi — Thu, 20 Aug 2026 12:56:22 +0000

1. The question that started everything

Three weeks into my internship, my supervisor sat down across from me and asked, very casually:

"OK your NLP pipeline extracts intentions and rules from legacy Java. Nice. And then what?"

I looked at him. I looked at my laptop. I looked back at him.

The whole project — Pulsar Modernizer — was supposed to eventually turn legacy Java into modern Spring Boot code. My part was the "understand the old code" part. F1 = 0.857 on the annotated corpus, a shiny React UI, everything humming in Docker.

But the "and then?" was doing a lot of work in that sentence.

That evening I wrote in my notes: "Nobody has actually tried the generation part. Everyone assumes it'll be easy because LLMs. That is very obviously wrong."

So I decided to try.

2. Why "just prompt an LLM to rewrite it" doesn't work

The naive move — feed the old code and the extracted rules to an LLM and say "please modernize this" — has three problems and I hit all of them in the first hour:

The model hallucinates. It happily invents helper classes that don't exist and calls methods with the wrong signature.
You have no criterion for stopping. The model tells you "it's done ". OK. Is it? By what test?
You have no criterion for equivalence. Even if it compiles, how do you know the new code actually does what the old one did?

I needed something more constrained than "prompt it and pray".

3. The setup — a chain, not a monolith

I ended up building three specialized agents in sequence:

IntentCard + RuleCards
        │
        ▼
[APIDesigner] ──► JSON contract (class, methods, DTOs, throws)
        │
        ├───────────────┐
        ▼               ▼
[CodeGenerator]    [TestGenerator]
        │               │
        ▼               ▼
     .java          *Test.java
        │               │
        └────► verifier (mvn test)

The key insight: each rule extracted from the legacy code should become a test that the generated code has to pass.

This flips the whole thing. I don't trust the LLM. I trust javac and JUnit.

I did all of this on a local model — Qwen 2.5 Coder 3B via Ollama. No cloud APIs, no data leaving my Mac. On a 3B model, the constraint is fair: if the pipeline needs GPT-4-level intelligence to hold together, it's not going to fly in a bank in production.

4. The three cracks I never expected

Here's where it got interesting.

I hand-wrote the JUnit test-oracles for four rules from my banking corpus. Then I asked the LLM to generate the code. Baseline result:

2 out of 3 oracles green with a "raw" prompt.
3 out of 3 with a "clarified" prompt.

The difference between the two prompts was three conventions I added at the top:

null inputs are always rejected (never silently ignored)
"cannot exceed N" means N is accepted, N + ε is rejected
Currency/country lists are case-sensitive by default

Every failure I observed traced back to one of these three assumptions being different in my head vs the LLM's head.

Which means: the three cracks were in my own annotations, not in the model. My RuleCards were technically valid English sentences that a human reader would interpret one way, and a language model would interpret the other way. Nobody had written down the tiebreaker.

That was the aha moment of the whole internship. Not "LLMs are amazing". Not "LLMs are terrible". Just: your dataset has hidden ambiguities, and a fluent model surfaces them by making the opposite choice from what you meant.

5. The Markov-1 pathological fixed point

I got greedy. I built a repair loop: if the generated code doesn't compile, feed the compiler error back into the prompt and ask the model to fix it. Up to 5 iterations.

Round 1: on a simple case (DevisePermise — accepted currencies), the 3B model produced code that treated null as .toUpperCase() and crashed at runtime. I fed the error back. Round 2: fixed. 2 iterations, converged. Beautiful.

Then I threw a harder rule at it: LimiteVelociteTransactions — count more than 20 transactions in a sliding hour window, add +30 to the fraud score.

Round 1: if (nowTimestamp == null) — but nowTimestamp is a Java long primitive. long can't be null. javac refused. I fed the error back.

Round 2: same code. Same error.
Round 3: same. Same.
Rounds 4, 5: exactly the same generated code.

The loop had converged — on broken code. The 3B model, presented with a clear javac message, could not figure out that long isn't Long. And because my repair prompt was Markov-1 (only the previous attempt), it kept producing the same fixed point.

The takeaway: the fact that a repair loop "converges" tells you nothing about whether it converges on something correct.

6. What I actually shipped

Full disclosure — this isn't a production system. It's a slice, on 4 rules, in a language (Java) that has parsers everywhere, with a corpus I annotated myself. The results are directional, not statistical.

But the plumbing works end-to-end. A user pastes a Git URL in the interface. The system clones, extracts intentions with the NLP pipeline, persists them in PostgreSQL as a Cognitive Knowledge Graph, and — from a single click — chains the three agents to produce a downloadable zip of modern Spring Boot code. On Spring PetClinic (30 files), the loop currently produces four working Java classes with tests in about 10 minutes.

You can compile the zip. You can read the tests. You can also see, in the output, that the model sometimes invents helper types the contract never mentioned. That's fine. That's what the human-in-the-loop is for.

7. What I'd do differently

Three things that I already know I got wrong and would fix on day one of a v2:

Escalate to a bigger model after N repair failures. 3B is fast; 7B has 4× the reasoning depth. Cascade them.
Use a validator of attribution. Before generating tests, check that every method signature the LLM invents is in the APIDesigner's contract. Reject and re-ask if not.
Amend the annotation doctrine with three explicit fields (null_policy, bound_inclusivity, case_sensitive) instead of relying on three "conventions" in the prompt. Right conventions in the wrong place.

8. The one thing I'm actually taking away from this

Every tutorial I read before starting this project was about how to prompt an LLM to write good code. None of them talked about how to know if the code is good.

That's the whole game. The LLM is the cheap part. The evaluator, the oracle, the human-in-the-loop, the annotation doctrine — that's where all the actual work lives.

A 3B model with a well-designed loop beats a 70B model with none.

Meryeme Ramdi is a second-year AI engineering student at ENSIAS, currently interning on the Pulsar Modernizer project at Pulsaride Solutions. She writes about ML systems that ship, at dev.to/meryyy.

How I Built an AI That Tells Doctors "Wait ...Are You Sure?"

Meryeme ramdi — Tue, 07 Jul 2026 14:13:39 +0000

How a student project turned into a lesson about humility, hard negatives, and why the best models know when to shut up.

The number that wouldn't leave me alone

I remember the exact article that ruined my week.

It was a study saying that diagnostic errors affect around 12 million adults in the US every year. Twelve million. That's roughly one in every twenty outpatient visits, and a decent chunk of them cause real harm.

I read it, closed my laptop, and then reopened it ten minutes later because my brain was doing the thing it does the "but what if you could just…" thing.

I'm a student. I'm not going to reform healthcare. But I do know how to train models. And the problem, when you squint at it, looks like a machine learning problem: given a set of symptoms and a proposed diagnosis, does this pairing actually make sense?

So I started building. The project became my final-year adventure. I called it Misdiagnosis Detector, which is a slightly dramatic name for what it actually does: it's a second opinion. A tiny, opinionated little AI that reads a case and says one of three things:

"Yeah, that fits."
"Hmm, maybe double-check."
"No, that doesn't add up."

That third option ; the confident "no" , is the fun part. But getting there was a saga.

Attempt 1: The Naive Version (and why it embarrassed me)

My first instinct was the same instinct every ML student has: throw a classifier at it.

I scraped clinical cases from PubMed. I extracted symptoms and diagnoses. I trained a model to predict "diagnosis given symptoms." Classic supervised learning.

It got ~90% accuracy on my test set.

I was ecstatic for about six minutes, which is roughly how long it took me to realize the model was cheating. It had learned that "chest pain + shortness of breath" almost always co-occurs with "myocardial infarction" in the data because PubMed publishes the interesting cases where the diagnosis was, in fact, an MI. My model wasn't verifying anything. It was pattern-matching on publication bias.

A doctor doesn't need an AI to tell them "chest pain probably means heart attack." A doctor needs an AI to tell them when the symptoms and the diagnosis they wrote down don't actually match.

That's a different problem. And it required a completely different framing.

Attempt 2: Learning what "wrong" looks like

The reframe: instead of predicting the diagnosis, I'd train a model to score coherence — how well any given symptom set matches any given diagnosis.

For that, I needed pairs of symptoms and diagnoses that don't match. Not made-up gibberish — plausible-but-wrong pairings, the kind a tired resident at 3am might actually write down.

This is where I met my new best friend: hard negatives.

Instead of pairing "chest pain" with "sprained ankle" (too obvious, model learns nothing), I mined titles from PubMed to find diagnoses that sound like they could fit the symptoms but don't. Pneumonia paired with heart failure. Migraine paired with meningitis. Anxiety paired with hyperthyroidism. The lookalikes. The gotchas.

The model that came out of this was more interesting, but the metric that mattered wasn't accuracy anymore , it was AUC, because I wanted a score, not a hard classification. I got to around 0.80 on my Phase 2 model.

Not bad. Not good enough to put in front of anyone.

Attempt 3: Standing on the shoulders of giants (aka PubMedBERT)

Here's the thing about building medical NLP as a student: you cannot possibly train a language model that understands medicine from scratch. Not with a laptop. Not with your university's GPU. Not in this lifetime.

But you don't have to. Microsoft trained one for you. It's called PubMedBERT, and it has already read basically every biomedical paper on Earth.

So I stopped trying to be clever with raw text. I ran everything — symptoms, diagnoses, case descriptions through PubMedBERT to get embeddings. Then, on top of those frozen embeddings, I trained a small projection head with a contrastive objective: pull matching (symptoms, diagnosis) pairs close in vector space, push mismatches apart.

Small model. Big shoulders. Suddenly my AUC jumped to ~0.92.

I stared at the number for a while. It felt too good. I retrained it four times to make sure I hadn't leaked test data. I hadn't.

The part where I almost shipped a lie

At 92% AUC, the temptation was to just… ship it. Slap a green/red output on it, call it a demo, put it on my CV.

But here's the thing about medicine that I kept slamming into: a model that's confidently wrong 8% of the time is worse than no model at all. If a doctor trusts your green light and a patient dies, "well, my AUC was 0.92" is not a defense.

So I did the least sexy but most important thing in the project: I made the model shut up when it wasn't sure.

I built a three-zone decision system:

Score below 0.15 → "This diagnosis doesn't fit the symptoms." (red)
Score above 0.89 → "This checks out." (green)
Everything in between → "I don't know. Verify manually." (yellow)

The middle zone is the whole point. It's the model saying "I refuse to bet on this." Around 36% of cases fall into it, which sounds bad until you realize the model's accuracy on the cases it does commit to is 81%.

I would rather build an AI that admits ignorance on a third of cases than one that fakes confidence on all of them. This is the lesson I want tattooed on every ML student's forehead: coverage is not the same as competence.

Making it useful, not just correct

A number on a screen isn't a product. So I built a Streamlit app around it, and added two things I'd want if I were the doctor using it:

A differential diagnosis. When the model rejects the proposed diagnosis, it doesn't just say "no" , it suggests the diagnoses that would fit the symptoms better, ranked by coherence score. Because "you're wrong" is unhelpful. "You're wrong, and here are three things it might actually be" is a second opinion.
Similar cases from PubMed. For every prediction, the app shows real published case reports with similar symptom profiles, retrieved via a semantic search over MiniLM embeddings. Because a doctor should never trust the AI blindly — they should trust the AI plus the evidence the AI is pointing at.

I also spent a stupid amount of time on vocabulary mapping. Doctors write "common cold." The model was trained on "URTI" (upper respiratory tract infection). Doctors write "heart attack." The training data says "STEMI" or "NSTEMI." I built a small clinical synonym mapper so the model doesn't get thrown by the fact that real people use real words.

What I actually learned (the parts that don't fit on a CV)

If you scroll to the top of my repo, you'll see a README with metrics and architecture diagrams. What you won't see is the stuff I actually take away from this project:

Models that abstain are better than models that guess. The most valuable thing I built wasn't the classifier , it was the yellow zone. Knowing when not to answer is a feature, not a bug.

Publication bias is a monster. Every dataset lies to you in some way. PubMed publishes the interesting cases; Reddit publishes the anxious ones; NEJM publishes the impossible ones. You have to know your data's personality before you can trust what your model learned from it.

"92% AUC" is a beginning, not an ending. It's the moment you start asking harder questions: 92% on which subgroup? At what threshold? With what failure modes? A model that looks great in aggregate can be catastrophic in the tails.

Frozen giants + a tiny learned head beats an ambitious student with a from-scratch model, every single time. Use PubMedBERT. Use SapBERT. Use whatever's out there. Your originality goes into the problem framing, not the architecture.

Boring engineering matters. Half my final commits were about handling common cold vs URTI, calibrating thresholds, and making the Streamlit UI not look terrible. None of that goes on the model card. All of it decides whether the tool is usable.

The version I actually want to build next

If I keep going with this — and I probably will — the next version needs three things:

Uncertainty that a doctor can act on. Right now the middle zone just says "verify." I want it to say why it's unsure. "The symptoms overlap with three other diagnoses" is more useful than "score = 0.42."

More languages. All my training data is in English. Medicine happens everywhere. This is a fixable problem and a real one.

A humbler evaluation. My 5-out-of-8 score on reference clinical cases is fine for a demo. It is not fine for anything real. I want to build a proper eval set with medical students grading outputs, and iterate against that.

Why I wrote this down

If you're a student staring at some overwhelming problem and thinking "I can't fix healthcare / climate / education / whatever" — you're right. You can't. Neither can I.

But you can pick one small, well-defined corner of the problem and build a thing that would make one specific person's life 3% easier. That is enough. That is, honestly, more than enough.

Twelve million diagnostic errors a year is not a number you fix with a Streamlit app. But it might be a number you chip at with tools that whisper "wait, are you sure?" at exactly the moment someone needed to hear it.

I built one of those tools. It's imperfect and it's mine and I learned more from it than from any course I've ever taken.

If you want to see it, the code is on GitHub and the live demo is on Hugging Face Spaces. If you want to build something similar and get stuck, my inbox is open.

Now go find your twelve-million-shaped problem.

Btw, this is my first time writing about my work publicly, but it definitely won't be the last. I built this project to learn, but I am always looking for the next thing to build.
I’m an AI student seeking an internship for next year, and I am eager to collaborate on new projects and tackle completely different datasets. My inbox is open for new ideas, feedback, or internship opportunities. Drop me a line at meryemeramdi05@gmail.com, and let's connect on LinkedIn : https://www.linkedin.com/in/meryeme-ramdi-5a879b388/