DEV Community: Chidozie Uzoegwu

I Swept All 33 Bedrock Regions So You Don't Have To

Chidozie Uzoegwu — Tue, 08 Sep 2026 18:11:46 +0000

The Amazon Bedrock pricing page publishes a training price for the Meta model you almost certainly are not fine-tuning.

Here is the customization pricing on the page, as of 8 September 2026:

Model	Published training price
Llama 2 Pretrained 13B	$1.49 per 1M tokens
Llama 2 Pretrained 70B	$7.99 per 1M tokens
Cohere Command	$0.004 per 1,000 tokens
Titan Image Generator	$0.005 per image seen
gpt-oss-20b	$80 per training hour
Qwen3 32B	$80 per training hour
Llama 3.1, Llama 3.3	none

Other providers are covered. Two models are even priced by the hour. The only Meta model with a published training price is Llama 2, and the Llama you would actually fine-tune today is not on the list.

I have a fine-tuned Llama 3.3 70B running in production on Bedrock, and a line item on my AWS bill saying it cost real money to train. So the capability exists, it bills, and I could not find its price published anywhere.

A caution while you are here: I have seen third-party guides quote "$0.00799 per 1,000 tokens" as the Llama 3.3 70B training rate. That is the Llama 2 70B figure restated and applied to a different model. If you are budgeting, price it from your own first job rather than from a number someone inferred.

That gap turned out to be the least surprising thing I learned. This article is the map I wish I had before I started: which regions can actually do this, what the documented way around it says versus what happened when I took it, and the one piece of job state that decides whether a stalled job is costing you money or nothing at all.

Everything below is either something I measured or something AWS documents, and I have labelled which. Where the two disagree, you get both.

Part 1: the sweep

I did not want to work from a documentation table. The Bedrock catalogue varies by region more than people expect, and a page describing 33 regions is a hard thing to keep current. So I asked the API directly, in every region the SDK knows about.

import boto3, concurrent.futures as cf
from botocore.config import Config

cfg = Config(connect_timeout=6, read_timeout=12, retries={"max_attempts": 1})
regions = boto3.Session().get_available_regions("bedrock")

def probe(region):
    try:
        client = boto3.client("bedrock", region_name=region, config=cfg)
        models = client.list_foundation_models(
            byCustomizationType="FINE_TUNING"
        )["modelSummaries"]
        ids = sorted({m["modelId"].split(":")[0] for m in models})
        return region, len(ids), ids
    except Exception as e:
        return region, "ERR " + type(e).__name__, []

with cf.ThreadPoolExecutor(max_workers=12) as ex:
    for region, count, ids in sorted(ex.map(probe, regions)):
        print(region, count, any("llama3-3-70b" in i for i in ids))

byCustomizationType="FINE_TUNING" is the important part. It returns only the models you can actually train, not the much longer list you can invoke.

33 regions. Here is what came back.

Result	Regions
Can fine-tune Llama 3.3 70B	1 (us-west-2)
Answered, some tunable models, no Llama	2 (us-east-1, eu-west-2)
Answered, zero tunable models	17
Opt-in regions, not enabled on my account	13

One region can fine-tune Llama 3.3 70B. Out of the 33 the SDK lists, and out of the 20 that gave me a real answer. Not a short list. One.

An honest caveat, because the difference matters. Those 13 are not a fine-tuning answer. They returned UnrecognizedClientException, which is what you get when you call an opt-in region your account has never enabled. I cannot rule them out from this account, and neither can you from yours unless you have opted in. Twenty regions gave a real answer.

The two regions that answered with something other than zero are more interesting than the seventeen that answered zero.

us-west-2, five tunable models:

amazon.titan-embed-image-v1
anthropic.claude-3-haiku-20240307-v1
meta.llama3-1-8b-instruct-v1
meta.llama3-1-70b-instruct-v1
meta.llama3-3-70b-instruct-v1

us-east-1, six tunable models:

amazon.nova-micro-v1
amazon.nova-lite-v1
amazon.nova-pro-v1
amazon.nova-2-lite-v1
amazon.nova-canvas-v1
amazon.titan-embed-image-v1

Read that second list again. In the region most people default to, every model you can fine-tune is an Amazon model. No Llama. No Anthropic. If you want to customise anything that is not Amazon's own, us-east-1 cannot do it.

The console agrees. Opening Create Fine-tuning job in us-east-1 on 8 September 2026, the model picker offers one category, "Serverless model providers", and one provider under it: Amazon. The six models it lists are the same six the API returns.

And eu-west-2, London, where a lot of UK and EU workloads want to sit for latency or residency reasons, has exactly one tunable model. It is not a Llama.

This has been stable. I first ran the sweep on 1 August 2026 and re-ran it on 8 September 2026 before publishing. Same answer both times. That is five weeks, not five years, so re-run it yourself rather than trusting my table. The snippet above takes about a minute.

Part 2: the escape hatch, and what happened when I tried it

The obvious way out is to train the model somewhere else and bring the weights in. Bedrock has Custom Model Import for exactly that, and on paper it fits.

The documentation is clear and it is worth quoting accurately, because my experience differed from it and I want you to have both.

Custom Model Import lists Llama 3.3 among its supported architectures. It states the ceiling in terms of weight size rather than parameter count: under 200GB for text models, with a maximum context length below 128K. It is available in eu-central-1, us-east-1, us-east-2 and us-west-2, which is four regions to native customization's one. AWS has published a walkthrough of importing DeepSeek-R1-Distill-Llama-70B through it.

By that description, importing a fine-tuned Llama 3.3 70B should work. My artifacts were 141GB in bf16, comfortably inside the documented 200GB.

The import failed:

too large to fit on available hardware

That is the whole error. It does not say which limit was hit, whether the constraint was my artifacts, the region, or capacity at that moment. Searching it turns up other people receiving the same generic message on far smaller models, including an 8B, which suggests it is a catch-all rather than a statement about 70B specifically.

So the honest position is narrower than "it cannot be done." The documented path supports the architecture and the size. It did not complete for me, on my artifacts, in my account, on the day I tried, and the error was not specific enough to tell me why. Someone else may well succeed with it, and AWS's own material suggests they do.

What that means practically, if you are planning:

Native customization is the constrained path. One region for Llama 70B, and no choice about it.
Import is the less constrained path on paper, including into eu-central-1, which matters if EU residency is a hard requirement for you. Try it before you conclude anything from my result.
Budget for the possibility that it does not complete, and for an error that will not tell you why.

For my own build, native customization is what worked, so that is where the model lives, and the application runs in a different region from the custom model deployment. To be precise about which half of that was forced: the customization and its deployment had to be in us-west-2. Where the application runs was my choice, and I could move it.

Part 3: InProgress means two completely different things

This is the part I have not found documented anywhere, and it is the one that costs money.

Launch a customization job and poll it. The status reads InProgress.

That tells you nothing useful, because InProgress covers two states that are not remotely the same:

The job is sitting in a capacity queue waiting for a GPU. No trainer has started. You are being charged nothing.
The job is training. You are burning GPU time.

From the outside these look identical. Same status, same API, same console. The only thing that separates them is nested one level down:

job = bedrock.get_model_customization_job(jobIdentifier=job_id)

job["status"]
# 'InProgress'   <- tells you nothing

job["statusDetails"]["trainingDetails"]["status"]
# 'NotStarted'   <- queued, free to stop
# 'InProgress'   <- training, stopping now pays for nothing

statusDetails.trainingDetails.status is the only honest signal. Check it before you stop anything.

Why it matters in cash terms. Over one week of capacity contention I stopped five jobs. Four had never left the queue.

Jobs stopped	Reached the trainer	Billed
4	No	$0.00
1	Yes	$29.89

Four jobs, each killed after many hours, cost nothing at all. Queue time is free. Had I believed the top-level status, I would have assumed all five cost me something and drawn exactly the wrong conclusion about how expensive iteration is.

This is consistent with what AWS documents, once you read it precisely. The guidance on stopping a job says Bedrock charges for the tokens it used to train the model before you stopped it. If training never started, no tokens were used, and there is nothing to charge for. The documentation and the invoice agree. What neither tells you is which of the two states your job is in, and that is the gap trainingDetails fills.

One more thing that trips people up here: validation completing does not mean training has started. Validation proves your S3 permissions and your data schema are fine. It says nothing about whether a GPU has been allocated.

Part 4: the threshold I invented

Here is the mistake, because it is more useful than the finding.

Jobs were stalling in the queue. I went back through my own job history and worked out that the longest queue wait ever followed by successful training was about 11 hours. So I adopted a rule: past 11 hours the job is hung, kill it and relaunch.

That rule was garbage, and it took me a while to see why.

Every job in my history that had waited longer than 11 hours had been killed by me before it could recover. The number did not measure Bedrock's behaviour. It measured my patience. I then used it to justify killing the next job, which fed the same number back into the same conclusion.

The data point that broke it: one job waited 23 hours and 15 minutes in the queue, then trained normally and produced a working model.

An earlier job had been stopped at 17 hours 31 minutes on the strength of a threshold that did not exist. It cost $0.00, because it had never left the queue, but it cost most of a day.

The queue has no published upper bound that I could find. The documented threshold applies to a different state. AWS's troubleshooting guidance says training time runs "between 3-4 hours, up to 24 hours, depending on configuration and traffic", and that you should contact Support if a job has been in Training for more than 24 hours. In Training, not in queue. The phrase "depending on configuration and traffic" is doing real work there: capacity is shared, and the wait before training is not the thing being bounded.

The correct response to a stalled customization job, given that queue time is free and every relaunch goes to the back of the queue: wait. Impatience has a real cost here. Patience has none.

Part 5: the things that will bite you next

Four more, briefly, all learned the expensive way.

The schema is per model family. The limits are not. Llama 3.3 takes the Converse format, with schemaVersion, system[] and messages[]. Llama 3.1 rejects it outright, with ValidationException: Unable to parse S3 file due to invalid data schema/format, and wants flat {"prompt": ..., "completion": ...} instead. Both share the same 16,000 token ceiling, the same 100 to 10,000 record range, the same epoch and learning rate bounds. Matching limits are not matching schemas, and I lost a launch to assuming otherwise.

prompt/completion has no system field. Move from 3.3 to 3.1 and your system prompt has to be prepended into every training record, then reproduced byte for byte at inference. Get one character wrong later and the model quietly degrades, with no error anywhere.

Your real concurrency limit may be smaller than you think. The quota that binds is Custom models with a creating status per account, which on my account is 2. Not the scheduled-customization-jobs quota, which on my account is 10 and looks like the constraint until you hit the other one. Check both in Service Quotas rather than assuming my numbers are the defaults.

Storage is per model per month. On my bill, a retained rollback model comes to about $1.95 a month to sit there. Cheap, and worth it for the rollback, but not free, and easy to accumulate.

The decision tree, compressed

If you are considering fine-tuning a Llama on Bedrock, the honest version is short.

For native customization, the region is decided for you. us-west-2 or nothing, for Llama 70B. If you go that route, design the cross-region call in from the start.
Try Custom Model Import before you accept that. It documents support for Llama 3.3, a 200GB text ceiling and four regions including eu-central-1. Mine failed with a generic hardware error inside the documented limit, so budget for that outcome, but do not take my result as the rule.
Check trainingDetails, never the top-level status, before you stop a job or conclude anything about cost.
When it stalls, wait. Queue time is free, relaunching is not, and any hang threshold you derive from your own history is measuring you.
Read the schema doc for your exact model version, not the family. The limits will match and the format will not.

None of it sits in one place, which is why it is here. The pricing page still says Llama 2.

Don't Fine-Tune. Unless You Can Answer These Three Questions.

Chidozie Uzoegwu — Tue, 25 Aug 2026 19:21:17 +0000

Twice now I have gone looking for a retrain and found the problem somewhere else.

The first time, a third of my output had collapsed into one template. Before spending anything I counted the pattern in my training data. 5 occurrences in 1,610 examples. Then I counted it in the output. 36%. The cause was a single example I had hardcoded into my own prompt months earlier, and the model was copying it.

The second time, quality had gone flat across the board. The system prompt had grown to 224,833 characters, roughly 56,000 tokens on every call, and ninety percent of it was accumulated rules about how the output should sound. The model was doing exactly what I had asked it to do, which was avoid things.

Neither needed new weights.

That is the pattern I would put in front of anyone about to fine-tune. It is the most expensive way to discover you had a prompt bug. About $30 and a five-hour job per run on a 70B model, and the compute is the small part. The real cost is that you now have two variables moving and no clean way to tell which one broke.

So, three questions.

1. Is this a knowledge problem or an identity problem?

Fine-tuning does not teach a model facts. It shifts a distribution.

If the model needs to know things it currently does not, your product details, your documentation, anything that was true last week and false today, that is retrieval. Training knowledge into weights means retraining every time the knowledge changes, which is a subscription you did not mean to sign.

Fine-tuning is for behaviour that stays put. How it writes. What register it falls back to. What it does when the input is ambiguous.

The test I use: if the right answer changes when the underlying data changes, it belongs in retrieval.

2. Have you stripped the prompt back first?

Most requests to fine-tune are prompt bloat wearing a disguise.

Prompts accrete. Something goes wrong, you add a rule, it improves, and nothing ever gets removed, because removing a rule feels like inviting the bug back. Six months later two hundred lines of prohibitions are competing with two lines describing the actual job, and the model attends across all of it.

Before you price a training run, cut the prompt to the smallest thing that states the task, and measure again. If the output improves, you never had a weights problem.

This is uncomfortable, because it means deleting work you were proud of. Do it anyway. You cannot diagnose a system you have only ever added to.

3. Can you actually produce the data?

This is the question that stops people, and it is where most of the effort goes.

Everyone has data. Almost nobody has a thousand or more curated examples of the exact behaviour they want, cleaned, and is prepared to keep cleaning them.

My last build started from 1,418 labelled examples and kept 884 after cleaning. The final training set reached 1,610 rows once other curated sources were added. Here is what got dropped from that base pool, and none of it was optional.

Rhetorical tics I did not want the model to learn: 107 examples dropped for one construction alone, plus smaller counts for two others. If a pattern sits in your training data at any volume, you are teaching it.

Length and register discipline: 302 dropped for being the wrong length for their category. A model trained on mixed lengths produces mixed lengths.

Opener variety: 45 dropped because more than two examples shared the same opening three words. Without that cap the model finds one opening it likes and reuses it everywhere.

Then there was the part I did not plan for.

83 examples were being deleted on every build

The build had a decontamination step. It removed hand-approved examples from one source file, on the assumption they had already been copied into a curated file elsewhere.

Nothing had ever copied them.

83 examples were dropped silently at build time, on every run, for months. They were disproportionately the best ones, the examples carrying concrete specifics rather than general statements, because those were exactly the ones I had approved by hand in the first place.

A row that never arrives leaves no trace. Nothing in a training run tells you what it did not receive.

So there was no error. The row counts looked plausible. The model trained, the evaluation ran, and the result was slightly worse than it should have been in a way no single test could isolate.

I found it by reading the builder line by line.

That is not really a lesson about checking your pipeline. A training pipeline fails quietly by design. Code that drops rows produces a smaller number, and a smaller number still looks like a number. So print what you dropped and why, broken down by reason, on every single build, and then read it.

The bar

Answer all three and fine-tuning is a reasonable thing to do, and it works. Voice in particular is something you can instruct a model toward but cannot make native to it, and that gap is real.

You will also need an evaluation set that can fail, held out and genuinely separate from anything you trained on, and a rollback path that restores the previous model together with its system prompt. Those are their own articles, and they are the difference between shipping a fine-tune and gambling on one.

But answer the three questions first. Two of them cost nothing except honesty.

Every Rule I Added Made It Worse: How Prompt Bloat Killed My Voice

Chidozie Uzoegwu — Fri, 14 Aug 2026 14:11:21 +0000

I measured the system prompt from my old setup last week. It came to 224,833 characters, roughly 56,000 tokens, on every single call.

Ninety percent of that was one block of accumulated rules about how the output should sound.

I did not set out to build that. Nobody does. It grew one reasonable decision at a time, and by the end it was actively making the output worse, which I spent months blaming on the model.

(That prompt-engineering path is no longer what I run in production. It survives in my codebase as a rollback target, which is exactly why I could still assemble it and measure it.)

How a prompt gets to 56,000 tokens

The system generates short-form written content in a specific voice. I started the way most people do: a good frontier model, a carefully written prompt, and iteration.

The loop that got me here is one every prompt engineer will recognise:

Output comes back with a cliché I hate.
Add a line to the prompt banning that cliché.
Output improves. Genuinely, for a while.
New failure appears. Add another rule.
Repeat for six months.

Each individual addition was justified. Each one was a real observed failure with an obvious textual fix. The categories that accumulated:

Banned phrases, specific clichés that kept reappearing
Banned structures, sentence shapes that read as machine-written
Opener bans, a list of ways it was not allowed to start
Register rules, tone and length and formality per content type
Anti-repetition context, the openers and closers used in recent output, injected so it wouldn't repeat itself
Worked examples, retrieved samples showing the target voice
Meta-rules, instructions about which of the above took priority when they conflicted

That last category is the tell. When your prompt needs rules about how to resolve conflicts between its own rules, the prompt has become a program, and nobody is testing it.

Then came the validators

Rules in a prompt are requests, not guarantees. So when a rule failed to hold, I did the obvious thing and enforced it in code afterwards.

That grew into a stack of post-generation checks: regex to catch a banned sentence shape, a stripper for a punctuation habit, a rejection pass for output that restated its input, a vocabulary blocklist, a fallback for when everything got rejected. At one point I shipped seven of these as a single bundle.

Each was a patch for something the prompt couldn't reliably enforce. And here's the part worth sitting with: I was compounding validators to compensate for a weakness in how the model was being steered. Every new validator was evidence the prompt wasn't working, and my response was another validator.

The symptom I misread

Quality got worse as the rules accumulated.

Not dramatically. Gradually. Output got flatter and more cautious, and it started sounding like something written by a committee avoiding mistakes rather than something with a point of view. Which, functionally, is exactly what it was.

My diagnosis at the time was that the model wasn't good enough. I planned a move to a more expensive tier and put a cost estimate together.

That would not have fixed anything. A stronger model given 56,000 tokens of conflicting constraints produces more expensive committee output.

The actual diagnosis

The rules were competing, with each other and with the task.

A language model attends across the whole context. A prompt is not a config file where each line executes independently. It is context the model weighs all at once. Two hundred lines of prohibitions against a two-line description of the actual job means the overwhelming signal is avoid things. The model optimises for whatever you have given it the most evidence you care about, and I had spent six months providing evidence that what I cared about was not breaking rules.

That produces output which breaks no rules and says nothing.

Worse, it doesn't converge. Adding a rule to fix flatness makes the ratio worse. The tool I was using to fix the problem was the thing causing it.

Why prompting couldn't solve this one

Here's the part I had underweighted for months.

You can instruct a model toward a style. You cannot make a style native to it.

Prompting is instruction. It operates on top of what the model already is. Ask for a voice and you get an impression of that voice, held in place by the instruction, degrading the moment the instruction competes with anything else. The rules were load-bearing scaffolding. Remove them and the voice collapsed. Keep them and they crowded out the task.

That is the signature of using the wrong technique. If a behaviour has to be re-specified in full on every single call, it does not belong in the prompt. It belongs in the weights.

What replaced it

I fine-tuned a model on the voice instead. Later articles in this series cover the how, the cost and the evaluation discipline, but the number relevant here is what happened to the prompt.

Approach	Assembled system prompt
Prompt-engineering	224,833 chars, roughly 56,000 tokens
Fine-tuned model	6,805 chars, roughly 1,700 tokens
Difference	About 33x smaller

You can measure your own the same way. Assemble the prompt exactly as your production path does and count it, rather than eyeballing the template file:

prompt = build_system_prompt(sample_input)   # your real assembly path
print(len(prompt), "chars |", len(prompt) // 4, "approx tokens")

Break it down by section too. Mine was 90% one block, which is what told me the problem was concentrated rather than spread evenly:

import re
sections = [s.strip() for s in re.split(r"={40,}", prompt) if s.strip()]
for n, head in sorted(((len(s), s.split("\n")[0][:50]) for s in sections), reverse=True)[:10]:
    print(f"{n:7d}  {head}")

The voice rules didn't move into a shorter prompt. They moved into the model. What is left in the prompt is what a prompt is actually good at: what to write about right now, how long, which mode. Instruction, not identity.

The lesson I'd give my past self

Strip before you add.

When output is wrong, the reflex is to append a rule, because appending is easy and feels like progress. But every rule is a permanent tax, paid on every call, forever, competing with every other rule, and never reviewed. I never once removed a rule. I only ever added.

Three questions I now ask before adding anything to a prompt:

Is this instruction, or identity? Instruction belongs in the prompt. Identity, meaning a voice or a consistent style or a way of reasoning that must hold every time, belongs in the weights.
Is this knowledge? Facts the model needs but cannot be expected to hold belong in retrieval, not permanently in the prompt.
What am I removing to make room? If the answer is nothing, you are not editing. You are accreting.

The rough division that came out of it, and that I would now design around from day one:

Fine-tuning owns identity. Prompting owns instruction. RAG owns knowledge.

Put a job in the wrong one and you will spend months adding rules that make it worse, which is a fairly expensive way to learn where the boundaries are.

0.3% In, 36% Out: Your Fine-Tuned Model Is Copying Your Prompt Examples

Chidozie Uzoegwu — Thu, 13 Aug 2026 17:08:00 +0000

I run a fine-tuned Llama 3.3 70B on Amazon Bedrock. It generates short first-person narrative posts: a setup, a few lines of story, and a closing line that lands the point.

Last week I noticed the closing lines had collapsed into a single template. Not similar in spirit, but literally the same grammatical shape, over and over:

and that's how [someone] [learns/teaches] [something].

Roughly a third of everything the model produced ended that way. The obvious diagnosis was overfitting: the training data must be saturated with that pattern, so the model learned it as the way to end a story. The obvious fix was equally clear. Scrub the training set, retrain, redeploy.

In my setup that runs to about $30 of training compute and a five-hour training job, plus the evaluation pass afterwards before I'd know whether it had worked. Not ruinous, but not free either, and this would have been the third retrain cycle.

Before spending it, I did something I should have done first. I counted.

Step one: count the pattern in your training data

My training file is JSONL, one example per line. The check took about ten seconds:

grep -o -i "that's how" train_v6_2.jsonl | wc -l
# 5

wc -l train_v6_2.jsonl
# 1610

Five occurrences in 1,610 training examples. 0.3%.

Then I counted the same pattern in what the model was actually producing. Every generated post gets persisted, so this was one SQL query over the last 25 rows:

select right(content, 200) as ending
from generated_stories
order by created_at desc
limit 25;

Nine of twenty-five. 36%.

Those two numbers are irreconcilable. Training data at 0.3% cannot produce output at 36%. Fine-tuning shifts a model's distribution, but it does not amplify a pattern by a factor of a hundred. Whatever was driving this, it was not the weights.

Which meant a retrain would have changed nothing, and I'd have concluded my fine-tune was broken.

Step two: read your own prompt

The generation prompt included a line I had written months earlier and never revisited. Structurally it looked like this:

THE CLOSER: lead with a thesis line that names what the story proves, e.g. "and that's how [X] teaches [Y] to expect [Z]."

There it is. I had handed the model one complete, well-formed example of exactly the thing I wanted, and it did the most reasonable thing available to it. It copied the example, template and all.

This is not the model misbehaving. Give a language model a single concrete instance of a pattern and ask it to produce that pattern, and the example becomes the strongest signal in the context window. It will out-compete a 0.3% tendency in the weights every time. I had effectively hardcoded my output and then blamed the training data.

Step three: fix the injection layer, not the model

The fix cost nothing and shipped in an afternoon.

Instead of one hardcoded example, I built a small pool of seven closing lines. The key property is that they are structurally dissimilar: a flat verdict, a two-beat reveal, a consequence line, a prescriptive line, an ironic callback. Critically, none of them uses the shape that was being copied.

My production lines come from a private archive, so here is an illustrative pool in a neutral domain that shows the same variety:

_CLOSER_EXEMPLARS = [
    "The process was the problem.",
    "Nobody had tested it. Everybody had approved it.",
    "We saved an hour that afternoon and spent the next year paying for it.",
    "Write the runbook before you need it. You will not be calm enough later.",
    "So much for the quick fix.",
    "We called it a deadline for four months before anyone admitted it was a guess.",
    "What would you have checked first?",
]

examples = "\n".join(
    f"- {c}" for c in random.sample(_CLOSER_EXEMPLARS, 3)
)

Three are sampled at random per call, plus one explicit instruction: study the function these serve, then write a new one for this story, and do not reuse their wording.

The reasoning is that one example teaches a template, while several dissimilar examples teach a function. If every example looks different but they all achieve the same effect, the only thing left to imitate is the effect.

The result

Measured on live inference against the same deployed model, same theme, no retraining:

Run	Template rate
Before the fix	36% (9 of 25)
After the fix	8% (1 of 12)

Sample sizes are small and I'd want more data before treating 8% as precise. But the direction is unambiguous, and it cost one prompt edit rather than a retrain cycle.

The remaining 8% is, I think, genuinely the weights. That 0.3% is still in there, and a prompt fix can't reach it. That residue is a legitimate item for the next training-data scrub. The difference is that it's now a small, well-understood cleanup rather than a retrain I'd have run on a wrong diagnosis.

The generalizable part

When a fine-tuned model produces something repetitive, there are three possible causes and they have very different price tags:

The prompt is handing it the pattern. Free to fix. Check this first.
The training data is saturated with the pattern. Needs a scrub and a retrain.
The base model has a strong prior. Needs data specifically counteracting it.

The diagnostic that separates them is one line of grep and one SQL query. Compare the rate in your training data to the rate in your output. If the output rate is dramatically higher, the cause sits in your prompt or your retrieval layer, not your weights. If the two rates are similar, it really is a data problem and a retrain is justified.

Two habits came out of this that I'd suggest to anyone running a fine-tune on Bedrock:

Persist your generations. I could only run the "after" comparison because every output was already in a table with a timestamp. Without that history there is no measurement, only impressions, and impressions are what sent me toward a retrain in the first place.
Treat prompt examples as training data. They are, functionally. Anything concrete you put in a prompt as an illustration will get copied, so either give none, or give enough variety that only the underlying function is imitable. One example is the worst of both worlds.

The uncomfortable version of this lesson: the model was fine. I nearly spent money and two days proving that the hard way, and the evidence that it was fine was sitting in a file I could have counted at any point.

My 9-Month Journey to 9 Cloud Certifications: Juggling Work, Deadlines, Projects, and a New Baby

Chidozie Uzoegwu — Fri, 10 Jan 2025 14:13:00 +0000

Certifications earned in chronological order:

AWS Certified Cloud Practitioner (CLF-C01)
Microsoft Azure Fundamentals (AZ-900)
Oracle Foundations Associate (OFA)
AWS Solutions Architect Associate (SAA)
AWS Developer Associate (DVA-C02)
AWS Data Engineer Associate (DEA-C01)
AWS Certified AI Practitioner (AIF-C01)
AWS SysOps Administrator Associate (SOA-C02)
AWS Machine Learning Engineer Associate (MLA-C01)

How It All Started
How did I even end up with 9 certifications when my initial target was just to focus on AWS Solution Architect Associate (SAA), get certified, build some projects, and rest? Oh boy, I was mesmerized and fell in love with cloud knowledge when I started! Solution Architect (SA) sounded interesting, and the concept of building solutions for products, brands, and companies using cloud infrastructure was exhilarating. To be frank, Solution Architect keeps coming up randomly as a highly sought-after skill set, and the job market needs people with such expertise. I'd just be watching random interviews where the presenter would ask, "What do you do for a living?" and the correspondent would say, "I'm a Solution Architect. I do this and that for companies…" The next day, I'd be skimming through LinkedIn or X or other social platforms, and there again, I'd see discussions and blog posts about Solution Architects. Then I'd say to myself, "This is the sign - SA is calling me!"

First Encounter with Cloud Computing
Let's retrace to the very beginning. My first encounter with the cloud was in one of my course modules during my Master of Data Science degree program called "Big Data and Cloud Computing." It amazed me that big data, programming, building solutions, storage, and many other activities could be done seamlessly on the cloud without needing to piece everything together - every service you need is already available on the platform like a one-size-fits-all solution!
The idea fascinated me. I wouldn't have to store data on a hard drive, manually upload or import it into my programming environment, worry about losing data entirely, or deal with different subscriptions and platforms to achieve a solution. The cloud could provide it all as Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) through a variety of services such as compute, storage, databases, machine learning, networking, analytics, and more. "Voilà," I said to myself, "Cloud is the way!"

The Immigration Journey and EB-2 NIW Motivation
Procrastination set in at first as I explored other paths - whether to pursue a Ph.D. immediately or get a job as a Data Scientist. As an immigrant in the UK with an expiring student visa, the visa challenges of finding skilled-worker sponsorship loomed large. I had just started a new family too, and the urge to move to countries where I could have more stability, along with other concerns I won't burden the readers with, occupied my mind rather than getting cloud certified - until early 2024.
In January, a Facebook friend recently shared his success story of obtaining USA EB-1 approval for a green card - a visa category for individuals with extraordinary abilities, outstanding professors, or multinational executives. This meant he would become a permanent resident in the USA while living in the UK on his previously approved Global Talent Visa. I reached out to him and discovered that I qualified for the EB-2 NIW (National Interest Waiver), a category for individuals with advanced degrees or exceptional abilities whose work benefits the U.S. However, I needed to demonstrate how "well-positioned" I was to advance my proposed endeavors. That sparked my motivation to pursue certifications, as they could serve as supporting documents to prove my exceptional abilities.

Starting with the Basics: Cloud Practitioner
Instead of jumping straight into the SAA certification, I needed to understand everything from scratch. Unlike Microsoft's Azure platform, which I had experienced in my Big Data and Cloud Computing module, I learned that AWS was the largest cloud provider with the biggest market share, so I focused on AWS. I started with the fundamentals through the AWS Certified Cloud Practitioner Certification (CLF-C02) from freeCodeCamp.org with instructor Andrew Brown on YouTube. The 14-hour video was comprehensive, but I couldn't follow it structurally as everything was mixed without sections, topics, and structure. So, I opted to subscribe to ExamPro, a platform developed by Andrew Brown, where I could find more structure in my learning and explore the practice exam questions - which are crucial for passing the main exam.
My next decision came upon entering the ExamPro platform. I discovered a course bundle with basic cloud foundation courses from top providers called "Cross-Cloud Fundamentals Package": AWS Certified Cloud Practitioner, Microsoft Azure Fundamentals, and Oracle Foundations Associate. I thought to myself, why not subscribe to this package? After all, it's not a bad idea to learn about diverse cloud concepts from the basics. I went for it at $69 USD and instantly became a serious student.
The Beast Mode Phase
Remember, during this period I was preparing my EB-2 NIW petition and gathering other documents for my application. I needed to speed up with my certifications to include as supporting documents, so my energy and "ginger" was on nitro! I was in beast mode and hyper-determined to succeed. I studied rigorously and mustered the wherewithal to meet my targets. I studied on my way to and from work, in the metro, during my free periods and breaks at work. At home, it was back-to-back studying if I was awake. Scratch that - I was waking up specifically to study, instead!

Foundation Certifications Success
In less than a month, I took and passed the AWS Certified Cloud Practitioner (CLF-C01) exam. Yay! My approach was simple: I covered all topics from Andrew Brown's course module and completed the hands-on practicals, but more importantly, I practiced the sample exam questions extensively, which prepared me for the real exam. Learning the course material alone wasn't enough - I needed to familiarize myself with the AWS exam structure, which played a major role in my success.
Immediately after passing the CLF-C01 exam, I gained momentum like a toddler hungry for its mother's milk and swung into action toward Microsoft Azure Fundamentals (AZ-900). I realized that I had already studied the CLF-C01 material extensively, which largely overlapped with the concepts in AZ-900, with only minor differences in terminology. As a result, I only needed to focus on practicing the AZ-900 exam questions. I took the AZ-900 and passed. Yay! The same happened with Oracle Foundations Associate (OFA). Yay! The OFA was a free exam, unlike the CLF-C01 and AZ-900 that cost about $100 USD each.

Tackling the Solution Architect Associate
Next came the certification that started it all - the SAA. I began researching study materials, and everywhere I turned, I encountered "Stephane Maarek" and success stories from those who had taken his Udemy course. This led me to sign up for Udemy and purchase my first course: "Ultimate AWS Certified Solutions Architect Associate" by Stephane Maarek. It came with some practice questions, but for deeper preparation, I also bought the SAA practice exam by Abhishek Singh and Stephane Maarek, which contained six full practice exams. This gave me plenty of opportunities to familiarize myself with the material and prepare for the exam. Both courses cost just over £28, as I bought them during a Udemy promotional sale. Always watch for those promos!
I prepared intensively for less than a month as my EB-2 NIW petition deadline approached. I passed! Yay! The exam usually costs $150 USD, but since I had passed the CLF-C01, I received a 50% discount on my next qualified exam. So, I paid just $75 USD for the SAA.

The Developer Journey and Life Changes
Upon passing the SAA, wild thoughts began to swirl in my mind: "Perhaps I can go for the AWS Developer certification and then the AWS Certified SysOps Administrator." Another voice chimed in, encouraging, "Why not complete all the associate-level AWS certifications and become more grounded in it?" Yet another joined, cheering me on, "Besides, you'll get half-price for the next exam since you passed the last one!"
I gave in because, truthfully, I had become magnetized by the endless possibilities of cloud knowledge and, like Oliver Twist, yearned for more. I recalled Stephane Maarek's advice from his FAQ about "In which order should you learn?": "Start with AWS Certified Solutions Architect Associate, then move on to AWS Certified Developer Associate and then AWS Certified SysOps Administrator. Afterwards, you can either do AWS Certified Solutions Architect Professional or AWS Certified DevOps Professional, or a specialty certification of your choosing."

The Developer Certification Challenge
So, I continued with AWS Developer (DVA-C02) certification. I purchased both the course and practice exams from Stephane Maarek at £28 each (promotional price - regular price over £60). DVA-C02 proved challenging, possibly due to its different approach and because I had a lot going on as I neared the final submission of my EB-2 NIW application. While preparing for the DVA-C02 certification, I was also preparing to submit my EB-2 application, and in three weeks, I took the DVA-C02 exam and failed. Nay! I was sad but not broken - I get motivated by failure; it sparks me as if I have something to prove. I can be stubborn like that!
The sadness was short-lived, though, as my EB-2 NIW was approved in May. Success! YAY! The next day, I welcomed my baby into this world. YAY! so I took 4 weeks off to give my baby all the attentions needed.

Building Real-World Experience: The Projects Phase
After taking a break, I returned to my studies as if I'd never left. I went back to the drawing board and realized I needed more hands-on experience to build up my profile and portfolio with projects. I began exploring both free and paid hands-on projects.
I started with free projects on YouTube:
From Tech with Lucy:

Launched a website on Amazon S3
Visualized data using Amazon QuickSight From Tiny Technical Tutorials:
Architected and built an end-to-end AWS web application from scratch
Built a full end-to-end web app with 7 AWS services
Created a free static website with AWS Amplify and Amazon S3
Built a React app with Amplify (Gen 1), Cognito, and CI/CD with GitHub Then came the paid projects: From Tech with Lucy - AWS Cloud Projects ($63 USD):
Built a daily task scheduler using Amazon PartyRock
Created an image labels generator using Amazon Rekognition
Developed a text narrator using Amazon Polly
Built a language translation bot using Amazon Lex
Deployed a bucket list tracker application on AWS Amplify From Azeez Salu through aosnote.com ($49.95 USD each):
Host a Dynamic Web App on AWS
Deploy a Dynamic Website on AWS with Terraform To ensure success on my next DVA-C02 attempt, I purchased another course by Jon Bonso. This provided additional learning opportunities, and combined with my hands-on project experience, I passed the exam in June after extra two weeks of study. Yay!

The Data Engineering Challenge
By now, my mission had expanded to completing all associate-level certifications. I wanted to be versatile and not limited to just one area of cloud computing. I aimed to become a full-fledged cloud engineer with comprehensive knowledge. When AWS launched the Data Engineer certification (DEA-C01), it was a perfect fit since data is my area of expertise. I made this my next target before tackling the most challenging one - AWS Certified SysOps Administrator Associate.
I purchased course materials from Frank Kane and Stephane Maarek, along with practice exams from Thomas Hass, both at the promotional price of £28. While studying, I attended the AWS Summit in London, where I met AWS partners and employees and gained first-hand experience with various AWS projects and services. This major event provided great insights into AWS. However, despite my preparation, I failed the DEA-C01 exam. NAY!
What went wrong? Data Engineer was complex, especially as a new certification with limited study materials available. I had approached it too quickly without thoroughly understanding the core concepts of data engineering, which focuses on moving and processing data. Many services appeared similar but differed in operation and approach. I needed to return to basics and take more time to understand the material properly.
For me, as I've mentioned before, failure doesn't weaken or depress me - it makes me more competitive and determined to win. I enjoy the challenge of overcoming failure. So, I purchased additional practice exams for AWS Certified Data Engineer from Abhishek Singh and Stephane Maarek, practiced for two more weeks, and passed! Yay!

The SysOps Administrator Battle
Next came the formidable AWS Certified SysOps Administrator Associate (SOA-C02). This proved to be the most challenging certification of all. Though I nearly gave up, I had to prove I could rise to the challenge. I started with Stephane Maarek's courses and practice exams but failed SOA-C02 twice! Nay! Nay!
I purchased additional materials from Neal Davis and Jon Bonso and continued studying. During this time, AWS launched two certifications in my field of expertise: AWS Certified AI Practitioner (AIF-C01) and AWS Certified Machine Learning Engineer Associate (MLA-C01). This excited me for two reasons: it aligned with my specialty, and AI and ML were advancing rapidly. I decided to use these as a refreshing break from my battle with SOA-C02.

AI and Machine Learning Certifications
AWS offered an "Early Adopter" badge to first-time earners of new certifications, so I tackled AIF-C01 first, purchasing course and practice exam materials from Stephane Maarek and Abhishek Singh. In just under three weeks, I passed - yay!
Next, I returned to finish my showdown with SOA-C02, and this time, it didn't stand a chance. I delivered the knockout punch, mastering monitoring and reporting, high availability and resilience, deployment and provisioning, storage and data management, networking, security and compliance, and cost and performance optimization. Passing SOA-C02 felt like a massive milestone.
Finally came MLA-C01. I purchased materials from Frank Kane and Stephane Maarek for both the course and practice exams, studied for less than a month, and passed - another yay! Although AIF-C01 and MLA-C01 were newly introduced, they turned out to be far less intimidating - my background in Data Science gave me a solid edge, and I happily breezed through both.

Cost Overview and Future Plans
It's worth mentioning that I utilized various AWS exam discounts, including the "Associate Challenge" promotion. These discounts helped significantly with retaking certifications. Generally, foundational certifications cost $100 USD at regular rates, while Associate Certifications cost $150 USD. However, passing one certification earns you a 50% discount on your next exam.
Speaking of next exams, I now have two bonus attempts available for professional certifications - either AWS Certified Solutions Architect Professional, AWS Certified DevOps Professional, or a specialty certification.

Looking Forward
Overall, I've thoroughly enjoyed the journey and experiences gained along the way. The ups and downs have shaped my current position as I continue learning and building. There's still so much to learn both in cloud computing and beyond. The world of Data Science is evolving at lightning speed, and I'm staying current with all the new technologies. What makes these experiences particularly exciting is the integration of AI, ML, and Data with Cloud technologies. I believe both are unstoppable, especially when combined. Currently, I'm working on a project using the latest AWS services - Sagemaker, Glue, Bedrock Agentic AI, Knowledge bases, Lambda Functions, Step Functions, CloudWatch, EC2, and others - as a solution for a brand. AWS continues to push boundaries, ensuring builders, users, and cloud engineers have all the resources they need as a platform-as-a-service. And that's truly the best part.
As I keep learning and building, I'll be sharing my journey along the way! Bye for now and Keep Learning!