DEV Community

Cover image for Learning to Build with LLMs the Framework-Free Way
Rijul Rajesh
Rijul Rajesh

Posted on

Learning to Build with LLMs the Framework-Free Way

#ai

Hello, I'm Rijul, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product.


Usually, when learning to build with LLMs, we start with a framework, follow a quickstart guide, and build a demo.

The problem is that once everything is set up, you may end up not fully understanding what is actually happening under the hood.

This can become a problem down the line when you eventually need to tune or modify things.

Because of this, I found a repository that is quite handy for learning:

AI Engineer Notebooks

You can find it here:

https://github.com/calmrocks/ai-engineer-notebooks

These are a set of clean, self-contained notebooks that promote more framework-free learning and let you focus on the actual concepts.

Checking the First Notebook

Frameworks may change from time to time, but the underlying concepts stay mostly the same.

So, if you want to try it out, you can go through each Colab notebook one by one, as mentioned in the README.

Here, I'll be going through one of the notebooks mentioned in the beginning.

It covers the fundamentals.

You can find the notebook here:

https://colab.research.google.com/github/calmrocks/ai-engineer-notebooks/blob/main/01-model-apis/00-prompting-basics.ipynb

You can first go through the setup.

You can create a free API key at Groq and add it using the key icon on the right.

The notebook then shows some of the basics, such as prompts and the difference they can make.

It also shows how specifying the output format can affect the response.

Going a Little Further

Now, let's look at another notebook that goes a bit further:

https://colab.research.google.com/github/calmrocks/ai-engineer-notebooks/blob/main/02-evals-basics/01-measuring-outputs.ipynb#scrollTo=K6anGbvMJwxm

This one is more about measurements.

Imagine you built a little tool.

Someone emails support, and your tool reads that email and outputs three things:

  • What the problem is
  • How urgent it is
  • What category it falls into, such as billing or a bug

That's it. That's the whole system we're testing.

Now, how do you know if it's actually good?

Most people just try a few emails, look at the output, think "Yeah, that looks right," and move on.

That's what we tend to do by default.

It feels fine until it isn't.

The notebook teaches you to do something slightly annoying but much more honest:

Write down the right answers first, before you even start improving anything.

Step 1: Golden Set

First, they create 6 fake emails and, next to each one, write down what the correct priority and category should be.

Think of it as an answer key.

For example:

"Someone got double-charged" → "urgent" and "billing"

"Someone is asking how to change their email" → "low" priority and "account" category

These are simple examples, but now the correct answers are written down instead of just living in your head.

Step 2: Run the Tests

Now you feed those same 6 emails into your AI tool and check each answer against what you wrote down.

Did it match?

Did it not match?

You add everything up and get a real number, rather than relying on a feeling.

In this case, the result is 83%.

That's your starting point.

Step 3: Change One Thing

Now, you change the prompt a little.

Step 4: Run the Same Test Again

Then you run the exact same test again.

Initially, the score was 83%.

Now, it has improved to 100%.

That's how you can systematically improve things.

Wrapping Up

These are some simple and practical ways of learning what goes into building with LLMs.

Following this kind of framework-free learning can give you a base-level understanding of the concepts, which can make it easier to handle more complex systems later.



Your team's attention is limited, and the deluge of AI-generated code is making it harder to keep production reliable and secure without slowing you down.

I'm building LiveReview, a blast-radius aware AI code review built for your business-critical systems.

Instead of presenting every diff with equal emphasis, LiveReview scores each change by blast radius — how far its impact reaches through your call graph — so you can focus attention where it actually matters.

Spend code review effort where business risk is highest — not spread evenly across every diff.

⭐ Star it on GitHub:

GitHub logo HexmosTech / LiveReview

Blast-Radius Aware AI Code Review for Business-Critical Systems

LiveReview

gitleaks.yml osv-scanner.yml govulncheck.yml semgrep.yml dependabot-enabled mcp-testcases.yml

LiveReview: Blast-Radius Aware AI Code Review for Business-Critical Systems

LiveReview is an AI code reviewer that scores every hunk of a diff by blast radius: how far a change reaches through your call graph, how much persistent state it touches, and how well-tested it is. A 3-line change to a shared auth check can outrank a 300-line UI tweak. Your team's attention goes to the highest-risk code first, not spread evenly across every diff.

blast-radius-demo.mp4

LiveReview's Blast Radius & Review Priority scoring, live in the diff viewer.

The exact math, not a black box Visualize blast radius at a glance Every factor that feeds the score
How does Blast Radius scoring work? (a more technical explanation)

Here's the goal:

  • A 3-line fix in a function used by 40 other files, that also writes to a database, should score high.
  • A 300-line UI change in one file, fully covered by…

Click below to try LiveReview with your codebase:

LiveReview Banner

Top comments (0)