DEV Community: Aeron

Blog #4: My First Ever Agent and Things That I Wish I Knew Before Building This

Aeron — Thu, 17 Sep 2026 04:39:02 +0000

Having an agent these days is not that hard with Claude, Hermes, ChatGPT, etc.

Most people these days, I believe, only see the tip of the iceberg. In most cases, they build and communicate with these agents through natural language.

I don’t say there’s anything wrong with it. Instead, it’s a very convenient and useful way to make the most of the technology we have available today.

But the tradeoff is that they never really know how exactly the machine runs behind the scenes.

Do they even need to know?

For general tasks, def no. But when it comes to a very specific use case for a specific person, I bet the answer is yes. Or at least, it's necessary to have someone who understands it deeply enough to handle that for them. Individually.

How the machine is running behind these agents

I never understood this until I built my first ever agent.

I called this one a Crypto Research Assistant. It does a super simple task: receive a question about crypto, then answer it. If the question is not covered in its provided sources, it’ll say that it doesn’t know.

Its scale is 0.0001% of every current agent you can randomly name out there, but even with this small agent, the work behind the scenes is way more complicated than I thought before.

Let me unpack them for you.

1/ First, the core agent loop

Do you know the fact that it’s never the case that an agent receives a question and gives you the answer in the very first response?

In practice, they call multiple tools, receive the results from those tools, and input those results back into the model. They’ll do this multiple times until they have enough information to answer your question.

That’s the agent loop.

2/ Second, the RAG pipeline

This is the core structure to handle source ingestion and find the answer to your question. There are 4 main steps:

Chunking -> Embedding -> Retrieval -> Generation

The RAG pipeline will then be wired as one of the tools in the core agent loop, so the agent can actually use it.

3/ Third, evals (also one of the most important things)

At this point, the agent can answer your question. The thing is you don’t even know whether its answer is right or not.

Maybe you do because you know all the information in the sources you give it, but users won’t.

So you need to build a system to check:

Leak: Is the agent’s answer actually from the sources, or is it guessing based on its memory?
Over-refusal: Does the agent refuse to answer even when it knows the answer?

4/ Fourth, the deployment

Ok, now you finally have a basic agent running. But if you want others to use it, you have to deploy it somewhere.

Keep in mind, you don’t want to publish the source documents when you deploy the app to the cloud. So here’s the stack I chose:

For code: my Github repo of course. This contains the source code only, no secrets or any private source documents
For document storage: an S3 bucket, read by a least-privilege IAM key
For UI: deploy to Streamlit Community Cloud from my public repo, password-gated

Yep, that’s the whole process.

That doesn’t even count the bugs that I encountered along the way. So if I could go back in time and build it again, here are the hardest lessons that I wish I knew earlier.

The Lessons

1/ Your evals won’t sustainably be 100% all the time

I used to spend a lot of time fixing the system prompt and tool description so that all my evals could sustainably pass. The benchmark I made was 3 times 100% in a row.

The reason why it’s so hard is that the LLM’s response is different every time, even when you don’t change the prompt.

So choose your target eval, improve only one thing at a time. When you achieve it, move on. Don’t ever fall into the rabbit hole like me.

2/ Watch out for your token usage

If it’s only a few LLM calls, that’s basically fine. But if you’re A/B testing your evals, watch out for it.

Shout out to the many devs in the comments who gave me advice on this.

Some of the solutions that you want to note about this:

Set a cap for token usage
Give the agent the latest response for context, so it won’t rerun the whole thing every time
Only test the eval you want to fix, not the whole set every time

3/ Don’t make a chunker that overfits to any specific article

Chunking is one of the hardest parts I got in this entire agent-building process tbh.

The hard part of chunking an article is:

It has to be not too long because the model has a token limit per call
Or too short because the chunks list would be too fragmented. The model then can’t find the exact chunk it needs to answer your question

I used to build a chunker that divided a specific article based on its content. The mean rank improved significantly, but when it came to multi-doc ingestion, that chunker just didn’t work anymore.

So I had to spend another morning just rebuilding the entire chunker and retesting the evals to make sure everything still worked.

A lot of work and time wasted, but lesson learned.

Final Thoughts

That’s everything I’ve learned so far from building an agent.

It sounds so complicated, but my thesis is that to get further in building these agents for specific use cases, you have to understand deeply how they work.

Or else, you’ll never get the output you want.

If you find this article interesting, pls drop a follow. Also, if you have any thoughts or suggestions, I’d love to hear them and chat more with ya.

Hope you enjoyed the read!

- Aeron

Blog #3: The Beginning of AI Learning

Aeron — Fri, 11 Sep 2026 08:52:35 +0000

Finally, the day has come. This week is the first week I started truly learning about AI basics.

For those who are hearing about me for the first time, the reason why I haven't started this sooner is because I basically started from zero. The last 3 months, I spent it entirely on learning how to code and the foundations of being an SWE.

The two biggest milestones this week: RAG pipeline and Evals.

1/ RAG Pipeline

A complete RAG pipeline basically has 4 parts: chunking, embedding, retrieval, and generation.

The hardest part for me is the chunking part.

Why tho?

There are a couple of reasons for this based on my honest experience:

* This is ultimately the input for the whole pipeline.

Think of this process like a cooking process. You simply can't have a good meal if you don't have the best ingredients.

* Not every file has the best format.

In reality, files are messy and usually you won't handle one file at once. In practice, it could be several dozen to hundreds, even more.

The goal is to split these into proper chunks that the model can easily read and search for the right information. It shouldn't be too short or too long. It has to be an appropriate size.

Two methods I tried this week with chunking: chunking by the file structure and by the word count.

From my experience, the file structure works much better than the word count since each chunk now contains specific content. The fact is the mean rank of this method is 15% lower than the word count one (1.7 compared to 2.0).

2/ Evals

This is one of the most interesting parts for me, as verifying an LLM's output is always one of my biggest concerns these days.

There are a couple of ways to build evals. This week, I tried to use the layer method, which separately measures the output of two layers: retrieval and generation.

That way, I could know two key things about these outputs:

If the output is wrong, I can identify which layer makes the wrong move and iterate on it.
If the output is right, I can know whether it's because the LLM is just guessing the answer from its memory or not.

One key lesson I learned with evals is that this is always a continuous improvement loop. There's no end, just how to get better each time.

Testing, improving, and repeat.

A Small Milestone

Other than these, I just want to spend a small part of the blog to mark this little achievement I achieved this week.

I gained like 10 followers on X in the last 7 days, and there are many supportive comments from the dev community.

I know it's a super small amount of followers, a super small milestone, but I'm so happy about it. It's proof that there are still people out there supporting me and eventually, that's the sign for me to keep moving forward.

Links

As usual, if you find this valuable to you, or you simply just think this is interesting and you want to read more articles like this, pls follow me on X.

For the code and tech stack I'm using, here's my GitHub.

Hope you enjoy the reading!

See you in the next blog very soon!

Blog #2: What I Learned After Building My First Real App

Aeron — Fri, 04 Sep 2026 01:07:57 +0000

Hey guys, it’s Aeron here.

This week officially marks 100 days in this journey, so I decided to write a blog about it.

This is not going to be an incredible learning lesson or a blog that will help you become a coder right away. Instead, this one is more about all the lessons I’ve learned throughout the journey so far.

If you’re also starting this journey from scratch like me, I hope this can save you some time.

The Context

Since my last blog, which I wrote after finishing the CS50 course, it’s been about two months now.

But tbh, my journey has been very up and down.

During the first month after CS50, around Day 30 to Day 60 or 70-ish, I was overwhelmed with my old job. That was my last month there, so there was a lot to do and a lot to transfer as well.

For anything I do in my life, I always try to do it with the highest level of responsibility and professionalism possible. So I was really packed back then, and that’s why there was a month-long gap in the journey where I barely coded anything.

This month, from around Day 70 until now, I’ve been fully focused on the journey. This is finally when I’ve been able to learn a lot.

If the first 30 days were mainly about learning how to code, this month has been more about learning how to actually build, test, and run a real app in production.

To do that, I first learned how to create and run an endpoint with FastAPI. Then I used Render to push it to public production. I also created a frontend with Streamlit.

The other half of the month was more about polishing the app and making it feel more professional, and more like a real app out there.

The three most important things were packaging the code with Docker, changing raw SQL to an ORM, and using Alembic to manage the database.

And of course, along with that learning curve came many lessons that I want to share in more detail below.

The Lessons

1. A small change could lead to an app crashing and several code refactors

I remember around two weeks ago, when I was in the middle of refactoring the raw SQL to ORM, I changed the class for the input information from class Transaction to class TransactionIn.

The initial purpose was simply to distinguish it from the other classes because after the refactor, I needed to add two more classes to identify the output: class TransactionOutMessage and class TransactionOut.

I barely thought about anything else back then, but it turned out to be a disaster.

The POST /transaction endpoint couldn’t run because it was still calling the old class.
Pytest started showing several failures because the tests could no longer call Transaction.

I needed to sit down and fix everything for a while.

That was the moment I really remembered.

After that, anytime I’m about to change something, I always think about what could be affected if I change this.

It sounds like a small thing, but for me, it was one of those lessons that only really makes sense after you break something yourself.

2. Pytest is green, but the new code actually hasn’t been tested yet

Before using SQLAlchemy like I do now, I was using psycopg2 for my main app.

That means for every request, I created a new connection, used it, and then closed it.

The code is much better now after I refactored everything to use session with SQLAlchemy. Instead of creating a new connection every time, the connection is already managed in the pool. I just borrow one, use it, and then return it back to the pool.

The thing is, after spending the whole day refactoring tons of psycopg2 code to use sessions in the main app, I checked pytest.

13/13 green.

I thought the code was good back then, until I suddenly realized:

Why haven’t I refactored the test app, but it’s still green?

It turned out that in the test code, I had dependency_overrides.

That meant my tests were never actually testing the code I had just refactored. They were still running against the old psycopg2 implementation.

That was another key lesson for me because I believe testing is always one of the most important parts, along with coding.

Getting the code to run is only part one.

Making it pass all the tests is the next step before putting it into production.

Thoughts

So yeah, those two above are the two biggest coding lessons I wanted to share from the last two months of this journey.

If there’s anything else I want to share, it would be this:

Always stick with your original plan, keep your promise, and never lose faith in yourself.

Tbh, there have been more than 10 times when I thought about quitting and going back to the blockchain industry.

But then I think, the journey has only just started.

Keep pushing, and the result will come sooner or later.

That’s it, guys.

See you soon in the next blog!

Blog #1: The First Milestone

Aeron — Wed, 01 Jul 2026 09:44:02 +0000

Last Month Was an Incredible Month

Last month was an incredible month for me.

I started this journey without a single coding skill, and here I am today. I built my own project and earned the CS50 certificate, which is the first coding certificate I've ever received.

If you had asked me back on May 22, when I wrote my very first tweet about this journey, I would never have imagined I could come this far, this fast.

Let Me Tell You About My Final Project

I actually came up with this project because it solves a problem I've had for a long time. I always struggled to manage all my crypto orders and positions because everything was so fragmented. My trades were scattered across different platforms, and it was difficult to know exactly how my portfolio was performing.

That's why I built the Crypto P&L Tracker. The goal is simple: help people manage their entire crypto portfolio in one place with real-time price updates.

How Does It Work?

Pretty simple.

Step 1

Every time you buy or sell a token, log the transaction into the tracker.

Step 2

That's it.

The tracker automatically calculates the average entry price, the performance of each position, and your overall portfolio ROI. Whenever you want to see how your portfolio is doing, just ask the tracker to display it.

Thanks to the CoinGecko API, all token prices are updated in real time.

I chose CoinGecko because even the free API offers 100 requests per minute, which is more than enough for this use case.

Project Structure

I divided the project into five key functions:

main()
- Handles the core logic
- Calculates average prices
- Computes the overall portfolio ROI
checker()
- Validates and filters user actions
buyAmount()
- Processes buy orders, including edge cases
sellAmount()
- Similar to buyAmount(), but for sell orders
getPrice()
- Fetches real-time token prices from the CoinGecko API

The Biggest Challenge

If there was one difficult part of building this project, it was definitely time management.

I still have a full-time job, so finding time to code wasn't easy. Fortunately, I had a day off during the final week of CS50, and I spent the entire day coding. That was how I managed to finish the project on time.

More importantly, this project made me realize how much I genuinely enjoy coding and solving problems. I haven't felt this level of passion and curiosity about learning in a long time.

Looking Ahead

I know this is still a very small milestone. Compared to the thousands of developers out there, I'm probably at the level of a second-year CS student.

But I always remind myself that everyone has to start somewhere.

I can become whoever I want to be, and I'll keep moving forward no matter what.

Now, it's time for month two.

I'm ready.