Finally, the day has come. This week is the first week I started truly learning about AI basics.
For those who are hearing about me for the first time, the reason why I haven't started this sooner is because I basically started from zero. The last 3 months, I spent it entirely on learning how to code and the foundations of being an SWE.
The two biggest milestones this week: RAG pipeline and Evals.
1/ RAG Pipeline
A complete RAG pipeline basically has 4 parts: chunking, embedding, retrieval, and generation.
The hardest part for me is the chunking part.
Why tho?
There are a couple of reasons for this based on my honest experience:
* This is ultimately the input for the whole pipeline.
Think of this process like a cooking process. You simply can't have a good meal if you don't have the best ingredients.
* Not every file has the best format.
In reality, files are messy and usually you won't handle one file at once. In practice, it could be several dozen to hundreds, even more.
The goal is to split these into proper chunks that the model can easily read and search for the right information. It shouldn't be too short or too long. It has to be an appropriate size.
Two methods I tried this week with chunking: chunking by the file structure and by the word count.
From my experience, the file structure works much better than the word count since each chunk now contains specific content. The fact is the mean rank of this method is 15% lower than the word count one (1.7 compared to 2.0).
2/ Evals
This is one of the most interesting parts for me, as verifying an LLM's output is always one of my biggest concerns these days.
There are a couple of ways to build evals. This week, I tried to use the layer method, which separately measures the output of two layers: retrieval and generation.
That way, I could know two key things about these outputs:
- If the output is wrong, I can identify which layer makes the wrong move and iterate on it.
- If the output is right, I can know whether it's because the LLM is just guessing the answer from its memory or not.
One key lesson I learned with evals is that this is always a continuous improvement loop. There's no end, just how to get better each time.
Testing, improving, and repeat.
A Small Milestone
Other than these, I just want to spend a small part of the blog to mark this little achievement I achieved this week.
I gained like 10 followers on X in the last 7 days, and there are many supportive comments from the dev community.
I know it's a super small amount of followers, a super small milestone, but I'm so happy about it. It's proof that there are still people out there supporting me and eventually, that's the sign for me to keep moving forward.
Links
As usual, if you find this valuable to you, or you simply just think this is interesting and you want to read more articles like this, pls follow me on X.
For the code and tech stack I'm using, here's my GitHub.
Hope you enjoy the reading!
See you in the next blog very soon!
Top comments (0)