How to LEARN with your own custom-made course

(https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/ju94efc6t245npprg8ku.png)
I haven't bought a course of any kind again. Don't get me wrong, I'm not against people who make courses or who build learning paths for others, I myself bought compulsively and learned a lot from courses bought on platforms like Udemy or Coursera, with very good results.

The thing is, I started to notice something. There were very good courses, with lots of stars or almost perfect ratings, that I bought hoping to absorb that knowledge and, once finished, be able to face real projects and work professionally in those fields, but strangely, when I finished the ones I managed to finish through sheer willpower, I ran into a problem that, if you're like me, you've probably noticed: I had the teacher's examples, but I had no idea how they had reached that conclusion.

I tried to come up with cases similar to the ones in the course, but if I changed something the teacher hadn't done, I got stuck immediately, because I had followed their steps but never knew how he or she had got there. For me, their thought process, or their scars from everything they had tried before getting there, was as hidden as the steps I needed to take to solve the problems I set for myself.

And that, folks, is one of the biggest frictions I lived through in my years of trying to learn only with guided courses.

And believe me, the problems I had facing reality were not few. I started looking for solutions to this, I tried buying more courses on how to learn, I used a thousand and one learning strategies, taking notes by hand, the Feynman method, etc. I'm not saying they don't work, but as soon as I went back to the blank page or to a problem I had never seen, it beat me again, I didn't even know where to start.

One day, in one of my many searches, I found what the problem was. I used methods, read docs, tried to change the teacher's solved examples little by little, but I had never learned to build a mental model.

The skill that changed how I learn

One night, browsing the internet, I came across Matt Pocock, an old acquaintance in the TS and education world. He had been teaching AI engineering for a while and talked about how he had poured his years of teaching into a skill called teach. In his explanation of how he used it I had an idea that I felt would help me with my learning problem.

Because it wasn't a course with a closed syllabus or a teacher guiding you to a conclusion reached off camera, it was a living learning system that, in my view, could lead me to build my own roadmap and develop that much-desired judgment.

teach: https://www.aihero.dev/skills-teach

How it works

In Matt's words:

It does not teach from what the model already knows. It treats parametric knowledge as untrusted.

Reach for it when the learning is the project.

This is where, for me, the key is. It's a living workspace where you can create your own learning, which isn't prebuilt and evolves one lesson at a time. This gives us a chance to solve the problem, because I was able to design my own workspace to learn by doing and develop the judgment I hadn't found in any course.

The skill builds the workspace with four pieces, and each one tackles a part of the problem.

• Mission: when you use the skill, this is the heart of your workspace. It explains why you want to learn this.
• Resources: which curated resources you'll use as the source of knowledge. The model teaches from there and not from what it thinks it knows, so if a lesson says something, you can go to the source and check it.
• Lessons: they don't come made in advance, they're created one by one based on what you need and what you've already learned.
• Learning records: the skill prepares an MD file as a memory of what you learn in each lesson. That way, if tomorrow you come back with another model or another harness to work on the same workspace, you don't need an initial prompt telling the model what you did before, because it has the source of truth for the decisions made right there.

But none of this is useful without an example of what I'm telling you, so here's a real approach I've used to learn a topic.

My case: a workspace to sharpen my AI engineering

Let's say we want to get better at AI, specifically AI engineering, solve real problems and develop the judgment that lets me do this professionally. That was my case, and my idea was to build a tool that solved a real problem while I polished my skills.

Mission. Some of the definitions I set for this workspace were these:

• «not as a prompt engineer, but as an engineer who understands why every architectural decision is made»
• «I can explain every decision in an interview without relying on the AI to think for me»
• «I learn best by building things I actually use, not toy examples»

I clearly defined what the goal was and what I didn't want. I didn't want to make «hello world» projects or not be able to explain every part of the code and every decision that had led me there. I wanted every problem I ran into to be solved not like a teacher who gives the solution, but like a guide who helps you build the mental model.

Resources. The skill will give you a curated list of resources that the model searches for on the internet, and it asks that they be relevant and from high-trust sources, but I added more things. I added books I knew as references, in this case AI Engineering, by Chip Huyen, as a theoretical source, among other books that in my research had, in my view, the syllabus and depth to be reference manuals. I also added blogs I had read that were very popular on the topic. All these resources serve as reference manuals in the lessons.

Alembic. In my workspace, while I was learning, I started building my own knowledge management system called Alembic (https://github.com/lamjesus/Alembic), a personal knowledge agent that saves web pages, PDFs, notes and videos, distills them and gives them back to me when I need them. It's a work in progress.

Before the lessons started, the open model provider I use shut down the platform where an assistant of mine was running, and with that I realized my project depended too much on a third party's infrastructure and wasn't resilient to their changes. That's why in the first lesson it went into my learning memory that from then on I would work with an abstraction layer over the models, so that any change in how they're served wouldn't break the whole architecture of Alembic.

This is one of the most important things and what answers the problem I raised at the beginning. I could only know this because I lived that problem myself while doing a lesson I created myself. Maybe in a course it would have been the teacher's recommendation to use pattern X or Y for this, but it would never have stuck with me like it has now if I hadn't had to iterate several times to find a solution.

The rules so the AI doesn't learn for me

The LLM can be wrong, and I don't think I'm discovering anything with that statement, so I created certain rules that helped me minimize these errors and control them, as well as know when I should take a risk.

The rules are few and I wrote them down in the workspace mission, the LLM can explain concepts to me, show me the shape of a pattern, give me exercises as skeletons with TODOs and review the code I wrote, but it can't give me the solution to an exercise or decide the architecture for me, and when I let an agent implement something, the specification, the tests and the decisions are mine. Before using any code I ask myself whether I can explain every line and every decision in an interview without help from an LLM, and if I can't, that code doesn't go in.

Try it yourself

I can't lie to you, it's not a perfect process, sometimes you have to iterate or change direction, but that's exactly what I was missing in courses. Before, I had the teacher's examples and didn't know how they got there, today I look at Alembic and I know why every piece is where it is, because I made the mistake and I made the decision.

If you want to try it, create a separate folder that isn't your work project and run /teach with the topic you want to learn, the first thing it will ask you is why you want to learn it, and that will be your mission.

If there's one thing all this left me with, it's that a course gives you the result of someone else's scars, and judgment only comes from your own. That's why I still use courses, but no longer to learn from them, now they're resources inside my own course, one I design myself and that makes mistakes with me.

In the next one I'll tell you how I designed each lesson so the AI doesn't learn for me.