Overengineering: A Series

I want to make a CI/CD pipeline. Going in to this project I understand that whatever solution I could make would not be nearly as usable, reliable, or feature rich as existing options. Despite this, I want to give it a go.

There are a few reasons for this. Firstly, from my (possibly incorrect) viewpoint as a junior dev, I think that what is most important for me is to understand the fundamentals and the tools that I use. I know how to go through the process of commiting, pushing, making a PR and merging, but given a blank terminal I would struggle to remake that process. Add to this the exposure to networking, scripting, and automated tests, and I think the picture becomes very clear.

Secondly, I think that self hosting where possible is a good thing. Sure, my GitHub profile probably won't be deleted and take with it all of my previous projects, but you never know. If I can host my repos on some device that I own then I won't need to rely on any third party, and I won't need to send packets halfway around the world to access my own projects.

Thirdly, I think it would make an interesting series of blog posts. My plan is to take on this project in weekly sprints, and to review progress at the end of each of these sprints. With this in mind, we could call this first post my sprint planning.

The first step of any project: figure out what I actually want to achieve. When I say that I want a CI/CD pipeline, what do I mean? Well,

• I need a place for my repos to live. GitHub and Gitlab exist, but as stated, I would prefer to keep this on my own network. Functionality here can be minimal, especially at the start. A directory on a machine somewhere on my LAN will do.
• I want some series of events to happen when I push to main.
• A build should be created. This will most likely be in a docker container, or at least it will be to start with.
• After a build is created, it should be tested. This should be everything from unit tests to end to end automated testing. Maybe even some SecOps stuff, just to spice it up.
• If anything fails (in the build or test stages), the whole pipeline should rollback. This probably means I need to keep builds somewhere in storage.
• If everything is ok, we are sorted. Deploy the build to some container.

Even in writing this list I have found a few areas that will require research. I guess this is kind of the point. I will spend the next week filling in any gaps in knowledge that would block initial progress, and also trying to figure out how much time I can actually afford to put into this project.

Thus far, no blockers.