I started with linear regression and did it properly this time, from scratch, with no sklearn doing the work.
Here's how the day went:
- Intuition first. What it means to fit a line through data, and what "best fit" is actually measuring.
- Cost function. Wrote out MSE and worked through why the errors get squared.
- Gradient descent by hand. Worked out the updates on paper before touching any code. It's a small loop: predict, check the error, adjust the weights, repeat.
- Implemented the whole thing in NumPy from scratch.
- Went through the assumptions the model relies on, mainly linearity and how the residuals behave.
- Ran the same data (California housing) through scikit-learn and compared it against my version.
I've used linear models in projects before. The ticket triage pipeline I built has a Linear SVM inside it. But I had never gone through the whole thing step by step myself, and I wanted that done before moving on to anything bigger.
I also spent some time on core Python, to make sure the basics under everything I write are solid.
Top comments (1)
Working through gradient descent by hand on paper before writing any code is exactly the right sequence. When you implement something before you've worked it through manually, you end up with code that runs but you don't fully own the intuition for why. Having the update loop in your head first — predict, check error, adjust weights, repeat — means the NumPy implementation is a translation of something you already understand, not a transcription of something you're hoping works.
Comparing your from-scratch version against sklearn on the same dataset is a good verification approach. If the coefficients and predictions are close, you've built the right thing. If they diverge, you have a concrete debugging target.
The point about having a Linear SVM in a real ticket triage pipeline but never having gone through the full math yourself is an honest framing. A lot of practitioners ship models without that foundation, which usually works until it doesn't and then the debugging gets hard. Good call to go back and build it properly before moving to more complex models. What's the plan for day 2?