Automations?
What did "automate this" really mean? It meant I was starting to get really annoyed with this day in and day out ritual experiment. It was yet "another" thing I had to do, and it was taking up a lot of my cognitive capability, which was already taking a pounding from everything else.
The ritual I described in Part 1 was doing three things at once:
- Preserving data past GitHub's 14-day cliff: The most obvious one. GitHub gives you two weeks and shrugs. If you didn't grab yesterday's numbers, they're gone. I was the human cron job. We've all been there, and the gods willing, we'll finally be able to leave those terrible days behind us.
- Filtering noise: PyPI download numbers are famously useless out of the box. At least, that's what everyone told me when I first started to look, because I sure as shit couldn't make any sense out of them. I feel like I have a doctorate on it now. Bots, mirrors (i.e. Bandersnatch), and CI pipelines. It's just a part of my recurring dreams, along with "everything has to be async and properly validated".
- Taking raw data, some context, and passing it to ChatGPT, Claude, and Gemini was an amazing, and eye-opening experience, and was pretty instrumental in me making a lot of sense of the above filtered data.
It also helped me start to visualize patterns, understand download behaviors, and more.
And it worked. Until it didn't. One day the chat just forgets. Contradicts something it told you last week. Confuses your March numbers with January. You realize the thing you've been trusting has quietly lost the plot, and you don't even know when it happened. So you start a new chat, port the context over, and the cycle repeats.
That's when I realized I didn't need a smarter chat. I needed a system that couldn't forget. One that runs against the real data every time, tells me "so what" without me asking, and doesn't drift because a context window got full.
The three AI chats taught me what questions to ask. Pulse is the thing that answers them without being asked.
The Reports
The insights I was getting from the various LLMs were exactly what I wanted. That was never the problem. The problem was that when I started building Pulse, I had to figure out how to present all of that to a user who doesn't have three AI chats preloaded with six months of context and a personal vendetta against bot downloads.
So I built reports that generate automatically, weekly, monthly, quarterly, from the same data the dashboard shows. No manual writing. Every number, every trend arrow, every narrative caption is computed.
It already knows that I care about Unique Cloners, so that is the thing it leads with, and then naturally brings up other hightlights and drivers.
And the part I'm most proud of: the week-by-week and month-by-month strips at the top.
Each block is a period. Teal means up. Red means down. Mixed means some metrics rose while others fell. You can see your entire project history in one glance, and I'll be honest with you: the first time I looked at that strip with a few red blocks in a row, my immediate reaction was "I need to get three teals in a row." I didn't design it to be motivating. I designed it to be informative. Turns out those are the same thing when you care about what you're building.
Defenders of Legacy
One thing (amongst many) I've learned during this AI transition is that everyone has their own pace at which they accept things (or not at all). So even though it's cool to use generative AI to generate high level summaries, there's still something to be said for getting to look at a good old chart and great UI/UX.
Pulse has both. The reports are there when you want the story. The dashboard is there when you just want to look at your numbers and draw your own conclusions. Neither one replaces the other, and I'm not going to tell you which one you should care about. That's your call.
They say it's best to build something you actually use. Well, I use the hell out of this. My multi-hour ritual is as short as 5 minutes, or as long as... I mean, who're we kidding, I can still pour over the data for hours. It's just not annoying now.
Till tomorrow...
Pulse is live if you want to poke around. Search any PyPI package and see what the data looks like: pulse.aegis-stack.io/search
Built on https://github.com/lbedner/aegis-stack, an open-source FastAPI scaffolding platform I maintain. One command, production-ready, everything pre-wired. The same infrastructure running Pulse is what you get out of the box.
Top comments (0)