How many decisions does the AI make without me knowing?

#claude #opensource #showdev #github

I ran an experiment after reading a couple of chapters from Alan Knox's book, AI Builders.

The Experiment: To keep track of the decisions the AI and I make in a log for a simple application.

To set up my project, I used the 'prompting for options' approach that Alan talks about to establish a strong foundation for my app.

I made around 20 decisions regarding scope, my data model, overall design, etc.. This gave me a great starting point to start building.

Once it was time to start implementing, I let off the reigns and let the AI use my 20 decisions to build the four screens out.

The Result: The AI made about 54 decisions without me.

I got a result that looked good overall. The app checked off the boxes in my original plan, but there were things implemented that I didn't agree to.

19 of those were substantive (search function, day-grouping, dark mode, undo delete, tag management, etc.)
~35 were smaller implementations (FAB style, field order, placeholder text, notification copy, data-layer storage details, etc.)

Alan brings up an excellent point that in building with AI, there is a significant number of decisions being made by the AI that we have no idea about. And these silent decisions slowly start to build technical debt.

Imagine if I hadn't made the first 20 decisions for the AI to go off of. Who knows what result I would've gotten.

The more informed decisions you make about your app, the better.

After the Initial Experiment

After seeing how many decisions get made without me knowing, I now take the 'decisions then log it' approach. I prompt for options, then I weigh the tradeoffs, then I decide, then I log that decision.

While building Today, I Learned, I also built Decision Explorer AI. It's a Claude skill that keeps track of all your decisions, their tradeoffs, and more.

I found this tool very valuable when I was adding new features because the tool references my existing decisions and reasonings to help me make the best choice for my app.

The image below shows a list of the decisions I accepted/made before building, while building the app, and after submitting the app for review to the App Store.