Hello from Japan 🇯🇵
AI has made it possible to build web applications much faster than before.
Paste in an error, and AI can help investigate the cause.
Say, "I want to add a feature like this," and it can write the code.
It can help create a Dockerfile, pytest tests, authentication logic, and more.
It's incredibly useful.
But while continuing to build my own applications, one question started bothering me:
Can I actually explain the system I built?
Code working correctly and understanding why it works are not the same thing.
So this time, I decided to audit my own understanding of the Flask application I'm developing.
The rule was simple:
Answer 20 questions in my own words—without looking at the code, without searching, and without asking AI.
Only after answering would I compare my understanding with the actual implementation.
The results were surprisingly interesting.
This article was originally published in Japanese on Qiita and has been translated and adapted for DEV Community.
The Web Application I Audited
The application is a bakery sales-management system I've been building as a personal project.
Its basic workflow is:
Register products
↓
Enter daily sales quantities
↓
Save sales data
↓
Visualize rankings and charts
↓
Send sales data to the Gemini API
↓
Display suggestions that may help with business decisions
The main technologies include:
- Python / Flask
- PostgreSQL
- SQLAlchemy
- Flask-Login
- Flask-WTF
- Alembic / Flask-Migrate
- Docker
- Gunicorn
- Render
- Gemini API
- pytest
- GitHub Actions
The application works.
At the time of this audit, it had 91 pytest tests.
But this time, I wasn't trying to answer:
"Do I know the names of the technologies I'm using?"
I wanted to answer:
"Can I explain why each one is there and what it actually does?"
I Defined Three Levels of Understanding
I evaluated myself using three levels.
| Rating | Meaning |
|---|---|
| 🟢 | I can explain it in my own words |
| 🟡 | I understand the general direction, but some parts are vague or misunderstood |
| 🔴 | I cannot explain it, or my understanding is incorrect |
The most important rule was:
Do not look at the code first.
If I looked at the implementation before answering, I could always say:
"It works this way because the code here says so."
But that would not tell me whether I actually understood the system beforehand.
So I first answered entirely from memory.
Only afterward did I inspect the real code.
The 20 Questions
These were the questions I used.
| # | Question | Initial Rating |
|---|---|---|
| 1 | Whose problem does this system solve, and what problem is it? | 🟡 |
| 2 | Can I explain it to a non-technical store manager in 30 seconds? | 🟡 |
| 3 | How is it different from managing everything in Excel? | 🟡 |
| 4 | What remains usable if the AI becomes unavailable? | 🟢 |
| 5 | Where does entered sales data go? | 🟡 |
| 6 | How does data travel from the browser to the database? | 🔴 |
| 7 | What is GitHub used for? | 🟡 |
| 8 | Why does the app keep running when my own PC is turned off? | 🟡 |
| 9 | Why am I using Docker? | 🟢 |
| 10 | What is the difference between Docker and Render? | 🟡 |
| 11 | What are the different roles of GitHub, Render, and PostgreSQL? | 🟢 |
| 12 | What does SQLAlchemy do? | 🔴 |
| 13 | What does the login system protect? | 🟡 |
| 14 | Why does the login state remain when moving between pages? | 🔴 |
| 15 | What exactly am I sending to Gemini? | 🟡 |
| 16 | What is an API? | 🔴 |
| 17 | Why shouldn't an API key be stored on GitHub? | 🟢 |
| 18 | What are the roles of environment variables and .gitignore? |
🟡 |
| 19 | Why did I grow pytest to 91 tests? | 🟢 |
| 20 | If all pytest tests are GREEN, does that mean there are no bugs? | 🟢 |
My initial result was:
🟢 6
🟡 10
🔴 4
The purpose was not to compete for a score.
I used it as a diagnostic tool to find:
where my own understanding stopped.
I Could Explain the User-Facing Purpose Relatively Well
One interesting result was that I could explain fairly well:
why I had built the system in the first place.
For example, I understood the role of AI as:
It is not a system where AI makes every decision. It provides reference suggestions based on sales data so that the store manager can think about what to try next.
I could also explain that:
Even if the AI feature becomes unavailable, the core functions—product registration, sales entry, rankings, and charts—still remain usable.
So I had a reasonable understanding of:
the purpose of the system from the user's perspective.
The problems appeared behind the scenes.
I Thought Sales Data Went "Through GitHub" to the Database
One question was:
How does sales data entered in the browser reach PostgreSQL?
My first explanation was something like:
It goes through GitHub and then reaches the database.
That was wrong.
The actual path is:
Browser
↓
Flask application
↓
Input validation
↓
SQLAlchemy
↓
PostgreSQL
GitHub is not part of this runtime data path.
I now separate the roles like this:
GitHub
= Storage for source code and change history
Render
= A place where the web application runs on the internet
PostgreSQL
= A warehouse that stores product and sales data
I knew all three technology names.
But I could not explain them as parts of one coherent system.
That was the gap.
I Also Thought Docker Sent My Finished Local Container Directly to Render
I had another misunderstanding about Docker.
My mental model was roughly:
I build a Docker container on my PC, then send that completed container directly to Render.
But when I went through the actual Dockerfile line by line, my understanding changed.
For example:
FROM python:3.12-slim AS builder
creates a builder stage based on Python 3.12 slim.
Then:
COPY requirements.txt .
RUN pip install --no-cache-dir --user -r requirements.txt
prepares the required Python packages.
After that:
FROM python:3.12-slim
creates another clean stage for production.
In other words, this is a multi-stage build.
I understood it using a Mini 4WD analogy:
builder
= Assembly pit
gcc / libpq-dev
= Tools such as cutters and files needed during assembly
requirements.txt
= Parts list
Production stage
= The actual machine that enters the race
There is no need to load every assembly tool onto the race car.
Only the required finished components are moved into the production stage:
COPY --from=builder /root/.local /root/.local
Then:
COPY . .
copies the application files from the Docker build context into the production image.
I now think of a Dockerfile as:
not the finished cargo itself, but a loading and assembly instruction sheet describing how the cargo should be prepared.
That mental model made much more sense to me.
I Knew the Name SQLAlchemy, but I Couldn't Explain It
SQLAlchemy was one of the biggest discoveries in this exercise.
I had seen the name many times.
It appears throughout my own application.
But when I asked myself:
What is SQLAlchemy?
I couldn't explain it.
Looking at the real code, I found:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
and then:
db.init_app(app)
to connect SQLAlchemy with the Flask application.
When saving sales data, the application uses:
db.session.add(sale)
db.session.commit()
My current mental model is:
PostgreSQL
= A warehouse where data is stored
SQLAlchemy
= An intermediary between Python and the database
Alchemy literally means "alchemy," so I decided to remember it as:
a SQL alchemist that helps Python work with database data in a convenient form.
I also started thinking of these operations like this:
db.session.add()
= Put something onto the shipping line as a candidate for storage
commit()
= Finalize this transaction
rollback()
= Cancel the uncommitted changes from this transaction
Strictly speaking, SQL can sometimes be sent to the database before commit().
But the transaction as a whole has not yet been committed.
Before this exercise, I could list SQLAlchemy under "technologies used."
I couldn't explain it this far.
I Was Confusing Cookies with Browser Cache
Authentication exposed another major misunderstanding.
At first, I explained login persistence as:
The browser stores the login information in its cache.
But browser cache and cookies are different things.
Then I inspected the actual implementation and found another mismatch:
the main branch at the time of this audit was not using Google OAuth.
It was using a single-administrator login system.
So even my belief that:
"My application uses Google login."
was out of sync with the implementation I was actually auditing.
I now think of the responsibilities like this:
Cookie
= Entry ticket or wristband held by the browser
Session
= State such as "this user is logged in"
Flask-Login
= Staff who check that state
@login_required
= A gate saying "only authenticated users may continue"
For example:
@login_required
def dashboard():
means that if an unauthenticated user directly visits /dashboard, Flask-Login checks their authentication state.
The implementation at the time of this audit also stored a SHA-256 fingerprint derived from ADMIN_PASSWORD_HASH in the session and compared it with a fingerprint derived from the current ADMIN_PASSWORD_HASH value.
A technical note about Flask sessions
Flask's default session is not necessarily a separate "box" stored on the server.
By default, Flask stores session data in a signed—not encrypted—cookie on the browser side.
For the purpose of understanding responsibilities, I separate the concepts like this:
Cookie = entry ticket
Session = entry state
This was another area where:
"It works" had made me think I understood it.
I Was Mixing Up APIs and API Keys
When I asked myself what an API was, my first answer was:
An API is something like a key.
That was another misunderstanding.
Now I separate them like this:
API
= A defined interface through which different applications or services communicate
API key
= Authentication information used to access that interface
Environment variable
= A way to keep values such as API keys outside the source code
The application retrieves the Gemini API key using:
api_key = os.environ.get("GEMINI_API_KEY")
and prepares the Gemini client with:
client = genai.Client(api_key=api_key)
So this does not mean:
Use the API key to log into the operating system.
os.environ.get() means:
Retrieve the value named
GEMINI_API_KEYfrom the environment variables available to the running application.
.gitignore Is Not a Feature That Hides Secrets
I also had an inaccurate understanding of .gitignore.
I thought of it as:
Something that hides variables people should not see.
That's not quite right.
.gitignore is:
a set of rules telling Git which untracked files and directories it should ignore.
The .gitignore I inspected during this audit included entries such as:
過去売上高/
*.xlsx
*.db
.env
Using a logistics analogy:
It is a "do not load" list for the truck heading to GitHub.
.gitignore itself does not encrypt anything.
And if secret information has already been committed to Git, adding the file to .gitignore afterward does not remove that secret from the existing Git history.
Trying to explain it forced me to understand that distinction much more clearly.
What Am I Actually Sending to Gemini?
I also inspected the AI part of the application.
Initially, I thought:
I'm sending the prompt written in the dashboard together with the sales numbers to Gemini.
In reality, the Gemini prompt-generation logic is separated into prompts.py.
For example:
def build_sales_prompt(sales_summary: str) -> str:
Then app.py creates a sales summary:
sales_summary = ", ".join(
[f"{name}: {qty}個" for name, qty in ranked_sales]
)
prompt = build_sales_prompt(sales_summary)
This inserts product names and sales quantities into the prompt.
For example:
Croissant: 80 units
White bread: 65 units
Melon bread: 42 units
Then the request is sent to Gemini:
response = client.models.generate_content(
model=config.GEMINI_MODEL,
contents=prompt,
)
and the returned text is displayed on the dashboard.
So the flow is:
PostgreSQL
↓
Aggregate sales data in Flask
↓
Product names + sales quantities
↓
Build instructions in prompts.py
↓
Gemini API
↓
Receive suggestions
↓
Display them on the dashboard
Saying "It Analyzes Current Social Media Trends" Was an Overstatement
This was one of the biggest discoveries in the audit.
The prompt I inspected at the time asked Gemini to consider things such as:
Recent food trends
Potential social media appeal
Patterns involving weekdays, seasons, and holidays
Because of that wording, I had started thinking:
Gemini is actively looking up current social media and internet trends and using them in its analysis.
But when I inspected the implementation, the store-specific information sent to Gemini by this feature was basically:
product names + aggregated sales quantities
There was also no implementation in this flow that retrieved current web-search results or social media data and passed them to Gemini.
So:
Writing "consider recent trends" in a prompt
and:
actually searching the current web and analyzing fresh information
are different things.
I noticed another related issue.
The prompt also asked Gemini to consider:
latent demand based on weekday, seasonal, and holiday patterns
But the store data sent to Gemini was primarily product names and aggregated sales quantities.
The application was not directly sending detailed daily or weekday-level sales records to Gemini in this flow.
So:
Asking the AI to analyze weekdays
and:
giving the AI enough data to perform weekday analysis
are also different things.
The lesson was:
Writing "search for it" in a prompt
≠
Implementing search
Writing "analyze weekdays" in a prompt
≠
Providing enough weekday-level data for that analysis
This was probably the clearest example in the entire audit of:
a difference between what I thought my working application could do and what the implementation actually did.
If I genuinely want future recommendations to include current web information or detailed weekday trends, I would need additional design work such as:
- Explicitly integrating a web-search capability
- Sending daily or weekday-level sales data
- Defining the analysis period and conditions clearly
Even 91 GREEN pytest Tests Do Not Mean "Safe"
One area I could explain relatively well was pytest.
At the time of this audit, the application had 91 pytest tests.
Why had I grown the suite that far?
Because I assumed:
Users will not always operate the system exactly the way I expect them to.
Examples include:
- Sending a product ID that does not exist
- Registering sales against a product from another month
- Registering sales for a discontinued product
- Directly accessing a protected URL without authentication
- Tampering with a CSRF token
- The Gemini API returning an error
- A database operation failing halfway through
In truck driving, I cannot think about safety only in terms of:
"Normally, nobody would do that."
Unexpected things happen.
So I try to think about dangerous scenarios before they occur.
I approach pytest in a similar way.
I have been growing it into:
an "incident prevention log" that records failure patterns before they can become repeat incidents.
But:
91 passed
does not mean:
There are no bugs.
It means:
All 91 expectations represented by the tests at that point passed in that run.
If I discover a new blind spot, I fix it and add another regression test.
I used to feel reassured simply by seeing GREEN.
Now I try to ask:
"What exactly does this GREEN result guarantee?"
Not Being Able to Explain Something Wasn't Embarrassing — It Showed Me What to Learn Next
At first, I felt a little embarrassed.
This was my own application, yet I discovered that:
- I couldn't explain SQLAlchemy
- I was confusing cookies with cache
- I was confusing APIs with API keys
- I thought GitHub was part of the sales-data path
- I thought my local Docker container was simply sent to Render
- I thought the application was retrieving current social media trends
But my perspective changed during the exercise.
Finding something I don't understand means I've found the next place to learn.
What seems more dangerous is:
continuing to use something while only thinking that I understand it.
Being Able to Explain Something Simply Does Not Mean Complete Understanding
There is an important limitation here.
Being able to say:
PostgreSQL is a warehouse.
does not mean I completely understand PostgreSQL.
Calling SQLAlchemy an intermediary does not mean I understand all of its internals.
What I did here was:
use explanation as a way to locate gaps in my understanding.
Being able to explain something is not the final goal.
I see it as:
one diagnostic method for checking understanding.
I Don't Need to Avoid Technical Terms
I also don't think technical terms need to be completely removed when explaining a system.
What matters is:
Can I translate them into words the listener can understand?
For example:
PostgreSQL
= A warehouse that stores application data
SQLAlchemy
= An intermediary between Python and the database
Dockerfile
= A loading and assembly instruction sheet for building the application
GitHub
= Storage for source code and change history
Render
= Rented infrastructure where the application runs
Cookie
= Entry ticket
Session
= Entry state
Flask-Login
= Entry-management staff
API
= A communication interface with another service
API key
= Authentication information used to access that interface
Combining:
the official technical term + language I personally understand
made the system much easier to organize in my head.
Writing Your Own Manual Can Be a Surprisingly Good Test
One thing I learned from this exercise is that asking:
Can I write the manual for my own system?
is a surprisingly powerful test.
For a user-facing manual, I need to explain:
What can the application do?
How do you operate it?
When is it useful?
What can it not do?
For a developer-facing explanation, I need to explain:
How does the data flow?
Why is this technology being used?
How does authentication work?
What is sent to the AI?
What remains available during failures?
What do the tests actually guarantee?
If I get stuck while trying to write one of these explanations, that may point to a gap in my understanding.
And if I find myself thinking:
"This operation is extremely difficult to explain."
the problem may not only be my understanding.
There may also be a UI/UX problem.
Writing documentation can itself become a system inspection.
This Is Not an Argument Against Using AI
This article is not saying:
Using AI to write code is bad.
I will continue using AI.
Without AI, I probably could not have experimented this much in such a short period of time.
But instead of stopping at:
AI created it
↓
It works
↓
Done
I think there is more value in continuing:
AI created it
↓
It works
↓
Why does it work?
↓
What does this component do?
↓
If I break it, will pytest detect it?
↓
Can I explain it to the user?
↓
Does my understanding match the actual code?
At that point, AI stops being only a code-generation machine.
It becomes a powerful learning tool.
Summary
I audited my own web application using 20 questions.
My initial result was:
🟢 6
🟡 10
🔴 4
But the score itself was not important.
Comparing my answers with the actual code revealed gaps such as:
- GitHub is not part of the runtime path for sales data
- A Dockerfile is an assembly instruction, not the finished product
- SQLAlchemy helps connect Python application logic with the database
- Cookies, sessions, and Flask-Login have different responsibilities
- An API and an API key are different things
-
.gitignoreis not an encryption feature - I verified what data was actually being sent to Gemini
- My belief that the application was analyzing current social media trends did not match the implementation
- What I ask an AI to analyze is not necessarily the same as the data I actually provide for that analysis
The biggest lesson was:
What I thought I knew was more dangerous than what I knew I didn't know.
In the AI era, the speed at which we can generate code will probably continue to increase.
That may make it even more important to spend time asking, in our own words:
What is this here for?
How does it work?
What happens if it breaks?
Can I explain it to the user?
I want to continue building with one rule in mind:
Don't stop at "it works." Keep going until I can explain it.
Top comments (0)