📝 Growing pytest into an “Incident Prevention Log” — Stage 3: Authentication, CSRF, and Access Control (51 69 Tests)

Hello from Japan 🇯🇵

I'm a truck driver in Japan, teaching myself web application development mainly with Python while continuing to work full-time.

In my personal Flask project, I've been trying to turn pytest into more than just a tool for checking whether the application works.

I think of it as an "incident prevention log":

a set of regression tests that helps keep previously discovered problems and near misses from silently returning.

This article was originally published in Japanese on Qiita and has been translated and adapted for DEV Community.

https://github.com/tosane932/sales_data_app

---

pytest Improvement Series

• Stage 1
  
  https://qiita.com/tosane932/items/f3de1e190873a90de39f

• Stage 2
  
  https://qiita.com/tosane932/items/b91261e7103df5792f7d

• Stage 3
  
  https://qiita.com/tosane932/items/6d1ca5490979c8cf9d62

• Stage 4
  
  https://qiita.com/tosane932/items/372270330e73583a227f

• Stage 5
  
  https://qiita.com/tosane932/items/85fd24c7baa6fe7c76a7

---

Introduction

Stage 3 became a fairly long article because it covers authentication, CSRF protection, and access control.

To make it easier to read, I've placed some of the implementation details inside expandable <details> sections.

You can follow the overall story without opening them, so feel free to expand only the sections that interest you.

So far, I've strengthened the test suite in stages:

• Stage 1: 3 → 9 tests
• Stage 2: 9 → 51 tests

In Stage 3, I focused mainly on:

authentication, CSRF protection, and access control

and increased the regression suite from:

51 passed
↓
69 passed

But once again, increasing the number of tests was not the goal itself.

What I wanted to check in Stage 3 was whether situations like these were possible:

Can anyone modify application data?

Even if someone is logged in,
can an external site cause an unintended POST request?

Can anonymous users access application pages or APIs
that should require authentication?

Can an anonymous user trigger the AI API?

As in the previous stages, I worked with Codex in this order:

Investigate
↓
RED
↓
Smallest necessary fix
↓
GREEN

---

Stage 3 Started with 51 Passing Tests

At the end of Stage 2:

51 passed

The existing pytest suite already covered areas such as:

• sales input validation
• product registration validation
• invalid product IDs
• products belonging to another month
• discontinued products
• database unique constraints
• rollback when commit() fails
• soft deletion and preservation of sales history
• dashboard aggregation

However, when I investigated authentication-related behavior, I found that:

Authentication
CSRF protection
Access control

were still not sufficiently implemented.

So I decided to work through Stage 3 in this order:

Authentication
↓
CSRF
↓
Access control for application pages and APIs

---

First, I Investigated the Current State Without Fixing Anything

The first thing I asked Codex to do was not to implement authentication.

I asked it only to:

investigate the current authentication state.

At that point, the application had:

• no login page
• no User model
• no Flask-Login
• no CSRF tokens
• anonymous access to application pages and APIs

For state-changing POST requests in particular:

POST /
POST /input

could be reached without authentication.

In other words:

an unauthenticated user could reach the code that modifies product and sales data.

Instead of fixing that immediately, I first decided to express the dangerous state as failing pytest tests.

---

Authentication: Make Unauthenticated POST Requests RED

The first tests I added were:

Reject unauthenticated product POST requests.

Reject unauthenticated sales POST requests.

The future behavior I wanted was:

anonymous
↓
POST
↓
302
↓
/login
↓
no database changes

Before implementing authentication, however:

Expected: 302
Actual:   200

and the request actually reached the database modification logic.

That was exactly the RED state I wanted to confirm.

🧪 Authentication: From RED Tests to Implementation

I Defined the Login Behavior with RED Tests First

Next, I added tests for the behavior I wanted:

• GET /login should render successfully
• correct credentials should log the user in
• an incorrect password should not log the user in
• the login state should persist in the same test client
• unauthenticated POST requests should be rejected

Because no login functionality existed yet:

5 failed

Only after confirming that RED state did I move on to implementing authentication.

I Chose a Single-Admin Login

For this application, I deliberately did not expand the scope into full multi-user management.

Instead, I chose:

single-admin authentication

using Flask-Login.

The credentials are loaded from environment variables:

SECRET_KEY
ADMIN_USERNAME
ADMIN_PASSWORD_HASH

Rather than storing a production plaintext password in the source code, I verify a password hash using:

check_password_hash()

At this stage, I did not add:

• a User database model
• general users
• roles
• tenants
• store-specific permissions

For the current specification:

authenticated user
=
single administrator

So Stage 3 does not implement detailed role-based authorization.

The target here was simply:

only the authenticated administrator can access protected application functionality.

After Authentication: 56 Tests GREEN

After implementing the single-admin authentication:

pytest test_auth.py -v
→ 5 passed

pytest -v
→ 56 passed

At that point, I created a local commit:

feat: add single-admin authentication

I wanted to preserve the point where authentication alone was GREEN instead of continuing straight into CSRF changes in the same step.

---

Next, I Investigated CSRF Protection

Adding authentication does not automatically mean:

"The application is now safe."

When authentication uses a session cookie, the browser automatically includes that cookie with matching requests.

That means I also needed to consider whether:

a logged-in user's session could be abused to send a POST request that the user did not intend.

So the next step was CSRF protection.

---

I Created RED Tests for CSRF Before Implementing It

I added six CSRF-related tests.

Before implementation:

6 failed, 56 passed

The forms had no CSRF token.

And while authenticated, product and sales POST requests still worked without a token.

Even the login POST succeeded with the correct username and password when no CSRF token was supplied.

🧪 The Six CSRF RED Tests and Implementation Details

CSRF Tests I Added

The login form contains a CSRF token.

The product form contains a CSRF token.

The sales form contains a CSRF token.

Reject a login POST without a token.

Reject a product POST without a token.

Reject a sales POST without a token.

Without a Token, the Request Could Still Modify the Database

Before adding CSRF protection, a product POST without a token resulted in:

status=200
products_unchanged=False

In other words:

a request without a CSRF token could still reach the Product modification logic.

The sales POST behaved similarly:

status=200
sales_unchanged=False

and DailySales was modified.

Instead of stopping at the observation:

There is no CSRF protection.

I wanted the RED tests to show:

how far the current request could actually proceed.

Adding CSRFProtect with Flask-WTF

I used Flask-WTF for CSRF protection.

CSRFProtect(app)

Then I added a hidden CSRF field to the three POST forms:

The protected forms were:

/login
/
/input

I did not add CSRF exemptions.

I also did not disable CSRF protection in the test environment.

Existing Tests Also Had to Become "Legitimate POST Requests"

Once CSRFProtect was enabled, existing positive-path tests also returned 400 if they posted without a token.

But tests intended to verify things such as:

product validation

sales rollback

need to reach the business logic.

If CSRF blocks them first, those tests are no longer testing what they are supposed to test.

So I changed the test flow to:

GET
↓
retrieve csrf_token
↓
POST with a valid token

I created a test fixture for retrieving CSRF tokens.

The authenticated test client also does not directly inject a logged-in state into the session.

Instead, it follows a flow closer to the real login process:

GET /login
↓
retrieve CSRF token
↓
username
password
csrf_token
↓
POST /login
↓
authenticated client

After CSRF Protection: 62 Tests GREEN

After implementation:

pytest test_csrf.py -v
→ 6 passed

pytest test_auth.py -v
→ 5 passed

pytest -v
→ 62 passed

For product and sales POST requests without a token:

CSRFProtect rejects the request with 400
↓
no database changes

For the product POST, I compare snapshots of both Product and DailySales.

For the sales POST, I verify that DailySales is unchanged.

For login without a token:

400
↓
no authenticated session is created

I then created another commit:

feat: add CSRF protection

---

Even After Adding Authentication and CSRF, Anonymous Users Could Still Access Application Pages

After adding authentication and CSRF protection, I investigated the GET routes and APIs.

The routes I checked were:

/login
/
/input
/dashboard
/api/dashboard-data
/api/ai-advice
/api/greeting

At that point, anonymous requests to all of them returned:

HTTP 200

/login should of course remain public.

But the application pages and APIs that were intended to require authentication were still accessible anonymously.

---

Anonymous Users Could Even Reach the AI API

The endpoints that concerned me most were:

/api/ai-advice
/api/greeting

During the investigation, I did not connect to the real Gemini API.

Instead, I mocked the Gemini client.

The result was that an anonymous request still caused:

Gemini Client
→ called once

So under the right conditions:

an unauthenticated user could reach the AI-processing logic.

---

Access Control Also Started with RED Tests

So I added a new test file:

test_authorization.py

The filename uses authorization, but I want to be precise about what it means in this project.

I was not testing detailed role-based authorization.

What I was testing was:

access control that prevents anonymous users from reaching application pages and APIs that should require authentication.

The expected behavior became:

GET /login
→ anonymous 200

All other application pages/APIs that require authentication
→ anonymous 302
→ /login

The initial result:

1 passed, 6 failed

Only the public /login route was GREEN.

The other six routes still returned HTTP 200 to anonymous users, so they were RED.

🔐 Access Control Implementation and Existing Test Adjustments

Codex Stopped When It Realized Existing Tests Needed Changes

Something memorable happened at this point.

While preparing to implement access control, Codex determined that:

some existing tests would also need to change.

And it stopped before implementing anything.

For example:

test_dashboard.py

was checking API aggregation using an anonymous client.

Once the API became authentication-protected, that client would receive a 302 redirect and never reach the aggregation logic.

Some CSRF tests similarly retrieved tokens from application pages while anonymous.

So I reorganized the responsibilities:

test_authorization.py
→ verifies anonymous users cannot enter protected areas

test_dashboard.py
→ verifies correct aggregation after authentication

test_csrf.py
→ verifies CSRF behavior

test_auth.py
→ verifies authentication behavior

I had already been instructing Codex:

if an unexpected change becomes necessary, do not continue automatically — report it first.

This time, it followed that rule.

It stopped instead of silently modifying existing tests, so I reviewed the reason and then allowed only the minimum necessary changes.

Protecting Six Routes with login_required

In the end, I added login_required to:

/
/input
/dashboard
/api/dashboard-data
/api/ai-advice
/api/greeting

After implementation:

GET /login
→ anonymous 200

Other six routes
→ anonymous 302
→ /login

For the AI endpoints, I also added regression tests confirming that, during anonymous access:

Gemini Client
→ 0 calls

So an anonymous user no longer reaches the AI-processing code.

---

Stage 3 Finished with 69 GREEN Tests

The progression was:

51 passed
↓
56 passed
↓
62 passed
↓
69 passed

Stage 3 directly added:

Authentication     5 tests
CSRF               6 tests
Access control     7 tests

for a total of 18 additional tests:

51
↓
69

✅ Final pytest Results and Test Responsibilities

Final pytest Results

pytest test_authorization.py -v
→ 7 passed

pytest test_auth.py -v
→ 5 passed

pytest test_csrf.py -v
→ 6 passed

pytest test_dashboard.py -v
→ 4 passed

pytest -v
→ 69 passed

The Tests Started to Develop Clear Responsibilities

One of the biggest things I noticed during Stage 3 was not simply that the test count increased.

It was that:

each group of tests started to have a clearer responsibility.

test_auth.py
→ login behavior and rejection of unauthenticated POST requests

test_csrf.py
→ CSRF tokens and rejection of tokenless POST requests

test_authorization.py
→ anonymous users cannot access protected pages/APIs

test_dashboard.py
→ aggregation after authentication

test_products.py
→ product behavior after authentication and CSRF checks

test_sales.py
→ sales behavior after authentication and CSRF checks

Earlier, many of my pytest tests were closer to:

A response came back.

Now they were gradually becoming:

"Which specific incident is this test responsible for stopping?"

---

I Also Used a Feature Branch and Pull Request

For Stage 3, I worked on:

feature/auth-hardening

After confirming all 69 tests were GREEN locally, I followed this flow:

feature/auth-hardening
↓
push to GitHub
↓
Pull Request
↓
GitHub Actions
↓
merge into main

I had already been using local pytest and GitHub Actions.

But this time, I also went through the process of:

creating a Pull Request from a feature branch, confirming CI success, and only then merging into main.

🚚 Commit, Pull Request, and GitHub Actions Details

I Split the Commits by Stage

I used separate commits:

feat: add single-admin authentication

feat: add CSRF protection

feat: protect authenticated routes

In the Pull Request, I checked:

• base: main
• compare: feature/auth-hardening
• three commits
• the diff
• whether there were merge conflicts
• GitHub Actions results

Only after that did I merge.

GitHub Actions Also Passed All 69 Tests

After creating the Pull Request, GitHub Actions ran Run Tests.

CI collected:

collected 69 items

and finished with:

69 passed

GitHub Actions reported:

Success

Only after confirming that result did I merge into main.

So the Stage 3 flow became:

Local pytest
69 passed
↓
Push feature branch
↓
Pull Request
↓
GitHub Actions
69 passed
↓
Merge into main

I Ran All 69 Tests Again After the Merge

After merging the Pull Request, I switched my local environment back to main and synchronized it with origin/main.

I confirmed that the local and remote main HEADs matched.

Then I ran:

pytest -v

one more time.

The result:

69 passed

The working tree was also clean.

At that point, I considered Stage 3 complete.

---

A Pull Request Feels Like a Shipping Checkpoint to Me

I've previously compared pytest to:

a pre-departure inspection

and GitHub Actions to:

an automated inspection at the shipping gate.

Using Pull Requests added another checkpoint:

working branch
↓
Pull Request
↓
CI
↓
main

Thinking about it through my work as a truck driver, it feels something like:

Inspect in the work area
↓
Check whether it is ready to ship
↓
Automated inspection
↓
Send it onto the main route

Instead of saying:

"It works locally, so put it straight into main."

I now have another place to review the diff and test results before the code enters main.

Using the process myself helped me understand why that extra checkpoint matters.

---

State of the Application at the End of Stage 3

At the end of Stage 3:

Single-admin authentication
✅

CSRF protection
✅

Anonymous access blocked for protected pages/APIs
✅

Anonymous AI API execution blocked
✅

pytest
69 passed

GitHub Actions
69 passed / Success

Pull Request
✅

Merged into main
✅

Post-merge main
69 passed

For the scope I defined:

Stage 3 was complete.

---

What I Deliberately Did Not Do in Stage 3

During the final review, I also found several things where I decided:

"I found this, but I'm not fixing it in this stage."

If I expanded into every improvement I discovered, the goal of Stage 3 would become unclear.

So I kept the scope to:

Single-admin authentication
CSRF protection
Blocking anonymous access to protected pages/APIs

📋 Improvements I Deliberately Left for Later

The final review identified possible future improvements such as:

logout
dedicated fail-closed tests for missing configuration
dedicated invalid-CSRF-token tests
Session Cookie settings
login attempt rate limiting
JSON 401 responses for APIs
cleaning up duplicated authentication checks

Logout Is Not Implemented Yet

At this point, there is still no:

logout route
logout_user()
logout button

Anonymous access protection itself works.

However, because the authentication system uses a Session Cookie, allowing an administrator to explicitly end a session is something I want to add later.

This becomes especially important if the application is used on a shared device.

I plan to reconsider it before a real production rollout.

There Is No Detailed Role-Based Authorization Yet

The current application still has no:

• User database model
• general users
• roles
• tenants
• store IDs
• store-level access control

The only user who can log in is the single administrator configured through environment variables.

So Stage 3 assumes:

authenticated user
=
administrator

and considers the following sufficient for this stage:

only the authenticated administrator can reach protected application functionality.

If I introduce multiple users in the future, I will need to design role-based and store-level permissions separately.

Production Requires Environment Variables

The authentication implementation reads:

SECRET_KEY
ADMIN_USERNAME
ADMIN_PASSWORD_HASH

from environment variables.

So these values must be configured correctly before using authentication in production.

In particular:

ADMIN_PASSWORD_HASH

expects a Werkzeug-compatible password hash, not a plaintext password.

Stage 3 did not include changing the production environment configuration on Render.

---

Summary

Stage 3 of strengthening pytest moved the suite from:

51 passed
↓
69 passed

The main additions were:

• single-admin authentication
• CSRF protection
• access control for protected pages and APIs
• preventing anonymous AI API execution
• regression tests for authentication, CSRF, and access control

Stage 1 began with:

turning only three pytest tests into the beginning of a regression suite.

Stage 2 expanded into:

database updates, rollback behavior, soft deletion, and aggregation.

And Stage 3 expanded the question again.

It was no longer only:

"Can the application save the correct data?"

It also became:

"Who is allowed to reach the code that performs that operation?"

This was also the first stage where I actually used the full flow:

feature branch
↓
Pull Request
↓
GitHub Actions
↓
merge into main

There are still areas I can improve.

But by continuing with:

Investigate
↓
RED
↓
Smallest necessary fix
↓
GREEN
↓
Commit
↓
Next problem

pytest has gradually started to feel less like:

something I run after an incident

and more like:

a mechanism that catches known failure conditions before they can cause the same problem again.

Next, I plan to move on to Stage 4.

---

pytest Improvement Series

• Stage 1
  
  https://qiita.com/tosane932/items/f3de1e190873a90de39f

• Stage 2
  
  https://qiita.com/tosane932/items/b91261e7103df5792f7d

• Stage 3
  
  https://qiita.com/tosane932/items/6d1ca5490979c8cf9d62

• Stage 4
  
  https://qiita.com/tosane932/items/372270330e73583a227f

• Stage 5
  
  https://qiita.com/tosane932/items/85fd24c7baa6fe7c76a7

---

https://github.com/tosane932/sales_data_app

https://qiita.com/tosane932

Comments

[Alex Shev]

Turning pytest into an incident prevention log is a strong framing. Tests become more valuable when they explain the failure class they protect against, not just whether the current implementation still passes.