Running Apache Airflow locally is the fastest way to test your DAGs, but it comes with a hidden cost: SQLite. Because local Airflow relies on a simple file-based database and background web workers, a single ungraceful shutdown can leave your environment completely deadlocked.
If you are fighting database is locked errors, port collisions, or mysterious 'Unauthorized' login failures, you don't need to rebuild your environment. You just need to know how to properly clear the cache. I had to learn this through first-hand experience, and here is how you can handle it:
1. The SQLite Death Grip
The Trade-off: Airflow uses SQLite by default because it requires zero configuration; it’s literally just a file on your hard drive. The massive downside is that SQLite handles concurrent connections poorly. If your standalone server hangs and you force-quit your terminal, background processes maintain a death grip on that file.
When you try to boot back up, you are met with a wall of error logs that looks like this:
The Fix: You have to nuke the invisible processes and delete the corrupted database to start fresh. Run this in your terminal:
# 1. Kill any hidden background processes
pkill -9 -f airflow
# 2. Delete the deadlocked database (and any journal files)
rm ~/airflow/airflow.db*
2. The Port 8080 Hostage Situation
The Trade-off: airflow standalone is actually a wrapper command that spins up four heavy components at once (scheduler, webserver, triggerer, database). Pushing this to the background or shutting it down ungracefully often leaves orphaned web workers alive.
You will know you hit this trap if your UI refuses to load and your logs spit out an Errno 98.Here's a snippet of what it might look like:
api-server | INFO: Started server process [25087]
api-server | INFO: Waiting for application startup.
api-server | INFO: Application startup complete.
api-server | ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8080): address already in use
api-server | INFO: Waiting for application shutdown.
api-server | INFO: Application shutdown complete.
The Fix: Those orphaned workers are usually gunicorn processes. You will need to terminate them so your new server can bind to the port.
pkill -9 -f gunicorn
3. The "Ghost User" Lockout
The Trade-off: To save you from having to create an admin profile manually every time you test locally, standalone Airflow automatically generates a password and saves it to a tiny hidden simple_auth_manager_passwords.json.generated file.
However, if you just deleted your airflow.db to fix the SQLite lock mentioned above, you also deleted your user account. But because that .generated text file still exists, Airflow assumes the user is still active and refuses to give you a new password. You will type the exact password into the UI and immediately hit a 401 Unauthorized error.
If you check your startup logs, you will see it completely bypassing the password generation:
The Fix: You need to delete that hidden file so Airflow is forced to mint a brand new admin user from scratch.
rm ~/airflow/simple_auth_manager_passwords.json.generated
The Ultimate Reset Protocol
If your local environment is completely frozen, chain it all together. This is the exact sequence to completely factory reset a jammed standalone server without touching a single line of your DAG code:
pkill -9 -f airflow
pkill -9 -f gunicorn
rm -f ~/airflow/airflow.db*
rm ~/airflow/simple_auth_manager_passwords.json.generated
airflow standalone
When you run that final command, Airflow will breathe easily, rebuild a clean database, bind to port 8080 without a fight, and print a new password right in your terminal.
And that's on another day of fighting errors in the terminal😂.
Top comments (0)