DEV Community

Cover image for Testing Data Persistence, Service Name Resolution, and stdout Logging in a Flask and PostgreSQL Multi-Container Environment
tosane932
tosane932

Posted on • Edited on • Originally published at qiita.com

Testing Data Persistence, Service Name Resolution, and stdout Logging in a Flask and PostgreSQL Multi-Container Environment

#docker #flask #postgres #python

This article was originally published in Japanese on Qiita and has been translated and adapted for DEV Community.

Overview

Hello from Japan! 🇯🇵

I am a professional truck driver teaching myself Python while working toward a career transition into web engineering.

After 108 cumulative hours of Python study, I conducted hands-on tests in a multi-container environment consisting of Flask as the web application and PostgreSQL as the database, managed with Docker Compose.

This article reports the observed results of the following three tests:

  1. Data persistence using Docker volumes
  2. Service-name resolution between containers
  3. Error-handling and logging design using Python's logging module

I also tested the troubleshooting process for an invalid or missing environment variable while integrating the application with the Gemini API.

Through this process, I examined the importance of logging in containerized application development.

📦 1. Testing Container Lifecycles and Data Persistence

What this section covers:

Does database data really remain after the containers are removed with docker compose down?

When containers are removed using docker compose down, data stored only inside those containers is also lost.

However, sales_data_app uses a named volume to preserve PostgreSQL data independently of the database container.

The following configuration is defined in docker-compose.yml:

services:
  db:
    image: postgres:16
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:
Enter fullscreen mode Exit fullscreen mode

The PostgreSQL data directory inside the container:

/var/lib/postgresql/data
Enter fullscreen mode Exit fullscreen mode

is connected to a named volume called:

postgres_data
Enter fullscreen mode Exit fullscreen mode

Because this storage exists independently of the container lifecycle, the data should remain even when the container is deleted and recreated.

🔍 Hands-On Test: Does the Data Remain After down and up?

First, I confirmed from the host machine that the volume had actually been created.

$ docker volume ls

DRIVER    VOLUME NAME
local     sales_data_app_postgres_data
Enter fullscreen mode Exit fullscreen mode

Next, I checked the existing number of database records.

I then intentionally removed and recreated the containers before checking the record count again.

Step Command Result
1. Check the record count before removal docker compose exec db psql -U postgres -d sales_db -c "SELECT COUNT(*) FROM daily_sales;" 5 records
2. Remove the containers docker compose down —
3. Recreate the containers docker compose up -d —
4. Check the record count again docker compose exec db psql -U postgres -d sales_db -c "SELECT COUNT(*) FROM daily_sales;" 5 records, unchanged

✅ Analysis of the Result

After removing the containers and creating new ones, the number of records remained at 5.

This result confirmed that docker compose down removes the containerized runtime environment, but does not remove data stored in a separately defined named volume.

In other words:

  • Containers are disposable runtime units
  • Persistent data is an asset with a separate lifecycle
  • Stateful data should be externalized using named volumes or bind mounts

By actually destroying and recreating the containers, I verified this fundamental principle of container design instead of relying only on documentation or assumptions.

🌐 2. Testing Service Name Resolution Between Containers

What this section covers:

Why does the web container connect to PostgreSQL using the hostname db instead of localhost?

In a Docker Compose environment, using localhost to connect from one container to another causes a connection failure.

This is because localhost inside a container refers to that container itself.

For example, from inside the web container:

localhost
Enter fullscreen mode Exit fullscreen mode

means the web container—not the PostgreSQL container.

When Docker Compose starts the services, it automatically creates a dedicated bridge network.

Within this network, service names defined in docker-compose.yml can be used as hostnames.

Docker automatically resolves those service names into internal IP addresses.

The database connection used by sales_data_app therefore specifies:

db:5432
Enter fullscreen mode Exit fullscreen mode

instead of directly specifying an IP address.

This allows the application to continue communicating with the database even if the internal IP address changes after the containers are recreated.

🔍 Hands-On Test: Is the Service Name Really Converted into an IP Address?

To confirm that this name-resolution mechanism was working, I entered the running web container and used Python's standard socket module to resolve the service name db.

# Open a Bash shell inside the web container
$ docker compose exec web bash

# Resolve the service name "db" using Python's socket library
root@1d708ec39a0e:/app# python -c "import socket; print(socket.gethostbyname('db'))"
Enter fullscreen mode Exit fullscreen mode

Output:

172.18.0.2
Enter fullscreen mode Exit fullscreen mode

✅ Analysis of the Result

Inside the Linux environment of the web container, the string:

db
Enter fullscreen mode Exit fullscreen mode

was successfully resolved to the concrete internal IP address:

172.18.0.2
Enter fullscreen mode Exit fullscreen mode

The IP address may change when the container is recreated.

The service name, however, remains stable.

This test confirmed why container connection settings should use service names instead of fixed internal IP addresses.

Using:

db
Enter fullscreen mode Exit fullscreen mode

allows the application to communicate without depending on an environment-specific IP address.

The result demonstrated that Docker Compose service discovery provides a stable connection method even when the underlying containers change.

📝 3. Logging Design: Sending Logs to stdout and stderr

What this section covers:

Why should container logs be sent to standard output instead of being stored only in files inside the container?

In containerized applications, logs should generally be sent to:

  • Standard output: stdout
  • Standard error output: stderr

rather than being stored only in log files inside the container.

This allows Docker and hosting platforms to collect and manage application logs centrally.

I introduced Python's standard logging module to capture:

  • Application lifecycle events
  • Database initialization results
  • Warnings
  • API requests
  • Errors
  • Stack traces

Application Code: Extract from app.py 

import logging
import os

from flask import Flask, jsonify, render_template, request

import config
from models import db


# Configure the log level and format for standard output.
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S",
)

logger = logging.getLogger(__name__)


app = Flask(__name__)
app.config["SQLALCHEMY_DATABASE_URI"] = config.SQLALCHEMY_DATABASE_URI
db.init_app(app)


# Record the database initialization process.
with app.app_context():
    try:
        db.create_all()
        logger.info("Database tables initialized successfully.")
    except Exception:
        logger.error(
            "Failed to initialize database tables.",
            exc_info=True,
        )


def _generate_ai_advice(ranked_sales):
    if not ranked_sales:
        logger.warning(
            "AI advice was requested, but ranked_sales is empty."
        )
        return "No sales data is available."

    try:
        api_key = os.environ.get("GEMINI_API_KEY")

        if not api_key:
            logger.error(
                "GEMINI_API_KEY is missing from the environment variables."
            )
            return "Configuration error."

        logger.info(
            "Requesting Gemini AI advice for %d products.",
            len(ranked_sales),
        )

        # Send the API request here.

        logger.info(
            "Gemini AI advice was generated successfully."
        )

        return response.text

    except Exception:
        # exc_info=True includes the full stack trace.
        logger.error(
            "Unexpected error during AI advice generation.",
            exc_info=True,
        )

        return "A temporary error occurred."
Enter fullscreen mode Exit fullscreen mode

Instead of using only print(), I assigned different levels according to the meaning of each event:

  • INFO: Normal application activity
  • WARNING: Unexpected but recoverable situations
  • ERROR: Failed processing or invalid configuration

The exc_info=True argument also makes it possible to output the full stack trace when an exception occurs.

🔧 4. Troubleshooting Using Actual Container Logs

What this section covers:

The complete process of identifying and fixing an actual API error using only the application logs.

I started the system and executed the external API feature through the browser.

The following output appeared in the container logs.

Actual Error Log 

web-1  | 2026-07-01 11:23:53 [INFO] Generating AI daily greeting...
web-1  | 2026-07-01 11:23:54 [ERROR] Failed to generate AI greeting: 400 INVALID_ARGUMENT. {'error': {'code': 400, 'message': 'API key not valid. Please pass a valid API key.'}}
web-1  | Traceback (most recent call last):
web-1  |   File "/app/app.py", line 231, in api_greeting
web-1  |   ...
web-1  | google.genai.errors.ClientError: 400 INVALID_ARGUMENT.
Enter fullscreen mode Exit fullscreen mode

Identifying the Cause and Applying the Fix

Step Action
Analyze the log The exception ClientError: 400 INVALID_ARGUMENT and the message API key not valid indicated that GEMINI_API_KEY was invalid or not configured correctly
Apply the fix Checked the .env file referenced by docker-compose.yml and replaced the incorrect value with the correct API key
Reapply the environment Ran docker compose down, followed by docker compose up, to recreate the containers with the corrected configuration

As confirmed in Section 1, recreating the containers did not delete the PostgreSQL data.

This was possible because the container runtime and persistent database volume had separate lifecycles.

The environment configuration could therefore be changed and reapplied without risking the loss of database records.

After recreating the containers, I executed the same feature again.

The log output changed to the following:

web-1  | 2026-07-01 11:24:26 [INFO] Requesting Gemini AI advice for products: 5 items.
web-1  | 2026-07-01 11:24:26 [INFO] Gemini AI advice generated successfully.
Enter fullscreen mode Exit fullscreen mode

The application changed from an error state to a successful execution state.

The troubleshooting process was:

  1. Observe the error log
  2. Read the exception type
  3. Identify the relevant error message
  4. Form a hypothesis about the environment variable
  5. Check and correct the .env file
  6. Recreate the containers
  7. Run the same operation again
  8. Confirm successful INFO output

This test demonstrated that properly designed logs can reveal the root cause without relying on guesswork.

🚩 Conclusion

In multi-container development, several important behaviors can easily become hidden inside a black box.

In this article, I made the following three areas visible through hands-on testing and stdout logging:

  1. The difference between the container lifecycle and the data lifecycle
  2. The behavior of communication and service-name resolution between containers
  3. The internal error state of the application

The tests confirmed that:

  • PostgreSQL data remained after the containers were recreated because it was stored in a named volume
  • The service name db was automatically resolved to the database container's internal IP address
  • Application and API errors could be investigated using structured logs and stack traces
  • Environment settings could be corrected and reapplied without losing persistent database data

Troubleshooting based on facts produced by the system is more reliable than troubleshooting based on assumptions.

By separating disposable runtime environments from persistent data, using stable service names for container communication, and sending structured logs to standard output, a containerized application becomes easier to maintain and diagnose.

sales_data_app Maintenance Series

Pre-Production Inspection Trilogy

Post-Deployment Maintenance Series

🍞 sales_data_app on GitHub

Top comments (2)

Collapse
 
mads_hansen_27b33ebfee4c9 profile image
Mads Hansen •

This is a good progression from observation to evidence. One distinction worth making explicit is that “the named volume survived docker compose down” proves persistence across container replacement, not backup or recoverability. I’d add three destructive tests: docker compose down -v in an isolated environment, restore into a fresh empty volume from pg_dump or a physical backup, and a PostgreSQL image-version upgrade with the same data. Measure restored row counts plus constraints or checksums, not only the original count.

For startup behavior, add a PostgreSQL healthcheck and make the app retry until the database is ready—DNS resolution only proves the name resolves, not that Postgres accepts connections or migrations are complete. The stdout logging is useful; structured JSON, request or trace IDs, and explicit redaction tests for API keys, connection strings, user data, and exception payloads would make those logs safer to centralize.

Collapse
 
tosane932 profile image
tosane932 •

Thank you very much for the detailed feedback.🥰

Your distinction between persistence and backup/recoverability is especially helpful. My test only confirmed that the named volume survived container replacement, so I should be more precise about what the result actually proves.

I also appreciate the suggestions for destructive and recovery testing. Testing docker compose down -v, restoring into a fresh volume with pg_dump, and verifying constraints or checksums would be a meaningful next step.

I currently use a PostgreSQL healthcheck with pg_isready and condition: service_healthy in Docker Compose, but I agree that DNS resolution alone only proves name resolution. It does not prove that PostgreSQL is ready to accept connections or that migrations have completed.

The suggestions about structured JSON logging, trace IDs, and redaction tests are also new learning points for me. I will keep them in mind as I continue improving the application.

Thank you for reading the article so carefully and for giving me concrete next steps.