DEV Community: Oleksandr Kashytskyi

Ever seen a docker-compose.test.yml that's just a full copy of docker-compose.yml? I wrote a short guide on fixing that with one shared base file, extends and profiles, so a new environment is a few lines instead of a copy-paste. Full code inside, ready to

Oleksandr Kashytskyi — Tue, 13 Oct 2026 10:27:53 +0000

Oleksandr Kashytskyi

Oct 13

Efficent usage of Docker

#architecture #devops #docker #softwareengineering

6 min read

Efficent usage of Docker

Oleksandr Kashytskyi — Tue, 13 Oct 2026 10:24:19 +0000

I've seen this more than once, even on production projects. Somebody needs a test environment, so they create docker-compose.test.yml, copy the whole docker-compose.yml into it, and declare every service again from scratch.

Ctrl+C and Ctrl+V are great shortcuts, until they end up in your infrastructure. Sooner or later the copies drift apart: one file gets a Postgres upgrade, the other doesn't, and your tests happily pass against a database version you don't actually run.

Docker Compose can do much better with two features:

A shared base file: every service is declared once and reused with extends.
Profiles: one docker-compose.yml holds all environments, and you choose which one to start.

Add one env file per environment and a well-chosen command, and adding a new environment becomes a few lines instead of a full copy-paste.

The snippets are simplified from my side project, a flower shop backend, but the pattern works for any stack.

The full setup
How to run it
The shared base file
Profiles
Env files
The command key
Try it yourself

The full setup

Here's everything at once, then we'll take it apart piece by piece. The example is a Node.js app with PostgreSQL and MailHog (a fake SMTP server that catches outgoing emails).

.
├── docker-compose.base.yml   # shared service settings
├── docker-compose.yml        # environments: dev and test
├── .env.dev
└── .env.test

docker-compose.base.yml

services:
  app:
    image: node:22.12.0
    ports:
      - ${PORT}:${PORT}
    volumes:
      - ./:/usr/src/app
    working_dir: /usr/src/app

  postgres:
    image: postgres:17.3
    ports:
      - ${POSTGRES_PORT}:${POSTGRES_PORT}
    environment:
      - POSTGRES_DB=${POSTGRES_DB}
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - PGPORT=${POSTGRES_PORT}

  mailhog:
    image: mailhog/mailhog
    ports:
      - ${EMAIL_SMTP_PORT}:${EMAIL_SMTP_PORT}  # SMTP
      - ${EMAIL_HTTP_PORT}:${EMAIL_HTTP_PORT}  # web UI
    environment:
      - MH_SMTP_BIND_ADDR=0.0.0.0:${EMAIL_SMTP_PORT}
      - MH_UI_BIND_ADDR=0.0.0.0:${EMAIL_HTTP_PORT}
      - MH_API_BIND_ADDR=0.0.0.0:${EMAIL_HTTP_PORT}

docker-compose.yml

services:
  dev_app:
    extends:
      file: docker-compose.base.yml
      service: app
    profiles:
      - dev
    env_file: .env.dev
    depends_on:
      - dev_postgres
      - mailhog
    command: sh -c "npm ci && npm run start"

  test_app:
    extends:
      file: docker-compose.base.yml
      service: app
    profiles:
      - test
    env_file: .env.test
    depends_on:
      - test_postgres
      - mailhog
    command: sh -c "npm ci && npm test"

  dev_postgres:
    extends:
      file: docker-compose.base.yml
      service: postgres
    profiles:
      - dev
    volumes:
      - ./docker/pgdata:/var/lib/postgresql/data

  test_postgres:
    extends:
      file: docker-compose.base.yml
      service: postgres
    profiles:
      - test
    tmpfs:
      - /var/lib/postgresql/data

  mailhog:
    extends:
      file: docker-compose.base.yml
      service: mailhog
    profiles:
      - dev
      - test

.env.dev

NODE_ENV=dev
PORT=3000
JWT_SECRET=dev_secret

POSTGRES_PORT=54321
POSTGRES_DB=development
POSTGRES_USER=admin
POSTGRES_PASSWORD=postgres
POSTGRES_URL=postgresql://admin:postgres@dev_postgres:54321/development

EMAIL_HOST=mailhog
EMAIL_SMTP_PORT=1025
EMAIL_HTTP_PORT=8025

.env.test

NODE_ENV=test
PORT=3001
JWT_SECRET=test_secret

POSTGRES_PORT=54322
POSTGRES_DB=test
POSTGRES_USER=test
POSTGRES_PASSWORD=test
POSTGRES_URL=postgresql://test:test@test_postgres:54322/test

EMAIL_HOST=mailhog
EMAIL_SMTP_PORT=1025
EMAIL_HTTP_PORT=8025

How to run it

One command per environment:

# Development
docker compose --env-file .env.dev --profile dev up

# Tests
docker compose --env-file .env.test --profile test up --exit-code-from test_app

That's the whole interface. Now let's see why it works.

The shared base file

docker-compose.base.yml describes each service once, with only the settings that are the same in every environment: image, ports, volumes and basic configuration. There are no profiles, no names and no commands in it. It's not meant to be started on its own; it's a library of building blocks.

docker-compose.yml then builds the real services from those blocks with extends:

dev_postgres:
  extends:
    file: docker-compose.base.yml
    service: postgres
  profiles:
    - dev

dev_postgres gets everything from the base postgres service and adds only what makes it different.

Why bother: the Postgres version, ports and settings now live in exactly one place. Upgrade the image once and every environment gets it. No more "which file is the right one?"

Tip: with extends, profiles and env files combined, it can be hard to see what Compose will actually run. docker compose config prints the final merged result with all variables filled in:

docker compose --env-file .env.dev --profile dev config

Profiles

A profile is a label on a service. When you run docker compose --profile dev up, only services labelled dev start:

Profile	Starts
`dev`	`dev_app`, `dev_postgres`, `mailhog`
`test`	`test_app`, `test_postgres`, `mailhog`

This is what replaces the copy-pasted docker-compose.test.yml. Both environments live in one file, side by side, and each one only adds what is actually different.

A few decisions worth explaining:

Postgres is per environment. dev_postgres mounts a folder, so my dev data survives restarts. test_postgres keeps its data in memory (tmpfs), so every test run starts with an empty database.
MailHog is shared. It's listed in both profiles, dev and test. It doesn't care which environment sends an email; it catches everything and judges nobody.

Why tmpfs and not just "no volume"? My first version simply left the volume out, and that's not enough. The postgres image declares its own volume, so Docker quietly creates one anyway. --exit-code-from stops the containers but doesn't remove them, so the next up starts the same container with the same data. A tmpfs mount is wiped every time the container stops, so the reset happens only for tests and never touches your dev data. Easy to check: insert a row in one test run, run the exact same command again without down, and the row should be gone.

One gotcha: a service without a profiles key starts with every profile. So give every service a profile, or you'll wonder why your test run also started something it shouldn't.

Env files

Each profile gets its own env file: .env.dev for dev and .env.test for test. The compose files stay the same; only the values change.

The variable names are identical in every environment. Dev and test both have PORT, JWT_SECRET, POSTGRES_URL and so on, just with different values:

Variable	.env.dev	.env.test
`PORT`	3000	3001
`JWT_SECRET`	dev_secret	test_secret
`POSTGRES_PORT`	54321	54322
`POSTGRES_DB`	development	test

So the structure of your config and secrets never changes, and your app never has to ask "which environment am I in?". It reads process.env and gets on with life.

A new environment is a new env file plus a small edit. Want staging? Create .env.staging, add a staging_app and staging_postgres that extend the base services, and give them a staging profile. That's it.

--env-file and env_file are two different things. They look almost the same but do different jobs:

--env-file .env.dev on the command line fills in the ${...} placeholders inside the compose files, like ports and passwords.
env_file: .env.dev inside a service passes the variables into the container, so the app can read them from process.env.

In this setup the app uses the same file for both, so everything about an environment lives in one place.

The same port inside and outside. A line like ${POSTGRES_PORT}:${POSTGRES_PORT} maps the host port to the same port inside the container. For that to work, the service has to listen on that port too, which is what PGPORT does for Postgres and the MH_*_BIND_ADDR variables do for MailHog. The result: one number per service, and dev and test never point at the same database.

Keep the real env files out of git. They hold secrets, and your future self will thank you.

The command key

command overrides what a container runs on start, and it's what lets one image do different jobs.

Both dev_app and test_app use the plain node image with the source code mounted in. No Dockerfile, no custom image to rebuild. The only difference is the command:

dev_app runs npm ci && npm run start and keeps the server running.
test_app runs npm ci && npm test, then exits.

Two small things matter here:

Use &&, not ;. With npm ci; npm test, the tests still run after a failed install and fail with a confusing error. && stops at the first failure.
--exit-code-from test_app stops the containers when the tests finish (it doesn't remove them) and returns the tests' exit code. Your terminal (or CI) gets a clear pass or fail.

Also, depends_on only controls the start order. It doesn't wait until Postgres is actually ready to accept connections. In practice npm ci is slow enough that Postgres is ready long before it finishes (finally, a use for slow installs). For a bulletproof setup, add a Postgres healthcheck and use condition: service_healthy.

Try it yourself

Take any Node.js project with start and test scripts in package.json.
Add the four files from The full setup, and add .env.dev and .env.test to .gitignore.
Start development:

   docker compose --env-file .env.dev --profile dev up

Your app is on http://localhost:3000, and every email it sends shows up at http://localhost:8025.

Run the tests:

   docker compose --env-file .env.test --profile test up --exit-code-from test_app

Clean up when you're done:

   docker compose --env-file .env.dev --profile dev --profile test down

A full working example lives in my repo: kot-1999/flowers_shop_be.

Copy-paste is still allowed. Just not for your infrastructure. 😉

How do you handle multiple environments with Docker Compose? Let me know in the comments.

Product Maintainability - design principles and practices

Oleksandr Kashytskyi — Sat, 24 May 2025 11:38:37 +0000

Introduction
Main 3 Design Principles
- Operability
- Simplicity
- Evolvability
Additional Maintainability Practices
- Code Readability
- Testing & Automation
- Decoupling Components
- Continuous Refactoring
- Monitoring & Logging
Conclusion

Introduction

Most of the total cost of software development and ownership is tied not to the initial build but to its maintenance. In fact, multiple industry studies, including those by IEEE and Gartner, indicate that 60–80% of a software system’s total lifecycle cost is spent on maintaining and evolving it. These costs encompass fixing bugs, enhancing functionality, adapting to new requirements, updating dependencies, and ensuring security compliance.

A well-maintained software system is easier to operate, understand, and extend over time. This not only boosts productivity but also improves developer morale and reduces turnover. Software maintainability is a key metric in software quality, and ensuring it requires a deliberate and thoughtful approach throughout the development lifecycle—from design to deployment.

So let's check what can increase maintainability of our product!!!

Main 3 Design Principles

Operability

Operability refers to how well the system supports day-to-day operations such as deployment, monitoring, and troubleshooting. A highly operable system:

Integrates with monitoring tools (e.g., Prometheus, Datadog, Sentry)
Has built-in health checks and metrics
Supports automated recovery and graceful failure handling
Enables fast incident diagnosis and resolution

Improving operability leads to lower Mean Time to Recovery (MTTR) and better system uptime, both of which are crucial for business continuity.

Simplicity

Simplicity aims to eliminate unnecessary complexity. Systems that are simpler:

Have clear responsibilities and minimal side effects
Are easier to test, reason about, and modify
Encourage consistent coding styles and patterns

As per software engineering research, complexity is the primary factor behind bugs and delayed development. Tools like static analyzers, linters, and code review checklists can help enforce simplicity at scale.

The best tool for removing complexity is abstraction. By encapsulating intricate logic behind well-defined interfaces, abstraction helps reduce cognitive load, prevent errors, and improve reusability.

Evolvability

Evolvability ensures the system is ready to change and grow with new requirements. Key practices include:

Modular design and domain-driven boundaries
Clear, stable APIs with versioning
Backward compatibility and migration support

NOTE: Except of backward compatibility there is forward compatibility, but it's implementation require very high skills and knowledge of the product.

Additional Maintainability Practices

Code Readability

Readable code is easier to debug and extend. It involves:

Clear naming conventions
Consistent formatting (e.g., via Prettier or Black)
Logical structure and separation of concerns
Inline comments and documentation for complex logic

Surveys show that developers spend over 70% of their time understanding existing code. Readable code is not a luxury—it's a necessity for long-term maintainability.

Testing & Automation

Automated tests form the backbone of a reliable system. Key strategies include:

Unit tests to validate business logic
Integration tests to catch cross-module bugs
End-to-end tests for user-facing flows
CI/CD pipelines for fast, reliable delivery

According to Capers Jones, defect rates drop by 60–90% in systems with strong test coverage and automation.

Decoupling Components

Reducing interdependencies makes systems easier to change. Strategies include:

Applying microservice, service-oriented, or hexagonal architecture
Using message queues or APIs for communication
Defining clear interfaces and contracts

Organizations that adopt modular and decoupled architectures report higher deployment frequencies and lower change failure rates, as seen in the State of DevOps reports.

Continuous Refactoring

Refactoring is essential to maintain code health. Benefits include:

Reduction of technical debt
Improved performance and maintainability
Easier onboarding of new developers

Unchecked technical debt can reduce development speed by 15–20% annually, compounding into major delays over time. Scheduled refactoring sprints or “engineering health” time allocations are vital.

Monitoring & Logging

Visibility into system behavior is critical for proactive maintenance. Good observability includes:

Structured logging with correlation IDs
Metrics collection and dashboards
Distributed tracing
Real-time alerts and anomaly detection

Teams that invest in observability tools report a 30%+ improvement in reliability and resolution times, according to research by Honeycomb and Google SRE practices.

Conclusions

Building maintainable software is not a one-time effort but a continuous commitment. By adhering to foundational design principles — operability, simplicity, and evolvability — and reinforcing them with proven best practices like code readability, testing, decoupling, refactoring, and monitoring, teams can ensure that their systems remain robust, adaptable, and cost-effective.

Ultimately, maintainability is a force multiplier — it enhances productivity, reduces risk, and positions software to evolve in harmony with business needs. Prioritizing it from day one is one of the most impactful investments in long-term software success.

Scalability in Data-Intensive applications - Fan-Out, Throughput, Twitter problem, Percentile

Oleksandr Kashytskyi — Tue, 25 Feb 2025 10:18:43 +0000

Introduction
Identifying Bottlenecks
The Twitter Problem
Measuring Response Time Effectively
Conclusion

Introduction

As applications grow, they need to handle more users, more data, and more requests efficiently. Scalability is a term used to describe a system's ability to cope with an increasing load. But how do we ensure that a system scales well? Let's explore some key concepts.

Identifying Bottlenecks

To scale a system effectively, it's essential to analyze its load parameters. Different systems have different constraints, and finding bottlenecks helps in optimizing performance. Here are several key factors to consider:

Fan-out: It's a term which describes the number of requests a service or endpoint makes to other services in order to serve a single incoming call. A high fan-out can lead to increased latency and system overload.
Throughput: In batch processing systems like Hadoop, the focus is on records processed per second rather than individual response times.
Response Time Distribution: Measuring response time is not just about average values but understanding the distribution of values.

The Twitter Problem

A classic example of scalability challenges is Twitter's timeline system. There are two endpoints, one to create a new post and another to fetch newest 20 posts.

A naive approach would be to query the database every time a user requests their home timeline. This results in expensive read operations and high latency.

Instead, Twitter solves this problem by maintaining a cache for each user's homepage. As cache memory is very expensive, there can't be stored all data needed for response, but it's possible to easily store IDs of last 20 posts in the timeline. This approach increases write complexity (It will be necessary to maintain both posts in a database and cache storage), but significantly improves GET request performance.

Measuring Response Time Effectively

Response time can vary significantly depending on system load. One of the best ways to analyze it is through percentiles, rather than averages.

99.9th percentile is often used to track performance (Some companies, like AWS AWS for example use 99.99th percentile). The reasoning behind this is that the top 0.1% of users are usually the most valuable customers, often transferring the most data or making the most critical requests.

Example: Sentry Monitoring: In observability tools like Sentry, response time percentiles help identify slowest transactions affecting real users, allowing engineers to optimize performance accordingly.

Conclusion

Scaling a system is not just about handling more traffic but ensuring efficiency, and optimal resource allocation.

Scalability is essential for handling growing demands in data-intensive applications. Identifying bottlenecks like fan-out, throughput, and response time distribution helps optimize performance. Finally, using percentiles instead of averages ensures a more accurate measure of system performance, helping engineers focus on critical optimizations.

🚀 Ever wondered why some systems never fail while others crash at the worst moments?

Oleksandr Kashytskyi — Sun, 16 Feb 2025 16:23:38 +0000

Oleksandr Kashytskyi

Feb 16 '25

Reliability in Data-Intensive Applications

#bigdata #software #computing #fault

3 min read

[Boost]

Oleksandr Kashytskyi — Sun, 16 Feb 2025 16:21:53 +0000

Oleksandr Kashytskyi

Feb 16 '25

Reliability in Data-Intensive Applications

#bigdata #software #computing #fault

3 min read

Reliability in Data-Intensive Applications

Oleksandr Kashytskyi — Sun, 16 Feb 2025 16:21:12 +0000

Introduction
What is Reliability?
Types of Faults in Data-Intensive Systems
- Hardware Faults
- Software Errors
- Human Errors
Visualizing Reliability in Systems
- Fault Isolation
- Observability Framework
Conclusion

Introduction

Data-intensive applications differ from compute-intensive ones by relying heavily on data storage, processing, and retrieval rather than raw computational power. These applications are typically built from standard building blocks, such as databases, caches, messaging systems, and distributed storage.

Beyond databases, maintaining a data-intensive system requires a suite of other tools to ensure reliability, performance, and fault tolerance.

What is Reliability?

A system is considered reliable if it:

Performs its intended function correctly as expected by the user.
Can tolerate user mistakes without severe failures.
Maintains good enough performance for the required use case.
Prevents unauthorized access to sensitive data.

Reliability is closely related to fault tolerance — the system’s ability to continue functioning despite faults.

Fault ≠ Failure

A fault occurs when a component stops working (e.g., a database node crashes).
A failure happens when the system as a whole can no longer function correctly.

Types of Faults in Data-Intensive Systems

Hardware Faults

Hardware failures include disk crashes, memory corruption, and power outages.

Modern distributed systems can tolerate hardware faults through redundancy and failover mechanisms (RAID for storage, replication for databases e.t.c.).

Software Errors

Software errors are trickier to handle than hardware faults. They can be caused by:

Crashes due to bad input or unhandled edge cases.
A runaway process consuming all system resources.
Failures in external services that the system depends on.
Cascading failures, where a small failure triggers larger system-wide outages.

To mitigate software errors:

Implement robust error handling and graceful degradation.
Use circuit breakers and retry mechanisms.
Employ canary releases and feature flags to minimize blast radius.

Human Errors

Studies show that only 10-25% of outages are due to server or network faults, meaning human errors are a major contributor to system failures.

Strategies to reduce human-induced faults:

Design for resilience – Make critical operations harder to break.
Decouple risky operations – Separate places where humans interact most.
Thorough testing – Include unit, integration, and system-level tests.
Quick and easy recovery – Provide rollback mechanisms and automated recovery.
Detailed monitoring and alerting – Detect anomalies early.
Training and process improvement – Foster good management practices and continuous learning.

Visualizing Reliability in Systems

Fault Isolation

A well-architected system uses fault isolation to prevent one failing component from bringing down the entire system.

Load balancer ensures traffic is distributed evenly.
Circuit breakers prevent overload from failed services.
Caching layers reduce direct dependencies on databases.

Observability Framework

A good monitoring and alerting system is essential:

Logs, metrics, and tracing should be unified for quick debugging.
Real-time dashboards help detect anomalies.
Automated alerts ensure rapid response to incidents.

Conclusion

Reliability is a key aspect of data-intensive applications. Achieving it requires:

Understanding and mitigating different types of faults (hardware, software, and human errors).
Designing systems with resilience in mind (e.g., fault isolation, circuit breakers, failover strategies).
Implementing strong observability tools (Sentry, AWS CloudWatch) to detect and resolve issues quickly.

By following these principles, data-intensive applications can achieve high availability, fault tolerance, and consistent performance, ensuring a smooth experience for users.

DEV Community: Oleksandr Kashytskyi

Ever seen a docker-compose.test.yml that's just a full copy of docker-compose.yml? I wrote a short guide on fixing that with one shared base file, extends and profiles, so a new environment is a few lines instead of a copy-paste. Full code inside, ready to

Efficent usage of Docker

Efficent usage of Docker

Contents

The full setup

How to run it

The shared base file

Profiles

Env files

The command key

Try it yourself

Product Maintainability - design principles and practices

Table Of Contents

Introduction

Main 3 Design Principles

Operability

Simplicity

Evolvability

Additional Maintainability Practices

Code Readability

Testing & Automation

Decoupling Components

Continuous Refactoring

Monitoring & Logging

Conclusions

Scalability in Data-Intensive applications - Fan-Out, Throughput, Twitter problem, Percentile

Table Of Contents

Introduction

Identifying Bottlenecks

The Twitter Problem

Measuring Response Time Effectively

Conclusion

🚀 Ever wondered why some systems never fail while others crash at the worst moments?

Reliability in Data-Intensive Applications

[Boost]

Reliability in Data-Intensive Applications

Reliability in Data-Intensive Applications

Table Of Contents

Introduction

What is Reliability?

Types of Faults in Data-Intensive Systems

Hardware Faults

Software Errors

Human Errors

Visualizing Reliability in Systems

Fault Isolation

Observability Framework

Conclusion