DEV Community: Javi Palacios

Why We Need Databases

Javi Palacios — Sat, 24 Oct 2026 22:41:39 +0000

You start a small project. Very small. So small that saving the data in a file feels perfectly reasonable.

A users.json here. An orders.csv there. Maybe a shared spreadsheet because “business can see it too.” Everything is calm. Everything is under control. Until someone asks: “can we see last month's orders grouped by customer, excluding cancelled ones, sorted by revenue?”

And suddenly your humble file becomes a crime scene investigation: manual filters, copies named final_v2_good_really_this_time.xlsx, formulas nobody wants to touch, and the very specific feeling a former coworker once summarized perfectly: “it works because God is good.”

That's where databases begin.

Not because they're more elegant. Not because they sound more professional. Because at some point, storing data is not enough. You need to query it, relate it, protect it, modify it safely, and let multiple people or processes work with it at the same time without turning your application into a social experiment.

The problem: storing data is not managing data

Storing data is easy.

[
  {
    "id": 1,
    "name": "Ada",
    "email": "ada@example.com"
  }
]

This works. For learning, for a small script, for a demo, for that personal project that starts on a Sunday afternoon and ends up living in production for three years because “we'll clean it up later.” Spoiler: they did not clean it up.

The problem doesn't show up when you have three users. It shows up when you have thousands. Or when two processes write at the same time. Or when you need to know which orders belong to which user. Or when someone deletes a line by mistake. Or when a query that used to be instant now takes long enough for you to reconsider your career choices.

A flat file is not designed to answer complex questions. It's designed to store bytes. That's important, but it's only the first step.

The difference between “I have data” and “I have a database” is the difference between having all the screws for a shelf in a plastic bag and having the shelf assembled, stable, and without that one leftover metal piece staring at you from the table like it knows something you don't.

What breaks when we use files for everything

Files are not bad. In fact, many databases store information in files internally. The difference is that a database adds a layer of organization, rules, queries, and guarantees on top.

When you use files directly, sooner or later these problems appear:

1. Querying data becomes painful

With a small file, finding something is trivial. Open the file, read line by line, find what you need.

Now imagine a CSV with one million orders:

order_id,user_id,total,status,created_at
1,42,99.90,paid,2026-07-01
2,57,19.50,cancelled,2026-07-02
3,42,149.00,paid,2026-07-03

Now you want to answer:

How much has each user spent?
What are the top 10 best-selling products?
Which orders are paid but not shipped?
How much did revenue grow compared to last month?

You can do it with scripts, yes. You can also assemble a shelf with a spoon if you have enough patience. Possible is not the same as reasonable.

SQL exists to ask questions of your data declaratively:

SELECT user_id, SUM(total) AS total_spent
FROM orders
WHERE status = 'paid'
GROUP BY user_id
ORDER BY total_spent DESC;

Don't worry about understanding every word yet. The important idea is this: instead of writing a program that walks through the file step by step, you tell the database what result you want. The engine decides how to get it.

That difference changes everything.

2. Relationships become manual

Real data almost never lives alone.

A user has orders. An order has line items. A line item points to a product. A product belongs to categories. A review belongs to a user and a book. Everything starts connecting to everything else, because the real world has the rude habit of not organizing itself into one pretty list.

With files, you manage those relationships yourself:

{
  "order_id": 1,
  "user_id": 42,
  "product_ids": [7, 12, 19]
}

Does user 42 exist? Do products 7, 12, and 19 exist? What happens if product 12 is deleted? Who prevents user_id from pointing to someone who no longer exists?

Exactly: you. Your code. Your discipline. Your teammate on Friday at 6:03 PM. Good luck.

A relational database lets you express those rules as part of the model itself:

CREATE TABLE orders (
  id INTEGER PRIMARY KEY,
  user_id INTEGER REFERENCES users(id)
);

Again, you don't need to master the syntax now. Keep the concept: the database can know that orders.user_id must point to a real user. That's not just storage. That's integrity.

And when data matters, integrity is not a fancy extra. It's the seatbelt.

3. Multiple writers at once is a dangerous party

Imagine two processes updating the same file at the same time.

One reads the file. The other reads it too. One writes changes. The other writes later with an older version. Result: lost data. The kind of bug that doesn't scream, doesn't explode, and doesn't leave a stack trace. It just eats information and walks away like nothing happened.

That's the worst kind of bug: the one that doesn't look like a bug until you do the math.

Databases are designed to handle concurrency: multiple clients reading and writing at the same time under clear rules. That's why concepts like transactions, locks, isolation levels, and ACID exist. Today we'll only name them; later we'll open them carefully, because there is enough material there to make a spreadsheet cry.

The basic idea is simple: a database doesn't just store data. It coordinates changes.

4. Security and guarantees disappear quickly

With files, you usually have operating system permissions: who can read or write the file. That's useful, but it doesn't go far enough.

In a real application, you might want rules like:

The backend can insert orders, but not delete users.
An analyst can read sales, but not emails.
A user can only modify their own reviews.
Nobody should be able to create an order with a negative total.

A database lets you model permissions, constraints, roles, validations, and access rules in a much finer-grained way. It doesn't replace application security, but it acts as your last line of defense.

And yes: you want that last line. Because trusting all application code to be perfect forever is a beautiful strategy, just like trusting that “this time I will definitely remember to make a backup before touching production.”

So what is a database?

A database is an organized system for storing, querying, modifying, and protecting persistent data.

The key word is not “storing.” A file already does that. The key is the full set:

Persistence: data survives when the application shuts down.
Structure: data follows an understandable model.
Querying: you can ask complex questions without writing the whole algorithm by hand.
Relationships: you can connect entities without relying on human memory and optimistic comments.
Integrity: you can define rules to avoid invalid states.
Concurrency: multiple processes can work at the same time without stepping on each other like they're editing the same Word document in 2007.
Security: you can control who accesses what.
Performance: you can search through millions of rows without reading everything every time.

Put all of that together and you no longer have “a place to put things.” You have a central piece of your system.

And here's an important point: a database does not magically fix bad design. If you model your data badly, the database will obey. With admirable professionalism and terrible consequences.

Types of databases from a distance

This course focuses on SQL and PostgreSQL, but it's worth knowing that not all databases use the same model.

Type	How it thinks about data	Examples	Where it appears
Relational	Tables, rows, columns, and relationships	PostgreSQL, MySQL, SQL Server	Business applications, structured data, strong integrity
Document	JSON/BSON documents	MongoDB, CouchDB	Flexible data, catalogs, semi-structured content
Key-value	Key → value	Redis, Valkey	Cache, sessions, counters, simple queues
Graph	Nodes and relationships	Neo4j	Complex relationships: social networks, recommendations, fraud
Time-series	Data indexed by time	TimescaleDB, InfluxDB	Metrics, sensors, events, observability

Don't stress about memorizing this now. The goal is to give you a map, not send you out explaining distributed databases on a napkin like you're in a backend interview at 9 AM.

The important idea: each type of database exists to solve different problems. But if you're learning from zero, starting with the relational model is the best investment. Many ideas about data, performance, querying, and integrity are easier to understand from there.

Why PostgreSQL

We'll use PostgreSQL 18 throughout the course.

Not because it's the only option. MySQL, MariaDB, SQL Server, Oracle, and SQLite exist and have real use cases. But PostgreSQL is an excellent database to learn with and to work with seriously because it combines several things that are hard to find together:

It's open source and widely used in the industry.
It implements SQL in a powerful and mostly standard way.
It has advanced types: JSONB, arrays, ranges, UUIDs, full-text search.
It supports constraints, transactions, views, materialized views, roles, extensions, and serious tooling.
It scales from small projects to large systems.
It lets you learn general SQL while also using modern features that appear in production.

PostgreSQL is a good school because it doesn't treat you like you only want to store four rows. It gives you grown-up database tools from the beginning. Some of them won't show up until much later, but it's good to know they're there, patiently waiting for you to stop writing SELECT * out of habit.

What learning SQL really means

Learning SQL is not memorizing commands.

Memorizing commands creates that dangerous feeling of “I know SQL” until a query appears with LEFT JOIN, GROUP BY, HAVING, a subquery, and a NULL staring from the corner. Then the castle starts wobbling.

Really learning SQL means understanding questions like these:

What shape does my data have?
What relationship exists between these tables?
Which rows am I filtering before grouping?
Why is this query reading ten million rows?
What index does this access pattern need?
What happens if two users modify the same thing at the same time?
What is my ORM actually generating?

That's why this course starts here, before installing anything. Because if you understand the problem, the tool stops feeling like magic. And when the tool stops feeling like magic, you start using it with judgment.

What you take from this lesson

If you understood this, you now have the mental foundation for the course:

A file can store data, but it doesn't manage querying, relationships, concurrency, and integrity well.
A database adds structure, rules, guarantees, and query capabilities.
SQL lets you ask for the result you want, not manually describe how to walk through the data step by step.
Relational databases remain fundamental because they model structured data and relationships very well.
PostgreSQL will be our main tool because it's modern, open, powerful, and widely used in production.

In the next lesson we'll set up the environment: we'll run PostgreSQL 18 with Docker, connect with psql and pgAdmin, and leave a database ready for real data. If Docker still looks at you funny, Mastering Docker from Scratch is the perfect warm-up; we'll use it here without turning this lesson into another container-moving day.

Never stop coding!

Docker Compose networking: networks, services, and isolation

Javi Palacios — Sat, 24 Oct 2026 07:46:27 +0000

There is a very Docker Compose moment: Postgres is running, the API is running, the logs look reasonable, and the API still says it can't connect to localhost:5432.

You check docker compose ps. Postgres is Up. You check the docker-compose.yml. The port is there. You stare at the screen with that specific mix of suspicion and tiredness that only appears when a tool is technically telling the truth in the least helpful way possible.

Is Postgres down? No. Is the port wrong? Maybe, but probably not. Is localhost a trap with good branding? Exactly.

In Docker Compose, each container lives in its own little network world. Once you understand that, the rest stops feeling like dark magic with YAML.

Compose's default network

When you run docker compose up, Compose automatically creates a network for the project. If your directory is called myapp, that network is usually named something like myapp_default.

All services in the docker-compose.yml join that network by default.

services:
  api:
    image: myapp/api

  db:
    image: postgres:16-alpine

Even though you didn't write networks: anywhere, Compose did some work for you:

docker compose up -d
docker network ls

You'll see a network like this:

NETWORK ID     NAME            DRIVER    SCOPE
abc123def456   myapp_default   bridge    local

Inside that network, services can find each other by service name. The API can connect to Postgres using db as the hostname.

postgresql://myapp:secret@db:5432/myapp

Don't use the container IP. Container IPs can change when services are recreated. Using container IPs in Compose is like writing down where you parked three weeks ago on a napkin: technically useful for a few minutes.

Use service names.

`localhost` inside a container is not your machine

This is the main trap.

From your machine, localhost means your machine.

From inside a container, localhost means that container.

services:
  api:
    image: myapp/api
    environment:
      DATABASE_URL: postgresql://myapp:secret@localhost:5432/myapp

  db:
    image: postgres:16-alpine

That is wrong for container-to-container communication. The API will look for Postgres inside the API container itself. Spoiler: it is not there. If it were, you'd have another problem, probably with a suspicious architecture smell.

The correct version uses the service name:

services:
  api:
    image: myapp/api
    environment:
      DATABASE_URL: postgresql://myapp:secret@db:5432/myapp

  db:
    image: postgres:16-alpine

Mental model:

Connecting from	Correct host
Your browser to the API	`localhost` if the port is published
API to Postgres	`db`
API to Redis	`redis`
One container to another	The service name

Don't stress if this feels counterintuitive at first. localhost has spent years looking innocent while ruining entire afternoons with a smile.

`ports` is not how containers talk to each other

Another classic confusion: thinking you need to publish a port so another container can connect to it.

You don't.

services:
  api:
    build: ./api
    ports:
      - "3000:3000"
    environment:
      DATABASE_URL: postgresql://myapp:secret@db:5432/myapp

  db:
    image: postgres:16-alpine

The ports: "3000:3000" line publishes the API port to your machine. It lets you open http://localhost:3000 from your browser.

But the API does not need Postgres to have ports: "5432:5432" to connect to db:5432. Both services are already on the internal Compose network.

This matters for security. If you publish Postgres like this:

services:
  db:
    image: postgres:16-alpine
    ports:
      - "5432:5432"

You're exposing the database to the host. In development that can be useful if you want to connect with TablePlus, DBeaver, local psql, or another external tool. But don't do it “just in case”. “Just in case” in infrastructure is a polite way of saying “I'll leave this door open and see what happens”.

If only the API needs to talk to the database, don't publish the port.

Custom networks: separate rooms

The default network is fine for small projects. All services can see each other and life goes on.

But in a more realistic application, you might have:

web: frontend or reverse proxy.
api: backend.
db: database.
redis: cache.

There is no reason for web to connect directly to db. The frontend talks to the API. The API talks to the database. The database should not be standing in the lobby waving at the entire building.

You can model that with custom networks:

services:
  web:
    image: nginx:alpine
    ports:
      - "8080:80"
    networks:
      - frontend

  api:
    image: myapp/api
    networks:
      - frontend
      - backend

  db:
    image: postgres:16-alpine
    networks:
      - backend

networks:
  frontend:
  backend:

Now the rules are clear:

Service	Networks	Can talk to
`web`	`frontend`	`api`
`api`	`frontend`, `backend`	`web`, `db`
`db`	`backend`	`api`

web and db do not share a network, so they cannot resolve each other by name or connect directly. This does not replace authentication, firewalls, permissions, or common sense, but it reduces surface area. And reducing surface area is one of those boring things that prevents interesting incidents. Interesting incidents are overrated.

`expose` vs `ports`

Compose also has expose:

services:
  api:
    image: myapp/api
    expose:
      - "3000"

expose documents that a service listens on a port for other services on the network, but it does not publish that port to your machine.

In practice, services on the same network can connect to a container port even if you don't write expose. That's why many teams don't use it. It can be useful as documentation inside the docker-compose.yml, but don't confuse it with security.

The important difference:

Field	Publishes to host	Main use
`ports`	Yes	Access from your machine or outside
`expose`	No	Document internal ports

If you want to open something from the browser, use ports. If only containers talk to each other, you usually don't need anything.

External networks

Sometimes you want several Compose projects to share a network. For example, a common reverse proxy (traefik, nginx, caddy) routing multiple local apps or services on the same server.

First create the network manually:

docker network create shared-proxy

Then declare it as external:

services:
  app:
    image: myapp/api
    networks:
      - shared-proxy

networks:
  shared-proxy:
    external: true

external: true means: “Compose, don't create this network; it already exists, use it.”

If the network doesn't exist, Compose fails. And that's good. Better to fail than silently create a new network and leave you wondering for half an hour why the proxy can't see your app. Docker already has enough material to confuse you without extra help.

Network aliases

By default, each service resolves by its name: api, db, redis.

Sometimes you want additional names. That's what aliases are for.

services:
  api:
    image: myapp/api
    networks:
      backend:
        aliases:
          - api-service
          - internal-api

  worker:
    image: myapp/worker
    networks:
      - backend

networks:
  backend:

From worker, these hostnames point to the api service:

api
api-service
internal-api

Use this carefully. An alias can help when a legacy application expects a specific hostname. But if you start adding three names per service “for convenience”, you'll end up with a tiny home-made DNS system inside YAML. Like having five nicknames for the same person in a group chat: funny until someone new tries to understand the conversation.

Complete example

A reasonable stack with reverse proxy, API, database, and cache:

services:
  web:
    image: nginx:alpine
    ports:
      - "8080:80"
    networks:
      - frontend

  api:
    build: ./api
    environment:
      DATABASE_URL: postgresql://myapp:secret@db:5432/myapp
      REDIS_URL: redis://redis:6379
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
    networks:
      - frontend
      - backend

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: myapp
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: myapp
    volumes:
      - db-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U myapp -d myapp"]
      interval: 5s
      timeout: 3s
      retries: 5
    networks:
      - backend

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 3
    networks:
      - backend

volumes:
  db-data:

networks:
  frontend:
  backend:

Notice three details:

Only web publishes a port to the host.
api talks to db and redis using service names.
db and redis are not on frontend, so web cannot talk to them directly.

This does not turn your app into Fort Knox. But you're no longer putting every service in the same room with a sticky note that says “be nice”.

Practical challenge

Take the docker-compose.yml from the previous tutorial and split it into two networks: frontend and backend. The API should be on both. Postgres and Redis should only be on backend. If you have a web or proxy service, put it only on frontend and publish only its port.

Then verify that your internal URLs use service names (db, redis, api) and not localhost. If you find a localhost inside an environment variable used between containers, don't look at it fondly: change it.

You now have the right mental map: Compose creates a default network, services discover each other by name, ports is for reaching your machine, and custom networks let you isolate what should not talk directly. In the next tutorial we'll close the Compose module with best practices: environment-specific files, secrets, resource limits, logging, and the small details that separate a useful docker-compose.yml from one that looks written during a power outage.

Never stop coding!

Docker Compose volumes and data persistence

Javi Palacios — Fri, 23 Oct 2026 05:38:35 +0000

Docker has a very effective teaching method: you start Postgres, insert data, stop the container, start it again, and everything is still there. Nice. You gain confidence. You clean things up. You run the wrong command. You open the app again and the database is emptier than a sprint retrospective where everyone says “all good”.

Where did the data live? Inside the container? Somewhere in Docker? In a secret folder guarded by a small daemon with trust issues?

Take a breath. Docker doesn't hate your data. Containers are disposable by design. If you want persistence, you need to declare it explicitly. That's what volumes are for.

The problem: containers are not hard drives

A container has its own filesystem. You can write files, modify data, create directories, and feel like everything is under control. The catch is that this writable layer belongs to the container.

Remove the container, and those changes go with it.

docker run --name temp-postgres -e POSTGRES_PASSWORD=secret -d postgres:16-alpine
docker rm -f temp-postgres

The container is gone. Its writable changes are gone too.

That's not a bug. That's the model. Containers should be safe to destroy and recreate. The image contains the application; the data lives outside.

Think of a container as a moving van. Useful, necessary, maybe smelling vaguely of cheap air freshener. But you don't store your lease, family photos, and the only copy of your production database inside the van. Those go in labeled boxes.

In Docker, those boxes are volumes.

The three volume types you'll actually use

In Compose, a service can mount several kinds of storage:

services:
  app:
    image: myapp
    volumes:
      - app-data:/app/data
      - ./src:/app/src
      - /app/temp
      - ./config:/app/config:ro

volumes:
  app-data:

Those four lines are not the same thing. Treating them as interchangeable is how you end up staring at docker volume ls like you're reading ancient ruins.

Named volumes

services:
  app:
    image: myapp
    volumes:
      - app-data:/app/data

volumes:
  app-data:

A named volume is storage managed by Docker. You give it a name (app-data), and Docker decides where it physically lives on the host.

This is the default choice for data you want to keep: databases, uploaded files, persistent caches, internal application state.

The upside: it works consistently across Linux, macOS, and Windows. You don't depend on a specific host path.

The downside: the files are not sitting neatly inside your project directory. Docker stores them internally. If you're like me when I started, that creates a very specific anxiety: “I know my data exists, but I don't know where it lives.” Correct. Welcome to the club.

You can inspect volumes with:

docker volume ls
docker volume inspect myproject_app-data

Compose usually prefixes volume names with the project name. If your directory is called myproject and your volume is app-data, the real volume might be myproject_app-data. Docker isn't trying to be mysterious. It just has a gift for making simple things look slightly like infrastructure archaeology.

Bind mounts

services:
  app:
    image: myapp
    volumes:
      - ./src:/app/src

A bind mount mounts a host directory into the container. Here, ./src on your machine appears as /app/src inside the container.

This is excellent for development: edit code in your editor, the container sees the change immediately, the dev server reloads, and for one beautiful second software feels like it was designed by someone on your side.

services:
  api:
    build: ./api
    volumes:
      - ./api/src:/app/src
    command: npm run dev

But bind mounts depend on the host filesystem. If ./api/src exists on your laptop but not on the server, deployment fails. If file synchronization is slow on macOS, you'll feel it. If someone changes the path, the container does not develop intuition.

The practical rule:

Development: bind mounts for source code and local configuration.
Persistent data: named volumes.
Production: avoid mounting source code from the host unless you have a VERY clear reason.

Not “interesting”. Clear.

Anonymous volumes

services:
  app:
    image: myapp
    volumes:
      - /app/temp

An anonymous volume has no explicit name. Docker creates one automatically for /app/temp.

This can be useful for internal container paths, but for important data it's a trap with good lighting. The volume exists, yes, but you didn't choose a stable name for it. Weeks later, you'll see a list of long IDs and have to guess which one contains what.

It's like having a drawer full of unlabelled cables. One of them is definitely important. Good luck finding the right charger before USB invents another connector.

For data you want to keep and understand, use named volumes.

Read-only volumes

services:
  app:
    image: myapp
    volumes:
      - ./config:/app/config:ro

The :ro suffix mounts the volume as read-only. The container can read /app/config, but it can't write there.

Use this for configuration, certificates, static files, or anything the container should not modify. If an application only needs to read a file, don't give it write access. Least privilege applies locally too. Your laptop is not a moral exception.

Database persistence

The most common Compose case is a database.

services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: myapp
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: myapp
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  db-data:

Postgres stores its data in /var/lib/postgresql/data. By mounting db-data there, the data lives in the volume, not in the container's writable layer.

Now you can remove and recreate the container without losing the database:

docker compose down
docker compose up -d

The container changes. The volume remains.

But pay attention: down and down -v are not polite cousins. docker compose down removes containers and networks, but keeps named volumes. docker compose down -v also removes the volumes declared by Compose.

docker compose down     # Keeps named volumes
docker compose down -v  # Removes named volumes too

That -v is tiny, quiet, and destructive. Like a red button without a safety cover. Use it when you deliberately want to delete data: resetting a local database, cleaning a test environment, starting from scratch.

Don't use it as “general cleanup” if the data matters.

For other databases, the idea is the same: mount the volume where the image stores its data.

services:
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_PASSWORD: secret
    volumes:
      - postgres-data:/var/lib/postgresql/data

  mongo:
    image: mongo:7
    volumes:
      - mongo-data:/data/db

  mysql:
    image: mysql:8
    environment:
      MYSQL_ROOT_PASSWORD: secret
    volumes:
      - mysql-data:/var/lib/mysql

volumes:
  postgres-data:
  mongo-data:
  mysql-data:

The path changes depending on the image. The pattern doesn't.

Initialization scripts

Many official database images can run initialization scripts the first time the database is created.

With Postgres, anything mounted into /docker-entrypoint-initdb.d runs when the data directory is initialized:

services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: myapp
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: myapp
    volumes:
      - db-data:/var/lib/postgresql/data
      - ./init-scripts:/docker-entrypoint-initdb.d:ro

volumes:
  db-data:

For example:

-- init-scripts/01-create-tables.sql
CREATE TABLE IF NOT EXISTS notes (
  id SERIAL PRIMARY KEY,
  title TEXT NOT NULL,
  created_at TIMESTAMP NOT NULL DEFAULT NOW()
);

Important detail: these scripts run when the data directory is empty. If the volume already exists and Postgres has already initialized, they do not run again.

Feeling the “but I changed the SQL and nothing happened” moment? Good. The volume already had data. Docker is not going to wipe your database because you edited a file. That would be helpful in a deeply criminal way.

To test initialization changes locally:

docker compose down -v
docker compose up -d

That deletes the volume and forces a fresh initialization. Deliberate, visible, consequences included.

Development vs production

Volumes also separate two very different needs: fast development and stable runtime behavior.

In development, you want hot reload:

# docker-compose.dev.yml
services:
  api:
    build:
      context: ./api
      target: development
    volumes:
      - ./api/src:/app/src
      - ./api/package.json:/app/package.json:ro
      - /app/node_modules
    command: npm run dev

The - /app/node_modules line creates an anonymous volume so the container's node_modules directory doesn't get overwritten by the host. This trick exists because JavaScript decided dependencies should have more emotional mass than the actual project. We're not fixing that today.

In production, you don't mount source code from your machine. You build an image and only mount the data that must persist:

# docker-compose.prod.yml
services:
  api:
    build:
      context: ./api
      target: production
    volumes:
      - api-data:/app/data
    command: npm start

volumes:
  api-data:

Then combine files depending on the environment:

# Development
docker compose -f docker-compose.yml -f docker-compose.dev.yml up

# Production
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

The base docker-compose.yml describes the shared stack, and the environment-specific files override what changes. Just like with Git, you commit what describes the project and keep machine-specific details out.

Backups: persistence is not a backup

A persistent volume is not a backup. Say it with me: persistence is not backup.

A named volume survives container recreation. It does not protect you from docker compose down -v, manual deletion, data corruption, an overconfident script, or you on a Friday afternoon thinking “this definitely won't break anything”.

For databases, the best backup is usually the database's own tool. For Postgres:

docker compose exec db pg_dump -U myapp myapp > backup.sql

That runs pg_dump inside the db container and writes the output to your machine. Clear, portable, and database-aware.

You can also make a low-level copy of a volume using a temporary container:

docker run --rm \
  -v myproject_db-data:/data:ro \
  -v "$(pwd)":/backup \
  alpine \
  tar czf /backup/db-data-backup.tar.gz -C /data .

What's happening here:

myproject_db-data:/data:ro mounts the volume read-only.
"$(pwd)":/backup mounts your current directory as the backup destination.
tar czf archives the volume contents.
--rm removes the temporary container when it exits.

If you've spent years fighting shell quotes, you noticed "$(pwd)". If not, don't worry: Bash will get its own course later, because this topic deserves more than “copy this and trust me”. That's not decoration. It keeps the command from breaking when the path contains spaces. The shell is always waiting for a chance to remind you that quotes are not optional.

To restore into an empty volume:

docker volume create myproject_db-data

docker run --rm \
  -v myproject_db-data:/data \
  -v "$(pwd)":/backup:ro \
  alpine \
  tar xzf /backup/db-data-backup.tar.gz -C /data

For databases that are actively running, don't blindly copy files while the engine is writing. It might help in development. In production, use pg_dump, consistent snapshots, or the official tooling for your database. Faith can move mountains, but it does not guarantee transactional integrity.

Cleanup without destroying what matters

These Compose and Docker commands have very different effects:

docker compose stop
docker compose down
docker compose down -v
docker volume ls
docker volume rm myproject_db-data
docker volume prune

Human translation:

Command	What it does
`docker compose stop`	Stops containers, removes nothing
`docker compose down`	Removes containers and networks, keeps named volumes
`docker compose down -v`	Removes containers, networks, and project volumes
`docker volume rm <name>`	Removes one specific volume
`docker volume prune`	Removes volumes not used by any container

docker volume prune asks for confirmation, but it does not know which volume was “emotionally important”. It only knows whether a container is currently using it. If you have an old volume with data you care about and no container is attached to it right now, it can be removed.

Before deleting, inspect:

docker volume inspect myproject_db-data

And if you're unsure, back it up first. Boring advice. So is wearing a seatbelt, until the day it stops being boring.

Practical challenge

Create a docker-compose.yml with Postgres and a fake API or any lightweight image you know. Add a named volume for Postgres, start the stack, create some data, run docker compose down, then start it again and confirm the data is still there.

After that, back up the volume using the temporary alpine container. You don't need an enterprise backup system yet; just understand the flow: volume → .tar.gz archive → restore into an empty volume.

You can now treat Docker Compose data with the respect it deserves: outside the container, with clear names, with clean separation between development and production, and with backups that don't depend on crossed fingers. In the next tutorial we'll go into Docker Compose networking: default networks, custom networks, service isolation, and why your database should not be cheerfully waving at the entire stack.

Never stop coding!

Setting up your DevOps work environment: Linux, terminal, and essential tools

Javi Palacios — Thu, 22 Oct 2026 06:29:04 +0000

The first real collision with DevOps usually isn't Kubernetes. Or Terraform. Or that YAML file that looks like it was written by someone with strong feelings against indentation.

The first collision is usually less heroic: you try to prepare your environment, open a terminal, see three different ways to “use Linux,” someone mentions SSH keys, someone else says “just use WSL,” another person says “spin up a cloud instance,” and suddenly installing a few tools feels like choosing a mortgage.

Should you use local Linux? A VM? WSL? AWS? GCP? Will your laptop explode? No. Unless you run too many browser tabs while VirtualBox is trying to breathe. But conceptually, no.

This lesson sets up a serious working environment for the course: real enough to learn DevOps properly, simple enough that you don't spend three days configuring the workshop before touching a single tool.

What a DevOps learning environment actually needs

A DevOps environment does not need to look like production from minute one. That would be like learning to drive by entering a six-lane roundabout with a truck behind you. Educational, sure. Also unnecessarily traumatic.

What it does need is a place where you can practice four things:

using a Linux terminal comfortably;
installing command-line tools;
running services, scripts, and processes;
connecting to remote machines through SSH.

That last point matters. DevOps lives a lot on servers. Even when we later work with containers, Kubernetes, or cloud providers, the base is still the same: a shell, permissions, processes, logs, networking, and files.

If you don't have a strong terminal foundation yet, that's fine. This course starts exactly there. And if Git still feels like a black box full of branches, Mastering Git from Scratch will save you several uncomfortable conversations with yourself.

First the judgment; then the options.

The recommended option: a local Ubuntu VM

To follow this course consistently, the recommended option is a local VM running Ubuntu 24.04 LTS.

Not because Ubuntu is magically better, but because it gives us a shared baseline: real Linux, systemd, systemctl, journalctl, networking, permissions, packages, and a machine you can break without touching your main system. Think of it as a small practice server at home. If you destroy it, delete it and create another one. Much better than learning electricity by sticking a screwdriver into the main panel.

We'll use Multipass, a Canonical tool for creating Ubuntu VMs from the CLI. The picture looks like this:

your machine ── multipass ──> Ubuntu VM devops-lab

Before continuing: Multipass is recommended because it lets us create an Ubuntu VM in a simple and reproducible way. But it is not a religious requirement.

If you work on a corporate machine, have security restrictions, already use a virtualization tool approved by your company, or your computer needs a special setup, use whatever method fits your environment: VirtualBox, VMware, UTM, Hyper-V, Parallels, libvirt, or anything else you are allowed to use.

The tool is not the goal. The goal is to end up with an Ubuntu 24.04 LTS VM —or an equivalent Linux distribution— that you can access, where you can install packages, run commands, manage services, and practice without touching your main system.

DevOps is about understanding systems, not marrying the first installer mentioned in a tutorial.

Installing Multipass

Install Multipass according to your system:

# macOS and Linux (Homebrew)
brew install --cask multipass

# Ubuntu / Debian
sudo snap install multipass

# Arch Linux (AUR)
paru -S canonical-multipass

Arch users already know the drill: if multipassd won't start, install dnsmasq and add ReadWritePaths=/usr/local/etc/multipassd to the service. If QEMU can't find OVMF.fd, symlink it from edk2-ovmf to /usr/share/qemu/.

On Windows, install it from PowerShell:

winget install -e --id Canonical.Multipass

If virtualization is disabled in your BIOS/UEFI, your company policies are strict, or Hyper-V is doing Hyper-V things, Multipass may complain. Don't stress: DevOps does not hate you personally. It's virtualization. Sometimes it feels like opening a door; sometimes it feels like getting approval from five departments to move a chair.

Once installed, check that it responds:

multipass version

Now we can create the VM.

Creating the course VM

The course VM will be called devops-lab:

multipass launch 24.04 --name devops-lab --cpus 2 --memory 4G --disk 30G

That command creates an Ubuntu 24.04 machine with:

2 CPUs;
4 GB of RAM;
30 GB of disk;
the name devops-lab.

If your computer is low on resources, you can reduce it to --memory 2G, but 4 GB will be more comfortable for what we'll do later. A tiny VM works, sure, like a chair with one loose leg works. The question is how much trust you want to place in it.

To list your VMs:

multipass list

To enter the VM:

multipass shell devops-lab

Multipass already knows how to enter the machine. You don't need to configure SSH manually for everyday course work. It's a convenient door managed by the tool.

To exit:

exit

To stop the VM:

multipass stop devops-lab

To start it again:

multipass start devops-lab

And if you break everything with educational enthusiasm:

multipass delete devops-lab
multipass purge

Breaking a lab VM is not a tragedy. It's almost part of the process. Like burning the first pancake: you don't serve it, but it teaches you how hot the pan is.

Accepted alternatives

Multipass will be the baseline for examples, but it is not the only valid way to follow the course.

Option	When to use it	Main trade-off
Multipass	Recommended course baseline	Depends on local virtualization
WSL2	You use Windows and Multipass does not fit your setup	Very practical, with a few Windows moments
Local Linux	You already use Linux and know what you're touching	You can break your real system
Cloud	You want real public infrastructure	More realistic, but billing enters chat

WSL2 on Windows

If you work on Windows and Multipass gives you trouble, WSL2 is a reasonable alternative. Microsoft documents the basic installation as a single command: wsl --install, which installs WSL and, by default, Ubuntu.

From PowerShell as administrator:

wsl --install

Restart if Windows asks for it. Windows asking for a restart: the ritual that keeps the ecosystem humble.

WSL2 works for many parts of the course, but it is not exactly a traditional VM. Filesystem, networking, and services can behave differently. Great to start; not always identical to a server.

Local Linux

If you already use Linux, perfect. You're home. You probably also have an opinion about your distro, terminal, and package manager. Arch users already know the drill.

You can follow the course on your real system, but at the beginning it helps to separate learning from your daily machine. You don't learn carpentry by removing your front door first.

Cloud later

Cloud will show up later when we actually need it: IAM, security groups, cloud networking, Terraform against real infrastructure, managed services, and other pieces with consequences.

Can free tier work? Yes, if you respect limits and conditions. Am I going to promise that the whole course will be free on cloud? No. That would be like saying “don't worry, this YAML will definitely work the first time.” Technically possible; emotionally irresponsible.

Entering with Multipass and practicing SSH

For course work, you'll usually enter like this:

multipass shell devops-lab

You can also run commands without opening a full shell:

multipass exec devops-lab -- lsb_release -a

But SSH is still fundamental in DevOps. On real servers, cloud instances, and many lab environments, SSH will be your front door.

So let's practice it against our local VM. First generate a key if you don't already have one:

ssh-keygen -t ed25519 -C "devops-course"

Accept the default path (~/.ssh/id_ed25519) unless you have a reason to change it. The public part (id_ed25519.pub) goes to the remote machine. The private part (id_ed25519) is not shared. Ever. Think house keys: you can give the lock a copy of your public identity, but you don't post your private key in a team chat. Obvious, until it happens. And it happens.

Copy the public key into the VM:

multipass transfer ~/.ssh/id_ed25519.pub devops-lab:/home/ubuntu/id_ed25519.pub

Enter the VM with Multipass:

multipass shell devops-lab

Inside the VM, add the key to authorized_keys:

mkdir -p ~/.ssh
cat ~/id_ed25519.pub >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
rm ~/id_ed25519.pub
exit

Now check the VM IP:

multipass info devops-lab

Look for the IPv4 line and connect through SSH:

ssh ubuntu@<vm-ip>

If you need a specific key:

ssh -i ~/.ssh/id_ed25519 ubuntu@<vm-ip>

If this does not work on your system, that's fine: some drivers, firewalls, or networking setups may block direct access by IP. We'll use multipass shell to follow the course; SSH is extra practice here, and we'll revisit it with cloud later.

The important idea is this: Multipass gives you the convenient door; SSH is the door you'll use on real servers. You should know both.

Now that you have a stable way into a Linux machine, you need the base tools.

Essential tools

We'll install the minimum reasonable toolkit for the first lessons:

git, for versioning files and scripts;
curl and wget, for making requests and downloading resources;
vim, because sooner or later you'll edit something in a terminal;
htop, for reading processes without suffering through top;
jq, for working with JSON;
net-tools or modern equivalents, for basic networking commands.

If Vim traps you and you can't exit, it's not a bug, it's an initiation ceremony. Mastering Vim from Scratch starts exactly from that kind of existential moment.

If you're following the recommended path, enter the VM first with multipass shell devops-lab and use the Ubuntu/Debian commands. If you're using another alternative, adapt the installation to your system:

# macOS and Linux (Homebrew)
brew install git curl wget vim htop jq

# Ubuntu / Debian
sudo apt update
sudo apt install -y git curl wget vim htop jq net-tools

# Arch Linux
sudo pacman -S git curl wget vim htop jq net-tools

Then verify everything responds:

git --version
curl --version
vim --version
jq --version

You don't need to memorize every tool now. git gives us traceability; curl lets us talk to services; jq lets us read JSON without crying; htop shows processes with less of a 1998 accounting spreadsheet vibe.

With tools ready, let's make the terminal more comfortable.

Configuring a comfortable terminal

You could do the entire course with the default shell, no plugins, and no colors. You could also write all your code in a tiny-font plain text editor. Possible is not the same as wise.

For now, bash is fine. In fact, it's what you'll find on most servers you connect to. But this is your work terminal, your personal environment — you can afford some luxuries here. If you want a nicer experience, install zsh and, optionally, Oh My Zsh. If you want the slower walkthrough, there is already a dedicated tutorial on how to install ZSH and Oh My ZSH! in GNU/Linux, which fits this Ubuntu VM especially well.

# macOS and Linux (Homebrew)
brew install zsh

# Ubuntu / Debian
sudo apt install -y zsh

# Arch Linux
sudo pacman -S zsh

Oh My Zsh is installed through a remote script. That's common, but it is still a remote script. The responsible version: read it before running it. The realistic version: at least run it from the official source and not from random-devops-blog-2020.example, please.

sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"

After installing, change your default shell to zsh. If you are using the recommended Multipass VM, the ubuntu user has no password, so use sudo:

sudo chsh -s $(which zsh) ubuntu

On other systems, chsh may ask for your password without sudo. Both achieve the same result.

Now add a few useful aliases to the end of ~/.zshrc or ~/.bashrc, depending on your shell:

# DevOps aliases
alias ll='ls -lah'
alias ports='sudo ss -tulpn'
alias logs='sudo journalctl -f'
alias c='clear'

Reload the configuration:

source ~/.zshrc

Or if you use Bash:

source ~/.bashrc

Don't fill your shell with 200 aliases on day one. That looks like productivity, but it's often decoration with side effects. Start with four or five you actually use. Your terminal should help you, not become an airplane cockpit where you don't know which button turns off the engine.

Creating the course workspace

Let's leave a clean structure for the next exercises:

mkdir -p ~/devops-course/{scripts,configs,logs,projects,notes}
cd ~/devops-course

touch notes/lesson-02.md
printf 'DevOps course workspace ready\n' > notes/lesson-02.md

Check the result (using the ll alias you just created):

ll

You should see something like this:

drwxr-xr-x  7 user user 4.0K Jul  6 10:00 .
drwxr-x--- 20 user user 4.0K Jul  6 10:00 ..
drwxr-xr-x  2 user user 4.0K Jul  6 10:00 configs
drwxr-xr-x  2 user user 4.0K Jul  6 10:00 logs
drwxr-xr-x  2 user user 4.0K Jul  6 10:00 notes
drwxr-xr-x  2 user user 4.0K Jul  6 10:00 projects
drwxr-xr-x  2 user user 4.0K Jul  6 10:00 scripts

Now initialize a Git repository to version your notes and scripts:

git init
git status

On branch main

No commits yet

Untracked files:
  notes/

Make the first commit. First, tell Git who you are (if this is your first time using Git on this machine):

git config user.email "you@email.com"
git config user.name "Your Name"

Add --global if you want this configuration applied to all your repositories.

Now for the actual commit:

git add notes/lesson-02.md
git commit -m "chore: initialize devops course workspace"

Yes, even for a learning workspace. Especially for a learning workspace. If you break something, Git gives you a way back without doing archaeology in your own directory.

Final check

Before closing, run this small checklist:

pwd
git --version
ssh -V
curl --version
jq --version

And confirm mentally:

I have a usable Linux terminal;
I have a course directory;
I can install tools;
Git works;
I know how to connect through SSH, or at least I understand what I need to do it.

If any of this fails, don't continue as if nothing happened. That's one of the big DevOps lessons: a half-broken environment today becomes a lost afternoon tomorrow. Like a chair with one loose leg: you can sit on it, yes, but part of your brain is already drafting the incident report.

Your environment is no longer “an open terminal where things happen.” You now have a minimal workshop: a shell, base tools, a working structure, Git, and a clear idea of how to enter a remote machine.

In the next lesson we'll go down into the Linux filesystem: directories, files, permissions, and basic commands. That's where /etc, /var, /home, and friends stop looking like names chosen by someone in a hurry.

Never stop coding!

Introduction to the DevOps world: culture, automation, and feedback

Javi Palacios — Wed, 21 Oct 2026 06:50:40 +0000

It's 5:43 PM on a Thursday. Development has just finished a feature. Operations needs to deploy it. The feature works locally, tests are green, the ticket is Done, and someone is already thinking about closing the laptop with dignity.

Then production shows up and says: “adorable, but no.”

The server has a different Node version. The environment variable is missing. The deploy script depends on a manual step known by exactly one person. Logs live on one machine, configuration on another, and the document explaining the process is called deploy-final-v2-good-now-for-real.md.

Whose fault is it? Dev? Ops? The VPN? That README nobody has updated since 2021? If you're just starting out, this may sound exaggerated. If you've worked in software before, you probably just remembered a specific face. Sorry about that.

Welcome to the DevOps world.

The problem before DevOps

For a long time, software delivery was organized around a very clear separation:

Development wrote the code.
Operations managed servers, networks, deployments, and production.

On paper, that sounds reasonable. Like separating kitchen and service in a restaurant: one team prepares the food, another serves it. The problem starts when the kitchen doesn't know how the dish is served, service doesn't know what's in the dish, and the customer is staring at lasagna that's still frozen in the middle.

In software, that separation produced familiar symptoms:

“It works on my machine.”
“Ops handles deployments, ask them.”
“We can't deploy until Friday because there's a change window.”
“Where are the logs?”
“Who touched production?”

“It works on my machine” is practically cultural heritage in this industry. It solves nothing technically, but it does buy three seconds of awkward silence. Like assembling IKEA furniture, finding one spare metal part, and deciding it was probably decorative. The shelf is standing. For now.

The real problem wasn't that Dev was irresponsible or Ops was slow. That caricature is convenient, but wrong. The problem was the system of work: separated teams, different goals, little automation, late feedback, and deployments treated as special events instead of a normal part of development.

That's where DevOps comes in.

What is DevOps?

DevOps is a culture and a set of practices for delivering software faster, more safely, and more reliably through collaboration, automation, and continuous feedback.

The important sentence is this: DevOps is not a tool.

It is not Jenkins. It is not Docker. It is not Kubernetes. It is not Terraform. It is not having a 400-line YAML file and hoping that if you look at it respectfully enough, it won't fail.

Tools help, of course. But DevOps starts earlier: it starts when development and operations stop working like two departments throwing packages over a fence and start sharing responsibility for the whole software lifecycle.

That full lifecycle looks roughly like this:

Plan → Code → Build → Test → Release → Deploy → Operate → Monitor

DevOps tries to make that cycle:

faster, because repetitive steps are automated;
safer, because manual changes and surprises are reduced;
more observable, because you know what is happening when something breaks;
more collaborative, because the team shares context and responsibility.

The uncomfortable bit: DevOps cannot be “installed.” It has to be practiced. If a company says “we bought DevOps,” what it probably bought is a tool, consulting, or a laptop sticker. Real DevOps changes how work happens.

And to change how work happens, you need three core ideas.

The three core ideas: collaboration, automation, and feedback

We could build a giant list of principles, but for now keep three. They are the tripod. If one leg fails, the camera falls and the picture comes out blurry. Humble metaphor, accurate result.

Collaboration

Dev and Ops should not meet for the first time on deployment day.

Collaboration means that people writing the code understand how it runs in production, and people operating the system understand what is changing in the product. Not so everyone does everything all the time — that ends in calendar-shaped chaos — but so there are no absurd walls.

A developer doesn't need to become a BGP networking expert to deploy an API. But they should understand what variables it needs, what ports it exposes, what logs it emits, and how to tell if it is healthy.

An operations person doesn't need to know every detail of the business domain. But they should be involved in decisions that affect deployment, capacity, observability, and recovery.

DevOps removes the “I already did my part” escape hatch. In real systems, that sentence is a trap. Your part does not end when you run git push; it ends when the software delivers value in a real environment.

Automation

Every repeated manual step is a future incident updating its résumé.

Building by hand, copying files over SSH, restarting services “in the correct order,” editing configuration directly in production... all of that can work once. You can also cross a road with your eyes closed once. That doesn't make it a methodology.

Automation turns fragile rituals into repeatable steps:

Code → pipeline → tests → build → deploy → monitoring

Instead of depending on someone remembering the exact ceremony, the process lives in scripts, pipelines, repositories, and infrastructure as code.

This is where tools we'll cover later enter the picture: Git, Jenkins, Docker, Kubernetes, Terraform, Prometheus, Grafana. But notice the mental order: problem first, tool second. Buying Kubernetes before understanding deployment is like buying an excavator because you want to hang a picture frame.

Feedback

Fast feedback prevents errors from arriving late, expensive, and dressed as production.

Automated tests, pipelines, alerts, logs, metrics, dashboards, postmortems: all of this exists to answer questions as early as possible:

Does the change work?
Is the system still healthy?
Did the deploy break something?
Are users suffering?
Can we recover quickly?

Without feedback, you work blind. And working blind in infrastructure has a technical name: Friday afternoon.

With collaboration, automation, and feedback, you can already see why DevOps is not “put someone in charge of Jenkins.” It is a way to reduce distance between writing software and operating it.

So, is DevOps a role?

Here is where the confusion gets fun. Fun in the sense of “some org charts look generated by a sleep-deprived AI.”

DevOps started as culture and practices, not a job title. Then the industry did what the industry does: it turned the idea into job postings.

Today you'll see titles like:

DevOps Engineer
Site Reliability Engineer
Platform Engineer
Cloud Engineer
Infrastructure Engineer
Build and Release Engineer

Are they the same? No. Do they overlap? A lot. Does it depend on the company? Absolutely. Does that help beginners? Not even slightly.

A practical way to frame it:

Focus	Main question
DevOps	How do we deliver software better and with less friction?
SRE	How do we make the system reliable in a measurable way?
Platform Engineering	How do we build a platform so other teams deliver software better?

DevOps usually focuses on collaboration, automation, CI/CD, and infrastructure. SRE takes many of those ideas into operational reliability: SLOs, error budgets, incident response, toil, postmortems. Platform Engineering climbs another level: internal tools, Golden Paths, and self-service so teams don't reinvent deployment in every repository.

If DevOps reduces the wall between Dev and Ops, SRE puts numbers on reliability, and Platform Engineering builds the road teams drive on. If the road is well built, nobody thinks about it. If it isn't, everyone ends up in an infinite roundabout holding a Jira ticket.

What a traditional DevOps Engineer does

In many companies, the DevOps Engineer role combines several responsibilities:

Designing and maintaining CI/CD pipelines.
Automating deployments.
Managing infrastructure as code.
Maintaining servers, environments, and configuration.
Setting up basic monitoring.
Helping teams deploy more safely.

That means touching many moving parts. Some pleasant. Some YAML-shaped.

The usual skills include:

Linux/Unix, because most servers live there.
Terminal and scripting, especially Bash and sometimes Python.
Git, because without version control there is no traceability. If Git still feels like a black box with branches, Mastering Git from Scratch will save you real pain.
Containers, usually Docker. If you want to enter there before Kubernetes, Mastering Docker from Scratch is the natural path.
Cloud providers, like AWS or GCP.
Observability, so you can see what is happening when the system stops smiling.

The key is not knowing all of this on day one. No serious person expects that. The key is understanding the map: which piece solves which problem and how the pieces talk to each other.

Because if you don't have a map, every tool looks like a universal solution. That's how you end up trying to fix a DNS problem with Terraform. Don't ask how I know.

The DevOps ecosystem

A simple way to understand the ecosystem is to follow a change from the moment it is written to the moment it is observed in production:

Development → Build → Test → Deploy → Monitor

Git → Jenkins → Docker → Kubernetes → Prometheus
      GitHub    Registry   Helm         Grafana
      GitLab    Artifacts  ArgoCD       Alerts

Don't memorize all the names yet. Seriously, don't stress. The list looks like a hardware store where someone mixed screwdrivers, saws, cables, and a welding machine in the same aisle.

What matters is understanding the categories:

Version control: where code and change history live.
CI/CD: how build, tests, and deploy are automated.
Packaging: how the application is packaged to run consistently across environments.
Orchestration: how multiple services and their lifecycle are managed.
Infrastructure as Code: how infrastructure is described in a versioned way.
Observability: how you see what is happening inside the system.

Each category answers a different question. That distinction saves you from a classic trap: learning tools without understanding the problem. That produces long résumés and fragile systems. Lots of technology, little judgment. Like buying a professional toolbox to tighten one screw and ending up disassembling the washing machine.

The course roadmap

This course is designed so the map appears gradually, not as a wall of acronyms on day one.

We'll move through three large phases.

Phase 1: Traditional DevOps

First we'll build fundamentals:

Linux and terminal.
Processes, services, logs, and basic networking.
Bash scripting.
Git applied to infrastructure.
CI/CD with Jenkins.
Docker and containers.
Kubernetes.
Terraform.

This phase answers the question: how do I automate and operate the software delivery lifecycle?

Here you'll get your hands dirty. Good. DevOps without touching a terminal is like learning to swim from a PowerPoint deck.

Phase 2: SRE

Then we'll move into Site Reliability Engineering:

Advanced observability.
SLOs and error budgets.
Incident management.
Blameless postmortems.
Chaos engineering.
Capacity planning.
Sustainable on-call.

This phase asks a different question: how do I make the system reliable without turning the team into a permanent fire brigade?

SRE introduces a fundamental idea: reliability with numbers. Not “we want it to work well,” but “this service must meet this SLO, and this is the acceptable error margin.” Less heroic intuition, more engineering.

Phase 3: Platform Engineering

Finally we reach Platform Engineering:

Golden Paths.
Developer Experience.
Self-service.
Backstage.
Service catalog.
Internal Developer Platforms.
Internal tooling.

This phase answers the mature question: how do I build a platform so other teams deliver software better without depending on me for every step?

That shift is huge. You move from putting out fires to designing systems that prevent fires from starting. Or, more realistically, systems that at least keep the fire in a controlled area, with alerts, a runbook, and an extinguisher that didn't expire in 2019.

What you should take from this first lesson

DevOps is not about tools. It is about reducing friction between building software and operating it in a real environment.

Tools matter, but they are the consequence. Git, Jenkins, Docker, Kubernetes, Terraform, Prometheus, or Backstage make sense when you understand what problem they solve. Without that context, they are just expensive names in an architecture nobody wants to touch.

Keep these ideas:

DevOps emerged to break silos between Development and Operations.
The foundation is collaboration, automation, and fast feedback.
DevOps is not a pure role, even though roles called DevOps Engineer exist.
SRE and Platform Engineering are related evolutions, not magic words.
The course will move from real fundamentals to modern internal platforms, without jumping straight into deep YAML.

💡 Challenge: Think of a real or imaginary project you know. Write a short note describing how a change is delivered today: who writes the code, how it is tested, how it is deployed, where logs are checked, and how the team knows it works. If at any point your answer is “I don't know” or “someone does it manually,” perfect: you just found a future lesson in this course.

In the next tutorial we'll prepare the working environment. We'll compare local virtual machines, cloud environments, and WSL2, then set up a Linux base so you can start using the terminal without relying on dark magic or “it was already configured on my laptop.”

Never stop coding!

Marks and jumps in Vim: navigate without losing the thread

Javi Palacios — Tue, 20 Oct 2026 12:31:21 +0000

You're editing a function at line 347. Before finishing, you need to check how another function works. You jump with /, get there, read it, realize you also need to see a constant you defined near the top. You head up there. When you try to get back to line 347 — you can't find it. You search for the function name, land close, move around a bit. You pick up the thread again. Now you need to check one more thing in another file.

This isn't editing anymore. This is sightseeing.

Vim has a full system for remembering where you've been: marks for manually saving positions, a jump list for the automatic history of your movements, and a changelist for the spots where you made edits. Once you work these into your flow, "where was I?" stops being a question.

Marks: leaving a signal before you go

A mark is a saved position. To create one, use m followed by a letter:

ma

That saves your current position under the key a. You can use any lowercase letter (a-z). Now move wherever you need to — another function, another section, another file. To return exactly where you were, use backtick followed by the letter:

`a

Cursor back to that exact line and column.

There's also the apostrophe version:

'a

The difference: `a (backtick) jumps to the exact position — line and column. 'a (apostrophe) jumps to the line, but to the first non-blank character. It's the difference between remembering "it was in the third drawer, under the old batteries, in the right corner" versus just "the third drawer." Both work. One is more precise.

Which one to use? Backtick. Almost always. Unless you specifically need the first character of the line, backtick puts you back exactly where you left the cursor.

Marks are movements too, which means they compose with Normal mode grammar:

d`a    " delete from cursor to mark a
y'b    " yank from cursor to the line of mark b

Vim gives you 26 local marks (a-z) and 26 global ones (A-Z) — fifty-two total. Exactly the number that makes you think you need a naming convention and maybe a spreadsheet to track them all. What you'll actually use is two or three: ma to mark where you jumped from, mb for something you need to check later, and ms for that other thing you've been putting off. Always the same two or three.

Global marks: across files

Uppercase letters (A-Z) are global marks: they work across files and persist between Vim sessions.

mA

That saves the current position under A. Tomorrow, after closing and reopening Vim, `A takes you right back there — opening the file if needed.

Useful when you're constantly jumping between two or three files: mA in the config file, mB in the routes file, mC in the main entry point. You jump between them without searching for anything.

Honest limitation: only one slot per letter. If you use mA in a different file, the previous A is gone. Fine with three files; not the right tool if you're trying to build a full project index with 26 active entries. For navigating many files, Vim has better tools you'll see later in the course.

Automatic marks: the ones Vim sets for you

Vim also saves certain positions automatically. You don't have to do anything — they're already there when you need them.

Mark	What it stores
`.	Position of the last change in the file
`^	Position where you left Insert mode last time
`[	Start of the last yanked or changed text
`]	End of the last yanked or changed text
`<	Start of the last visual selection
`>	End of the last visual selection

The most useful one day-to-day: `. jumps to the last place you made a change. No setup, no thinking. `. and you're there.

Then there's '' (two apostrophes), or (two backticks) for the exact-position version. Both store where you were before your last big jump — a search, gg, G, jumping to a mark. It's the retroactive "take me back to the start" command. You don't activate it in advance; Vim records it automatically on every large jump.

To see all active marks at any point:

:marks

It shows a table with name, line, column, and the text at that line. Useful when you've been working for a while and can't remember which mark points where — which is exactly when you need it.

The jump list: the editor's history

Beyond manual marks, Vim keeps an automatic position history called the jump list. Every large jump — searching with /, going to a mark, gg, G, jumping to a definition — gets recorded.

To navigate that history:

Ctrl+o    " go back (older jump)
Ctrl+i    " go forward (newer jump)

Ctrl+o is your browser's back button, but for positions inside the editor. If you've made four jumps from where you started, four Ctrl+o presses bring you back one step at a time. Ctrl+i goes the other way.

To see the full list:

:jumps

The jump list holds 100 positions. A hundred. Just enough to make it look like something you need to actively manage — you don't. When it fills up, the oldest entry quietly disappears. Vim doesn't send a notification. You keep editing.

A note about searches with / and ?: every jump to a result registers in the jump list. If you move through six matches with n, you have six entries. Ctrl+o undoes the trail.

The changelist: where you edited

The jump list tracks positions. The changelist tracks specifically the spots where you made a change in the file.

g;    " go to older change
g,    " go to newer change

To see the full list:

:changes

The common case: you edited something, jumped to another spot to check a variable, and now you want to get back exactly to where you were typing. g; and you're there.

The changelist stores the last 100 changes with exact positions — a detailed audit trail of everything you've touched in the file since you opened it. In practice, you'll use g; once, to get back to the last place you typed something. That's what it's there for.

Special jumps

`%`: the matching pair

Place the cursor on a parenthesis, brace, or bracket, and press %:

It jumps to the matching character. On ( → jumps to ). On } → jumps to the { that opens it. You can bounce back and forth with repeated %.

Useful in heavily nested code: instead of visually counting where a block ends, % takes you there directly.

`gd` and `gD`: go to definition

gd    " go to local definition (within the current scope)
gD    " go to global definition (from the top of the file)

gd finds the first occurrence of the word under the cursor within the local scope — useful for jumping to where a variable is declared. gD does the same but searches from the beginning of the file.

Not the same as LSP, which you'll see later in the course and which jumps with actual semantic context. gd and gD are text searches. In files without LSP, or for local variables with clear names, they work without any configuration.

`gf`: open the file under the cursor

The most underrated command in this lesson.

Place your cursor over a file path in the text:

import { config } from '../config/app'
require('./utils/format')

And press:

gf

Vim opens the file. No extra commands, no file explorer, no copying the path and pasting it into :e. If it exists, gf goes there. To come back, use Ctrl+o — the jump was already recorded in the jump list.

One caveat: gf works with the literal text under the cursor. Paths using bundler aliases (@/components) or implicit extensions might need extra configuration to resolve. On explicit paths, it works out of the box.

Key concepts from this lesson

m{a-z} creates a local mark; `{letter} jumps to the exact line and column; '{letter} jumps to the line
m{A-Z} creates a global mark that persists across sessions and works across files
`. jumps to the last change; '' jumps to the position before the last big jump
:marks lists all active marks
Ctrl+o / Ctrl+i move backward and forward through the jump list
g; / g, move through the changelist (edit history)
% jumps between matching open and close characters
gd / gD jump to the local or global first occurrence of the word under the cursor
gf opens the file whose path is under the cursor

💡 Challenge: Open a large file from a real project. Set ma where you are, jump to another section with /, come back with `a. Make a small edit, jump with gg, come back with g;. If there's any file path referenced in the code, place your cursor on it and try gf. Then run :marks and :jumps to see both lists — everything you just did is recorded there.

Marks and jumps close out Vim's core navigation system. You now have the tools to move through any codebase without losing the thread. In the next tutorial, we'll cover text objects — what many consider Vim's actual superpower: the ability to operate on "the content inside these quotes," "the block within these braces," or "this whole word" with a single two-key command.

Never stop coding!

Docker Compose file structure: health checks, environment variables, and restart

Javi Palacios — Mon, 19 Oct 2026 10:03:37 +0000

The docker-compose.yml from the last lesson had a subtle lie built into it. depends_on: db sounds like "wait for Postgres to be ready." What it actually means is "wait for the Postgres container to be running." These are not the same thing.

On your machine, Postgres initializes in under a second and you never notice the difference. On a CI runner with a slower disk, your API tries to connect to the database while Postgres is still writing its initial data directory. The connection fails, the container restarts, and you're left with an intermittent error you can't reproduce locally — the worst kind of bug to debug.

The previous tutorial mentioned condition: service_healthy as the real solution and left it there. This lesson covers it in full, along with environment variables from files and restart policies. Three additions that take a docker-compose.yml from "works on my machine" to "works everywhere."

Every field a service can have

Before diving into health checks, here's a reference of what a service definition can include. The previous lesson used image, ports, environment, depends_on, and volumes. There are more:

services:
  api:
    # Image or build (not both)
    image: myapp/api:latest
    build:
      context: ./api # Directory with the Dockerfile
      dockerfile: Dockerfile # Dockerfile name (default: Dockerfile)
      args:
        NODE_ENV: production # Build-time arguments

    # Optional: explicit container name (auto-generated if omitted)
    container_name: myapp-api

    # Port mapping: host:container
    ports:
      - "3000:3000"

    # Environment variables inline
    environment:
      NODE_ENV: production
      DATABASE_URL: postgresql://db:5432/myapp

    # Environment variables from a file
    env_file:
      - .env

    # Dependencies
    depends_on:
      - db

    # Volumes
    volumes:
      - ./api/src:/app/src # Bind mount (development hot reload)
      - api-data:/app/data # Named volume

    # Networks this service connects to
    networks:
      - backend
      - frontend

    # Health check
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 3s
      retries: 3
      start_period: 40s

    # Restart policy
    restart: unless-stopped

Not every service needs all of these. Most use five or six fields. This is the full reference — the most relevant ones get detailed treatment in this and the following lessons.

One note on build: the short form (build: ./api) assumes there's a Dockerfile in that directory. The long form with context, dockerfile, and args is for when you need build arguments or a differently named Dockerfile.

Health checks: waiting for the service, not just the container

A health check tells Docker how to verify that a service is actually ready — not just running. Docker runs that command on a schedule and marks the service healthy when it succeeds consistently.

For Postgres, the standard health check uses pg_isready, a utility that ships with the image and returns 0 when the database accepts connections:

services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: myapp
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: myapp
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U myapp -d myapp"]
      interval: 5s # How often Docker runs the check
      timeout: 3s # Max time allowed per check
      retries: 5 # Fail after this many consecutive failures
      start_period: 10s # Grace period before failures count against retries

start_period is worth understanding: during that window, health check failures don't count toward the retries limit. This gives Postgres time to finish initializing before Docker starts caring whether the check passes. Without it, a health check could fail during normal startup and prematurely mark the service as unhealthy.

With the health check defined, update depends_on to use it:

services:
  api:
    build: ./api
    depends_on:
      db:
        condition: service_healthy # Wait until db passes its health check

The syntax changes from a list to a map when you add a condition. condition: service_healthy holds the API back until Postgres's health check returns healthy. This is what depends_on: [db] implied it was doing.

Redis gets a similarly simple check:

redis:
  image: redis:7-alpine
  healthcheck:
    test: ["CMD", "redis-cli", "ping"]
    interval: 5s
    timeout: 3s
    retries: 3

And for an HTTP service you control:

api:
  build: ./api
  healthcheck:
    test: ["CMD-SHELL", "curl -sf http://localhost:3000/health || exit 1"]
    interval: 10s
    timeout: 5s
    retries: 3
    start_period: 15s

The || exit 1 ensures the check fails if curl can't connect, times out, or gets a non-2xx response. Without it, the shell returns 0 even when curl reports an error, and Docker happily marks the service healthy when it isn't.

Environment variables: the .env file

Three environment variables inline in a docker-compose.yml are fine. Twelve in staging is manageable. Forty-seven in production, where someone hardcoded a password two years ago and there's a JWT secret nobody remembers generating — that's a different situation.

The solution is to move variables out of the YAML.

Interpolation from .env

Compose automatically reads a .env file in the project root and makes its variables available for interpolation in the docker-compose.yml:

# .env
POSTGRES_USER=myapp
POSTGRES_PASSWORD=secret
POSTGRES_DB=myapp
API_PORT=3000

# docker-compose.yml
services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}

  api:
    build: ./api
    ports:
      - "${API_PORT}:3000"

The ${VARIABLE} syntax pulls values from .env or the shell environment. If a variable is missing, Compose substitutes an empty string without complaining. To make it fail loudly when a critical variable is absent:

POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required}

The .env file goes in .gitignore. Always. What you do version is a .env.example with variable names and placeholder values — so anyone cloning the repo knows what they need to configure.

env_file: passing a file directly to the container

env_file is an alternative that passes all variables from a file directly to the container, without declaring them one by one under environment:

services:
  api:
    build: ./api
    env_file:
      - .env
      - .env.local # Local overrides, not versioned

The difference from interpolation: env_file injects variables into the container's environment, but they're not available for substitution elsewhere in the YAML. If you need a variable in ports, image, or healthcheck, use interpolation (${VAR}). If you just need it inside the container and want a cleaner YAML, env_file is the right choice.

Restart policies

Without a restart policy, a failed container stays down. In development you'll notice immediately. On a server with no active monitoring, it might be down for hours.

The four options:

Policy	Behavior
`"no"`	Never restart automatically (default)
`always`	Restart automatically whenever it exits; after a manual `docker stop`, restart when the daemon restarts or it is started manually
`on-failure`	Restart only when the process exits with an error
`unless-stopped`	Like `always`, but respects manual `docker compose stop`

For most cases, unless-stopped is the right call: the service recovers if the process crashes, but you can stop it manually without it coming back on its own.

always has a behavior that surprises people the first time: if a container was running when you restarted the server, Docker starts it automatically when the Docker daemon starts. In production, that's usually what you want. On a laptop with ten projects, you'll find Docker starting six development stacks you forgot you had.

A complete stack

Everything together: Node.js API, Postgres, and Redis — with health checks, variables from .env, custom networks, and restart policies.

The .env file (add to .gitignore):

POSTGRES_USER=myapp
POSTGRES_PASSWORD=changeme
POSTGRES_DB=myapp
REDIS_PASSWORD=redispass
API_PORT=3000

The .env.example (commit this one):

POSTGRES_USER=
POSTGRES_PASSWORD=
POSTGRES_DB=
REDIS_PASSWORD=
API_PORT=3000

The docker-compose.yml:

services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    volumes:
      - db-data:/var/lib/postgresql/data
    networks:
      - backend
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
      interval: 5s
      timeout: 3s
      retries: 5
      start_period: 10s
    restart: unless-stopped

  redis:
    image: redis:7-alpine
    command: redis-server --requirepass ${REDIS_PASSWORD} --loglevel warning
    networks:
      - backend
    healthcheck:
      test: ["CMD-SHELL", "redis-cli -a ${REDIS_PASSWORD} --no-auth-warning ping"]
      interval: 5s
      timeout: 3s
      retries: 3
    restart: unless-stopped

  api:
    build: ./api
    ports:
      - "${API_PORT}:3000"
    environment:
      DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
      REDIS_URL: redis://:${REDIS_PASSWORD}@redis:6379
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
    networks:
      - backend
    restart: unless-stopped

volumes:
  db-data:

networks:
  backend:

With this file, docker compose up -d starts Postgres and Redis, waits for both to pass their health checks, then starts the API. If the API process crashes, unless-stopped restarts it. Credentials live in .env, outside the repository.

The networks field controls what can talk to what. All three services here share backend, so the API can resolve db and redis by name. If you added a frontend service, it would connect to a separate frontend network — able to reach the API, but not the database directly. Networking in Compose is its own lesson.

With health checks, externalized environment variables, and restart policies, a docker-compose.yml stops being a development convenience and starts behaving like real infrastructure. The next lesson goes deep on volumes: how data persists across container restarts, the difference between bind mounts and named volumes, and how to handle backups from Compose itself.

Never stop coding!

💡 Challenge: Take the docker-compose.yml from the previous lesson's challenge and add a Postgres health check using pg_isready. Update depends_on to use condition: service_healthy. Run docker compose up -d and watch docker compose ps as Postgres transitions from starting to healthy before the dependent services start.

Introduction to Docker Compose: one file, one command

Javi Palacios — Sun, 18 Oct 2026 11:58:37 +0000

There's a moment where managing Docker manually stops scaling. Not when you add the second container — that one you can keep in your head. It's when you add the third, and the startup order starts to matter, and the API tries to connect to the database before it's ready, and you have four terminal tabs open, and you're running the same four commands from memory in a slightly different order every time.

That moment is when Docker Compose starts making sense.

Instead of remembering which flags you passed to Postgres last week, you describe the entire stack in a single docker-compose.yml and run one command. The file lives in your repository. The next person who clones it runs docker compose up -d and gets the same environment you have. No terminal history, no sticky notes, no "I have the command somewhere".

What is Docker Compose?

Docker Compose is a tool for defining and running multi-container applications. You describe your services, their relationships, and their configuration in a YAML file. Compose handles creating the network, starting services in the right order, and managing their lifecycle together.

The approach is declarative: you describe what your stack looks like, not the step-by-step how of starting it.

Here's what running a database, a cache, and an API looked like before Compose:

docker network create myapp-network

docker run -d --name db --network myapp-network \
  -e POSTGRES_PASSWORD=secret postgres

docker run -d --name redis --network myapp-network redis

docker run -d --name api --network myapp-network \
  -p 3000:3000 -e DATABASE_URL=postgresql://db:5432 myapp-api

Four commands. In the right order. With the exact flags. On the right network. And that's three services — not the twelve you'll have in production.

With Compose:

# docker-compose.yml
services:
  db:
    image: postgres
    environment:
      POSTGRES_PASSWORD: secret

  redis:
    image: redis

  api:
    image: myapp-api
    ports:
      - "3000:3000"
    environment:
      DATABASE_URL: postgresql://db:5432
    depends_on:
      - db
      - redis

docker compose up -d

One file. One command.

Installation

Docker Compose v2 ships with Docker Desktop on macOS and Windows. On Linux, it depends on how you installed Docker.

Check if you already have it:

docker compose version

If that returns Docker Compose version 5.1.4, you're set. If it returns an error, or you have the old docker-compose binary (with a hyphen — the v1 Python tool), install the modern version:

# macOS and Linux (Homebrew)
brew install docker-compose

# Ubuntu / Debian
apt install docker-compose-plugin

# Arch Linux
pacman -S docker-compose

The modern command is docker compose (space, no hyphen). v1 was a separate Python binary, v2 is an official plugin integrated into the Docker CLI. If you installed Docker in 2023 or later, you already have v2.

Your first docker-compose.yml

The docker-compose.yml has three top-level sections:

services: # The containers that make up your application
volumes: # Named persistent storage
networks: # Custom networks (optional if the default is enough)

A realistic example — a Node.js API with a Postgres database:

services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: myapp
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: myapp
    volumes:
      - db-data:/var/lib/postgresql/data

  api:
    build: ./api # Build from the Dockerfile in ./api
    ports:
      - "3000:3000"
    environment:
      DATABASE_URL: postgresql://myapp:secret@db:5432/myapp
    depends_on:
      - db

volumes:
  db-data:

A few things worth noting:

image: postgres:16-alpine pulls a public image, same as docker run.
build: ./api builds the image from that directory's Dockerfile. No separate docker build step needed.
In DATABASE_URL, the host is db — the service name. Compose creates a default network where services resolve each other by name automatically.
db-data under volumes: declares a named volume that Docker manages. Postgres data persists across docker compose down. The volume stays until you explicitly remove it.

The file belongs at the root of your project, versioned alongside your code. If you're using Git, docker-compose.yml is just another configuration file — and it's exactly what eliminates the mental overhead of remembering how to start the project.

Essential commands

These cover the vast majority of what you'll actually need:

# Start all services in the background
docker compose up -d

# Start and rebuild images that have a Dockerfile
docker compose up -d --build

# Stop containers without removing them
docker compose stop

# Stop and remove containers + networks (volumes intact)
docker compose down

# Stop and remove everything, including named volumes
docker compose down -v

# Show running services and their status
docker compose ps

# Show logs from all services
docker compose logs

# Follow logs from a specific service
docker compose logs -f api

# Run a command inside a running service
docker compose exec api sh
docker compose exec db psql -U myapp

The distinction between stop and down: stop halts containers but keeps them, down removes them. Named volumes survive down — they only get removed if you pass -v. Worth knowing before you accidentally wipe a local database.

depends_on: startup ordering

Without depends_on, Compose starts all services roughly in parallel. That means your API can try to connect to Postgres while Postgres is still initializing — which, depending on how patient your connection retry logic is, will either work on the third attempt or fail immediately with a confusing error.

services:
  db:
    image: postgres:16-alpine

  api:
    build: ./api
    depends_on:
      - db # Wait for db container to be running before starting api

Here's the nuance worth understanding upfront: depends_on waits for the container to be running, not for the service to be ready. The Postgres container can be up and running while Postgres itself is still writing its initial data directory. From Docker's perspective, started. From Postgres's perspective, not yet accepting connections.

The proper solution — waiting until the service is genuinely healthy — uses health checks with condition: service_healthy. That's the topic for the next lesson, along with the full docker-compose.yml structure.

You now have everything needed to run a real development stack with Docker Compose: a declarative file, the commands to manage it, and the basic dependency model. The next lesson covers the complete docker-compose.yml structure — environment variables from files, health checks, restart policies, and how to split development and production configuration in the same project.

Never stop coding!

💡 Challenge: Write a docker-compose.yml with three services: Postgres, Redis, and any image you're familiar with. Start the stack with docker compose up -d, verify all three services show Up in docker compose ps, then follow the combined logs with docker compose logs -f. When you're done, bring everything down with docker compose down and confirm docker compose ps shows nothing running.

Git in CI/CD pipelines: from commit to automated deploy

Javi Palacios — Sat, 17 Oct 2026 08:53:18 +0000

The CI runner is the most disoriented machine on the team. It arrives at each run knowing nothing about you: no .gitconfig, no custom pager, no commit graph precomputed overnight by git maintenance. An empty container that clones the repository, runs what you tell it, and disappears. Like assembling an IKEA shelf from scratch every time you need to put a book on it — the box arrives complete, no memory of the last time, no leftover screw you set aside just in case. If you know what you're doing, you can pre-stage the pieces for the next build. That's what caching is.

This is the final lesson in the course. What we'll cover here is how Git fits inside an automated pipeline: what changes compared to local use, how to configure GitHub Actions and GitLab CI so that Git doesn't become the bottleneck, and which practices keep the integration between code and deployment free of surprises.

Git in a pipeline: what changes

On your machine, Git has context: your global config, your local history, packed objects, configured remotes. On a CI runner, none of that exists. Each run starts from zero and needs to:

Authenticate to clone the repository
Clone (or restore from cache)
Execute what it needs to execute
Optionally push or deploy something

Authentication is what differs most from local use. Locally you use SSH or HTTPS with saved credentials. In CI, platforms inject a temporary token — GITHUB_TOKEN in GitHub Actions, CI_JOB_TOKEN in GitLab — that has read permissions (and write if you configure it) only for that run. You don't manage credentials manually: the platform handles it.

The clone is where there's usually room for improvement.

Shallow clones: the first adjustment to make

By default, git clone downloads the full repository history. In a project with five years of commits, that can be hundreds of megabytes of objects that the runner will completely ignore — it'll run the tests, generate the build, and forget they ever existed.

The solution is a shallow clone:

# Only fetch the last commit
git clone --depth=1 <url>

In GitHub Actions, the actions/checkout action does this by default (fetch-depth: 1). In GitLab CI, you configure the GIT_DEPTH variable:

# GitHub Actions — default behavior (shallow)
- uses: actions/checkout@v4

# GitHub Actions — full history when you need it
- uses: actions/checkout@v4
  with:
    fetch-depth: 0

# GitLab CI — shallow clone
variables:
  GIT_DEPTH: "1"

# GitLab CI — full history
variables:
  GIT_DEPTH: "0"

⚠️ Shallow clones have a limitation you'll discover at exactly the wrong moment: if your build runs git log, git describe, or any comparison with older commits — for version number generation, changelog calculation, or comparing against a base commit in a PR — Git will tell you it doesn't have those commits. The fix is fetch-depth: 0. Now you know this before learning it in production.

Caching: pre-staging the pieces

A pipeline that takes four minutes and runs eighty times a day adds up to over five hours of distributed waiting across the team. Of those five hours, twenty minutes might be npm install downloading node_modules from scratch on every single run. With caching, that disappears.

CI platforms have built-in caching systems that persist directories between runs — not the container itself, but specific folders you save and restore. The cache key typically includes a hash of the dependency lockfile, so it automatically invalidates when you change a dependency.

GitHub Actions

name: CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/cache@v4
        with:
          path: ~/.npm
          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-node-

      - name: Install dependencies
        run: npm ci

      - name: Run tests
        run: npm test

The key is built from the hash of package-lock.json. If the lockfile hasn't changed, the cache is exactly what the previous build left behind and npm ci takes seconds instead of minutes. If it changed — new dependency, updated version — the cache invalidates and rebuilds.

GitLab CI

stages:
  - test
  - deploy

variables:
  GIT_DEPTH: "1"

test:
  stage: test
  image: node:20-slim
  cache:
    key: $CI_COMMIT_REF_SLUG
    paths:
      - node_modules/
  script:
    - npm ci
    - npm test

deploy:
  stage: deploy
  script:
    - ./deploy.sh
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

GitLab CI uses cache.key to identify the cache. $CI_COMMIT_REF_SLUG is the branch name in a filesystem-safe format — each branch gets its own cache, which prevents one branch from corrupting another's.

Automated tests on every commit

The whole point of connecting Git to a pipeline is this: every commit gets verified automatically before it reaches main. The standard flow:

Developer pushes to a feature branch
The pipeline clones, installs, and runs tests
If tests fail, the PR/MR can't be merged
If they pass, the code can be reviewed and merged with confidence

name: CI

on:
  push:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test
      - run: npm run lint

Branch protection in GitHub (Settings → Branches → Require status checks) makes the pipeline a mandatory condition for merging. Without green tests, the merge button is disabled. Simple in theory, career-saving in practice.

Automated deploy on merge

The next step is connecting the merge to main with deployment. The most common pattern:

name: Deploy

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy to production
        env:
          DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }}
        run: ./scripts/deploy.sh

The deploy only runs on pushes to main — not on PRs, not on feature branches. Secrets (DEPLOY_KEY, cloud credentials, access tokens) live in the platform's secrets configuration and get injected as environment variables at runtime. Never in the repository. Never in the code. If you ever see a hardcoded key in a deploy script, that repository has a problem that isn't a CI problem.

Best practices for Git in CI

Things most people learn the hard way:

Use fetch-depth: 0 only when you need it. Shallow clone is the right default. If a specific step needs full history, add fetch-depth: 0 to that job only, not everywhere.

git fetch --tags if your build depends on tags. Shallow clones don't fetch tags by default. If you use git describe or generate version numbers from tags, you need an explicit fetch:

- uses: actions/checkout@v4
  with:
    fetch-depth: 0
    fetch-tags: true

Minimum permissions for the token. GITHUB_TOKEN defaults to write permissions on the repository. If your workflow only reads, reduce them:

permissions:
  contents: read

Don't cache the .git directory. It might seem like a good idea to skip the clone entirely, but .git changes on every run and the cache is almost never valid. What makes sense to cache is your dependencies: node_modules, .pip, ~/.gradle.

The pipeline that works locally and fails in CI has a cause. Always. It might be an environment variable you assumed was present, a system dependency installed globally on your machine, or — if the runner is windows-latest — mixed line endings because core.autocrlf isn't consistently configured across the team (the exact setting from the Git configuration lesson). The runner doesn't lie: it runs exactly what you tell it. If it fails there and works locally, local has something implicit that isn't in the repository.

Key concepts from this lesson

In CI, each run starts from zero: no local configuration, no previous history, authentication injected by the platform.
fetch-depth: 1 (shallow clone) is the right default; use fetch-depth: 0 only when you need full history.
Cache dependencies by lockfile hash — not the .git directory.
Secrets go in the platform's secrets configuration, never in the repository.
Branch protection + required status checks = merges only happen when tests pass.
If the pipeline fails and local works, local has something implicit you haven't declared.

Forty lessons. From git init in an empty folder to signed commits, worktrees, automated hooks, LFS, submodules, and now pipelines that deploy code without anyone pressing a button. Git is the same tool it was in lesson one. What changed is the size of what you can do with it.

A closing note on who makes it here:

Arch Linux users probably had delta, rerere, and fsmonitor configured before this course mentioned them, installed lazygit from the AUR before we got to the productivity section, and have been running split panes in the terminal the entire time. They don't need the acknowledgment, but they've earned it.

Windows users: same ground covered, with the minor detail that somewhere in these forty lessons there was an unexpected \r\n in the history, Git Bash and WSL2 coexist on the same machine, and the question of which one to use is technically still open. Spoiler: WSL2.

And those who made it here with VS Code, the Source Control panel, and a mouse: Git is exactly the same underneath the buttons. You now know what happens when you click "Sync." That counts.

What comes next — if the curiosity doesn't stop — is the infrastructure that gives context to all of this. The same git push you've just learned to connect to a pipeline can end up updating a Kubernetes cluster, applying a Terraform plan, or notifying a monitoring system that a new version is live. That's exactly what the DevOps and Platform Engineering course — coming soon — is about: Linux, networking, CI with Jenkins, Docker, Kubernetes, Terraform, observability, and everything between the code and the server. Git is the mother tool. Now you know how to use it.

Never stop coding!

git switch and git restore: the end of the all-purpose checkout

Javi Palacios — Fri, 16 Oct 2026 13:59:32 +0000

git checkout is Git's Swiss Army knife: five different tools folded on top of each other, and when you need the scissors, you pull out the corkscrew. It switches branches, creates new branches, restores files, restores files to a specific commit's version, and drops you into a detached HEAD state if you give it a hash. All valid. All the same command. None of them particularly obvious about what the others do.

If you've ever felt that git checkout was confusing — that it made no sense for the same command to handle branch navigation and file recovery — you were right. It wasn't just you. It was the design. Enough so that in 2019, with Git 2.23, two new commands arrived to split it apart: git switch for branches, git restore for files. They've been stable for years. This tutorial is about understanding them and making the switch.

git switch: changing branches without ambiguity

git switch does one thing: move HEAD to another branch. That's it. The syntax is straightforward:

# Switch to an existing branch
git switch main

# Create a new branch and switch to it
git switch -c feature/new-functionality

# Create from a specific base
git switch -c feature/my-branch origin/develop

# Return to the previous branch
git switch -

If you're like me after years of muscle memory, git checkout -b is wired into your left hand and git switch -c will feel redundant for the first few weeks. That's normal. The difference isn't in what it does — it's in what it refuses to do: git switch can't restore files, can't go to a specific commit, can't enter detached HEAD unless you explicitly ask for it. It switches branches. That's what makes it predictable.

What happens if you try git switch abc123 to inspect a specific commit? Git doesn't guess: it gives you an error and tells you to use --detach if that's what you want. You have to be intentional:

# You have to mean this
git switch --detach abc123

With git checkout, ending up in a detached HEAD was too easy — you'd type a hash without thinking and suddenly be in a repository state that required explaining what HEAD was and why it was floating alone in space. With git switch, that doesn't happen by accident.

git restore: recovering files without the mysterious double dash

The classic way to discard changes in a file was git checkout -- file.txt. The -- there comes from a Unix convention for separating flags from arguments — without it, Git can't tell if you're passing a branch name or a path. Technically necessary. Practically, one of those details you've spent years typing without knowing what that double dash is doing there, because nobody stopped to explain it and you already had it in your muscle memory before you thought to ask.

git restore makes it explicit:

# Discard working directory changes (restore to last committed version)
git restore file.txt

# Restore multiple paths at once
git restore src/ tests/

# Discard all tracked changes in working directory
git restore .

# Restore a file to its state at a specific commit
git restore --source=abc123 file.txt

# Unstage a file (move it back from index to working directory)
git restore --staged file.txt

# Unstage and discard working directory changes in one shot
git restore --staged --worktree file.txt

The --staged flag is the one that makes the biggest difference day-to-day. Before, removing a file from the staging area meant git reset HEAD file.txt — which works, but uses the word "reset" for something that isn't a reset, which caused that brief hesitation before hitting Enter. git restore --staged does the same thing and says exactly what it does. No hesitation required.

⚠️ git restore without --staged discards working directory changes permanently. They don't go to the reflog because they were never committed — they're just overwritten. If you're not sure, git stash before git restore.

The conversion table

For when muscle memory still reaches for the old command:

Operation	Before	Now
Switch branch	`git checkout main`	`git switch main`
Create and switch	`git checkout -b new`	`git switch -c new`
Restore file	`git checkout -- file.txt`	`git restore file.txt`
Restore from commit	`git checkout abc123 -- file`	`git restore --source=abc123 file`
Unstage file	`git reset HEAD file.txt`	`git restore --staged file.txt`
Detached HEAD	`git checkout abc123`	`git switch --detach abc123`

git checkout isn't going anywhere — there's too much code, too many scripts, too much muscle memory in the world to remove it. But if you're onboarding someone new or starting a fresh project, git switch and git restore are the commands worth teaching first. Not because they're modern, but because each one does exactly one thing.

git sparse-checkout: when you don't need the whole repository

In a monorepo with fifty packages, you're usually working on two or three of them. Downloading the other forty-seven every time gives you nothing except slower clones and wasted disk space.

git sparse-checkout solves that: it lets you check out only the directories you need:

# Initialize sparse-checkout (cone mode is the standard)
git sparse-checkout init --cone

# Specify which directories to include
git sparse-checkout set packages/my-app src/shared

# Check what's currently configured
git sparse-checkout list

# Add more directories without replacing the current set
git sparse-checkout add packages/another-lib

# Disable and restore full checkout
git sparse-checkout disable

Cone mode (Git 2.26) works with whole directories instead of arbitrary globs, which makes it significantly faster and more predictable. It's the mode you'll want to use in practice.

Combined with --filter at clone time, it goes even further:

# Only download metadata; file contents are fetched on demand
git clone --filter=blob:none --sparse https://github.com/org/monorepo
cd monorepo
git sparse-checkout set packages/my-app

With --filter=blob:none, Git downloads commits and trees but defers file contents until they're actually needed. In a monorepo with years of history, the difference between this and a full clone can be several minutes. In a CI pipeline that runs a hundred times a day, those minutes add up fast.

git maintenance: the work Git does while you're not looking

As a repository grows, Git's internal operations get slower: the commit graph gets recalculated from scratch, loose objects accumulate, the index grows. git gc existed to clean all of that up, but it required either remembering to run it manually or trusting Git to trigger it automatically at the most inconvenient moment.

git maintenance (Git 2.31) registers periodic background tasks with the operating system's scheduler to keep the repository healthy without interrupting your work:

# Register this repo for background maintenance
git maintenance start

# Run all maintenance tasks manually
git maintenance run

# Stop background maintenance for this repo
git maintenance stop

With git maintenance start, Git registers a task in cron (Linux/macOS) or Task Scheduler (Windows) that periodically runs four tasks:

commit-graph: precomputes the commit graph so git log and git merge-base run faster.
loose-objects: packs loose objects accumulated since the last pack.
incremental-repack: reorganizes object packs incrementally, without the cost of a full git gc.
prefetch: downloads objects from the remote in the background so the next git fetch is faster.

On a normal-sized repository, the effect is imperceptible. On a repository with years of history and tens of thousands of commits, it can make git log and git status noticeably faster without changing anything about how you work. Which is, essentially, the best version of any optimization.

Key concepts from this lesson

git switch replaces git checkout for changing branches: it does only that, and detached HEAD requires explicit --detach.
git restore replaces git checkout -- file and git reset HEAD file: --staged moves files out of the index, without the flag it discards working directory changes.
git checkout will keep working, but git switch and git restore are what makes sense to use and teach in new projects.
git sparse-checkout --cone lets you check out only part of the repository: essential in monorepos or repos with large histories.
git maintenance start registers background maintenance with no cost to your normal workflow.

This is the second-to-last lesson in the course. In the final tutorial, we'll look at how Git fits into CI/CD pipelines: shallow clones so builds don't download the full commit history, object caching between runs, and the specific operations that make Git behave well when it's being run by an automated server instead of a developer at a terminal.

Never stop coding!

Prompts for Code Generation with AI

Javi Palacios — Wed, 14 Oct 2026 10:13:56 +0000

You type "write a function to manage users" and hit Enter.

The AI comes back with 847 lines. An entity, a repository, a service, a controller, a DTO, a mapper, a validation decorator, and a README explaining the architecture. You wanted to filter by name.

This is what happens when you ask for code without specifying what you actually want: the AI doesn't fill gaps with nothing — it fills them with competence. Impressive, thorough, and completely wrong for your use case.

The problem isn't the AI. It's that "manage users" isn't a specification — it's an open invitation. And without constraints, AI tends to show everything it knows. Like that coworker who, when you ask what time it is, also explains how atomic clocks work.

In the previous tutorial, you used AI as a learning tutor to understand programming concepts. Now the goal shifts: instead of asking for explanations, you're asking for code you can actually use. That requires a different way of asking.

Specification first, code second

The rule is straightforward: the better you describe what you want, the closer the result is to what you wanted.

It sounds obvious. It is, until you see the difference in practice:

❌ Vague specification:
"Write a function to process orders"

✅ Concrete specification:
"Write a Python function process_order() that:
- Accepts an Order object with fields: id, user_id, items (list
  of dicts with product_id and quantity), and status
- Calculates the total by summing price * quantity for each item
- Returns the Order object with total calculated and status set to 'processed'
- Raises ValueError if items is empty
- Raises OrderAlreadyProcessedError if status is already 'processed'
- Includes full type hints
- Uses no external dependencies"

Why all the detail? Because "process orders" can mean forty different things depending on the system. The function generated from the vague prompt might work perfectly in a different context than yours.

Isn't the AI supposed to be smart? It is. But smart isn't the same as psychic. Your most capable colleague would also ask three questions before touching the keyboard. The AI can't ask because you already told it to start.

Elements that belong in a solid specification:

Name and signature of the function or module
What it receives (types, input constraints)
What it returns (return type, shape of the result)
Error behavior (which exceptions to raise, when)
Technical constraints (language version, available libraries, project conventions)
What NOT to do — sometimes more important than everything else

That last one deserves its own example:

"Implement without recursion.
Don't introduce external dependencies.
Don't use type syntax below Python 3.10.
Don't generate test code yet — just the function."

Negative constraints save time. Without them, the AI makes decisions that are reasonable from its perspective but may not be from yours. Then you spend ten minutes wondering why there's a functools.reduce in a function you thought was simple.

The scaffold approach: structure first, code second

Asking for code directly is like building a house by starting with the curtains. The curtains look great. Then you discover the windows aren't standard size, the frames don't fit, and someone designed a bathroom with no door.

The scaffold approach works the other way: structure first, then implement piece by piece.

Prompt 1:
"Give me the complete file structure for a Flask REST API
for a todo app. Just the structure: filenames and one line
describing what each contains. No code yet."

Prompt 2:
"Now implement models.py with the models you described."

Prompt 3:
"Implement routes.py connecting to the models from the previous step."

Prompt 4:
"Write tests for routes.py."

Each step is verifiable before moving forward. If the structure doesn't work for you, you fix it at prompt 1, not at prompt 4 when you already have 200 lines of code that depend on a flawed decision.

There's a non-obvious benefit here too: breaking work into short steps gives the AI more room to reason. A specific, focused prompt consistently outperforms "do everything at once."

A useful variation is a context block — a chunk of project context you paste at the start of each important conversation. The AI doesn't remember previous sessions, but it does know everything in the current context. Use that:

"Project context:
- Django 5 backend with Django REST Framework
- PostgreSQL 15, Redis for caching, Celery for async tasks
- Python 3.12, strict typing with mypy
- All models follow snake_case, tests with pytest
- Domain separated into apps: orders/, products/, users/

With this context, implement..."

Once per conversation, at the start. You don't need to repeat it every prompt.

TDD with AI: tests first

The test-first approach with AI is, arguably, the most honest way to generate code. Not because it's more principled, but because it forces you to specify behavior before seeing the implementation.

The logic: tests are easier to verify than code. You can read three lines of a test and know exactly what's expected. The 40 lines of implementation that make them pass are more opaque.

Prompt 1:
"Write failing pytest tests for a function add_user(name, email) that should:
- Accept a non-empty name and a valid email
- Raise ValueError if name is empty
- Raise ValueError if email has invalid format
- Return a dict with generated id, name, and email
Don't implement the function yet."

Prompt 2:
"Now implement add_user() to make these tests pass."

Prompt 3:
"What edge cases are missing from my tests?"

The third prompt is the most valuable step in the process. The AI typically surfaces scenarios you didn't consider: emails with Unicode characters, names with 500 characters, None passed instead of a string, emails with leading spaces. Some will be irrelevant to your use case. Others will be the exact bug you'd have found in production two weeks later.

If you're also working through the Python from scratch course, this pattern maps naturally to the functions lesson: define the signature, specify behavior through tests, let the implementation come after. It's a way to practice thinking in interfaces before thinking in code.

The problems with generated code

Generated code works. It compiles, sometimes has tests, the variable names are reasonable. What it doesn't have is context about what happened the last time someone tried something similar in production.

You do.

The most common problems that slip through unnoticed:

Stale APIs. The library changed eight months ago and the AI doesn't know. The generated code uses the previous version. It works in development because you have the old version pinned. It breaks on the production server because that one has the new version. The error shows up on a Friday afternoon.

Hardcoded values. Database credentials, API tokens, internal service URLs — all in the source code. Like hiding your house key under the welcome mat with a note that says "emergency key." Everybody knows about the welcome mat. So does your git history on GitHub.

Partial error handling. The AI handles the cases you described. The ones you didn't mention don't exist to it. If you didn't say what should happen when the external service doesn't respond, there's a silent timeout waiting for its moment in production.

Security issues. SQL injection when inputs aren't sanitized correctly. Secrets leaking into logs. Validations that look complete but don't cover every execution path.

Performance that doesn't scale. A query that works with 100 records and takes 45 seconds with 50,000. The N+1 that looks innocent until traffic grows and someone messages you at 3 AM.

None of these are the AI's fault. They're a consequence of you having context it doesn't: the project's history, previous architectural decisions, bugs that already happened, the client's security requirements. That knowledge isn't transmitted in a prompt — it gets applied in review.

The review prompt

Once you have generated code, this prompt catches most issues before they reach production:

"Review the code you just generated, specifically looking for:
1. Security vulnerabilities (injection, exposed secrets, input validation)
2. Performance issues (inefficient queries, unnecessary memory load)
3. Missing or incomplete error handling
4. Hardcoded values that should be configurable
5. Assumptions about the environment that might not hold

For each issue found: describe the problem, explain why it matters,
and propose the specific fix."

The AI runs through the same checks you'd do in a code review, before you do. And it sometimes catches things you'd have missed. Not always — especially problems that depend on your system's context. The review prompt reduces noise; your judgment as a developer is still the final filter.

Key concepts from this lesson

A concrete specification produces concrete code; a vague one produces architecture nobody asked for
The scaffold approach (structure → models → routes → tests) avoids rewriting from the middle
TDD with AI: write failing tests first, ask for an implementation that passes them, then ask what edge cases you missed
Generated code works for the cases you specified — the ones you didn't mention are your responsibility
The review prompt is a second pair of eyes before code review, not a replacement for it

There's a pattern in all of these approaches: the first response is rarely the final one. The scaffold takes multiple rounds, TDD has three prompts, the review adds another pass. That's not a problem with the process — it's how building code with AI actually works. In the next tutorial, we'll look at exactly that: how to manage the refinement dialogue without losing the thread or ending up with a function that no longer resembles what you asked for in the first place.

Never stop coding!

Advanced function parameters in Python: defaults, *args, and **kwargs

Javi Palacios — Tue, 13 Oct 2026 11:06:36 +0000

The function you wrote in the previous tutorial had one or two parameters. The one you'll write in six months will have eight. Someone on the team added four without telling anyone, one has a name nobody remembers the origin of, the seventh is a boolean called flag whose behavior depends on the value of the parameter before it, and there's a mode that "makes sense in context." This isn't a prediction; it's just how software grows.

Python has tools to keep this from becoming a disaster: default parameters, keyword arguments, *args, and **kwargs. This lesson covers all of them — including why one of them has a trap that's caught more programmers than will admit it.

Default parameters

A default parameter has a value Python uses when you don't pass an argument for it:

def greet(name, greeting="Hello"):
    print(f"{greeting}, {name}")

greet("Alice")               # Hello, Alice
greet("Bob", "Hey")          # Hey, Bob

greeting is optional. If you don't pass it, Python uses "Hello". If you do, Python uses what you send.

The rule: default parameters always come after non-default ones. Python needs to know what's required and what's optional, and it figures that out by position.

def calculate(price, discount=0, tax=0.21):   # ✅
    base = price - discount
    return base + (base * tax)

def calculate(discount=0, price, tax=0.21):   # ❌ SyntaxError
    ...

If you put the default parameter first, Python can't tell whether the first argument you pass is discount or price. The question makes sense; the answer is a syntax error.

The trap nobody warns you about

Default values are evaluated once, when Python defines the function — not each time you call it. For strings, numbers, and booleans that's fine because they're immutable. For lists and dictionaries, it matters a lot.

def add_item(item, items=[]):
    items.append(item)
    return items

print(add_item("a"))   # ['a']
print(add_item("b"))   # ['a', 'b']  ← wait
print(add_item("c"))   # ['a', 'b', 'c']  ← this wasn't the plan

The [] in the definition is created once. Every call that doesn't pass items shares the exact same object. You're accumulating items across calls even though it looks like you're starting fresh each time.

Like a shared whiteboard in a meeting room: you walk in expecting a blank surface, but the diagrams from the previous meeting are still there. The whiteboard is the list; the diagrams are the elements from previous calls; and "I thought it started fresh" is wrong every time.

The fix:

def add_item(item, items=None):
    if items is None:
        items = []  # New list for each call without an explicit list
    items.append(item)
    return items

The rule: never use a mutable object as a default value — list, dict, or set. Use None and create the object inside the function. No exceptions.

Positional and keyword arguments

So far you've been passing arguments in order — those are positional arguments. Python assigns them by position: first to the first parameter, second to the second, and so on.

With two or three parameters, that's fine:

def show_user(name, age, city):
    print(f"{name}, {age}, {city}")

show_user("Alice", 34, "London")

With six:

create_report(True, "pdf", "en", None, False, "full")

What does that True do? And the False? Is None intentional or did someone forget to pass something? You can't know without reading the function definition. And reading the function definition every time you call something isn't development — it's archaeology.

Keyword arguments solve this:

show_user(name="Alice", age=34, city="London")

Now each value carries its label. You can also change the order:

show_user(city="London", name="Alice", age=34)

And mix positional and keyword — always with positional first:

show_user("Alice", city="London", age=34)   # ✅
show_user(name="Alice", 34, "London")       # ❌ SyntaxError

Why bother? For the day your function has four boolean parameters and the person reading the code six months from now isn't you. include_signature=True says what it does. True in position four says nothing.

*args: variable number of positional arguments

What if you need a function that accepts any number of arguments? The * before the parameter name tells Python to pack all the extra positional arguments into a tuple:

def add(*args):
    return sum(args)

print(add(1, 2))              # 3
print(add(1, 2, 3, 4, 5))    # 15
print(add())                  # 0

Inside the function, args is just a tuple with everything that arrived. The name args is convention — the * does the work. You could call it *numbers or *values. In practice, *args is what everyone expects to see.

def inspect(*args):
    print(type(args))   # <class 'tuple'>
    print(args)

inspect(10, 20, "hello")
# <class 'tuple'>
# (10, 20, 'hello')

You can combine regular parameters with *args, as long as *args comes after the positional ones:

def show_list(title, *items):
    print(f"{title}:")
    for item in items:
        print(f"  - {item}")

show_list("Fruits", "apple", "orange", "banana")

Fruits:
  - apple
  - orange
  - banana

Think of *args like a function that takes however many pizza toppings you want — you don't hardcode three slots for topping_1, topping_2, topping_3. Whatever shows up goes in.

**kwargs: variable keyword arguments

**kwargs does the same thing as *args, but for keyword arguments. Instead of a tuple, Python packs everything into a dictionary:

def configure(**kwargs):
    for key, value in kwargs.items():
        print(f"{key}: {value}")

configure(color="blue", size="large", border=True)

color: blue
size: large
border: True

Same as *args, the name kwargs is convention. The ** is what counts.

**kwargs shows up a lot in code that builds or configures objects, in function wrappers, and in generic utilities that don't know in advance what options they'll receive.

Honest warning: **kwargs can make your function very flexible or completely opaque. If someone calls configure(name="test", temperature=37, pizza=True) and the function accepts it without complaint, there's no way to know from the outside which parameters are actually valid. Use it when the flexibility makes real sense; not as a substitute for thinking through your function's interface. The warning sign: if you're calling kwargs.get() in eight different places inside the same function, that function is probably doing too many things.

Combining everything

You can combine regular parameters, *args, and **kwargs in one function. The order is always:

Regular positional parameters
*args
Keyword-only parameters (come after *args and can only be passed by name)
**kwargs

def log(level, *messages, timestamp=True, **extra):
    print(f"[{level}] timestamp={timestamp}")
    for msg in messages:
        print(f"  {msg}")
    if extra:
        print(f"  extra: {extra}")

log("INFO", "Connection established", "Database ready",
    timestamp=False, user="admin", ip="192.168.1.1")

[INFO] timestamp=False
  Connection established
  Database ready
  extra: {'user': 'admin', 'ip': '192.168.1.1'}

timestamp isn't a regular positional parameter or part of **kwargs — it sits between *args and **kwargs, which makes it keyword-only: it can only be passed by name, never by position.

When to use what

Default parameters: when a reasonable value works for most cases. Don't put a default on everything "just in case" — a function where everything is optional has no clear contract.
Keyword arguments: when the signature has more than three parameters or when the argument types don't make their purpose obvious.
*args: when you genuinely don't know how many positional arguments will arrive. If you know, list them explicitly — an explicit signature always communicates more than *args.
**kwargs: for wrappers, decorators, and generic configuration utilities. Not as a substitute for thinking through the interface.

Key concepts from this lesson

Default parameters always go after required ones.
Default values are evaluated once: never use a mutable object as a default value.
Keyword arguments make code more readable when functions have many parameters.
*args packs positional arguments into a tuple; **kwargs packs keyword arguments into a dictionary.
Parameters after *args are keyword-only: they cannot be passed by position.
The order is: positional → *args → keyword-only → **kwargs.

💡 Challenge: Create a function log(message, level="INFO", *tags, timestamp=True, **extra). Call it in several ways: just with message, adding level, passing extra tags, with timestamp=False, and with extra keyword arguments like user="admin". What does it print each time? What happens if you try to pass timestamp by position?

Next up: scope — how Python decides which variable is visible from where, what happens when a function defines a variable with the same name as one outside it, and why global exists even though most of the time you're better off not using it.

Never stop coding!

DEV Community: Javi Palacios

Why We Need Databases

The problem: storing data is not managing data

What breaks when we use files for everything

1. Querying data becomes painful

2. Relationships become manual

3. Multiple writers at once is a dangerous party

4. Security and guarantees disappear quickly

So what is a database?

Types of databases from a distance

Why PostgreSQL

What learning SQL really means

What you take from this lesson

Docker Compose networking: networks, services, and isolation

Compose's default network

localhost inside a container is not your machine

ports is not how containers talk to each other

Custom networks: separate rooms

expose vs ports

External networks

Network aliases

Complete example

Practical challenge

Docker Compose volumes and data persistence

The problem: containers are not hard drives

The three volume types you'll actually use

Named volumes

Bind mounts

Anonymous volumes

Read-only volumes

Database persistence

Initialization scripts

Development vs production

Backups: persistence is not a backup

Cleanup without destroying what matters

Practical challenge

Setting up your DevOps work environment: Linux, terminal, and essential tools

What a DevOps learning environment actually needs

The recommended option: a local Ubuntu VM

Installing Multipass

Creating the course VM

Accepted alternatives

WSL2 on Windows

Local Linux

Cloud later

Entering with Multipass and practicing SSH

Essential tools

Configuring a comfortable terminal

Creating the course workspace

Final check

Introduction to the DevOps world: culture, automation, and feedback

The problem before DevOps

What is DevOps?

The three core ideas: collaboration, automation, and feedback

Collaboration

Automation

Feedback

So, is DevOps a role?

What a traditional DevOps Engineer does

The DevOps ecosystem

The course roadmap

Phase 1: Traditional DevOps

Phase 2: SRE

Phase 3: Platform Engineering

What you should take from this first lesson

Marks and jumps in Vim: navigate without losing the thread

Marks: leaving a signal before you go

Global marks: across files

Automatic marks: the ones Vim sets for you

The jump list: the editor's history

The changelist: where you edited

Special jumps

%: the matching pair

gd and gD: go to definition

gf: open the file under the cursor

Key concepts from this lesson

Docker Compose file structure: health checks, environment variables, and restart

Every field a service can have

Health checks: waiting for the service, not just the container

Environment variables: the .env file

`localhost` inside a container is not your machine

`ports` is not how containers talk to each other

`expose` vs `ports`

`%`: the matching pair

`gd` and `gD`: go to definition

`gf`: open the file under the cursor