DEV Community: Wallace Espindola

Python Just Turned 35, Here's What Kept It Alive All These Years

Wallace Espindola — Fri, 20 Feb 2026 21:52:01 +0000

Today is February 20, 2026. Exactly 35 years ago, Guido van Rossum released the first public version of Python, a language he'd been hacking on during the Christmas holidays as a side project. Let that sink in for a second. A side project from 1991 is now arguably the most popular programming language on the planet.

I've been thinking about this milestone all week, and honestly, the more I dig into Python's story, the more impressed I get. Not because Python is perfect, it's definitely not, but because it kept finding ways to matter, decade after decade, even when the tech world shifted underneath it. That's rare. Languages come and go all the time. Most of them peak and fade within ten years. Python just... kept going.

So let's talk about how a hobby project turned into the backbone of modern AI, web development, data science and pretty much everything in between.

A Quick Trip Through Four Eras

Python's 35-year history breaks down pretty neatly into four phases. Each one could've been the end of the road, but instead each one became a springboard.

Phase 1: The Scripting Era (1991-2000)

Back in the early '90s, Python was "a better scripting language." That was the pitch. If you were writing shell scripts or wrestling with C for quick automation tasks, Python gave you something cleaner and more readable. It showed up in universities and Unix shops, quietly building a reputation as the language that didn't make you hate your life when you read someone else's code six months later.

Was it glamorous? Not even close. But it was useful, and people kept coming back to it.

Phase 2: The Web Era (2000-2010)

Then the web happened, and suddenly everybody needed a way to build websites that wasn't PHP spaghetti code. Ruby had Rails and that was the cool kid on the block for a while, but Python answered with Django in 2005 and Flask in 2010.

Django gave you the full kitchen, ORM, admin panel, authentication, the works. Flask gave you a knife and a cutting board and said "go build whatever you want." Between the two of them, Python carved out real territory in web development. It wasn't the dominant web language, but it was a serious contender, and startups loved it for getting things shipped fast.

Phase 3: The Data Science Revolution (2010-2020)

This is where things went nuclear. NumPy and SciPy had been around for a while, but when Pandas matured and Jupyter Notebooks made exploratory analysis actually pleasant, Python became the default language for anyone touching data. Then TensorFlow and PyTorch showed up, and suddenly Python wasn't just a data language, it was the AI language.

Researchers, data scientists and machine learning engineers all converged on Python. The network effects were enormous. If you wanted to do ML, you used Python. Period. The ecosystem was too deep, the community too large, the tooling too mature to ignore.

Python's identity shifted from "solid web language" to "the language of data and AI." And that identity has only gotten stronger since.

Phase 4: The Platform Era (2020-Present)

Now we're in the era where Python is everywhere, and I mean everywhere. FastAPI brought modern, async-first API development with automatic OpenAPI docs. Python runs DevOps pipelines, orchestrates cloud infrastructure, powers AI agents and serves as the glue language connecting systems across the entire stack.

It's not just a language anymore, it's a universal integration platform. You can prototype an idea in the morning, deploy an API by lunch, hook it up to an AI model in the afternoon and automate the whole thing before dinner. That's the power of what Python has become.

So What Actually Kept Python Alive for 35 Years?

Plenty of languages had momentum at some point. Perl was huge. Ruby was everywhere for a while. CoffeeScript had its moment. They all faded or shrank. Python didn't. Why?

Human-Centered Design

This is the big one, and it goes back to Guido's original philosophy. Python optimized for the person reading the code, not the person writing it. That sounds like a small thing, but it's massive over time. Code is read far more often than it's written, and Python's insistence on readability meant that projects stayed maintainable longer, onboarding new developers was easier and collaboration just worked better.

The famous "there should be one obvious way to do it" mantra meant less time arguing about style and more time building things.

Batteries Included

Python's standard library was always surprisingly complete. Need to parse JSON? It's there. HTTP requests? There. CSV files, email, regex, unit testing, threading? All there. You could get a lot done before you ever needed to pip install anything.

This mattered enormously in the early days when package management was rough, and it still matters today when you just want to write a quick script without pulling in half the internet.

The Ecosystem Flywheel

Once Python hit critical mass in universities, the flywheel started spinning. Students learned Python, graduated, used Python at work, wrote libraries in Python, taught Python to the next generation. Researchers published papers with Python code. Startups built MVPs in Python. Every new library made the ecosystem richer, which attracted more developers, which produced more libraries.

It's the classic network effect, and Python benefited from it more than almost any other language in history.

Low Barrier of Entry

Python never stopped being approachable. You can teach a complete beginner to write meaningful Python code in a day. That constant influx of new developers kept the community fresh, energetic and growing. Languages that become too complex or too niche eventually stagnate. Python avoided that trap by always keeping the front door wide open.

Adaptability

This might be the most underrated factor. Python didn't fight paradigm shifts, it absorbed them. Web frameworks? Python adapted. Data science? Python adapted. Machine learning? Python adapted. Cloud-native microservices? Adapted again. Every time the industry moved, Python found a way to be relevant in the new landscape.

What's Strong Right Now

Let's be honest about where Python genuinely excels today, because the list is impressive:

Developer productivity is unmatched. For getting from idea to working code, nothing beats Python. The read-write-test cycle is incredibly fast, and the language stays out of your way.

AI and ML dominance is real. PyTorch, TensorFlow, scikit-learn, Hugging Face, LangChain, the entire AI stack runs on Python. If you're doing anything with AI, you're writing Python or you're calling Python.

The ecosystem is massive. PyPI has hundreds of thousands of packages. Whatever problem you're trying to solve, someone's probably already built a library for it.

Cross-domain reach. Python is one of the very few languages where the same syntax works for web backends, data analysis, automation scripts, AI models and system administration. That versatility is genuinely rare.

Prototyping speed. When you need to validate an idea quickly, Python lets you move fast without accumulating too much technical debt in the prototype phase.

Orchestration. Python has become the default "glue language", the thing that ties together databases, APIs, AI models, cloud services and everything else. FastAPI made this even more compelling with its modern async approach.

What Could Still Get Better

I love Python, but I'm not going to pretend it's flawless. There are real pain points that the community has been wrestling with for years.

Performance

Let's get the elephant out of the room. Python is slow compared to compiled languages. It uses more memory and CPU time than Go, Rust, Java or C++. Yes, there's Cython, PyPy and the ongoing work on faster CPython. Yes, you can drop into C extensions for hot paths. But the baseline performance story is still a weakness, especially as workloads scale up.

The good news is that the core team is actively working on this. The "Faster CPython" project has already delivered meaningful speedups in recent releases, and JIT compilation is on the roadmap. It won't match Rust anytime soon, but the gap is narrowing.

The Packaging Mess

Oh, the packaging. We've got pip, poetry, conda, pipenv, pdm, hatch and now uv. Each one has its fans, each one does things slightly differently, and none of them has emerged as the single obvious choice. Compare this to npm in JavaScript or cargo in Rust, where there's basically one answer, and you can see why this frustrates people.

Yes, uv is making a strong case as the future here, it's fast, it's well-designed, and it's backed by the Astral team that gave us Ruff. But we're still in the "multiple competing standards" phase, and it creates real friction for teams and newcomers alike.

The GIL

The Global Interpreter Lock has been Python's concurrency bottleneck for decades. It prevents true parallel execution of Python threads on multiple CPU cores. For I/O-bound work it doesn't matter much, but for CPU-intensive parallel processing it's a real limitation.

The exciting development is PEP 703 and the experimental no-GIL builds. Python 3.13 shipped with an experimental free-threaded mode, and the community is cautiously optimistic. This could be a game-changer, but it'll take years for the ecosystem to fully adapt.

Enterprise Governance

Dynamic typing is great for speed but can be a headache at scale. Large Python codebases without disciplined use of type hints, linters and testing can become maintenance nightmares. Java and C# have built-in guardrails that Python relies on developers to self-impose.

The gradual typing story with mypy, pyright and type hints has improved enormously, but it's still opt-in. In enterprise environments where dozens of teams touch the same codebase, that "opt-in" nature can be a real challenge.

The Java Angle: Rivals or Partners?

Java turned 30 last year. Python turns 35 today. And people still love asking "which one should I learn?" as if it's a zero-sum game. It's not.

Here's the thing, Python and Java don't actually compete for the same jobs in most modern architectures. Java is the infrastructure backbone. It runs the banking systems, the telecom platforms, the enterprise middleware that needs to be rock-solid, handle massive concurrency and run for years without surprises. The JVM is a masterpiece of engineering, and Java's static typing and mature tooling make it ideal for large-scale systems where reliability matters more than development speed.

Python is the intelligence layer. It runs the AI models, the data pipelines, the automation scripts and the rapid-prototype APIs. It's where experimentation happens, where new ideas get tested and where developers move fast.

In practice, modern platforms often use both. Java handles the core transactional processing. Python handles the analytics, ML inference and orchestration. They complement each other beautifully, and arguing about which one is "better" misses the point entirely. They're optimized for different things, Java for systems, Python for humans.

If you're early in your career, learn both. Seriously. Having Python and Java in your toolkit makes you valuable in almost any engineering organization.

My Take on Python's Future

I've been writing Python for years, alongside Java and a handful of other languages. Here's what I think the next decade looks like.

Python isn't going anywhere. The AI wave alone guarantees its relevance for at least another ten years. But more than that, Python has demonstrated something powerful: the ability to reinvent itself without losing its soul. It's still readable, still approachable, still the language where you can teach a beginner in the morning and deploy production AI in the afternoon.

The things that need fixing, performance, packaging, the GIL, are all actively being worked on. The community is engaged, the core team is focused and the commercial investment (from companies like Microsoft, Google and Meta) is substantial. Python's governance model, with its steering council and PEP process, has proven resilient and effective.

My prediction? Python at 40 will be faster, better packaged, truly concurrent and even more deeply embedded in AI infrastructure. The language itself will probably look mostly the same, and that's the point. Python's superpower was never about being the most innovative syntax or the fastest runtime. It was about being the most useful language for the most people, and that hasn't changed in 35 years.

Happy birthday, Python. Here's to 35 more.

Let's Discuss

I'd love to hear your perspective on Python's journey:

When did you first start using Python, and what was the first thing you built with it?
What's your biggest frustration with Python today?
Do you use Python alongside another language? How do they complement each other?
Where do you think Python will be in 10 years?

Drop your thoughts in the comments. I'm genuinely curious what this community thinks.

Happy coding!

Written by Wallace Espindola, Software Engineer and Python Enthusiast.

Connect with me:

LinkedIn
GitHub

Contract-First Integration: A Practical Guide for Developers

Wallace Espindola — Tue, 10 Feb 2026 20:12:47 +0000

Ever had that moment where you finish building an API, tell the frontend team "it's ready," and then spend two days debugging why their requests keep failing? Yeah, me too.

Turns out there's a better way: Define the contract first, then build the code to match it. In this guide, I'll show you exactly how to do contract-first integration with Spring Boot, OpenAPI, and Kafka.

No theory. Just practical steps you can use today.

What Is Contract-First Integration?

Simple version: You write the API specification before you write any code.

Instead of:

Code the API
Test it
Document it
Hope it matches what consumers expected

You do:

Write the contract (OpenAPI spec or Avro schema)
Get everyone to agree on it
Generate code from the contract
Implement the logic

The contract is the single source of truth. Code, tests, and docs all come from it.

Why Should You Care?

Three reasons:

1. No more "but I thought you meant..." conversations

When you define the contract upfront, there's no ambiguity. The spec says exactly what fields are required, what types they are, and what errors to expect.

2. Teams can work in parallel

Frontend doesn't have to wait for backend. They can generate a client SDK from the contract and develop against a mock
server. When the real API is ready, they just switch the URL. No code changes.

3. Breaking changes get caught in CI, not production

You set up automated checks that fail the build if someone tries to remove a required field or change a response type. No more "oops, we broke all the mobile clients" moments.

Quick Start: Your First OpenAPI Contract

Let's build a simple orders API. We'll start with the contract.

Create a file: contracts/openapi/orders-api.yaml

openapi: 3.2.0
info:
  title: Orders API
  version: 1.0.0
  description: Contract-first REST API for order management

paths:
  /v1/orders:
    post:
      summary: Create a new order
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CreateOrderRequest'
      responses:
        '201':
          description: Order created successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/OrderResponse'
        '400':
          description: Invalid request
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'

  /v1/orders/{orderId}:
    get:
      summary: Get an order by ID
      parameters:
        - name: orderId
          in: path
          required: true
          schema:
            type: string
      responses:
        '200':
          description: Order found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/OrderResponse'
        '404':
          description: Order not found

components:
  schemas:
    CreateOrderRequest:
      type: object
      required:
        - customerId
        - items
      properties:
        customerId:
          type: string
          example: "CUST-123"
        items:
          type: array
          minItems: 1
          items:
            $ref: '#/components/schemas/OrderItem'

    OrderItem:
      type: object
      required:
        - sku
        - quantity
      properties:
        sku:
          type: string
          example: "SKU-001"
        quantity:
          type: integer
          minimum: 1
          example: 2

    OrderResponse:
      type: object
      required:
        - orderId
        - customerId
        - status
        - items
        - timestamp
      properties:
        orderId:
          type: string
          example: "ORD-10001"
        customerId:
          type: string
        status:
          type: string
          enum: [ CREATED, REJECTED ]
        items:
          type: array
          items:
            $ref: '#/components/schemas/OrderItem'
        timestamp:
          type: string
          format: date-time

    ErrorResponse:
      type: object
      required:
        - code
        - message
        - timestamp
      properties:
        code:
          type: string
          example: "VALIDATION_ERROR"
        message:
          type: string
          example: "items must not be empty"
        timestamp:
          type: string
          format: date-time

That's your contract. It defines:

Two endpoints: POST /v1/orders and GET /v1/orders/{orderId}
Request and response structures
Error responses
Validation rules (like minItems: 1 for the items array)

Implementing the API in Spring Boot

Now let's build the actual API. Create a Spring Boot controller:


@RestController
@RequestMapping("/v1/orders")
@RequiredArgsConstructor
@Slf4j
public class OrderController {

    private final OrderService orderService;

    @PostMapping
    public ResponseEntity<OrderResponse> createOrder(
            @Valid @RequestBody CreateOrderRequest request) {

        log.info("Creating order for customer: {}", request.customerId());

        OrderResponse response = orderService.createOrder(request);

        return ResponseEntity
                .status(HttpStatus.CREATED)
                .body(response);
    }

    @GetMapping("/{orderId}")
    public ResponseEntity<OrderResponse> getOrder(@PathVariable String orderId) {
        OrderResponse response = orderService.getOrder(orderId)
                .orElseThrow(() ->
                        new ResourceNotFoundException("Order not found: " + orderId));

        return ResponseEntity.ok(response);
    }
}

Key points:

@Valid triggers validation based on the contract constraints
HTTP status codes match what the contract specifies (201 for created, 404 for not found)
DTOs (CreateOrderRequest, OrderResponse) match the contract schemas exactly

Adding Contract Validation to CI

Here's how to catch breaking changes before they reach production.

Add this to your GitHub Actions workflow:

name: Validate API Contract

on: [ pull_request ]

jobs:
  contract-validation:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: Check for breaking changes
        run: |
          npx openapi-diff \
            origin/main:contracts/openapi/orders-api.yaml \
            HEAD:contracts/openapi/orders-api.yaml \
            --fail-on-breaking

Now if someone tries to:

Remove a required field
Change a field type
Remove an endpoint
Change HTTP status codes

The PR build will fail. Breaking changes get caught in code review.

Contract-First for Kafka Events

APIs are one thing, but what about event-driven systems? Let's look at Kafka.

For Kafka, you need two contracts:

1. Topic Semantics (Human-Readable)

Create: contracts/events/topics.md

## orders.order-created.v1

- Purpose: Emitted when an order is created
- Key: orderId (for partition affinity)
- Delivery: At-least-once (consumers must be idempotent)
- Deduplication: Use eventId field
- DLQ: orders.order-created.v1.dlq

This documents the operational behavior.

2. Avro Schema (Machine-Validated)

Create: contracts/events/avro/OrderCreated.v1.avsc

{
  "type": "record",
  "name": "OrderCreated",
  "namespace": "com.acme.events",
  "fields": [
    {
      "name": "eventId",
      "type": "string",
      "doc": "Unique event ID for deduplication"
    },
    {
      "name": "occurredAt",
      "type": "string",
      "doc": "ISO 8601 timestamp"
    },
    {
      "name": "orderId",
      "type": "string"
    },
    {
      "name": "customerId",
      "type": "string"
    },
    {
      "name": "items",
      "type": {
        "type": "array",
        "items": {
          "type": "record",
          "name": "OrderItem",
          "fields": [
            {
              "name": "sku",
              "type": "string"
            },
            {
              "name": "quantity",
              "type": "int"
            }
          ]
        }
      }
    }
  ]
}

Publishing Kafka Events

Here's how to publish events in Spring Boot:


@Component
@RequiredArgsConstructor
@Slf4j
public class OrderEventPublisher {

    private final KafkaTemplate<String, Object> kafkaTemplate;

    public void publishOrderCreated(OrderCreated event) {
        String key = event.getOrderId();

        log.debug("Publishing OrderCreated: orderId={}", key);

        CompletableFuture<SendResult<String, Object>> future =
                kafkaTemplate.send("orders.order-created.v1", key, event);

        future.whenComplete((result, ex) -> {
            if (ex == null) {
                log.info("Published OrderCreated: partition={}, offset={}",
                        result.getRecordMetadata().partition(),
                        result.getRecordMetadata().offset());
            } else {
                log.error("Failed to publish OrderCreated", ex);
            }
        });
    }
}

Why use orderId as the key?

Kafka partitions messages by key. Using orderId ensures all events for the same order go to the same partition, preserving order.

Consuming Kafka Events (The Right Way)

Kafka delivers messages at-least-once, which means duplicates are possible. Your consumer must be idempotent:


@KafkaListener(topics = "orders.order-created.v1", groupId = "billing-service")
public void onOrderCreated(OrderCreated event) {
    // Check if we already processed this event
    if (processedEventsRepo.existsByEventId(event.getEventId())) {
        log.debug("Skipping duplicate event: {}", event.getEventId());
        return;
    }

    // Process the event
    billingService.createInvoice(
            event.getOrderId(),
            event.getCustomerId(),
            event.getItems()
    );

    // Mark as processed
    processedEventsRepo.save(
            new ProcessedEvent(event.getEventId(), Instant.now())
    );
}

This pattern prevents duplicate processing. Even if Kafka redelivers the same event, you check the eventId and skip it if you've seen it before.

Schema Evolution Done Right

Eventually you'll need to add fields to your events. Here's how to do it without breaking consumers.

Wrong way (breaks consumers):

{
  "name": "source",
  "type": "string"
}

This makes source required. Old events don't have it. New consumers crash when reading old events.

Right way (backward compatible):

{
  "name": "source",
  "type": [
    "null",
    "string"
  ],
  "default": null
}

Now:

Old consumers reading new events: They ignore the source field
New consumers reading old events: They get null for source
Nobody breaks

Always make new fields optional with defaults.

Setting Up Schema Registry

Schema Registry validates that your schema changes are backward compatible.

In application.yml:

spring:
  kafka:
    bootstrap-servers: localhost:9092
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: io.confluent.kafka.serializers.KafkaAvroSerializer
      properties:
        schema.registry.url: http://localhost:8081
        auto.register.schemas: true
        use.latest.version: true

When your producer starts, it tries to register the schema. If the schema isn't backward compatible with the previous version, registration fails and your service won't start.

This prevents breaking changes from reaching production.

Database Contracts With Flyway

Database schemas are contracts too. Use Flyway to version your migrations.

Create: src/main/resources/db/migration/V1__create_orders.sql

CREATE TABLE orders
(
    id          VARCHAR(32) PRIMARY KEY,
    customer_id VARCHAR(32) NOT NULL,
    status      VARCHAR(16) NOT NULL,
    created_at  TIMESTAMP   NOT NULL DEFAULT now()
);

CREATE TABLE order_items
(
    order_id VARCHAR(32) NOT NULL REFERENCES orders (id),
    sku      VARCHAR(64) NOT NULL,
    quantity INT         NOT NULL CHECK (quantity > 0),
    PRIMARY KEY (order_id, sku)
);

CREATE INDEX idx_orders_customer_id ON orders (customer_id);

Flyway runs migrations in order on startup. If someone manually modified the schema, Flyway detects the mismatch and fails the deployment.

Quick Wins Checklist

Here's what you can do this week:

Day 1: Write one OpenAPI contract

Pick your simplest API endpoint
Write the OpenAPI spec for it
Validate it with editor.swagger.io

Day 2: Set up contract validation in CI

Add openapi-diff to your GitHub Actions
Test it by trying to make a breaking change

Day 3: Create a mock server

Use Prism or Wiremock to serve responses based on your contract
Give the mock URL to your frontend team

Day 4: Implement the real API

Build the Spring Boot controller that matches the contract
Run your contract validation to ensure it matches

Day 5: Measure the difference

Count integration bugs before and after
Measure time to integration

Common Mistakes to Avoid

Mistake 1: Generating contracts from code

Don't use annotations to generate OpenAPI specs. Write the spec first, then implement code that matches it.

Mistake 2: Skipping CI validation

Contracts are useless if they're not enforced. Always add automated checks.

Mistake 3: Breaking changes without versioning

If you need to make a breaking change, create a new version (/v2/orders) and deprecate the old one gradually.

Mistake 4: Forgetting about idempotency

Always include an idempotencyKey field for POST requests and an eventId for Kafka events. Retries happen. Your code must handle them gracefully.

Tools That Help

OpenAPI:

Swagger Editor - Validate your specs
Prism - Mock server from OpenAPI
openapi-generator - Generate client SDKs
openapi-diff - Detect breaking changes

Kafka:

Confluent Schema Registry - Schema validation
Avro Tools - Generate Java classes from schemas

Database:

Flyway - Database migrations
Liquibase - Alternative to Flyway

What's Your Experience?

Have you tried contract-first integration? What worked? What didn't?

Drop a comment below. I'd love to hear about your experiences and answer any questions.

Want the full code?

Check out the complete working example: github.com/wallaceespindola/contract-first-integrations

It includes:

Full OpenAPI specs
Spring Boot implementation
Kafka producer and consumer with Avro
Flyway migrations
CI/CD validation setup

If this helped you, hit the ❤️ button and share it with your team. Follow me for more practical Spring Boot and Kafka tutorials.

Related posts you might like:

Need more tech insights?

Check out my GitHub, LinkedIn, and Speaker Deck.

Support my work: Buy Me A Coffee ☕

Happy coding!

Generating Realistic Fake Data in Java with Quarkus, DataFaker & EasyRandom

Wallace Espindola — Tue, 07 Oct 2025 17:47:43 +0000

Let’s be honest — your test data probably looks a little… robotic.
You know the drill: “John Doe”, “test@example.com”, “123 Main Street”.

It works, but it’s painfully obvious that it’s fake.

And if you’re demoing your API, seeding a database, or stress-testing your app, that kind of data just doesn’t cut it anymore.

The good news? With Quarkus, DataFaker, and EasyRandom, you can generate realistic, varied, and structured data — fast, elegant, and believable.

By the end of this article, you’ll have a Quarkus REST API that returns lifelike fake users — names, emails, phones, and addresses — all neatly timestamped.

Full code example here:
👉 github.com/wallaceespindola/fake-data-quarkus

Let’s dive in.

Why Fake Data Matters More Than Ever

If you’re a backend dev, you’ve probably needed mock data for one of these:

Testing APIs or UI components
Populating demo environments
Running performance or load tests
Anonymizing real data for staging environments

And sure, you could throw in “John Doe” again.
But what if you could generate realistic, localized names, addresses, and emails that feel like real-world data — and do it dynamically, in a fast Quarkus service?

That’s what we’re building.

The Stack

We’ll use:

Quarkus — the fast, lightweight Java framework for modern apps
DataFaker — for generating realistic values
EasyRandom — for filling entire objects automatically
Lombok — to cut boilerplate
OpenAPI + Swagger UI — for quick API testing

Project Setup

Generate a Quarkus project using the CLI or code.quarkus.io:

mvn io.quarkus.platform:quarkus-maven-plugin:3.12.1:create \
  -DprojectGroupId=com.example \
  -DprojectArtifactId=fake-data-quarkus \
  -DclassName="com.example.fakedata.UserResource" \
  -Dpath="/api/users"

Then, add dependencies to your pom.xml:

<dependencies>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-resteasy-reactive</artifactId>
  </dependency>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-smallrye-openapi</artifactId>
  </dependency>
  <dependency>
    <groupId>net.datafaker</groupId>
    <artifactId>datafaker</artifactId>
    <version>2.2.2</version>
  </dependency>
  <dependency>
    <groupId>org.jeasy</groupId>
    <artifactId>easy-random-core</artifactId>
    <version>5.0.0</version>
  </dependency>
  <dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <optional>true</optional>
  </dependency>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-junit5</artifactId>
    <scope>test</scope>
  </dependency>
</dependencies>

The Domain

Let’s start small — our fake “User” object:

package com.example.fakedata.domain;

import lombok.Builder;
import lombok.Data;

@Data
@Builder
public class User {
  private String id;
  private String fullName;
  private String email;
  private String phone;
  private String address;
}

And a DTO version for API responses:

package com.example.fakedata.dto;

public record UserDto(String id, String fullName, String email, String phone, String address) { }

We’re using Lombok for the model and a Java Record for the DTO to keep things clean.

The Data Generation Service

Here’s where we mix DataFaker and EasyRandom for maximum awesomeness:

package com.example.fakedata.service;

import com.example.fakedata.domain.User;
import jakarta.enterprise.context.ApplicationScoped;
import net.datafaker.Faker;
import org.jeasy.random.EasyRandom;
import org.jeasy.random.EasyRandomParameters;

import java.util.List;
import java.util.Locale;
import java.util.UUID;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

@ApplicationScoped
public class DataGenService {

  private final Faker faker = new Faker(Locale.ENGLISH);
  private final EasyRandom easyRandom;

  public DataGenService() {
    EasyRandomParameters params = new EasyRandomParameters()
        .seed(System.currentTimeMillis())
        .stringLengthRange(5, 20);
    this.easyRandom = new EasyRandom(params);
  }

  public User randomUserViaDatafaker() {
    return User.builder()
        .id(UUID.randomUUID().toString())
        .fullName(faker.name().fullName())
        .email(faker.internet().emailAddress())
        .phone(faker.phoneNumber().cellPhone())
        .address(faker.address().fullAddress())
        .build();
  }

  public User randomUserViaEasyRandom() {
    User u = easyRandom.nextObject(User.class);
    if (u.getId() == null || u.getId().isBlank())
      u.setId(UUID.randomUUID().toString());
    u.setFullName(faker.name().fullName());
    u.setEmail(faker.internet().emailAddress());
    return u;
  }

  public List<User> manyUsers(int count, boolean useEasyRandom) {
    return IntStream.range(0, Math.max(0, count))
        .mapToObj(i -> useEasyRandom ? randomUserViaEasyRandom() : randomUserViaDatafaker())
        .collect(Collectors.toList());
  }
}

Here’s the beauty of it:

DataFaker generates names, addresses, and emails that sound real.
EasyRandom handles the object creation for you.

Combine them, and you’ve got structured data that looks human.

The REST API

Now, let’s make it accessible with Quarkus RESTEasy Reactive.

package com.example.fakedata;

import com.example.fakedata.domain.User;
import com.example.fakedata.dto.UserDto;
import com.example.fakedata.service.DataGenService;
import jakarta.inject.Inject;
import jakarta.ws.rs.*;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.Response;
import org.eclipse.microprofile.openapi.annotations.Operation;
import org.eclipse.microprofile.openapi.annotations.tags.Tag;

import java.time.Instant;
import java.util.List;
import java.util.stream.Collectors;

@Path("/api/users")
@Produces(MediaType.APPLICATION_JSON)
@Tag(name = "Users", description = "Generate fake users with DataFaker + EasyRandom")
public class UserResource {

  @Inject
  DataGenService service;

  record ApiResponse<T>(T data, Instant timestamp) {
    static <T> ApiResponse<T> of(T data) {
      return new ApiResponse<>(data, Instant.now());
    }
  }

  @GET
  @Path("/{count}")
  @Operation(summary = "Generate a list of fake users")
  public Response generateUsers(@PathParam("count") int count,
                                @QueryParam("easy") @DefaultValue("false") boolean easy) {
    List<UserDto> users = service.manyUsers(count, easy)
        .stream()
        .map(u -> new UserDto(u.getId(), u.getFullName(), u.getEmail(), u.getPhone(), u.getAddress()))
        .collect(Collectors.toList());
    return Response.ok(ApiResponse.of(users)).build();
  }

  @GET
  @Path("/one")
  @Operation(summary = "Generate one fake user")
  public Response oneUser() {
    User u = service.randomUserViaDatafaker();
    return Response.ok(ApiResponse.of(new UserDto(u.getId(), u.getFullName(), u.getEmail(), u.getPhone(), u.getAddress()))).build();
  }
}

This API gives you two simple endpoints:

GET /api/users/{count} — returns N users
GET /api/users/one — returns a single fake user

Every response includes a timestamp, so you know exactly when it was generated.

Run It

Start Quarkus in dev mode:

./mvnw quarkus:dev

Visit Home Test Page:

👉 http://localhost:8080/

Visit Swagger Page for Endpoints:

👉 http://localhost:8080/q/swagger-ui

Try:

GET /api/users/3

Sample output:

{
  "data": [
    {
      "id": "a8a5fbb3-47b6-4387-9827-8c9b8ac6e9dc",
      "fullName": "Rosa Bianchi",
      "email": "rosa.bianchi@example.it",
      "phone": "+39 349 123 4567",
      "address": "Via Roma 45, Milano"
    },
    {
      "id": "c48de9e7-92f4-4b15-8bcb-3c1b3a710c0c",
      "fullName": "James Holloway",
      "email": "james.holloway@demo.io",
      "phone": "+1 415 555 1829",
      "address": "77 Market Street, San Francisco"
    }
  ],
  "timestamp": "2025-10-06T18:45:33.982Z"
}

Looks real enough to demo your next project, right?

Bonus: Built-In Swagger & OpenAPI

Quarkus comes with OpenAPI + Swagger UI ready to go.

No config hell, no dependencies to chase. Just hit /q/swagger-ui and explore your endpoints visually.

Want to tweak the OpenAPI path?

Just update application.properties:

quarkus.smallrye-openapi.path=/openapi
quarkus.swagger-ui.always-include=true

Why This Combo Works

DataFaker brings authenticity — localized names, realistic emails, believable data.
EasyRandom handles complexity — fills entire objects without effort.
Quarkus ties it together with speed and simplicity — minimal setup, instant hot reloads, and small footprint.

You end up with a lightning-fast, production-like data generator that’s perfect for modern development.

Real Use Cases

Preloading a test database
Filling front-end prototypes with real-looking data
Stress-testing APIs with variety
Creating synthetic datasets for analytics
Replacing sensitive info in QA or sandbox environments

Going Native (Optional)

Want to go wild? Build it as a native binary using GraalVM:

./mvnw package -Pnative
./target/fake-data-quarkus-1.0.0-runner

Startup time? Instant.
Memory footprint? Tiny.
Speed? Ridiculous.

Perfect for containers, CI pipelines, or quick mock environments.

Final Thoughts

Fake data doesn’t have to be boring.
With Quarkus, DataFaker, and EasyRandom, you can generate realistic, rich, and structured data that makes your applications — and demos — feel alive.

This setup is fast, modern, and ready to use for everything from testing to production seeding.

So next time you’re building or testing an API, forget “John Doe”.
Give your fake data a little soul.

Need more tech insights?

Check out my GitHub, LinkedIn, and SpeakerDeck.

Support my work: Buy Me A Coffee ☕

Cassandra vs PostgreSQL: A Developer’s Guide to Choose the Right Database

Wallace Espindola — Wed, 09 Jul 2025 15:48:42 +0000

Choosing the right database can feel a bit like picking the right tool for a job—you wouldn't use a hammer to tighten a screw, right? In the world of databases, two heavy-weight options often come up: Apache Cassandra and PostgreSQL. Both are powerful, but they shine in different scenarios. Let's dive into their strengths, weaknesses, and ideal use cases to help you make an informed decision.

Understanding the Basics

Apache Cassandra

Cassandra is a distributed NoSQL database designed for handling large amounts of data across many servers. It's known for its high availability and scalability. Companies like Apple and Netflix rely on Cassandra to manage massive datasets.

Apple: Reportedly runs over 75,000 Cassandra nodes, storing more than 10 petabytes of data.

Netflix: Uses Cassandra to handle its ever-growing persistence needs.

PostgreSQL

PostgreSQL is a powerful, open-source relational database system known for its robustness and standards compliance. It's widely used in various applications, from web development to data analysis. GitLab, for instance, uses PostgreSQL as its primary database system.

Key Differences

Feature	Cassandra	PostgreSQL
Data Model	Wide-column store (NoSQL)	Relational (SQL)
Scalability	Horizontally scalable across many servers	Vertically scalable; horizontal scaling possible with extensions
Consistency	Eventual consistency (tunable)	Strong consistency (ACID compliant)
Query Language	CQL (Cassandra Query Language)	SQL (Structured Query Language)
Use Cases	High write throughput, IoT, real-time analytics	Complex queries, transactional systems, analytics
Community Support	Active community with enterprise backing	Large, active open-source community

When to Choose Cassandra

Cassandra excels in scenarios where you need to handle large volumes of data with high availability and scalability. Consider Cassandra if:

High Write Throughput: Your application requires writing large amounts of data quickly, such as logging or sensor data.

Distributed Architecture: You need a database that can run across multiple data centers or cloud regions.

Fault Tolerance: Your system must remain operational even if parts of it fail.

Example: A global e-commerce platform tracking user activity in real-time across various regions.

When to Choose PostgreSQL

PostgreSQL is ideal when your application requires complex queries, transactions, and data integrity. Opt for PostgreSQL if:

Complex Queries: You need to perform joins, aggregations, and subqueries.

Data Integrity: Your application requires strict adherence to ACID properties.

Extensibility: You plan to use extensions like PostGIS for geospatial data or TimescaleDB for time-series data.

Example: A financial application managing transactions and generating detailed reports.

Real-World Use Cases

Cassandra at Apple

Apple uses Cassandra to manage services like iMessage and iTunes, running over 75,000 nodes and storing more than 10 petabytes of data.

PostgreSQL at GitLab

GitLab relies on PostgreSQL for its database needs, emphasizing its robustness and reliability.

Performance and Scalability

Cassandra

Scalability: Designed for horizontal scaling; adding more nodes increases capacity.

Performance: Optimized for write-heavy workloads; reads can be fast with proper data modeling.

PostgreSQL

Scalability: Primarily scales vertically; horizontal scaling achievable with tools like Citus.

Performance: Excels in read-heavy workloads and complex queries; write performance is strong but may require tuning for very high volumes.

Community and Ecosystem

Cassandra

Backed by the Apache Software Foundation, Cassandra has a robust ecosystem with tools for monitoring, management, and integration.

PostgreSQL

PostgreSQL boasts a vast array of extensions and tools, such as PostGIS for geospatial data and TimescaleDB for time-series data. Its active community ensures continuous improvement and support.

Final Thoughts

Choosing between Cassandra and PostgreSQL depends on your specific needs:

Opt for Cassandra if you require a highly scalable, fault-tolerant system capable of handling massive write loads across distributed environments.

Choose PostgreSQL when your application demands complex queries, strong data integrity, and a rich set of features out of the box.

Both databases are powerful in their domains. Understanding your application's requirements will guide you to the right choice.

References

Cassandra's documentation: https://cassandra.apache.org/
PostgreSQL's documentation: https://www.postgresql.org/

Need more tech insights?

Check out my GitHub repo and my LinkedIn page.
Some of my presentation slides are available here.
Do you want to buy me a coffee to elevate my energy? You can do it here.

Happy coding!

FastAPI Unleashed: Building Modern and High-Performance APIs

Wallace Espindola — Tue, 20 May 2025 17:41:01 +0000

If you’re coding in Python and need to whip up a blazing-fast API with minimal fuss, FastAPI is the framework you’ve been waiting for. It brings together ease of use, powerful data validation, and auto-generated API docs into one neat package. Below, we’ll walk through why FastAPI rocks and use code snippets to highlight its best features.

What is FastAPI?

FastAPI is a Python web framework designed for building APIs quickly and efficiently. Leveraging Python’s async and await keywords, it takes advantage of non-blocking I/O to handle high loads. Under the hood, it uses Starlette for HTTP and Pydantic for data validation, making the development process clean and robust.

Key Highlights

High-performance: Comparable to Node.js and Go in many benchmarks.

Type Hints: Strongly tied to type hints for validating and documenting your APIs.

Interactive Docs: Automatic generation of docs using Swagger UI and ReDoc right out of the box.

Basic API: Hello, FastAPI!

Let’s start simple, so here’s a basic hello-world to start:

from fastapi import FastAPI

app = FastAPI()

# A simple GET endpoint
@app.get("/")
async def read_root():
    return {"message": "Hello, FastAPI!"}

In this example, you can see how FastAPI simple to implement and very compact.

Now a simple API with GET and POST Endpoints

Here’s a basic example with GET/POST. Let's suppose you want to build an API to store item data in a hypothetical e-commerce application. With FastAPI, you can define routes in just a few lines of code:

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class Item(BaseModel):
    name: str
    description: str = None
    price: float
    tax: float = None

@app.get("/")
async def read_root():
    return {"message": "Hello from root path!"}

@app.get("/items/{item_id}")
async def read_item(item_id: int, q: str = None):
    return {"item_id": item_id, "q": q}

@app.post("/items/")
async def create_item(item: Item):
    return {"item": item.dict()}

What’s Going On?

app = FastAPI(): Instantiates your FastAPI application.

Item class: Uses Pydantic to define the data model. The BaseModel approach means any JSON coming into the endpoint will be validated automatically.

@app.get("/"): A simple GET endpoint that returns a JSON greeting.

@app.get("/items/{item_id}"): Parameterizing the endpoint to retrieve a specific item (by item_id) and an optional query parameter q.

@app.post("/items/"): A POST endpoint to create a new item. Because it’s defined with item: Item, FastAPI will expect a JSON body matching the Item schema.

Making Life Easier with Dependency Injection

Now, imagine you need the same parameter—like a search query—across multiple endpoints. Instead of duplicating logic, you can lean on FastAPI’s built-in dependency injection:

from fastapi import Depends

def common_parameters(q: str = None):
    return {"q": q}

@app.get("/items/")
async def read_items(commons: dict = Depends(common_parameters)):
    return commons

@app.get("/users/")
async def read_users(commons: dict = Depends(common_parameters)):
    return {"users_query": commons}

Why Dependency Injection?

Reusability: Define a common function once and share it with multiple routes.

Maintainability: Centralize logic (like parsing query parameters or handling authentication) in one function.

Clean Code: Keep your endpoint functions minimal and focused on core logic.

Background Tasks: Handling Work Behind the Scenes
Sometimes you want to do something without making the user wait—like sending a notification or writing to a log. That’s where background tasks come in:

from fastapi import BackgroundTasks

def write_log(message: str):
    with open("log.txt", "a") as log:
        log.write(message + "\n")

@app.post("/send-notification/")
async def send_notification(background_tasks: BackgroundTasks):
    background_tasks.add_task(write_log, "Notification sent!")
    return {"message": "Notification will be sent in the background"}

How Does It Work?

Background Tasks: A special FastAPI parameter that queues up tasks to run after your endpoint returns a response.

Immediate Responses: Your user gets a success message right away, while the server quietly handles the heavy lifting or logging behind the scenes.

Why Should You Use FastAPI?

Developer Productivity:
The intuitive design and interactive docs mean you spend less time on boilerplate and more time on what matters—your business logic.

Performance & Scalability:
By using async/await, FastAPI manages high throughput gracefully, making it a strong choice for microservices or real-time data apps.

Automatic Documentation:
Just hit /docs in your browser, and you’ll see a Swagger UI ready to go. Testing endpoints is a piece of cake.

Data Validation & Serialization:
Pydantic models (like our Item example) ensure the data coming in (and going out) matches your defined schema. It catches errors early, so you’re not left debugging invalid payloads later.

Easy Integrations:
Want to integrate a database like PostgreSQL, or a Machine Learning model built in scikit-learn? FastAPI’s modular design and dependency injection make it super straightforward.

Wrapping Up

FastAPI is a breath of fresh air in the Python ecosystem for building APIs. Its combination of modern Python features, high performance, and user-friendly documentation sets it apart from other frameworks. Whether you’re a data scientist deploying a predictive model or a developer writing microservices, FastAPI is definitely worth a look.

Ready to give it a spin? Install FastAPI with pip install fastapi uvicorn, throw in the examples above, and open your browser at http://127.0.0.1:8000/docs to see your new API in action.

You can find all the code in this article in this GitHub repo.

References, of course

This project uses FastAPI, Starlette, Pydantic, Swagger and Redoc in it. For detailed documentation, take a look at:

FastAPI docs
Starlette docs
Pydantic docs
Swagger docs
Redoc docs

Need more tech insights?

Check out my GitHub repo and my LinkedIn page. Some slides here.
Do you want to buy me a coffee to elevate my energy? You can do it here.

Happy coding!

NoSQL Fighters Arena: The Battle of Data Titans

Wallace Espindola — Thu, 24 Apr 2025 22:38:38 +0000

Welcome to the NoSQL Arena, where mighty heroes battle not with fists, but with queries, scalability, and performance. Each warrior brings their own superpowers, tactics, and secret weaknesses. Let's meet our combatants!

🌀 MongoDB — The Shapeshifter

Codename: The Document Bender

Type: Document

Class: Agile Technomancer

Superpowers:

Master of flexible schemas — adapts to any data form
Fluent in JSON-like spellcasting (BSON)
Summons indexes for fast lookup in semi-structured chaos

Weaknesses:

Needs configuration for true consistency
Can get sluggish under massive joins

Best Battlegrounds:

CMS realms, e-commerce jungles, and fast-changing development fields.

⚔️ Cassandra — The Warlord of Writes

Codename: The Column Crusher

Type: Wide-column

Class: Distributed Commander

Superpowers:

Unmatched in handling high write volume
Stretches across continents with ease (multi-region champion)
Resilient under pressure — no single point of failure

Weaknesses:

Requires deep strategic planning (data modeling isn't beginner-friendly)
Limited in query flexibility

Best Battlegrounds:

Real-time analytics warzones, IoT strongholds, social networks under siege.

⚡ Redis — The Speedster

Codename: The Memory Phantom

Type: Key-Value

Class: Temporal Trickster

Superpowers:

Blinding in-memory speed
Time-bending queues, pub/sub, and leaderboards
Can freeze and persist memory snapshots

Weaknesses:

Short-term memory — must be carefully managed for durability
Not ideal for complex data battles

Best Battlegrounds:

Caching towers, session dungeons, messaging arenas.

🕸️ Neo4j — The Webweaver

Codename: The Graph Seer

Type: Graph

Class: Relational Mystic

Superpowers:

Sees hidden connections — perfect for deep relationship logic
Controls the Cypher of the Graph Realm
Swift pathfinding through data networks

Weaknesses:

Doesn’t scale horizontally like the others
Not built for heavy write storms

Best Battlegrounds:

Fraud rings, social kingdoms, and recommendation labyrinths.

🔄 Couchbase — The Syncmaster

Codename: The Hybrid Hacker

Type: Document + Key-Value

Class: Cloud Rogue

Superpowers:

Fuses document power with key-value speed
Offline-first magic for mobile warriors
Speaks SQL-like N1QL incantations

Weaknesses:

Complex gear needed for full potential
High cost if not optimized

Best Battlegrounds:

Real-time mobile raids, distributed cloud ops, and sync-centric conflicts.

☁️ DynamoDB — The Cloudborn

Codename: The Serverless Sentinel

Type: Document + Key-Value

Class: Scalable Guardian

Superpowers:

Instantly scales to cosmic proportions
Configurable consistency (strong or eventual)
Immune to infrastructure failure

Weaknesses:

Costs can skyrocket in long campaigns
Limited in query magic — needs indexes and design forethought

Best Battlegrounds:

Serverless realms, battle carts, gaming arenas.

🎯 Elasticsearch — The Searchblade

Codename: The Index Warden

Type: Search / Document

Class: Text Archer

Superpowers:

Precision full-text strikes
Instant retrieval across massive scrolls
Data dashboard conjuring with Kibana

Weaknesses:

Not a transactional fighter
Can become a memory hog

Best Battlegrounds:

Search quests, log-dungeons, observability citadels.

🗿 HBase — The Ancient Colossus

Codename: The Big Data Behemoth

Type: Wide-column

Class: Batch Titan

Superpowers:

Handles ancient, massive archives with ease
Integrates with Hadoop scrolls and Spark fire
Excels at sequential time-series combat

Weaknesses:

Slow to move, hard to master
Requires lots of provisioning and allies

Best Battlegrounds:

Telecom vaults, financial archives, historical timelines.

NoSQL Fighters Arena — Comparison Summary

DB Hero	Spell Type	Superpower	Best Battlegrounds
🌀 MongoDB — The Shapeshifter	Document	Schema flexibility, fast querying	Web apps, CMS, e-commerce
⚔️ Cassandra — The Warlord of Writes	Wide-column	High write throughput, distributed scalability	Real-time analytics, IoT, social networks
⚡ Redis — The Speedster	Key-Value	In-memory speed, pub/sub magic	Caching, messaging systems, leaderboards
🕸️ Neo4j — The Webweaver	Graph	Powerful relationship navigation (Cypher)	Social graphs, recommendations, fraud detection
🔄 Couchbase — The Syncmaster	Document + Key-Value	Mobile sync, hybrid model flexibility	Offline-first apps, real-time APIs, caching
☁️ DynamoDB — The Cloudborn	Document + Key-Value	Auto-scaling, managed infrastructure	Serverless apps, gaming, shopping carts
🎯 Elasticsearch — The Searchblade	Search / Document	Full-text search precision, analytics power	App/site search, log analytics, observability
🗿 HBase — The Ancient Colossus	Wide-column	Batch data processing, massive scale	Telecom, financial archives, Hadoop ecosystems

Final Word: Choose Your Champion

In this arena, there's no single victor—each fighter dominates their specific battlefield. Whether you're crafting a social empire, building lightning-fast apps, or managing global dataflows, there's a NoSQL warrior ready to fight for you.

Who will be the winner? Who would you choose for your next data battle? Tell me in the comments!

References of fighters:

MongoDB
Apache Cassandra
Redis
Neo4j
Couchbase
Amazon DynamoDB
Elasticsearch
Apache HBase

Need more tech insights?

Check out my GitHub repo and my LinkedIn page. Slides here.
Do you want to buy me a coffee to elevate my energy? You can do it here.

Happy coding!

Test Python Code Like a Pro with Poetry, Tox, Nox and CI/CD

Wallace Espindola — Mon, 06 Jan 2025 17:59:19 +0000

Hey there!

Got a Python project and need to make sure it works on every version of Python out there? Trust me, that can be a HUGE headache. But don't worry, I’ve got your back. In this guide, I’ll show you how to use Tox, Nox and CI/CD, awesome tools, to test your code across multiple Python versions.

And guess what? It’s easier than you think.

By the time you’re done reading this, you’ll be running tests like a pro across Python 3.8 to 3.13. We’ll keep things simple, fun, and totally actionable. Sounds good? Let’s dive in.

Why Should You Even Care About Multi-Version Testing?

Picture this: You write some cool Python code, and it works on your computer. But then, BAM! A user emails you, saying it breaks on Python 3.9. You try it, and sure enough, something’s off.

Why?

Because Python’s got all these versions, and each one has its quirks. If you don’t test your code on multiple versions, you’re flying blind.

But the GOOD NEWS is, you don’t have to manually install a bunch of Python versions and run tests on each. That’s where Tox and Nox swoop in like superheroes. And CI/CD standout, of course.

What Are Tox, Nox and CI/CD?

Let’s break it down:

Tox: Think of it as a robot that tests your code in different Python environments. It’s super organized and follows your instructions from a simple tox.ini file. You tell Tox what to do, and it does it.
Nox: It’s like Tox, but cooler in some ways. Why? Because instead of a config file, you get to write a Python script (noxfile.py). Want to add custom logic or conditions? Nox has your back.
CI/CD: Imagine an automated assembly line for your code. Continuous Integration (CI) ensures that every change you make is automatically built and tested, catching issues early. Continuous Deployment (CD) takes it a step further by automatically deploying your tested code to production or staging environments. This seamless pipeline ensures that your software is always in a deployable state, reduces manual intervention, and accelerates the delivery of new features and fixes.

So which one’s better? Honestly, it depends. If you like things neat and straightforward, go with Tox. If you’re the creative type and love flexibility, Nox is your jam. And if you want to automate your entire development workflow, from testing to deployment, CI/CD is the way to go.

Let’s Build Something Cool

Here’s the deal:

We’re gonna create a mini project with two simple functions:

Add two numbers.
Subtract one number from another.

We’ll write some tests to make sure they work, and then we’ll use Tox, Nox and CI/CD to test them on Python versions from 3.8 to 3.13.

Sounds fun, right?

Here’s the file structure we’re working with:

tox-nox-python-tests/
├── .github
│   └── workflows
│      └── python-tests-matrix.yml
├── tox_nox_python_test_automation/
│   ├── __init__.py
│   ├── main.py
│   └── calculator.py
├── tests/
│   ├── __init__.py
│   └── test_calculator.py
├── .gitlab-ci.yml
├── pyproject.toml
├── tox.ini
├── noxfile.py
└── README.md

Step 1: Write the Code

Here’s our calculator.py:

def add(a, b):
    """Returns the sum of two numbers."""
    return a + b

def subtract(a, b):
    """Returns the difference of two numbers."""
    return a - b

Simple, right? Let’s keep it that way.

Step 2: Write Some Tests

Time to make sure our code works. Here’s our test_calculator.py:

import pytest
from tox_nox_python_test_automation.calculator import add, subtract

@pytest.mark.parametrize("a, b, expected", [
    (1, 2, 3),
    (-1, 1, 0),
    (0, 0, 0),
])
def test_add(a, b, expected):
    assert add(a, b) == expected

@pytest.mark.parametrize("a, b, expected", [
    (5, 3, 2),
    (10, 5, 5),
    (-1, -1, 0),
])

def test_subtract(a, b, expected):
    assert subtract(a, b) == expected

We’re using pytest, a testing tool that’s basically the MVP of Python testing. If you’ve never used it, don’t sweat it, it’s super easy to pick up.

Step 3: Manage Dependencies with Poetry

Okay, so how do we make sure everyone working on this project uses the same dependencies? We use Poetry, which is like a supercharged requirements.txt file.

Here’s what our pyproject.toml looks like:

[tool.poetry]
name = "tox_nox_python_tests"
version = "0.1.0"
description = "Testing with multiple Python versions using Tox, Nox and CI/CD."
authors = ["Wallace Espindola <wallace.espindola@gmail.com>"]
license = "MIT"

[tool.poetry.dependencies]
python = "^3.8"
pytest = "^8.3"
nox = "^2024.10.9"
tox = "^4.23.2"

To install everything, just run:

poetry install

Step 4: Run unit tests with Pytest

You can run the basic unit tests this way:

poetry run pytest --verbose

And will see a standard unit test running output.

Step 5: Test with Tox

Tox is all about automation. Here’s our tox.ini:

[tox]
envlist = py38, py39, py310, py311, py312, py313

[testenv]
allowlist_externals = poetry
commands_pre =
    poetry install --no-interaction --no-root
commands =
    poetry run pytest

Run Tox with one command:

poetry run tox

And boom! Tox will test your code across every version of Python listed. See and example output here:

Step 6: Test with Nox

Want more control? Nox lets you get creative. Here’s our noxfile.py:

import nox

@nox.session(python=["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"])
def tests(session):
    session.install("poetry")
    session.run("poetry", "install", "--no-interaction", "--no-root")
    session.run("pytest")

Run Nox with:

poetry run nox

Now you’ve got full flexibility to add logic, skip environments, or do whatever else you need. See and example output here:

Step 7: Automate with CI/CD

Why stop at local testing? Let’s set this up to run automatically on GitHub Actions and GitLab CI/CD.

GitHub Actions

Here’s a workflow file .github/workflows/python-tests.yml for Tox or Nox:

name: Python Tests Tox (or Nox)

on:
  push:
    branches: [ main, develop ]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: ${PYTHON_VERSION}

      - name: Install Poetry
        run: pip install poetry

      - name: Install Dependencies
        run: poetry install --no-interaction --no-root

      - name: Run Tests with Tox (or Nox)
        run: poetry run tox (or nox)

And another example for parallel matrix (pure CI/CD):

name: Python Tests with Matrix

on:
  push:
    branches: [ main, develop ]

jobs:
  test:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        python-version: [3.8, 3.9, 3.10, 3.11, 3.12, 3.13]

    steps:
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install Poetry
        run: pip install poetry

      - name: Install Dependencies
        run: poetry install --no-interaction --no-root

      - name: Run Tests with Pytest
        run: poetry run pytest --verbose

GitLab CI/CD

Here’s an example .gitlab-ci.yml for Tox or Nox:

stages:
  - test

tox_tests:
  stage: test
  image: python:${PYTHON_VERSION}
  variables:
    PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
  before_script:
    - pip install poetry
    - poetry install --no-interaction --no-root
  script:
    - poetry run tox (or nox)
  cache:
    paths:
      - .cache/pip/

And here’s an example for a parallel matrix tests (pure CI/CD):

stages:
  - test

matrix_tests:
  stage: test
  image: python:${PYTHON_VERSION}
  variables:
    PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
  before_script:
    - pip install poetry
    - poetry install --no-interaction --no-root
  script:
    - poetry run pytest --verbose
  cache:
    paths:
      - .cache/pip/
  parallel:
    matrix:
      - PYTHON_VERSION: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]

Let’s Wrap It Up

You did it! You now know how to test Python code across multiple versions using Tox, Nox, and Poetry.

Here’s what to remember:

Tox is your go-to for simple, automated testing.
Nox gives you the freedom to customize.
Poetry makes managing dependencies a breeze.
CI/CD ensures your tests run automatically.

References, of course

This project uses Tox, Nox, Poetry, and Pytest for test automation. For detailed documentation, take a look at:

Tox Documentation
Nox Documentation
Poetry Documentation
Pytest Documentation
GitLab CI/CD Documentation.
GitHub Actions CI/CD Documentation.

Need the full code and examples?

Check out the repo on GitHub: tox-nox-python-tests.

For other interesting subjects and technical discussions, check my LinkedIn page.

For some slides, take a look here.

Do you want to buy me a coffee to elevate my energy? You can do it here.

Now go out there and make your Python projects bulletproof! 🚀

Python Multithreading: Unlocking Concurrency for Better Performance

Wallace Espindola — Fri, 17 May 2024 21:23:43 +0000

Hey, let's dive into the world of Python multithreading! Whether you're an intermediate or advanced developer, or maybe even coming from another programming language, mastering this will boost your skills and your applications' performance. Ready to tackle it? Let's go!

Why Multithreading?

Multithreading can seem daunting, but it's a game-changer for performing multiple operations at once, especially when dealing with I/O-bound or high-latency operations. Think of it as hiring more workers for your shop when it gets busy. Each worker handles a different customer simultaneously, speeding up service. That's what threads do for your programs.

Setting Up Your Workshop – The Basics

First, let's set up our environment. Python provides a built-in module for threading called threading. Here’s how you get started:

import threading

def do_something():
    print("Look, I'm running in a thread!")

# Create a thread
thread = threading.Thread(target=do_something)

# Start the thread
thread.start()

# Wait for the thread to complete
thread.join()

print("All done!")

This basic setup creates and starts a thread that runs a function. The join() method is crucial, as it tells Python to wait for the thread to complete before moving on.

Best Practices for Healthy Threading

1. Keep the GIL in mind: Python’s Global Interpreter Lock (GIL) allows only one thread to execute at a time in a single process, which can be a bottleneck. But don’t worry! This mainly affects CPU-bound tasks. For I/O-bound tasks, threading can still be very beneficial.

2. Use ThreadPoolExecutor: This is a fantastic tool from the concurrent.futures module. It simplifies managing a pool of threads and handling tasks. Here’s how you can use it:

from concurrent.futures import ThreadPoolExecutor

def task(n):
    print(f"Processing {n}")

with ThreadPoolExecutor(max_workers=3) as executor:
    results = executor.map(task, range(1, 4))

This example creates a pool of three threads, each processing a part of a range of numbers. Using ThreadPoolExecutor not only simplifies thread management but also handles task distribution and collection elegantly.

3. Don’t share state if possible: Sharing data between threads can lead to data corruption and other nasty bugs if not handled properly. If you must share data, use locks:

from threading import Lock

lock = Lock()
shared_resource = []

def safely_add(item):
    with lock:
        shared_resource.append(item)

threads = [
    threading.Thread(target=safely_add, args=(i,)) for i in range(10)
]

for thread in threads:
    thread.start()

for thread in threads:
    thread.join()

print(shared_resource)

Each thread in this example safely adds an item to a shared list, thanks to the lock that ensures only one thread modifies the list at a time.

When Not to Use Multithreading

Gotcha! While multithreading is powerful, it’s not a silver bullet. For CPU-bound tasks that require heavy computation and little I/O, consider using multiprocessing or asynchronous programming (like asyncio) instead. These tools can bypass the GIL and truly run code in parallel, maximizing CPU usage.

Debugging Multithreaded Applications

Yes, it can be tricky! Here are a couple of quick tips:

- Logging is your friend: Use Python’s logging module to help track down what each thread is doing.

- Use debuggers that understand threads: Tools like IntelliJ, PyCharm and VS Code have debuggers that can handle threading well.

Wrapping Up with a Real-World Example

Imagine you're building a web scraper that downloads content from multiple URLs. Multithreading can significantly speed up this process since network requests are I/O-bound. Here’s a sketch of how you might set this up:

from concurrent.futures import ThreadPoolExecutor
import requests

def fetch(url):
    response = requests.get(url)
    return response.text

urls = ["https://example.com", "https://example.org", "https://example.net"]

with ThreadPoolExecutor(max_workers=5) as executor:
    results = list(executor.map(fetch, urls))

print("Fetched the content of all URLs!")

This example fetches data from multiple URLs in parallel. Each thread handles downloading the content from one URL, which is typically a slow operation due to network latency.

Keep Learning and Experimenting

Multithreading is a vast topic, and mastering it can take some time. Keep experimenting with different scenarios and tweak your approach as you learn more about Python's capabilities and limitations. Remember, the more you practice, the better you’ll get!

Now, why not try implementing a multithreaded solution in your next project? Dive in, break things, fix them, and learn. Go ahead!

You may check this python-multithreading project I created on GitHub with examples and tests, and also, for other interesting technical discussions and subjects, check my LinkedIn page. Happy coding!

Best API Performance Testing Tools in 2024

Wallace Espindola — Wed, 31 Jan 2024 19:54:00 +0000

In the dynamic world of software development, ensuring the performance and reliability of APIs is primordial. As applications grow in complexity and scale, developers and QA engineers turn to a variety of processes and tools to rigorously test API performance under various conditions. This article explores some of the most common processes and tools in the market, offering insights into their capabilities, ease of use, and how they can be integrated into development workflows.

First of all, how and what to test

Testing your API performance involves several key steps and tools designed to evaluate its responsiveness, throughput, stability under load, and more. Here's a structured approach to doing so:

Define Performance Goals

Latency: Maximum acceptable time for the API to respond.
Throughput: Number of requests per second the API should handle.
Concurrency: Number of simultaneous connections the API must support.
Availability: Expected uptime percentage.

Understand Your API Workload

Identify the Most Common Operations: Pinpoint the most frequently used endpoints and functions within your API.
Determine Peak Usage Times and Patterns: Analyze when your API experiences the highest traffic and the nature of this usage.
Analyze User Demographics and Geographies: Understand where your users are located and how geographic distribution affects API usage.
Identify Dependent Systems and Integrations: Know which external or internal systems interact with your API and how these dependencies affect performance.
Monitor Mobile vs. Desktop Usage: Be aware of the differences in API usage between mobile and desktop clients, as they can have different performance characteristics.

Set Up Your Testing Environment

Similarity to production: Ensure the testing environment closely mimics the production environment to get accurate results.
Environment isolation: Isolate the test environment to avoid impacting production systems.
Make it reliable: Avoid instable server environments where tests can be impacted by external variables.

Create Test Cases

Baseline Testing: Determine how the API performs under normal conditions.
Load Testing: Gradually increase the load on the API until it reaches the expected maximum.
Stress Testing: Push the API beyond its expected maximum until it breaks to understand its upper limits.
Spike Testing: Test the API with sudden increases in load to simulate real-world spikes in usage.
Endurance Testing: Evaluate how the API performs under a sustained load over a long period.

Monitor and Measure Performance

Response Time: Time taken for the API to respond.
Throughput: Number of requests handled per second.
Error Rate: Percentage of failed requests.
Resource Utilization: CPU, memory, disk, and network usage.

Analyze Results and Optimize

Identify bottlenecks and limitations in your API and infrastructure.
Optimize code, database queries, server configuration, etc.
Repeat tests to measure improvements.

Continuous Monitoring and Testing

Integrate performance testing into your continuous integration/continuous deployment (CI/CD) pipeline.
Regularly monitor API performance in production to detect and address new performance issues.

Second, choose the right tools

Here are some comprehensive and widely used performance and load testing tools for APIs to excel in 2024.

JMeter
Apache JMeter stands out for its robustness and flexibility, catering to a wide range of testing needs from load testing to stress and spike testing. While its graphical user interface (GUI) enhances accessibility, making it suitable for both technical and non-technical users, mastering its advanced features can require a deeper dive. JMeter's ability to simulate heavy loads across multiple machines makes it a go-to for large-scale performance testing. The tool enjoys strong community support, ensuring an abundance of resources and documentation for users.

Postman
Originally designed for API development, Postman has evolved to include performance testing features. It's celebrated for its user-friendly interface, appealing to developers and testers at all levels. Although its performance testing capabilities are more rudimentary compared to dedicated tools, it excels in API development and testing workflows. Postman's community is vast, providing extensive support and resources.

Locust
Locust differentiates itself with a code-driven approach, allowing testers to write tests in Python to simulate millions of users. This flexibility is particularly appealing to developers comfortable with scripting. Its distributed testing capability is robust, backed by real-time monitoring through a web UI. While detailed analysis may require external tools, Locust's scalability is a key strength. The community around Locust is actively growing, contributing to its development and support.

Gatling
Gatling impresses with high-performance tests and detailed reporting, thanks to its Scala-based Domain-Specific Language (DSL). Although learning Scala might be a barrier for some, the investment pays off with Gatling's efficient and scalable performance testing capabilities. Its comprehensive reports offer in-depth insights into application behavior under load. Gatling has a supportive community, providing ample learning resources.

Grafana k6
Grafana k6 is a modern tool designed with developers in mind, offering a JavaScript environment for scripting tests. It's optimized for minimal resource consumption while delivering powerful performance testing features. k6's integration with Grafana for advanced data visualization and analysis stands out, providing clear insights into performance metrics. The tool's community is rapidly expanding, offering strong support and documentation.

Enhancing Performance Testing with Monitoring and CI/CD Integration

Beyond testing tools, monitoring solutions like New Relic, Datadog, Prometheus, Splunk, and Grafana play a critical role in real-time performance monitoring, offering insights that guide optimization efforts.

Incorporating these tests into CI/CD pipelines automates performance testing, ensuring continuous delivery of high-quality software. Tools like Jenkins, GitLab CI, CircleCI, Travis CI, GitHub Actions, JetBrains TeamCity, and Bamboo integrate seamlessly with testing workflows, enabling automated builds, tests, and deployments.

Conclusions

The choice of API performance testing tools and processes depends on specific project requirements, team expertise, and the scale of testing needed. Whether it's the comprehensive testing capabilities of JMeter and Gatling, the developer-friendly nature of Locust and k6, or the API development integration of Postman, each tool offers unique strengths. Coupled with effective monitoring and CI/CD integration, these tools empower teams to deliver robust, high-performing applications that stand the test of user expectations and system demands.

For some interesting performance tests topics, you may take a look on my GitHub repo, and/or on my LinkedIn profile.