DEV Community: Nicolas Bonnici

Say hello to GoREST v0.3

Nicolas Bonnici — Thu, 04 Dec 2025 21:12:46 +0000

What's New Under the Hood?

Plugins

Plugins for all, plugins everywhere, plugins for everything. Whether
custom or core features there's a plugin for that. Want to create
your own? Let's go!

With the release of v0.3, every feature of the GoREST core is now optional and fully configurable.

Each plugin has its own configuration in the gorest.yaml plugins
section.

Here's the list of the first available core plugins:

Authentication with JWT support
Content-type negotiation
Logger
Rate limiter
Request identifier
Security
CORS management
Health check
Benchmark

If you need something specific, you can build your own plugin. Here's an
example of a simple customplugin that calculates request duration time
and adds it to the response headers.

package customplugin

import (
    "fmt"
    "time"

    "github.com/gofiber/fiber/v2"
    "github.com/nicolasbonnici/gorest/plugin"
)

// TimingPlugin adds request execution time headers to all responses
type TimingPlugin struct {
    enabled bool
}

// NewTimingPlugin creates a new timing plugin instance
func NewTimingPlugin() plugin.Plugin {
    return &TimingPlugin{enabled: true}
}

func (p *TimingPlugin) Name() string {
    return "timing"
}

func (p *TimingPlugin) Initialize(config map[string]interface{}) error {
    if enabled, ok := config["enabled"].(bool); ok {
        p.enabled = enabled
    }
    return nil
}

func (p *TimingPlugin) Handler() fiber.Handler {
    return func(c *fiber.Ctx) error {
        if !p.enabled {
            return c.Next()
        }

        start := time.Now()

        // Process request
        err := c.Next()

        // Calculate duration
        duration := time.Since(start)

        // Add timing headers
        c.Set("X-Response-Time", fmt.Sprintf("%v", duration))
        c.Set("X-Response-Time-Ms", fmt.Sprintf("%.2f", float64(duration.Microseconds())/1000.0))

        return err
    }
}

Main advantages of this new plugin system:

Plugins can be applied globally across the API or restricted to specific endpoints and methods.
Plugins can create new API endpoints.
Plugin can also provide CLI commands.

Plugins vs Existing Hook System

Plugins are designed for generic API functionalities, while the hook
system lets you implement business-specific logic. You can share GoREST plugins publicly, but hooks are tied to your business objects and logic.

Flexibility Is Key

Every API has its own needs and specificities. The GoREST core focuses only on essential API behaviors ** resource interactions ** while everything else can be modular.

Performance

Performance matters. Does all this flexibility come with a cost?
Absolutely not. Modularity increases performance by removing unused plugins that were previously tightly coupled to the core.

Benchmark

GoREST includes a built‑in benchmarking plugin. Simply run:

make benchmark

Below is the output, executed locally on my laptop with the following
specs, while running Debian, GNOME Shell, and several applications like a browser, IDE, and Insomnia:

11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
16GB RAM

=========================================
      API Performance Benchmark
=========================================

[INFO] Setting up benchmark table...
[INFO] Generating test data...
[INFO] Generating models and resources...
[INFO] Building API server...
[INFO] Starting API server...
[INFO] Waiting for server to be ready...
[INFO] Server is ready

=========================================
       Running Benchmarks
=========================================

Benchmarking GET /benchmarkitems?limit=10
─────────────────────────────────────────────────────────────────
  Concurrency: 1   | RPS:       1 | p50: 871.26µs | p95: 1.433519ms | p99: 1.433519ms | Errors: 0 | Total: 5
  Concurrency: 10  | RPS:      10 | p50: 821.151µs | p95: 1.289257ms | p99: 1.597766ms | Errors: 0 | Total: 50
  Concurrency: 50  | RPS:      50 | p50: 817.202µs | p95: 1.569706ms | p99: 2.275461ms | Errors: 0 | Total: 250

Benchmarking GET /benchmarkitems?limit=100
─────────────────────────────────────────────────────────────────
  Concurrency: 1   | RPS:       1 | p50: 1.965903ms | p95: 2.383311ms | p99: 2.383311ms | Errors: 0 | Total: 5
  Concurrency: 10  | RPS:      10 | p50: 1.872685ms | p95: 2.449289ms | p99: 2.95571ms | Errors: 0 | Total: 50
  Concurrency: 50  | RPS:      50 | p50: 1.706943ms | p95: 2.682877ms | p99: 5.437662ms | Errors: 0 | Total: 250

Benchmarking GET /benchmarkitems?limit=1000
─────────────────────────────────────────────────────────────────
  Concurrency: 1   | RPS:       1 | p50: 11.371178ms | p95: 14.578225ms | p99: 14.578225ms | Errors: 0 | Total: 5
  Concurrency: 10  | RPS:      10 | p50: 11.772652ms | p95: 13.573469ms | p99: 14.813448ms | Errors: 0 | Total: 50
  Concurrency: 50  | RPS:      50 | p50: 11.403417ms | p95: 13.064978ms | p99: 16.583982ms | Errors: 0 | Total: 250

=========================================
       Benchmark Complete
=========================================

[INFO] Cleaning up...
[INFO] Restoring original schema...
[INFO] Cleanup complete

Performance Summary

Small Page Size (limit=10)

Concurrency 1: p50 871µs, p95 1.4ms, p99 1.4ms
Concurrency 10: p50 821µs, p95 1.3ms, p99 1.6ms
Concurrency 50: p50 817µs, p95 1.6ms, p99 2.3ms

Excellent --- sub‑millisecond median response times even under load.

Medium Page Size (limit=100)

Concurrency 1: p50 1.97ms, p95 2.4ms, p99 2.4ms
Concurrency 10: p50 1.87ms, p95 2.4ms, p99 3.0ms
Concurrency 50: p50 1.71ms, p95 2.7ms, p99 5.4ms

Very good --- under 2ms median for 100 items.

Large Page Size (limit=1000)

Concurrency 1: p50 11.4ms, p95 14.6ms, p99 14.6ms
Concurrency 10: p50 11.8ms, p95 13.6ms, p99 14.8ms
Concurrency 50: p50 11.4ms, p95 13.1ms, p99 16.6ms

Good --- around 11ms for 1000 items is reasonable.

Key Observations

Zero errors across all tests
Excellent scalability
Consistently low latency
Strong connection pooling
Linear performance growth

Security

Thanks to the core security plugin, building a REST API with GoREST
gives you a highly secure framework.

We performed a security audit and penetration test using Claude AI from
Anthropic. It checked for OWASP Top 10 vulnerabilities, test coverage,
attack surface analysis, pentesting, and stress testing.

Standard Requirement GoREST Status

OWASP ASVS L2 Security headers ✅ Complete
OWASP ASVS L2 TRACE disabled ✅ Complete
OWASP ASVS L2 Rate limiting ✅ Complete
OWASP ASVS L2 JWT security ✅ Complete
OWASP ASVS L2 SQL injection prevention ✅ Complete
OWASP ASVS L2 XSS prevention ✅ Complete
OWASP ASVS L2 CSRF protection ✅ N/A (API)
CIS Benchmarks Secure headers ✅ Complete
CIS Benchmarks Method whitelisting ✅ Complete
Mozilla Observatory Security grade ✅ A+

Compliance Level: OWASP ASVS Level 2 (Standard Web Applications)

Results

Security Score: 10/10
Vulnerabilities: 0
Tests Passed: 27/27
Production Status: ✅ Approved

GoREST is now production‑ready with industry‑leading security.

What's Coming Next?

Roadmap to the v1 milestone:

Plugins repository
OAuth2 support
gRPC plugin
RBAC plugin
Development profiler plugin
Complete documentation

Contribute

Feel free to create new plugins and contribute to the library. Even a
single ⭐ can motivate continued development toward the v1 milestone.

Let's create a production grade REST API from an existing database or from scratch using Go in seconds

Nicolas Bonnici — Sat, 15 Nov 2025 21:15:59 +0000

Nicolas Bonnici

Nov 15 '25

How to create a REST API in seconds with Go

#howto #api #rest #go

Comments

8 min read

[Boost]

Nicolas Bonnici — Sat, 15 Nov 2025 18:17:52 +0000

Nicolas Bonnici

Nov 15 '25

How to create a REST API in seconds with Go

#howto #api #rest #go

Comments

8 min read

How to create a REST API in seconds with Go

Nicolas Bonnici — Sat, 15 Nov 2025 13:15:59 +0000

Quick Start (30 Seconds to Running API)

# Clone and setup
git clone https://github.com/nicolasbonnici/gorest.git
cd examples/basic-api
cp .env.dist .env
# Start database and generate API
docker compose up -d 
make generate
make run

Your API is now running at http://localhost:3000 with interactive documentation at http://localhost:3000/openapi.

More informations on the basic-api example README.

Start with Your Data Model

Define your business objects and their relationships using standard SQL:

-- schema.sql
CREATE TABLE users (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    firstname TEXT NOT NULL,
    lastname TEXT NOT NULL,
    email TEXT UNIQUE NOT NULL,
    password TEXT,
    created_at TIMESTAMP(0) WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE todo (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id UUID REFERENCES users(id),
    title TEXT NOT NULL,
    content TEXT NOT NULL,
    created_at TIMESTAMP(0) WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
);

Two tables with a foreign key relationship. That's our entire starting point in this post.

What Gets Generated Automatically

Complete REST API Endpoints

For Users:

POST /users - Create user
GET /users - List users (with pagination, filtering, ordering)
GET /users/{id} - Get user
PUT /users/{id} - Update user
DELETE /users/{id} - Delete user

For Todos:

POST /todos - Create todo
GET /todos - List todos
GET /todos/{id} - Get todo
PUT /todos/{id} - Update todo
DELETE /todos/{id} - Delete todo

Code Structure

gorest/
├── cmd/
│   └── server/
│       └── main.go      # Application entry point
├── internal/
│   ├── models/          # Database models
│   │   ├── user.go
│   │   └── todo.go
│   ├── dto/             # Data Transfer Objects
│   │   ├── user_dto.go
│   │   └── todo_dto.go
│   ├── hooks/           # Your business logic goes here
│   │   ├── user.go
│   │   └── todo.go
│   └── resources/       # REST resource configurations
│       ├── user.go
│       └── todo.go
└── openapi.json         # API documentation

Key point: Generated files (models, DTOs, resources) are recreated when you run generate. Your custom business logic goes in hooks/, which are never overwritten.

Interactive API Documentation

Access at http://localhost:3000/openapi:

Browse all endpoints with request/response schemas
Test API calls directly from browser
Copy code examples in multiple languages
JSON spec available at /openapi.json for Postman, Insomnia, or client SDK generation

Authentication & Security

Built-in JWT Authentication

Registration and login endpoints are automatically generated:

# Register a new user
curl -X POST http://localhost:3000/register \
  -H "Content-Type: application/json" \
  -d '{
    "firstname": "John",
    "lastname": "Doe",
    "email": "john@example.com",
    "password": "secure123"
  }'

# Login and get JWT token
curl -X POST http://localhost:3000/login \
  -H "Content-Type: application/json" \
  -d '{
    "email": "john@example.com",
    "password": "secure123"
  }'

Response:

{
  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "user": {
    "id": "abc-123",
    "firstname": "John",
    "lastname": "Doe",
    "email": "john@example.com"
  }
}

All protected endpoints automatically require the JWT token:

curl -H "Authorization: Bearer YOUR_TOKEN" \
     http://localhost:3000/todos

Securing Sensitive Data with DTOs

Control exactly what data is exposed through Data Transfer Objects:

// Database model - contains ALL fields
type Todo struct {
    gorest.Model
    ID          string    `json:"id" gorm:"primaryKey"`
    Title       string    `json:"title"`
    Content     string    `json:"content"`
    Password    string    `json:"-"` // Never serialized
    UserID      string    `json:"user_id"`
    CreatedAt   time.Time `json:"created_at"`
}

// What clients see when reading
type TodoReadDTO struct {
    ID        string    `json:"id"`
    Title     string    `json:"title"`
    Content   string    `json:"content"`
    UserID    string    `json:"user_id"`
    CreatedAt time.Time `json:"created_at"`
    // Password is completely absent - type-safe
}

// What clients can send when creating
type TodoWriteDTO struct {
    Title    string `json:"title" validate:"required,min=3"`
    Content  string `json:"content"`
    Password string `json:"password" validate:"required,min=8"`
    // UserID is absent - auto-populated server-side
}

Benefits:

Type-safe: Compiler prevents exposing sensitive fields
Flexible: Different DTOs per operation (list, detail, create, update)
Self-documenting: API contract is explicit
Zero overhead: No runtime transformation

Advanced Querying Out of the Box

Filtering

GET /todos?title[like]=meeting              # Pattern matching
GET /todos?priority[gte]=5                  # Greater than or equal
GET /todos?status[]=active&status[]=pending # Multiple values (OR)

Ordering

GET /todos?order[priority]=desc&order[created_at]=asc

Pagination

GET /todos?page=2&limit=20

Relationship Expansion

# Automatically expand foreign keys
GET /todos?expand=user

# Response includes nested user data
{
  "id": "todo-123",
  "title": "Buy groceries",
  "user": {
    "id": "user-456",
    "firstname": "John",
    "email": "john@example.com"
  }
}

Adding Business Logic With Hooks

Hooks execute at key moments in the request lifecycle without modifying generated code.

Available Hooks

StateProcessor - Validate/transform data before Create/Update/Delete
SQLQueryOverride - Replace default queries with custom SQL
SQLQueryListener - Intercept queries before/after execution
Authorize - Add authentication/authorization logic

Example 1: Validation and Auto-Population

func (h *TodoHooks) StateProcessor(ctx context.Context, operation hooks.Operation, id any, todo *models.Todo) error {
    switch operation {
    case hooks.OperationCreate:
        // Validate
        if todo.Title == "" {
            return fmt.Errorf("title is required")
        }
        if len(todo.Title) < 3 {
            return fmt.Errorf("title must be at least 3 characters")
        }

        // Auto-populate user_id from JWT authentication
        // This happens server-side - clients cannot spoof it
        if userID := ctx.Value("user_id"); userID != nil {
            todo.UserID = userID.(string)
        }
    }
    return nil
}

Result: Users can't assign todos to each other, and data is validated before reaching the database.

Example 2: Custom Query Logic

func (h *TodoHooks) SQLQueryOverride(ctx context.Context, operation hooks.Operation, id any) (*sqlbuilder.SelectQuery, error) {
    if operation == hooks.OperationList {
        // Only show the authenticated user's incomplete todos
        userID := ctx.Value("user_id")

        query := sqlbuilder.
            Select("id", "title", "content", "user_id").
            From("todo").
            Where("completed = FALSE AND user_id = ?", userID)

        return query, nil
    }
    return nil, nil
}

Result: Every GET /todos automatically filters to the authenticated user's incomplete tasks.

Example 3: Authorization Control

func (h *TodoHooks) Authorize(ctx context.Context, operation hooks.Operation, id any) error {
    userID := ctx.Value("user_id")
    if userID == nil {
        return fmt.Errorf("authentication required")
    }

    // Custom permission checks
    if operation == hooks.OperationDelete {
        // Only admins can delete
        if !isAdmin(userID) {
            return fmt.Errorf("insufficient permissions")
        }
    }

    return nil
}

Why Go and GoREST?

Language and Framework Choice

To build a REST API, you need both a language and framework. We chose Go with GoREST based on these technical advantages:

Performance Benchmarks

Response times: Sub-millisecond under normal load
Concurrent requests: Handle thousands simultaneously via goroutines
Startup time: ~1ms (vs. 1-2 seconds for interpreted languages)
Compilation: Direct to machine code with no runtime overhead

Development Experience

Clean syntax and strong typing reduce bugs
Built-in testing frameworks and profiling tools
Excellent standard library for HTTP, JSON, and database operations
Simple concurrency with goroutines and channels
Code remains maintainable months later

Production Benefits

Single binary deployment (detailed in next section)
Cross-platform compilation built-in
No dependency version conflicts
Trivial CI/CD pipelines

GoREST Framework Goals

GoREST focuses on eliminating boilerplate while maintaining flexibility:

Code Generation - Define your schema once, get a complete API
Consistency - Predictable routes, uniform behavior, automatic validation
Extensibility - Add business logic via hooks without modifying generated code
Performance - Leverage Go's speed and efficiency
Database Flexibility - Works with PostgreSQL, MySQL, SQLite

Deployment: The Single Binary Advantage

One of Go's most powerful features is that your entire application compiles into a single, self-contained executable. This fundamentally changes how you deploy and manage applications.

Production Deployment Scenarios

GoREST deployment:

# Build once
make build

# Deploy
scp ./bin/gorest user@server:/opt/gorest/
./gorest

# That's it. No dependencies. No runtime.

1. Bare Metal / VM with systemd

[Unit]
Description=GoREST API Server
After=network.target postgresql.service

[Service]
Type=simple
User=api
WorkingDirectory=/opt/gorest
EnvironmentFile=/opt/gorest/.env
ExecStart=/opt/gorest/bin/gorest
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

sudo systemctl enable gorest
sudo systemctl start gorest

2. Docker (Multi-stage Build)

# Build stage
FROM golang:1.22 AS builder
WORKDIR /app
COPY . .
RUN make build

# Final stage - just the binary
FROM scratch
COPY --from=builder /app/bin/gorest /gorest
EXPOSE 3000
CMD ["/gorest"]

Result: 10-20MB Docker image (vs. 500MB+ for Node.js/Python apps)

3. Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gorest-api
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: gorest
        image: your-registry/gorest:latest
        ports:
        - containerPort: 3000
        resources:
          requests:
            memory: "20Mi"
            cpu: "100m"
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: db-secret
              key: url

Each pod starts in milliseconds and uses minimal resources.

4. AWS Lambda (Custom Runtime)

# Package as custom runtime
zip function.zip gorest
aws lambda create-function --runtime provided.al2

# Cold start: ~100ms (vs. seconds for interpreted languages)

5. Edge & Serverless Platforms

Deploy to Cloudflare Workers, Fly.io, or any edge platform. Small binary size and instant startup are ideal for serverless environments.

Cross-Platform Compilation

Build for any platform from any platform:

# Build for Linux from your Mac
GOOS=linux GOARCH=amd64 make build

# Build for Windows
GOOS=windows GOARCH=amd64 make build

# Build for ARM (Raspberry Pi, M1 Mac)
GOOS=linux GOARCH=arm64 make build

CI/CD Pipeline Example

# GitHub Actions
name: Deploy

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Build
        run: make build

      - name: Test
        run: ./bin/gorest --version

      - name: Deploy
        run: scp ./bin/gorest server:/app/

No dependency caching, no runtime installation, no complex build pipelines.

Version Management

# No runtime version conflicts
# Each binary is independent and portable
/opt/api-v1/gorest  # Old version
/opt/api-v2/gorest  # New version

# Instant rollback
ln -sf /opt/api-v1/gorest /usr/local/bin/gorest

# Instant upgrade
ln -sf /opt/api-v2/gorest /usr/local/bin/gorest

Why This Matters

Zero Dependencies: No runtime installation, no package managers, no dependency hell
Consistent Behavior: Same binary runs identically across dev, staging, and production
Reduced Attack Surface: No runtime vulnerabilities, minimal OS dependencies
Cost Efficiency: Smaller instances, cheaper hosting, more instances per server
Faster Deployment: Copy one file, run it
Simplified Operations: No dependency updates, no runtime patches

Complete Example: From Schema to Production

Let's walk through a complete workflow:

Step 1: Define Schema

CREATE TABLE todo (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    title TEXT NOT NULL,
    content TEXT NOT NULL,
    user_id UUID REFERENCES users(id),
    created_at TIMESTAMP(0) WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
);

Step 2: Generate and Build

make generate  # Generates models, DTOs, resources, OpenAPI docs
make build     # Compiles to single binary in ./bin/gorest

Step 3: Deploy

# Copy to server
scp ./bin/gorest user@server:/opt/api/

# Run
ssh user@server
cd /opt/api
./gorest

Step 4: Use Your API

# Register and login (see Authentication section for details)
TOKEN="your-jwt-token-here"

# Create todo
curl -X POST http://your-server:3000/todos \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"title":"Buy groceries","content":"Milk, eggs, bread"}'

# List todos with filtering and ordering
curl -H "Authorization: Bearer $TOKEN" \
     "http://your-server:3000/todos?title[like]=groceries&order[created_at]=desc"

# Get specific todo
curl -H "Authorization: Bearer $TOKEN" \
     "http://your-server:3000/todos/{id}"

# Update todo
curl -X PUT http://your-server:3000/todos/{id} \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"title":"Buy groceries ASAP"}'

# Delete todo
curl -X DELETE http://your-server:3000/todos/{id} \
  -H "Authorization: Bearer $TOKEN"

View interactive documentation at http://your-server:3000/openapi.

Getting Started

Ready to try GoREST?

View the basic-api example

Explore the full source code:

GitHub Repository

Found this helpful? Please star the project to show your support!

Summary

GoREST transforms your database schema into a production-ready REST API compiled as a single executable binary.

What you get:

Complete CRUD operations for all tables
JWT authentication with register/login endpoints
Advanced filtering, ordering, pagination, and relationship expansion
Interactive OpenAPI documentation
Type-safe DTOs for security and flexibility
Extensible hooks system for business logic
Production-ready error handling and validation

Deployment benefits:

Single self-contained binary with no dependencies
Deploy anywhere: bare metal, Docker, Kubernetes, serverless, edge
10-20MB total size with sub-millisecond response times
Instant startup and minimal resource footprint
Simple CI/CD and version management

Your database schema becomes a production-grade REST API. Define your data model, run make generate && make build, and ship anywhere.

Contribute to GoREST

All contributions welcome - code, documentation, ideas, suggestions:

Thanks for reading! 🚀

[Boost]

Nicolas Bonnici — Wed, 12 Nov 2025 23:13:52 +0000

GoREST is dead...

Nicolas Bonnici ・ Nov 12

#go #sql

GoREST is dead...

Nicolas Bonnici — Wed, 12 Nov 2025 21:31:11 +0000

This Is the End

Recently, I shared a project skeleton and a post that allows you to easily turn almost any relational database schema into a REST API.

It took me about a month to develop the first release, just a few hours at night when the kids were asleep and during weekends, since I’m currently working full-time.

The feedback was mixed. Some people liked it, others simply hated it. At some point on Reddit, people even downvoted the project without explaining why. Fortunately, some benevolent users also took the time to tell me what they liked or didn’t about the project.

The Need and the Vision Behind the First Release

My initial goal was to solve a problem I had: reusing an existing, functional relational data model and its data from a monolithic application, and turning it into a headless architecture, where the backend is a REST API, and the resources are consumed by both a web app and an admin portal using Nuxt UI and Flutter respectively.

After benchmarking the first release candidate, I was impressed by the performance and decided to share the code under an MIT license for anyone facing the same challenge.

I’m not a Go expert. I recently learned the basics through Grok, and this was my first real project involving more advanced use cases. I managed to build something genuinely useful, and overall, the performance was quite decent, so why not share it?

So, Why Kill It So Fast?

Programming languages are often more than mere instruction sets for machines. They embody implicit rules, conventions, structural philosophies, and specific subtleties. Go is no exception.

The first and most important issue with GoREST v0.1 wasn’t specific to Go: separation of concerns. Nobody wants to mix their business logic into a skeleton project unless that skeleton is meant to evolve directly into their final product.

Then there’s maintainability. Having a boilerplate that requires regular updates quickly becomes a burden and makes long-term maintenance painful.

And finally, flexibility. I wanted to offer multiple ways to use and implement GoREST, either as a library integrated into an existing project or as the foundation of a brand-new one. There are thousands of ways to code business logic, and I don’t want to impose just one.

So, there’s no reason to maintain this skeleton anymore.

Let’s cut off its head (no headless architecture related nerdy joke intended, literally).

RIP GoREST v0.1.0

Long live GoREST v0.2.0!

Yup, from the fresh ashes of GoREST v0.1 comes a complete rewrite as a library and toolbox. It aims to help you quickly scaffold your business objects and write your business logic with ease.

A new release marks a new chapter for GoREST. Yes, I know this is a total breaking change and should be a major version, but since no one is using it yet except me, and we’re still far from version v1.0.0, there’s no harm in making it a minor upgrade.

In my opinion, this new v0.2 release is a healthy foundation to iterate from. It keeps the same features and benefits as the previous version but is now decoupled into a library of Go modules and useful commands.

Road to v1.0.0

Here’s what I want to accomplish before releasing the first stable, long-term support version:

Better performance
A fully modular middleware layer with install/uninstall hooks
100% test coverage (currently around 80%)
gRPC support
Support for more database engines
Scaffolded clients
Some kickass new features I haven’t even thought of yet...

Contribute

Wanna join the adventure and contribute to the project?

All kinds of contributions are welcome, even just ideas or suggestions.

Read the contributing section.

You can fork it and/or star the project if you like it.

Documentation

Full documention is available on pkg.go

Thanks for reading.

[Boost]

Nicolas Bonnici — Thu, 06 Nov 2025 09:41:25 +0000

GoREST turn any database to a production grade REST API

Nicolas Bonnici ・ Nov 5

#go #rest #sql #gorest

GoREST turn any database to a production grade REST API

Nicolas Bonnici — Wed, 05 Nov 2025 21:29:37 +0000

GoREST v0.1.0

🎉 Introducing GoREST: The First Open Source Release!

I'm very proud to announce the first release of GoREST (v0.1.0) - This project aim to easily transform any relational database to a production grade REST API and it's written in Go! After months of development, testing, and refinement, I'm proud to release it under the MIT license, making it completely free and open source for everyone.

🔗 Get it now on GitHub!

Tired of writing CRUD endpoints by hand? Copy-pasting Go structs? Spending hours on Swagger documentation? Reimplementing the same authentication flow for the hundredth time? GoREST solves all of that.

With its first release, GoREST offers a production-grade REST API generator that:

🔎 Auto-discovery of tables, relations, columns & types
🛠 Scaffold REST endpoints for each table
⚡ Type-safe generic CRUD operations with hooks system
🔐 Full DTO support with field-level control (dto tags)
🔑 JWT authentication with context-aware middleware
🎭 Hook layer to add your business logic onto your API resources
🌐 JSON-LD support with semantic web context (@context, @type, @id)
🔗 Automatic relation to IRI conversion (e.g., /users/{uuid})
🔍 Advanced filtering & ordering
📄 Page-based pagination with Hydra collections
👨🏻‍💻 DAL for PostgreSQL, MySQL and SQLite engines
🛡️ Production-grade error and process management
🐳 Docker support with multi-database testing
🧪 Full test coverage with automated testing
💚 Health check endpoint (/health)
📜 OpenAPI 3 spec generation

🎯 Why GoREST?

GoREST isn't just another CRUD generator. It's a complete solution for building production-ready REST APIs:

Feature	GoREST	Manual Implementation
Time to API	2 commands (~30 seconds)	Days/weeks
Schema Discovery	Auto-discovery of tables, relations, columns & types	Manual schema mapping
Database Support	PostgreSQL, MySQL, SQLite (DAL)	Pick one, code for one
Type Safety	Auto-generated models with type-safe CRUD	Manual struct definitions
REST Endpoints	Scaffold for each table	Write every endpoint
OpenAPI Docs	Auto-generated 3.0 spec	Manual Swagger annotations
Authentication	Built-in JWT + context-aware middleware	Implement from scratch
Field Auto-Population	Auto-populate from auth context	Manual implementation
DTOs	Auto-generated with field-level security (`dto` tags)	Manual for every endpoint
JSON-LD Support	Built-in semantic web (@context, @type, @id)	Complex manual implementation
Foreign Key IRIs	Automatic conversion (e.g., `/users/{uuid}`)	Manual hypermedia links
Filtering	Equality, comparison, text search, multiple fields	Hours of coding
Ordering	Multi-field sorting out of the box	Manual SQL ORDER BY logic
Pagination	Hydra collections with page-based navigation	Manual cursor/offset implementation
Hooks System	StateProcessor, SQLQueryOverride, Serializer	Custom middleware for each case
Production Features	Health checks, graceful shutdown, structured logging	One-by-one implementation
Error Handling	Production-grade error management	Custom error handling
Testing	Full test coverage with Docker support	Write all tests manually

📌 Real-World Example: Todo API in 60 Seconds

1. Start with Your Existing Database

No need to change your schema - GoREST works with what you have.

CREATE TABLE users (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    firstname TEXT NOT NULL,
    lastname TEXT NOT NULL,
    email TEXT UNIQUE NOT NULL,
    password TEXT,
    created_at TIMESTAMP(0) WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE todo (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id UUID REFERENCES users(id),
    title TEXT NOT NULL,
    content TEXT NOT NULL,
    created_at TIMESTAMP(0) WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
);

2. Generate Everything (Models, DTOs, Routes, OpenAPI)

git clone https://github.com/nicolasbonnici/gorest.git
cd gorest
cp .env.example .env
# Edit .env with your database connection

make generate     # Generates models, DTOs, resources, routes, OpenAPI spec
make run         # API is live!

That's it. Your API is running at http://localhost:3000 with:

✅ Full CRUD endpoints for todos and users
✅ OpenAPI documentation at /openapi.json
✅ Health check at /health
✅ JWT authentication ready to use
✅ Type-safe DTOs with automatic validation
✅ JSON-LD support for semantic web compatibility

You can find the API spec here:

Web version (using Scalar UI): http://localhost:3000/openapi
JSON format: http://localhost:3000/openapi.json

Generated Endpoints:

Method	Endpoint	Description
`GET`	`/todos`	List todos (with filtering, ordering, pagination)
`POST`	`/todos`	Create todo
`GET`	`/todos/:id`	Get specific todo
`PUT`	`/todos/:id`	Update todo
`DELETE`	`/todos/:id`	Delete todo
`GET`	`/users`	List users
`POST`	`/users`	Create user
`GET`	`/users/:id`	Get user
`PUT`	`/users/:id`	Update user
`DELETE`	`/users/:id`	Delete user

🔐 Security Done Right: DTOs & Field Control

Here's where GoREST shines. Ever had a client send user_id in a request to access someone else's data? Not anymore.

Automatic DTO Generation

GoREST generates three DTO types for every model:

CreateDTO - For POST requests (excludes auto-generated fields like id, created_at)
UpdateDTO - For PUT requests (excludes auto-generated and protected fields)
ResponseDTO - For GET responses (shows everything the client should see)

Field-Level Control with `dto` Tags

Control what clients can send and receive using model tags:

type Todo struct {
    Id        string     `json:"id" db:"id"`
    UserId    *string    `json:"user_id" db:"user_id" dto:"read"`  // 🔒 Read-only!
    Title     string     `json:"title" db:"title"`
    Content   string     `json:"content" db:"content"`
    CreatedAt *time.Time `json:"created_at" db:"created_at"`
}

Result: Clients cannot send user_id in POST/PUT requests. GoREST populates it server-side from JWT authentication:

// Client sends (TodoCreateDTO):
{
  "title": "Buy groceries",
  "content": "Milk, eggs, bread"
  // No user_id - field not in DTO
}

// Server populates user_id from JWT token (happens in hooks)
// Client receives (TodoDTO):
{
  "id": "abc-123",
  "user_id": "xyz-789",  // ✅ Populated server-side
  "title": "Buy groceries",
  "content": "Milk, eggs, bread",
  "created_at": "2024-01-15T10:30:00Z"
}

Security guarantee: Client cannot spoof user_id. It's extracted from the JWT token by the server. Period.

🔍 Advanced Filtering, Ordering & Pagination

GoREST provides production-grade querying out of the box. No code needed - it's all generated for you.

Filtering

Filter your data using intuitive query parameters with support for various operators:

# Equality
GET /todos?status=active

# Multiple values (OR)
GET /todos?status[]=active&status[]=pending

# Comparison operators
GET /todos?priority[gte]=5           # Greater than or equal
GET /todos?priority[lte]=10          # Less than or equal
GET /todos?priority[gt]=3            # Greater than
GET /todos?priority[lt]=8            # Less than
GET /todos?priority[ne]=0            # Not equal

# Text search
GET /todos?title[like]=meeting       # Case-sensitive
GET /todos?title[ilike]=MEETING      # Case-insensitive

# Date filtering
GET /todos?created_at[gte]=2024-01-01
GET /todos?updated_at[lt]=2024-12-31

# Combine filters (AND)
GET /todos?status=active&priority[gte]=7&created_at[gte]=2024-01-01

Ordering

Sort results by one or multiple fields:

# Single field
GET /todos?order[priority]=desc
GET /todos?order[created_at]=asc

# Multiple fields (order matters)
GET /todos?order[priority]=desc&order[created_at]=asc

The first order parameter becomes the primary sort, second becomes secondary, etc.

Pagination with Hydra Collections

GoREST uses Hydra for semantic pagination:

GET /todos?page=2&limit=20

Response:

{
  "@context": "http://www.w3.org/ns/hydra/context.jsonld",
  "@type": "hydra:Collection",
  "hydra:totalItems": 42,
  "hydra:member": [
    {
      "@id": "/todos/abc-123",
      "@type": "TodoDTO",
      "id": "abc-123",
      "title": "Buy groceries",
      "user_id": "/users/xyz-789"
    }
  ],
  "hydra:view": {
    "@type": "hydra:PartialCollectionView",
    "hydra:first": "/todos",
    "hydra:previous": "/todos",
    "hydra:next": "/todos?page=3",
    "hydra:last": "/todos?page=5"
  }
}

Real-World Combined Query

GET /todos?status[]=active&status[]=pending&priority[gte]=5&order[priority]=desc&order[created_at]=desc&page=2&limit=20

This single query:

✅ Filters todos with status "active" OR "pending"
✅ AND priority >= 5
✅ Sorts by priority (descending), then created_at (descending)
✅ Returns page 2 with 20 items per page
✅ Includes navigation links (first, previous, next, last)

All of this is auto-generated. Zero code required.

Security

Filtering and ordering are secure by design:

🔒 Only database fields (with db tags) can be filtered/sorted
🔒 SQL injection protection via parameterized queries
🔒 Invalid fields are silently ignored
🔒 No arbitrary SQL execution possible

🪝 The Hooks System: Resources Lifecycle Hooks

GoREST generates code, but you control the business logic through hooks. Here's the real power:

Hook Types

StateProcessor - Validate/transform data before Create/Update/Delete
SQLQueryOverride - Replace default queries with custom SQL
SQLQueryListener - Intercept queries before/after execution
Serializer - Transform data before API response

Example 1: Auto-Populate User ID (Server-Side Security)

type TodoHooks struct {
    hooks.NoOpHooks[models.Todo]
}

func (h *TodoHooks) StateProcessor(ctx context.Context, operation hooks.Operation, id any, todo *models.Todo) error {
    if operation == hooks.OperationCreate {
        // Validate
        if len(todo.Title) < 3 {
            return fmt.Errorf("title must be at least 3 characters")
        }

        // Auto-populate user_id from JWT context (server-side!)
        if userID := ctx.Value("user_id"); userID != nil {
            if uid, ok := userID.(string); ok {
                todo.UserId = &uid
            }
        }
    }
    return nil
}

Example 2: Custom SQL Queries

func (h *TodoHooks) SQLQueryOverride(ctx context.Context, operation hooks.Operation, id any) (*sqlbuilder.SelectQuery, error) {
    if operation == hooks.OperationList {
        // Only show incomplete todos
        query := sqlbuilder.
            Select("id", "title", "content", "user_id").
            From("todo").
            Where("completed = FALSE")
        return query, nil
    }
    return nil, nil
}

Example 3: Per-Resource Authorization

func (h *TodoHooks) Authorize(ctx context.Context, operation hooks.Operation, id any) error {
    userID := ctx.Value("user_id")
    if userID == nil {
        return fmt.Errorf("authentication required")
    }

    // Add custom role checks, permissions, etc.
    if operation == hooks.OperationDelete {
        // Only admins can delete
        if !isAdmin(userID) {
            return fmt.Errorf("insufficient permissions")
        }
    }

    return nil
}

Key Benefit: Hooks live in separate files. Regenerate your API anytime without losing business logic.

⚡ Performance: Blazing Fast Under Load

GoREST is built for production workloads. Here are real benchmark results from concurrent load testing with Vegeta (Framework 14" laptop: i7 12th gen, 16GB RAM, PostgreSQL):

Load Test Results

Records	Concurrency	RPS	p50 (Median)	p95	p99 (Tail)	Success Rate
10	1	1	810µs	1.27ms	1.27ms	100%
10	10	10	744µs	959µs	1.04ms	98%
10	50	50	229µs	338µs	518µs	99.6%
100	1	1	257µs	455µs	455µs	80%
100	10	10	262µs	412µs	451µs	98%
100	50	50	231µs	329µs	373µs	99.6%
1000	1	1	228µs	386µs	386µs	80%
1000	10	10	297µs	379µs	475µs	98%
1000	50	50	230µs	329µs	378µs	99.6%

Performance Summary

✅ Best p50 (median): 228µs - typical response time
✅ Best p95: 329µs - 95% of requests complete faster
✅ Worst p99 (tail latency): 1.27ms - even worst-case is sub-2ms
✅ All requests under 10ms - Excellent production performance

What This Means

⚡ Sub-millisecond responses for most queries (even with 1000 records)
🚀 Scales horizontally - performance stays consistent at 50 concurrent clients
📦 Including JSON-LD transformation - semantic web overhead is negligible
🎯 Production-ready - p99 latency under 2ms means happy users

Benchmark It Yourself

make benchmark

The benchmark:

Creates a test table with 1000 records
Generates models and API endpoints
Starts the API server
Runs Vegeta load tests at 1, 10, and 50 concurrent clients
Tests fetching 10, 100, and 1000 records
Measures p50, p95, p99 latency percentiles
Cleans up and restores original schema

Note: Results depend on your hardware and database configuration. The above benchmarks use PostgreSQL with default settings.

🌐 JSON-LD: Semantic Web Out of the Box

GoREST automatically supports JSON-LD (Linked Data), making your API:

🤖 Machine-readable - semantic context for AI/ML tools
🔗 Interoperable - works with semantic web technologies
📈 SEO-friendly - structured data for search engines

Regular JSON vs JSON-LD:

// Accept: application/json
{
  "id": "abc-123",
  "user_id": "xyz-789",
  "title": "Buy groceries"
}

// Accept: application/ld+json
{
  "@context": "https://schema.org/",
  "@type": "TodoDTO",
  "@id": "/todos/abc-123",
  "id": "abc-123",
  "user_id": "/users/xyz-789",  // 🔗 Auto-converted to IRI
  "title": "Buy groceries"
}

Foreign keys automatically become IRIs:

user_id: "xyz-789" → user_id: "/users/xyz-789"
Perfect for hypermedia-driven APIs (HATEOAS)
Zero configuration required

🛡️ Production Features

Health Checks

curl http://localhost:3000/health

{
  "status": "healthy",
  "database": {
    "status": "up"
  }
}

Perfect for:

Load balancer health checks
Kubernetes readiness/liveness probes
Monitoring systems (Prometheus, Datadog)

Graceful Shutdown

Listens for SIGTERM and SIGINT
30-second timeout for in-flight requests
Cleanly closes database connections
Zero data corruption risk

Structured Logging

import "github.com/nicolasbonnici/gorest/internal/logger"

logger.Log.Info("User created", "user_id", userID, "email", email)
logger.Log.Error("Database error", "error", err, "query", query)

JSON output perfect for log aggregation (ELK, Splunk, etc.)

🤝 Contribute & Help Shape GoREST

GoREST is 100% open-source and we need YOU to make it better!

🌟 Why Contribute?

Learn Go, SQL, code generation, REST APIs in a real-world project
Your code helps thousands of developers build better APIs faster
Build your portfolio with production-grade open-source contributions
Join a welcoming community of passionate developers
Support the free software movement

💡 How to Contribute

1. Test & Report Bugs

git clone https://github.com/nicolasbonnici/gorest.git
cd gorest
make test-up && make test-schema
make generate && make run

Found a bug? Open an issue!

2. Contribute Code

For Beginners:

Fix typos in documentation
Add code comments
Write examples and tutorials
Improve error messages

Intermediate:

Add tests for existing features
Fix bugs from the issue tracker
Improve code generation templates
Add validation helpers

Advanced:

Add support for new databases (MariaDB, CockroachDB?)
Implement GraphQL support alongside REST
Add gRPC code generation
Create monitoring/metrics integrations
Build a web UI for API management

3. Improve Documentation

Write blog posts about your GoREST experience
Create video tutorials
Translate documentation to other languages
Add architecture diagrams
Write migration guides

4. Share Ideas

Have a feature idea? Open a GitHub Discussion or Issue!

🚀 Easy First Contributions

Add Database Support: GoREST supports PostgreSQL, MySQL, SQLite - what about MariaDB or CockroachDB?
Write Hook Examples: Share your real-world hook implementations
Create Middleware: Rate limiting, CORS, request logging, etc.
Improve Benchmarks: Test with different databases and workloads
Build Integrations: Prometheus metrics, OpenTelemetry, etc.

📋 Contribution Guidelines

Fork the repo and create a feature branch
Write tests for new features
Follow Go conventions (gofmt, golint)
Use conventional commits (feat:, fix:, docs:, etc.)
Update documentation for user-facing changes
Open a PR with a clear description

See CONTRIBUTING.md for detailed guidelines.

🎯 What Could YOU Build with GoREST?

💼 SaaS Backend - Multi-tenant APIs in minutes
📱 Mobile App Backend - REST APIs ready for iOS/Android
🤖 Microservices - Generate APIs for each service
🧪 Prototypes - Validate ideas fast
📊 Data APIs - Expose your database as a service
🎮 Game Backends - Player data, leaderboards, etc.

What will you build? Tell us in the comments!

🔗 Links & Resources

GitHub: github.com/nicolasbonnici/gorest
Documentation: Full README with detailed guides
Issues: Report bugs or request features
Contributing: CONTRIBUTING.md

⭐ Support the Project

Star the repo - it helps others discover GoREST
Share this article - Twitter, LinkedIn, Reddit, Dev.to
Write about your experience - blog posts, tweets, videos
Contribute code or docs - every bit helps!

💬 Join the Conversation

Have you tried GoREST?

What's your use case?
What features would you like to see?
Want to contribute but don't know where to start?

Drop a comment below! We reply to everyone.

Ready to generate your first API?

git clone https://github.com/nicolasbonnici/gorest.git
cd gorest
make generate && make run

It's that simple. 🚀

GoREST v0.1.0 is MIT licensed and free to use in your projects. Built with ❤️ in Go, for developers who value their time.

Build a Symfony 7 boilerplate using FrankenPHP, Docker, PostgreSQL and php 8.4

Nicolas Bonnici — Mon, 23 Dec 2024 08:08:47 +0000

What are we cooking?

Hi everyone, in this post we're going to build a boilerplate to start any kind of Symfony project, such like a monolith or an API. We'll use the top tier app server FrankenPHP written in Go. The boilerplate will also use PostgreSQL SGDB for relational database.

Compose the stack using Docker and Compose

First thing first to orchestrate all the containers we will use Compose, we're going to write the stack containers definition.

The directory structure will be very simple, one folder for all docker related files and an other for the Symfony project source code.

We'll add a compose.yml file directly at the project root.

services:
  boilerplate-database:
    image: postgres:16
    container_name: boilerplate-database
    env_file:
      - symfony/.env
    restart: always
    environment:
      POSTGRES_DB: ${DATABASE_NAME}
      POSTGRES_PASSWORD: ${DATABASE_PWD}
    ports:
        - 15432:5432
    volumes:
      - database_data:/var/lib/postgresql/data:rw

  boilerplate-app:
    env_file:
      - symfony/.env
    container_name: boilerplate-app
    build:
      context: ./
      dockerfile: docker/api/Dockerfile
      target: frankenphp_dev
    depends_on:
      - boilerplate-database
    image: ${IMAGES_PREFIX:-}boilerplate-app
    restart: unless-stopped
    environment:
      SERVER_NAME: ${SERVER_NAME:-http://localhost}, boilerplate-app:80
      MERCURE_PUBLISHER_JWT_KEY: ${CADDY_MERCURE_JWT_SECRET:-!ChangeThisMercureHubJWTSecretKey!}
      MERCURE_SUBSCRIBER_JWT_KEY: ${CADDY_MERCURE_JWT_SECRET:-!ChangeThisMercureHubJWTSecretKey!}
      TRUSTED_PROXIES: ${TRUSTED_PROXIES:-127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16}
      TRUSTED_HOSTS: ${TRUSTED_HOSTS:-^${SERVER_NAME:-nbonnici\.info|localhost}|php$$}
      DATABASE_URL: postgresql://${POSTGRES_USER:-app}:${POSTGRES_PASSWORD:-!ChangeMe!}@database:5432/${POSTGRES_DB:-app}?serverVersion=${POSTGRES_VERSION:-16}&charset=${POSTGRES_CHARSET:-utf8}
      MERCURE_URL: ${CADDY_MERCURE_URL:-http://php/.well-known/mercure}
      MERCURE_PUBLIC_URL: ${CADDY_MERCURE_PUBLIC_URL:-http://${SERVER_NAME:-localhost}/.well-known/mercure}
      MERCURE_JWT_SECRET: ${CADDY_MERCURE_JWT_SECRET:-!ChangeThisMercureHubJWTSecretKey!}
    volumes:
      - ./symfony:/app:cached
      - caddy_data:/data
      - caddy_config:/config
    # comment the following line in production, it allows to have nice human-readable logs in dev
    tty: true

networks:
  default:
    external: true
    name: proxies

volumes:
  database_data:
  caddy_data:
  caddy_config:

Here nothing fancy, we create on a custom network a database container using the latest PostgreSQL version and another container built using frankenphp containing the Symfony app.

We can override it this way for development purpose using a compose.override.yml at project root

# Development environment override
services:
  boilerplate-app:
    build:
      context: ./
      dockerfile: docker/api/Dockerfile
      target: frankenphp_dev
    ports:
      # HTTP
      - target: 80
        published: ${HTTP_PORT:-80}
        protocol: tcp
      # HTTPS
      - target: 443
        published: ${HTTPS_PORT:-443}
        protocol: tcp
      # HTTP/3
      - target: 443
        published: ${HTTP3_PORT:-443}
        protocol: udp
    volumes:
      - ./symfony:/app
      - /symfony/var
      - ./docker/frankenphp/Caddyfile:/etc/caddy/Caddyfile:ro
      - ./docker/frankenphp/conf.d/app.dev.ini:/usr/local/etc/php/conf.d/app.dev.ini:ro
      # If you develop on Mac or Windows you can remove the vendor/ directory
      #  from the bind-mount for better performance by enabling the next line:
      #- /app/vendor
    environment:
      MERCURE_EXTRA_DIRECTIVES: demo
      # See https://xdebug.org/docs/all_settings#mode
      XDEBUG_MODE: "${XDEBUG_MODE:-off}"
    extra_hosts:
      # Ensure that host.docker.internal is correctly defined on Linux
      - host.docker.internal:host-gateway
    tty: true

Now let's take a closer look at the app container Dockerfile located in docker/api/Dockerfile to discover how this image is built.

FROM dunglas/frankenphp:1-php8.4-bookworm AS frankenphp_upstream

FROM frankenphp_upstream AS frankenphp_base

WORKDIR /app

# persistent / runtime deps
# hadolint ignore=DL3008
RUN apt-get update && apt-get install --no-install-recommends -y \
    acl \
    file \
    gettext \
    git \
    && rm -rf /var/lib/apt/lists/*

# https://getcomposer.org/doc/03-cli.md#composer-allow-superuser
ENV COMPOSER_ALLOW_SUPERUSER=1

RUN set -eux; \
    install-php-extensions \
        @composer \
        apcu \
        intl \
        opcache \
        zip \
        pdo_mysql \
        pdo_pgsql \
        gd \
        intl \
        xdebug \
    ;

COPY --link docker/frankenphp/conf.d/app.ini $PHP_INI_DIR/conf.d/
COPY --link --chmod=755 docker/frankenphp/docker-entrypoint.sh /usr/local/bin/docker-entrypoint
COPY --link docker/frankenphp/Caddyfile /etc/caddy/Caddyfile

ENTRYPOINT ["docker-entrypoint"]

HEALTHCHECK --start-period=60s CMD curl -f http://localhost:2019/metrics || exit 1

# Dev
FROM frankenphp_base AS frankenphp_dev

ENV APP_ENV=dev XDEBUG_MODE=off
VOLUME /app/var/

RUN mv "$PHP_INI_DIR/php.ini-development" "$PHP_INI_DIR/php.ini"

RUN set -eux; \
    install-php-extensions \
        xdebug \
    ;

COPY --link docker/frankenphp/conf.d/app.dev.ini $PHP_INI_DIR/conf.d/

CMD [ "frankenphp", "run", "--config", "/etc/caddy/Caddyfile", "--watch" ]

Here again nothing fancy unless the multi stages of this Dockerfile, first we build a base image using Debian Bookworm based Frankenphp image, install container's dependencies and docker entrypoint. Then we can build and configure the dev image from it and production ready optimized image.

I use the Debian Bookworm based image since i don't recommend using the Alpine one, the perfs seems a little less stable and fast. This is related to the musl libc library and JIT AKA just in time compilation used by php core, more information here on the official Frankenphp document.

The docker entrypoint, located in docker/frankenphp look like this:

#!/bin/sh
set -e

if [ "$1" = 'frankenphp' ] || [ "$1" = 'php' ] || [ "$1" = 'bin/console' ]; then
    if [ -z "$(ls -A 'vendor/' 2>/dev/null)" ]; then
        composer install --optimize-autoloader --prefer-dist --no-progress --no-interaction
    fi

    if grep -q ^DATABASE_URL= .env; then
        echo "Waiting for database to be ready..."
        ATTEMPTS_LEFT_TO_REACH_DATABASE=60
        until [ $ATTEMPTS_LEFT_TO_REACH_DATABASE -eq 0 ] || DATABASE_ERROR=$(php bin/console dbal:run-sql -q "SELECT 1" 2>&1); do
            if [ $? -eq 255 ]; then
                # If the Doctrine command exits with 255, an unrecoverable error occurred
                ATTEMPTS_LEFT_TO_REACH_DATABASE=0
                break
            fi
            sleep 1
            ATTEMPTS_LEFT_TO_REACH_DATABASE=$((ATTEMPTS_LEFT_TO_REACH_DATABASE - 1))
            echo "Still waiting for database to be ready... Or maybe the database is not reachable. $ATTEMPTS_LEFT_TO_REACH_DATABASE attempts left."
        done

        if [ $ATTEMPTS_LEFT_TO_REACH_DATABASE -eq 0 ]; then
            echo "The database is not up or not reachable:"
            echo "$DATABASE_ERROR"
            exit 1
        else
            echo "The database is now ready and reachable"
        fi

        if [ "$( find ./migrations -iname '*.php' -print -quit )" ]; then
            php bin/console doctrine:migrations:migrate --no-interaction --all-or-nothing
        fi
    fi

    setfacl -R -m u:www-data:rwX -m u:"$(whoami)":rwX var
    setfacl -dR -m u:www-data:rwX -m u:"$(whoami)":rwX var
fi

exec docker-php-entrypoint "$@"

This is exactly the same provided by the Symfony part of the FrankenPHP documentation.

Configure Frankenphp

FrankenPHP use Caddy as proxy server, so we'll need a Caddyfile to configure it and also provide basic php configurations. Here again we'll stick to the FrankenPHP documention. You can find it in the docker/frankenphp folder.

By default FrankenPHP will work on worker mode, by launching two proc by CPU core which can be tweak according to your project needs and also your hosting type.

Symfony project

Here's a minimal list of dependencies to offer a first class developer experience. But first thing first we need to install the FrankenPHP runtime, the same we configure on the worker.Caddyfile configuration:

worker {
    file ./public/index.php
    env APP_RUNTIME Runtime\FrankenPhpSymfony\Runtime
}

To do so simply install the runtime/frankenphp-symfony composer package. Then we install the bare minimum for a kick ass developer experience, a linter using Code Sniffer, phpstan as code quality audit tool, Rector to ease and automate code maintenance, some useful Symfony components and package and of course the Doctrine ORM. Here the composer.json file located at the symfony folder root.

{
    "type": "project",
    "license": "proprietary",
    "minimum-stability": "stable",
    "prefer-stable": true,
    "require": {
        "php": ">=8.1",
        "ext-ctype": "*",
        "ext-iconv": "*",
        "doctrine/dbal": "^3",
        "doctrine/doctrine-bundle": "^2.13",
        "doctrine/doctrine-migrations-bundle": "^3.3",
        "doctrine/orm": "^3.2",
        "nelmio/cors-bundle": "^2.5",
        "phpdocumentor/reflection-docblock": "^5.4",
        "phpstan/phpdoc-parser": "^1.30",
        "ramsey/uuid-doctrine": "^2.1",
        "runtime/frankenphp-symfony": "^0.2.0",
        "symfony/asset": "7.2.*",
        "symfony/console": "7.2.*",
        "symfony/dotenv": "7.2.*",
        "symfony/expression-language": "7.2.*",
        "symfony/flex": "^2",
        "symfony/framework-bundle": "7.2.*",
        "symfony/password-hasher": "7.2.*",
        "symfony/property-access": "7.2.*",
        "symfony/property-info": "7.2.*",
        "symfony/runtime": "7.2.*",
        "symfony/security-bundle": "7.2.*",
        "symfony/serializer": "7.2.*",
        "symfony/twig-bundle": "7.2.*",
        "symfony/validator": "7.2.*",
        "symfony/yaml": "7.2.*"
    },
    "require-dev": {
        "doctrine/doctrine-fixtures-bundle": "^3.5",
        "friendsofphp/php-cs-fixer": "^3.65",
        "phpunit/phpunit": "^9.5",
        "rector/rector": "^1.2",
        "symfony/browser-kit": "7.2.*",
        "symfony/css-selector": "7.2.*",
        "symfony/maker-bundle": "^1.61",
        "symfony/phpunit-bridge": "^7.2",
        "symfony/stopwatch": "7.2.*",
        "symfony/var-dumper": "7.2.*",
        "symfony/web-profiler-bundle": "7.2.*"
    },
    "config": {
        "allow-plugins": {
            "php-http/discovery": true,
            "symfony/flex": true,
            "symfony/runtime": true
        },
        "sort-packages": true
    },
    "autoload": {
        "psr-4": {
            "App\\": "src/"
        }
    },
    "autoload-dev": {
        "psr-4": {
            "App\\Tests\\": "tests/"
        }
    },
    "replace": {
        "symfony/polyfill-ctype": "*",
        "symfony/polyfill-iconv": "*",
        "symfony/polyfill-php72": "*",
        "symfony/polyfill-php73": "*",
        "symfony/polyfill-php74": "*",
        "symfony/polyfill-php80": "*",
        "symfony/polyfill-php81": "*"
    },
    "scripts": {
        "auto-scripts": {
            "cache:clear": "symfony-cmd",
            "assets:install %PUBLIC_DIR%": "symfony-cmd"
        },
        "post-install-cmd": [
            "@auto-scripts"
        ],
        "post-update-cmd": [
            "@auto-scripts"
        ]
    },
    "conflict": {
        "symfony/symfony": "*"
    },
    "extra": {
        "symfony": {
            "allow-contrib": false,
            "require": "7.2.*"
        }
    }
}

Why using the latest version of Symfony 7.2 and not waiting on the last LTS which is the 6.4 for now? Because this is not the way Symfony behave, let's listen to what Nicolas Grekas said the Forum PHP 2024 event:

so first thing first, it will be more easy to maintain a project by updating it once monthly than migrating from the last LTS to the next one, which can be in some project very painless and time consuming. Tool like Rector can really help by the way this kind of migration and many others too.

Leverage all the power of Composer package manager

In this project Composer is used to handle class autoloading, dependencies and also manage the project itself. Lets add a set of usefull scripts in the dedicated section of our composer.jsonconfiguration file.

{

    ...

    "scripts": {

        ...

        "setup": [
            "composer run up",
            "composer run deps:install",
            "composer run database",
            "composer run migrate",
            "composer run fixtures"
        ],
        "up": [
            "docker compose --env-file symfony/.env up -d --build"
        ],
        "stop": [
            "docker compose --env-file symfony/.env stop"
        ],
        "down": [
            "docker compose --env-file symfony/.env down"
        ],
        "build": [
            "docker compose --env-file symfony/.env build"
        ],
        "deps:install": [
            "docker exec -it boilerplate-app bin/composer install -o"
        ],
        "database": [
            "docker exec -it boilerplate-app bin/console doctrine:database:create -n --if-not-exists"
        ],
        "migrate": [
            "docker exec -it boilerplate-app bin/console doctrine:migration:migrate -n"
        ],
        "fixtures": [
            "docker exec -it boilerplate-app bin/console doctrine:fixtures:load -n"
        ],
        "tests": [
            "Composer\\Config::disableProcessTimeout",
            "docker exec -t boilerplate-app bash -c 'clear && ./vendor/bin/phpunit --testdox --exclude=smoke'"
        ],
        "lint": [
            "docker exec -t boilerplate-app ./vendor/bin/php-cs-fixer ./src/"
        ],
        "lint:fix": [
            "docker exec -t boilerplate-app ./vendor/bin/php-cs-fixer fix ./src/"
        ],
        "db": [
            "Composer\\Config::disableProcessTimeout",
            "psql postgresql://postgres:password@127.0.0.1:15432/boilerplate"
        ],
        "logs": [
            "Composer\\Config::disableProcessTimeout",
            "docker compose logs -f"
        ],
        "generate-keypair": [
            "docker exec -t boilerplate-app bin/console lexik:jwt:generate-keypair"
        ],
        "cache-clear": [
            "docker exec -t boilerplate-app bin/console c:c"
        ]
    }
}

Here's an overview of available composer commands:

To setup project's containers simply run:

composer setup

Once built, you can start or stop project's containers like this:

composer up

composer stop

Destroys containers (but keep volumes)

composer down

Migrate database

composer migrate

Load fixtures

composer fixtures

Connect to postgresql database

composer db

Show logs

composer logs

Fix code lint

composer lint:fix

The Composer\Config::disableProcessTimeout allow some composer scripts to disable the default process timeout behavior.

Optimize for production

Symfony development mode cost a lot by caching nothing and add a lot of debug everywhere. On production environment we will use OPCache to store in cache class content, dump composer class autoload in a more optimized way and get rid of development related dependencies. You can find on the docker/frankephp/conf.d the different configurations for php.

First let's add a production specific stage on our Dockerfile

# Prod
FROM frankenphp_base AS frankenphp_prod

ENV APP_ENV=prod
ENV FRANKENPHP_CONFIG="import worker.Caddyfile"

RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini"

COPY --link docker/frankenphp/conf.d/app.prod.ini $PHP_INI_DIR/conf.d/
COPY --link docker/frankenphp/worker.Caddyfile /etc/caddy/worker.Caddyfile

COPY symfony/ .

RUN set -eux; \
    composer install --no-cache --prefer-dist --no-dev --no-autoloader --no-scripts --no-progress

RUN set -eux; \
    mkdir -p var/cache var/log; \
    composer dump-autoload --classmap-authoritative --no-dev; \
    composer dump-env prod; \
    composer run-script --no-dev post-install-cmd; \
    chmod +x bin/console; sync;

CMD [ "frankenphp", "run", "--config", "/etc/caddy/Caddyfile", "--watch" ]

Then create a specific override for compose to use in production, at the root of the project create a new composer.override.prod.yml with the following content:

# Development environment override
services:
  boilerplate-app:
    build:
      context: ./
      dockerfile: ./docker/api/Dockerfile
      target: frankenphp_prod
    expose:
      - 80
    volumes:
      - ./docker/frankenphp/Caddyfile:/etc/caddy/Caddyfile:ro
      - ./docker/frankenphp/conf.d/app.prod.ini:/usr/local/etc/php/conf.d/app.prod.ini:ro
      # If you develop on Mac or Windows you can remove the vendor/ directory
      #  from the bind-mount for better performance by enabling the next line:
      #- /app/vendor
    environment:
      SERVER_NAME: ${SERVER_NAME:-http://api.nbonnici.info}, boilerplate-app:80
      MERCURE_EXTRA_DIRECTIVES: demo
      # See https://xdebug.org/docs/all_settings#mode
      XDEBUG_MODE: "${XDEBUG_MODE:-off}"
    extra_hosts:
      # Ensure that host.docker.internal is correctly defined on Linux
      - host.docker.internal:host-gateway

The purpose is to specify the new build target which is now frankenphp_prod and only expose the http port of the container without any forwarding. This is this spcecific port 80 on your container that your hostname with SSL support will target after a reverse proxy.

Benchmark

Now it's time to benchmark, is FrankenPHP as blazing fast as most people stated? Short answer yes it is, but each project having his own needs you'll need to tweak it, and it's very flexible so no problem doing it.

For this test we'll create a dead simple Todo entity containing a few columns and a foreign key to our User entity.

Using fixtures we will create one thousand of todos and using the top tier REST api creation bundle API Platform we will load them in json format.

For this test i'am using a local docker container on a 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz CPU with 16go of RAM.

The test itself consist to make an HTTP request to a RESTFUL API to retrieve a collection of todo resources, with more items per page from a page of 10 to 1000.

So the project must route the request then use API Platform layers and the ORM to query from the database the todos, then serialize the objects in a json response.

The container directly run on the host I send the request so there's almost no network latency and I use Insomnia to measure the response time.

GET /todos 10 / 50 / 100 / 500 / 1000 resource in dev staging

Page of 10 resources: 177 ms
Page of 50 resources: 188 ms
Page of 100 resources: 211 ms
Page of 500 resources: 259 ms
Page of 1000 resources: 346 ms

GET 10 / 50 / 100 / 500 / 1000 resource in production staging

Page of 10 resources: 9.03 ms
Page of 50 resources: 15.1 ms
Page of 100 resources: 29.2 ms
Page of 500 resources: 106 ms
Page of 1000 resources: 170 ms

Conclusion the gap is huge, between development and production nothing new here. By creating the same REST API without Symfony and API Platform and all the confort they bring you can win let's say a few milliseconds more which is totally nothing and almost impossible to detect from a human perception. Frankenphp per default using mecanism such like early hint http code, the go routines and many modern and blazing fast concept can really improve your project performances.

Going further

Security notes

We can secure things a little more by not using root user container side, this is a bad practice.

To do so, we need to follow the official Frankenphp documentation.

Migrate Symfony up to the incoming 7.4 LTS

Here again keep your project freshly updated each month and also pay attention to the deprecated warning you can find.

Configuration

All depend on your need, are you working on a CLI app, an API, a monolith? How do you host your app, on a cluster, just one bare metal onto lambda in all those cases you need to find the better settings by tweaking the worker thread number by core and also the right php configuration.

Conclusion

This boilerplate can literally boot up any project from a monolith to a REST API, almost everything you can build with php and Symfony. Using top tier service like PostgreSQL and easily scalable using Kubernetes and Karpenter for instance, as well as a Gateway API to proxy and absorb mostly GET http incoming requests for high demand projects. You can also use it to migrate an existing project to Frankenphp.

You can find the final boilerplate source code here on Gitlab. Feel free to contribute on it, i will maintain and update this post as well as the boilerplate, thank you for reading.

Efficient code review process

Nicolas Bonnici — Mon, 09 Sep 2024 05:46:59 +0000

Welcome. If you're here reading this post, it means you already contributed to some projects where you were not alone, or you plan to. If you want to know more about what we call code review (aka code contribution reviewed by peers before merging it into a common trunk), read on.

Mindset

In this process, the main purpose is to challenge a technical response to a problem within the project context.

If a reviewer suggests another approach, you must ask yourself: is it a better approach than yours? If the project can benefit from the suggested change in any domain—user experience, developer experience, security, performance, or code clarity—you must consider it seriously and push the requested changes. Otherwise, at least ask for more information and, if needed, engage in a direct discussion.

The whole process should be taken as nothing but a win-win. The more care you put into the review process, the more the produced code quality will improve, and thus the overall project quality will benefit as well.

Code hygiene and stability are the main goals. You guarantee best practices, security, maintainability, and performance. Sometimes, on some projects, this is not already the case and you have to maintain a legacy project anyway. Apply strictly the same process to all new contributions; then you can leverage a technical roadmap to clean what needs to be cleaned.

Of course, but it’s important to mention again: be respectful, kind, and never get upset—unless you're Linus Torvalds. Be open to suggested changes.

Interact with the reviewed developer

Using the most popular Git web interfaces such as GitHub or GitLab, there are many ways to interact with the code author. Sometimes the changes you request are not mandatory or can be postponed on a technical roadmap. Sometimes the contribution will directly alter the project's code quality, security, or performance at merge time, so they must be addressed beforehand.

When you open a thread and ask the author to implicitly push some changes, a suggestion is always easier to understand than a long explanation without one.

It's your responsibility to follow up and resolve threads as soon as possible to avoid blocking the next steps in the process. Using common web apps such as GitHub or GitLab, you have administration panels where all your pending reviews can be found. Be sure to use those pages: on GitLab it's https://gitlab.com/dashboard/todos and on GitHub https://github.com/pulls/review-requested.

Sometimes the best way to understand each other is to simply start talking in person or in a one-to-one online chat.

Project size considerations

The more contributors your project has, the stricter the review and merge process must be. Be careful not to be overkill, but also not to be too permissive.

At least 20% positive reviews from all contributors, combined with robust CI steps such as code audits, linting, and tests, should be sufficient. You can leverage role concepts in Git web apps to ensure at least one core maintainer approval for projects with many contributors.

Again, your process, your rules—just make sure they are documented somewhere. Pull/merge request title formats, description metadata, and any other rules are the foundation of this process.

The process

First of all, the smaller the contributions are, the better this process works. Avoid TLTR reviews (Too Long To Read) by splitting task scopes into small chunks.

Checklist

First of all, if you take time to review a merge request, the developer who assigned you must deliver a stable branch and a finished contribution. All CI and code quality steps must be green.
Another important point is to check that the reviewed code is correctly rebased on top of the target branch HEAD and contains no conflicts at all. Otherwise, ask the contributor to rebase their branch before reviewing the code.
The code lint must be valid, all kinds of test suites must pass, and all code quality analysis tools must succeed. Otherwise, ask the developer for related changes before reviewing the code.
Now it's time to analyze the changes: which files were edited or added? You need a global point of view first. The more context the developer adds—such as explanations, links to tickets, or even media like videos or screen captures—the better the reviewer will understand the purpose of the code. Document or override Git commit templates to set the standards your team needs.
Does the code add a new feature or modify an existing one? Is it a fix? In any case, the code should be backed by tests to ensure it works as expected, fails correctly, and does not regress later.
Is this contribution sufficiently documented? Is the specification properly updated?
Does the contribution properly handle errors or anything that could break normal code behavior? Is it possible to understand application behavior through logs?
Can this contribution be simplified in any way?
Is this contribution at the right abstraction level?
Does this project have solid code analysis steps? Is the test coverage at least 80%, and are critical user stories tested end to end? If not, you must check out the branch locally and rebuild it from scratch. Ensure all available tools for testing, quality, and linting are used. A clean CI/CD pipeline is the core of a clean project.
Does this code alter the database or persistence layer? Is the migration process safe when the CD pipeline applies it to production with existing large datasets?
Can this contribution expose unwanted sensitive data?
Does this code alter global project performance? Is there a risk with high data volumes? Are new database indexes needed? Will it scale?
Is this contribution safe, or does it introduce a new security flaw? Bad practices, potential XSS vulnerabilities, unsafe secrets committed to Git, or backdoors introduced by malicious contributors. Security tools should be part of your CI pipeline.
Does the project have contribution rules (often in a CONTRIBUTING.md file)? Does this contribution respect them?

Do and don't

Do

Respect the checklist above and review code only when the contributor considers it ready.
Take the time to review thoroughly.
Ask the developer to flag their contribution as WIP or draft if the code is not ready.
Add context to your review threads. Sometimes your request is not obvious. Saying something is “cleaner” or “better” without explanation is not helpful. Example: “Can you split this service into two—one in domain X and one in domain Y—to avoid mixing responsibilities?”
Ask for small contributions. Split tasks into small chunks to avoid TLTR reviews.
Always assign the best possible reviewers. Find experts for each part of the stack you touch.

Don't

Assume a developer is so senior that you can approve blindly. No one is perfect—even AI makes mistakes. Doing so defeats the purpose of code review.
Review WIP code. Such contributions should be marked as drafts.
Approve untested contributions.
Approve commented-out code.
Approve unused code.
Approve something you don’t understand.
Approve hacks or workarounds instead of proper fixes.
Repeat yourself for typos. If a typo appears everywhere, point it out once instead of opening dozens of threads. It wastes time and helps no one.

Gatekeeping

One phenomenon you will encounter is known as gatekeeping—in other words, someone being extremely strict about contributions, or wanting everyone to code exactly the way they do.

The best way to deal with gatekeeping is to define strict rules and document them, especially around contributions.

You can also improve developer experience with code quality rules, templates, linting, and automated checks directly in your CI process.

Most of the time, gatekeepers are more about rigid adherence to project rules. If it concerns project rules rather than personal preferences, thank that reviewer—they are fulfilling the core purpose of this process.

Word of end

Keep in mind that code reviews are one of the best and fastest ways to grow as a developer. A single thoughtful comment from a generous reviewer can significantly improve your skills.

Thank you for reading. Feel free to share your own best practices for reviewing code in the comments below.

Dockerized Symfony 6.4 project boilerplate

Nicolas Bonnici — Sat, 17 Feb 2024 16:12:37 +0000

Repository and post updated September 22 2024

The perfect Dockerized Symfony 6.4 boilerplate doesn't exists... But wait, what if I create and share my own vision of it?!!

Leggooo

We gonna build a classic stack that i personaly love, with such cutting edge technologies like PostgreSQL 16, Redis and Nginx HTTP server and also PHP 8.3 and FPM and Symfony framwork in LTS version 6.4.

First of all, I want something really easy to use, so let's leverage docker compose plugin without any extra argument to start and initialize all needed containers.

composer setup

Project treeview

logs
docker
symfony

Everything is at project root to do so, no fancy parameter needed since everything use default project path convention.

Simply 3 folders, the first one "docker/" to store all Docker related configurations, one other "logs/" for the whole containers logs and a "symfony/" last one directory to put your Symfony project's source code.

Go further

Good but wait I have other developers to sync in that project and do they can scaffold it on their local development env without knowing any clue about Symfony and server side development or never work on containerized projects?

Nope just kidding, ain't no voodoo involved in that process, we're not gonna reinvent the wheel and make a revolution. Not at all, just a simple leverage of composer and more specially his scripts section.

Ok if you don't rage quit this post or punch directly your screen you'll enjoy a little magic, with this simple one line command to run all needed containers then setup the database, create the data model before populating it if you agree the prompt by responding "yes" at the end.

All that using one dead simple command:

composer setup

Explanation we first build and launch all needed containers, the run them in demon mode, this the first part of the command: docker compose up --build -d. The second part will then run the "install-project" composer script. See the symfony/composer.json "scripts" section.

{
    "scripts": {
        ...
        "setup": [
            "composer run up",
            "composer run deps:install",
            "composer run database",
            "composer run migrate",
            "composer run fixtures"
        ],
        "up": [
            "docker compose --env-file symfony/.env up -d --build"
        ],
        "down": [
            "docker compose --env-file symfony/.env down"
        ],
        "stop": [
            "docker compose --env-file symfony/.env stop"
        ],
        "build": [
            "docker compose --env-file symfony/.env build"
        ],
        "deps:install": [
            "docker exec -it php-fpm bin/composer install -o"
        ],
        "database": [
            "docker exec -it php-fpm bin/console doctrine:database:create -n --if-not-exists",
            "docker exec -it php-fpm bin/console doctrine:database:create -n --if-not-exists --env=test"
        ],
        "migrate": [
            "docker exec -it php-fpm bin/console doctrine:migration:migrate -n",
            "docker exec -it php-fpm bin/console doctrine:migration:migrate -n --env=test"
        ],
        "fixtures": [
            "docker exec -it php-fpm bin/console doctrine:fixtures:load -n",
            "docker exec -it php-fpm bin/console doctrine:fixtures:load -n --env=test"
        ],
        "tests": [
            "docker exec -t php-fpm bash -c 'clear && ./vendor/bin/phpunit --testdox --exclude=smoke'"
        ],
        "lint": [
            "docker exec -t php-fpm ./vendor/bin/php-cs-fixer ./src/"
        ],
        "lint:fix": [
            "docker exec -t php-fpm ./vendor/bin/php-cs-fixer fix ./src/"
        ],
        "db": [
            "psql postgresql://postgres:password@127.0.0.1:15432/dbtest"
        ],
        "logs": [
            "docker compose logs -f"
        ],
        "cache-clear": [
            "docker exec -t php-fpm bin/console c:c"
        ]
    }
}

It will first install all needed composer dependencies and optimize classes autoloader. Then execute migrations up to the latest version then populate database with fixtures respectively with "doctrine/doctrine-migrations-bundle" and "doctrine/doctrine-fixtures-bundle" bundles.

The actual composer scripts available:

composer [
    setup
    up
    stop
    build
    deps:install
    database
    migrate
    fixtures
    tests: launch phpunit tests suite
    lint
    db: connect to database container via CLI client 
    logs: show containers logs
    cache-clear
]

Talk is cheap, show me the code

Feel free to fork, contribute and maintain this boilerplate using this Gitlab repository: nicolasbonnici/symfony-docker-boilerplate

Dockerize Nextcloud with Postgre and Redis

Nicolas Bonnici — Fri, 11 Nov 2022 23:02:09 +0000

Summary

In this post we gonna build from scratch a dockerized Nextcloud instance. Nextcloud is a powerfull self hosted collaborative cloud platform.

Key features

Nextcloud 24.0.7
PostgreSQL for the database
Redis for cache
Nginx as proxy
Out of the box A+ SSL grade
Easy backup

We gonna use a PostgreSQL 14 database container in our stack. We also create another container running a Redis instance for cache purposes.

Since Nextcloud is a web app, the frontend will be serve by an Nginx container in front of a reverse proxy to PHP FPM. We also need a to run Nextcloud related commands or cron job using an another container connected to the same volume.

We'll also need some other containers to generate and also renew the SSL certificate using Letsencrypt.

Legggo!

Docker compose

To build this complexe stack let's leverage the power of docker-compose and begin to write the docker-compose YAML file.

version: '3'

services:
  db:
    container_name: nc-db
    hostname: nc_db
    image: postgres:14
    restart: always
    environment:
      POSTGRES_DB: ${DATABASE_NAME}
      POSTGRES_PASSWORD: ${DATABASE_PWD}
    volumes:
      - db:/var/lib/postgresql/data
    ports:
      - "15432:5432"

  app:
    container_name: nc-app
    build: ./
    volumes:
      - nextcloud:/usr/src/nextcloud
    environment:
      - POSTGRES_HOST=${DATABASE_HOST}
      - POSTGRES_DB=${DATABASE_NAME}
      - POSTGRES_USER=${DATABASE_USER}
      - POSTGRES_PASSWORD=${DATABASE_PWD}
      - REDIS_HOST=${REDIS_HOST}
      - REDIS_HOST_PASSWORD=${REDIS_PASSWORD}
      - DEFAULT_PHONE_REGION=FR
    depends_on:
      - db
      - redis

  cron:
    container_name: nc-cron
    image: nextcloud:fpm-alpine
    restart: always
    volumes:
      - nextcloud:/var/www/html
    entrypoint: /cron.sh
    depends_on:
      - db
      - redis

  nc-web:
    container_name: nc-web
    build:
      context: ./nginx
    restart: always
    environment:
      - VIRTUAL_HOST=${CLOUD_HOST}
    volumes:
      - ./nginx/conf.d/:/etc/nginx/conf.d
      - ./certbot/conf/:/etc/nginx/ssl/:ro
      - ./certbot/www:/var/www/certbot/:ro
      - nextcloud:/var/www/html
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/sites/:/etc/nginx/sites-available
      - ./nginx/logs:/var/log/nginx
      - /var/run/docker.sock:/tmp/docker.sock:ro
    depends_on:
      - app
    networks:
      - proxy-tier
      - default
    ports:
      - "8888:80"
      - "443:443"

  redis:
    container_name: nc-redis
    image: "redis:alpine"
    command: redis-server --requirepass ${REDIS_PASSWORD}
    ports:
      - "6379:6379"
    volumes:
      - ./redis/data:/var/lib/redis
      - ./redis/redis.conf:/usr/local/etc/redis/redis.conf
    environment:
      - REDIS_REPLICATION_MODE=master

  certbot:
    container_name: nc-certbot
    image: certbot/certbot:latest
    volumes:
      - ./certbot/www/:/var/www/certbot/:rw
      - ./certbot/conf/:/etc/letsencrypt/:rw
    networks:
      - proxy-tier
    depends_on:
      - nc-web

volumes:
  db:
  nextcloud:

networks:
  proxy-tier:

Nothing fancy here, we configure all the needed containers and mount handful volumes for configuration and logs directly to the host file system.

Environment variables like DATABASE_HOST will be resolved using dotenv, something like this for example IN .env at project root:

CLOUD_HOST=cloud.example.localhost

DATABASE_HOST=nc_db
DATABASE_NAME=nextcloud
DATABASE_USER=postgres
DATABASE_PWD=example!

REDIS_HOST=redis
REDIS_PASSWORD=example!

Lets break this in smaller parts

Nextcloud

First let's talk about the Nextcloud container, here's his DockerFile largely inspired by the official documentation.

# DO NOT EDIT: created by update.sh from Dockerfile-alpine.template
FROM php:8.0-fpm-alpine3.16

# entrypoint.sh and cron.sh dependencies
RUN set -ex; \
    \
    apk add --no-cache \
        rsync \
    ; \
    \
    rm /var/spool/cron/crontabs/root; \
    echo '*/5 * * * * php -f /var/www/html/cron.php' > /var/spool/cron/crontabs/www-data

# install the PHP extensions we need
# see https://docs.nextcloud.com/server/stable/admin_manual/installation/source_installation.html
RUN set -ex; \
    \
    apk add --no-cache --virtual .build-deps \
        $PHPIZE_DEPS \
        autoconf \
        freetype-dev \
        icu-dev \
        libevent-dev \
        libjpeg-turbo-dev \
        libmcrypt-dev \
        libpng-dev \
        libmemcached-dev \
        libxml2-dev \
        libzip-dev \
        openldap-dev \
        pcre-dev \
        postgresql-dev \
        imagemagick-dev \
        libwebp-dev \
        gmp-dev \
    ; \
    \
    docker-php-ext-configure gd --with-freetype --with-jpeg --with-webp; \
    docker-php-ext-configure ldap; \
    docker-php-ext-install -j "$(nproc)" \
        bcmath \
        exif \
        gd \
        intl \
        ldap \
        opcache \
        pcntl \
        pdo_mysql \
        pdo_pgsql \
        zip \
        gmp \
    ; \
    \
# pecl will claim success even if one install fails, so we need to perform each install separately
    pecl install APCu-5.1.21; \
    pecl install memcached-3.2.0; \
    pecl install redis-5.3.7; \
    pecl install imagick-3.7.0; \
    \
    docker-php-ext-enable \
        apcu \
        memcached \
        redis \
        imagick \
    ; \
    rm -r /tmp/pear; \
    \
    runDeps="$( \
        scanelf --needed --nobanner --format '%n#p' --recursive /usr/local/lib/php/extensions \
            | tr ',' '\n' \
            | sort -u \
            | awk 'system("[ -e /usr/local/lib/" $1 " ]") == 0 { next } { print "so:" $1 }' \
    )"; \
    apk add --virtual .nextcloud-phpext-rundeps $runDeps; \
    apk del .build-deps

# set recommended PHP.ini settings
# see https://docs.nextcloud.com/server/latest/admin_manual/installation/server_tuning.html#enable-php-opcache
ENV PHP_MEMORY_LIMIT 512M
ENV PHP_UPLOAD_LIMIT 10G
RUN { \
        echo 'opcache.enable=1'; \
        echo 'opcache.interned_strings_buffer=16'; \
        echo 'opcache.max_accelerated_files=10000'; \
        echo 'opcache.memory_consumption=128'; \
        echo 'opcache.save_comments=1'; \
        echo 'opcache.revalidate_freq=60'; \
    } > "${PHP_INI_DIR}/conf.d/opcache-recommended.ini"; \
    \
    echo 'apc.enable_cli=1' >> "${PHP_INI_DIR}/conf.d/docker-php-ext-apcu.ini"; \
    \
    { \
        echo 'memory_limit=${PHP_MEMORY_LIMIT}'; \
        echo 'upload_max_filesize=${PHP_UPLOAD_LIMIT}'; \
        echo 'post_max_size=${PHP_UPLOAD_LIMIT}'; \
    } > "${PHP_INI_DIR}/conf.d/nextcloud.ini"; \
    \
    mkdir /var/www/data; \
    chown -R www-data:root /var/www; \
    chmod -R g=u /var/www

VOLUME /var/www/html

ENV NEXTCLOUD_VERSION 24.0.7

RUN set -ex; \
    apk add --no-cache --virtual .fetch-deps \
        bzip2 \
        gnupg \
    ; \
    \
    curl -fsSL -o nextcloud.tar.bz2 \
        "https://download.nextcloud.com/server/releases/nextcloud-${NEXTCLOUD_VERSION}.tar.bz2"; \
    curl -fsSL -o nextcloud.tar.bz2.asc \
        "https://download.nextcloud.com/server/releases/nextcloud-${NEXTCLOUD_VERSION}.tar.bz2.asc"; \
    export GNUPGHOME="$(mktemp -d)"; \
# gpg key from https://nextcloud.com/nextcloud.asc
    gpg --batch --keyserver keyserver.ubuntu.com  --recv-keys 28806A878AE423A28372792ED75899B9A724937A; \
    gpg --batch --verify nextcloud.tar.bz2.asc nextcloud.tar.bz2; \
    tar -xjf nextcloud.tar.bz2 -C /usr/src/; \
    gpgconf --kill all; \
    rm nextcloud.tar.bz2.asc nextcloud.tar.bz2; \
    rm -rf "$GNUPGHOME" /usr/src/nextcloud/updater; \
    mkdir -p /usr/src/nextcloud/data; \
    mkdir -p /usr/src/nextcloud/custom_apps; \
    chmod +x /usr/src/nextcloud/occ; \
    apk del .fetch-deps

COPY *.sh upgrade.exclude /
COPY config/* /usr/src/nextcloud/config/

ENTRYPOINT ["/entrypoint.sh"]
CMD ["php-fpm"]

EXPOSE 9000

Here we use the 24.0.7 Nextcloud version, to build a PHP-FPM container from php:8.0-fpm-alpine3.16, install all needed dependencies, configure PHP then run the process on port 9000.

As i write those lines, Nextcloud still don't support PHP 8.1 version

We also configure PHP to accept upload up to 10go according to Nginx configuration.

Nginx

Nginx main role will be to serve http/s requests from browser then forward the request to our previous PHP-FPM nc-app container on port 9000.

I chose to mount local folder nginx/logs to retrieve the access as well as error logs, the other volumes are useful to tweak the web server configuration.

Here is the dead simple container Dockerfile, where the nginx instance will expose HTTP ports.

FROM nginx:alpine

WORKDIR /var/www

CMD ["nginx"]

EXPOSE 80 443

Then here's the nginx.conf, with the global nginx configuration.

user  nginx;
worker_processes  auto;
daemon off;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    access_log  /var/log/nginx/access.log;
    #access_log /dev/stdout;
    #error_log /dev/stderr;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    gzip  on;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-available/*.conf;
}

Finally the default Nextcloud nginx block and his upstream from the nc-app container.

# Set the `immutable` cache control options only for assets with a cache busting `v` argument
map $arg_v $asset_immutable {
    "" "";
    default "immutable";
}

upstream php-handler {
    server app:9000;
}

# server {
#     listen 80;
#     listen [::]:80;

#     server_name cloud.example.localhost;
#     server_tokens off;

#     location /.well-known/acme-challenge/ {
#         root /var/www/certbot;
#     }

#     location / {
#         return 301 https://cloud.example.localhost$request_uri;
#     }
# }

server {
    # listen 443 default_server ssl http2;
    # listen [::]:443 ssl http2;

    listen 80;
    server_name cloud.example.localhost;

    # Path to the root of your installation
    root /var/www/html;

    # Use Mozilla's guidelines for SSL/TLS settings
    # https://mozilla.github.io/server-side-tls/ssl-config-generator/
    # ssl_certificate     /etc/nginx/ssl/live/cloud.example.localhost/fullchain.pem;
    # ssl_certificate_key /etc/nginx/ssl/live/cloud.example.localhost/privkey.pem;
    # ssl_protocols TLSv1.2 TLSv1.3;

    # Prevent nginx HTTP Server Detection
    server_tokens off;

    # ECDHE forward secrecy
    # ssl_ciphers "HIGH:!aNULL:!MD5:!ADH:!RC4:!DH";
    # ssl_prefer_server_ciphers on;

    # HSTS settings
    # WARNING: Only add the preload option once you read about
    # the consequences in https://hstspreload.org/. This option
    # will add the domain to a hardcoded list that is shipped
    # in all major browsers and getting removed from this list
    # could take several months.
    # add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" always;

    # set max upload size and increase upload timeout:
    client_max_body_size 10G;
    client_body_timeout 300s;
    fastcgi_buffers 64 4K;

    # Enable gzip but do not remove ETag headers
    gzip on;
    gzip_vary on;
    gzip_comp_level 4;
    gzip_min_length 256;
    gzip_proxied expired no-cache no-store private no_last_modified no_etag auth;
    gzip_types application/atom+xml application/javascript application/json application/ld+json application/manifest+json application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject application/wasm application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml image/x-icon text/cache-manifest text/css text/plain text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy;

    # Pagespeed is not supported by Nextcloud, so if your server is built
    # with the `ngx_pagespeed` module, uncomment this line to disable it.
    #pagespeed off;

    # The settings allows you to optimize the HTTP2 bandwitdth.
    # See https://blog.cloudflare.com/delivering-http-2-upload-speed-improvements/
    # for tunning hints
    client_body_buffer_size 512k;

    # HTTP response headers borrowed from Nextcloud `.htaccess`
    add_header Referrer-Policy                      "no-referrer"   always;
    add_header X-Content-Type-Options               "nosniff"       always;
    add_header X-Download-Options                   "noopen"        always;
    add_header X-Frame-Options                      "SAMEORIGIN"    always;
    add_header X-Permitted-Cross-Domain-Policies    "none"          always;
    add_header X-Robots-Tag                         "none"          always;
    add_header X-XSS-Protection                     "1; mode=block" always;

    # Remove X-Powered-By, which is an information leak
    fastcgi_hide_header X-Powered-By;

    # Specify how to handle directories -- specifying `/index.php$request_uri`
    # here as the fallback means that Nginx always exhibits the desired behaviour
    # when a client requests a path that corresponds to a directory that exists
    # on the server. In particular, if that directory contains an index.php file,
    # that file is correctly served; if it doesn't, then the request is passed to
    # the front-end controller. This consistent behaviour means that we don't need
    # to specify custom rules for certain paths (e.g. images and other assets,
    # `/updater`, `/ocm-provider`, `/ocs-provider`), and thus
    # `try_files $uri $uri/ /index.php$request_uri`
    # always provides the desired behaviour.
    index index.php index.html /index.php$request_uri;

    # Rule borrowed from `.htaccess` to handle Microsoft DAV clients
    location = / {
        if ( $http_user_agent ~ ^DavClnt ) {
            return 302 /remote.php/webdav/$is_args$args;
        }
    }

    location = /robots.txt {
        allow all;
        log_not_found off;
        access_log off;
    }

    # Make a regex exception for `/.well-known` so that clients can still
    # access it despite the existence of the regex rule
    # `location ~ /(\.|autotest|...)` which would otherwise handle requests
    # for `/.well-known`.
    location ^~ /.well-known {
        # The rules in this block are an adaptation of the rules
        # in `.htaccess` that concern `/.well-known`.

        location = /.well-known/carddav { return 301 /remote.php/dav/; }
        location = /.well-known/caldav  { return 301 /remote.php/dav/; }

        location /.well-known/acme-challenge    { try_files $uri $uri/ =404; }
        location /.well-known/pki-validation    { try_files $uri $uri/ =404; }

        # Let Nextcloud's API for `/.well-known` URIs handle all other
        # requests by passing them to the front-end controller.
        return 301 /index.php$request_uri;
    }

    # Rules borrowed from `.htaccess` to hide certain paths from clients
    location ~ ^/(?:build|tests|config|lib|3rdparty|templates|data)(?:$|/)  { return 404; }
    location ~ ^/(?:\.|autotest|occ|issue|indie|db_|console)                { return 404; }

    # Ensure this block, which passes PHP files to the PHP process, is above the blocks
    # which handle static assets (as seen below). If this block is not declared first,
    # then Nginx will encounter an infinite rewriting loop when it prepends `/index.php`
    # to the URI, resulting in a HTTP 500 error response.
    location ~ \.php(?:$|/) {
        # Required for legacy support
        rewrite ^/(?!index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|oc[ms]-provider\/.+|.+\/richdocumentscode\/proxy) /index.php$request_uri;

        fastcgi_split_path_info ^(.+?\.php)(/.*)$;
        set $path_info $fastcgi_path_info;

        try_files $fastcgi_script_name =404;

        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $path_info;
        fastcgi_param HTTPS on;

        fastcgi_param modHeadersAvailable true;         # Avoid sending the security headers twice
        fastcgi_param front_controller_active true;     # Enable pretty urls
        fastcgi_pass php-handler;

        fastcgi_intercept_errors on;
        fastcgi_request_buffering off;

        fastcgi_max_temp_file_size 0;
    }

    location ~ \.(?:css|js|svg|gif|png|jpg|ico|wasm|tflite|map)$ {
        try_files $uri /index.php$request_uri;
        add_header Cache-Control "public, max-age=15778463, $asset_immutable";
        access_log off;     # Optional: Don't log access to assets

        location ~ \.wasm$ {
            default_type application/wasm;
        }
    }

    location ~ \.woff2?$ {
        try_files $uri /index.php$request_uri;
        expires 7d;         # Cache-Control policy borrowed from `.htaccess`
        access_log off;     # Optional: Don't log access to assets
    }

    # Rule borrowed from `.htaccess`
    location /remote {
        return 301 /remote.php$request_uri;
    }

    location / {
        try_files $uri $uri/ /index.php$request_uri;
    }
}

Redis

We just secure the instance using a password and use the latest redis:alpine image.

Certbot

We use the latest certbot image, to easiky and freely generate SSL certificate for our cloud. The important point here is the certificate will be generated under the certbot/conf and this same folder will be mapped to Nginx nc-web container /etc/nginx/ssl/ path.

To resolve LetsEncrypt host validation challenge we also need to serve /var/www/certbot from certbot container in the Nginx container nc-web nad using the commented server block in nginx/sites/default.conf.

Time to launch

At this point we already have a fully functional dockerized Nextcloud instance, let's use docker-compose to build and run it in background.

docker-compose up -d --build

This is it! Let's install and configure your own Nextcloud instance on http://localhost:8888 or http://cloud.example.localhost:8888 if you added cloud.example.localhost hostname to your local etc/hosts.

Et voila! keep in mind that this instance run without SSL, some errors can occur for instance while setup you'll be redirected to https:// url scheme, just remove the "s".

Security consideration

Nextcloud offer a built in security diagnosis very useful to keep your instance secure.

SSL

First we need to generate an SSL certificate using letsencrypt then configure the nginx instance to only listen on 443 default https port. To do so we need a real domain name (at least subdomain) since .localhost extension is not valid.

You need to replace all project's occurence of cloud.example.localhost with your own domain name.

Generate your SSL certificate

To use the certbot container and generate your domain name SSL certificate you first need to uncomment those lines in nginx/sites/default.conf

server {
    listen 80;
    listen [::]:80;

    server_name cloud.example.localhost;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://cloud.example.localhost$request_uri;
    }
}

Dont forget to replace cloud.example.localhost by your real domain name too. This way we can validate our host via http challenge.

And we need to use default http port 80 too rather than our previous custom 8888 in our nginx proxy container nc-web.

In docker-compose.yml, replace

    ports:
      - "8888:80"
      - "443:443"

    ports:
      - "80:80"
      - "443:443"

Dont forget to restart the stack:

docker-compose restart

Now we can generate it, first let's try in dry-run mode:

docker-compose run --rm  certbot certonly --webroot --webroot-path /var/www/certbot/ --dry-run -d votredomaine.com

If all went smooth the, directly generate them by removing the dry-run flag.

docker-compose run --rm  certbot certonly --webroot --webroot-path /var/www/certbot/ -d votredomaine.com

Finally we need to update our nginx/sites/default.conf to listen on SSL default port and also load SSL certificates and uncommented those lines:

server {
    listen 443 default_server ssl http2;
    listen [::]:443 ssl http2;

    server_name yourdomainenamehere.com;

    # Path to the root of your installation
    root /var/www/html;

    # Use Mozilla's guidelines for SSL/TLS settings
    # https://mozilla.github.io/server-side-tls/ssl-config-generator/
    ssl_certificate     /etc/nginx/ssl/live/yourdomainenamehere.com/fullchain.pem;
    ssl_certificate_key /etc/nginx/ssl/live/yourdomainenamehere.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;


    # Prevent nginx HTTP Server Detection
    server_tokens off;

    # ECDHE forward secrecy
    ssl_ciphers "HIGH:!aNULL:!MD5:!ADH:!RC4:!DH";
    ssl_prefer_server_ciphers on;

    # HSTS settings
    # WARNING: Only add the preload option once you read about
    # the consequences in https://hstspreload.org/. This option
    # will add the domain to a hardcoded list that is shipped
    # in all major browsers and getting removed from this list
    # could take several months.
    add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" always;

This way you'll end up with a A+ SSL grade according to SSL Labs.

Backup

Since we created two docker volumes, one for the PostgreSQL database and an antoher for the actual nextcloud script path also containing the data folder. All you have to backup the nextcloud instance is to dump the database and backup the cloud files onto the data folder.

To backup or restore those two volumes, just refer to the official docker documentation about volumes.

Conclusion

We built a cutting edge dockerized Nextcloud instance.

You can find all the related code on this Github repository, feel free to contribute.

Thank you for reading.