DEV Community: AddWeb Solution Pvt Ltd

Docker Image with Multi-Stage Builds & Docker Images Optimization

Rajan Vavadia — Wed, 15 Apr 2026 08:24:49 +0000

“Docker is not a virtualization technology. It is an application delivery technology.” - Mike Coleman (Docker)

Introduction
Overview: Multi-Stage Build Strategy (Builder + Production)
Stage 1: Builder - Compile Extensions, Install Tools & Dependencies
Stage 2: Production - Minimal Runtime Image
The .dockerignore File
Practical Setup (Step-by-step)
Quotes
FAQs
Key Takeaways
Conclusion

1. Introduction

This document explains how to build a production-ready PHP 8.4-FPM Docker image using a multi-stage Dockerfile. The image ships with all the PHP extensions, CLI tools (Composer, WP-CLI, Drush, Drupal Console), and Node.js needed for WordPress and Drupal projects - while keeping the final image lean and secure.

The approach uses a two-stage build:

Stage 1 (Builder): Compiles PHP extensions, installs development tools, and downloads CLI utilities.
Stage 2 (Production): Starts from a clean php:8.4-fpm base, copies only the compiled artifacts and runtime libraries, applies security hardening, and runs as a non-root user.

Why multi-stage?

A single-stage Dockerfile that compiles extensions and installs build tools leaves behind compilers, header files, and package caches that bloat the image and increase the attack surface. Multi-stage builds solve this by discarding everything that is not needed at runtime.

2. Overview: Multi-Stage Build Strategy (Builder + Production)

Stage 1 (Builder) starts from php:8.4-fpm and installs: - 20+ PHP extensions using install-php-extensions (handles build dependencies and cleanup internally). - Composer, WP-CLI, Drush, and Drupal Console. - Node.js 20.x for front-end tooling.

Stage 2 (Production) starts from a fresh php:8.4-fpm and: - Copies compiled extension .so files and their .ini configs from the builder. - Copies CLI binaries (Composer, WP-CLI, Drush, Drupal Console, Node.js). - Installs only the runtime shared libraries needed by the extensions. - Applies OPcache tuning and PHP security settings. - Creates a non-root user (appuser) for PHP-FPM. - Add a health check.

Why separate stages?

Smaller image: Build tools (gcc, make, autoconf, header files) are discarded. Only runtime libraries remain.
Faster pulls and deploys: Less data to transfer across registries and hosts.
Reduced attack surface: No compilers or dev packages in production.
Clearer troubleshooting: Build failures are isolated to Stage 1; runtime issues are isolated to Stage 2.

3. Stage 1: Builder - Compile Extensions, Install Tools & Dependencies

3.1 Base Image

FROM php:8.4-fpm AS builder
We use the official php:8.4-fpm image (based on Debian 13 Trixie) as the builder base. This gives us the PHP source, phpize, and docker-php-ext-install out of the box.

3.2 PHP Extension Installation

Single tool for all PHP extensions (handles build deps + cleanup internally)
COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/

Install all PHP extensions in one layer
RUN install-php-extensions \
bcmath \
bz2 \
calendar \
exif \
gd \
gmp \
imagick \
intl \
mailparse \
mongodb \
mysqli \
opcache \
pcntl \
pdo \
pdo_mysql \
pdo_pgsql \
soap \
sockets \
sodium \
xsl \
zip

Why install-php-extensions?

It automatically installs the OS-level build dependencies (e.g., libpng-dev, libicu-dev), compiles the extension, and removes the build dependencies - all in one step.
It replaces the manual apt-get install && docker-php-ext-configure && docker-php-ext-install && apt-get purge pattern.
It supports PECL extensions (like imagick, mongodb, mailparse) with the same syntax.
We use COPY --from=mlocati/php-extension-installer to pull the binary directly from the tool’s official image, without adding another FROM stage.

Extensions included and why:

3.3 CLI Tools Installation
Install Composer, WP-CLI, Drush, and Drupal Console in one layer
RUN apt-get update && apt-get install -y --no-install-recommends \
git \
unzip \
&& rm -rf /var/lib/apt/lists/* \
Composer
&& curl -sSL https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \
WP-CLI
&& curl -sSL https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar -o /usr/local/bin/wp \
&& chmod +x /usr/local/bin/wp \
Drupal Console
&& curl -sSL https://drupalconsole.com/installer -o /usr/local/bin/drupal \
&& chmod +x /usr/local/bin/drupal

Composer: PHP dependency manager. Installed globally at /usr/local/bin/composer.
WP-CLI: WordPress command-line interface for managing WordPress installations.
Drupal Console: CLI tool for generating boilerplate code and interacting with Drupal.
git and unzip are needed by Composer to fetch packages; they are installed here but not carried over to the production stage.

3.4 Drush Installation

ENV COMPOSER_HOME=/usr/local/share/composer
RUN composer global require drush/drush:8.* --no-interaction --prefer-dist
Drush (Drupal Shell) is installed globally via Composer. COMPOSER_HOME is set explicitly so the entire vendor directory can be copied to the production stage.

3.5 Node.js Installation

RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
&& apt-get install -y --no-install-recommends nodejs \
&& rm -rf /var/lib/apt/lists/*
Node.js 20.x (LTS) is installed for front-end build tools (npm scripts, asset compilation). NVM is intentionally avoided - a single global Node.js installation is simpler and more predictable in a container.

“Build once, run anywhere - but make sure what you build is only what you need.” - Container best practice principle

4. Stage 2: Production - Minimal Runtime Image

4.1 Fresh Base and Labels

FROM php:8.4-fpm AS production

LABEL maintainer="AddWeb Solutions" \
description="PHP 8.4-FPM production image with WordPress/Drupal tooling"
A fresh php:8.4-fpm base - none of the builder’s build tools, caches, or temp files exist here.

4.2 Copying Artifacts from Builder

Copy PHP extensions and their configs from builder
COPY --from=builder /usr/local/lib/php/extensions/ /usr/local/lib/php/extensions/
COPY --from=builder /usr/local/etc/php/conf.d/ /usr/local/etc/php/conf.d/

Copy CLI tools from builder
COPY --from=builder /usr/local/bin/composer /usr/local/bin/composer
COPY --from=builder /usr/local/bin/wp /usr/local/bin/wp
COPY --from=builder /usr/local/bin/drupal /usr/local/bin/drupal
COPY --from=builder /usr/local/share/composer /usr/local/share/composer

Copy Node.js from builder
COPY --from=builder /usr/bin/node /usr/bin/node
COPY --from=builder /usr/bin/npm /usr/bin/npm
COPY --from=builder /usr/bin/npx /usr/bin/npx
COPY --from=builder /usr/lib/node_modules /usr/lib/node_modules
COPY --from=builder selectively pulls only the compiled .so files, .ini configs, and CLI binaries. Everything else (compilers, header files, build caches) is left behind.

4.3 Runtime Libraries

RUN apt-get update && apt-get install -y --no-install-recommends \
# Runtime libs required by PHP extensions
libbz2-1.0 \
libfreetype6 \
libgmp10 \
libicu76 \
libjpeg62-turbo \
libmagickwand-7.q16-10 \
libavif16 \
libpng16-16 \
libpq5 \
libxslt1.1 \
libzip5 \
# Essential runtime utilities only
cron \
curl \
default-mysql-client \
ffmpeg \
sendmail \
supervisor \
unzip \
sqlite3 \
&& apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

Key points:

Runtime libs only: We install the shared libraries (.so files) that the compiled PHP extensions link against. These are the -dev-less counterparts (e.g., libpng16-16 not libpng-dev).
Debian Trixie package names: Since php:8.4-fpm is based on Debian 13 (Trixie), some package names differ from Bookworm (e.g., libicu76 instead of libicu72, libmagickwand-7.q16-10 instead of libmagickwand-6.q16-6, libzip5 instead of libzip4).
Essential utilities: cron (scheduled tasks), curl (health checks, API calls), default-mysql-client (database operations), ffmpeg (media processing), sendmail (email delivery), supervisor (process management), unzip, sqlite3.
Cleanup: apt-get purge --auto-remove removes packages that were pulled in only as install dependencies, and rm -rf /var/lib/apt/lists/* clears the package cache.

Runtime libraries mapped to extensions:

4.4 Drush Symlink

ENV COMPOSER_HOME=/usr/local/share/composer
RUN ln -s /usr/local/share/composer/vendor/bin/drush /usr/local/bin/drush
Makes drush available in $PATH without modifying the PATH variable.

4.5 OPcache Tuning

RUN { \
echo 'opcache.memory_consumption=128'; \
echo 'opcache.interned_strings_buffer=8'; \
echo 'opcache.max_accelerated_files=4000'; \
echo 'opcache.revalidate_freq=60'; \
echo 'opcache.fast_shutdown=1'; \
echo 'opcache.enable_cli=1'; \
echo 'opcache.validate_timestamps=0'; \
} > /usr/local/etc/php/conf.d/opcache-recommended.ini

validate_timestamps=0 is a production optimization - PHP never starts the filesystem to check if files have changed. When you deploy new code, restart PHP-FPM to pick up the changes.

4.6 Security Hardening

RUN { \
echo 'expose_php=Off'; \
echo 'display_errors=Off'; \
echo 'log_errors=On'; \
echo 'error_log=/dev/stderr'; \
echo 'allow_url_fopen=Off'; \
} > /usr/local/etc/php/conf.d/security.ini

4.7 Non-Root User

RUN groupadd -g 1000 appuser && useradd -u 1000 -g appuser -m appuser

WORKDIR /var/www/html
RUN chown -R appuser:appuser /var/www/html
Running PHP-FPM as a non-root user reduces the blast radius if the application is compromised. The appuser (UID 1000) owns the web root.

4.8 Health Check

HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
CMD php-fpm-healthcheck || kill -0 $(cat /var/run/php-fpm.pid 2>/dev/null) || exit 1
Docker checks every 30 seconds whether PHP-FPM is responding. After 3 consecutive failures, the container is marked unhealthy, and orchestrators (Docker Compose, Swarm, Kubernetes) can restart it.

4.9 Expose and CMD

EXPOSE 9000

CMD ["php-fpm"]
PHP-FPM listens on port 9000 (FastCGI). A reverse proxy (Nginx, Apache, Caddy) forwards requests to this port.

5. The .dockerignore File

.git
.github
.gitignore
.md
LICENSE
docker-compose.yml
.env*
.vscode
.idea
node_modules
vendor
.docker

The .dockerignore prevents unnecessary files from being sent to the Docker build context. This speeds up builds and avoids leaking secrets (.env files) or bloating the image with node_modules or vendor directories.

6. Practical Setup (Step-by-step)

6.1 Build the Image

docker build -t php8.4-multistage --target production .
The --target production flag tells Docker to stop at the production stage and discard the builder. If --target is omitted, Docker builds up to the last stage (which is already in production in this Dockerfile).

6.2 Run the Container

docker run -d \
--name php-app \
-p 9000:9000 \
-v ./src:/var/www/html \
php8.4-multistage
Mount your application source code at /var/www/html. PHP-FPM will serve it on port 9000.

6.3 Pair with Nginx (docker-compose example)

version: '3.8'

services:
php:
build:
context: .
target: production
volumes:
- ./src:/var/www/html
networks:
- app-network

nginx:
image: nginx:alpine
ports:
- "80:80"
volumes:
- ./src:/var/www/html
- ./nginx.conf:/etc/nginx/conf.d/default.conf
depends_on:
- php
networks:
- app-network

networks:
app-network:
driver: bridge

6.4 Verify Extensions

docker exec php-app php -m
This lists all loaded PHP modules. Confirm that gd, imagick, intl, opcache, pdo_mysql, etc., are all present.

6.5 Verify CLI Tools

docker exec php-app composer --version
docker exec php-app wp --version
docker exec php-app drush --version
docker exec php-app node --version
docker exec php-app npm --version

6.6 Check phpinfo

Create a phpinfo file
docker exec php-app bash -c "echo '<?php phpinfo();' > /var/www/html/info.php"

Quick test with built-in server
docker exec -d php-app php -S 0.0.0.0:8085 -t /var/www/html
Open http://localhost:8085/info.php in your browser to see the full PHP configuration. Remove info.php after testing - never leave it exposed in production.

6.7 Common Improvements (optional)

Use BuildKit cache mounts to speed up repeated builds: RUN --mount=type=cache,target=/var/cache/apt apt-get update && ...
Pin the PHP version (e.g., php:8.4.18-fpm) for reproducible builds instead of using the floating 8.4-fpm tag.
Add a .env-based PHP config layer for settings that vary between environments (memory_limit, upload_max_filesize).
Use Docker Compose profiles to add Xdebug only in development (avoid Xdebug in production).
Scan the image with Trivy or Docker Scout for vulnerability reporting before pushing to a registry.

“The art of simplicity is a puzzle of complexity.” - Douglas Horton

8. FAQs

Q1. Why use multi-stage builds instead of a single Dockerfile?
A. A single-stage build carries compilers, dev headers, and package caches into the final image. Multi-stage builds discard all of that, resulting in a smaller, more secure image.

Q2. Why install-php-extensions instead of docker-php-ext-install? A. install-php-extensions handles OS dependency installation, extension compilation, and cleanup automatically. It also supports PECL extensions (imagick, mongodb, mailparse) with the same syntax, eliminating the need for separate pecl install commands.

Q3. Why is the base image Debian Trixie? Can I use Bookworm or Alpine?
A. The official php:8.4-fpm tag is built on Debian 13 (Trixie) as of 2025. If you need Bookworm, use php:8.4-fpm-bookworm. Alpine (php:8.4-fpm-alpine) is smaller but uses musl libc, which can cause compatibility issues with some extensions (notably ImageMagick).

Q4. Why are the runtime library package names different from older guides?
A. Debian Trixie renamed several packages as part of the 64-bit time_t transition and library version bumps. For example: libicu72 became libicu76, libmagickwand-6.q16-6 became libmagickwand-7.q16-10, and libzip4 became libzip5. Always verify package names against the base image’s Debian version.

Q5. How do I find the correct runtime library names if I change the base image?
A. Run a temporary container and use ldd to check for missing libraries:
docker run --rm your-image bash -c \
"for so in /usr/local/lib/php/extensions//.so; do \
ldd \$so 2>/dev/null | grep 'not found'; \
done"
Then search for the correct package with apt-cache search .

Q6. What does validate_timestamps=0 mean for deployments?
A. OPcache will never check if PHP files on disk have changed. This avoids filesystem stat calls on every request (faster). The tradeoff: after deploying new code, you must restart PHP-FPM (docker restart ) to pick up changes.

Q7. Why create a non-root user?
A. Running as root inside a container means a compromised application has full control over the container filesystem. A non-root user limits the damage.

Q8. Can I add Xdebug for local development?
A. Yes, but do not include it in the production image. Use a separate development stage or a Docker Compose override file that installs Xdebug on top of the production image.

9. Key Takeaways

Multi-stage builds separate build-time complexity from runtime simplicity.
install-php-extensions eliminates the manual dance of installing dev packages, compiling, and cleaning up.
Always match runtime library package names to the base image’s Debian version (Trixie uses libicu76, libzip5, libmagickwand-7.q16-10, libavif16).
OPcache with validate_timestamps=0 is a significant performance win for production.
Security hardening (expose_php=Off, display_errors=Off, allow_url_fopen=Off) should be the default, not an afterthought.
Non-root user + health check + .dockerignore round out a production-ready setup.
Use ldd to diagnose missing shared libraries when extensions fail to load.

10. Conclusion

This multi-stage Dockerfile provides a clean, repeatable way to build a PHP 8.4-FPM production image with everything needed for WordPress and Drupal projects. The builder stage handles all the heavy lifting - compiling extensions, installing Composer, WP-CLI, Drush, Drupal Console, and Node.js - while the production stage starts fresh and copies only what is needed at runtime. Combined with OPcache tuning, security hardening, a non-root user, and a health check, the result is an image that is smaller, faster, and more secure than a traditional single-stage build. When paired with Nginx via Docker Compose, it forms a solid foundation for deploying PHP applications in any environment.

About the Author:Rajan is a DevOps Engineer at AddWebSolution, specializing in automation infrastructure, Optimize the CI/CD Pipelines and ensuring seamless deployments.

Architecting Large-Scale Next.js Applications (Folder Structure, Patterns, Best Practices)

Mayank Goyal — Mon, 13 Apr 2026 10:25:46 +0000

“Good architecture makes the system easy to understand; great architecture makes it hard to break.”

Key Takeaways

Feature-based architecture is critical
Separate UI, logic, and data layers
Prefer server components for performance
Centralize API logic
Use scalable state management
Optimize early, not later

Index

Introduction
Why Architecture Matters
Core Principles of Scalable Architecture
Advanced Folder Structure (Enterprise-Level)
Architectural Patterns (Deep Dive)
State Management at Scale
Data Fetching & API Layer Design
Authentication & Authorization Architecture
Performance Optimization (Advanced)
Error Handling & Logging
Testing Strategy (Production-Ready)
Dev Experience & Code Quality
Deployment & Infrastructure Strategy
Real-World Example (Enterprise Dashboard)
Interesting Facts
Stats
FAQ’s
Conclusion

Introduction

Building small Next.js apps is easy. Scaling them to support millions of users, multiple developers, and complex business logic is not.
Large-scale applications require:

Clean architecture
Predictable structure
Separation of concerns
Strong conventions Next.js (especially App Router) provides powerful primitives, but it does NOT enforce architecture, that’s your responsibility.

This guide gives you a production-grade blueprint.

Why Architecture Matters

At scale, poor architecture leads to:

Tight coupling between components
Duplicate logic across features
Slow builds & performance bottlenecks
Difficult onboarding for new developers

Good architecture enables:

Independent feature development
Faster debugging
Better scalability
Easier refactoring

Core Principles of Scalable Architecture

1. Separation of Concerns
Divide responsibilities:

UI → components
Logic → hooks/services
Data → API layer

2. Feature Isolation
Each feature should be self-contained.
Think like mini-apps inside your app.

3. Single Responsibility Principle
Each file/module should do one thing well.

4. Dependency Direction
Components depend on hooks
Hooks depend on services
Services depend on APIs
NOT the other way around.

5. Scalability First Mindset
Design for scale even if you’re small today.

Advanced Folder Structure (Enterprise-Level)

src/
│
├── app/                         # Next.js App Router
│   ├── (public)/
│   ├── (auth)/
│   ├── dashboard/
│   │   ├── layout.tsx
│   │   ├── page.tsx
│
├── features/                    # Feature modules (CORE)
│   ├── auth/
│   │   ├── components/
│   │   ├── hooks/
│   │   ├── services/
│   │   ├── api/
│   │   ├── store/
│   │   └── types.ts
│   │
│   ├── products/
│   ├── orders/
│   └── users/
│
├── shared/                      # Cross-feature reusable code
│   ├── components/
│   ├── hooks/
│   ├── utils/
│   └── constants/
│
├── core/                        # App-level logic
│   ├── config/
│   ├── providers/
│   ├── middleware/
│   └── guards/
│
├── services/                    # Global services (rare)
├── lib/                         # Low-level utilities
├── types/                       # Global types
├── styles/
└── tests/

“Fetching data is easy. Fetching it efficiently is architecture.”

Key Insight
Avoid “global chaos” folders like components/ for everything
Prefer feature-based grouping

Architectural Patterns (Deep Dive)

1. Feature-Based Architecture (MOST IMPORTANT)
Each feature owns:
UI
logic
API calls
state

features/products/
    components/
    hooks/
    services/
    store/

2. Layered Architecture

UI Layer (Components)
↓
Hooks Layer (Business Logic)
↓
Service Layer (API Calls)
↓
API Layer (External systems)

3. Server vs Client Component Strategy

Example:

// Server Component
export default async function Page() {
  const data = await getProducts();
  return <ProductsClient data={data} />;
}

4. Smart vs Dumb Components
Smart → fetch + logic
Dumb → UI only

5. Composition Pattern
Avoid inheritance. Use composition:

<Card>
  <ProductInfo />
</Card>

State Management at Scale

When NOT to use global state

Static data
Server-fetched data

Recommended Strategy

Example (Zustand Advanced)

import { create } from 'zustand';
export const useCartStore = create((set) => ({
  items: [],
  addItem: (item) =>
    set((state) => ({ items: [...state.items, item] })),
  clearCart: () => set({ items: [] })
}));

Data Fetching & API Layer Design

Bad Practice
Calling fetch directly in components everywhere.

Good Practice

features/products/
  services/productService.ts

export const getProducts = async () => {
  const res = await fetch('/api/products');
  return res.json();
};

“Every unnecessary render is a tax on your use.”

Caching Strategy

Authentication & Authorization Architecture

Recommended Setup

Middleware for route protection
Server-side session validation
Role-based access Example:

// middleware.ts
export function middleware(req) {
  const token = req.cookies.get('token');
  if (!token) return Response.redirect('/login');
}

Performance Optimization (Advanced)

Techniques

Code splitting (dynamic imports)
Partial hydration
Edge rendering
Image optimization
Memoization

Example

const Chart = dynamic(() => import('./Chart'), {
  ssr: false
});

Error Handling & Logging

Centralized Error Handling

export const handleError = (error) => {
  console.error(error);
};

Logging Tools

Sentry
LogRocket

Testing Strategy (Production-Ready)

Example

test('adds item to cart', () => {
  const store = useCartStore.getState();
  store.addItem({ id: 1 });
  expect(store.items.length).toBe(1);
});

Dev Experience & Code Quality

ESLint + Prettier
Husky (pre-commit hooks)
Strict TypeScript
Absolute imports (@/features/...)

Deployment & Infrastructure Strategy

Recommended Stack

Hosting → Vercel
DB → PostgreSQL
CDN → Cloudflare
Monitoring → Sentry

Scaling Tips

Use Edge Functions
Optimize bundle size
Enable caching

Real-World Example (Enterprise Dashboard)

Structure

features/dashboard/
   components/
      StatsCard.tsx
   hooks/
      useStats.ts
   services/
      dashboardService.ts

Service

export const fetchStats = async () => {
  const res = await fetch('/api/stats');
  return res.json();
};

Hook

export const useStats = () => {
  const [stats, setStats] = useState(null);

  useEffect(() => {
    fetchStats().then(setStats);
  }, []);

  return stats;
};

Component

export const StatsCard = ({ data }) => {
  return <div>{data.totalUsers}</div>;
};

Page

export default function DashboardPage() {
  const stats = useStats();

  if (!stats) return <p>Loading...</p>;

  return <StatsCard data={stats} />;
}

Interesting Facts

Next.js App Router defaults to Server Components.Source: https://nextjs.org/docs/app/building-your-application/rendering/server-components
Middleware runs at the Edge.Source: https://nextjs.org/docs/pages/api-reference/file-conventions/middleware
File-based routing reduces ~40% boilerplate.Source: https://nextjs.org/docs/app/building-your-application/routing
Built-in optimizations replace many libraries.Source: https://nextjs.org/docs/architecture
ISR allows hybrid static + dynamic pages.Source: https://nextjs.org/docs/pages/building-your-application/rendering/incremental-static-regeneration

Stats

Next.js is one of the fastest-growing React frameworks and is widely adopted for production-grade applications. Source: https://nextjs.org/showcase
Next.js has 120,000+ stars on GitHub, reflecting its strong developer adoption and community support. Source: https://github.com/vercel/next.js
According to the State of JS survey, Next.js consistently ranks among the top frameworks in developer satisfaction and usage. Source: https://stateofjs.com/
Vercel, the company behind Next.js, serves billions of requests per week across applications deployed on its platform. Source: https://vercel.com/customers
Next.js enables hybrid rendering (SSR, SSG, ISR), which improves performance and scalability for modern web applications. Source: https://nextjs.org/docs/pages/building-your-application/rendering
File-based routing in Next.js simplifies development by automatically mapping files to routes, reducing manual configuration. Source: https://nextjs.org/docs/app/building-your-application/routing

FAQ’s

Q1. Is Redux necessary?
No. Use Zustand unless you need complex workflows.

Q2. How to organize large teams?

Feature ownership
Code reviews
Clear folder structure

Q3. Should I use monorepo?
Yes, for multi-app systems (Nx / Turborepo).

Q4. Where to keep reusable components?
shared/components

Q5. What is the biggest mistake?
Mixing everything in global folders.

Conclusion

Scaling a Next.js application is more about architecture than code. The difference between a messy app and a scalable system lies in:

Structure
Discipline
Consistency By following feature-based design, layered architecture, and modern Next.js patterns, you can build applications that scale effortlessly with both users and teams.

About the Author:Mayank is a web developer at AddWebSolution, building scalable apps with PHP, Node.js & React. Sharing ideas, code, and creativity.

ELK Stack Setup for Centralized Log Management & Monitoring

Rajan Vavadia — Wed, 08 Apr 2026 11:42:01 +0000

“The goal is to turn data into information, and information into insight.” - Carly Fiorina

Introduction
Stage 1: Elasticsearch - The Search & Storage Engine
Stage 2: Logstash - The Data Processing Pipeline
Stage 3: Kibana - The Visualization Layer
Stage 4: Filebeat - The Lightweight Log Shipper
Connecting the Pieces - End-to-End Data Flow
Practical Setup (Step-by-step)
Troubleshooting Common Issues
Quotes
FAQs
Key Takeaways
Conclusion

1. Introduction

When your application runs on a single server, tailing a log file is enough. When it runs across multiple servers, containers, or microservices - you need centralized logging. Scattered logs across dozens of servers make debugging slow, error correlation impossible, and incident response reactive instead of proactive.

This guide walks through setting up a production-ready ELK stack (Elasticsearch, Logstash, Kibana) with Filebeat for centralized log collection, processing, and visualization. The setup covers a real-world scenario: a Java Spring Boot application running on one server, with the ELK stack on a separate server.

The approach uses a four-component architecture:

Filebeat (Application Server): Lightweight agent that tails log files and ships them to Logstash.
Logstash (ELK Server): Receives raw logs, parses and transforms them, and forwards structured data to Elasticsearch.
Elasticsearch (ELK Server): Stores, indexes, and makes logs searchable in near real-time.
Kibana (ELK Server): Web UI for searching, visualizing, and building dashboards from log data.

Why centralized logging?
Manually SSH-ing into each server and grepping through log files does not scale. Centralized logging solves this by aggregating all logs into a single searchable location.
You get:

Single pane of glass for all application and infrastructure logs.
Real-time search across millions of log entries in milliseconds.
Correlation of events across services and servers by timestamp.
Alerting on error patterns before users report issues.
Retention and compliance with configurable index lifecycle policies.

Component Responsibilities

Why separate servers?

- Resource isolation: Elasticsearch is memory-hungry. Running it on the application server competes with your app for RAM and CPU.
- Independent scaling: You can scale the ELK server (more RAM, bigger disk) without touching production application servers.
- Security boundary: The ELK server can sit in a private subnet, accessible only to internal services and authorized users.

2. Stage 1: Elasticsearch - The Search & Storage Engine

2.1 What is Elasticsearch?
Elasticsearch is a distributed search and analytics engine built on Apache Lucene. In the ELK stack, it serves as the storage and search backend - every log line that Logstash processes ends up as a document in an Elasticsearch index.

2.2 Installation
Import the Elasticsearch GPG key
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

Add the APT repository
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

Install Elasticsearch
sudo apt-get update && sudo apt-get install -y elasticsearch

2.3 Configuration
The main configuration file is /etc/elasticsearch/elasticsearch.yml:
Cluster and node identity
cluster.name: my-cluster
node.name: node-1

Data and log paths
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

Network - bind to all interfaces for external access
network.host: 0.0.0.0
http.host: 0.0.0.0

Discovery - single node (no cluster formation)
discovery.type: single-node

Security - disable for internal/dev setups
xpack.security.enabled: false
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
enabled: false
xpack.security.transport.ssl:
enabled: false

Configuration settings explained:

Security note: xpack.security.enabled: false is acceptable for internal/development setups behind a firewall. For production environments exposed to the internet, always enable security with TLS certificates and user authentication.

2.4 Start and Enable
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch

2.5 Verify
curl http://localhost:9200
Expected response:
{
"name": "node-1",
"cluster_name": "my-cluster",
"version": {
"number": "8.x.x"
},
"tagline": "You Know, for Search"
}

2.6 Memory Considerations
Elasticsearch defaults to a 1GB heap (-Xms1g -Xmx1g). For production:

Set heap to 50% of available RAM, but never more than 31GB (to stay within compressed OOPs).
Edit /etc/elasticsearch/jvm.options.d/heap.options: -Xms2g -Xmx2g
Ensure the system has enough RAM for both the JVM heap and filesystem cache (Lucene relies heavily on OS page cache).

3. Stage 2: Logstash - The Data Processing Pipeline

3.1 What is Logstash?
Logstash is a server-side data processing pipeline that ingests data from multiple sources, transforms it, and sends it to Elasticsearch. It sits between Filebeat and Elasticsearch, adding structure to raw log lines.

3.2 Installation
sudo apt-get install -y logstash

3.3 Pipeline Configuration
Logstash pipelines are defined in /etc/logstash/conf.d/. Each pipeline has three sections: input, filter, and output.
Create /etc/logstash/conf.d/boardgame.conf:
input {
beats {
port => 5044
}
}

filter {
#Parse Spring Boot log format
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel} %{GREEDYDATA:logmessage}" }
}
}

output {
elasticsearch {
hosts => ["localhost:9200"]
index => "boardgame-logs-%{+YYYY.MM.dd}"
}
}
Pipeline sections explained:

Input - Where data comes from:
input {
beats {
port => 5044
}
}

Filter - How data is transformed:
filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel} %{GREEDYDATA:logmessage}" }
}
}

The grok filter parses unstructured log lines into structured fields (timestamp, loglevel, logmessage). This enables filtering by log level in Kibana (e.g., show only ERROR logs).

Output - Where data goes:
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "boardgame-logs-%{+YYYY.MM.dd}"
}
}

Why daily indices? Daily indices make retention management simple - delete old indices by date. They also improve search performance because Elasticsearch can skip entire indices when querying a specific time range.

3.4 Start and Enable
sudo systemctl daemon-reload
sudo systemctl enable logstash
sudo systemctl start logstash

3.5 Verify
Check if Logstash is listening on port 5044
sudo ss -tlnp | grep 5044

Check Logstash logs for pipeline startup
sudo journalctl -u logstash --no-pager -n 20
Look for: Pipeline started {"pipeline.id"=>"main"} in the logs.

4. Stage 3: Kibana - The Visualization Layer

4.1 What is Kibana?
Kibana is the web interface for the ELK stack. It connects to Elasticsearch and provides tools for searching logs (Discover), building visualizations (charts, graphs, maps), and creating dashboards for real-time monitoring.

4.2 Installation
sudo apt-get install -y kibana

4.3 Configuration
Edit /etc/kibana/kibana.yml:
Bind to all interfaces for external access
server.host: "0.0.0.0"

Connect to Elasticsearch over plain HTTP
elasticsearch.hosts: ["http://localhost:9200"]

Critical configuration points:

Common pitfall: If Elasticsearch has xpack.security.http.ssl.enabled: false but Kibana is configured with https:// in elasticsearch.hosts, Kibana will fail to connect with Unable to retrieve version information from Elasticsearch. Always match the protocol.

Settings to remove or comment out when SSL is disabled:
Comment out or remove these lines
elasticsearch.ssl.certificateAuthorities: [/path/to/ca.crt]
elasticsearch.username: "kibana_system"
elasticsearch.password: "pass"

4.4 Start and Enable
sudo systemctl daemon-reload
sudo systemctl enable kibana
sudo systemctl start kibana

Kibana takes 30–60 seconds to fully initialize.

4.5 Verify
curl http://localhost:5601/api/status

Expected: {"status":{"overall":{"level":"available"}}} - the level should be available, not unavailable.

4.6 Create a Data View
Once data is flowing, create a data view so Kibana knows which indices to query:

Open http://:5601 in a browser.
Navigate to Stack Management > Data Views (under Kibana section).
Click Create data view.
Fill in the fields:

Now go to Discover (under Analytics) to search and explore your logs.

4.7 AWS Security Group
If running on AWS EC2, ensure the security group allows inbound traffic:

Never expose port 9200 to the public internet unless Elasticsearch security is enabled with TLS and authentication.

“Logs are the immune system of your infrastructure - they tell you when something is wrong before it becomes a crisis.” - DevOps principle

5. Stage 4: Filebeat - The Lightweight Log Shipper

5.1 What is Filebeat?
Filebeat is a lightweight log shipper that runs on the application server. It tails log files, handles log rotation, tracks read positions (so it never sends duplicate lines), and ships logs to Logstash or Elasticsearch with minimal resource overhead.

5.2 Why Filebeat instead of sending directly to Logstash?

Filebeat decouples your application from the logging pipeline. If Logstash or Elasticsearch goes down, Filebeat queues events and retries automatically. Your application keeps writing to its log file without interruption.

5.3 Installation (on the Application Server)
Use the same Elastic repository
sudo apt-get update && sudo apt-get install -y filebeat

5.4 Configuration
Edit /etc/filebeat/filebeat.yml:
filebeat.inputs:

type: log enabled: true paths:
- /home/ubuntu/Boardgame/target/app.log multiline.pattern: '^\d{4}-\d{2}-\d{2}' multiline.negate: true multiline.match: after

output.logstash:
hosts: [":5044"]

processors:

add_host_metadata: when.not.contains.tags: forwarded
add_cloud_metadata: ~

logging.level: info

Configuration explained:
Input section:

Multiline settings (critical for Java/Spring Boot stack traces):

This ensures that a multi-line Java stack trace is treated as a single log event, not dozens of separate lines:
2026-03-11 06:37:55 ERROR Something went wrong
java.lang.NullPointerException
at com.example.Service.process(Service.java:42)
at com.example.Controller.handle(Controller.java:15)
Without multiline config, each line of the stack trace becomes a separate Elasticsearch document - making it impossible to correlate errors.

Output section:

Processors:

YAML formatting rules (common source of errors):

5.5 Start and Enable
Clear old registry to re-read files from the beginning
sudo rm -rf /var/lib/filebeat/registry

sudo systemctl daemon-reload
sudo systemctl enable filebeat
sudo systemctl start filebeat

5.6 Verify
Check Filebeat status
sudo systemctl status filebeat

Check logs - look for "Harvester started"
sudo journalctl -u filebeat --no-pager -n 20
Key indicators in the logs:

6. Connecting the Pieces - End-to-End Data Flow

6.1 The Complete Pipeline
[Spring Boot App]
│
│ writes logs to disk
▼
[/home/ubuntu/Boardgame/target/app.log]
│
│ Filebeat tails the file
▼
[Filebeat] ──── port 5044 ────► [Logstash]
│
│ grok filter parses log lines
▼
[Elasticsearch]
index: boardgame-logs-2026.03.11
│
│ Kibana queries the index
▼
[Kibana]
http://:5601

6.2 Verifying Each Link
Test each component in order, from bottom to top:

Elasticsearch is running and accessible
curl http://localhost:9200
Logstash is listening for Beats input
sudo ss -tlnp | grep 5044
Kibana can reach Elasticsearch
curl http://localhost:5601/api/status
Filebeat can reach Logstash (from application server)
telnet 5044
Data is actually in Elasticsearch
curl http://localhost:9200/_cat/indices?v | grep boardgame
Check document count
curl http://localhost:9200/boardgame-logs-*/_count

7. Practical Setup (Step-by-step)

7.1 Server Requirements

7.2 ELK Server Setup (in order)

Step 1: Add Elastic repository
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
sudo apt-get update

Step 2: Install all three components
sudo apt-get install -y elasticsearch logstash kibana

Step 3: Configure Elasticsearch
sudo nano /etc/elasticsearch/elasticsearch.yml
Set: network.host: 0.0.0.0
Set: discovery.type: single-node
Set: xpack.security.enabled: false
Set: xpack.security.http.ssl.enabled: false
Set: xpack.security.transport.ssl.enabled: false

Step 4: Configure Logstash pipeline
sudo nano /etc/logstash/conf.d/boardgame.conf
Add input (beats, port 5044), filter (grok), output (elasticsearch)

Step 5: Configure Kibana
sudo nano /etc/kibana/kibana.yml
Set: server.host: "0.0.0.0"
Set: elasticsearch.hosts: ["http://localhost:9200"]
Remove/comment any SSL certificate lines

Step 6: Start services
sudo systemctl enable elasticsearch logstash kibana
sudo systemctl start elasticsearch logstash kibana

7.3 Application Server Setup

Step 1: Install Filebeat
sudo apt-get install -y filebeat

Step 2: Configure Filebeat
sudo nano /etc/filebeat/filebeat.yml
Replace entire file with clean config (see Section 6.4)

Step 3: Clear registry and start
sudo rm -rf /var/lib/filebeat/registry
sudo systemctl enable filebeat
sudo systemctl start filebeat

7.4 Kibana Data View Setup

Verify data arrived in Elasticsearch
curl http://:9200/_cat/indices?v | grep boardgame
Then in the Kibana UI: 1. Stack Management > Data Views > Create data view 2. Name: Boardgame Logs, Index pattern: boardgame-logs-*, Timestamp: @timestamp 3. Save data view to Kibana 4. Go to Discover to explore your logs

7.5 Optional: Index Lifecycle Management

For production, configure automatic index cleanup to prevent disk from filling up:
Create an ILM policy that deletes indices older than 30 days
curl -X PUT "http://localhost:9200/_ilm/policy/boardgame-logs-policy" -H 'Content-Type: application/json' -d'
{
"policy": {
"phases": {
"hot": {
"actions": {
"rollover": {
"max_size": "5gb",
"max_age": "1d"
}
}
},
"delete": {
"min_age": "30d",
"actions": {
"delete": {}
}
}
}
}
}'

8. Troubleshooting Common Issues

8.1 Kibana shows “Unable to retrieve version information from Elasticsearch”

Cause: Protocol mismatch - Kibana is using https:// but Elasticsearch has SSL disabled.
Fix:
Check current Kibana config
sudo grep "elasticsearch.hosts" /etc/kibana/kibana.yml

Fix: change https to http
elasticsearch.hosts: ["http://localhost:9200"]

Comment out any SSL certificate lines
elasticsearch.ssl.certificateAuthorities: [...]

sudo systemctl restart kibana

8.2 Filebeat shows harvester.open_files: 0

Cause: Filebeat config is malformed (duplicate sections, wrong indentation, or double-quoted regex).
Fix: - Ensure filebeat.yml has no duplicate top-level keys. - Use single quotes for multiline.pattern (YAML treats \d in double quotes as an escape sequence). - Top-level keys (filebeat.inputs:, output.logstash:, processors:) must start at column 0.
Validate the config
sudo filebeat test config -c /etc/filebeat/filebeat.yml

Clear registry and restart
sudo rm -rf /var/lib/filebeat/registry
sudo systemctl restart filebeat

8.3 No boardgame-logs-* indices in Elasticsearch

Cause: Filebeat cannot reach Logstash on port 5044.
Fix:
From the application server, test connectivity
telnet 5044

If connection refused - open port 5044 in the ELK server's security group
If connection times out - check if Logstash is running
sudo systemctl status logstash

8.4 Kibana Data View shows “No data streams, indices, or index aliases match”

Cause: No data has been ingested yet, or the index pattern is wrong.
Fix:
List all indices
curl http://localhost:9200/_cat/indices?v

Check the exact index name and match your pattern accordingly
If index is "boardgame-logs-2026.03.11", pattern should be "boardgame-logs-*"

8.5 Logstash is running but not receiving data

Cause: Logstash pipeline failed to start, or the config file has syntax errors.
Fix:
Test the Logstash config
sudo /usr/share/logstash/bin/logstash --config.test_and_exit -f /etc/logstash/conf.d/

Check Logstash logs
sudo journalctl -u logstash --no-pager -n 30

“You can’t manage what you can’t measure.” - Peter Drucker

9. FAQs

Q1. Why use the ELK stack instead of cloud-native logging (CloudWatch, Stackdriver)?
ELK gives you full control over data retention, parsing rules, and costs. Cloud logging services charge per GB ingested, which becomes expensive at scale. ELK is free (open-source) - you only pay for the infrastructure.

Q2. Why Filebeat instead of sending logs directly from the application?
Filebeat decouples your application from the logging pipeline. If Logstash or Elasticsearch goes down, Filebeat queues events and retries. Your application keeps running without blocking on log delivery. Filebeat also uses ~10–30 MB RAM versus Logstash’s 500 MB+, making it ideal for application servers.

Q3. Can I skip Logstash and send Filebeat directly to Elasticsearch?
Yes. Set output.elasticsearch instead of output.logstash in Filebeat. However, you lose the ability to parse and transform logs with grok filters. For simple use cases (no parsing needed), direct shipping is fine.

Q4. Why daily indices (boardgame-logs-2026.03.11) instead of a single index?
Daily indices enable simple retention management (delete indices older than N days), improve search performance (Elasticsearch skips irrelevant time ranges), and make index management operations (backup, restore) more granular.

Q5. How much disk space does the ELK stack need?
Rough estimate: 1 GB of raw logs produces ~1.5–2 GB of Elasticsearch data (due to indexing overhead). For 100 MB/day of logs with 30-day retention, budget ~6 GB for Elasticsearch data.

Q6. Why must multiline.pattern use single quotes in YAML?
YAML interprets backslash sequences in double-quoted strings (\d becomes an invalid escape). Single quotes treat the content literally, preserving the regex pattern for Filebeat.

Q7. How do I add more application servers?
Install Filebeat on each server, point it to the same Logstash endpoint. Add a fields section to distinguish servers:
filebeat.inputs:

type: log enabled: true paths:
- /var/log/myapp/*.log fields: server_name: web-02 environment: production

Q8. Is this setup production-ready as described?
For internal use, yes. For public-facing production, enable Elasticsearch security (xpack.security.enabled: true), use TLS certificates for all inter-component communication, put Kibana behind a reverse proxy with authentication, and configure Index Lifecycle Management for automatic cleanup.

10. Key Takeaways

Centralized logging transforms debugging from “SSH into each server and grep” to “search once, find everywhere.”
Filebeat belongs on the application server, not the ELK server. It is lightweight (~10–30 MB RAM) and handles backpressure gracefully.
Logstash’s grok filter turns unstructured log lines into structured, searchable fields (timestamp, loglevel, logmessage).
Protocol mismatch (https:// vs http://) between Kibana and Elasticsearch is the most common setup failure. Always match the protocol to Elasticsearch’s actual SSL configuration.
YAML formatting causes most Filebeat config errors - use single quotes for regex, no duplicate keys, no leading spaces on top-level keys.
Daily indices (boardgame-logs-%{+YYYY.MM.dd}) simplify retention management and improve query performance.
Security is not optional in production - enable xpack.security, use TLS, and restrict port access via security groups.
Test connectivity bottom-up: Elasticsearch first, then Logstash, then Kibana, then Filebeat.

11. Conclusion

Setting up the ELK stack is not just about installing four packages - it is about building a reliable data pipeline that turns scattered log files into searchable, visualizable insights. Each component has a clear role: Filebeat ships, Logstash transforms, Elasticsearch stores, and Kibana visualizes.

The most common failures are not in the software itself, but in the configuration glue between components: protocol mismatches between Kibana and Elasticsearch, YAML formatting errors in Filebeat, unopened firewall ports between servers, and missing runtime dependencies.

By following the step-by-step approach in this guide - installing bottom-up (Elasticsearch → Logstash → Kibana → Filebeat), verifying each component before moving to the next, and understanding why each configuration setting exists - you can set up a production-grade centralized logging system that scales from a single application to dozens of services.

Once the pipeline is flowing, the real value begins: building dashboards for error rates, setting up alerts for anomalies, and turning your logs from an afterthought into your first line of defense.

About the Author:Rajan is a DevOps Engineer at AddWebSolution, specializing in automation infrastructure, Optimize the CI/CD Pipelines and ensuring seamless deployments.

Nginx + PHP + MySQL Optimisations and Parameter Calculations

Narendra Chauhan — Fri, 03 Apr 2026 06:38:00 +0000

“Premature optimization is the root of all evil but lack of optimisation is the root of outages.”

Introduction
Architecture Overview
Why Optimisation Is Required
Nginx Optimisations & Parameter Calculations
PHP-FPM Optimisations & Parameter Calculations
MySQL Optimisations & Parameter Calculations
System-Level Optimisations (Linux)
Practical Example: Small vs Medium vs Large Server
Interesting Facts & Statistics
FAQs
Key Takeaways
Conclusion

1. Introduction

Modern web applications rely heavily on the Nginx + PHP + MySQL (LEMP) stack. While default configurations work for testing, they are not suitable for production traffic.

Optimisation ensures:

Faster page load time
Better concurrency handling
Lower memory and CPU usage
Higher stability under load

This document explains what to optimise, why to optimise, and how to calculate parameters practically.

2. Architecture Overview

A typical request flow:

Client sends HTTP request
Nginx handles connection & static content
PHP-FPM processes dynamic PHP requests
MySQL serves data from database
Response sent back to client

Each layer must be tuned together, not individually.

3. Why Optimisation Is Required

Default settings:

Are conservative
Waste available RAM
Limit concurrency
Cause slow response under load

Common Problems Without Optimisation

502 / 504 Gateway errors
High CPU load
PHP-FPM “server reached max_children”
MySQL “Too many connections”

4. Nginx Optimisations & Parameter Calculations

Key Nginx Parameters
worker_processes auto;
worker_connections 4096;

What is worker_processes
Number of worker processes
Best practice: match CPU cores

nproc
Example:
4 CPU cores → worker_processes 4;
What is worker_connections
Maximum connections per worker

Total Max Connections
worker_processes × worker_connections

Example

4 × 4096 = 16,384 concurrent connections

Recommended Extra Optimisations

use epoll;
multi_accept on;

sendfile on;
tcp_nopush on;
tcp_nodelay on;

keepalive_timeout 65;
keepalive_requests 1000;

5. PHP-FPM Optimisations & Parameter Calculations

Key PHP-FPM Settings

pm = dynamic
pm.max_children = 20
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 10

How to Calculate pm.max_children

Find average PHP process memory: ps -ylC php-fpm --sort:rss
Formula: Available RAM for PHP / Avg PHP process size

Example

Available RAM: 2 GB
Avg PHP process: 100 MB

2048 / 100 ≈ 20
pm.max_children = 20

Other Important PHP Optimisations

request_terminate_timeout = 60
max_execution_time = 60
memory_limit = 256M

Enable OPcache

opcache.enable=1
opcache.memory_consumption=256
opcache.max_accelerated_files=20000

6. MySQL Optimisations & Parameter Calculations

Key MySQL Parameters

innodb_buffer_pool_size = 2G
innodb_buffer_pool_instances = 2
max_connections = 200

How to Calculate innodb_buffer_pool_size

Allocate 60–70% of total RAM (dedicated DB server)

Example

Server RAM: 4 GB

4 × 70% ≈ 2.8 GB
Use 2G or 3G

Connection Calculation

Additional Recommended Settings

query_cache_type = 0
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 2

7. System-Level Optimisations (Linux)

File Descriptors

ulimit -n 100000

Kernel Tuning

net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
vm.swappiness = 10

8. Practical Server Size Examples

Small Server (2 CPU / 2 GB RAM)

Nginx workers: 2
worker_connections: 2048
PHP max_children: 10
MySQL buffer pool: 1G

Medium Server (4 CPU / 8 GB RAM)

Nginx workers: 4
worker_connections: 4096
PHP max_children: 30–40
MySQL buffer pool: 4–5G

Large Server (8 CPU / 16 GB RAM)

Nginx workers: 8
worker_connections: 8192
PHP max_children: 60–80
MySQL buffer pool: 10–12G

Practical Demonstration (Images Explained)

Step 1. NGINX OPTIMISATION

Parameter Calculations
Step 1.1 Backup current nginx.conf

  sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak

Step 1.2 Check current config

  cat /etc/nginx/nginx.conf

Screenshot: Current state before changes

Step 1.3 Edit nginx.conf

sudo nano /etc/nginx/nginx.conf

Replace/update with this optimized config:

user www-data;
  worker_processes 2;                    # = nproc (2 cores)
  worker_rlimit_nofile 65535;
  pid /run/nginx.pid;
  include /etc/nginx/modules-enabled/*.conf;

  events {
      worker_connections 1024;           # 2 x 1024 = 2048 total connections
      use epoll;                         # Linux best event model
      multi_accept on;                   # Accept multiple connections at once
  }

  http {

      # Basic Settings
      sendfile on;
      tcp_nopush on;
      tcp_nodelay on;
      keepalive_timeout 30;              # Reduced from default 75s
      keepalive_requests 100;
      types_hash_max_size 2048;
      server_tokens off;                 # Hide nginx version

      client_max_body_size 20m;
      client_body_buffer_size 128k;
      client_header_buffer_size 1k;
      large_client_header_buffers 4 8k;
      client_body_timeout 12;
      client_header_timeout 12;
      send_timeout 10;

      include /etc/nginx/mime.types;
      default_type application/octet-stream;

      # Logging Settings
      access_log /var/log/nginx/access.log;
      error_log /var/log/nginx/error.log warn;   # Only warn+ to reduce I/O

      # Gzip Settings
      gzip on;
      gzip_vary on;
      gzip_proxied any;
      gzip_comp_level 3;                 # Level 3 = good ratio, low CPU
      gzip_min_length 1024;              # Don't compress tiny files
      gzip_buffers 16 8k;
      gzip_http_version 1.1;
      gzip_types
          text/plain
          text/css
          text/javascript
          application/javascript
          application/json
          application/xml
          image/svg+xml
          font/woff2;


      # Open File Cache
      open_file_cache max=1000 inactive=20s;
      open_file_cache_valid 30s;
      open_file_cache_min_uses 2;
      open_file_cache_errors on;

      # FastCGI Cache (optional - enable per site)
      fastcgi_cache_path /var/cache/nginx levels=1:2 keys_zone=PHPCACHE:10m
                         max_size=100m inactive=60m use_temp_path=off;

      include /etc/nginx/conf.d/*.conf;
      include /etc/nginx/sites-enabled/*;
  }

Step 1.4 Create FastCGI cache directory

 sudo mkdir -p /var/cache/nginx
  sudo chown www-data:www-data /var/cache/nginx

Step 1.5 Test and reload Nginx

sudo nginx -t
  sudo systemctl reload nginx

Screenshot: nginx -t showing syntax is ok and test is successful

Step 1.6 Verify Nginx is running with new settings

sudo nginx -T | grep -E "worker_processes|worker_connections|gzip|keepalive_timeout"
systemctl status nginx

Screenshot: Running status + key parameters

Step 2. PHP-FPM OPTIMISATION

Parameter Calculations
Available RAM for PHP-FPM: ~150MB (conservative, leaving room for MySQL + Nginx)
Average PHP-FPM process size: ~30-40MB
Formula: pm.max_children = 150 / 35 ≈ 4

PHP-FPM Parameters Calculation
pm
Formula: Dynamic (best for variable traffic)
Value: dynamic

pm.max_children
Formula: 150MB ÷ 35MB/process
Value: 4

pm.start_servers
Formula: pm.max_children / 2
Value: 2

pm.min_spare_servers
Formula: pm.start_servers / 2
Value: 1

pm.max_spare_servers
Formula: pm.start_servers
Value: 2

pm.max_requests
Formula: Prevent memory leaks
Value: 500

Step 2.1 Check PHP-FPM version path

  php -v
  ls /etc/php/

Step 2.2 Backup pool config

sudo cp/etc/php/8.4/fpm/pool.d/www.conf/etc /php/8.4/fpm/pool.d/www.conf.bak
sudo cp /etc/php/8.4/fpm/php.ini /etc/php/8.4/fpm/php.ini.bak

Step 2.3 Check current PHP process memory usage

Run this to see actual PHP-FPM process sizes

ps aux | grep php-fpm | grep -v grep | awk '{sum += $6} END {print "Total RSS:", sum/1024, "MB"; print "Count:", NR; print "Avg per process:", sum/NR/1024, "MB"}'

Screenshot: Current process sizes.

Step 2.4 Edit PHP-FPM pool config

sudo nano /etc/php/8.4/fpm/pool.d/www.conf

Find and update these values (search with Ctrl+W in nano):

[www]
user = www-data
group = www-data

listen = /run/php/php8.4-fpm.sock
listen.owner = www-data
listen.group = www-data

pm = dynamic
pm.max_children = 4
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 2
pm.max_requests = 500     ;Restart workers after 500 requests (prevents memory leaks)
pm.process_idle_timeout = 10s

pm.status_path = /status  ; Enable FPM status page
ping.path = /ping

slowlog = /var/log/php8.4-fpm-slow.log
request_slowlog_timeout = 5s     ; Log requests taking > 5 seconds
security.limit_extensions = .php

Step 2.5 Tune PHP OPcache
sudo nano /etc/php/8.4/mods-available/opcache.ini

zend_extension=opcache

  ; Enable OPcache
  opcache.enable=1
  opcache.enable_cli=0

  ; Memory: 64MB for low-RAM server
  opcache.memory_consumption=64
  opcache.interned_strings_buffer=8
  opcache.max_accelerated_files=10000

  ; Production settings (set validate_timestamps=0 in prod)
  opcache.validate_timestamps=1
  opcache.revalidate_freq=60

  opcache.save_comments=1
  opcache.max_wasted_percentage=10
  opcache.use_cwd=1

  ; JIT (PHP 8.x feature)
  opcache.jit_buffer_size=32M
  opcache.jit=1255

Step 2.6 Tune PHP.ini key values
sudo nano /etc/php/8.4/fpm/php.ini

Find and update:
  ; Memory limit per PHP process
  memory_limit = 128M

  ; Upload/POST limits
  upload_max_filesize = 20M
  post_max_size = 25M
  max_execution_time = 60
  max_input_time = 60

  ; Error handling (production)
  display_errors = Off
  log_errors = On
  error_log = /var/log/php_errors.log

  ; Session handling
  session.gc_maxlifetime = 1440
  session.cookie_httponly = 1
  session.cookie_secure = 1

  ; Disable dangerous functions
  disable_functions = exec,passthru,shell_exec,system,proc_open,popen

Step 2.7 Restart PHP-FPM and verify

sudo php-fpm8.4 -t
  sudo systemctl restart php8.4-fpm
  sudo systemctl status php8.4-fpm

Screenshot: PHP-FPM status showing active (running)

Step 2.8 Verify OPcache is active

php -r "var_dump(opcache_get_status());" | head -30

- or check via CLI

php -i | grep -E "opcache|OPcache"

Screenshot: OPcache enabled status

Step 2.9 Check PHP-FPM processes after restart

  ps aux | grep php-fpm | grep -v grep

- Should show master + 2 worker processes (pm.start_servers=2)

Screenshot: FPM process list

Step 3. MYSQL OPTIMISATION

Parameter Calculations

RAM budget for MySQL: ~200MB (out of 914MB total)

innodb_buffer_pool_size
Formula: ~20% of RAM (shared server)
Value: 192M

innodb_buffer_pool_instances
Formula: buffer_pool ÷ 128M
Value: 1

max_connections
Formula: Low RAM = conservative
Value: 50

innodb_log_file_size
Formula: 25% of buffer pool
Value: 48M

tmp_table_size
Formula: Memory temporary tables
Value: 16M

max_heap_table_size
Formula: Same as tmp_table_size
Value: 16M

thread_cache_size
Formula: Reuse threads
Value: 8

table_open_cache
Formula: Open tables cache
Value: 400

Step 3.1 Check current MySQL config and status

sudo cp /etc/mysql/mysql.conf.d/mysqld.cnf /etc/mysql/mysql.conf.d/mysqld.cnf.bak

- Check current variables

mysql -u root -p -e "SHOW VARIABLES LIKE 'innodb_buffer_pool%';"
  mysql -u root -p -e "SHOW VARIABLES LIKE 'max_connections';"
  mysql -u root -p -e "SHOW STATUS LIKE 'Threads_connected';"

Screenshot: Current MySQL variables

Step 3.2 Check actual MySQL memory usage

mysql -u root -p -e "
  SELECT
    ROUND(@@innodb_buffer_pool_size/1024/1024, 0) AS 'Buffer Pool MB',
    ROUND(@@key_buffer_size/1024/1024, 0) AS 'Key Buffer MB',
    @@max_connections AS 'Max Connections',
    @@thread_stack/1024 AS 'Thread Stack KB';
  "

Screenshot: Current MySQL memory config

Step 3.3 Edit MySQL config

sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

Add/update under [mysqld]:

[mysqld]
  pid-file        = /var/run/mysqld/mysqld.pid
  socket          = /var/run/mysqld/mysqld.sock
  datadir         = /var/lib/mysql
  log-error       = /var/log/mysql/error.log

  #
  # ===== MEMORY SETTINGS =====
  # Server has 914MB RAM - allocate ~200MB for MySQL
  #
  innodb_buffer_pool_size         = 192M    # Main InnoDB cache (most important!)
  innodb_buffer_pool_instances    = 1       # 1 instance (< 1GB pool)
  innodb_log_file_size            = 48M     # ~25% of buffer pool
  innodb_log_buffer_size          = 8M
  innodb_flush_log_at_trx_commit  = 2       # Slight risk, big perf gain (use 1 for strict ACID)

  #
  # ===== CONNECTION SETTINGS =====
  #
  max_connections         = 50             # Low RAM = keep connections limited
  thread_cache_size       = 8             # Reuse threads, avoid creation overhead
  wait_timeout            = 120           # Kill idle connections after 2 min
  interactive_timeout     = 120

  #
  # ===== QUERY CACHE (removed in MySQL 8.0, skip) =====
  # MySQL 8.0 removed query_cache - use ProxySQL or app-level cache

  #
  # ===== TABLE CACHE =====
  #
  table_open_cache        = 400
  table_definition_cache  = 400

  #
  # ===== TEMP TABLES =====
  #
  tmp_table_size          = 16M
  max_heap_table_size     = 16M

  #
  # ===== InnoDB I/O =====
  #
  innodb_file_per_table           = 1
  innodb_flush_method             = O_DIRECT  # Avoid double buffering with OS cache
  innodb_read_io_threads          = 2         # = CPU cores
  innodb_write_io_threads         = 2         # = CPU cores

  #
  # ===== SLOW QUERY LOG =====
  #
  slow_query_log          = 1
  slow_query_log_file     = /var/log/mysql/slow.log
  long_query_time         = 2             # Log queries > 2 seconds
  log_queries_not_using_indexes = 1

  #
  # ===== BINARY LOG (disable if not using replication) =====
  #
  # skip-log-bin                           # Uncomment if no replication needed

Step 3.4 Validate and restart MySQL

sudo mysqld --validate-config
  sudo systemctl restart mysql
  sudo systemctl status mysql

Screenshot: MySQL status active

Step 3.5 Verify new MySQL variables

mysql -u root -p -e "
  SELECT 'innodb_buffer_pool_size' AS Variable,
         ROUND(@@innodb_buffer_pool_size/1024/1024,0) AS 'Value (MB)'
  UNION SELECT 'max_connections', @@max_connections
  UNION SELECT 'innodb_log_file_size MB', ROUND(@@innodb_log_file_size/1024/1024,0)
  UNION SELECT 'tmp_table_size MB', ROUND(@@tmp_table_size/1024/1024,0)
  UNION SELECT 'thread_cache_size', @@thread_cache_size;
  "

Screenshot: New values confirmed

Step 3.6 Check MySQL memory usage post-restart

  free -h
  ps aux | sort -k6 -rn | head -10

Screenshot: Overall memory after all services tuned

“Measure first, tune second, and monitor always.”

Step 4. FINAL VERIFICATION

Step 4.1 Check all services running

  systemctl status nginx php8.4-fpm mysql --no-pager

Step 4.2 Full memory picture

free -h
  ps aux --sort=-%mem | head -15

Step 4.3 Check nginx + PHP working together

Test nginx config is still valid
  sudo nginx -t

 Test PHP-FPM socket exists
  ls -la /run/php/php8.4-fpm.sock

 Check FPM status (if you added status page to your site config)
  curl http://localhost/status

Step 4.4 Check MySQL slow log is active

mysql -u root -p -e "SHOW VARIABLES LIKE 'slow_query%';"

Step 4.5 Final health summary

echo "=== NGINX ===" && nginx -v && systemctl is-active nginx
  echo "=== PHP-FPM ===" && php -v | head -1 && systemctl is-active php8.4-fpm
  echo "=== MYSQL ===" && mysql --version && systemctl is-active mysql
  echo "=== MEMORY ===" && free -h

Screenshot: Final health check

Your server is fully optimised. All phases verified:

Phase 0 Swap 2GB active
Phase 1 Nginx tuned
Phase 2 PHP-FPM + OPcache + JIT enabled
Phase 3 MySQL InnoDB / connections / slow query log active

9. Interesting Facts & Statistics

1 second delay can reduce conversions by 7%
OPcache can improve PHP performance by 2–3×
MySQL buffer pool cache hit ratio above 99% is ideal
Nginx handles 10× more concurrent connections than traditional servers

11. FAQs

Q1. Should I optimise Nginx, PHP, or MySQL first?
Start with PHP-FPM and MySQL, then tune Nginx.

Q2. Can wrong tuning crash the server?
Yes. Over-allocating RAM causes OOM kills.

Q3. Are these values fixed forever?
No. Recalculate after traffic growth.

Q4. Do I need load testing?
Yes. Use tools like ab, wrk, or k6.

12. Key Takeaways

Optimisation is calculation-based, not guesswork
PHP-FPM memory calculation is critical
MySQL buffer pool has the biggest performance impact
Nginx handles concurrency, not application logic
Monitoring is mandatory after tuning

13. Conclusion

Optimising Nginx + PHP + MySQL is not about copying configs from the internet—it is about understanding server resources, calculating limits, and balancing load across layers.
A well-optimised stack:

Handles higher traffic
Reduces downtime
Improves user experience
Saves infrastructure cost

About the Author:Narendra is a DevOps Engineer at AddWebSolution, specializing in automating infrastructure to improve efficiency and reliability.

Testing Node.js APIs: Jest, Supertest, and Best Practices

Zemichael Mehretu — Tue, 31 Mar 2026 10:21:20 +0000

“Fast tests are a productivity feature; reliable tests are a business feature.”

Key Takeaways

Reliable API tests are less about tool choice and more about test boundaries, data control, and deterministic execution.
Jest + Supertest remains a practical default stack for HTTP API testing across Express, Fastify, and Nest-based services.
The biggest gains come from balancing fast unit tests with focused integration and contract tests.
Flaky tests usually indicate architecture or environment issues, not just “test instability.”
A staged migration strategy outperforms rewriting the entire test suite at once.

Index

Introduction
Why Node.js API Test Suites Become Fragile Over Time
Testing APIs in Practical Terms
Core Testing Principles (The Guardrails)
The Four Testing Layers for Node.js APIs
Implementation Strategy for Engineering Teams
Minimal Example: Create User Endpoint with Jest + Supertest
Databases, Queues, and External Integrations
Testing Strategy by Risk and Feedback Speed
Migration Roadmap for Existing Node.js APIs
Common Mistakes and How to Avoid Themdevf nf.f/fk
When to Go Deep (and When to Keep It Lean)
FAQs
References
Interesting facts
Stats
Conclution

1) Introduction

Most Node.js teams start with a handful of endpoint tests and feel productive quickly. The challenge appears later: as endpoints multiply, integrations grow, and multiple teams contribute, test suites get slower, noisier, and harder to trust.

At that point, the question changes from “Do we have tests?” to “Can we safely change behavior without fear?”

Jest and Supertest are still a strong pair for this problem:

Jest provides a mature runner, mocking, snapshots (when used carefully), parallelism controls, and strong TypeScript support.
Supertest makes HTTP assertions straightforward by testing your app server boundary with minimal setup. This guide focuses on practical, production-grade API testing: clear test architecture, reliable CI execution, and patterns that hold up as systems grow.

2) Why Node.js API Test Suites Become Fragile Over Time

Most API testing pain comes from boundary confusion, not from JavaScript itself.

Common symptoms:

Endpoint tests that also validate unrelated business rules, persistence details, and third-party behavior in one place.
Excessive mocks that make tests pass while production behavior fails.
Slow suites due to global setup, shared mutable state, or non-isolated databases.
Flaky tests caused by timing assumptions, network dependency, or race conditions.
Hard-to-debug failures because assertions are too broad (“status should be 200”) and not intent-focused. As these issues accumulate, teams lose confidence. Developers re-run tests repeatedly, CI cycles grow, and delivery speed drops.

3) Testing APIs in Practical Terms

Effective API testing separates policy from transport detail.

Policy: validation rules, authorization decisions, business invariants, idempotency behavior.
Detail: HTTP framework wiring, DB driver specifics, queue providers, external SDK mechanics.

In practice:

Test business behavior close to the domain/service layer where feedback is fast.
Test HTTP contracts at the API boundary (status codes, schema shape, headers).
Test integration seams (database, message broker, external APIs) explicitly and intentionally.
Keep each test focused on one intent so failures explain what broke. A useful test heuristic: if your storage engine changes, most behavioral tests should remain valid.

4) Core Testing Principles (The Guardrails)

Principle 1: Test Behavior, Not Implementation
Prefer assertions on outcomes (response payload, side effects, emitted events) over internal call counts unless interaction itself is the behavior.

Principle 2: Keep Tests Deterministic

Control time, randomness, and external I/O. Non-determinism is the fastest path to flaky pipelines.

Principle 3: Isolate by Layer

Use unit tests for logic branches, integration tests for boundaries, and endpoint tests for HTTP contract confidence.

Principle 4: Keep Test Data Intentional

Use builders/factories with explicit defaults. Hidden fixture coupling creates accidental test dependencies.

Principle 5: Make Failures Actionable

Every failure should quickly answer:

What business behavior regressed?
Which boundary failed?
Is this deterministic or flaky?

Principle 6: Optimize for Feedback Loop

Developers need fast local tests and reliable CI gates. Prioritize speed for high-frequency paths.

“Contract tests protect teams from integration surprises better than shared assumptions.”

5) The Four Testing Layers for Node.js APIs

A) Unit Tests (Core Behavior)
Purpose: verify pure or near-pure logic quickly.
Test here:

Validation utilities
Domain services
Policy evaluators (authz/eligibility)
Data transformation helpers

Characteristics:

No network I/O
Minimal mocking
Millisecond-level runtime

B) Service/Use-Case Tests (Application Behavior)
Purpose: validate business workflows with mocked boundaries.
Test here:

Use-case orchestration
Branching by role/plan/state
Error mapping from domain to application outcomes

Characteristics:

Mock repository/gateway interfaces
Clear input-output assertions
Strong signal for business regressions

C) API Integration Tests (HTTP Boundary)
Purpose: verify endpoint contracts through real routing/middleware.
Test here:

Status and payload schema
Auth middleware effects
Validation and error format
Idempotency and headers

Characteristics:

Run through supertest(app)
Use real request pipeline
Keep counts limited to meaningful scenarios

D) Infrastructure Integration Tests (Real Dependencies)
Purpose: validate adapter correctness with real systems.
Test here:

Repository against real DB (often containerized)
Outbound gateway behavior against sandbox/stub
Queue publish/consume adapter correctness

Characteristics:

Slower and fewer
High confidence at critical seams
Isolated environments required

6) Implementation Strategy for Engineering Teams

Strategy 1: Start with High-Risk Endpoints
Prioritize money movement, authentication, billing, entitlement, and state-transition APIs.

Strategy 2: Define a Test Matrix per Endpoint

For each endpoint, define:

happy path,
validation failures,
authorization failures,
business rule failures,
idempotency/concurrency behavior.

Strategy 3: Standardize Test Utilities

Create shared helpers for:

app bootstrap,
authenticated request context,
test data factories,
DB reset/seed.
This reduces duplicated setup and inconsistent patterns.

Strategy 4: Separate Fast vs Slow Pipelines

Run fast tests on every push; run heavier integration suites on merge or scheduled jobs.

Strategy 5: Enforce Test Ownership in Reviews

Review checklist:

Is the behavior covered at the right layer?
Are assertions business-meaningful?
Is the test deterministic?
Does this duplicate lower-layer coverage?

Strategy 6: Track Quality Metrics
Monitor:

flaky test rate,
mean CI duration,
re-run frequency,
escaped defect categories. Good testing architecture should improve these over time.

7) Minimal Example: Create User Endpoint with Jest + Supertest

The goal is clarity of boundaries, not framework-specific depth.

src/app.ts

import express from 'express';

const app = express();
app.use(express.json());

app.post('/users', (req, res) => {
  const { email, name } = req.body ?? {};

  if (!email || !name) {
    return res.status(422).json({
      error: {
        code: 'VALIDATION_ERROR',
        message: 'email and name are required'
      }
    });
  }

  return res.status(201).json({
    data: {
      id: 'usr_123',
      email,
      name
    }
  });
});

export { app };

test/users.create.spec.ts

import request from 'supertest';
import { app } from '../src/app';

describe('POST /users', () => {
  it('creates a user when payload is valid', async () => {
    const response = await request(app)
      .post('/users')
      .send({ email: 'sam@example.com', name: 'Sam' });

    expect(response.status).toBe(201);
    expect(response.body).toEqual({
      data: {
        id: expect.any(String),
        email: 'sam@example.com',
        name: 'Sam'
      }
    });
  });

  it('returns 422 for invalid payload', async () => {
    const response = await request(app)
      .post('/users')
      .send({ email: 'sam@example.com' });

    expect(response.status).toBe(422);
    expect(response.body.error.code).toBe('VALIDATION_ERROR');
  });
});

package.json (relevant scripts)

{
  "scripts": {
    "test": "jest --runInBand",
    "test:watch": "jest --watch",
    "test:ci": "jest --coverage --maxWorkers=50%"
  }
}

In real systems, route handlers should delegate to use cases/services. Tests then become cleaner: unit tests cover rules, while Supertest verifies HTTP contract and middleware behavior.

8) Databases, Queues, and External Integrations

These boundaries create most false confidence if not tested deliberately.

Database: prefer isolated test DB instances, deterministic seeders, and teardown per test or per suite.
Queues/events: test enqueue intent at service level, then test adapter correctness separately.
External APIs: avoid live dependencies in regular CI; use stable stubs, contract checks, or sandbox environments.
Retries/timeouts: assert observable outcomes (e.g., mapped errors, retry caps), not private implementation internals.

A practical split:

regular CI: no real internet dependency,
nightly/extended pipeline: sandbox integration checks.

9) Testing Strategy by Risk and Feedback Speed

Treat tests as a portfolio.

Fast lane (minutes): unit + selected service tests.
Core lane (merge gate): API integration for critical endpoints.
Extended lane (scheduled): infra/sandbox checks, broader resilience scenarios. This balances confidence and developer velocity.

Recommended safeguards:

Fail build on flaky test detection thresholds.
Quarantine unstable tests with owner + fix-by date.
Keep coverage goals directional, not vanity metrics.
Prefer mutation testing selectively on high-value modules.

10) Migration Roadmap for Existing Node.js APIs

Avoid all-at-once rewrites.

Identify top 5 risky endpoints by incident history or business criticality.
Add baseline Supertest contract tests for current behavior.
Extract business logic from route handlers into testable services/use cases.
Add unit/service tests around extracted rules.
Introduce integration tests for critical DB/external adapters.
Reduce over-mocked endpoint tests and remove duplicate coverage.
Enforce testing checklist in pull requests.
Repeat by vertical slice.

This path allows continuous delivery while improving reliability incrementally.

11) Common Mistakes and How to Avoid Them

Over-mocking everything: tests become fiction; keep meaningful boundaries real.
Only endpoint tests: slow suites and weak failure localization.
Shared mutable fixtures: hidden coupling and random failures.
Ignoring time/state transitions: retries, TTLs, and async workflows go untested.
Snapshot overuse: brittle assertions with low business signal.
No ownership model: flaky tests linger and trust decays. Fixes are process + architecture: test at the right layer, isolate state, and treat flaky tests as production bugs.

12) When to Go Deep (and When to Keep It Lean)

Go deeper when:

APIs control payments, access, compliance, or irreversible workflows.
multiple teams ship to the same service.
incident cost is high and regressions are expensive.

Keep it leaner when:

service is internal and low-risk,
behavior is straightforward CRUD,
change frequency and impact are low.
Testing depth should match business risk, not trend pressure.

“A stable test suite is not a luxury in API teams; it is delivery infrastructure.”

13) FAQs

Is Jest still a good default for Node.js API testing?
Yes. It remains a practical default for most teams due to ecosystem maturity, tooling support, and straightforward CI integration.

Do we need Supertest if we already unit test services?
Yes, for HTTP contract confidence. Unit tests do not validate routing, middleware, serialization, and error envelope behavior.

How much mocking is too much?
If tests pass while real integration repeatedly fails, mocking is likely hiding boundary problems. Mock at external seams, not core behavior.

Should we test against real third-party APIs in CI?
Usually not in the main pipeline. Prefer deterministic stubs/contracts in regular CI and schedule sandbox verification separately.

What is a healthy target for coverage?
There is no universal number. Prioritize meaningful coverage of critical flows and failure modes over headline percentages.

14) References

Jest Documentation: https://jestjs.io
Supertest (GitHub): https://github.com/ladjs/supertest
Node.js Documentation (Testing & Runtime): https://nodejs.org
Martin Fowler Test Pyramid: https://martinfowler.com/articles/practical-test-pyramid.html
Google SRE Workbook (Testing in Production Concepts): https://sre.google/workbook/

15) Interesting Facts

JavaScript continues to be one of the most widely used languages globally, and Node.js remains one of the most commonly used web technologies among professional developers. Source: https://survey.stackoverflow.co/2024/
Testing has become a first-class topic in modern JavaScript ecosystems, with dedicated community tracking for framework usage, satisfaction, and retention trends. Source: https://stateofjs.com/
Node.js now includes an official built-in test runner, which reflects how central testing has become in backend JavaScript workflows. Source: https://nodejs.org/api/test.html
Jest provides native support for parallel execution and coverage reporting, which is why many teams keep it as a CI-friendly default for API and service testing. Source: https://jestjs.io/docs/cli
Supertest is designed to test HTTP servers without opening a real network port, making endpoint tests faster and more deterministic in local and CI environments. Source: https://github.com/ladjs/supertest
Jest is an all-in-one testing solution that includes a test runner, assertion library, and mocking capabilities out of the box.
Supertest can simulate HTTP requests directly against an Express app, making tests faster and more reliable.
Jest runs tests in parallel by default, significantly improving execution speed for large test suites.
API testing in Node.js often focuses more on integration tests rather than pure unit tests due to the nature of backend systems.
Best practice emphasizes testing API behavior (status codes, responses) rather than internal implementation details.

16) Stats

According to the Stack Overflow Developer Survey, a majority of Node.js developers use automated testing in production environments.Source: https://survey.stackoverflow.co/
Jest has 40k+ stars on GitHub and millions of weekly npm downloads, making it one of the most widely used JavaScript testing frameworks.Source: https://github.com/facebook/jest
Tools like Supertest are commonly used with Express.js for API testing due to their ability to test endpoints without starting a live server.Source: https://github.com/visionmedia/supertest

17) Conclusion

Testing Node.js APIs effectively is not about maximizing test count; it is about placing the right tests at the right boundaries. With Jest and Supertest, teams can build a fast, trustworthy feedback loop by combining deterministic unit/service tests, focused API contract tests, and intentional integration checks. Over time, this reduces incident risk, shortens debugging cycles, and makes API evolution safer.

About the Author: Zemichael is a fullstackDeveloper at AddWeb Solution.Crafting web experiences with robust architecture, performance focus, and design thinking.

Common Mistakes Laravel Developers Make (And How to Avoid Them)

Lakashya Upadhyay — Fri, 27 Mar 2026 08:23:53 +0000

“Controllers should coordinate, not decide.”

Every Laravel developer, from juniors to seasoned engineers, has written code that felt “good enough” but created performance bottlenecks, deep technical debt, or hard‑to‑debug bugs months later. Laravel’s expressive API makes it easy to ship fast, but it also makes it easy to fall into common anti‑patterns.

This guide walks through the most frequent mistakes Laravel developers make in production, explains why they’re harmful, and shows how to refactor them into clean, maintainable structures.

Key Takeaways

Avoid putting business logic in controllers; move it into services, actions, or domain classes.
Use Eloquent relationships, eager loading, and proper indexing to prevent N+1 queries.
Organize your code with proper folders, route groups, and naming conventions.
Separate validation from controllers and reuse it via Form Requests or validation rules.
Use migrations, tests, and documentation to keep your app scalable and on‑team‑friendly.
Treat database design, authentication, and security as first‑class concerns, not afterthoughts.

Index

Why This Matters
Mistake 1: Putting Too Much Logic in Controllers
Mistake 2: Ignoring Eloquent Relationships and N+1 Queries
Mistake 3: Poor Database Design and Indexing
Mistake 4: Not Using Form Requests or Validation Properly
Mistake 5: Ignoring Migrations and Model Structure
Mistake 6: Messy Routes and Poor Naming Conventions
Mistake 7: Overusing Helper/Static Methods and Global Helpers
Mistake 8: Skipping Tests and Documentation
Mistake 9: Hard‑coding Logic in Migrations or Seeders
Mistake 10: Poor Error Handling and Logging
Frequently Asked Questions (FAQs)
Interesting Facts & Stats
Conclusion

1. Why This Matters

Laravel is designed to help you ship fast while keeping your codebase sane. But if you ignore structure, performance, and maintainability, your app can quickly become a “Laravel monolith” with tangled controllers, slow queries, and brittle validation.

Common mistakes Laravel developers make often look harmless at first:

A controller method that grows from 10 to 100 lines.
A route file that crosses 500 lines.
A model that eagerly loads every relation on every query. These small decisions compound over time, leading to slower performance, harder debugging, and more expensive refactoring later.

This guide helps you avoid the most common pitfalls and build Laravel apps that are:

Readable to other developers
Testable with clear separation of concerns
Scalable as the user base and feature set grow

2. Mistake 1: Putting Too Much Logic in Controllers

Controllers are meant to handle HTTP concerns:

Accepting requests
Running validation
Returning responses
When you put business logic directly in controllers, you end up with:
Fat controllers anyone is afraid to touch
Duplicated logic across multiple controller methods
Hard‑to‑test code that depends on the HTTP request
Example of the mistake:

class SubscriptionController extends Controller
{
    public function upgrade(Request $request)
    {
        $user = Auth::user();

        if ($user->isPremium()) {
            return back()->withErrors('You are already premium.');
        }

        $price = $request->input('plan') === 'annual'
            ? 1000 : 200;

        $gateway = new StripeGateway();
        $gateway->charge($user, $price);

        $user->role = 'premium';
        $user->premium_until = now()->addYear();
        $user->save();

        event(new UserUpgraded($user));

        return redirect('/dashboard');
    }
}

How to fix it:

Move charge and subscription logic into a SubscriptionService or SubscriptionAction.
Keep the controller action thin and focused on the request/response pipeline.

class SubscriptionController extends Controller
{
    public function upgrade(Request $request, UpgradeSubscriptionAction $action)
    {
        $result = $action->execute(
            user: Auth::user(),
            plan: $request->input('plan'),
        );

        if ($result->failed()) {
            return back()->withErrors($result->message);
        }

        return redirect('/dashboard');
    }
}

Benefits:

Business logic is reusable across APIs, commands, and jobs.
Easier to unit test the subscription logic without touching HTTP.
Cleaner diffs and easier on‑boarding for new team members.

3. Mistake 2: Ignoring Eloquent Relationships and N+1 Queries

“Eloquent is powerful, but misuse kills performance.”

Many Laravel developers fetch data manually instead of using Eloquent relationships. This leads to:

N+1 queries (one query per related record)
Slow page loads
Database bottlenecks

Example of the mistake:

$posts = Post::all();

foreach ($posts as $post) {
    echo $post->user->name; // Each access triggers a separate query
}

How to fix it:

Define relationships in your models (User belongsToMany Posts, etc.).
Use with() to eager load related data.

$posts = Post::with('user')->get();

foreach ($posts as $post) {
    echo $post->user->name; // No extra query
}

Additional tips:

Use select() to limit columns when you don’t need the full model.
Avoid all() on large tables; paginate instead with paginate().
Use tools like laravel-debugbar or clockwork to detect N+1 queries early.

4. Mistake 3: Poor Database Design and Indexing

“Bad database design will always be your bottleneck.”

Many Laravel apps start with poorly normalized tables, missing foreign keys, or no indexes on frequently queried columns. This leads to:

Slow queries
Complex joins and filters in PHP instead of SQL
Inconsistent or denormalized data

Common anti‑patterns:

Storing JSON in a single column instead of proper relationships
Using integers for IDs but not defining foreign‑key constraints
Not indexing columns used in where, order, or join clauses

How to fix it:

Use Laravel migrations to define tables, indexes, and foreign keys.
Normalize your data where it makes sense (e.g., users → posts → comments).

Index frequently filtered columns:

$table->index('status');
$table->foreignId('user_id')->constrained();

Benefits:

Faster queries as datasets grow
Easier to maintain data consistency
Cleaner, more predictable Eloquent relationships

5. Mistake 4: Not Using Form Requests or Validation Properly

“Validation should be reusable, not copy‑pasted.”

Laravel’s validation is powerful, but it’s often embedded directly in controllers as inline arrays or repeated in multiple methods.

Example of the mistake:

class UserController extends Controller
{
    public function update(Request $request)
    {
        $request->validate([
            'name' => 'required',
            'email' => 'required|email',
            'age' => 'nullable|integer|min:18',
        ]);

        // ...
    }

    public function store(Request $request)
    {
        $request->validate([
            'name' => 'required',
            'email' => 'required|email',
            'age' => 'nullable|integer|min:18',
        ]);

        // ...
    }
}

How to fix it:

Create Form Request classes:

php artisan make:request UserStoreRequest
php artisan make:request UserUpdateRequest

class UserStoreRequest extends FormRequest
{
    public function rules()
    {
        return [
            'name' => 'required',
            'email' => 'required|unique:users,email',
            'age' => 'nullable|integer|min:18',
        ];
    }
}

Then inject the form request into your controller:

public function store(UserStoreRequest $request)
{
    $validated = $request->validated();

    // ...
}

Benefits:

Validation rules are reusable and centralized
You can add custom validation logic or authorization inside the form request
Controllers stay lean and focused

6. Mistake 5: Ignoring Migrations and Model Structure

“They removed the model directory after the first migration.”

Migrations are Laravel’s way to version‑control your database. Ignoring them or using them incorrectly leads to:

Divergent schemas across environments
Manual SQL changes in production
Broken deployments
Common mistakes:
Not using down() methods in migrations
Writing business logic inside migration files (e.g., looping through users and updating them)
Skipping foreign keys or using unsigned() on integer foreign keys
How to fix it:
Always define proper foreign keys:

$table->foreignId('user_id')->constrained();

Use rollback‑safe migrations with clear up() and down() methods.
Avoid complex business logic in migrations; move it to seeders or jobs.

Best practice:

Treat migrations as “immutable” after they’re pushed to production.
Use Schema::table() for alterations, not Schema::dropIfExists() for simple changes.

7. Mistake 6: Messy Routes and Poor Naming Conventions

“Routes are the contract of your app.”

Laravel’s routing system is powerful, but it’s easy to end up with a monolithic routes/web.php file and inconsistently named routes.

Common issues:

Mixing API and web routes in one file
Not using route groups (prefix, middleware, namespace)
Using inconsistent route names like userShow, showUser, show_user

How to fix it:

Split routes into:
- routes/web.php (HTML pages)
- routes/api.php (REST or JSON endpoints)
Use route groups to apply middleware and prefixes:

Route::middleware('auth')->prefix('admin')->group(function () {
    Route::get('users', [UserController::class, 'index'])->name('admin.users.index');
    Route::post('users', [UserController::class, 'store'])->name('admin.users.store');
});

Follow consistent naming conventions (e.g.,

resource_route_name.action, snake_case):

Route::name('users.')->group(function () {
    Route::get('/users', [UserController::class, 'index'])->name('index');
    Route::get('/users/{user}', [UserController::class, 'show'])->name('show');
});

Benefits:

Routes are easier to scan and maintain
Route names are predictable for other developers
Auth and middleware configuration is centralized

8. Mistake 7: Overusing Helper/Static Methods and Global Helpers

“Helper methods are helpers, not your business logic.”

Many Laravel apps accumulate global helper functions in app/Helpers or config/helpers.php. This leads to:

Hidden dependencies and hard‑to‑trace behavior
Testing nightmare (no dependency injection)
Spaghetti‑style code scattered across functions

How to fix it:

Use services, actions, or managers instead of global helpers.
Use dependency injection where possible. For example, instead of:

// app/Helpers/general.php
function calculateDiscount($price, $user)
{
    return $price * 0.9;
}

Use a service:

class DiscountService
{
    public function calculate($price, User $user)
    {
        return $price * 0.9;
    }
}

Inject it into your controller or action:

public function checkout(Request $request, DiscountService $discount)
{
    $price = $request->input('price');
    $discounted = $discount->calculate($price, $user);

    // ...
}

Benefits:

Clear dependencies and easier testing
Reusable across services, jobs, and controllers
Easier to refactor or replace logic

9. Mistake 8: Skipping Tests and Documentation

“Tests are your safety net.”

Skipping tests leads to:

Fear of refactoring
More bugs in production
Longer on‑boarding for new developers

How to fix it:

Write feature tests for critical user flows.
Write unit tests for business logic and services. Use Laravel’s built‑in testing helpers:

public function test_user_can_upgrade_subscription()
{
    $user = User::factory()->create();

    $response = $this
        ->actingAs($user)
        ->post(route('subscriptions.upgrade'), [
            'plan' => 'annual',
        ]);

    $response->assertRedirect('/dashboard');
    $this->assertTrue($user->fresh()->isPremium());
}

Best practices:

Use expectsDatabaseTransactions() for database‑intensive tests.
Keep your tests fast and focused.
Add high‑level documentation for architecture, routes, and key classes.

10. Mistake 9: Hard‑coding Logic in Migrations or Seeders

“Migrations are not meant to run business logic.”

People sometimes write loops, call services, or send notifications inside migrations or seeders. This is dangerous because:

Migrations are meant to be repeatable and safe to roll back
Running business logic here can break future deployments

How to fix it:

Use migrations only for:
- Schema changes
- Simple data seeding (if needed)
Move business logic into:
- Artisan commands
- Jobs
- Seeders with proper rollback support

Example:

php artisan make:command MigrateOldUserSubscriptions

Then run it explicitly instead of hiding it in a migration.

Benefits:

Migrations stay predictable and safe
Business logic can be retried or rolled back cleanly

11. Mistake 10: Poor Error Handling and Logging

“Exceptions are information, not noise.”

Ignoring error handling and logging in Laravel apps leads to:

Silent failures
Difficult debugging in production
Unpredictable behavior after exceptions

How to fix it:

Use try/catch blocks where appropriate.
Use Log::error(), Log::info(), or report() to log errors.
Configure app/Exceptions/Handler.php to:
- Report critical errors to services like Sentry, Bugsnag, or LogRocket
- Transform exceptions into user‑friendly messages

Example:

public function report(Throwable $exception)
{
    if ($exception instanceof CustomException) {
        // Log or notify specific teams
        Log::channel('slack')->error($exception->getMessage());
    }

    parent::report($exception);
}

Benefits:

Easier debugging and faster incident response
Better visibility into production issues

12. Frequently Asked Questions (FAQs)

Q. Should I always move logic from controllers to services?
A. Yes, if it’s reusable business logic. Keep controllers focused on HTTP concerns and use services, actions, or domain classes for the rest.

Q. Is it okay to keep small apps without tests?
A. For small prototypes, maybe. But even tiny apps benefit from a basic test suite once they go to production.

Q. Can I mix web and API routes in one file?
A. Technically yes, but it’s cleaner to separate them into web.php and api.php with proper prefixing and middleware.

Q. Are migrations really necessary for small changes?
A. Yes, especially after going live. Migrations help keep your team and environment in sync.

Q. How do I know if my app has N+1 queries?
A. Use tools like laravel-debugbar or clockwork to analyse queries or enable Laravel’s built‑in query logging in development.

13. Interesting Facts & Stats

Well-structured Laravel apps with services, form requests, and proper Eloquent usage can reduce controller code by 40-60% in larger projects. Source: https://laravel.com/docs/eloquent-resources
Teams that consistently use migrations and tests spend 25-35% less time debugging in production compared to teams that don’t. Source: https://martinfowler.com/articles/practical-test-pyramid.html
Avoiding N+1 queries often reduces page load time by 2-10x for list-style pages with relationships.Source: https://laravel.com/docs/eloquent-relationships#eager-loading
Laravel’s Form Request system and built-in validation rules have become industry standards for clean, maintainable validation logic in web frameworks.Source: https://laravel.com/docs/validation

14. Conclusion

Laravel is a powerful framework that encourages fast development, but it also amplifies the impact of bad habits. Common mistakes like fat controllers, poor database design, and ignoring migrations can slow your app and your team over time.

By consistently:

Moving business logic out of controllers
Using Eloquent properly and avoiding N+1 queries
Organizing routes, migrations, and helpers
Writing tests and proper logging

You turn Laravel’s “easy to start” advantage into a long‑term maintenance win.

Clean, maintainable Laravel code is not about syntax alone; it’s about structure, separation of concerns, and respecting the framework’s conventions. Stick to these patterns, and you’ll avoid most of the common pitfalls Laravel developers face in production.

About the Author: Lakashya is a full‑stack Laravel developer at AddWeb Solution specializing in scalable, real‑time applications with PHP and modern frontends.

References

Laravel Documentation. (2025). Best Practices. https://laravel.com/docs/12.x
Laravel.io. (2026). Common Laravel Mistakes I See in Production. https://laravel.io/articles/common-laravel-mistakes-i-see-in-production-and-how-to-avoid-them
Laravel Daily. (2025). Laravel: 9 Typical Mistakes Juniors Make. https://laraveldaily.com/post/laravel-typical-mistakes-juniors-make

Practical Techniques for Optimizing React Performance in Production: A Developer’s Guide

Mayank Goyal — Thu, 26 Mar 2026 10:34:57 +0000

"Premature optimization is the root of all evil." - Donald Knuth

Introduction

As modern web applications grow in complexity, maintaining high performance becomes increasingly important. React is known for its efficient rendering and component-based architecture, but poorly optimized applications can still suffer from issues such as slow rendering, unnecessary re-renders, large bundle sizes, and delayed user interactions.

Optimizing React applications in production environments requires a combination of architectural decisions, efficient state management, and performance monitoring.

This guide explores practical techniques developers can use to optimize React performance in production, helping applications remain fast, scalable, and responsive even as they grow.

Key Takeaways

Avoid unnecessary component re-renders using React.memo, useMemo, and useCallback.
Use code splitting and lazy loading to reduce initial bundle size.
Implement list virtualization when rendering large datasets.
Optimize state management to prevent unnecessary updates.
Use debouncing and throttling for expensive operations like search or scroll events.
Regularly monitor performance using React DevTools and browser profiling tools.

Index

Understanding React Performance Optimization
Detailed Comparison of Optimization Techniques
Implementation Overview
When to Use Each Optimization Strategy
Developer Recommendations
Code Examples
FAQ
Interesting Facts
Conclusion

1. Understanding React Performance Optimization

1.1 Preventing Unnecessary Re-Renders
React re-renders components whenever their state or props change. While React's Virtual DOM minimizes DOM updates, unnecessary re-renders can still affect performance.

Key Techniques

React.memo
Prevents functional components from re-rendering when props have not changed.

useMemo
Caches the result of expensive computations.

useCallback
Prevents functions from being recreated on every render.

Best For

Components that render frequently
Expensive calculations inside components
Preventing unnecessary child component updates

1.2 Code Splitting and Lazy Loading
Large JavaScript bundles increase page load time and negatively impact performance.
Code splitting allows loading only the code required for the current view.
React provides built-in support through:

React.lazy
Suspense

Benefits

Faster initial page load
Reduced bundle size
Improved performance on slower networks

Best For

Large applications
Route-based components
Feature-heavy dashboards

1.3 Virtualizing Large Lists
Rendering thousands of DOM elements simultaneously can slow down the browser.
List virtualization renders only the items visible in the viewport.
Popular libraries include:

react-window
react-virtualized

Best For

Large tables
Infinite scrolling lists
Analytics dashboards

2. Detailed Comparison

2.1 Rendering Optimization

2.2 Bundle Optimization

2.3 Event Optimization

3. Implementation Overview

3.1 Memoizing Components
Flow:

Component receives props
React compares previous and new props
If props are unchanged, rendering is skipped

Pros

Reduces unnecessary rendering
Improves UI responsiveness
Useful in component-heavy applications

3.2 Code Splitting in React
Flow:

User visits application
Initial bundle loads
Additional components load on demand
Suspense displays fallback UI

Pros

Faster application startup
Reduced network usage
Better scalability for large apps

3.3 Virtualizing Lists
Flow:

Large dataset fetched
Only visible rows rendered
Rows dynamically mounted/unmounted during scroll

Pros

Reduces DOM nodes
Smooth scrolling
Improves memory usage

4. When to Use Each Optimization Technique

Use Memoization If

Components render frequently
Parent components trigger unnecessary updates
Expensive calculations occur during rendering

Use Code Splitting If

Your bundle size is large
Your app has multiple pages
You want faster initial load time

Use Virtualization If

Rendering thousands of items
Working with large datasets
Building analytics dashboards or tables

5. Developer Recommendation

For most production React applications, the following performance practices provide the biggest impact:

Memoize components when necessary
Implement route-based code splitting
Use virtualization for large lists
Debounce expensive API requests
Continuously monitor performance using profiling tools

Optimizing too early can introduce unnecessary complexity, so focus first on identifying performance bottlenecks before applying optimizations.

6. Code Examples

6.1 Preventing Re-renders with React.memo

import React from "react";

const UserCard = React.memo(({ name }) => {
  console.log("Rendering UserCard");

  return <div>{name}</div>;
});

export default UserCard;

6.2 Using useMemo for Expensive Calculations

import { useMemo } from "react";

function ExpensiveComponent({ items }) {

  const sortedItems = useMemo(() => {
    return items.sort((a, b) => a.price - b.price);
  }, [items]);

  return (
    <ul>
      {sortedItems.map(item => (
        <li key={item.id}>{item.name}</li>
      ))}
    </ul>
  );
}

6.3 Lazy Loading Components

import React, { Suspense } from "react";

const Dashboard = React.lazy(() => import("./Dashboard"));

function App() {
  return (
    <Suspense fallback={<div>Loading...</div>}>
      <Dashboard />
    </Suspense>
  );
}

6.4 Debouncing API Calls

import debounce from "lodash/debounce";

const searchUsers = debounce(async (query) => {
  const response = await fetch(`/api/users?q=${query}`);
  return response.json();
}, 300);

7. FAQ

1. What causes poor React performance?
Common causes include:

Unnecessary re-renders
Large bundle sizes
Rendering large lists
Heavy computations during rendering

2. Should every component use React.memo?
No. Memoization adds comparison overhead. Use it only when performance issues exist.

3. What tool helps identify slow components?
React DevTools Profiler helps identify slow rendering components.

4. Is lazy loading good for SEO?
It can be, but server-side rendering (SSR) may be required for SEO-critical pages.

5. What is the biggest performance win in React apps?
Reducing bundle size and unnecessary re-renders typically provides the largest improvements.

8. Interesting Facts

React’s Virtual DOM reduces direct DOM manipulation by updating only the elements that changed.Source:https://react.dev/learn/render-and-commit
The React Profiler tool allows developers to measure rendering performance and detect unnecessary updates.Source:https://react.dev/reference/react/Profiler
Libraries like react-window can reduce thousands of DOM nodes to just a few dozen visible elements.Source:https://github.com/bvaughn/react-window

"Programs must be written for people to read, and only incidentally for machines to execute." – Harold Abelson

9. Conclusion

Optimizing React performance in production requires thoughtful architecture and strategic use of optimization techniques. While React already provides efficient rendering through the Virtual DOM, developers must still manage component updates, bundle sizes, and expensive computations carefully.

By applying techniques such as memoization, code splitting, virtualization, and event optimization, developers can significantly improve application responsiveness and scalability.
The key is to identify real performance bottlenecks first and apply targeted optimizations, ensuring the application remains both maintainable and performant.

About the Author: Mayank is a web developer at AddWebSolution, building scalable apps with PHP, Node.js & React. Sharing ideas, code, and creativity.

Secure File Uploads in Laravel: Validation, Storage & Basic Virus Protection

Abodh Kumar — Fri, 20 Mar 2026 07:54:18 +0000

Security is not a product, but a process.- Bruce Schneier, Security Technologist

File uploads are a common requirement in modern web applications - whether it's profile pictures, documents, invoices, or media files. However, insecure file uploads can expose your application to serious threats such as malware injection, remote code execution, and data breaches.

Key Takeaway

Validate both file extension (mimes) and content-based MIME type (mimetypes) - never trust one alone.
Always generate a random UUID-based filename - never persist client-supplied filenames to disk.
Store files outside the public directory on a private disk; serve through authenticated controllers.
Use temporary signed URLs for file downloads to prevent unauthorized access and link sharing.
Integrate ClamAV or another AV engine; fail closed if the scanner is unavailable.
Strip EXIF metadata from images to protect user privacy and prevent location data leaks.

Introduction
Understanding File Upload Risks
Laravel File Validation - A Deep Dive
Secure Storage Strategies
Basic Virus Protection with ClamAV
Advanced Security Techniques
Stats & Interesting Facts
FAQ
Conclusion

1. Introduction

document managers to medical portals and e-commerce platforms. Yet despite their ubiquity, secure file handling remains one of the most misunderstood areas of web development.

Laravel, PHP's premier web framework, equips developers with an elegant and expressive set of tools for handling file uploads. However, leveraging those tools securely requires a deliberate, layered approach: validating the incoming file, storing it safely, and scanning it for malicious content before it ever reaches your users or your system.

This article walks you through a complete, production-ready strategy for secure file uploads in Laravel - covering everything from MIME type validation and file size limits to randomized storage paths and ClamAV antivirus integration.

2. Understanding File Upload Risks

Before writing a single line of code, it is essential to understand why file uploads are dangerous. Attackers routinely exploit careless upload implementations to compromise servers and steal data. The most common attack vectors include:

2.1 Malicious File Execution
An attacker uploads a PHP file disguised as an image (e.g., evil.php.jpg). If the server is misconfigured or the MIME type is not properly validated, the file may be executed as a script, granting the attacker remote code execution (RCE).

2.2 Denial of Service via Large Uploads
Without file size restrictions, an attacker can upload multi-gigabyte files to exhaust disk space or memory, effectively taking down the server.

2.3 Path Traversal Attacks
If user-supplied filenames are stored directly, an attacker might submit a filename like ../../etc/passwd to write or overwrite sensitive files on the server.

2.4 Virus & Malware Distribution
Even with correct validation, a seemingly valid PDF or DOCX file might contain embedded malware. If your application serves these files to other users, you inadvertently become a malware distribution vector.

2.5 Metadata & Privacy Leaks
Uploaded images may contain EXIF metadata including GPS coordinates, device information, or personally identifiable information - a significant privacy risk if files are served publicly without stripping metadata.

3. Laravel File Validation - A Deep Dive

Laravel's validation system is remarkably expressive. The file() and image() validation rules, combined with additional constraints, let you define exactly what constitutes an acceptable upload.

3.1 Basic Validation Setup
Start with a dedicated Form Request class to keep your controller lean:

    php artisan make:request SecureFileUploadRequest

In your SecureFileUploadRequest class:

<?php

namespace App\Http\Requests;
use Illuminate\Foundation\Http\FormRequest;
class SecureFileUploadRequest extends FormRequest
{
   public function authorize(): bool
   {
       return auth()->check();
   }

   public function rules(): array
   {
       return [
           'document' => [
               'required',
               'file',
               'mimes:pdf,docx,xlsx,png,jpg,jpeg',
               'mimetypes:application/pdf,image/png,image/jpeg,
                          application/vnd.openxmlformats-officedocument
                          .wordprocessingml.document',
               'max:10240',  
               'min:1', 
           ],
       ];
   }
   public function messages(): array
   {
       return [
           'document.mimes'    => 'Only PDF, DOCX, XLSX, PNG, and JPG                   files are allowed.',
           'document.max'      => 'File size must not exceed 10 MB.',
           'document.mimetypes'=> 'The file type does not match the expected MIME type.',
       ];
   }
}

3.2 Understanding mimes vs. mimetypes
This is a common source of confusion. The mimes rule validates using the file extension, while mimetypes validates the actual MIME type detected by PHP's Fileinfo extension. Using both together is a stronger approach:

mimes:pdf,png - Laravel maps the extension to an expected MIME type and checks it.
mimetypes:application/pdf - Directly checks the MIME type detected from file content.
Using both ensures neither the extension nor the MIME type is spoofed independently.

3.3 Validating Dimensions for Images
For image uploads specifically, Laravel allows you to enforce dimension constraints:

'avatar' => [
   'required',
   'image',
   'mimes:jpeg,png,webp',
   'max:2048',
   'dimensions:min_width=100,min_height=100,max_width=3000,max_height=3000',
],

4. Secure Storage Strategies

Validating a file is only half the battle. Where and how you store it is equally critical to your application's security posture.

4.1 Never Store in the Public Directory
A common beginner mistake is storing uploads directly in public/uploads. This makes files web-accessible by default, which is dangerous. Use the storage/app/private directory instead:

use Illuminate\Support\Facades\Storage;
use Illuminate\Support\Str;

public function store(SecureFileUploadRequest $request)
{
   $file = $request->file('document');

   $filename = Str::uuid() . '.' . $file->getClientOriginalExtension();

   $path = $file->storeAs(
       'uploads/' . auth()->id(),
       $filename,
       'private' 
   );

   Document::create([
       'user_id'        => auth()->id(),
       'original_name'  => $file->getClientOriginalName(),
       'stored_path'    => $path,
       'mime_type'      => $file->getMimeType(),
       'file_size'      => $file->getSize(),
   ]);
   return response()->json(['message' => 'File uploaded successfully.']);
}

4.2 Configuring a Private Disk
In config/filesystems.php, define a private disk that lives outside the public web root:

'disks' => [
   'private' => [
       'driver'     => 'local',
       'root'       => storage_path('app/private'),
       'visibility' => 'private',
   ],
   's3_private' => [
       'driver'     => 's3',
       'key'        => env('AWS_ACCESS_KEY_ID'),
       'secret'     => env('AWS_SECRET_ACCESS_KEY'),
       'region'     => env('AWS_DEFAULT_REGION'),
       'bucket'     => env('AWS_BUCKET'),
       'visibility' => 'private',
   ],
],

Never trust user input when handling uploaded files. Always validate file types, sizes, and storage locations on the server side.- Stephen Rees-Carter

4.3 Serving Files Securely via Signed URLs
Since files are not publicly accessible, you need a controller endpoint to serve them. Use signed URLs to prevent unauthorized access:

use Illuminate\Support\Facades\URL;
$signedUrl = URL::temporarySignedRoute(
   'file.download',
   now()->addMinutes(30),
   ['document' => $document->id]
);
Route::get('/files/{document}', function (Document $document) {
   abort_unless(
       request()->hasValidSignature() &&
       $document->user_id === auth()->id(),
       403
   );
   return Storage::disk('private')->download($document->stored_path);
})->name('file.download')->middleware(['auth', 'signed']);

5. Basic Virus Protection with ClamAV

Even perfectly validated files can harbor malicious payloads. A PDF can contain JavaScript exploits; a DOCX can embed macros. Integrating an antivirus scanner is the professional-grade answer.

5.1 Installing ClamAV
On Ubuntu/Debian servers:

sudo apt-get update
sudo apt-get install -y clamav clamav-daemon
sudo freshclam   # Update virus definitions
sudo systemctl enable clamav-daemon
sudo systemctl start clamav-daemon

5.2 Using the Laravel ClamAV Package
The most popular Laravel integration is via the laravel-clamav package:

composer require clamav/clamav

Or use a custom wrapper with the PHP socket connection:

composer require sunspikes/clamav-validator

5.3 Building a ClamAV Service
Here is a clean, reusable ClamAV service class:

<?php

namespace App\Services;

use Illuminate\Http\UploadedFile;
use Illuminate\Support\Facades\Log;

class VirusScanService
{
   protected string $clamdHost;
   protected int $clamdPort;

   public function __construct()
   {
       $this->clamdHost = config('services.clamav.host', '127.0.0.1');
       $this->clamdPort = config('services.clamav.port', 3310);
   }

   public function scan(UploadedFile $file): bool
   {
       $socket = @fsockopen(
           $this->clamdHost,
           $this->clamdPort,
           $errno,
           $errstr,
           5
       );

       if (!$socket) {
           Log::error('ClamAV unavailable', ['error' => $errstr]);
           // Fail closed: reject upload if scanner unavailable
           return false;
       }

       $fileContent = file_get_contents($file->getRealPath());
       $length = strlen($fileContent);

       fwrite($socket, "nINSTREAM\n");
       fwrite($socket, pack('N', $length) . $fileContent);
       fwrite($socket, pack('N', 0));

       $response = trim(fgets($socket));
       fclose($socket);
       // 'stream: OK' means clean; anything else is infected
       return str_contains($response, 'OK');
   }
}

5.4 Integrating the Scanner in Your Controller

<?php
use App\Services\VirusScanService;

public function store(SecureFileUploadRequest $request, VirusScanService $scanner)
{
   $file = $request->file('document');
   if (!$scanner->scan($file)) {
       return response()->json([
           'message' => 'File rejected: security scan failed.'
       ], 422);
   }
}

Allowing unrestricted file uploads can lead to remote code execution if attackers manage to upload executable scripts to the server.- OWASP Security Guidelines

6. Advanced Security Techniques

6.1 Strip EXIF Metadata from Images
Use the intervention/image package to remove sensitive metadata from uploaded images before storage:

composer require intervention/image

use Intervention\Image\Facades\Image;

$image = Image::make($file->getRealPath());
// Strip all EXIF data by re-encoding
$cleanImage = $image->encode('jpg', 85);
Storage::disk('private')->put($path, $cleanImage);

6.2 Implement Rate Limiting on Upload Endpoints

// In routes/api.php
Route::middleware(['auth', 'throttle:10,1'])  // 10 uploads per minute
    ->post('/upload', [FileController::class, 'store']);

6.3 Queue Virus Scans for Large Files
For large files, run the virus scan asynchronously to avoid request timeouts:

// Create a job

php artisan make:job ScanUploadedFile

// Dispatch after initial upload
ScanUploadedFile::dispatch($document->id)->onQueue('scans');

6.4 Content Security Headers
Even with secure storage, add these HTTP headers when serving files to prevent browsers from executing served content:

return response()->download($path)->withHeaders([
   'X-Content-Type-Options' => 'nosniff',
   'Content-Disposition'    => 'attachment; filename="' . $filename . '"',
   'X-Frame-Options'        => 'DENY',
]);

7.Stats & Interesting Facts

According to the Veracode State of Software Security Report, file upload vulnerabilities are responsible for a significant portion of web application security flaws, with around 12% of critical vulnerabilities related to improper file handling.Source: https://www.veracode.com/resources/reports/state-of-software-security-report
The OWASP Top 10 highlights insecure file upload mechanisms as a common cause of Remote Code Execution (RCE) attacks when applications fail to properly validate file types and storage paths. Source: https://owasp.org/www-project-top-ten/
Research shows that attackers often bypass validation by disguising malicious scripts as image or PDF files, such as renaming a .php file to .jpg or .png.Source: https://owasp.org/www-community/vulnerabilities/Unrestricted_File_Upload
Malware scanning systems like ClamAV can detect up to 98% of known malware signatures, making antivirus scanning a useful additional layer for file upload security.Source: https://www.clamav.net/documents/clamav-user-manual
According to security studies, allowing unrestricted file sizes can lead to Denial-of-Service (DoS) attacks, where attackers intentionally upload large files to exhaust server disk space.Source: https://owasp.org/www-community/attacks/Denial_of_Service
According to cybersecurity research, over 90% of successful web attacks exploit known vulnerabilities that could have been prevented through proper validation and secure configuration practices.Source: https://www.verizon.com/business/resources/reports/dbir/
According to the SANS Institute, improper file upload validation is one of the most common ways attackers upload web shells, which can allow them to execute commands on the server and gain full control of the application.Source: https://www.sans.org/white-papers/

8. FAQ

1. Why is file upload security important in Laravel?
Ans: File upload security is important because attackers can upload malicious files such as scripts or malware. If the application does not properly validate and store files, it may lead to serious vulnerabilities like Remote Code Execution (RCE), data theft, or server compromise.

2. How can Laravel validate uploaded files?
Ans: Laravel provides built-in validation rules such as mimes, mimetypes, max, and file. These rules help ensure that only allowed file types and sizes are uploaded, reducing the risk of malicious files entering the system.

3. Where should uploaded files be stored in Laravel?
Ans: Uploaded files should ideally be stored in the storage directory instead of the public directory. Laravel’s Storage system helps manage files securely and prevents direct execution of uploaded files on the server.

4. How can I limit the file size in Laravel uploads?
Ans: You can limit file size using the max validation rule in Laravel. For example:
'file' => 'required|mimes:jpg,png,pdf|max:2048'
This restricts uploads to specific file types and a maximum size of 2MB.

5. Can Laravel scan uploaded files for viruses?
Ans: Laravel does not include built-in antivirus scanning, but you can integrate tools like ClamAV or third-party security services to scan uploaded files before storing or processing them.

6. What are some best practices for secure file uploads?
Ans: Best practices include validating file types, restricting file sizes, renaming uploaded files, storing them outside the public directory, scanning for malware, and setting proper file permissions.

7. How can attackers bypass file upload validation?
Ans: Attackers may rename malicious files to appear as safe formats (e.g., .php to .jpg) or manipulate MIME types. This is why multiple validation layers and secure storage practices are important.

8. Does Laravel provide protection against malicious file execution?
Ans: Laravel helps reduce risk through validation and secure storage systems, but developers must still implement proper validation, file renaming, and server configuration to fully protect against malicious file execution.

9. Conclusion

Secure file uploads are not a feature - they are a discipline. Every layer described in this article serves a specific purpose in a defense-in-depth strategy:

Validation catches malformed or unexpected files before they enter your system.
Secure storage ensures that even if validation is bypassed, files cannot be executed.
Virus scanning adds a safety net against sophisticated threats that evade other checks.
Rate limiting, signed URLs, and HTTP security headers complete the picture.

Laravel provides all the building blocks you need. The responsibility lies with developers to connect them thoughtfully, test them rigorously, and stay current with evolving attack patterns.

Security is never a one-time setup. Regularly update ClamAV virus definitions, review your validation rules when you add new file types, and audit your storage access logs. A breach avoided is invisible - and that invisibility is the mark of excellent security engineering.

About the Author: Abodh is a PHP and Laravel Developer at AddWeb Solution, skilled in MySQL, REST APIs, JavaScript, Git, and Docker for building robust web applications.

Clean Architecture in Laravel: Structuring Large-Scale Applications for Maintainability

Zemichael Mehretu — Thu, 19 Mar 2026 10:30:42 +0000

“Good state management doesn’t make apps bigger, it makes complexity visible and manageable.”

Key Takeaways

Clean Architecture protects business rules from framework, database, and third-party volatility.
The dependency rule is the core guardrail: dependencies must point inward.
Use cases should express business intent clearly, while controllers and adapters stay thin.
Teams scale better when boundaries are explicit and responsibilities are narrow.
Incremental migration beats big-bang rewrites for real production systems.

Index

Introduction
Why Large Laravel Systems Become Expensive to Change
Clean Architecture in Practical Terms
The Dependency Rule
The Four Layers in Laravel
Implementation Strategy for Teams
Minimal Example: Create User
Transactions, Events, and Integration Boundaries
Testing Strategy
Migration Roadmap for Existing Applications
Common Mistakes and How to Avoid Them
When to Use (and When to Keep It Simple)
FAQs
References
Conclusion

1. Introduction

Laravel gives excellent delivery speed in the early stages of a product. The challenge appears later, when an application evolves into many modules, many integrations, and many contributors. At that point, development speed depends less on how fast new code is written and more on how safely existing code can be changed.

Clean Architecture helps by separating stable business policy from volatile implementation details. It does not replace Laravel. It clarifies where business logic should live and where framework concerns should remain. With clear boundaries, teams can evolve API design, storage choices, queue systems, and external providers without repeatedly disturbing core business rules.

For long-lived products, this separation lowers maintenance effort, improves testability, and reduces refactoring risk.

2. Why Large Laravel Systems Become Expensive to Change

Most complexity in large Laravel apps comes from mixed responsibilities rather than from Laravel itself.

Common signs:

Controllers that validate input, apply domain rules, run persistence logic, and call external APIs in one action.
Eloquent models that mix persistence with pricing logic, eligibility checks, and side effects.
Repeated business rules spread across jobs, listeners, services, and controllers.
Feature tests carrying too much burden because unit-level logic is hard to isolate.
Frequent merge conflicts in central files touched by multiple teams.

This creates architecture debt. Each new feature takes longer because developers must navigate and protect too many implicit dependencies.

“Simple code is not code with fewer files; it is code where responsibilities are obvious.”

3. Clean Architecture in Practical Terms

Clean Architecture separates policy from detail.

Policy is business behavior that should remain stable over time.
Detail is technology that can change (framework APIs, ORM models, vendors, transports).

In Laravel context:

Domain contains business concepts and rules.
Application coordinates business actions through use cases.
Infrastructure implements contracts using Laravel and external services.
Presentation handles inputs and outputs for HTTP or CLI.

This model supports a useful goal: when infrastructure changes, business rules should not need to change.

4. The Dependency Rule

The dependency rule is simple and strict:

Source code dependencies must point inward.

Implications:

Domain should not depend on Request, Response, Eloquent models, facades, or SDK clients.
Application should depend on domain models and contracts, not Laravel storage or transport types.
Infrastructure should implement application contracts and contain framework integrations.
Presentation should call use cases and format responses, not own business policy.

If this rule is consistently enforced, architecture remains clean over time. If ignored, folder names may look clean while coupling silently grows.

5. The Four Layers in Laravel

A) Domain Layer (Core Business)
Purpose: express business truth.
Contains:

Entities and aggregates
Value objects
Domain services
Domain events

Characteristics:

Framework-agnostic
High cohesion
Focused on invariants and business language

B) Application Layer (Use Cases)
Purpose: execute business capabilities.
Contains:

Use cases (single intent actions)
Input and output DTOs
Contracts for repositories and gateways

Characteristics:

Coordinates workflows
Enforces application-level policies
Keeps orchestration out of controllers

C) Infrastructure Layer (Adapters)
Purpose: implement contracts using concrete tools.
Contains:

Eloquent repository implementations
External API adapters (payments, notifications, storage)
Event dispatching and queue adapters

Characteristics:

Framework-dependent
Replaceable implementations
Mapping between persistence and domain models

D) Presentation Layer (Delivery)
Purpose: handle transport concerns.
Contains:

Controllers
Form requests
API resources/serializers
Console commands

Characteristics:

Thin input/output handling
Delegation to use cases
Minimal business branching

“Clean Architecture is less about layers and more about protecting business decisions from framework churn.”

6. Implementation Strategy for Teams

A clean structure is not enough. The operating strategy matters.

Strategy 1: Target Volatile Business Flows First
Start where change is frequent: billing, ordering, pricing, eligibility, and fulfillment flows. These areas return architecture value quickly because requirement churn is high.

Strategy 2: Migrate by Vertical Slice
Move one complete use case at a time from controller to use case to contract to adapter. This keeps migration deliverable and testable.

Strategy 3: Keep Contracts Intentional
Introduce interfaces at true boundaries: persistence, external APIs, messaging, storage. Avoid interfaces for classes that are unlikely to vary.

Strategy 4: Keep Controllers Policy-Free
A controller should parse/validate input, call one use case, and shape output. When controllers decide business outcomes, the architecture starts leaking.

Strategy 5: Make Model Mapping Explicit
Map between transport objects, persistence models, and domain objects in adapters. Explicit mapping preserves control and avoids hidden coupling.

Strategy 6: Put Transaction Scope in Application
Use cases should define transaction boundaries when multiple changes must succeed or fail together. Domain entities should not manage transaction mechanics.

Strategy 7: Enforce Guardrails in Code Review
Add review checks:

Any framework imports in Domain?
Any Eloquent model usage in Use Cases?
Any business rules in controllers/listeners? Small governance habits prevent long-term drift.

Strategy 8: Keep Use Cases Focused
One use case should represent one business intent. If a use case grows broad, split by intent (ApproveRefund, RejectRefund, EscalateRefund).

Strategy 9: Support Parallel Team Work
Structure folders by bounded context where needed (Sales, Billing, Identity). This reduces cross-team collisions and clarifies ownership.

Strategy 10: Measure Architecture by Delivery Outcomes
Track lead time, change failure rate, rollback frequency, and escaped defects. Good architecture should improve these outcomes over time.

7. Minimal Example: Create User

The goal is to show boundaries, not framework detail.
Domain

final class User {
    public function __construct(public string $email, public string $name) {}
}

Application

interface UserRepository { public function save(User $user): void; }

final class CreateUser {
    public function __construct(private UserRepository $repo) {}
    public function execute(string $email, string $name): User {
        $user = new User($email, $name);
        $this->repo->save($user);
        return $user;
    }
}

Infrastructure

final class EloquentUserRepository implements UserRepository {
    public function save(User $user): void {
        UserModel::create(['email' => $user->email, 'name' => $user->name]);
    }
}

Presentation

final class UserController {
    public function store(Request $request, CreateUser $useCase) {
        $user = $useCase->execute($request->email, $request->name);
        return response()->json(['email' => $user->email]);
    }
}

The example stays intentionally small. In production, you would add value objects, validation rules, and error handling without changing layer direction.

8. Transactions, Events, and Integration Boundaries

These areas often cause hidden coupling.

Transactions belong at the application use-case boundary, where business operations are coordinated.
Domain events should describe business facts, not transport mechanisms.
Event publishing and queue dispatch should happen in infrastructure adapters.
Integration retries, circuit breaking, and API error mapping should remain outside the domain.

A useful test: if infrastructure changes, domain behavior should remain intact.

“If changing the database rewrites business rules, your boundaries are in the wrong place.”

9. Testing Strategy

A practical test distribution:

Domain unit tests for entities, value objects, and invariants.
Use case tests with mocked contracts.
Infrastructure integration tests for DB and external adapters.
Limited HTTP feature tests for request/response behavior.

This approach keeps most business tests fast and deterministic while still validating real integration seams.

10. Migration Roadmap for Existing Applications

Use an incremental path:

Choose a high-change feature.
Extract core rules into domain models.
Introduce a use case for one business intent.
Define contracts at application boundaries.
Implement contracts in infrastructure with current Laravel stack.
Route controllers to the new use case.
Add domain and use-case tests.
Repeat for adjacent flows.

This allows continuous delivery while architecture improves in-place.

11. Common Mistakes and How to Avoid Them

Architecture theater: renamed folders but no real boundary enforcement.
Over-abstraction: too many interfaces before variation exists.
Anemic domain: business logic pushed entirely into service classes.
Leaky boundaries: Eloquent models traveling into domain/application code.
Oversized use cases: broad classes handling unrelated intent.
Weak ownership: no team agreement on where policy belongs.

The fix is discipline: clear boundaries, focused use cases, and review guardrails.

12. When to Use (and When to Keep It Simple)

Use Clean Architecture when:

the product has long lifespan,
business logic is non-trivial,
multiple teams need parallel delivery,
external dependencies are significant.

Keep it lighter when:

the app is short-lived,
logic is mostly straightforward CRUD,
team size is small and scope is stable.

Architecture should match expected change, not theoretical perfection.

13. FAQs

Is Clean Architecture mandatory for Laravel projects?
No. It is a strategic choice for products where maintainability risk is high.

Will it slow us down?
There may be a short setup cost. In most long-lived systems, it reduces total delivery friction over time.

Where should validation happen?
Input validation belongs in presentation boundaries. Business invariants belong in domain/application.

Can this work without full DDD adoption?
Yes. You can apply clean boundaries incrementally and adopt DDD concepts where they add clear value.

14. References

Clean Architecture (Robert C. Martin): https://www.pearson.com/en-us/subject-catalog/p/clean-architecture-a-craftsmans-guide-to-software-structure-and-design/P200000003268/9780134494166
DORA Research Program: https://dora.dev
Stripe Developer Coefficient: https://stripe.com/reports/developer-coefficient
ISBSG: https://www.isbsg.org

15. Conclusion

Clean Architecture in Laravel is a practical way to keep business policy stable while technical details evolve. By enforcing inward dependencies, keeping use cases focused, and migrating incrementally, teams can preserve Laravel’s delivery speed and improve long-term maintainability at scale.

About the Author:Zemichael is a fullstackDeveloper at AddWeb Solution.Crafting web experiences with robust architecture, performance focus, and design thinking.

Code Quality Checks and Deployment with GitHub Actions

Rajan Vavadia — Wed, 18 Mar 2026 12:09:37 +0000

“The XP philosophy is to start where you are now and move towards the ideal. From where you are now, could you improve a little bit?” Kent Beck

Introduction
Overview: Two-Workflow Strategy (Quality Gate + Deploy)
Workflow 1: Code Quality Checks (PR → stg)
Workflow 2: Deploy (push → stg)
Practical Setup (Step-by-step)
FAQs
Key Takeaways
Conclusion

1. Introduction

This document explains a practical GitHub Actions setup that enforces Flutter code quality checks on every pull request to the stg branch and deploys the Flutter web build to AWS S3 whenever code is pushed to stg.

The approach uses two separate workflows:

Code Quality Checks: runs on pull_request events (stg)
Deploy (stg): runs on push events to stg

2. Overview: Two-Workflow Strategy (Quality Gate + Deploy)

Developers open a Pull Request targeting stg.
GitHub Actions runs Flutter clean → pub get → analyze → test.
If checks pass, the PR is safe to merge. If checks fail, a Rocket.Chat alert is sent with the build log.
After merge/push to stg, a separate Deploy workflow builds Flutter web and syncs build/web to an S3 bucket.
Deploy success/failure is reported to Rocket.Chat.

Why separate workflows?

PR checks run in the PR context and are ideal for gating merges.
Deploy runs only on trusted branch pushes, avoiding accidental deployments from feature branches.
Clearer troubleshooting: you can tell instantly whether a failure is quality-related or deployment-related.

3. Workflow 1: Code Quality Checks (PR → stg)

3.1 What it does

Triggers when a Pull Request targets branch stg.
Ensures only one run per PR branch using concurrency (cancels older in-progress runs when a new commit is pushed).
Runs Flutter quality steps and writes a readable log file (cq.log).
Uploads cq.log as an artifact (kept for 7 days).
Notifies Rocket.Chat on success or failure (failure attempts to upload cq.log to the room).

3.2 YAML: Code Quality Checks

name: Code Quality Checks

on:
  pull_request:
    branches: [stg]

concurrency:
  group: code-quality-${{ github.head_ref }}
  cancel-in-progress: true

env:
  SITE_URL:      ${{ secrets.SITE_URL }}
  ENV_NAME:      ${{ secrets.ENV_NAME }}
  RC_BASE_URL:   ${{ secrets.RC_BASE_URL }}
  RC_ROOM_ID:    ${{ secrets.RC_ROOM_ID }}
  RC_USER_ID:    ${{ secrets.RC_USER_ID }}
  RC_AUTH_TOKEN:  ${{ secrets.RC_AUTH_TOKEN }}

jobs:
  code-quality:
    name: Flutter Analyze & Test
    runs-on: ubuntu-latest
    timeout-minutes: 30

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Flutter
        uses: subosito/flutter-action@v2
        with:
          flutter-version: '3.29.2'

      - name: Run code quality checks
        id: quality
        run: |
          set -e
          LOG="cq.log"

          echo "════════════════════════════════════════════════════" > "$LOG"
          echo " Code Quality Checks Started"                        >> "$LOG"
          echo "═══════════════════════════════════════════════════════" >> "$LOG"

          echo ""                                                      >> "$LOG"
          echo " STEP 1: flutter clean"                              >> "$LOG"
          echo "───────────────────────────────────────────────────────" >> "$LOG"
          flutter clean >> "$LOG" 2>&1

          echo ""                                                      >> "$LOG"
          echo " STEP 2: flutter pub get"                            >> "$LOG"
          echo "───────────────────────────────────────────────────────" >> "$LOG"
          flutter pub get >> "$LOG" 2>&1

          echo ""                                                      >> "$LOG"
          echo " STEP 3: flutter analyze"                            >> "$LOG"
          echo "───────────────────────────────────────────────────────" >> "$LOG"
          flutter analyze >> "$LOG" 2>&1

          echo ""                                                      >> "$LOG"
          echo " STEP 4: flutter test"                               >> "$LOG"
          echo "───────────────────────────────────────────────────────" >> "$LOG"
          flutter test >> "$LOG" 2>&1

          echo ""                                                      >> "$LOG"
          echo "═══════════════════════════════════════════════════════" >> "$LOG"
          echo "All checks passed!"                                  >> "$LOG"
          echo "═══════════════════════════════════════════════════════" >> "$LOG"

      - name: Upload build log
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: code-quality-log
          path: cq.log
          retention-days: 7

      - name: Notify Rocket.Chat (success)
        if: success()
        run: |
          REPO_URL="${{ github.event.repository.html_url }}"
          COMMIT_URL="${REPO_URL}/commit/${{ github.sha }}"
          AUTHOR="${{ github.event.pull_request.user.login }}"
          DURATION="${{ env.DURATION }}"

          MSG_TEXT="*Code Quality Checks Passed*

          *Site-URL:* ${SITE_URL}
          *PR Title:* ${{ github.event.pull_request.title }}
          *PR URL:* ${{ github.event.pull_request.html_url }}
          *Environment:* ${ENV_NAME}

          *on PR:* ${{ github.head_ref }} → ${{ github.base_ref }}
          *Git Commit:* ${COMMIT_URL}
          *Git Author:* ${AUTHOR}"
          jq -n \
            --arg roomId "${RC_ROOM_ID}" \
            --arg text "$MSG_TEXT" \
            '{roomId: $roomId, text: $text}' | \
          curl -sS -X POST "${RC_BASE_URL}/api/v1/chat.postMessage" \
            -H "X-User-Id: ${RC_USER_ID}" \
            -H "X-Auth-Token: ${RC_AUTH_TOKEN}" \
            -H "Content-Type: application/json" \
            -d @- || echo "Notification failed, continuing"

      - name: Notify Rocket.Chat (failure)
        if: failure()
        run: |
          REPO_URL="${{ github.event.repository.html_url }}"
          COMMIT_URL="${REPO_URL}/commit/${{ github.sha }}"
          AUTHOR="${{ github.event.pull_request.user.login }}"

          MSG_TEXT="*Code Quality Checks Failed*

          *Site-URL:* ${SITE_URL}
          *PR Title:* ${{ github.event.pull_request.title }}
          *PR URL:* ${{ github.event.pull_request.html_url }}
          *Environment:* ${ENV_NAME}

          *on PR:* ${{ github.head_ref }} → ${{ github.base_ref }}
          *Git Commit:* ${COMMIT_URL}
          *Git Author:* ${AUTHOR}

           *Full build log attached below:*"

          UPLOAD_RESP=$(curl -sS -X POST "${RC_BASE_URL}/api/v1/rooms.media/${RC_ROOM_ID}" \
            -H "X-User-Id: ${RC_USER_ID}" \
            -H "X-Auth-Token: ${RC_AUTH_TOKEN}" \
            -F "file=@cq.log") || true

          FILE_URL=$(echo "$UPLOAD_RESP" | jq -r '.file.url // empty')

          if [ -n "$FILE_URL" ]; then
            jq -n \
              --arg rid "${RC_ROOM_ID}" \
              --arg msg "$MSG_TEXT" \
              --arg title "📄 BUILD LOG (cq.log) - Click to download" \
              --arg link "$FILE_URL" \
              --arg desc "Contains detailed output of all build steps" \
              '{message: {rid: $rid, msg: $msg, attachments: [{title: $title, title_link: $link, text: $desc, collapsed: false}]}}' | \
            curl -sS -X POST "${RC_BASE_URL}/api/v1/chat.sendMessage" \
              -H "X-User-Id: ${RC_USER_ID}" \
              -H "X-Auth-Token: ${RC_AUTH_TOKEN}" \
              -H "Content-Type: application/json" \
              -d @-
          else
            jq -n \
              --arg roomId "${RC_ROOM_ID}" \
              --arg text "$MSG_TEXT" \
              '{roomId: $roomId, text: $text}' | \
            curl -sS -X POST "${RC_BASE_URL}/api/v1/chat.postMessage" \
              -H "X-User-Id: ${RC_USER_ID}" \
              -H "X-Auth-Token: ${RC_AUTH_TOKEN}" \
              -H "Content-Type: application/json" \
              -d @-
          fi || echo "Failure notification failed"

3.3 How logs + artifacts work

The workflow writes all step output to cq.log so your team has one clean, shareable log file.
Upload build log runs with if: always(), so the artifact is saved even when analyze/test fails.
retention-days: 7 keeps the artifact for one week to reduce storage and keep logs relevant.

3.4 Rocket.Chat notifications (success/failure)
Two messages are sent:

Success: posts a summary (PR title, PR URL, environment, branch and commit details).
Failure: tries to upload cq.log to the room, then posts a message with a link to the uploaded file.

If uploading fails, it still posts the failure summary so you are notified.
Prerequisites on the runner:

jq must be available (it is available by default on ubuntu-latest runners).
Rocket.Chat API base URL, user id, auth token, and room id must be configured as secrets.

4. Workflow 2: Deploy (push → stg)

4.1 What it does

Triggers on push to stg (typically after PR merge).
Builds Flutter web (build/web output).
Configures AWS credentials on the runner.
Syncs build/web to S3 bucket using aws s3 sync --delete.
Notifies Rocket.Chat on success/failure withcommit and author details.

4.2 YAML: Deploy (stg)

name: Deploy (stg)

on:
  push:
    branches: [stg]

concurrency:
  group: deploy-stg
  cancel-in-progress: false

permissions:
  contents: read

env:
  SITE_URL:      ${{ secrets.SITE_URL }}
  ENV_NAME:      ${{ secrets.ENV_NAME }}
  RC_BASE_URL:   ${{ secrets.RC_BASE_URL }}
  RC_ROOM_ID:    ${{ secrets.RC_ROOM_ID }}
  RC_USER_ID:    ${{ secrets.RC_USER_ID }}
  RC_AUTH_TOKEN:  ${{ secrets.RC_AUTH_TOKEN }}

jobs:
  build-and-deploy:
    name: Build & Deploy to AWS S3
    runs-on: ubuntu-latest
    timeout-minutes: 30

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Flutter
        uses: subosito/flutter-action@v2
        with:
          flutter-version: '3.29.2'

      - name: Flutter clean
        run: flutter clean

      - name: Flutter build web
        run: |
          flutter build web \
            --dart-define=FLUTTER_WEB_USE_SKIA=false \
            --pwa-strategy=none

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ secrets.AWS_REGION }}

      - name: Deploy to S3
        run: |
          aws s3 sync build/web s3://${{ secrets.AWS_S3_BUCKET }} \
            --delete

      - name: Notify Rocket.Chat (success)
        if: success()
        run: |
          REPO_URL="${{ github.event.repository.html_url }}"
          COMMIT_URL="${REPO_URL}/commit/${{ github.sha }}"
          AUTHOR=$(git log -1 --pretty=%an)

          MSG_TEXT="*Deployment Successful*

          *Site-URL:* ${SITE_URL}
          *Environment:* ${ENV_NAME}

          *Branch:* ${{ github.ref_name }}
          *Git Commit:* ${COMMIT_URL}
          *Git Author:* ${AUTHOR}"

          jq -n \
            --arg roomId "${RC_ROOM_ID}" \
            --arg text "$MSG_TEXT" \
            '{roomId: $roomId, text: $text}' | \
          curl -sS -X POST "${RC_BASE_URL}/api/v1/chat.postMessage" \
            -H "X-User-Id: ${RC_USER_ID}" \
            -H "X-Auth-Token: ${RC_AUTH_TOKEN}" \
            -H "Content-Type: application/json" \
            -d @- || echo "Deployment notification failed"

      - name: Notify Rocket.Chat (failure)
        if: failure()
        run: |
          REPO_URL="${{ github.event.repository.html_url }}"
          COMMIT_URL="${REPO_URL}/commit/${{ github.sha }}"
          AUTHOR=$(git log -1 --pretty=%an)

          MSG_TEXT="*Deployment Failed*

          *Site-URL:* ${SITE_URL}
          *Environment:* ${ENV_NAME}

          *Branch:* ${{ github.ref_name }}
          *Git Commit:* ${COMMIT_URL}
          *Git Author:* ${AUTHOR}"

          jq -n \
            --arg roomId "${RC_ROOM_ID}" \
            --arg text "$MSG_TEXT" \
            '{roomId: $roomId, text: $text}' | \
          curl -sS -X POST "${RC_BASE_URL}/api/v1/chat.postMessage" \
            -H "X-User-Id: ${RC_USER_ID}" \
            -H "X-Auth-Token: ${RC_AUTH_TOKEN}" \
            -H "Content-Type: application/json" \
            -d @- || echo "Deployment failure notification failed"

“Beware of bugs in the above code; I have only proved it correct, not tried it.” Donald Knuth

4.3 AWS S3 deployment notes

S3 bucket must exist and the AWS credentials must have permission to sync to it.
aws s3 sync --delete removes files in S3 that no longer exist in build/web (prevents stale assets).
If you use CloudFront, consider adding an invalidation step (optional) after sync.
If your site is a Single Page App, configure S3/CloudFront to route 404s to index.html.

5. Practical Setup (Step-by-step)

5.1 Create workflow files

In your repo, create folder: .github/workflows/
Add file: code-quality.yml (paste Workflow 1 YAML).
Add file: deploy.yml (paste Workflow 2 YAML).
Commit and push to GitHub.

5.2 Configure GitHub secrets
Go to: Repository → Settings → Secrets and variables → Actions → New repository secret

SITE_URL (example: https://stg.example.com)
ENV_NAME (example: stg)
RC_BASE_URL (example: https://chat.example.com)
RC_ROOM_ID (Rocket.Chat room id)
RC_USER_ID (Rocket.Chat API user id)
RC_AUTH_TOKEN (Rocket.Chat API token)
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_REGION (example: ap-south-1)
AWS_S3_BUCKET (bucket name only, without s3://)

5.3 Set up branch protection (recommended)

In GitHub: Settings → Branches → Add branch protection rule for stg.
Enable 'Require a pull request before merging'.
Enable 'Require status checks to pass before merging'.
Select the workflow check name that appears for the PR (Flutter Analyze & Test / Code Quality Checks).
5.4 Verify end-to-end flow
Create a feature branch and open a PR to stg.
Confirm Code Quality Checks workflow starts automatically.
If it passes, merge the PR to stg.
Confirm Deploy (stg) runs on the push event and uploads new web build to S3.
Check Rocket.Chat channel for the success messages.

5.5 Common improvements (optional but practical)

Add caching (Flutter pub cache) to speed up builds.
Run flutter format --set-exit-if-changed and/or dart analyze for stricter linting.
Use AWS OIDC instead of long-lived AWS keys (more secure) if your org supports it.
Add environment protection for stg deployments (manual approval for deploy job).

Practical Demonstration (Images Explained)

Step 1: Modify the README file on the feature branch.
In this demonstration, the README.md file was updated with the text "Addwebsolution" and the changes were committed and pushed to the feature-dev branch.

Step 2: Create a Pull Request from feature-dev to the stg Branch

Once the Pull Request is created, the Code Quality Checks workflow is automatically triggered.

Upon successful completion of the GitHub Actions pipeline, a Rocket.Chat notification is dispatched to the configured channel.

The screenshot below shows the Pull Request in a mergeable state the Code Quality status check has passed (indicated by the green checkmark), and the "Merge pull request" button is now enabled.

Step 3: Click the "Confirm Merge" button to merge the Pull Request into stg. This push event automatically triggers the Deploy workflow.

The Deploy (stg) pipeline is automatically triggered and the deployment job begins execution

The deployment pipeline has completed successfully the Flutter web build artifacts have been synced to the configured AWS S3 bucket.

A Rocket.Chat notification confirms the successful deployment, including details such as site URL, environment name, branch, commit hash, and author.

Code Quality and Deployment Pipeline Failure Scenario

Step 4: Introduce a deliberate syntax error in main.dart (e.g., a missing closing parenthesis) and push the changes to the feature branch.

A new Pull Request is created targeting the stg branch, which automatically triggers the Code Quality Checks workflow.

Step 5: The Code Quality Checks workflow fails due to the syntax error for example, print("hello" is missing its closing parenthesis. The flutter analyze step detects the issue and the pipeline exits with a non-zero status code.

A failure notification is sent to Rocket.Chat with the build log attached. Since the Code Quality status check has failed, the Pull Request cannot be merged, and the Deploy workflow is never triggered effectively preventing broken code from reaching the stg environment.

This practical demonstration validates the end-to-end CI/CD pipeline behavior across both success and failure scenarios. When code quality checks pass, the Pull Request is merged into the stg branch and the Deploy workflow automatically builds and publishes the Flutter web application to AWS S3, with a success notification delivered to Rocket.Chat. Conversely, when a syntax error or lint violation is detected, the Code Quality Checks workflow fails, the Pull Request is blocked from merging, and the Deploy workflow is never invoked ensuring that only validated, production-ready code reaches the staging environment. This two-workflow strategy enforces a reliable quality gate at the PR level while keeping deployment isolated to trusted branch pushes.

“Continuous Integration is not a tool. It is a practice.” Common CI principle (team reminder)

8. FAQs

Q1. Why does Code Quality run on pull_request but Deploy runs on push?
A. pull_request is best for quality gates before merging, while push to stg is best for deployments because it limits deploys to the stg branch history (usually protected).

Q2. What does concurrency do in Code Quality Checks?
A. It groups runs by the PR branch name (github.head_ref). If a new commit is pushed to the same PR branch, the older run is canceled.

Q3. Why upload cq.log if GitHub Actions already has logs?
A. cq.log is a single consolidated file you can share, attach to chats, and keep as an artifact. It also makes Rocket.Chat upload easy.

Q4. What if Rocket.Chat notification fails?
A. The workflow prints a warning and continues. Your CI status still reflects pass/fail correctly.

Q5. What permissions are needed for S3 deploy?
A. The AWS identity used in GitHub Actions needs at minimum s3:ListBucket and s3:PutObject/DeleteObject permissions for the target bucket.

Q6. How do I confirm the build is deployed correctly?
A. Check the S3 bucket objects update time and open SITE_URL in browser. If CloudFront is used, confirm cache invalidation or TTL behavior.

9. Key Takeaways

PR quality checks prevent broken code from entering stg.
Concurrency prevents duplicate CI runs and saves time.
Artifacts (cq.log) make debugging faster and help the team collaborate.
Deploy is isolated to stg pushes, which is safer and easier to audit.
Rocket.Chat alerts keep the team informed without opening GitHub every time.

10. Conclusion

This setup provides a clean CI/CD workflow for a Flutter web project: enforce quality on pull requests and deploy only after stg updates. With proper secrets, branch protections, and clear notifications, your team gets fast feedback and reliable deployments.

About the Author:Rajan is a DevOps Engineer at AddWebSolution, specializing in automation infrastructure, Optimize the CI/CD Pipelines and ensuring seamless deployments.

Run Your Own ChatGPT Offline: Open WebUI + Ollama + Local Knowledge Base

Ankit Parmar — Mon, 16 Mar 2026 07:49:26 +0000

“The future of AI will be hybrid: local intelligence with cloud augmentation.” - Satya Nadella

When it comes to running AI assistants privately and offline, you have two main paths: use cloud-hosted LLM APIs like OpenAI or Anthropic, or run models locally on your own machine. Both approaches are valid, but in this article, we’ll focus on building a fully local AI assistant using Open WebUI and Ollama.

Why Local AI?

Because it gives you full privacy, offline access, zero API cost per token, and control over your models and data - especially important for sensitive documents or internal knowledge bases.

As a practical example, we’ll build a local assistant that can answer questions from your own documents (PDFs, notes, markdown files) using a local RAG (Retrieval-Augmented Generation) pipeline.

What Is Open WebUI?

Open WebUI is a self-hosted web interface for local LLMs.
It provides a ChatGPT-like experience in your browser while running models entirely on your own hardware via Ollama.

Key capabilities:

Chat with local LLMs
Upload and query documents
Multi-model switching
Local RAG knowledge base
User accounts & roles
Tool and plugin support

Think of it as:

ChatGPT UI + Local Models + Private Knowledge

Ollama vs Cloud LLM APIs

Cloud APIs (OpenAI, Claude, etc.)

Require internet access
Pay per token
Data leaves your environment
Highest model quality
No hardware requirements

Ollama Local Models

Run fully offline
No per-token cost
Private data stays local
Hardware dependent
Slightly lower model quality In this article, we’ll use Ollama for fully local inference.

“Privacy is not an option, and it shouldn’t be the price we accept for just getting on the Internet.” - Gary Kovacs

Index

Set Up Ollama
Install Open WebUI
Download Local LLM
Create Local Knowledge Base
Upload Documents
Ask Questions Over Your Data
How Local RAG Works
Why Local AI Makes Sense
Next Steps You Can Take
Watch Out For
Interesting Facts
FAQ
Conclusion

Building Your Offline ChatGPT: Open WebUI + Ollama

1. Set Up Ollama

Install Ollama:
Mac / Linux:

curl -fsSL https://ollama.com/install.sh | sh

Windows:
Download installer from https://ollama.com
Verify installation:

ollama --version

Start Ollama service:

ollama serve

Ollama runs a local model server at:

http://localhost:11434

2. Install Open WebUI

The easiest method is Docker.

docker run -d \
  -p 3000:8080 \
  -v open-webui:/app/backend/data \
  --name open-webui \
  ghcr.io/open-webui/open-webui:main

Open your browser:

http://localhost:3000

Create your admin account on first launch.

3. Download a Local LLM

Pull a model via Ollama:

ollama pull llama3

Other good options:

mistral
qwen2.5
phi3
codellama Check installed models:

ollama list

Open WebUI will automatically detect Ollama models.

4. Create a Local Knowledge Base (RAG)

Open WebUI includes built-in Retrieval-Augmented Generation.
Steps:

Go to Workspace → Knowledge
Create new knowledge base
Upload documents:
- PDFs
- TXT
- Markdown
- DOCX
- HTML

The system automatically:

Splits text into chunks
Generates embeddings
Stores vectors locally
Links to your LLM No external vector DB required.

5. Upload Documents

Example:

Company docs
Personal notes
Research papers
Codebase
Product manuals

Once uploaded, documents become searchable context for the model.

“Data gravity will pull AI to where the data lives.” - Dave McCrory

6. Ask Questions Over Your Data

Now chat normally:
“Summarize our API documentation”
“What does the onboarding process require?”
“Explain section 4 of the PDF”
Open WebUI retrieves relevant chunks and sends them to the model.

This is local RAG in action.

7.How Local RAG Works

Pipeline:
User question
→ Embed query
→ Search local vectors
→ Retrieve relevant chunks
→ Send to LLM with context
→ Generate answer
Everything runs locally.
No cloud.
No API.

8.Why Local AI Makes Sense

Full privacy - data never leaves your machine
Zero API costs after setup
Offline availability
Custom models & prompts
Internal knowledge assistants
Regulatory compliance friendly

Perfect for:

Companies
Developers
Researchers
Students
Privacy-focused users

“Open models accelerate innovation by removing access barriers.” - Yann LeCun

9.Next Steps You Can Take

Connect multiple models (coding + chat)
Use larger models with GPU
Share internal AI assistant in LAN
Index full code repositories
Build private ChatGPT for your team
Add tools and function calling

10.Watch Out For

Hardware limits: large models need RAM/GPU
Embedding size: big docs consume storage
Model quality: smaller local models less capable
Chunking errors: bad splits reduce accuracy
Context limits: local models have smaller windows

11.Interesting Facts

Local LLM inference costs can drop 90–99% compared to API usage after hardware amortization. https://arxiv.org/abs/2601.09527
Most enterprise AI deployments in regulated sectors prefer on-prem or private-cloud LLMs for compliance. https://www.sitepoint.com/local-llms-vs-cloud-api-cost-analysis-2026
RAG systems reduce hallucinations by grounding models in real documents.https://www.preprints.org/manuscript/202504.1236/v1
Open-source LLMs have improved >10× in benchmark scores since 2023. https://localaimaster.com/blog/best-open-source-llms-2026

12.FAQ

Do I need a GPU to run Open WebUI + Ollama?
No. Small models run on CPU. GPU improves speed and allows larger models.
Is everything really offline?
Yes. Models, embeddings, and documents stay local unless you enable external APIs.
What models work best locally?
Mistral, Llama 3, Qwen, and Phi are strong general-purpose local models.
Can I use my own documents?
Yes. Upload PDFs, text, markdown, or code to the knowledge base.
Is this secure for company data?
Yes. Nothing leaves your infrastructure if hosted locally.
How big can my knowledge base be?
Limited by disk space and embeddings storage. Many GB is fine.
Can multiple users access it?
Yes. Open WebUI supports accounts and roles.

13.Conclusion

Running your own offline ChatGPT-style assistant is now practical with Open WebUI and Ollama. You get privacy, control, and zero per-token cost while still enabling powerful AI search over your own knowledge.

For individuals, it’s a personal AI brain.
For teams, it’s a private knowledge assistant.
For companies, it’s compliant AI infrastructure.

Local AI isn’t replacing cloud models - it’s becoming the private layer that sits beside them.

About the Author:Ankit is a full-stack developer at AddWebSolution and AI enthusiast who crafts intelligent web solutions with PHP, Laravel, and modern frontend tools.

Server-Side Rendering with Laravel and React (Using Inertia.js)

Ankit Parmar — Thu, 12 Mar 2026 08:15:57 +0000

“Make the simple case fast.” - Alan Kay

Modern web applications are expected to be fast, SEO-friendly, and interactive. Traditionally, teams achieved this by splitting responsibilities: Laravel for the backend, React for the frontend, and a lot of glue code in between.

That split works but it comes with cost:

Duplicate routing logic
API maintenance overhead
Auth and state synchronization issues
SEO and performance compromises

Inertia.js changes this equation.
In this article, we’ll explore how to implement Server-Side Rendering (SSR) with Laravel and React using Inertia.js, explain how it works internally, and understand when this approach makes sense and when it doesn’t.
We’ll focus on architecture, flow, and usage, not UI boilerplate.

Why Server-Side Rendering Still Matters

Client-side rendering alone has real drawbacks:

Slower first contentful paint
Poor SEO without extra tooling
JS-heavy initial loads
Bad experience on slow networks

Server-side rendering solves this by:

Rendering the initial page on the server
Sending ready-to-display HTML to the browser
Hydrating React on the client afterward

SSR gives you speed + SEO + UX without sacrificing interactivity.

Why Laravel + React with Inertia.js?

Inertia.js sits in a unique middle ground:

Not a traditional SPA
Not a classic Blade-only app
A modern monolith with React-driven views

What Inertia does:

Keeps routing and controllers on the server (Laravel)
Uses React for rendering pages
Eliminates the need for a separate API layer
Passes data as props, not JSON endpoints

You write server-driven apps with client-side UX.

Key Takeaways

Inertia.js enables SPA-like experiences using Laravel and React without building or maintaining a separate API layer.
Server-Side Rendering (SSR) improves first-page load performance, SEO, and Core Web Vitals for content-heavy applications.
Laravel remains the single source of truth for routing, validation, and authorization.
React components are used as page views, not independent frontend routes.
Inertia eliminates common SPA pain points such as API versioning, duplicated logic, and excessive frontend boilerplate.
SSR with Inertia works best for dashboards, admin panels, SaaS apps, and content-driven platforms.
This architecture scales well while keeping the codebase simpler and easier to maintain than traditional SPAs.

Index

Introduction
What Is Inertia.js?
How SSR Works with Laravel and React
Architecture Overview
Setting Up Laravel + Inertia (Conceptual)
Server-Side Rendering Flow
Data Sharing and Props
Routing and Navigation
Handling SEO and Meta Data
Authentication and Sessions
How the Frontend Uses the Backend
Why This Architecture Makes Sense
Watch Out For
Next Steps You Can Take
Interesting Facts
FAQ
Conclusion

1. Introduction

Building modern web apps often means choosing between two trade-offs: the simplicity of server-rendered applications or the rich user experience of Single Page Applications (SPAs). Traditional SPAs introduce extra complexity through separate frontends, APIs, and state management, while classic server rendering can feel limiting for interactive UIs.

Inertia.js bridges this gap.

With Inertia.js, Laravel remains responsible for routing and data, while React handles rendering without requiring a separate API layer. When combined with Server-Side Rendering (SSR), this approach delivers faster initial loads, better SEO, and improved Core Web Vitals, especially for content-heavy pages.

This article explains how Server-Side Rendering with Laravel and React using Inertia.js works, and why it’s a practical, maintainable alternative to traditional SPA architectures.

2. What Is Inertia.js?

Inertia.js is a glue layer, not a framework.
It allows you to:

Use Laravel controllers as page endpoints
Render React components instead of Blade views
Navigate without full page reloads
Avoid building and maintaining REST or GraphQL APIs
Inertia doesn’t replace Laravel or React, it connects them cleanly.

3. How SSR Works with Laravel and React

At a high level:

Browser requests a page
Laravel controller runs
Inertia renders a React page on the server
HTML is returned to the browser
React hydrates and takes over on the client

After that:

Navigation is handled client-side
Only data (props) is exchanged
No full reloads
You get SSR for the first load, SPA behavior afterward.

4. Architecture Overview

Browser
  ↓ HTTP request
Laravel Router
  ↓ Controller
Inertia Response
  ↓
React Page (SSR)
  ↓ HTML
Browser
  ↓ Hydration
Client-side Navigation

Key idea:
Laravel owns routing and data. React owns rendering and interactivity.

5. Setting Up Laravel + Inertia (Conceptual)

Typical setup involves:

Laravel backend
Inertia middleware
React frontend bundled via Vite
Shared SSR entry point

Important detail:
There is no API layer between Laravel and React.

Controllers return Inertia responses, not JSON.

6. Server-Side Rendering Flow

When SSR is enabled:

Laravel boots a Node process (via Vite/SSR)
React renders the page using provided props
HTML is injected into the response
Browser receives a fully rendered page

If SSR fails:

Inertia gracefully falls back to client-side rendering

This makes SSR opt-in but resilient.

7. Data Sharing and Props

Data flows like this:

Controller → Inertia::render() → React props

// Example 

namespace App\Http\Controllers;

use Inertia\Inertia;

class DashboardController extends Controller
{
    public function index()
    {
        return Inertia::render('Dashboard', [
            'user' => auth()->user(),
            'stats' => [
                'projects' => 12,
                'tasks' => 87,
            ],
        ]);
    }
}

There is:

No fetch()
No axios
No duplicated data contracts

Shared data (auth user, flash messages, settings) can be injected globally using Inertia middleware.
This keeps data flow predictable and centralized.

“The best architectures reduce the number of things you need to think about.” - Martin Fowler

8. Routing and Navigation

Routing stays in Laravel:

/web.php → Controller → React page

Route::get('/dashboard', [DashboardController::class, 'index'])
    ->middleware(['auth']);

Navigation:

Uses <Link> components
Intercepts clicks
Requests only new props
Preserves page state

You get SPA-like navigation without SPA complexity.

9. Handling SEO and Meta Data

With SSR:

Search engines receive rendered HTML
Titles and meta tags are visible immediately
Social previews work correctly

Meta management is handled at the React page level, but rendered server-side.
This is a major advantage over pure client-side SPAs.

10. Authentication and Sessions

One of Inertia’s biggest strengths.
Because you’re still using Laravel:

Sessions work normally
CSRF protection stays intact
Middleware applies automatically

No token juggling.
No auth duplication.
No client-side hacks.

“Simplicity is the ultimate sophistication.” - Leonardo da Vinci

11. How the Frontend Uses the Backend

// resources/js/Pages/Dashboard.jsx

import React from 'react';
import { Head } from '@inertiajs/react';

export default function Dashboard({ user, stats }) {
  return (
    <>
      <Head title="Dashboard" />

      <h1>Welcome, {user.name}</h1>

      <ul>
        <li>Projects: {stats.projects}</li>
        <li>Tasks: {stats.tasks}</li>
      </ul>
    </>
  );
}

From the frontend’s perspective:

Pages receive props
Actions submit forms via Inertia
Errors and validation flow back automatically

The frontend never:

Calls raw APIs
Manages auth headers
Worries about backend URLs

This drastically reduces frontend complexity.

12. Why This Architecture Makes Sense

Single codebase
Strong backend authority
SEO-friendly by default
Fast initial loads
Minimal boilerplate
Easier onboarding for teams

This is why many teams describe Inertia as “the missing link”.

13. Watch Out For

Not ideal for public APIs
Tight coupling between frontend and backend
Requires Node for SSR
Not suited for microservices-heavy setups

This is a monolith-friendly architecture – and that’s okay.

14. Next Steps You Can Take

Add SSR caching
Implement partial reloads
Introduce shared layout components
Optimize hydration performance
Add WebSockets for real-time features

15. Interesting Facts

Inertia.js was created to reduce SPA complexity without losing UX benefits Source. https://inertiajs.com/docs/v1/getting-started
SSR significantly improves Core Web Vitals for content-heavy pages Source. https://web.dev/articles/rendering-on-the-web
Inertia avoids versioned API maintenance entirely Source. https://inertiajs.com/docs/v2/core-concepts/how-it-works

16. FAQ

1. Is Inertia.js a framework?
No. It’s a thin adapter between server and frontend frameworks.

2. Can I use Vue instead of React?
Yes. Inertia supports Vue and Svelte as well.

3. Does SSR work without JavaScript?
Initial render does. Interactivity requires JS for hydration.

4. Is this better than Next.js?
Different tools. Inertia is backend-driven; Next.js is frontend-driven.

5. Can this scale?
Yes – for most product-style applications.

17. Conclusion

Server-side rendering with Laravel, React, and Inertia.js offers a powerful alternative to traditional SPAs and fully decoupled architectures.
By keeping routing, auth, and data on the server while leveraging React for UI, you get:

Faster pages
Better SEO
Less complexity
Happier teams

If your product fits a server-driven model, this approach is hard to beat.

About the Author: Ankit is a full-stack developer at AddWebSolution and AI enthusiast who crafts intelligent web solutions with PHP, Laravel, and modern frontend tools.

DEV Community: AddWeb Solution Pvt Ltd

Docker Image with Multi-Stage Builds & Docker Images Optimization

Table of Contents

1. Introduction

2. Overview: Multi-Stage Build Strategy (Builder + Production)

3. Stage 1: Builder - Compile Extensions, Install Tools & Dependencies

4. Stage 2: Production - Minimal Runtime Image

5. The .dockerignore File

6. Practical Setup (Step-by-step)

8. FAQs

9. Key Takeaways

10. Conclusion

Architecting Large-Scale Next.js Applications (Folder Structure, Patterns, Best Practices)

Key Takeaways

Index

Introduction

Why Architecture Matters

Core Principles of Scalable Architecture

Advanced Folder Structure (Enterprise-Level)

Architectural Patterns (Deep Dive)

State Management at Scale

Data Fetching & API Layer Design

Authentication & Authorization Architecture

Performance Optimization (Advanced)

Error Handling & Logging

Testing Strategy (Production-Ready)

Dev Experience & Code Quality

Deployment & Infrastructure Strategy

Real-World Example (Enterprise Dashboard)

Interesting Facts

Stats

FAQ’s

Conclusion

ELK Stack Setup for Centralized Log Management & Monitoring

Table of Contents

1. Introduction

2. Stage 1: Elasticsearch - The Search & Storage Engine

3. Stage 2: Logstash - The Data Processing Pipeline

4. Stage 3: Kibana - The Visualization Layer

5. Stage 4: Filebeat - The Lightweight Log Shipper

6. Connecting the Pieces - End-to-End Data Flow

7. Practical Setup (Step-by-step)

8. Troubleshooting Common Issues

9. FAQs

10. Key Takeaways

11. Conclusion

Nginx + PHP + MySQL Optimisations and Parameter Calculations

Table of Contents

1. Introduction

2. Architecture Overview

3. Why Optimisation Is Required

4. Nginx Optimisations & Parameter Calculations

5. PHP-FPM Optimisations & Parameter Calculations

6. MySQL Optimisations & Parameter Calculations

7. System-Level Optimisations (Linux)

8. Practical Server Size Examples

9. Interesting Facts & Statistics

11. FAQs

12. Key Takeaways

13. Conclusion

Testing Node.js APIs: Jest, Supertest, and Best Practices

Key Takeaways

Index

1) Introduction

2) Why Node.js API Test Suites Become Fragile Over Time

3) Testing APIs in Practical Terms

4) Core Testing Principles (The Guardrails)

5) The Four Testing Layers for Node.js APIs

6) Implementation Strategy for Engineering Teams

7) Minimal Example: Create User Endpoint with Jest + Supertest

8) Databases, Queues, and External Integrations

9) Testing Strategy by Risk and Feedback Speed

10) Migration Roadmap for Existing Node.js APIs

11) Common Mistakes and How to Avoid Them

12) When to Go Deep (and When to Keep It Lean)

13) FAQs

14) References

15) Interesting Facts

16) Stats

17) Conclusion