<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Ze Horta</title>
    <description>The latest articles on DEV Community by Ze Horta (@zhoorta).</description>
    <link>https://dev.to/zhoorta</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4171712%2Ff0948dbc-fd21-4b2e-8af7-2a985cd3735a.png</url>
      <title>DEV Community: Ze Horta</title>
      <link>https://dev.to/zhoorta</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/zhoorta"/>
    <language>en</language>
    <item>
      <title>Does a GPU let you fit more users on a shared Linux desktop server? I measured it</title>
      <dc:creator>Ze Horta</dc:creator>
      <pubDate>Thu, 29 Oct 2026 18:41:32 +0000</pubDate>
      <link>https://dev.to/zhoorta/does-a-gpu-let-you-fit-more-users-on-a-shared-linux-desktop-server-i-measured-it-4k7f</link>
      <guid>https://dev.to/zhoorta/does-a-gpu-let-you-fit-more-users-on-a-shared-linux-desktop-server-i-measured-it-4k7f</guid>
      <description>&lt;p&gt;I'm building &lt;a href="https://waydesk.dev" rel="noopener noreferrer"&gt;Waydesk&lt;/a&gt;, a free, open-source remote desktop service for Debian and Ubuntu. The goal is narrow: &lt;strong&gt;a light, responsive RDP service where every account on a server gets its own desktop, built on Sway and Wayland.&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Standard RDP,&lt;/strong&gt; so people connect with the client they already have (Windows App, mstsc, Remmina, FreeRDP). Nothing to install on their side.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One complete desktop per account:&lt;/strong&gt; a headless Sway session with a bar, dock, launcher, file manager, terminal and browser. It is assembled from Sway and a Qt shell rather than shipped as one GNOME or KDE package. Sessions persist when you disconnect.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Aimed at shared servers:&lt;/strong&gt; labs, courses, support desks, small teams, and service providers who run Linux servers for others, anywhere a ready desktop per person on a server you already run is easier than a laptop each.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No GPU required,&lt;/strong&gt; but one is used when there is one.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The usual alternatives are a full desktop environment (GNOME, KDE, Xfce) behind xrdp, or VNC. "Light" and "responsive" are claims, so I wanted numbers behind them. The question that decides whether this is practical: &lt;strong&gt;how many people fit on one server?&lt;/strong&gt; And the one I most wanted to answer: &lt;strong&gt;does a GPU change that answer?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I tested both on rented servers by adding simulated users one at a time until the server refused one. Short version: memory decides how many fit, and the GPU did something different from what I expected. I did not compare against other remote-desktop stacks, so these numbers show what Waydesk does, not that it beats anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  First, what limits you: memory
&lt;/h2&gt;

&lt;p&gt;Every logged-in user keeps a session alive, with their programs open, whether or not they're looking at the screen. So the number that counts is how many accounts are logged in, not how many are typing.&lt;/p&gt;

&lt;p&gt;Per user, measured:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Kind of user&lt;/th&gt;
&lt;th&gt;Memory each&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Desktop only&lt;/td&gt;
&lt;td&gt;about 0.4 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Plus Chromium with three pages&lt;/td&gt;
&lt;td&gt;about 0.7 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Heavy: Chromium, a 300,000-row spreadsheet, VS Code&lt;/td&gt;
&lt;td&gt;about 1.6 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;And how many fit, without a GPU:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Server&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;4 GB&lt;/td&gt;
&lt;td&gt;5 users with a browser&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;8 GB&lt;/td&gt;
&lt;td&gt;9 users with a browser, or 8 heavy users&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;23 GB, 12 cores&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;12 heavy users; the 13th was refused&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Without protection the 4 GB server crashed at five users: the kernel's out-of-memory killer shut down parts of people's desktops. Two things fixed it: a new login is turned away when too little memory is free (someone already logged in is never kicked out for a newcomer), and compressed swap (zram) squeezes idle memory about 4.5 to 1. On the 8 GB server that took heavy users from 6 to 8. The details are in &lt;a href="https://github.com/zhoorta/waydesk/blob/main/docs/MEASUREMENTS.md" rel="noopener noreferrer"&gt;the measurements&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;In these runs the programs were opened and then left idle, so they are best-case numbers.&lt;/p&gt;

&lt;h2&gt;
  
  
  The GPU test
&lt;/h2&gt;

&lt;p&gt;For this I used a dedicated Intel i7-7700 (4 cores, 8 threads) with its integrated HD 630 GPU and 64 GB of memory. The users were &lt;strong&gt;active&lt;/strong&gt;, not idle. Each one had:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Chromium on a page that scrolls by itself,&lt;/li&gt;
&lt;li&gt;a terminal typing and printing output,&lt;/li&gt;
&lt;li&gt;LibreOffice Calc with a 300,000-row sheet and VS Code, both receiving keystrokes,&lt;/li&gt;
&lt;li&gt;window switches every 15 to 30 seconds,&lt;/li&gt;
&lt;li&gt;a muted looping video for every fourth user, GIMP on a large image for every fourth, and sound for three.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The RDP clients ran on a separate cloud server in the same data centre, so decoding 20 video streams wouldn't eat the CPU I was measuring.&lt;/p&gt;

&lt;p&gt;I ran the ramp twice: GPU on (sway rendering on the GPU, VAAPI hardware H.264 encoding, zero-copy capture) and GPU off (software rendering, OpenH264 on the CPU).&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Active heavy users&lt;/th&gt;
&lt;th&gt;CPU, GPU on&lt;/th&gt;
&lt;th&gt;CPU, GPU off&lt;/th&gt;
&lt;th&gt;free memory, GPU on&lt;/th&gt;
&lt;th&gt;free memory, GPU off&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;23 %&lt;/td&gt;
&lt;td&gt;21 %&lt;/td&gt;
&lt;td&gt;53.4 GB&lt;/td&gt;
&lt;td&gt;53.6 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;47 %&lt;/td&gt;
&lt;td&gt;45 %&lt;/td&gt;
&lt;td&gt;32.7 GB&lt;/td&gt;
&lt;td&gt;35.8 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;20&lt;/td&gt;
&lt;td&gt;50 %&lt;/td&gt;
&lt;td&gt;69 %&lt;/td&gt;
&lt;td&gt;10.7 GB&lt;/td&gt;
&lt;td&gt;16.6 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;24&lt;/td&gt;
&lt;td&gt;(22nd refused)&lt;/td&gt;
&lt;td&gt;94 %&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;8.5 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;GPU on: 21 users fit, the 22nd was refused. GPU off: 24 fit, the 25th was refused.&lt;/strong&gt; Memory ended both runs.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fuocj4by2w3dx5moenb52.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fuocj4by2w3dx5moenb52.png" alt="Line chart of CPU use against active users. With the GPU on, CPU stays near 50 percent from 10 to 20 users. With the GPU off it climbs from about 45 to 94 percent at 24 users." width="800" height="480"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;So the GPU did not let more people in, but it changed two other things:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;It took the load off the CPU.&lt;/strong&gt; With the GPU, CPU use stayed near 50 % from 15 users on. Without it, it climbed 3 to 4 points per user. The capture-and-encode processes used 4.8 % of the machine with the GPU at 20 users, against 15.4 % without (about 70 % less). On a machine with more memory, that headroom should fit more users. I estimated that; I did not measure it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It used more memory.&lt;/strong&gt; 2.24 GB per user against 1.95 GB, because an integrated GPU takes its buffers from system RAM. A card with its own video memory wouldn't do this, but I haven't tested one under many users.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Nothing failed in either run: no errors, no dropped frames, no out-of-memory kills.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the GPU clearly matters: graphics-heavy pages
&lt;/h2&gt;

&lt;p&gt;I also ran a full-screen WebGL noise page for each user, which is a deliberately brutal case:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Users&lt;/th&gt;
&lt;th&gt;Frame rate per page (median)&lt;/th&gt;
&lt;th&gt;GPU render engine&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;60&lt;/td&gt;
&lt;td&gt;53 %&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;60&lt;/td&gt;
&lt;td&gt;93 %&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;47&lt;/td&gt;
&lt;td&gt;100 %&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;20.5&lt;/td&gt;
&lt;td&gt;100 %&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;td&gt;17&lt;/td&gt;
&lt;td&gt;100 %&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;An HD 630 gives two users 60 fps, and then its render engine is the limit: the frames get shared about evenly. Lighter pages share among more people.&lt;/p&gt;

&lt;p&gt;And &lt;strong&gt;without a GPU the page didn't animate at all&lt;/strong&gt; in the browser. As Waydesk starts Chromium, it uses &lt;code&gt;--disable-gpu&lt;/code&gt; when there is no usable render node. If your users need WebGL, a GPU isn't an optimisation but a requirement.&lt;/p&gt;

&lt;h2&gt;
  
  
  What happens when the GPU can't help
&lt;/h2&gt;

&lt;p&gt;Two cases I checked, because they affect real deployments:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;GPU present but no hardware encoder.&lt;/strong&gt; The session still works; the H.264 encoding falls back to OpenH264 on the CPU, and the log says so. One user on the WebGL page went from 2.9 % of the machine (VAAPI) to 12.3 % (CPU encoder). This is what an NVIDIA card looks like to VAAPI, so I wrote an NVENC encoder as well: on an RTX 4000 SFF Ada, one user ran at 61 fps with the card's encoder 4 % busy. That was one user on one card; several users on one card are not measured. It also needed NVIDIA's own 615 driver, since Debian's 550 can't give a headless sway its buffers.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Render node present but the GPU switched off.&lt;/strong&gt; An OpenGL app (&lt;code&gt;glxgears&lt;/code&gt;) falls back to the CPU and opens; a Vulkan one (&lt;code&gt;vkcube&lt;/code&gt;) fails to present its window. On a server with no render node at all, &lt;code&gt;vkcube&lt;/code&gt; opened on Mesa's software drivers. Odd, but worth knowing before you debug it.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What I'd tell someone sizing a server
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Memory first.&lt;/strong&gt; About 0.4 to 0.5 GB per session for the desktop, plus what the programs use, plus about 1 GB for the system. Heavy users cost 1.6 to 1.85 GB.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Add zram and size it to RAM,&lt;/strong&gt; and refuse new sessions when the machine is full instead of degrading everyone.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A GPU buys CPU headroom, not more users,&lt;/strong&gt; unless you have the memory to use it. Graphics-heavy work (WebGL, video, 3D) is where it becomes necessary.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Check your own hardware.&lt;/strong&gt; My results were one user off between a VM and a real server.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What I didn't measure
&lt;/h2&gt;

&lt;p&gt;How fast individual applications feel with and without the GPU (apart from WebGL), the delay between a click and its picture, a discrete card under many users, AMD, long runs (each ramp took about 45 minutes), and far-away clients on slow links with many users.&lt;/p&gt;

&lt;p&gt;Waydesk is early and has had no independent review. If you want to try it, or compare your numbers with mine, the testing checklist is in &lt;a href="https://github.com/zhoorta/waydesk/issues/1" rel="noopener noreferrer"&gt;issue #1&lt;/a&gt;. The full method, tables and raw data are in &lt;a href="https://github.com/zhoorta/waydesk/blob/main/docs/MEASUREMENTS.md" rel="noopener noreferrer"&gt;&lt;code&gt;docs/MEASUREMENTS.md&lt;/code&gt;&lt;/a&gt; and &lt;a href="https://github.com/zhoorta/waydesk/tree/main/docs/measure-data" rel="noopener noreferrer"&gt;&lt;code&gt;docs/measure-data/&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>opensource</category>
      <category>performance</category>
    </item>
    <item>
      <title>Multi Shelter Manager: an open-source, multi-tenant app for animal shelters, with Laravel 13, Livewire and a one-click demo</title>
      <dc:creator>Ze Horta</dc:creator>
      <pubDate>Sat, 24 Oct 2026 17:03:28 +0000</pubDate>
      <link>https://dev.to/zhoorta/multi-shelter-manager-an-open-source-multi-tenant-app-for-animal-shelters-with-laravel-13-1jbp</link>
      <guid>https://dev.to/zhoorta/multi-shelter-manager-an-open-source-multi-tenant-app-for-animal-shelters-with-laravel-13-1jbp</guid>
      <description>&lt;p&gt;Small animal shelters usually run on spreadsheets, paper cards and WhatsApp groups. Vaccination dates get lost, nobody knows which cage a dog is in, and adoption requests arrive as Instagram DMs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Multi Shelter Manager&lt;/strong&gt; is a free, MIT-licensed web app that replaces all of that. Several shelters share one installation, each fully isolated, and it already runs three real shelters, from the Azores to Lisbon and Porto. It's made in Portugal with Laravel 13 and Livewire.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;👉 Try it now: &lt;a href="https://demo.focinhos.pt" rel="noopener noreferrer"&gt;https://demo.focinhos.pt&lt;/a&gt;&lt;/strong&gt; (one click, no sign-up, you get your own sandbox shelter)&lt;/p&gt;

&lt;p&gt;Code: &lt;a href="https://github.com/zhoorta/multi-shelter-manager" rel="noopener noreferrer"&gt;https://github.com/zhoorta/multi-shelter-manager&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1alsdw5nfoz0w9496jvg.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1alsdw5nfoz0w9496jvg.png" alt="Dashboard" width="800" height="1518"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Pets&lt;/strong&gt;: identification, microchip, health, photos, and a status (available / adopted / ...) worked out automatically.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Facilities → wings → cages&lt;/strong&gt;: model the physical space, see occupancy, and always know where every animal is.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Vaccinations and treatments&lt;/strong&gt; with due dates and daily e-mail reminders.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Adoptions, sponsorships, volunteers, members and their fees.&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Foster families&lt;/strong&gt; and &lt;strong&gt;cat colonies&lt;/strong&gt; (CED/CER programmes).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Reports&lt;/strong&gt; for managers, with printable yearly activity report.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Public adoption portal&lt;/strong&gt; with filters and an application form.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;10 languages&lt;/strong&gt; and light/dark mode.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The interesting parts, technically
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Multi-tenancy by &lt;code&gt;shelter_id&lt;/code&gt;.&lt;/strong&gt; One database, one install, any number of shelters. Every shelter-owned table carries a &lt;code&gt;shelter_id&lt;/code&gt;, and a trait scopes queries so a shelter can never see another's data. Users can belong to several shelters with a different role in each (admin, manager, staff, viewer).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Livewire + Flux UI + Tailwind.&lt;/strong&gt; Classic split components (class plus Blade view), no JavaScript framework. Tables, modals and forms come from Flux, so the UI stays consistent with very little custom CSS.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Auditing for free.&lt;/strong&gt; A &lt;code&gt;Blameable&lt;/code&gt; trait fills &lt;code&gt;created_by&lt;/code&gt;, &lt;code&gt;updated_by&lt;/code&gt; and &lt;code&gt;deleted_by&lt;/code&gt; on the core models, which matters when volunteers and staff share an account of what happened to an animal.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Ten languages from day one.&lt;/strong&gt; All UI strings go through &lt;code&gt;__()&lt;/code&gt; with a JSON file per locale. Retrofitting this later is painful; doing it from the start was cheap.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A public demo that costs almost nothing.&lt;/strong&gt; People should be able to try it without installing anything, but a shared demo database turns into a mess in minutes. So in &lt;code&gt;DEMO_MODE&lt;/code&gt; every visitor gets a throwaway shelter cloned from a template, and a &lt;code&gt;demo:prune&lt;/code&gt; command (every 10 minutes) deletes old ones. It runs in a single FrankenPHP container with SQLite on a RAM-backed volume, behind nginx on a small server. A restart wipes everything and re-seeds in about a minute. The one rule to remember: every new shelter-owned table must be added to the sandbox's table list, or it won't be copied or cleaned up.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why open source
&lt;/h2&gt;

&lt;p&gt;Shelters have no budget for software, and each country has a handful of tools that are either paid per animal or abandoned. This is a cause project, not a business: if it helps another shelter, great. If you run one, or know someone who does, the README has a quick start, and the docs folder has a step-by-step guide with screenshots.&lt;/p&gt;

&lt;h2&gt;
  
  
  Feedback welcome
&lt;/h2&gt;

&lt;p&gt;Feedback is very welcome, from other Laravel developers (what would you do differently in the tenancy layer?), and from anyone who works with shelters (what's missing?). Issues and PRs are open.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Demo: &lt;a href="https://demo.focinhos.pt" rel="noopener noreferrer"&gt;https://demo.focinhos.pt&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Repo: &lt;a href="https://github.com/zhoorta/multi-shelter-manager" rel="noopener noreferrer"&gt;https://github.com/zhoorta/multi-shelter-manager&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>laravel</category>
      <category>livewire</category>
      <category>opensource</category>
      <category>nonprofit</category>
    </item>
    <item>
      <title>Full-text search in Laravel on both SQLite and MySQL: FTS5, FULLTEXT and one interface</title>
      <dc:creator>Ze Horta</dc:creator>
      <pubDate>Mon, 19 Oct 2026 16:48:22 +0000</pubDate>
      <link>https://dev.to/zhoorta/full-text-search-in-laravel-on-both-sqlite-and-mysql-fts5-fulltext-and-one-interface-f9f</link>
      <guid>https://dev.to/zhoorta/full-text-search-in-laravel-on-both-sqlite-and-mysql-fts5-fulltext-and-one-interface-f9f</guid>
      <description>&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; you can add real search inside files to a Laravel app without Meilisearch or Elasticsearch. Keep the extracted text in one table, give each database its own small driver (FTS5 on SQLite, a FULLTEXT index on MySQL and MariaDB), and join the results to your normal tables so permissions stay in one place.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://github.com/zhoorta/ferrite" rel="noopener noreferrer"&gt;Ferrite&lt;/a&gt;, a self-hosted file server. It runs on SQLite by default and on MySQL or MariaDB if you prefer. People wanted to find a file by a word inside it, not only by its name, and I did not want the install to grow another service. This is how I did it, including the places where the two databases differ enough to bite.&lt;/p&gt;

&lt;p&gt;(If you can run a search service, Laravel Scout with Meilisearch or Typesense is a perfectly good answer. This is for when you can't, or won't.)&lt;/p&gt;

&lt;h2&gt;
  
  
  One table, keyed on the content
&lt;/h2&gt;

&lt;p&gt;Extracted text goes into a &lt;code&gt;node_contents&lt;/code&gt; table, one row per &lt;em&gt;distinct content&lt;/em&gt; (the file's SHA-256), not per file. Two identical files are read once. A queued job on its own &lt;code&gt;search&lt;/code&gt; queue reads the file after it is stored: text files directly (the first 2 MB, valid UTF-8 only) and PDFs through &lt;code&gt;pdftotext&lt;/code&gt;. A row has a status (&lt;code&gt;indexed&lt;/code&gt;, &lt;code&gt;skipped&lt;/code&gt; with a reason, or &lt;code&gt;failed&lt;/code&gt;), so a binary file or a broken PDF is not retried forever.&lt;/p&gt;

&lt;p&gt;The index is built on top of that table, and that is the part that differs per database.&lt;/p&gt;

&lt;h2&gt;
  
  
  One interface, two drivers
&lt;/h2&gt;

&lt;p&gt;The search code does not know which database it is on. It asks for a driver:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="kd"&gt;interface&lt;/span&gt; &lt;span class="nc"&gt;ContentSearchDriver&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// Narrow $nodes to the files whose text matches, best first, with a `snippet` column.&lt;/span&gt;
    &lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="k"&gt;function&lt;/span&gt; &lt;span class="n"&gt;match&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;Builder&lt;/span&gt; &lt;span class="nv"&gt;$nodes&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="nv"&gt;$term&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt; &lt;span class="kt"&gt;?Builder&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="c1"&gt;// The snippet with the matched words marked.&lt;/span&gt;
    &lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="k"&gt;function&lt;/span&gt; &lt;span class="n"&gt;present&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="nv"&gt;$snippet&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="nv"&gt;$term&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;match&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;DB&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;connection&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;getDriverName&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s1"&gt;'sqlite'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;SqliteContentSearch&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'mysql'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'mariadb'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;MysqlContentSearch&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="k"&gt;default&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;   &lt;span class="c1"&gt;// other databases keep the table but have no content search&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;match()&lt;/code&gt; receives a query that is already limited to what the user may see, and adds the join, the full-text condition, the ranking and the snippet. More on that in a moment.&lt;/p&gt;

&lt;h2&gt;
  
  
  SQLite: an FTS5 table kept in step by triggers
&lt;/h2&gt;

&lt;p&gt;FTS5 is built into SQLite. I use an &lt;em&gt;external content&lt;/em&gt; table, so the text is stored once (in &lt;code&gt;node_contents&lt;/code&gt;) and the FTS table only holds the index. Three triggers keep it in step:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="no"&gt;DB&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;unprepared&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"create virtual table node_contents_fts using fts5(
    text, content='node_contents', content_rowid='id',
    tokenize='unicode61 remove_diacritics 2')"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="no"&gt;DB&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;unprepared&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"create trigger node_contents_ai after insert on node_contents begin
    insert into node_contents_fts(rowid, text) values (new.id, coalesce(new.text, ''));
end"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="c1"&gt;// plus an 'after delete' and an 'after update' trigger that remove the old entry first&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;remove_diacritics 2&lt;/code&gt; makes &lt;code&gt;reuniao&lt;/code&gt; find &lt;code&gt;reunião&lt;/code&gt;, which matters if your users are not English-speaking. The query itself is short, with &lt;code&gt;bm25&lt;/code&gt; for ranking and &lt;code&gt;snippet()&lt;/code&gt; for the excerpt:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nv"&gt;$nodes&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nb"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'node_contents as nc'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'nc.sha256'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'='&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'nodes.sha256'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nb"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'node_contents_fts'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'node_contents_fts.rowid'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'='&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'nc.id'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;whereRaw&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'node_contents_fts match ?'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;$query&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;select&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'nodes.*'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;selectRaw&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"snippet(node_contents_fts, 0, char(1), char(2), '…', 18) as snippet"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;orderByRaw&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'bm25(node_contents_fts)'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I mark the matched words with two control characters (&lt;code&gt;char(1)&lt;/code&gt; and &lt;code&gt;char(2)&lt;/code&gt;), so the view can escape the whole snippet as HTML and only then replace the markers with &lt;code&gt;&amp;lt;mark&amp;gt;&lt;/code&gt;. That keeps the snippet safe even when the file's text contains &lt;code&gt;&amp;lt;script&amp;gt;&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  MySQL and MariaDB: a FULLTEXT index
&lt;/h2&gt;

&lt;p&gt;On MySQL the migration is one line: &lt;code&gt;$table-&amp;gt;fullText('text')&lt;/code&gt;, on a &lt;code&gt;LONGTEXT&lt;/code&gt; column. The query uses boolean mode:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nv"&gt;$nodes&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nb"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'node_contents as nc'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'nc.sha256'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'='&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'nodes.sha256'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;whereRaw&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'match(nc.text) against (? in boolean mode)'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;$query&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;select&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'nodes.*'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;selectRaw&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'substring(nc.text, greatest(locate(?, nc.text) - 60, 1), 240) as snippet'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;$words&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]])&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;selectRaw&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'match(nc.text) against (? in boolean mode) as content_rank'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;$query&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;orderByDesc&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'content_rank'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Three differences from SQLite:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;No snippet function.&lt;/strong&gt; I cut a 240-character window around the first word in SQL and mark the words in PHP afterwards, with the same control characters, so the view code is identical.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Short words and stopwords.&lt;/strong&gt; InnoDB ignores words shorter than three characters and a built-in list of stopwords. If you make such a word &lt;em&gt;required&lt;/em&gt; (&lt;code&gt;+the*&lt;/code&gt;) the search can come back empty. I don't rely on the behaviour: I drop those words from the query, and if nothing is left I return no content matches.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Ranking differs.&lt;/strong&gt; &lt;code&gt;bm25&lt;/code&gt; on SQLite, InnoDB's relevance score on MySQL. The order of results is not the same, and I documented that instead of pretending.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Never let the user's text reach the search syntax
&lt;/h2&gt;

&lt;p&gt;Both engines have a query language (quotes, &lt;code&gt;OR&lt;/code&gt;, &lt;code&gt;NEAR(&lt;/code&gt;, &lt;code&gt;-&lt;/code&gt;, &lt;code&gt;*&lt;/code&gt;, &lt;code&gt;:&lt;/code&gt; in FTS5). If you pass the raw input through, &lt;code&gt;plain OR (&lt;/code&gt; becomes a syntax error, or worse, a way to run operators you did not intend. So the input is reduced first to letters and digits:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="nb"&gt;preg_match_all&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'/[\p{L}\p{N}]+/u'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;mb_strtolower&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$term&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="nv"&gt;$found&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each word then becomes &lt;code&gt;"word"*&lt;/code&gt; on SQLite or &lt;code&gt;+word*&lt;/code&gt; on MySQL: every word required, each matched as a prefix. The tests throw odd inputs at it (&lt;code&gt;"plain&lt;/code&gt;, &lt;code&gt;col:plain&lt;/code&gt;, &lt;code&gt;NEAR(plain words)&lt;/code&gt;, &lt;code&gt;*&lt;/code&gt;, a lone quote) and assert that nothing throws.&lt;/p&gt;

&lt;h2&gt;
  
  
  Permissions stay where they were
&lt;/h2&gt;

&lt;p&gt;The index never decides who sees what. Search starts from the same visibility query as the file list (your own files, plus everything below folders shared with you, trashed items excluded) and then &lt;code&gt;match()&lt;/code&gt; narrows it. A user only ever gets snippets of nodes they could already open, even though identical content is stored once across users.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="nv"&gt;$query&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nv"&gt;$driver&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="k"&gt;match&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="nv"&gt;$this&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;visibleTo&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;Node&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;query&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="nv"&gt;$user&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;where&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'nodes.type'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;NodeType&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nc"&gt;File&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;whereNull&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'nodes.trashed_at'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="nv"&gt;$term&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Testing on both databases
&lt;/h2&gt;

&lt;p&gt;The search tests are one Pest file. By default it runs against in-memory SQLite. To run the same file on MySQL or MariaDB you point the suite at an empty test database, and CI does that with a MySQL service container.&lt;/p&gt;

&lt;p&gt;One trap: &lt;strong&gt;an InnoDB FULLTEXT index only sees committed rows&lt;/strong&gt;, and Laravel's &lt;code&gt;RefreshDatabase&lt;/code&gt; wraps each test in a transaction. On MySQL every search found nothing. The fix, in the test file only: commit at the start of each test and empty the tables afterwards.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="c1"&gt;// An InnoDB FULLTEXT index only sees committed rows, so on MySQL these tests commit&lt;/span&gt;
&lt;span class="c1"&gt;// the transaction the suite wraps them in and empty the tables afterwards instead.&lt;/span&gt;
&lt;span class="nv"&gt;$this&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;commits&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;in_array&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;DB&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;connection&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;getDriverName&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'mysql'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'mariadb'&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$this&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;commits&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="no"&gt;DB&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;commit&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I ran the file on SQLite, MySQL 8.4 and 9, and MariaDB 11.&lt;/p&gt;

&lt;h2&gt;
  
  
  Gotchas to know about
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Accents on MySQL depend on the collation.&lt;/strong&gt; With &lt;code&gt;utf8mb4_unicode_ci&lt;/code&gt; (Laravel's default) accents and case are ignored. With a &lt;code&gt;_bin&lt;/code&gt; or &lt;code&gt;_as_cs&lt;/code&gt; collation they are not.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The queue worker has to listen to the new queue.&lt;/strong&gt; Mine is &lt;code&gt;queue:work --queue=default,search&lt;/code&gt;. Without it the extraction jobs are queued and never run. With the &lt;code&gt;sync&lt;/code&gt; queue driver, PDFs are better left to a scheduled command so a slow PDF can't hit a time limit inside a request.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;PDFs need &lt;code&gt;pdftotext&lt;/code&gt;&lt;/strong&gt; (poppler-utils). Scanned documents have no text layer, so there is nothing to index. I did not add OCR.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The index holds the file's text unencrypted&lt;/strong&gt; in your database. If that matters, make indexing switchable. Mine is one environment variable.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What I would do differently
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;If you already run Meilisearch or Typesense, use them through Scout. Typo tolerance and facets are something this setup does not give you.&lt;/li&gt;
&lt;li&gt;A Postgres driver would be a third class behind the same interface (&lt;code&gt;tsvector&lt;/code&gt;), and the structure makes it easy to add. I did not need it.&lt;/li&gt;
&lt;li&gt;I would test the very large case (millions of rows) before promising anything. This is built for a person or a small team.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The code is open source (AGPL-3.0). If you want to read it rather than my summary:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The drivers: &lt;a href="https://github.com/zhoorta/ferrite/blob/main/app/Support/Search/SqliteContentSearch.php" rel="noopener noreferrer"&gt;&lt;code&gt;SqliteContentSearch&lt;/code&gt;&lt;/a&gt; and &lt;a href="https://github.com/zhoorta/ferrite/blob/main/app/Support/Search/MysqlContentSearch.php" rel="noopener noreferrer"&gt;&lt;code&gt;MysqlContentSearch&lt;/code&gt;&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;The design notes: &lt;a href="https://github.com/zhoorta/ferrite/blob/main/docs/content-search.md" rel="noopener noreferrer"&gt;&lt;code&gt;docs/content-search.md&lt;/code&gt;&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;The project: &lt;a href="https://github.com/zhoorta/ferrite" rel="noopener noreferrer"&gt;github.com/zhoorta/ferrite&lt;/a&gt;, with a &lt;a href="https://demo.ferrite.stackcare.pt" rel="noopener noreferrer"&gt;live demo&lt;/a&gt; (throwaway account, wiped after an hour) where you can upload a text file or PDF and search for a word in it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Questions or corrections are welcome in the comments.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Disclosure: I built Ferrite and I use an AI coding assistant (Claude) while developing it; I review and test what it writes. An AI assistant also helped me draft this article, and I edited it and checked the claims against the code. I chose the "AI-assisted" option when publishing.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>laravel</category>
      <category>php</category>
      <category>sqlite</category>
      <category>mysql</category>
    </item>
    <item>
      <title>Ferrite: self-hosted file storage in Laravel, Livewire and Flux UI (open source, with a live demo)</title>
      <dc:creator>Ze Horta</dc:creator>
      <pubDate>Tue, 13 Oct 2026 15:15:22 +0000</pubDate>
      <link>https://dev.to/zhoorta/ferrite-self-hosted-file-storage-in-laravel-livewire-and-flux-ui-open-source-with-a-live-demo-8ld</link>
      <guid>https://dev.to/zhoorta/ferrite-self-hosted-file-storage-in-laravel-livewire-and-flux-ui-open-source-with-a-live-demo-8ld</guid>
      <description>&lt;p&gt;I've built a small self-hosted file server called &lt;strong&gt;Ferrite&lt;/strong&gt;. It is web-only file storage for one person or a small team that you run on your own server, and it is open source (AGPL-3.0). You can try it without installing anything: &lt;strong&gt;&lt;a href="https://demo.ferrite.stackcare.pt" rel="noopener noreferrer"&gt;demo.ferrite.stackcare.pt&lt;/a&gt;&lt;/strong&gt; (one click, a throwaway account that is wiped after an hour; small images, PDFs and text only).&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fzhoorta%2Fferrite%2Fmain%2Fdocs%2Fimg%2Fdemo.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fzhoorta%2Fferrite%2Fmain%2Fdocs%2Fimg%2Fdemo.gif" alt="A short tour of Ferrite: upload, search inside files, replace a file, switch themes" width="719" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Why I built it
&lt;/h2&gt;

&lt;p&gt;I come from a sysadmin background, and I wanted private file storage that is boring to run: one app, a database, a folder, and nothing else to keep alive. Most of what I found was either a full cloud suite or a sync tool, and I wanted something smaller. I also wanted to see how far a small, opinionated Laravel app could go when it only does one job.&lt;/p&gt;

&lt;p&gt;It is deliberately narrow: no sync clients and no WebDAV. You open it in a browser, upload, preview, share and search.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Uploads that survive bad connections.&lt;/strong&gt; Chunked and resumable, so multi-gigabyte files and flaky Wi-Fi are fine. Whole folders by drag and drop.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Previews and sharing.&lt;/strong&gt; Images, PDF, video, audio and text open in the browser. Share by link with a password or an expiry, with other users as viewer or editor, or give a guest an upload-only link to one folder.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Search inside your files.&lt;/strong&gt; Names, and the text of text files, Markdown, code, CSV and PDFs, with highlighted snippets.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A small API for scripts and other apps&lt;/strong&gt;: personal access tokens made under Settings &amp;gt; API tokens, optionally limited to one folder, with list, Range download and chunked upload. Write tokens never replace or rename; a taken name answers 409.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;When a name already exists,&lt;/strong&gt; it asks: replace (same file, its links and favorites stay), keep both, skip, or cancel.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Users and storage.&lt;/strong&gt; Several users with quotas, 2FA and passkeys; files on a local folder, S3 or SFTP.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fifteen themes,&lt;/strong&gt; because I like retro interfaces: Classic Mac, Amiga, a Nokia 3310 screen, Miami Vice and more. They are optional, and you can switch them in the demo under Settings &amp;gt; Appearance.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  How it is built
&lt;/h2&gt;

&lt;p&gt;Laravel 13, Livewire, Flux UI, Tailwind and Pest, with about 550 tests. A few decisions I found interesting:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The slow part never blocks a request.&lt;/strong&gt; The last upload chunk only queues a job. The job hashes the file and copies it to its disk, which can take minutes on SFTP or S3. It runs on its own queue connection with a &lt;code&gt;retry_after&lt;/code&gt; longer than the job's timeout, so a long copy is never started twice.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Search works on SQLite and MySQL.&lt;/strong&gt; It uses FTS5 on SQLite and a FULLTEXT index on MySQL and MariaDB, behind one small interface. The CI runs the search tests on both.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Serving users' files from your own origin needs care.&lt;/strong&gt; Files only display inline for an allowlist of safe types, everything else is a download, with &lt;code&gt;nosniff&lt;/code&gt; on every response. There is a written security review in the repository.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Names and folders live only in the database.&lt;/strong&gt; The files themselves are blobs under random keys, so renaming or moving something never touches the disk.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;No versioning, sync clients, WebDAV, office previews or real-time collaboration.&lt;/li&gt;
&lt;li&gt;S3 is tested up to the adapter, not against a real S3 service. SFTP was tested on a Hetzner Storage Box.&lt;/li&gt;
&lt;li&gt;No independent security audit yet, and no encryption at rest (it is planned).&lt;/li&gt;
&lt;li&gt;It is young: version 0.3.0.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What I'd like from you
&lt;/h2&gt;

&lt;p&gt;If you try the demo or read the install docs, tell me what felt confusing or broke. I'd especially like to hear where the install guide lost you, and whether the security notes in the repository (&lt;code&gt;docs/security.md&lt;/code&gt;) cover what you would worry about before putting your files on it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Demo:&lt;/strong&gt; &lt;a href="https://demo.ferrite.stackcare.pt" rel="noopener noreferrer"&gt;https://demo.ferrite.stackcare.pt&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Project page:&lt;/strong&gt; &lt;a href="https://ferrite.stackcare.pt" rel="noopener noreferrer"&gt;https://ferrite.stackcare.pt&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Code:&lt;/strong&gt; &lt;a href="https://github.com/zhoorta/ferrite" rel="noopener noreferrer"&gt;https://github.com/zhoorta/ferrite&lt;/a&gt; (AGPL-3.0)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Install:&lt;/strong&gt; &lt;code&gt;docker compose up -d&lt;/code&gt;, or a normal Laravel install. The README has both routes.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I built Ferrite and I use an AI coding assistant (Claude) while developing it; I review and test what it writes. An AI assistant also helped me draft this post, and I edited it and checked the claims against the code. I chose the "AI-assisted" option when publishing.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>showdev</category>
      <category>laravel</category>
      <category>selfhosted</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Resumable uploads in Laravel without a package: chunks, offsets and a queue job</title>
      <dc:creator>Ze Horta</dc:creator>
      <pubDate>Thu, 08 Oct 2026 16:18:26 +0000</pubDate>
      <link>https://dev.to/zhoorta/resumable-uploads-in-laravel-without-a-package-chunks-offsets-and-a-queue-job-4nc6</link>
      <guid>https://dev.to/zhoorta/resumable-uploads-in-laravel-without-a-package-chunks-offsets-and-a-queue-job-4nc6</guid>
      <description>&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; send a file in 5 MB chunks, let the &lt;em&gt;server's&lt;/em&gt; offset decide where to continue, and do the slow final copy in a queue job on its own queue connection, with a &lt;code&gt;retry_after&lt;/code&gt; longer than the job. That is the whole design, and it needs no package.&lt;/p&gt;

&lt;p&gt;Uploading a 2 GB file through a normal PHP form is a bad idea. A proxy times out, PHP's limits get in the way, and one dropped connection at 95% means starting again. When I built a self-hosted file server (Laravel, Livewire, Flux UI) I wanted uploads that survive all of that, including a final copy to a remote disk (SFTP, S3) that can take minutes.&lt;/p&gt;

&lt;p&gt;If you need interoperability with existing clients, the &lt;a href="https://tus.io" rel="noopener noreferrer"&gt;tus protocol&lt;/a&gt; is the standard and there are Laravel packages for it. What follows is a simpler, application-specific protocol that I could test end to end.&lt;/p&gt;

&lt;h2&gt;
  
  
  The protocol: four endpoints
&lt;/h2&gt;

&lt;p&gt;All uploads, big or small, use the same flow, so there is no separate "simple" path to maintain.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;code&gt;POST /uploads&lt;/code&gt; with the file name, size and a fingerprint. The server creates an &lt;code&gt;uploads&lt;/code&gt; row and answers with an &lt;code&gt;id&lt;/code&gt;, the current &lt;code&gt;offset&lt;/code&gt; (0) and the &lt;code&gt;chunk_size&lt;/code&gt;. Starting the same file again resumes the unfinished upload instead of creating a new one.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;PATCH /uploads/{id}&lt;/code&gt; with the raw chunk as the request body and an &lt;code&gt;Upload-Offset&lt;/code&gt; header.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;GET /uploads/{id}&lt;/code&gt; returns the state: &lt;code&gt;receiving&lt;/code&gt;, &lt;code&gt;processing&lt;/code&gt;, &lt;code&gt;done&lt;/code&gt; or &lt;code&gt;failed&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;DELETE /uploads/{id}&lt;/code&gt; cancels.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Chunks are appended to a temporary file named after the upload id. The default chunk size is 5 MB, so each request only has to fit your proxy's body limit, not the whole file. &lt;code&gt;upload_max_filesize&lt;/code&gt; stops mattering because the body is not a form upload, but keep &lt;code&gt;post_max_size&lt;/code&gt; and your proxy's body limit above the chunk size.&lt;/p&gt;

&lt;h2&gt;
  
  
  The client: ask the server where you are
&lt;/h2&gt;

&lt;p&gt;The browser slices the file and sends one chunk at a time. The key idea is that the client never trusts its own idea of progress. The server's offset is the truth.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;end&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;chunk_size&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;file&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;size&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;`/uploads/&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;method&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;PATCH&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;body&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;file&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;offset&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;end&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="na"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Content-Type&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;application/octet-stream&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Upload-Offset&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;offset&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;status&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="mi"&gt;409&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// Out of step: the server says where it really is, and we carry on from there.&lt;/span&gt;
    &lt;span class="nx"&gt;state&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;&lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;...(&lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt; &lt;span class="p"&gt;};&lt;/span&gt;
    &lt;span class="k"&gt;continue&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If a request fails, the client waits (1, 2, 4 seconds and so on, up to five retries), asks &lt;code&gt;GET /uploads/{id}&lt;/code&gt; for the real offset and continues from it. That one rule gives you resume after a network drop, after a tab reload (the fingerprint, here the file's last-modified time, finds the unfinished upload) and after a duplicated request.&lt;/p&gt;

&lt;h2&gt;
  
  
  The server: one writer at a time
&lt;/h2&gt;

&lt;p&gt;Two requests for the same upload must never write at the same time, and a chunk must only be accepted at the exact offset the server expects:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="nv"&gt;$lock&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Cache&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;lock&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"upload:&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nv"&gt;$upload&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;120&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt; &lt;span class="nv"&gt;$lock&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;response&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$this&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;state&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$upload&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="mi"&gt;409&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nv"&gt;$upload&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;refresh&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="nv"&gt;$header&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nv"&gt;$request&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nb"&gt;header&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'Upload-Offset'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nv"&gt;$offset&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;is_numeric&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$header&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;?&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nv"&gt;$header&lt;/span&gt; &lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="c1"&gt;// Wrong offset, or already handed over: tell the client where we are.&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt; &lt;span class="nv"&gt;$upload&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;isReceiving&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nv"&gt;$offset&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="nv"&gt;$upload&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;offset&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;response&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$this&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;state&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$upload&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="mi"&gt;409&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c1"&gt;// append the chunk to the temporary file at that offset ...&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;finally&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nv"&gt;$lock&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;release&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A &lt;code&gt;409&lt;/code&gt; carrying the real state is not an error to the client. It is how it finds its place. I also refuse a chunk that would go past the declared size (&lt;code&gt;413&lt;/code&gt;), and every chunk request checks that the upload belongs to the current user.&lt;/p&gt;

&lt;h2&gt;
  
  
  Finishing: do the slow part in a queue job
&lt;/h2&gt;

&lt;p&gt;When the last chunk arrives, the request does one thing: it marks the upload &lt;code&gt;processing&lt;/code&gt;, queues a job and answers &lt;code&gt;202&lt;/code&gt; at once. The job hashes the file (SHA-256), sniffs the MIME type from the content (the client's claim is ignored), copies it to its disk under a random key and creates the database record.&lt;/p&gt;

&lt;p&gt;Why not in the request? On a remote disk the final copy can take minutes, and a proxy will cut the request long before that. Meanwhile the browser shows "Storing the file…", starts sending the next file, and polls &lt;code&gt;GET /uploads/{id}&lt;/code&gt; (every second at first, slowing to five) until the state says &lt;code&gt;done&lt;/code&gt; or &lt;code&gt;failed&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The job has two unusual settings:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="kd"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;FinalizeUpload&lt;/span&gt; &lt;span class="kd"&gt;implements&lt;/span&gt; &lt;span class="nc"&gt;ShouldQueue&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="nv"&gt;$tries&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;          &lt;span class="c1"&gt;// a half-finished copy is cleaned up; the user retries&lt;/span&gt;
    &lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="nv"&gt;$timeout&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;21600&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;    &lt;span class="c1"&gt;// six hours&lt;/span&gt;
    &lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="kt"&gt;bool&lt;/span&gt; &lt;span class="nv"&gt;$failOnTimeout&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="k"&gt;function&lt;/span&gt; &lt;span class="n"&gt;__construct&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="nv"&gt;$uploadId&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nv"&gt;$this&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;onConnection&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'uploads'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The gotcha that cost me the most time: &lt;code&gt;retry_after&lt;/code&gt;
&lt;/h2&gt;

&lt;p&gt;A queue worker puts a job back on the queue if it has not finished within the connection's &lt;code&gt;retry_after&lt;/code&gt; seconds. The default is 90. A copy that legitimately takes an hour would be picked up and started a second time while the first is still running.&lt;/p&gt;

&lt;p&gt;The fix is a separate queue connection just for these jobs, with a &lt;code&gt;retry_after&lt;/code&gt; larger than the job's timeout, and a worker started on that connection:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="c1"&gt;// config/queue.php&lt;/span&gt;
&lt;span class="s1"&gt;'uploads'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="s1"&gt;'driver'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'database'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'table'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'jobs'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'queue'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'uploads'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'retry_after'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;21700&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;   &lt;span class="c1"&gt;// above the job's 21600 s timeout&lt;/span&gt;
&lt;span class="p"&gt;],&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;php artisan queue:work uploads &lt;span class="nt"&gt;--queue&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;uploads &lt;span class="nt"&gt;--tries&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;1 &lt;span class="nt"&gt;--timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The first &lt;code&gt;uploads&lt;/code&gt; is the connection name, not the queue name. A worker started without it would use the default connection and its 90-second window.&lt;/p&gt;

&lt;h2&gt;
  
  
  Other things that bit me
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Clean up on failure.&lt;/strong&gt; Note the destination key on the upload row before you write, so a crash halfway can still remove the partial file. A failed job deletes the temporary file and any file that no database record uses, and marks the upload &lt;code&gt;failed&lt;/code&gt; with a reason the browser can show next to a Retry button.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Quota.&lt;/strong&gt; I check it when the upload starts and again, inside a transaction that locks the user's row, when the file is stored. Parallel uploads can each pass the first check, and the second check keeps the books right.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Temporary space.&lt;/strong&gt; The temp folder needs room for the largest file times the uploads in flight, and the target disk needs the file again.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Deploys.&lt;/strong&gt; Workers keep the old code in memory, so restart them after every deploy. A scheduled prune command removes unfinished uploads older than a day and gives up on jobs that died.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Proxy limits.&lt;/strong&gt; The proxy's body limit and timeout only have to cover one chunk. Long timeouts are for downloads, not uploads.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What I would still do differently
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;If you need to resume across different clients and tools, use tus instead of a custom protocol.&lt;/li&gt;
&lt;li&gt;Hash on the client if you want to skip uploads of files you already have. I did not.&lt;/li&gt;
&lt;li&gt;I tested the remote path against one SFTP server (a Hetzner Storage Box) and a local OpenSSH server, not against a real S3 service, so treat that part as less proven.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The full implementation (controller, job, the browser store) is open source in the project where I built this: &lt;a href="https://github.com/zhoorta/ferrite" rel="noopener noreferrer"&gt;Ferrite&lt;/a&gt;. The upload design is written up in &lt;code&gt;docs/uploads.md&lt;/code&gt;.&lt;/p&gt;




&lt;p&gt;*Disclosure: I wrote the code. An AI assistant helped me draft and edit this article, and I checked every claim against the code.&lt;/p&gt;

</description>
      <category>laravel</category>
      <category>php</category>
      <category>livewire</category>
      <category>queues</category>
    </item>
  </channel>
</rss>
