DEV Community

shangyin
shangyin

Posted on

Building a Lightweight Knowledge Base for a Small Team: Migrating from Nextcloud to Cloudreve

#cloud #docker #performance #software

One server, two containers, 1.6 GB RAM—handling online preview and streaming for 100 GB of company assets.

Plus three gnarly bugs that took me the better part of a day to track down.

  1. Why We Ditched Nextcloud

Our company has about two dozen people. Over the years, we accumulated more than 100 GB of data on Nextcloud: product docs, project artifacts, training videos, and presales decks. The data is valuable, but the system became increasingly painful to use:

• Heavy. Nextcloud (PHP) + OnlyOffice Document Server routinely consumed over 4 GB of RAM, with OnlyOffice alone eating 2–3 GB.

• Wrong mental model. Nextcloud assumes "everyone gets their own cloud drive." What we needed was "the company has one shared knowledge base." The workaround? Everything lived in a single shared account; the other hundred-odd accounts were effectively dead weight.

• Feature bloat. Calendar, chat, tasks, mail, Kanban—none of it used.

Our actual needs were simple: browse directories, preview documents and videos online, stream videos, and download files. Paying 4 GB of RAM and a pile of unused modules for that was a bad trade.

  1. Choosing the Stack: Where the Lightweight Wins Come From

The final setup is Cloudreve v4 + fileview, running as two Docker containers.

Old Stack New Stack

  1. Core Nextcloud (PHP) Cloudreve v4 (Go single binary)
  2. Office Preview OnlyOffice Document Server fileview (Java)

RAM Usage ~4 GB ~1.6 GB

Core Process RAM ~800 MB (php-fpm multiprocess) ~170 MB

Cloudreve is written in Go. A single binary, ~145 MB idle, ~170 MB under load with 100 GB of data. It’s a Chinese project with a native UI, built-in directory management, share links, a video player (Artplayer), and WebDAV support.

A pleasant surprise: Cloudreve v4’s official image ships with ffmpeg, LibreOffice, and libvips (which explains the 1.3 GB image size). Video thumbnails and Office file thumbnails work out of the box—no extra installs.

A honest note on “lightweight”: High-fidelity online Office preview always requires a separate conversion service, no matter which file manager you use. Fileview takes ~1.25 GB, better than OnlyOffice’s 2–3 GB, but not zero cost. If you can live with “click to download, open locally,” the whole stack can run under 200 MB.

The architecture is straightforward:
├─→ :5212 Cloudreve Main entry: browsing, download, video playback, sharing
└─→ :8012 fileview Loaded only when opening Office files

No Nginx reverse proxy. For internal HTTP use, it’s just extra config and a failure point. Cloudreve’s built-in HTTP server is sufficient. We’ll add HTTPS when exposing it to the public internet—no architectural changes needed.

  1. Data Migration: Recon Before You Move

With 100+ GB to move, the biggest mistake is “start copying immediately.” We spent 20 minutes doing read-only reconnaissance, and it changed our plan.

What the Recon Revealed

Running du -sm */ on the Nextcloud data directory showed the distribution across 87 user directories:

117 GB   one shared account      ← 93% of total
3.2 GB   userA
1.5 GB   userB
1.2 GB   userC
...
37 MB    userD
37 MB    userE                  ← many accounts at exactly 37 MB
37 MB    ...
Enter fullscreen mode Exit fullscreen mode

Those 37 MB accounts? Default sample files (welcome PDF, sample images) created when Nextcloud provisions a new user. Many accounts were never actually used.

Drilling into the 117 GB shared account:

89 GB   files/              ← real files
26 GB   files_trashbin/     ← trash
1.8 GB  files_versions/     ← version history
Enter fullscreen mode Exit fullscreen mode

Trash and version history accounted for 28 GB—none of which needed migrating.

Another finding: several accounts that looked like they had a few GB of data only had tens of MB in files/; the rest was trash. Three accounts had zero files.

Conclusion: Of the apparent 126 GB, only 87 GB needed to move. Of 87 accounts, only 4 had real content.

We also tallied file types, which informed the preview strategy:

758  mp4      ← browser-playable, no transcoding needed
671  docx
326  pdf
237  pptx
199  xlsx
143  md
131  xmind    ← no previewer, download only
Enter fullscreen mode Exit fullscreen mode

Why You Can’t Just Copy Files

The naive approach: “just copy files into Cloudreve’s storage directory.” Doesn’t work.

Cloudreve stores filenames and directory structures in a database. On disk, files have randomized prefixes:

uploads/1/some-dir/1_MYEphK73_filename.docx
              ↑ user ID  ↑ random string
Enter fullscreen mode Exit fullscreen mode

Copying files directly leaves Cloudreve unaware of them. You must go through the upload interface.

Cloudreve v4 has a POST /api/v4/workflow/import endpoint that claims to scan existing directories and build indexes. I burned a lot of time here: src requires a relative path (discovered by trial and error), dst formatting never quite worked, and user_id expects a hashed ID, not a numeric one. The closest I got was a log showing Importing 303 physical files, but files landed in a malformed location—and each failure left “imported” markers in the DB, causing retries to be skipped. Orphaned records everywhere.

I abandoned this and fell back to the simplest, most verifiable approach: rsync to a staging directory + rclone upload via WebDAV. Slower, but every step is inspectable, retryable, and reversible.

1. rsync from old server to local staging (gigabit LAN, ~90–100 MB/s) 

rsync -a --info=progress2 --rsync-path="sudo rsync" \
  user@old-server:/path/to/nextcloud/data/shared-account/files/ /srv/staging/
Enter fullscreen mode Exit fullscreen mode

2. rclone upload via WebDAV (observed ~40–75 MB/s) 

rclone copy /srv/staging cr: \
  --transfers 16 --ignore-existing \
  --exclude ".attachments.*/**" --exclude "**/.DS_Store"
Enter fullscreen mode Exit fullscreen mode

Note: The WebDAV password must be created separately in Cloudreve—it’s not your login password.

Total time: rsync 15 min + upload 49 min. 3,199 files, zero failures.

Post-migration verification: directory-by-directory file counts matched (11 directories), and random MD5 checks on 10 files (including a 1.6 GB video) all matched. Only then did we delete the 88 GB staging directory.

  1. Three Gotchas

The real time sinks weren’t setup—they were these three issues.

Gotcha 1: Misdiagnosing IO Contention as a Performance Bottleneck

During upload testing, I observed 1 file every 12 seconds. At that rate, 3,000 files would take 10 hours.

My first instinct: “Cloudreve spawns LibreOffice/ffmpeg per file for thumbnails; CPU is saturated.” I was about to disable thumbnails.

Then I checked actual resource usage:

cloudreve: MEM 192MiB CPU 0.06%

CPU at 0.06%—the service was practically idle. Thumbnails weren’t the bottleneck.

The real cause: rsync was saturating disk IO at 100 MB/s, and the upload was fighting for the same disk. After rsync finished, I retested:

116 files / 161 MB, 4.3 seconds, ~38 MB/s

From 12 seconds/file to 27 files/second. No configuration changes—just no longer fighting for IO.

Lesson: Check resource usage before “optimizing.” If I’d disabled thumbnails, I’d have wasted effort, concluded “that didn’t help,” and gone down a rabbit hole of false leads.

Gotcha 2: “File Encrypted, Enter Password”—Except It Wasn’t

This one was fun. After wiring up Office preview, every Word/Excel/PPT file opened with: “File is encrypted, please enter password.” These files opened fine locally and had never been password-protected.

Checking fileview logs:

📊 File size: 91 bytes
✅ File download success - LocalPath: /opt/fileview/data/downloads/xxx.docx
⚠️ Error: Invalid header signature; read 0x3A2265646F63227B,
         expected 0xE11AB1A1E011CFD0
🔒 Office document requires password
Enter fullscreen mode Exit fullscreen mode

91-byte docx? That header 0x3A2265646F63227B decodes to {"code":.

It wasn’t a docx—it was JSON. Catting those 91 bytes:
{"code":40071,"msg":"sign expired"}

Sign expired. But I could manually download the same URL minutes later without issue. Comparing the browser’s request to my manual one:

Mine:    sign=U14xJjCuRZ2PBwObURMH0Hk3qyn2k14LTO2Ba1gTcLY%3D%3A1786345482
Browser: sign=ZJYidJHcBN4qVby-0Y4K6-yx60DlQbo2KFXDawt1aCw
                                                    ↑ stops here
Enter fullscreen mode Exit fullscreen mode

The signature itself matched, but the trailing %3D%3A was missing (%3D%3A decodes to =:).

Digging into fileview’s frontend bundle, I found the URL parser:

const s = n.split("&").map(l => {
    const [c, d] = l.split("=");   // ← here
    ...
    return "".concat(v, "=").concat(p);
});
Enter fullscreen mode Exit fullscreen mode

sign==: split on = produces three parts: ["sign", "", ":"]. Destructuring [c, d] only captures the first two; the third is permanently lost. Reassembly only reconstructs c=d.

The full causal chain:

Signature truncated by frontend
→ Cloudreve receives a broken signature, returns 91-byte "sign expired" JSON
→ fileview treats the JSON as a docx
→ detects it’s neither ZIP (docx is a zip) nor OLE2 (legacy doc)
→ code says "conservatively assume encrypted"
→ frontend shows "file encrypted, enter password"

A URL parsing bug, manifesting as “file encryption.” Highly misleading—without logs, you’d chase file encoding, permissions, or corruption.

The fix is simple: split only on the first =:

const _i = l.indexOf("=");
const c = l.slice(0, _i), d = l.slice(_i + 1);
Enter fullscreen mode Exit fullscreen mode

Since this patches a bundled JS file inside a container, persistence matters. Mount the patched file as a read-only volume in docker-compose:

volumes:
  - ./patched-js/index-XXXX.js:/opt/fileview/frontend/dist/js/index-XXXX.js:ro
Enter fullscreen mode Exit fullscreen mode

(Note: frontend filenames include hashes; after an image upgrade, the hash changes. Document this or the mount will silently break on upgrade.)

One more detail: fileview caches failure states. After patching, old files still reported “password required” because it served cached results. Clear the cache or re-trigger conversion.

Gotcha 3: An Unauthenticated SSRF

Before exposing the service publicly, I ran a security check and found fileview’s /preview/api/netFile endpoint: no authentication, accepts arbitrary URLs.

Its intended purpose: “download this file from a URL and convert it.” With no source restrictions, it’s an SSRF vector. Testing against an internal server:

curl -X POST http://server:8012/preview/api/netFile \
  -d '{"networkFileUrl":"http://192.168.x.x/","fileName":"t.docx"}'
Enter fullscreen mode Exit fullscreen mode

{"code":0, ...} accepted

Logs confirmed the outbound request:

🌐 Starting network file download - URL: http://192.168.x.x
❗ Download failed - Error: HTTP connect timed out

Classic SSRF. Error messages (timeout vs. connection refused vs. 404) leak port status.

Fortunately, fileview ships with a whitelist—no code changes needed:
environment:

  • FILEVIEW_NETWORK_SECURITY_TRUSTEDSITES=,127.0.0.1

This maps to fileview.network.security.trusted-sites (comma-separated; empty means unrestricted).

Small pitfall: including the port (e.g., IP:5212) causes even legitimate requests to be rejected—the comparison is host-only, no port.

Post-fix verification: internal hosts, external addresses, gateways, and cloud metadata endpoints (169.254.169.254) all returned “access denied.” Six common bypass techniques failed—@ userinfo confusion (http://trusted@target/), decimal IPs (http://3232261131/), hex IPs, DNS tricks like .nip.io, and file:// protocol attempts.

  1. Pre-Public Internet Security Checklist

For internal use, the setup is fine. Before exposing it publicly, these items are mandatory:

Issue Reality Check

Preview service port must not be exposed fileview has zero authentication. Beyond SSRF, all its endpoints are open.

No brute-force protection on login 8+ failed passwords all return “wrong password”—no rate limiting, no lockout, no CAPTCHA. Must be handled at the reverse proxy layer.

Plain HTTP Passwords and file contents transmitted in cleartext.

Admin interfaces Authenticated, but best practice: block external access at the proxy to reduce attack surface.

What’s already safe: Cloudreve’s own authentication is solid (unauthenticated file access returns 401; admin endpoints return 404). User registration is disabled by default.

  1. The Rough Edges

Write-ups tend to focus on wins. Here are the real shortcomings of this setup:

  1. Video format limitations. Browsers only play mp4 (H.264) / webm. AVI/MKV/FLV/WMV can be uploaded and downloaded, but not streamed—there’s no transcoding pipeline in the image. We got lucky: all 758 videos were mp4/mov.
  2. No true “team space.” Cloudreve v4 Community Edition embeds user IDs in storage paths, enforcing physical isolation. There’s no shared container concept. To share, you use share links (directory-level; subdirectories are included automatically). Recipients can “save shortcut” to their own file list, approximating a shared space—but there’s no “share directly with account X” workflow.
  3. Accounts as ownership boundaries. Files belong to the creator. To transfer ownership, you either change the account’s email or re-upload—because storage paths include the user ID.
  4. No redundancy. This is the biggest gap. 100 GB lives on a single disk. Disk failure = total loss. Local backups help little; an external backup target is essential.
  5. No preview for niche formats like XMind—download only.

Hope this helps someone evaluating a lighter alternative to Nextcloud. Happy to answer questions about the migration or the patches.

Top comments (0)