I Built a Photo Management SaaS with FastAPI, Qdrant, CLIP and Automatic Duplicate Detection

ChRainG — Fri, 25 Sep 2026 11:03:50 +0000

Managing a few hundred photos is easy.

Managing 20,000, 50,000 or 100,000 photos is a different problem.

At some point you stop remembering where a photo is stored, whether you already have another copy of it, or why there are 15 almost identical images from the same moment.

That problem is what led me to build Renvumi.

Renvumi is a web service for analyzing and organizing large photo libraries.

It is available here:

https://renvumi.ru/

The interface is now available in both English and Russian.

The original problem

The first version was supposed to do one thing:

find duplicate photos

Exact duplicates are relatively easy.

If two files have the same content, a cryptographic hash can identify them.

But real photo libraries are much messier.

You often have:

resized copies
recompressed JPEGs
edited versions
burst shots
nearly identical frames
the same photo stored in several albums
files with identical names but completely different contents

So exact hashing solved only a small part of the problem.

Exact duplicates

For exact duplicates, Renvumi uses file hashes.

This is useful when several backup drives or old folders have been merged together.

If two files have identical content, they can be grouped immediately without expensive AI processing.

Near duplicates

Near duplicates are more interesting.

Two files can look almost identical but have different binary contents.

For this case I use perceptual hashing.

This works well for things like:

resized images
recompressed images
small edits
slightly modified copies

But perceptual hashes are not enough for broader semantic similarity.

Similar image search with CLIP

For more flexible similarity search, Renvumi uses CLIP embeddings.

Each analyzed image gets a vector representation.

Those vectors are stored in Qdrant.

This makes it possible to search for visually related photos even when they are not near duplicates.

For example, you can select one photo and search for similar photos across the entire library.

That can work for:

the same car photographed at different times
the same building
similar landscapes
related scenes
different photos from the same event

Renvumi uses absolute CLIP cosine similarity for this type of search.

Perceptual hashes are kept for near-duplicate detection.

I intentionally separated these two concepts because they solve different problems.

Text search

CLIP also makes text-to-image search possible.

Instead of remembering filenames, you can search for something like:

dog on the beach

or:

red car near a house

The system converts the query into the same embedding space and searches the existing image vectors.

This is one of the features that makes a large photo archive feel less like a filesystem and more like a searchable database.

Global search across albums

Originally, every operation was scoped to one album.

That quickly became a limitation.

Imagine this:

Phone Backup/
    IMG_1001.JPG

Old HDD/
    IMG_1001.JPG

Vacation/
    copy_IMG_1001.JPG

The same image may exist in several places.

So Renvumi now has a Global mode.

It aggregates results from all analyzed albums.

From there users can work with:

exact duplicates
near duplicates
similar photos
unique photos
people
best shots
cleanup candidates
filename conflicts

The original album name is still shown for every result.

Same filename, different file

This turned out to be a surprisingly useful feature.

Cameras and phones reuse filenames over time.

After several years, a library may contain many files called:

IMG_0001.JPG

But they may all be different photos.

Renvumi detects this separately.

The rule is simple:

same filename + different SHA-256 = filename conflict

If both filename and content are identical, it is treated as an ordinary exact duplicate instead.

Choosing the best shots in a series

Another common problem is burst photography.

You may have 10 or 20 images taken within a few seconds.

They are all different files, but from a user's perspective they form one series.

Renvumi groups related shots and helps identify stronger candidates.

It can use signals such as:

sharpness
face detection
closed-eye detection
visual similarity

But I deliberately do not let the software automatically decide what the user must delete.

A technically imperfect photo can still be the best photo emotionally.

So the system narrows the set.

The user makes the final decision.

Face detection and people grouping

For people detection I use OpenCV components including YuNet and face recognition logic.

The pipeline also contains additional checks to reduce obvious false positives.

The goal is to group images of the same person across an album and then aggregate those results globally.

This part turned out to require much more tuning than I initially expected.

Face detection itself is easy.

Reliable grouping across thousands of real photos is not.

Automatic analysis of new uploads

One workflow decision changed recently.

Previously the process was:

upload photos
click Analyze
wait

That felt unnecessary.

A photo analysis service without analysis is not very useful.

Now the workflow is:

upload photos
analysis starts automatically

But there is an important optimization.

Only new, not-yet-analyzed photos are processed.

For example:

Day 1:
Upload 5,000 photos
Analyze 5,000 photos

Day 7:
Upload 100 photos
Analyze only those 100 photos

The existing 5,000 images are not analyzed again.

This saves GPU time and user quota.

If another upload happens while analysis is already running, the new photos remain pending and are picked up by the next analysis job.

Background processing

Image analysis is expensive enough that I did not want HTTP requests doing the heavy work.

Renvumi uses Celery with Redis for background jobs.

Different types of work are separated.

For example:

GPU analysis
import operations
maintenance operations
scheduled jobs

The GPU worker currently runs with concurrency set to one.

That prevents multiple large model workloads from fighting for GPU memory.

Interactive work also gets higher queue priority than large background analysis jobs.

Avoiding O(n²) comparisons

One of the less glamorous problems appears when a photo library becomes large.

A naive pairwise comparison of all images grows very quickly.

For 10,000 photos:

10,000 × 9,999 / 2

is almost 50 million pairs.

That is not something I wanted to do every time a user opened a result page.

So candidate generation happens first.

For near-duplicate processing I use perceptual-hash-based indexing to narrow the candidate set before applying more expensive comparisons.

This was one of the biggest performance improvements for the global archive mode.

The backend stack

The current backend includes:

FastAPI
PostgreSQL 16
Redis
Celery
Qdrant
OpenCV
CLIP
LibreTranslate
Docker Compose
Caddy

The entire service runs in Docker.

Caddy handles HTTPS.

PostgreSQL stores application data.

Qdrant stores image embeddings.

Redis handles Celery, caching and temporary runtime state.

Why LibreTranslate?

Renvumi supports text-to-image search.

But CLIP works more predictably for my use case when queries are normalized into English.

So I run a private LibreTranslate container inside the Docker network.

For example:

жираф

can be translated internally to:

giraffe

before going to the search pipeline.

No external translation API is required.

The same internal translation system is also now used as part of the English localization workflow.

English and Russian interface

Renvumi originally started as a Russian-language product.

I recently added full language switching:

🇷🇺 Russian
🇬🇧 English

The selected language applies to:

main UI
account pages
Help
API documentation
examples
UI messages
PWA metadata

The language is remembered in a cookie.

On the first visit, the service can use the browser language as the default.

Storage and safety

Photo management software has one dangerous feature:

it can delete photos.

That means recovery matters.

Renvumi uses a trash workflow rather than immediately destroying files.

There are also backup and restore mechanisms.

I have spent more time than expected on things like:

interrupted jobs
database pool exhaustion
backup consistency
recovery
storage encryption
failed analysis
user quotas
long-running imports

These parts are much less exciting than AI models, but they are what make the service usable in practice.

API

Renvumi also has an external API for paid plans.

The API documentation includes examples for:

cURL
Python
PowerShell

I wanted the API docs to describe not just the endpoint name, but also:

what the function does
parameters
returned data
request example
response example

The documentation is available in both English and Russian.

The infrastructure is currently self-hosted

One unusual part of the project is that Renvumi currently runs on my own server.

The machine has:

72 CPU threads
256 GB RAM
NVIDIA GPU

This gives me much more control over GPU workloads than using an expensive managed inference API for every image.

It also makes performance tuning interesting because I can see exactly where the bottlenecks are.

What I learned building it

One thing became clear fairly quickly:

AI was not the hardest part.

Getting an embedding from CLIP is easy.

Building a real service around it is much harder.

The difficult parts were things like:

making large jobs resumable
not analyzing the same photo twice
keeping interactive searches responsive
limiting database connections
cleaning up thousands of images safely
keeping backup operations isolated
handling users with large libraries
making analysis results understandable

The model is only one component.

The product is everything around it.

Current status

Renvumi is live:

https://renvumi.ru/

The free tier currently includes:

1,000 analyses
1 GB storage

My main goal now is not to keep adding features.

I want to see how the system behaves on real photo libraries.

Especially:

10,000 photos
50,000 photos
100,000+ photos

Test datasets are useful, but real archives always contain weird cases you did not expect.

I would like feedback

If you have a large photo library, I would be interested in hearing how you manage it today.

Do you use:

Lightroom?
Google Photos?
Apple Photos?
local folders?
a NAS?
custom scripts?
something else?

And what would stop you from using a service like Renvumi?

For example:

privacy
upload speed
cloud storage
price
wanting a desktop version
not trusting automatic similarity detection

Those answers are probably more valuable to me right now than another feature request.

Renvumi:

https://renvumi.ru/

DEV Community: ChRainG

I Built a Photo Management SaaS with FastAPI, Qdrant, CLIP and Automatic Duplicate Detection

The original problem

Exact duplicates

Near duplicates

Similar image search with CLIP

Text search

Global search across albums

Same filename, different file

Choosing the best shots in a series

Face detection and people grouping

Automatic analysis of new uploads

Background processing

Avoiding O(n²) comparisons

The backend stack

Why LibreTranslate?

English and Russian interface

Storage and safety

API

The infrastructure is currently self-hosted

What I learned building it

Current status

I would like feedback