Mustafa ERBAY

Posted on Jun 18 • Originally published at mustafaerbay.com.tr

I Deleted Google Photos: All My Memories to My Own Server with Immich

#selfhosting #linux #docker #postgres

Ever since Google Photos changed its free storage limits, the idea of hosting all my digital memories under my own control, on my own server, has been on my mind. Last month, I finally pulled my nearly 400 GB photo and video archive entirely from Google and migrated it to my own VPS using the open-source Immich project. This process not only involved moving my data but also taught me important lessons about data sovereignty and long-term costs.

In this post, I will detail why I left Google Photos, why I chose Immich, the setup and data migration steps, the technical challenges I encountered, and the maintenance and security strategies I apply to keep my own photo server running.

Why I Left Google Photos: Data Sovereignty and Cost

For years, I got used to the convenience of Google Photos. Everything I shot on my phone was automatically backed up, and I could easily find anything I was looking for. However, with the end of unlimited storage in 2021, I realized that this convenience would come at a cost. The turning point for me was having nearly 400 GB of photo and video data and exceeding Google's 100 GB free quota.

When I saw the approximately 100 USD I would have to pay annually for 2 TB of storage, I calculated that this cost would be more expensive in the long run than buying a disk for my own server. Moreover, I was concerned about how my data was processed by Google's AI algorithms and how much my privacy was protected. I had a similar data sovereignty discussion while working on a production ERP; there, too, the risks and costs of keeping critical company data in the cloud pushed us to invest in our own infrastructure. My personal data was at least as critical to me.

💡 Cost Analysis

When calculating long-term costs, it's important to consider not only cloud storage fees but also potential data transfer fees and the risk of changes in the cloud service provider's pricing policies. The one-time cost I paid for a 2 TB disk on my own server would be more affordable than the total amount I would pay to Google over a few years.

What is Immich and Why Did I Choose Immich?

My search for an alternative to Google Photos led me to a number of open-source solutions. I examined options like PhotoPrism and Nextcloud Photos, but Immich, with its modern interface and mobile app support, met my expectations the most. Immich offers a self-hosted photo and video backup solution; it stands out with features like AI-powered object and face recognition, automatic backup, and multi-user support.

The main reasons I chose Immich were:

Mobile App: Thanks to its native apps for both Android and iOS, I can automatically back up from my phone. The user experience is quite similar to Google Photos.
Active Development: The project's GitHub repository is constantly updated, new features are rapidly added, and the community is very active. This gave me confidence for future support and development.
Microservice Architecture: In the background, PostgreSQL, Redis, and multiple Immich services run via Docker containers. This structure felt familiar to me in terms of performance and scalability; since we use similar approaches in enterprise software architectures, it would be easier to manage.
AI Features: Thanks to AI algorithms running on my own server, I can perform object and face recognition in my photos. This allows me to perform smart searches without sending my data to a third-party service.

Other solutions either lacked sufficient mobile app support or had very outdated interfaces. Immich stood out with its promise of a modern experience under my own control.

Immich Installation Process: Step-by-Step with Docker Compose

I used Docker Compose to install Immich on my own VPS. For me, running Docker containers on a bare-metal server is always my favorite hybrid deployment method, both for flexibility and resource management. Before installation, I had an Ubuntu 22.04 LTS VPS with 4GB RAM and 2vCPUs ready. Docker and Docker Compose were installed on this VPS.

The installation steps generally proceeded as follows:

Requirements:
- A Linux server (Ubuntu recommended)
- Docker and Docker Compose
- Sufficient storage space (in my case, a 2 TB NVMe disk)
- Nginx (for reverse proxy and SSL)
- A domain name (e.g., photos.mysite.com)

Pulling Immich Project Files:
I downloaded the docker-compose.yml file from Immich's official GitHub repository. This file defines all Immich services (server, microservices, web, machine learning, PostgreSQL, Redis).

mkdir -p /opt/immich && cd /opt/immich
wget -O docker-compose.yml https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
wget -O .env https://github.com/immich-app/immich/releases/latest/download/.env

Setting Environment Variables (.env):
I edited the .env file to make adjustments according to my own requirements. Variables such as DB_PASSWORD, UPLOAD_LOCATION (the directory where photos will be stored), and TZ (timezone) were particularly important. I used the /mnt/data/immich_uploads directory for photos on my server.
```
# Example .env file content
DB_PASSWORD=very_strong_password
UPLOAD_LOCATION=/mnt/data/immich_uploads
TZ=Europe/Istanbul
# ... other Immich settings
```
Starting Containers:
After making all the settings, I started the Immich services with a single command.
```
docker compose up -d
```
This command started all services in the background and downloaded the container images. The first launch may take some time, as multiple images are downloaded and the PostgreSQL database is initially set up. I monitored the status of the services with the docker compose logs -f command.

The architecture of Immich services is quite modular. The Mermaid diagram below shows how the main components interact with each other:

This structure ensures that each component fulfills its own responsibility and helps prevent a single component failure from affecting the entire system.

Importing Google Takeout Data to Immich and Challenges Faced

After completing the Immich setup, the most critical step was importing the data I downloaded from Google Photos to Immich. I had downloaded my entire archive as ZIP files using Google Takeout. However, properly importing this data into Immich was a more complex process than I expected.

Google Takeout creates a separate JSON file (.json) for each photo or video to store metadata information (capture date, location, etc.). Immich's own import tool can read these JSON files, but Takeout's folder structure and sometimes missing or incorrect metadata information can cause problems.

For import, I used a CLI tool called immich-go. This tool is specifically designed to import Google Takeout data into Immich.

Preparing Google Takeout Data:
I unzipped all the ZIP files and placed them under a single root directory as expected by the immich-go tool. For example: /mnt/data/google_takeout_data.
Installing the immich-go Tool:
I installed the tool on a system with Go installed (I installed it on my laptop) with the following command:
```
go install github.com/alextran1502/immich-go@latest
```
Importing Data to Immich:
I started the import with immich-go by specifying my Immich server address and API key.
```
immich-go upload --server https://photos.mysite.com --api-key <IMMICH_API_KEY> --recursive /mnt/data/google_takeout_data
```
One of the biggest challenges I encountered here was that the metadata of some photos in Takeout JSONs was corrupted. Especially for very old photos or videos uploaded from different devices, EXIF information could be missing. This prevented Immich from indexing these files with the correct timestamp. Some files appeared with default dates like 2000-01-01. This reminded me of the date format and data integrity issues we faced when integrating IFRS into a production ERP; data quality is always the biggest source of problems.

⚠️ Metadata Loss and Timestamps

It is possible for some metadata information in Google Takeout data to be missing or incorrect. This is often the case for old photos or photos from different sources. After importing into Immich, check the timestamps of important photos and correct them manually if necessary. Otherwise, your memories may not be in chronological order.

To detect such issues, I carefully examined the "Recently Uploaded" section in the Immich interface after the import and manually found and corrected files with strange-looking dates. Sometimes I also updated the EXIF data of photos using tools like exiftool with the information from the JSON.

Keeping the Immich Infrastructure Running: Maintenance, Security, and Performance

Setting up my own photo server is not just a one-time setup. It requires continuous maintenance, security updates, and performance optimization. This was another example of the "setup never ends, operations begin" principle I learned from my 20 years of system administration experience.

Maintenance and Updates

Since Immich is actively developed, it's important to keep up with updates regularly. I usually run docker compose pull && docker compose up -d commands weekly to update the containers to the latest versions. This allows me to get new features as well as critical security patches.

Security

I adopted a multi-layered approach to secure my Immich server:

Nginx Reverse Proxy and SSL: Instead of direct access to Immich, I routed all traffic through Nginx. I used a free SSL certificate from Let's Encrypt to ensure all traffic is encrypted. This is an approach I frequently use in L7 load balancing scenarios as well.

# /etc/nginx/sites-available/immich.conf
server {
    listen 80;
    server_name photos.mysite.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    server_name photos.mysite.com;

    ssl_certificate /etc/letsencrypt/live/photos.mysite.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/photos.mysite.com/privkey.pem;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
    ssl_prefer_server_ciphers on;

    location / {
        proxy_pass http://immich_web:3000; # Immich web container name and port
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering off; # Important for large file uploads
        client_max_body_size 0; # Unlimited file size
    }

    location /api {
        proxy_pass http://immich_server:3001; # Immich API container name and port
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering off;
        client_max_body_size 0;
    }
}

Fail2ban: I installed fail2ban to block failed login attempts to the server. It automatically blocks brute-force attacks by monitoring Nginx logs. This is always one of the first security measures I take.
Firewall (UFW): I ensured that only the necessary ports (80, 443) were open, making other server ports inaccessible from the outside. I also isolated communication between Immich's containers via Docker's own network.
System Security: I monitor Linux kernel modules (with auditd) and track known CVEs. For example, I previously blacklisted a kernel module for a security vulnerability in the algif_aead module. Such measures increase the overall security of the server.

Performance Optimization

I made some performance adjustments for Immich to run fast and smoothly:

PostgreSQL Tuning: Since PostgreSQL is the heart of Immich, I optimized the database settings. In particular, I adjusted shared_buffers, work_mem, and max_connections parameters according to my server's RAM and usage model. I also optimized checkpoint_completion_target and max_wal_size settings to prevent WAL bloat issues.

-- Some adjustments made in postgresql.conf
shared_buffers = 1GB          -- Around 25% of total RAM
work_mem = 64MB               -- Memory per query
maintenance_work_mem = 256MB  -- For operations like VACUUM
max_connections = 100         -- According to application's connection needs
wal_level = replica
max_wal_size = 4GB            -- Maximum size of WAL files
checkpoint_completion_target = 0.9 -- To extend checkpoint duration

Redis Eviction Policy: Redis is used by Immich for caching. I set the maxmemory-policy setting, which determines which data to evict when memory is full, to allkeys-lru. This ensures that the least recently used keys are evicted, keeping important data in memory.

Container Resource Limits: I limited the RAM and CPU usage of Immich containers using cgroup settings within Docker Compose. This was critical, especially since the immich-machine-learning service can consume high CPU and RAM. The memory.high soft limit prevents a service from overusing memory while tolerating momentary spikes.

# Example for immich-machine-learning service in docker-compose.yml
immich-machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:latest
    restart: always
    environment:
        # ...
    deploy:
        resources:
            limits:
                cpus: '1.0'
                memory: 2G
            reservations:
                cpus: '0.5'
                memory: 1G

Monitoring: I continuously monitor the performance of my server and Immich services using Prometheus and Grafana. I track metrics such as disk usage (I'm sensitive about this after experiencing a "docker disk fire"), CPU, RAM, and network traffic. I receive notifications when an alarm threshold is exceeded.

Backup Strategy

The most important aspect of my own photo server is backup. Losing all my memories would be a disaster. Therefore, I regularly back up Immich data:

PostgreSQL Database Backup: Every night, I back up the database using pg_dump and copy it to a separate storage area (an SMB share on a different server).
Photo and Video Files: I synchronize the directory I specified as UPLOAD_LOCATION (i.e., /mnt/data/immich_uploads) with rsync every night, backing it up to a separate storage unit.

These two backup strategies allow me to fully restore my Immich setup in case of any disaster.

Trade-offs and Future Plans for Photo Management on My Own Server

Migrating from Google Photos to Immich was more than just a technical challenge for me; it meant regaining control over my digital assets. However, as with every technological decision, this approach has its own advantages and disadvantages.

Advantages:

Full Control and Privacy: I know exactly where my data is stored, who can access it, and which algorithms process it. I am not dependent on a third-party company's privacy policies.
Cost-Effectiveness: Although there is an initial setup cost (VPS, disk), it is more affordable in the long run than annual subscription fees. This difference becomes even more pronounced for those with high storage needs.
Flexibility: Since it's on my own server, I can customize Immich according to my needs, and integrate it with different services (for example, I can use an AI model from my own side project to tag my photos).
Learning Opportunity: This process reinforced my practical experience in Docker, PostgreSQL tuning, Nginx configuration, and Linux system administration.

Disadvantages:

Maintenance Burden: System updates, backups, security patches, and potential troubleshooting are entirely my responsibility. This requires time and technical knowledge.
Initial Setup Complexity: After the "plug and play" ease of Google Photos, Immich's setup and data migration required more technical knowledge and patience.
Internet Connection: Since my home internet upload speed is limited, initial synchronization and uploading large video files took days. A good upload speed is critical for high-resolution photos and 4K videos.
Backup Responsibility: A robust backup strategy must be established and regularly checked to prevent data loss. This is an additional responsibility for those accustomed to the automatic backup convenience of cloud services.

Future Plans

My journey with Immich is just beginning. In the future, I aim to further develop Immich's AI capabilities. In particular, I plan to integrate some of my own AI models (using prompt engineering and RAG patterns) into Immich to add smarter search and automatic tagging features. Also, writing scripts to automatically synchronize photos from different devices (action cameras, drones) to Immich is among my plans.

This transition, like the "monolith vs microservice" debate, once again reminded me of the benefits and responsibilities that come with having full control over a system. My preference, when it comes to my personal data, has always been to take control.

I am quite satisfied with my Immich experience. Hosting my own data on my own server has given me both peace of mind and technological satisfaction. If you also want to take control of your digital memories, Immich is definitely a project you should consider. In the next post, I will explain how I integrated this Immich setup with a Zero-Trust Network Access (ZTNA) architecture.