DEV Community

Cover image for restic: Designing a "Restorable Development Environment" While Excluding node_modules and .git
tumf
tumf

Posted on • Originally published at blog.tumf.dev

restic: Designing a "Restorable Development Environment" While Excluding node_modules and .git

#linux #security #devops

Originally published on 2026-01-09
Original article (Japanese): restic: node_modules と .git を除外しつつ「復元可能な開発環境」を維持する設計

Are you properly backing up your development machine?

I used to think, "I have Time Machine, so I'm fine," or "I can restore node_modules with git clone..." But backups that cannot be restored when truly needed are meaningless.

In my year-end cleanup article (Part 2), I introduced restic + resticprofile, but I received questions like, "How exactly should I configure it?" In this article, I will delve into the practical design of development environment backups using restic, explaining the fundamentals. Let's build a balanced backup strategy that avoids wasting space while ensuring that necessary items can be reliably restored.

What is restic?

restic is a modern backup tool developed in the Go programming language. Since its introduction in 2015, it has gained support for its simplicity and power.

Key Features

  • Encryption: All data is encrypted with AES-256
  • Deduplication: Efficient differential management at the chunk level rather than the file level
  • Incremental Backups: Only changes are transferred after the initial backup
  • Compression Support: Supports zstd compression since version 0.14 (2022)
  • Diverse Backends: Supports over 20 types of storage, including local disks, S3, SFTP, and B2
  • Cross-Platform: Consistent user experience on macOS, Linux, and Windows

Differences from Time Machine and borg

Tool Compression Multithreaded Cloud Support Main Use Case
Time Machine △ × × Local backups for macOS
borg ○ × △ For Linux servers
restic ○ ○ ◎ Cross-platform

While Time Machine is convenient because it is integrated into macOS, it requires an external disk and lacks flexibility. borg has excellent compression rates but operates in single-threaded mode, making it slower than restic on fast local disks.

restic combines "multithreading + compression + cloud support," making it particularly suitable for developers in diverse environments.

Installation

macOS 

brew install restic
Enter fullscreen mode Exit fullscreen mode

Linux (Ubuntu/Debian) 

sudo apt install restic
Enter fullscreen mode Exit fullscreen mode

Check Version 

restic version
# restic 0.18.1 compiled with go1.23.x on linux/amd64
Enter fullscreen mode Exit fullscreen mode

Basic Concept: The 3-2-1 Backup Rule

A fundamental principle of backup design is the "3-2-1 rule":

  • 3 copies (production data + 2 backups)
  • 2 different media types (local + cloud, etc.)
  • 1 offsite (physically separate location)

With restic, this rule can be relatively easily implemented.

Step 1: Initialize the Repository

In restic, the location where backup data is stored is called a "repository." Let's create a repository on a local disk.

Create a Local Repository 

# Create the backup destination directory
mkdir -p /Volumes/Backup/restic-repo

# Initialize the repository
restic init --repo /Volumes/Backup/restic-repo
Enter fullscreen mode Exit fullscreen mode

You will be prompted for a password. This will be used to generate the encryption key, so make sure to store it securely.

enter password for new repository: 
enter password again: 
created restic repository 6a3e8a9b at /Volumes/Backup/restic-repo

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
Enter fullscreen mode Exit fullscreen mode

Password Management Options

Entering the password every time can be cumbersome. You can automate this using the following methods.

Method 1: Password File (Recommended) 

# Create a password file (set strict permissions)
echo "your-secure-password" > ~/.restic-password
chmod 600 ~/.restic-password

# Backup using the password file
restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       backup ~/work
Enter fullscreen mode Exit fullscreen mode

Method 2: Environment Variables 

# Add to .zshrc or .bashrc
export RESTIC_REPOSITORY="/Volumes/Backup/restic-repo"
export RESTIC_PASSWORD="your-secure-password"

# You can now run with shorter commands
restic backup ~/work
Enter fullscreen mode Exit fullscreen mode

Security Note: Environment variables can be visible with commands like ps, so the password file method is safer.

Step 2: Initial Backup

Let's back up the ~/work directory in its simplest form.

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       backup ~/work
Enter fullscreen mode Exit fullscreen mode

Execution result:

repository 6a3e8a9b opened (version 2, compression level auto)
created new cache in /Users/tumf/.cache/restic

Files:       12543 new,     0 changed,     0 unmodified
Dirs:         1823 new,     0 changed,     0 unmodified
Added to the repository: 4.2 GiB (3.1 GiB stored)

processed 12543 files, 5.8 GiB in 2:15
snapshot 9a3d7f2e saved
Enter fullscreen mode Exit fullscreen mode

Key points to note:

  • Compression Effect: 5.8 GiB of data compressed to 3.1 GiB (approximately 47% reduction)
  • Processing Speed: Processed 12,543 files in 2 minutes and 15 seconds
  • Snapshot ID: 9a3d7f2e is the ID that identifies this backup

Step 3: Directories to Exclude in Development Environments

Now we get to the main topic. Development environments often contain a large number of temporary files and build artifacts, which can take up significant space if backed up as is.

Typical Directories to Exclude

Node.js/JavaScript Projects 

node_modules/       # Dependency packages (can be restored from package.json)
.next/              # Next.js build artifacts
dist/               # Build output
build/              # Build output
coverage/           # Test coverage
.turbo/             # Turbopack cache
Enter fullscreen mode Exit fullscreen mode

Python Projects 

__pycache__/        # Python bytecode
.venv/              # Virtual environment (can be restored from requirements.txt)
venv/
.pytest_cache/      # pytest cache
.mypy_cache/        # mypy cache
.ruff_cache/        # ruff cache
*.egg-info/         # Package metadata
Enter fullscreen mode Exit fullscreen mode

Rust Projects 

target/             # Build artifacts (can be restored from Cargo.toml)
Enter fullscreen mode Exit fullscreen mode

Go Projects 

vendor/             # Dependency packages (can be restored from go.mod)
Enter fullscreen mode Exit fullscreen mode

Git Repositories 

.git/objects/       # Git objects (can be restored from remote)
Enter fullscreen mode Exit fullscreen mode

However, you must not exclude .git/config or .git/hooks as they contain local settings.

Criteria for Determining Restorability

Criteria for determining whether something can be excluded:

  1. Can it be fully restored from definition files?

    • package.json → Restorable with npm install ✅
    • requirements.txt → Restorable with pip install -r ✅
    • Cargo.toml → Restorable with cargo build ✅

  2. Can it be obtained from remote?

    • .git/objects/ → Restorable with git clone or git fetch ✅
    • Local uncommitted changes → Not restorable ❌

  3. Does regeneration take too long?

    • Build time is a few minutes → Can be excluded ✅
    • Build time is several hours → Depends on the situation △

Step 4: Designing Exclude Patterns

Create Exclude File

Write the patterns in ~/.restic-excludes:

# ~/.restic-excludes

# Node.js
**/node_modules
**/.next
**/dist
**/build
**/coverage
**/.turbo

# Python
**/__pycache__
**/.venv
**/venv
**/.pytest_cache
**/.mypy_cache
**/.ruff_cache
**/*.egg-info

# Rust
**/target

# Go
**/vendor

# Git objects (keep config)
**/.git/objects
**/.git/logs
**/.git/refs/remotes

# Editors/IDEs
**/.vscode/.history
**/.idea/workspace.xml
**/.idea/tasks.xml

# macOS
**/.DS_Store
**/._*

# Temporary files
**/*.tmp
**/*.log
**/.cache
Enter fullscreen mode Exit fullscreen mode

How to Write Patterns

  • **/ matches any depth
  • *.log matches by extension
  • !pattern specifies exceptions to the exclusion (to include specific files)

Backup with Exclude File 

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       --exclude-file ~/.restic-excludes \
       backup ~/work
Enter fullscreen mode Exit fullscreen mode

Execution result:

Files:        8234 new,     0 changed,     0 unmodified
Dirs:          983 new,     0 changed,     0 unmodified
Added to the repository: 1.2 GiB (892 MiB stored)

processed 8234 files, 1.8 GiB in 0:45
snapshot b7e2c9a1 saved
Enter fullscreen mode Exit fullscreen mode

The size reduced from 5.8 GiB to 1.8 GiB after exclusions (approximately 69% reduction).

Utilizing exclude-caches

Many build tools place a CACHEDIR.TAG file in their cache directories. You can automatically detect and exclude these:

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       --exclude-file ~/.restic-excludes \
       --exclude-caches \
       backup ~/work
Enter fullscreen mode Exit fullscreen mode

The --exclude-caches option excludes directories that contain files adhering to the Cache Directory Tagging Standard:

Signature: 8a477f597d28d172789f06886806bc55
# This file is a cache directory tag.
Enter fullscreen mode Exit fullscreen mode

Cache directories for Docker and Homebrew will also be automatically excluded.

Step 5: Verify Backups and Restore

List Snapshots 

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       snapshots
Enter fullscreen mode Exit fullscreen mode

Example output:

repository 6a3e8a9b opened (version 2, compression level auto)
ID        Time                 Host        Tags        Paths
-------------------------------------------------------------------------
9a3d7f2e  2025-12-28 15:30:00  macbook                 /Users/tumf/work
b7e2c9a1  2025-12-28 16:45:00  macbook                 /Users/tumf/work
-------------------------------------------------------------------------
2 snapshots
Enter fullscreen mode Exit fullscreen mode

Search for Specific Files 

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       find "package.json"
Enter fullscreen mode Exit fullscreen mode

Restore Files (Partial Restore)

Restore only specific files:

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       restore b7e2c9a1 \
       --target ~/restore \
       --include "project-name/package.json"
Enter fullscreen mode Exit fullscreen mode

Full Restore

Restore the entire snapshot:

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       restore latest \
       --target ~/restore
Enter fullscreen mode Exit fullscreen mode

latest refers to the most recent snapshot.

Step 6: Manage Generations (Retention Policy)

As you continue to back up, the storage will increase. Set up "generation management" to automatically delete old snapshots.

forget Command 

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       forget \
       --keep-last 5 \
       --keep-daily 7 \
       --keep-weekly 4 \
       --keep-monthly 6 \
       --prune
Enter fullscreen mode Exit fullscreen mode

Meaning of the options:

  • --keep-last 5: Keep the latest 5 generations
  • --keep-daily 7: Keep daily backups for the past 7 days
  • --keep-weekly 4: Keep weekly backups for the past 4 weeks
  • --keep-monthly 6: Keep monthly backups for the past 6 months
  • --prune: Physically delete unnecessary data blocks after deletion

Best Practices for Generation Management

Recommended settings for development environments:

# If backing up frequently
--keep-last 3 \
--keep-daily 7 \
--keep-weekly 4 \
--keep-monthly 3

# If storage capacity allows
--keep-last 10 \
--keep-daily 14 \
--keep-weekly 8 \
--keep-monthly 12
Enter fullscreen mode Exit fullscreen mode

Step 7: Automation (Scheduling)

It's easy to forget to back up manually. Let's automate it.

macOS (Using launchd)

Create ~/Library/LaunchAgents/dev.tumf.restic-backup.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>dev.tumf.restic-backup</string>

    <key>ProgramArguments</key>
    <array>
        <string>/opt/homebrew/bin/restic</string>
        <string>--repo</string>
        <string>/Volumes/Backup/restic-repo</string>
        <string>--password-file</string>
        <string>/Users/tumf/.restic-password</string>
        <string>--exclude-file</string>
        <string>/Users/tumf/.restic-excludes</string>
        <string>--exclude-caches</string>
        <string>backup</string>
        <string>/Users/tumf/work</string>
    </array>

    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>3</integer>
        <key>Minute</key>
        <integer>0</integer>
    </dict>

    <key>StandardOutPath</key>
    <string>/Users/tumf/.restic-backup.log</string>

    <key>StandardErrorPath</key>
    <string>/Users/tumf/.restic-backup-error.log</string>
</dict>
</plist>
Enter fullscreen mode Exit fullscreen mode

Start it:

launchctl load ~/Library/LaunchAgents/dev.tumf.restic-backup.plist
Enter fullscreen mode Exit fullscreen mode

This will execute automatic backups daily at 3 AM.

Linux (Using systemd)

Create ~/.config/systemd/user/restic-backup.service:

[Unit]
Description=Restic backup service

[Service]
Type=oneshot
ExecStart=/usr/bin/restic --repo /backup/restic-repo \
    --password-file /home/tumf/.restic-password \
    --exclude-file /home/tumf/.restic-excludes \
    --exclude-caches \
    backup /home/tumf/work

[Install]
WantedBy=default.target
Enter fullscreen mode Exit fullscreen mode

Create ~/.config/systemd/user/restic-backup.timer:

[Unit]
Description=Restic backup timer

[Timer]
OnCalendar=*-*-* 03:00:00
Persistent=true

[Install]
WantedBy=timers.target
Enter fullscreen mode Exit fullscreen mode

Enable it:

systemctl --user daemon-reload
systemctl --user enable restic-backup.timer
systemctl --user start restic-backup.timer
Enter fullscreen mode Exit fullscreen mode

Step 8: Key Management with Multiple Passwords

Considering team development and emergency recovery, you can register multiple passwords.

Register Additional Passwords 

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       key add
Enter fullscreen mode Exit fullscreen mode

When you enter a new password, another key will be added.

List Passwords 

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       key list
Enter fullscreen mode Exit fullscreen mode

Example output:

 ID          User        Host        Created
--------------------------------------------------------------
*eb2e1c89    tumf        macbook     2025-12-28 15:30:00
 a3d7f9b2    tumf        macbook     2025-12-28 16:50:00
--------------------------------------------------------------
Enter fullscreen mode Exit fullscreen mode

The * indicates the currently used key.

Use Cases

  1. Team Sharing: Issue different passwords for each member
  2. Emergency Key: Store a backup password in a secure location that is not used regularly
  3. Rotation: Change passwords regularly and remove old keys

Remove Old Keys 

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       key remove a3d7f9b2
Enter fullscreen mode Exit fullscreen mode

Step 9: Utilizing Cloud Backends

Local disks alone do not satisfy the "offsite" requirement of the 3-2-1 rule. Let's add cloud storage.

Backblaze B2 (Recommended for Low Cost)

Backblaze B2 is S3 compatible, inexpensive, and works well with restic.

# Set environment variables
export B2_ACCOUNT_ID="your-account-id"
export B2_ACCOUNT_KEY="your-account-key"

# Initialize repository
restic init --repo b2:bucket-name:restic-repo

# Backup
restic --repo b2:bucket-name:restic-repo \
       --password-file ~/.restic-password \
       --exclude-file ~/.restic-excludes \
       --exclude-caches \
       backup ~/work
Enter fullscreen mode Exit fullscreen mode

AWS S3 

export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"

restic init --repo s3:s3.amazonaws.com/bucket-name

restic --repo s3:s3.amazonaws.com/bucket-name \
       --password-file ~/.restic-password \
       backup ~/work
Enter fullscreen mode Exit fullscreen mode

SFTP (e.g., Home NAS) 

restic init --repo sftp:user@nas.local:/backup/restic-repo

restic --repo sftp:user@nas.local:/backup/restic-repo \
       --password-file ~/.restic-password \
       backup ~/work
Enter fullscreen mode Exit fullscreen mode

Step 10: Verify Backups

Regularly verify that backups are not corrupted.

check Command 

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       check
Enter fullscreen mode Exit fullscreen mode

Example output:

using temporary cache in /var/folders/...
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
no errors were found
Enter fullscreen mode Exit fullscreen mode

read-data Option (Full Verification) 

restic --repo /Volumes/Backup/restic-repo \
       --password-file ~/.restic-password \
       check --read-data
Enter fullscreen mode Exit fullscreen mode

This reads and verifies all data blocks (it takes time). It is recommended to run this about once a month.

Practical Example: My Development Environment Backup Configuration

Finally, I will share the configuration I actually use.

Directory Structure 

~/
├── work/                    # All projects
├── .restic-password         # Password file
├── .restic-excludes         # Exclude patterns
└── .zshrc                   # Environment variable settings
Enter fullscreen mode Exit fullscreen mode

.zshrc Configuration 

# restic aliases
export RESTIC_REPOSITORY="/Volumes/Backup/restic-repo"
export RESTIC_PASSWORD_FILE="$HOME/.restic-password"

alias rb='restic backup ~/work --exclude-file ~/.restic-excludes --exclude-caches'
alias rs='restic snapshots'
alias rf='restic forget --keep-last 5 --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune'
alias rc='restic check'
Enter fullscreen mode Exit fullscreen mode

Daily Operations 

# Execute backup (using alias)
rb

# Check snapshots
rs

# Delete old snapshots
rf

# Monthly verification
rc --read-data
Enter fullscreen mode Exit fullscreen mode

It's simple, but it allows me to focus on development with peace of mind.

Conclusion

I have explained the design of development environment backups using restic.

Key Points:

  1. Exclude Patterns: Exclude items like node_modules and .venv that can be restored
  2. Generation Management: Automatically delete old snapshots with forget
  3. Multiple Passwords: Register multiple keys for team sharing and emergencies
  4. Automation: Schedule with launchd/systemd
  5. Offsite: Achieve the 3-2-1 rule with cloud backends
  6. Regular Verification: Check data integrity with the check command

"Backups are meaningless if they cannot be restored" — keep this in mind and don't forget to conduct regular restore tests.

Next time, I plan to discuss managing multiple profiles using resticprofile and monitoring integrations using HTTP Hooks.

Reference Links

Top comments (0)