DEV Community

Royce
Royce

Posted on Originally published at ossalt.com

Self-Host Your Email with Mailu

#mailu #email #selfhosting #docker

TL;DR

You can replace Google Workspace ($6/user/month) with Mailu — a Docker-based open source mail server — and pay only server costs (~$6-12/month). It's a legitimate path for technical teams prioritizing privacy and cost control. But self-hosting email is genuinely hard: DNS misconfiguration means your emails go to spam, IP reputation takes weeks to build, and a single mistake can get your IP blacklisted. This guide walks through the full setup and helps you decide if it's worth it.

Key Takeaways

  • Mailu is a Docker-first mail server — SMTP, IMAP, webmail, spam filtering, and admin UI in one stack
  • DNS records are the hardest part — MX, SPF, DKIM, DMARC, and PTR records must all be correct for deliverability
  • IP warm-up is mandatory — Gmail and Outlook apply strict filtering to new IPs; ramp up slowly
  • Mailu vs Mailcow: Mailu = simpler, better docs; Mailcow = more features, bigger community
  • When to self-host email: privacy-critical orgs, developers learning the stack, personal domains
  • When NOT to: transactional email (use Resend/Postmark instead), teams without a sysadmin

Should You Self-Host Email?

Be honest about the trade-offs before starting.

Self-hosting email is right for:

  • Personal or organizational domains where privacy matters (no Google scanning your emails)
  • Technical teams with a dedicated sysadmin
  • Learning how email infrastructure works
  • Cost reduction at scale (10+ mailboxes)

Don't self-host email if:

  • You need transactional email (welcome emails, password resets) — use Resend or Postmark instead
  • You don't have time to maintain it (spam filters update constantly)
  • Your IP address changes or you're on a residential connection
  • You need Google Workspace's collaborative features (Docs, Calendar, Meet)

Rule of thumb: Self-host email only if you're willing to own the deliverability problem. It works, but it requires ongoing attention.

Mailu vs Mailcow: Which Self-Hosted Email Server?

The two dominant Docker-based mail servers:

Feature Mailu Mailcow
Setup complexity ⭐ Simpler More complex
GitHub Stars ~6,500 ~9,000
Webmail Roundcube included SOGo + Roundcube
Admin UI ✅ Clean ✅ Feature-rich
Spam filtering Rspamd Rspamd
2FA ✅ ✅
ARM support ✅ Limited
Kubernetes support ✅ ❌
Documentation Good Excellent
Community Medium Large
Best for Simplicity Full-featured deployments

Recommendation: Start with Mailu if you want a simpler setup. Switch to Mailcow if you need more features or find Mailu limiting.

Part 1: Server Prerequisites

VPS Requirements

Email requires a dedicated static IP — not a shared host:

  • Minimum: 2 vCPU, 2 GB RAM, 20 GB storage
  • Recommended: Hetzner CX22 (€4.35/month) — dedicated IP included
  • Critical: Your provider must allow port 25 (SMTP). Some cloud providers block it by default. Hetzner, OVH, and most VPS providers allow it; AWS, GCP, and Azure block port 25 by default.

Check Port 25

Before installing anything, verify your VPS can send on port 25:

# Check if port 25 is open (from your VPS):
telnet smtp.gmail.com 25
# If it times out, your provider is blocking it
Enter fullscreen mode Exit fullscreen mode

Reverse DNS (PTR Record)

This is non-negotiable. Your IP address must resolve back to your mail server hostname. Set this in your VPS provider's control panel:

YOUR_SERVER_IP → mail.yourdomain.com
Enter fullscreen mode Exit fullscreen mode

Hetzner: Server → Networking → Edit PTR record

DigitalOcean: Networking → Domains → Add reverse record

Part 2: DNS Configuration

Before installing Mailu, configure all DNS records (these take up to 48 hours to propagate):

Required Records 

# MX record — tells other servers where to deliver email
yourdomain.com.    IN  MX  10  mail.yourdomain.com.

# A record — IP of your mail server
mail.yourdomain.com.  IN  A  YOUR_SERVER_IP

# SPF — which servers can send mail for your domain
yourdomain.com.    IN  TXT  "v=spf1 ip4:YOUR_SERVER_IP ~all"

# DMARC — policy for failed authentication (start permissive)
_dmarc.yourdomain.com.  IN  TXT  "v=DMARC1; p=none; rua=mailto:dmarc@yourdomain.com"
Enter fullscreen mode Exit fullscreen mode

DKIM: Generated by Mailu after installation — add after setup.

Autoconfig Records (optional but recommended)

These allow email clients (Thunderbird, Outlook) to auto-configure:

autoconfig.yourdomain.com.  IN  CNAME  mail.yourdomain.com.
autodiscover.yourdomain.com.  IN  CNAME  mail.yourdomain.com.
Enter fullscreen mode Exit fullscreen mode

Part 3: Install Mailu

Generate Configuration

Use Mailu's official setup wizard at setup.mailu.io to generate your docker-compose.yml and mailu.env.

Key settings to configure:

  • Domain: yourdomain.com
  • Hostname: mail.yourdomain.com
  • TLS flavor: letsencrypt (automatic SSL)
  • Webmail: roundcube (recommended)

Download both generated files to your server:

mkdir /mailu && cd /mailu
# Download the generated files from setup.mailu.io
# Or use the config generator CLI
Enter fullscreen mode Exit fullscreen mode

Deploy 

cd /mailu
docker compose up -d
Enter fullscreen mode Exit fullscreen mode

Mailu starts the following services:

  • front — Nginx reverse proxy + SSL termination
  • admin — Web admin interface
  • imap — Dovecot IMAP server
  • smtp — Postfix SMTP server
  • antispam — Rspamd
  • webmail — Roundcube

Create Your First Admin Account 

docker compose exec admin flask mailu admin admin yourdomain.com 'SecurePassword123'
# Creates admin@yourdomain.com with admin privileges
Enter fullscreen mode Exit fullscreen mode

Access the admin UI at https://mail.yourdomain.com/admin.

Part 4: Configure DKIM

DKIM keys are generated by Mailu on startup. Retrieve your public key:

docker compose exec admin flask mailu dkim yourdomain.com
Enter fullscreen mode Exit fullscreen mode

This outputs a DNS TXT record like:

dkim._domainkey.yourdomain.com.  IN  TXT  "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0..."
Enter fullscreen mode Exit fullscreen mode

Add this to your DNS. After propagation (up to 48 hours), verify:

# Verify DKIM is publishing correctly:
dig TXT dkim._domainkey.yourdomain.com
Enter fullscreen mode Exit fullscreen mode

Tighten DMARC After Verification

Once SPF and DKIM are working (check via mail-tester.com):

# Phase 1 — monitor only:
_dmarc.yourdomain.com.  IN  TXT  "v=DMARC1; p=none; rua=mailto:dmarc@yourdomain.com"

# Phase 2 — quarantine (after 2 weeks of clean reports):
_dmarc.yourdomain.com.  IN  TXT  "v=DMARC1; p=quarantine; rua=mailto:dmarc@yourdomain.com"

# Phase 3 — reject (full enforcement):
_dmarc.yourdomain.com.  IN  TXT  "v=DMARC1; p=reject; rua=mailto:dmarc@yourdomain.com"
Enter fullscreen mode Exit fullscreen mode

Part 5: Test Deliverability

Before sending any real email, verify your setup:

# 1. Test DNS records:
dig MX yourdomain.com
dig TXT yourdomain.com          # Should show SPF
dig TXT dkim._domainkey.yourdomain.com  # Should show DKIM
dig TXT _dmarc.yourdomain.com   # Should show DMARC

# 2. Check blacklists (run from your server):
curl -s "https://api.hetrixtools.com/v2/blacklist-check/YOUR_SERVER_IP/"

# 3. Send a test email to mail-tester.com:
# Visit mail-tester.com, get a unique address, send to it, check your score
# Aim for 9/10 or higher before sending real email
Enter fullscreen mode Exit fullscreen mode

IP Warm-Up Schedule

Gmail and Outlook treat new IPs with heavy suspicion. Follow this ramp-up:

Week 1:  10-50 emails/day (to engaged recipients only)
Week 2:  50-200 emails/day
Week 3:  200-500 emails/day
Week 4+: Scale gradually based on bounce/spam rates
Enter fullscreen mode Exit fullscreen mode

Never blast a large list from a new IP. The first emails should go to people who know you and will mark them as "not spam" if needed.

Part 6: Adding Mailboxes

Via the admin UI at https://mail.yourdomain.com/admin:

  1. Users → Add User → Set email, password, quota
  2. Aliases → Add alias (e.g., hello@ → admin@)
  3. Domains → Add additional domains (multi-domain setup)

Or via CLI:

# Add a user:
docker compose exec admin flask mailu user jane yourdomain.com 'Password123'

# Add an alias:
docker compose exec admin flask mailu alias info yourdomain.com jane
Enter fullscreen mode Exit fullscreen mode

Connecting Email Clients

Mailu auto-generates client configuration for:

  • IMAP: mail.yourdomain.com:993 (SSL/TLS)
  • SMTP: mail.yourdomain.com:587 (STARTTLS)
  • Webmail: https://mail.yourdomain.com

For Thunderbird, the autoconfig endpoint auto-detects settings when you enter your email address.

Ongoing Maintenance

Self-hosting email requires ongoing attention:

Monthly tasks:

When updates release:

cd /mailu
docker compose pull && docker compose up -d
Enter fullscreen mode Exit fullscreen mode

Backup email data:

# Backup mailboxes (Dovecot IMAP storage):
tar -czf mail-backup-$(date +%Y%m%d).tar.gz /mailu/mail/
Enter fullscreen mode Exit fullscreen mode

Cost Comparison

Option Monthly Cost (5 users) Notes
Google Workspace Starter $30/mo $6/user
Microsoft 365 Business Basic $30/mo $6/user
Fastmail $25/mo $5/user
Mailu (self-hosted) ~$6-12/mo Server cost only
Proton Mail Plus $50/mo Privacy-focused managed

Self-hosting saves ~$25-50/month for a 5-person team — meaningful for bootstrapped startups, but less meaningful once you factor in maintenance time.

Methodology

  • Mailu documentation: mailu.io/docs
  • Mailu GitHub: github.com/Mailu/Mailu
  • Email authentication specs: Google, DMARCLY, RFC standards
  • Comparison sources: RunCloud, GeekFlare, It's FOSS (2025-2026)

Compare all open source Google Workspace alternatives at OSSAlt.com.

Top comments (1)

Collapse
 
aliah_meddev profile image
Ali Ahmed •

We run Postfix and Dovecot with Roundcube in our panel, and the part that still trips people up is PTR. I have seen people fix SPF and DKIM for hours while the real problem was that the PTR was still set to the default VPS hostname at the provider, not at the registrar. I now check dig -x first, before I touch any other record.