Pagekite: expose your team's localhosts to the world under your own domain

I've been using ngrok for years to expose my dev servers to the world, usually so I can receive webhooks or test OAuth flows. It works amazingly well, and the price is reasonable.

It was a lifesaver as a sole dev building Skoach because, by nature of the app being a chatbot, every local server needs to receive webhooks from Slack and other channels. As the team grows and more people touch code, it translates to an additional ngrok account at ~$8/mo per dev. But startups are supposed to be frugal, right?

Enter Pagekite. It's got more features than ngrok with an arguably more complex interface, but it's open source and we can host our own "frontend" (i.e.: the public server that tunnels our requests). I figured I'd set this up once on a cheap server somewhere, and have a bunch of tunnels available for everyone on the team, with the bonus of using our own domain. Plus, I wrote it all down for posterity 😉.

In this article we'll explore how to

• Generate a wildcard certificate for a domain using LetsEncrypt (you can skip this section if you don't need Pagekite to handle encryption for your https tunnels)
• Set up a Pagekite "frontend" on a publicly facing linux server with https support
• Configure a Pagekite "backend" (the client your run locally) to use our server for tunneling

Pre-requisites

1. A public linux server somewhere, hopefully geographically near your team for lower latencies. A $5 DigitalOcean droplet will do (cheaper than ngrok even!), but EC2, Azure, or equivalent will work. My choice of distro was ubuntu in this particular case.
2. Your own domain, and DNS access to it.

Generate wildcard certificate

ssh into your server and make sure certbot is installed:

$ sudo apt install certbot

And build your certificate + key for the domain you selected:

$ sudo certbot certonly --manual \
  --preferred-challenges=dns \
  --email YOUR_EMAIL \
  --server https://acme-v02.api.letsencrypt.org/directory \
  --agree-tos \
  --manual-public-ip-logging-ok \
  -d "*.dev.skoach.com"

You should pick a domain under which all your tunnels will be created. For example, *.dev.skoach.com would let us create alice.dev.skoach.com, bob.dev.skoach.com and so on. We'll use this as an example going forward.

This command will start the certificate creation & domain validation flow. It'll ask you to add a DNS record to your domain, the scope of which is beyond this article. There are other ways to verify your domain as well.

You should end up with 4 files under /etc/letsencrypt/live/dev.skoach.com:

total 12
-rw-r--r-- 1 root root  692 Aug 12 18:36 README
lrwxrwxrwx 1 root root   43 Aug 12 18:36 cert.pem -> ../../archive/dev.skoach.com/cert1.pem
lrwxrwxrwx 1 root root   44 Aug 12 18:36 chain.pem -> ../../archive/dev.skoach.com/chain1.pem
lrwxrwxrwx 1 root root   48 Aug 12 18:36 fullchain.pem -> ../../archive/dev.skoach.com/fullchain1.pem
lrwxrwxrwx 1 root root   46 Aug 12 18:36 privkey.pem -> ../../archive/dev.skoach.com/privkey1.pem

Generate a cert + key file for Pagekite

An additional step is required here because Pagekite expects the certificate and its private key to be in the same file, but LetsEncrypt generates them separately. This is easy enough:

$ sudo cat \
  /etc/letsencrypt/live/dev.skoach.com/fullchain.pem \
    /etc/letsencrypt/live/dev.skoach.com/privkey.pem \
    | sudo tee /etc/letsencrypt/live/dev.skoach.com/keycert.pem > /dev/null

Just remember to run this after every time you renew your certificates. Take special care if you're auto-renewing.

Set up your DNS

In case you haven't already, you'll have to point your wildcard domain to your Pagekite server. The details can vary a bit with your DNS & server hosts and is beyond the scope of this article, but as an example, with CloudFlare DNS and a DigitalOcean droplet you'd have to add an A record with *.dev as the hostname pointing to your droplet's IP address.

Set up Pagekite

We've got our certs, so lets get Pagekite going. Download all the things:

$ sudo apt-get update
$ sudo apt-get install dirmngr
$ echo deb http://pagekite.net/pk/deb/ pagekite main | sudo tee -a /etc/apt/sources.list
$ sudo apt-key adv --recv-keys --keyserver keys.gnupg.net AED248B1C7B2CAC3
$ sudo apt-get update
$ sudo apt-get install pagekite

See this guide if you're curious about what each of these commands do.

Pagekite includes several config files and some defaults for being used as a backend server. This isn't what we want, so lets go into:

$ sudo nano /etc/pagekite.d/10_account.rc

And comment out everything:

#################################[ This file is placed in the Public Domain. ]#
# Replace the following with your account details.

# kitename   = NAME.pagekite.me
# kitesecret = YOURSECRET

# Delete this line!
# abort_not_configured

Then configure the frontend in:

$ sudo nano /etc/pagekite.d/20_frontends.rc

Comment out the defaults line, and add the following (make sure to change the domains):

isfrontend
ports=80,443
protos=http,https
domain=http,https:*.dev.skoach.com:ASECRETFORYOURKITES

# Skip the next line if you don't need HTTPS encryption handled by Pagekite
tls_endpoint=*.dev.skoach.com:/etc/letsencrypt/live/dev.skoach.com/keycert.pem

You'll use ASECRETFORYOURKITES to authenticate your backends later. Now, restart Pagekite to effect these changes:

$ sudo systemctl restart pagekite.service

And set it up to start on boot:

$ sudo systemctl enable pagekite.service

That's it!

If your server's behind a firewall (as it should be) remember to open ports 80 and 443. With ufw this would be sudo ufw allow 80 and sudo ufw allow 443.

Configure your Pagekite backend

All we need to do now is set up our local Pagekite to connect to our new frontend. You can download it here if you haven't yet.

Although it says Python 2.7 is required, the team has made all required changes for it to work on Python 3 (and I've used it as such). If you have issues running the script directly, try running it with $ python3 pagekite.py instead.

Run it once so it'll create default configs:

$ pagekite something-unique.pagekite.me

This will create the config file that you'll edit next. It should be ~/.pagekite.rc under MacOS and linux, or C:\Users\your-username\pagekite.cfg under Windows. Add the following lines:

# kitename is the URL for your tunnel. Should be *something*.YOUR.DOMAIN.TLD,
# and unique among your tunnels/devs
kitename   = jf.dev.skoach.com
# kitesecret should match what you picked for your frontend under the "domain" config
kitesecret = ASECRETFORYOURKITES

# URL for your frontend
frontend = dev.skoach.com:80

# Your tunnel. This will expose whatever's on port 8000 to the public URL picked above 
service_on = http:@kitename:localhost:8000:@kitesecret

Save and run pagekite again. It should connect to your frontend and expose your local port:

$ pagekite
>>> Hello! This is pagekite.py v1.5.2.200725.                   [CTRL+C = Stop]
    Connecting to front-end relay 134.209.88.195:80 ...
     - Relay supports 2 protocols on 2 public ports.
     - To enable more logging, add option: --logfile=/path/to/logfile
~<> Flying localhost:8000 as https://jf.dev.skoach.com/
 << pagekite.py [flying]   Kites are flying and all is well.

You're all set! Just repeat this chapter for each tunnel & dev that you want to set up and you're done 💪.

Acknowledgements

Thanks to whoever wrote this tutorial, Renato Candido for this guide, and to the Pagekite team for building awesome software 👌.

Thanks to Norberto and Vanessa for reviewing this article.

Thanks to Allec Gomes via Unsplash for the cover photo.

Useful References

• The Pagekite README is more complete than their website
• Pagekite's technical manual (i.e. manpage)
• Certbot (LetsEncrypt) user guide

Comments

[voarsh2]

Thanks for the useful tutorial, can you explain this?

Generate a cert + key file for Pagekite
An additional step is required here because Pagekite expects the certificate and its private key to be in the same file, but LetsEncrypt generates them separately. This is easy enough:
$ sudo cat \
/etc/letsencrypt/live/dev.skoach.com/fullchain.pem \
/etc/letsencrypt/live/dev.skoach.com/privkey.pem \
| sudo tee /etc/letsencrypt/live/dev.skoach.com/keycert.pem > /dev/null
Just remember to run this after every time you renew your certificates. Take special care if you're auto-renewing.

Does this command need to be run at renewal?

[João Ferreira]

Yep! It does. Pagekite expects a single file with the SSL certificate and private key, but letsencrypt creates separate files for all those. They're just text files though, so we can just concatenate their contents.

It's a bit of a chore, but you can add hooks to letsencrypt to do that: certbot.eff.org/docs/using.html#pr...