I've been using ngrok for years to expose my dev servers to the world, usually so I can receive webhooks or test OAuth flows. It works amazingly well, and the price is reasonable.
It was a lifesaver as a sole dev building Skoach because, by nature of the app being a chatbot, every local server needs to receive webhooks from Slack and other channels. As the team grows and more people touch code, it translates to an additional ngrok account at ~$8/mo per dev. But startups are supposed to be frugal, right?
Enter Pagekite. It's got more features than ngrok with an arguably more complex interface, but it's open source and we can host our own "frontend" (i.e.: the public server that tunnels our requests). I figured I'd set this up once on a cheap server somewhere, and have a bunch of tunnels available for everyone on the team, with the bonus of using our own domain. Plus, I wrote it all down for posterity 😉.
- Generate a wildcard certificate for a domain using LetsEncrypt (you can skip this section if you don't need Pagekite to handle encryption for your https tunnels)
- Set up a Pagekite "frontend" on a publicly facing linux server with https support
- Configure a Pagekite "backend" (the client your run locally) to use our server for tunneling
- A public linux server somewhere, hopefully geographically near your team for lower latencies. A $5 DigitalOcean droplet will do (cheaper than ngrok even!), but EC2, Azure, or equivalent will work. My choice of distro was ubuntu in this particular case.
- Your own domain, and DNS access to it.
ssh into your server and make sure certbot is installed:
$ sudo apt install certbot
And build your certificate + key for the domain you selected:
$ sudo certbot certonly --manual \ --preferred-challenges=dns \ --email YOUR_EMAIL \ --server https://acme-v02.api.letsencrypt.org/directory \ --agree-tos \ --manual-public-ip-logging-ok \ -d "*.dev.skoach.com"
You should pick a domain under which all your tunnels will be created. For example,
*.dev.skoach.comwould let us create
bob.dev.skoach.comand so on. We'll use this as an example going forward.
This command will start the certificate creation & domain validation flow. It'll ask you to add a DNS record to your domain, the scope of which is beyond this article. There are other ways to verify your domain as well.
You should end up with 4 files under
total 12 -rw-r--r-- 1 root root 692 Aug 12 18:36 README lrwxrwxrwx 1 root root 43 Aug 12 18:36 cert.pem -> ../../archive/dev.skoach.com/cert1.pem lrwxrwxrwx 1 root root 44 Aug 12 18:36 chain.pem -> ../../archive/dev.skoach.com/chain1.pem lrwxrwxrwx 1 root root 48 Aug 12 18:36 fullchain.pem -> ../../archive/dev.skoach.com/fullchain1.pem lrwxrwxrwx 1 root root 46 Aug 12 18:36 privkey.pem -> ../../archive/dev.skoach.com/privkey1.pem
An additional step is required here because Pagekite expects the certificate and its private key to be in the same file, but LetsEncrypt generates them separately. This is easy enough:
$ sudo cat \ /etc/letsencrypt/live/dev.skoach.com/fullchain.pem \ /etc/letsencrypt/live/dev.skoach.com/privkey.pem \ | sudo tee /etc/letsencrypt/live/dev.skoach.com/keycert.pem > /dev/null
Just remember to run this after every time you renew your certificates. Take special care if you're auto-renewing.
In case you haven't already, you'll have to point your wildcard domain to your Pagekite server. The details can vary a bit with your DNS & server hosts and is beyond the scope of this article, but as an example, with CloudFlare DNS and a DigitalOcean droplet you'd have to add an A record with
*.dev as the hostname pointing to your droplet's IP address.
We've got our certs, so lets get Pagekite going. Download all the things:
$ sudo apt-get update $ sudo apt-get install dirmngr $ echo deb http://pagekite.net/pk/deb/ pagekite main | sudo tee -a /etc/apt/sources.list $ sudo apt-key adv --recv-keys --keyserver keys.gnupg.net AED248B1C7B2CAC3 $ sudo apt-get update $ sudo apt-get install pagekite
See this guide if you're curious about what each of these commands do.
Pagekite includes several config files and some defaults for being used as a backend server. This isn't what we want, so lets go into:
$ sudo nano /etc/pagekite.d/10_account.rc
And comment out everything:
#################################[ This file is placed in the Public Domain. ]# # Replace the following with your account details. # kitename = NAME.pagekite.me # kitesecret = YOURSECRET # Delete this line! # abort_not_configured
Then configure the frontend in:
$ sudo nano /etc/pagekite.d/20_frontends.rc
Comment out the
defaults line, and add the following (make sure to change the domains):
isfrontend ports=80,443 protos=http,https domain=http,https:*.dev.skoach.com:ASECRETFORYOURKITES # Skip the next line if you don't need HTTPS encryption handled by Pagekite tls_endpoint=*.dev.skoach.com:/etc/letsencrypt/live/dev.skoach.com/keycert.pem
ASECRETFORYOURKITES to authenticate your backends later. Now, restart Pagekite to effect these changes:
$ sudo systemctl restart pagekite.service
And set it up to start on boot:
$ sudo systemctl enable pagekite.service
If your server's behind a firewall (as it should be) remember to open ports 80 and 443. With ufw this would be
sudo ufw allow 80and
sudo ufw allow 443.
All we need to do now is set up our local Pagekite to connect to our new frontend. You can download it here if you haven't yet.
Although it says Python 2.7 is required, the team has made all required changes for it to work on Python 3 (and I've used it as such). If you have issues running the script directly, try running it with
$ python3 pagekite.pyinstead.
Run it once so it'll create default configs:
$ pagekite something-unique.pagekite.me
This will create the config file that you'll edit next. It should be
~/.pagekite.rc under MacOS and linux, or
C:\Users\your-username\pagekite.cfg under Windows. Add the following lines:
# kitename is the URL for your tunnel. Should be *something*.YOUR.DOMAIN.TLD, # and unique among your tunnels/devs kitename = jf.dev.skoach.com # kitesecret should match what you picked for your frontend under the "domain" config kitesecret = ASECRETFORYOURKITES # URL for your frontend frontend = dev.skoach.com:80 # Your tunnel. This will expose whatever's on port 8000 to the public URL picked above service_on = http:@kitename:localhost:8000:@kitesecret
Save and run pagekite again. It should connect to your frontend and expose your local port:
$ pagekite >>> Hello! This is pagekite.py v188.8.131.52725. [CTRL+C = Stop] Connecting to front-end relay 184.108.40.206:80 ... - Relay supports 2 protocols on 2 public ports. - To enable more logging, add option: --logfile=/path/to/logfile ~<> Flying localhost:8000 as https://jf.dev.skoach.com/ << pagekite.py [flying] Kites are flying and all is well.
You're all set! Just repeat this chapter for each tunnel & dev that you want to set up and you're done 💪.