Introducing KittenRouter - Serverless cousins to InboxKitten
Since we were using Cloudflare serverless kittens for our API. Why not expand this for the UI assets as well?
- Log every web request on cloudflare (This is an enterprise only feature), helping us to track server failures.
- Automated failover on server errors, to maintain overall site availability (High Availability!).
- All as a reusable class module
Keeping the project small along the "do one thing and do it well" concept, the pseudo code could be summarized as below.
when receiving a request {
for( each backend configuration ) {
if (invalid backend configuration - wrong domain, etc) {
continue - to next backend configuration iteration
}
perform cachable request to the backend
log request to ElasticSearch backend - or alternative log provider
if(
request result is valid ||
is last configuration ||
default backend configuration
) {
return result - Ends request processing
}
continue - to next backend configuration iteration
}
}
# PS: The above is conceptually the synchronous flow. In the actual
# implementation, logging should occur asynchronously without blocking
This allows us to configure redundancies within the backend, with commons.host servers coming in first, followed by firebase as "backup".
Background context (why we made this?)
When we migrated our UI frontend from Firebase to Commons.Host (an opensource and public CDN/Static hosting providers), one of the issues we faced was random 404's in random remote locations. Like Russia - possibly due to the lack of servers there.
And because we are supporters of the growing 🇸🇬 Commons Host project, we decided to stand by them together, instead of ditching them for alternatives.
We wanted a better way to debug such issues while improving the overall reliability of our service.
Side note to any big-brother authorities, we only track static assets loading with IP address masking on inboxkitten.com. No logging is done on the API. So yea, do not bother asking me for such data.
Elasticsearch Monitoring: Commons Host
What's awesome, is using the logging done by KittenRouter, of our commons host deployment, we can start having hard data on its status.
Is that currently in such a setup, we have 99.35% of our traffic routed through the free public servers of commons.host.
And for the remaining 0.65% of request which fails, They automatically fall back onto firebase infrastructure, under its free tier.
While it's not yet 9 nines, considering that these commons.host servers and cloudflare servers are scattered around the world. It's well within the 98% of real world user metrics of most CDNs
Furthermore, with the header information provided, we can keep track of the error rate on a per server basis. And with the KittenRouter setup on Cloudflare, you can configure such monitoring for any existing site without application code change.
We have since provided the commons.host team access to the above elasticsearch cluster for extracting useful data on their infrastructure under real user load.
Clarification notes, for privacy reasons, commons.host do not perform such level of logging on their systems for sites hosted on their platform.
The intention here for us at Uilicious, is to provide them with useful real-world usage metrics of their servers, and effectively exempting our website, inboxkitten.com, from such privacy protection, to help them improve their service.
Elasticsearch Monitoring: InboxKitten
Finally, we are able to easily use cloudflare country codes to help us keep track of our Kitten world domination map - live!. Meow-hahaha!
Greenland, and Madagascar - I got my
plague incInboxKitten eyes set on you!
Cool! Where can I get the code
For deployment and configuration details...
What is KittenRouter
KittenRouter is a routing script for Cloudflare Workers that attempts to connect to a list of specified servers and redirect the request to whichever server that is currently 'alive' at that point of time. It is extremely useful when you have servers that may go down or are unavailable to process the request and KittenRouter can automatically attempt to redirect the request to the next configured URL for processing.
At the same time, it can be configured to log down information to your ElasticSearch server for analytical purposes. Some of the information logged are the status of the servers, country of the request and etc. For the full details, see the index.js file.
How to use KittenRouter
Ultimately, KittenRouter is used together with Cloudflare workers. There are two ways in which you can use KittenRouter on your Cloudflare worker script,
- Using NPM modules
- Adding KittenRouter manually
1)
…That's cool, but Why do I even need Inboxkitten disposable email for again?
One of the key use cases currently, and why we built this project, is to perform email validations as part of our automated test scripts. Such as the following...
// Lets goto inbox kitten
I.goTo("https://inboxkitten.com");
I.see("Open-Source Disposable Email");
// Go to a random inbox inbox
I.fill("email", SAMPLE.id(22));
I.click("Get Mail Nyow!");
// Check that its empty
I.see("There for no messages for this kitten :(");
// Testing for regular email
// (sent using a jenkins perodic build)
I.goTo("https://inboxkitten.com");
I.see("Open-Source Disposable Email");
I.fill("email", "ik-reciever-f7s1g28");
I.click("Get Mail Nyow!");
// See an email we expect, nyow
I.see("Testing inboxkitten subject");
With sharable test results such as
Plus it's simple, cool, and fun to tinker around with.
What's next for KittenRouter?
Very rough roadmap of improvements needed for KittenRouter
#3
What's next then? For KittenRouter
- More configurable backend options
- Request domain, path, region
- Request timeout
- Error / Passing HTTP code control
- Custom headers control
- Perform request as specified domain on specific IP
- More logging options
- parameters logging whitelist/blacklist for
- Headers
- Cookies
- parameters logging whitelist/blacklist for
- More logging providers
- ???
Let's see where this InboxKitten journey goes next... Till then we will be taking a cat nap!
Top comments (6)
How was integrating with elastic search and kibana? Do they have free plans?
Elasticsearch + Kibana do have a free self hosted version. You do have to secure it using a http auth server, and https certificate for security. Which we run on our own.
That being said the process of setting that up isn't as one click as most would hope, and I do plan to do a seperate guide on it at some point.
Alternatively they do have cloud hosting options: elastic.co/cloud/elasticsearch-ser... which does so out the box.
Thanks for the reply. I'll look more into it. Great job btw documenting your journey will KittenRouter!
Wow, great example of using the right tools for the right jobs and focusing on your core concepts. Looks great!
+100 for the name 😸
😼 we plan to make all our future opensource projects based on animals, with their own cute mascots