As I commented on my Challenges, Solutions and more Challenges and more Solutions blog post. I believe a serverless solution would be a better fit, instead of a continuously running service.
If you want to get an overview of the problem I wanted to solve and more on the challenges and solution, do give it a read.
Readdressing the solution, I got the service up and running as a serverless function with Digital Ocean's Functions offering. I had never done a serverless function before doing this one and it required a lot of experimentation and I faced multiple challenges, luckily I also came across proper solution. Describing the solution I will highlight some of the challenges.
The main goal of the solution would be the capability of redirecting from a short URL to a long URL.
Example: https://releases.llvm.org/13.0.0/tools/clang/docs/DiagnosticsReference.html#wall
It should be reachable via something along the line of: https://pxy.fi/13/all, this was implemented for the service, so this had to be ported to the serverless function.
The serverless functions with Digital Ocean are based on Apache Open Whisk, so the service has additional name space, which need to go into the URL.
First version looked like this:
/proxy/redirect/13/all
I decided to cut the namespace down to single letters and ended up with:
/p/r/13/all
When functions are deployed that work out of the box, but they do have very long URLs, so I wanted and need to get them served from my own short domain name: pxy.fi instead of: https://urchin-app-c8tyu.ondigitalocean.app/, which would kind of defeat the whole purpose of the solution, since brevity is essential.
It is however possible to configure the functions with DigitalOcean to be served from your own domain name, so after reading the documentation, asking around I got it to work.
Another concept with serverless functions seem very much to be one of serverless functions to be called in a controlled environment, so they are fed JSON as input and they return JSON.
I however wanted to consume a URL with it's path and return a redirect via HTTP, the latter actionally being doable quite easily (I found an example).
function main() {
return {
headers: { location: 'http://openwhisk.org' },
statusCode: 302
}
}
Example from: apache/openwhisk
As mentioned above, the package name space of the serverless function is included in the URL and since I used the URL to communicate my intent (or input) data, I needed to get the data/parameters out of the URL.
This accomplished done line this:
path := args["__ow_path"].(string)
This was actually how I found out that the DigitalOcean functions where based on Apache Open Whisk, due to the "ow" prefix. I did not know Apache Open Whisk at all, but getting this information opened up for locating more blog posts and more documentation and examples, not being limited to the DigitalOcean documentation and examples.
The documentation from DigitalOcean is fine, but it is limited and I do not think Open Whisk was mentioned anywhere, but I could have missed it.
Due to challenges with the Go implementation I had already changed the format of the URL for consumption to avoid the use of fragments to be a path.
I found this response on StackOverflow to the question I was asking myself
where the helicopter did the fragment go?
And I eventually found it in the Go net/url documentation
Web browsers strip #fragment before sending the URL to a web server.
Where the target URL uses a fragment resembling e.g.: #wall the short version used a form mimicking a path, like so: /<version number>/<fragment> e.g. /13/all. Where the end results is translates the last part of the URL (back) into a fragment. By "back" I mean that the URLs I used are actually parsed from the original URLs including fragement and translated into a format that was transportable, see: clang diagnostic flags matrix generator.
A funny thing about the URL of the service, https://pxy.fi/p/r/13/all (example), was that I could lift the code from the service implementation, since the root was not where I expected it to be: https://pxy.fi/, but actually here: https://pxy.fi/p/r/
So the code parsing the URL and extracting the parameters was unchanged.
From the service:
func assembleNewURL(url *url.URL) (string, error) {
s := strings.SplitN(url.Path, "/", 3)
From the function:
func assembleRedirectURL(url *url.URL) (string, error) {
s := strings.SplitN(url.Path, "/", 3)
By now the service had been ported to a serverless function and it was working.
I could have been done, but... nn Twitter and later in several other places I fell over this article on good error messages, it is really a good read and worth a recommendation.
I had for the service implemented some pages to communicate error scenarios using HTML, a robots.txt and a favicon.ico.
I set out to do something similar for the serverless function. This however was not as easy as expected since I could not serve files as easily as I could for the service, where the Go net/http package with it's ServeFile made it very easy.
I ended up serving the HTML from strings embedded in the function code, so all is contained in a single file, Go, CSS, HTML and all.
The complete implementation is available on GitHub and the function can be seen in service from:
I decided to use some marvellous web layout for the error messages by Swarup Kumar Kuila (see Codepen.io).
Please let me know what you think of the error messages. I know there are plenty of error scenarios I am not handling, where you get the standard errors from the serverless platform like:
https://pxy.fi/
If you want to see examples of my errors try this, this or this.
I will need to do some more reading and experiementation in order to handle these scenarios, I am thinking whether a load-balancer in front of the serverless function can do the job - perhaps I will try to find out, perhaps I will just leave it as is, since all of this was actually just to avoid exceeding the limit on the size of properly rendered Markdown files on GitHub and I do have other side-projects that need my attention.
Thanks to the DigitalOcean employees helping out on their Slack channel dedicated to the functions offering (and the ones writing the documentation). All of the people blogging and asking questions on StackOverflow, which allow me to find answers more easily.
Feedback, suggestions, corrections and questions are most welcome.
Top comments (2)
I am just echoing a small note I added on another one of my posts:
Since this post was written much have changed, so I am working/writing on an update.
The URL have been shortened to:
https://pxy.fi/<version>/<anchor>for example:pxy.fi/4/wall
I am have eliminated all non-GET HTTP requests
I am working on improving the operational dashboard and the handling of error messages
So lots of stuff to write about, but for now the hands-on part has taken most of my time, that and I gave a presentation (PDF) on my serverless solution yesterday to the Copenhagen Gophers
I shared my blog post with the DigitalOcean employees which had helped me out. They are working on updates to their documentation, so perhaps some of the issues I ran into will be included.
They also mentioned that integration with a CDN for external content could be an alternative solution.
Kudos to @digitalocean_staff