API Framework, the moving parts

raisr — Thu, 11 Nov 2021 21:55:19 +0000

In my last post "Starting my API journey..." i wrote about my plans to create a framework that eases the development of web APIs.

In this post i'm going to outline my current ideas about the parts the framework should include and how they relate to each other.

So what parts should an API framework include? Of course, my ideas are very opinionated. But this are the parts that always made my life as a developer hard.

Gateway

The gateway acts as a reverse proxy that builds the boundary between the clients and our services. When a client calls an endpoint the gateway redirects the call to an internal service and returns the response. If desired it can provide additional features that so that we don't need to implement them in our services.

Possible additional features are:

Throttling
Authentication / Authorization (using an identity provider)
SSL Termination
Analytics (using a telemetry service)

In my framework i will use YARP. Yarp is an open source reverse proxy built by microsoft. You can read a short summary about YARP in the announcement post.

Logging / Telemetry

Logging, telemetry and analytics are an important part for every API. Identifying problems and bottlenecks early is important to keep our customers happy and the system healthy. The telemetry service should handle task like:

Storing and analyzing logs / errors
Provide usage statistics, execution times, etc.

For the logging part i will use serilog.

Identity Provider

The most APIs need some kind of authentication / authorization. Unfortunately this is one of the hardest parts. There are a lot of libraries / frameworks out there to do this. But either they are not able to handle multi tenancy or they are very complex. And mostly you need to have more security know how than you want. I want to provide a security solution that is easy to use and provides a lot of features. I think this will be the hardest part in my framework.

I want this part to provide the following features:

OAuth2 / OpenID Connect workflows
API Key Authentication
Session based authentication (so that we can use it for simple websites)
Multi tenancy support (one user can have different credentials for different APIs)

The biggest problem here is that there is no real open source solution that could provide this features. Microsoft does not provide something good. There is ASP.Net Identity with no support for OAuth and multi tenancy. So i guess i will go for Duende Identity Server. It's not really free, but open source projects und small companies will not have to pay for it. It is really sad that there is no really open source alternative in the .Net world like Keycloak in the java world. If you know an alternative just leave me a comment.

Task Scheduling

Task scheduling isn't really a crucial part of an API framework but it can make your life very convenient. Often you need to run tasks on a regular basis. Then you start creating shell scripts, batch files, etc. that you trigger by the windows task scheduler or linux cron jobs. It is far more convenient to trigger an API endpoint on a schedule.

For this part i will use hangfire.

Configuration

We have to deal with a lot of configuration settings in our projects. Usually we store stuff in the configuration files of our APIs (web.config, appsettings.json, etc.) or in some database tables. Using config files has the disadvantage of redeploying our services when settings change. Storing the stuff in the database of our services clutters everything. The goal is to provide a central configuration system that you can use. Currently i have no idea which library i can use. Maybe you have some ideas.

Common UI

All the features are very nice. But we want to view logs and telemetry data. We want to configure the gateway, identity provider and all the other parts. Of course we can do this by editing database tables or config files by hand. Ok, no! There should be a UI where we can do all this stuff. For this i want to create a Blazor SPA that provides a UI to do all this.

Next stop

So whats next? Until now i didn't talk about naming. One of the hardest parts in software development :). I will think about some name and then create an initial github or gitlab repo.

After that the first thing i will implement will be a simple gateway and telemetry (logging first) solution.

^{Photo by Thomas Hawk. License CC BY-NC 2.0}

Starting my API journey...

raisr — Fri, 29 Oct 2021 10:25:56 +0000

During my career i mostly worked in small to medium companies. In most of them i had to create some kind of web based apis for a lot of different use cases. Being it as a backend for SPAs, public facing apis for customers and so on. Creating a simple rest api using the asp.net stack is quite simple. But very soon you have to tackle a lot more complicated / sophisticated problems like Authentication, load balancing, routing, configuration management, scheduled tasks and so on.

There are a lot of free and or open source tools, libraries and frameworks that help you to solve those problems. But then the real challenge starts. The available solutions a often very powerful, written in different languages and require some effort to get started with. So you have to integrate them, manage the deployment / hosting on different OSes and keep all of this up to date. That is extremely challenging for a small to medium company. You just don't have all the skills on your team or the time and budget.

So, what can i do? I want to create yet another framework :)

No, i don't want to reinvent the wheel. My goal is to create some kind of framework that incorporates existing tools where possible and hides the magic behind but gives you the power to extend if required. In the end you should have everything at hand to create apis that provide your business value without havening to much to care about the rest of the infrastructure.

So what does that mean. Lets say you want to have a logging solution. Then you could just do something like this pseudo code.

   app.UseLoggingServer(); // Enable logging server on this machine
   app.UseLogging("logging server uri"); //Send logs to the logging server
   app.UseLoggingUI(); // Provide some ui to show the logs

Under the hood it could use for example serilog. This code would configure serilog with settings that should be sufficient in most cases. But if not you could change the serilog configuration directly.

These are some of the key aspects that i want the framework to follow:

Modular

Cherrypick what you want. Just need an api gateway or logging? Pick this from the framework.

Scalable

It should be possible to run everything within a single web application or scale out stuff that is hitting the limits.

No rules for your code

The framework should handle the infrastructure stuff. It should not force you how to write your code in any way.

OS agnostic

The whole framework has to run on windows, linux, mac os.

Here my journey starts...

^{Photo by Clemens van Lay on Unsplash}

DEV Community: raisr