I have a project that has three components:
- A server that exposes a WebSocket interface.
- A GUI that connects to the server.
- Client library that is used in user programs and also connect to the server.
The idea is that the host program controls the GUI through the server.
This is a pet project I barely work on, but I like to dream big. I want the three parts to run on different hosts (which mean I'll need credentials and user authentication, which I haven't implemented yet because I still run them all on the same machine and only connect via the loopback). I also want the client library to be implemented in many different languages, so that they can all use my project.
The Problem
Now, while I'm trying to keep the API simple, adding all the security to each implementation of the client library could be tedious, and if I ever need to do security changes (how often are these necessary anyway?) they'll have to be done in all the implementations.
Another problem is concurrency. My WebSocket API is asynchronous - you send a request with an ID and eventually you get a response with the same ID, but meanwhile other messages from other threads can flow in the channel. This works well for the server and for the GUI, because both use an actor system, but the clients are not going to use actors. This kind of concurrency can still be implemented without actors, but it takes work - I managed to do it in Python (my first and currently only implementation of the client library) but it was a bit tricky and I don't want to repeat this for every language...
The Solution
I'm thinking about adding a relay between the client and the server. The relay will be an executable that client library invokes as a child process and pass it a configuration file with all it needs to know to connect to the server. The relay process will login to the server, and also open a 127.0.0.1-only port and print its number (and maybe a security token?) to STDOUT. The client library will store this information - and that's all it has to implement authentication-wise.
For actual usage, the client library will simply connect to the local port. Since this is a loopback and there are no security constraints (except maybe passing a secret token to avoid other processes on the same machine discovering the port and using it), and because there is no need for a handshake (the relay process will be a single session), this should be relatively fast and cheap, and also easy to implement in any language. For concurrency, I'll just open multiple connections to the same port. Maybe even add an option to invoke the relay executable with different parameters to send a request and receive a response.
Another benefit of this architecture is that in the future I'll be able to implement more advanced features (dreaming big!) without having to implement them for each client library implementation. For example - HTTP tunneling between the GUI and the client via the server.
The Drawback
While the relay executable will make the client library implementations easier to write - it'll also make them harder to distribute. Most modern languages have their own package manager built into their build system, capable of easily adding dependencies written in that language, but adding binary dependencies is harder. Especially if you want to support multiple platforms.
It might be easier to install the relay separately, but that would make it harder for the end users. And while my project is going to be the greatest thing since sliced bread I still don't want to scare people away with complex installation instructions...
Are my concerns legit? What would be the best design decision here?
Top comments (2)
Actually, one common use case I expect this software to have is running the controlling program and the GUI on the same computer. In that case, it'll also make sense to run the server on that computer. My original plan was to have the users run the GUI and the server manually and then have the controlling program connect to the server - which would be a bit cumbersome, but later I could provide a script.
But... if use a relay program, maybe instead of building three separate executables I can build one big executable that can be launched as either GUI, server, or relay? If I do that, I can add an option to launch it as all three - the controlling application will launch it that way and connect to it. This will cause the GUI to pop up for the human user to use. And the server part will just be internal and make the other parts think they talk to each other over a network.
If I pass the connection information to the relay via an environment variable, the controlling applications and the client libraries won't even have to recognize the network capabilities - they'll just run the relay locally, and if the user wants to connect to a server they can just set the environment variable to a connection configuration file.
Hi @idanarye , I think that what you want is a JavaScript SDK with the following features:
If you need to provide a backend SDK you can use the same on both backend and frontend for JavaScript/TypeScript world (unless you need to use backend or frontend specific APIs) and provide platform/language specific SDKs as well (depending on your client's needs).
For the credentials thingy if you use an API key you just need to code a service for creating API keys, storing them in a DB and checking it's validity on each request.
This process will be standalone and the integration with the rest of your code should be pretty easy.
This way you can also control which features your clients are paying for (modularity) and bring some monitoring tool implemented to know and/or restrict the usage of your API based on a given quota and so on.
Not sure if I answered your question entirely, if not let me know.
Best regards