Good Bye Web APIs

When building a single-page application or a mobile application, we usually need to implement a web API (REST, GraphQL, etc.) to connect the frontend and the backend. Technically, it's not very difficult, but it has some unfortunate consequences.

Imagine two planets. The planet "frontend" speaks JavaScript and the planet "backend" also speaks JavaScript or any other advanced language.

Now let's say that these planets need to collaborate extensively to form a whole called "application".

Unfortunately, the planets are unable to communicate with each other directly using their native language and they have to rely on a third party called "web API" which speaks a much less sophisticated language.

Indeed, the language of most web APIs is limited to a combination of URLs, a few HTTP verbs (GET, POST, DELETE, etc.), and some JSON.

The web APIs that speak GraphQL are more advanced but they remain far behind the possibilities of a programming language such as JavaScript:

• The programming paradigm is procedural or functional (no object-oriented programming).
• Only the most basic types are supported (forget about Date, Map, Set, etc.).
• The concept of reference is missing (you can only pass objects by value).

Placing a rudimentary language between the frontend and the backend adds a lot of boilerplate and ruins the development experience.

Another problem is that a web API is an extra layer to worry about. It must be designed, implemented, tested, documented, etc. And all this is frankly a pain in the ass.

But the worst thing is that building a web API generally forces you to degrade the quality of your codebase. Indeed, it's quite challenging to keep your code DRY and cohesive when your frontend and your backend are separated by a web API.

Now imagine that we could get rid of the web API. Imagine that the frontend could communicate directly with the backend using its native language. Wouldn't it be great?

The good news is that it's possible today thanks to a set of libraries called Layr.

Hello, Layr!

With Layr, the frontend and the backend are physically separated (they run in different environments) but logically reunited (it's as if they were in the same environment).

How does it work?

1. The backend is composed of one or more classes whose some of their attributes and methods are explicitly exposed to the frontend.
2. The frontend generates some proxies to the backend classes and can use these proxies as if they were regular JavaScript classes.

Under the hood, Layr relies on an RPC mechanism. So, superficially, it can be seen as something like CORBA, Java RMI, or .NET CWF.

But Layr is radically different:

• It's not a distributed object system. A Layr backend is stateless, so there are no shared objects across the stack.
• It doesn't involve any boilerplate code, generated code, configuration files, or artifacts.
• It uses a simple but powerful serialization protocol (Deepr) that enables unique features such as chained invocation, automatic batching, or partial execution.

Layr starts its journey in JavaScript/TypeScript, but the problem it tackles is universal, and it could be ported to any object-oriented language.

Example

Let's implement the classic "Counter" example to see what it looks like to build a full-stack application with Layer.

First, we implement the "data model" and the "business logic" in the backend:

// backend.js

import {
  Component,
  primaryIdentifier,
  attribute,
  method,
  expose
} from '@layr/component';
import {ComponentHTTPServer} from '@layr/component-http-server';

class Counter extends Component {
  // We need a primary identifier so a Counter instance
  // can be transported between the frontend and the backend
  // while keeping it's identity
  @expose({get: true, set: true}) @primaryIdentifier() id;

  // The counter value is exposed to the frontend
  @expose({get: true, set: true}) @attribute() value = 0;

  // And the "business logic" is exposed as well
  @expose({call: true}) @method() increment() {
    this.value++;
  }
}

// Lastly, we serve the Counter class through an HTTP server
const server = new ComponentHTTPServer(Counter, {port: 3210});
server.start();

Oh my! All that code just for a simple "Counter" example? Sure, it seems overkill, but we've actually implemented a full-grade backend with a data model, some business logic, and an HTTP server exposing the whole thing.

Now that we have a backend, we can consume it from a frontend:

// frontend.js

import {ComponentHTTPClient} from '@layr/component-http-client';

(async () => {
  // We create a client to connect to the backend server
  const client = new ComponentHTTPClient('http://localhost:3210');

  // We get a proxy to the Counter backend class
  const Counter = await client.getComponent();

  // Lastly, we consume the Counter
  const counter = new Counter();
  console.log(counter.value); // => 0
  await counter.increment();
  console.log(counter.value); // => 1
  await counter.increment();
  console.log(counter.value); // => 2
})();

What's going on here? By invoking the counter.increment() method the counter value is incremented. Note that this method does not exist in the frontend. It is implemented in the backend and is therefore executed in this environment. But from the perspective of the frontend, the actual execution environment doesn't matter. The fact that the method is executed remotely can be seen as an implementation detail.

The Counter class in the frontend can be extended to implement features that are specific to the frontend. Here's an example of how to override the increment() method to display a message when the counter reaches a certain value:

class ExtendedCounter extends Counter {
  async increment() {
    // We call the `increment()` method in the backend
    await super.increment();

    // We execute some additional code in the frontend
    if (this.value === 3)
      console.log('The counter value is 3');
    }
  }
}

This is what it looks like when the frontend and the backend are reunited. Pretty cool isn't it?

What's the Catch?

Why does everyone build web APIs when we could do without them?

There is one good reason to implement a web API, it's when you want to expose your backend to some external developers through an established protocol such as REST. But let's be honest, the vast majority of applications don't have this requirement. And if it turns out that you need a web API, it is possible to add it afterward while continuing to use the "API-less" approach for all your internal needs.

Another reason is if you work on a large-scale application with millions of users. Indeed, the convenience provided by Layr doesn't come without a cost, so if you want the most optimized application possible, you'd better go with a lower-level solution.

Finally, if you want to implement a frontend or a backend in a language other than JavaScript, you can still use Layr on one side of the stack, but you will then have to implement an API client or server that can speak the Deepr protocol on the other side of the stack.

Conclusion

Removing the web API allows you to build a full-stack application much faster while increasing the quality of your codebase.

By using Layr on several projects, including some production projects, I was able to reduce the amount of code by 50% on average and greatly increase my productivity.

Another important aspect is the development experience. Since the frontend and the backend are no longer separated by a web API, you get a feeling similar to developing a standalone application, and it's a lot more fun.

Comments

[Magne]

How does Layr compare to Inertia.js ?

inertiajs.com/how-it-works

[Ivan Pozderac]

Personally I don't see it using it for my projects in the near future but this is really cool solution and approach is very interesting.

However I really disagree with this statement:

"But the worst thing is that building a web API generally forces you to degrade the quality of your codebase."

Not sure what you meant with that, but I had very sweet tiny and dry codebase on frontend that used gargantuan REST API. The architecture of API and frontend are important and you can make great apps with pretty slick codebases if you have great developers both on the backend and frontend or if you are fullstack with respectable skills on both ends.

I also written over engineered app that run smoothly like a swiss watch but was very small and applying MVVM was overkill.

[Rodrigo González Castillo]

Too many word to said JSON-RPC

[danielcmorris]

This is basically what SOAP did. I still have some old apps where you just expose the class on the server and the client-side application reads the entire class object and you get to use them. It was awesome and easy. I still miss it.

People complained it was heavy, and XML Based, and JSON was so much lighter.... although now the speed difference is akin to old guys saving space by only storing 2 digit years in the 1900's

Personally, I've completely given up on REST standards with my systems. I've also given up on GraphSQL, which was a nice idea, but I find a bit messy.

As a compromise, I cobbled together something similar to layr, but one step further down the stack. I created a simple object, in which I pass the names of stored procedures directly through to the API server as a package with parameters. A single API clears the bearer token, then reads the procedures requested, builds the statement, appends the token as a parameter, then runs it against the database server, where it executes it and returns it to the API server. The api server then returns an array of JSON to the front-end.

It's actually much in the same vein of Layr, but I bypass the API server entirely (except in anomolies that require more complex work) and send it directly to SQL Server, where security and logic can be handled in Stored Procedures.

Another reason: How long as SQL been around compared to other languages? How many of us have had to update PHP to a very different version, or go from ASP classic, to ASP.net to ASP.net Core, et al. How many went from simple HTML with some javascript, to DHTML, to AngularJS to Angular2?

But still: I gotta say, Layr is pretty cool. If I wasn't running a business and was still just a developer, I'd be loving it.

[Alan Bosco]

So meteor js?

[Manuel Vila]

Please take a look at my answers here.

[⚫️ nothingness negates itself]

REST is web, graphql is not

[Dan Moore]

Sounds like you've reinvented RPC! I remember doing similar things with java RMI in the early 2000s.

That's great, and is good in some use cases. I'm not super familiar with the javascript world, but was there no existing standards that you could leverage?

[Manuel Vila]

No. To my knowledge, the "cross-layer inheritance" ability is unique to Layr.

[Elias]

Sometimes one can think this post would look as kind of "revolutionary" idea but I think this is far to be the case. Post titles like this one usually leads to misinformation.

[Tomas Rimkus]

This reminds me a bit of Firebase callable functions. In case of Firebase, you can only export functions, but not whole classes.
Basically on the backend side (Firebase function) you export the function:

export const doSomething = functions.https.onCall((data, context) => {
  // do something
  return {baz: data.foo};
});

and then you can call it on the fronted:

const doSomethingFn = firebase.functions().httpsCallable('doSomething');
const result = await doSomehtingFn({foo: bar});

console.log(result.data.baz);

I like your way more because it's OOP instead of functional programming. Also, I really like how you use TypeScript decorators for exposing class properties and methods. I just think it would be better if Component was also a decorator like in Angular (Component) and NestJS (Controller) instead of a class you have to extend. This way it would be a lot more flexible.

[Manuel Vila]

I just think it would be better if Component was also a decorator like in Angular (Component) and NestJS (Controller) instead of a class you have to extend. This way it would be a lot more flexible.

It's feasible and I agree that it would bring more flexibility. I'll put more thought into it. Thanks, @hakimio !

[Tomas Rimkus]

Some recommendations for inspiration:

• How mixins are used in NestJS (presentation by NestJS creator): youtu.be/jo-1EUxMmxc?t=2501
• NestJS CRUD module: github.com/nestjsx/crud

Also, I think it would be nice if the frontend library would be framework agnostic and would work with all major SPA frameworks/libraries: React, Angular and Vue.

Anyway, I really like your project and I think it has a lot of potential. Hope, it will get more recognition from the community and you'll have motivation to continue developing it.

[Manuel Vila]

Also, I think it would be nice if the frontend library would be framework agnostic and would work with all major SPA frameworks/libraries: React, Angular and Vue.

Layr is designed to be UI framework agnostic but for now, there is only a React integration.

Thanks for your links and your encouraging words.

[Abod Micheal (he/him)]

I'll stick with api for now but nice library tho, really good idea

[o xelou]

I think this is really interesting and worth checking out right now. I'm also kinda disappointed at how people can be so passive-agressive in the comments right here. Why are you guys getting so personal about it?

[Andrew Robida]

Jesus Christ when someone provides free code can we refrain from bashing it? This might not solve your problem but it could be very helpful to people that want to build web apps quickly that are not really worried about scaling it. Example a website that is primarily for booking clients. It is not necessary to use something like graphql or to build a rest API for a local business to book clients. I am pretty sure barbershops or saloons don't need to worry about this type of scaling. This would be perfect for something like that but serverless is a good option as well. Good job Manuel Vila thank you for sharing!

[leob]

There's no bashing going on from what I can see, just people pointing out the pros and cons of this approach.

[Amer Mallah]

It's not really bashing... He led with a headline and article designed to spark interest, made some broad generalizations in his description and is personally engaging with the debate. His answers are a bit defensive, sure, but I think he (and other noobs who are reading this) are getting a lot of information about the theory behind of separation of concerns and the pros/cons of various remoting tech. I think this both improves his product and educates bystanders!

[Andy Maleh]

You realize that's just a clone of Meteor right? Meteor innovated this idea already many years ago.

[Manuel Vila]

Please have a look at the documentation and see by yourself how far Layr is from Meteor.

[Andy Maleh]

I already did, and didn’t see anything novel compared to Meteor or even isomorphic frameworks in other languages. And since it’s new, I bet it doesn’t handle as many edge cases yet, so I don’t trust it. That’s my opinion, take it or leave it. In any case, I’m a Ruby/Java developer. I don’t do much JS these days anyways except inside Opal libraries/frameworks (Ruby compiled to JavaScript in the browser). Cheers.

[Manuel Vila]

didn’t see anything novel compared to Meteor

Let me give you a hint: cross-layer inheritance.

[Andy Maleh]

Saw that too. Developers could already do that before since classes/prototypes are shared in Meteor anyways with a toggle for writing extra code as client only (or server only), enabling one to do extra inheritance on the client (or server) when needed. And, that was back in 2014 or earlier if I remember right.

[Manuel Vila]

What you're talking about is very different. Layr doesn't share code across layers, it transports the state of the objects when you call a method. To see what I mean, try to implement the example given in the article with Meteor.

[Gerard Klijs]

The main problem with solutions like these I see, is that if something else also want to use the same backend it becomes very hard.

In my opinion it makes a lot of sense to realy on protocols like Rest or GraphQL so you can easily use the frontend or the backend. For example you could move the backend to AWS async, or you can build some widget with Rust. You can't do those things if you tightly couple your backend to your frontend. But for some apps that might not be a problem.

[Manuel Vila]

Please see my answer here.

[Gerard Klijs]

Having actual json for the queries is pretty nice. But it does seem rather niche. Same for similar solutions like fulcro.

[Christopher Wray]

I have to say, that I love this post because of the discusions it has created! Thanks for making it.

Now here is my thought.
You mentioned something about API's being too simple, and that it was unfortunate. From my experience I have loved working with API's exactly for that reason. The main reason I am requesting the server is because I want data. Actually, normally, that is the only reason. Sending back simple json data just makes sense.

I do like what you wrote, and I am interested in checking this project out.