When building a single-page application or a mobile application, we usually need to implement a web API (REST, GraphQL, etc.) to connect the frontend and the backend. Technically, it's not very difficult, but it has some unfortunate consequences.
Imagine two planets. The planet "frontend" speaks JavaScript and the planet "backend" also speaks JavaScript or any other advanced language.
Now let's say that these planets need to collaborate extensively to form a whole called "application".
Unfortunately, the planets are unable to communicate with each other directly using their native language and they have to rely on a third party called "web API" which speaks a much less sophisticated language.
Indeed, the language of most web APIs is limited to a combination of URLs, a few HTTP verbs (GET, POST, DELETE, etc.), and some JSON.
The web APIs that speak GraphQL are more advanced but they remain far behind the possibilities of a programming language such as JavaScript:
- The programming paradigm is procedural or functional (no object-oriented programming).
- Only the most basic types are supported (forget about Date, Map, Set, etc.).
- The concept of reference is missing (you can only pass objects by value).
Placing a rudimentary language between the frontend and the backend adds a lot of boilerplate and ruins the development experience.
Another problem is that a web API is an extra layer to worry about. It must be designed, implemented, tested, documented, etc. And all this is frankly a pain in the ass.
But the worst thing is that building a web API generally forces you to degrade the quality of your codebase. Indeed, it's quite challenging to keep your code DRY and cohesive when your frontend and your backend are separated by a web API.
Now imagine that we could get rid of the web API. Imagine that the frontend could communicate directly with the backend using its native language. Wouldn't it be great?
The good news is that it's possible today thanks to a set of libraries called Layr.
Hello, Layr!
With Layr, the frontend and the backend are physically separated (they run in different environments) but logically reunited (it's as if they were in the same environment).
How does it work?
- The backend is composed of one or more classes whose some of their attributes and methods are explicitly exposed to the frontend.
- The frontend generates some proxies to the backend classes and can use these proxies as if they were regular JavaScript classes.
Under the hood, Layr relies on an RPC mechanism. So, superficially, it can be seen as something like CORBA, Java RMI, or .NET CWF.
But Layr is radically different:
- It's not a distributed object system. A Layr backend is stateless, so there are no shared objects across the stack.
- It doesn't involve any boilerplate code, generated code, configuration files, or artifacts.
- It uses a simple but powerful serialization protocol (Deepr) that enables unique features such as chained invocation, automatic batching, or partial execution.
Layr starts its journey in JavaScript/TypeScript, but the problem it tackles is universal, and it could be ported to any object-oriented language.
Example
Let's implement the classic "Counter" example to see what it looks like to build a full-stack application with Layer.
First, we implement the "data model" and the "business logic" in the backend:
// backend.js
import {
Component,
primaryIdentifier,
attribute,
method,
expose
} from '@layr/component';
import {ComponentHTTPServer} from '@layr/component-http-server';
class Counter extends Component {
// We need a primary identifier so a Counter instance
// can be transported between the frontend and the backend
// while keeping it's identity
@expose({get: true, set: true}) @primaryIdentifier() id;
// The counter value is exposed to the frontend
@expose({get: true, set: true}) @attribute() value = 0;
// And the "business logic" is exposed as well
@expose({call: true}) @method() increment() {
this.value++;
}
}
// Lastly, we serve the Counter class through an HTTP server
const server = new ComponentHTTPServer(Counter, {port: 3210});
server.start();
Oh my! All that code just for a simple "Counter" example? Sure, it seems overkill, but we've actually implemented a full-grade backend with a data model, some business logic, and an HTTP server exposing the whole thing.
Now that we have a backend, we can consume it from a frontend:
// frontend.js
import {ComponentHTTPClient} from '@layr/component-http-client';
(async () => {
// We create a client to connect to the backend server
const client = new ComponentHTTPClient('http://localhost:3210');
// We get a proxy to the Counter backend class
const Counter = await client.getComponent();
// Lastly, we consume the Counter
const counter = new Counter();
console.log(counter.value); // => 0
await counter.increment();
console.log(counter.value); // => 1
await counter.increment();
console.log(counter.value); // => 2
})();
What's going on here? By invoking the counter.increment() method the counter value is incremented. Note that this method does not exist in the frontend. It is implemented in the backend and is therefore executed in this environment. But from the perspective of the frontend, the actual execution environment doesn't matter. The fact that the method is executed remotely can be seen as an implementation detail.
The Counter class in the frontend can be extended to implement features that are specific to the frontend. Here's an example of how to override the increment() method to display a message when the counter reaches a certain value:
class ExtendedCounter extends Counter {
async increment() {
// We call the `increment()` method in the backend
await super.increment();
// We execute some additional code in the frontend
if (this.value === 3)
console.log('The counter value is 3');
}
}
}
This is what it looks like when the frontend and the backend are reunited. Pretty cool isn't it?
What's the Catch?
Why does everyone build web APIs when we could do without them?
There is one good reason to implement a web API, it's when you want to expose your backend to some external developers through an established protocol such as REST. But let's be honest, the vast majority of applications don't have this requirement. And if it turns out that you need a web API, it is possible to add it afterward while continuing to use the "API-less" approach for all your internal needs.
Another reason is if you work on a large-scale application with millions of users. Indeed, the convenience provided by Layr doesn't come without a cost, so if you want the most optimized application possible, you'd better go with a lower-level solution.
Finally, if you want to implement a frontend or a backend in a language other than JavaScript, you can still use Layr on one side of the stack, but you will then have to implement an API client or server that can speak the Deepr protocol on the other side of the stack.
Conclusion
Removing the web API allows you to build a full-stack application much faster while increasing the quality of your codebase.
By using Layr on several projects, including some production projects, I was able to reduce the amount of code by 50% on average and greatly increase my productivity.
Another important aspect is the development experience. Since the frontend and the backend are no longer separated by a web API, you get a feeling similar to developing a standalone application, and it's a lot more fun.
Oldest comments (143)
There is always a catch.
Thanks, @stereobooster , you raise important points.
The server only exposes the type of the attributes and the signature of the methods. So it's no different than a traditional web API. If you change your API endpoints, you break the client. You can make incremental changes though. In case of breaking changes (which is generally not recommended), there is a way to specify a version number when you expose your backend so that the frontend can automatically reload itself.
Yes, all database related methods offer a
reloadoption.The frontend and the backend communicate with the Deepr protocol. For now, you can send Deepr queries manually (like you would do with GraphQL) to solve the n+1 query problem, but in a near future, I'd like to implement sugar to make the operation easier.
Not yet, but it is on the road map.
Nice. This make sense. Deepr looks very similar to GraphQL. Does it mean it has the same "problem" with client side cache resolution? e.g. when you have list of objects and then you update one of them, you need to have resolution otherwise update won't be represented in the list
The cache problem is out of the scope of Deepr, but Layr (which is built on top of Deepr) partially solves this problem with an identity map. So, all object updates should be automatically synchronized. What remains to be solved though is the case of object additions or removals in collections. For that, I need to add the concept of query subscriptions. It is on the road map but it is not an easy task and it will take a while to be implemented.
Traditional APIs are very different. For example, versioning through content-negotiation has none of these problems. Having a server do content-negotiation is a very powerful method to server different agents (for example browsers, or app version, or xxx) different responses, based on their capabilities. This seems completely impossible with Layr.
Even when you use versioning, what happens if you do a rolling update with Layr? That is, you've updated half your servers, but not the other half, with a load balancer in front of it? Will half of it fail?
It also sounds like the entire HTTP Caching protocol would conflict with Layr, that is, Layr will try to do its own synchronisation, ignoring or conflicting with what was previously possible using HTTP Caching. Why is that important? CDN and proxy caching. GraphQL in general suffers from this.
Isn't this conflating use cases of SSR or those where the backend is very similar to the front-end? In the majority of the work I did the past 10 years, the backend and frontend are completely different, and should be. It feels like the authors of the library haven't had good experiences with trying to write maintainable backend and frontend code and ran into these issues (which is understandable, I've had similar issues too!), but are trying to solve it on the incorrect/unfortunate abstraction.
Let's be real. In practice, how many apps have multiple API versions? My guess is that 95% of the apps have only one API version that serves browsers, mobile, etc. API versioning is a pain in the ass to implement and maintain so we usually go with a single API that supports backward-compatible changes.
Layr is made to build web apps, not websites, so HTTP Caching is not an essential feature. It might be supported in the future, but it is not on the priority list.
The author of the library (me) has more than 25 years of experience building full-stack applications.
We write applications for millions of daily users, with clients that can be as old as 2 years (updating them is not in our control). Maintenance of up to 5 versions of the representations of the resources of our endpoints is trivial, because we barely ever have to touch old versions. They just keep working.
I think that the reason so many APIs only have 1 version is because versioning is hard, not because they shouldn't / don't want to have them. So yes, I agree with you that many APIs only have one version, but that's probably more a consequence of tooling not allowing for it, such as this, than that people shouldn't have it.
Having backwards compatible APIs (for example by only adding fields, and never removing/changing fields, which is definitely good practice) doesn't mean you shouldn't/won't have versioning.
That said, the statement that traditional APIs are no different than Layr-enabled APIs is just false. It just doesn't hold up. I don't know why you're stating anecdotal experience or opinion as fact.
Dismissing HTTP Caching because you think web apps don't need it / don't primarily benefit from it means we can't discuss this.
Then your post and this library seems more out-of-place than I thought before.
I think the misunderstanding comes from the fact that we are not building the same kind of applications.
I cannot speak about building an application for millions of users. I've never done that. I build small-to-medium applications and I designed Layr for this use case.
So Layr is probably not good for Stripe, Facebook, Twitter, etc. But I believe it is great for the vast majority of developers that don't work on large-scale applications.
Thanks for pointing that out. I edited the "What's the catch?" section to make it clear that Layr is not made for large-scale applications.
It would be helpful if experienced people stopped making statements that are opinions as if they're fact. This only spreads misinformation and less experienced people are going to take it at face value and run with it.
Good luck with the endeavour.
Yeah, that is a MAJOR massive catch.. That pretty much turns this completely unusable for anything other than meaningless apps that that store a users data in a silo. EG, a note keeping app that doesn't have the ability to share notes between users.
@wormss , I think you misunderstood. Not being able to synchronize clients in real-time with web sockets doesn't mean there is no shared storage. It's no different than any web app, the backend stores data in a database, so any data can be shared between users.
If I am reading your example above correctly, "value" is synchronous, so it would never know if someone else has incremented it? So reading it client side will be wrong the moment someone else adds it.
The backend is stateless, so in the example, the counter is unique for each user. To share the counter among all users, the backend must save it in a database.
And again, I ask how the UI gets to know that it has changed.
When you execute a remote method on an object, the attributes of the object are automatically transported between the frontend and the backend. So, when you change an attribute in the backend, the change is reflected in the frontend.
Some similar thing I seen in Meteor, where server and client code lives together.
Sure, Meteor and Layr share the "API-less" approach, but the way it is implemented is very different. Layr is much more low level. Rather than providing a framework, it extends the capabilities of JavaScript classes to allow remote execution.
Would love to see this integration with php (Symfony in particular but bare php will do it). This sounds very good, the frontend and backend api issue has always been a headache to solve.
That's called RPC, or SOAP, and pretty much everyone hates them. Cross language solutions need to be more verbose because everything must be described. Maybe an opinionated approach could simplify those solutions, but still I'd prefer a data only transfer layer; sure, the typing system is poor, ma probably a solution using protobuf could be the answer.
Interesting approach, though!
I haven't used PHP for years, but I am pretty sure it is possible to port Layr to PHP in an elegant way.
Possible? Sure (it's my first language, I know what I say). Useful? I don't know, it seems to me that the logic shared between two projects written in two languages isn't too much at the end.
While I'm intrigued from a conceptual point of view, I don't think I'll use this approach for a production project.
What about all the boilerplate you can avoid by removing the web API?
The concept is awesome! I apprechiate it.
I will defiantly give a try...
How does this approach address communications from both ends from a security standpoint? Would there be a way to authenticate with RCP?
How can you ensure your client isn't being intercepted a-la man in the middle?
How do you ensure your communication is encrypted?
Conceptually, authentication works the same as with typical web APIs. Instead of passing a token through a cookie or an HTTP header, you pass it through a static class attribute. You can find an example of implementation here: github.com/layrjs/react-layr-realw...
About security concerns, you can expose your Layr backend through HTTPS.
Just out of curiosity, does the framework have parameters for ordering your requests such as etag? That's important for things like versioning.
I am sorry, but I am not sure I understand your question. What do you mean by "ordering your requests"?
Something like this:
developer.mozilla.org/en-US/docs/W...
Then your concern is about caching? Layr is a solution for building web apps, not websites. Caching backend responses at the HTTP level is essential for websites but not so useful for web apps. Layr might support ETags in the future, but it is not something on the priority list.
There are various use cases where Caching is implemented at the API level, like metadata which does not change too often. These are usually cached at API level and returned to clients with out hitting database to fetch the metadata. Do Layr support this?
Not yet but this may be implemented in the future.
Library relies on decorators (I guess previous version of proposal). Current proposal is in stage 2, it will take 6 months to collect responses and get it to stage 3. github.com/tc39/proposal-decorators
I hope so. 😀
The separation of concerns in applications is intentional. I don't think most devs would have a problem with blurring the lines a bit, or further abstracting some of the lower-level minutiae, but there's a purpose behind the separation between the two. The nasty, ugly mess that PHP-based platforms became, and in some ways continue to be is a prime example. Maintaining data separate from presentation lets us not only develop each separately, but also allow some failover. The frontend doesn't care what the backend does. You can step in, completely replace the backend with an entirely different platform, and if the data looks the same, the frontend doesn't care. If you completely swap out the frontend, the backend doesn't care as long as the requests look the same.
It's fine if you want to push a new platform you've found (or made) that shakes things up a bit. That's how we innovate - by questioning what we already have. However, none of that means better paradigms suddenly become obsolete, or that this shiny new thing is necessarily better.
I agree that separating the frontend and the backend is a good thing and Layr does not question this.
With Layr, the frontend and the backend can still be developed and deployed independently. You simply avoid a lot of boilerplate.
see also: SQL is a better API language than GraphQL – Convince me otherwise (twitter.com/simonw)
via news.ycombinator 348 points by edward 7 months ago | hide | past | favorite | 152 comments
more detail on SimonWillison's project docs.datasette.io/en/stable/
I think that using SQL as an API language is a terrible idea but it is nice to see someone arguing against GraphQL. 😉
I think SQL is a brilliant idea, providing of course the proper constraints are in place. Simonw goes on to show how it actually works based on his own experiments--he was skeptical too. I imagine it really depends on the application. And I agree with the author that GraphQL is reinventing the wheel while not appearing to improve on REST for resource usage (perhaps it's easier to develop relative to rolling restful and rpc endpoints--initially, not sure about maintenance, I think it's a wash or worse there). See also Owen Rubel's related answer--I tend to totally appreciate his insights, he has a lot of good material in this area: rest vs rpc; API chaining part 1of2 and part 2.
Thanks, @jimmont , I will check this out.
If I understand this correctly, the client pushes it's state over the "wire" to the server. Isn't this what ASPX Forms did in the past? Everyone loved the convinience but everyone ran away at the end?
ASPX is almost 25 years old and it was build for wired intranet and failed with the advance of the public internet. One of the reasons was that the client had to transmit often too much data and that was slow and had other problems as well. Back then java script was not easy but js evolved just to be able to drive a client's inteligence within the browser and only ask for the data that is necessary to drive the inteligence. ASPX Forms was also customizable and you could control what goes in and what goes out but we all know how it goes. Even before google development became a thing, people would just copy code if it seemed to work somewhere.
Obviously things are different nowadays and MS is doing something similar with the Blazor, where you get to choose if "server-side" rendering happens. As far as I understood Layr, doesn't do server side rendering, only server side processing.
I personally consider a frontend and backend API the same. I don't understand why people make these distinctions. Same nfr rules should apply, like versioning. The only difference is who is the audience of the API or what is the business it tries to serve. With front end, that is data optimized for one audience (the client) and the backend is for all. This difference changes many of the nfr and lifecycle as well but conceptually it is the same.
I always consider layers important. Maybe not convienient to developers but still very important with longivity in mind. Layr takes this transparency away and that makes it a problem for this requirement. I expect that, for most with architectual background, removing/hiding the FE API would raise red flags.
The counter example is very easy and doesn't automatically raise the alarms for many but experience does and my advice is to market this with more complicated examples and make sure that you show that primarily you are in control of what is put in the wire. Gut feeling, this code will lead you back to the
I hope you find the feedback useful, even if I'm not correct. History has lots of good paradigms to learn from. I'm not this is silly. God knows when I was young some attempts I had to framework some things and I learned from the effort, the feedback, the failures and the success.
Thanks, @safarian, for this long comment but I'm not sure to fully understand your point.
Except for the ability to expose a backend to some third-party developers, what are the benefits provided by an API layer?
Structure, control, real segregation of concern.
I'm not sure as what you are asking. I mentioned that BE/FE API are both important and if you try to hide/remove the FE API on the principal of developer convienience, then many architects would get worried.
Very simplified, just because it is FE API, it is not there to remove because it is inconvienient. :)
Though some words might be considered judgemental, I'll remind you that they are not used like that.
I am sorry but you didn't convince me. I can agree that for some complex projects with a large team an API layer can provide some useful guardrails, but for 95% of the web apps, I believe that an API layer is just an unnecessary burden.
In my original comment I mentioned
long termand there is our different in mental approach and probably our professional engagement.There is no argument against the context you describe but I consider them under the classifications of "quick and dirty", "fire and forget".
I'm not trying to convince you. Just sharing experience: ) :) . I probably wouldn't have convinced my younger self as well.
I won't bet you're older than me. :)
You and everyone who commented raised important concepts! Thank you!
I like your approach to the problem but I'm not sure there was a problem with web API, yet. But I'll dig deeper and I'll try Layr for sure!
Are we trying to recreate JSP and PHP again?
It even looks very much like .Net Remoting to me !
Except for the "API-less" approach, Layr is very different than .Net Remoting. Layr removes the API layer but it keeps the client/server (with a stateless server) architecture of web applications.
Indeed 😄, I got that, one cannot really compare it with an almost pre-web techno, but the feeling of it remains (it looks like it ... attributes, proxies & stuff).
I guess that's something that object-oriented RPC libs have in common.
Not at all. Layr is quite the opposite. The frontend (UI) and the backend (data model and business logic) are physically separated.
With NextJS getServersideProps you can write directly server code in your frontend React component, you could also directly talk to a database.
Next.js is great to build websites but not so great to build web apps. Layr puts its focus on single-page apps (with no server-side rendering) and mobile apps.
I disagree, Nextjs is perfect for web apps and you don't need ssr because it supports csr and ssg also.
The whole point of using Next.js is server-side rendering. If you don't need SSR, I think you'd better build a regular single-page application.
The whole point was SSR not anymore. Nextjs 10 is different
actually, seeing all the decorators, it absolutely looks like implementing an api. looks pretty much like nest.js.
how is it actually with data? does this keeps everything in memory? is there a seperate database?
The backend is stateless and all data comes either from the frontend or the database.
Sooo like Fulcro/Pathom?
Unfortunately, I don't know much about the Clojure ecosystem, but I will check these libraries out, thanks!
This is interesting, but the way you're presenting it doesn't work - the messaging is too negative, which I think pushes people AWAY from a potentially very interesting solution ... :-)
Basically, you're telling people "Web APIs suck, you've been doing it wrong, my solution is superior" ... I think psychologically that doesn't work - people will reject your stuff because they're perceiving you as unfairly trashing REST and GraphQL, meaning they'll be biased against it from the start.
So the problem is you're focusing on "your stuff is bad" rather than "my stuff is good". It works much better if your message is a positive one, rather than negative.
In other words, don't tell people "Web APIs suck", tell them that you've developed a great full-stack framework which can (potentially) vastly improve a developer's productivity. THEN you'll get them interested and they'll start listening to you.
(my first reaction when reading your article was, okay, so what is this - it's just RPC - remember CORBA, remember Enterprise JavaBeans, remember SOAP, remember Java RMI and so on? All of that was "RPC", and we moved away from it because there were problems with it ... so that gave me a serious "deja vue" feeling)
Just my 2 cents :-)
You should read the post more carefully.
Hold on, I've read your post, but more importantly I've modified and expanded my original comment - read it again :-)
The problem (as I try to explain) is not that your framework isn't good, I think it's very interesting. The problem is that your message comes across as negative, as unfairly trashing REST and GraphQL. I think that hurts your goal of getting people interesting in your solution.
I've looked at your docs and the fact that you don't have an explicit API layer but "direct object execution" is only a part of the whole thing, all of the features built around and on top of it are probably much more interesting.
People aren't interested in you telling them that they've been doing it wrong, they're interested to hear what value you can add - and looking at your docs, adding value is what you're able to do.
I'd say dump the polarizing title "Goodbye Web APIs" and the divisive message "Web APIs suck". Well as clickbait to get people to read your post the title does work, but if you'd change it to "Web APIs? Maybe there's a better way" then people might be more inclined to take you seriously.
Thanks, @leob , for explaining why you rejected my post in the first place. I agree that the title is a bit "click-baity". But I don't regret it. It's sad, but it's the only thing that works today.
I understand what you're saying, and you do you, but I don't think I'd use a budding technology solution with someone who openly has this negative mentality at the helm.
"It's the only thing that works today."
"You should read the post more carefully."
These are not good responses to carefully crafted, constructive criticism. If you look at the likes for @leob 's comments vs. the likes for yours, I think it's pretty apparent that this isn't the best approach for you to take.
Well you're absolutely right that your click-baity title worked, because your post attracted a lot of views and comments. But at the same time I think it's off-putting to many people, which is a shame because I think the work you've done is very interesting. Anyway just my 2 cents, no worries :-)
Couldn't have said it better Rob Sherling, this is exactly the point - the project might be fantastic but someone with a negative or defensive attitude at the helm is going to put people off ... really what I'd sincerely advise is, flip the switch, lose the negativity, and the project might be doing well and might attract enthusiastic users and collaborators!
Thanks, @leob , I got your point. I wish I could use a less provocative tone, but I feel this is a necessary evil to get this project off the ground. This is not the first I write about Layr (see my previous posts). I used a more consensual tone then, and it didn't work at all.
Well you're right about that and you absolutely do get attention this way - you see that people start discussing, no doubt about it. So well yes, maybe sometimes this is the only way ... but, now that you do have the attention, maybe you should consider trying to change the tone of the discussion - make it less provocative :-)
OK. I'll do my best! :)
Haha good, you rock, you're a star!
The title can be click-baity but the content doesn't have to be.
True and that's the case, to a degree :-)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.