The Problem
Well, it is no news for us that developing software is expensive, and takes A LOT of time and effort, after all, we're talking about code, good code.
It is that kind of code which solves a problem, and it does so while being elegant and readable... But doing that is hard, and here is where we touch the surface of a difficult problem.
Good code is really expensive, and inaccessible for most people and companies.
Not only bad/faulty code often fails to solve the problem it should, it also becomes the own problem. I bet you once had to understand the incredibly messy class system of a legacy code base, just because you were tasked to fix a "simple" bug.
My theory
I have a fun personal theory, that all ever created about code can be somewhat traced back to the code accessibility problem, with the sole purpose of making it bit by bit a little easier.
People started using C so they do not need to move bytes manually so often. On the web, the rise of JQuery led to more and more complex and capable websites, which was a thing only to dream about. Then came Java, C#, ReactJS, tools like Docker, and uncountable others goodies to make our life better.
However, the "writing good code" problem still persists, though adapted to the current scenarios.
Honestly, I don't think this problem has a definitive solution. As our society and technology gets more capable and complex, so does our problems; but I do think we can do better to make software more accessible.
Making code Accessible and Cheaper
Now, what if such "good code" could be written only once, and all of us could freely use that code for our own purposes? What if we can chain multiple good codes together, while not needing to write boilerplate code for that?
Perhaps, even better, we could be writing only the absolutely necessary code for our use case.
Here's where that JSON comes in.
Code as Data
Before we begin, for "code as data" to exist, the following must be true:
If It is possible to represent any code as a piece of information, it is possible to organize it to compose features and functionality.
Luckily for us, data can represent anything.
Don't believe me? Try giving this a read:
{
"variables": [{ "name": "highestAllowedNumber", "value": 3 }],
"code": [
{
"procedureName": "if",
"boolean": {
"procedureName": "higherThan",
"input": "functionInput1",
"targetValue": "highestAllowedNumber"
},
"then": { "procedureName": "stdOut", "message": "too high!" },
"else": { "procedureName": "stdOut", "message": "you're fine." }
}
]
}
When the user input is higher than 3, we will get a message: "Too high!"
Now, perhaps you're asking yourself, what is the difference from this to regular code?
At first glance not too much, however, in practice they are fundamentally different. You cannot execute this data, but you can parse it into code (good code too!), then execute it.
Data as Code
If we think about it, a good part of programming is converting information of a business process into a language the computer can more readily work with.
Try making this simple exercise. Get that data from the last section and write it in JavaScript, then C#, then C++.
After completing it, I think you can somewhat visualize that we can tell a computer make this conversion for us.
Data and You Making Code Accessible
Oversimplifying for the sake of comprehension, let's say you've written the best, unrivaled, if statement there is, and it could be represented by the same data structure we saw above.
If we manage to get data which correctly represents our intentions, it can become the best code we have, while not even thinking about its implementation at all. For best results, make it open source.
How a single JSON file could become your entire code base
Right now, I bet the title is not that far from reality, comparing what it initially seemed to be for you. Well, actually, such thing already exists, and here's a WIP example.
This was made possible by using Meta-System, an open source software in which I had the pleasure of working on. It makes software accessible, while also providing you a way to contribute to such accessibility.
Check the Repository, and join the discord, where we talk about making the world of software more welcoming and less challenging.
Oldest comments (48)
Awesome, I'll have a better look and join the Discord!
Code is data. That's why arguing about layout is so silly. Ultimately we're just writing a code dom using a user friendlier syntax. Metadata as baseline code is what we need more of.
I’m exploring something similar to this in github.com/samhinton88/code-fold
Quite interesting project, and a related concept too. Keep up :)
I had the privilege of you convincing me of this concept on Discord. I'd love to see this pick up and widely used to build reproducable services. Keep it up!
Please don't do this without typescript and strong types.
At first config objects seems like a great idea to simplify and write less repetitive code. However, leaning too heavily on them can lead to rigid janky code to handle new cases and edge cases.
The dev experience becomes terrible when your coworkers have to read through hundreds of lines of config object parsing methods.
The time you initially thought you'd save is lost again to debugging and trying to find some undocumented setting or side effect that's not working as you expect.
IMO config objects are best when they are simple, self documenting, and no more than one or two layers deep
Edit:typo
This is absolutely a great call! Given the amount of data that can be condensed on such objects, skipping one or 2 props will eventually happen. Having typescript (or any other type system) is a MUST.
FYI, Meta-System is made with typescript under the hood, and I'm glad I used it.
Nice! If your config objects are created and consumed strictly by software (which I assume is likely the case if you're doing language translation) then these problems probably won't put as much burden on developers.
My feelings mostly come from projects I've implemented where developers were intended to create the configs. Not having strong enough types as a POC lead to needing a big refactor to improve discoverability of features for developers.
Common case of a POC mutating from "hey this could save us time" into "all my time is spent maintaining and documenting" 🤣
You're 100% correct, the idea is nice and will definitely save some time at the beginning, but with time and team growth, it becomes quite a mess specially if there's no proper documentation on how to use such files, and no one documents their work... Right now I'm working for a company that uses this approach, with old javascript split in different repositories, it is really impossible to debug and sometimes I take days just to find the right piece of code to update.
Using typescript for this seems a very good idea indeed ... JSON-schema maybe?
And I can imagine that at some point you'd also want some sort of a graphical editor, so that you can edit a "model" in a meaningful way (MDD, model-driven development) ...
I must say that when I saw the big blob of JSON that's needed for even a pretty simple app it gave me a bit of a headache, I can't imagine that it will really be a joy to "program" like that. Some sort of higher-level editor support would alleviate that ... and the ability to split the JSON declaration into separate files or "components".
The way I could really see this working is that you'd use a JSON model to generate the repetitive, boiler plate kind of stuff within your app, then the more "interesting" logic would still be coded in a conventional programming language.
Well this does look like MDD (Model Driven Development) all over again.
I can definitely say that we're on the same page :)
All of this is on the Meta-System's Roadmap, which you can check it here.
I see it! "Meta-System configuration can be split in multiple files"
I worked at a multibillion dollar company that had a single Google sheet control software operations. It was done this way so finance, sales, and C-level emoyees could configure our platform. Even if it was only dev controlling it, the adapter code for it grew exponentially with each update request. It ended up being a nightmare for everyone and eventually the company restructured communication to avoid that mess in the future.
Yeah, I think this emphasizes the problem of "good code" accessibility, It's hard even for a multi billion dollar company.
For this specific kind of issue though, I'd say the problem was with the lack of a predictable interface. Can't say for sure since I don't know the code...
I’ve been moving one of my react projects to a similar predetermined or templated structure using firebase and a “Core” collection. Basically like a schema for your schema. It’s a lot of work upfront but definitely pays dividends in the long run not having to chase down irregularities with how your data interacts as your project grows or changes. Also very helpful with data relationships as changing how those relationships function or are defined is as easy as changing the Core schema.
Word of caution though, abstraction always comes with the temptation to further abstract and can become a time sink for no actual improvement. I know because I’ve ended up there at times. Don’t let that dessuade you though as I’ve learned an invaluable amount about Js and React data handling in the process.
Don’t play too hard!!
I'm having a similar approach. I have a few predefined/builder components and I define layout and hierarchy in JSON. With a single instace of this codebase I serve 100+ completely different websites, with a 150k React bundle size, including styles!
This is an amazing work! Keep up <3
Love this concept. I've been working on a "Code as JSON" project also which you might want to check out! github.com/satelite-digital/engineer
Yes, you can make a Turing-complete programming language whose syntax is a subset of JSON. Your if statement example demonstrates this nicely.
But what problem does that solve?
You can write any program as a JSON data structure, but you could write that same program in C# or JavaScript and it will be far more readable and succinct.
JSON can be used to represent just about any kind of data, including a computer program, but it's not optimized for this. Like JSON, C# and JavaScript can be viewed as formats for storing data — the difference being that C# and JavaScript were designed specifically to represent computer algorithms while JSON was not.
Couldn't agree more
Hey Sam, thanks for coming up with this!
Well, I do agree with you. For simple tasks, such as the
if statementin the article, it surely can take up more space than we would usually need with our own and beloved usual programming, and this is a good sign of the development of good languages for writing such rules in an easy manner.I feel like data cannot simply replace our traditional programming, this is impossible given that the data needs to be eventually resolved in code. However, the great usability of this comes when we can abstract good chunks of repetitive logic in a simple API. If we combine that with a predictable API across multiple modules/libraries, we have come up with a way to greatly reduce programming effort.
Now, whether we want to represent our logic with data or not is up to the developer, and it also requires a good understanding of the problem (and its complexity) we are trying to solve. If we have enough flexibility within the data model, or the code is organized enough, this is likely influence our decision.
I love this concept, but it would benefit greatly from being more extensible and allow for higher level features. Client needs such as "Render a paginated, filterable, sortable list of objects" is the kind of need which absolutely should be programmable by config.
I think that when you start implementing loops and branching with your JSON, you've gone too far in a weird direction.
You would love to see Meta-System in depth then! We made the core to be extensible as you wish, so you can implement such high-level features and use them in your JSON file. Come say hi at the discord, I love to discuss about this sort of things :)
I started tinkering with a similar concept for blazor. Write a bunch of json files to create components. Handles state fairly well. Much more form oriented than whole website type of thing. Allows some json to be injected so that users can create forms on the fly, validated and possibly passed to some api for processing.
so a compiler? I don't see why we need json to write a compiler that operates on a high level easy to understand language.
You are just reinventing what Json schema and hyperschema are already offering. This design has been utilized in the industry for almost two decade and commonly referred as config driven development.
I would stop programming and go find another career if I had to deal with contrived indirection like this which I believe was ultimately conceived not to democratize the process of algorithm creation (which becomes an enjoyable process once one achieves flow) but to exclude “the unwashed masses” from “mucking up our pristine code wonderland” the original intention of abstraction-in-the-cloud seldom serves the noble goal of abstraction as labor saving device and is increasingly observed as a mechanism for implementation of INaccessibility (while perhaps not the direct intention of this author) is a theme often manifested in the web programming world where life is easier than all other manifestations of software engineering.
Meanwhile in the engineering world where software devs meet electrical engineers and summarily get dehumanized merely for being software specialists, things like the downsides of IF statements do not happen because the comparator is an elegant construct.
We’re often battling to justify our work to engineers of physical systems who feel as though we make nothing but fluff.
Then I get recommended articles like this…
No, procedural languages are beautiful, C,C++,JavaScript,Python,Rust and so on are beautiful and the art of programming is also beautiful.
JSON is also beautiful but I’ll not be making a programming language out of it.