The Problem
Well, it is no news for us that developing software is expensive, and takes A LOT of time and effort, after all, we're talking about code, good code.
It is that kind of code which solves a problem, and it does so while being elegant and readable... But doing that is hard, and here is where we touch the surface of a difficult problem.
Good code is really expensive, and inaccessible for most people and companies.
Not only bad/faulty code often fails to solve the problem it should, it also becomes the own problem. I bet you once had to understand the incredibly messy class system of a legacy code base, just because you were tasked to fix a "simple" bug.
My theory
I have a fun personal theory, that all ever created about code can be somewhat traced back to the code accessibility problem, with the sole purpose of making it bit by bit a little easier.
People started using C so they do not need to move bytes manually so often. On the web, the rise of JQuery led to more and more complex and capable websites, which was a thing only to dream about. Then came Java, C#, ReactJS, tools like Docker, and uncountable others goodies to make our life better.
However, the "writing good code" problem still persists, though adapted to the current scenarios.
Honestly, I don't think this problem has a definitive solution. As our society and technology gets more capable and complex, so does our problems; but I do think we can do better to make software more accessible.
Making code Accessible and Cheaper
Now, what if such "good code" could be written only once, and all of us could freely use that code for our own purposes? What if we can chain multiple good codes together, while not needing to write boilerplate code for that?
Perhaps, even better, we could be writing only the absolutely necessary code for our use case.
Here's where that JSON comes in.
Code as Data
Before we begin, for "code as data" to exist, the following must be true:
If It is possible to represent any code as a piece of information, it is possible to organize it to compose features and functionality.
Luckily for us, data can represent anything.
Don't believe me? Try giving this a read:
{
"variables": [{ "name": "highestAllowedNumber", "value": 3 }],
"code": [
{
"procedureName": "if",
"boolean": {
"procedureName": "higherThan",
"input": "functionInput1",
"targetValue": "highestAllowedNumber"
},
"then": { "procedureName": "stdOut", "message": "too high!" },
"else": { "procedureName": "stdOut", "message": "you're fine." }
}
]
}
When the user input is higher than 3, we will get a message: "Too high!"
Now, perhaps you're asking yourself, what is the difference from this to regular code?
At first glance not too much, however, in practice they are fundamentally different. You cannot execute this data, but you can parse it into code (good code too!), then execute it.
Data as Code
If we think about it, a good part of programming is converting information of a business process into a language the computer can more readily work with.
Try making this simple exercise. Get that data from the last section and write it in JavaScript, then C#, then C++.
After completing it, I think you can somewhat visualize that we can tell a computer make this conversion for us.
Data and You Making Code Accessible
Oversimplifying for the sake of comprehension, let's say you've written the best, unrivaled, if statement there is, and it could be represented by the same data structure we saw above.
If we manage to get data which correctly represents our intentions, it can become the best code we have, while not even thinking about its implementation at all. For best results, make it open source.
How a single JSON file could become your entire code base
Right now, I bet the title is not that far from reality, comparing what it initially seemed to be for you. Well, actually, such thing already exists, and here's a WIP example.
This was made possible by using Meta-System, an open source software in which I had the pleasure of working on. It makes software accessible, while also providing you a way to contribute to such accessibility.
Check the Repository, and join the discord, where we talk about making the world of software more welcoming and less challenging.
Latest comments (48)
You are on the road to making a Lisp-like system ;) . This article from 2006 goes on a similar journey starting from XML of all things to show the power of data as code which is the fundamental principle of lisp.
I find the proposition a little confusing I admit. Data is code, and code is data, always was always will be, which is more or less the fundamental proposition of IT from the outset. What new observation is being tabled here? I'm not so sure. There is also nothing new in abstracting code so it can be translated into another code easily. Which is all I see in the JSON example is that, a new code defined in such a way that other codes can easily be generated from it.
I mean by all means, such a thing is an interesting idea and if you can justify developing one, go for it and see if you can win uptake and adoption. But make no mistake code is data and data is code, and all you are doing is defining a new language: Meta-System.
If 10 years hence it proved to have been a success (the way Python has for example), it will just be seen as the new language on the block "Meta-System", perhaps with a new name by then or not. As it is just a new language on the face of it.
Or, by all means, let me know if I've misunderstood.
Hell no!
This is just Lisp but without parens.
I think the idea itself can be good, I don't think I personally like this approach though. There are real languages out there that aims to provide features like this, for instance functional programming languages like Haskell, OCaml etc. The code is very much structured like data and provides much of the same benefit, while still being actual code. I can see a few problems with this implementation, at first glance.
Any programming language today, does not involve programming "against the computer", meaning, we don't write actual binary machine code. We program against so called abstract machines (the C++ standard for instance is a good resource to check out, if one wants to learn more). As such, generalizing this "code as data" is hard. Is it impossible? No. But suddenly we will be left with config files for config files, and that's just one level of indirection, that I could think of, off the cuff. With "normal" code, compilers do this, they are essentially code generators, with the difference being, when we need to, we can delve into the actual code and handle edge cases. With "code as data" this becomes pretty involved. Reading package.json for a NodeJS project today, is more cumbersome than following actual code for instance. Even though to the computer - everything is data - to the person reading it, it isn't.
Reading code is what we all do and what we all are good at. It's "easy" to step through, it's "easy" to build mental models (when it boils down to functions, or some part of some system). With code as data, like this, suddenly "following the thread" becomes harder. How would we debug a system like this? What code would be generated? The problem with many of today's "code generators" is that debugging them becomes incredibly hard. One of the many questions become, "what goes where?". Suddenly we've introduced (or at least made a problem harder than it was before) problems we did not have before.
We have no up front way of evaluating the product. We have no "debug builds" or "release builds" or "release with debug info" builds. We just have the spit out code from the system and that's that. This I think is a major problem. This will lead to bloat (like any framework today does, I don't think I've ever seen a framework that doesn't involve pretty substantial amounts of bloat), unforeseen inefficiencies and so forth. And in the end, I think we'd all be able to blow our own foot off with a system like this, just like we do with "normal programming" on a regular basis. Part of our jobs are fixing our own mistakes and I think code as data, doesn't prohibit us from making those mistakes.
It's an interesting approach none the less, even though it's not in my taste so to speak.
The example you give is not "Data as code", is just code. A more declarative aproach shouldn't have instructios
I'll tell you what. Our company had a similar idea to create what today would be called a "low code" solution: a drag and drop interface for building business web-applications. Page definitions are nothing more than JSON, you can think of it as a React application, but the JSX is expressed as only JSON. Several of these apps are deployed in production and do fairly well. The main benefit is that it is fast to configure something new and it is great for standardisation (since only a limited set of building blocks can be used).
Now what you might expect: as business requirements become more complex, things get nasty real quick. You will need to define all sorts of conditional logic inside the JSON tree to fulfill all those crazy business requirements. The JSON structure becomes a programming language of its own, which is impossible to debug, let alone do proper version management (merge conflicts...). When things get too complex you fall back to just writing JS again, so why bother in the first place?
Oh and by the way, since this is all proprietary stuff, you need to maintain the application builder as well. New developers need to learn this highly non-standard tooling.
This sort of stuff is great for letting the end-user create dashboards with simple configurable widgets, but building whole apps this way is a Sisyphean task and can be a huge risk. We're moving away from this approach and writing apps the 'old fashioned way' now that the whole React ecosystem has become more mature.
Oh I'm working on something very similar myself but a slightly different spec than you have here. I almost think of it more like js arrays written as a LISP. Very similar concept just written with less keys etc and using functional paradigms.
Reminds me a lot of the 4GL's of the 80s, or of "Model Driven Development" (and it sounds a lot like the "no code" or "light code" movement too) ...
But, I do see the value of the idea of 'automating away' the repetitive, boiler-plate like parts of coding/development, and of replacing "imperative programming" by "declarative programming".
The point is - this has been tried in so many different ways, shapes and forms (see my opening sentence), but reality is we're still writing code "in the old way".
But, never say never, who knows how we'll construct software in 10 or 15 years from now - I can't really imagine that we'll be doing it exactly as we are doing it now, by endlessly coding our React hooks and components and whatnot - I do expect a large part of it to be done somehow in a more "high level", abstracted, declarative, model-driven way.
Nice article, and definitely you are on a point too little considered by the community that generate it.
I have touched code as data on a recent project. It may be interesting for you since it show another perspectives on that.
You may look at: github.com/HRI-EU/JSEN
You may also have a look at an article on that: ronpub.com/OJWT_2021v8i1n01_Ceravo...
This is called an Abstract Syntax Tree (AST) but why would you want to write out raw syntax trees by hand?
If for some reason you needed an AST, you could probably create a real syntax for your language and have a compiler mode that emits the syntax tree as JSON.
You're kind of starting with the middle state of a language/compiler and working your way backwards. This idea is very much missing a "why".
The idea is to represent the code as data not to write the data by hand. Nor to read it in JSON format. This aproach enables projectional editing (term popularized by Intellij in their MPS) that has much more potential then traditional text editors.
Why? Because we can write the code once and generate multiple versions of binaries choosing implementation language that suites the best the target evironments with their constraints. For example runs in the browser and also super fast on the desktop.
I actually faced this when migrating a gui component from SWT (Java) to Javascript). It felt such a waste of time to rewrite everything.
But it's still a language.
What you put in the JSON is effectively just a serialized AST for a specific language. Yes, it's JSON, but it's not just JSON - the structures and values have syntax and form that goes beyond JSON, much the same as how source languages go beyond ASCII or Unicode.
Plenty of compilers transform the AST in the middle, applying optimizations and so forth - applying transformations to JSON isn't fundamentally different from applying transformations to an AST. It's a model, with a specific schema, and you transform it. The back-end of the compiler ultimately transforms the final AST into some other source language or binary form.
That just explains why we have high-level languages and compilers.
From what you've explained, all you're proposing is a high-level language with a syntax that happens to be a superset of JSON.
I mean, yes, you'd be able to parse it using a standard JSON parser - it would work sort of like a high-level lexer, but you would still need to build all of the remaining parts of a compiler. (And probably a highly specialized editor as well, since working with these bulky JSON structures in a text editor would be... less than appealing.)
I was going to say the same thing
For a complex and large code it will be never helpful. For smaller jobs or for product managers it can be helpful. When you already know the stuff why do you need so much simplification. Write good code with well documentation. That's is it.
Cheers!
Um, just parse code into an AST instead. Terser 🐈⬛
I cringed all the way to the end. Whatever you are trying to achieve exists as "magic" in many opinionated frameworks.
JSON is not expressive enough for whatever you mentioned. It will look ugly and unreadable while making anything meaningful.
I would have cringed less if it was say YAML.
There exists a lot of configuration driven plug and play frameworks, all of it starts to bite you in the ass while straying away from what the author wants you to do.
I use java and rely on opinionated frameworks along with open source libraries like apache commons lang to do even string comparisons.
This basically accomplishes the same thing with a window for pulling out big guns if you want to.