When we think about the next generation of programming, it can be helpful to look back at how the personal computer has evolved.
To see what I mean, let's take a quick trip back to the 70s.
The origin of personal computers
1975 was a revolutionary year for personal computing. The Altair 8800 was released, the first commercially successful personal computer. This was shortly followed by Altair Basic - the first programming language for the machine, developed by Bill Gates and Paul Allen.
These, combined with a teletype, produced an early computer terminal. It functioned much like a modern terminal developers use today, but it was a hardcopy terminal. The video below demonstrates using Altair Basic on an Altair 8800 with a teletype. You type into a typewriter, hit enter, and the computer processes the input and types a response
Typing Altair Basic on an Altair 8800 with a teletype - what I like to call the "ghost typewriter" computer. Credit
Adding a screen
The Apple II was released in 1977, another landmark in personal computing. It was a key part of a new wave of introducing whole new visual concepts to computing using a graphical display.
The same year, the original digital spreadsheet, VisiCalc, was released for the Apple II. This program alone was considered by Steve Jobs to have "propelled the Apple II to the success that it achieved" (source).
VisiCalc on the Apple II. Credit
Enter the GUI
The original GUI (Graphical User Interface) was first developed by Xerox in 1973. The Xerox Alto was the first computer to use a GUI, the desktop metaphor, and the mouse. The Alto was a major influence for both the Apple Macintosh and the original Windows OS, released in 1984 and 1985, respectively.
Not only did this pave the way for making computers drastically more intuitive and approachable to everyone, it brought us this incredible ad:
Programming with text
Fast forward to today: we interact with computers constantly, essentially forgetting there was ever a day when the GUI did not exist. Can you imagine using an iPhone without being able to navigate and interact with apps by tapping and using gestures, and instead you had to type in commands?
Oddly enough, when it comes to programming for computers, that is where we still are. We aren't that much farther from the original Altair 8800 and teletype. We type commands into a console, and type structured text-based instructions to a compiler/interpreter.
Some might argue this is surprisingly elegant - and it is in certain ways. But at the same time - it's 2021 and we are still using text editors and terminals to code. Shouldn't we have figured out a better way by now?
The benefits of visualization
The benefits of visual computing are obvious. It is approachable, efficient, and elegant - yet still very powerful.
The beauty of using a GUI is that every use case can have its own purpose-built experience. Unlike a programming language that has one simple construct (it's syntax and grammar) to accomplish all tasks, a UI can provide a unique experience optimized for each type of task.
Everything from querying data to analyzing it, there are better tools than text:
Same goes for creating UIs:
The challenges of visualization
So, why are we still writing programs as text like we did 50 years ago? Some have even called this the "great stagnation"
The challenge of visual programming lies in its benefits - there is no one way to do everything. So as a result, we still lean on text-based coding, as its simple yet flexible constructs leave no gaps unfulfilled. In a way this leads text-based coding to being a jack of all trades, and master of none.
To take things back to our examples from the 70s and 80s, a metaphor for the majority of current no-code tooling is an arcade game. Arcade games were single purpose. They had all the things that seemed magical about the revolutionary Macintosh - they had a visual display, they were intuitive for even children to use, etc. Much like the current generation of no-code tools.
But they lacked one key ingredient - they were not general purpose. Don't get me wrong, single purpose computing has its benefit, but a revolution in software development doesn't come from such technology, it comes from generalizability. Aka building something that is intuitive and powerful and is unbounded in what you can create with it.
How do we solve this?
New programming generations are created as layers on top of the prior generations, not as wholly separate and new concepts. New technology is created by standing on the backs of giants, not reinventing the world.
In order to create a visual programming experience unbounded by the constraints of a single problem, we must connect visualization to existing software systems. In other words, we don't need to reinvent the wheel for a single purpose, but connect to it as it is.
Brett Victor, in his incredible talk "Inventing on principle", shows us some examples.
Who is doing this now?
There are 2 main categories - visually enhanced developer tools (developer tools like IDEs with visual features), and low-code tools (visual tools that connect to existing APIs and codebases).
Visually enhanced developer tools
One industry that is really pushing visual coding is game development. Games are created by huge teams, and have huge production value that cannot depend on legacy methods like app/web devs use - things like passing off a design to a developer and asking them to code it up in CSS by hand. A world as intricate as those found in modern games would be a nightmare to build line by line manually.
Credit: Ghost of Tsushima
Would you want to code this landscape by hand like web devs code CSS? Yeah, didn't think so.
The need to take games beyond what could be coded by hand led the industry to invest heavily in visual tools that connect directly to code. Unreal Engine is a great example that you can try yourself today:
Another great example of this is the latest SwiftUI in Xcode
Low-code tools
In web and application software, low-code tools are starting to emerge and grow rapidly. Tools like Airtable, Zapier, Builder, and Retool are showing how we can elegantly allow visual editing connected to existing code, data, and APIs
These work great because they build on top of existing infrastructure - your existing React components, databases, and APIs - and you can can granularly set permissions as to who can edit what and where.
So - what's next? Where is this going?
As we see it, the connection between code and no code will only grow tighter and stronger. We are only at the beginning, you could call it the Apple II days of visual software development. We still have our version of the Macintosh (truly easy and powerful visual development) to get to, and ultimately the iPhone (easy for everyone).
Here are a few projects out of many we are particularly excited about right now - Storybook, JSX Lite, Blockly, and Build
Are there any other exciting visual programming developments that you are excited about or want to see? Drop me a comment below!
Top comments (27)
Not sure about this - where I agree with you is that it's likely to see a movement to visual tooling in areas that are inherently visual, e.g. front-end (and then especially the presentation layer of the front-end), but in other areas I'm not so sure.
What we need (I would think) is increasing the abstraction level, meaning that we'd not be replacing code with pictures, but we'd replace code with less code. I can definitely see that - the amount of code that currently needs to be written even for a trivial app is staggering.
I can hardly imagine that in 10 years from now we'll still be coding all of the pretty low-level and mundane stuff like user onboarding, login/signup and so on, which we're still implementing again and again, ad nauseam ... instead we'll probably just tick a few checkboxes somewhere, or add a switch to a console command, and voila it's all there.
And somehow I suspect that a lot (most?) of the software we're writing now won't have to be written at all, in 10 years from now - as in many other areas it might very well be the case that a lot of the work will be automated away by things like AI and so on, especially the more mundane and repetitive parts.
But, regarding code (text), it still has a number of clear advantages compared to diagrams and 'pictures' - easier to manage in version control systems, in many cases easier to reason about, and so on.
What's really interesting is to consider why these mundane repetitive tasks are not already abstracted away into code packages with config. What we see is that some of the work can be abstracted away e.g. Auth0 service or a React login form component. The problem is integrating these reuse elements vertically across the horizontal layers, and also how they interact with custom code and other similar components. In this example, how does the login component know where to find the Auth0 service? Does it tie you into Auth0 as a third party provider? Where are your users stored and how will your code and other components find them? Classic ASP.Net had a go at this as its components where server/client integrated but the resulting abstraction was leaky as a sieve and horrible to use. We need some innovations around code structure, reuse and interoperability to make this work. I look at some of this in this article: restspace.io/blog/Web%20reusabilit.... I'm also working on a way of having front end NPM component packages (like the React login component) express their back end API and infrastructure requirements declaratively and having a system which fulfils those requirements automatically on the back end.
That's very interesting ... having looked at Restspace, one thing that jumped out to me was the principle "Uniform interfaces to functionality" - that's very essential, example:
Something like Auth0 gives us a "chunk" of functionality (a service or a component, whatever you want to call it) that we can 'simply' drop into our app - this means writing less code, but we still write code, talking to the Auth0 API ... now, what if we don't want Auth0 but another service or component - we have to change our app, because that other service will have a different API - there is no standard!
In other words, because standard interfaces don't exist, we have to write "glue" code or adapters every single time. Even though we get a chunk of pre-made functionality, we still spend time because we need to fit a square peg into a round hole. This is what causes the "reinventing the wheel" syndrome, the feeling that as developers we're reimplementing the same trivial stuff time and again.
Compare that to the situation in other "industries", for instance construction or plumbing or you name it - those industries are chock full of standards, with names like DIN or ISO and so on. A plumber or an electrician buys a component (a pipe, or a faucet, or a transistor) with certain "specs" and then he/she can be sure that it will fit onto the other components he's purchased before.
Once we have these standard interfaces, then you can start inventing a very high level "specification language" which declaratively states what the components are and how they should interact. And then there would be a runtime which interprets this declarative "pseudo program" and makes the app or system come alive.
And for specific customizations or specialized parts of the system/app you could have "escapes" which allow you to hook into the program/runtime with snippets of code that you write in a conventional language (like Javascript).
Note that all of this becomes orders of magnitude easier if we reduce the degrees of freedom, so if we limit the options. So if we say, for instance, that we only support one target language (for instance, only Javascript, no PHP or Ruby or Python) then we reduce the complexity of this "platform" in a huge way. Same story if we'd say that we support only one database, and so on. So if we'd limit the number of options then we'd vastly simplify things.
(so part of the problem is that in the software world everyone and their brother (or sister) likes to invent their own programming language, database, frontend framework, CMS, and so on)
But, all of this is nice and good, but:
(a) pulling this off will require a monumental effort and a huge investment, and
(b) fragmentation won't end if a dozen alternative platforms like Restspace are popping up (in other words, there would still be no industry standard unless we make it so), and
(c) actually I'm afraid that there's not that much of an incentive to make this work, because you could very well argue that as an industry (developers) we'd be shooting ourselves in the foot!
In other words, the current situation where we're spending a lot of time eternally reinventing the wheel is of course a great job creator :-) and, in general we really like what we're doing, right now - so, in yet other words, in whose interest would this be ... at the same time I'm convinced that at some point in the future this status quo WILL change, whether we like it or not.
Great response! Yes this is pretty much what I'm trying to do with Restspace. One way of standardising the connection between components is that even small components running on the same server all use the data structure of an HTTP message to communicate. Within the same server, code level routing sends these HTTP data structures without using the wire, but the runtime determines if the URL is external and goes across the net if necessary. This creates internal interoperability, allows the components to be distributed across servers however you like, and potentially pulls in every API available on the internet. Then the structure of the API is standardised across services providing the same functionality by adapting internal and third party services. For instance, the basic data API is simply to do a GET to read data or a PUT/POST to write data to a url. Data structures are defined by JSON Schemas. The service providing this API can have an adapter plugged into it which talks to the file system, MongoDb, S3, a SQL database or whatever.
This then enables a module/package to know where and how it needs to store data. For instance the 'full stack' auth package can be told a url in the data store API for storing user records, and it immediately knows how to do this, without restricting what underlying technology you want to use. An email package then also knows how to get user data and can just be wired up to the url in order to get the user data to customise an email template.
To your points, having been working on this for a while, the effort is actually not that massive, at least to get a set of core functions together. There would likely be some fragmentation initially but like most of these things, there are network effects at work and the winner of an initial battle would become a de facto standard. Dev has constantly been pushing effort up the stack and away from the generic into the more distinctive aspects of applications. I recently saw a Twitter threads where devs listed the things they hated doing repeatedly, and those are the things a system like this would be ideal for.
Right, I see what your getting at, very interesting ... looks very well thought out with these services, pipelines, data transforms and so on:
restspace.io/
Very cool project and a great effort, I've saved this to my notes, looks worth trying out. By the way is it open source? I saw a couple of Github repos (github.com/restspace) but looks like that covers only a small part of it. Ah I see the answer here in the docs:
"We are also considering open-sourcing the runtime so you can run it wherever you wish"
So that would be "not yet but maybe in the future".
Many thanks for the kind words! Yes, basically I'd love to open source this but it will never fulfil its promise unless there is some cashflow to support development. Quite likely it will work as an open source project with a paid for hosting like the model Wordpress has, however it needs some thought how to do that without losing the chance to make some revenue. I'll definitely open source components which it makes sense to do.
Yes I definitely see what you mean, I get your point, totally.
Ah yes this is 100% fair and valid. I think to maintain the advantages of text I'm a big fan of visualizations that derive from the code, like in the case of JSX Lite and Build so you can keep using all your code tooling and workflows and visualization is purely optional.
But also agree that some things are not as inherently visual like game and frontend dev. That said I've seen some interesting stuff around for visualizing data flow in code, which brings me back to the idea of (optional) visualizations on top of (text based) code, which can have multiple formats too (eg visualize and edit the same code as a data graph, text based, maybe even block based, based on what type of editing or debugging you are doing)
Right, I see what you mean - I'd call that round-trip engineering:
en.wikipedia.org/wiki/Round-trip_e...
Well I don't have a crystal ball, so I don't know exactly what the future is going to look like, but I think it's safe to assume that things will be really different in let's say 10 years from now - I can't imagine that we'll still be doing things the same way, writing tons of code for mundane stuff like user onboarding and so on.
Probably somehow the abstraction level and the 'automation level' will be much higher. I wouldn't be surprised if we'd be coding in a very high level pseudo-code language, and/or with abstract diagrams - and these two would then be just two representations of the same "model" (round-trip engineering).
Yes, I feel that that might very well be our future - we'll just design a "model" and voila we'll have our app or your system - all of the low level details will be managed by some sort of runtime. And maybe AI will be thrown in the mix.
So, to a large extent this actually sounds a lot like what your post was suggesting? :-)
Ah yes absolutely. Also worth noting there are ways to do this without additional artifacts generated that could go out of sync too. Eg in the case of JSX Lite and Build they parse the (javascript) source file to an AST, store and modify that in memory only when editing visually, and changes are applied back to the source code without anything else generated or saved
There are diverging trends in this space:
There are a lot of good examples of visual or no-code tools for transforming data or managing infrastructure which is very visual and requires no code. They can be really good if itś truly turn-key and you get things done right now.
However ... a lot of data transformation and infrastructure management is migrating from visual tools to writing it as code. The reason is that systems for doing this are expensive, unreliable, moving targets, proprietory, hard to automate, hard to validate, slow, obsolete, and often not a good long-term investment if you want to learn these tools.
Agreed, I think tools like Airtable and Builder.io have their place and strike a good balance for their use cases, but for general purpose visual programming I think that will lie in tools like JSX lite and Build that can add a visual layer directly on code so you don't lose the benefit of code and existing code tooling
Generally no code tools ARE visual development tools. A common characteristic of these tools is a graph idiom, where nodes represent processing units and lines, data paths between them. I think what they represent is less a visual approach to coding than a graphical approach to coding. If your primitives are processing nodes, it can be very powerful and quick to compose those nodes into networks. The extra work required to leverage preprogrammed units is much less than with a programming language.
This is because of the simplicity and universality of a data pipeline as a connector compared to a call signature. We find this graph idiom again in the attraction of microservices, with the edges being HTTP connections. Reconfiguration of very loosely coupled cohesive units is a powerful way of achieving new functionality with little work. Unix pipelines leverage this too.
I'm working on a framework which lets you construct pipelines of HTTP services declaratively with scripts. This has the advantage of being open to and inclusive of any existing HTTP service on the net, but provides a core set of primitive mini services which enable building a lot of powerful and useful back end functionality. Also it (will) let you build your own mini services rather like serverless functions. Here's an article where I explore the ideas around this further: restspace.io/blog/Web%20reusabilit...
Great points and super well put.
One additional space that heavily uses graph nodes/paths is Reaktor, which I was tempted to add a whole diatribe on flow based programming and its uses like in the music industry native-instruments.com/en/products...
An HTTP service composer using this structure sounds amazing
Yes indeed the GUIs on synth/sequencing tools have always impressed the hell out of me. The general paradigm of the modular synthesiser is exactly what I'm describing: a graph of control/signal cables with the nodes being sound processors. Thanks for the term 'flow based programming', I'm going to read up on that!
nice informative article and the projection is very realistic and fact-based.
Scratch is another visual programming tool.
Thanks! And Scratch is a perfect example of visual programming
For those interested: scratch.mit.edu/
I believe Blockly is heavily inspired by Scratch. As is PXT by Microsoft which is another very cool one for teaching code visually, esp in how it can integrate deeper with actual code
github.com/microsoft/pxt
Great work!
Nice!
What about things like Luna? It does not fit your categories, I think, but I might be closer to next-gen programming, at least for data-heavy stuff.
Ah yes absolutely, Luna is very cool and I'd definitely include it in the where things are going category. Very similar in mindset to Build and JSX Lite, in terms of deeper/tighter code and visual integration
I am very eager for this group of tools to really take off
Nice read - you make some interesting points about the challenges of visualization.
This post would not be complete without mentioning APEX: Low-Code since 20 years... so not day I :)
Very quick overview: youtu.be/klm8M4e5fEM
Main portal: apex.oracle.com/
I think you would also like Node-RED (nodered.org). Visual flow based programming - but allows you to drop down to node.js (javascript) or call out for anything else you may wish to do.
Ah yes, Node-RED is a perfect example!