This article categorizes the code components used to build a modern web application, dividing them into Frameworks, Libraries and Languages. I'll describe each of these and offer an opinion about why the way we structure a project frequently results in things not turning out as intended.
A Framework is software analogous to a new house without windows, doors or furniture. The structure is all there but you have to supply all the detail yourself; all the bits that make it into a home. It's quite easy to spot a software framework; it follows the Hollywood principle of "Don't call us; we'll call you". The framework is very much in control; you are not.
A Library is completely the opposite; it's code that is called directly by your application, providing "black box" functionality as required by applications. Here you are in control. Libraries are a vital part of any software system and the value of a library is often closely related to the care taken in defining its API.
A Language is neither of the above. It's an agreed set of words and a syntax that uses them to convey meaning. Structurally it's the same house complete with windows, doors, furniture, the table laid and a chicken casserole in the oven. In a limited sense the syntax rules amount to a kind of framework, but it's one that puts you in control, and in human languages such as English the concept of a "library", in the sense of something that implements added features, is meaningless. If we encounter a concept that requires an elaborate description we encapsulate it behind a new vocabulary word, with its own syntax that conforms to the general rules for the language, and this then becomes a seamless part of that language. Examples are "laser" and "smartphone". Whatever allows us to understand these terms might be regarded as an implicit library function. Something of this kind happens when JavaScript gets a new release; new words and syntax rules arrive that work alongside those previously used.
Now for some examples of each of the three categories.
Angular is a framework and proudly announces itself as such. Although I have very little experience of using it I believe few would regard it as anything else. In order to create a viable Angular product the structure must be defined up front and then everything made to conform to that structure. This is for some excessively rigid but for others a guarantee of a superior outcome.
React is also a framework, but - as I understand it - with a nod to being a library as well. The framework is less rigid that than of Angular, and for anyone brought up on imperative coding it may be somewhat easier to learn. React was developed by and for Facebook; some believe that makes it good for everything while others place it with Angular as overkill for smaller projects.
JQuery is a library, but with a hint of some language features. For example, selectors provide no new functionality, just an easier way to express what could already be done in JavaScript. JQuery is quick and dirty but compact, effective and easy to learn, and has no pretensions about being any more than it is.
JavaScript is a language, and like human languages is fluid and ever-evolving. In its early days it lacked structural features and this provided the initial impetus for the development of the frameworks and libraries we see today. Some have argued that recent improvements to the language, such as the addition of classes, make the need for frameworks less pressing, that most things can now be done without the need for one.
So, what is the match between the three concepts and the kinds of product being requested by customers?
Client side programming ranges from very simple interactivity added to a static web page through to huge commercial, industrial or social media websites. It's rather foolish to assume that the same tools are equally applicable in all cases, though looking at the skills requested by potential employers that tends very much to be the assumption. I put most of that down to ignorance and the herd instinct.
As with houses, changing the structure of a large software system is an expensive business, so frameworks work best where the structure is unlikely to change. Paradoxically, that makes a library component the ideal place to deploy a framework to maximum effect. Once written, the structure of the component will rarely be changed, and if it's a popular library it will have a development team dedicated to its maintenance over a long period. One of the best examples is Google Maps. Few users of this amazing library have the slightest interest in knowing what goes on inside it; all that concerns them is the API it presents to the world. It would be inconceivable for Google not to maintain the expertise needed to support and continue the development of Maps.
Websites themselves, with relatively few exceptions, do not remain unchanged for long. When change comes it's frequently structural, as often as not driven by a desire to "look different", and this presents a problem. In a nutshell it's that software tools are designed to help developers become more productive. All the emphasis is on creating a product, the assumption presumably being that once developed and deployed the same team will continue to provide maintenance. But this is only true for companies who are themselves software houses. For everyone else, the development team, whether in-house or contracted, will move on. It's hard to justify maintaining expensive developers when you only have occasional work for them.
As I said above, structural changes are expensive. Before you can even start you have to get the people up to speed on how the system currently works, in order to then make it do something different. It can take months for even the best programmers to get to grips fully with a complex system, and in most cases months is what we don't have. The result is inevitable; changes are made by people who lack a complete understanding of the system, resulting in code that will baffle future maintainers and damage that will be even harder to deal with next time a change is needed. And so on, a spiral of bloat, decline and "death by a thousand cuts", until the cost of maintenance exceeds that of abandoning the product and starting again. Usually years before coming to the original projected life of the product.
I've seen this first-hand more than once. In one case I joined a large pharmacy chain whose main software product was maybe 5 years old and was rapidly becoming an unruly pile of spaghetti. There were only 3 people who really understood it; one left a year after I joined and the other two were contractors. With a change of ownership the company then decided to cut back on the latter because of the cost, so those who remained never had a perfect understanding of how the system worked, nor was there anyone left to ask. Naturally there was no software documentation - we were expected to be experts who could "read the code". We couldn't, of course; few can, any more than the general population can read and understand legal documents, whether written in English or a foreign language. Eventually the product was abandoned and its place taken by a standard off-the-shelf system; expensive and with fewer features but none of the problems of maintenance.
No matter how hard the industry works at creating new structural entities, the results will never be understandable by the great majority of those charged with maintaining the products. They're simply too complex and the tools used to build them go out of fashion too quickly. As long as we continue with the same mindset, projects will continue to fail early.
Which brings me to a diagnosis. The problem isn't one of structure; it's one of language. We speak English; the computer speaks JavaScript. Our human requirements are translated into computer code, and along the way misunderstandings are bound to occur. It's largely an open-ended process with no feedback loop. When we want to modify code we're not fully familiar with we have to translate JavaScript into English so we can discuss it with our clients, then translate the agreed changes back into JavaScript. Each of these translations is imperfect and will generate errors. Faulty systems aren't just unfortunate; they're inevitable.
The only solution to this problem is for humans and computers to speak the same language. Since humans will never be able to speak JavaScript this means computers will have to speak English, or at least some form of it that acts as a common ground. Before you dismiss this as impractical, consider; it's already done on a huge scale and very successfully. If you are a database expert (and not necessarily a programmer) you will be familiar with SQL and able to read or even construct quite elaborate descriptions of what you want to achieve. The good news is, the computer will understand you. The overall error rate will be low because all the parties involved - client, database expert, programmer and computer - speak the same language. I expect a similar claim can also be made for finance, where spreadsheets containing Excel macros commonly bind clients, financial experts, programmers and the computer.
And that's the basis of my plea for an industrial-strength language for browsers, that uses English-like scripts to convey meaning and preserves readability by everyone from the owner of the site to the browser it runs on. Since I believe in practicing what I preach I built my own compiler and I get very good results with it, but I'm not an expert in that field, nor am I in the same intellectual class as the people currently developing frameworks like Angular and React. I can barely imagine the power and beauty of what might appear if that amount of effort were to be expended on devising and implementing a form of English that allows people of all abilities to truly "write" websites and web apps and then keep then running for a decade and beyond, but I long for the day such a system arrives.
Title photo by Nick Fewings on Unsplash
Latest comments (10)
Your mention of SQL as an English language everyone speaks is a great example as to why that isn't true.
In its basic form sure, but table structure does not get built to represent the business owners language, there are technical requirements and performance that play into its structure. Even if the end goal is to make the transition easier, it requires a good understanding of those original requirements (ones the client changes their mind/desires on).
OK, I accept that SQL isn't really English, and it can get pretty hard to follow, but it's a lot closer to English than are most regular computer languages, with their heavy reliance on symbols and constructs that are impenetrable to the uninitiated.
Once you get away from the basic form, even English isn't always understandable by English speakers. Legal jargon, for example. We'll never get a true universal language, and I'd never advocate one. In spite of my enthusiasm for English-like scripting languages I don't regard them as a universal panacea, but never underestimate how far plain speaking can take you.
Too much language is used in part to exclude those who don't match the author's idea of who is able to appreciate the revelations contained therein, and my desire is to see tools that will give the lie to the claim that obtuse language is always necessary. Two books on any given subject can present what is essentially the same content in very different ways, one accessible and the other impenetrable, and I would like to see more of the former. Having the language with which to do it is a good start.
I would claim that the issue is the same as the uncanny valley except for code.
It is what I was actually going for in my response to regex.
There is a similar question going around, but it just got the same old answers that people always give about Regex syntax, but that's not the point here, so please try to not knee jerk the same old answers about regex syntax. Try to be a little more original and…
Interesting discussion, that shows Regex to be a good example of the limitations of plain speaking. Sometimes there really is no alternative to out-and-out "code". Elsewhere, though, plain language is better than dense code, for the same reason we avoid one-letter variable names. It's all to do with human comprehension. As a rule of thumb I'd say if it can be described clearly in English then perhaps it ought to be coded in English - or as close as you can get to that.
Well the uncanny valley is the idea that the closer you are to realism, or in this case English, the harder it is for people to believe it when things are still inaccurate. (You mentioned a compiler, maybe some synthetic examples could help in this discussion.)
Let's consider Javascript and some of its evaluation.
That is nice but why not
Clearly we would want a subset of the English language.
It is not like Javascript is clear on by its choice but does follow some really really normal emphasis convention. But I don't even know where to go with English. Sure maybe Javascript should have use
likein the first one, but hindsight is 20/20.What about some bash
It could be argued that
gtis close enough, but that isn't English what if it means 'goes into'Sure we don't know of a language with this semantics, but if someone wants that maybe that is what they believe 'gt' to stand for.
And while we're at it, what are these quotes and what is a string.
But I've included 'as' after 'writing' how does the language know which articles to expect when? Maybe we don't need the articles.
Well that isn't English. We've hit the uncanny valley. If we don't handle all of English we will come up with our own language which looks like English but does not live by English rules.
When theory shows a thing to be X, practice often demonstrates it's Y. I'm not a theoretician, just an old-fashioned engineer who makes things that work. In this case, user interfaces, which today mostly means websites. Any website that can't be described clearly in English probably can't be used by human beings, so I code them entirely using my own formal English-like imperative syntax originally inspired by Apple's HyperTalk. Like English, this is unambiguous if coded sensibly and it can be used by any person with an understanding of what the UI is for and what it does, whether their native tongue is English, French, German or whatever. Devising a clear and unambiguous syntax and then writing a compiler for it are not trivial tasks, and the future can come along and bite you, as the authors of JavaScript have found. But that's no reason for not doing what works best now.
A typical example might be "while Count is less than the json count of TheDataArray gosub to AddAnotherRow". One might argue about its linguistic purity but ten years down the road it will still be instantly readable while a React or Angular solution will have the reader diving into the discarded book pile to find out where to locate and how to set up the IDE, never mind changing the code. I exaggerate, perhaps, but not by much.
And that's what lies behind the articles I write on the subject of appropriate language - real-world experience of something that works now and goes on working indefinitely while the rest of the world changes. And I have satisfied customers able to testify to the effectiveness of the approach.
"while Count is less than the json count of TheDataArray gosub to AddAnotherRow"
Any chance you could explain what this is checking?
It's the head of a 'while' loop that counts from the current value of Count up to the number of elements in a string that's a JSON-encoded array (held in a variable called TheDataArray). Each time through the loop it executes a statement, which might be a single command or a block (these use begin...end); here it's a subroutine call; the code assumes Count will be incremented there. (I'm not saying I would actually code like that; it's just an artificial one-liner.) All variables are global but there are rarely more than a few dozen in any one script. The language allows scripts to invoke other scripts and share variables with them; unshared variables remain private.
I originally wrote the compiler and runtime in Java some 20 years ago. Last year I rewrote it in JavaScript as a website coding engine, and I'm now doing a version in Python (as a learning exercise). None of them have ever been widely used by anyone but myself, but that's the fate of most custom languages. The performance is quite good, especially when complicated functionality is wrapped up as script objects (as happens in English). I accept it breaks all sorts of conventions regarding the 'right' way to do things but I'm only interested in what works. Most importantly I can return years later and quickly pick it up again, something I struggle to do even with my own code written in Java or PHP, for example.
You absolutely right, I think nowadays programing is more short term oriented, and we do not take time to think "if I build this, what will be the technical debt for the next 2, 5, 10 years?".
You definitively raise for me the biggest challenge in IT: can we improve our human to computer bond by making the computer do more efforts to understand the human natural language? And I am convinced part of this answer is your work on EasyCoder!
I think it's not so much short-term as goal oriented, and the goal is too narrowly defined to consider only the requirement to deliver to the spec. There's an assumption that everyone will learn React (or whatever) as needed, but my experience is that this doesn't happen and projects get orphaned. Development teams should be more realistic about this; I know they love using their toys but there's a real world out there that doesn't share that love. Better to compromise and produce something that has a future not dependent on skills that have only been around for a short time and for which there is no guarantee of continuity.