DEV Community: Simple Thread

Your Text Editor Plugins Belong in a Lock File

Simple Thread — Tue, 11 Jun 2019 13:00:42 +0000

I just updated all of your text editor plugins. What’s your reaction? Something like this?

It’s been years since you’ve updated all those plugins, hasn’t it? Who cares about those new features, bug fixes, or performance and security improvements? Just cross your fingers and hope your laptop doesn’t die forcing you to upgrade. Better yet, you’ve got a full image backup, right? That way you can keep those crusty old plugins forever.

It’s understandable to wince at the thought of updating those plugins. Most developers have been burned for one reason or another. Besides, who has time for that crap? After you update you run into some obscure conflict between plugins, or the author pushed a beta version. Even worse, that fancy new version of the plugin is a real performance bottleneck.

And of course the issues never seem to crop up immediately after doing the update but instead at the most inopportune time. It’s several days later and you’ve got a deadline approaching. You don’t have an easy way to revert your update. Heck you don’t even know which plugin is the culprit. Now you have to make a choice. Do you slog through debugging your editor or disable plugins, continuing your work with one hand tied behind your back?

Updating your plugins doesn’t have to be this painful though. Wouldn’t it be nice if your editor plugins had a lock file like your application dependencies do? (You lock your application dependencies, don’t you?) Then you could add that file to version control. Going back to your old setup would be like snapping your fingers. Just checkout the previous commit and run the install command. Sounds obvious in hindsight, right?

I’ve kept my plugins in a plugin.lock file for about a year now, and it has been wonderful. It works exactly like you would expect. I update my plugins every couple months. If I run into an issue and don’t feel like dealing with it immediately then I can quickly revert. For debugging, I can do a quick git diff to see which plugins were recently updated. If one plugin is misbehaving I pull the old version from my git history and continue on my way. If I’m desperate to determine which plugin is misbehaving I can even do a binary search (Recursively revert half the versions and check if the problem still exists).

Creating and using your very own plugin.lock file is really easy. I have a few examples below for popular editors.

VIM

Using the popular VIM plugin manager vim-plug

Create your lock file…

:PlugSnapshot plugin.lock

Restore from your lock file…

:source plugin.lock

Atom

Create your lock file…

apm list --installed --bare | xargs -L 1 echo apm install > plugin.lock

Restore from your lock file…

sh plugin.lock

VS Code

The insiders version of Visual Studio Code Supports this.

Disable auto extension updating (if you haven’t done so already)

Create your lock file…

code-insiders --list-extensions --show-versions | xargs -L 1 echo code-insiders --install-extension > plugin.lock

Restore from your lock file…

sh plugin.lock

———————-

If you have instructions for creating a plugin.lock in other editors please let me know in the comments!

The post Your Text Editor Plugins Belong in a Lock File appeared first on Simple Thread.

Was MongoDB Ever the Right Choice?

Simple Thread — Tue, 26 Mar 2019 15:06:05 +0000

I was reading a post recently aboutRed Hat removing MongoDB support from Satellite (and yes, some folks say it is because of the license changes). It made me think how often over the last few years I’ve seenpost afterangry post about how terrible MongoDB is and how no one should ever use. However, in all this time, MongoDB has become a much more mature product. So what happened? Did all of the hate truly come from mistakes made in the early implementation/marketing of MongoDB? Or is the problem that people are blaming MongoDB for their own lack of efforts when evaluating if it was a good fit?

If you’re finding yourself at this point foaming at the mouth because it appears that I’m defending MongoDB, please jump to the end of this post and read my disclaimer.

Trendspotting

I’ve been working with software for more years at this point than I like to admit, but even then I’ve only experienced a tiny fraction of the trends that have buffeted our industry. I’ve witnessed the rise and fall of 4GL, AOP, Agile, SOA, Web 2.0, AJAX, blockchain… the list never ends. Every year there are new trends that pop up in the world of software engineering. Some fizzle quickly, while others fundamentally change the way software development is performed.

With any new innovation that starts to gain traction, you’re going to see a general excitement appear around it as people start to pile on board, or see the buzz being generated by others and decide that they too want to get in on the action. This process was codified by Gartner with itshype cycle, which (while controversial) is a decent approximation of what happens with technologies that are eventually found to be valuable.

But every once in a while, a new innovation appears (or in this case reappears) that is driven by one particular implementation of that innovation. In the case of the NoSQL movement, it was driven heavily by the appearance, and meteoric rise, of MongoDB. MongoDB didn’t start the movement, it was the data challenges at large internet companies that really drove the return to non-relational databases. Projects like Google’s Bigtable and Facebook’s Cassandra kicked it off, but MongoDB was the most visible and accessible implementation of a NoSQL database that most developers had access to.

Aside: You might be thinking right now that I’m conflating document databases with columnar databases, key/value stores, or any of the numerous other datastore types that fall under the generic NoSQL banner. And you are correct. But this was happening wildly at the time. Everyone was jumping into the NoSQL craze and they knew that they absolutely needed NoSQL, but didn’t really understand the different technologies involved. To many people, MongoDB was NoSQL.

And developers pounced on it. The idea of a schema-less database that magically scaled to meet any challenge was quite alluring. Around 2014 or so, it seemed like everywhere you looked someone was implementing MongoDB in a place where just a year earlier a relational database like MySQL, Postgres, or SQL Server would have been used. When asked why Mongo was being used there were responses ranging from the banal “it’s web scale” to the more thoughtful “my data is very loosely structured and fits well into a schema-less database”.

It is important to remember that MongoDB, and document databases in general, solve a number of problems people had with traditional relational databases:

Strict Schema – With a relational database, if you had dynamically shaped data you were forced to either create a bunch of random “miscellaneous” data columns, shove data in as a blob of data, or use anEAV setup… all of which had significant downsides.

Difficult Scalability – With a relational database, if your data was so large that you couldn’t fit it easily into one server MongoDB had built in mechanisms for allowing you to scale that data across multiple machines.

Difficult Schema Modifications – No migrations! With a relational database, changing the structure of the database can be a huge challenge (especially once your data gets really big). MongoDB promised to make this dramatically more simple. And it made it soooo easy to get started, you could just keep updating your schema and move really quickly.

Write Performance – MongoDB’s performance was good, especially when configured in certain ways. MongoDB’s out-of-the-box write configuration, which is one of the big things it was criticized for, allowed it to put up some impressive performance numbers.

Caveat Emptor

The potential benefits MongoDB provided were huge, especially for people facing certain classes of problems. Reading the list above without context, or experience, would lead you to believe that it truly was a game-changer when it came to database systems. The only problem was that the benefits listed above came with a number of caveats, some of which I’ve listed below.

To be fair, no one at 10gen/MongoDB Inc. would claim the items below aren’t true, they are just tradeoffs.

Loss of transactions – Transactions are a core feature of many relational databases (no, not all, but most). Having a transaction means that you can perform multiple operations atomically and you can ensure that your data will stay consistent. Sure, with a NoSQL database you can have a transaction within a single document, or you can use tactics like two-phase commits to get transaction-like semantics. But the point is you have to do this work yourself… and it can be challenging and labor intensive to get right. Often you don’t realize how much you’re giving up here until you start seeing data in your database get into invalid states because you couldn’t guarantee the atomicity of operations.

Loss of relational integrity (foreign keys) – If your data has relationships, then you’re going to have relations. Almost all data has some kind of relations, and if your database doesn’t enforce them, then your application is going to have to. Having a database enforce these relationships can offload a lot of work from your application, and therefore from your engineers.

Lack of ability to enforce data structure – Strong schemas might be a pain in the ass sometimes, but they can also be a powerful mechanism for ensuring that your data is well structured. If you leverage them appropriately, it provides a powerful mechanism for ensuring your data is in the shape you expect. Document databases like MongoDB allow an incredible amount of flexibility around the schema, but that flexibility offloads the responsibility onto the maintainer to keep their data clean. If you don’t put in that effort, then you end up putting a lot of code into your application to account for data that might not be in the shape you expect. As we often like to say at Simple Thread… your app is going to be rewritten one day, your data will live forever.

Custom query language/Loss of tooling ecosystem – SQL was an absolute revolution when it came out, and nothing has changed since then. SQL is an incredibly powerful language, but one that can also be challenging. Having to query a database using a custom query language composed of JSON snippets would be considered a big step backwards by folks experienced with SQL. There is a whole world of tools that interoperate with SQL databases. Everything from IDEs to reporting tools. Moving to a database that doesn’t support SQL means you can’t use most of these tools, or you have to find a way to get your data into a SQL database so that these tools can be used, and this can be harder than you think.

Many developers who reached for MongoDB didn’t deeply understand the tradeoffs they were making, and often they dove in head-first by using it as the primary datastore for their applications. This meant that is was often incredibly costly to go back on this decision.

What could have been done differently?

Not everyone jumped in head first and slammed into the bottom of the deep end. But enough did that there will be projects for years to come removing MongoDB from places where it just didn’t fit. If many of these organizations had taken a bit of time to think methodically about the technology choices they were making, it is likely that many of them wouldn’t have made the decisions they did.

So how do you decide what technology makes sense for your use case? There have been a few attempts at creating systematic frameworks for evaluating technologies such as “A Framework for Technology Introduction in Software Organizations” and “A Framework for Evaluating Software Technologies”, but I don’t think it needs to be that complicated.

Many technologies can be reasonably evaluated by asking just two main questions, but the challenge is finding individuals who can responsibly answer them, dedicate time to answering them, and answer them without bias.

If you’re not facing some kind of problem, you don’t need a new tool. Full stop.

Question 1: What problems am I trying to solve?

If you’re not facing some kind of problem, you don’t need a new tool. Full stop. Don’t look for solutions and then back into problems. If you’re not facing a problem that a new technology doesn’t solve significantly better than your existing technology, then your decision is over. If you’re considering using this technology because you’ve seen others using it, it might be useful to think about what problems they are facing, and ask yourself if you’re facing the same problems. It is often easy to reach for a technology because you see another company using it, the difficulty is in determining whether or not you’re facing the same challenges.

Question 2: What am I giving up?

This is definitely the harder of the two questions to answer, because you have to dig in and have a good understanding of both the old technology and the new technology. Sometimes you can’t really understand a new technology until you’ve built something with it, or have access to someone who has spent significant time with the technology.

If you don’t have either, then you should be considering what is the smallest investment you can make to determine if this tool is valuable. And if you make the investment, how hard would it be to undo the decision?

Humans Always Messing Things Up

One thing you’ll have to keep in mind is that you’re going to be fighting human nature when you’re trying to answer these questions as unbiased as possible. There are a number of cognitive biases that must be overcome in order to effectively evaluate a technology, but just to name a few:

Bandwagon effect – Everyone knows this, and yet it is still hard to fight against. Just make sure that you’re choosing a technology because it solves real needs for you, not because the cool kids are doing it.

Mere newness bias – Many software developers tend to undervalue technologies they have worked with for a long time, and overvalue the benefits of a new technology. This isn’t specific to software engineers, everyone has the tendency to do this.

Feature-positive effect – We tend to see what is present, and overlook what isn’t there. This can wreak havoc when working in concert with the “Mere newness bias”, since not only are you inherently putting more value on the new technology, but you’re also overlooking the gaps of the new tech.

Looking at things objectively is a challenge, but understanding the biases that may affect you will help you make more rational decisions.

Wrap Up

When a new innovation appears (or reappears), we need to be very careful in answering two questions:

Does this tool solve a real problem for us?
Do we thoroughly understand the tradeoffs?

If you can’t confidently answer those two questions, take a few steps back and reevaluate.

So was MongoDB ever the right choice? Yes, of course it was; like most things in engineering, it depends. For teams that answered those two questions, many found value and continue to find value in MongoDB. For those who didn’t, hopefully they learned a valuable, not-too-painful lesson about navigating the hype cycle.

Disclaimer

I want to clarify that I neither love nor hate MongoDB. I simply haven’t run into many problems that I thought it would be the best fit for. I know that 10gen/MongoDB Inc. didn’t do themselves any favors early-on by setting unsafe defaults and promoting MongoDB everywhere (especially at hackathons) as the be-all end-all solution for every data need. Yes these were probably bad decisions, but I think it backs up the point I’m making here because these were issues that could be uncovered very quickly with even a cursory evaluation of the technology.

The post Was MongoDB Ever the Right Choice? appeared first on Simple Thread.

Great Scott! Timing Attack Demo for the Everyday Webdev

Simple Thread — Tue, 05 Mar 2019 13:45:39 +0000

One of the more insidious ways your web application’s data can be comprised is through a timing attack. The time it takes your app to serve a request can reveal more information than you might think. While certain ingredients need to be present for a successful timing attack, the surface area is large and we can’t depend on a library to protect us against all cases.

After you understand the ins and outs of a timing attack, you might be surprised at what you find auditing your own web application. What follows below is a complete walk through extracting secret data from a vulnerable demo application.

The scripts are written in Ruby, and while we love it if that’s not your language don’t worry! Written explanations are provided for each code snippet.

The demo app and all code in this timing attack walkthrough can be found here on the Simple Thread Github page.

Demo App Overview

Before we get to the good stuff, here is the vulnerable demo app, which we will call Great Scott, a standard CRUD blog. Any user that visits the site can search posts by title (case insensitive) or author. When you log in as an admin you have the ability to create and publish posts. Only the admin can see unpublished blog posts.

The timing attack vector will be abusing the search feature to grab the title of unpublished blog posts by measuring the time it takes for the server to respond.

Minimal Proof

As the attacker, the easiest place to start is creating a minimal test to see if a timing attack is possible. The goal of a minimal test isn’t to fully extract the secret title. We’re going to pass two search queries and measure the results. One will match a piece of the title and the other won’t.

Let’s set up our lab environment to remove as many variables as possible for the timing attack. We’ll be running our exploit scripts and app server on the same box so jitter caused by going over the open internet won’t be an issue. Also we’ll kill any resource hungry processes. Finally we want only one blog post to exist in our application and it should be unpublished. The rest of the guide assumes the title of this post is “Great Scott! We Must Get Back to the Future”

Before running any scripts yourself make sure you only have a single unpublished post! Additional posts adds “noise” and we’ll discuss overcoming that later. You can run the setup commands listed in the README to reset the database and create this single unpublished post.

To start we’ll build a simple method to wrap the search route. For all intents and purposes this is basically curl.

Next we want to measure how long it takes for the server to respond. We’ll wrap our get method above with a benchmark to measure the time it takes for the server to respond in milliseconds:

Now we’re ready to actually build our test case. We’ll test two search terms: ‘scott,’ and ‘blake’. ‘scott’ is in the unpublished post’s title while ‘blake’ is not. We’ll run 100 trials each by passing the term into the previously defined benchmark_get method. The results variable contains the search term and an array of runtimes.

See full script here

Finally let’s do some number crunching on the results.

Term: scott Trials: 100 Min: 87.3 Max: 135.3 Mean: 95.7 Standard Deviation: 6.7

Term: blake Trials: 100 Min: 75.6 Max: 161.1 Mean: 84.7 Standard Deviation: 11.5

( Results will vary )

The first thing to note is that there is a noticeable gap between the averages. “Scott” took ever slightly longer to run. Approximately 10ms longer to run then “blake”. That’s seems like a positive result but how can we be sure? Request times can vary after all, just take a look at the minimum and maximum result for each term. Maybe if we ran this script again “blake” might come back with a higher average? Maybe this was just dumb luck?

Rephrasing those questions in a more scientific way: how do we know the results are statistically significant? That’s where computing a confidence interval comes in. A confidence interval gives us a range that likely includes the true mean. You can read more about how this value is computed here.

Term: scott Confidence Interval 95%: [94.3, 96.9] Confidence Interval 99%: [93.9, 97.4]

Term: blake Confidence Interval 95%: [82.4, 87.0] Confidence Interval 99%: [81.6, 87.7]

For “scott” we are 99% sure that our confidence interval of 93.9ms to 97.4ms contains the true mean. For “blake” we are 99% sure that our confidence interval of 81.6ms to 87.7ms contains the true mean. Since there is a gap between the low end of scott’s range and the high end of blake’s range we can draw a reasonable conclusion. Yes there indeed is a measurable difference when a search query contains text from an unpublished post title.

Extracting Title Length

Now that we’ve established that there’s a measurable difference, how do we extract the full title? We could just start by feeding single characters into a query. Doing so would allow us to confirm there’s say one or more ‘A’ characters somewhere in the title but we wouldn’t know the location. Also how would we know when we’ve extracted the full title and can stop? Wouldn’t it be nice if we could determine the length of the secret title first?

Take a closer look at those search options. “Use a ‘?’ for single character wildcards” Let’s put it to good use.

See full script here

We feed in a series of queries with an increasing number of question marks. e.g. “?”, “??”, “???”

Let’s chart the mean:

Our title is “Great Scott! We Must Get Back to the Future” which is 43 characters. Notice the dip in results between between passing queries with 43 and 44 question marks? We’ve determined the character length of the secret blog post from the attacker’s perspective.

There’s an additional benefit to extracting the title length first. When we move out of the lab environment and have multiple posts in the system, then published posts will add processing time and thus help “cover up” unpublished posts. As the attacker, being able to filter by title length means an unpublished post with a unique character count is ripe for the picking.

Coup De Grâce

To get the first letter of our hidden post title we can iterate from ‘a’ to ‘z’ with 42 wildmark characters (question marks). That makes 43 characters total.

Let’s chart the mean result for each letter:

Notice the “g” stands out? We’ve extracted the first letter of the post. Before we go adding a loop and calling it a day we should consider two additional things.

First is that we are only querying for alphabetic letters. What happens when our script hits the “!” character? We need a way to differentiate when the character we’re scanning for doesn’t exist in our list. Ultimately we can skip these characters and come back after extracting the alphabetic characters with a list of educated guess characters.

If the leading result doesn’t stand out then we can conclude that the character doesn’t exist in our list. An easy way to check this is to calculate the difference (delta) between the leading character’s mean and mean of all results. In our algorithm we’ll set a delta threshold value that determines whether we accept the winning result or leave the character as a wildcard “?” and continue on.

The second issue is the huge number of requests this is going to take. So far we’ve been using 100 trials/requests per a query term. The minimal proof script resulted in 200 requests total. Extracting the title length took 6,000 requests. If you’ve been following along with the repo you will have noticed a significant jump in time to run those two scripts. Now if we stay our current course it’s going to take us 111,800 requests. (43 characters in the title * 26 characters * 100 trials )

Can we dial back the number of requests we’re running per character? Well, as we decrease the number of requests the outlying results are going to have a greater impact. Take a look back at the results for the initial proof of concept test. “blake” actually had the higher maximum result even though it’s average was significantly less than “scott”. Over 100 trials this result was “smoothed” over by a series of typical results into a mean that was less than “scott”. As we decrease the number of trials we’re going to get less of this smoothing effect and our mean will include more noise.

One way of working with less data is checking whether the data is tightly grouped or contains big outliers. If the data contains huge outliers we can throw out the results and try again. We can use standard deviation for this but that value will scale as the run time changes and we would need to keep constantly adjusting our threshold. We can normalize this value as a percentage by dividing the standard deviation against the mean. This is called Coefficient of Variation.

By doing this check we can reduce the number of trials per character to 10. Sometimes the winning character isn’t going to be clear. In those cases we’ll have to throw out the results and try again. Even so, this is going to come out to considerably less requests than sticking with 100 trials each for our upcoming timing attack.

Let’s put it all together in the final code.

See final code here

Note that depending on your environment you may need to tweak the constants at the top of the script. Here’s a snippet of output:

Trial 1 of 10 Trial 2 of 10 Trial 3 of 10 Trial 4 of 10 Trial 5 of 10 Trial 6 of 10 Trial 7 of 10 Trial 8 of 10 Trial 9 of 10 Trial 10 of 10 Exporting results to csv... tmp/1548447356.csv Delta: 16.7114306526752 Coefficient Variation: 9.705968406065965 Current Secret: great sc*??????????????????????????????????

Additional Optimizations

There’s plenty more you could do to optimize this further.

The attacking script could be optimized by short circuiting after one character’s result has an anomaly.
The attacker could use letter frequencies to start with more likely characters instead of iterating A-Z.
For performing the attack over a network, the demo app freely gives out the the X-RUNTIME header which reports how long the server took to process the request. The attacker can use that header instead of benchmarks to get jitter-free results over a network.

DE—FENSE!

Fundamentally there are three ways to mitigate a timing attack.

Eliminate or decrease the time difference between conditional branches that depend on private information.
Decrease number of requests an attacker can make.
Increase variance which forces the attacker to make more requests to overcome additional variance. (e.g. Adding jitter with sleep rand)

Ultimately if the response time from the server is the same regardless of private information then a timing attack is not possible. Increasing jitter should not be considered effective by itself, because jitter can be cancelled out through statistical means.

Specific to our demo application here’s the culprit code:

More Context Here

First let’s note that this is poorly written code for the sake of demonstration. We collect a list of posts filtered by title. This is done using a SQL query, so it’s fast. Then we inefficiently iterate over each individual post to filter by author. Finally we decide whether the posts should be visible if the post is published or if the user is logged in as an admin.

This code can be optimized to use a single query instead of iterating — which avoids those extra milliseconds of delay the attacker needs. If we absolutely must iterate then we should swap the order of these filters. Gather a list of published posts first, then filter by title. Doing so avoids iterating over hidden posts by their title and thus can’t reveal information about the title.

Running our minimal test from earlier results in:

Name: scott Trials: 100 Min: 76.8 Max: 111.9 Mean: 82.3 Standard Deviation: 4.8 Confidence Interval 95%: [81.38422898750433, 83.26669763389363] Confidence Interval 99%: [81.07523792180912, 83.57568869958884]

Name: blake Trials: 100 Min: 74.0 Max: 109.4 Mean: 81.8 Standard Deviation: 5.7 Confidence Interval 95%: [80.72481552206241, 82.96866577575187] Confidence Interval 99%: [80.35650676830025, 83.33697452951404]

This is a good sign.

The confidence intervals no longer have a gap like they did prior to our server side change. However, this does not definitively prove that a timing attack is impossible.

For the attacker, a gap between two search terms is a positive result but lack of a gap isn’t necessarily a negative result. As the number of trials increases the confidence interval will get narrower. It’s possible that the attacker could run a higher number of trials and then there would be a statistically significant gap.

100 requests is a rather small amount. Let’s bump that up:

Name: scott Trials: 100000 Min: 69.6 Max: 8226.2 Mean: 94.5 Standard Deviation: 197.6 Confidence Interval 95%: [93.30207952605443, 95.7513753943521] Confidence Interval 99%: [92.91469089382367, 96.13876402658285]

Name: blake Trials: 100000 Min: 68.6 Max: 5974.0 Mean: 94.1 Standard Deviation: 200.2 Confidence Interval 95%: [92.81101468120839, 95.2932841341303] Confidence Interval 99%: [92.41841083916462, 95.68588797617407]

Even with 100,000 requests there still is no statistically significant finding. At this point the attacker would be entering DoS territory. In order to call our demo app “fixed” we should add a request throttling library. (In the Ruby world that would be rack-attack) Thus even if an attack is theoretically possible with a higher number of requests it becomes impractical because of the limited throughput.

Conclusion

Most people either don’t understand a timing attack, or see them as an esoteric issue with which they don’t need to concern themselves. I hope that with this post you now see that innocent decisions made during an application’s implementation could lead to potential data leaks vulnerable to a timing attack, and give you some ideas on how to address one.

Drop me a line, or submit a pull request if you want to dive in further!

The post Great Scott! Timing Attack Demo for the Everyday Webdev appeared first on Simple Thread.

A Fictional Account of How I Invented React: A Simplified Perspective of How React Works

Simple Thread — Tue, 19 Feb 2019 13:30:36 +0000

This past year I decided to catch up on the latest in JavaScript frameworks and learn React. Actually, it was it bit more than just learning React as I also was eager to pick up ES6 and Node. Anyway, after experiencing the complexities of Angular, Backbone, Ember, and Knockout, I found React’s clarity amazing. That said, I did, however, think that the couple books and dozens of articles I read overly complicated the simplicity of React.

I adopted a technology comprehension strategy from a mentor, Eduardo Ross (VP of Technology at ASNA), who said: “I consider how I might have coded it. And, if it’s well designed, I’m usually right.” Anyway, as I began to read more about the intricacies of React that were developed to handle edge cases my mind began to create a simplified view of how React works. I began to ignore these added features and instead develop a simplified perspective of why React was created and how it was implemented.

The following is a work of fiction but it is based on the elegance of the React framework. The story is false but the method names and behaviors are real — names of the innocent were not changed to protect them.

One Language and No Salad Dressing

I’ve been at this Web thing since before Perl scripts, PHP, and the Java Servlet API. I’ve been an author and trainer and, I’ll tell you, this mix-and-match of HTML and INSERT_LANGUAGE_HERE in Java Server Pages (JSP) or Groovy Server Pages (GSP) or Embedded RuBy (ERB) was madness. Then LiveScript came out (sorry, JavaScript) and things got yet more complicated. Using multiple languages in one file is like mixing oil and vinegar — in that they don’t mix.

Actually, that’s salad dressing, but, to make it palatable, you have to add zest and vigorously shake. It’s like your web code needs a comment: Shake brain vigorously before trying to maintain.

What I wanted (what we all wanted) was one language. One language that would rule them all. A language used to create the browser’s Document Object Model (DOM). The obvious language to use is the world’s most prevalent and pervasive — JavaScript. But its syntax would need to be similar to XHTML. So let’s call it JSX for JavaScript XHTML. The following shows the simplest React JSX you might write:

You can try this code on Codepen.io. Codepen manages the loading of the required React libraries. ReactDOM, in the above code, makes the Header 1 element render into a DOM node with the ID of root. Your React apps might have a minimal HTML page as follows:

<div id="root"></div>

But you could use React on any HTML. The <h1> contained within the element variable above looks like HTML. It is not. It is a JavaScript function called h1, implemented with the React libraries. In fact, during my delusions of creating React, I wrote JavaScript implementations for all the HTML tags. Like I said: no HTML, only JavaScript. (If you are still in ES5-land seeing ES6 transpiled to ES5 might be interesting)

The trivial example above shows the simplicity of React but, it really is not yet very useful. The pages of our applications need to be dynamic.

How might I pass data to one of these JSX components? Following our conceptual mapping of HTML syntax to JavaScript, we could use attributes. The image (img) tag, for example, has, besides the ever present id attribute, src, alt, height, and width. But, for our custom components, we will be creating our own non-HTML JSX functions and we can use any set of attributive name-value pairs. Consider the following:

The above code defines a custom JSX element called Welcome. The usage of the Welcome JSX element is in the first argument of the call to ReactDOM.render specifies two attributes: name and onClick. Name is set to simple text but onClick is an inline definition of an ES6 fat arrow function. So, using any name-value pairs we can pass any kind of data including function implementations.

I said name-value pairs but, actually, all the attributes with their values are passed as a JavaScript object. So, it’s more correct to say key-value pair.

If you try the Welcome code on codepen you’ll see the value passed via the name key is used in the “Hello from” string. Furthermore, if you click on the text, you’ll see the following in the console log:

click [object HTMLDivElement] div-id

Understand that the function executed from within the Welcome JSX function. The above code passed a JavaScript ES6 anonymous function inline with the attribute name of onClick. After all, JavaScript functions, as first class objects or citizens, can be passed around via arguments.

I’m sure you know all about JavaScript handlers for HTML events, such as those listed at w3schools. Various DOM elements generate these events. Before React, we set onclick attributes directly in HTML source code to specify JS event handlers. This gave way to unobtrusive JavaScript and the binding of element events of DOM nodes in separate scripts using jQuery or addEventListener. With React you specify the event in either inline JS code or pass a reference to a function.

Note: I did have one problem with the naming of the attribute: I found I had to use mixed-case for the handler to avoid confusion. In fact, stick with camelCase for all your JSX attributes. (Check out the React docs for DOM elements)

My use of the term, attribute, is no longer broad enough to describe key-value pairs passed to React JavaScript functions. Let’s call them properties, or just props. Properties are the magic that turns components into dynamic elements — they allow the embedding, nesting, and reuse of components.

JavaScript eXpressions

JSX statements are expressions. Their name and properties describe what behaviors their JavaScript implementations provide. Earlier I said JSX stood for JavaScript XHTML but JavaScript eXpression is more articulate.

As React is all JavaScript, your custom React component can contain other custom JSX components (as well as the standard HTML components like h1, p, ul, li, etc.).

To keep functional React components in the pure function category, their props parameter should not be modified inside the component. Or, in functional programming terms, they are considered to be immutable objects. You can always re-invoke the function with a new props object, But that gets tedious.

Class

Our complex business requirements necessitate the maintenance of application state. In object oriented programming a class is a template used to instantiate objects that contain state and methods/functions that manipulate that state. So, begrudgingly, in my invention of React, I’m going to need classes. When the state of an object instance of a React class changes, React will automatically update the browser’s DOM. To be very specific about the state that a React class is prepared to react to, those values must be contained within an instance property called, well… state.

Codepen.io hosts a simple example of a React component. The following shows the implementation of the constructor function for the component:

JustMe React application in Codepen.io

JustMe’s state contains mental state and state locality. (Your applications may be a bit more complex.) State is a JavaScript object that can be as complex as you’d like. What I envisioned was for React to respond to changes in the state object by turning on the properties watering hose and dowsing its nested JSX with the updated values. React would then update the DOM to reflect the values in the modified state. To manage the massive process of updating the DOM with state changes, I minimized it by requiring changes to state to be done in a function called setState. After calls to the setState method complete, the DOM refresh runs.

Consider coding state management yourself…. You have to implement some implementation of the Observable pattern. Just pondering on it now, brings back of painful memories working with updates to Angular 1’s $scope.

The handleStateChange method below embeds a call to setState change. I’ll explain in a bit why the state key is defined as a computed property. For now, suffice to say that, after setState runs, all nested JSX that display values for mental or locality will be refreshed.

Cascading Properties

The render method of your React class is where you tell React what JSX is to be, well, rendered as DOM. All the JSX functions therein are invoked passing, not just the key-value pairs of attributes in the props variable. but optionally also the state.

The codepen was created to show the flow of state through <JustMe>, <LevelTwo>, and <LevelThree>. The first line of the return clause of the two Level functions does a console.log of the passed properties:

console.log('LevelTwo props: %O', props)

The result of which, when the page loads, is:

▾ "LevelTwo props: %O" Object { locality: "VA", mental: "hanging on", name: "LevelTwo", ▸ onClick: function click(e) {↔} ▸ onStateChange: function (e) {↔ }

Notice the state values of mental and locality (that were initially set in JustMe’s constructor) are passed. When these state values change in their parent component (JustMe), those props values are passed down to children and trigger a re-render of descendant components that use them (LevelTwo and LevelThree.) So in addition to cascading properties, you also get cascading updates and selective re-rendering of stuff that’s changed. Pretty cool. As I said, properties are the magic that enables the propagation of values to nested components.

Lifting State Up

LevelThree has inputs for the mental and locality values. That component is, well…, three levels down from JustMe. I explained how the values cascaded down the nested component tree, but how is the state object — maintained by the root component JustMe — going to be updated?

Property values cascade down but state updates are lifted up!

JustMe passes its handleStateChange method to LevelOne with:

onStateChange={this.handleStateChange}

I could have used any object key to hold a reference to handleStateChange, including handleStateChange but I used onStateChange to show you can use whatever name you prefer for your property’s object keys.

LevelTwo then did the same and passed everything to LevelThree. But, rather than explicitly specifying the key-value pairs as JustMe did when it called LevelThree, in LevelThree I got lazy and passed the full properties object:

Actually, it used ES6’s spread operator (...props) to create a new object. Remember: The key-value pairs specified in a JSX are passed as a JavaScript object to the function that implements the JSX.

At any rate, the LevelThree method has a reference to JustMe’s handleStateChange function. So, when the user modifies the mental or locality input fields, their DOM’s onchange event is handled by the passed properties onStateChange reference. Which, we know, came originally from JustMe. So state changes are “Lifted Up” from the following LevelThree input fields to LevelTwo back up to JustMe:

When It’s Time To StageChange (You’ve Got to Rearrange)

Change is tough, just ask Peter Brady. Now I can explain to you the implementation of handleStateChange:

It takes one argument, e, which, when onchange calls it, will be the event object. The event object has a bunch of properties one of which is target. Target has a property called name. Well, here’s the gimmick: LevelTwo defined the values for the input fields to match the object key names defined in JustMe. So, when mental state is changed to “completely nuts” the following code in:

obj[e.target.name] = e.target.value;

Executes essentially as:

state[mental]: “completely nuts”

Where the brackets contain any JavaScript expression that converts to an object property name.

This is really crazy, and I’m so impressed with my React code (which I delusionally created) that updates the DOM: but, as you type “completel” — “y nuts” not yet typed — in the mental input field, you can watch the text displayed in the JSX in JustMe as:

{this.state.mental} in {this.state.locality}

Changes from:

JustMe’s state:hanging on in VA

To:

JustMe’s state:completel in VA

For added kicks, you can click on the text displayed on the page in the div sections of LevelOne or LevelTwo and the console will display:

“LevelTwo-div clicked” or “LevelThree-div clicked”.

Note: I defined the click function stand-alone just to be different as it wasn’t necessarily a behavior specific to the JustMe component.

No Thanks to Dad

Why no inheritance? Inheritance for React provides little. In fact, React is more about nesting JSX (composition) than inheritance. A well accepted principle of OOP is: “Prefer composition over inheritance.” Actually, even though the power of classes is available in React, I recommend you start with functional components and only use class components when state is required.You will find use of pure functions will make your code easier to understand, test, reuse, and maintain.

Getting Real

OK, OK, I didn’t invent nor have anything to do with the development of React. But you already knew that because you’ve already checked Wikipedia and React’s commit log.

But, as I said, React’s clarity is amazing. It just makes sense — especially when compared to other JavaScript frameworks. React certainly has its own complexities (e.g. callbacks and context) but, for the most part, your code will be more clear without them. That said, you probably should look at React lifecycle events. I’m also quite excited about React Hooks as they let coders package state and side effects in pure functional components.

A few recommendations:

Use codepen.io :Read (and copy) other mini React apps. I want to point out that, often, when React code looks complicated, it is ES6 (or advanced JS) not React that is complicated. I had just picked up ES6 before learning React, which was good because ES6 features like spread, rest, and de-structuring are used heavily in React apps.

Get comfortable with ES6 : I highly recommend the book Simplifying JavaScript.

The reactjs.org Getting Started is great and makes great use of codepen.io. I also got a lot out of the vides at the Material UI getting started.

Hey! I even came up with a clever logo for this new library!

The post A Fictional Account of How I Invented React: A Simplified Perspective of How React Works appeared first on Simple Thread.