Hidden complexity in cute animal pictures

You head over to your favourite link site and click one of the pictures -- or let's be honest, you ctrl+click all the pictures. Your browser is filled with tabs of cute and cuddly little creatures. For all their simple attractiveness, getting those images on your screen is a technological marvel. Have you ever considered how much is involved?

I brought this up in a recent discussion about software development, in particular about the low quality of software nowadays. We wondered about the complexity of flying an airplane compared to something like viewing an image. Though airplane programming is by no means easy, viewing an image is a mind-boggling interplay of thousands of little bits. It's magical that it ever works!

Calling the cuddles

Let's start by clicking a link to a puppy.

URL: https‍://mortoray.files.wordpress.com/2017/10/pet-2571552_640.jpg

1. There's a thumbnail of an adorable animal but you'd like to indulge in its full cuteness. You do this by clicking on the image, or the title of the item. Glossing over a lot of details here, somehow the computer has managed to track the cursor, determine that you are clicking as opposed to scrolling, figure out which element you clicked on, and know that it is a link and should be loaded.

2. The "link" resolves to a URL, as shown with the image above. This link encodes severals bits of useful information. To make sense of it the computer first needs to parse it. It pulls out the scheme, or protocol, https. It also gets the domain mortoray.files.wordpress.com and the path /2017/10/pet-2571552_640.jpg.

3. The domain mortoray.files.wordpress.com tells us the location of the image. As the internet loves numbers more than names, we first need to translate this name into a number: an IP address. Translation is done with a service called DNS; these are remote computers that keep tables of names and numbers. Let's just skip the part about how your computer figures out which DNS server to use.

4. With the address, your computer can start talking to the remote server. As there's a lot more to the internet than just web pages, communication happens over a variety of layers. The browser speaks to the remote server with https, which is the web protocol: it knows about documents and types. This runs on top of TCP/IP which gets data from point A to B and back. If you like metaphors, you can consider TCP/IP to be a moving van. It takes stuff from one location to another never caring what that stuff is. HTTP would be the contents inside that moving van. One of those vans will arrive back at us carrying the image file.

5. A file is binary data: a long sequence of numbers. Files could contain anything, so the HTTP layer tells us what it is: a JPEG image, or rather image/jpeg in technical MIME lingo. The browser can't use this file directly, the actual image is specially encoded. It must use a tool, a software library, that can understand the JPEG format and convert it to a series of coloured pixels.

6. It's not really coloured pixels yet; it's still just more numbers sitting in the computer's memory. These need to get to your display. The browser, using one out of many possible graphics libraries, will send this data to the video card. It also instructs the video card where it should draw the graphic, including how big it is, possibly stretching the image to fill the screen. The video card further processes this data, along with data from all other running programs, and assembles what you see on the screen.

There, now we know what it takes to get charming creatures on your screen...

What else?

...or do we?

I've glossed over a lot of details. Each of those steps hides many levels of detail.

I've ignored the hardware the browser is running on. The browser doesn't get to talk directly to the video card, or to the network; it makes requests via the operating system (OS) instead. All that network stuff runs on physical cables and boxes. Suffice to say there are more protocols involved than just HTTP and TCP/IP.

I've conveniently ignored what the remote computer does. We sent a request for an image, but it has a lot of work involved in getting that file back to us. I figure you've had enough acronyms for now.

The point is that seemingly simple tasks involve a large number of components, both software and hardware. As time goes on this complexity continues to increase, as users continue to expect new features. Easy-to-use devices, like smartphones, despite their flaws, are doing an excellent job of hiding the complexity in their implementation.

Comments

[Ben Halpern]

The power of abstraction! Besides the inherent complexity, there is also the politics, business and human interplay that got us to this current state. I say this because this reminds me of this email exchange between Marc Andreessen and Tim Berners-Lee where they debate the merits of the <img> tag.

1997.webhistory.org/lists/www-talk...

[edA‑qa mort‑ora‑y]

Yes, our layers are not purely by design, more by evolution. Anytime somebody steps in to fix things, somebody else steps in to make it flashier instead.

If all this stuff wasn't abstracted it'd be impossible to write any software anymore. Just sure wish those libraries were a bit more stable. :)

[Ben Halpern]

Yeah, good abstractions are the only reason we can get anything done.