DEV Community: MichaelPaulKunz

What We Can Learn From Sir Charles Antony Richard Hoare

MichaelPaulKunz — Thu, 20 May 2021 20:22:43 +0000

Sir Charles Antony Richard Hoare, inventor of quick sort and the null reference, is a giant in the world of computing and technology. His name, tailor-made for minimum word counts, can be shortened to C.A.R. Hoare or simply Sir Tony Hoare, his knighthood differentiating him from the cyclist.

Hoare was born in present-day Sri Lanka to British parents and studied philosophy at Oxford. He now teaches computer science there. On his journey from Oxford back to Oxford, he lived in the Soviet Union for some time and became fluent in Russian. He developed the quick sort algorithm and the null reference -- for which he has apologized -- among other algorithmic innovations and models such as the Hoare Triple, and the Dining Philosophers Problem used to describe concurrent processes in multi-threaded languages.

Hoare has received many awards and nominations, including the 1980 A.M. Turing Award recognizing his "contributions to the definition and design of programming languages." Sir Tony received the Faraday Medal in 1985 and the Kyoto Prize in 2000. Also in 2000, he was knighted by Queen Elizabeth II, certainly a rare honor for a coder. In addition to his supreme intellect, Sir Tony is a bastion of humility and grace, in many ways pioneering the "soft skills" that we strive for in order to perfect our culture of code. Here are a couple of values we can learn from Sir Tony's example.

Simplicity

You have to be highly intelligent to succeed in the field of software engineering. Routinely, concepts like block-chain technology are wheeled out on late-night talk shows as examples of impenetrable puzzles that nobody understands. But intelligence has a dark side. Trying to one-up other coders and position yourself as the smartest person in the room can tempt you to talk over somebody's head when direct communication is more effective. Or to write overly complex code that could invite unexpected consequences down the line. To this impulse, Sir Tony says the following:

"There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies."

There are many ways to make your code impenetrable. There's the "spaghetti JavaScript" method of chaining together higher-order functions until it reads like several lines of arrows with no context. Insufficient commenting. Using recursion when a dynamic solution would work just as well. It might feel good to get your code as concise as possible. Solving something complex in just a couple of lines is surely a feat to be proud of. But it's more useful to write code in a way that another developer could look at and instantly understand, even if it doesn't flex the algorithmic muscles you've spent your career developing. Hoare is aware that as one's resources expand, the tendency is to do as many cool things with them as possible. And so he elaborates on the importance of simplicity even in the face of expanding ability:

"The price of reliability is the pursuit of the utmost simplicity. It is a price which the very rich find most hard to pay."

Humility

In spite of all his accolades, Sir Tony can still admit when he's wrong. This can be one of the most difficult things for a successful person to do. He has called the null reference his billion dollar mistake, citing the lost time and money null-related problems have cost developers and technology companies throughout the years. I'm sure Joseph Tartaro would agree with Sir Tony on this point, after getting "NULL" on his license plate caused every parking ticket with an unknown license plate to fall on his shoulders.

This is an important lesson for coders everywhere with impostor syndrome. And for every coder who makes a silly mistake and loses hours to it. Imagine being on the front lines of writing code and developing stuff that gets used for decades. And you mess something up. And watch it go into effect and spiral out into more and more problems every year. That would be devastating, but Hoare takes it in stride, giving speeches and writing volumes on how big of a mistake it was. Definitely makes forgetting that "this" references the outer scope inside an arrow function seem pretty tame in comparison.

And that's Tony Hoare in a nutshell. An immensely important figure in the tech world who is humble and plainspoken despite his looming status. We can all learn a lot from his words and his example.

Cited

Eloquent JavaScript
Antony Hoare Wikipedia
Similarly-named Cyclist
Hoare Triple
Dining Philosophers Problem
Turing Award
Other Acknowledgments
Billion Dollar Mistake
NULL License Plate

HTTPS

MichaelPaulKunz — Tue, 11 May 2021 00:34:00 +0000

Most websites have a url that starts with https://. To the left of the url, there's a padlock icon that opens up an info box when you click it.

The info box lets you know that the site's connection is secure. You can send your credit card number through the site, and your information will be safe.

Some older sites are missing the 's'. They have urls that start with http:// and instead of the lock, they have a triangle with an 'i' inside. Clicking it brings up a different info box.

It tells you that your connection to the site is insecure, and advises against divulging any private information like credit card numbers or passwords.

Security

On July 24, 2018, Chrome started labeling any sites that didn't use https:// with the insecure tag you see now[1]. At the time, many prominent sites including BBC.com and Hilton.com were still using the insecure http://. Early developers will notice that their localhost projects start with an http url, as do their first deployed projects. But you'd be hard-pressed to find a major site these days with an http instead of an https url. This could probably be attributed to the stigma of having a big "insecure" thumb-print at the top of your page. Also, most news sites are looking for subscribers now, and they need to securely handle credit-card information so that users can sign up. It's weird to think that as recently as three years ago, prominent sites were still hosting insecure connections, but now any serious website needs to use https. In this blog, I'll try to explain how https works and lay out some starting points for adding the S to your url.

HTTP Requests

HTTP stands for HyperText Transfer Protocol. It uses Transmission Control Protocol (TLS) and User Datagram Protocol (UDP) to send information from one computer to another[2].

The entire internet runs on http requests. If you have Postman, open it up and do a get request on the url for any website.

You'll receive the html for that website. When you visit a page, your computer sends a get request to that page, which sends back the html, and your browser processes it to display the website.

HTTPS

HTTPS requests are in essence the same as HTTP requests, but with an added layer of security. While HTTP runs on port 80, HTTPS runs on port 443, and encrypts the request in Transport Layer Security (TCP)[2]. HTTPS stands for HyperText Transfer Protocol Secure or HTTP over TLS / HTTP over SSL.

HTTPS requests are encrypted by a method called public key cryptography. When your browser sends the get request to a url, the url first sends back a certification from some Certificate Authority (CA) that your browser trusts. The Certificate Authority also has a public key that your browser knows and can use to encrypt the information in the get request so that any third party listeners will receive an untranslatable jumble of characters instead of your request's exact text. The site's server can then decrypt the encoded message with a private key that only it has. For the duration of your https session, all communication is sent back and forth encrypted with these private and public keys. The kind folks over at kubucation created this info-graphic that breaks down a request to youtube.com. [3]. In youtube's case, the Certificate Authority comes from Google.

Converting to HTTPS

Michal O'neal has a great article about converting your site from HTTP to HTTPS[4]. It involves four main steps :

Buying an SSL Certificate (preferably from your web hosting service)
Installing the web hosting certificate on your account
Switching all internal linking to HTTPS
Notifying search engines with 301 redirects.

The final step can be automated with a CMS plugin. Notifying search engines is important. Especially since one extra advantage of using HTTPS is that you get an SEO boost.

Citations:

1: You'll Never Believe the Sites Still on HTTP

2: Difference Between HTTP and HTTPS

3: How Does HTTPS Work?

4: How To Convert HTTP to HTTPS

Sequelize's "Super Many-to-Many" Associations

MichaelPaulKunz — Tue, 04 May 2021 03:31:10 +0000

A common way to explain many-to-many models in relational databases is the book-author relationship. An author can write many books. And a book can be co-written by two or more authors. If we draw lines connecting a collection of books to their authors, we'd have multiple lines matching some books to many authors and most authors (J.D. Salinger not included) to their many books.

If we map this in Sequelize, our schema for books looks like this:

const Books = sequelize.define('Books', {
  id: {
    type: DataTypes.INTEGER,
    primaryKey: true,
    allowNull: false,
    autoIncrement: true,
    unique: true,
  },
  title: {
    type: DataTypes.STRING,
    allowNull: false
  },
});

And our schema for authors looks like this:

const Authors = sequelize.define('Authors', {
  id: {
    type: DataTypes.INTEGER,
    primaryKey: true,
    allowNull: false,
    autoIncrement: true,
    unique: true,
  },
  name: {
    type: DataTypes.STRING,
    allowNull: false
  },
});

"Books" doesn't have an "Authors" key, and "Authors" doesn't have a "Books" key. Their relationship is defined in a separate join table that references the book and author IDs.

const Writings = sequelize.define('Writings', {
  id: {
    type: DataTypes.INTEGER,
    primaryKey: true,
    allowNull: false,
    autoIncrement: true,
    unique: true,
  },
  BookId: {
    type: DataTypes.INTEGER,
    references: {
      model: Users,
      key: Users.id,
    }
  },
  AuthorId: {
    type: DataTypes.INTEGER,
    references: {
      model: Merchants,
      key: Merchants.id,
    }
  },
});

In Sequelize, we define many-to-many relationships with "belongsToMany" and "through."

Books.belongsToMany(Authors, { through:  Writings});
Authors.belongsToMany(Books, { through: Writings });

Once this is all tied up in our schemas, and we've filled out our tables, we can query the Authors table, including the books they've written, with the following command:

Authors.findAll({
    where: {},
    include: {
      model: Writings,
      include: Books
    }
})

This will give us all of our authors, including an array of the books they've written under the key "Books." This is all fine and good if there is only one relationship between authors and books. But authors can also READ books. What if we were mapping a database that collected authors, the books they've written, and also the books they've read? Sequelize's usual many-to-many model doesn't cover this situation, since the joined table will come back as its original table name (Books) instead of its join table name (Writings). This is a job for Sequelize's Super Many-to-Many Association schema.

Whereas "belongsToMany" is used to map many-to-many relationships, "hasMany" and "belongsTo" map one-to-many relationships. Instead of directly linking books and authors with belongsToMany, we can link "Books" and "Writings" with a "hasMany" command, and also link "Authors" and "Writings" the same way. That code looks like this:

Books.hasMany(Writings);
Writings.belongsTo(Books);
Authors.hasMany(Writings);
Writings.belongsTo(Authors);

We never explicitly bind Books to Authors, but once we've done this, we can run the same query command from earlier, and the array of books each author has written will come back under the key "Writings" instead of the key "Books." This means that we can declare another join table called "Readings" and map out a totally separate many-to-many relationship.

const Readings = sequelize.define('Readings', {
  id: {
    type: DataTypes.INTEGER,
    primaryKey: true,
    allowNull: false,
    autoIncrement: true,
    unique: true,
  },
  BookId: {
    type: DataTypes.INTEGER,
    references: {
      model: Users,
      key: Users.id,
    }
  },
  AuthorId: {
    type: DataTypes.INTEGER,
    references: {
      model: Merchants,
      key: Merchants.id,
    }
  },
});

We do everything we did for Writings, but with Readings instead:

Books.hasMany(Readings);
Readings.belongsTo(Books);
Authors.hasMany(Readings);
Readings.belongsTo(Authors);

And likewise change "Writings" to "Readings" in our earlier query to get back a list of authors with the books they've read:

Authors.findAll({
    where: {},
    include: {
      model: Readings,
      include: Books
    }
})

I'm working on a project where I needed to plot multiple many-to-many relationships between tables, and the Sequelize docs only ever got me half the way there. They explain how to implement the "super many-to-many" association, but they don't really explain what it's useful for. So I had to follow a hunch that maybe it was the thing I needed. And then I had to search for old requests in help forums to get the right syntax for the calls.

You can reference multiple join tables in your query by defining your includes key as an array. So you could query the Authors table and have it include both the books they've written and the books they've read with this command:

Authors.findAll({
    where: {email: email},
    include:
    [{
      model: Writings,
      include: Books,
    },
    {
      model: Readings,
      include: Books
    }
  ]})

You can also nest "include" fields. So if you wanted to get back a list of authors, including the books they've read, and have each book include its own authors in its own list, you could write this:

Authors.findAll({
    where: {},
    include:
    {model: Readings,
      include: {
        model: Books,
        include: {
          model: Writings,
          include: Authors
      }
     }
    }
  })

Helpful links:

Wikipedia Many-To-Many

Sequelize Docs

Writers Who Only Wrote One Novel

Create Native Apps with React Native and Android Studio

MichaelPaulKunz — Mon, 26 Apr 2021 02:12:29 +0000

If you've been making web pages with JavaScript and React, and you want to try apps that users can download and install onto their phones, then React Native is the path of least resistance. React Native is a library that converts JavaScript and React code into the native langauges used by both Android and Mac devices. It takes the place of ReactDOM, which you use for regular web pages. You can install it globally with the following command:

npm install -g react-native-cli

This tutorial won't cover deploying your app to the Mac or Android store where a user can download it. But we will go over getting started, explore some basic React Native syntax, and test our code with Android Studio. I'm working in Ubuntu 18.04 and some of my commands are Linux specific. Also, I'll focus on running an Android app instead of an iOS app.

Android Studio

Since we're not making web pages that will render in a browser, we won't be able to simply run a local server and test our code in Chrome. Instead we'll use Android Studio to display our output to a cell phone emulator. To run Android Studio, you'll first need a Java Development Kit. Linux users can install OpenJDK and Android STudio with the following two commands:

sudo apt install openjdk-8-jdk

sudo snap install android-studio --classic

I should mention that installing Android Studio is not always easy. You can find more detailed instructions for installing to Ubuntu 18.04 here, but even then, you'll probably do a good bit of troubleshooting. I tried to keep a log of all the error messages I received along the way and the steps I took to fix them, but it proved too exhaustive. You'll deal with your own unique bouquet of complications anyway, so just know that having set-backs doesn't mean you're doing it wrong.

One thing you'll probably want to do is configure a hardware accelerator for your virtual machine. The Android developer page provides detailed instructions for how to do that on Mac, Windows, or Linux. Android Studio can really slow your computer down without an accelerator.

Once it's fully installed, running the command android-studio in your terminal will bring up this window:

Click configure on the bottom right and choose "AVD Manager" to set up your Android Virtual Device. One may already be created for you, or you may need to make your own. I created a Pixel 2. Once it's created, you should have a line in your window displaying its name, resolution properties, and some other attributes. All the way to the right of the line are the actions. If everything is configured right, hitting the play button in actions launches your virtual machine. But you might still have to set up some environment variables. You're doing it right if you see a replica of an Android phone on your screen:

React Native
You're pretty much finished with Android Studio at this point. The rest is in VS Code with React Native. Once you have it installed globally, you can initiate a React-Native app with the following command:

react-native init <projectName>

It will create a folder named after whatever you put for projectName. You can cd into that folder and run code . to explore your boilerplate app. There will be a lot of folders and a few dependencies installed for React-Native. Go into package.json to and check your scripts object to see what commands you need to run in the terminal. You'll be running "start" and "android" if you're working on an Android app:

npm start
npm android

If everything is configured properly, your cell phone emulator will pop up whether you have Android Studio running or not, and you'll see some sample text on the cell phone screen. But it probably won't be configured properly. I know I got at least four errors when initially running the commands. Three saying I didn't have an emulator, or that it couldn't find the emulator. And one saying I had the wrong version of OpenJDK-- I had only just installed it specifically to work on this very project. In a lot of cases, getting these errors to go away was a matter of searching them on Google, finding solutions on Stack Overflow and other help forums, and trying them out until one worked. More often than not, the changes wouldn't take until I rebooted my computer, so keep that in mind if it seems like nothing's working. You've officially made it if you see your virtual cell phone running this screen:

Follow the instructions on the screen and make some minor edits to App.js so you can see the results change. Coding in React Native isn't much different from coding in React. You can still use classes or functions with hooks. You import and export files the same. Your return statement will still be JSX rendering. One big difference is you can't use <div> tags. Instead of traditional html <div> tags, React Native provides us with a predefined library of components that we need to wrap around our code. You'll import them from 'react-native' at the top of the file and use them like you'd use any other React Component. Wrap your text in <Text></Text> component tags and then wrap that in <View></View> tags to control screen placement. Use a <TextInput /> component for a form. And <TouchableOpacity /> to make an item that responds visually to being pressed on the touch screen. Give it an onPress property to make it dynamic.

Traversy Media has a very good React Native Crash Course where they walk you through building a Shopping List. You can pull or peak at their code here to see the way these Native components are used. Or check out my not-at-all derivative to-do list code instead.

I hope you've found this helpful. It's far from exhaustive, and you'll definitely need to supplement it with other sources to get up and running. React Native is a pretty useful skill to have. It can almost certainly get you hired. Here's a list of some of the resources I used to build my first React Native app and write this article.

Cites:

React Native Crash Course
React Native Docs
Install Android Studio
Install KVM Acceleration
KVM Acceleration Ubuntu

Write Your First Python-Powered Web Page

MichaelPaulKunz — Mon, 19 Apr 2021 05:22:10 +0000

Mark Wahlbeck from Devslopes thinks beginners shouldn't learn Python. I don't know if he's right or not, but I do want to stress that this tutorial is for people who already have some experience with JavaScript, have built a server with Node and Express, and want to try something new. If nothing else, building your site with Python and Django might cement your understanding of how your Node server is working. And if you get comfortable with Python, it can be a good catch-all language for automating your daily tasks, even if it never gets you hired as a junior developer.

Setting Up Your Dev Environment

Because this tutorial is for JavaScript coders, we'll be working in VS Code instead of PyCharm. I'm working in Ubuntu 18.04, so ideally this is for other Linux users, but I will include alternative commands for Mac and Windows where possible.

The first thing you'll need to do is fire up VS Code. I created a directory named "python" to work in. Install the Python extension for VS Code.

Next you'll need a Python package manager and interpreter. Ubuntu users can install Pip with the commands sudo apt update followed by sudo apt install python3-pip in the terminal. Enter pip3 --version to confirm you did it right. You'll get back something like this: pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6). Windows and Mac users can follow the respective links for instructions on installing package managers.

Linux users should have Python3 as a default Python interpreter. If you're not sure if you have it, or if you'd like to change it, you can press ctrl+shft+P and type "Python: Select Interpreter" into the search box. Click the one you want. I'm using Python 3.6.9. If you have an interpreter set up, you should have a ".vscode" folder in your project folder with a "settings.json" file. If your interpreter is set up properly, the json file will have a "python.pythonPath" key. Its value is the file path to your interpreter.

Testing Python

In your root directory, create a folder called "test.py" (touch test.py for Linux users), and enter the following two lines:

msg = "Its Alive"
print(msg)

Save the file, and run it by pressing the play button at the top-right of your VS Code window. You've probably never used this thing if you're used to writing JavaScript, so here's a picture with the button highlighted.

Pressing it should have the same effect as running node [somefile].js. If you see the text "Its Alive" in your terminal, you are officially coding in Python.

Virtual Environment

Before we install Django, we're going to create a virtual envrironment. I'm running Linux, so I'll run the following two commands in my terminal:

sudo apt-get install python3-venv
python3 -m venv env

Here are the MacOS and Windows alternatives:

python3 -m venv env (for Mac)
python -m venv env (for Windows)

Say yes to all the prompts. Your project folder should now contain an "env" directory with some folders in it ("bin", "include", "lib", ect). It should sort of look like your standard JavaScript / Node project. You're going to redirect your Python interpreter to this folder. So once again press ctrl+shft+P and search for "Python: Select Interpreter". You should see a new option with ('env') at the end. Click it, and it should update your "settings.json" file to direct to the new file path ("env/bin/python" on my end).

Press ctr+shift+P again, and this time type "Terminal: Create New Integrated Terminal" into the search bar. Click the result, and your terminal window should display a command line starting with "(env)". This is your integrated terminal window, where we'll install Django. If at any point you close your project and come back, your terminal will not default to the integrated terminal window. The following commands will bring you back into your integrated terminal window:

source env/bin/activate (Linux or Mac)
env\Scripts\Activate.ps1 (Windows)

Django

In your integrated terminal window, enter the following two commands:

python -m pip install --upgrade pip
python -m pip install django

If you're using Windows, and these commands don't work, try troubleshooting with this link.

When the installation is complete, you're ready to make a Django app. You'll start your project by running the following command in your integrated terminal:

django-admin startproject web_project .

This will create a new folder called web_project in your parent directory as well as a few utility files. Next, create your database and start your server:

python manage.py migrate
python manage.py runserver

Your terminal will give you a link to http://127.0.0.1:8000/ and you'll see the following default message on your page:

Next step is to replace the default success screen with some personalized content, which we'll do with the startapp command:

python manage.py startapp myfirstpython

This will create a "myfirstpython" folder with a "views.py" file. Replace the contents of "views.py" with the following code and save:

from django.http import HttpResponse

def home(request):
    return HttpResponse("Boilerplate Django App")

Now, in your "myfirstpython" folder, create a file called ursl.py with the following code:

from django.urls import path
from hello import views

urlpatterns = [
    path("", views.home, name="home"),
]

Your web_project folder also contains a urls.py file. Replace its code with this:

from django.contrib import admin
from django.urls import include, path

urlpatterns = [
    path("", include("myfirstpython.urls")),
    path('admin/', admin.site.urls)
]

Save and re-run the command to start your server in your integrated terminal:

python manage.py runserver

Following the link to http://127.0.0.1:8000/ should bring up a plain white page with the text "Boilerplate Django App." And there you have it. Your first webpage without any JavaScript. It's not much, but if you study Python, and follow some of the links below, you can make a fully interactive page with Django. Feel free to fork this boilerplate and run with it.

Citations:

Installing Python
Installing Pip
Django
For Windows Users
For Mac Users
More Windows Troubleshooting
Cooler and more complex Django App Code-along

Introduction to Machine Learning with TensorFlow

MichaelPaulKunz — Mon, 15 Mar 2021 04:40:58 +0000

Machine Learning

When we program a computer to "think" in ways that superficially resemble human thought, it's called machine learning. Machine learning is the building block of artificial intelligence, which powers auto-complete and fraud detection software as well as automated playlists and recommended videos for Spotify and YouTube. Machine learning is famously used in facial recognition software and in targeted ad algorithms.

Complex Algebra

Recognizing images is something the human mind does effortlessly, but it takes a lot for a computer to perform the same task. Image recognition software begins with a series of neurons, which are defined as numbers ranging from 0 to 1. Whichever number a neuron holds is its activation, and the closer to 1, the more activated it is.

We can model any grey-scale image as a matrix of neurons. Each neuron corresponds to a pixel, and its activation corresponds to its darkness of shade, with darker pixels being closer to 0 and lighter pixels being closer to 1. An AI algorithm tasked with identifying an image would start with the matrix of pixels and pass them through a series of hidden layers with progressively fewer neurons until it reaches the output layer. These layers act kind of like a game of 21 questions. The first layer may determine if the image is of a plant, animal, person, or object. If it determines it is an animal, the next layer may determine what species, and so on. In theory, the more layers involved, the more accurate the image identification will be, but each layer consumes a lot of run time memory. 3Blue1brown goes into detail about a four-layered neural network tasked with visually identifying numbers written in a wide range of styles.

The neurons in a neural network are connected through layers with different "weights" that in turn have different "biases." Each of these terms translates to a variable in a few complex algebraic equations. Two foundational equations for machine learning are the Sigmoid function, represented as s(x) = 1 / 1 + e^-x and ReLU -- rectified linear unit -- represented by the simpler and more accurate equation ReLU(a) = max(0,a). All of this heady math is beyond the scope of this article and above my pay-grade. Fortunately in computer science, we stand on the shoulders of giants, and there are libraries and APIs out there that give developers access to complex machine learning algorithms without all the number crunching.

TensorFlow

TensorFlow is an open source library developed by the Google Brain team that affords lay people the powers of machine learning. It was written in C++ and can be utilized in JavaScript or Python. You know that daunting diagram in the banner image? TensorFlow reduces the most complicated parts of that network to just a few lines of code.

TensorFlow derives its name from the tensor data structure that can store any number of scalers, vectors, or matrices. While TensorFlow's job is to abstract away the algebraic details of machine learning, it's important to note that an image is itself a matrix of pixels, and so machine learning can be powered by a GPU instead of a CPU. TensorFlow is accelerated by WebGL, an open-source JavaScript graphics API.

ml5.js

ml5 is an open source library and community that builds upon the TensorFlow library. ml5 doesn't require any other dependencies besides TensorFlow, so it's surprisingly easy to get started with. You can fork this image-recognition app from their GitHub and have the power of machine learning at your fingertips.

Conclusion

Machine learning is a rapidly developing field that increasingly powers our daily lives
Machine learning algorithms are powered by complex algebraic equations
Open source libraries like TensorFlow and ml5 give us the power of machine learning without needing a PHD in advanced mathematics

Sources:

Master of Puppets: Using Headless Chrome

MichaelPaulKunz — Sun, 07 Mar 2021 17:59:45 +0000

Imagine browsing the web without a graphical interface. This is Chrome in headless mode, without the point-and-click windows we're all used to. Accessing a page in headless mode is more efficient because your browser doesn't have to process layout, images, video, etc. Headless Chrome is useful for front-end testing. It also lets search engines and other web crawlers access the full DOM without rendering the full page. Sometimes hackers use headless mode to bypass XSS restrictions and inject malware.

In the Terminal

You can run headless mode from the command line. Append the --headless tag to a $google-chrome command in your bash terminal.

google-chrome --headless

You'll notice nothing happens. Without Chrome's user interface, we have nothing but a terminal to type commands in. The dump-dom command will display a full text-rendering of the DOM for any URL you enter after it.

google-chrome --headless --dump-dom https://example.com

Try it yourself. Even a simple page like example.com has a pretty lengthy DOM, so I included the text at this link to avoid bulking up the article. If your terminal is displaying similar text to what's in the link, you've successfully accessed example.com in headless mode.

In VS Code with Puppeteer

You aren't limited to the terminal window when browsing in headless mode. There are APIs that let you access it in your JavaScript code. This article focuses on Puppeteer, a node library with an API that allows you to perform most browser actions in your code. You'll need some version of Node to run Puppeteer. I'm using Node v14.15.4. To install Puppeteer in your project, enter npm i puppeteer into the terminal. It should add a node_modules folder and a package-lock.json folder to your parent directory. Puppeteer bundles all its necessary dependencies, so your json file will be about 400 lines long, and you won't need to worry about running any other terminal commands for it to work.

npm i puppeteer

Create a JavaScript file and start writing your Puppeteer code. You can create a screenshot of any website by entering the URL. First use node's require command to assign Puppeteer to a variable. Then use an asynchronous function to launch Puppeteer, open a headless browser, navigate to the desired site, and take a screenshot. Finally, close the browser. In this example, we take a screenshot of the Google Developer's page for Puppeteer:

const ventriloquist = require('puppeteer');

(async() => {
  const startUp = await ventriloquist.launch();
  const virtualBrowser = await startUp.newPage({headless: true});
  await virtualBrowser.goto('https://developers.google.com/web/tools/puppeteer');
  await virtualBrowser.screenshot({path: 'puppetmaster.png'});

  await virtualBrowser.close();
})();

You can enter the above code into a JavaScript file -- puppet.js -- and run the file with node puppet.js. After running, you'll have a new file in your parent directory called puppetmaster.png. It will look like this (until Google changes their developer page layout or the contents of their Puppeteer page).

Testing

Developers use Puppeteer to test the front end of their design and to do end-to-end testing. Headless mode allows us all the functionality of our browser without the costly layout rendering, so it's ideal for setting up efficient tests. Puppeteer lets us test our front-end server-side instead of client-side which is four times faster. Going into detail about testing with Puppeteer is beyond the scope of this article, but Akshay Kadam wrote a tutorial for Sitepoint about end-to-end testing with Puppeteer and Yarn.

Web Indexing

Web pages these days are written mostly in JavaScript or JSX, with the HTML page serving as a blank canvas for Angular or React to add content. This presents a problem for web indexing. If a search engine wants to collect data about a site, it can no longer just read its HTML page. Crawling websites with a headless browser is a good way to get all the relevant DOM information, not just what's in the HTML file. Eric Bidelman goes into more detail here.

Malicious Use

The same features that make Puppeteer so useful for web indexing make it a potential tool for hackers. You can bypass XSS restrictions by directly accessing a site in headless mode. While this doesn't necessarily invite scripting attacks, it does allow for easier creation of web crawlers that can mass scan sites for vulnerabilities. It is not common practice for servers to block Headless Chrome. For the legitimate developer, that means you can use its features without fear of 404 errors. Read more from David Bekerman at Imperva.

Summary

Headless Chrome is Chrome minus the window
You can access it from the terminal or in VS Code with APIs
Puppeteer is a node library with a great headless API
You can use Puppeteer for testing and web indexing
Some people who use Puppeteer are up to no good, but they haven't yet ruined it for the rest of us
Works Cited

Master of Puppets: Using Headless Chrome

MichaelPaulKunz — Sun, 07 Mar 2021 17:50:37 +0000

In the Terminal

You can run headless mode from the command line. Append the --headless tag to a $google-chrome command in your bash terminal.

google-chrome --headless

google-chrome --headless --dump-dom https://example.com

In VS Code with Puppeteer

npm i puppeteer

const ventriloquist = require('puppeteer');

(async() => {
  const startUp = await ventriloquist.launch();
  const virtualBrowser = await startUp.newPage({headless: true});
  await virtualBrowser.goto('https://developers.google.com/web/tools/puppeteer');
  await virtualBrowser.screenshot({path: 'puppetmaster.png'});

  await virtualBrowser.close();
})();

Testing

Web Indexing

Malicious Use

Summary

Headless Chrome is Chrome minus the window
You can access it from the terminal or in VS Code with APIs
Puppeteer is a node library with a great headless API
You can use Puppeteer for testing and web indexing
Some people who use Puppeteer are up to no good, but they haven't yet ruined it for the rest of us
Works Cited

Writing JavaScript Permutation Functions

MichaelPaulKunz — Mon, 01 Mar 2021 05:10:21 +0000

Let's say you're browsing a crowded outdoor market, and a wild-eyed man in a top hat waves you over. Standing behind a table with three cups in a straight line, he places a ball under the center cup and switches it with the cup to your left. He switches the right and left cups. The center cup with the right cup, the center with the left. It's too fast, you can't keep up. He says for $10, you can guess which cup the ball is under. If you guess correctly, you'll win $100.

You think you like those odds, but you're not sure. Luckily, you have your laptop and a WiFi signal. You open Chrome and enter the developer's tools to do some quick arithmetic in JavaScript.

const cups = ['left', 'center', 'right'];
let odds = 1 / cups.length;
let overUnder = 10 / 100;
console.log(odds > overUnder);

Your odds are greater than your over-under, so it's a good bet. You pay $10 and choose the center cup. No dice! Should've picked left. You can keep playing, but there's a catch. He'll double the prize money for the next round, but you have to guess correctly twice in a row. To sweeten the deal, you'll only have to pay $5. Figuring out your over-under for multiple rounds is a little trickier. With one round, there are exactly three possibilities: left, center, or right. But what about two? You express the results of each round as the item of an array. One possibility, where the ball is under the left cup both times, would look like this.

[left, left];

The second round could also result in center or right.

  [left, left]
  [left, center]
  [left, right]

Since you have to guess twice, the first round result is still unknown. It could be be right or center. So your list of potential outcomes looks like this:

  [left, left]
  [left, center]
  [left, right]
  [center, left]
  [center, center]
  [center, right]
  [right, left]
  [right, center]
  [right, right]

There are nine possibilities. You stand to win $200, and your total buy-in is $15 ($5 plus the original $10). So you consult your dev tools.

odds = 1 / 9;
overUnder = 15 / 200;
console.log(odds > overUnder);

The odds are still in your favor, but for how long? You can keep playing the game for $5 a round, and he'll keep doubling the prize money. But if you fail the second round, you'll have to guess correctly three times in a row. And four times if you fail the third round. At which round do the odds stack up too high?

To figure this out, you generate a permutation function that outputs the possible results over any given number of rounds. Permutation functions generate every possible variation of a set of elements. They are confusing and inefficient, with a factorial O(n!) time complexity. But they are the only way to obtain some data, like a word's complete list of anagrams. Or the total possible outcomes of a number of cup tricks. The skyrocketing time required to compute large, complex permutation is what keeps your passwords safe. It's possible to create so many variations that even a fast computer would take weeks or even years to exhaust all of them.

You create the shell of your flip-cup-outcomes-generating function.

function flipCup(tries) {
  const cups = ['left', 'center', 'right'];
  const outcomeList = [];
  let outcomes = [];
  /**
   * A BUNCH OF LOOPS
   * OR SOMETHING
   **/
  return outcomeList;
}

You eventually realize that no matter how you arrange a series of nested loops, they just won't get the job done. You have to use recursion. So you start building your recursive function to create each array of game results, starting with the base case.

function generate(outcome) {
    if (outcome.length === tries) {
      outcomeList.push(outcome);
      return;
    }
}

This function will only change the value of outcomes. It doesn't return anything, but you include a return statement anyway to terminate it. Now you have to loop through the array of possible outcomes, calling the function within the loop.

for (let i = 0; i < cups.length; i++) {
      outcomes = outcome.concat(cups[i]);
      generate(outcomes);
    }

You put the for-loop in your nested "generate" function, and you put "generate" in your parent flipCup function. Immediately invoke "generate" and return your completed array. Your final code looks like this.

function flipCup(tries) {
  const cups = ['left', 'center', 'right'];
  //parent array
  const outcomeList = [];
  //nested array
  let outcomes = [];
  function generate(outcome) {
    //base case
    if (outcome.length === tries) {
      outcomeList.push(outcome);
      return;
    }
    //loop through cups array, recursively call generate
    for (let i = 0; i < cups.length; i++) {
      outcomes = outcome.concat(cups[i]);
      generate(outcomes);
    }
  }
  generate(outcomes);
  return outcomeList;
}

You plug the number 3 into your function and get quite the avalanche of possibilities.

  [ 'left', 'left', 'left' ],
  [ 'left', 'left', 'center' ],
  [ 'left', 'left', 'right' ],
  [ 'left', 'center', 'left' ],
  [ 'left', 'center', 'center' ],
  [ 'left', 'center', 'right' ],
  [ 'left', 'right', 'left' ],
  [ 'left', 'right', 'center' ],
  [ 'left', 'right', 'right' ],
  [ 'center', 'left', 'left' ],
  [ 'center', 'left', 'center' ],
  [ 'center', 'left', 'right' ],
  [ 'center', 'center', 'left' ],
  [ 'center', 'center', 'center' ],
  [ 'center', 'center', 'right' ],
  [ 'center', 'right', 'left' ],
  [ 'center', 'right', 'center' ],
  [ 'center', 'right', 'right' ],
  [ 'right', 'left', 'left' ],
  [ 'right', 'left', 'center' ],
  [ 'right', 'left', 'right' ],
  [ 'right', 'center', 'left' ],
  [ 'right', 'center', 'center' ],
  [ 'right', 'center', 'right' ],
  [ 'right', 'right', 'left' ],
  [ 'right', 'right', 'center' ],
  [ 'right', 'right', 'right' ]

Twenty-seven total. You consult your over-under algorithm again.

let odds = 1 / 27;
let overUnder = 20 / 300;
console.log(odds > overUnder);

And the over-under has already overtaken your odds. No matter how many times this swindler doubles the pot, it's still dwarfed by the exponential increase in possible results. You are trying to be careful with your money, so you choose not to play another round.

Conclusion:

Permutations are taxing for both human and machine
They have a factorial 0(n!) time complexity, which is bad
Understand them, but avoid them if you can
To find the total number of permutations, without the full list, you could write a function that returns 3 raised to the power of n.
Don't give your money to shady carnival barkers

JavaScript's Broken Promise: A Single Asynchronous Thread

MichaelPaulKunz — Mon, 22 Feb 2021 06:21:49 +0000

Most JavaScript code is synchronous, executing one line at a time. This happens linearly, from top to bottom. Line 1 executes, followed by line 2. So this logs "Hello" to the console:

const hello = "Hello";
console.log(hello);

But this throws a reference error:

console.log(hello);
const hello = "Hello";

The binding at hello doesn't exist yet when we try to log it because the second line can't run before the first line. Not in this case, at least. It's easy to think of examples where JavaScript isn't quite so linear. Like basic function declarations:

const greeting = function(){console.log("Hello")};
console.log("Goodbye");
greeting();

"Goodbye" logs before "Hello" even though the "log Hello" command appears first in the code. We wrapped it in a function and then called that function later. This example is not linear, but it's still synchronous. We've simply designated a block of code to execute further down. The order of execution is still determined by each command's order in the script, and the interpreter is still handling one line at a time. Asynchronous native methods like setTimeout do away with this simplicity.

setTimeout(function(){console.log("Hello")}, 2000);
console.log("Goodbye");

This will log "Goodbye" to the console immediately and then "Hello" two seconds later. The setTimeout method takes as arguments a callback function and a number, and runs the callback function after the number of milliseconds has elapsed. At the instant that "Goodbye" is logged, it appears our interpreter is handling two commands at once, logging "Goodbye" while counting down the seconds until it logs "Hello." This is tricky because JavaScript is a single-threaded language.

By single-threaded, we mean that JavaScript uses only one interpreter, which consists of an event loop (aka callback queue) and a call stack. Synchronous code is pushed one line at a time into the call stack and then executed. Function declarations are skipped over and only added to the stack when they are called.

When setTimout delays a function and the next line runs instantly, we're not passing either into a separate interpreter. So how does "Goodbye" manage to log while "Hello" is still processing? Shouldn't the entire execution freeze for two seconds, and only log "Goodbye" after "Hello" is logged? We need some back channel either to store our delayed function or fast-track the code that comes after it.

That back channel comes in the form of an API that works with the event loop. When the interpreter reads certain functions (setTimeout, setInterval, event handlers), it passes them off to the API, which stores them in the event loop while subsequent code is sent to the call stack. Philip Roberts made an impressive visualization of this called Loupe.

Run the following code in Loupe:


console.log("Most code starts at the call stack...");
console.log("...and then runs.")


setTimeout(function asynchronous(){
    console.log("the stack sends asyncs to the API");
    console.log("API sends it to the queue");
    console.log("Queue waits for stack to empty")

}, 0);

function alinear (){
 console.log("... aren't added ....");
 console.log("... to the call stack... ");
}
console.log("declared functions...");
alinear(); 
console.log("...until they are called");

Loupe will reorganize the logged text to run in order and explain how the call stack works with the event queue and the API to handle synchronous and asynchronous functions. Two things become readily apparent. First, the number argument in our setTimeout call is 0. The function still executes last, even when it's instructed to wait zero milliseconds. Second, every subsequent line of code runs before our setTimeout finally executes. How useful is it to have a function that delays some code for a few seconds if that function always has to be the very last part of your code that will run?

Returning to our first setTimeout example, how could we rewrite it so that "Goodbye" logs after "Hello"? We could move "Goodbye" into the setTimeout callback:

setTimeout(function(){
  console.log("Hello");
  console.log("Goodbye");
}, 2000);

Problem solved. Now both "Hello" and "Goodbye" will log (in that order) after a two-second pause. This is fine for a two-line operation, but it's increasingly problematic as our code grows in size. If we were to take this approach in a larger project, all the code we want to run after the first instance of a delayed or event-triggered function would need to be stuffed into that function's callback. I learned this the hard way the first time I tried to design an interactive page with JQUERY. Each new action taken by the user required a new callback function with a new call to every event-triggered function from earlier in the code. This is known as callback hell.

JavaScript offers a better way to deal with asynchronous code via promises. JavaScript Tutorial defines a promise as "an object that returns a value which you hope to receive in the future, but not now." All this optimistic talk about the future makes promises sound like an ideal solution for issues around time-sensitive asynchronous callbacks. Indeed, couching an asynchronous callback in a promise object saves us from nested callback hell.

const hello = "Hello";

const goodbye = function(){
  console.log("Goodbye");
};

const noGreet = function(){
  console.log("I don't know why you'd say goodbye.");
  console.log("I never said hello.");
};

const greeting = new Promise(function (resolve, reject) {
  setTimeout(function(){
    if (hello) {
        console.log(hello);
        resolve(goodbye());
    } else {
        reject(noGreet());
    }
  }, 2000);
});

The above code waits two seconds and then logs "Hello" and "Goodbye" in that order. By splitting the progression of our code into two possible outcomes-- "resolve" if some conditional is passed and "reject" if it is not-- promises let us continue to build our code after calling an asynchronous function without nesting the rest of our project inside the asynchronous function call. With an example this small, there is no real advantage over nesting "Goodbye" in the asynchronous call. But for larger projects, this step is necessary to keep code readable.

Summary:

Asynchronous callback functions make our projects time and event sensitive
An API feeds these function calls to the event loop which holds them until the call stack is clear
Because the rest of the code must run before the event loop gives our asynchronous calls to the stack, we might be tempted to nest too much code inside one function
We can avoid this ugly fate with promises

Graduating From Repl.it to Dev Tools

MichaelPaulKunz — Wed, 20 Jan 2021 10:21:23 +0000

If you're like me, you're a wiz in a school setting, but real world applications test your stress levels and make you lose focus. The leap from classroom to office is a nosedive from the guiding hand of a teacher to the grimace of an impatient boss. And it means leaving the comfort of repl.it or Python Tutor for the cold reality of Chrome's developer tools. Fortunately, with some practice, dev tools can be as useful or more than those safe and easy sandboxes.

Go to any website and press Ctrl+Shift+I (in Windows or Linux) or Command+Option+I (on a Mac), and a new frame will take over the right side of your screen, pinning your page to the left. These are the developer tools. Look at the top right corner of the dev tools window. At the end of the "Elements, Console, Sources..." panel you'll see a gear icon followed by three dots in a vertical line. Click the dots. A small window should appear with a few options, the first one saying "Dock side" with four icons next to it. Click the icon that says "Dock to bottom" when you hover over it. Now the dev tools are on the bottom of your screen. Changing the dev tools' position is easy, so it's good to get the least annoying placement for whatever you're trying to do.

The best-known dev tools tabs are probably Console and Sources. Console lets you write JavaScript and compiles it for you. Sources shows the text files powering your page. If you can't find the file you're looking for, press Ctrl+P (Windows, Linux) or Command+P (Mac), and all of the source files should appear in a drop-down menu. In the main html file, you can change the text and layout of the page. The changes are only visible to you, and they revert to normal when you refresh.

If you're developing a webpage, you can load your unpublished work on a private http server and use dev tools to see what your code is doing. If you save your changes and reload your page but your changes don't stick, do a hard refresh. On Windows or Linux, that's Ctrl+Shift+R. On a Mac, Command+Shift+R. Always hard refresh when checking your code.

The Elements panel is wildly useful. It's like Sources, but it shows you the html of the DOM as it's been modified by JavaScript. This html does not exist in any actual file. It is the end result of the html file and the JavaScript files that are enacting changes on it. You can access Elements by right-clicking (Ctrl-click for Mac) any part of the page and clicking "Inspect." It will show you the html code for the part of the page you clicked and highlight the part of the page produced by the code you hover over. This is really useful if you've introduced some JavaScript that's messed up the layout of your page, and you need to figure out which element you accidentally made block-level.

In your JavaScript file, you can run a "console.log" statement, and it will show up in your console when you refresh the page. The console also displays error messages. You can supercharge the error-checking ability of your console by adding a "debugger" line to your code. Go anywhere in your code and add a line that says:

debugger;

Now save your file and refresh the page. The code will come to a hard stop on the line you put the debugger on. Since each var, let, and const you declare is function or block-scoped, you can't just type any binding name into your console and expect to see its value. You need to guide your code to the scope level where the binding is held. So you stick a debugger line inside a function declaration, conditional, or loop and then check your local bindings while the code is frozen in their scope.

Let's say you have some code like this:

let i = 0;
while (i < 10){
  i++;
}

And you want to use the console and debugger to get the value of i. You put a debugger line in the while loop:

while (i < 10){
  i++;
  debugger;
}

And then you run your code. It hits the debugger line and stops, and you can now type "i" into your console to get the value. But which iteration of the loop is it giving you the value for? The code stops as soon as it reaches the debugger line, so initially, i will be 1. But what if you want to know the value of i at the third iteration? There is a panel in your dev tools that stays put whether you're in Elements, Console, or Sources. That panel will say "Debugger paused" while your code is frozen, and it will have a few icons above that line. A play button, a dot with a loopy arrow above it. An arrow pointing down, one pointing up, and one pointing to the right. The right arrow is your "step" feature. It will walk through your paused code line-by-line every time you press it. So if you press it once, it will take you back to the top of your while loop and check the conditional. i equals 1, which is less than 10, so it proceeds into the loop on the next click. Then adds 1 to i on the next click, etc etc. A few clicks later, you can get to the third iteration of the loop and see that i = 3. This is useful for values that are being sent to functions or recursive calls and coming back different from what you expected. Put the debugger right before the function call and walk through the steps until you figure out what's going wrong.

This is just the tip of the iceberg with dev tools' ability to freeze and pick apart your code. You can go into Sources and highlight a line number of one of your source code files. The number will turn blue. This does the same thing as adding a debugger line in your actual source code. Refresh the page and it will freeze on that line. If your code never freezes, that means your interpreter is never reaching the debugger line. Perhaps it's a conditional whose condition is never met.

if (false){
  debugger;
} //your code will never pause

If you dig a little deeper, you can even trigger pauses with events like mouse clicks. Go ahead and play around in dev tools. It's not as user friendly as a pure learning environment, and sometimes it seems deliberately confusing. Why does the next line always say "undefined" after you enter a command into the compiler? Is it just to make you think you've done something wrong? Maybe. But don't let that discourage you from making the most of the tools at your disposal.