DEV Community: Valerie Foster

Getting Started With Vue.js: Directives Part 2

Valerie Foster — Mon, 21 Dec 2020 22:39:49 +0000

In my last few blog posts, I’ve been talking about how to get started using Vue.js by giving an introduction to the basic things Vue has to offer. I’ve previously covered the Vue instance, and last week I talked about a few of the Vue directives for handling how things appear on a webpage. I will now talk about the directives that make your Vue app reactive and interactive to a user.

If you know anything about basic, vanilla JavaScript, you should know that the way to make a webpage reactive is by adding event listeners. There are all different kinds of events, including clicking on an HTML element, scrolling through a page, and pushing keys on the keyboard. The v-on directive is Vue’s built in way to listen for events. Here’s an example that also uses the v-if directive, which I covered in my last post:

//  in index.js
new Vue({
  el: '#toggle',
  data: {
    condition: true
  }
})
//  in index.html
<div id="toggle">
  <button v-on:click="condition = !condition">Click Me!</button>
  <p v-if="condition">Click the Button to toggle me in and out</p>
</div>

In the example above, you can see the v-on directive in action. The part after the colon tells Vue what type of event to listen for; in this case, click. If I had instead put mouseover, then it would change every time I moved my cursor over the button. In this example, the v-on directive points to an action to make the condition variable swap between true and false. So with the v-if, the p element will toggle in and out of the page. I can also have v-on point to a function in my instance:

//  in index.js
new Vue({
  el: "#message",
  data: {
    message: "I'm a message!"
  },
  methods: {
    alertMessage: function () {
      alert(this.message)
    }
  }
})
//  in index.html
<div id="message">
  <button v-on:click="alertMessage">Click Me!</button>
</div>

This example will show an alert at the top of the page with whatever the message variable says, like this:

There are also all kinds of modifiers for v-on. Modifiers are “directive postfixes denoted by a dot”. To see a full list of modifiers, you can go to Vue’s documentation on Event Handling. As an example, there is one modifier for v-on called .once that ensures that the event can only be triggered once. So if I add it to the example from earlier:

<div id="counter">
  <button v-on:click.once="alertMessage">Click Me!</button>
</div>

The alert will only pop up the first time you click the button. Clicking the button any time afterwards will do nothing.

Other than v-on, there is one more directive that lets users interact with the data in your Vue instance. That is v-model, and it is arguably the most magic of all of the directives Vue gives you. It is magical because it does a lot of work, handles all of the interaction between an input, textarea, or select element, and a variable to associate them with. For a basic example, assuming we have a message variable with an empty string in our Vue instance:

<div id="model">
  <input v-model="message" placeholder="Type something...">
  <p>You typed: {{ message }}</p>
</div>

Just by adding v-model, this input will always show the current state of the message variable, any changes to the input will change the message variable, and any changes to message will change what appears in the input element.

There are many different ways to use v-model depending on what type of form element you are using and how you want to use the variable. When using the input types of checkbox or radio, whatever you put as the value of the input will be stored in the variable from v-model. Here are some examples:

<!-- The picked variable will change to the string value of
    "Jack" or "Jill" depending on which is selected -->
<input type="radio" id="jack" value="Jack" v-model="picked">
<label for="jack">Jack</label>
<br>
<input type="radio" id="jill" value="Jill" v-model="picked">
<label for="jill">Jill</label>
<p>Picked: {{ picked }}</p>

<!-- The colors variable is an array of checked colors, because
    you can have more than one checkbox selected at a time -->
<input type="checkbox" id="red" value="Red" v-model="colors">
<label for="red">Red</label>
<input type="checkbox" id="yellow" value="Yellow" v-model="colors">
<label for="yellow">Yellow</label>
<input type="checkbox" id="blue" value="Blue" v-model="colors">
<label for="blue">Blue</label>
<p>Checked colors: {{ colors }}</p>

<!-- The selected variable will change to whatever string is
    in the option you select -->
<select v-model="selected">
  <option disabled value="">Please pick one</option>
  <option>Lion</option>
  <option>Tiger</option>
  <option>Bear</option>
</select>
<p>You picked: {{ selected }}</p>

You can see from these examples that v-model can be used in a lot of different ways. But this is just the tip of the iceberg, and you can checkout Vue’s documentation about Form Inputs to find out more.

With this I have finished going over all the main directives that Vue.js gives you. Using these, you can make a webpage interactive and interesting to a user relatively easily because Vue does all the hard work for you, by hooking up your data to elements on the page. So far, I’ve gone over how to do things on a small scale, like having one element on a page react a certain way. But this doesn’t really work well to scale up for large applications. That’s where Vue Components come in. I’ll go over what they are and how to use them another time.

Getting Started With Vue.js: Directives

Valerie Foster — Wed, 16 Dec 2020 01:46:00 +0000

Continuing on from my last two blog posts, I’ll be talking more about some of the basics of Vue.js. I have covered how to start a project using Vue and some of the ways you can use the Vue instance in a project. In this post, I’ll go over Vue’s directives; the way to make your webpages interactive with the data in your Vue instances.

Directives are special attributes provided by Vue. They are prefixed with v- and each one applies specific reactive behavior to the HTML element you add it to on the page. The main Vue directives are v-bind, v-if, v-for, and v-on. An example using v-bind looks like this:

<div v-bind:id="someVariable"></div>

The v-bind attribute is used to access Vue instance variables from inside an HTML attribute. The example above shows the v-bind attribute being used to assign the id of the div to a variable called someVariable. Unlike referencing a variable in between HTML elements, you cannot use double curly braces to interpret variables in HTML attributes, you have to use v-bind instead:

<!-- Doesn't work, causes an error! -->
<a href={{ url }}>Vue.js</a>
<!-- This is the correct way to assign the href attribute to url -->
<a v-bind:href="url">Vue.js</a>

You can use v-bind to assign any attribute an element might have to a variable. But v-bind can get more confusing when you need to attach more than one variable to an attribute. When adding styling to a page, the class and style attributes might need to have many variables associated with them. In that case, you can use array or object syntax with v-bind, like these examples with multiple classes on each element:

<div v-bind:class="[activeClass, errorClass]">
  <!-- This will have the class names of whatever is in
    the 'activeClass' and 'errorClass' variables -->
</div>
<div v-bind:class="{ active: isActive, 'text-danger': hasError }" >
  <!-- This will have the classes of 'active' and 'text-danger'
    depending on if their variables are true or false -->
</div>

When using the style attribute, it’s often better to create a style object in the instance to bind to the attribute:

// in index.html
<div v-bind:style="styleObject">
  <!-- content -->
</div>
// in index.js
data: {
  styleObject: {
    color: 'red',
    fontSize: '13px'
  }
}

This way, if your style object gets too big, it’s not taking up space in your HTML.

The next directive uses a concept that should be very familiar to any programmer. The v-if directive is used for conditional rendering, and it evaluates the condition it is given the same way as an if-statement does in code. For example:

<div id="if-directive">
  <h1 v-if="condition">I'm conditionally rendered!</h1>
</div>

In this example, the condition that v-if points to should be some data that you have defined that can be evaluated to true or false. Now, if the condition is true, you will see the h1 tag on the webpage, and if it is false, this will be an empty div. You can test this out by creating a Vue instance attached to this div:

let ifDirective = new Vue({
  el: "#if-directive",
  data: {
    condition: true
  }
})

Now, if you have the page open in your browser, you should see the h1 tag, but if you open the console and set ifDirective.condition to false, you can see the header disappear.

Just like if-statements with any programming language, v-if also has the associated v-else and v-else-if so you can give other options for what to render depending on a variety if conditions, like this:

<div id="gender">
  <p v-if="person.gender === 'M'">
    Male
  </p>
  <p v-else-if="person.gender === 'F'">
    Female
  </p>
  <p v-else>
    Other
  </p>
</div>

And the last directive I’ll talk about in this post is v-for. This should also sound familiar to any programmer, and as you might expect, the v-for attribute is designed to loop over a collection or list. Here’s an example of what a Vue instance with a list could look like:

const forDirective = new Vue({
  el: "#for-directive",
  data: {
    todos: [
      { text: 'Wash dishes', isComplete: false },
      { text: 'Do laundry', isComplete: true },
      { text: 'Check email', isComplete: false }
    ]
  }
})

And the HTML would look like this:

<ul id="for-directive">
  <li v-for="todo in todos">
    {{ todo.text }}
  </li>
</ul>

This will render a list like the one above. If you have the index.html page open in your browser and you open the console, you can add or remove items in the todos array and see the list update on the page. The list will always update whenever there is a change to the list in your instance. The v-for attribute also gives you the ability to loop over an object, and a way to access the index and/or key of each of the elements in the collection:

// Using the index of each item in the array
v-for="(item, index) in items"
// Looping over an object
v-for="value in object"
// Looping over an object with it's key names and index values
v-for="(value, name, index) in object"

One thing to note is that it is not advised to use v-if with v-for. The main reason to try to use them together is when you only want to display items in the list that fit a condition. But the better way to do this is to use a computed property. You should use computed properties to display a sorted or filtered part of your original list. If we use the example from earlier with the todo list, we could add a computed property to the instance that filters the list by items that are not completed, like so:

computed: {
  completedTodos: function () {
    return this.todos.filter(function (todo) {
      return !todo.isComplete
    })
  }
}

Then you can use v-for with this computed property to only display the todos that are not completed to the page:

<li v-for="todo in completedTodos">
  {{ todo.text }}
</li>

Another thing to note about the v-for directive is that you should give each element in the list a unique key attribute. This is so that Vue can properly keep track of each element’s identity so it will reuse existing elements, instead of recreating them when there is a change to the collection:

<div v-for="item in items" v-bind:key="item.id">
  <!-- content -->
</div>

Now, with everything I have talked about in my blog posts on Vue, I have covered most of the different ways to display data from your Vue instances to a webpage. And because of the the way Vue is set up, the page will change according to any updates that are made to the data. But I still haven’t really covered how to allow a user to interact with your webpage. That requires the use of the v-on directive, and I’ll spend time talking about that in my next blog post.

Getting Started With Vue.js: The Vue Instance

Valerie Foster — Mon, 14 Dec 2020 20:05:18 +0000

In my last blog post, I gave an introduction of how to get started learning Vue. I talked about how it is similar to React, the framework I’ve used most often, and I gave some instructions for one of the ways to get a project set-up to try out Vue on your own computer. I ended my post by talking about the Vue instance and how to show data from it on a webpage. The instance is what Vue is all about, and I’ll go into some of the things it can do and how to use it in this blog post.

As a reminder from last time, a basic Vue instance looks like this:

new Vue({
  data: {
    // put properties you will use here
  }
})

The data part of the Vue instance is arguably the most important. Any properties that you put into the data object are automatically tracked by Vue, and any changes you make to any of these properties will cause a re-render to your page so it will always show the updated values. Any sort of data that you want to use and keep track of should go in here. The data you keep in your data object can be of most any type; numbers, a String, a boolean value (true or false), an array, or even a function. It can also be something evaluated like this:

new Vue({
  data: {
    date: new Date().toLocaleString()
  }
})

This example uses JavaScript’s built in Date class to get the exact date and time when it is called, then uses a method to turn the date object into a human readable String. As another refresher, the way to use this property in a webpage is by wrapping it in double brackets in your HTML and adding the el property to the instance with a reference to the HTML element:

<div id="name">
  {{ date }}
</div>

Now we can see the date String with the date and time on the page like this:

This date will be the exact date and time from when you loaded the page, and it will not update unless you refresh the page. When interpolating data inside these double brackets, you are not just limited to referencing a predefined piece of data, you can also use a piece of code to change what shows up on the page. For example, if you wanted to only show the date without the time, you could split the date property at the comma and only display the first part, like so:

{{ date.split(',')[0] }}

One thing to note: you can only put one expression in each of the pairs of brackets, so you can’t do anything too complicated to the data in your HTML. But there are things you can add to your Vue instance to write longer or more complicated code.

Another property you can add to the Vue instance besides el and data is methods. Up till now, I’ve only talked about opening the console to manually change the data in your instance, but you’ll often want to use functions to do this. Here’s an example of an instance with the method property:

new Vue({
  el: "#counter",
  data: {
    count: 0
  },
  methods: {
    add: function () {
      this.count++
    }
  }
})

Now, using a directive, something I’ll talk more about in my next blog post, you can attach this function to a button on the DOM:

<div id="counter">
  <p>Count: {{ count }}</p>
  <button v-on:click="add">Add</button>
</div>

This example uses the v-on directive to create an event listener when you click the button, and it attaches the add function to the listener. Now, if you click the Add button you can see the count go up on the screen.

The last property for Vue instances that I’ll talk about is the computed property. It is very similar to the methods property, in that it also uses functions. For example:

new Vue({
  data: {
    firstName: "Jane",
    lastName: "Doe"
  },
  computed: {
    fullName: function () {
      return `${this.firstName} ${this.lastName}`
    }
  }
})

This computed property will always return the firstName and lastName properties with a space between, and it will update whenever one of the other properties changes. You might think it would be the same thing to have this function in the methods property or to make this function a part of data, but it is slightly different. For one thing, if you use a computed property, you can reference it in the DOM the same as you would any piece of data, instead of using parentheses to call it like a function. Also, a computed property only updates when some of the data it uses updates, as opposed to functions that re-compute the result any time it’s referenced in the DOM. So, if you are doing a complicated and time consuming evaluation, it will be more expensive time-wise not to use a computed property.

Ultimately, Vue instances are designed to help you reference and change data on your webpages by making it easy to keep track of updates and giving you a variety of ways to change the data in any way you might need. So far, I’ve only talked a little about all the cool ways Vue gives you to render that data to the DOM. That will be the focus of my next blog post, where I’ll talk about Vue directives.

Getting Started With Vue.js

Valerie Foster — Tue, 08 Dec 2020 02:31:48 +0000

Designing things for the web nowadays requires that sites be interactive and react quickly. There have been many studies done about how users are likely to leave a site if it takes too long to load and update. That is where frontend software development comes in. And the most popular coding language to make a website immediately responsive is JavaScript. Because JavaScript has become so popular, there are now a large number of frameworks that make it easy to make your website interactive. Arguably the most popular one is React.js, and this is what I was taught to use in my bootcamp. But I decided it would be good to learn another one, at the very least to see what it is like to teach myself how to use a framework. So I decided to look into Vue.js.

One thing I’d heard is that, since I know React, learning another framework isn’t as hard, since they all have similar concepts. From what I’ve seen, that is mostly true. Both React and Vue have a way to keep track of some sort of “state” that affects what is displayed on a webpage. Using this state, both React and Vue have ways to add things to html so you can cycle through a list and have html elements show up on the page for each thing in the list. They both have ways to make something only show up on a webpage depending on a condition. And they both have ways to make certain elements react to the events you specify.

In comparison to React, it is very easy to get started with adding Vue to a project, you just need to add the script that Vue provides in their guide to your HTML file:

<script src="https://cdn.jsdelivr.net/npm/vue@2/dist/vue.js">
</script>

Another comparison is that, in React, everything is written in JSX, which is a syntax extension to JavaScript that was created to be used with React. I personally like JSX, but it may be hard for some people to get a handle on, so Vue is better for beginners in this way as well. Vue doesn’t have its own syntax, it just uses HTML and JavaScript.

I taught myself about Vue through the guide they have on their website. Similar to React’s create-react-app, they have a way to scaffold a new project with all the structure you need for a full project with vue-cli. But to start out with learning Vue’s, I decided to test all of the concepts out using a very basic project with a simple structure. It just has three files: index.html, index.js, and index.css (the css file isn’t necessary, I just added it so I can style some things). The index.html file looks something like this:


   <html>
     <head>
       <link rel="stylesheet" href="index.css">
       <script src="https://cdn.jsdelivr.net/npm/vue@2/dist/vue.js">
       </script>
     </head>
     <body>
       . . .    
     <script src="index.js"></script>
     </body>
   </html>

You could also put all of your JavaScript in the script tag at the bottom of the body tag in your HTML file instead of linking the index.js file in, but I think this makes the HTML file to long and involved so I prefer to put all of the JavaScript in a separate index.js file. To see the index.html file in my browser, I just run this command in my terminal:

// ♥ > open index.html

The main thing you need to get started using Vue is the Vue instance. A basic example, using the classic “Hello, World!”, would look like this:

var hello = new Vue({
  el: '#hello',
  data: {
    message: 'Hello, World!'
  }
})

The data part of a Vue instance is an object with keys and values with data you want to use, and Vue will keep track of it as it changes. In a Vue instance, you can access anything in the data object directly from the instance, like this:

hello.message  // => "Hello, World!"

You use el to attach the instance to an HTML element, in this case something with an id of hello. This way, inside the HTML element you can just use message directly to reference it:

<div id="hello">
  {{ message }}
</div>

The double brackets are the way we link the data from the Vue instance to the DOM, so if you open the console and change hello.message to equal something else, you will immediately see the page update to include the new value.

But just being able to show some data from a Vue instance on the screen, and only being able to change it though the console isn’t very interesting. Vue gets a lot more useful when you start adding directives and add functions so you can interact with the data that appears on the screen. I’ll go into the basics of what Vue directives are, some of the ones that are available in Vue, and how to use them in my next blog post.

Intro To JavaScript Testing With Mocha: Part 2

Valerie Foster — Wed, 02 Dec 2020 19:13:53 +0000

In my last blog post I talked about how to get started with writing tests for a JavaScript project using the Mocha testing library. As I mentioned at the end of that post, this time I will talk about some of the more interesting and less intuitive things you can do when testing with Mocha.

One interesting thing that Mocha gives you is the hooks before(), after(), beforeEach(), and afterEach(). You can use these inside a describe block, and each of them has a different purpose: before and after are only run once, before or after all of your tests in describe, while beforeEach and afterEach are run before or after each test. One way this can come in handy is if you want to run all of your tests on the same piece(s) of data, like an array. You could define an array at the top level of the describe block, but if your tests do a lot of transformations to the array, you could use the beforeEach method to reset the array before each new test like this:

describe('Array', function () {
  let a

  beforeEach(function () {
    a = ["reset", "the", "array", "to", "be", "the", "same"]
  })

  // all of your tests go here
})

Each hook takes a callback function and an optional description string, the same as the it function. It also doesn’t matter what order you put your hooks or tests in, the hooks will still be run at the time they are intended, and your tests will run in the order you have written them in your file.

Another useful thing that Mocha gives you is the ability to write tests that don’t run. This may sound silly, but it can be very helpful. Tests that don’t run are called pending tests and there are a few ways to write them. One way is making a test without a callback. The idea of this type of test is that someone should write a test case for it eventually. For example:

describe('Array', function () {
  describe('#includes()', function () {
    // pending test below
    it('should return false when the value is not present')
  })
})

Pending tests will be included in the test results, and marked as pending. A pending test is not considered as passing or failing. When you run your tests, the result will tell you how many pending tests you have, along with the passing and failing ones. The result would look something like this:

Another way that you can make pending tests is by writing tests that are skipped. One reason you would do this is if you wrote out a test, but it doesn’t pass and you don’t know why, or you don’t want to take the time to fix it right now, then you can skip it temporarily. This is better than commenting out the test because, if it you get a pending result every time you run your tests, you won’t forget to come back to it later. Skipped tests work by appending .skip() to the test functions. A single skipped test will look like this:

describe('#includes()', function () {
  it.skip('should return true when present', function () {
    // this test will not be run
  })

  it('should return false when not present', function () {
    // this test will be run
  })
})

Or, to skip an entire suite, you can use the skip keyword on the describe function like this:

describe.skip('#includes()', function () {
  it('should return true when present', function () {
    // this test will not be run
  })

  it('should return false when not present', function () {
    // neither will this one
  })
})

The last place you could use skip is inside your test function to skip the test at runtime using this.skip(). If a test requires an environment or configuration that you wouldn’t be able to know about beforehand, using skip at runtime is the best way to handle the problem. For example:

it('should only test in the correct environment', function() {
  if (/* check test environment */) {
    // make assertions
  } else {
    this.skip()
  }
})

If the environment isn’t right, the result of this test will be pending, and it will essentially abort the test, instead of completely skipping it. The example above is also a best practice, compared to leaving the space in the else block blank, because a blank test will be reported as passing, and it would be much more useful for us to know the test was aborted.

Opposite but similar to skip, you can also append .only() to it or describe to choose to run only one distinct test or test suite. This can be useful if you are working to make one specific test pass and you don’t want to waste time waiting while all the other tests are run. Both only and skip can be added to any number of tests you like (but adding them to too many would defeat the purpose). One thing to note: any Hooks you have will still be run.

One other thing that is necessary to consider when writing anything in JavaScript is how to deal with asynchronous code. Mocha has a few ways to deal with asynchronous code in your tests. One way is by adding an argument to the test callback, usually called done. For example:

describe('Tea', function () {
  describe('#save()', function () {
    it('should save without error', function (done) {
      let tea = new Tea('Chai')
      tea.save(done)
    })
  })
})

Using done like this will ensure that Mocha knows it should wait for this function to be called to complete the test. This format will also handle any errors in the save function. But if the APIs you are testing return promises instead of taking callbacks, rather than using the done() callback, you can use Chai as Promised, and have something like this in your test:

return expect(Promise.resolve({ foo: "bar" }))
                               .to.eventually.have.property("foo")

The essential piece that Chai as Promised gives you is the eventually property. It also gives you more than the .have.property() chains, so you can check for a variety of things. There are more examples of how to use it in their documentation.

The last thing I will mention is that Mocha has a wide variety of optional flags you can add to the end of your command to run your tests. The flags are all listed in their documentation, but I’ll mention a few here. One is --bail, and it causes the test suite to abort after the first failed test, which is handy if you want to work on fixing one test at a time. There is also the --parallel flag, which will cause the tests to be run in parallel mode. This mode is designed to help your test suite run faster by making your tests run somewhat at the same time, instead of one after another, which can be very practical and save time if you have a large number of tests to run. There are a few drawbacks to parallel mode though, so make sure you know enough about parallel mode to recognize if it will work for you.

And this is the end of my introduction to testing with Mocha. There are other JavaScript testing libraries, but Mocha is one that is well used, thoroughly documented, and, as I hope you can tell from my posts, easy enough to work with. Tests are very useful, and I hope my posts have convinced you to consider using Mocha to write tests for your next JavaScript project.

Intro to JavaScript Testing With Mocha

Valerie Foster — Wed, 02 Dec 2020 19:06:09 +0000

With my last few blog posts, I spent time talking about the importance of testing with software development, and I gave an intro to writing tests with Rails. For this post, I’m going to talk about how to write tests in one of the foremost JavaScript testing frameworks.

With plain old vanilla JavaScript, not using any frameworks like React or Angular, there are a variety of testing frameworks you can choose from, but I am going to talk about Mocha. The reason I chose Mocha is that the syntax is similar to the tests I was writing with Rails, and it is the framework my Bootcamp used for writing tests, so it feels familiar to me. It also has very good documentation with clear examples and explanations, and it has been around for a long time so they’ve had a chance to iron out all the bugs, and there are a lot examples of it being used.

To get started with using Mocha for your JavaScript project, run these commands in the top level of your project directory:

~ // ♥ > npm install mocha
~ // ♥ > mkdir test
~ // ♥ > touch test/test.js

These commands will add Mocha to your project, create a test directory, and make a file to write your tests in. The test file doesn’t have to be called test.js, you can name it whatever you like, and if you’re going to be testing multiple files, you should have a test file for each one with a name referencing the file you’re testing, like <filename>Test.js.

Now for writing you first test. Using Node.js’ built-in assert module, you could write a simple test like this:

const assert = require('assert') // from Node.js' assert module

describe('Array', function () {
  describe('#indexOf()', function () {
    it('should return -1 when the value is not present', function(){
      assert.equal([1, 2, 3].indexOf(4), -1)
    })
    it('should return the index when present', function(){
      assert.equal([1, 2, 3].indexOf(2), 1)
    })
  })
})

It is pretty clear from the function names what is going on here. The describe function is given a name that tells you what all of the tests inside it are going to be testing; in this example JavaScript’s Array indexOf function. The it function is given a name that describes exactly what the test is checking for, as well as a callback function. Inside the callback is an assertion. Assertions are designed to evaluate something for expected results. In this example we check that each call of the indexOf function on the given array returns the number we expect.

Now to run the tests from the command line, and see the results:

~ // ♥ > ./node_modules/mocha/bin/mocha

  Array
    #indexOf()
      ✓ should return -1 when the value is not present
      ✓ should return the index when present

  2 passing (7ms)

One thing to note here: you can see from the result that tests were run in the order that they were written in the file. This will always be the case for all tests that are written to run normally (I’ll get into running tests “abnormally” another time).

Back to running the tests, you can see that the command to run them is long and annoying to type out, so to make it easier you can set up a test script in your package.json file:

"scripts": {
  "test": "mocha"
}

Then you can run your tests with a simple command (you use test because it is the key you put in your script):

~ // ♥ > npm test

Another thing you can add to your script for running your tests is a reporter. Reporters change the way the results of your tests look in your terminal. The example I have above uses spec, which is the default. There is a wide range of reporters, some with the function descriptions for passing tests included, like span, and some that just include the name of the function if it fails. There is a full list of the reporters Mocha has here. My favorite is nyan and it looks like this:

To change your reporter to something other than span you just have to change the script in your package.json to look like this:

"scripts": {
  "test": "mocha --reporter <reporter-name>"
}

The example I have above works just fine, but there are other options for how to write tests. For example, Mocha allows you to use any assertion library you want. One that I like because it has very useful and clear types of assertions is called chai; it has expect(), assert() and should-style assertions, and you can choose to use any of these you like. After installing chai with npm install chai, I could rewrite the tests from before to look like this:

const expect = require('chai').expect; // add to the top of the file

describe('Array', function () {
  describe('#indexOf()', function () {
    it('should return -1 when the value is not present', function(){
      expect([1, 2, 3].indexOf(4)).to.equal(-1)
    })
    it('should return the index when present', function(){
      expect([1, 2, 3].indexOf(2)).to.equal(1)
    })
  })
})

I personally like this better because I think expect(a).to.equal(b) makes it clearer to someone reading it what you’re checking for, as opposed to assert.equal(a, b). There are also a lot more methods than just equal() that you can use in your assertions, for example, expect(foo).to.be.a(‘string’). You can see all of them listed out in chai’s documentation.

Another thing to note: Passing arrow functions to Mocha is discouraged. Arrow functions bind this so you cannot access the Mocha context. This can sometimes lead to some errors in your tests, and while arrow functions will usually work, avoiding them will cause you less problems down the line.

And that is all I’m going to talk about in writing JavaScript tests with Mocha today. I think I’ve laid a good foundation for how to get started with writing tests for a basic JavaScript project. In my next post, I’ll go into some of the more interesting things that you can do with Mocha tests.

Intro to Testing with Rails: Controller Tests

Valerie Foster — Tue, 24 Nov 2020 01:21:10 +0000

This is the final part in my series of posts about testing with Rails. Previously, I talked about getting started with Rails testing, and I talked about writing tests for the Model and View parts of the MVC pattern. This post will talk about how to write tests for the Controller part of the MVC framework, testing your routes and controller actions. This is the last important piece of testing your Rails application.

Testing can open up a whole new way to think about developing code. One way to think about development is using something called Test Driven Development, or TDD. TDD is a software development process where you write a test with the desired behavior you want your application to have, then you write the code to make the test pass. This way of developing code can be very efficient. For one thing, it forces you to think about what functionality you want to add and exactly what result you want it to have. This also makes sure you don’t have any pieces of untested code — if you write your tests after the fact, you could forget to check the functionality for some sections of code.

Also, TDD can make it easier to figure out why your tests are failing. When you write functionality to pass the test you just wrote, you have a better idea of what sections of code come into play in making your test pass. Therefore, if the test is still failing after you’ve added the code to make it pass, you know where in your code to look to find the part that doesn’t work. This can also help you realize if your tests are trying to cover too wide a range of functionality, and if you might want to split the test into multiple parts to test smaller pieces of functionality.

Now, about Rails controller tests. Controller tests are used to verify that all of your controller actions do as intended as well as checking that your routes function correctly. So naturally, they rely heavily on request types like get, post, patch, and delete. All of these request types have equivalent methods that you can use to check if the result of the request is successful, and that a post, patch, or delete updated the database and redirected to the proper page.

An example test for checking that the index action in the authors_contoller works could use the get method with the path for the Author’s index page, then check that the url response is a success. The method would look like this:

test "should get index" do
   get authors_path
   assert_response :success
end

In contrast to the get method, post and patch need to use parameters to give the method the new information to create or update a record with. You would pass these params to the post method like this:

post authors_path, params: {
   author: { name: "Philip Pullman", age: 73 }
}

Another thing controller tests can check for is the three Hash objects you have access to after a request has been made and processed; cookies, flash, and session. Say, for example, in my author’s create action I used a flash notice to add an alert whenever an author was created successfully. I could check it was working by adding a line like this to my test for creating new authors:

assert_equal 'Author was successfully created.', flash[:notice]

One more useful thing that controller tests give you is access to three instance variables after a request is made: @controller (the controller processing the request), @request (the request object), and @response (the response object). So, another way you could check that the index action was working would be like this:

test "should get index" do
   get authors_path
   assert_equal "index", @controller.action_name
   assert_match "Authors", @response.body
end

Of all of the different things I have talked about that you can add to your controller tests, you will probably find some more useful than others. I have only used a few in the example controller test file that I have here (the file is mostly the same as the one generated from the scaffold command I ran for authors):

# in controllers/authors_controller_test.rb
require 'test_helper'
class AuthorsControllerTest < ActionDispatch::IntegrationTest
   # method that is called before every test
   setup do
      @author = authors(:rowling)
   end
   test "should get index" do
      get authors_url
      assert_response :success
   end
   test "should get new" do
      get new_author_url
      assert_response :success
   end
   test "should create author" do
      assert_difference('Author.count') do
         post authors_url, params: {
            author: { name: "Philip Pullman", age: 73 }
         }
      end
      assert_redirected_to author_url(Author.last)
   end
   test "should show author" do
      get author_url(@author)
      assert_response :success
   end
   test "should get edit" do
      get edit_author_url(@author)
      assert_response :success
   end
   test "should update author" do
      patch author_url(@author), params: { author: { age: 56 } }
      assert_redirected_to author_url(@author)
   end
   test "should destroy author" do
      assert_difference('Author.count', -1) do
         delete author_url(@author)
      end
      assert_redirected_to authors_url
   end
end

With all of the types of tests I have covered in my four blog posts on Rails testing, I have given a pretty thorough introduction to writing tests for Rails applications. I have also given reasons for why writing tests is useful and practical when creating any kind of application. Ultimately, I have tried to convince you that you should consider writing tests for the next Rails app you create. Tests are very useful, and I hope my posts have made it seem doable to implement them in Rails.

Intro to Testing With Rails: System Tests

Valerie Foster — Tue, 24 Nov 2020 01:20:55 +0000

Writing full and complete tests for a Rails application requires testing each of the parts of the MVC pattern. For the Model component, you test that the methods in your models do all the things you want them to and that the relationships between your models work properly. I have gone over the basics of how you can write tests for you models in my previous two blog posts. In this one I will talk about how to start testing the View part of the MVC architecture by talking about how to test the way a user interacts with the webpages in your application.

But first, I’m going to talk a bit about ways to figure out how many tests you should write. It is of course impossible to test for every possible thing that could happen when someone uses your app, and too many tests would take too long to run. But you also want to make sure you have enough tests to cover each range of possibilities. A good way to do this is by using edge cases and partitioning.

Edge cases occur at an extreme, like a maximum or a minimum. For example, an empty string would be an edge case for strings. Partitioning is separating the inputs you are testing into groups of a similar type. An example of partitioning when testing numbers would be separating inputs into negative, positive, and zero. Then you only have to write three tests, each using one number in each partition. Using these methods you can write tests that hopefully cover every type of input a user could give while not writing a whole bunch of unnecessary tests.

Getting back to Rails, the way to write tests for your views in Rails is system testing. All the files for this are kept in the system folder under the test directory. Now, unlike all the other types of tests I’ve talked about so far, a system test file is not generated by rails g resource. But Rails does provide a generator to make system tests for each of your models:

~ // ♥ > rails g system_test authors
      invoke  test_unit
      create    test/system/authors_test.rb

This command creates a file for system tests for the authors model (with a sample method). But since I used rails g scaffold, and scaffold does generate system test files, my authors_test.rb file already had a variety of system tests written for authors. The only thing I had to change was the line in the setup method to reference the rowling fixture I had written earlier:

# in system/authors_test.rb
require "application_system_test_case"
class AuthorsTest < ApplicationSystemTestCase
   setup do
      @author = authors(:rowling)
   end
   test "visiting the index" do
      visit authors_url
      assert_selector "h1", text: "Authors"
   end
   test "creating a Author" do
      visit authors_url
      click_on "New Author"
      fill_in "Age", with: @author.age
      fill_in "Name", with: @author.name
      click_on "Create Author"
      assert_text "Author was successfully created"
      click_on "Back"
   end
   test "updating a Author" do
      visit authors_url
      click_on "Edit", match: :first
      fill_in "Age", with: @author.age
      fill_in "Name", with: @author.name
      click_on "Update Author"
      assert_text "Author was successfully updated"
      click_on "Back"
   end
   test "destroying an Author" do
      visit authors_url
      page.accept_confirm do
         click_on "Destroy", match: :first
      end
      assert_text "Author was successfully destroyed"
   end
end

System tests are noticeably different from the other tests I have talked about before. For one thing, they are not automatically run when you run the rails test command. You have to either specify the system test file you want to run, like rails test test/system/authors_test.rb or, to run all you system tests, in the command line run:

~ // ♥ > rails test:system

Another thing different about system tests is how you are not just using assertions anymore, you are using Capybara to interact with the webpage. From the tests in authors_test.rb above, you can see from the intuitively named methods that you can tell the test to visit a given url path, click on certain buttons, and fill in form inputs. All of this is supposed to mirror how a user would interact with a webpage. For more information on what kinds of methods Capybara has, you can look through their documentation.

Another interesting thing about system tests is that when they are run, they use a browser window. When I ran rails test:system it opened a new google chrome window for each of the tests, and if you watch one window you can see it doing the things defined in the test, like navigating from page to page and filling in and submitting forms. But it all moved so fast it was difficult to see what happened, and to help with this Rails has the ScreenshotHelper. You can add take_screenshot or take_failed_screenshot at any point in your tests to take a screenshot of the browser window running the test. ScreenshotHelper is very helpful in debugging how something went wrong.

Now, the reason my system tests opened a chrome window was because it was configured that way. There is a file in the top level of the test directory that is generated every time you start a Rails application (as long as it’s not an API) called application_system_test_case.rb. This file is for configuring your system tests and the default configuration looks like this:

require "test_helper"
class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
   driven_by :selenium, using: :chrome, screen_size: [1400, 1400]
end

The “driver” in driven_by determines where the tests are run, in a browser window or in your terminal or another type of window. You can also specify what browser you would like to use with using. The screen_size argument defines what size you’d like the browser window to be. This argument can be very useful if you want to test how your app looks on mobile. If you plan on having your app used on mobile you can create a separate file with configurations for the mobile screen_size and have some tests for mobile that require and inherit from your mobile configured file instead of ApplicationSystemTestCase.

I think it’s clear that system tests are a very important part of thoroughly testing a Rails application. I have now covered how to start writing tests for both the M and V parts of the MVC framework. In my next post I’ll go into testing the Controller component of the MVC pattern by going into how to begin writing tests for your controller actions.

Intro to Testing With Rails: The Test Database and Fixtures

Valerie Foster — Tue, 24 Nov 2020 01:19:52 +0000

Testing is undeniably an important part of web development. Continuing on from my last blog post, I am going to give more information about how to write tests with Ruby on Rails in this post. This post will be easier to understand if you read my last one, Intro to Testing With Rails. This one picks up where I left off, and explains more of the structure of the testing framework that Rails provides.

But before I get into that, I’ll talk a bit about the mindset a developer should have when writing tests. Sometimes it can be hard for a developer to write tests because you need to have a different mindset from when you are creating parts of your application. When writing out the features of an app, you want to focus on all ways your app will work. But, when writing tests you have to think about all the ways your app could break. Sometimes it can be hard to think about breaking your app when you worked so hard to make it function the way you want it to, but you need to think in this mindset to write good tests.

Now back to Rails testing. One thing you should have is a separate database specifically for testing. Most of the parts of the your application are going to interact heavily with a database, but when you are testing you are usually going to be writing bad data in an attempt to see if things break. Also, you will most likely be running the tests over and over again, and you don’t want all of the data you made in your tests to be added to the database multiple times. This is why we have a test database. Once again, Rails has already taken care of this problem for you. By default, every Rails app has three environments: development, test, and production, and there is a database for each of them. They are configured in config/database.yml and created when you run rails db:setup.

One way Rails uses the test database is with fixtures. Fixtures are sample data that Rails will automatically add to the test database each time your tests are run. Fixtures can be very useful because you don’t have to create a new instance of a model for every test you write, you can just reference one that’s already in your database. Fixtures are written in a file with a .yml file extension and they look like this:

# in fixtures/authors.yml
rowling:
  name: J. K. Rowling
  age: 55
# in fixtures/books.yml
harry:
  title: Harry Potter
  author: rowling

The authors.yml and books.yml files these fixtures are written in were generated when I used the rails g scaffold commands. Whenever you generate a new model Rails will automatically create a new file for fixtures of that model (with examples) in the test/fixtures directory.

As you can see from the examples, there is a specific format to fixtures. Each is given a name (anything you want to call it) which is followed by an indented list of the model’s attributes followed by a colon and whatever value you want to give the attribute. Also, notice how the books fixture references the author fixture. This is completely valid and, just by writing rowling, Rails will know that you are referencing the rowling fixture in the authors.yml file.

Now, to use the fixtures in your tests you have to reference them like this: authors(:rowling) or, to get a list, authors(:rowling, :author_two) will return an array with two fixtures. Once you have a reference to a fixture, you can treat it the same as any other instance of the class, such as using any of the methods the class might have on the fixture.

This is an example of a passing test using fixtures:
# in models/book_test.rb
test "fixtures properly save to database" do
  assert_not_nil books(:harry), 
      "book fixture did not save to database"
  assert_equal authors(:rowling), books(:harry).author, 
      "book fixture does not reference author fixture"
end

Now, so far I’ve only changed Rails-generated files in two folders in the test directory: models and fixtures. As I just explained, files in the fixtures folder are for creating sample data to run your tests on. In the models folder you write tests to test the methods in all your model classes using assertions. So, as yet, I have only written tests for my models in the models folder. But testing the behavior of your models is by no means sufficient to prove that a Rails app works as intended.You also need to test how a user can interact with the app. In other words, test the controllers and routing.

Testing you controller actions and what shows up on a web page uses a different system of writing tests using a gem called Capybara. In my next bog post on testing with Rails I will focus on how you can use Capybara to test a user’s interaction with web pages in a Rails app.

Intro to Testing With Rails

Valerie Foster — Tue, 24 Nov 2020 01:19:40 +0000

Many developers will rely on interacting with an app to prove it works the way they want. But this is not the most efficient way to do things. Writing tests for your code is a good practice, and it can save time finding bugs down the line. In this series of blog posts, I’m going to talk about how to write tests for a Rails application.

First, I’ll talk a bit about why testing is so important. Writing tests is useful because you can use them to check that a small piece of your code works the way it was intended. This is also a good way to become more familiar with your code, because it forces you to thinks about what exactly you want each piece of your code to do. And when you find a bug in your code you can have a better idea of where it is, because you know for sure that certain things work. Also, once you’ve written a test, you can run it over and over to check that everything still works as expected, even after you have made a major change to your app.

Writing tests in Rails is relatively easy. A Rails application started with rails new already has a testing framework set up for you, in a folder called test. It has a file structure like this:

- test
  - channels/
  - controllers/
  - fixtures/
  - helpers/
  - integration/
  - mailers/
  - models/
  - system/
  - application_system_test_case.rb
  - test_helper.rb

Each of the folders in test are designed to test a specific part of your application, and I’ll go into each of them in a later blog post. For this one, I’ll just focus on the basics of writing and running tests.

As an example, I’ll make references to files in a project created with these commands:

~ // ♥ > rails g scaffold Author name age:integer
~ // ♥ > rails g scaffold Book title author_id:integer

When you use a rails generator like rails g scaffold or rails g resource to create files for your models, rails will automatically create some test files for these models that you can fill in. In the models folder I have a file like this:

# author_test.rb
require 'test_helper'
class AuthorTest < ActiveSupport::TestCase
# test "the truth" do
#   assert true
# end
end

The class AuthorTest inherits from ActiveSupport::TestCase which gives us a number of useful methods, including the one used in the example commented out here. The test method takes a test name(here, the name is “the truth”) and a block(everything between the do and end), and if you give it a descriptive name, it can be very clear what the test is designed to look for.

In the do…end block is an assertion. Assertions are designed to evaluate something for expected results. There are a variety of available assertions, and there is a full list of them here. Each test can have as many assertions as you want, and the test will only pass if all the assertions are successful. An example assertion for my Authors/Books project test that an author will not save without a name. A test to check for this could look like this:

test "author should not save without name" do
  author = Author.new
  assert_not author.save
end

Now that we have a test written, we need to be able to run it and see the results. There are a variety of ways to run your tests; you could run all of them at once with rails test, but this isn’t very efficient. Running all of your tests can take longer, so if you just want to check the results of a specific test, the best way is to drill down to run just the tests in the file, or even the test on a given line. So the command to run the test I wrote would look like this (7 being the line the test starts on):

~ // ♥ > rails test test/models/author_test.rb:7

But, because I never wrote the validation to require an author to have a name into my Author model class, the test fails, and it denotes the failure with an F:

# Running:
F
Failure:
AuthorTest#test_author_should_not_save_without_name 
[/path/rails-testing/test/models/author_test.rb:17]:
Expected true to be nil or false
rails test test/models/author_test.rb:7
Finished in 1.116573s, 0.8956 runs/s, 0.8956 assertions/s.
1 runs, 1 assertions, 1 failures, 0 errors, 0 skips

Unfortunately, this failure isn’t very descriptive about what exactly went wrong. That’s because I didn’t give the assertion a massage to output so it used the default, “Expected true to be nil or false”. To make the test clearer I can add a message, and I’ll also add another assertion to check that an author cannot be saved with an age that is not positive:

test "should not save if validations fail" do
  author = Author.new
  assert_not author.save, "saved the author without a name"
  author = Author.new(name: "J. K. Rowling", age: -10)
  assert_not author.save, "saved the author with a non-positive age"
end

Then I’ll add the validations to the model file like so:

#author.rb
class Author < ApplicationRecord
  has_many :books
  validates :name, presence: true
  validates :age, numericality: { greater_than: 0 }
end

Running the test now gives me a passing result:

# Running:
.
Finished in 0.267856s, 3.7333 runs/s, 7.4667 assertions/s.
1 runs, 2 assertions, 0 failures, 0 errors, 0 skips

There is one more result you can get from running a test, and that is an error. The same as in any other code, errors can occur from spelling mistakes or references to variables that don’t exist. For example, if I had misspelled assert_not as asset_not (which I did), the result would look like this, with an E to designate the error:

# Running:
E
Error:
AuthorTest#test_should_not_save_if_validations_fail:
NoMethodError: undefined method `asset_not' for 
#<AuthorTest:0x00007fea2ffdd178>
    test/models/author_test.rb:12:in `block in <class:AuthorTest>'
rails test test/models/author_test.rb:7
Finished in 0.766280s, 1.3050 runs/s, 1.3050 assertions/s.
1 runs, 1 assertions, 0 failures, 1 errors, 0 skips

The error helpfully points to the line where it occurred, on line 12 in author_test.rb. It also ran the first assertion before it found the error in the second one and stopped.

And this is most of what you need to know about the basics of testing in rails. In summary, you write tests in blocks that use assertions to verify the results you expect. Then you run the tests and, if you are clear in your test massages, the results give helpful information about any tests that didn’t pass. In my next blog post I’ll go into the types of tests to write in each of the directories that Rails gives you in the test folder.

Routing in React With React Router

Valerie Foster — Sat, 14 Nov 2020 01:08:15 +0000

People who work with React should know that it is essentially a single page application (SPA). But but many React applications don’t present that way to a user. The user interacts with things on the page, and different components appear and disappear. A user interacts with the application as if it has many pages, so it makes sense for the URL to reflect this. This is where React Router comes in.

First of all, since React is a SPA, all routing is Client-Side routing. This is in comparison to server side routing where each different URL makes a different GET request to the server. In Client-Side routing, the only thing the server does is render the HTML with the 'root' div for React to render it’s virtual DOM. One benefit of this is the speed with which the different “pages” will appear for the user. The Client-Side route will just swap which component is rendered to the page instead of making a new server call, which can take some time. But on the flip side, the first page may take longer to load.

The best way to use Client-Side routing in React is to use React Router. This is a React library that uses specific URLs to tell React what components to render at each URL. To use it you have to install react-router-dom to your React app by running:

npm install react-router-dom

Now, to use react-router-dom, you have to decide what file or files you want to define the routes in your React app. The most common places for this is in index.js or the App component if you need to pass state down the component tree. App is usually where you keep the logic of how to organize the application, so it makes sense that it is the component that decides which components to show at each route. You also want to decide what routes you want to have.

For example, lets say you have a React application with a three main components: Home, About, and FriendList, which shows info about each friend in a list you have. At the top of the App.js file you would need to import all the components along with react-router-dom:

import Home from './Home'
import About from './About'
import FriendList from './FriendList'
import { BrowserRouter as Router, Route } from 'react-router-dom'

Importing BrowserRouter as Router is a common convention, so instead of using the term BrowserRouter in the component, you refer to it as Router. There are other things you can import from react-router-dom, but these are the main ones you’ll want to use to apply Client-Side routing.

Now that you’ve got everything imported, you’ll want to use the Router and Route components in the JSX returned from the App component (either the render method of a class component, or the return of a functional component):

return (
  <Router>
    <div>
      <Route exact path=”/” component={Home} />
      <Route exact path=”/about” component={About} />
      <Route exact path=”/friends” component={FriendList} />
    </div>
  </Router>
)

To unpack this example, first we’ll talk about the Router component. It is the base of our application’s routing, so it is where we declare how React Router will be used. It can also only have one child element, so that is why all the Route components are wrapped in a <div>.

Next we’ll talk about the Route component. In this example, they are taking two props, which basically equate to them saying “when the URL matches this path, render the given component.” The exact part just makes sure that the component is only rendered when the URL matches the path exactly. If we omitted the exact from all of the Routes, the Home component would render at any path with a “/” in it, in other words at every path.

Route can also be given the prop of render instead of component. Render takes a callback function as input so our example Routes would look like this:

<Route exact path=”/” render={() => <Home />} />

Using render is best when you have props from App that you want to send to it’s child components. It also has default props that you can pass through the callback function.

Another helpful thing react-router-dom has is the NavLink and Link components. When you import and use either of these components you can add links throughout your pages to other pages in the application. Both components function almost the same way, except you can add styling to NavLinks that show what page you are currently on. So for our example you could have a component called Navbar that you render on every page that looks like this:

import { NavLink } from 'react-router-dom'
const Navbar = () => {
  return (
    <div>
      <NavLink to=”/” exact>Home</NavLink>
      <NavLink to=”/about” exact>About</NavLink>
      <NavLink to=”/friends” exact>Friends</NavLink>
    </div>
  )
}

There is one last piece of routing functionality we are missing, and that is the ability to show the individual friends from our friend list at an individual page. The URL for this is commonly '/friends/1', the 1 being the id of the friend being shown. Writing individual routes for each friend would be ridiculous, so the way we do this is with nested routes.

This requires some refactoring, so first off, we have to change App’s FriendList route to look like this:

<Route
  path='/friends'
  render={props => {
    <FriendList {...props} friends={this.state.friends}/>
  }}
/>

This Route uses render so that it can have access to a list of movies from state and the list of props with information about the route.

Then the FriendList component will render a list of Links that each go to show page for a friend with the URL '/friends/:id' with the id being the id of the friend that’s being shown. It also defines a new Route that uses the match prop passed down:

const FriendList = ({ match, friends }) => {
  const renderFriends => {
    return friends.map(friend => {
      return <Link key={friend.id} to={`/friends/${friend.id}`}>
        {friend.name}
      </Link>
    })
  };
  return (
    <div>
      {renderFriends()}
      <Route
        path={`${match.url}/:friendId`}
        render={props => <Friend {...props} friends={friends} />}
      />
    </div>
  )
}

Now we need a way for the Friend component to know which friend from the list it should display. We do this through the match prop again:

const Friend = ({ match, friends }) => {
  return (
    <div>
      <h3>{ friends[match.params.friendId].name }</h3>
    </div>
  )
}

And with this we have fully functioning routes for visiting a a specific friend from the list.

Routing for an application may seem minor when thinking about the entirety of an application, but it can actually be very important. A user could have a favorite page in an app, and if they want to bookmark it, they need a URL to save. Routes can also be expressive to the user about what a certain page of an app is doing. Ultimately, routing is something all developers should take into careful consideration when designing their applications.

The Basics of Scraping with Ruby

Valerie Foster — Sat, 14 Nov 2020 00:47:15 +0000

There will be a number of times when you will want to get data from the internet for a program you are writing. APIs are a great resource, but only if one exists with the subject matter you want and if the data you are looking for is static and unchanging. If you want information about the upcoming events for your favorite celebrity or the approaching games for the Seahawks or Mariners, APIs don’t cut it. Scraping is a great way to get data from a webpage.

Scraping is the process of extracting data from a webpage. It can refer to manually copying the data from the page, but it usually refers programmatically parsing the data you need from the HTML of the webpage. The best way to scrape with Ruby is to use the Nokogiri gem and Open-URI. Open-URI is just used to get the HTML from the link you pass it then Nokogiri does all of the interesting stuff.

The name Nokogiri comes from Japanese meaning “a fine-toothed saw” that was used for the rough ripping of logs into boards for carpenters and cabinetmakers. And thats how the Nokogiri gem is used. It precisely scrapes the only the necessary data that we are looking for from a webpage.

To use Nokogiri you first have to install the gem then require it and Open-URI at the top of your file:

require 'nokogiri'
require 'open-uri'

Then to start you need to get the HTML from whatever website you want data from using the url:

html = open('http://www.google.com')

and once you have this Nokogiri will take the string of HTML returned and translate it into a NodeSet that we can mess with (it is conventional to call this ‘doc’):

doc = Nokogiri::HTML(html)

Now doc has all the HTML converted into nested nodes, which is similar to the structure of nested hashes, so we can iterate over the Nokogiri object to find the information we want.

Now that we have the fundamentals done can get to the interesting stuff: getting the data we’re looking for from ‘doc’. Nokogiri uses css selectors to return data from a specific part of the HTML document. Css selectors look like this:

<div id="example-div">
  <p class="example-paragraph">Example paragraph</p>
</div>

In this example you would refer to the div element with #example-div (# for ids) and the paragraph with .example-paragraph (. for class). In HTML an id has to be unique and only refers to one element, while a class can be applied to a group of similar elements.

Using the Nokogiri in the doc variable to look for the element you want is possible, but not very practical seeing as it is a very long list of information. The best way to find what you want is to find it on the original webpage, right click it, and choose inspect. This will show you the HTML of the whole webpage, but it will highlight the item you clicked on.

Back to Ruby, Nokogiri has a .css method that is called on the doc variable. It takes an argument of the css selector you want to find using the approach from the previous paragraph, with classes and ids. So getting the paragraph from the example above would look like:

doc.css(".example-paragraph")

But this actually returns something similar to an array with one Nokogiri object with a bunch of information about the whole paragraph element. If you want to get only the text part of the paragraph you would call .text on the end:

doc.css(".example-paragraph").text
  => "Example paragraph"

But it isn’t very interesting to grab information from only one part of a webpage, and this is where iteration comes in. If an animal shelter website had a list of div elements for every adoptable pet, you could find the class name for all of those divs and call pets = doc.css(“.class-name”) to get a collection of elements, then iterate over them using an enumerable to get the names of all of the pets, for example:

pets.map do |pet|
   pet.css(".pet-name").text
end
  => [array of pet names]

It’s clear to see that scraping is a powerful tool to get data from the internet, but there are a number of drawbacks. One is that it might take time to find what you’re looking for. Another is that you might have to write vastly different code to scrape from one website compared to another. But the biggest problem is that if a website changes the scraper you wrote probably won’t work anymore. But even with these stumbling blocks scraping is still much faster and easier than having to manually enter data.