DEV Community: Katie Larson

Fruit Cart: Writing Tests, Part I

Katie Larson — Sun, 07 Jul 2019 01:23:15 +0000

Database up, the framework and tools in place, TDD process hammered out, all the magic ready--time to cast the spells!

Unit . . . or so I thought

While we will be TDDing our code, we still had to decide on a basic architecture. For instance, if we want to test a Fruit model, well, then we have to decide that we need a Fruit model in the first place. There's a bit of a chicken or the egg problem: we're operating on the principle that the tests will drive production, but yet to even write the test we have to know what we're testing . . .

So, in the example of the Fruit model, what comes first: the model or the tests? The chicken or the egg? The answer: the model. Sort of. In our collective brain it comes first. Tests and production code are dependent on each other. The way the tests are written certainly does determine the way the code is written, but for the tests to even be written in the first place, we have to decide on the basic architecture we're testing. In other words, we have to decide what we're testing.

Jeff and I actually never went ahead and wrote a file without having the tests for it set up first, but we knew in our brain that what we file we would be testing.

So we have to figure out what we're going to need and, perhaps more importantly, what we're not going to need. And that's where we run into YAGNI: You Aren't Gonna Need It.

YAGNI comes from the Extreme Programming methodology and means that we should only write the code that we need now, not what we think we'll need in the future. Yes, we want our code to be extensible, but we also don't want to build out future functionality that is not what we're looking to deliver as a part of our minimum viable product. While this may not seem forward thinking, if we do too much anticipation outside the scope of the functioning product, we'll likely end up writing code that we don't need.

For instance, say eventually user accounts and login is something we want Fruit Cart users to have and do. I go in and install the OAuth library with a Google strategy. Then I start building out functionality for creating a fruit. Suddenly I realize I didn't have a reason for users to log in. Why would they need to log in to create fruits? If all it takes to create fruit is a user account and anyone can have a user account, what is the point of having them log in? Hmmmm . . . seems like for MVP we're only going to need users to create fruit; they don't need an account.

In the future, we may want this functionality if, say, we're going in the direction of an online store. But that's not what we're building at the moment. But, you may say, there is no harm in letting OAuth library or whatever login code we've written hang out in our codebase, so why not? And the answer is twofold: yes, there is a harm, but also, why waste our energy and resources on functionality that is not a part of the task at hand?

First the harm: the more dependencies you have, the more attack surfaces you have for bad actors. Although OAuth is a library we trust, it could be deprecated at some point and/or have vulnerabilities about which we know nothing. Besides, if you're not using it, why is it there?

Also, any code we write does run the risk of tripping up our integration and functional tests. There may be unintended consequences to the way parts of our code interact with each other or subtle changes to the user journey. Keep the tests green!

Second: cost of build. Why spend time on something we're specifically not building out right now? Building new functionality has a cost of time and human labor, so let's focus on what we know we want right now. We can address authorization and log in when we need those features.

For more information on YAGNI, checkout this great Martin Fowler post. He uses a shipping insurance company from Minas Tirith as an example. Hope it fares better than those in Osgiliath.

So let's take a look at the structure of our API so we can even know what-in-the-what we're testing:

ACTION: A call hits an endpoint, let's call it "/fruits", which triggers a database call that has us getAll() fruits. Actions and routing to the correct calls will be handled by a controller, FruitCartController.
LOGIC: Since all the FruitCartController does is route and return responses, we need to something to handle actual business logic. For that, we have a FruitService. FruitService decides how we're going to get information from the database (not what information--that's decided by the controller). So we're going to have a handy method called getAllFruits() which will do just that: get all fruits from the database.
DATA: So FruitService is going to call the database using a built-in JPA Repository method called findAll(). This FruitCartRepository is a mapper, and magically transforms data into the Java object of our choosing, in this case Fruit.

That's it.

So let's start at the beginning, with Marcus Aurelius . . . or the FruitCartController.

Because I watched Silence of the Lambs too many times, the quote "What is it in itself", which Hannibal Lector ascribes to Marcus Aurelius, continually pops into my head when trying to test. The full passage from Meditations Book 8 reads: "What is this thing in itself, in its own constitution? What are its elements of substance and material, and of cause? What is its function in the world? What is its duration?"

And in general, these are good things to ask code. What do we want this piece of code to do? What do we need it to do and what pieces of code do we need it to have? How long should it take to do these things? Expectations like these will guide our test-making

Here's what we need from FruitCartController:

To have an endpoint that successfully returns an HTTP response with status of OK.
This endpoint will also need to return a list of all the fruits in the database.

Cool. Let's write a test for the first bullet.

Q: So in an HTTP request, how do we know a response was successful?
A: A status code of 200, or OK.

So we made a test for that in a class called FruitCartControllerTests. Spring provides us with MockMVC, a servlet that mocks HTTP CRUD requests. If you're using IntelliJ (which I highly recommend for Java), you'll be prompted to import the correct library once you start writing code. Our final test reads:

@Test
public void shouldReturnHttpStatusOk() throws Exception {
  this.mockMvc.perform(get("/api/fruits").accept(MediaType.APPLICATION_JSON))
    .andExpect(status().isOk());
}

And when we run it, it fails. Great: we have our red. Time to get the green.

So now that we have the test, we need to write the code to make it pass.

The obvious file needs to be created (FruitCartController) along with the obvious route ("/fruits"--the "/api" portion is namespacing since we'll also have routing with similar names for the front end and want to be as clear as possible when naming). Then all we do is pass it in the most obvious way, going back to Mr. Aurelius--what is it in itself? What is its behavior? Well, all we're requiring at the moment is that it return a 200 status, so let's go ahead and make an endpoint that does just that.

@Controller
@RestController
@RequestMapping(value="/api")
public class FruitCartController {

    @RequestMapping(value="/fruits")
    public ResponseEntity getAll(){
        return new ResponseEntity(HttpStatus.OK);
    }
}

Run those tests and it should pass! Hurray! We have our green.

Now normally we'd refactor, either our test or our code, but in this case, the test is so small, it's hard to see where we would do that. The solution is pretty elegant and does exactly what it needs to do.

So let's take the test one step further; after all, we don't just want a status--we want fruit!

We'll alter our test to require the return of an array of JSON fruit objects. It should look like this:

[{id: 1, name: "apple", description: "A fleshy red fruit"}, {id: 2, name: "banana", description: "A minion's favorite fruit"}]

We'll expect to be able to access the first element in the array and get a name of "apple". Here's the code:

@WebMvcTest(FruitCartController.class)
public class FruitCartControllerTests {

  @Autowired
  MockMvc mockMvc;

  @Test
  public void shouldReturnAnArray() throws Exception {   
    this.mockMvc.perform(get("/api/fruits").accept
(MediaType.APPLICATION_JSON))
       .andExpect(status().isOk())
       .andExpect(MockMvcResultMatchers.jsonPath("$").isArray())
       .andExpect(MockMvcResultMatchers.jsonPath("$[0].id").value("1"))
  .andExpect(MockMvcResultMatchers.jsonPath("$[0].name").value("apple"));
    }

MockMVC performs that GET request, but on top of the status code of OK, we're also expecting some actual data. So of course, this promptly fails.

Back at red. Let's make it pass.

Getting the test to pass is actually going to be pretty easy: we make a Fruit model with some standard setters, getters, and a constructor, and then just hardcode the response we want directly into the return result in FruitCartController:

    @RequestMapping(value="/fruits")
    public ResponseEntity getAll(){
        Fruit fruit = new Fruit("1", "apple");
        return new ResponseEntity(Arrays.asList(fruit), HttpStatus.OK);
    }

It's green. Time for the refactor.

But is this really the way we want our system to work? The database isn't being queried, and we'd have to instantiate a ton of different Fruit objects within the controller, which is not its job. No. We need to make that database query to get all fruits.

Now remember in our outline of the system architecture, the controller only handles the routing, telling the rest of the backend what data it actually wants to get. But it doesn't actually get the data. Thus, its ability to fulfill its function depends on another layer of code: the service. Currently, FruitService doesn't even exist; we haven't written the code for it. But it sounds like we might have to start if we're going to get anywhere near how we actually want our controller to work.

And now our little test has moved beyond unit testing. It's an integration test: we're testing that two layers--controllers and services--are interacting appropriately so as to return the desired result.

So let's write some true unit tests; let's create our FruitService.

ABSOLUTE UNIT

Initially we thought FruitService would also be an integration test. After all, it is going to call on the FruitCartRepository, and that would need testing too, right? Well, not so much.

FruitCartRepository, while indeed a file, is an interface. It's a contract between files or layers. And all the functionality we're using is contained by the JPA Repository. So in the end all we have is a file that looks like this:

@Repository
public interface FruitCartRepository extends JpaRepository<Fruit, Integer> {

}

There's nothing inside of it. All we have to do is tell Spring that this interface is a repository (the @Repository handles that) and basically create it, specifying that it is mapping Fruit data from the fruit table onto Fruit objects and that the primary key in the database is an Integer datatype. All the actual data calls are provided by the library.

So if we were to test our FruitCartRepository, we wouldn't actually be testing any code we wrote. We would be testing the JPA Repository's code. And that, friends, has already been tested. We just need to make sure it's being called right in our own custom code, which means testing the FruitService.

Q: What is the core behavior of FruitService we need right now?
A: We need it to return an array of fruits.

What is the simplest thing it can return? An empty array! Frequently null or empty return values are the easiest way to start with testing. Baby steps.

So we made another test file for our FruitService class and made a test to get an empty array:

public class FruitServiceTests {

    FruitService fruitService;

    @Before
    public void setUp() {
        this.fruitService = new FruitService();
    }

    @Test
    public void shouldReturnEmptyArray() {
        assertThat(fruitService.getAllFruits(), is(Arrays.asList()));
    }
}

First we initialize a new FruitService called, originally enough, fruitService. Then we assert that it calls a method getAllFruits() and that we expect that value to be an empty array.

Run those tests, and boom: RED LIGHT.

Time for that refactor.

We go in, create that FruitService class, and create a method called getAllFruits() to return Arrays.asList(). That's it.

Run those tests, and boom: GREEN LIGHT.

But that's not really what we want, right? We want those fruits!

So we re-write the test to look for a fruit in that array:

@Test
public void shouldReturnArrayOfFruit() {
  assertThat(fruitservice.getAllFruits().get(0).getId(), is("1"));
}

That's it. We're trying to get the id from the first element in the array, which should be one.

Run that test, and we're back at red.

Now let's fix our FruitService. We replace the return value of Arrays.asList() with the following:

        Fruit fruit = new Fruit("1", "apple");
        return Arrays.asList(fruit);

And that's it: run the test, and the light is green.

But is this really how we want it to do this?

We're running into the same problem with our FruitService that we did with our FruitCartController. It's not querying the database, but creating objects right in its own logic, which is not what it should be doing. It should be querying the mapper (FruitCartRepository), which should query the database to get what it needs. It should then return an array of these objects. So what we need to test for is if it's calling the FruitCartRepository. We want to test it to be sure it's returning not only what we need, but doing it in the way we need it to. And for that we need to up our testing game.

We need a couple of new superheroes: Mocks & Stubs.

Fruit Cart: Testing Rules of the Game

Katie Larson — Tue, 25 Jun 2019 23:03:42 +0000

After a config full of "code holes" and discussion about what, if anything, we needed to add to our original stack, we were ready to start our first user story: the Landing Page.

Our initial page is designed to show the navbar along with a list of all the fruits in our database. The user story is pretty simple, done in the "As, wants to, so that" format: As a user, I want to visit the landing page so that I can see a list of all current fruits. We have a context (user), an action (visiting the landing page), and a value (seeing the current fruits).

Acceptance criteria are few and equally simple: if there are no fruits, user should should see an "Add Fruits" message; if there are fruits, the user should see the fruits. (In retrospect, error handling should also have been an AC: if the page fails to load, an error message is displayed.)

Fruits currently have just an id, name, and description--pretty simple data. Now the navbar is a separate card, so we're just focusing on the listing of fruit.

Next step: task it out. What do we need to do and what files do we need to touch to fulfill the ACs and finish the card?

With the database up, we need a fruit model (our backend is object-oriented), fruit repository to handle mapping of SQL to the model, fruit service to handle business logic, and fruit controller to provide actual endpoints and handle calls. We'll have to create directories and folders for all of that in the same package. But first things first: the tests.

The primary goal of this project is to create a simple CRUD app using TDD, or test drive development. Here's what this means in practice:

We write tests first Tests are the first lines of code we write whenever we start a new piece of functionality. Hence . . .
Tests define production code They define what code is written in the first place. They drive the code. For example: we know our fruit cart service needs to return a fruit item with a name "apple" when the getAllFruits() method is called. So we write a test that calls getAllFruits(), with the result expected to include the string "apple". This allows us to keep results and expectations in mind, and, ultimately become more value driven. After all, if you don't know what you want your code to do, why write it in the first place?
Tests are just as important as production code We spend equal time writing good tests and good code. They're not an after thought, and they allow us to write cleaner, more focused code in an incremental manner called . . .

Baby steps We take it one step at a time. If it's easiest to make the service pass the above test by hardcoding a return value of a string "apple", then that's what we do. The test will pass, and we can move on to the next step: refactoring.
Red, Green, Refactor Ah, the core of TDD. Write your test, make it fail (red). Write your code, and, in the easiest way possible, make it pass (green). This initially may mean hardcoding, just like in the example above. Then refactor your code. Hardcoding values is a code smell, so in this case, you would refactor it out. Every time you make a change that could alter the return value, run the tests.

Change either your test or code, not both Now tests can be refactored just as much as code can. But you don't want to make changes to both, run your tests, and watch them fail. Then you won't know what actually failed: the newly refactored code or the newly refactored tests. Furthermore, you may actually be changing what behavior your tests are expecting, making previously solid code fail.

The Testing Pyramid We'll be following the classic testing pyramid.
- Unit tests will make up the bulk of our test suite These tests run against discrete methods and have concrete expectations. For example, if I run the getAllFruits() method that gets all fruit information from the database, I should have a solid expectation of its return value (eg. an array of fruit objects, one of which contains the name "apple"). Their advantages are that they are fast and small: when they fail, they fail quickly, so we get near instant feedback (assuming we run our tests regularly); and when they fail, they fail for a single piece of functionality so it's quite easy to target what line of code failed. Plus, they take fewer resources to run. JUnit with the Hamcrest library will be our friends here for the backend, along with Jest and Enzyme for the React frontend.
- Integration tests form our middle ranks These are slightly more expensive in terms of computing resources, but they test the connections between the moving parts. For instance, our controller tests are actually integration tests--when we hit an endpoint, the controller calls the service which calls the repository which calls the database to get the fruits. Several methods across multiple classes are called here, and we want to make sure everything works. These are larger, require mocking and stubbing (which we will return to in a different post), and span across multiple classes. They make sure everything works together, but when they fail, they take longer to do so, and it can be hard to tell what, exactly, is failing. We'll add Mockito to JUnit for these tests.
- Functional or UI tests sit at the top They test everything--user interface, database, services, etc--to make sure it all works together correctly. For instance, when a user lands on our page, they should see all the fruits listed. That's the front and backend pieces all working together for a single result. These tests take longer and may require a zombie web browser to be spun up and simulate user interactions. When they fail, it can be extremely difficult to tell why. Selenium will be our tool of choice to automate these.

Those are the rules! Let the (fruity) games begin!

Fruit Cart: Config

Katie Larson — Fri, 21 Jun 2019 16:09:06 +0000

Stories sliced, goals set, it is now time to spin up the cart.

Config and project start has always been awkward for me. You only start a project once during its duration, so all the steps you have to take to get it code-ready are in neither my muscle memory nor my brain memory. After all, I'm not constantly starting greenfield projects.

But start we must. So in the beginning, here's what we did:

Bootstrapped a Gradle project using https://start.spring.io/ We added the JPA, Devtools, and Postgres dependencies, which you can search for on the site. We will be using MyBatis to run the migrations, but not as a mapper; that will be handled "under the hood" by the JPA repository library.
Initiated a Git repo (remote and local) for Fruit Cart and performed our first commit of the initial codebase.
Installed and ran a Postgres server. Followed these instructions to create a new database 'fruitcart' with superuser 'fruitcart':

Start postgres server and setup database * check that the fruitcart superuser exists run "psql -c '\du'" * if user does not exist run "c" (no need to set a password, defaults to 'postgres') * create database fruitcart run "./gradlew initdb" * create migrations for fruit table with columns id, description, and name. Run "./gradlew createdb" * to add some data run "psql -h localhost -U fruitcart --password -d fruitcart < db/backup.sql"

Ran a clean build to see if there were any errors (spoiler alert: there were--database wasn't configured properly).
Created Gradle tasks to initialize and create our database on clean build. After we got our database up correctly and running on port :5432, we were ready to start with our first tests.

But let's back up a second. Let's talk takeaways.

First, migrations must be created and run from the command line in the same directory where your database environment is located. I did know this before, but it bears repeating because I routinely forget it (hey, this blog is a resource for me too).

Command is:
./mybatis/bin/migrate new name-of-migration

This should create a file (the name of which begins with a timestamp--migrations run in the order in which they were created) in the scripts directory. Use this to add you SQL statements.

Second: If you treat it right, .gitignore is your friend.

So there are certain files we don't want to appear in our remote repo. I usually think of .gitignore as a convenient place to hide secrets, but I think more practically it allows us to hide those pieces of our project that are specific to our local builds and not necessary for building our project in other environments, such as those that track changes in our workspace or build.

We used https://gitignore.io/ to generated the correct .gitignore file for IntelliJ, our IDE for this project. It's a little overkill: we don't have Jira or the Crashalytics plugin enabled (pure IntelliJ CE level for us), but it gives us a good sense of what should be included. So we just copied and pasted that into our very own .gitignore. And all is well.

Well, not so much for Jeff.

One of the .idea files--workspace.xml--kept escaping from the .gitignore. He would try and commit/push his code, and there would be a failure: there were changes to his workspace.xml file that weren't tracked. But of course there were and of course it wouldn't be tracked: it's a file that tracks his location in the IDE and other internal structures particular to his workspace. For some reason, .gitignore was not telling git that this file shouldn't be tracked.

Turns out, if you already committed that file and pushed it to your remote repo, it will haunt you. In Jeff's words, "It's a piece sh**".

Solution: Delete the .idea folder (it will be generated automatically by IntelliJ anyway), commit your changes, then add .idea/workspace.xml to your .gitignore, and add/commit/push all that up.

tl;dr: if a file has already been committed to your repo it will be automatically tracked regardless of whether or not you put it in .gitignore

Third: there may be manual set up of SQL with your Postgres DB.

Our build was failing. Turns out our DB wasn't being created. So weird, given that Postgres was running (I think it always is on my machine; that and Docker). But the database wasn't being created. It just wasn't there.

Turns out we had to create a shell script to actually create the database, and when we did we created an owner, fruitcart, that didn't actually exist.

Our shell script made the createdb function just fine, and we executed the task in our build, but the database would actually fail because there was no superuser or even user with write permissions. Basically we hadn't followed the first two * of the instructions at the beginning of this post. Once we had, our fruit cart snapped into existence.

Now all this took so much longer than we thought. And this is going to be a running theme with us: we spike, go into internet wormholes, and spend a lot of time exploring. We looked at getting Jetty happening, discarded Jetty after numerous failures. We spent a long time with workspace.xml. We tinkered in Postgres. It's getting better. But it's hard to even know what questions to ask when you're learning. It's hard to even know where to look when you don't know what you don't know.

And after all, that's what this is: learning.

Fruit Cart: Inception

Katie Larson — Wed, 12 Jun 2019 22:24:40 +0000

The Fruit Cart project was born from one three-hour training course, the desire to create an entire application strictly test-driven development (TDD) style, and, well, Jeff. Jeff is my pair for this project. It's his little blueberry.

In this series, I'm going to be walking through our process of creating a SPA using ReactJS that fetches info from a Java/Spring Boot API and Postgres database. It's entirely test-driven: our tests write our code and take just as much primacy in the process as functionality. We'll be using JUnit with the Hamcrest library, Jest with Enzyme, and (eventually) Selenium.

Now it's not my intention to create a tutorial that can be repeated step-by-step. My aim is to reflect on the process and reinforce what I'm learning. For instance, what were the pain points? What did we have to research? Which resources were valuable and why? Where were their pain points? What do we wish we had?

So here's where we started:

Inception

I'm a fairly organized person, so I like to know what I need to do, when I need to do it, and why I need to do it. Otherwise I find myself in what I call a "code hole": an internet wormhole composed of nothing but docs, Stack Overflow posts, Baeldung pages, and DZone tutorials. By the time I emerge, I frequently have lost sight of what I was researching in the first place.

To avoid a code hole, we split our very basic CRUD cart into user stories. It's an easy way to task out what, exactly, we want to happen.

We ended up with 5 stories detailing the 7 RESTful routes (index, show, new, create, edit, update, destroy), 1 story for navigation, and 2 cards related to image uploads. Here's an overview of what we want the user to do on the cart:

When a user reaches the landing page and there are fruits in the
database, they should see a list (name, description) of all those
fruits. Otherwise, if there are no fruits, they see the Banana Man and
are prompted to add a fruit.

The user should also see a navbar with "Add Fruit" and "Home" as
options; the navbar will be visible on any "page" in the Fruit Cart
app. Each fruit description and name is a link that routes users to
the show page for that particular fruit (see #4 below).

When the user clicks on the "Add Fruit" message or "Add Fruit" option
in the navbar, it takes them to a form with name and description
fields with the save button.

Upon submission of a completed fruit form (i.e. when the user clicks
"save"), the fruit is created, and the user is taken to the show page
for that fruit.

The show page displays the name and description of the fruit, as well
as a button to edit the fruit.

When the user clicks the "edit" button, they are directed to the edit
form with pre-filled information in the name and description fields.
They edit, then submit the form to update the fruit. They are then
redirected to the newly-updated show page for the fruit they just
edited. If they were to click on the home link in the navbar, they
would be redirected to the landing page, now with any new or updated
fruits they have created.

And that's it! Additional functionality will include the ability to upload and insert images, but that's likely for version 2.0.

Now it's not super complicated; there's no log in or log out functionality or social media aspect, but that's not the point. The basics are the point: TDD and RESTful APIs, small React components and minimal dependencies, Java and Spring Boot. The experience is the point.

Next up: Landing page! Let's see all those beautiful fruits. 🍉 🍌🍎 🍍 🍒