DEV Community: Joe McCall

Grails 5 no longer runs my tests!

Joe McCall — Mon, 21 Mar 2022 19:56:05 +0000

Problem

I just followed the guide to upgrade from Grails 4.0 to Grails 5.1 here. I now try to run my unit and integration tests and Gradle is reporting there are no tests to run!

Solution

In our case, we need to add a new line in build.gradle:

tasks.withType(Test) {
    useJUnitPlatform() // <-- Here
    systemProperty "geb.env", System.getProperty('geb.env')
    systemProperty "geb.build.reportsDir", reporting.file("geb/integrationTest")
    systemProperty "webdriver.chrome.driver", System.getProperty('webdriver.chrome.driver')
    systemProperty "webdriver.gecko.driver", System.getProperty('webdriver.gecko.driver')
}

This tells Gradle that we're using JUnit-style tests.

"Repository Not Found" when accessing a GitHub remote

Joe McCall — Wed, 09 Dec 2020 14:55:57 +0000

Boy was this frustrating! Basically I have several keys in ~/.ssh/ that I use to access various projects. It turned out that one of them was added to GitHub on another account (likely a bot account I had created for another project).

I would try to run a git pull but would be greeted with an error:

$ git pull
ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

This would only happen on my org-owned projects, since my personal projects worked just fine. So what was the issue?

We get a hint by following the troubleshooting tips here: https://docs.github.com/en/free-pro-team@latest/github/creating-cloning-and-archiving-repositories/error-repository-not-found

$ ssh -T git@github.com
Hi <not my expected username>! You've successfully authenticated, but GitHub does not provide shell access.

Basically my SSH agent (gnome-keyring) was presenting a key that belonged to a valid GitHub user (so authentication passed), but not one with access to my project (so GitHub denied access).

In other words, even though I had the correct key in my keychain, the SSH agent presented the wrong one, so I couldn't access the repository.

The solution: add this to my ~/.ssh/config:

Host github.com
    IdentityFile /path/to/the/correct/key

Now when I perform a git pull it authenticates correctly:

$ git pull
Already up to date.

On simplicity

Joe McCall — Wed, 27 May 2020 12:15:24 +0000

We're all familiar with the phrase, "Keep it simple, stupid!" However over the years I have heard this uttered in a rather dismissive tone. What is really being communicated is, "My criticism of your solution relates to its apparent complexity, therefore you are unintelligent." This is not only mean, but also incorrect. I argue that simplicity is in the eyes of the beholder, so to speak.

Let us examine a hypothetical "pet-store" example. Note that some of the details moving forward are intentionally exaggerated for the sake of making a point. In this pet store application, the existing code base is standard HTML/CSS, with a small amount of javascript (to drive things like analytics, or a twitter feed). It allows the user to browse available pets for adoption, read descriptions, leave comments, etc. Authenticated users can log in to add new pets and remove adopted pets from the list.

Now, as of writing this post, everyone is encouraged to stay home for public safety. So there are far fewer customers than normal at our pet store in-person, where adoptions typically take place. So we decide to add a "virtual" adoption experience with real-time video and chat.

Let's keep it simple! Our stack is HTML/CSS/JS, so we should just write our chat application from scratch using javascript, rendered on our existing HTML page. No need to introduce new libraries or languages into our stack.

Wait, that's re-inventing the wheel! We don't want to write all that new code! It will be a mess! What about cross-browser compatibility? I guess there's a thing called WebRTC for video chat. Is my team prepared to embark on solving unknown problems by writing all this code from scratch?

Let's keep it simple! This problem has been solved before. In 2020 people use React/Vue to program real-time applications. There are several examples and ready-to-use libraries that do exactly what we want. We could easily throw together some isolated testable components, and render them on our page by including a bundle.js.

But now our code base has a mix of HTML/CSS/JS for our main Pet Store, and JS/JSX for our chat application. If there's an issue with one of our pages in the pet store app, we now have two different places to look. We also need to perform an additional step to generate a bundle before we deploy, but only for the javascript we wrote for the chat app (i.e., operational complexity was introduced). It's inconsistent. Some developers of our team are much more comfortable editing the static portion of our site, while other developers prefer working with JS/JSX. This inconsistency has now split the team!

Let's keep it simple! All of our code should be in one language/framework for ease of maintenance. We re-implement the HTML/CSS/JS sections of our site to be react components, and our page is a simple HTML file that serves the "uber-bundle.js." All CSS rules are expressed as inline react stylesheets. All developers can now focus on one standardized way of doing things.

Hold on though, now we've completely bypassed standard HTML. Our users with screen readers have all complained that our site is completely broken. Furthermore, our search engine score has dropped significantly (someone from our team mentioned something about needing server-side rendering for this?). We also have to worry about maintaining browser history and populating the URL bar for bookmarks- something the browser used to do for us! We have introduced problems here that we didn't need to solve before.

In reality, the simplest answer is going to be a compromise. Any new code will introduce at least the complexity of learning/maintaining a "new" way of doing things. We all want to arrive at the simplest solution in the end, and it benefits nobody good to call each other "stupid." Often times complexity is just pushed around from one component to the next. When evaluating a seemingly complex solution, let us all keep in mind that simplicity is not all that simple to discover.

Until next time!

Terraform - InvalidID on aws_eip

Joe McCall — Tue, 28 Apr 2020 17:39:23 +0000

I was trying to apply a terraform template today and ran into this error:

module.network.aws_eip.eip_nat_az2: Creating...

Error: error adding tags: error tagging resource (18.214.131.208): InvalidID: The ID '18.214.131.208' is not valid
        status code: 400, request id: 49aab8a6-8c26-4327-90a5-389a2e5962bd

  on ../modules/network/main.tf line 104, in resource "aws_eip" "eip_nat_az1":
 104: resource "aws_eip" "eip_nat_az1" {

Error: error adding tags: error tagging resource (100.24.169.116): InvalidID: The ID '100.24.169.116' is not valid
        status code: 400, request id: 212e2891-6500-4150-94e1-76f28603a2b2

  on ../modules/network/main.tf line 111, in resource "aws_eip" "eip_nat_az2":
 111: resource "aws_eip" "eip_nat_az2" {

Error: error adding tags: error tagging resource (52.206.41.2): InvalidID: The ID '52.206.41.2' is not valid
        status code: 400, request id: 38dbad9b-8985-428d-ae67-064fdd4f0f93

NOTE: These IPs are just random from the AWS pool and don’t really point to anything useful.

Searching for this issue yielded no results, so I figured I should post the solution.

It turns out my module had defined eips like so:

resource "aws_eip" "eip_nat_az1" {
    tags {
        Name = "foo"
        Environment = "bar"
    }
}

When I examined the created IP addresses in the AWS console I saw that they were “Classic” EIPs and not VPC.

I had forgotten to specify vpc = true. The module now looks like this:

resource "aws_eip" "eip_nat_az1" {
    vpc = true
    tags {
        Name = "foo"
        Environment = "bar"
    }
}

Now terraform apply runs cleanly!

Groovy’s compareTo operator and Equality

Joe McCall — Sat, 11 May 2019 16:54:23 +0000

Let’s assume we have a Book class that looks like this:

@CompileStatic
class Book {
  String name
  String author
  Float price
}

Within our app we have several places where a widget is presented to the end-user, such as an API that drives a mobile app, an API that drives a single-page-application (like react/vue/etc), or even something rendered server-side like a GSP.

The data is represented well, and the customer is happy, but now there’s a business rule that states that all books should be presented sorted alphabetically. We can do this several ways:

Sort it in a controller
Sort it in the view itself
(Edit) Specify the sort order in the domain class

Edit: If you are using a data source that supports it, you can specify sort columns and order directly in the mapping closure. The point of this post still stands outside the context of GORM though. If you have a list of any POGO you could be affected.

If we sort it in the controller it looks like this:

@ReadOnly
def index() {
   // old code
   //respond Book.list()

   // new code to sort by name
   respond Book.list().sort { it.name }
}

This will work, but some developers may want to record the fact that a book’s “natural order” is that of being sorted by name. In other words, the fact is stored on the domain itself of how it should be sorted. Let’s impliment this using java’s compareTo method:

@CompileStatic
class Book implements Comparable<Book> {
    String name
    String author
    Float price

    @Override
    int compareTo(Book other) {
        // Sort by name ascending
        return this.name <=> other.name
    }
}

Now our controller method looks like this:

@ReadOnly
def index() {
    // old code
    //respond Book.list()

    // new code to sort, this time by using the pre-defined "natural order"
    respond Book.list().sort()
}

THIS HAS A VERY SERIOUS SIDE-EFFECT

The above is not an ideal solution, and it’s best illustrated with a concrete example.

def book1 = new Book(name: 'The Gadfly', author: "Ethel Voynich", price: 19.95)
def book2 = new Book(name: 'The Gadfly', author: "Johnny Copycat", price: 9.95)

assert book1 != book2 // THIS FAILS!!! GROOVY THINKS THEY ARE EQUAL!

It’s a bit worrisome that the two books are equal. This is clearly not the case.

What’s happening?

I’ll get to the technical reasons in a minute, but based on observation, it appears that it’s only checking the name field of our class for equality, and stopping there.

At this point all we know is the language is incorrectly interpreting the idea of equality between two books. It thinks that just because the book names match, that the book objects must be equal.

Let’s try to fix this

We introduced natrual ordering, but it has clearly affected the definition of equality for books. Therefore let’s try to rectify this situation be defining an equals method so there’s no confusion:

@CompileStatic
class Book implements Comparable<Book> {
    String name
    String author
    Float price

    @Override
    int compareTo(Book other) {
        // Sort by name ascending
        return this.name <=> other.name
    }

    @Override
    boolean equals(Book other) {
        this.name == other.name &&
            this.author == other.author &&
            this.price == other.price
    }
}

Now we’re expressing the idea that a book can only equal another book if the name, author, and price match.

Let’s test this:

def book1 = new Book(name: 'The Gadfly', author: "Ethel Voynich", price: 19.95)
def book2 = new Book(name: 'The Gadfly', author: "Johnny Copycat", price: 9.95)

assert !book1.equals(book2) // This passes, we should be ok
assert book1 != book2 // THIS STILL FAILS!!! WHY IS THIS?!

Why didn’t defining `equals` fix `==`?

After all, doesn’t the == operator just delegate to the equals method in groovy?

Let’s look closer at the documentation: http://docs.groovy-lang.org/latest/html/documentation/index.html#_behaviour_of_code_code

In Java == means equality of primitive types or identity for objects. In Groovy == translates to a.compareTo(b)==0, if they are Comparable, and a.equals(b) otherwise. To check for identity, there is is. E.g. a.is(b).

Oof. This really hinders our ability to define a natural order. Even if we define equals, it will be ignored if our class implements Comparable.

What is the solution?

Let’s look at the Java recommendations for the Comparable interface: https://docs.oracle.com/javase/7/docs/api/java/lang/Comparable.html

It is strongly recommended (though not required) that natural orderings be consistent with equals. This is so because sorted sets (and sorted maps) without explicit comparators behave “strangely” when they are used with elements (or keys) whose natural ordering is inconsistent with equals. In particular, such a sorted set (or sorted map) violates the general contract for set (or map), which is defined in terms of the equals method.

In groovy, if we follow this recommendataion everything works fine. So let’s make compareTo consistent with equals:

@CompileStatic
class Book implements Comparable<Book> {
    String name
    String author
    Float price

    @Override
    int compareTo(Book other) {
        // Sort by name ascending
        int cmp = this.name <=> other.name

        // Then sort by author, then price
        if (!cmp) {
            cmp = this.author <=> other.author
        }

        if (!cmp) {
            cmp = this.price <=> other.price
        }

        return cmp
    }

    @Override
    boolean equals(Book other) {
        this.name == other.name &&
            this.author == other.author &&
            this.price == other.price
    }
}

Now when we call compareTo on another instance it will only return 0 when the instances are equal.

def book1 = new Book(name: 'The Gadfly', author: "Ethel Voynich", price: 19.95)
def book2 = new Book(name: 'The Gadfly', author: "Johnny Copycat", price: 9.95)

assert book1.compareTo(book2) != 0 // passes
assert !book1.equals(book2) // passes
assert book1 != book2 // passes

That’s too much code. Is there a better way?

We can use Groovy’s built-in transforms to achieve the same result with far less code:

@CompileStatic
@Sortable(includes = ['name', 'author', 'price'])
@EqualsAndHashCode // Not required, but here for completeness
class Book {
    String name
    String author
    Float price
}

The @Sortable annotation implements the Comparable interface for us. Furthermore, if the includes argument is used it will check the fields in the order they are listed.

I’ve listed the @EqualsAndHashCode transformation here as well to generate those methods, simply because we had them in the above snippets. They aren’t strictly necessary to make our test pass since Groovy will only look at compareTo, but I think that documenting that we are defining equality somewhere in this class is important, and this is a way to achieve that.

Just a word of caution: be sure to include all fields in the @Sortable includes list. Otherwise the natural order of the class becomes inconsistent with the equality.

What if I want to use a custom `equals` method?

I can’t think of a reason you would need this, but it should be able to do something like this:

    @Override
    boolean equals(Book other) {
        // custom definition of equality
        // ...
        return equalsObject(other)
    }

    @Override
    int compareTo(Book other) {
        if (this.equals(other)) {
            return 0

        } else {
            // Normal sort logic from above:

            // Sort by name ascending
            int cmp = this.name <=> other.name

            // Then sort by author, then price
            if (!cmp) {
                cmp = this.author <=> other.author
            }

            if (!cmp) {
                cmp = this.price <=> other.price
            }

            return cmp
        }
    }

Remember, the key is to ensure that the natural order is consistent with equality.

I hope this helps someone out!

DataBinding and GORM objects

Joe McCall — Wed, 20 Feb 2019 20:54:23 +0000

In Groovy, you are able to do this:

class Widget {
  String name
}

def widget = new Widget(name: '')
assert widget.name == ''

When this widget is a GORM object (in a Grails application), the default configuration doesn’t allow for this:

// Inside grails-app/domain
class PendingDeletion {
  String name
  String path
}

// Elsewhere in code
def pendingDeletion = new PendingDeletion(name: 'foo', path: '')
assert pendingDeletion.path == '' // Fails; pendingDeletion.path is null

This is because the default configuration instructs the Grails data-binder to convert empty strings to null ¹.

I’m not exactly sure why this is the default. In order to make the above snippet work in your Grails application, specify the following in application.yml:

grails:
  databinding:
    convertEmptyStringsToNull: false

This allows the GORM map constructor to behave the same as a normal POGO map constructor.

Alternatively, just do not use the GORM map constructor when the field values can be null.

http://docs.grails.org/3.3.9/ref/Constraints/nullable.html ↩

Grails Views - Rendering Empty Lists

Joe McCall — Fri, 25 Jan 2019 20:54:23 +0000

Do you have a controller action that looks like this?

def index() {
  respond myService.listItems()
}

Does that controller show a view like this?

model {
  List<Item> itemList
}

json g.render(itemList)

This may appear to work fine when itemList is populated, but you may notice that the API is broken when the index method is run when there are no items in the list:

$ curl http://localhost:8080/api/items
[
   null
]

What? This is supposed to render [] because we’re responding with an empty list. What happened?

Let’s look at this from the documentation found here: http://views.grails.org/latest/

If the collection is empty, emptyCollection will be used as the default model name. This is due to not being able to inspect the first object’s type.

So the itemList field on our model is null because the auto model-binding computes to emptyCollection. There are technical reasons for this. The fastest way to fix this is to change how we call respond:

def index() {
  respond([itemList: myService.listItems()])
}

This forces the itemList in the model to be bound manually, whether it is empty or not.