Grails 3.3.10 to 4.0.2 Postmortem

Tucker Pelletier — Tue, 12 May 2020 02:04:32 +0000

The Grails 3 to 4 upgrade was nowhere near as daunting as upgrading from 2.5.x. However, I thought I would do a postmortem for the upgrade just the same. This is for a project that has about 80,000 lines of Groovy/Grails, and 40,000 lines of GSP, which is quickly becoming deprecated for Angular.

Tasks:

Custom password salt implementation finding temp workaround then replacing, with bcrypt.
Removing a duplicate class wonder why this didn't cause issues before
Removed a class that was just a script, which means it had a main method which caused errors
Fixed unit test the fall back for the mixins didn't seem to work anymore
Gorm/HQL had to replace all ? params with positional ?0,?1,..., etc.
Had to add enabled = true to war task
groovy.json.internal.LazyMap was being used with instanceof had to update to org.apache.groovy.json.internal.LazyMap
Many instances where hibernate proxies were not being unwrapped. Fixed with GrailsHibernateUtil.unwrapIfProxy or fetch:'join' depending on circumstances
DB migration plugin no longer has access to system properties so I had to add a fallback candidate to the config.
Actuators and h2 console changed so I had to update the security around them
Had to move where I defined the sourceSets for the resources for the db migrations
Had to exclude old converters from grails-datastore-rest-client, which we should upgrade
Had to fix spring security ldap: https://www.baeldung.com/spring-boot-security-autoconfiguration spring.autoconfigure.exclude=['org.springframework.boot.autoconfigure.ldap.LdapAutoConfiguration', 'org.springframework.boot.actuate.autoconfigure.security.servlet.ManagementWebSecurityAutoConfiguration']

Other resources:

Grails 2.5.5 to Grails 3.3.10 Postmortem

Tucker Pelletier — Tue, 12 May 2020 01:44:43 +0000

Here's my postmortem of what I did in an upgrade from Grails 2.5.5 to Grails 3.3.10 I figured this may help someone... The app I was working on was fair-sized about 80,000 lines of Groogy/Grails and 40,000 lines of GSP, not including angular code, which is where the app is moving to. How hard a Grails 2 to 3 upgrade is a function of:
How many bad practices your codebase had * the number of lines of code.

Tasks/commits:

Moved a lot of db writes in controllers to services with transactions
Using the resource plugin had to upgrade to the Asset Pipeline
Layouts, being used as templates
Inline plugins that were deprecated, needed to be detangled and removed
Moving a lot of code from web-app to src/main/webapp and fixing hardcoded links in js files
Repackaged controllers, services, and domain objects from one or no package, into something more manageable.
Porting JQgrid for older GSP code because the plugin wasn't ported to Grails 3 *Reworking the special password encoder that uses a hard-coded salt, one from the db, and one passed in from config. Later ditch for Bcrypt
Porting a custom constraint to Grails 3
Porting log4j to logback
Refactoring a Util class that used GrailsDomainClass API to the Persistent API, and several classes that used it, Did changes several times, Was Like a game of wack a mole.
Replacing ../ with / in gsp files
Replacing a deprecated MD5Codec with encodeAsMD5 Ported AsyncEventPublisher to use Grails events
Replacing CommonsMultiPartFile with CommonsMultiPartFile
Fixing some static compilation issues
Converting Filters to Interceptors
Two versions on Angular
Old GSPs using Jquery instead of Angular
Fixing a class that was using an @Override that wasn't overriding anything...
Fixing a bunch of custom validators that broke on upgrade
Removing a lot of @Slf4j annotations that were unnecessary, and caused issues with some traits
Fixed imports app, unit tests, and integration tests.
Grails Mocks needed to be replaced
GrailsUnitTestCase needed to be replaced.
Validateable annotation had to be replaced with the Validateable trait
Removing unused dependency that was causing build errors.
Fixing a bunch of queries that were using string interpolation, but were also parameterized
Fixed Spring security Roles missing the ROLE_ prefix, several times, all over the app
Rolled back some of the Role prefix changes, because the role table was used for security roles, and roles in a different sense.
Ported integration tests to use @Integration and @Rollback
Added the external config plugin, to maintain the external configuration. And convert the YML, to Groovy config.
JWT didn't have a secret key set in Grails 2 but needed it in Grails3
Added @Transactional in some places that were missing it...
Fixed integration tests, that were doing setup calls outside the setup methods
Making integration helper classes that generate data in setup methods @Transactional
Adding missing @Mock([
Replacing constraints with constrainedProperties
Eliminating with{} in Spock tests
Removed autowire from Domain classes
Added pathingJar = true for windows people
Dealt with various Gradle commands not having access to System properties, which are needed for licensing validation.
Fought with a test that was failing because of @Memoized Sank a lot of time trying to get the test reports in the right place for the Jenkins jobs to pick them up
A method that was annotated with @Transactional(readOnly=true) was doing a delete
Fixed bouncy castle Gradle excludes
Replaced PEMReader with PEMParser
Fixed the way we got metadata for the app version
Replaced uses of jdbcTemplacte to fix some integration tests
Fix some HQL queries that were using id instead of the domain object
Replaced some uses of a custom ApplicationContextHolder with Holders, which solved some issues
Removed some deprecated logic dealing with releaseLocalThreadMemory() DomainClassGrailsPlugin.PROPERTY_INSTANCE_MAP.get().clear()
Added Try catch to the interceptor and fixed the issue so the logs don't blow up.
Moved a cascade from constraints to mapping
Found several times that params don't have to be included on redirects and forwards, and if they are they can be duplicated into an array
Fixed a GSP that could include properties or not... wrapped with a <g:if tag
Updated Quartz jobs to work with new plugin swapping static with def
Fixed command objects trait to ignore certain properties like constraintsMap, metaClass, class, constraints
Fixed a test that was missing a controller context...
Fixed several issues where associated properties of a domain class were not accessible in GSP files, so passed then as part of the model
Started to have memory issues with the DB migration, This was cleared up by removing a lot of legacy changes from the changelog, and rebasing to a new "gold" image to start.
All DB migration scripts in one folder(updated in the middle of a sprint right before a release) to be by version.
Fixed a bunch of URLs, that didn't have the application context *Command Object not binding body of delete, because it's not supposed to have one, used a util method that does the binding for me.
Several places used the query parameter format, that didn't work the way it did before. used withFormat in several cases, some that I shouldn't have *Refactored the places that I shouldn't have used withFormat to use command objects, because the params could come in as ajax post, or form post.
Found a melody plugin issue between @Transactional and a Property of type class
Swapped from failOnError:false to failOnError:true, and making a lot of code changes because of it. Don't do this, this is wrong.
Fixed imports in GSP files

Things that held up or slowed down progress, things hopefully you can avoid:

Had to continue to merge changes coming from the Grails 2 branch that was actively being developed, and fix things as they broke.
Moved new tests/integration test/web-app/src/groovy files to there new locations after merges
Still had to work on pull request periodically
Two versions on Angular, which was later reduced to one
Got pulled back to work on the previous version of the app is several cases
About 60+ security issues(XSS, SQL injection, CSRF) which took over the team for quite a while
After which we stopped merging, between branches and had to cherry-pick between the three branches were are maintaining(two Grails 2, and the new Grails 3)
Was left without QA for over a month.

More resources:

This wasn't my first Grails 2 to 3 upgrade, I did another one that wasn't anywhere near as complicated, but the codebase was under 10,000 line of Groovy code. I'll publish another article for my Grails 4 postmortem, which is much shorter.

DEV Community: Tucker Pelletier

Grails 3.3.10 to 4.0.2 Postmortem

Grails 2.5.5 to Grails 3.3.10 Postmortem