DEV Community: Vladimir Nemergut

Splitting Liquibase changelog? No problem.

Vladimir Nemergut — Tue, 24 Mar 2020 11:03:17 +0000

What is Liquibase?

Liquibase is an open-source database-independent library for tracking, managing and applying database schema changes. It was started in 2006 to allow easier tracking of database changes, especially in an agile software development environment.

I find Liquibase as a neat tool to migrate your database automatically. DB migration itself is a very complicated topic anyway, outside the scope of this article.

Liquibase can run as a standalone tool or it can be integrated into your application. It's easy to add it to Spring context.

Spring Boot makes it even easier. Liquibase is autoconfigured if you enable it in the properties file, you have Liquibase in the classpath and you have DataSource in the context.

liquibase.change-log=classpath:changelog.xml
liquibase.enabled=true

Liquibase makes the MockMvc testing very simple, too. One can configure it to create the H2 database for the testing purposes.

How Liquibase works?

Liquibase reads the xml changelog file and figures out what changesets it needs to apply. It uses the DATABASECHANGELOG table in your DB (DataSource) for this purpose. The DATABASECHANGELOG contains the list of changesets that are already applied with their ID, FILENAME, MD5SUM, AUTHOR and few other properties.

The logic is relatively simple. Just by comparing the changelog with the table Liquibase knows what changesets it needs to apply. There are, however, few gotchas ...

Liquibase can only take one changelog file
Liquibase determines the list of changesets to apply before applying them
the actual DB can get out of sync with the DATABASECHANGELOG table. E.g. if you manually modify database, or so on
if Liquibase fails to apply a changeset, it fails immediately and won't continue with next datasets
if Liquibase is running in Spring app as a bean, it executes during application startup, hence if it fails, then the application won't start
changesets are not atomic. It can happen that part of the changeset passes, it modifies the DB properly, and next part fails. The changeset record won't go into DATABASECHANGELOG table. Hence it leaves the DB in the state that requires manual repair (e.g. reverting the part of the changeset and letting Liquibase to run again)
the changesets can't be modified. If you modify changeset after it was applied in your db, then Liquibase fails stating that the MD5SUM doesn't match.
The ID is not the unique identifier of the changeset. It is in fact the combination of ID, FILENAME and AUTHOR
the changesets that are in the DATABASECHANGELOG and are not in the changelog files are ignored

Of course, Liquibase has much more functionality. Just read the documentation. This is also out of scope of this article.

The changelog file grows over time

Yep, if you don't define some strategy at the beginning, your changelog file will just grow bigger and bigger. On a large project it can be a couple of thousands of lines long with hundreds of changesets. There is a high code churn on the changelog file, too, so it will cause you some merging effort.

There are a few alternatives that you should consider early on to avoid this.

Define multiple changelog files

.. and <include> them in the master changelog, e.g.:

<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog dbchangelog-3.5.xsd">

    <include file="feature1.xml" relativeToChangelogFile="true"/>
    <include file="feature2.xml" relativeToChangelogFile="true"/>
    <include file="feature3.xml" relativeToChangelogFile="true"/>
</databaseChangeLog>

The benefit of this one is obvious - less code churn, better organization. The problem comes if there are any logical dependencies between changesets across the files - e.g. if there are any relations defined between tables of multiple files. Since the Liquibase executes the changesets in sequence, it starts with feature1.xml, continues with feature2.xml.

Perhaps you can find out a better split key - based on target releases perhaps?

    <include file="release_0.1.0.0.xml" relativeToChangelogFile="true"/>
    <include file="release_0.1.0.1.xml" relativeToChangelogFile="true"/>
    <include file="release_1.0.0.0.xml" relativeToChangelogFile="true"/>

Configure multiple Liquibase runs

Since one run can only take one changelog file, just define multiple changelog files and let the Liquibase run multiple times.

In your Spring (Boot) app just define multiple liquibase beans:

import liquibase.integration.spring.SpringLiquibase;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.DependsOn;

import javax.sql.DataSource;

@Configuration
public class MultipleLiquiaseConfiguration {

    @Bean
    public SpringLiquibase liquibaseRelease1(DataSource dataSource) {
        SpringLiquibase liquibase = new SpringLiquibase();
        liquibase.setDataSource(dataSource);
        liquibase.setChangeLog("classpath:release_v1.xml");

        return liquibase;
    }

    @Bean
    public SpringLiquibase liquibaseRelease2(DataSource dataSource) {
        SpringLiquibase liquibase = new SpringLiquibase();
        liquibase.setDataSource(dataSource);
        liquibase.setChangeLog("classpath:release_v2.xml");

        return liquibase;
    }
}

Both beans will be created in the context, hence 2 Liquibase runs will be performed. If you rely on the Spring Boot's autoconfiguration, your entityManager bean will force you to have one bean called liquibase. This is easy to do. Also, if your changelogs need to run in a certain order, you can solve this with @DependsOn:

@Configuration
public class MultipleLiquiaseConfiguration {

    @Bean
    public SpringLiquibase liquibaseV1(DataSource dataSource) {
        SpringLiquibase liquibase = new SpringLiquibase();
        liquibase.setDataSource(dataSource);
        liquibase.setChangeLog("classpath:release_v1.xml");

        return liquibase;
    }

    @Bean
    @DependsOn("liquibaseV1")
    public SpringLiquibase liquibase(DataSource dataSource) {
        SpringLiquibase liquibase = new SpringLiquibase();
        liquibase.setDataSource(dataSource);
        liquibase.setChangeLog("classpath:release_v2.xml");

        return liquibase;
    }
}

Note, that the last to run is called liquibase (which your entityManager depends on) and it points to the previous-to-run with @DependsOn annotation.

How to deal with long changelog?

If you haven't applied any strategy early on, or you just joined a running project with legacy code, your changelog is already too big. Now, how to reduce it?

You might say - well, I just split it to multiple files and use either of the 2 strategies as mentioned above. Well, not so fast! :) I mentioned earlier that the filename is important as it is used to determine if a changeset was applied or not. If you simply move existing changesets to another file, Liquibase would think that those changesets were not applied and in fact will try to apply them again. And it will fail as the DB already contains the changes.

To describe the issue a bit better, just imagine a model situation having this changelog.xml:

<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog dbchangelog-3.5.xsd">

    <changeSet author="me" id="changeset1">
        <createTable tableName="TABLE1">
            <column name="COLUMN1" type="VARCHAR2(10)"/>
        </createTable>
    </changeSet>

    <changeSet author="me" id="changeset2">
        <createTable tableName="TABLE2">
            <column name="COLUMN1" type="VARCHAR2(10)"/>
        </createTable>
    </changeSet>
</databaseChangeLog>

And you do move the second changeset to changelog2.xml and include changelog2.xml in the changelog.xml. Starting your app will fail with similar exception:

Table "TABLE1" already exists;

Ok, it will work just fine in your unit tests, since the DB is created from scratch, but will fail if you run Liquibase to migrate the DB of your deployed instance. We all agree that this is bad ;)

Luckily, we still have a few options left ;)

Change the `logicalFilePath`

Liquibase allows you to define so called logical file path of your changelog. This allows you to fake Liquibase that the changesets actually come from the same file. Imagine the changelog2.xml would look like this now:

<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog dbchangelog-3.5.xsd"
                   logicalFilePath="classpath:changelog.xml">

    <changeSet author="me" id="changeset2">
        <createTable tableName="TABLE2">
            <column name="COLUMN1" type="VARCHAR2(10)"/>
        </createTable>
    </changeSet>
</databaseChangeLog>

Note the loficalFilePath value there. Yes, this will work, Liquibase will treat this changeset2 as if it was previously defined in changelog.xml. Perfect.

Actually, this approach has also a few drawbacks that might (but might not) stop you from applying. If you don't store your changelog in the resources, but rather elsewhere in filesystem, your DATABASECHANGELOG will contain the full path to the file. If you then have multiple environments where you want to migrate DB and your changelog file location vary, you have no way how to set the logicalFilePath. Remember that it must match the previous value.

Another issue is that this approach is not the best if your intent to split the changelog is to move the part of it to another package, module, and so on.

Use intermediate changelog

If you intend to move part of your changelog to another module (e.g. you finally want to break that nasty monolith of yours to a few microservices having their own database), this approach might suite you the best. It contains some intermediate and temporary steps, but the outcome is what you want :)

The first step is to move all the relevant changesets to another file elsewhere. In our example above we just move the changeset2 to changelog2.xml. Now we need to fake Liquibase that those changesets didn't change. We do it by modifying the FILENAME value in the database as part of the Liquibase changelog itself ;)

Create one more (intermediate/temporary) changelog (let's call it tmp-migration.xml) with just this one changeset:

<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog dbchangelog-3.5.xsd">

    <changeSet id="moving changesets from changelog to changelog2" author="Maros Kovacme">
        <sql>
            UPDATE DATABASECHANGELOG
            SET
            FILENAME = REPLACE(FILENAME, 'changelog.xml', 'changelog2.xml'))
            WHERE
            ID IN (
            'changeset2'
            );
        </sql>
    </changeSet>

</databaseChangeLog>

This changeset will replace the FILENAME column value in the DB from classpath:changelog.xml to classpath:changelog2.xml. When we then run Liquibase with the changelog2.xml, it will think that all changesets are already applied. It is not possible to use just 2 changelog files for this purpose. Liquibase first calculates the list of changesets to be applied (per changelog file) and only then it will apply them. We need to modify the FILENAME before it processes the second file.

The last step we have to apply is to define the corresponding beans in our context in the right order:

@Configuration
public class MultipleLiquiaseConfiguration {

    @Bean
    public SpringLiquibase liquibaseChangelog(DataSource dataSource) {
        SpringLiquibase liquibase = new SpringLiquibase();
        liquibase.setDataSource(dataSource);
        liquibase.setChangeLog("classpath:changelog.xml");

        return liquibase;
    }

    @Bean
    @DependsOn("liquibaseChangelog")
    public SpringLiquibase liquibaseMigration(DataSource dataSource) {
        SpringLiquibase liquibase = new SpringLiquibase();
        liquibase.setDataSource(dataSource);
        liquibase.setChangeLog("classpath:tmp-migration.xml");

        return liquibase;
    }

    @Bean("liquibase")
    @DependsOn("liquibaseMigration")
    public SpringLiquibase liquibaseChangelog2(DataSource dataSource) {
        SpringLiquibase liquibase = new SpringLiquibase();
        liquibase.setDataSource(dataSource);
        liquibase.setChangeLog("classpath:changelog2.xml");

        return liquibase;
    }
}

The changelog.xml will run first. The changeset2 exists in the DATABASECHANGELOG but not in the file, hence it is ignored. Then the tmp-migration.xml runs and changes the FILENAME column. The last will run the changelog2.xml, but Liquibase will treat the changeset2 as already applied.

Some time later (when you believe that all affected databases are already migrated) you might remove the tmp-migration.xml together with it's bean. The changeset will stay in the DATABASECHANGELOG table but that's just a minor thing I believe.

And then the next step could be to move the definition of beans to the contexts of your concrete microservices.

Conclusion

There is always some way ;)

Reducing the maven build execution time

Vladimir Nemergut — Mon, 16 Mar 2020 10:47:09 +0000

Current state

We have a multi-module Spring Web MVC 4 application with about 100k lines of code. We use Maven and Azure DevOps. A simple build pipeline builds and runs all the unit tests - about 2.8k of them. Well, honestly, I would call most of them component tests or even integration tests. Let's clear out some definition at the beginning.

What is a unit test?

There are many definitions of a unit test. I like the one by Roy Osherove:

“A unit test is an automated piece of code that invokes the unit of work being tested, and then checks some assumptions about a single end result of that unit. A unit test is almost always written using a unit testing framework. It can be written easily and runs quickly. It’s trustworthy, readable, and maintainable. It’s consistent in its results as long as production code hasn’t changed.”

There is a nice article about unit tests in Martin Fowler's bliki so I won't go into much details.

Let's say that we expect the following from a unit test:

fast feedback (has to run fast)
short (small amount of code)
tests one thing (one logical assert)
fails for a good reason

What is a component test?

Again a nice article in bliki.

Since our application uses Spring Data JPA, we have lots of repositories. Many of them contain custom query methods. We use Spring Test DBUnit to help us out testing them. The idea is simple - you setup a mockup database (H2), import some test data before a test, run a test, assert and then cleanup the DB end the end. Since this approach creates the DB and some (small) spring context with all necessary beans, I treat these tests as component tests.

These tests run much longer than unit tests.

What is an integration test?

The bliki defines the purpose of integration tests as:

The point of integration testing, as the name suggests, is to test whether many separately developed modules work together as expected.

In our case, we have the the Spring Web MVC controllers. And those controllers expose RESTful API. For those we usually create a full spring context (with mocked H2 database) and use MockMvc to examine them. The beauty of such tests is that you test the API, including all underlying services (business logic) and repositories (persistence). The huge drawback is the execution time. It takes really long time to create Spring context (depends on how large your app is). And in many cases the context is destroyed and created fresh. This all adds up to the total build time.

What tests do I need?

If you are now biased to lean towards just unit tests, as they are fast to run, and want to ditch your component/integration tests as they are slow - hang on ... not so fast. You need The tests at all levels. In a reasonable ratio. In large applications you might split the test runs to fast/slow to speedup the feedback loop. You also need UI driven tests, system tests, performance tests ... Ok, the whole topic about high quality testing of a software is very complex and you might start reading here if you are interested. Let's get back to our topic for now.

The path to improve the situation

There is a nice article written about what you can do. It tells you to re-think whether you need that many integration tests and rewrite them with unit tests. As described above, do it with caution.

It's important to define the test strategy early on. Otherwise the issue will just grow too big to finally conclude that you deal with lots of legacy tests that is expensive to change now. In companies and large projects this happens too often to simply ignore it. Legacy or not, there is still something we can do with reasonable effort and positive return of investment.

Spring context

Due to many reasons our spring context is created in total 17 times during the test run. Every time it takes about 30 seconds to start. It is 8.5 minutes added to the test run. You can make use of the advice from the article above to make sure that the context is only created once. We could save 8 minutes here.

DBUnit

If using DBUnit, each test starts with importing data to DB and ends with cleanup. There are several strategies for this. The Spring Test DBUnit expects these annotations on test method:

@DatabaseSetup(value = "data.xml")
@DatabaseTearDown(value = "data.xml")

The default strategy is CLEAN_INSERT for both Setup and Teardown which in fact deletes all data from affected tables and insert them again. With pure DBUnit, you configure it explicitly, e.g.:

AbstractDatabaseTester databaseTester = new DataSourceDatabaseTester(dataSource);
databaseTester.setSetUpOperation(DatabaseOperation.CLEAN_INSERT);
databaseTester.setTearDownOperation(DatabaseOperation.DELETE_ALL);

Consider changing these strategies to something faster, e.g. instead of CLEAN_INSERT for Setup, use just INSERT. Of course, this will cause problems if your tables are not cleaned up properly by previous tests. The CLEAN_INSERT of Teardown can be replaced with just DELETE_ALL or even faster one TRUNCATE_TABLE. There is one catch with TRUNCATE_TABLE on H2 database, more details in the appendix.

We have about 1200 tests that are running with DBUnit. Just imagine if we saved 100 milliseconds for setup/teardown. It would be 4 minutes saved with relatively low effort.

Build time analysis

However, our situation was a bit more complicated. Our build execution time grew from about 9 minutes to something about 25 minutes in avarage over the past 6 months.

(the sudden peaks are caused by the fact that we recycle the build agents every now and then and hence the maven cache is gone).

Moreover, we experience some builds taking as long as 45 minutes and some of them even more then 1 hour.

We did a simple analysis of the build logs. The tests are configured to output to the console with TRACE level. There we could see that some operations took suddenly much longer than usual. For instance the DB teardown:

2020-03-13T12:59:21.9660309Z INFO  DefaultPrepAndExpectedTestCase - cleanupData: about to clean up 13 tables=[...]
2020-03-13T12:59:29.0545094Z INFO  MockServletContext - Initializing Spring FrameworkServlet ''

The first message is the Teardown of DB from one test, the second message is starting a new spring context for another test. Note the timestamps. 7 seconds to cleanup the DB! Remember we do it 1200 times. After analyzing one build output (the build took 45 minutes) we calculated the total time spent on cleaning up the DB - 1800 seconds. 2/3 of the build spent on this. It surely can't take that long.

Of course we already checked the infrastructure. The build agents are pretty decent t2.large EC2 instances with 2 vCPUs and 8GiB RAM powering Ubuntu. Should be OK.

I like quick and easy solutions for complex problems. We analyzed the build output using simple tools - shell and excel. Let's do it step by step:

extract the timestamps when cleanup DB starts:

cat buildoutput.log | grep "DefaultPrepAndExpectedTestCase - cleanupData" | sed 's/\([0123456789T.:-]*\).*DefaultPrepAndExpectedTestCase.*/\1/g' > cleanup_start.out

(grep for the lines matching the substring, then using sed to only output the timestamp)

extract the timestamps of the next messages - this will roughly be the end time of the cleanups:
```
cat buildoutput.log | grep "DefaultPrepAndExpectedTestCase - cleanupData" -A 1 | grep -v "\-\-" | grep -v "DefaultPrepAndExpectedTestCase" | sed 's/\([0123456789T.:-]*\).*/\1/g' > cleanup_end.out
```
(grep for the lines matching the substring and 1 line after, grep out the first line and the line with '--', then using sed to only output the timestamp)
Load both files to Excel
- choose Data -> From Text/CSV (Alt+A, FT)
- select the first file cleanup_start.out
- Source and Change type steps should be added automatically, add a few more:
- the new Added column contains Time.Hour([Column1])*60*60+Time.Minute([Column1])*60+Time.Second([Column1])
- the Removed columns step just deletes the Column1
- now the excel contains the amount of seconds (incl. fraction) elapsed from the day start
- repeat the same for second file cleanup_end.out
- add diff column
Now the diff column contains roughly the duration of the DB cleanup. I simply put a sum at the end of the column to calculate the 1800 seconds mentioned above.
plot a chart
- select the whole third column
- add a new line chart
The chart now shows how log it took the DB cleanup during the build. You can see important information here. It was all fine and then we see sudden blocks of peaks. The pattern looks very suspicious.

Short stare at the chart, some experience and the 'blink' moment - the Garbage Collector!!! This explains why it takes 7 seconds to cleanup the DB. Because the full GC runs in the background.

Improving the situation

We checked the POM of the affected module and ... guess the surefire plugin configuration:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${surefire.version}</version>
        <configuration>
            <argLine>${surefireArgLine} -Dfile.encoding=UTF-8 -Xmx1024m
            </argLine>
        </configuration>
</plugin>

Yep, the -Xmx1024m. The maximum Java heap space explicitly set to 1024m. This is not enough for out tests any longer. Even though we set the Xmx in the build pipeline, it is not picked up. This explains everything - when the heap space is reaching the maximum, the full GC is invoked to free up some memory. This radically slows down the test execution.

First thing we did - increase the Xmx to at least 2048m (or leave it empty to default to 1/4 of RAM, whichever is smaller).

Since we are here, let me show you one more thing. If your tests are good enough to run in isolation properly, and you have the HW that is powerful enough, play around with this configuration of surefire:

<configuration>
    <forkCount>2</forkCount>
    <parallel>classes</parallel>
    <threadCountClasses>2</threadCountClasses>
</configuration>

This settings means, that there will be 2 forks of surefire booter (this guy actually executes the tests in isolated JVM), it will use 2 threads to run the tests and it will break down the test suite to classes to distribute them over. Important to remember here is that the threadCountClasses is used within the same (forked) JVM, meaning that there will be 2 threads per fork. So if you run into race conditions, just decrease the threadCountClasses to 1. The tests will still run in parallel, but in 1 thread per fork. And because H2 is typically 1 per JVM, you should be fine. Unless you have another kind shared data outside JVM.

The outcome that we reached - down to 11 minutes build time from 25 minutes, which is only 44% of previous duration.

Conclusion

Understand the tools that you use (e.g. MockMvc, DBUnit, Spring Test DBUnit, Surefire plugin). Analyse your build outputs. Do not close the case with just stating that your application grows, your test suite grows, hence the build time gets longer. Sure this is true, but you should always reserve some time for refactoring and improving your code.

Appendix

DBUnit H2 `TRUNCATE_TABLE` operation

Truncate table is usually faster operation than delete from. There are some catches though. Read more about it here. H2 won't allow you to truncate table if there are foreign keys to that table. However, there is a special H2 syntax that you can make use of. We implemented the following DBUnit operation:

class H2TruncateOperation extends org.dbunit.operation.AbstractOperation
{
    private final Logger logger = LoggerFactory.getLogger(H2TruncateOperation.class);

    @Override
    public void execute(IDatabaseConnection connection, IDataSet dataSet)
            throws DatabaseUnitException, SQLException
    {
        logger.debug("execute(connection={}, dataSet={}) - start", connection, dataSet);
        IDataSet databaseDataSet = connection.createDataSet();
        DatabaseConfig databaseConfig = connection.getConfig();
        IStatementFactory statementFactory = (IStatementFactory) databaseConfig
                .getProperty("http://www.dbunit.org/properties/statementFactory");
        IBatchStatement statement = statementFactory.createBatchStatement(connection);

        try
        {
            int count = 0;
            Stack<String> tableNames = new Stack<>();
            Set<String> tablesSeen = new HashSet<>();
            ITableIterator iterator = dataSet.iterator();

            String tableName;
            while(iterator.next())
            {
                tableName = iterator.getTableMetaData().getTableName();
                if(!tablesSeen.contains(tableName))
                {
                    tableNames.push(tableName);
                    tablesSeen.add(tableName);
                }
            }

            if(!tableNames.isEmpty())
            {
                statement.addBatch("SET FOREIGN_KEY_CHECKS=0");
            }

            for(; !tableNames.isEmpty(); ++count)
            {
                tableName = tableNames.pop();
                ITableMetaData databaseMetaData = databaseDataSet.getTableMetaData(tableName);
                tableName = databaseMetaData.getTableName();
                String sql = "TRUNCATE TABLE " +
                             this.getQualifiedName(connection.getSchema(), tableName,
                                                   connection) +
                             " RESTART IDENTITY";
                statement.addBatch(sql);
                if(logger.isDebugEnabled())
                {
                    logger.debug("Added SQL: {}", sql);
                }
            }

            if(count > 0)
            {
                statement.addBatch("SET FOREIGN_KEY_CHECKS=1");
                statement.executeBatch();
                statement.clearBatch();
            }
        }
        finally
        {
            statement.close();
        }
    }
}

An example SQL statements:

SET FOREIGN_KEY_CHECKS=0;
TRUNCATE TABLE table1 RESTART IDENTITY;
TRUNCATE TABLE table2 RESTART IDENTITY;
SET FOREIGN_KEY_CHECKS=1;

You can use pass this operation to plain DB unit configuration like this:

AbstractDatabaseTester databaseTester = new DataSourceDatabaseTester(dataSource);
databaseTester.setTearDownOperation(new H2TruncateOperation());

or implement a new DB Operation lookup if you are using Spring Test DBUnit:

public class H2SpecificDatabaseOperationLookup extends DefaultDatabaseOperationLookup
{
    @Override
    public org.dbunit.operation.DatabaseOperation get(DatabaseOperation operation)
    {
        return operation == DatabaseOperation.TRUNCATE_TABLE ?
               new H2TruncateOperation() :
               super.get(operation);
    }
}

and use annotations:

@DbUnitConfiguration(databaseOperationLookup = H2SpecificDatabaseOperationLookup.class)

Zorin OS for a web developer? Yes please!

Vladimir Nemergut — Fri, 10 Jan 2020 16:41:12 +0000

Background

I've been a developer for many years. My beginnings were on Windows 95. Since then my primary development environment has been Windows OS based.

Some time back I had an opportunity to switch my work web development environment to MacOS. MacOS is great for that. Switching back to Windows was not necessarilly any comparable experience 😄

So what about trying something else, shall we?

In this chapter you get some introduction and I share my feelings and struggles.

Zorin OS?

When searching over a dozen of modern Linux distributions, I suddenly found Zorin OS hidden behind Ubuntu, Debian, Mint (from a popularity perspective) and elementary OS (from UX perspective). It's great looking desktop OS. Very visually appealing. There are a few editions you might choose from:

Ultimate
Core
Lite
Education and Education Lite

All but Ultimate are free of charge.

I've got a spare Fujitsu Celsius H710 laptop, so I decided to install Core edition on it.

Installation

I don't want to go through the whole installation procedure. You can find many tutorials on internet. After burning the installation DVD of Zorin OS 15.1, I just followed the installation steps. It is super easy (if you manage to boot from DVD).

Struggle with NVidia drivers

The first installation went through, all fine. Then I wanted to tweak the OS a little bit, that was a mistake 😄. The H710 has hybrid graphics. So actually 2 graphics adapters - Intel and NVidia. I tried to install the additional NVidia drivers from the Software & Updates

Neither of the NVidia drivers worked perfectly. With the proprietary driver you get this annoying NVidia logo while booting and the booting seemed to take longer. Both proprietary and open source NVidia drivers caused the brightness adjustment to stop working. I mean the functional key still work, but the brightness always stays on 100%. Switching back to Nouveau drivers was a nightmare, too. The screen resolution went to 640x480 and no way to change it properly.

If you now get a cool idea to switch to Intel graphics adapter ...



sudo prime-select intel

... don't do this 😏. At least I was not able to make it work with Intel adapter. Mind you, even reinstall won't help as you need to switch back to NVidia.

The only remedy that worked for me was:

switch to NVidia ```

sudo prime-select nvidia

- uninstall NVidia drivers completely:

sudo apt purge nvidia-*


After that Zorin OS experience was back to the top level:
- Zorin OS logo when booting in high-res
- resolution back to screen native

# Laptop power management

Power management is surely important if you want to use your laptop on battery. The most advanced tool is the [TLP](https://linrunner.de/en/tlp/tlp.html). 

Just install TLP on your laptop:

sudo add-apt-repository ppa:linrunner/tlp
sudo apt update
sudo apt install tlp tlp-rdw


You might find TLP GUI handy:
``
sudo add-apt-repository ppa:linuxuprising/apps
sudo apt update
sudo apt install tlpui
```

![TLP GUI](https://thepracticaldev.s3.amazonaws.com/i/sjfa7zk099tx0kqocg2u.png)

Then go to Software app and search for TLP Extension. This will give you an easy way to switch between power management profiles without opening a terminal window and typing in something.

![TLP Extension](https://thepracticaldev.s3.amazonaws.com/i/bo9v0905fnxxh4bxy84n.png)

The TLP installs the default TLP profile under `/etc/defaults/tlp`. You might follow these steps to create your custom profiles.

- copy the `/etc/defaults/tlp` to `~/.tlp/<Your profile name>` as many times as many profiles you want
- open the respective profile file in TLP GUI
- make all modifications to your liking and save it

# What next?

In the next chapters I'd share my impressions of using Zorin OS as web developer.

DEV Community: Vladimir Nemergut

Splitting Liquibase changelog? No problem.

What is Liquibase?

How Liquibase works?

The changelog file grows over time

Define multiple changelog files

Configure multiple Liquibase runs

How to deal with long changelog?

Change the logicalFilePath

Use intermediate changelog

Conclusion

Reducing the maven build execution time

Current state

What is a unit test?

What is a component test?

What is an integration test?

What tests do I need?

The path to improve the situation

Spring context

DBUnit

Build time analysis

Improving the situation

Conclusion

Appendix

DBUnit H2 TRUNCATE_TABLE operation

Zorin OS for a web developer? Yes please!

Background

Zorin OS?

Installation

Struggle with NVidia drivers

Change the `logicalFilePath`

DBUnit H2 `TRUNCATE_TABLE` operation