DEV Community: Base Blocks tech

How-to #2. Audit trail using envers

Oleksii Kondratiuk — Sun, 27 Sep 2020 10:24:04 +0000

Motivation

Consider case when changes that user or some system makes to the data in the database need to be tracked. Sometimes it is the requirement from the officials to have ability to get history of this changes, sometimes we just want to have the view on what happened in the system for error processing/debug/monitoring purposes or it might be we just want to have historical data about how, for example, height of a kid changed. All this cases can be covered with pretty simple solution - logging of the latest state of the data at the moment of changes. It is called audit trail.

Requirements

Changes in tables are tracked automatically whenever we call save, update or delete operations on our repository level;
Ability to easily query audit trail.

Tech solution

We are going to work with our project User service which was used for previous article in the series. Our goal to have audit trail of all the changes of the user data.

Spring Data has the mechanism called Spring Data Envers which allow access to entity revisions managed by Hibernate Envers. It will automatically write entities snapshots of the User entity on the update, save and delete operation into separate table(user_history in our case). Let's have a look how it can be implemented.

Respectful dependency should be added to enable it in our project:
compile group: 'org.springframework.data', name: 'spring-data-envers'

Repository factory bean should be changed. It can be done by adding EnableJpaRepositories(or modifying if you already have it) on the Application class(class marked by @Configuration) with the EnversRevisionRepositoryFactoryBean as factory bean class:
@EnableJpaRepositories(repositoryFactoryBeanClass = EnversRevisionRepositoryFactoryBean.class)

Audited annotation should be added to enable audit trail of the particular entity:
@Entity @Data @Table(name = "bb_user") @EqualsAndHashCode(onlyExplicitlyIncluded = true) @Audited public class User { ...

Repository should extend one more interface - RevisionRepository. It enables repo to access revisions of the entity when you will want to get all the data revisions or some particular one:
public interface UserRepository extends RevisionRepository<User,Long, Integer>, CrudRepository<User, Long> { ...

There is no need to define user_history entity. You should just specify the suffix that Spring Data will add automatically add to the name of the entity and will use as a name of the table to which audit trail will be written. It is configurable in application.yaml:
spring.jpa.properties.org.hibernate.envers.audit_table_suffix: _history

In our case user_history is the table that will be used for audit trail. Let's configure Spring JPA to create tables on the application startup(set spring.jpa.generate-ddl to true and spring.jpa.hibernate.ddl-auto to create), start the application and have a look at this table. Now we have user and usera_history tables in the database. user_history table has two additional columns in comparison to the user table - rev and revtype. Revision is store in the rev column. revtype defines the type of the operation that was performed on the entity:

0 - ADD - A database table row was inserted;
1 - MOD - A database table row was updated;
2 - DEL - A database table row was deleted.

Moving to the test. First user needs to be created. In this case new records will be created in the user database with revtype 0 and first revision. revision is one per table, not per entity, this is important. So if I insert next user then the last revision (let`s say 2) will be taken and incremented by 1(3). Then if i insert next user then its revision will be 4. And so on.
User record will be updated, but there will be new record in user_history table with next revision in case of update operation on the existing user.

Conclusion

Spring Data Envers is a very convenient mechanism for the audit trail. It supports more complicated mappings as well. For instance, Audited annotation can be put not on the class level, but on some specific field and only that field will be audited. I encourage you to check its Javadoc and check Hibernate doc to get more insights.

How-to #1. Data encryption.

Oleksii Kondratiuk — Sun, 06 Sep 2020 16:14:32 +0000

Preface

Pretty much every developer faces a lot of challenges along his career. Especially if you work on a lot of different projects and assigned a variety of diverse tasks. Of course, most of them are pretty obvious and requires knowledge that you already have. But some amount of them requires an understanding of the specifics. That is why I start a series of how-to blog posts. I want to share my practical experience in frameworks, databases, servers, clouds, etc. I hope this will help someone get a better knowledge of them, spend less time wondering how to do this or that, and be more productive in general.

Data encryption

Each and every user wants their data to be in safe. No matter if it is his progress in learning something in some online system or his money in online banking. We all want this data to be protected. That is also why GDPR and other regulations appeared. I will cover how to make sensitive data encrypted in the SQL database if you use Java and such technologies like Spring and Hibernate.

Project overview

It is a practical guide, so let`s build some small project which we will utilize to master this topic. Assume you build a microservice that is responsible to store user data. Not authenticate, but just storing whatever is specific to the user.

Motivation

Birthdate, country of origin, address, etc. All this is sensitive data that is not subject to sharing to third parties and even to the employees of the company where you work. Obviously you should restrict access to production servers and databases, make sure that your APIs do not allow you to get data of some other person, but these measures restrict access to the data. Data itself is still vulnerable - if someone gets access to the database then we are in trouble. Let`s see how we can fix this.

Requirements

Data should be stored encrypted in the database.
Encryption should happen when data stored in the database.
Decryption should happen each time we get data from the database.

Tech solution

There are different ways how to implement this. We will focus on one specific now. Hibernate converters and Java encryption capabilities are going to be used for the solution.

Let`s start with our model:

@Entity
@Data
@Table(name = "bb_user")
@EqualsAndHashCode(onlyExplicitlyIncluded = true)
public class User {

    @Id
    @GeneratedValue
    @EqualsAndHashCode.Include
    private Long id;

    @Convert(converter = StringAttributeConverter.class)
    private String name;
    private String token;
}

As you most probably noticed, there is Convert annotation from javax.persistence package. Its role here to trigger StringAttributeConverter exactly at the moments when we need it. Here its source code:

@Component
public class StringAttributeConverter implements 
AttributeConverter<String, String> {
    @Autowired
    private Encryptor encryptor;

    @Override
    public String convertToDatabaseColumn(String attribute) {
        return encryptor.encrypt(attribute);
    }

    @Override
    public String convertToEntityAttribute(String dbData) {
         return encryptor.decrypt(dbData);
    }
}

convertToDatabaseColumn method is triggered when the entity is saved to the database or an existing record is updated. In our case encryptor is triggered to encrypt data. The field value for which the method is executed(name in our case) will be substituted with the value produced by this method before the flush of the transaction.
convertToEntityAttribute method obviously is triggered at the moment when the entity is obtained from the database. The result of method execution is decrypted name.
The repository for this entity is a simple implementor of the Spring Data`s CrudRepository, no additional implementation needed on the repository level:

public interface UserRepository extends CrudRepository<User, Long> {

    Optional<User> findByToken(String token);
}

Encryptor interface is created in case if we will want to change encryption implementation:

public interface Encryptor {

    String encrypt(String plainText);

    String decrypt(String encryptedString);
}

Standard Java encryption capabilities are used with AES encryption for the implementer:

@Component
public class AesEncryptor implements Encryptor {
    private static final Logger LOGGER = 
LoggerFactory.getLogger(AesEncryptor.class);
    private static final String TRANSFORMATION = 
"AES/ECB/PKCS5Padding";
    private static final String UTF_8 = "UTF-8";

    @Autowired
    private SecretKeySpec secretKey;

    @Override
    public String encrypt(String plainText) {
        LOGGER.info("Encrypted field {}", plainText);

        try {
            Cipher cipher = 
Cipher.getInstance(TRANSFORMATION);
            cipher.init(Cipher.ENCRYPT_MODE, secretKey);
            Base64.Encoder encoder = Base64.getEncoder();
            byte[] stringBytes = plainText.getBytes(UTF_8);
            return 
 encoder.encodeToString(cipher.doFinal(stringBytes));
        } catch (Exception e) {
            throw new RuntimeException("Could not handle the encryption");
        }
    }

    @Override
    public String decrypt(String encryptedString) {
        LOGGER.info("Decrypted field {}", encryptedString);
        try {
            Cipher cipher = 
Cipher.getInstance(TRANSFORMATION);
            cipher.init(Cipher.DECRYPT_MODE, secretKey);
            Base64.Decoder decoder = Base64.getDecoder();
            return new 
String(cipher.doFinal(decoder.decode(encryptedString)));
        } catch (Exception e) {
            throw new RuntimeException("Could not handle the decryption", e); 
        }
    }
}

And the last piece here is the secret key that is used for the encryption/decryption process. Here is the configuration class which includes secret key bean creation:

@Configuration
public class ApplicationConfiguration {

    @Value("${encryption.key}")
    private String encryptionKey;

    @Bean
    public SecretKeySpec secretKey() throws 
UnsupportedEncodingException, NoSuchAlgorithmException {
        byte[] keyBytes = encryptionKey.getBytes("UTF-8");
        MessageDigest sha = MessageDigest.getInstance("SHA-1");
        keyBytes = sha.digest(keyBytes);
        keyBytes = Arrays.copyOf(keyBytes, 16);
        return new SecretKeySpec(keyBytes, "AES");
    }
}

The full source code is available in GitHub repo