1. Introduction
Caching data is a very interested topic along with performance of system. There are very strategies to implement caching that you can hear at somewhere: write-through, write-back, read-through, cache-aside,... Today, we will create the simplest example for lazy-load also known as a cache-aside strategy by Spring Boot, MongoDB and Redis.
Redis is an open-source, in-memory data structure store, often used as a database, cache, and message broker. Redis is regularly used in web applications to cache frequently accessed data and reduce the number of database queries, thereby improving performance.
MongoDB is a popular, open-source NoSQL document-oriented database that is designed to store and manage unstructured data. Instead of using tables and rows like traditional relational databases, MongoDB stores data as JSON-like documents with dynamic schemas, which means that each document can have its own unique structure and fields.
In this example, we not use self-managed for Redis. Instead of, we will use AWS ElastiCache Redis. This is a service that AWS provide that can help us reduce much time to set up Redis server on local machine.
2. Implementation
2.1. Create AWS resources
We need to create Redis cluster. You can refer this below link for step-by-step guide: https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/GettingStarted.CreateCluster.html#Clusters.Create.CON.Redis-gs
After creating resource, we have one Redis cluster. Information of resources like this
2.2. Create Spring Boot project
In this demo, we will create APIs to interact with a local MongoDB instance and Redis remote server hosted in AWS. You can use the Spring Start Project in Eclipse or Spring Initializer to create a Spring Boot project. These are required maven dependencies that you must add to POM file in our example.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
2.3. Create API to interact with MongoDB & Redis
To work with AWS Elasticache Redis server, we need to add some configurations to property file
spring.data.redis.host=<redis-primary-endpoint>
spring.data.redis.port=<redis-port>
You must replace the placeholder by your Redis primary endpoint and port, default port normally is 6379. After adding configurations, we create a Bean to interact with Redis server by Redis template.
@Bean
public LettuceConnectionFactory redisConnectionFactory() {
return new LettuceConnectionFactory();
}
@Bean
RedisTemplate < String, Object > redisTemplate(RedisConnectionFactory redisConnectionFactory) {
RedisTemplate < String, Object > template = new RedisTemplate < > ();
template.setConnectionFactory(redisConnectionFactory);
return template;
}
Now, creating an entity named is Product and this entity has some attributes like this
@Document("product")
@Data
public class Product {
private String id;
private String name;
private String category;
private int quantity;
}
And, we create a data access object that directly work with database and cache from application.
// function work with MongoDB
public Product addProduct(Product p) {
return mongoTemplate.save(p);
}
public Product getProductById(String id) {
Class<?> productClass = Product.class;
Product product = (Product) mongoTemplate.findById(id, productClass);
log.info("Got product from Database {}", product);
return product;
}
// function work with Redis
public ValueOperations < String, Object > addProductToRedis(Product p) {
try {
ValueOperations < String, Object > cachedProduct = redisTemplate.opsForValue();
objectMapper = getObjectMapper();
Map < ? , ? > map = objectMapper.convertValue(p, Map.class);
cachedProduct.set(p.getId(), map, 10, TimeUnit.SECONDS);
log.info("Add product to cache {}", map);
return cachedProduct;
} catch (RedisConnectionFailureException e) {
log.info("Cannot add product, Redis service go down...");
}
return null;
}
public Product getProductFromRedis(String key) {
ValueOperations < String, Object > cachedProduct = redisTemplate.opsForValue();
try {
Object result = cachedProduct.getAndExpire(key, 10, TimeUnit.SECONDS);
objectMapper = getObjectMapper();
Product product = objectMapper.convertValue(result, Product.class);
log.info("Got product from cache {}", product);
return product;
} catch (RedisConnectionFailureException e) {
log.info("Cannot get product, Redis service go down...");
}
return null;
}
Then, creating a service to invoke functions in DAO layer
public Product addProduct(Product p) {
return productDao.addProduct(p);
}
public Product getProductById(String id) {
Product product = productDao.getProductFromRedis(id);
if (product == null) {
product = productDao.getProductById(id);
productDao.addProductToRedis(product);
}
return product;
}
The getProductById function has logic that follow with the strategy like this
- When Spring App needs to read data from MongoDB, it checks the cache in Redis first to determine whether the data is available.
- If the data is available (a cache hit), the cached data is returned, and the response is issued to the caller.
- If the data isn’t available (a cache miss), the database is queried for the data. The cache is then populated with the data that is retrieved from the database, and the data is returned to the caller.
Moreover, because we don't want to store data that not be frequently access, store infrequently accessed data is waste. So, we will add a Time To Live (TTL) for each data which be initially cached by 10 seconds. And for every time this data be accessed, we refresh the TTL value. In the above code, you can see we already used getAndExpire method to accomplish this idea.
Finally, we create some endpoints to test our functions
@PostMapping("/products")
public Product addProduct(@RequestBody Product p) {
return productService.addProduct(p);
}
@GetMapping("/products/{id}")
public Product getProduct(@PathVariable String id) {
return productService.getProductById(id);
}
2.4. Testing API
For testing purpose, we use Postman to create a product
Then, getting the product by ID and view the log in Eclipse console
Because this is the first time to get product, you can see Redis return null value and database was being queried to get product. Afterward, product is added to Redis. We also see the time that API respond is 731ms. Now, we test to see how much time to take response with Redis
Only 8ms and by the log we can confirm product was got from cache instead of database.
3. Summary
We already implemented an example for Lazy-loading also known as Cache-Aside strategy caching using AWS Elasticache Redis. We can see the performance was significantly increased by caching. But it just is the simplest example to give a viewing for implement caching approach. To use it in the real world, for a large system, we must handle many situations that can occur like Redis goes down, memory fragmentation issues, TTL management,...
The implementation of all these examples can be found in my GitHub
References: https://docs.aws.amazon.com/whitepapers/latest/database-caching-strategies-using-redis/caching-patterns.html
Happy Coding :)
Top comments (0)