DEV Community: Yogesh Mali

Upload File to S3 Using NestJS Application

Yogesh Mali — Sun, 07 Jan 2024 04:30:20 +0000

Introduction

In this post, I will show how to upload a file to S3 using a NestJS application and Multer. S3 is Amazon's Simple Storage Service (S3). In old systems, one could upload files to a database. However, with storage services, it is easier to upload and retrieve files. This is also more performant.

If you are a beginner with the NestJS framework, I would recommend reading their documentation. I have written some posts about NestJS previously here, here, and here.

Simple Storage Service

Simple storage service is also known as S3. Irrespective of what kind of application you build, you have to store static files somewhere. AWS offers a simple and effective service called S3. As previously mentioned in the post, application developers used to save files in the database, but that is not very performant.

With S3, now you can store the file in a storage service and create a file link that you can save in the database.

To understand S3 is to understand a hash table. Usually, when you store any file on S3, the service generates a random string as a key to identify that file. The file is the blob format data.

S3 is targeted towards application builders to individual users. The advantages of S3 are scalability, high availability, performance, and security. S3 can also be used for redundancy. One thing to remember is that you can not use S3 to host a static website, especially if you want to use it with HTTPS.

Setting up an S3 bucket

For this demo, let's set up an S3 bucket in the AWS S3 service. I would assume that you have an AWS account with enough permissions to manage resources like creating or deleting buckets. If you don't, you can always create an account for AWS with a free tier. Over the years, AWS has done a great job to help developers play with their services and improve them.

Once you have your user, make sure to download the AWS Access Key and AWS Secret Key. We will need these keys to call AWS services programmatically.

Let's create a bucket in S3.

NestJS Application

Let's create a nestjs application. If you do not have nest-cli downloaded, I would recommend use npm i @nestjs/cli.

1. Set up Application

Create a new application

nest new fileuploaddemo

Once we have the facade of the application, we will create an API, service, and a database table. If you are wondering, why a database table, then to answer that - We will use a database to store file metadata information including the link and when it was created.

Just like previous articles, we will use prisma as our ORM for creating this nestjs application.

Install Prisma dependencies

npm install prisma --save-dev

npm install @prisma/client

Create initial Prisma set up

npx prisma init

If you are using Windows environment, you might run into an error while running above command. So set environment variables

set PRISMA_CLI_QUERY_ENGINE_TYPE=binary

set PRISMA_CLIENT_ENGINE_TYPE=binary

Add the database table details in schema.prisma file

// This is your Prisma schema file,
// learn more about it in the docs: https://pris.ly/d/prisma-schema

generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "mysql"
  url      = env("DATABASE_URL")
}

model FileEntity {
  id Int @default(autoincrement()) @id
  fileName String
  fileUrl  String 
  key      String
  createdAt   DateTime @default(now())
  updatedAt   DateTime @updatedAt
}

And make sure to set DATABASE_URLas the environment variable before running Prisma migrations.

Run prisma migrations

npm prisma migrate dev

2. Set up environment

We will need a few environment variables to make sure our application is functioning.

So create .env file in your application and add the following variables.

AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=accessKey
AWS_SECRET_ACCESS_KEY=secretKey
AWS_BUCKET_NAME='nestjsfileuploaddemo'
DATABASE_URL="mysql://user:password@localhost:3306/fileuploaddemo?schema=public"

3. Upload API

Usually, a client application will send us file details through form-data. For this demo, we will create an API that the client application can call. This will be our upload API.

import { Controller, Post, UploadedFile, UseInterceptors } from "@nestjs/common";
import { FileInterceptor } from "@nestjs/platform-express";
import { FileUploadService } from "src/services/fileupload.service";
import { Express } from 'express';


@Controller('/v1/api/fileUpload')
export class FileController {
    constructor(private fileUploadService: FileUploadService) {}

    @Post()
    @UseInterceptors(FileInterceptor('file'))
    async uploadFile(@UploadedFile() file: Express.Multer.File): Promise {
        const uploadedFile = await this.fileUploadService.uploadFile(file.buffer, file.originalname);
        console.log('File has been uploaded,', uploadedFile.fileName);        
    }

}

In this API, we are using FileInterceptor from NestJS which extracts file details from the request.

The API is straightforward. We pass the file metadata information to FileUploadService and that does the job of uploading the file to S3 as well as saving the data in the database.

4. FileInterceptor for File Upload

An interceptor is one of the fundamental concepts in NestJS. Interceptor is a class with annotation @Injectable() and implements NestInterceptor interface.

5. Create a FileUploadService

FileUploadService will perform two tasks. One is to upload the file to S3 and the second is to store the metadata in the database.

import { Injectable } from "@nestjs/common";
import { ConfigService } from "@nestjs/config";
import { FileEntity, Prisma } from "@prisma/client";
import { S3 } from "aws-sdk";
import { PrismaService } from "src/common/prisma.service";
import { v4 as uuid } from 'uuid';

@Injectable()
export class FileUploadService {
    constructor(private prismaService: PrismaService,
        private readonly configService: ConfigService,){}

    async uploadFile(dataBuffer: Buffer, fileName: string): Promise {
        const s3 = new S3();
        const uploadResult = await s3.upload({
            Bucket: this.configService.get('AWS_BUCKET_NAME'),
            Body: dataBuffer,
            Key: `${uuid()}-${fileName}`,
        }).promise();

        const fileStorageInDB = ({
            fileName: fileName,
            fileUrl: uploadResult.Location,
            key: uploadResult.Key,
        });

        const filestored = await this.prismaService.fileEntity.create({
            data: fileStorageInDB
        });

        return filestored;
    }
}

We have a method called uploadFile in this service class. It takes two parameters one for a data buffer and the other for a file name.

We create an S3 instance using S3() and use that to upload files to S3. It needs a bucket name, a data buffer in the body, and a unique key for storing the file.

We use the location as fileUrl from the uploaded file. And then save the metadata information in the database.

6. Final Demo

Let's run our application to see how it is working now.

npm run start - will start the application at default port 3000.

We will use postman to call our upload API.

Once we upload the file, we will see a console message of file uploaded.

And similarly, a database entry

Just to make sure that we have our files uploaded in S3, let's look at the bucket.

You will see two entries in the bucket.

Conclusion

In this post, we discussed how we can upload a file to AWS Simple Storage Service (S3) using NestJS application. We only covered the surface of this vast topic of S3. You can also manage to upload private files, encrypt, and decrypt those files. You can also subscribe to events of file upload. There are a lot of options when it comes to AWS S3. We will cover more in the upcoming series of NestJS applications.

How to use Circuit Breaker in a Spring Boot Application

Yogesh Mali — Sat, 27 Feb 2021 06:31:12 +0000

In this post, I will show how we can use the Circuit Breaker pattern in a Spring Boot Application. When I say Circuit Breaker pattern, it is an architectural pattern. Netflix had published a library Hysterix for handling circuit breakers. As part of this post, I will show how we can use a circuit breaker pattern using the resilence4j library in a Spring Boot Application.

In other news, I recently released my book Simplifying Spring Security. If you are interested to learn about Spring Security, you can buy it here.

What is Circuit Breaker?

The concept of Circuit Breaker comes from Electrical Engineering. In most electricity networks, circuit breakers are switches that protect the network from damage caused by an overload of current or short circuit.

Similarly, in software, a circuit breaker stops the call to a remote service if we know the call to that remote service is either going to fail or time out. The advantage of this is to save resources and being proactive in our troubleshooting of the remote procedure calls.

The circuit breaker makes the decision of stopping the call based on the previous history of the calls. But there are alternative ways how it can handle the calls. Usually, it will keep track of previous calls. Suppose 4 out of 5 calls have failed or timed out, then the next call will fail. This helps to be more proactive in handling the errors with calling service and caller service can handle the response in a different way, allowing users to experience the application differently than an error page.

Another way circuit breaker can act if calls to remote service are failing in a particular time duration. A circuit breaker will open and will not allow the next call till remote service improves on error.

Resilience4J Library

We have our code where we call remote service. The circuit breaker module from resilience4j library will have a lambda expression for a call to remote service OR a supplier to retrieve values from the remote service call. I will show this as part of the example. The circuit breaker decorates this remote service call in such a way so it can keep track of responses and switch states.

Different configurations of Resilience4j Library

To understand the circuit breaker concept, we will look at different configurations this library offers.

slidingWindowType() – This configuration basically helps in making a decision on how the circuit breaker will operate. There are two types COUNT_BASED and TIME_BASED. COUNT_BASED circuit breaker sliding window will take into account the number of calls to remote service while TIME_BASED circuit breaker sliding window will take into account the calls to remote service in a certain time duration.

failureRateThreshold() – This configures the failure rate threshold in percentage. If x percentage of calls are failing, then the circuit breaker will open.

slidingWindowSize() – This setting helps in deciding the number of calls to take into account when closing a circuit breaker.

slowCallRateThreshold() – This configures the slow call rate threshold in percentage. If x percentage of calls are slow, then the circuit breaker will open.

slowCallDurationThreshold – Time duration threshold about which calls are considered slow.

minimumNumberOfCalls() – Minimum number of calls required before which circuit breaker can calculate the error rate.

ignoreException() – This setting allows you to configure an exception that a circuit breaker can ignore and will not count towards success or failure of call of remote service.

waitDurationInOpenState() – Duration for which the circuit breaker should remain in the open state before transitioning into a half-open state. The default value is 60 seconds.

Count-Based Circuit Breaker

While using resilience4j library, one can always use default configurations that the circuit breaker offers. Default configurations are based on the COUNT-BASED sliding window type.

So how do we create a circuit breaker for COUNT-BASED sliding window type?

CircuitBreakerConfig circuitBreakerConfig = CircuitBreakerConfig.custom()               .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.COUNT_BASED).slidingWindowSize(10)
    .slowCallRateThreshold(65.0f)
    .slowCallDurationThreshold(Duration.ofSeconds(3))
    .build();

CircuitBreakerRegistry circuitBreakerRegistry =
                CircuitBreakerRegistry.of(circuitBreakerConfig);

CircuitBreaker cb = circuitBreakerRegistry.circuitBreaker("BooksSearchServiceBasedOnCount");

In the above example, we are creating a circuit breaker configuration that includes sliding window of type COUNT_BASED. This circuit breaker will record the outcome of 10 calls to switch circuit breaker to the closed state. If 65 percent of calls are slow with slow being of duration of more than 3 seconds, the circuit breaker will open.

CircuitBreakerRegistry is a factory to create a circuit breaker.

Time-Based Circuit Breaker

Now on Time-Based circuit breaker.

CircuitBreakerConfig circuitBreakerConfig = CircuitBreakerConfig.custom()                .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.TIME_BASED)             .minimumNumberOfCalls(3)
                .slidingWindowSize(10)
                .failureRateThreshold(70.0f)
                .build();

CircuitBreakerRegistry circuitBreakerRegistry =
                CircuitBreakerRegistry.of(circuitBreakerConfig);

CircuitBreaker cb = CircuitBreakerRegistry.circuitBreaker("BookSearchServiceBasedOnTime");

In the above example, we are creating a circuit breaker configuration that includes a sliding window of type TIME_BASED. Circuit breaker will record the failure of calls after a minimum of 3 calls. If 70 percent of calls fail, the circuit breaker will open.

Example of Circuit Breaker in Spring Boot Application

We have covered the required concepts about the circuit breaker. Now, I will show we can use a circuit breaker in a Spring Boot application.

On one side, we have a REST application BooksApplication that basically stores details of library books. On the other side, we have an application Circuitbreakerdemo that calls the REST application using RestTemplate. We will decorate our REST call through the circuit breaker.

BooksApplication stores information about books in a MySQL database table librarybooks. The REST Controller for this application has GET and POST methods.


package com.betterjavacode.books.controllers;

import com.betterjavacode.books.daos.BookDao;
import com.betterjavacode.books.models.Book;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import java.util.ArrayList;
import java.util.List;
import java.util.Optional;

@CrossOrigin("https://localhost:8443")
@RestController
@RequestMapping("/v1/library")
public class BookController
{
    @Autowired
    BookDao bookDao;

    @GetMapping("/books")
    public ResponseEntity<List> getAllBooks(@RequestParam(required = false) String bookTitle)
    {
        try
        {
            List listOfBooks = new ArrayList<>();
            if(bookTitle == null || bookTitle.isEmpty())
            {
                bookDao.findAll().forEach(listOfBooks::add);
            }
            else
            {
                bookDao.findByTitleContaining(bookTitle).forEach(listOfBooks::add);
            }

            if(listOfBooks.isEmpty())
            {
                return new ResponseEntity<>(HttpStatus.NO_CONTENT);
            }

            return new ResponseEntity<>(listOfBooks, HttpStatus.OK);
        }
        catch (Exception e)
        {
            return new ResponseEntity<>(null, HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @GetMapping("/books/{id}")
    public ResponseEntity getBookById(@PathVariable("id") long id)
    {
        try
        {
            Optional bookOptional = bookDao.findById(id);

            return new ResponseEntity<>(bookOptional.get(), HttpStatus.OK);
        }
        catch (Exception e)
        {
            return new ResponseEntity<>(null, HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @PostMapping("/books")
    public ResponseEntity addABookToLibrary(@RequestBody Book book)
    {
        try
        {
            Book createdBook = bookDao.save(new Book(book.getTitle(), book.getAuthor(),
                    book.getIsbn()));
            return new ResponseEntity<>(createdBook, HttpStatus.CREATED);
        }
        catch (Exception e)
        {
            return new ResponseEntity<>(null, HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @PutMapping("/books/{id}")
    public ResponseEntity updateABook(@PathVariable("id") long id, @RequestBody Book book)
    {
        Optional bookOptional = bookDao.findById(id);

        if(bookOptional.isPresent())
        {
            Book updatedBook = bookOptional.get();
            updatedBook.setTitle(book.getTitle());
            updatedBook.setAuthor(book.getAuthor());
            updatedBook.setIsbn(book.getIsbn());
            return new ResponseEntity<>(bookDao.save(updatedBook), HttpStatus.OK);
        }
        else
        {
            return new ResponseEntity<>(HttpStatus.NOT_FOUND);
        }
    }

    @DeleteMapping("/books/{id}")
    public ResponseEntity deleteABook(@PathVariable("id") long id)
    {
        try
        {
            bookDao.deleteById(id);
            return new ResponseEntity<>(HttpStatus.NO_CONTENT);
        }
        catch (Exception e)
        {
            return new ResponseEntity<>(HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }
}

On the other side, our application Circuitbreakerdemo has a controller with thymeleaf template so a user can access the application in a browser.

For the demo purpose, I have defined CircuitBreaker in a separate bean that I will use in my service class.

    @Bean
    public CircuitBreaker countCircuitBreaker()
    {
        CircuitBreakerConfig circuitBreakerConfig = CircuitBreakerConfig.custom()
                .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.COUNT_BASED)
                .slidingWindowSize(10)
                .slowCallRateThreshold(65.0f)
                .slowCallDurationThreshold(Duration.ofSeconds(3))
                .build();

        CircuitBreakerRegistry circuitBreakerRegistry =
                CircuitBreakerRegistry.of(circuitBreakerConfig);

        CircuitBreaker cb = circuitBreakerRegistry.circuitBreaker("BooksSearchServiceBasedOnCount");

        return cb;
    }

    @Bean
    public CircuitBreaker timeCircuitBreaker()
    {
        CircuitBreakerConfig circuitBreakerConfig = CircuitBreakerConfig.custom()
                .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.TIME_BASED)
                .minimumNumberOfCalls(3)
                .slidingWindowSize(10)
                .failureRateThreshold(70.0f)
                .build();

        CircuitBreakerRegistry circuitBreakerRegistry =
                CircuitBreakerRegistry.of(circuitBreakerConfig);

        CircuitBreaker cb = circuitBreakerRegistry.circuitBreaker("BookSearchServiceBasedOnTime");
        return cb;
    }

I have defined two beans one for the count-based circuit breaker and another one for time-based.

The BookStoreService will contain a calling BooksApplication and showing books that are available. This service will look like below:

@Controller
public class BookStoreService
{

    private static final Logger LOGGER = LoggerFactory.getLogger(BookStoreService.class);

    @Autowired
    public BookManager bookManager;

    @Autowired
    private CircuitBreaker countCircuitBreaker;

    @RequestMapping(value = "/home", method= RequestMethod.GET)
    public String home(HttpServletRequest request, Model model)
    {
        return "home";
    }

    @RequestMapping(value = "/books", method=RequestMethod.GET)
    public String books(HttpServletRequest request, Model model)
    {
        Supplier<List> booksSupplier =
                countCircuitBreaker.decorateSupplier(() -> bookManager.getAllBooksFromLibrary());

        LOGGER.info("Going to start calling the REST service with Circuit Breaker");
        List books = null;
        for(int i = 0; i < 15; i++)
        {
            try
            {
                LOGGER.info("Retrieving books from returned supplier");
                books = booksSupplier.get();
            }
            catch(Exception e)
            {
                LOGGER.error("Could not retrieve books from supplier", e);
            }
        }
        model.addAttribute("books", books);

        return "books";
    }
}

So when the user clicks on the books page, we retrieve books from our BooksApplication REST Service.

I have autowired the bean for countCircuitBreaker. For demo purposes – I will be calling the REST service 15 times in a loop to get all the books. This way, I can simulate interruption on my REST service side.

Our circuit breaker decorates a supplier that does REST call to remote service and the supplier stores the result of our remote service call.

In this demo, we are calling our REST service in a sequential manner, but remote service calls can happen parallelly also. The circuit breaker will still keep track of results irrespective of sequential or parallel calls.

Demo

Let’s look at how the circuit breaker will function in a live demo now. My REST service is running on port 8443 and my Circuitbreakerdemo application is running on port 8743.

Initially, I start both of the applications and access the home page of Circuitbreakerdemo application. The home page contains the link for viewing all the books from the store.

Now to simulate some errors, I have added the following code in my RestTemplate call that basically sleeps for 3 seconds before returning the result of the REST call.

    public List getAllBooksFromLibrary ()
    {
        HttpHeaders httpHeaders = new HttpHeaders();
        httpHeaders.setContentType(MediaType.APPLICATION_JSON);

        ResponseEntity<List> responseEntity;
        long startTime = System.currentTimeMillis();
        LOGGER.info("Start time = {}", startTime);
        try
        {
            responseEntity= restTemplate.exchange(buildUrl(),
                    HttpMethod.GET, null, new ParameterizedTypeReference<List>()
                    {});
            if(responseEntity != null && responseEntity.hasBody())
            {
                Thread.sleep(3000);
                LOGGER.info("Total time to retrieve results = {}",
                        System.currentTimeMillis() - startTime);
                return responseEntity.getBody();
            }
        }
        catch (URISyntaxException | InterruptedException e)
        {
            LOGGER.error("URI has a wrong syntax", e);
        }

        LOGGER.info("No result found, returning an empty list");
        return new ArrayList<>();
    }

In short, my circuit breaker loop will call the service enough times to pass the threshold of 65 percent of slow calls that are of duration more than 3 seconds. Once I click on the link for here, I will receive the result, but my circuit breaker will be open and will not allow future calls till it is in either half-open state or closed state.

You will notice that we started getting an exception CallNotPermittedException when the circuit breaker was in the OPEN state. Also, the circuit breaker was opened when the 10 calls were performed. This is because our sliding window size is 10.

Another way, I can simulate the error by shutting down my REST service or database service. That way REST calls can take longer than required.

Now, let’s switch the COUNT_BASED circuit breaker to TIME_BASED circuit breaker. In TIME_BASED circuit breaker, we will switch off our REST service after a second, and then we will click on here link from the home page. If 70 percent of calls in the last 10 seconds fail, our circuit breaker will open.

Since REST Service is closed, we will see the following errors in Circuitbreakdemo application

We will see the number of errors before the circuit breaker will be in OPEN state.

One configuration we can always add how long we want to keep the circuit breaker in the open state. For the demo, I have added the circuit breaker will be in an open state for 10 seconds.

How to handle OPEN circuit breakers?

One question arises, how do you handle OPEN circuit breakers? Luckily, resilience4j offers a fallback configuration with Decorators utility. In most cases, you can always configure this to get the result from previous successful results so that users can still work with the application.

Conclusion

In this post, I have covered how to use a circuit breaker in a Spring Boot application. The code for this demo is available here.

In this demo, I have not covered how to monitor these circuit breaker events as resilience4j the library allows storing these events with metrics that one can monitor with a monitoring system.

If you enjoyed this post, consider subscribing to my blog here.

Simplifying Spring Security

Yogesh Mali — Fri, 19 Feb 2021 04:51:29 +0000

Finally, the book is here. Simplifying Spring Security.

Why I wrote this book?

As part of writing this blog, I also follow few communities on Facebook. Most of these communities are related to Spring Framework and Spring Boot. The number of users asks questions related to Spring Security. Hence, I wondered why not write a book about it.

Also as a developer, when I’m writing a Spring Boot application, I often use Spring Security. Accordingly, I always felt like I was using this mysterious library that solves my authentication problems. I wanted to understand the fundamentals and how Spring Security dealt with authentication and authorization.

What do I cover?

In the book, I cover from fundamentals of authentication, authorization, and how to use Spring Security for different authentication flows. Also, I show these flows with examples. As part of the book, you will also get access to a source code repository that you can play with.

In short, I cover the following topics in the book:

Introduction
1. What is Spring Security?
2. How Spring Security fits in with Spring Boot Application?
3. Why you need Spring Security?
Authentication
1. What is authentication?
2. Authentication Architecture
3. Types of Authentication
4. Implementation of Different Flows
Authorization
1. What is authorization?
2. How does Spring Security handle authorization?
3. What are GrantedAuthorities?
4. Implementation of Authorization in an application
Protection against common exploits
1. Introduction
2. Transport Layer Security
3. Security HTTP Response Headers
4. Clickjacking Attack
5. Cross-site Request Forgery Attack (CSRF)
Miscellaneous

Simplifying Spring Security

Why should you buy this book?

First, it is a technical book and if you are a developer, it will easily help you improve your career. You’ll learn a lot about authentication and can solve some crucial security problems that many applications face.

Most importantly, you can also build your own application and use any of these authentication mechanisms for the application.

Subsequently, if you are getting started for a job in Spring Boot or Spring Framework, the book will also help you in preparing for Spring Security interviews.

Step by Step Spring Batch Tutorial

Yogesh Mali — Sun, 14 Feb 2021 20:22:02 +0000

In this post, I want to show how you can use Spring Batch. This is a step by step Spring Batch Tutorial.

In enterprise applications, batch processing is common. But with data becoming more prevalent on the internet, it has also become important how we process this data. There are multiple solutions available. Apache Storm or Apache Spark helps with processing and transforming the data in the required format. In this post, we will be looking at Spring Batch more closely.

What is Spring Batch?

Spring Batch is a lightweight framework designed to facilitate batch processing. It allows developers to create batch applications. In turn, these batch applications process the incoming data and transform it for further usage.

Another big advantage of using the Spring Batch is that it allows for high-performance processing of this data. The applications that rely upon data heavily, it is of utmost importance that data becomes instantly available.

Spring Batch allows a developer to use POJO based approach. In this approach, a developer can transform the batch-processed data into data models that she can further use for application business logic.

In this post, I will cover an example where we will batch process a data-intensive CSV file for employee records and transform, validate that data to load into our database.

What is Batch Processing?

Batch processing is a data processing mode. It involves consuming all the data, processing that data, transforming it, and then sending it to another data source. Usually, this is done through an automated job. Either a triggering system or a user triggers a job and that job processes the job definition. Job definition will be about consuming the data from its source.

The key advantage of batch processing is it handles a large volume of data. Nevertheless, this operation can be asynchronous. Most applications perform batch processing separately from real-time user interaction.

Next, we will learn about the Spring Batch framework and what it comprises.

Spring Batch Framework

The following architecture shows the components of the Spring Batch framework.

First, the batch process involves a job. User schedules a job to be run at a certain time or based on a certain condition. This can also involve a job trigger.

Spring Batch framework also includes

logging and tracing
transaction management
job processing statistics
job restart
resource management

Usually, when you configure a job, it will be saved in the job repository. Job Repository keeps the metadata information of all the jobs. A trigger starts these jobs at their scheduled time.

A job launcher is an interface to launch a job or runs a job when the jobs’ scheduled time arrives.

Job is defined with job parameters. When a job starts, a job instance runs for that job. Every execution of job instance has job execution and it keeps track status of the job. A job can have multiple steps.

Step is an independent phase of a job. A job can be comprised of more than one step. Similar to the job, each step has step execution that executes the step and keeps track of the status of the step.

Each step has an item reader that basically reads the input data, an item processor that processes the data and transforms it, and an item writer that takes the processed data and output it.

Now, let’s see all these components in our demo.

Step by Step Spring Batch Tutorial with an example

As part of the demo, we will be uploading a csv file through Spring Batch Framework. So to start with, create the spring project and add the following dependency:

implementation 'org.springframework.boot:spring-boot-starter-batch'

This is the main dependency of our project. Also out main application will look like below:

package com.betterjavacode.springbatchdemo;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;


@SpringBootApplication
public class SpringbatchdemoApplication
{

    public static void main(String[] args)
    {
        SpringApplication.run(SpringbatchdemoApplication.class, args);
    }

}

Create DTO Object

I will be uploading employee data through a CSV file, so I will have my DTO object for Employee created as below:

package com.betterjavacode.springbatchdemo.dtos;

import com.betterjavacode.springbatchdemo.models.Company;
import com.betterjavacode.springbatchdemo.models.Employee;
import com.betterjavacode.springbatchdemo.repositories.CompanyRepository;
import org.springframework.beans.factory.annotation.Autowired;


import java.io.Serializable;

public class EmployeeDto implements Serializable
{
    private static final long serialVersionUID = 710566148641281929L;

    @Autowired
    public CompanyRepository companyRepository;

    private int employeeId;
    private int companyId;
    private String firstName;
    private String lastName;
    private String email;
    private String jobTitle;

    public EmployeeDto()
    {

    }

    public EmployeeDto(int employeeId, String firstName, String lastName, String email,
                        String jobTitle, int companyId)
    {
        this.employeeId = employeeId;
        this.firstName = firstName;
        this.lastName = lastName;
        this.email = email;
        this.jobTitle = jobTitle;
        this.companyId = companyId;
    }

    public Employee employeeDtoToEmployee()
    {
        Employee employee = new Employee();
        employee.setEmployeeId(this.employeeId);
        employee.setFirstName(this.firstName);
        employee.setLastName(this.lastName);
        employee.setEmail(this.email);
        Company company = companyRepository.findById(this.companyId).get();
        employee.setCompany(company);
        employee.setJobTitle(this.jobTitle);
        return employee;
    }

    public int getEmployeeId ()
    {
        return employeeId;
    }

    public void setEmployeeId (int employeeId)
    {
        this.employeeId = employeeId;
    }

    public int getCompanyId ()
    {
        return companyId;
    }

    public void setCompanyId (int companyId)
    {
        this.companyId = companyId;
    }

    public String getFirstName ()
    {
        return firstName;
    }

    public void setFirstName (String firstName)
    {
        this.firstName = firstName;
    }

    public String getLastName ()
    {
        return lastName;
    }

    public void setLastName (String lastName)
    {
        this.lastName = lastName;
    }

    public String getEmail ()
    {
        return email;
    }

    public void setEmail (String email)
    {
        this.email = email;
    }

    public String getJobTitle ()
    {
        return jobTitle;
    }

    public void setJobTitle (String jobTitle)
    {
        this.jobTitle = jobTitle;
    }
}

This DTO class also uses a repository CompanyRepository to get a company object and convert DTO to a database object.

Setting up Spring Batch Configuration

Now, we will set up a batch configuration for our job that will run to upload a CSV file into the database. Our class BatchConfig contain an annotation @EnableBatchProcessing. This annotation enables Spring Batch features and provides a base configuration to set up batch jobs in a @Configuration class.

@Configuration
@EnableBatchProcessing
public class BatchConfig
{

}

This Batch Configuration will include a definition of our job, steps involved in the job. It will also include how we want to read our file data and process it further.

    @Bean
    public Job processJob(Step step)
    {
        return jobBuilderFactory.get("processJob")
                .incrementer(new RunIdIncrementer())
                .listener(listener())
                .flow(step).end().build();
    }

    @Bean
    public Step orderStep1(JdbcBatchItemWriter writer)
    {
        return stepBuilderFactory.get("orderStep1").<EmployeeDto, EmployeeDto> chunk(10)
                .reader(flatFileItemReader())
                .processor(employeeItemProcessor())
                .writer(writer).build();
    }

Above bean declares the job processJob.incrementer adds job parameters. listener will listen to job and handle job status. The bean for listener will handle job completion or job failure notification. As discussed in Spring Batch architecture, every job includes more than one step.

@Bean for step uses stepBuilderFactory to create a step. This step processes a chunk of data in a size of 10. It has a Flat File Reader flatFileItemReader(). A processor employeeItemReader will process the data that has been read by Flat File Item Reader.

    @Bean
    public FlatFileItemReader flatFileItemReader()
    {
        return new FlatFileItemReaderBuilder()
                .name("flatFileItemReader")
                .resource(new ClassPathResource("input/employeedata.csv"))
                .delimited()
                .names(format)
                .linesToSkip(1)
                .lineMapper(lineMapper())
                .fieldSetMapper(new BeanWrapperFieldSetMapper(){{
                    setTargetType(EmployeeDto.class);
                }})
                .build();
    }

    @Bean
    public LineMapper lineMapper()
    {
        final DefaultLineMapper defaultLineMapper = new DefaultLineMapper<>();
        final DelimitedLineTokenizer delimitedLineTokenizer = new DelimitedLineTokenizer();
        delimitedLineTokenizer.setDelimiter(",");
        delimitedLineTokenizer.setStrict(false);
        delimitedLineTokenizer.setNames(format);

        defaultLineMapper.setLineTokenizer(delimitedLineTokenizer);
        defaultLineMapper.setFieldSetMapper(employeeDtoFieldSetMapper);

        return defaultLineMapper;
    }

    @Bean
    public EmployeeItemProcessor employeeItemProcessor()
    {
        return new EmployeeItemProcessor();
    }

    @Bean
    public JobExecutionListener listener()
    {
        return new JobCompletionListener();
    }

    @Bean
    public JdbcBatchItemWriter writer(final DataSource dataSource)
    {
        return new JdbcBatchItemWriterBuilder()
                .itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
                .sql("INSERT INTO employee(employeeId, firstName, lastName, jobTitle, email, " +
                        "companyId) VALUES(:employeeId, :firstName, :lastName, :jobTitle, :email," +
                        " " +
                        ":companyId)")
                .dataSource(dataSource)
                .build();
    }

We will take a look at each of these beans now.

FlatFileItemReader will read the data from the flat file. We are using a FlatFileItemReaderBuilder to create a FlatFileItemReader of type EmployeeDto.

resource indicates the location of the file.

delimited – This builds a delimited tokenizer.

names – will show the order of fields in the file.

lineMapper is an interface to map lines from file to domain object.

fieldSetMapper will map the data from fieldset to an object.

lineMapper bean needs tokenizer and fieldsetmapper.

employeeDtoFieldSetMapper is another bean that we have autowired in this class.


package com.betterjavacode.springbatchdemo.configurations.processor;

import com.betterjavacode.springbatchdemo.dtos.EmployeeDto;
import org.springframework.batch.item.file.mapping.FieldSetMapper;
import org.springframework.batch.item.file.transform.FieldSet;
import org.springframework.stereotype.Component;
import org.springframework.validation.BindException;

@Component
public class EmployeeDtoFieldSetMapper implements FieldSetMapper
{

    @Override
    public EmployeeDto mapFieldSet (FieldSet fieldSet) throws BindException
    {
        int employeeId = fieldSet.readInt("employeeId");
        String firstName = fieldSet.readRawString("firstName");
        String lastName = fieldSet.readRawString("lastName");
        String jobTitle = fieldSet.readRawString("jobTitle");
        String email = fieldSet.readRawString("email");
        int companyId = fieldSet.readInt("companyId");

        return new EmployeeDto(employeeId, firstName, lastName, jobTitle, email, companyId);
    }
}

As you can see, this FieldSetMapper maps fields to individual objects to create an EmployeeDto.

EmployeeItemProcessor implements the interface ItemProcessor. Basically in this class, we validate EmployeeDto data to verify if the company, the employee belongs to, exists.

JobCompletionListener checks for job completion status.

    @Override
    public void afterJob(JobExecution jobExecution)
    {
        if (jobExecution.getStatus() == BatchStatus.COMPLETED)
        {
            // Log statement
            System.out.println("BATCH JOB COMPLETED SUCCESSFULLY");
        }
    }

Now, let’s look at ItemWriter. This bean basically uses JdbcBatchItemWriter. JdbcBatchItemWriter uses INSERT sql statement to insert processed EmployeeDto data into the configured data source.

Configuring Application Properties

Before we run our application to process a file, let’s look at application.properties.


spring.datasource.url=jdbc:mysql://127.0.0.1/springbatchdemo?autoReconnect=true&useSSL=false
spring.datasource.username = root
spring.datasource.password=*******
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
spring.datasource.hikari.connection-test-query=SELECT 1
spring.batch.initialize-schema=ALWAYS

Other than regular data source properties, we should understand the property spring.batch.initialize-schema=ALWAYS. If we don’t use this property and start the application, the application will complain Table batch_job_instance doesn't exist.

To avoid this error, we are basically telling to create batch job-related metadata during startup. This property will create additional database tables in your database like batch_job_execution, batch_job_execution_context, batch_job_execution_params, batch_job_instance etc.

Demo

Now if I execute my Spring Boot Application, it will run and execute the job. There are different ways to trigger a job. In an enterprise application, you will receive a file or data in some kind of storage place (S3 or Amazon SNS-SQS), you will have a job that will be monitoring this location to trigger file loading Spring Batch job.

You can see in the execution a message about job completion – “BATCH JOB COMPLETED SUCCESSFULLY“. If we check our database table, we will see the data loaded.

You can download code for this demo from my github repository.

What more?

I have covered a Spring Batch tutorial here, but this is not all. There is more to Spring Batch than this introductory part. You can have different input data sources or you can also load the data from file to file with various data processing rules.

There are also ways to automate these jobs and process a high volume of data in a performant manner.

Conclusion

In this post, I showed a step by step Spring Batch Tutorial. There are many ways to handle batch jobs, but Spring Batch has made this very easy.

In other news, I recently released my new book – Simplifying Spring Security. If you are looking to learn about Spring Security, you can buy the book here. Accompany this book with this post of Spring Boot Interview questions and you will be ready for your next job interview.

Spring Boot CRUD Application Example with MongoDB

Yogesh Mali — Sat, 23 Jan 2021 17:14:00 +0000

Introduction

In this post, I will show how we can use Spring Boot to build a simple CRUD REST application example with MongoDB. I know your first question will be what is MongoDB?

What is MongoDB?

MongoDB is a NoSQL document database. In this database, records are documents that behave a lot like JSON objects. So it is mostly key-value pair.

The key advantages of using MongoDB as a database are:

MongoDB is a schema less document database. One collection holds different documents.
Structure of a single object is clear.
No complex joins.
Deep query ability.
Easy to scale-out

Here are a few reasons why you would use MongoDB or similar NoSQL databases in enterprise applications:

If you need faster data retrieval in JSON style
Easy to add index on an attribute
For Sharding purposes – Sharding is the process of storing data records across multiple machines. You will usually partition this data according to some criteria while storing across multiple machines.
Fast in-place updates
Easier to query

Prerequisites

To create this sample application, you need:

Spring Boot (version 2.4.1)
MongoDB
Gradle
Java

Spring Boot CRUD Application

As part of this post, I will be building a REST CRUD application.

This includes:

A book library – We will be creating a collection library in our MongoDB database.
A library where we will be storing the books by author
A user can call the REST API to retrieve the book by author
A user can call the REST API to retrieve all the books from the library
POST – /v1/mongodbapp/books – To add a book to library
GET – /v1/mongodbapp/books – To retrieve all the books from the library
GET – /v1/mongodbapp/books/id – To retrieve a specific book
DELETE – /v1/mongodbapp/books/id – To remove a book from the library

How to use Spring Boot CRUD with MongoDB

To get started with creating this application, we will be using gradle to handle our dependencies and build the application. Add the following dependencies in our Spring Boot application:

implementation 'org.springframework.boot:spring-boot-starter-data-mongodb'
implementation 'org.springframework.boot:spring-boot-starter-web'

Once we have these dependencies, we will be able to connect to mongo db database. But we still need to add where our database is located. We will add the required properties in our application.properties file as below:

spring.data.mongodb.host = localhost
spring.data.mongodb.port=27017
spring.data.mongodb.database=library

This will allow us to connect to MongoDB database running at host localhost on the port 27017 and the database schema is library.

Define Data Model

As part of the library, we will need books. So our main data object is Book. The data model for this will include the book title, author, and ISBN. This will be as follows:

package com.betterjavacode.mongodbdemo.mongodbapp.models;

import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;



@Document(collection = "books")
public class Book
{
    @Id
    private String id;

    private String title;
    private String author;
    private String isbn;

    public Book()
    {

    }

    public Book(String title, String author, String isbn)
    {
        this.title = title;
        this.author = author;
        this.isbn = isbn;
    }

    public String getId ()
    {
        return id;
    }

    public void setId (String id)
    {
        this.id = id;
    }

    public String getTitle ()
    {
        return title;
    }

    public void setTitle (String title)
    {
        this.title = title;
    }

    public String getAuthor ()
    {
        return author;
    }

    public void setAuthor (String author)
    {
        this.author = author;
    }

    public String getIsbn ()
    {
        return isbn;
    }

    public void setIsbn (String isbn)
    {
        this.isbn = isbn;
    }
}

Since we are using MongoDB, @Document annotation overrides the collection books.

Add Repository Interface

We will need a repository interface to fetch, save or delete our book object. In repositories package, we will add BookRepository interface

package com.betterjavacode.mongodbdemo.mongodbapp.repositories;

import com.betterjavacode.mongodbdemo.mongodbapp.models.Book;
import org.springframework.data.mongodb.repository.MongoRepository;

import java.util.List;

public interface BookRepository extends MongoRepository<Book, String>
{
    List findByTitleContaining(String title);
    List findByAuthor(String name);
}

This repository has two methods to fetch list of books by a title or by author name.

Add Spring REST API Controller

Now to make our application REST CRUD, we will add a REST controller. This controller will contain POST, PUT, GET and DELETE APIs.

package com.betterjavacode.mongodbdemo.mongodbapp.controller;

import com.betterjavacode.mongodbdemo.mongodbapp.models.Book;
import com.betterjavacode.mongodbdemo.mongodbapp.repositories.BookRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import java.util.ArrayList;
import java.util.List;
import java.util.Optional;

@CrossOrigin("http://localhost:8080")
@RestController
@RequestMapping("/v1/mongodbapp")
public class BookController
{
    @Autowired
    BookRepository bookRepository;

    @GetMapping("/books")
    public ResponseEntity<List> getAllBooks(@RequestParam(required = false) String bookTitle)
    {
        try
        {
            List listOfBooks = new ArrayList<>();
            if(bookTitle == null || bookTitle.isEmpty())
            {
                bookRepository.findAll().forEach(listOfBooks::add);
            }
            else
            {
                bookRepository.findByTitleContaining(bookTitle).forEach(listOfBooks::add);
            }

            if(listOfBooks.isEmpty())
            {
                return new ResponseEntity<>(HttpStatus.NO_CONTENT);
            }

            return new ResponseEntity<>(listOfBooks, HttpStatus.OK);
        }
        catch (Exception e)
        {
            return new ResponseEntity<>(null, HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @GetMapping("/books/{id}")
    public ResponseEntity getBookById(@PathVariable("id") String id)
    {
        try
        {
            Optional bookOptional = bookRepository.findById(id);

            return new ResponseEntity<>(bookOptional.get(), HttpStatus.OK);
        }
        catch (Exception e)
        {
            return new ResponseEntity<>(null, HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @PostMapping("/books")
    public ResponseEntity addABookToLibrary(@RequestBody Book book)
    {
        try
        {
            Book createdBook = bookRepository.save(new Book(book.getTitle(), book.getAuthor(),
                    book.getIsbn()));
            return new ResponseEntity<>(createdBook, HttpStatus.CREATED);
        }
        catch (Exception e)
        {
            return new ResponseEntity<>(null, HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @PutMapping("/books/{id}")
    public ResponseEntity updateABook(@PathVariable("id") String id, @RequestBody Book book)
    {
        Optional bookOptional = bookRepository.findById(id);

        if(bookOptional.isPresent())
        {
            Book updatedBook = bookOptional.get();
            updatedBook.setTitle(book.getTitle());
            updatedBook.setAuthor(book.getAuthor());
            updatedBook.setIsbn(book.getIsbn());
            return new ResponseEntity<>(bookRepository.save(updatedBook), HttpStatus.OK);
        }
        else
        {
            return new ResponseEntity<>(HttpStatus.NOT_FOUND);
        }
    }

    @DeleteMapping("/books/{id}")
    public ResponseEntity deleteABook(@PathVariable("id") String id)
    {
        try
        {
            bookRepository.deleteById(id);
            return new ResponseEntity<>(HttpStatus.NO_CONTENT);
        }
        catch (Exception e)
        {
            return new ResponseEntity<>(HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }
}

BookController is our REST Controller class. It includes

* @RestController – to mark this as a REST controller.


@CrossOrigin – This annotation allows cross-origin resource sharing (CORS). This will add CORS access control headers in the REST response. These headers are necessary because it allows servers to specify not only who can access the resources, but how they can be accessed.
We have added different methods – addABookToLibrary adds the book to database, getAllBooks retrieves the book from database.

The Complete Demo

Now we have added our REST controller, repository methods, we will build and run this application. As part of the demo, I will use POSTMAN to access APIs.

You can build the application from command line or from the code editor you are using. I prefer command line, so I already built the application. Once built, you can run the program as follows:

java -jar mongodbapp-0.0.1-SNAPSHOT.jar.

Let’s use POSTMAN to add books to our library.

As shown above, we added a book and the response shows the added book in the database.

The following API is a GET API to get all the books from the database.

Now to show delete the book from library, we will use DELETE API.

The code for this sample application is available in my github repository.

If you want to see this data in MongoDB database, you can access MongoDB compass tool and access the collection books in library database.

Conclusion

In this post, I showed how to use Spring Boot to create REST API while using MongoDB database. The approach we used is very similar to what we have while using a regular SQL database. There are some changes that we have to add while using NoSQL database. If you find this post useful or have any questions that I can help you with, you can subscribe to my blog here.

Spring WebClient vs RestTemplate – Comparison and Features

Yogesh Mali — Sun, 27 Dec 2020 18:30:38 +0000

Introduction

Spring 5 introduced a new reactive web client called WebClient. In this post, I will show when and how we can use Spring WebClient vs RestTemplate. I will also describe what features WebClient offers.

What is RestTemplate?

RestTemplate is a central Spring class that allows HTTP access from the client-side. RestTemplate offers POST, GET, PUT, DELETE, HEAD, and OPTIONS HTTP methods. The simple use case of RestTemplate is to consume Restful web services.

You can create a bean that provides the instance of RestTemplate. You can then @autowire this bean in any class where you plan to call REST services. RestTemplate is the class that implements the interface RestOperations.

The following code shows the declaration of the bean:

    @Bean
    public RestOperations restOperations()
    {
        return new RestTemplate();
    }

The following code shows a REST client YelpClient calling Yelp’s REST API to get rental property reviews.

   @Autowired
   private final RestOperations restOperations;

   public List getRentalPropertyReviews(String address)
   {
        String url = buildRestUrl(businessId);
        HttpHeaders httpHeaders = new HttpHeaders();
        String apiKey = getApiKey(YELP);
        httpHeaders.add("Authorization","Bearer " + apiKey);
        httpHeaders.setContentType(MediaType.APPLICATION_JSON);

        HttpEntity entity = new HttpEntity("parameters", httpHeaders);
        ResponseEntity response;

        try
        {
            response = restOperations.exchange(url, HttpMethod.GET,entity, String.class);
        }
        catch(RestClientException e)
        {
            throw new RuntimeException("Unable to retrieve reviews", e);
        }

    }

In the above code, we are building HTTP Headers by adding Yelp’s REST API key as part of the authorization. We call the GET method to get review data.

Basically, one has to do

Autowire the RestTemplate object
Build HTTP Headers with authorization and Content Type
Use HttpEntity to wrap the request object
Provide URL, Http Method, and the Return type for exchange method.

What is WebClient?

Spring 5 introduced a reactive web client called WebClient. It’s an interface to perform web requests. It is part of the Spring web reactive module. WebClient will be replacing RestTemplate eventually.

Most importantly, WebClient is reactive, nonblocking, asynchronous, and works over HTTP protocol Http/1.1.

To use WebClient, one has to do

Create an instance of WebClient
Make a request to the REST endpoint
Handle the response

WebClient webClient = WebClient
       .builder()
       .baseUrl("https://localhost:8443")
       .defaultCookie("cookieKey", "cookieValue")
       .defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE) 
       .defaultUriVariables(Collections.singletonMap("url", "https://localhost:8443"))
       .build();

The above code shows one way to instantiate WebClient. You can also create an instance by simply using WebClient webClient = WebClient.create();

WebClient provides two methods exchange and retrieve. exchange method usually fetches the response along with status and headers. retrieve method gets the response body directly. It’s easier to use.

Also depending on if you are trying to fetch a single object in response or a list of objects, you can use mono or flux.

this.webClient =
                webClientBuilder.baseUrl("http://localhost:8080/v1/betterjavacode/").build();

this.webClient.get()
                .uri("users")
                .accept(MediaType.APPLICATION_JSON)
                .retrieve().bodyToFlux(UserDto.class).collectList();

The above code basically uses webClient to fetch a list of users from the REST API.

Spring WebClient vs RestTemplate

We already know the one key difference between these two features. WebClient is a non-blocking client and RestTemplate is a blocking client.

RestTemplate uses Java Servlet API under the hood. Servlet API is a synchronous caller. Because it is synchronous, the thread will block until webclient responds to the request.

Consequently, Requests waiting for results will increase. This will result in an increase in memory.

On the other hand, WebClient is an asynchronous non-blocking client. It uses Spring’s reactive framework under the hood. WebClient is a part of the Spring-WebFlux module.

Spring WebFlux uses reactor library. It provides Mono and Flux API to work data sequences. Reactor is a reactive streams library. And, all of its operators support non-blocking back pressure.

Example of how to use WebClient in a Spring Boot Application

We can combine the capabilities of Spring Web MVC and Spring WebFlux. In this section, I will create a sample application. This application will call a REST API using WebFlux and we will build a response to show a web page with a list of users.

RestController for this example is an API to get a list of users:


package com.betterjavacode.webclientdemo.controllers;

import com.betterjavacode.webclientdemo.dto.UserDto;
import com.betterjavacode.webclientdemo.managers.UserManager;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.List;

@RestController
@RequestMapping("v1/betterjavacode")
public class UserController
{
    @Autowired
    public UserManager userManager;

    @GetMapping(value = "/users")
    public List getUsers()
    {
        return userManager.getAllUsers();
    }
}

Controller class that uses a WebClient to call REST API looks like below:

package com.betterjavacode.webclientdemo.controllers;

import com.betterjavacode.webclientdemo.clients.UserClient;
import com.betterjavacode.webclientdemo.dto.UserDto;
import com.betterjavacode.webclientdemo.managers.UserManager;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;

import java.util.List;

@Controller
public class MainController
{
    @Autowired
    UserClient userClient;

    @GetMapping(value = "/")
    public String home()
    {
        return "home";
    }

    @GetMapping(value = "/users")
    public String getUsers(Model model)
    {
        List users = userClient.getUsers().block();

        model.addAttribute("userslist", users);
        return "users";
    }
}

Now, the important piece of code of UserClient is where we will be using WebClient to call REST API.

package com.betterjavacode.webclientdemo.clients;

import com.betterjavacode.webclientdemo.dto.UserDto;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Service;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;

import java.util.List;

@Service
public class UserClient
{

    private WebClient webClient;

    public UserClient(WebClient.Builder webClientBuilder)
    {
        this.webClient =
                webClientBuilder.baseUrl("http://localhost:8080/v1/betterjavacode/").build();
    }

    public Mono<List> getUsers()
    {
        return this.webClient.get()
                .uri("users")
                .accept(MediaType.APPLICATION_JSON)
                .retrieve().bodyToFlux(UserDto.class).collectList();
    }
}

Above code shows first building the WebClient and then using it to retrieve response from REST API. retrieve method offers two options of mono or flux. Since we have more than one user to get, we are using flux.

This shows we can use reactive, non-blocking WebClient which is part of WebFlux in Spring Web MVC framework.

What Else Is There in Spring WebClient?

Spring WebClient is part of Spring WebFlux framework. The major advantage of this API is that the developer doesn’t have to worry about concurrency or threads. WebClient takes care of that.

WebClient has a built-in HTTP Client library support to perform requests with. That includes Apache HttpComponents, Jetty Reactive HttpClient, or Reactor Netty.

WebClient.builder() offers following options:

uriBuilderFactory – customized uriBuilderFactory to use base URL
defaultHeader – Headers for every request
defaultCookie – Cookies for every request
defaultRequest – To customize every request
filter – Client filter for every request
exchangeStrategies – HTTP Message reader/writer customizations
I already showed retrieve method in the above code demo.

WebClient also offers a method exchange with varients like exchangeToMono andexchangeToFlux`.

With attribute(), we can also add attributes to the request.

Alternatively, one can use WebClient for synchronous use also. In my example above MainController, I use block to get the final result. This basically blocks parallel calls till we get the result.

One key feature that WebClient offers is retryWhen(). For more resilient system, it is a great feature that you can add while using WebClient.

     webClient
        .get()
        .uri(String.join("", "/users", id))
        .retrieve()
        .bodyToMono(UserDto.class)
        .retryWhen(Retry.fixedDelay(5, Duration.ofMillis(100)))
        .block();

retryWhen takes Retry class as a parameter.

WebClient also offers a feature for error handling. doOnError() allows you to handle the error. It is triggered when mono ends with an error. onErrorResume() is a fallback based on the error.

Conclusion

In this post, I showed what is Spring WebClient is, how we can use Spring WebClient vs RestTemplate, and what different features it offers.

The Complete Guide to use Docker Compose

Yogesh Mali — Sun, 20 Dec 2020 02:13:30 +0000

In this post, I will cover the complete guide to using docker compose. You can use it to build a multi-container application. But what is a docker compose and why one should use it?

What is Docker Compose?

If you don’t know what a docker is, you can read about that here. If you have an application that is running on a docker and if that application is using multiple other services like database, web-server, load balancer, then you can write multiple docker files and run multiple containers. It can be cumbersome to manage these files. And if you have to change something, you might have to change in all files.

Docker compose solves this problem by allowing you to write a YAML file to define multiple containers in a single file. You write one docker file and build and run that file for all the containers.

Installing Docker Compose

Based on the definition from docker, docker compose is a tool for defining and running multiple Docker containers.

Depending on your environment, you will have to use the instructions to install docker compose. You will also need docker engine before you can install docker compose. I use the Windows environment, so I will show those instructions here.

Launch Power shell in administrator mode
Run this command – [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Then run the following command – Invoke-WebRequest “https://github.com/docker/compose/releases/download/1.27.4/docker-compose-Windows-x86_64.exe” -UseBasicParsing -OutFile $Env:ProgramFiles\Docker\docker-compose.exe This will install docker compose. Open a new command prompt and type the first command

docker-compose -v

This should provide the docker-compose version if your installation has run without any issues.

Setting up a Spring Boot application with Docker

To show the power of docker-compose, we will be using a simple To-Do list spring boot app. I will share this app in a GitHub repository along with docker compose file. But this app includes the following applications that we will be using in docker compose:

Spring Boot application
Java version 8
MySQL for database
Keycloak for Authentication
Postgres for database

So I won’t show implementing the Spring Boot application. If you want to download this application, you can visit the Github repository or you can read my previous post here.

We will create a docker file for this Spring Boot application and this will run in its own container. Now this application connects to Keycloak and MySQL database for authentication. Keycloak will use Postgres database instead of using the same MySQL database.

The docker file for the Spring Boot application will look like below:

FROM openjdk:8-jdk-alpine
VOLUME /tmp
COPY ./build/libs/*.jar app.jar
ENTRYPOINT ["java", "-jar", "/app.jar"]

This docker file basically downloads Open JDK 8. It mounts the disk at /tmp. It copies an application jar file as app.jar. And of course, it will start the application by running java -jar .

How to write Docker Compose file

Now comes the docker-compose.yml file. This will look like below:

version: "3.8"

services:
  web:
    build: .
    ports:
      - "8080:8080"
    depends_on:
      - db
      - keycloak
    environment:
      SPRING_DATASOURCE_URL: jdbc:mysql://db:3306/todolist?autoReconnect=true&useSSL=false
      SPRING_DATASOURCE_USERNAME: betterjavacode
      SPRING_DATASOURCE_PASSWORD: betterjavacode
      KEYCLOAK_URI: http://keycloak:8180/auth
      REALM: SpringBootKeycloakApp
    networks:
      - common-network
  db:
    image: mysql:5.7
    ports:
      - "3307:3306"
    restart: always
    environment:
      MYSQL_DATABASE: todolist
      MYSQL_USER: betterjavacode
      MYSQL_PASSWORD: betterjavacode
      MYSQL_ROOT_PASSWORD: root
    volumes:
      - db-data:/var/lib/mysql
    networks:
      - common-network
  postgres:
    image: postgres
    volumes:
      - postgres_data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: keycloak
      POSTGRES_USER: keycloak
      POSTGRES_PASSWORD: password
    networks:
      - common-network
  keycloak:
    image: jboss/keycloak
    ports:
      - "8180:8180"
    command: ["-Djboss.socket.binding.port-offset=100"]
    environment:
      DB_VENDOR: POSTGRES
      DB_ADDR: postgres
      DB_DATABASE: keycloak
      DB_USER: keycloak
      DB_PASSWORD: password
      DB_SCHEMA: public
      KEYCLOAK_USER: admin
      KEYCLOAK_PASSWORD: Pa55w0rd
    depends_on:
      - postgres
    networks:
      - common-network
networks:
  common-network:
    driver: bridge
volumes:
  db-data:
    driver: local
  postgres_data:
    driver: local

The first line in this docker-compose file is the version of your docker-compose.

services define different types of services that we will use to build our docker container. web service uses an image that builds from a docker file. In our case, we are building a docker image of our Spring Boot application. This application will run on port 8080. We also have to make sure to pass the required environment variables. As you see in the file, we are using our database as db and the variable SPRING_DATASOURCE_URL shows that. db is the name of our database service that our application will connect to.

Our database service db runs on host port of 3307, but uses port 3306 (default port) on the container. This is because I have MySQL running on my host machine at port 3306, so to avoid port conflict, I am using 3307.

We have another database service postgres in our docker compose file. That uses default ports of 5432 and that’s why not specified here. Keycloak uses postgres as part of this entire application. If you don’t specify postgres, Keycloak will use an in-memory H2 database by default. The problem with an in-memory database is once you stop your container, it will lose all the data. To avoid that, I am using a real database that will save our realm and users’ data.

Another service, that we are using is keycloak. This is our IDP for authentication. The service is running on port 8180. It uses the Postgres database to connect to. The command part of keycloak service instructs to run the service on port 8180 in the container instead of default 8080.

networks service defines that all these containers are part of the same network common-network with a driver of type bridge.
To make sure we can use the database, we need to mount the disk volume for both MySQL and Postgres databases. We mount these volumes locally.

Running the containers

Now to execute the containers with the application, execute the following command (make sure you build your application)

docker-compose up

This will build Docker containers for all our services and start them. Now if we access our application at http://localhost:8080

If a user clicks on Get all tasks, user will see keycloak login screen as below:

Enter the username and password, and the user will see the tasks for the logged-in user.

Useful commands

docker-compose up – This command will build the docker containers and start them.

docker-compose up -d – This is a similar command as above, except it will run all the processes in the background.

docker-compose stop – Stop the docker services. This will retain the previous state of containers even after you have stopped the containers.

docker-compose start – Start the docker services

docker-compose logs – Show the logs from docker containers

docker-compose ps – List the Docker containers

docker-compose run – Run one-off command. Example – docker-compose run web env – List the environment variables of web service.

Advantages of Docker Compose

By running most of the services in docker, you don’t have to install those services in your environment.
It’s easier to collaborate the development environment with other developers by checking in the source in version control with docker-compose.
Quick and easy configuration. You can run your services across platforms.

Advance use of docker compose

Something I have not covered in this post is using network as a service that you can really extend with docker compose. It also allows you to run a load balancer (or reverse proxy like nginx) and manage the load with multiple hosts.

Instead of using environment variables, you can also use .env file for environment variables and load it while starting the containers.

Conclusion

In this post, I showed how you can use docker compose to run multiple containers with a single docker compose file. It also allows you to easily manage your environment.

References

Docker Compose – docker compose
Keycloak – Keycloak containers

7 AWS Services Every Developer Should Know About

Yogesh Mali — Wed, 18 Nov 2020 17:57:05 +0000

In this post, I will describe the 7 AWS Services a developer should know about. As a developer, it is important to understand when and how to use these services.

Even though moving the infrastructure to the cloud movement began in the last decade, it has picked up the speed in the last 5-6 years. As always, Amazon had been leading on this front. Now the most companies use cloud whether AWS, Google, or Microsoft Azure services for cloud infrastructure.

Amazon Web Services offer a number of services, but we will look at the 7 AWS services only. These services help users to administrator their applications smoothly. If you are building an application and using old on-premise infrastructure, you will be better off moving to the cloud. The dilemma comes what services to use, which Cloud service to choose.

7 AWS Services to Know

Elastic Container Service (ECS)
Lambda
Simple Storage Service (S3)
DynamoDB
Route53
Elastic Load Balancer(ELB)
Kinesis

Elastic Container Service

ECS is a container orchestration service. What does container orchestration mean?

Basically, you build your application.
Deploy that application on a docker container.
Push the docker image to ECR (Elastic Container Repository).
Build your elastic container service while using the docker image from ECR.

ECS does a good job of choosing what EC2 instance to use, how to identify a load balancer to use, and how many instances to run for your service.

ECS easily integrates with other cloud services that Amazon offers. These services include Amazon Route53, Secrets Manager, Identity and Access Management, and Cloudwatch.

While deploying your service on ECS, you can choose an EC2 container or AWS Fargate container. AWS Fargate is a serverless compute engine that works with both Elastic Container Service or Elastic Kubernetes Service (EKS).

EC2 is an Elastic Cloud Computing service that offers a server instance. You can choose what you want on your server instance installed.

Lambda

If you have heard the word “Serverless”, it is the AWS Lambda that has made it popular. You can write your code without provisioning or managing servers. The selling point of Lambda is that you don’t have to worry about backend infrastructure. Lambda takes care of that, you only pay for the compute time you use. So this makes writing efficient code even more important.

The alacrity with which Lambda handles running your code, it makes Lambda one of the most useful services. Lambda is definitely the most useful when you want to automate repetitive tasks.

Running an entire application on Lambda may not be the best idea. If it’s a static website, then surely you can use Lambda. But a scalable and dynamic application, you will be better off using an EC2 instance than Lambda.

One heuristic to consider to decide when to use Lambda – If you have a code that will not change, but will also perform repetitive tasks without much monitoring efforts, then use Lambda.

Simple Storage Service (S3)

S3 is a Simple Cloud Storage. Irrespective of what kind of application you build, you have to store static files somewhere. AWS offers a simple and effective service called S3.

To understand S3 is to understand a hash table. Usually, when you store any file on S3, the service generates a random string as a key to identify that file. The file is the blob format data.

S3 is targeted towards application builders to individual users. The advantages of S3 are scalability, high availability, performance, and security. S3 can also be used for redundancy. One thing to remember that you can not use S3 to host a static website, especially if you want to use it with HTTPS.

DynamoDB

DyanmoDB is a No-SQL database service. The advantages of DynamoDB over regular DB are:

High performance even at a scale
A simple API allowing for simple key-value access Dynamo DB is a partitioned B-Tree Data structure. The performance in DynamoDB remains consistent irrespective of the data you are inserting in it. Key-Value pair makes it easy to access Dyanmo DB.

You can get aggregated or sorted data from a regular relational database. But with DynamoDB, your application has to do everything on its own. Once the application fetches the data from DynamoDB, it can get a single data or a contiguous range of data. This is the key point in deciding if you want to use DynamoDB or not.

Route53

Route53 as the name suggests is a DNS service that translates a domain name to IP address. When a request comes from an application to AWS infrastructure, Route53 translates that request to either load balancers or EC2 instance or S3 bucket.

Route53 may not have many advantages over any other infrastructure, but as a developer knowing about Route53 is important. Once you deploy your application in AWS, you will need to make sure it is accessible to users. From a network perspective, understanding Route53 is important.

Route53 integrates well with ELB (Elastic Load Balancer).

ELB

ELB is an elastic load balancer service. It offers a load balancer for application and for the network. So we can say application load balancer (ALB) or network load balancer (NLB).

Application load balancer basically routes the traffic from the internet to your application and vice versa. ALBs offer different features like routing rules, sticky sessions, authentications.

Network load balancers route the network packets. NLB is a network router and not an HTTP request router.

Both NLBs and ALBs support TLS/HTTPS and integrates with AWS Certificate Manager. NLBs and ALBs don’t validate certificates, but since these load balancers run in a VPC, there is a protection from spoofing and man-in-the-middle attack.

One disadvantage of ALBs is that it adds a few milliseconds to each request, increasing the latency in the process. AWS handles scaling ALB automatically based on the demand for your service.

Kinesis

Kinesis is a streaming service. It basically collects, processes, and analyzes real-time data. Once the data arrives in the streaming service, you can build an event to handle the data. Kinesis steam is a highly durable linked list.

You can have multiple applications consuming data from Kinesis. As data streaming getting more and more popular, kinesis offers a lot of flexibility.

Conclusion

There are a lot of other AWS services like SQS, Cloud Formation, Cognito, API Gateway, Step Functions, etc. Here I described 7 AWS services that you will end up using the most frequently. Also, I want to thank Daniel Vassallo and his insightful book The Good Parts of AWS.

Original post for this was published on my blog betterjavacode

How to Use Keycloak With a Spring Boot Application

Yogesh Mali — Fri, 25 Sep 2020 23:25:42 +0000

In this post, I will show how to use Keycloak in a Spring Boot application. Before we use Keycloak, we will cover some basics about what Keycloak is and why we use it.

To get started with this demo, you will need the following things:

A Code Editor – IntelliJ
Database – MySQL
Keycloak
Java 8

What is Keycloak?

Keycloak is an open-source identity and access management solution for modern applications and services. Keycloak provides both SAML and OpenID protocol solutions.

Why do we use Keycloak?

As mentioned, Keycloak provides identity and access management, it is also open source. SAML and OpenID protocols are industry standards. Building an application that is integrated with Keycloak will only provide you a more secure and stable solution. There are definitely other solutions available like Gluu, Shibboleth, WSO2.

For this post, we will be using Keycloak.

Securing Spring Boot Application with Keycloak

There are two parts in this demo. One is about Keycloak. The second is about securing the Spring Boot Application with Keycloak.

Install Keycloak

Download the Keycloak on your machine. Unzip the downloaded file and run the server with the following command from bin directory on your command prompt (Note – I’m on a windows machine):

standalone.bat -Djboss.socket.binding.port-offset=100

This will start the Wildfly server for your Keycloak on your local machine. We can access the server by executing the URL http://localhost:8180. If you just use standalone.bat to execute without that parameter, the server will run on the port 8080.

Once you start the server, the first thing you will have to do is to create an admin user. We will create a user admin and password d#n3q2b .

Now we will access the administration console and enter our user details. Once we login as an admin user, we will see the first screen as below:

Adding Application

Initial screen shows the default realm. For our demo purposes, we will create a new realm SpringBootKeycloakApp. In this realm, we will add our Spring Boot application as a client. Create a new client on Clients tab. We will name our client application as SpringBootApp.

Now in settings, we will add redirect url for our Spring Boot Application. This is the URL where Keycloak will redirect to our app after authentication. Also, we are using openid connect as a protocol as part of this implementation.

Adding User

Now we will add a user that we will use to authenticate. We will use this user to login to our sample Spring Boot application.

Add a role that you want for this user ROLE_User on the roles tab in Keycloak. Once that is done, let’s go to the Users tab and add a new user.

On the Role Mappings tab, make sure to add the newly created role for this user.

Create A Spring Boot Application

Now, we will create a simple Spring Boot application that will use Keycloak for security. As part of this application, we will be showing a list of to-do list tasks for the user who will authenticate with the application.

To build this app, we need the following dependencies:

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
    implementation 'org.springframework.boot:spring-boot-starter-security'
    implementation 'org.springframework.boot:spring-boot-starter-thymeleaf'
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.springframework.boot:spring-boot-starter-jdbc'
    implementation 'org.keycloak:keycloak-spring-boot-starter'
    runtimeOnly 'mysql:mysql-connector-java'
    testImplementation('org.springframework.boot:spring-boot-starter-test') {
        exclude group: 'org.junit.vintage', module: 'junit-vintage-engine'
    }
    testImplementation 'org.springframework.security:spring-security-test'
}

As you can see we are using spring-boot and spring-security along with keycloak-spring-boot-starter dependency.

The keycloak dependency includes Keycloak client adapters. We will use these adapters for authentication purposes. They will replace our standard Spring Security adapters. To make sure this keycloak-spring-boot-starter dependency works correctly, we will need one more dependency to be added in our gradle file as below:

dependencyManagement {
    imports {
        mavenBom "org.keycloak.bom:keycloak-adapter-bom:11.0.2"
    }
}

To read more about this, you can visit the official documentation of keycloak.

Our Controller class will have two important methods, one to get the home page which will be accessible for anyone, and another getting the list of tasks which will be accessible to only authenticated users with a role ROLE_User. The code for this TaskController will look like below:

package com.betterjavacode.keycloakdemo.keycloakdemo.controllers;

import com.betterjavacode.keycloakdemo.keycloakdemo.dto.TaskDto;
import com.betterjavacode.keycloakdemo.keycloakdemo.managers.TaskManager;
import org.keycloak.KeycloakSecurityContext;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;

import javax.servlet.http.HttpServletRequest;
import java.util.List;

@Controller
public class TaskController
{
    private final HttpServletRequest request;

    @Autowired
    public TaskController(HttpServletRequest request)
    {
        this.request = request;
    }

    @Autowired
    private TaskManager taskManager;

    @GetMapping(value="/")
    public String home()
    {
        return "index";
    }

    @GetMapping(value="/tasks")
    public String getTasks(Model model)
    {
        List tasks = taskManager.getAllTasks();
        model.addAttribute("tasks", tasks);
        model.addAttribute("name", getKeycloakSecurityContext().getIdToken().getGivenName());

        return "tasks";
    }

    private KeycloakSecurityContext getKeycloakSecurityContext()
    {
        return (KeycloakSecurityContext) request.getAttribute(KeycloakSecurityContext.class.getName());
    }

}

In this controller class, we use TaskManager to get all tasks. I will explain KeyCloakSecurityContext when I will show about SecurityConfig.

With or without Spring-Security

We can leverage this application and use Keycloak for authentication with or without Spring-Security. As part of this demo, we are using Spring-Security. To use the same application without Spring-Security, you can just remove the Spring-Security dependency and add security configuration through application.properties file.

We will need the following properties in application.properties to use Keycloak for authentication in this app.

keycloak.auth-server-url=http://localhost:8180/auth
keycloak.realm=SpringBootKeycloakApp
keycloak.resource=SpringBootApp
keycloak.public-client=true
keycloak.principal-attribute=preferred_username

If we wanted to use this application without Spring-Security, we will need the following two properties also:

keycloak.security-constraints[0].authRoles[0]=ROLE_User
keycloak.security-constraints[0].securityCollections[0].patterns[0]=/tasks

Since we are using Spring-Security, we will configure the security configuration through a Java class SecurityConfig.

This SecurityConfig class will extend KeyCloakWebSecurityConfigurerAdapter.

Our configure method will look like below:

    @Override
    protected void configure(HttpSecurity httpSecurity) throws Exception
    {
        super.configure(httpSecurity);
        httpSecurity.authorizeRequests()
                .antMatchers("/tasks").hasRole("User")
                .anyRequest().permitAll();
    }

Basically any requests coming to /tasks endpoint, should have user role as ROLE_User. The prefix of ROLE_ is assumed here. Other than any other request will be permitted without any authorization. In this case, we will be calling our index page.

We will be using annotation @KeyCloakConfiguration which is basically covers @Configuration and @EnableWebSecurity annotations.

Since our SecurityConfig extends KeycloakWebSecurityConfigurerAdapter, we have to implement sessionAuthenticationStrategy and httpSessionManager. We will also have to register our idp Keycloak with Spring Security Authentication Manager.

So our SecurityConfig will look like below:

package com.betterjavacode.keycloakdemo.keycloakdemo.config;

import org.keycloak.adapters.springsecurity.KeycloakConfiguration;
import org.keycloak.adapters.springsecurity.authentication.KeycloakAuthenticationProvider;
import org.keycloak.adapters.springsecurity.config.KeycloakWebSecurityConfigurerAdapter;
import org.keycloak.adapters.springsecurity.management.HttpSessionManager;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;

import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.core.authority.mapping.SimpleAuthorityMapper;
import org.springframework.security.core.session.SessionRegistryImpl;
import org.springframework.security.web.authentication.session.RegisterSessionAuthenticationStrategy;
import org.springframework.security.web.authentication.session.SessionAuthenticationStrategy;


@KeycloakConfiguration
public class SecurityConfig extends KeycloakWebSecurityConfigurerAdapter
{
    @Autowired
    public void configureGlobal(AuthenticationManagerBuilder authenticationManagerBuilder)
    {
        SimpleAuthorityMapper simpleAuthorityMapper = new SimpleAuthorityMapper();
        simpleAuthorityMapper.setPrefix("ROLE_");

        KeycloakAuthenticationProvider keycloakAuthenticationProvider =
                keycloakAuthenticationProvider();
        keycloakAuthenticationProvider.setGrantedAuthoritiesMapper(simpleAuthorityMapper);
        authenticationManagerBuilder.authenticationProvider(keycloakAuthenticationProvider);
    }

    @Bean
    @Override
    protected SessionAuthenticationStrategy sessionAuthenticationStrategy ()
    {
        return new RegisterSessionAuthenticationStrategy(new SessionRegistryImpl());
    }

    @Bean
    @Override
    @ConditionalOnMissingBean(HttpSessionManager.class)
    protected HttpSessionManager httpSessionManager()
    {
        return new HttpSessionManager();
    }

    @Override
    protected void configure(HttpSecurity httpSecurity) throws Exception
    {
        super.configure(httpSecurity);
        httpSecurity.authorizeRequests()
                .antMatchers("/tasks").hasRole("User")
                .anyRequest().permitAll();
    }
}

So Spring Security uses roles in upper case like ROLE_USER and always use ROLE_ prefix. To handle that, I have added a user with a role ROLE_User in Keycloak, but we will only verify a prefix as our http configuration will verify the role anyhow.

Since we will be authenticating with Keycloak, we will need a session for user’s state. We are using RegisterSessionAuthenticationStrategy here. HttpSessionManager is a conditional bean because Keycloak already implements that bean.

To implement Keycloak Spring Boot adapter, we will add a KeyCloakSpringBootConfigResolver bean as follows:

package com.betterjavacode.keycloakdemo.keycloakdemo.config;

import org.keycloak.adapters.springboot.KeycloakSpringBootConfigResolver;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class KeycloakConfig
{
    @Bean
    public KeycloakSpringBootConfigResolver keycloakSpringBootConfigResolver()
    {
        return new KeycloakSpringBootConfigResolver();
    }
}

I have not shown the rest of the application build, but the code is available on GitHub for this project.

Demo of the application

Run our keycloak application, it will be running on http://localhost:8180. Our Spring Boot application will be running at http://localhost:8080.

Our first screen of the Spring Boot application will look like below:

Now if a user clicks on Get all tasks, he will be redirected to Keycloak login screen as below:

Now, I will enter my user betterjavacode username and password and it will show us our list of tasks as follows:

Authentication Flow

When the user clicks on Get all tasks, the user is redirected to Spring Security’s sso/login endpoint which KeycloakSpringBootConfigResolver handles and sends an authorization code flow request to Keycloak

http://localhost:8180/auth/realms/SpringBootKeycloakApp/protocol/openid-connect/auth?response_type=code&client_id=SpringBootApp&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Fsso%2Flogin&state=70bd4e28-89e6-43b8-8bea-94c6d057a5cf&login=true&scope=openid

Keycloak will process the request to respond with a session code and show the login screen.

Once the user enters credentials and keycloak validates those, it will respond with an authorization code, and this code is exchanged for a token, and the user is logged in.

Conclusion

In this post, I showed how to secure your Spring Boot application using Keycloak as an identity provider.

The original post for this was posted on my blog betterjavacode

How to use Basic Authentication with Rest Template in Spring Boot

Yogesh Mali — Tue, 18 Aug 2020 02:16:24 +0000

In this post, I will show how to use Rest Template to consume RESTful API secured with Basic Authentication. As part of this post, I will show how to build a REST API that is secured with Basic Authentication.

Overview

Basic Authentication is one of the mechanisms that you can use to secure your REST API. In my previous post, I showed how to secure REST API with Json Web Token.

Secure a REST API with Basic Authentication

Configure a REST API

Firstly, we will show a simple REST API to create users or retrieve users from the database. Then, we will secure this REST API with a Basic Authentication mechanism. Lastly, we will show how to use Basic Authentication with Rest Template to call this REST API.

Our REST controller class for this API to create or retrieve users will look like below:



package com.betterjavacode.restdemo.controllers;

import com.betterjavacode.restdemo.dto.UserDto;
import com.betterjavacode.restdemo.managers.UserManager;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import java.util.List;

@RestController
public class UserController
{
    @Autowired
    private UserManager userManager;

    @RequestMapping(value = "/user/", method = RequestMethod.GET)
    public ResponseEntity<List<UserDto>> listAllUsers()
    {
        List<UserDto> users = userManager.getAllUsers();
        if(users.isEmpty())
        {
            return new ResponseEntity<List<UserDto>>(HttpStatus.NO_CONTENT);
        }

        return new ResponseEntity<>(users, HttpStatus.OK);
    }

    @RequestMapping(value = "/user/{id}", method = RequestMethod.GET, produces =
            MediaType.APPLICATION_JSON_VALUE)
    public ResponseEntity<UserDto> getUser(@PathVariable("id") long id)
    {
        UserDto userDto = userManager.getUser(id);
        if(userDto == null)
        {
            return new ResponseEntity<>(HttpStatus.NOT_FOUND);
        }
        return new ResponseEntity<>(userDto, HttpStatus.OK);
    }


    @RequestMapping(value = "/user/", method= RequestMethod.POST)
    public ResponseEntity<UserDto> createUser(@RequestBody UserDto userDto)
    {
        UserDto user = userManager.createUser(userDto);

        return new ResponseEntity<>(user, HttpStatus.OK);
    }

    @RequestMapping(value = "/user/{id}", method=RequestMethod.DELETE)
    public ResponseEntity<UserDto> deleteUser(@PathVariable("id") long id)
    {
        UserDto user = userManager.getUser(id);

        if(user == null)
        {
            return new ResponseEntity<>(HttpStatus.NOT_FOUND);
        }

        userManager.deleteUser(id);

        return new ResponseEntity<>(HttpStatus.NO_CONTENT);
    }
}

Our database model class for User will look like below:



package com.betterjavacode.restdemo.models;

import javax.persistence.*;
import java.io.Serializable;

@Entity(name = "User")
@Table(name = "users")
public class User implements Serializable
{
    private static final long serialVersionUID = 20200816121023L;

    public User()
    {

    }

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    @Column(name="id", nullable=false)
    private long id;

    @Column(name="firstname", length=100)
    private String firstname;

    @Column(name="lastname", length=100)
    private String lastname;

    @Column(name="email", length=100)
    private String email;

    @Column(name="role", length=45)
    private String role;

    @Column(name="enabled")
    private boolean enabled;

    public long getId ()
    {
        return id;
    }

    public void setId (long id)
    {
        this.id = id;
    }

    public String getFirstname ()
    {
        return firstname;
    }

    public void setFirstname (String firstname)
    {
        this.firstname = firstname;
    }

    public String getLastname ()
    {
        return lastname;
    }

    public void setLastname (String lastname)
    {
        this.lastname = lastname;
    }

    public String getEmail ()
    {
        return email;
    }

    public void setEmail (String email)
    {
        this.email = email;
    }

    public String getRole ()
    {
        return role;
    }

    public void setRole (String role)
    {
        this.role = role;
    }

    public boolean isEnabled ()
    {
        return enabled;
    }

    public void setEnabled (boolean enabled)
    {
        this.enabled = enabled;
    }
}

Just to make sure we understand here that, we are using a DTO object UserDto to create and retrieve the data from the database. User is our database model object.

The UserDto object will be as follows:



package com.betterjavacode.restdemo.dto;

import com.betterjavacode.restdemo.models.User;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;

@JsonIgnoreProperties(ignoreUnknown = true)
public class UserDto
{
    private String firstname;
    private String lastname;
    private String email;

    public UserDto(){}

    public UserDto(User user)
    {
        this.setEmail(user.getEmail());
        this.setFirstname(user.getFirstname());
        this.setLastname(user.getLastname());
    }

    public String getFirstname ()
    {
        return firstname;
    }

    public void setFirstname (String firstname)
    {
        this.firstname = firstname;
    }

    public String getLastname ()
    {
        return lastname;
    }

    public void setLastname (String lastname)
    {
        this.lastname = lastname;
    }

    public String getEmail ()
    {
        return email;
    }

    public void setEmail (String email)
    {
        this.email = email;
    }

}

Once we configure our application properties and create the required database table, we will start the application.

Now if we execute the API through a client like Postman, we will be able to retrieve or create the User object.

The goal is to secure this API.

So add Spring-Security in our project build.

implementation "org.springframework.boot:spring-boot-starter-security"

Now, if we add the annotation @EnableWebSecurity in our main application class like below:



package com.betterjavacode.restdemo;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;

@SpringBootApplication
@EnableWebSecurity
public class RestdemoApplication
{
    public static void main(String[] args)
    {
        SpringApplication.run(RestdemoApplication.class, args);
    }
}

and if we access the API to create user, we will get 401 unauthorized error like below:

Basic Authentication

Traditionally, access to REST API will happen on the server-side once the user has logged in with authentication.

Basic authentication provides one of the ways to secure REST API. It’s not the most secure way compared to OAuth or JWT based security. In Basic Authentication, a client sends Base64 encoded credentials with each request using HTTP Authorization Header.

The client will send the Authorization header with each request. There is always a possibility of compromising these credentials even when they are Base64 encoded. To avoid that, we can use HTTPS.

Now from our implementation perspective, we will add a SecurityConfig class to configure security for our REST API.



package com.betterjavacode.restdemo;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter
{
    @Override
    protected void configure(HttpSecurity httpSecurity) throws Exception
    {
        httpSecurity
                .csrf().disable()
                .authorizeRequests().anyRequest().authenticated()
                .and()
                .httpBasic();
    }

    @Autowired
    public void configureGlobal(AuthenticationManagerBuilder auth)
            throws Exception
    {
        auth.inMemoryAuthentication()
                .withUser("adminuser")
                .password("{noop}adminpassword")
                .roles("USER");
    }
}

configure method in this class will configure basic authentication and every request coming to our controller will need to be authorized.

configureGlobal method will add authentication of the incoming request. The requests coming through the controller will be validated for these credentials that we have configured for in-memory authentication.

WARNING – This is not the most secure way to secure your API. Definitely not with in-memory authentication. Do not use it in production.

Now if we execute REST API through POSTMAN, we will see the successful response as below:

Rest Template with Basic Authentication Example

Initially, we used POSTMAN as a client to call our REST APIs. But in a real scenario, we won’t be using POSTMAN, you will have to call these APIs programmatically.

We will create a class RestClient and that will call our APIs while building Basic Authentication.

While using RestTemplate that Spring Boot provides, you need to pass HttpHeaders with a RequestEntity.



private static HttpHeaders getHeaders ()
    {
        String adminuserCredentials = "adminuser:adminpassword";
        String encodedCredentials =
                new String(Base64.encodeBase64(adminuserCredentials.getBytes()));

        HttpHeaders httpHeaders = new HttpHeaders();
        httpHeaders.add("Authorization", "Basic " + encodedCredentials);
        httpHeaders.setAccept(Arrays.asList(MediaType.APPLICATION_JSON));
        return httpHeaders;
    }

We use exchange method from RestTemplate to call our API and HttpHeaders that contain Basic Authentication.

The whole class RestClient will look like below:



package com.betterjavacode.restdemo;


import com.betterjavacode.restdemo.dto.UserDto;
import org.apache.tomcat.util.codec.binary.Base64;
import org.json.JSONObject;
import org.springframework.http.*;
import org.springframework.web.client.RestTemplate;

import java.util.Arrays;
import java.util.LinkedHashMap;
import java.util.List;

public class RestClient
{
    public static final String REST_SERVICE_URL = "http://localhost:8080/user/";

    private static HttpHeaders getHeaders ()
    {
        String adminuserCredentials = "adminuser:adminpassword";
        String encodedCredentials =
                new String(Base64.encodeBase64(adminuserCredentials.getBytes()));

        HttpHeaders httpHeaders = new HttpHeaders();
        httpHeaders.add("Authorization", "Basic " + encodedCredentials);
        httpHeaders.setAccept(Arrays.asList(MediaType.APPLICATION_JSON));
        return httpHeaders;
    }

    private static void listAllUsers()
    {
        System.out.println("Getting all users");
        RestTemplate restTemplate = new RestTemplate();

        HttpHeaders httpHeaders = getHeaders();

        HttpEntity<String> httpEntity = new HttpEntity<>(httpHeaders);

        ResponseEntity<List> responseEntity = restTemplate.exchange(REST_SERVICE_URL,
                HttpMethod.GET, httpEntity, List.class);

        if(responseEntity.hasBody())
        {
            List<LinkedHashMap<String, Object>> users = responseEntity.getBody();

            if(users != null)
            {
                for(LinkedHashMap<String, Object> userMap: users)
                {
                    System.out.println("User is " + userMap.get("firstname") + " " + userMap.get(
                            "lastname"));
                }
            }
        }
        else
        {
            System.out.println("User not found");
        }

    }

    public static void main (String[] args)
    {
        listAllUsers();

        getUser(1);
    }



    private static void getUser(long id)
    {
        System.out.println("Getting a user ");

        String restUrl = REST_SERVICE_URL  + id;

        RestTemplate restTemplate = new RestTemplate();

        HttpHeaders httpHeaders = getHeaders();

        HttpEntity<String> httpEntity = new HttpEntity<>(httpHeaders);

        ResponseEntity<String> responseEntity = restTemplate.exchange(restUrl,
                HttpMethod.GET, httpEntity, String.class);

        if(responseEntity.hasBody())
        {
            JSONObject jsonObject = new JSONObject(responseEntity.getBody());

            System.out.println(jsonObject.get("firstname"));
            System.out.println(jsonObject.get("lastname"));
        }
        else
        {
            System.out.println("User not found");
        }

    }
}

Now if we execute the program, we will see the output as below:

In this post, we showed how to secure REST API with Basic Authentication. The original post was published on my blog Betterjavacode

Json Web Token: How to secure Spring Boot REST API

Yogesh Mali — Sat, 08 Aug 2020 05:50:30 +0000

In this post, I will show how to secure your spring boot based REST API. It has been more of a trend to secure REST APIs to avoid any unnecessary calls to public APIs. We will be using some Spring boot features for Spring security along with JSON WebTokens for authorization.

User flow in this case is

User logs in
We validate user credentials
A token is sent back to user agent.
User tries to access a protected resource.
User sends JWT when accessing the protected resource. We validate JWT.
If JWT is valid, we allow the user to access the resource.

JSON WebTokens, known as JWTs are used for forming authorization for users. This helps us to build secure APIs and it is also easy to scale. During authentication, a JSON web token is returned. Whenever the user wants to access a protected resource, the browser must send JWTs in the Authorization header along with the request. One thing to understand here is that it is a good security practice to secure REST API.

Basically, we will show

Verify JSON WebToken
Validate the signature
Check the client permissions

What you will need?

Java 8,
MySQL Database
IntelliJ Editor
Gradle

Note - This won't be a full-fledged app, but REST APIs based on Spring boot, Spring security.

Spring Boot Based REST API

I will be securing REST API for company that I created in this blog post REST API. This API also includes caching. A user will try to access /cachedemo/v1/companies/ and since APIs are protected, he will get a response like below:

Now we will implement how to protect this API and how to access it when it is protected.

Adding User and User registration

Since we want to add authorization for APIs, we will need where the user is able to log in and send credentials. These credentials will be validated and a token will be generated. This token then will be transmitted in a request to an API call. The token will be validated in the Spring security authorization filter that we will add. If a valid token, the user will be able to access the API.

Create a user model

package com.betterjavacode.models;

import javax.persistence.*;
import java.io.Serializable;

@Entity(name = "User")
@Table(name = "user")
public class User implements Serializable
{
    public User()
    {

    }

    @Id
    @GeneratedValue(strategy =  GenerationType.IDENTITY)
    private long id;

    @Column(name = "username")
    private String username;

    @Column(name = "password")
    private String password;

    public long getId()
    {
        return id;
    }

    public void setId(long id)
    {
        this.id = id;
    }

    public String getUsername()
    {
        return username;
    }

    public void setUsername(String username)
    {
        this.username = username;
    }

    public String getPassword()
    {
        return password;
    }

    public void setPassword(String password)
    {
        this.password = password;
    }
}

We will add a controller where a user can register with its details for username and password.

package com.betterjavacode.resources;

import com.betterjavacode.models.User;
import com.betterjavacode.repositories.UserRepository;
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping(value = "/cachedemo/v1/users")
public class UserController
{
    private UserRepository userRepository;
    private BCryptPasswordEncoder bCryptPasswordEncoder;

    public UserController(UserRepository userRepository, BCryptPasswordEncoder bCryptPasswordEncoder)
    {
        this.userRepository = userRepository;
        this.bCryptPasswordEncoder = bCryptPasswordEncoder;
    }

    @PostMapping("/signup")
    public void signUp(@RequestBody User user)
    {
        user.setPassword(bCryptPasswordEncoder.encode(user.getPassword()));
        userRepository.save(user);
    }

}

Now when we POST a request to /cachedemo/v1/users/signup , a user will be saved in the database. Password for the user will be saved in encrypted format as we are using BCryptPasswordEncoder. We will show how a user can log in to create a token.

User Login

To handle user login, we will add an AuthenticationFilter which will get added in FilterChain and Spring boot will handle the execution of it appropriately. This filter will look like below:

package com.betterjavacode.SpringAppCache;


import com.fasterxml.jackson.databind.ObjectMapper;
import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.SignatureAlgorithm;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.AuthenticationException;
import org.springframework.security.core.userdetails.User;
import org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter;

import javax.servlet.FilterChain;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Date;

public class AuthenticationFilter extends UsernamePasswordAuthenticationFilter
{
    private AuthenticationManager authenticationManager;

    public AuthenticationFilter(AuthenticationManager authenticationManager)
    {
        this.authenticationManager = authenticationManager;
        setFilterProcessesUrl("/login");
    }

    @Override
    public Authentication attemptAuthentication(HttpServletRequest request, HttpServletResponse response) throws AuthenticationException
    {
        try
        {
            com.betterjavacode.models.User creds = new ObjectMapper().readValue(request.getInputStream(), com.betterjavacode .models.User.class);
            return authenticationManager.authenticate(new UsernamePasswordAuthenticationToken(creds.getUsername(), creds.getPassword(),new ArrayList<>()));
        }
        catch(IOException e)
        {
            throw new RuntimeException("Could not read request" + e);
        }
    }

    protected void successfulAuthentication(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain, Authentication authentication)
    {
        String token = Jwts.builder()
                .setSubject(((User) authentication.getPrincipal()).getUsername())
                .setExpiration(new Date(System.currentTimeMillis() + 864_000_000))
                .signWith(SignatureAlgorithm.HS512, "SecretKeyToGenJWTs".getBytes())
                .compact();
        response.addHeader("Authorization","Bearer " + token);
    }
}

Basically, a user will send credentials in a request to URL ending with /login . This filter will help to authenticate the user, if there is successful authentication, a Token will be added in response header with the key Authorization.

Token Validation and Authorization

We add another filter AuthorizationFilter to validate the token that we passed through AuthenticationFilter earlier. This filter will look like below:

package com.betterjavacode.SpringAppCache;

import io.jsonwebtoken.Jwts;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.web.authentication.www.BasicAuthenticationFilter;

import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.ArrayList;


public class AuthorizationFilter extends BasicAuthenticationFilter
{
    public AuthorizationFilter(AuthenticationManager authenticationManager)
    {
        super(authenticationManager);
    }

    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
            throws IOException, ServletException
    {
        String header = request.getHeader("Authorization");
        if(header == null || !header.startsWith("Bearer"))
        {
            filterChain.doFilter(request,response);
            return;
        }

        UsernamePasswordAuthenticationToken authenticationToken = getAuthentication(request);
        SecurityContextHolder.getContext().setAuthentication(authenticationToken);
        filterChain.doFilter(request,response);
    }

    private UsernamePasswordAuthenticationToken getAuthentication(HttpServletRequest request)
    {
        String token = request.getHeader("Authorization");
        if(token != null)
        {
            String user = Jwts.parser().setSigningKey("SecretKeyToGenJWTs".getBytes())
                    .parseClaimsJws(token.replace("Bearer",""))
                    .getBody()
                    .getSubject();
            if(user != null)
            {
                return new UsernamePasswordAuthenticationToken(user, null, new ArrayList<>());
            }
            return null;
        }
        return null;
    }
}

If the validation of the token is successful, a user is returned and assigned to a security context.

To enable Spring security, we will add a new class WebSecurityConfiguration with annotation @EnableWebSecurity. This class will extend the standard WebSecurityConfigurerAdapter. In this class, we will restrict our APIs and also add some whitelisted URLs that we will need access without any authorization token. This will look like below:

package com.betterjavacode.SpringAppCache;

import org.springframework.context.annotation.Bean;
import org.springframework.http.HttpMethod;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.builders.WebSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.config.http.SessionCreationPolicy;
import org.springframework.security.core.userdetails.UserDetailsService;
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
import org.springframework.web.cors.CorsConfiguration;
import org.springframework.web.cors.CorsConfigurationSource;
import org.springframework.web.cors.UrlBasedCorsConfigurationSource;

@EnableWebSecurity
public class WebSecurityConfiguration extends WebSecurityConfigurerAdapter
{
    private BCryptPasswordEncoder bCryptPasswordEncoder;
    private UserDetailsService userDetailsService;

    private static final String[] AUTH_WHITELIST = {
            "/v2/api-docs",
            "/swagger-resources",
            "/swagger-resources/**",
            "/configuration/ui",
            "/configuration/security",
            "/swagger-ui.html",
            "/webjars/**"
    };

    public WebSecurityConfiguration(UserDetailsService userDetailsService, BCryptPasswordEncoder bCryptPasswordEncoder)
    {
        this.bCryptPasswordEncoder = bCryptPasswordEncoder;
        this.userDetailsService = userDetailsService;
    }


    protected void configure(HttpSecurity httpSecurity) throws Exception
    {
        httpSecurity.cors().and().csrf().disable().authorizeRequests()
                .antMatchers(AUTH_WHITELIST).permitAll()
                .antMatchers(HttpMethod.POST, "/cachedemo/v1/users/signup").permitAll()
                .anyRequest().authenticated()
                .and().addFilter(new AuthenticationFilter(authenticationManager()))
                .addFilter(new AuthorizationFilter(authenticationManager()))
                .sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS);
    }

    public void configure(AuthenticationManagerBuilder authenticationManagerBuilder) throws Exception
    {
        authenticationManagerBuilder.userDetailsService(userDetailsService).passwordEncoder(bCryptPasswordEncoder);
    }

    @Bean
    CorsConfigurationSource corsConfigurationSource()
    {
        final UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
        source.registerCorsConfiguration("/**",new CorsConfiguration().applyPermitDefaultValues());
        return source;
    }
}

In method configure we have restricted most APIs, only allowing Swagger URLs and signup URL. We also add filters to HttpSecurity. We will add our own UserDetailsServiceImpl class to validate user credentials.

package com.betterjavacode.services;

import com.betterjavacode.models.User;
import com.betterjavacode.repositories.UserRepository;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.security.core.userdetails.UserDetailsService;
import org.springframework.security.core.userdetails.UsernameNotFoundException;
import org.springframework.stereotype.Component;

import java.util.Collections;

@Component
public class UserDetailsServiceImpl implements UserDetailsService
{
    private UserRepository userRepository;

    public UserDetailsServiceImpl(UserRepository userRepository)
    {
        this.userRepository = userRepository;
    }

    @Override
    public UserDetails loadUserByUsername(String username) throws UsernameNotFoundException
    {
        User user = userRepository.findByUsername(username);
        if(user == null)
        {
            throw new UsernameNotFoundException(username);
        }
        return new org.springframework.security.core.userdetails.User(user.getUsername(), user.getPassword(), Collections.emptyList());
    }
}

Demo

With the all the code changes, now we are ready to create a user, login and access secured REST APIs. From the image above, a user gets Access Denied error for accessing secured APIs. To demo this, I have already registered a user with username test1 and password test@123.

The POST request for login will give us Authorization token in response. Now using this token in our GET request to retrieve companies data. This GET request will look like below:

In this way, we showed how to secure REST API using JSON web token.

The original blog post was posted on my blog betterjavacode

DEV Community: Yogesh Mali

Upload File to S3 Using NestJS Application

Introduction

Simple Storage Service

Setting up an S3 bucket

NestJS Application

1. Set up Application

2. Set up environment

3. Upload API

4. FileInterceptor for File Upload

5. Create a FileUploadService

6. Final Demo

Conclusion

How to use Circuit Breaker in a Spring Boot Application

What is Circuit Breaker?

Resilience4J Library

Different configurations of Resilience4j Library

Count-Based Circuit Breaker

Time-Based Circuit Breaker

Example of Circuit Breaker in Spring Boot Application

Demo

How to handle OPEN circuit breakers?

Conclusion

Simplifying Spring Security

Why I wrote this book?

What do I cover?

Why should you buy this book?

Step by Step Spring Batch Tutorial

What is Spring Batch?

What is Batch Processing?

Spring Batch Framework

Step by Step Spring Batch Tutorial with an example

Create DTO Object

Setting up Spring Batch Configuration

Configuring Application Properties

Demo

What more?

Conclusion

Spring Boot CRUD Application Example with MongoDB

Introduction

What is MongoDB?

Prerequisites

Spring Boot CRUD Application

How to use Spring Boot CRUD with MongoDB

Define Data Model

Add Repository Interface

Add Spring REST API Controller

The Complete Demo

Conclusion

Top 21 Spring Boot Interview Questions

Spring Boot Interview Questions for Novice Developers

1. What is Spring Boot and How is it useful in Web Application Development?

2. What are the differences between Spring Boot and Spring?

3. How can you set up the Spring Boot application with maven or gradle?

4. How can you change the default port of Spring Boot Application?

5. How would you set up an application with HTTPS port with Spring Boot?

6. What embedded servers does Spring Boot support?

7. What are the advantages of Spring Boot?

Spring Boot Interview Questions for Intermediate Developers

8. How to deploy Spring Boot Web Application as a jar and war files?

9. Why do we use the annotation @SpringBootApplication in the main Spring Boot application class?

10. What are @RestController and @RequestMapping annotation used for?

11. Why do we need Spring Profiles?

12. How to disable specific auto-configuration?

13. What is Thymeleaf and how to use it?

14. What are the different Spring Boot starters out there and what purpose you can use them?

Spring Boot Interview Questions for Experienced Developers

15. How do you connect Spring Boot Application to the database using JPA?

16. Why do you use the Spring Boot Actuator?

17. What is Spring Boot Devtools used for?

18. What is the advantage of using Spring Data JPA?

19. Explain Spring Boot supports relaxed binding.

20. If you want to use Spring Security in your application, how do you use it, and how it makes the application secure?

21. What do you use the annotation @ControllerAdvice for?

Microservices Interview Questions

1. What is Microservices?

2. What are the common tools used to build and deploy microservices?

3. How does Spring help in Microservices development?

4. What are the challenges with Microservices?

Conclusion

9. Why do we use the annotation `@SpringBootApplication` in the main Spring Boot application class?

10. What are `@RestController` and `@RequestMapping` annotation used for?

21. What do you use the annotation `@ControllerAdvice` for?