DEV Community: Suraj

How to update the default exception body thrown by Sping-Boot

Suraj — Sun, 14 Jun 2020 08:40:38 +0000

The default exception in spring, if thrown (and not handled by ControllerAdvice) follows the exception format as

{
    "timestamp": "2020-06-14T05:46:37.538+00:00",
    "status": 401,
    "error": "Unauthorized",
    "message": "Unauthorized",
    "path": "/path_requested" 
}

To change this default exception body to return your own custom fields, you just have to override the BasicErrorController(provided by the framework) and its error method.

How to override

@RestController
@RequestMapping({"${server.error.path:${error.path:/error}}"})
@Slf4j
public class BasicErrorControllerOverride extends AbstractErrorController {

  public BasicErrorControllerOverride(ErrorAttributes errorAttributes) {
    super(errorAttributes);
  }

  @RequestMapping
  public ResponseEntity<Map<String, Object>> error(HttpServletRequest request) {
    HttpStatus status = this.getStatus(request);
    Map<String, Object> errorCustomAttribute = new HashMap<>();
    errorCustomAttribute.put("custome_message", "error");
    errorCustomAttribute.put("custom_status_field", status.name());
    return new ResponseEntity(errorCustomAttribute, status);
  }

  @Override
  public String getErrorPath() {
    return "/error";
  }
}

How the error will now look like:

{
    "custome_message": "UNAUTHORIZED",
    "custom_status_field": "error"
}

🍻 If you enjoyed this story, please click the ❤️ button and share it to help others find it! Feel free to leave a comment below.

Grouping APIs in Swagger

Suraj — Tue, 09 Jun 2020 20:16:18 +0000

We develop and manage lot of APIs but when it comes to sharing apis to other teams, most used approach is to expose them using swagger.

But, listing so many APIs in a single page in swagger can create confusion for the consumer. And also, there are some critical APIs that might change our database state or some configuration and we might not want to expose them.

To solve this, we can group APIs in Swagger (using springdoc-openapi) so that only limited APIs are displayed in one group.

Let' see how to accomplish this using spring-boot and springdoc-openapi.

Include springdoc-openapi-ui in pom.xml



        <dependency>
            <groupId>org.springdoc</groupId>
            <artifactId>springdoc-openapi-ui</artifactId>
            <version>1.4.0</version>
        </dependency>

Make a simple ToDoController with 6 APIs i.e 2 for user,1 for admin, 3 for ops



@RequestMapping("/todo")
@RestController
public class ToDoController {

  private final Map<Integer, String> todoMap = new HashMap<>(30);
  private final Random random = new Random();

  @GetMapping(value = "/user")
  @Operation(summary = "Get All ToDo List")
  public Map<Integer, String> getAllTodoList() {
    return todoMap;
  }

  @GetMapping(value = "/user/{todo_id}")
  @Operation(summary = "Get ToDo item for todo id")
  public String getTodo(@PathVariable("todo_id") Integer todoId) {
    return todoMap.get(todoId);
  }

  @PostMapping(value = "/admin")
  @Operation(summary = "Post new ToDo")
  public Integer postTodo(@RequestParam("todo") String todo) {
    int todoId = random.nextInt();
    todoMap.put(todoId, todo);
    return todoId;
  }

  @GetMapping(value = "/operation")
  @Operation(summary = "Get All ToDo List")
  public Map<Integer, String> getAllTodoListByOps() {
    return todoMap;
  }

  @GetMapping(value = "/operation/{todo_id}")
  @Operation(summary = "Get ToDo item for todo id")
  public String getTodoByOps(@PathVariable("todo_id") Integer todoId) {
    return todoMap.get(todoId);
  }

  @PostMapping(value = "/operation")
  @Operation(summary = "Post new ToDo")
  public Integer postTodoByOps(@RequestParam("todo") String todo) {
    int todoId = random.nextInt();
    todoMap.put(todoId, todo);
    return todoId;
  }
}

Create three GroupedOpenApi beans i.e for user,admin and ops



  @Bean
  GroupedOpenApi userApis() { // group all APIs with `user` in the path
    return GroupedOpenApi.builder().group("user").pathsToMatch("/**/user/**").build();
  }

  @Bean
  GroupedOpenApi adminApis() { // group all APIs with `admin` in the path
    return GroupedOpenApi.builder().group("admin").pathsToMatch("/**/admin/**").build();
  }

  @Bean
  GroupedOpenApi opsApis() { // group all APIs with `operation` in the path
    return GroupedOpenApi.builder().group("operation").pathsToMatch("/**/operation/**").build();
  }

Note -> pathsToMatch() in above code accepts a regular expression of the API path which we are grouping.

Now, run spring-boot application and open localhost:8080/swagger-ui.html

Let's see the three groups created in Swagger

User Group

Operation Group

Admin Group

Complete Code 👇

s2agrahari / grouping-in-swagger-springdoc-openapi

grouping-in-swagger-springdoc-openapi

🍻 If you enjoyed this story, please click the ❤️ button and share it to help others find it! Feel free to leave a comment below.

Authentication between microservices - Part II

Suraj — Wed, 03 Jun 2020 12:04:52 +0000

This is a continuation of Authentication between microservices - Part I.

Please read part I to know the idea behind this implementation.

We will use spring-boot to make two services and see how authentication can be implemented.

Let's build caller service A

First, configure the keys in application.yml that will be used to create JWT token

service:
  jwtAlgorithmKey: sampleJwtAlgorithmKey #will be used as key in JWT Algo
  serviceBSecretKey: serviceB-SecretKey-To-Be-Provided-By-ServiceA #this is the key that called service (`service B`) will compare on decoding token

spring:
  application:
    name: service-a #application-name

Make a config class which will generate jwt tokens at the time of service startup using the values provided in application.yml file

@Configuration
@ConfigurationProperties(prefix = "service")
public class ServiceAuthConfig {

  @Setter @Getter private String jwtAlgorithmKey;

  @Setter @Getter private String serviceBSecretKey;

  @Value("${spring.application.name}")
  private String applicationName;

  @Getter private String serviceBAuthToken;

  /*
    Generate auth token which will be passed in header when calling 
    API of service B
   */
  @PostConstruct
  public void load() {
    serviceBAuthToken = createJwtToken(serviceBSecretKey);
  }

  /*
   Create JWT Token with the algorithm key (common for both the called and caller service)
   Service Secret key is to be generate by caller service and then shared
   and stored in called service yml
  */
  private String createJwtToken(String serviceSecretKey) {
    return JWT.create()
        .withIssuer(applicationName)
        .withSubject(serviceSecretKey)
        .sign(Algorithm.HMAC512(jwtAlgorithmKey));
  }
}

Make a feign client to invoke service B's apis

@FeignClient(name = "serviceBClient", url = "http://127.0.0.1:8081/service-b")
public interface ServiceBFeignClient {

  @GetMapping(value = "/get-greetings")
  @ApiOperation("Get Greetings from service B")
  String getGreetings(@RequestHeader("SERVICE-AUTH-TOKEN") String serviceAuthToken);
}

Make an api which will call service B's getGreetings API

@RestController
@RequestMapping(value = "/service-a")
@Slf4j
public class ServiceAController {

  @Autowired private ServiceAuthConfig serviceAuthConfig;

  @Autowired private ServiceBFeignClient serviceBFeignClient;

  @GetMapping(value = "/get-greeting-from-service-b")
  public ResponseEntity getGreetingFromServiceB() {
    try {
      String serviceBGreetingResponse =
          serviceBFeignClient.getGreetings(serviceAuthConfig.getServiceBAuthToken()); // passing jwt auth token for service B
      return ResponseEntity.ok(serviceBGreetingResponse);
    } catch (FeignException exception) { 
      /*
        Extract the error thrown from service B 
        and if the error is `INVALID_TOKEN`, then return 401
       */
      String error = new String(exception.responseBody().get().array());
      log.error("Error {}", error);
      if ("INVALID_TOKEN".equals(error)) {
        return ResponseEntity.status(401).body(error);
      }
      throw exception;
    }
  }
}

Now, Let's create a called service B

Configure the jwt algo key and the secret key provided by caller service A in application.yml

service:
  jwtAlgorithmKey: sampleJwtAlgorithmKey
  registeredSecretKeys:
    - serviceB-SecretKey-To-Be-Provided-By-ServiceA

spring:
  application:
    name: service-b

Expose an api which will generate a greeting

@RestController
@RequestMapping(value = "/service-b")
public class ServiceBController {

  @GetMapping(value = "/get-greetings")
  public ResponseEntity sayGreeting() {
    return ResponseEntity.ok("Hello There from Service B!!!");
  }
}

Create a config component to auto read the application.yml properties

@Configuration
@ConfigurationProperties(prefix = "service")
@Slf4j
@Setter
@Getter
public class ServiceAuthConfig {

  private String jwtAlgorithmKey;

  private List<String> registeredSecretKeys;
}

Finally, create a ServiceAuthFilter which will read the token from SERVICE-AUTH-TOKEN header, decode and validate it against the keys stored in config

@Component
@Slf4j
public class ServiceAuthFilter implements Filter {

  @Autowired private ServiceAuthConfig serviceAuthConfig;

  @Override
  public void doFilter(
      ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain)
      throws IOException, ServletException {

    HttpServletRequest httpServletRequest = ((HttpServletRequest) servletRequest);
    String serviceAuthToken = httpServletRequest.getHeader("SERVICE-AUTH-TOKEN");
    boolean isTokenValid = validateJwtToken(serviceAuthToken);
    if (!isTokenValid) {
      HttpServletResponse httpServletResponse = (HttpServletResponse) servletResponse;
      httpServletResponse.setStatus(401);
      httpServletResponse.getWriter().write("INVALID_TOKEN");
    } else {
      filterChain.doFilter(servletRequest, servletResponse);
    }
  }

  /*
   Validate the token by comparing the secret key in it against list 
    of secret keys registered in service B
  */
  private boolean validateJwtToken(String jwtToken) {
    if (StringUtils.isEmpty(jwtToken)) {
      return false;
    }
    DecodedJWT decodedJWT =
        JWT.require(Algorithm.HMAC512(serviceAuthConfig.getJwtAlgorithmKey()))
            .build()
            .verify(jwtToken);
    log.info("Request Initiated from {}", decodedJWT.getIssuer());
    return serviceAuthConfig.getRegisteredSecretKeys().contains(decodedJWT.getSubject());
  }
}

Now, how to validate what has been built?

Build and start both, service A and service B.
Invoke get-greeting-from-service-b api from caller service A and see the output by keeping the secret keys same in both services and then by changing it in one of them.

s2agrahari / authentication-between-microservices

authentication-between-microservices

View on GitHub

🍻 If you enjoyed this story, please click the ❤️ button and share it to help others find it! Feel free to leave a comment below.

Authentication between microservices - Part I

Suraj — Wed, 03 Jun 2020 12:04:37 +0000

How to establish your own custom authentication between microservices?

What and Why? 🤔

Let's assume the interaction between 3 microservices.

User Service - Handles user's registration/login and other user data.
Product Service - Handles the product list present in the system.
Order Service- Handles Order, invokes product service to validate the order and invokes user service to attach required user-details to the order.

Why Authentication is needed?

Let's say you want to keep control over which service is calling another service.

Example here - User service shouldn't be calling order service or product service. And similarly, product service shouldn't be calling order service.

Explanation - Sometimes, a service ends up invoking an API of other services, which it should not (because that is not the service responsibility or may be to implement a new feature, etc).
Say, user-service starts invoking orders API of order-service to know if a user has placed order and wants to send an email to that user. And similarly, any other service can call this API of order-service.
But say, if we want to know, what all services call this order API, we won't be able to tell. To do this kind of tracing or to
restrict this type of API calls, order-service service should know from which service the call is originating.

But how will one service know, that it shouldn't allow any other service to use it's APIs except a few?
If you are using container orchestration like k8s, this could be achieved by using namespace restriction.

But how to do it independent of k8s?

Let's make one more assumption here:

Service A - Caller service
Service B - Called service i.e interaction happening from service A->B

For B to know if A is authorized to invoke its APIs, we can deploy one of the logics which is used for user authorization i.e sharing an authorization token while calling each API.

Now, who will own the generation of the authorization token?

Each caller service(service A) will generate a token (using a secret key) and pass to the called service (service B).

Note - Secret key that caller service (service A) uses to generate the token will also be present with called service (service B) for validation.

How the called service will validate the token it receives?

The called service(service B) will validate by decoding the token it received and comparing the secret key in the token with the stored secret key list.

How to implement it? Authentication between micro-services - Part II

If you enjoyed this story, please click the ❤️ button and share it to help others find it! Feel free to leave a comment below.

Global Header in Swagger-Ui Spring-Boot

Suraj — Sat, 23 May 2020 11:49:02 +0000

This article is about adding custom headers in all Apis globally without writing in each API method.

Why do we need it?

There are cases where we need some header value from consumers to validate the request.

Example - Basic-Auth-Token, which can be validated from a servlet filter, so we don’t need to add it as a header parameter in each API method.

But, on exposing the API, we would want that the Basic-Auth-Token parameter reflects in each API under Swagger-UI to make testing easier.

Like in below image

So, let's get on how to do it

The first step would be to create a docket bean as below



  @Bean
  public Docket docket() {
    return new Docket(DocumentationType.SWAGGER_2)
        .forCodeGeneration(true)
        .globalOperationParameters(globalParameterList()) 
// ^ here we are passing the global paramters to inlcude in swagger-ui
        .select()
        .apis( RequestHandlerSelectors.withClassAnnotation(RestController.class))
        .paths(PathSelectors.any())
        .build();
  }

Make sure that @EnableSwagger2 is added in the Application file

Now, let's add global parameters that we need to pass in above docket bean



  private List<Parameter> globalParameterList() {

    val authTokenHeader =
        new ParameterBuilder()
            .name("AUTH-TOKEN") // name of the header
            .modelRef(new ModelRef("string")) // data-type of the header
            .required(true) // required/optional
            .parameterType("header") // for query-param, this value can be 'query'
            .description("Basic Auth Token")
            .build();

    return Collections.singletonList(authTokenHeader);
  }

Add a RequestFilter to log the incoming request header that we just added.



@Component
@Slf4j
public class RequestFilter implements Filter {

  @Override
  public void doFilter(
      ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain)
      throws IOException, ServletException {
    log.info(
        "Value of AUTH-TOKEN  -> {}",
        ((HttpServletRequest) servletRequest).getHeader("AUTH-TOKEN"));
    filterChain.doFilter(servletRequest, servletResponse);
  }
}

Finally, a simple hello-world controller to test what we just made.



@RestController
@RequestMapping("/hello-world")
public class HelloWorldController {

  @GetMapping
  public ResponseEntity sayHello() { // No header needed here - that got covered in docket
    return ResponseEntity.ok("Hello There!!!");
  }
}

How the output will look like

Sample Project can be found at

s2agrahari / swagger-global-header-spring-boot

Global Header in Swagger without writing in Controller

swagger-global-header-spring-boot

Global Header in Swagger without writing in Controller

Blog link - https://dev.to/s2agrahari/global-header-in-swagger-ui-spring-boot-5188

View on GitHub

If you enjoyed this story, please click the ❤️ button and share it to help others find it! Feel free to leave a comment below.

Moving from Hibernate Envers to Debezium

Suraj — Mon, 06 Jan 2020 17:35:09 +0000

This article is about logging data change events asynchronously.

What is Hibernate Envers ?

Envers provide an easy auditing/versioning solution for entity classes.

Why to move from envers?

Envers is a synchronous audit logging mechanism. So, there is an additional commit transaction required for the log table created by envers. This takes an additional millisecond for the write operation.

And, if there is an error while writing into the log table, then whole data is rolled back (even the actual changes).

What is Debezium ?

Debezium is an open source distributed platform for change data capture.

It lets your apps react every time your data changes. It continuously monitors your databases and streams every row-level change in the same order they were committed in the database to any streaming platform i.e Kafka

It uses Kafka-Connect to stream captured events to Kafka and then from Kafka to other systems.

How debezium uses Kafka connect to stream data change events.

Why Debezium ?

Data change is captured asynchronously
Reads from the WAL. All changes, even the ones made from SQL console will be captured.
Has a range of connectors to capture from (MySql, Postgres) and sink data to (MySql, Postgres, Elastic, etc)

Let's start on a POC, where we will capture data change events from MySql DB and then store the emitted events in a different MySql DB.

We need to first install/download

Steps

Extract the downloaded Debezium Connector to get the jars.
Move the extracted jars to any classpath as specified in plugin.path
variable in /kafka/bin/config/connect-distributed.poperties file.
Or, move the jars to Kafka libs directory

mv debezium-connector-mysql/* kafka/libs/
Extract the Kafka Jbdc Sink Connector jars
and place them at the same location as we did for Debezium Connector jars.

Start zookeeper. Zookeeper is shipped with Kafka, so we don't need to download separately.

/bin/zookeeper-server-start.sh /config/zookeeper.properties

Start Kafka

/bin/kafka-server-start.sh /config/server.properties

Start kafka-connect

/bin/connected-distributed.sh /config/connected-distributed.properties

Now we have all the tools needed installed and running.

Let's create connectors

Let us assume the default initial data in the employee table be as shown in the above fig.

Give following access to debezium user, so that it can scan and create a snapshot of the table and database at the time of creating a connector.

GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'debezium'@'%'

We will first create a source connector that will capture the data change events of the employee table.

curl -X POST \ http://localhost:8083/connectors/ \ -H 'Content-Type: application/json' \ -H 'Postman-Token: f4367baf-f2b3-4763-84d4-21ae2f68faeb' \ -H 'cache-control: no-cache' \ -d '{ "name": "producer-connector", "config": { "connector.class": "io.debezium.connector.mysql.MySqlConnector", "database.hostname": "127.0.0.1", "database.port": "3306", "database.user": "debezium", "database.password": "dbz", "database.server.id": "12345", "database.server.name": "localhost", "database.history.kafka.bootstrap.servers": "localhost:9092", "database.history.kafka.topic": "dbhistory", "include.schema.changes": "true", "table.whitelist": "employee", "transforms": "Reroute", "transforms.Reroute.type":"io.debezium.transforms.ByLogicalTableRouter", "transforms.Reroute.topic.regex": "([^.]+)\\.([^.]+)\\.([^.]+)", "transforms.Reroute.topic.replacement": "$3" } }'

Little about not so obvious input data

i) Data change for each table is published in Kafka topic in the format
server_name.database_name.table_name.
ii) So, here we will transform the topic before they are created using transform. The new topics that will be created will be of form table_name

We now will create the sink connector which will consume events from the employee topic and persist in the sink DB

curl -X POST \ http://localhost6:8083/connectors/ \ -H 'Content-Type: application/json' \ -H 'Postman-Token: 453bb3ff-ac42-4934-a45f-4afb51052609' \ -H 'cache-control: no-cache' \ -d '{ "name": "sink-connector", "config": { "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector", "tasks.max": "1", "connection.url": "jdbc:mysql://localhost:3306/sink_db?user=sink&password=snk", "transforms": "unwrap", "transforms.unwrap.type": "io.debezium.transforms.UnwrapFromEnvelope", "auto.create": "true", "insert.mode": "insert", "topics.regex": "employee" } }'

Now, let's make a few updates to the employee table

update employee set salary = 10000 where emp_id = 1;
update employee set emp_name = 'CLAIRE' where emp_id = 3;

You can see in below image that the data changes are captured for the above two update queries

Logged data change events