DEV Community: sanjay shivanna

Springboot+Docker+Minikube+Kubectl - Continuous Integration(CI) Local

sanjay shivanna — Fri, 08 Oct 2021 13:15:25 +0000

In this blog, we will try to learn springboot with docker and deploying it to local k8s cluster hosted with minikube.

Setup

Make sure you have minikube installed
Make sure you have kubectl installed
Make sure you have docker installed
Dockerhub account to push the images

Refer my blog for installing via chocolatey for windows

Steps

1.Create springboot project with following dependencies
from start.spring.io
2.Sample Docker file reference to create the image

FROM openjdk:13
EXPOSE 9000
ARG JAR_FILE= ./target/spring-dockr-k8s-0.0.1-SNAPSHOT.jar
ADD ./target/spring-dockr-k8s-0.0.1-SNAPSHOT.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

Pom file changes to build and push the image to dockerhub

<plugin>
 <groupId>com.spotify</groupId>
 <artifactId>dockerfile-maven-plugin</artifactId>
 <version>${dockerfile-maven-version}</version>         <executions>
    <execution>
     <id>default</id>
     <goals>
      <goal>build</goal>
      <goal>push</goal>
     </goals>
     </execution>
</executions>
    <configuration>
         <username>username</username>
    <password>******</password>
    <repository>sanjaybsm/spring-dockr-k8s</repository>
    <tag>latest</tag>
       <buildArgs>
    <JAR_FILE>${project.build.finalName}.jar</JAR_FILE>
    </buildArgs>
    </configuration>
    </plugin>

Build, Install and Push

If your pom changes are correct then,

mvn clean install' should already build an image for you, and you should see below logs.[INFO] Successfully built sanjaybsm/spring-dockr-k8s:latest`
Push
Below snap from inteliij, but you can manually also run the commands from command line to push the image,

[INFO] Image 54ab11ed5591: Preparing [INFO] Image b8cb43830a76: Preparing [INFO] Image 93c304238ab8: Preparing [INFO] Image 12a9cd7d069e: Preparing [INFO] Image 93c304238ab8: Layer already exists [INFO] Image b8cb43830a76: Layer already exists [INFO] Image 54ab11ed5591: Pushing [INFO] Image 12a9cd7d069e: Layer already exists [INFO] Image 54ab11ed5591: Pushed [INFO] latest: digest: sha256:f2ddde51cbd4005ec90c25698649e1cf960cd1fbeb308f1de4b05b9a53606c48 size: 1166

Start minikube and run the image locally

Start minikube minikube start

Start the minikube tunnel so that you can access the app outside k8s cluster minikube tunnel
keep the tunnel open, and this change is needed for windows.

Create deployment
kubectl create deployment testspringk8s --image={dockerhub_username}/{image_name}:{version}
e.g.kubectl create deployment testspringk8s --image=sanjaybsm/spring-dockr-k8s

Expose Deployment
kubectl expose deployment testspringk8s --type=LoadBalancer --port=9000

Run this command to know the externalIp assigned
kubectl get svc
It should show something like below

if it's pending it means that your tunnel is not running or some issue with your minikube

Happy learning!

reference

https://minikube.sigs.k8s.io/docs/handbook/accessing/

Installing minikube,kubectl with chocolatey - windows

sanjay shivanna — Mon, 04 Oct 2021 19:00:49 +0000

In this blog we will try to learn how to install minikube and kubectl for windows.

The easiest for any software to be installed on windows would be to use chocolatey

One instance chocolatey immense helped us was in a situation where we were asked to switch laptop with better configuration. In an enterprise you would need to download software from 'software centre' or ask IT teams to install it for you. In this case we provided this case we provided the chocolatey script to admin team to run it as a script! it helped both the teams i.e. dev and IT team.

Coming back on how to install minikube..., it's very easy, run below command.

choco install minikube

make sure you run the above command in powershell with admin privileges.

Similarly you can install kubectl using below command.

choco install kubernetes-cli

Basically you can install any software for windows easily using chocolatey, above are just example.

https://community.chocolatey.org/

Springboot Project Structure

sanjay shivanna — Wed, 28 Apr 2021 12:32:16 +0000

In this short article we will try to explore some of the discussions around best practices for packages

Project by code structure

Above structure defined is more a classic structure followed mostly :) ... (At least on the projects that I worked on)

Project by feature structure

package by feature

Follow this article which more are less explains the spring pet clinic project structure.
https://www.baeldung.com/spring-boot-package-structure

package by code structure

Below is the good article I found about project by code structure, if you find anything better, please comment.
https://medium.com/the-resonant-web/spring-boot-2-0-project-structure-and-best-practices-part-2-7137bdcba7d3

which ever approach you choose from above for enterprise you would like to standardize the project structure for all the development team and provide some template classes. If you are using maven you can do this by arch-type.

Maven Archtype

We can use maven archtype to create structure. Personally I feel it's important in enterprise to have a common structure defined for consistency and readability perspective.

Good Articles to follow for creating a maven archtype.

https://rieckpil.de/create-your-own-maven-archetype-in-5-simple-steps/

https://maven.apache.org/guides/mini/guide-creating-archetypes.html

https://medium.com/@inthiraj1994/creating-your-own-archetype-840136fc7818

https://www.javacodegeeks.com/2019/05/creating-custom-maven-archetype.html

you can also use IntellIJ feature for storing project templates locally for future reference.
https://www.jetbrains.com/help/idea/saving-project-as-template.html

In this article I try to summarize the package structure and how we can use maven archtype to create our package structures and templates.

Timeouts and how to handline in Java

sanjay shivanna — Fri, 26 Mar 2021 13:10:22 +0000

In this article we will try to cover why it’s important to define timeouts for out bound rest calls.

Before configuring any timeout let’s understand below some common exceptions for http outbound calls,

Connection timeout

maximum time to wait for the other side to answer "yes, I'm here, let's talk" when creating a new connection, (ConnectTimeout eventually calls socket.connect(address, timeout), If the connection is not established within the ConnectTimeout specified by you or the library you are using then, you get an error 'connect timedout

Socket timeout

Is the timeout for waiting for data or, put differently, a maximum period inactivity between two consecutive data packets

Read timeout

Read timeout can happen when there is successful connection established between client and the server and there is an inactivity between data packets while waiting for the server response.

Write timeout

Similar to Read timeout, write timeout can happen when there is successful connection established between client and server, and there is inactivity between data packets while sending the request to the server.

The important topic to remember here is that based on the choice of library we use for outbound calls it’s very important that we configure the properties to handle the above mentioned one’s and handle the exception gracefully.

Apache Camel

If we are using Apache Camel ‘http’ component to make the outbound calls then we can configure these properties in following ways, please note that if we don’t define this properties the default values is -1! means connection will never timeout and can have advert effect on the application performance overall.

camel.property

 http.urlProxy = http4://ThirdPartyServers?throwExceptionOnFailure=false&httpClient.socketTimeout=${rest.timeout.millis}&httpClient.connectTimeout=${rest.timeout.millis}

Route Builder Class

    @Override
    public void configure() throws Exception {

        configureTimeout();

    }

    private void configureTimeout() {
        HttpComponent httpComponent = getContext().getComponent("http4", HttpComponent.class);
        httpComponent.setConnectionTimeToLive(VALUE_IN_MILI);// for closing the idle connection - in milliseconds
        httpComponent.setSocketTimeout(VALUE_IN_MILI); //socket timeout - in milliseconds
        httpComponent.setConnectTimeout(VALUE_IN_MILI); // connection timeout - in milliseconds*/
    }

Rest Template

In a similar manner for rest template we could define as follows,

final RequestConfig requestConfig = RequestConfig.custom()
                .setConnectionRequestTimeout(VALUE_IN_MILI)
                .setConnectTimeout(VALUE_IN_MILI)
                .setSocketTimeout(VALUE_IN_MILIs)
                .build();

        final HttpClient httpClient = HttpClients.custom()
                 .setConnectionTimeToLive(VALUE_IN_MILI, SECONDS)
                .setRetryHandler((IOException exception, int executionCount, HttpContext context) -> {
                    return executionCount <= 3;
                })
                .setServiceUnavailableRetryStrategy(new DefaultServiceUnavailableRetryStrategy(3, 1))
                .setDefaultRequestConfig(requestConfig)
                .build();

        final RestTemplate restTemplate = new RestTemplate(new HttpComponentsClientHttpRequestFactory(httpClient));

WebClient

And in a similar manner below is how we could define it in web client,

    public WebClient getWebClient()
    {
        HttpClient httpClient = HttpClient.create()
                .tcpConfiguration(client ->
                        client.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, VALUE_IN_MILI)
                        .doOnConnected(conn -> conn
                                .addHandlerLast(new ReadTimeoutHandler(rest.timeout.millis))
                                .addHandlerLast(new WriteTimeoutHandler(rest.timeout.millis))));

        ClientHttpConnector connector = new ReactorClientHttpConnector(httpClient.wiretap(true));       

        return WebClient.builder()
                .baseUrl("http://localhost:3000")
                .clientConnector(connector)
                .defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
                .build();
          }

Summary

All in all it’s very important to configure these values i.e. connection timeout, read timeout, socket timeout etc so as to terminate the connections after waiting for a specific amount of time rather keeping the connection open indefinitely which can bring issues to overall application performance and stability.

References

HTTP4 :: Apache Camel

A Guide to the RestTemplate | Baeldung

Moving from Java8 to JavaX...

sanjay shivanna — Thu, 25 Feb 2021 12:43:30 +0000

I would like to keep this article to list out following

Key features in each version beginning from Java9 till Java12.
Challenges faced in migration from Java8 to JavaX
Good Blogs and Vlog available to refer to while migrating.

Refer this wiki page for complete details on JAVA releases and features and timelines.

Key Features by release:

Java9

Java Module System
Linking
Jshell - runtime console
Javadoc improvments
Collection Factory Methods ex: List stringsOf = List.of("apple","orange","grapes");
Stream API improvements
private interface methods
Http - new classes to make request/response
Multi release Jar - where you can specify which class file to pick based on the JRE.

Good Blogs to refer pluralsight, baeldung and journaldev

Java 10:

Local-Variable Type Inference ex: var = "number", var = 1;
Garbage-Collector Interface
Parallel Full GC for G1
Application Class-Data Sharing
Thread-Local Handshakes
Remove the Native-Header Generation Tool (javah)
Additional Unicode Language-Tag Extensions
Heap Allocation on Alternative Memory Devices
Experimental Java-Based JIT Compiler ex: Graal vm
Root Certificates
Time-Based Release Versioning

Good blogs : Journaldev

Java11:

Java11 is long term support(LTS) release. If we download JDK11 from oracle and use it in production then it's enterprise level and have to be paid.

Nest-Based Access Control
Dynamic Class-File Constants
Improve Aarch64 Intrinsics
Epsilon: A No-Op Garbage Collector
Remove the Java EE and CORBA Modules
HTTP Client (Standard)
Local-Variable Syntax for Lambda Parameters
Key Agreement with Curve25519 and Curve448
Unicode 10
Flight Recorder
ChaCha20 and Poly1305 Cryptographic Algorithms
Launch Single-File Source-Code Programs
Low-Overhead Heap Profiling
Transport Layer Security (TLS) 1.3
ZGC: A Scalable Low-Latency Garbage Collector
(Experimental) -
Deprecate the Nashorn JavaScript Engine
Deprecate the Pack200 Tools and API
Java12:
Shenandoah: A Low-Pause-Time Garbage Collector (Experimental)
Microbenchmark Suite
Switch Expressions (Preview)
JVM Constants API
One AArch64 Port, Not Two
Default CDS Archives
Abortable Mixed Collections for G1
Promptly Return Unused Committed Memory from G1

Good Blogs : whizlabs journaldev

Java13:

Dynamic CDS Archives
ZGC: Uncommit Unused Memory
Reimplement the Legacy Socket API
Switch Expressions (Preview)
Text Blocks (Preview)

Spring boot +Docker+ Gitlab - Continuous Integration(CI)

sanjay shivanna — Fri, 12 Feb 2021 16:11:57 +0000

A simple example to demonstrate, how we can use spring boot, docker and gitlab for build and test using maven, create an image and push it to gitlab repository.

Step 1 - Create a Gitlab account
If you already familiar with gitlab then feel free to skip this section, otherwise create a gitlab account by following this link Link

Step 2 - Create a New Project in Gitlab
Once you successfully created Gitlab account then create a new project

Step 3 - Create a New Spring boot project
Simple and easiest way to create spring boot project is to visit start.spring.io

Once you have the project created, use the choice of your editor to open the project.

Step 4 - Add Dockerfile and Update gitlab.ci.yml file
Docker file

FROM openjdk:13-alpine
Volume /tmp
ADD /target/*.jar springbootgitlab-0.0.1-SNAPSHOT.jar
ENTRYPOINT ["java","-jar","/springbootgitlab-0.0.1-SNAPSHOT.jar"]

gitlab-ci yml file

image: gitlab/dind
services:
  - docker:dind

variables:
  DOCKER_DRIVER: overlay
  SPRING_PROFILES_ACTIVE: gitlab-ci
  USER_GITLAB: sanjaybsm
  APP_NAME: springbootgitlab
  REPO: springbootlabtest

stages:
  - build
  - test
  - docker

maven-build:
  image: maven:3-jdk-8
  stage: build
  script: "mvn clean package -B"
  artifacts:
    paths:
      - target/*.jar

maven-test:
  image: maven:3-jdk-8
  stage: test
  script: "mvn test"
  artifacts:
    paths:
      - target/*.jar

docker-build:
  stage: docker
  script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker build -t registry.gitlab.com/yournameingitlab/springbootlabtest .
    - docker push registry.gitlab.com/yournameingitlab/springbootlabtest

please note test mvn test is added just as an example

For above project I used packaging as jar since my dockerfile and gitlab yml files configured like that.

Step 5 - Commit and Push to gitlab!
git add .
git commit -m "initial commit"
git push origin master

After above step, you should see your gitlab pipeline kicking in!!

and also you should see your image available in container registry!

Voila!! you have successfully used gitlab ci/cd pipeline to build, test, create image and push image to gitlab container registry!

In the next blog I will try to explain how we can download docker image from gitlab and run it from local machine!

Java Streams filter, distinct and map

sanjay shivanna — Wed, 19 Aug 2020 08:07:16 +0000

Simple example to illustrate using Java stream with filter,distinct and map.

 public static void main(String[] args) {
        Person lokesh = new Person(1, "Lokesh", "Gupta");
        Person brian = new Person(2, "Brian", "Clooney");
        Person alex = new Person(3, "Alex", "Kolen");
        Person ram = new Person(1, "Ram", "Mani");
        Person rahim = new Person(2, "Rahim", "sab");
        Person jhon = new Person(3, "Jhon", "son");

        List<String> personList = Arrays.asList("Lokesh", "Alex", "Peter");


        Collection<Person> list = Arrays.asList(lokesh, brian, alex, lokesh, brian, lokesh, ram, rahim, jhon);
        // Filter out fnames and then get distinct lastNames
        List<String> listWithLastName = list.stream()
                .filter(person -> !personList.contains(person.getFname()))
                .distinct()
                .map(person -> person.getLname())
                .collect(Collectors.toList());

        // Out put - [Clooney, Mani, sab, son]
        System.out.println(listWithLastName);
    }

After running above program you will see output like below
[Clooney, Mani, sab, son]

--
create pojo person class with the fields mentioned above

The example above is referred from https://howtodoinjava.com/java8/java-stream-distinct-examples/ and modified to explain filter,distinct and collect.

Spring boot : Rest project with lomback and logback

sanjay shivanna — Sat, 15 Aug 2020 15:50:35 +0000

A simple spring boot REST example with lomback and logback configuration for logging
Pom.xml file

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.3.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.sanjay.springrest</groupId>
    <artifactId>spring-rest</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>spring-rest</name>
    <description>Demo project for Spring Boot rest services</description>

    <properties>
        <java.version>11</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <version>2.3.3.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

Rest Controller class

package com.sanjay.springrest.springrest.rest;

import com.sanjay.springrest.springrest.model.Users;
import com.sanjay.springrest.springrest.service.LoginService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

@Slf4j
@RestController
@RequestMapping("/users")
public class LoginController {

    @Autowired
    LoginService loginService;

    @GetMapping("/all")
    public ResponseEntity getAllUsers() {
        log.debug("Inside GetAll users");
        return new ResponseEntity(loginService.getAllUsers(), HttpStatus.OK);
    }

    @PostMapping()
    public ResponseEntity addUsers(@RequestBody Users user) {
        log.debug("Inside add all users");
        loginService.addUser(user.getEmail(), user.getPassword());
        return new ResponseEntity(HttpStatus.CREATED);
    }

    @GetMapping()
    public ResponseEntity getUser(@RequestParam String email) {
        log.debug("Inside Get user {}", email);
        return new ResponseEntity(loginService.getUser(email), HttpStatus.OK);
    }

    @DeleteMapping
    public ResponseEntity deleteUser(@RequestParam("emailId") String email) {
        log.debug("Delete user {}", email);
        loginService.deleteUser(email);
        return new ResponseEntity(HttpStatus.OK);
    }


}

Service class

package com.sanjay.springrest.springrest.service;

import org.springframework.stereotype.Service;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;

@Service
public class LoginService {

    HashMap<String, String> users = new HashMap<>();

    public void addUser(String email, String password) {
        users.put(email, password);
    }

    public void deleteUser(String email) {
        users.remove(email);
    }

    public List<String> getAllUsers() {
        return new ArrayList<>(users.keySet());
    }

    public String getUser(String email) {
        return users.get(email);
    }

}

Model class with lombock

package com.sanjay.springrest.springrest.model;

import lombok.Data;
import lombok.Getter;
import lombok.Setter;

@Data
public class Users {
    String email;
    String password;

}

logback-spring.xml configuration file

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <property name="LOGS" value="./logs" />

    <appender name="Console"
              class="ch.qos.logback.core.ConsoleAppender">
        <layout class="ch.qos.logback.classic.PatternLayout">
            <Pattern>
                %black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %msg%n%throwable
            </Pattern>
        </layout>
    </appender>

    <appender name="RollingFile"
              class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${LOGS}/spring-boot-logger.log</file>
        <encoder
                class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <Pattern>%d %p %C{1.} [%t] %m%n</Pattern>
        </encoder>

        <rollingPolicy
                class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- rollover daily and when the file reaches 10 MegaBytes -->
            <fileNamePattern>${LOGS}/archived/spring-boot-logger-%d{yyyy-MM-dd}.%i.log
            </fileNamePattern>
            <timeBasedFileNamingAndTriggeringPolicy
                    class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
                <maxFileSize>10MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>
        </rollingPolicy>
    </appender>

    <!-- LOG everything at INFO level -->
    <root level="info">
        <appender-ref ref="RollingFile" />
        <appender-ref ref="Console" />
    </root>

    <!-- LOG "com.sanjay*" at TRACE level -->
    <logger name="com.sanjay" level="trace" additivity="false">
        <appender-ref ref="RollingFile" />
        <appender-ref ref="Console" />
    </logger>

</configuration>

Project structure

Github link for complete project
https://github.com/sanjaybsm/spring-rest

How to verify Mail exchange records(MX) of DNS configuration

sanjay shivanna — Sat, 18 Jul 2020 09:13:25 +0000

This is a short note explaining how to verify your MX records configurations are correct or not using 'nslookup'

C:\Users>nslookup
Default Server: UnKnown
Address: 2a02:2788:fff0:7::3

set q=mx
example.com.
Server: UnKnown
Address: 2a02:2788:fff0:7::3

Non-authoritative answer:
example.com MX preference = 0, mail exchanger = (root)

example.com nameserver = a.iana-servers.net
example.com nameserver = b.iana-servers.net
a.iana-servers.net AAAA IPv6 address = 2001:500:8f::53
b.iana-servers.net AAAA IPv6 address = 2001:500:8d::53
a.iana-servers.net internet address = 199.43.135.53

hope this helps.