DEV Community: Eduardo Issao Ito

Resumo Kubernetes

Eduardo Issao Ito — Tue, 25 Mar 2025 12:56:31 +0000

Configurar cluster

Configurar kubectl para acessar um Azure AKS

az aks get-credentials -g ${GROUP} -n ${CLUSTER}

Ver configurações do kubectl

kubectl config view

Adicionando um novo cluster ao kubectl

# Add a user/principal that will be used when connecting to the cluster
kubectl config set-credentials kubeuser/foo.com --username=kubeuser --password=kubepassword

# Point to a cluster
kubectl config set-cluster foo.com --insecure-skip-tls-verify=true --server=https://foo.com

# This context points to the cluster with a specific user
kubectl config set-context default/foo.com/kubeuser --user=kubeuser/foo.com --namespace=default --cluster=foo.com

# Use this specific context
kubectl config use-context default/foo.com/kubeuser

Trocando de cluster

#  Mostrando os clusters configurados no ~/.kube
kubectl config get-contexts

# Mostrando o contexto atual
kubectl config current-context

# Trocando de contexto
kubectl config use-context CONTEXT_NAME

Namespaces

Criando um namespace

kubectl create namespace ${NAMESPACE}

Listando os namespaces existentes:

kubectl get namespaces

Trocando o namespace corrente

kubectl config set-context --current --namespace=${NAMESPACE}

Pods

Procurar (por label)

kubectl get pods -l app=MY-APP

POD_NAME=$(kubectl get pods -o=jsonpath='{.items[?(@.metadata.labels.app=="MY-APP")].metadata.name}')

Describe (ver eventos)

kubectl describe pod $POD_NAME

Logs (da aplicação)

kubectl logs $POD_NAME

Mostrar os nodes de execução dos pods

kubectl get pods -o=wide

# Filtrar pelo nome do node
kubectl get pods --field-selector spec.nodeName=$NODE_NAME

Terminal

Abrir terminal num pod

kubectl exec --stdin --tty $POD_NAME -- /bin/bash

Se estiver usando um terminal Git-bash ou MinGW, colocar a variável MSYS_NO_PATHCONV:
MSYS_NO_PATHCONV=1 kubectl exec --stdin --tty $POD_NAME -- /bin/bash

Service

Acessando um serviço

kubectl get svc ${SERVICE_NAME}

EXTERNAL_IP=$(kubectl get svc ${SERVICE_NAME} -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')

curl http: ${EXTERNAL_IP}/api/v1/hello

Expondo um endereço:

kubectl port-forward service/${SERVICE_NAME} 9200:9200

Logs

kubectl logs -p $POD --all-containers --previous=false

Restart pods

Sem downtime:

kubectl rollout restart deployment <deployment_name>

Parando todos pods e depois reiniciando todos:

kubectl scale deployment <deployment name> - replicas=0
kubectl scale deployment <deployment name> - replicas=1

Alterando uma variável associada ao pod:

kubectl set env deployment <deployment name> LAST_START_DATE="$(date)"

Selecionando um pod específico:

kubectl delete pod <pod_name>

Selecionando todos pods com label:

kubectl delete pod -l "app:myapp"

Performance

Mostra consumo de CPU e MEMORY de pods

kubectl top pod
kubectl top pod POD_ID

Check for newer versions of dependencies in pom.xml

Eduardo Issao Ito — Wed, 08 May 2024 22:59:30 +0000

I made a script in Groovy language to look for newer versions of all declared dependencies in a Maven project. The script supports multi-module projects.

It searches the current folder for pom.xml files and checks the central.sonatype.com website for a newer versions. Only dependencies that have an explicit version defined in pom will be checked. Versions defined by properties will be resolved if the properties were declared in one of the pom files. (If a property is defined more than once, the result is undefined.)

Example of use:

I recommend installing Java 17 (or newer) and Groovy 4 with sdkman!

Put these two files in a folder that is included in PATH.

check-versions.sh:



#!/usr/bin/env bash
BASEDIR=$(unset CDPATH; cd `dirname $0`; pwd)

JAVA_VER=$(java -version 2>&1 | awk -F '"' '/version/ {print $2}' | awk -F '.' '{sub("^$", "0", $2); print $1$2}')
if [ "$JAVA_VER" -lt 170 ]; then
    echo "Java 17 necessario"
    exit 1
fi

#$PROXY="-DproxySet=true -DproxyHost=10.0.0.24 -DproxyPort=8080"

groovy -Dgroovy.grape.report.downloads=true $PROXY $BASEDIR/check-versions.groovy 2>/dev/null

check-versions.groovy:



import groovy.json.JsonSlurper
import groovy.xml.XmlParser
@Grab('org.jodd:jodd-lagarto:6.0.6')
import jodd.jerry.Jerry
@Grab('org.fusesource.jansi:jansi:2.4.1')
import org.fusesource.jansi.AnsiConsole
@Grab('org.slf4j:slf4j-jdk-platform-logging:2.0.13')
import org.slf4j.Logger

import static org.fusesource.jansi.Ansi.Color.GREEN
import static org.fusesource.jansi.Ansi.Color.RED
import static org.fusesource.jansi.Ansi.ansi

AnsiConsole.systemInstall()

record Dependency(String groupId, String artifactId, String version) {}

static def findAllDependencies() {
    //println("-- findAllDependencies")
    def allDeps = []
    def mvnProperties = [:]
    new File(".").traverse(type: groovy.io.FileType.FILES, nameFilter: ~/pom.xml/) {
        processPom(new XmlParser().parse(it), allDeps, mvnProperties)
    }
    return allDeps.unique().sort { k1, k2 -> k1.groupId + k1.artifactId <=> k2.groupId + k2.artifactId }
}

static def processPom(def project, def allDeps, def mvnProperties) {
    collectProperties(project, mvnProperties)
    collectDependencies(project.'**'.dependencies.dependency, allDeps, mvnProperties)
    collectDependencies(project.parent, allDeps, mvnProperties)
    collectDependencies(project.'**'.plugin, allDeps, mvnProperties)
}

static def collectProperties(def project, def mvnProperties) {
    //println("-- collectProperties")
    for (def p : project.properties) {
        for (def c : p.children()) {
            def name = c.name().localPart
            def value = c.text()
            mvnProperties[name] = value
        }
    }
}

static def collectDependencies(def dependencies, def allDeps, def mvnProperties) {
    //println("-- collectDependencies")
    for (def d : dependencies) {
        def groupId = d.groupId.text()
        if (groupId.isEmpty()) {
            // assume default groupId for maven plugins
            groupId = "org.apache.maven.plugins"
        }
        def artifactId = d.artifactId.text()
        def version = d.version.text()
        if (version.isEmpty())
            continue
        if (version.startsWith("\$")) {
            def matcher = (version =~ /\$\{(.+?)\}/)
            if (matcher.find()) {
                def property = matcher.group(1)
                version = mvnProperties[property]
                if (version == null) {
                    continue
                }
            }
        }
        def x = new Dependency(groupId, artifactId, version)
        allDeps.add(x)
    }
}

/**
 * This method is highly dependable on the html format returned by central.sonatype.com
 *
 * Return this map:
 * [name:logstash-logback-encoder, namespace:net.logstash.logback, version:7.4, versions:[7.4, 7.3, 7.2, 7.1.1, 7.1, 7.0.1, 7.0, 6.6, 6.5, 6.4, 6.3, 6.2, 6.1, 6.0, 5.3, 5.2, 5.1, 5.0, 4.11, 4.10], tags:[], usedIn:1600, tab:versions]
 */
def findLatestVersion(String groupId, String artifactId) {
    //println("-- findLatestVersion ${groupId}:${artifactId}")
    def url = "https://central.sonatype.com/artifact/${groupId}/${artifactId}/versions"
    def body = url.toURL().getText()
    def doc = Jerry.of(body)
    def scripts = doc.find('html.nx-html.nx-html--page-scrolling body.nx-body script')

    def text = ""
    scripts.each {
        def element ->
            if (element.text().contains('self.__next_f.push')) {
                text = element.text()
            }
    }

    int i = text.indexOf(':')
    int j = text.lastIndexOf('"')
    text = text.substring(i + 1, j - 2)
    text = text.replaceAll('\\\\"', '"')
    text = text.replaceAll('\\\\"', '"')

    def json = new JsonSlurper().parseText(text)
    try {
        def result = json[0][3].children[0][3]
        return result
    }
    catch (Exception ignored) {
        return null
    }
}

//skipVersions = [:]
skipVersions = [
        "org.springframework.boot:spring-boot-starter-parent"  : "3.2",
        "org.springframework.boot:spring-boot-starter-data-jpa": "3.2",
        "org.springframework.boot:spring-boot-starter-web"     : "3.2",
        "org.springframework.boot:spring-boot-dependencies"    : "3.2",
        "org.springframework.cloud:spring-cloud-dependencies"  : "2023",
]

def getLatestVersion(String groupId, String artifactId) {
    //println("-- getLatestVersion")
    def versions = findLatestVersion(groupId, artifactId)
    def skip = skipVersions["${groupId}:${artifactId}"]
    if (skip == null) {
        return versions?.version
    }

    for (def v in versions.versions) {
        if (v.startsWith(skip)) {
            continue
        }
        return v
    }
}

def allDeps = findAllDependencies()

for (def d : allDeps) {
    def latest = getLatestVersion(d.groupId, d.artifactId)
    if (latest == null) {
        continue
    }

    print("${d.groupId}:${d.artifactId}:${d.version} ")

    if (d.version.equals(latest)) {
        println("${ansi().fg(GREEN)}[OK]${ansi().reset()}")
    } else {
        println("${ansi().fg(RED)}[${latest}]${ansi().reset()}")
    }
}

By changing the skipVersions variable, it is possible to ignore a version of some dependencies.

For example: I'm stuck with Spring Boot 3.1, so the config below will ignore version 3.2, but will still notify for newer 3.1.x version.



skipVersions = [
        "org.springframework.boot:spring-boot-starter-parent"  : "3.2",
        "org.springframework.boot:spring-boot-starter-data-jpa": "3.2",
        "org.springframework.boot:spring-boot-starter-web"     : "3.2",
        "org.springframework.boot:spring-boot-dependencies"    : "3.2",
        "org.springframework.cloud:spring-cloud-dependencies"  : "2023",
]

If there is no dependency to skip, just use an empty map:



skipVersions = [:]

Changing the JVM in spring-boot:build-image

Eduardo Issao Ito — Wed, 12 Apr 2023 17:46:04 +0000

The spring-boot-maven-plugin can create an OCI image from a jar or war file using Cloud Native Buildpacks.

By default it uses a buildpack called paketo-buildpacks/bellsoft-liberica which is based on Bellsoft Liberica JDK.

The list of all Java buildpacks is available in Paketo Buildpacks Reference.

To choose another JVM in spring-boot-maven-plugin, first you need to find the buildpacks that are used by default. Run mvn spring-boot:build-image and look for something like this:

[INFO]     [creator]     ===> DETECTING
[INFO]     [creator]     6 of 26 buildpacks participating
[INFO]     [creator]     paketo-buildpacks/ca-certificates   3.6.1
[INFO]     [creator]     paketo-buildpacks/bellsoft-liberica 10.1.0
[INFO]     [creator]     paketo-buildpacks/syft              1.27.0
[INFO]     [creator]     paketo-buildpacks/executable-jar    6.6.3
[INFO]     [creator]     paketo-buildpacks/dist-zip          5.5.2
[INFO]     [creator]     paketo-buildpacks/spring-boot       5.23.0

That´s the list of buildpacks being used. We need to change Bellsoft Liberica to the desired JVM.

So, to use Microsoft OpenJDK, copy the buildpacks from above, but change paketo-buildpacks/bellsoft-liberica to gcr.io/paketo-buildpacks/microsoft-openjdk. The pom.xml should be:

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <configuration>
        <image>
            <buildpacks>
                <buildpack>paketo-buildpacks/ca-certificates</buildpack>
                <buildpack>gcr.io/paketo-buildpacks/microsoft-openjdk</buildpack>
                <buildpack>paketo-buildpacks/syft</buildpack>
                <buildpack>paketo-buildpacks/executable-jar</buildpack>
                <buildpack>paketo-buildpacks/dist-zip</buildpack>
                <buildpack>paketo-buildpacks/spring-boot</buildpack>
            </buildpacks>
        </image>
    </configuration>
</plugin>

Now, run mvn spring-boot:build-image and verify that Microsoft OpenJDK is being used:

[INFO]     [creator]     Paketo Buildpack for Microsoft OpenJDK 3.0.1
[INFO]     [creator]       https://github.com/paketo-buildpacks/microsoft-openjdk
...
[INFO]     [creator]       Microsoft OpenJDK 17.0.6: Contributing to layer
[INFO]     [creator]         Downloading from https://aka.ms/download-jdk/microsoft-jdk-17.0.6-linux-x64.tar.gz

Changing the base Linux image in spring-boot:build-image

Eduardo Issao Ito — Wed, 12 Apr 2023 16:59:12 +0000

The spring-boot-maven-plugin can create an OCI image from a jar or war file using Cloud Native Buildpacks.

By default it uses a builder called paketobuildpacks/builder:base which is based on Ubuntu 18.04 and is suitable for running a Spring Boot application.

But you can choose another builder, for example:

paketobuildpacks/builder:tiny is a lot smaller because it has the minimum requirements to run you application, even sh is absent (you can't open a terminal in this image)
paketobuildpacks/builder:full its the oposite of tiny, containing a lot of tools and commands.
paketobuildpacks/builder-jammy-base is based on Ubuntu 22.04

The list of all builders is available in Paketo Builders Reference.

It's just a matter of configuring the builder in pom.xml:

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <configuration>
        <image>
            <builder>paketobuildpacks/builder-jammy-tiny</builder>
        </image>
    </configuration>
</plugin>

Custom Root CA in spring-boot:build-image

Eduardo Issao Ito — Wed, 12 Apr 2023 14:46:48 +0000

The Spring Framework has a very useful feature which is the generation of a docker image through the spring-boot-maven-plugin. Simply running mvn spring-boot:build-image will create a docker compatible OCI image.

But if you are behind a corporate proxy, this error is likely to happen:

[INFO]     [creator]       BellSoft Liberica JRE 17.0.6: Contributing to layer

[INFO]     [creator]         Downloading from https://github.com/bell-sw/Liberica/releases/download/17.0.6+10/bellsoft-jre17.0.6+10-linux-amd64.tar.gz

[INFO]     [creator]     unable to invoke layer creator

[INFO]     [creator]     unable to get dependency jre

[INFO]     [creator]     unable to download https://github.com/bell-sw/Liberica/releases/download/17.0.6+10/bellsoft-jre17.0.6+10-linux-amd64.tar.gz

[INFO]     [creator]     unable to request https://github.com/bell-sw/Liberica/releases/download/17.0.6+10/bellsoft-jre17.0.6+10-linux-amd64.tar.gz

[INFO]     [creator]     Get https://github.com/bell-sw/Liberica/releases/download/17.0.6+10/bellsoft-jre17.0.6+10-linux-amd64.tar.gz: x509: certificate signed by unknown authority

[INFO]     [creator]     ERROR: failed to build: exit status 1

This happens because the corporation's certificate used in the proxy server is not known by the build process. When the buildpack tries download needed artifacts used inside the build process, it stops because the certificate is not trusted.

spring-boot-maven-plugin uses Cloud Native Buildpacks under the hood, and it allows some customization of the build process.

We need to put our corporate root CA certificates into the buildpack. For this we will create the files mycert.cer and type in the structure below:

.
├── pom.xml
└── src
    └── main
        └── bindings
            └── ca-certificates
                ├── mycert.cer
                └── type

src/main/bindings/ca-certificates/mycert.cer:

-----BEGIN CERTIFICATE-----
Base64–encoded certificate
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Base64–encoded certificate
-----END CERTIFICATE-----

src/main/bindings/ca-certificates/type:

ca-certificates

The following Maven configuration will add the certificate to the buildpack.

pom.xml:

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <configuration>
        <image>
            <env>
                <SERVICE_BINDING_ROOT>/bindings</SERVICE_BINDING_ROOT>
            </env>
            <bindings>
                <binding>${project.basedir}/src/main/bindings/ca-certificates:/bindings/ca-certificates</binding>
            </bindings>
        </image>
    </configuration>
</plugin>

Now, mvn spring-boot:build-image should work!

Getting http status in curl

Eduardo Issao Ito — Tue, 24 May 2022 19:43:58 +0000

How to get the http status code of curl in a script?


my_curl() {
  Red='\033[0;31m'
  Green='\033[0;32m'
  Color_Off='\033[0m'

  TEMP_FILE=$( mktemp )
  trap "rm -f $TEMP_FILE" EXIT
  curl -w "\n%{http_code}" -s "$@" > ${TEMP_FILE}
  body=$(cat ${TEMP_FILE} | sed '$d')
  status=$(cat ${TEMP_FILE} | sed -n '$p')

  if [[ $status = 2* ]]; then
    Color=$Green
  else
    Color=$Red
  fi

  echo -e "${Color}Status: $status${Color_Off}"
  echo $body
}

my_curl -X GET http://jservice.io/api/random

DB2 backup/restore schema

Eduardo Issao Ito — Thu, 14 Apr 2022 13:07:58 +0000

Creating the backup

Exporting DDL from EXAMPLE schema:

db2look -d MYDB -z EXAMPLE -e -o EXAMPLE.sql

Exporting data from schema:

db2move MYDB export -sn EXAMPLE

The db2move command will create a lot of files, incluing db2move.lst.

It contains internal tables like EXPLAIN_* and ADVISE_*. If you don't need them just delete de corresponding lines from db2move.lst.

Restoring backup

Creating objects:

db2 -t -f EXAMPLE.sql

Restoring data:

db2move MYDB import

db2move MYDB import -io insert_update

Using MTLS

Eduardo Issao Ito — Tue, 12 Apr 2022 17:02:35 +0000

Using MTLS

It is common for the server to be authenticated by a certificate, but in this case the client also has one. With MTLS the server can be sure who the client is.

MTLS means "mutual TLS authentication" or "two-way TLS authentication".

The scenario is: client (Amy) accessing server (Bob) using MTLS.

The diagram below shows how the server and client certificates are created.

The CA (certification authority) is a third party responsible for signing the certificates with a trusted root certificate. But it is common to use a CA that belongs to one of the two parties (Amy ou Bob).

It is also possible that Amy uses one CA and Bob chooses another (it will be explained later).

Note that the procedure for creating the server certificate is the same as for the client certificate. The only difference is in the common name used in the certificate subject:

The CN of the server must be the name that the Amy uses to access Bob.
The CN of the client can be anything, although the server can use it to do some additional validation.

1. [CA] creation of a self-signed root CA X.509 certificate

openssl req \
  -new \
  -x509 \
  -nodes \
  -days 365 \
  -subj '/CN=my-ca' \
  -keyout ca.key \
  -out ca.crt

2. [Bob, server side] create the server’s key

openssl genrsa -out bob_server.key 2048

3. [Bob, server side] request the server’s certificate

The Common Name in the subject should reflect the server name that will be used by clients, like srv.bob.com. For
testing purposes, we will simply assume localhost.

openssl req \
  -new \
  -key bob_server.key \
  -subj '/CN=localhost' \
  -out bob_server.csr

The Certificate Signing Request (csr file) generated in this step must be sent to the CA to proceed with the creation of
the certificate.

4. [CA, on behalf of Bob] Using the CSR, the CA creates the signed server certificate:

openssl x509 \
  -req \
  -in bob_server.csr \
  -CA ca.crt \
  -CAkey ca.key \
  -CAcreateserial \
  -days 365 \
  -out bob_server.crt

The certificate (crt file) is sent to Bob.

5. [Amy, client side] create the clients’s key

openssl genrsa -out amy_client.key 2048

6. [Amy, client side] request the server’s certificate

openssl req \
  -new \
  -key amy_client.key \
  -subj '/CN=amy-client' \
  -out amy_client.csr

The Certificate Signing Request (csr file) generated in this step must be sent to the CA to proceed with the creation of
the certificate.

7. [CA, on behalf of Amy] Using the CSR, the CA creates the signed client certificate:

openssl x509 \
  -req \
  -in amy_client.csr \
  -CA ca.crt \
  -CAkey ca.key \
  -CAcreateserial \
  -days 365 \
  -out amy_client.crt

The certificate (crt file) is sent to Amy.

Test

Start Bob's server:

Start node application:

node app_bob.js

If you want to allow only clients with the more secure TLS v1.3:

node --tls-min-v1.3 app_bob.js

Execute Amy's client:

curl \
  --cacert ca.crt \
  --key amy_client.key \
  --cert amy_client.crt \
  https://localhost:3000

Using distinct CAs

If Amy and Bob are using distinct CAs, they need to know each other root certificates.

Assume that Amy's and Bob's CAs have certificates amy_ca.crt and bob_ca.crt, respectively.

So, Bob must load the certificate used by Amy:

const options = {
    ca: fs.readFileSync('amy_ca.crt'),

And Amy's client need to know the certificate used by Bob:

curl \
  --cacert bob_ca.crt \

These are complete instructions to create this scenario:

Bob (server side):

openssl req \
  -new \
  -x509 \
  -nodes \
  -days 365 \
  -subj '/CN=bob-ca' \
  -keyout bob_ca.key \
  -out bob_ca.crt

openssl genrsa -out bob_server.key 2048

openssl req \
  -new \
  -key bob_server.key \
  -subj '/CN=localhost' \
  -out bob_server.csr

openssl x509 \
  -req \
  -in bob_server.csr \
  -CA bob_ca.crt \
  -CAkey bob_ca.key \
  -CAcreateserial \
  -days 365 \
  -out bob_server.crt

Amy (client side):

openssl req \
  -new \
  -x509 \
  -nodes \
  -days 365 \
  -subj '/CN=amy-ca' \
  -keyout amy_ca.key \
  -out amy_ca.crt

openssl genrsa -out amy_client.key 2048

openssl req \
  -new \
  -key amy_client.key \
  -subj '/CN=amy-client' \
  -out amy_client.csr

openssl x509 \
  -req \
  -in amy_client.csr \
  -CA amy_ca.crt \
  -CAkey amy_ca.key \
  -CAcreateserial \
  -days 365 \
  -out amy_client.crt

Reference

https://github.com/adzubla/mtls-test

Convert OpenAPI spec to JSON Schema

Eduardo Issao Ito — Tue, 09 Feb 2021 14:05:13 +0000

I've created a Groovy script to convert from "OpenAPI" specification (aka Swagger file) to "JSON Schema".

https://github.com/adzubla/schema-convert

groovy SchemaConvert.groovy openapi-spec.yaml

You can add the -m option to generate a schema compatible with MongoDB.

groovy SchemaConvert.groovy openapi-spec.yaml -m

Sample execution

groovy SchemaConvert.groovy petstore.yaml

Input file petstore.yaml:

openapi: "3.0.0"
info:
  version: 1.0.0
  title: Swagger Petstore
  license:
    name: MIT
servers:
  - url: http://petstore.swagger.io/v1
paths:
  /pets:
    get:
      summary: List all pets
      operationId: listPets
      tags:
        - pets
      parameters:
        - name: limit
          in: query
          description: How many items to return at one time (max 100)
          required: false
          schema:
            type: integer
            format: int32
      responses:
        '200':
          description: A paged array of pets
          headers:
            x-next:
              description: A link to the next page of responses
              schema:
                type: string
          content:
            application/json:    
              schema:
                $ref: "#/components/schemas/Pets"
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
    post:
      summary: Create a pet
      operationId: createPets
      tags:
        - pets
      responses:
        '201':
          description: Null response
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
  /pets/{petId}:
    get:
      summary: Info for a specific pet
      operationId: showPetById
      tags:
        - pets
      parameters:
        - name: petId
          in: path
          required: true
          description: The id of the pet to retrieve
          schema:
            type: string
      responses:
        '200':
          description: Expected response to a valid request
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Pets"
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
components:
  schemas:
    Pets:
      type: array
      items:
        $ref: "#/components/schemas/Pet"
    Pet:
      required:
        - id
        - name
      properties:
        id:
          type: integer
          format: int64
        name:
          type: string
        tag:
          type: string
    Error:
      required:
        - code
        - message
      properties:
        code:
          type: integer
          format: int32
        message:
          type: string

Output file petstore.json:

{"key": "value"}

Json validation with OpenAPI Schema

Eduardo Issao Ito — Tue, 09 Feb 2021 13:55:26 +0000

How to validate a json file against a Schema that is defined in an OpenAPI definition?

Using the open source library openapi4j!

Add the dependencies in your pom.xml file:

        <dependency>
            <groupId>org.openapi4j</groupId>
            <artifactId>openapi-parser</artifactId>
            <version>1.0.5</version>
        </dependency>

        <dependency>
            <groupId>org.openapi4j</groupId>
            <artifactId>openapi-schema-validator</artifactId>
            <version>1.0.5</version>
        </dependency>

Load the OpenAPI definition (petstore.yaml) and the json file (pets.json) to validate against the Pets schema:

        URL specURL = new URL("file:src/main/resources/petstore.yaml");
        URL contentURL = new URL("file:src/main/resources/pets.json");
        String schemaName = "Pets";

        OpenApi3 api = new OpenApi3Parser().parse(specURL, true);
        JsonNode contentNode = TreeUtil.load(contentURL);
        Schema schema = api.getComponents().getSchema(schemaName);
        JsonNode schemaNode = schema.toNode();

        SchemaValidator schemaValidator = new SchemaValidator(new ValidationContext<>(api.getContext()), null, schemaNode);

        ValidationData<Void> validation = new ValidationData<>();
        schemaValidator.validate(contentNode, validation);
        if (validation.isValid()) {
            System.out.println("ok");
        } else {
            System.out.println(validation.results());
        }

If there is any inconsistency, the ValidationData object will have a list of all errors found.

SFTP server in Ubuntu

Eduardo Issao Ito — Mon, 08 Feb 2021 16:56:32 +0000

How to setup a basic SFTP server in Ubuntu.

Requirements

No login
Read and write permissions
Access only to a base directory
Default directory should be the base directory

Instalation

Install the sftp server package

sudo apt-get install openssh-server

Create sftp group and user

sudo groupadd sftpgroup
sudo useradd myuser -m -G sftpgroup -s /usr/sbin/nologin
sudo passwd myuser

Prepare base sftp directory

sudo chown root:root /home/myuser
sudo mkdir /home/myuser/data
sudo chown myuser:sftpgroup /home/myuser/data

Configure sftp server

Edit the file /etc/ssh/sshd_config

Comment the following line:

#Subsystem sftp /usr/lib/openssh/sftp-server

Add the folowing lines at the end of the file:

Subsystem sftp internal-sftp
Match group sftpgroup
ChrootDirectory %h
X11Forwarding no
AllowTcpForwarding no
ForceCommand internal-sftp -d /data

Restart the server

sudo service ssh restart

Test

$ sftp myuser@localhost
myuser@localhost's password:
Connected to localhost.
sftp> pwd
Remote working directory: /data
sftp> put test.txt
Uploading test.txt to /data/test.txt
test.txt                                                                              100%    0     0.0KB/s   00:00
sftp> ls
test.txt
sftp> quit

Using multiple JMS servers with Spring Boot

Eduardo Issao Ito — Thu, 13 Aug 2020 18:16:34 +0000

Spring Boot support for JMS is straightforward to use. But the default configuration is limited to one server.

Let's see how to use more than one server. (In this example I will be using IBM MQ, but the same principle can be applied to other products.)

First we need to add configuration properties for two servers:

qm1.queueManager=QM1
qm1.channel=DEV.ADMIN.SVRCONN
qm1.connName=localhost(1414)
qm1.user=admin
qm1.password=passw0rd
qm1.pool.enabled=true

qm2.queueManager=QM2
qm2.channel=DEV.ADMIN.SVRCONN
qm2.connName=localhost(1415)
qm2.user=admin
qm2.password=passw0rd
qm2.pool.enabled=true

In this example we added the "qm1" and "qm2" prefixes to the standard properties, to configure the two servers.

For each one of the servers we need to read these properties (using @ConfigurationProperties annotation with the prefix) and create an specific ConnectionFactory and JmsListenerContainerFactory.

The Qm1Config and Qm2Config configure these beans.

@Configuration
public class Qm1Config {

    @Bean
    @ConfigurationProperties("qm1")
    public MQConfigurationProperties qm1ConfigProperties() {
        return new MQConfigurationProperties();
    }

    @Bean
    public MQConnectionFactory qm1ConnectionFactory(@Qualifier("qm1ConfigProperties") MQConfigurationProperties properties, ObjectProvider<List<MQConnectionFactoryCustomizer>> factoryCustomizers) {
        return new MQConnectionFactoryFactory(properties, factoryCustomizers.getIfAvailable()).createConnectionFactory(MQConnectionFactory.class);
    }

    @Bean
    public JmsListenerContainerFactory<?> qm1JmsListenerContainerFactory(@Qualifier("qm1ConnectionFactory") ConnectionFactory connectionFactory, DefaultJmsListenerContainerFactoryConfigurer configurer) {
        DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
        configurer.configure(factory, connectionFactory);
        return factory;
    }

}

@Configuration
public class Qm2Config {

    @Bean
    @ConfigurationProperties("qm2")
    public MQConfigurationProperties qm2ConfigProperties() {
        return new MQConfigurationProperties();
    }

    @Bean
    public MQConnectionFactory qm2ConnectionFactory(@Qualifier("qm2ConfigProperties") MQConfigurationProperties properties, ObjectProvider<List<MQConnectionFactoryCustomizer>> factoryCustomizers) {
        return new MQConnectionFactoryFactory(properties, factoryCustomizers.getIfAvailable()).createConnectionFactory(MQConnectionFactory.class);
    }

    @Bean
    public JmsListenerContainerFactory<?> qm2JmsListenerContainerFactory(@Qualifier("qm2ConnectionFactory") ConnectionFactory connectionFactory, DefaultJmsListenerContainerFactoryConfigurer configurer) {
        DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
        configurer.configure(factory, connectionFactory);
        return factory;
    }

}

Then, the listener must specify which queue manager is to be used:

@Component
public class QueueConsumer {

    @JmsListener(destination = "${example.queue}", containerFactory = "qm1JmsListenerContainerFactory")
    public void receive1(String text) {
        System.out.println("Received from qm1: " + text);
    }

    @JmsListener(destination = "${example.queue}", containerFactory = "qm2JmsListenerContainerFactory")
    public void receive2(String text) {
        System.out.println("Received from qm2: " + text);
    }

}

And that's it!

Example

A complete runnable example is available in GitHub:
https://github.com/adzubla/qmgrs

Dynamic creation of listeners

The above solution hardwires the JmsListeners and ConnectionFactories in source code. To have a configurable number of queue managers we can use the approach that follows.

The new properties file:

qm.list.0.queueManager=QM1
qm.list.0.channel=DEV.ADMIN.SVRCONN
qm.list.0.connName=localhost(1414)
qm.list.0.user=admin
qm.list.0.password=passw0rd
qm.list.0.pool.enabled=true
qm.list.1.queueManager=QM2
qm.list.1.channel=DEV.ADMIN.SVRCONN
qm.list.1.connName=localhost(1415)
qm.list.1.user=admin
qm.list.1.password=passw0rd
qm.list.1.pool.enabled=true

To read these properties in a List of properties, we use the configuration below:

@ConfigurationProperties("qm")
@Configuration
public class QmProperties {

    private List<MQConfigurationProperties> list;

    public List<MQConfigurationProperties> getList() {
        return list;
    }

    public void setList(List<MQConfigurationProperties> list) {
        this.list = list;
    }

}

As in the previous example, for each one of the servers we need to create an specific ConnectionFactory and JmsListenerContainerFactory. This is done using JmsListenerConfigurer that allows the creation of listeners programatically.

@Configuration
@EnableJms
public class JmsConfig implements JmsListenerConfigurer {

    @Autowired
    private ObjectProvider<List<MQConnectionFactoryCustomizer>> factoryCustomizers;

    @Autowired
    private DefaultJmsListenerContainerFactoryConfigurer configurer;

    @Autowired
    private QmProperties qmProperties;

    @Autowired
    private QueueConsumer queueConsumer;

    @Value("${example.queue}")
    private String destination;

    @Override
    public void configureJmsListeners(JmsListenerEndpointRegistrar registrar) {
        for (MQConfigurationProperties properties : qmProperties.getList()) {
            String queueManager = properties.getQueueManager();

            MQConnectionFactory connectionFactory = new MQConnectionFactoryFactory(properties, factoryCustomizers.getIfAvailable()).createConnectionFactory(MQConnectionFactory.class);

            DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
            configurer.configure(factory, connectionFactory);

            SimpleJmsListenerEndpoint endpoint = new SimpleJmsListenerEndpoint();
            endpoint.setId("jmsEndpoint-" + queueManager);
            endpoint.setDestination(destination);
            endpoint.setMessageListener(message -> {
                try {
                    queueConsumer.receive(queueManager, message.getBody(String.class));
                } catch (JMSException e) {
                    throw new RuntimeException(e);
                }
            });
            registrar.registerEndpoint(endpoint, factory);
        }
    }

}

The QueueConsumer bean will be registered for each of the queue managers the present in the configuration.

Note that we removed the @JmsListener because the QueueConsumer bean will be registered by our JmsConfig.

@Component
public class QueueConsumer {

    public void receive(String queueManeger, String text) {
        System.out.println("Received from " + queueManeger + ": " + text);
    }

}