DEV Community: Thomas Schühly

How to publish a Kotlin/Java Spring Boot library with Gradle to Maven Central - Complete Guide

Thomas Schühly — Wed, 08 Nov 2023 12:03:35 +0000

This is an opinionated step-by-step guide on how to publish a Kotlin/Java library with Gradle to Maven Central
repository.

It assumes that:

the project is built with Gradle (look at Maciej Guide if you want to do it with Maven)
the project code is hosted on GitHub and GitHub Actions are used to trigger the release

It uses JReleaser - I believe this is the simplest and the most straightforward way of signing
and uploading artifacts.

This guide is based on the excellent
article How to publish a Java library to Maven Central - Complete Guide
but uses Gradle instead of Maven.

Create an account in Sonatype JIRA
Create a "New Project" ticket
1. If a custom domain is used as a group id
2. If GitHub is used as a group id
3. Set ticket to "Open"
Create GPG keys
1. Export key to a key server
Export public and secret key to GitHub secrets
1. Create GitHub secrets with UI
2. Create secrets with GitHub CLI
Adjust pom.xml
1. Generate javadocs and sources JARs
2. Configure JReleaser Maven Plugin
Create a GitHub action
Get familiar with Sonatype Nexus UI
When is the library actually available to use?

1. Create an account in Sonatype JIRA

You do it only once - no matter how many projects you want to release or how many group ids you own.

2. Create a "New Project" ticket

Create a "New Project ticket in
Sonatype JIRA.

This step is done once per group id. Meaning, for each domain you want to use as a group id - you must create a new
project request.

Although the official Sonatype guide claims that normally, the process takes less than 2 business days. In my case it
took just a few minutes.

Once the ticket is created, a Sonatype JIRA bot will post comments with instructions what to do next:

2.1. If a custom domain is used as a group id

When you want to use a domain like de.tschuehly as a group id - you must own the domain - and be able to prove it. *
You must add a DNS TXT record with a JIRA ticket id* to your domain - this is done in the admin panel where your
domain is hosted.

Once you have added the record, verify that it is added with the following command:

$ dig -t txt tschuehly.de

2.2. If GitHub is used as a group id

If you don't own the domain it is possible to use your GitHub coordinates as a group id. For example, my GitHub account
name is tschuehly, so I can use io.github.tschuehly as a group id.

To prove that you own that GitHub account, create a temporary repository with a name reflecting the JIRA ticket id.

This can be done via github.com/new or with GitHub CLI:

$ gh repo create OSSRH-91026 --public

2.3. Set ticket to "Open"

The comment posted by Sonatype bot says that once you are done with either creating a DNS record or creating a GitHub
repository, "Edit this ticket and set Status to Open.".

I did not find any way to change status to "Open" in the edit form, but instead I had to click one of the buttons at the
top of JIRA ticket, right next to "Agile Board" and "More" (unfortunately I did not make a screenshot on time).

Once you do it, another comment will be posted by Sonatype bot:

This means that our job in the Sonatype JIRA is done. Congratulations 🎉

(you can now drop the temporary GitHub repository if you've created one)

3. Create GPG keys

Artifacts sent to Maven Central must be signed. To sign artifacts you need to generate GPG keys.

This must be done only once - all artifacts you publish to Maven Central can be signed with the same pair of keys.

Create a key pair with:

$ gpg --gen-key

Put your name, email address and passphrase.

List keys with command:

$ gpg --list-keys

You will see output like this:

pub   ed25519 2022-11-05 [SC] [expires: 2024-11-04]
      05342E4134D1F7C1B08F900FC2377C0DD0494024
uid           [ultimate] john@doe.com
sub   cv25519 2022-11-05 [E] [expires: 2024-11-04]

In this example - 05342E4134D1F7C1B08F900FC2377C0DD0494024 is the key id. Find your own key id and copy it to
clipboard.

If you can't find it, you probably used a wrong version of gpg. It didn't work on my Windows machine but worked on my
linux server

3.1 Export key to a key server

Next you need to export the public key to a key server with command:

$ gpg --keyserver keyserver.ubuntu.com --send-keys yourKeyId

4. Export public and secret key to GitHub secrets

JReleaser needs public and secret key to sign artifacts. Since signing will be done by a GitHub action, you need to
export these keys as GitHub secrets.

Secrets can be set either on the GitHub repository website or with a GitHub CLI.

4.1. Create GitHub secrets with UI

Go to repository Settings:

Create a repository secret JRELEASER_GPG_PUBLIC_KEY with a value from running:

$ gpg --export yourKeyId | base64

Create a key JRELEASER_GPG_SECRET_KEY with a value from running:

$ gpg --export-secret-keys yourKeyId | base64

Create a key JRELEASER_GPG_PASSPHRASE with a value that is a passphrase you used when creating your gpg key.

Two more secrets unrelated to GPG are needed to release to Maven Central:

Create a key JRELEASER_NEXUS2_USERNAME with the username you use to log in to Sonatype JIRA.

Create a key JRELEASER_NEXUS2_PASSWORD with the password you use to log in to Sonatype JIRA.

4.2. Create secrets with GitHub CLI

If you choose to use the CLI instead, run the following commands (replace things in < brackets > with real values) from the
directory where your project is cloned:

$ gh secret set JRELEASER_GPG_PUBLIC_KEY -b $(gpg --export <key id> | base64)
$ gh secret set JRELEASER_GPG_SECRET_KEY -b $(gpg --export-secret-keys <key id> | base64)
$ gh secret set JRELEASER_GPG_PASSPHRASE -b <passphrase>
$ gh secret set JRELEASER_NEXUS2_USERNAME -b <sonatype-jira-username>
$ gh secret set JRELEASER_NEXUS2_PASSWORD -b <sonatype-jira-password>

5. Create publishing config

Here is an example config you can adjust to your needs:

publishing{
  publications {
    create<MavenPublication>("Maven") {
      from(components["java"])
      groupId = "de.tschuehly"
      artifactId = "spring-view-component-thymeleaf"
      description = "Create server rendered components with thymeleaf"
    }
    withType<MavenPublication> {
      pom {
        packaging = "jar"
        name.set("spring-view-component-thymeleaf")
        description.set("Spring ViewComponent Thymeleaf")
        url.set("https://github.com/tschuehly/spring-view-component/")
        inceptionYear.set("2023")
        licenses {
          license {
            name.set("MIT license")
            url.set("https://opensource.org/licenses/MIT")
          }
        }
        developers {
          developer {
            id.set("tschuehly")
            name.set("Thomas Schuehly")
            email.set("thomas.schuehly@outlook.com")
          }
        }
        scm {
          connection.set("scm:git:git@github.com:tschuehly/spring-view-component.git")
          developerConnection.set("scm:git:ssh:git@github.com:tschuehly/spring-view-component.git")
          url.set("https://github.com/tschuehly/spring-view-component")
        }
      }
    }
  }
  repositories {
    maven {
        url = layout.buildDirectory.dir("staging-deploy").get().asFile.toURI()
    }
  }
}

5.1. Generate javadocs and sources JARs

Artifacts uploaded to Maven Central must have two extra jars: one with sources and one with Javadocs. Both are created
by Gradle.

java {
  withJavadocJar()
  withSourcesJar()
}

tasks.jar{
  enabled = true
  // Remove `plain` postfix from jar file name
  archiveClassifier.set("")
}

5.2 Configure JReleaser Maven Plugin

JReleaser can be invoked either as a standalone CLI application or a Gradle Plugin. To use the Gradle Plugin you need to add these plugins:

plugins {
  id("maven-publish")
  id("org.jreleaser") version "1.5.1"
  id("signing")
}

Add following plugin configuration to the plugins section of the release profile:

jreleaser {
  project {
    copyright.set("Thomas Schuehly")
  }
  gitRootSearch.set(true)
  signing {
    active.set(Active.ALWAYS)
    armored.set(true)
  }
  deploy {
    maven {
      nexus2 {
        create("maven-central") {
          active.set(Active.ALWAYS)
          url.set("https://s01.oss.sonatype.org/service/local")
          closeRepository.set(true)
          releaseRepository.set(true)
          stagingRepositories.add("build/staging-deploy")
        }  
      }
    }
  }
}

I recommend to set temporarily closeRepository and releaseRepository to false. At the end once you successfully
release the first version to staging repository in Sonatype Nexus you can switch it to true.

6. Create a GitHub action

The GitHub action will trigger the release each time a tag that starts with v is created, like v1.0, v1.1 etc.

Create a file in your project directory under .github/workflows/release.yml:

name: Publish package to the Maven Central Repository
on:
  push:
    tags:
      - v*
  pull_request:
    branches: [ main ]
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up Java
        uses: actions/setup-java@v3
        with:
          java-version: '17'
          distribution: 'adopt'
      - name: Run chmod to make gradlew executable
        run: chmod +x ./gradlew
      - name: Publish package to local staging directory
        run: ./gradlew :publish
      - name: Publish package to maven central
        env:
          JRELEASER_NEXUS2_USERNAME: ${{ secrets.JRELEASER_NEXUS2_USERNAME }}
          JRELEASER_NEXUS2_PASSWORD: ${{ secrets.JRELEASER_NEXUS2_PASSWORD }}
          JRELEASER_GPG_PASSPHRASE: ${{ secrets.JRELEASER_GPG_PASSPHRASE }}
          JRELEASER_GPG_SECRET_KEY: ${{ secrets.JRELEASER_GPG_SECRET_KEY }}
          JRELEASER_GPG_PUBLIC_KEY: ${{ secrets.JRELEASER_GPG_PUBLIC_KEY }}
          JRELEASER_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: ./gradlew :jreleaserDeploy -DaltDeploymentRepository=local::file:./build/staging-deploy

Adjust the Java version and the distribution to your needs.

The action will stage artifact and then
run jreleaser:deploy goal to publish artifact to Sonatype Nexus.

7. Get familiar with Sonatype Nexus UI

Once you create and push first tag and the GitHub Action finishes with success, you can log in
to Sonatype Nexus with your Sonatype JIRA credentials to preview your staging
repository.

In the Staging Profiles section you will see all the group ids you own:

If you set closeRepository and releaseRepository to false in JReleaser configuration, in
the Staging Repositories section you will see an entry for the version that was released with a GitHub action:

(image
from https://help.sonatype.com/repomanager2/staging-releases/managing-staging-repositories)

The first time I did it I needed to wait a long time and there were quite a few timeouts.

Here you can Close the repository and Release. Both actions trigger series of verifications - if your gradle.build.kts
meets criteria, if packages are properly signed, if your GPG key is uploaded to the key server.

I recommend triggering these actions manually for the first version you release just to see if everything is fine. Once
the Release action finishes with success, your library is considered as published to Maven Central.
Congratulations 🎉

You can now set closeRepository and releaseRepository to true in JReleaser configuration.

8. When is the library actually available to use?

The library is not immediately available after it is released. Official documentation says that it may take up to 30
minutes before the package is available, some folks claim that it can take few hours. In my case it took just 10
minutes.

Now your artifact can be referenced in build.gradle.kts and Gradle will successfully download it. If you try to do it before it
is available, Gradle will mark this library as unavailable and will not try to re-download it until the cache expires.
Use --refresh-dependencies flag to .\gradlew command to force Gradle to check for updates:

$ ./gradlew build --refresh-dependencies

Don't be fooled by the results in search.maven.org
or mvnrepository.com. Here your artifact or even a new version of the artifact will appear
after around 24 hours.

Conclusion

I hope this guide was useful, and it helped you to release a library to Maven Central.

If you find anything unclear drop me a message on Twitter.

Most of this guide is based on Maciej Walkowiak excellent guide so drop him a thanks!

I would like to thank Andres Almiray for creating JReleaser. This library significantly simplifes the whole process to the point that it's not terribly overcomplicated anymore.

Toasts with Thymeleaf, HTMX and Spring Boot

Thomas Schühly — Mon, 20 Feb 2023 08:36:57 +0000

In this example, I will show you a way to return Toast Notifications from your server and render them interactively.

The HTML:

We will start with an HTMX Element that creates a request to our spring server:

<form class="" hx-target="#toast" hx-put="/api/someRoute"></form>

As you can see we target an element with the id toast. To make this work I created a div with the id that is inside my
footer. The footer is shown on every page.

<footer>
    <div id="toast"></div>
</footer>

But what if you don't want to target the toast element, and instead you want to target another element?
I will show you later how you can change the HTMX target on the server.

Next is our toast element, to make them generic we use parameterized Thymeleaf fragments.
In this example I use the awesome Alpine.js library, but this is also easily doable with
normal javascript.

We create a toast.html file in our root directory to use it later as a return view.


<div th:fragment="info(message,duration)"
     th:attr="x-init='setTimeout(() => $el.style.display = \'none\',' + ${duration} + ')'"
     class="fixed bottom-10 z-40 left-1/2 -translate-x-1/2"
     x-data="{}"
     x-ref="info">
    <div class="flex">
        <div class="alert alert-info shadow-lg my-6">
            <div>
                <span th:text="${message}"></span>
            </div>
            <button @click="$refs.info.style.display = 'none'">
                <img src="/svg/x-circle.svg">
            </button>
        </div>
    </div>
</div>

To break it down:

th:fragment="info(message,duration) attribute declares that this is a parameterized Thymeleaf fragment we can later use in our spring backend.
th:attr="x-init='setTimeout(() => $el.style.display = \'none\',' + ${duration} + ')'" creates a timeout function on initialization that hides the toast after the duration we specified in the fragment parameter
Next, we position the element with the tailwind classes fixed bottom-10 z-40 left-1/2 -translate-x-1/2
The <span th:text="${message}"></span> element renders our text
With the <button @click="$refs.info.style.display = 'none'"> our user can click the toast away before it expires.

I also duplicated this element and changed the info to error and changed the styling to return info or error toasts.

The backend code:

To specify our toast I used an enum class:

enum class Toast {
    INFO {
        override fun ModelAndView(message: String, durationInMs: Int) =
            ModelAndView("toast :: info(message = '${message}', duration = '$durationInMs')")
    },
    ERROR {
        override fun ModelAndView(message: String, durationInMs: Int) =
            ModelAndView("toast :: error(message = '${message}', duration = '$durationInMs')")
    };

    abstract fun ModelAndView(message: String, durationInMs: Int): ModelAndView
}

We can return this enum in our Controller if we want to show either an info message or an error message.

As you can see we return the parameterized fragment with a message we can extract from our business context and give our
user valuable information.

@Controller
ExampleController() {
    @RequestMapping("/api/someRoute")
    fun someRoute(): ModelAndView {
        try {
            // do Something
        } catch (e: Exception) {
            return Toast.ERROR.ModelAndView("An Error occurred: ${e.message}", 5000)
        }

    }
}

Change the HTMX target on the server

But what do you do when you want to swap the element itself and only show a toast if an error occurs?


<form hx-put="/api/someRoute"></form>

We can change the behavior of HTMX easily with two Response Headers:

@RequestMapping("/api/someRoute")
fun someRoute(
    httpServletResponse: HttpServletResponse
): ModelAndView {
    try {
        // do Something
    } catch (e: Exception) {
        httpServletResponse.addHeader("HX-Retarget", "#errors");
        httpServletResponse.addHeader("HX-Reswap", "innerHTML");
        return Toast.ERROR.ModelAndView("An Error occurred: ${e.message}", 5000)
    }

}

We just include the httpServletResponse: HttpServletResponse in the constructor of our function, and then

With HX-Retarget we can change the hx-target property
And with HX-Reswap we can change the hx-swap property

There are many more HTMX Response Headers you can change to influence the behavior of your application.

My Indiehacker product is built with htmx:
PhotoQuest

Fixing the Spring Boot LiveReload Server with Gradle for Thymeleaf and TailwindCSS

Thomas Schühly — Tue, 05 Jul 2022 12:25:28 +0000

LiveReloading with Thymeleaf

I couldn't get Live Reload with Spring Boot and Thymeleaf to work. The problem seems to be so widespread that even @wimdeblauwe the author of Taming Thymeleaf uses gulp/npm to get Live Reloading working.

I thought why do we need to use Javascript to get our Spring Boot Developer Experience up to par with the big Javascript Frameworks? We have a perfectly fine Build Tool with Gradle!

LiveReload.js

First we need to include a script tag which points to the LiveReload Server:

<script src="http://localhost:35729/livereload.js"></script>

Gradle Sync

The plan is the same as with the npm scripts. Sync the templates into the build output so LiveReload triggers properly. This is done with the Gradle Sync Task:

// build.gradle.kts
tasks.register("sync"){
    inputs.files("./src/main/resources/static","./src/main/resources/templates",)
    doLast {
        sync {
            from("./src/main/resources/static")
            into("build/resources/main/static")
        }
        sync {
            from("./src/main/resources/templates")
            into("build/resources/main/templates")
        }
    }
}

Now start a continuous build with:

.\gradlew -t sync

Each time one of the inputs.files directory changes the sync task is rerun and LiveReload works.

Tailwind CSS

To use TailwindCSS we use the node-gradle plugin and create a npx task:

// build.gradle.kts
plugins {
    ...
    id("com.github.node-gradle.node") version "3.4.0"
}

tasks.register<com.github.gradle.node.npm.task.NpxTask>("tailwind"){
    command.set("tailwindcss")
    args.set(listOf("-i", "./src/main/resources/static/styles.css",  
        "-o" ,"./src/main/resources/static/output.css", "--watch"))
}

We then need to create a package.json:

{
  "devDependencies": {
    "@tailwindcss/forms": "^0.5.2",
    "tailwindcss": "^3.1.4"
  }
}

Add a tailwind.config.js:

/** @type {import('tailwindcss').Config} */
module.exports = {
  content: ["./src/**/*.{html,js}"],
}

Add a styles.css in resources/static/:

@tailwind base;
@tailwind components;
@tailwind utilities;

and include output.css in our html:

<link href="/output.css" rel="stylesheet">

Now we can start the tailwind npx task in a new terminal:

.\gradlew tailwind

And it works:

You can see an example at: github.com/tschuehly/thymeleaf-livereload-gradle

Implementing Worker Pools in Kotlin to upload Images over WebDav to a Hetzner StorageBox

Thomas Schühly — Wed, 22 Jun 2022 16:02:56 +0000

Worker pools

Recently while doing a project with Go I came across Worker Pools on GoByExample to do parallel processing. I didn't find many resources for implementing Worker Pools in Kotlin, but it seemed a good idea for my current Spring Boot + Kotlin webapp.

Kotlin

Kotlin uses coroutines for concurrency which are fairly similiar to goroutines.

Coroutines use structured concurrency to delimit the lifetime of each coroutine to a certain scope.

To be able to create a workergroup we need to create a coroutinescope that is persistent over the livetime of our application. We achieve this behaviour with the SupervisorJob() context.

private val coroutineScope = CoroutineScope(SupervisorJob() + Dispatchers.Default)

We then create a buffered channel as queue for our image data and the url where we want to upload it.

val channel = Channel<Pair<String, ByteArray>>(10000)

I'm using the Spring @PostConstruct annotation to create the worker group and listen to the channel for new data.
Each time a item is in the queue we launch the upload function, if no item is in the queue the function is suspended.

@PostConstruct
    fun createWorkerGroup() {
        coroutineScope.launch {
            for (x in 1..5) {
                launch {
                    println("Create Worker $x")
                    while (true) {
                        uploadImage(channel.receive())
                    }
                }
            }
        }
    }

Finally we can send our data to our channel inside a runBlocking coroutine scope:

runBlocking {
  uploadService.channel.send(Pair(url, image.bytes))
}

WebDav

In my webapp users upload images from their mobile phone to my webserver, afterwards I want to upload these pictures to a Hetzner Storage Box over webdav as a cheap alternative to an S3 object storage.

I use the sardine java webdav client library for its simplicity.

The usage is very straightforward, you configure the client with:

val sardine = SardineFactory.begin("webDavUsername", "webDavPassword")

The uploadImage Function is called every time a new image is sent over the channel we created earlier. In this function we call sarding.put() to save the image file.

sardine.put("https://username.your-storagebox.de/foldername/imagename.jpg", ImageByteArray)

That is all we need to have a highly parallel File Service

You can view the Sourcecode at Github: UploadService.kt ImageService.kt

Creating a GO GUI with Alpine.js and Webview

Thomas Schühly — Mon, 14 Mar 2022 16:55:59 +0000

There are a lot of options for building a GUI for Go applications.
Coming from the web development world building the frontend with HTML seems as a no-brainer.

Webview

Webview is a tiny cross-platform library for C/C++/Golang to build modern cross-platform GUIs. The goal of the project is to create a common HTML5 UI abstraction layer for the most widely used platforms.

To start using webview you need to install webview:
go get github.com/webview/webview

On windows you need to have these two dlls in the project root folder.

It supports two-way JavaScript bindings (to call JavaScript from C/C++/Go and to call C/C++/Go from JavaScript).
But writing pure javascript code for the interactivity (imo) is awful.

Alpine.js to the rescue

"Alpine.js is a rugged, minimal tool for composing behavior directly in your markup." It fits perfectly for our usecase.

You can load alpine inline or from a file. The newest version is available at unpkg.com/alpinejs

func loadAlpine() string {
    return "paste alpine.js source here"
}

First you must initialize webview.

func main() {
    webView := webview.New(true)
    defer webView.Destroy()

    webView.SetSize(600, 600, webview.HintNone)
    webView.Init(loadAlpine())

To execute go code with alpine we need to call webView.bind("functionName").

webView.Bind("extractSubDirectories", func(sourceFolder string) string {
  folderUrls = extractSubDirectories(sourceFolder)
    tmpl := template.Must(template.New("html").Parse(
      // language=GoTemplate
    `<div>
      {{range $vendor, $folderDetailsArray := .}}
        <div>
          <h3>Vendor: {{$vendor}}</h2>
        {{range $folderDetails := $folderDetailsArray}}
          <ul>
            <li>{{ .Path }} filecount:: {{ .FileCount }}</li>
          </ul>
        {{end}}
        </div>
      {{end}}
     </div>`))
  var html bytes.Buffer
  err := tmpl.Execute(&html, folderUrls)
  if err != nil {
    logger.WritePrint("ERROR: " + err.Error())
  }
  return html.String()
})

To create your first page you call webView.Navigate() and supply it with your HTML. Then call webView.Run()

webView.Navigate(`data:text/html` + `<!doctype html>
<html lang="de" x-data="{ pathInput: '', table : ''}">
    <body style="padding: 2rem">
        <h1>JPEG Sorter</h1>
        <p>Input the folder where the images are stored</p>
        <input type="text" x-model="pathInput"/>

        <button @click="table = ''; table = await extractSubDirectories(pathInput);">analyse folder</button>

        <div x-html=table></div>
    </body>
</html>`)
webView.Run()

Alpine.js

As you can see there are quite a lot of non standard html attributes, this is the magic of alpine.js.
You can create local alpine data variables in the scope of the element:

<html lang="de" x-data="{ pathInput: '', table : ''}">

You can bind input data to the local variables with x-model

<input type="text" x-model="pathInput"/>

But the coolest part comes now. With an @click alpine attribute we can call our go functions from the html. The extractSubDirectories() function we binded earlier in this example.

<button @click="table = await extractSubDirectories(pathInput);">
  analyse folder
</button>

With x-html we can bind the returned html from the go function into our gui.

<div x-html=table></div>

These are the basic steps to get webview and alpine.js working with Go.

GUI Example

You can look at my recent freelance project for a complete example on Github.

JWT authentication for Spring Boot simplified using GoTrue and Supabase

Thomas Schühly — Tue, 01 Mar 2022 22:23:06 +0000

In a quest to have a simpler JWT Authentication flow and not have to deal with security related userdata in my backend I discovered Supabase Auth which is an implementation of Netlify GoTrue.

For Kotlin there is the awesome supabase gotrue-kt library.

In your User Registration and Login Services you need to create a GoTrueClient

val goTrueClient = GoTrueClient.defaultGoTrueClient(
    url = "<base-url>",
    headers = mapOf("Authorization" to "foo", "apiKey" to "bar")
)

If you are using supabase, the base URL will be:
https://<your-project-id>.supabase.co/auth/v1

Then in your signup method you can just call signUpWithEmail().

val authDTO = goTrueClient()
    .signUpWithEmail(credentials["email"]!!, credentials["password"]!!)
websiteUserRepository.save(WebsiteUser(authDTO))

With the default client this returns a GoTrueUserResponse which most importantly contains a id which you then can persist in a WebsiteUser Authentication Pojo which holds information related to the user

With the goTrue Kotlin Library you can also specify a custom return type for example if you turned email confirmation off.

We define our DTO:

data class AuthDTO(
    val accessToken: String,
    val tokenType: String,
    val refreshToken: String,
    val user: User
)

data class User(
    val id: UUID,
    val email: String,
    val phone: String

)

and then create a Client where we pass this dto:

return GoTrueClient.customApacheJacksonGoTrueClient<AuthDTO, GoTrueTokenResponse>(url,headers)

In our Login Method we call signInWithEmail and then return the JWT from the GoTrue Response as Cookie

val repsonse = goTrueClient().signInWithEmail(
  credentials["email"],
  credentials["password"]
)
response.addCookie(
    Cookie("JWT", resp.accessToken).also {
        it.secure = true
        it.isHttpOnly = true
        it.path = "/"
        it.maxAge = 6000
    }
)

But we need to actually verify that the JWT is correct when a User request a page and that the user has the required access rights.

We do this in a JWT Filter that overrides the doFilterInternal method from the OncePerRequestFilter().

When our current SecurityContext Authentication is empty we need to extract the JWT from the Cookie and get the UserID from GoTrue to find the WebsiteUser we persisted earlier.
We then set the SecurityContext with the retrieved WebsiteUser

@Component
class JwtFilter(
    val websiteUserRepository: WebsiteUserRepository
) : OncePerRequestFilter() {
    override fun doFilterInternal(
        request: HttpServletRequest,
        response: HttpServletResponse,
        filterChain: FilterChain
    ) {
        if (SecurityContextHolder.getContext().authentication == null) {
            val auth = SecurityContextHolder.getContext()
            request.cookies?.find { it.name == "JWT" }?.let { cookie ->
                try {
                    goTrueClient.getUser(cookie.value).let {
                       SecurityContextHolder.getContext().authentication = websiteUserRepository.findByIdWithJPQL(it.id)
                    }
                } catch (e: GoTrueHttpException) {
                    if (e.data?.contains("Invalid token") == true) {
                        val oldCookie = request.cookies.find { it.name == "JWT" }
                        oldCookie?.maxAge = 0

                        response.addCookie(oldCookie)
                        response.sendRedirect("/")
                    }
                }
            }
        }

        filterChain.doFilter(request, response)
    }
}

At last we add this filter in our WebSecurityConfiguration:

@Configuration
@EnableWebSecurity(debug = false)
class SpringSecurityConfig(
    val jwtFilter: JwtFilter
) : WebSecurityConfigurerAdapter() {
    override fun configure(http: HttpSecurity) {
        http.formLogin()
            .loginPage("/login")
            .and()
            .logout()
            .deleteCookies("JWT","authenticated")
            .logoutUrl("/logout")
            .logoutSuccessUrl("/")
            // Our private endpoints
            .antMatchers("/konto").hasRole("USER")
            .antMatchers("/admin/**").hasRole("ADMIN")
            .anyRequest().authenticated()
            .and()
            .addFilterBefore(jwtFilter, UsernamePasswordAuthenticationFilter::class.java)
    }

Spring Content - The better way to save and serve files and images with Spring Boot

Thomas Schühly — Thu, 18 Nov 2021 08:01:35 +0000

Problems with saving images directly
How to do it using Spring Content

Save images directly into the DB

Recently I added a feature to my web app where my customer could upload images and display them inside the app.

At first I stored them directly into the db as ByteArray inside an JPA Entity with a @lob annotation

@Entity
class Picture(
    val name: String?,
    val type: String?,
    @Lob
    val data: ByteArray
)

I then returned them as Byte64 encoded string inside my JSON API Response.

Issues

It was working fine in my local dev enviroment, but as soon as I pushed it to prod I encountered an error where my whole json structure needed for my frontend wasn't displayed properly.
The Byte64 encoded image string just stopped in the middle.

I cloned the prod db locally into a postgres container and then the issue occured also locally.

HHH000100: Fail-safe cleanup (collections) : org.hibernate.engine.loading.internal.CollectionLoadContext@2555100f<rs=HikariProxyResultSet@1189576917 wrapping org.postgresql.jdbc.PgResultSet@545635a4>

After some 2am debugging I got a JacksonMapping exception and found the error on stackoverflow.

I thought the solution was too complex and looked for a simpler way.

Learnings

Lessons: Always test with an environment as similiar as production. I can recommend testcontainers for easier integration tests.
Lesson: Use a database versioning tool such as liquibase instead of spring.jpa.hibernate.ddl-auto = update.

With h2 the @lob was correctly saved as bytearray. With postgres the type was assumed to be oid which caused a issue mapping between @lob and the bytea column type.

This was fixed with the hibernate Annotation

@Type(type = "org.hibernate.type.MaterializedBlobType")

Lesson: Dont serve images directly as Base64 encoded string.

Instead use a seperate controller and serve the imagedata raw.

Serialize the entity which stores the data and the metadata.

@JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator::class, property = "id")
@JsonIdentityReference(alwaysAsId=true)

This avoids huge json responses and slowdowns to your website.

Spring Content to the rescue

Then I discovered spring-content which looked like a better alternative than saving them directly to a db, as I could switch between db, filesystem, AWS S3 and MongoDB GridFs by just switching the dependency.

To use Spring Content you just need to add the depencency in your build.gradle.kts

implementation("com.github.paulcwarren:content-jpa-spring-boot-starter:1.2.5")

Add the @ContentId annotation to your file entity. In my case i replaced:

    @JsonIgnore
    @Type(type = "org.hibernate.type.MaterializedBlobType")
    val data: ByteArray

with

@ContentId var imageId: String

Create a ContentStore

interface PictureContentStore : ContentStore<Picture, String>

In your method where you save the Files you need to use the contentstore.setContent() method. It is important to first save the entity.

val picture = pictureRepository.save(Picture(pictureFile.originalFilename, pictureFile.contentType,null))
pictureContentStore.setContent(picture, pictureFile.inputStream)
pictureRepository.save(picture)

Add the handling of pictures to your file entity

@RestController
@RequestMapping("api/picture")
class PictureController(
    val pictureService: PictureService,
    val pictureContentStore: PictureContentStore,
) {
    @GetMapping("/{id}")
    fun getById(
        @PathVariable id: Long,
    ): ResponseEntity<InputStreamResource> {
        pictureService.getById(id)
            ?.let { pic ->
                val content = pictureContentStore.getContent(pic)
                val inputStreamResource = InputStreamResource(content)

                return ResponseEntity.ok()
                    .contentType(MediaType.parseMediaType(pic.type.toString()))
                    .body(inputStreamResource)
            }
        throw NoSuchElementException("There is no Image with the corresponding id")
    }
}

and you are done.

You can look up my source code on Github.

This is my first project as a freelancer. Any suggestions or feedback is very welcome

Deploying docker-compose the easy way, without registry or scp

Thomas Schühly — Sun, 28 Mar 2021 14:29:44 +0000

Ever wondered how you can deploy your locally running docker-compose project to a remote server?
There are several options:

Pushing the images to a container registry like dockerhub, github container registry and pulling them on your server
Saving the images to a .tar archive and copying it over to your server and loading them there explained here

There is a easier way using docker remote host:

The only thing you have to make sure is to install docker, docker-compose onto your server and have a valid ssh key

docker-compose.yml

version: '3.9'
services:
  backend:
    build: spring-backend
    container_name: spring-backend
    image: spring-backend:0.0.1
    expose:
      - "8088"
  frontend:
    build: angular-frontend
    image: angular-frontend:0.0.1
    container_name: angular-frontend
    ports:
      - 80:80
    depends_on:
      - backend
    command: [nginx-debug, '-g', 'daemon off;']

deploy.sh

#!/bin/bash

docker-compose build

for img in $(docker-compose config | awk '{if ($1 == "image:") print $2;}'); do
  images="$images $img"
done

echo $images


docker image save $images | docker -H "ssh://user@serverIp" image load
docker-compose -H "ssh://user@serverIp" up --force-recreate -d
docker-compose -H "ssh://user@serverIp" logs -f
read -p "Press any key to continue... " -n1 -s

With a simple deploy.sh you can build your current state, load them onto your server, run them and attach to the output.