DEV Community: Daniel Santos

🎯 Sending flat files to Kafka

Daniel Santos — Wed, 03 May 2023 06:41:54 +0000

Hello 🖐️ !

Following the mechanisms described here to send CSV files into Kafka in JSON using kcat through shell pipelines, we packed 📦 all of that in a new home-made tool named producer-file. Here are the features :

💨 Sending line by line
💪 CSV to JSON conversion
🔦 Support message headers by file and/or by line
⚡️ Delta-changes mode (only send changes occurred after last send)

☝️ More about the shell way :

⌨️ Pipe xlsx files into/from Kafka... From cli with (k)cat 🙀

adriens for opt-nc ・ Oct 10 '22

#tutorial #productivity #kafka #linux

Motivations

Using jq + kcat with shell pipelines do the job well, but this doesn't fit the company's requirements today about software packaging and logging capabilities 🤷🏻‍♂️.

📽️ Demo

Short video demonstrating how to use this tool to send a CSV file in JSON into Kafka within a Docker environnement :

📖 Ressources

Finally, my first steps on Quarkus 🚀, as an Spring-boot developper

Daniel Santos — Fri, 22 Jul 2022 04:47:35 +0000

Hi everyone 🙋🏻‍♂️,

We just started trying Quarkus 🙏

api-transitaires was the first, it's time to do some quick feedback now 📢

👉 First, I know that it's already a 3 years old framework (some npm libs don't live that long), but in some JCompanies the investissement to set up some Spring[-Boot] 🍃 developers has been challenging, so the replacement has to be valuable (or fin valab)

Why to move from Spring-Boot ?

We have a lot of Sping-Boot apps and we are going forward to k8s in serverless way, so we did some tests with Spring-Native.

It worked 👍, but not so smoothly because of Spring's long history 🏭 which led us to more recent alternatives like 👉 Quarkus.

Why this app ?

api-transitaires it's a very basic one-endpoint REST API that serves the list of freight forwarders that manage customs clearance on New-Caledonia 🇫🇷.

The main goal was to try Quarkus in a end to end way, in our GitHub CI : native app, Docker..

Ok, let's go now..

1# IDE integration 📝 - VSCode plugin

I used since almost 2 years VScode, I was happy to discover a Quarkus's plugin who does the job :

generate new project : like SprinBoot does 😉
launch and debug : ok, relies to mvn but it works most of the time 🙄.. sometimes it switched off without saying goodbye _(ex: after adding a Quarkus's extension). Otherwise, I don't like that they preferred to use a specific menu for that rather than VSCode's classic "run and debug" 😒

⭐️ What I appreciate :

faster code change / test cycle out-of-the box 📦 : do the change, switch to browser and the change is already applied (I experienced a refresh the browser twice cycle with Spring-boot Devtools)

☝️ Nice to have :

autocompletion on application.properties as Spring-Boot's plugin does

2# Coding a Rest API 👨🏻‍💻 - the standard way

While Spring has chosen to make its own annotations (almost a copy of the standard), with Quarkus we go back to the Java standard's ones. Ok, JSR's are not so fancy 🌈, but less intrusive 🐙.

3# Docs 📖 - self sufficient 👍

No need to rely on other documentations to do basic things, Quarkus's guides are in my opinion succinct, practical (getting started style) and well referenced from google 🤓.. 1st result is the official one !

Game changing after years of Spring's empty shell getting started guides or JavaDoc style reference documentation 😵‍💫

👉 I used :

4# Fast 🏃💨 - minimum included

Lighter & faster, some fresh metrics from a recent migration from Spring-Boot (colisnc-api) :
- startup time: from 1s8 to 0.5s (is native really needed with Quarkus ? 🤔)
- CI native build : from 14min to 5min
- Docker Native image size : from 40Mo to 30Mo
Minimum included : it's a philosophy and I like it ❤️ (even the yaml configuration is not included : an extension is needed)

5# Native ⚡️ - the way is paved 🛣

It seems that it's was on the Quarkus cookbook since the beginning, not added at the end : Spring Native is still in experimentation.
Light by default : some basic things are not provided by default like SSL protocol support, but it's easy to do quarkus.ssl.native=true

6# Monitoring / dev tooling - Side dishes 🥗

less than actuator, but enough in my mind : health, Swagger
Nice to have online web dev that gives some basic but useful metrics (as others do since a while, like Symfony 😉)

Resources

api-transitaires : first app
colisnc-api : first migration from Springboot done within 1h30 ⏱

📑 #jfokus2022 : Quarkus [...] back from the Future

adriens for opt-nc ・ Jul 12 '22

#quarkus #java #kubernetes #showdev

Cheers 🍻

Knative : the easy way to serverless a java app

Daniel Santos — Fri, 04 Mar 2022 11:01:57 +0000

Hi everyone 🙋🏻‍♂️,

Let me introduce you Knative, a friend (or extension) of Kubernetes ☸, an easy way to stop apps when not used and so, save our planet 🌏 from CO₂ attack!

Works out of the box for a Docker/OCI image of a serving app (like a REST app), but needs some changes for event consumers one's.

As the app starts on first request, is better to have a rabbit 🐰 startup performance level degree than a tortoise's 🐢 one (only works in this case 🏁). In Java ☕️ apps, native-images comes to rescue 🚀 (more about that in the next post).

You can see a working example of Knative serving a native java REST app in this interactive Katacoda tutorial, or if you are a lucky 🇫🇷French🇫🇷 speaker, you can simply watch this demo video: .

Cheers 🍻