DEV Community: Andrei

Dockerizing Java 10 Spring Boot app

Andrei — Wed, 25 Apr 2018 20:42:54 +0000

In my previous article I’ve been talking about using Java 10 to build and run your Spring Boot app. Now it is time to talk how to put it in Docker container. Better support of containerization was one of the main features of Java 10 release. So, let’s go!

That is fairly simple process, so just take a look on following code snippet:

Here, as you can see, I use multi-stage Docker build file. Quick note: multi-stage builds are supported with Docker 17.05 or higher but result image is backward compatible(at least from my experience). On the first stage it downloads JDK from Oracle’s, but it can make sense to store your own copy somewhere closer to your build server since it isn’t that small — a little bit less then 200mb. So, it is downloaded, unziped and ready to be used. But not so quick.

As you probably remember, Java 9 gave us module system and JDK itself is also modularized. That means we don’t have to bring whole JDK to production but we could build our own striped version specifically for our app’s needs. There is jlink tool provided for that. Just take a look how it is used in the build process. The modules listed there should be enough to run a typical Spring Boot application. In my production case I had to put java.scripting, jdk.scripting.nashorn to the list as well.

On the second stage it copies results from previous one, defines some environment variables, exposes port, copy launch.sh and app’s artifact. Not a rocket science at all.

And here is launch.sh script I’ve just mentioned:

That is just a small helper to ensure that there are all important things provided, etc. But, sure, it is optional.

And that should be enough to put your app to the production. Please, put some comments down if you have some thoughts how to make it better!

Java 10 — migration story

Andrei — Mon, 26 Mar 2018 21:56:57 +0000

As you probably know Java 10 was recently released. Not so many new features. At least for Kotlin developers… But there are several improvements to JVM itself. And, I guess, there will be more in upcoming versions. Which we will get twice a year from now! So, I think it is better to take some time and sync your projects to make it easier in future to migrate with the same speed.

The main project I’m involved into is SpringBoot 1.5.X based service made with Kotlin and it uses Postgres as a data base. At this moment it uses Java 8. Such a great chance to skip Java 9 and jump directly to 10! Being inspired by this thought I have opened my laptop.

I use SDKman to install Java but for Java 10 it delivers broken version for some unknown reason. So this time I had to download bundle from the official website manually.

The biggest issue is that some libraries are no longer part of JDK. That is good news — it is getting cleaner. But as a result, you need to define additional dependencies for your project. You can also include them with command line parameters since they are actually still in JDK but I think it is better to move them into dependencies because as far as I’ve understood they gonna be removed in future releases completely. In my case it was:

The second issue was Jacoco Gradle plugin. If you use it to measure the code coverage there is good news— Jacoco itself supports Java 10. But to make Jacoco Gradle plugin work I had to specify the tool version like this in my gradle build file:

And the last part that was broken — Mockito. That issue could be solved by adding a fresh version of Bytebuddy library like this:

After these modifications, I was able to build and run my project and all tests were green. Don’t be afraid to migrate! Next step would be to prepare Dockerbuild file. But that is a topic for the next article.

Karma.js, Headless Chrome and Docker

Andrei — Sat, 03 Mar 2018 08:50:57 +0000

Today I’ve come back to frontend project I’m involved. I have no idea what have happened while I was away for different things but our Karma.js tests stop working.

The project is done with Vue.js and it becomes for some reason impossible to pass slots to the components as strings. Well. And there was error message suggesting to use Puppeteer instead.

And here is a short snippet to how actually do that. Quick notice, I run my tests inside Docker container…

remove Phantomjs from your package.json
add dependencies to puppeteer and karma-chrome-launcher
update karma.conf.js according to the snippet below

also Chrome requires some system libraries and doesn’t work on default nodejs Docker image. So, I’ve found geekykaran/headless-chrome-node-docker.

And after all that changes my tests are green again!

Kotlin for Pebble

Andrei — Fri, 16 Feb 2018 09:59:15 +0000

The best Smartwatchs are Pebble. Even taking into account that the company producing them was sold, and there will be no more new devices it is still best smartwatchs you can buy.

It is specially true if you are a software developer or just love to play with code. Pebbles provides you with powerful but simple SDK for C and Javascript. And cool web IDE — CloudPebble. As a result, it is super easy to develop apps and watchfaces.

But you probably know I am a big fan of Kotlin, so C and Javascript were not definitely enough for me! Since Kotlin-native at this moment doesn’t support the platforms are used in Pebble I’ve decided to go with KotlinJS.

KotlinJS is a way to transpile Kotlin code to Javascript. As a starting point, I took this repo — https://github.com/pebble/pebblejs. This repo provides you the way to develop apps for Pebble in Javascript locally.

First, you need to follow this https://developer.pebble.com/sdk/install/ instructions to setup PebbleSDK. Also, you need to setup Kotlin command line tools. I am using SDKMan for that. As soon as you have all that you are good to go.

So, I’ve made a fork of pebblejs repo — https://github.com/gimlet2/pebblekt. The first thing to check here is build.sh script. It has three steps:

Build Kotlin code to Javascript with kotlinc-js
Remove unused code with help of kotlin-dce-js
And pebble build to actually produce artifact for Pebble
The last step which is not the part of the script — installation — you can use different options of pebble install command.

The last thing to focus on — src folder:

in js folder you will find original PebbleJS SDK. Also a result of Kotlin transpiling is also could be found here in app.js file.
KotlinJs standard library could be found in kotlinjs folder. It is huge but kotlin-dce-js is there to fix this.
And finally, kotlin folder contains bindings to js code(for example ui.kt ), some helper functions in pebble.kt and app.kt — the application code.

If you build and run this code you’ll get typical Hello world app.

With this project I want to try to expand Kotlin to a new platform. Worth to mention the latest release of Kotlin-native (0.6.0) with a support of STM32 chips. But that is a story for another post.

Prometheus, Python, Flask

Andrei — Fri, 02 Feb 2018 16:43:38 +0000

Python based apps are significant part of the ecosystem in our company. In the era of microservices there is really no difference which technology is in use under the hood. But each service should provide its own API and the way to monitor its state.

In our setup we use Prometheus to aggregate metrics from all our services. And it works fine. Especially with our JVM apps. But that was not a case for all out Pythons till today.

As a historical tradition our Pythons apps are mostly done with Flask. And it is a nice framework. In some cases it is also wrapped with Gunicorn. And here is a tutorial how to make it work with Prometheus:

Add the dependency to your project setup.py to prometheus_client==0.1.1
Inside your application define some metrics you want to collect, for example: FLASK_REQUEST_LATENCY = Histogram(__name__.replace(‘.’, ‘_’) + ‘_request_latency_seconds’, ‘Flask Request Latency’) .
Annotate a method you want to measure with @FLASK_REQUEST_LATENCY.time()
Add endpoint to expose the statistics:

And that’s it! If your app is just plain Flask without Gunicorn. To make it work in multiprocessor scenario you need to make several additional steps:

Create config for Gunicorn:

This config can be provided to gunicorn cli with -c flag.
Add Environment variable prometheus_multiproc_dir that should point to a directory where Prometheus can temporarily store the metrics. If you use Kubernetes update the deployment descriptor to mount Volume with emptyDir type.
And the last thing to do — update code for your /stats endpoint like this:

I hope this short guide can be useful and will help you to make your apps measurable and system more reliable.

Setup Go project - for JVM developers

Andrei — Sat, 06 Jan 2018 00:36:59 +0000

image from https://github.com/zxh0/jvm.go

I have been doing development for JVM platform for quite some time already. It was Java back in time, now it is Kotlin. And everything seems logical here — here is your Maven , Gradle or whatever project description with versioned dependencies, when you do build — the artifact is right here. It gives you a sense of workspace. It is easy to develop multiple projects in parallel — they all are isolated from each other.

But then the new player comes in that perfect world — Go.

$GOROOT? $GOPATH? WTF? Where are project’s dependencies? Is there something like versions? And where are my binaries?!

I believe that such design of the ecosystem has some rational core and reasoning. But from my(probably not just my) experience it looks like completely parallel universe.

To deal with it my first approach was to replicate expected $GOPATH folders structure in the project directory — bin , src and pkg. Then create correct import path inside src folder — like github.com/gimlet2/<someproject> and checkout project itself there. After this manipulations updating of $GOPATH environment variable to point to the project directory. Well, too complicated.

To improve it a little bit I’ve decided to use dep package manager. And it became better. Now for each project, I was able to provide a list of dependencies with exact versions. Maybe that isn’t really the Go way, but I like it.

But still, it was far from ideal. Maybe virtual environment can be helpful here? It is widely used in Python to fix the similar issue. And the first thing I’ve found on Github was goenv. This project isn’t under active development but it does the job.

It is pretty easy to use:

Install it go get github.com/crsmithdev/goenv
checkout your project
then, in your project folder do goenv init github.com/me/myproject
active virtual environment . goenv/activate

Cool, it works! But dep is an issue now. It verifies that your project is located inside appropriate $GOPATH. And goenv creates virtual $GOPATH in ~/.goenv/<project name>. Inside this folder, it also creates bin , pkg and src folders and folders structure that you’ve specified with goenv init command. Almost the same steps I was doing manually before. And it makes a symlink to your actual project folder. So, I’ve improved goenv a little bit to cd into this virtual project folder on activating of the environment.

Now you can run dep ensure right away or start your code editor(personally I use Visual Studio Code for Go development), work with git, etc. I’ve provided this change as a pull request to the original project. But you can also install it from my fork — go get github.com/gimlet2/goenv .

Maybe that setup isn’t the best but it works for me. At least now I know what is where and can setup new project in one command.

8 minutes of fame — fzf talk

Andrei — Wed, 15 Nov 2017 18:36:19 +0000

Today we’ve started a new thing in our company(just in case you wondering it’s Outfittery Gmbh) — “8 minutes of fame”. This thing has one idea behind — small lightning talks 8 minutes each. It expected to be periodical. For now, we’ve agreed to do it once a month but will see how it’ll go. Each session contains 4 slots. No slides allowed * — show code or live demo, or you could just talk for 8 minutes(* — on some exceptional topics it’s ok to have).

But here I want to summarize the talk I gave — How to fzf.

First of all, what is fzf? That is command line tool that should simplify your daily life with power of fuzzy finding. That actually the way how it’s pronounced — FuzzyFinder. It is written in Go lang and could be easily found on Github. The recommended way to install it — checkout the repo to ~/.fzf folder and run install script. It will ask you several questions and you are ready to go!(don’t forget to apply shell config modifications with source command)

What is so cool about it? Well, it’s doing one thing but doing it well. As each Unix tools should. It takes a list of lines to STD/IN and produces a filtered list of lines to STD/OUT. But magic happens in the middle =). Under the hood it supports several filter options: default one — fuzzy finding — you type and it tries to understand what you actually mean, strict match, the strict match from the beginning of the line and strict match on the end of the line. Forgot to mention, it is also possible to filter something out — just prepend your query with !(bang).

After you’ve found what you need(with a query or by manual navigation with cursor buttons) just hit enter or even just pick what you need my mouse click.With -m flag passed it’s allowed to pick several items from the list(just use Tab or Shift+Tab).

Just with this functionality, this tool becomes useful for integration to you interactive scripts. But it provides more — bunch of cool keybindings(and you can extend it). Here is my favorite:

Ctrl+R — fuzzy search on your terminal history;
Ctrl+T — find files in current directory and deep inside and return the result to the command line. Super useful when you type some command(let’s say less) and then you hit Ctrl+T and pick file you want;
Alt+C — cd to selected directory;
ssh ** and then Tab — selecting a host from your hosts file or ssh config;
kill -9 and then Tab — selecting process to kill;

And there is more, those are just my top list I use daily.

There are several more things worse to mention: Vim integration and new feature — preview. But, I hope, if you haven’t tried(or maybe you are using fzf already for many years…), just check it. More details you always can find in README file. And I’m going to prepare my next talk for “8 minutes of fame”.

Summarize on “Spring Microservices” book by Rajesh RV

Andrei — Sat, 30 Sep 2017 12:33:11 +0000

Recently I’ve read “Spring Microservices” book by Rajesh RV and here I want to try to summarize it. The book was published in June 2016. But, it seems, it is still relevant.

Microservices isn’t just for developers pleasure but also business demand.

Yes, sure, it’s extremely interesting and fun to plan and implement scalable and distributed systems but developers should always keep in mind why we are doing that — to evaluate faster on business processes, reduce the cost of the experiment.

Meaningful health check is important

Self-healing is significant for microservice architecture. And to provide it your service has to report its health state accurate. For example, your service is deployed in Kubernetes cluster and exposes /health endpoint. Then it will be possible to restart it in case of failure and reroute traffic to a health node instead. Also, it helps you to understand your app better, know all significant dependencies and requirements.

Spring Cloud is a good choice for microservices

…if you do not afraid of such vendor-lock. In case if your platform is 100% based on Spring stack it seems pretty logical to try to solve microservices challenges with help of Spring Cloud. Bunch of cool project from Spring Cloud Config till Spring Cloud Streams helps you to build reliable and manageable system. But if you run your services for example on top of Kubernetes you will get alternative solutions for those issues for free(like service discovery, load balancing, etc.).

Distributed tracing brings transparency into the system

On that field Spring provides us Spring Cloud Sleuth solution. It works on top of Zipkin , Cloudera or Dapper. As result of integration, you will be able to add to your log messages what’s called Trace-Id and Span-Id, so you can track request chain across entire system for example in Kibana. Super useful for debugging, dependency analysis, etc.

Autoscaling

…isn’t trivial process. Beside of technical support of autoscaling from your platform(for example, Kubernetes has it but limited, but there are some third-party extensions to improve it), you really need to think about scaling policies, pay attention what limits your app performance.

Of course, there are much more useful tips in this book. Here is only some point I focus my attention on. In total — it worth to spend some time and read.

Sentri.io — errors handling

Andrei — Sat, 23 Sep 2017 01:49:47 +0000

Today I want to talk about one service I’ve discovered for myself last week. As you probably already got from article title it is Sentry.io. This open source project helps you to aggregate and monitor information regarding errors happening in your application. Beside opensource version, there is also hosted paid(with free Hobbyist plan) version which supposes to be the same.

This service provides wide support for different programming languages like Java, C#, Elixir, Go, Javascript, Node, Perl, PHP, Python, etc. The integration process is super easy — you just need to add proper dependency into your project and provide a little bit of configuration(at least access token), and you are ready to go.

In my case, it was a Spring-Boot application with logback logging framework. Both those technologies are supported by Sentry.io. So, I’ve placed io.sentry:sentry-spring and io.sentry:sentry-logback libraries into projects Gradle script. Then I’ve created a new project inside Sentry’s dashboard and generated a token for it. Next step — configuration inside the project. And that is something I don’t like about all that story. Maybe there is some way around but I can’t find it — so, I had to duplicate configuration twice — for logback and for spring-boot respectively. I’ve added sentry.properties file in project’s resource folder with dsn key inside and sentry.dsn key to my application.yaml. Also, the configuration of logback appenders was updated with the following block:

And as a final step additional beans for Spring to handle unhandled exceptions and provide additional request context(yes, a little bit of Kotlin ):

That’s it.

But there is much more than just aggregation of errors. Sentry provides useful report channels integrations. For example, it can send notifications by email or to S lack when there is a new type of error. Or notify you if same error happened more then 100 times per hour. All those rules are extremely flexible and can be configured with Sentry’s dashboard.

And there is integration with popular issue trackers like Jira. So, it will create a new bug ticket for each new error with stacktrace and all context it has. That should make it easier to fix them.

So far looks good.

Kafka usage — tips&hints

Andrei — Sun, 13 Aug 2017 21:30:42 +0000

Today we will talk about Apache Kafka. As you probably already know, Kafka is “a distributed streaming platform”, so it could be used as Queue, Message Bus, etc.

But this story isn’t about Kafka it self. It is about issues I faced recently developing one Kafka based service. Small application consuming messages from defined topic and validating them according to the schema. Nothing like rocket science. But.

Upfront check version of your Kafka server — even minor version difference could break API. Like server version 0.10.0.1 isn’t compatible with client 0.10.1.1. Just pay attention.
Check which compression mechanism is used on your Kafka server. GZip and Snappy are supported. In my case it was Snappy, and you know what? It requires glibc. And my base docker image was Alpine. And Alpine does not have support of glibc. So, choose the right base image.
If you use Kafka Streams API and want to split one stream into two based on some predicate your could pass this predicate to _branch_ method along with predicate that always returns true — some sort of _everithingElsePredicate(_Predicate { _, _ -> true }).
kafka-console-consumer and kafka-console-producer — your best friends to debug your Kafka app.
If you are going to scale you app to process some topic in parallel way don’t forget to increase number of partitions on that topic. But keep in mind that it is only possible to increase number of partitions but there is no way back.

That’s all I have in mind to mention.

Code sharing between Kotlin and Kotlin-Native

Andrei — Thu, 20 Jul 2017 22:40:39 +0000

Nowadays when popularity of Kotlin grows faster then ever JetBrains puts a lot of energy to bring us new Kotlin universe — Kotlin-Native. Kotlin-Native allows us to build native apps. It based on LLVM compiler and supports various of platforms. It still somewhere far from being production ready but we already have the third release — version 0.3

I have one let’s say pet-project I’ve already wrote about — kottpd. It is simple HTTP-server written entirely in Kotlin. From this two facts I’ve made obvious decision — I need to try to port my library to Kotlin-Native! But that is not only for fun. There is one dream behind it — to make it possible to build web apps in Kotlin but with possibility to run them with native performance.

Well, to be honest, it is only an experiment for now. But let’s see where this experiment will lead later.

As a first step I’ve cut all unimportant parts out — SSL support, files serving, exception handling and even multi-thread support. And it still was able to serve HTTP requests. Great! Let’s move on.

The second step was a little bit harder. I had to go through all codebase and remove all direct references to JVM classes. In my case it was java.net. ServerSocket , java.net. Socket , java.io. OutputStream , java.io. OutputStreamWriter , java.io. PrintWriter , java.io. BufferedReader , java.io. InputStream , java.io. InputStreamReader , java.io. IOException , java.lang. System. Not that big list.

Instead of all those classes I’ve introduced corresponding interfaces and exceptions. And that help me to isolate all platform dependent logic from, let’s say, business logic of the library. On that moment I did check how it works once again. And it was fine. Time to move to Kotlin-Native!

The code was copied to separate folder. And I had no idea what to do next. But, thanks to JetBrains team, we have some examples that come along with Kotlin-Native. For inspiration I choose socket sample. There I took build.gradle file and sockets.def for configuring C interop. On that moment I realized how far Kotlin-Native is from production use right now — there is no IDE support. That remind me old good times at school with Pascal — change code, compile, read error messages, fix code =)! And that what we have now with Kotlin-Native. Most of the time error messages give you enough information to actually get what is wrong.

Beside pure tooling I’ve faced not so many issues — fun main(args: Array<String>) — must be in a file without package, type conversions from native world and back are two biggest of them.

But anyway I made it work! In one thread, crashing time to time with segmentation fault but still serving HTTP requests!

There is more thoughts to improve it — bring all pieces removed due to simplification back, introduce multi-threading and improve build process to make it actually one application with common code base!

If you interested in this experiment please follow on github.

Kottpd — HTTP Server in pure Kotlin

Andrei — Wed, 16 Nov 2016 22:31:51 +0000

In today’s post I would like to talk about one of the projects I’m working on. I define it as light HTTP framework and it is made in pure Kotlin. Based on these two facts I’ve named it kottpd.

This project was born during long bus ride. There was no Internet and I decided to write something without external dependencies. What could be better then implementing well-known text-based protocol?!

Not all pieces were easy to find in the palaces of the mind but in couple of hours first version was ready to serve standard HTTP queries. The design of API was made with inspiration from Spark-Java. So, it looks quite similar.

What we could find under the hood? First of all it is Cached ThreadsPool. It helps to allocate and reuse threads for each HTTP request. Then in infinite loop the server waits for incoming connections, reads it, tries to match against configured routes and passes to corresponding handler if found. Beside request handlers you could also specify Before and After filters to implement some sort of middleware(for example — login, transactions, etc.).

What for is it useful?

Well, it could be used if you want to make a web application, or an REST API , or just to serve a static content(it supports that too!). But you want to do it in a lightweight way. And you like Kotlin! That it is a perfect choice for you. Have I mentioned it supports SSL?

To explain it better let me show you an example that could be easily found in the README file. But first of all you need to add it as a dependency to your project. The library is published into maven central repository, so you should not face any issues with it.

Here we create an instance of Server class, define path to the folder with static content we want our server to serve. Then we define several request handlers(yes, you could use regular expressions for that). And then there is a block with filters. They could be global — when there is no specific path defined — and path specific. And of course you could define some exception handlers too. The only thing is left to be done — call start() method. By default your application will listen on the port 9000. But you could just pass any port value you want as argument to start() method or define it with server.port system property(-Dserver.port=8080).

Conclusion

I hope you got interested with my project. Please, try to use it in your applications and I will be really happy to see any feedback from you. If you have never used Kotlin before, it is a good moment to start!

Do Kotlin — do the Web!