DEV Community: Waldemar Panas

BlockHound can save you from waking up at midnight

Waldemar Panas — Sun, 29 Aug 2021 00:00:00 +0000

BlockHound detects problems that take down your reactive applications

It might not be that self-evident, but your non-blocking reactive application can block and eventually run out of computing power. It's simple to make that error.

All you have to do is use a blocking operation, and everything
falls apart like a house of cards. It can disguise itself as an undocumented method with an unintended side-effect or a misused dependency.

BlockHound can save you, so use it. All you need to do is to add it to your tests suite. Just like that.

class SuperImportantServiceTest {
    @BeforeAll
    fun setupAll() {
        BlockHound.install();
    }

    ...
}

For more information on adding a dependency to your project, you can check an official getting starter guide.

BlockHound works with Project Reactor & Kotlin coroutines

BlockHound is a tool created by Project Reactor's team, but you can use it with many different technologies. Here you can see examples of using BlockHound with Reactor & Kotlin coroutines. It supports the JVM's SPI mechanism for easy integration with any technology.

Sadly, RxJava 3 is unsupported at the moment. BlockHound still supports RxJava 2.

Nonetheless, BlockHound's versatile, and there is no reason you can't use it right now. Unless you use RxJava 3 😉

Together with Testcontainers can even detect problems with your external dependencies

Some blocking operations might hide inside dependencies that you use. Let's say that you use blocking MongoDB operations by mistake. If you only mock operations on a database, then BlockHound won't help you.

Use Testcontainers to solve this problem. It's a tool that spawns containers with dependencies during running tests. In doing so, you connect to an authentic MongoDB instance just for your tests. BlockHound instantly catches that you use blocking API and warns you about it.

Summary

Putting it all together, go and use BlockHound immediately. It's an essential tool for every software developer that creates reactive applications.

BlockHound is:

easy to use & saves your good night sleep
works with the most popular non-blocking JVM technologies
together with Testcontainers is a bulletproof solution

Best Machine Learning tutorials

Waldemar Panas — Sun, 26 Jan 2020 00:00:00 +0000

Introduction

The best way to learn something is to start doing it. This thought guides me through my career and it never failed me.In this short article, I will give you a quickstart to master machine learning.

Before you start doing tutorials about machine learning, you should learn Python. There are many resources to do that. For newbies I recommend learnpython.org. For those, who are familiar with programming, go straight to Learn Python 3 in Y minutes.

The sad truth of machine learning is that the most work is with preparing the data. The best library to do it in Python is pandas. Fortunately, you can learn it in 10 minutes tops.

Main Part

Best tutorials are not enough to start doing machine learning. You must have data, a lot of data. It is the heart and blood of machine learning.

Fear not! You will find tutorials and the data at kaggle. All you need to do is sign up, and you can start learning right away. You don’t need to install any software at your computer. All the tutorials are interactive Python environments called notebooks. They are runnable in the browser without preparation.

Here you can find the most liked tutorials prepared for beginners. If you aren’t a beginner, then kaggle gets you covered as well. You can take a part in competition and win money! Here is the list of current competitions. If taking part in a competition is too much for you right now, you can challenge yourself another way. Try to find solution for a competition that is already over. You can also pick any of available datasets and start training estimators on your own.

Conclusion

Learning new skill in 2020 is super easy. One of the new skills that you will learn this year can be machine learning. Follow my article and start learning yourself and your machine right now.

Last, if you want to expand your knowledge even more, be sure to follow the blog Machine Learning Mastery. The author is Jason Brownlee PhD. Reading his posts is a great way to complement the knowledge gained from tutorials.

How to add Disqus to your GitHub Pages

Waldemar Panas — Sun, 19 Jan 2020 00:00:00 +0000

Add a partial with Disqus

Create a file _includes/disqus.html with following content. Then you need to replace YOUR_DISQUS_NAME with shortname that is identifying your website on Disqus.

{% if page.comments %}
<div id="disqus_thread"></div>
<script>
var disqus_shortname = YOUR_DISQUS_NAME;
var disqus_config = function () {
this.page.url = "{{page.url | absolute_url }}";
this.page.identifier = "{{page.id}}";
};

(function() {
var d = document, s = d.createElement('script');
s.src = 'https://' + disqus_shortname + '.disqus.com/embed.js';
s.setAttribute('data-timestamp', +new Date());
(d.head || d.body).appendChild(s);
})();
</script>
<noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>
{% endif %}

Create a layout with comments

In folder _layouts create a new HTML file blog.html. This layout will be extending default post layout with previously created partial.

---
layout: 'post'
---

{{ content }}

{% include disqus.html %}

Configure the new layout for all your posts

Now all you need to do to enable Disqus in your post is:

set blog as default layout for your posts,
enable flag comments by default.

To do so append file _config.yml with that configuration.

defaults:
  -
    scope:
      path: ""
      type: "posts"
    values:
      layout: "blog"
      comments: true

How to disable comments in selected post

Change comments flag to false at the begging of post file.

---
title: "My post without comments"
comments: false
---

Getting started with RSocket

Waldemar Panas — Mon, 22 Jul 2019 00:00:00 +0000

In this article I will present you how to use RSocket with Spring Boot.

What is RSocket?

RSocket is a bi-directional message-driven communication protocol. It features an advanced communication flow beyond standard request/response. You can response with a single message, a stream of messages or don’t response at all. A server can also start a communication on an established channel with a client.

RSocket allows you to communicate using following transport protocols:

TCP
WebSocket
Argon
HTTP/2 Stream

If you want to know more, you can watch Ben Hale’s presentation from Spring I/0 2019.

RSocket TCP Server

RSocket server is the easiest part of the whole project. All you need to do is to go to Spring Initializr and select Spring Boot, at least version 2.2.x. Then, select RSocket as a dependency.

To configure RSocket server with TCP protocol, add following lines to application.properties. This configuration will open RSocket at port 7000.

spring.rsocket.server.transport=tcp
spring.rsocket.server.port=7000

Last step required is to add some controllers and you are ready with the server. You can try something like that:

@Controller
class ServerController {
    @MessageMapping("queue")
    fun helloQueue(incomingMessage: Message): Mono<Message> 
        = Mono.just(Message("Hello ${incomingMessage.message}"))
}

data class Message(val message: String)

RSocket TCP Client

Client application is very similar. Go to Spring Initializr and bootstrap it with RSocket and Spring Reactive Web.

To communicate with RSocket over a TCP port 7000, you need to create a RSocketRequester. For the TCP protocol it can look like this.

@Configuration
class ClientConfiguration {
    @Bean
    internal fun rSocketRequester(rSocketStrategies: RSocketStrategies): RSocketRequester 
        = RSocketRequester.builder()
            .rsocketStrategies(rSocketStrategies)
            .connect(TcpClientTransport.create(7000))
            .block()!!
}

Your client application can be a standard reactive web application that uses HTTP. To test communication with configured server add a following endpoint.

@RestController
class ClientController(private val rSocketRequester: RSocketRequester) {
    @GetMapping("/hello/{name}")
    fun hello(@PathVariable("name") name: String): Mono<String> = rSocketRequester.route("queue")
        .data(Message(name))
        .retrieveMono(Message::class.java)
        .map { it.message }
}

data class Message(val message: String)

RSocket WebSocket server

WebSocket requires you to add Spring Reactive Web dependency to server application. RSocket port will be the same as Web application port. Instead configuring port, you need to define mapping for RSocket endpoint. In this example, you will use /rsocket path. Configure following properties in application.properties file.

spring.rsocket.server.transport=websocket
spring.rsocket.server.mapping-path=/rsocket

Controllers that you created for TCP server don’t need to change.

RSocket WebSocket client

Your server might be already started at port 8080. Change client port to another in application.properties, i.e. 8081.

server.port=8081

Configuration for client with WebSocket transport protocol is like previous one. All you need to do is provide ClientTransport.

@Configuration
class ClientConfiguration {
    @Bean
    internal fun rSocketRequester(rSocketStrategies: RSocketStrategies): RSocketRequester {
        val uri = URI.create("ws://localhost:8080/rsocket")
        return RSocketRequester.builder()
            .rsocketStrategies(rSocketStrategies)
            .connect(WebsocketClientTransport.create(uri))
            .block()!!
    }
}

Controllers that you created for TCP client don’t need to change.

Summary

Spring Boot made it simple to create application that use RSocket. I hope that those examples show you how to do it. For more information about RSocket in Spring Boot visit official reference guide.

Using Dagger 2 in Android and Kotlin

Waldemar Panas — Thu, 20 Jun 2019 00:00:00 +0000

What made me do this article?

I started programming Android applications few months ago. I went through fundamental and more advanced guides. None of them mentioned dependency injection. Without it I struggled to make proper unit tests. I decided that I have to make it work.

I looked around and found Dagger 2. The project developed by Google that provides dependency injection for Android applications. Sadly, I had a hard time making it work with Kotlin.

I have read many articles, but they had only scraps of necessary information. Combining them together I made it work. This post is a summary of my small research. It won’t be about Dagger 2 basics. If you want to know more, read official user’s guide. Those are articles that inspired me:

How to setup Dagger 2 with Kotlin?

First of all, add dependencies to app/build.gradle and enable kapt.

...

apply plugin: 'kotlin-kapt'

android {
    ...
}

dependencies {
    ...
    implementation 'com.google.dagger:dagger-android:2.x'
    implementation 'com.google.dagger:dagger-android-support:2.x'
    kapt 'com.google.dagger:dagger-android-processor:2.x'
    kapt 'com.google.dagger:dagger-compiler:2.x'
}

Provide modules to inject

Organize your code in modules that provide dependencies. Activities and fragments are dependencies too. They need to be in special abstract modules that use @ContributesAndroidInjector annotation on methods.

@Module
abstract class ActivityProvider {
    @ContributesAndroidInjector
    abstract fun mainActivity(): MainActivity
}

For other objects write modules with methods annotated with @Provides.

@Module
class CoffeeModule {
    @Provides
    @Singleton
    fun coffeeService() = CoffeeService()
}

class CoffeeService {
    fun prepareCoffee() = "Here! Espresso for you"
    fun fetchCoffee() = "espresso"
}

OrderService depends on CoffeeService. You don’t need to use @Inject annotation to do this with Dagger.

@Module
class OrderModule {
    @Provides
    @Singleton
    fun orderService(coffeeService: CoffeeService)
     = OrderService(coffeeService)
}

class OrderService(private val coffeeService: CoffeeService) {
    fun askForOrder() = "Would you like some ${coffeeService.fetchCoffee()}?"
}

Prepare your Application

Next add AppComponent that extends AndroidInjector<DaggerApplication>. It is vital that one of the included modules is AndroidSupportInjectionModule.

@Singleton
@Component(modules = [AndroidSupportInjectionModule::class, ActivityProvider::class, CoffeeModule::class, OrderModule::class])
interface AppComponent : AndroidInjector<DaggerApplication>

Add your own DaggerApplication implementation. This will be the starting point of your Android application.

class InjectionExampleApplication : DaggerApplication() {
    override fun applicationInjector(): AndroidInjector<out DaggerApplication> {
        val appComponent = DaggerAppComponent.builder()
            .build()

        appComponent.inject(this)

        return appComponent
    }
}

Don’t forget to add attribute android:name in AndroidManifest.xml.

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
          package="com.github.wpanas.injectionexample">

    <application
        android:name=".InjectionExampleApplication"
        ...
    >
        ...
    </application>

</manifest>

From now on, dependencies registered by modules in AppComponent will be available for injection.

Inject dependencies

The last step is injecting those dependencies into your activities. Change parent class of your MainActivity from AppCompatActivity to DaggerAppCompatActivity. Then add your dependencies as lateinit var properties with @Inject annotation.

class MainActivity : DaggerAppCompatActivity() {

    @Inject
    lateinit var coffeeService: CoffeeService

    @Inject
    lateinit var orderService: OrderService

    ...
}

The same you can do with all activities and fragments. Add prefix Dagger- to theirs parents class names. This will ensure that dependencies are injected. Don’t forget to add those activities and fragments to module like in ActivityProvider.

Summary

Those simple steps enables you to make Dagger 2 work with Kotlin. All this code is captured in this repository. For more complex example you can also check out this project of mine.

Spring Batch - JPA to XLSX with Apache POI

Waldemar Panas — Sat, 24 Feb 2018 00:00:00 +0000

What is Apache POI?

Apache POI is a Java library that allows reading and writing Microsoft Documents. It supports Excel, Word, PowerPoint and even Visio.

In this article I will focus on Excel and its current file format XLSX. Apache POI provides low-memory footprint API, SXSSF, for writing to XLSX files. You can specify how many rows are processed in the memory by setting a sliding window. If number of rows exceeds the size of the window, they are flushed to temporary files. At the end all data is transferred to specified by you destination file.

Spring Framework has its own library for processing large amounts of data. It is called Spring Batch. Spring Batch has support for CSV files, but natively doesn't support XLSX files. You can achieve storing data to XLSX with support of Apache POI SXSSF API.

Spring Batch SXSSF Example

Let's say you have a bookstore. You also have an application that allows you to store information about books available for sale.
You want to make an order for new books. Your deliverer wants you to send him an XLSX file with list of books to deliver.

Below is JPA representation of your bookstore's application logic. You have a Book model and a repository that keep information about books in a database.

@Data annotation is part of Project Lombok. It generates toString, equals,
hashCode, getters and setters methods for all properties.

@Data
@Entity
public class Book {
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;
    private String title;
    private String author;
    private String isbn;
}

@Repository
public interface BookRepository extends PagingAndSortingRepository<Book, Long> {
}

To process data from a JPA repository you need to create a RepositoryItemReader and a configuration that enables batch processing and JPA repositories.

@Configuration
@EnableBatchProcessing
@EnableJpaRepositories
public class BatchConfiguration {
    private static final int CHUNK = 100;

    @Bean
    public ItemReader<Book> bookReader(BookRepository repository) {
        RepositoryItemReader<Book> reader = new RepositoryItemReader<>();
        reader.setRepository(repository);
        reader.setMethodName("findAll");
        reader.setPageSize(CHUNK);
        reader.setSort(singletonMap("id", Sort.Direction.ASC));
        return reader;
    }
}

Fetched books are going to be written in a spreadsheet. All spreadsheets are kept in a Workbook, so before that you have to create one.

@Bean
public SXSSFWorkbook workbook() {
    return new SXSSFWorkbook(CHUNK);
}

SXSSFWorkbook number of stored in memory rows is explicitly set to CHUNK. With Workbook you can create a spreadsheet called Books. It will be passed to BookWriter.

@Bean
public ItemWriter<Book> bookWriter(SXSSFWorkbook workbook) {
    SXSSFSheet sheet = workbook.createSheet("Books");
    return new BookWriter(sheet);
}

BookWriter is a class that adds a new row to a spreadsheet for each book
in the repository. Book properties are saved as cells in columns.

public class BookWriter implements ItemWriter<Book> {
    private final Sheet sheet;

    BookWriter(Sheet sheet) {
        this.sheet = sheet;
    }

    @Override
    public void write(List<? extends Book> list) {
        for (int i = 0; i < list.size(); i++) {
            writeRow(i, list.get(i));
        }
    }

    private void writeRow(int currentRowNumber, Book book) {
        List<String> columns = prepareColumns(book);
        Row row = this.sheet.createRow(currentRowNumber);
        for (int i = 0; i < columns.size(); i++) {
            writeCell(row, i, columns.get(i));
        }
    }

    private List<String> prepareColumns(Book book) {
        return asList(
                book.getId().toString(),
                book.getAuthor(),
                book.getTitle(),
                book.getIsbn()
        );
    }

    private void writeCell(Row row, int currentColumnNumber, String value) {
        Cell cell = row.createCell(currentColumnNumber);
        cell.setCellValue(value);
    }
}

To complete transfer of books from a JPA repository to a XLSX file, you need to create a Job. Every Job has at least one Step to make.

@Bean
public Step step(ItemReader<Book> reader, ItemWriter<Book> writer) {
    return stepBuilderFactory.get("export")
            .<Book, Book>chunk(CHUNK)
            .reader(reader)
            .writer(writer)
            .build();
}

@Bean
public Job job(Step step, JobExecutionListener listener) {
    return jobBuilderFactory.get("exportBooksToXlsx")
            .start(step)
            .listener(listener)
            .build();
}

When a Job status changes, a JobExecutionListener is called. At the completion of the Job, you can save books from Workbook to selected XLSX file using FileOutputStream and dispose created Workbook.

@Slf4j
public class JobListener implements JobExecutionListener {
    private final SXSSFWorkbook workbook;
    private final FileOutputStream fileOutputStream;

    JobListener(SXSSFWorkbook workbook, FileOutputStream fileOutputStream) {
        this.workbook = workbook;
        this.fileOutputStream = fileOutputStream;
    }

    @Override
    public void afterJob(JobExecution jobExecution) {
        BatchStatus batchStatus = jobExecution.getStatus().getBatchStatus();
        if (batchStatus == COMPLETED) {
            try {
                workbook.write(fileOutputStream);
                fileOutputStream.close();
                workbook.dispose();
            } catch (IOException e) {
                log.error(e.getMessage(), e);
            }
        }
    }

    @Override
    public void beforeJob(JobExecution jobExecution) {

    }
}

Summary

Spring Batch is very elastic, and it can help you process big data chunks with any available library. This example shows how to do it with Apache POI, but you can easily changed it for your own purposes. Remember to maximize your performance with setting chunks. When the RAM memory will be exceeded, the speed of data processing will drop significantly.

Project created for this article is available on GitHub.

SDKMAN! - Multiple versions of Java

Waldemar Panas — Mon, 25 Dec 2017 00:00:00 +0000

What is SDKMAN!

SDKMAN! is a development tool that allows you to install and manage Java, Scala, Groovy, Maven and many other tools, libraries and programming languages united under JVM platform. It is created for GNU/Linux's and MacOS' users.

Installation

The first step is to paste this command in your terminal and submit it.

curl -s "https://get.sdkman.io" | bash

This will launch the process of installation, which might require your input. After successful installation paste this command and you are ready to work.

source "$HOME/.sdkman/bin/sdkman-init.sh"

Usage

You can list all available versions of desired SDK using below command:

sdk list <sdk>

Here is output of this command for Groovy.

sdk list groovy

================================================================================
Available Groovy Versions
================================================================================
     3.0.0-beta-3        2.4.8               2.1.4               1.7.6          
     3.0.0-beta-2        2.4.7               2.1.3               1.7.5          
     3.0.0-beta-1        2.4.6               2.1.2               1.7.4          
     3.0.0-alpha-4       2.4.5               2.1.1               1.7.3          
     3.0.0-alpha-3       2.4.4               2.1.0               1.7.2          
     3.0.0-alpha-2       2.4.3               2.0.8               1.7.1          
     3.0.0-alpha-1       2.4.2               2.0.7               1.7.0          
     2.6.0-alpha-4       2.4.1               2.0.6               1.6.9          
     2.6.0-alpha-3       2.4.0               2.0.5               1.6.8          
     2.6.0-alpha-2       2.3.11              2.0.4               1.6.7          
     2.6.0-alpha-1       2.3.10              2.0.3               1.6.6          
     2.5.8               2.3.9               2.0.2               1.6.5          
     2.5.7               2.3.8               2.0.1               1.6.4          
     2.5.6               2.3.7               2.0.0               1.6.3          
     2.5.5               2.3.6               1.8.9               1.6.2          
     2.5.4               2.3.5               1.8.8               1.6.1          
     2.5.3               2.3.4               1.8.7               1.6.0          
     2.5.2               2.3.3               1.8.6               1.5.8          
     2.5.1               2.3.2               1.8.5               1.5.7          
     2.5.0               2.3.1               1.8.4               1.5.6          
     2.4.17              2.3.0               1.8.3               1.5.5          
     2.4.16              2.2.2               1.8.2               1.5.4          
     2.4.15              2.2.1               1.8.1               1.5.3          
     2.4.14              2.2.0               1.8.0               1.5.2          
     2.4.13              2.1.9               1.7.11              1.5.1          
     2.4.12              2.1.8               1.7.10              1.5.0          
     2.4.11              2.1.7               1.7.9                              
     2.4.10              2.1.6               1.7.8                              
     2.4.9               2.1.5               1.7.7                              

================================================================================
+ - local version
* - installed
> - currently in use
================================================================================

Installation is effortless. Select any of available versions or just skip it and install the currently stable version.

sdk install <sdk> (<version>)

Let's try install Java 11. Paste sdk install java 11.0.4-hs-adpt and hit Enter.

sdk install java 11.0.4.hs-adpt

Downloading: java 11.0.4.hs-adpt

In progress...

######################################################################## 100,0%

Repackaging Java 11.0.4.hs-adpt...

Done repackaging...

Installing: java 11.0.4.hs-adpt
Done installing!

Do you want java 11.0.4.hs-adpt to be set as default? (Y/n): 

Setting java 11.0.4.hs-adpt as default.

All installed Java versions will be available in the folder $SDKMAN_DIR/candiadates/java.

ls $SDKMAN_DIR/candidates/java -l
total 8
drwxr-xr-x.  9 wpanas wpanas 4096 10-18 17:44 8u152-zulu
drwxr-xr-x. 10 wpanas wpanas 4096 11-04 16:22 9.0.1-zulu
lrwxrwxrwx.  1 wpanas wpanas   47 12-25 17:23 current -> /home/wpanas/.sdkman/candidates/java/11.0.4.hs-adpt

You can check if installation was successful by checking current Java version. The command java --version will not work in Java 8 or any previous, so let's try it.

java --version
openjdk 11.0.4 2019-07-16
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.4+11)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.4+11, mixed mode)

As you can see Java 11 is installed and ready to work. If you want to switch default Java's version use command sdk default java <version>, but you can also switch version only for current terminal session using sdk use java <version>. It's very convenient, if you want to check, how your code would behave on different Java version.

Summary

SDKMAN! is an essential tool for developers using Java, Groovy or any other JVM language. Check it out and you will not be disappointed.