DEV Community: Marko Milenkovic

Best Practices That You Should Follow When Extracting Zip Archive in Java

Marko Milenkovic — Mon, 17 Apr 2023 18:17:10 +0000

Extracting archive files without controlling resource consumption is security-sensitive and can lead to denial of service.

Our code executes on servers, but you should know that servers have limits. Based on this, check how many hardware resources your code can consume. Resources are CPU, RAM, disk, network... Understand those limits and, based on those limits, put thresholds in your code.

Here are some important thresholds you should put in a code when extracting a zip archive.

1. Limit the size of an extracted archive

This is the most basic and critical threshold you should introduce when extracting a zip archive.

How to limit extracted size?

You can check the size after you extract the archive, but this can be too late. If the extracted content is too big, it can drain your server resources.

You should include a size threshold check in the extraction process. Each time you extract and get some bytes from an archive, compare the total extracted size with the threshold.

Zip archives also have metadata. In metadata, you can find each entry's file size. But, as another application wrote metadata, this information doesn't need to be correct.

2. Limit the number of entries in an archive

Each file system has some limitations in terms of the number of files and directories that can handle in a directory/partition/hard drive... Those limits are usually huge numbers. Even so, this is a limited resource. In the Sonar article there is the following statement:

Too many entries in an archive, can lead to inodes exhaustion of the system.

Inodes are indexes that point to the actual location of a file on a disk. If you have too many small files on the file system, it can drain out available inodes. Consequently, your system cannot store new files.

Inodes are used on the linux/unix file systems, but a similar limit also exists for windows. Source: stackoverflow.

3. Protect yourself against zip bomb

Successful Zip Bomb attacks occur when an application expands untrusted archive files without controlling the size of the expanded data, which can lead to denial of service. Source: Sonar article

A Zip bomb is usually a malicious archive file of a few kilobytes with extracted content measured in gigabytes. To achieve this extreme compression ratio, attackers will compress irrelevant data (e.g., a long string of repeated bytes).

You can have a total archive file size threshold, but you should always strive to fail fast. The compression ratio can give you an idea of whether the data in the archive is relevant. The data compression ratio for most legit archives is 1 to 3. And in the example from the sonar article threshold is 10, which should be a good value.

Here you can find example zip bomb files.

4. Sanitize zip files

Sure, here is the revised version in Markdown format with grammatical corrections:

Run a virus scan on a zip file before performing any other action.

This can help prevent malicious files from being uploaded and potentially causing harm to the server or other users.

Check the content of the zip archive.

The content of a zip archive is stored sequentially as a sequence of bytes,
followed by a central directory at the end of the file.
The central directory contains metadata about all zip content.
For each file, it contains a relative path inside the zip archive.
This can create a problem as it can allow a ZipSlip attack
because a user can enter the file name in the form ..\..\file,
leading to the extraction of a zip file in an unwanted location.

To fix this, add the following check:
the canonical path of the file does not start with the path of the target directory.

   File file = new File(extractDirPath, zipEntry.getName());
   if( !file.getCanonicalPath().startsWith(extractDirPath) ) {
       throw new SecurityException("ZipEntry not within target directory!");
   }

Check file extensions inside the zip file.

They should match the file extensions that are allowed by your application.

Change the filename to something generated by the application.

You can use UUID to generate the file name.
This can help prevent conflicts between files and make it harder for an attacker to predict filenames,
which can be useful in certain types of attacks.

5. Don't extract archives inside an archive

This will also create the same problem as with extracting folder content. It is better to go with one of the following options:

Treat archive files in the archive as all other files - don't extract them.
Check with the business if you can forbid the archive files inside an archive. You can ask this question because compressing an already compressed archive will not lead to a smaller file. \ You can forbid archive entries by checking entry extension and mime type during the extraction process.

6. Don't rely only on archive entry metadata

Archives contain metadata that you can read during the extraction process. Compressing application wrote this data in an archive during the compression process.

The problem with this is that if you rely only on those properties, you need to trust a compressing application that is not in your control. Hackers can create fake metadata and crush your application.

Be careful when using zip archive metadata. Always ask yourself how attackers can abuse it to hack your extraction algorithm.

Sources:

Meetings Best Practices

Marko Milenkovic — Wed, 14 Dec 2022 18:31:59 +0000

In the previous period, I had a lot of meetings, so I came up with a list of best meeting practices that can help you be better at meetings:

Before a meeting:

Check if you can resolve the issue you have without organizing a meeting. Create a chat group, or email, try to understand the problem better, and try to solve the issue by yourself. There are several reasons for this.
1. The more people you need, the harder it is to find a time that is suitable for everybody.
2. Meetings are expensive for the company.
3. People are usually busy with their work.
If the subject is urgent, ignore the first point.
Prepare the meeting agenda.
Check all participants' calendars and choose the best time for a meeting.
Schedule a meeting and send the meeting agenda.
Prepare yourself for the meeting - do your homework/research. You can do this by scheduling another meeting for yourself (book the time).
Everybody needs to prepare for the meeting (even if you're not the meeting organizer).

During the meeting:

Don't be distracted by something else (chat messages, phone, other tickets...). Follow the meeting and be active. (This is very hard sometimes :D)
If you think that the subject is not relevant to you, then you should check with a meeting organizer why you're called to the meeting. (Mistakes can happen)

After the meeting:

The meeting organizer needs to send meeting follow-up notes with actions defined in the meeting.
The meeting organizer can ask for feedback from some or all the participants.
If you think a meeting was not productive, inform the organizer and give him proposals for improvement.

I'm sure that this list is not complete. If you have good meeting practices, please share them with me.

If you are needed for some part of the meeting, you can ask the group to leave the meeting after the discussion on your subject is over.
The good idea is to write your meeting notes (even if you're not an organizer).
At the end of the meeting, you should have a defined list of actions.

Setting up Junit 5 Parallel Test Execution With Maven

Marko Milenkovic — Thu, 03 Nov 2022 06:40:14 +0000

Introduction

We need fast-build pipelines. This is because we need fast feedback from our pipeline in order to be more productive. If something is wrong with our code, then we want our pipeline to fail fast. To accomplish that, you may decide to allow parallel test execution for tests in your project. This post describes how to do that with Maven and Junit 5.

Getting Started With Junit 5 and Maven

To get started with Junit 5, you need to import the following dependency into the project:


<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.9.0</version>
    <scope>test</scope>
</dependency>

At the time of writing this article, version 5.9.0 is the newest. You should check maven repository for a new version. Notice, that we imported the library in test scope as there is no need for this library to be in production code because we only use it for running tests.

Running Tests in Parallel

Maven offers you two ways for running tests in parallel:

By starting multiple JVM instances
By using multiple threads in the same JVM instance

1. By Using the Parallel Parameter

The setup for parallel test execution is straightforward. Include the following parameter:

<forkCount>2</forkCount>

It must have a value greater than 1 to enable parallel execution. It is important to know that parallel execution is achieved by starting multiple JVM child processes. This has multiple consequences, and it can affect your decision to use this method. Maven starts multiple JVM instances, and this means that it consumes more memory. Each thread has its own memory space. But it accomplishes a greater level of test independence, as processes cannot share data. You can use this method when you need to use an in-memory database for your unit tests.

The parameter forkCount has a fixed positive integer value which represents the number of forks. Of course, it is usual that multiple developers work on the same project, and we don't know how many cores other developers will have on their machines. To better use hardware, it would be great if we can set this parameter to create a number of forks that depends on the number of cores on CPU. Maven supports this, and you can configure it in the following way:

<forkCount>1C</forkCount>

This configuration instructs Maven to create one fork for each core. If your CPU has two virtual threads per core, you can also enter 2Cto use most CPU resources. Or if you don't want to use 100% of your CPU, you can enter 0.5C which will instruct Maven to create forks for the half of CPU cores.

2. By Setting up Junit Multi-Thread Execution

Maven uses plugins for everything. This is also the case for running project tests. To run project tests, Maven uses the Surefire plugin. Unfortunately, it uses the old version of the plugin by default, which doesn't support Junit5 test execution. So, we need to import a newer version of this plugin to Maven. To include a newer version of this plugin, we need to configure Maven. We can do this by adding the following XML to the pom.xml:


<build>
    <pluginManagement>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M7</version>
                <configuration>
                    We will add configuration here
                </configuration>
            </plugin>
        </plugins>
    </pluginManagement>
</build>

The XML is quite self-descriptive. We add new plugin that is in build->pluginManagement->plugins section of Maven pom.xml. We import the newest version of the surefire plugin at the time of writing this article. You can check this url for a newer version.

Now, when we have a newer version of the surefire plugin loaded, we can continue to configure Maven for multithreaded test execution.

In the official Maven documentation, to achieve this, you need to use parallel, threadCount and useUnlimitedThreads parameters. I tried all these parameters, but nothing worked with Junit 5. By checking official documentation for Maven Surefire and Junit 5 we can conclude that Junit 5 supports a new multithreaded execution model (which is still in the preview phase). So, to configure multithreaded execution, we need to configure Junit5 as well. We can do this by setting up properties in the Maven configuration.


<configuration>
    <properties>
        <configurationParameters>
            junit.jupiter.execution.parallel.enabled=true
            ...
        </configurationParameters>
    </properties>
</configuration>

Another option is to create junit-platform.properties file. Put it into test/resources folder, and put all the properties in that file instead of pom.xml.

It is enough to set junit.jupiter.execution.parallel.enabled=true to allow parallel test execution. But, if you try running the test after setting up this parameter value to true it will surprise you to see that tests are still running sequentially. The reason for this is that there are 2 strategies in which you can allow parallel execution in tests.

Junit can assume that all tests are sequential, and only ones that are annotated with @Execution(CONCURRENT) will be executed in parallel. You can use this annotation on a class or test method to enable parallel execution.
You can set parameter junit.jupiter.execution.parallel.mode.default = concurrent. Here, all tests will run in parallel by default except ones annotated with ExecutionMode.SAME_THREAD.

How Many Threads Junit Will Use for Parallel Execution?

This is also configurable property junit.jupiter.execution.parallel.config.strategy=dynamic and it also has default value dynamic. Strategy can also be fixed or custom. If we set up fixed value for this property, then we also need to set up the value for the junit.jupiter.execution.parallel.config.fixed.parallelism=<positive integer>. This is not a scalable solution, and we should use dynamic value to better utilize the hardware.

dynamic property also has a connected property that influences how many threads JUnit will create. junit.jupiter.execution.parallel.config.dynamic.factor=<positive decimal number>. Factor to multiply by the number of processors/cores to determine the desired parallelism for the dynamic configuration strategy.

Conclusion

Parallel test execution can speed up your build, but you must be careful when enabling it, as it will not always provide you with better performance. This is because the tests may use the same resource, and they can wait for each other to access it. Because of that, it is always a good idea to test and check everything.

Another more significant problem with parallel tests is if the tests are not independent, there can be flaky tests. They are very hard to debug and identify.

If you liked this post, you can follow me on Twitter or LinkedIn for more content.

Resources

Maven documentation

Junit guide

JUnit5 Parallel Execution of tests

I Asked a Java Champion: What Is the Biggest Problem That Professional Java Developers Are Facing Today?

Marko Milenkovic — Tue, 26 Oct 2021 15:00:41 +0000

I have attended the workshop that is organized by the ITKonekt. The workshop was about efficient coding practices for best performance in Java. Java Champion Victor Rentea did a great job explaining all java internals, tools, hibernate, threads, collections, garbage collections, and much more. It was very enjoyable to learn from him and I decided to ask him the question:

What is the biggest problem that professional Java developers are facing today?

His initial answer was: "Not writing enough Java. They write Kubernetes scripts, dev-ops scripts, python scripts…"

My understanding of this is that those people are very experienced developers. He also mentioned that this is what his colleagues are working on currently. But if you are just starting your career, he has advice for you:

If you are starting a career, then you should learn the language and frameworks that you are using. You should master the language and frameworks. This is the first challenge.
Don't be disgusted by the front-end technologies, try to be full stack after you master Java. This is after you are comfortable with Java. Go a bit full stack, not "full full" stack, but "just a bit" full stack.
Try to learn more about what surrounds you. Learn some other language. Kotlin is a good one. * Learn dev ops. Try to learn a bit of everything. He also gave me this advice: As long as you are surrounded by people smarter than you, you are good. And as long as you love to work with your team, you are good. Doesn’t matter project or technology, it can be a terrible legacy system. If you have super cool surroundings and you keep learning and you keep having fun, it’s perfect.

If you liked this post, you can follow me on Twitter or LinkedIn.

Victor is a great trainer and one of the best presenters. You can find him at the following links:

Web site victorrentea.ro you can find a blog, some recorded talks, plus other goodies.
Bucharest-software-craftsmanship-community (3000+ developers).
Twitter, LinkedIN, Facebook.
Victor’s YouTube Channel there is more recorded and live content, including coding katas, past meetups, and more.

10 mistakes that java developers make that prevent them from being successful developers

Marko Milenkovic — Fri, 11 Jun 2021 09:54:23 +0000

Based on my previous experience, I created a list of 10 mistakes that developers made, preventing them from being a great developer. Here is the list:

1. Not writing unit tests

Developers that don’t write unit tests produce more bugs from the code they write and maintain. That leads to unstable products and client dissatisfaction.

If you are not familiar with writing unit tests, there are some resources to get started with:

2. Not manually testing code

Even if you completely cover your code with unit tests, there is still a chance that you missed something out. It happens in practice that some error pushes through.

It is always good practice to manually test code before pushing it for code review. By doing this, you will look at your solution from the client’s perspective. And not only that you can detect bugs, but you can also identify design problems in the development stage.

3. Having the mindset “This will never happen”

Developers often make mistakes when they write new code by thinking that certain scenarios in code will never happen. Eventually, it turns out that they are wrong. In those situations, applications can behave unpredictably and it can lead to bugs. Handle every scenario that code can go into.

Defensive programming techniques will help you in that. If you are not familiar with defensive programming, you can check the following Pluralsight course: https://www.pluralsight.com/courses/defensive-programming-java

4. Not asking for feedback and not giving feedback

To improve yourself, regularly ask for feedback. You can as for feedback when you finish a ticket, or after finishing a project, or when you do a presentation… There is no bad time to ask for feedback.

Give feedback to your colleagues. And not by telling them they are great even if you think they are not so good. Tell them areas where they can improve themselves. If the feedback is honest, they will appreciate you more.

5. Not checking the performance of code

Often, developers write their code, but they don’t check for performance. When code goes to production, it creates various problems. Poor performance can even crush the server.

6. Writing long procedural code

It is very easy to write long methods with a bunch of logic. By doing this, programmers put the same logic in many places.
Projects with a lot of small methods have much greater code reusability and are much easier to maintain.

7. Not being familiar with the tools

Tools are extensions of your hands. The better you know them, the more productive you will be. You should be very familiar with the IDE you use.

Learn shortcuts, they will make you much more productive. Try learning one shortcut to a day and create your personal cheat sheet.

Research plugins, usually you will find a plugin that will help you be even more productive. Plugins that will help you write better code in Intellij Idea are Sonar Lint, Spot bugs, and Code Metrics.

8. Ignoring problems in code

Developers that are working on the most successful products are changing the code all the time. Don’t be afraid to refactor code. If your code is unit tested, then there is a low probability of introducing a regression.

But, don’t stop there. Developers often ignore problematic code that is not part of their ticket. As a developer, you are responsible to maintain an application and keep it in good shape. Because of that, fix all problems that you find.

The best way to proceed with fixing the problem is to create a ticket and work on it with your team. The following story emphasizes why it is important not to ignore problems in code: https://blog.codinghorror.com/the-broken-window-theory/.

9. Coding by accident

Developers should NEVER do a code modification and push that code in production without understanding the consequences of it. Code can produce correct results for given test values. However, there can be scenarios where it can produce unpredicted results and create serious problems.

Coding by accident often happens when developers use features from libraries that don’t completely understand. It can also happen when the developer solves the problem without understanding the solution.

10. Not asking for help

Developers are not very communicative people. They like to solve problems by themselves. The era where one developer creates a complete solution from start to end is over.

Developing software is a team activity. When you encounter a problem during programming, try to solve it by yourself. But don’t waste too much time if you can’t figure out the solution. There is a high probability that some of your colleagues already encounter the same problem and know a solution.

If it is not the case, then you will get help and the team will understand that the problem is complex and that you need time to solve it. Involving more people will help you resolve complex problems faster. Developers that don’t ask for help, usually spend too much time on a ticket.

Help others when you see they have problems with their ticket. As a result, the team will be more productive and people will like you more.

If you like this content, you can follow me on Twitter or LinkedIn.

How do I become proficient with functional programming in Java

Marko Milenkovic — Mon, 05 Apr 2021 13:50:19 +0000

As a Java developer, I always look for ways to improve my coding skills. I heard about functional programming (FP) back when I was a student. Then, FP was not very popular and most developers considered code written using FP to be slow.

Time has changed and today FP is very popular. Some developers are considering it to be the future of how developers write code.

FP is the most useful concept that I learned in the last several years. It helped me to become a better developer. I started writing cleaner code with fewer bugs.

Here are the immediate benefits I got from FP:

I sharpen my skills related to using Java Streams. It is much easier for me to work with Streams when I understand FP concepts.
My functions that follow FP concepts are easy to understand and maintain. There is no risk to using them in a concurrent environment. The reusability of those functions is much greater.
If you are not a fan of “if” and “for” statements, then you will like FP. It can help you write more understandable code that doesn’t include those statements. I can do more with fewer lines of code.
The FP is a declarative paradigm. With FP you describe what you want, rather than how to get it. This means that the code is more readable, reusable, and it is easier to maintain.

So, how I started with functional programming?

We can find a ton of material on the internet related to FP. I usually like to watch video material when I want to learn about a new subject.

I attended an online presentation hold by Venkat Subramaniam. It was a great and inspiring presentation. The good news is that you can watch it on the following link: Functional Programming with Java 8.

After that presentation, I gained an interest in FP, so I decided to study more about FP. I watched the Pluralsight course Functional Programming: The Big Picture. This course helped me understand the big picture of FP and why it matters.

The next natural step was to check if there is a course that's subject is related to implementing FP concepts in Java. I found the course Applying Functional Programming Techniques in Java. It is a great course. It helped me learn, understand, and apply new FP concepts in Java.

After watching those courses, I wanted to know more about Monads. They are very important in FP. Following two videos helped me understand Monads: Brian Beckman: Don't fear the Monad and What the ƒ is a Monad?.

Learn from Haskell

I started learning Haskell. At the end of Venkat Subramaniam's presentation, I asked him if learning Haskell will help me better understand FP concepts. The answer was something like: “Not only Haskell helped me to understand FP, Haskell LEARNED ME how to write good code”. Some developers say that learning Haskell is like learning programming again from scratch. I learned great stuff from Haskell, and it was easy and fun to get started with it. Learn You a Haskell for Great Good! is a great tutorial to get started with Haskell.

Learn some FP language

If you don’t like Haskell, don’t worry, you can learn another FP language. You can check Kotlin, Scala, F#, Clojure, Elixir, Erlang...

If you want to stick with JVM, then you can learn Kotlin, Scala, or Clojure. Kotlin and Scala support functional and OOP paradigms and Java developers can easily get started with them. Kotlin is a new language, Spring framework supports it, and some Java developers started switching to it. Because of those reasons, I suggest trying Kotlin.

Clojure is a modern Lisp variant that runs on JVM. It is useful to know that Lisp is the oldest FP language. Also, Uncle Bob is using Clojure.

My experience with Functional Programming

To get familiar with FP, I studied and applied in practice FP concepts. The most important concepts that I learned are:

Immutability - I have as many as possible immutable objects in my codebase. This leads to fewer places where I can change the state of the program. And that leads to fewer bugs.
Referential transparency - I write as much as possible pure functions. Those functions are like mathematical functions. For the same input, they always have the same output.
Pure and unpure functions - before FP, I was not aware of this concept. Now, I separate pure functions from unpure. It allowed me to easier test the code and improved my code reusability.
Function Composition - promotes better code readability and it is easier to write code by composing functions.
Curried Functions - brilliant concept, but it is not natural to use it in Java like in other FP languages. I don’t use them for now.
Lazy evaluation - evaluate values when they are needed. Lambdas are the way to do a lazy evaluation in Java.
Higher-Order functions - receive other functions as parameters in existing functions. Those functions are usually utility functions.
Map, filter, reduce pattern - Java Stream API implements this pattern.
Monads - helped me understand how to handle unpure functions safely.
Optional class - FP provides an efficient solution on how to work with nullable objects. I always return the Optional object instead of null in a method that can return null.
Railway programming - helped me understand how Stream API and Optional class works.

Using FP in code doesn’t prevent us, developers, from writing bad code. We still need to write unit tests, have a good understanding of our task and our codebase. We still need to apply all the best practices that we learned in the past. FP is promoting good practices and makes it easier for us developers to write good and maintainable code.

I plan to further learn about FP and Java. I will continue to write posts on this subject. For updates, you can follow me on Twitter or LinkedIn.