DEV Community: krud.dev

Ostara Version 0.10.0 Has Been Released

Idan Elhalwani — Mon, 12 Jun 2023 08:48:00 +0000

Hello and welcome to the first major update of Ostara!

We've been working hard the last 3 weeks on our latest release and I'm happy to say it's finally here: we are super excited to release version 0.10.0 of Ostara and bring it one step closer out of Alpha. This release is full of new features and improvements to make your experience smoother than ever. Among the many awesome updates, the one we are most thrilled about is the newly introduced Metrics Notifications!

Before diving into the specifics, let's quickly brush up on Ostara.

A Quick Recap

If you're new around here or just need a quick memory jog, Ostara is your go-to solution for managing and monitoring Spring Boot Actuator-enabled microservices.

With a primary focus on simplicity and user-friendliness, Ostara provides you with real-time insights into the health and performance of your microservices, allowing you to monitor critical metrics such as CPU and memory usage.

For more information, see my previous article.

Metric Notifications

Managing microservices can get a tad complex, and that's where Ostara's new feature comes into play - Metric Notifications. Let's take a deep dive into this key update in version 0.10.0.

Metric Notifications in Ostara serves as your personalized alarm system, alerting you when something goes amiss with your metrics. This feature introduces a level of proactive monitoring that is set to redefine how you interact with your metrics data.

In the Metric Notifications interface, you can now create custom alert rules for your application metrics. Once these rules are in place, Ostara will monitor your instances for metric updates and evaluate them, notifying you if the value has crossed the threshold that was set.

You can define the metric you wish to monitor and set the trigger value. Ostara currently supports two types of rules:

Simple rule

Monitors a single metric and triggers when the metric crosses your defined trigger value.

Relative rule

Tracks the relative value between two metrics, triggering when this value crosses your defined trigger value. For instance, you can set a rule for free disk space on your instance, where the first metric is disk.free and the relative value is disk.total. You can set a trigger for when the relative value drops below 20%.

Now, let's talk about how Ostara evaluates these rules. Ostara provides three comparison operations:

Lower than '<': Triggers a notification if the metric value (or relative value) falls below the trigger value.
Greater than '>': Triggers a notification if the metric value (or relative value) exceeds the trigger value.
Between: Triggers a notification if the metric value (or relative value) falls within the set trigger values.

If this is all too confusing and you're just looking to play around with the new rules, we've also added Predefined Rules that will allow you to instantly create a variety of default rules to play around with;

For more information, visit Metric Notifications

All The Other Goodies in Version 0.10.0

We've managed to achieve quite a lot in this version, here are some of the other features you will encounter:

Application Health Notifications

This feature sends immediate alerts if your application's health status changes. It's designed to help you react quickly to potential issues, reducing manual monitoring.

Notification settings

With the addition of OS level notifications, we've also added the option to turn them off completely in Settings, or just their sound.

Global, App and Folder Dashboard

There are three new dashboards for you to explore - the Global Dashboard, the App Dashboard and the Folder Dashboard. These dashboards provide a concise overview of the status of your applications and folders.

Export/Import Configurations

For those who manage multiple systems, want a backup of your configuration, or want to share your configurations with your friends and colleagues, you can now export and import Ostara configurations.

Instance Shutdown

We've added support for the shutdown capability on Actuator, allowing you to shutdown your instances directly from Ostara;

And a lot more! You can find the full changelog at the end of this post.

Quick Start with Ostara

If you're ready to dive into Ostara 0.10.0, head over to our website, download the version for your OS, and follow the Quick Start guide. You'll be up and monitoring in no time!

Contributions

Ostara is in pre release so we would greatly appreciate any feedback, recommendations and/or requests as well as any contributions. We would also love to hear from other Spring Boot developers. Head to the repository to participate and keep an eye out for our upcoming Beta program!

Links

Changelog: https://docs.ostara.dev/getting-started/changelog

Ostara home: https://ostara.dev

Ostara repository: https://github.com/krud-dev/ostara

Ostara documentation: https://docs.ostara.dev/

Join us on Discord! https://discord.gg/8pUtGUYncD

Introducing Ostara

Idan Elhalwani — Tue, 16 May 2023 13:09:11 +0000

(A previous version of this article introduced Ostara under its previous name, Boost, the article has been amended to reflect this)

What is Ostara

Ostara is a modern tool for managing and monitoring actuator-enabled microservices that aims to make the process more user-friendly and straightforward.

One of our main goals with Ostara was to create a tool that just works out of the box, without needing anything besides a functioning Actuator API on the other end.

Ostara allows you to gain insights into the performance and health of your microservices by providing real-time monitoring of critical metrics such as CPU and memory usage.

One of the key benefits of Ostara is its simplicity. The tool is designed to be easy to use and requires minimal setup. All you need is a running instance of your microservice with Actuator enabled, and you can start monitoring and managing it with Ostara.

Why?

The first question to ask is why even use Ostara, when excellent tools such as Spring Boot Admin already exist?

As Spring Boot Admin users, we had to ask ourselves this question before embarking on the development journey of Ostara.

For all its greatness, one thing Spring Boot Admin that cannot be said about Spring Boot Admin is that it's plug and play or 'off-the-shelf'.

To setup Spring Boot Admin, we start by having to create Spring project for the server itself (In our case, which is true for many, also meant we have to setup a full CI/CD flow, complete with Helm charts and Docker images in order to create a full production flow)

If we needed additional features such as authentication, notifications and so on, we would need to add them directly to the project.

All of this, despite the infinite versatility it offers (because it is essentially a blank slate), raises the startup costs of Spring Boot Admin considerably and even more so when you remember that you have to continually maintain this project as with any other project within your code ecosystem, either due to security updates to unrelated dependencies you may have needed to add, or in order to update Spring to match your main project(s).

But aren't there ready-made images of Spring Boot Admin Server? A quick Google search will show a few (often outdated) pre-built ones that people in the community have created for different scenarios, including this official codecentric/spring-boot-admin Docker image which states:

This repository contains pre-build Docker Images containing basic and hence not production-ready builds of Spring Boot Admin.

Now onto the client, in order to register instances on Spring Boot Admin, you have to add the spring-boot-admin-starter-client dependency to your app and point it at the admin server. From personal production experience in mature projects, this isn't always a pleasant, straightforward experience.

If your project guards Actuator's endpoints with Spring Security, you also have to configure the client to send the server the credentials it will need when registering itself.

Alternatively, if you use Spring Cloud Discovery within your ecosystem you will be able to use that in order to discover instances, dependency free, but not hassle free, as you will again have to add this to your server's project and configure on both ends.

The second part is performance. Spring Boot Admin lacks in a lot of areas when it comes to performance. The simplest example is the Thread Dump page, where if you stick around long enough without refreshing, it will eventually crash your tab.

With Ostara, we aimed for a solution that just works performantly with your existing setups, no matter their size
or scale and without requiring any prior modifications in your code or environment with the only prerequisite being the presence of Spring Actuator.

Feature Overview

A robust and user-friendly UI for an ever-growing list of Actuator endpoints (Full list)
Health monitoring of your applications and their individual instances
Neatly organize your applications and instances into different folders, with color coding and unique icons
Modify the loggers of your applications and individual instances on the fly
Evict caches from applications and individual instances on the fly
View cache statistics
View HTTP request statistics
Bean dependency visualization
Spring Integration Graph visualization
Application Security built in, interact with your secured Actuator instances with no extra code

The Future

Some of our planned features for future releases:

Service discovery support
Notifications
Custom themes
A plugin ecosystem for custom Actuator endpoints
Support for non-official actuator endpoints, such as Togglz
Native support for non-Spring Actuator facades such as gohealth, express-actuator and pyctuator

Setting up Ostara

Setting up Ostara is super simple, simply head over and download the latest release for your OS and follow the Quick Start to create your first instance and begin monitoring with Ostara

Contributions

Spring Boot + Electron, a case study

Idan Elhalwani — Mon, 15 May 2023 11:45:08 +0000

Background

We recently finished writing our desktop app for Spring Actuator, Ostara. Initially, we decided to rely on Electron's main process and write our "backend" in Node. We quickly hit roadblock after roadblock, until a decision was made to ditch the backend entirely, rewrite it in Spring Boot and Kotlin in the JVM ecosystem, and have the renderer communicate with it via REST.

We did this for a multitude of reasons:

First and foremost, we wanted the versatility of being able to run the project on and off Electron, and possibly with a remote backend. Our initial Node implementation relied heavily on IPC between the main and renderer and we knew that, for the most part, had to go.
It's a mature and robust ecosystem, and especially one where we knew we would be able to make rapid progress
We would be able to leverage our own JVM libraries, namely ShapeShift and a yet-unreleased framework we use for CRUD operations with traditional ORMs and ODMs.

Ecosystem Overview

Ostara is based off of electron-react-boilerplate and uses electron-builder to package the application.

The daemon uses JDK 17 with Kotlin, Spring Boot and Gradle

Project Daemon

Before we set out to research and implement this daemon for Ostara, we laid out a list of requirements:

A user should not need to install Java.
No significant downgrades to the developer experience.
The daemon process should be robust and resilient.
Support two-way communication in a manner that is similar to IPC, but doesn't tie us to Electron.
The startup time for the daemon should be under 20 seconds at all times.

Let's break these down.

A user should not need to install Java

The rationale here is simple, right? The user installed your app, they don't really care about your app's dependencies, and they certainly don't want to be given (either automatically or otherwise) a shopping list before they can use it.

Now, this may sound like a silly argument to make when you realize Ostara's primary target audience is JVM developers who certainly have JDK on their workstation. But which JDK? Spring Boot developers come in all shapes and sizes, some will still run JDK 8 on their workstation while others will be on the cutting edge.

Overall, the goal of this requirement was to reduce our footprint and (visible) overhead as much as possible while allowing maximum versatility.

Now right off the bat, let me just say that an alternative to this process called jlink exists. We opted not to use it, so I will not elaborate on it further.

The app is packaged 4 times, once for Windows (x64), once for Linux (x64), and twice for Mac (ARM and x64). Each one of these has its own unique JDK.

The goal would be to download the respective JDK, where it would eventually find its way inside the finished package for each operating system and architecture.

An additional goal we set was to avoid bloating the repository with zip files that we can easily download on the spot and being forced to use Git LFS, so committing the JDKs was not an option.

To achieve the procurement part, we wrote jdkutil, a pure Python tool with no external dependencies.

The tool is very simple in its operation. Given a list of categorized URLs (In our case, this csv), it will download them, verify the checksum matches, and unzip them into a directory tree matching this pattern: {platform}/{arch}/jdk

As a visual example, for the four variants above we will receive the following tree:

jdks/
├── linux
│   └── x64
│       └── jdk
├── mac
│   ├── arm64
│   │   └── jdk
│   └── x64
│       └── jdk
└── windows
    └── x64
        └── jdk

Once the JDKs were ready in this format, all we had to do was add the following to the extraResources block of the build section within our package.json:

{
    "build": {
        "extraResources": [
            {
              "from": "jdks/${os}/${arch}",
              "filter": [
                "**/*"
              ]
            }
          ]
    }
}

When electron-builder runs, it will populate ${os} and ${arch} according to the operating system and architecture being built, so at the end of this process we should have an arch specific, OS specific folder called jdk in every package that we create, with the correct JDK for the designated platform.

No significant downgrades to the developer experience

While a developer is expected to have JDK 17 installed, if they are only working on the Electron side there should be no additional steps to run or perform.

Setup

A "frontend" developer shouldn't need to setup a working environment to work on the daemon, but they will still need to be able to run the current development version and build it from scratch.

Likewise, a "backend" developer should be able to run their daemon from their IDE of choice and be able to hook it up to the process.

Finally, in a packaged state we will have a jar as well as a custom JDK that we will need to use.

This lead us to split the way we start the daemon into two ways.

The first is the "Packaged" approach, where Electron already knows the location of a prebuilt JAR and simply runs the process.

The second is the "Remote" approach. In this approach, Electron doesn't actually start the process, but simply receives a predetermined port and address (Usually localhost) and creates a connection with the daemon over HTTP.

The remote approach covers both development use-cases. In order to achieve this, we split our start command into two:

    "start": "concurrently \"npm run start:daemon\"  \"ts-node ./.erb/scripts/check-port-in-use.js && npm run start:renderer\"",
    "start:thin": "ts-node ./.erb/scripts/check-port-in-use.js && npm run start:renderer"
}

A "frontend" developer in this case would run start, which would then in turn compile and run the daemon locally. Since the app isn't packaged, the app will automatically determine that it needs to use a Remote daemon with the default host and port.

A "backend" developer would run start:thin which essentially does everything except start the daemon, with the expectation that the daemon is already running. Like the previous example, the app will automatically determine that it needs to use a Remote daemon with the default host and port.

Implementing health checks

Since the Daemon, like any process had the possibility of crashing or freezing, the next step was to implement health checks. We opted for a simple polling HTTP health check from the Electron main process to the daemon.

For this process we created 3 events in the Electron main process:

daemon-ready
daemon-healthy
daemon-unhealthy

The flow created around these events is simple.

Upon app start, the user will be sent to the splash screen. Once the daemon-ready event is fired, the splash screen is closed and is replaced by the main screen. If at any point during this time we receive the daemon-unhealthy event, the user's screen is replaced with the following:

If, during this time, daemon-healthy is fired, then the screen returns to normal.

Support two-way communication in a manner that is similar to IPC, but doesn't tie us to Electron

To achieve two-way communication between the Electron renderer and the daemon, we opted to use Websockets with the STOMP protocol. This approach decoupled the communication between the renderer and the daemon from the Electron framework, allowing for greater flexibility in the future and the ability to break away from Electron entirely if needed

Conclusion

In conclusion, despite the unorthodox choice, we successfully developed a robust and resilient daemon for our Electorn app using Spring Boot, all while addressing our primary goals.

By doing so, we have created a flexible and scalable app that can run on and off Electron with the potential for remote backend functionality. Additionally, we have opened the door for further expansion and improvements in the future, should the need arise.

Strategy Design Pattern with Spring Boot + Kotlin - Cleaner code with no boilerplate

Aviv Mor — Thu, 02 Mar 2023 09:48:50 +0000

The Strategy Pattern

If you already know the strategy pattern just jump here.

What is it?

According to Wikipedia, the strategy pattern is a behavioral software design pattern that enables selecting an algorithm at runtime. Instead of implementing a single algorithm directly, code receives run-time instructions as to which in a family of algorithms to use.

In simple words, the strategy pattern is used to specify how something should be done by providing a specific algorithm.
For example, a strategy for payment could be Credit Card, PayPal, or any other payment provider.

Why should we use it?

The Strategy Pattern defines a family of algorithms, encapsulates each one, and makes them interchangeable. Strategy lets the algorithm vary independently from clients that use it (source).

In short, it makes our code cleaner, easier to maintain, and reusable.

The Strategy Pattern in Code

Implementations of the pattern consist of an interface defining the strategy, multiple concrete implementations of the strategy and a context using the strategy.

By Vanderjoe - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=60733582

Code Example

Let's look at the following simplified scenario:

We have an application with users and we need to send them verification codes for 2FA.
The application supports multiple communication options for receiving the verification code: Email, SMS, Notifications...
Each user selects its preferred communication method.

First we will define our strategy interface:

interface VerificationCodeCommunicationStrategy {
    fun sendVerificationCode(user: User, code: String)
}

Now let's add our implementations:

class EmailCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override fun sendVerificationCode(user: User, code: String) {
        emailClient.sendEmail(user.email, "Verification code", "Your verification is $code.")
    }
}

class SmsCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override fun sendVerificationCode(user: User, code: String) {
        smsClient.sendSMS(user.phone, "Your verification is $code.")
    }
}

// ...

All that's left is to use the strategy.

enum class CommunicationMethod {
    Email, SMS, Notification
}

class VerificationCodeService {
    val verificationCodeStrategyMap: Map<CommunicationMethod, VerificationCodeCommunicationStrategy> = mapOf(
        CommunicationMethod.Email to EmailCommunicationStrategy(),
        CommunicationMethod.SMS to SmsCommunicationStrategy(),
        // ...
    )

    fun sendVerificationCode(user: User) {
        val code = generateVerificationCode()

        val verificationCodeStrategy = verificationCodeStrategyMap[user.preferredCommunicationMethod]
        verificationCodeStrategy?.sendVerificationCode(user, code)
    }
}

We can see how the code is clean and easy to maintain. Adding a new communication method would be easy and will not need to change existing code other than adding it to the map of strategies.

The Library

We use the strategy pattern a-looot in our projects. Moreover, like all developers we are lazy and want to make our life as easy as possible. This is why we created our library.

Code Example WITHOUT the library

Let's revise the previous example for a regular Spring Boot implementation.

When using the strategy pattern in a Spring Boot application the context and strategy implementations are usually Spring components.

We will start with the strategies:

interface VerificationCodeCommunicationStrategy {
    fun sendVerificationCode(user: User, code: String)
}

@Component
class EmailCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override fun sendVerificationCode(user: User, code: String) {
        emailClient.sendEmail(user.email, "Verification code", "Your verification is $code.")
    }
}

@Component
class SmsCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override fun sendVerificationCode(user: User, code: String) {
        smsClient.sendSMS(user.phone, "Your verification is $code.")
    }
}

// ...

The interface remains the same and the implementations receive the @Component annotation.

Now let's move on to the verification service.

@Service
class VerificationCodeServiceImpl(
    private val emailCommunicationStrategy: EmailCommunicationStrategy,
    private val smsCommunicationStrategy: SmsCommunicationStrategy
) {
    private val verificationCodeStrategyMap: Map<CommunicationMethod, VerificationCodeCommunicationStrategy> = mapOf(
        CommunicationMethod.Email to emailCommunicationStrategy,
        CommunicationMethod.SMS to smsCommunicationStrategy
    )

    fun sendVerificationCode(user: User) {
        val code = generateVerificationCode()

        val verificationCodeStrategy = verificationCodeStrategyMap[user.preferredCommunicationMethod]
        verificationCodeStrategy?.sendVerificationCode(user, code)
    }
}

We need to add the strategy implementations to the constructor and build the map manually. That's no fun!

Code Example WITH the library

The entire objective of the library is to make our life easier and the code cleaner and less bug prone. It does it by simply building the strategy map automatically.

We will start by modifying the strategy interface.

interface VerificationCodeCommunicationStrategy {
    @get:ComponentMapKey
    val method: CommunicationMethod

    fun sendVerificationCode(user: User, code: String)
}

As you can see we have added a new field method with the annotation @ComponentMapKey. This informs the library which field is the key for the strategy map.

Let's update the implementations according to the modified interface.

@Component
class EmailCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override val method = CommunicationMethod.Email

    override fun sendVerificationCode(user: User, code: String) {
        emailClient.sendEmail(user.email, "Verification code", "Your verification is $code.")
    }
}

@Component
class SmsCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override val method = CommunicationMethod.SMS

    override fun sendVerificationCode(user: User, code: String) {
        smsClient.sendSMS(user.phone, "Your verification is $code.")
    }
}

// ...

Now let's modify the service to use the library.

@Service
class VerificationCodeServiceImpl {
    @ComponentMap
    private lateinit var verificationCodeStrategyMap: Map<CommunicationMethod, VerificationCodeCommunicationStrategy>

    fun sendVerificationCode(user: User) {
        val code = generateVerificationCode()

        val verificationCodeStrategy = verificationCodeStrategyMap[user.preferredCommunicationMethod]
        verificationCodeStrategy?.sendVerificationCode(user, code)
    }
}

That's it! All we had to do is to add the @ComponentMap annotation to the strategy map and voila! The map is built automatically from the components. No boiler-plate code. To add a new strategy all you need to do is to create a new implementation and it will be "magically" added to all relevant strategy maps.

Final Words

This library is the most used library in our projects. Although it's not life changing, it's just useful, helps us keep our code clean and prevents lots of recurring bugs from forgetting to add new strategies to their appropriate maps.

If you would like to use it you can start with the quick start guide.

We develop open source because we love it and want to contribute to the cause. If you would like to support us please consider starring the repository.

Introducing ShapeShift: A lightweight, modular, performant and extensible object mapping library for Kotlin/Java

Aviv Mor — Wed, 12 Oct 2022 12:07:23 +0000

WHY

The short answer

We needed a simple to use, minimal boiler plate mapping engine, that is also flexible and supports the most advanced use cases.

The long answer

A few years ago we got tired of developing the same boiler plate code for CRUD and decided to develop a generic CRUD framework.

As a part of this framework we needed an object mapping library and could not find one that answered all of our requirements.

The first iteration of the library was part of the entire CRUD framework. However, some time ago we decided we want to start contributing to the open source community and we thought "What is the best way to contribute if not to release our CRUD framework?". The issue with this thought is that the framework is huge and contains lots of code, polishing it and getting it ready to be published as open source would take a lot of time (documentation anyone?).

Our solution was to separate the framework into multiple smaller libraries we can polish and publish in smaller time frames. The first of these libraries is ShapeShift.

WHAT

The short answer

ShapeShift will help you transform object of type A to object of type B easily, fast and without boiler plate code.

ShapeShift main features:

Auto mapping
Custom field transformers
Default transformers
Deep mapping
Multiple mapping targets
Conditional mapping
Mapping decorators
Seamless spring integration
Native Kotlin DSL API
Java builder API
Annotations API
Native Android support

The longer answer

ShapeShift's job seems simple, convert object of type A to object of type B. But to do its job in the best way possible, the library offers lots of features, to name a few:

Advanced auto mapping - Automatically map fields with the same name from the source object to the target object. Control wether to map only fields of the same type or use the advanced default transformers feature to auto map fields of different types!
Seamless Spring integration - ShapeShift's Spring Boot library allows to integrate it with Spring easily. Use Spring's power to create custom transformers beans with access to DB or other resources.

More about the advanced features with code examples is available in Kotlin version and Java version.

HOW

The short answer

ShapeShift supports multiple APIs to configure your mappings.
Annotations, Kotlin DSL and Java Builder.

The long answer

ShapeShift's documentation is available here. You can check our Quick Start guide to get it up and running in 5 minutes.

Built with Kotlin in mind, ShapeShift was designed around its ecosystem and best practices. The library APIs for mapping with Kotlin:

Annotations - Fully featured annotation based mapping, just add annotations to your objects and ShapeShift handles the rest. Including using custom field transformers, conditional mapping, advanced object decoration and much more.
Kotlin DSL - Allowing you to define the relations between objects. This allows you to map objects you can't change (or don't want to), like objects from 3rd party libraries. Additionally you can define inline transformations, conditions and decorations, enabling deep customization and very advanced mapping.

ShapeShift also has full Java support available with multiple APIs for mapping:

Annotations - Same API as Kotlin with full Java support.
Java Builder - The equivalent Java API for the Kotlin DSL. Allowing you to define the relations between objects without modifying their code, and map objects you can't change (or don't want to).

Final words

We poured our love, time, knowledge and what not into ShapeShift. I'm proud of the result and our greatest achievement is to be able to help others with their work.

If you would like to contribute, please feel free to open issues, join as a contributor or star the repository. Thanks in advance!

Good code is like an onion, it has layers

TheWizKid — Tue, 11 Oct 2022 13:17:39 +0000

The stack — Java, GWT, Mysql

The Experience — 11 Years

The Job — Senior Developer at Startup

—

Teaching some new recruits

It was my 3rd year at the startup, things were going well, the company was growing. We needed more working hands, so like any company, we hired a new group of devs, straight off the assembly line (because why spread it out, or get someone with experience). I don't exactly remember how it came to be, but I was in charge of getting them up to speed. This was my first true taste of teaching and leading.

I was always good at winging explanations, and presentations, so it was no different here. I showed them the ropes, the way things were put together, and what talked to what and how.

The problem came when I needed to explain the way the code was written, and why it was written that way. We were using GWT, and for those who don't know, it allowed us to write the client code as JAVA and use the same entities as we do with the DB.

To say that it was useful was an understatement, but like everything in life, this had its drawbacks. Namely everything you save in the DB is sent to the client. This could and did cause many performance issues, and looking back I'm pretty sure there were security issues stemming from this behavior. This code was not layered correctly, but I didn't quite know it yet.

The code that forced me to look and understand layers

It was around this time that my friend started working on another startup, there he worked with this outsourced dev. She had written the main structure of the server code, and had very distinct layers and separation.

I try not to go too deep in technical terms in these posts, but I kind of need to here. There are a few different actors in our story, and each has some specific needs and talents.

The DB

The job of the DB is simple, save the objects you need to, and allow you to query them when needed. So naturally, this is where you need to think of how to save things, how they are connected to the other entities in the system, and how you are going to query them.

The Server

Its main job is to process things, take requests from the client, do some logic, and save it to the DB. This is the man in the middle, between the client and the DB.

The Client

This is where things are displayed, not everything you need to do the job needs to be displayed. Sometimes you want to display things as a mix of other things.

Now I want to focus on the server, this is where my expertise is, and this is where the layers are most prevalent. It's the server's job to get tasks from the client and process them, update the DB, and send the results back for display. Let's break this down into meaningful layers.

[API] ← RO (Representation Object)→ [Service] ← Entity/RO → [Handler] ← Entity → [Dao/DB]

The client does requests against the server, this is deserialized to a RO.

This RO is then used by the service to do logic, this is where processes are done.

The service in turn works with the handlers, where short and transactional logic can be performed.

They in turn can use the DAO (Data Access Object) to persist and fetch data from the DB.

I'll dig deeper into the different layers, and what they are in charge of, and why they use the Objects they do to talk to the other layers.

The API

This is a facade, facing the client. Not all data needs to be transferred in order to do the requested action. For example, there could be fields on an object that are immutable, and there is no reason for the client to send them over when updating an object. Or there could be fields in the server that are only required for server logic, no reason to send them back via the API. This is why this layer works with ROs aka Representation Objects. They can have less fields, or aggregated fields, that can be used by the client.

This layer should be kept as “stupid” as you can, allowing the system to have multiple different types of APIs that call the same logic. The logic is based on the Service layer, and the API calls upon that layer to achieve its goals.

Service Layer

This is where most of the business logic lives, the different processes that constitute a system. These processes can be long, and can require multiple entities within the system. It's the service layer's responsibility to map the ROs into something more useful for handling the logic.

The service layer calls different Handlers, each with its own domain of responsibility to run the logic. These processes can be long, and should not be transactional (Create locks in the DB).

Handlers

Handlers have very well defined responsibilities, they usually work with only 1 entity, and they are the ones that can do transactional actions. They are also the only ones that talk to the DAO/DB.

As you know, DB transactions can cause locks. One of the things we strive for in good system design, is to minimize lock time and scope. This is why the handlers are separated from the services. In the handlers is where you will find the code most optimized, and single goal oriented.

DAO/DB

This is where persistence occurs. This layer is similar to the API layer in that it should be as simple and “stupid” as possible. Correct implementation of this allows us to simply (relatively) change DB by just re-implementing the DAO layer.

An example:

Lets take for example a process of publishing a new version of a post. This process is triggered by a request from the client, all that is needed is the post_id and version_number. So we have an RO with those fields only.

The process starts with some validation

Does that post_id exist?

Can the user requesting the publishing perform that action?

Is there a version_number like that?

Is it already published?

You can think of some more….

Now of course at the end of this, we want to transactionally update the is_published flag at the same time on the old version as the new. But during the time we are doing validations there is no reason to open a write transaction in the DB, and lock the entities. It is much better to do the validation process, call the handler just to do the swap, then continue after the transaction to create the result for the client.

This way we gained a few key points.

Shortest transaction time possible.

No connection to DB used if the request fails in the validation phase.

Code reuse - the method that publishes in the handler can later be used by different services (for example an automated time publisher).

Clear single responsibilities for each layer.

Practice what I preach

Over the years I have honed this layered design (I will get into it in later posts), but the basics remain. This code forced me to look at my code differently, and implement the layers, and I have to tell you, this has made my code much more readable and maintainable as well. It has also increased my development speed, because I can write each layer's interface, and then start filling in the methods.

I won't lie, it took me a while to get used to working this way, and even longer to get it right (I would put things in the handler that could have been in a service and split among multiple handlers), but it's a process. I urge you to try and split some code you are working on using these principals. Everyone has that 1 method that is so big that nobody wants to touch it. Go ahead, give it a go, I'm confident that after some time (and banging your head against the wall) you will be able to split it up and have much better code at the end.

Please check out my open source library and maybe even star it. I would appreciate it greatly!

Object Mapping advanced features & QoL with Java

Aviv Mor — Thu, 08 Sep 2022 12:04:01 +0000

This is the Java version of the Kotlin article.

When working with multi-layered applications, external libraries, a legacy code base or external APIs, we are often required to map between different objects or data structures.

In this tutorial, we will check out some Object Mapping libraries advanced features to simplify this task while saving development and maintenance time.

In our examples we will use the library ShapeShift. It's a light-weight object mapping library for Java/Kotlin with lots of cool features.

Auto Mapping

We will start with a bang. Auto mapping can and will save you lots of time and boiler-plate code. Some applications require manual mapping between objects, but most applications will save tons of time working on boring boiler-plate by just using this one feature. And it gets even better, with ShapeShift's default transformers we can even use automatic mapping between different data types.

Simple Mapping

Let's start with a simple example for auto mapping. We have our two objects, imagine they could also have tens, hundreds, or even thousands (for the crazy people here) of fields.

public class User {
    private String id;
    private String name;
    private String email;
    private String phone;
    // Getters, Setters, Constructors, Equals...
}

public class UserDTO {
    private String id;
    private String name;
    private String email;
    private String phone;
    // Getters, Setters, Constructors, Equals...
}

We want to map all the field from User to UserDTO. Using auto mapping we don't need to write any boiler-plate code. The mapper will be defined as follow:

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(User.class, UserDTO.class)
        .autoMap(AutoMappingStrategy.BY_NAME_AND_TYPE)
        .build();

Voila! All the fields will be mapped automatically without any manual boiler-plate code.

Advanced Mapping

In this example we will use the power of default transformers to take auto mapping even further.

public class User {
    private String id;
    private String name;
    private Date birthDate;
    // Getters, Setters, Constructors, Equals...
}

public class UserDTO {
    private String id;
    private String fullName;
    private Long birthDate;
    // Getters, Setters, Constructors, Equals...
}

Note that the types of the birthDate field are different in the source and destination classes. But using the power of default transformers we can still use auto mapping here.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(User.class, UserDTO.class)
        .autoMap(AutoMappingStrategy.BY_NAME)
        .build();

We changed the auto mapping strategy to BY_NAME so it will map fields also with different types. Now we need to register a default transformer to the ShapeShift instance in order for it to know how to transform Date to Long.

ShapeShift shapeShift = new ShapeShiftBuilder()
        .withTransformer(
                Date.class, 
                Long.class, 
                new DateToLongMappingTransformer(), 
                true // Default Transformer
        )
        .build();
    }

We can also add manual mapping on top of the auto mapping in order to add/change behavior. The source and destination classes have different names for the name field so we will add manual mapping for it.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(User.class, UserDTO.class)
        .autoMap(AutoMappingStrategy.BY_NAME)
        .mapField("name", "fullName")
        .build();

Auto mapping is great for use cases that does not require specific mapping. It helps reduce the amount manual boiler-plate code needed to configure mapping and also helps you keep your sanity.

Transformers

Transformers are very useful feature that allows you to transform the type/value of a field to a different type/value when mapping a field.

Some use cases we have been using widely:

Transform date to long and vice versa between server and client objects.
Transform JSON string to it's actual type and vice versa between server and client objects.
Transform comma separated string to list of enums.
Transform another object id to its object or one of its fields from the DB using Spring transformers.

Basic Transformers

We will start with a simple transformer example. Date-to-Long and Long-to-Date transformers:

public class DateToLongMappingTransformer implements MappingTransformer<Date, Long> {
    @Nullable
    @Override
    public Long transform(@NonNull MappingTransformerContext<? extends Date> context) {
        return context.getOriginalValue() != null ? context.getOriginalValue().getTime() : null;
    }
}

public class LongToDateMappingTransformer implements MappingTransformer<Long, Date> {
    @Nullable
    @Override
    public Date transform(@NonNull MappingTransformerContext<? extends Long> context) {
        return context.getOriginalValue() != null ? new Date(context.getOriginalValue()) : null;
    }
}

All we need to do now is to register them.

ShapeShift shapeShift = new ShapeShiftBuilder()
        .withTransformer(Date.class, Long.class, new DateToLongMappingTransformer(), true) // "true" is optional, we are registering the transformers as default transformers, more on that later.
        .withTransformer(Long.class, Date.class, new LongToDateMappingTransformer(), true)
        .build();

That's it! We can now use the transformers when mapping objects.

public class User {
    private String id;
    private String name;
    private Date birthDate;
    // Getters, Setters, Constructors, Equals...
}

public class UserDTO {
    private String id;
    private String name;
    private Long birthDate;
    // Getters, Setters, Constructors, Equals...
}

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(User.class, UserDTO.class)
        .mapField("id", "id")
        .mapField("name", "name")
        .mapField("birthDate", "birthDate").withTransformer(DateToLongMappingTransformer.class) // We don't have to state the transformer here because it is a default transformer
        .build();

Inline Transformers

In some use cases we want to transform the value but we don't need a reusable transformer and we don't want to create a class just for a one time use.

Inline transformers for the rescue! Inline transformers allow to transform the value without the need to create and register and transformer.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(Source.class, Target.class)
        .mapField("birthDate", "birthYear").withTransformer(context -> context.getOriginalValue() != null ? ((Date) context.getOriginalValue()).getYear() : null)
        .build();

Advanced Transformers

Transformers also allow us to do transformations with the DB or other data sources.

In this example we will use the power of the Spring Boot integration to create transformers with DB access.

We have three models:

Job - DB entity.
User - DB entity.
UserDTO - Client model.

public class Job {
    private String id;
    private String name;
    // Getters, Setters, Constructors, Equals...
}

public class User {
    private String id;
    private String jobId;
    // Getters, Setters, Constructors, Equals...
}

public class UserDTO {
    private String id;
    private String jobName;
    // Getters, Setters, Constructors, Equals...
}

We want to convert the jobId on User to jobName on UserDTO by querying the job from the DB and setting it on the DTO.

In Spring's case, you generally avoid interaction with the application context from static functions or functions on domain objects.

We will use a ShapeShift's Spring integration to create a component as a transformer to access our DAO bean.

@Component
public class JobIdToNameTransformer implements MappingTransformer<String, String> {

    @Autowired
    private JobDao jobDao;

    @Nullable
    @Override
    public String transform(@NonNull MappingTransformerContext<? extends String> context) {
        if (context.getOriginalValue() == null) {
            return null;
        }
        Job job = jobDao.findJobById(context.getOriginalValue());
        return job.getName();
    }
}

All that's left to do is to use this transformer in our mapping.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(User.class, UserDTO.class)
        .mapField("id", "id")
        .mapField("jobId", "jobName").withTransformer(JobIdToNameTransformer.class)
        .build();

Another bonus of using transformers is their reusability. In some use cases, We could create more generic transformers that will have application-wide usage.

Default Transformers

When registering transformers you can indicate wether a transformer is a default transformer. A default transformer of types <A, B> is used when you map a field of type <A> to field of type <B> without specifying a transformer to be used.

As we already seen, default transformers are useful for recurring transformations and especially for automatic mapping.

Deep Mapping

What if we want to map from/to fields that available inside a field which is an object? We can even do that, easily.

In order to access child classes we can use the full path of a field. Let's look at the following example.

public class From {

    private Child child = new Child();
    // Getters, Setters, Constructors, Equals...

    class Child {
        private String value;
        // Getters, Setters, Constructors, Equals...
    }
}

public class To {
    private String childValue;
    // Getters, Setters, Constructors, Equals...
}

We want to map the value field in Child class inside the From class to the childValue field in the To class. We will use the full path of value which is child.value.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(From.class, To.class)
        .mapField("child.value", "childValue")
        .build();

The full path is supported in both source and destination fields, it also supports multi level depth (e.g. x.y.z).

Conditional Mapping

Conditions allow us to add a predicate to a specific field mapping to determine whether this field should be mapped.

Using this feature is as easy as creating a condition.

public class NotBlankStringCondition implements MappingCondition<String> {
    @Override
    public boolean isValid(@NonNull MappingConditionContext<String> context) {
        return context.getOriginalValue() != null && !context.getOriginalValue().trim().isEmpty();
    }
}

And adding the condition to the desired field mapping.

public class SimpleEntity {
    private String name;
    // Getters, Setters, Constructors, Equals...
}

public class SimpleEntityDisplay {
    private String name = "";
    // Getters, Setters, Constructors, Equals...
}

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(SimpleEntity.class, SimpleEntityDisplay.class)
        .mapField("name", "name")
        .withCondition(NotBlankStringCondition.class)
        .build();

Inline Conditions

Like transformers, conditions can also be added inline using a function.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(SimpleEntity.class, SimpleEntityDisplay.class)
        .mapField("name", "name")
        .withCondition(context -> context.getOriginalValue() != null && !((String) context.getOriginalValue()).trim().isEmpty())
        .build();

Annotations Mapping

This specific feature receives lots of hate because it breaks the separation of concerns principle. Agreed, this could be an issue in some applications, but in some use cases where all objects are part of the same application it can also be very useful to configure the mapping logic on top of the object. Check out the documentation and decide for yourself.

Conclusion

Object mapping libraries are not the solution for every application. For small, simple applications using boiler-plate mapping functions are more than enough. But, when developing larger, more complex applications, object mapping libraries can take your code to the next level, saving you development and maintenance time. All of these while reducing the amount of boiler-plate code and overall improving the development experience.

On a personal note, I used to work with manual mapping functions and was ok with it. It was "just" some simple lines of code. After upgrading our applications to use object mapping as part of our "boiler-plate free" framework (We will discuss that framework at a later time), I can't go back. Now we spend more time on what's important and interesting and almost no time on boring boiler-plate code.

My First Real Job

TheWizKid — Tue, 06 Sep 2022 13:24:33 +0000

The stack — Java, Mysql

The Experience — 9 Years

The Job — Junior Developer at Startup

—

My first job interview

In my last year of uni Aviv (my now best friend and business partner) told me he is working at a startup that is now growing, and I should join him working there. I was nervous for my first job interview, they were working with a different stack than I had experience with, and let's just say I'm a little weird. A little background, I live and had grown up in a rural farming area, I wear shorts, and sandals (even in the dead of winter), not exactly “professional” looking. Anyway I vividly remember taking long pants and shoes in the car with me to the interview, thinking I would change right before I went in, to look the part. At the last moment I changed my mind, and told myself, “You should not try to pretend you are something you are not, they will either take you with your quirks or it's not the right place for you”. So I walked into the interview with the CEO of the company, in shorts and sandals. Lucky for me, the CEO was wearing torn jeans, and dirty worn sneakers, I felt right at home.

All this was to say something simple, but not easy. Be true to yourself, if you go work somewhere you won’t give your 100% if you feel out of place. I'm not saying you need to reject any job that isn't perfect, it's more like, find a job that doesn't compromise on your core, and be flexible about the “less important” stuff. In the long run this will be much better for you and the company.

Java you say

My first day on the job was an interesting one, I got to the new office, they gave me a laptop, and told me to install my environment. Easier said than done, till now it was a visual studio, now I needed this weird eclipse thing. Then I got the code and nothing runs :/ weird. After some back and forth, I learned that for java and eclipse I need to download the JDK (Java development toolkit), fine. Still nothing… Just a reminder it was my first day on the job, in a language/stack I didn't know, and I didnt want to be the new “idiot” they hired. Safe to say it took me the whole day or maybe even 2 just to get the code up and running.

Later that week I got my first task, I needed to copy some functionality that existed in the client area to the back office. This was actually a perfect first task, for many reasons, most of all it was 70% copy paste. This was great because I had barely knew how to read the java code I was looking at. Additionally the adaptation needed in the back office project was simple and existed in one form or another in different areas of the code. So basically the task was like a big jigsaw puzzle, I got 1 big piece, now I need to go hunt down some smaller ones and put it all together.

A small tangent

Over the years this “jigsaw puzzle” approach has really sunk in, and today I can tell you with confidence, if you structure the tasks given to developers (especially juniors) in this way, a few good things happen:

They get into the code much faster.
They write less buggy code.
They build confidence in their abilities. Finally as a business owner I can tell you from my experience, it takes a lot less time to see a positive ROI for their time.

This has led me to a core belief of mine (in regards to development). A good programmer needs to know 3 things:

How to read and understand code.
Be proficient in google searches for solutions to the problem at hand.
Know how to adapt other code to meet their current needs.

A great programmer is one that knows how/why things are done in this way, and understands optimization and deep diving into frameworks they use. While a good programmer just needs to know how to use the framework, and not how it works.

What is this Spring magic - using frameworks

The project we were writing was built upon Spring (a hugely popular dependency injection framework in Java). Now frameworks are powerful tools, but they hide many things under the hood. Over the years I have learned in depth how/why Spring works, but truth be told, as a junior developer back then it seemed like magic to me.

I want to take you through a few different stages of understanding that I use today when I am faced with a new framework.

Step 1 - Basics (The what)

In this step it is most important NOT to try to understand how it works, just to understand the tools and capabilities it provides. I have seen it many times that a junior gets “stuck” on trying to understand the depths of the framework, and the truth is they are just not ready to. They don’t fully understand WHAT the framework can do, so they have no chance to understand how.

Step 2 - Advanced Usage (The how)

Almost every framework I have used has many different features that compliment each other (makes sense). This stage is focused on that, using the framework to its fullest. Mixing and matching different features, making the whole greater than the sum of its parts. This is where the developer truly learns HOW to use the framework correctly.

Step 3 - Down the rabbit hole (The why)

Here is where things get truly interesting. There comes a time when you reach the limits of the framework and try to do something you think should be possible, but turns out it isn't. This is when you download the source code and start the deep dive to understand the WHY. Sometimes it's something at the core of how it's built, but other times it's just on the fringes of what the framework can do, and that means, no examples, no documentation. Then the only answer is to understand why the framework works the way it does.

Step 4 - Building upon the framework

This is the final stage, where you can start using the tools the framework gives you to build your own mini framework upon. Or even build your own libraries from scratch. This is where you start contributing to the framework, or start releasing some open source of your own.

Generalization is a powerful tool

After a year or 2 in the company, I got a new task, implementing a new PSP (Payment service provider), basically adding something like paypal to the clients payment options. This was a new area in the code for me, something I had no experience with. So off to the races, I went and looked at the implementation of a few different PSPs, and to my shock, I saw a ton of very similar code, makes sense since it doesn't really matter how the money comes in, we still need to update billing, create the relevant logs, and credit the account. All this was done time and time again, in the different PSPs, this was mainly because the objects we were getting back from the PSP api were so different.

Interfaces to the rescue, this is one of my first epiphanies, this has led me to proper code layers, and a few “new” design patterns (really just classes that give me a mix of design patterns with no boilerplate code). So I told my boss I needed more time to rewrite it correctly, and set to work. I created a handler that was in charge of the process of communication with the PSPs. It had methods for the different actions, like charge or refund and so forth. It managed the different steps of the process and called upon the different classes of the PSP for interpretation of the response received via API.

I don't want to get into the code (this is not the point of the post), but basically there was a PspResponse that all PSPs needed to inherit from, and then methods like:
getIsSuccess()
getErrorText()
getAmount()

This took some time to refactor the old code, but in the long run reduced implementation time for new PSPs drastically, all the while reducing the amount of bugs (mainly because someone would always forget to update the new logic in some old PSP).

The experience I gained designing and implementing this led me down the path to becoming an architect. I felt so proud from my new creation, and caused me to look at code in a different way. This is a pivotal point in my career, and has caused me to think about how to write code correctly in the long run, and not just this feature.

Please check out my open source library and maybe even star it. I would appreciate it greatly!

What I learned in Uni

TheWizKid — Thu, 25 Aug 2022 13:21:09 +0000

The stack — C#

The Experience — 7 Years

The Job — Uni Student + Freelancer

—

My first paying job

Let me set the mood, poor uni student, first year physics major, tooo many courses at the same time, not enough sleep; I think you get the idea (and possibly what it feels like). Now I know we would all love to have some money, to go out with, and enjoy our time. We can all find some minimum wage 9-5, but I was looking for something better, and thought I could use my skills as a developer to land some work.

Luckily for me, a family friend was looking for someone to develop something for them. The job was a good mix of my knowledge and skills. This friend owned a factory that builds custom water treatment and purification equipment, and he had a bottleneck in his quoting division. Every quote the sales team needed required an engineer to write up plans, then someone needed to calculate not only the costs of the different parts, but things like length of welds, amount of paint needed, amount of bolts needed, and so forth. Till this point it was done in excels with lookups and required a lot of know-how.

My job (should I choose to accept it) was to create a visual tool, that the sales agent can drag and drop to the parts needed, including piping and such, and the output would be a detailed quote, including everything mentioned above and more.

Working with/for a client as a freelancer

Now for all you developers out there, I'm sure you are thinking, “OK, he got some requirement documents on how/what to build”. That couldn't be farther from the truth (and surprisingly more common than I had expected). I had to basically interview all the people involved in the company, to try and get a feel of what was needed. After that I would sit down and write my own basic requirement documents, that I would then go over with the client.

If there is one thing you take away from this: “Always go over the requirements with the client in writing”

This has saved my @$$ many times.

It took some time learning how to deal with clients, and I learned a ton. This first client helped me learn, the development skills aren't the only thing important, it is important to understand the clients needs, pain points, and where to focus my efforts, this has rang true throughout my career.

I was lucky (in a sadistic kind of way) that my first client had no idea on how to write requirements, and how to handle a development cycle. In the years to come, I would receive many “requirements” of the type “put a button there that does that”; and 9/10 times there is a better way to do things that answers the clients problems in a better way. I urge you to look at anything you are developing, and try to understand the business use case behind it, and look at the bigger picture. You will be surprised to find that this will help you tremendously write better code.

Just for an example, I was told to make a button for export of the quote, sounds reasonable enough, but then it would be the sales agent's job to send copies to the client, the management, and the project manager. I wouldn't have known this if I just would have developed the button. Instead I suggested that the system have projects in it, with the project managers info, and have a “send to email” button. This may seem minor, but after talking to the CEO I found out it saved him tons of heartache, because more than once, the wrong quote was sent to the wrong client, just because the agent attached the wrong file.

What you can’t learn in Uni

During my time at uni, I learned a ton of things, in many different subjects, including programming. A few caveats to this is the fact that most of the time, the languages that were taught were out of date, and the professors didn't really know how to program a product, more like write blocks of code. Sadly there is more truth than I had hoped for in the old saying “Those who can't, teach”.

I think I can confidently say I learned more in my first freelance job, then everything uni could teach. There is nothing like getting your hands “dirty” and just doing some work. Here is a small list of the top things I “learned on the job”:

Understand the business - Like I stated before, understand “why” your client needs the feature, not just how they see the feature implemented
80/20 principle - Try and spend your time wisely covering most of the needs, and leaving “known issues” for the really rare edge cases.
KISS - “Keep it, simple stupid” (or sometimes I think it’s “Keep it simple, stupid”), try to simplify your code, break it into smaller bite sized pieces, that will pay dividends in the long run.
Performance isn't everything (or even important most of the time) - Today's computers are fast (like really really fast) and that fancy Binary search algorithm you learned, doesn't really matter in real life (most of the time), when you filter a list of even a few 10s of thousands, it’s not really going to change the runtime.
Use open source wherever you can - There are so many amazing open source projects out there, and they can cut your programming time by a huge factor. Additionally they can (usually) do it better then if you tried to write it yourself.

Please check out my open source library and maybe even star it. I would appreciate it greatly!

They don't call it intelligence for nothing

TheWizKid — Wed, 17 Aug 2022 14:20:08 +0000

The stack — C#, Oracle

The Experience — 4 Years

The Job — The intelligence corps

It's my time to serve

This may be doxing myself a little, but where I live there is a mandatory military service, for 3 f%$#ing years. Luckily for me, I had some skills, computer skills. It works here like this, you go do a lot of tests, intelligence tests, aptitude tests, physical tests, and at the end you get stationed somewhere (usually NOT where you want to be). I was in the running for a few different positions, and then one day out of the blue, I got a phone call. Come now to interview for a new position, nothing that I ever heard of, well of course I went. They were looking for someone with programming experience that they didn't need to send to the military academy to learn, and I fit the bill…

One thing led to another, and within the week I found myself in basic training on my way to the intelligence corps, as a developer in a small unit. After a few weeks of basic training (even us developers go through boot camp), I arrived at my unit. This is where I found out that just me and 1 other guy are the only people in the unit that can “speak computer”, and no I don't just mean program. We were in charge of the whole system there, from the network, to the servers, and of course the software.

How to learn C# in 3 easy steps

Get enlisted to the military, to a plush desk job as a programmer.
Be 1 of 2 people in charge of the whole system.
Never forget that if you don't get on top of things within a few months, they will ship you out to somewhere much worse.

Easy right?

But in all seriousness, I had to teach myself C# on the job. At least I had the running code, and the coveted PC on the internet. You see in the intelligence corps, and probably most military installations, no one has internet access on their PC. Well technically neither did I, but what I did have was access to the only PC (that was air gapped from the network) which had internet access.

By this time of my life (luckily for me), google was a thing already, and there were more and more resources online about programming. Even more for things like C#, because it was Microsoft's. But (I hate it that there is always a “but”)… The computer in question is air gapped, so no copy paste.

Into the deep end

This was the first time I truly worked with a DB (I don't really count MS Access). It was my first time, and they threw me into the deep end (without floats). Our systems used Oracle, till this day I think it's an amazing DB, but it's a beast. Our system was complex, and today, I would have never built it this way (I didn't build it back then either, the system was operational when I got there).

I know I'm going to spark some debate about this, but I dont think business logic is supposed to be in stored procedures (there I said it). This architecture caused some very weird side effects, for example, releasing versions requiring simultaneous release of new stored procedures, or updates to old ones. Triggers that can cause chain reactions of events, and you can't quite see them in any IDE.

If that wasn't bad enough, we didn't really have a server side. The C# was a desktop app, installed via group policy to all PCs on the network. The server was basically the stored procedures in Oracle. So things like creating thread pools to handle large tasks, or locks on editing were all out of the question.

This is where I started thinking about how code should be built, and it was the start of the path to architecture. Even if it still took a few years to find out what I should do, at least now, I know what I shouldn't. It was an important realization, one that I have since seen on some of the juniors I mentored. Writing code without bugs is important, but writing code in a maintainable way, is just as important.

My partner in crime

About half way through my service, the other guy I was with finished his tour. It was up to me to find a replacement. I had access to candidates (and it's not like they had CVs, they are all 18 year old fresh out of high school), I gave them tests to do, asked them questions. Remember back then I couldn't send them anything online, all by phone, or in person.

After a few weeks, and at least 50 candidates, I had found him, my new partner. Again we couldn't send him to learn in any course, so I had to find someone with the knowhow from home. Additionally even after I selected him, it took months for him to get to me. He had to get security clearance, and go through boot camp. So months later he was finally here, I would like to introduce you to a new character (that will have a recurring role in the articles to come), my business partner (and best friend).

Like any relationship it was rocky at first. I won't go into the details, but what I can say is that we went through a lot together. Things that built our relationship strong. Little did I know (spoilers) that we would be working together till this day (over 16 years).

Please check out our open source library and maybe even star it ;-)
We would appreciate it greatly.

So the journey begins

TheWizKid — Wed, 10 Aug 2022 12:52:04 +0000

The stack — VB6, C
The Experience — 2 Years
The Job — High school

Isn’t learning fun?

To set the scene, it was early 2000s in a small farming village. Yes we had internet, but it was dial up, and no stack overflow or anything close yet, the best you could hope for is a forum post.

So how did we learn? TL;DR — Books and lots of them…

I remember going to the library looking for books, swapping with friends. If you got the expensive ones you even got a CD (you know the round thing with a hole in it) with them with code examples and lesson projects, not that the CD ever worked (or was even there) from the library. No searching the book for your problem or an example, no copy paste, nothing. Just read and hope you remember later where you saw something about that problem you are having.

Yes the internet existed, but there wasn’t much in the way or programming guides, tutorials, or even examples. One of the most useful tools was IRC channels (for you kids out there, a really old chat). Tell you the truth, I don’t even remember how I found what channel to write to, but I remember talking to people from around the world about the most silly things, things that today you will find the solution to as the top search result on google.

I know this is weird, but you can learn at school

When I reached the last year of high school, I took a tech class. In this class we learned things from electrical engineering basics, mechanical design, and of course programming. The main problem was, I had taught myself more than what the teacher reached in class. Nonetheless I gained a great resource, my teacher could answer questions, point me in the right direction.

First of all, I would like to say, not all teachers are bad (yes many are, but not all). Luckily for me our high school tech teacher saw my potential (especially after I broke the school’s network). I used to get bored in class, everyone was learning C and VB6 at a level I passed a long time ago on my own. That is when my teacher came to my rescue.

My first real project

Computer vision, you know when computers look at an image and try to understand what is there. Well for that time it was no AI models or anything, it was purely algorithmic. My mission should I choose to accept it (over being thrown out of class) was to take this “new” library that allowed me to capture an image from a webcam in VB6 and process it (pixel by pixel), and try and make out a shape in an image, simple geometric shapes (circle, square, triangle).

So my best friend and I set out to this task, it was soooo interesting we would have all nighters trying to solve it. Let me take you through the basics.

Go over the image one pixel at a time to try and find a place with a big change in color (black shape on white paper).
When I reach it, go in a pattern to find its neighbor that is also a border pixel.
Do this in a loop till I can’t find any more border pixels.
This worked (in theory). There is always a “but”… It all went to hell when we would hit these edge cases of a few pixels out of place.

That’s when it hit me, the solution may be obvious to you, but to me it wasn’t. Well, it turns out there is a use for the data structure “stack” we’ve been taught. We put all pixels in the stack, and each time we would hit a brick wall, we would pop a pixel and try again. This would continue till we popped them all out and got back to our starting pixel.

This turned out to be very effective, and solved our biggest problem.

We actually found a few solutions for understanding what shape it was.

From the trivial, see how many “straight” lines made up the shape, which worked pretty well for simple shapes. To the most advanced solution, that I am still proud of till this day, calculate the area of the shape, and the perimeter of the shape, and see what constant best defines the relation.

The moral of the story

I told you this whole story to come to a small point, one that shaped the next few years of my development life. Data structures & design patterns are there for a reason, and even though it might not be obvious, you ask yourself “why would I ever need that”. At some point you will, and it’s good to keep them handy.

If you would like to check out our open source library and leave feedback we would appreciate it greatly.

Object Mapping advanced features & QoL with Kotlin

Aviv Mor — Tue, 09 Aug 2022 18:33:31 +0000

When working with multi-layered applications, external libraries, a legacy code base or external APIs, we are often required to map between different objects or data structures.

In this tutorial, we will check out some Object Mapping libraries advanced features to simplify this task while saving development and maintenance time.

In our examples we will use the library ShapeShift. It's a light-weight object mapping library for Kotlin/Java with lots of cool features.

Auto Mapping

Simple Mapping

Let's start with a simple example for auto mapping. We have our two objects, imagine they could also have tens, hundreds, or even thousands (for the crazy people here) of fields.

class User {
    var id: String = ""
    var name: String? = null
    var email: String? = null
    var phone: String? = null
}

class UserDTO {
    var id: String = ""
    var name: String? = null
    var email: String? = null
    var phone: String? = null
}

We want to map all the field from User to UserDTO. Using auto mapping we don't need to write any boiler-plate code. The mapper will be defined as follow:

val mapper = mapper<User, UserDTO> {
    autoMap(AutoMappingStrategy.BY_NAME_AND_TYPE)
}

Voila! All the fields will be mapped automatically without any manual boiler-plate code.

Advanced Mapping

In this example we will use the power of default transformers to take auto mapping even further.

class User {
    var id: String = ""
    var name: String? = null
    var birthDate: Date? = null
}

class UserDTO {
    var id: String = ""
    var fullName: String? = null
    var birthDate: Long? = null
}

Note that the types of the birthDate field are different in the source and destination classes. But using the power of default transformers we can still use auto mapping here.

val mapper = mapper<User, UserDTO> {
    autoMap(AutoMappingStrategy.BY_NAME)
}

val shapeShift = ShapeShiftBuilder()
    .withTransformer(DateToLongMappingTransformer(), true)
    .build()

val mapper = mapper<User, UserDTO> {
    autoMap(AutoMappingStrategy.BY_NAME)
    User::name mappedTo UserDTO::fullName
}

Auto mapping is great for use cases that does not require specific mapping. It helps reduce the amount manual boiler-plate code needed to configure mapping and also helps you keep your sanity.

Transformers

Transformers are very useful feature that allows you to transform the type/value of a field to a different type/value when mapping a field.

Some use cases we have been using widely:

Transform date to long and vice versa between server and client objects.
Transform JSON string to it's actual type and vice versa between server and client objects.
Transform comma separated string to list of enums.
Transform another object id to its object or one of its fields from the DB using Spring transformers.

Basic Transformers

We will start with a simple transformer example. Date-to-Long and Long-to-Date transformers:

class DateToLongMappingTransformer : MappingTransformer<Date, Long> {
    override fun transform(context: MappingTransformerContext<out Date>): Long? {
        return context.originalValue?.time
    }
}

class LongToDateMappingTransformer : MappingTransformer<Long, Date> {
    override fun transform(context: MappingTransformerContext<out Long>): Date? {
        context.originalValue ?: return null
        return Date(context.originalValue)
    }
}

All we need to do now is to register them.

val shapeShift = ShapeShiftBuilder()
    .withTransformer(DateToLongMappingTransformer(), true) // "true" is optional, we are registering the transformers as default transformers, more on that later.
    .withTransformer(LongToDateMappingTransformer(), true)
    .build()

That's it! We can now use the transformers when mapping objects.

class User {
    var id: String = ""
    var name: String? = null
    var birthDate: Date? = null
}

class UserDTO {
    var id: String = ""
    var name: String? = null
    var birthDate: Long? = null
}

val mapper = mapper<User, UserDTO> {
    User::id mappedTo UserDTO::id
    User::name mappedTo UserDTO::name
    User::birthDate mappedTo UserDTO::birthDate withTransformer DateToLongMappingTransformer::class // We don't have to state the transformer here because it is a default transformer
}

Inline Transformers

In some use cases we want to transform the value but we don't need a reusable transformer and we don't want to create a class just for a one time use.

Inline transformers for the rescue! Inline transformers allow to transform the value without the need to create and register and transformer.

val shapeShift = ShapeShiftBuilder()
        .withMapping<Source, Target> {
            // Map birthDate to birthYear with a transformation function
            Source::birthDate mappedTo Target::birthYear withTransformer { (originalValue) ->
                originalValue?.year
            }
        }
        .build()

Advanced Transformers

Transformers also allow us to do transformations with the DB or other data sources.

In this example we will use the power of the Spring Boot integration to create transformers with DB access.

We have three models:

Job - DB entity.
User - DB entity.
UserDTO - Client model.

class Job {
    var id: String = ""
    var name: String = ""
}

class User {
    var id: String = ""
    var jobId: String? = null
}

class UserDTO {
    var id: String = ""
    var jobName: String? = null
}

We want to convert the jobId on User to jobName on UserDTO by querying the job from the DB and setting it on the DTO.

In Spring's case, you generally avoid interaction with the application context from static functions or functions on domain objects.

We will use a ShapeShift's Spring integration to create a component as a transformer to access our DAO bean.

@Component
class JobIdToNameTransformer(
    private val jobDao: JobDao
) : MappingTransformer<String, String>() {
    override fun transform(context: MappingTransformerContext<out String>): String? {
        context.originalValue ?: return null
        val job = jobDao.findJobById(context.originalValue!!)
        return job.name 
    }
}

All that's left to do is to use this transformer in our mapping.

val mapper = mapper<User, UserDTO> {
    User::id mappedTo UserDTO::id
    User::jobId mappedTo UserDTO::jobName withTransformer JobIdToNameTransformer::class
}

Another bonus of using transformers is their reusability. In some use cases, We could create more generic transformers that will have application-wide usage.

Default Transformers

As we already seen, default transformers are useful for recurring transformations and especially for automatic mapping.

Deep Mapping

What if we want to map from/to fields that available inside a field which is an object? We can even do that, easily.

In order to access child classes we can use the .. operator. Let's look at the following example.

class From {
    var child: Child = Child()

    class Child {
        var value: String?
    }
}

class To {
    var childValue: String?
}

We want to map the value field in Child class inside the From class to the childValue field in the To class. We will create a mapper with the .. operator.

val mapper = mapper<From, To> {
    From::child..From.Child::value mappedTo To::childValue
}

Let's take it one step further with multi level depth.

class From {
    var grandChildValue: String?
}

class To {
    var child: Child = Child()

    class Child {
        var grandChild: GrandChild = GrandChild()
    }

    class GrandChild {
        var value: String?
    }
}

To access the grand child field we just use the .. operator twice.

val mapper = mapper<From, To> {
    From::grandChildValue mappedTo To::child..To.Child::grandChild..To.GrandChild::value
}

Conditional Mapping

Conditions allow us to add a predicate to a specific field mapping to determine whether this field should be mapped.

Using this feature is as easy as creating a condition.

class NotBlankStringCondition : MappingCondition<String> {
    override fun isValid(context: MappingConditionContext<String>): Boolean {
        return !context.originalValue.isNullOrBlank()
    }
}

And adding the condition to the desired field mapping.

data class SimpleEntity(
    val name: String
)

data class SimpleEntityDisplay(
    val name: String = ""
)

val mapper = mapper<SimpleEntity, SimpleEntityDisplay> {
    SimpleEntity::name mappedTo SimpleEntityDisplay::name withCondition NotBlankStringCondition::class
}

Inline Conditions

Like transformers, conditions can also be added inline using a function.

val mapper = mapper<SimpleEntity, SimpleEntityDisplay> {
    SimpleEntity::name mappedTo SimpleEntityDisplay::name withCondition {
        !it.originalValue.isNullOrBlank()
    }
}