DEV Community: Matteo Joliveau

Why we went from Slack to Discord

Matteo Joliveau — Thu, 09 Apr 2020 15:56:01 +0000

The beginning of 2020 was one of the most difficult moments of the last decade. The rapid spread of SARS-COVID-2 and the consequent quarantine imposed by the various countries of the world has led many companies to discover and approach remote work for the first time, in order to protect their employees while still being able to work without attending the office. However, choosing the right tools to support remote collaboration is almost as difficult a challenge as adopting "smart" working methodologies that allow you to keep productivity high. The panorama of available choices is vast, from open-source tools such as Jitsi and Nextcloud to commercial services such as Slack and Google GSuite, each with its own strengths and peculiarities.

At Mikamai we have always been used to working in a "smart" way, preferring asynchronous and written collaboration wherever possible to make our business more streamlined and agile. This allows us to expand our operational area to the entire world, both in terms of customers and collaborators. In fact, many colleagues do not work from our main office in Milan, Italy, but from more comfortable places for them, such as other Italian or European cities, even other continents. We have long been GSuite users for shared documents, GitHub for hosting our projects, Trello for managing activities and Slack for internal communication.

However, we recently started to feel the limitations of Slack’s free plan. Mikamai has grown, the number of people and messages written every day has grown significantly with it, and we have found ourselves to frequently exceed the limit of 10,000 messages, thus losing potentially important conversations and messages. The lack of group video calls forces us to use Google Meet, an excellent video conferencing solution but which lacks the immediacy of starting a call directly from the text chat. However, paid plans, the price of which increases for each employee present in the workspace, are not very appealing to us. So we started scouting for a tool that would better meet our needs.

In the past few months we have tried several online communication tools, in particular Mattermost, an open source alternative to Slack, JetBrains Space, a collaboration and management service that includes a chat very similar to Slack, and Discord, a communication platform designed for gamers and online communities, but with very interesting features also for companies that work in a distributed way.

The latter has caught our attention, and here are the reasons that led us to adopt it as our corporate communication tool.

Voice chat is only one click away

Despite working mainly in an asynchronous way through email, Trello cards and comments on the code repositories, direct conversation is a very important tool especially for creative activities that benefit from a rapid and continuous exchange of ideas between the participants. The ability of immediately entering a group voice chat, with excellent audio quality and with the support of text chat and screen sharing is perhaps Discord's main strength, and what really convinced us in attempting the transition.

Brainstorming and planning sessions can be started quickly without the overhead that changing tools entails when using services outside the chat, as was our case with Google Meet. Everything is much more natural and immediate, promoting communication and the exchange of ideas rather than discouraging them.

It is also very convenient for organizing virtual coffee breaks, allowing you to maintain human contact with colleagues that you risk losing by working remotely.

Granular control over roles and permissions

Discord was created to host online communities, mainly in the gaming world but extending over time to all those groups of users who need a safe digital space in which to meet and chat. For this reason, the platform provides a wide range of permissions, which can be grouped into roles to be assigned to users, allowing you to granularly control which actions are granted to which users, and in which channels.
In our particular case we do not manage a public space but a private server, however it is not used solely for business purposes. We have dedicated some channels to external activities, like video games or role-playing games, which involve the presence of users who are not part of the company staff, such as family members and playmates, former colleagues and partners of other companies. Thanks to Discord's powerful permission system, these users have access only to the channels for which they were authorized, while the company's internal chats are hidden and inaccessible.

This also allows us to invite our customers to join the server to facilitate the exchange of information with them. For each project, a text channel and a corresponding voice room are created, along with a role that allows access to them. When the customer enters the server and is assigned the role, he can then access these channels and actively collaborate with us, without being able to see reserved areas and spaces dedicated to other active projects.

We have also created a dedicated announcement channel, in which only Mikamai's administrative staff can write, to support the internal mailing list in quickly sharing official news and communications with colleagues.

External integrations

Slack boasts a multitude of integrations with third-party systems, from ticketing and project management platforms to infrastructure alarms and monitoring systems. Discord does not have the same spread in the business and development world, but provides the possibility to configure the integrations in a Slack compatible mode, basically allowing to integrate any system that can interact with Slack.
Discord also has a robust and diverse chatbot ecosystem, small applications that act like automatic users and increase native chat features. For example, a very successful bot in Mikamai’s server is Groovy, a bot that allows you to play music to all users connected to a voice channel as a kind of virtual jukebox. Thanks to it, we can share our music with colleagues, allowing everyone to add songs to the playlist being played.

Cost

Obviously one aspect that we have not underestimated during our evaluation is the monetary cost compared to the competition. All of Discord’s core features are completely free, providing paid additions mainly aimed at the gaming world, especially streamers and youtubers, such as the ability to add custom emojis, customize the profile and improve audio and video quality. This, added to the points previously discussed, was one of the key points for Mikamai in making the decision to migrate to Discord.

Testimonials

Given that I’m not a very techy person, which often limits me in discovering the full potential of this type of tool, I must say that I find Discord very versatile. I really appreciate the possibility of enabling audio channels that allow an immediate switch between text communication and voice communication (especially group chats). Having an unlimited number of messages always available free of charge is, of course, another nice advantage.

Debora, Office Manager

Discord adds syntax highlighting for code snippets, unlimited message history, integrated calls, screenshare with up to 50 people (during the COVID-19 period), a really advanced role and permission system, and is very easy to integrate with external tools. I don't know what more you could ask for.

Nicola Racco, Tech Lead

Nicola RaccoFollow

I have been using Discord for more than a year mainly for gaming but also to talk or see my friends. I immediately found it intuitive and with a youthful graphic feeling.
Its strength is absolutely being free, I still remember when I had to communicate with other players simultaneously, we used TeamSpeak and it required us to spend a lot of money every month for a dedicated server, with an increasing cost based on the amount of users who could connect at once, a real nightmare.
With Slack, I suffered (and still suffer) because of the limited chat history, 10,000 messages or a month of history and then that’s it. How many lost links, how many chats that have fallen by the wayside ("I wrote it to you on Slack", "When ???") ...
Discord is, in my opinion, the tool par excellence of remote communication both for gaming, chatting and companies.

Daniela, Frontend Developer

DanielaFollow

Conclusions

Discord has proved to be an excellent corporate communication platform despite being mainly geared towards the world of gaming and digital communities. The speed with which it is possible to start talking with colleagues and organize virtual meetings has allowed us to facilitate the transition even for those who were not previously used to working completely remotely, and has given us back a part of the warmth that sharing the day with our coworkers transmit, reducing the alienation due to the emergency situation in which we find ourselves. We will have the opportunity to test it thoroughly in the coming months, but we are sure that it will prove itself to be a valid and central tool for our daily work.

DEV Milan has started, and it was awesome!

Matteo Joliveau — Tue, 24 Sep 2019 13:12:34 +0000

On Monday 23rd September we kicked off the first event of the DEV Community Milanese chapter, and it was a blast! 🇮🇹

We as Mikamai have fostered and helped to gather many tech communities in Milan over the course of the years, and it with great pleasure that we organized this one ourselves to thank all these awesome people and their daily effort in bringing Italian developers together.

We hosted two talks (actually one big talk divided into two sections) exploring the magical world of GraphQL APIs first on the backend side, and then on the frontend.

The Mighty Backend

I started the evening with the backend talk, introducing new people to what GraphQL is and isn't and quickly explaining the inner working of the language, then moved on to more practical advices and strategies for implementing clean and effective GraphQL APIs.

We touched topics like types and scalars, datasources, batch loading and evolving schemas, and tried to give actionable suggestions on how to cleanly and efficiently tackle all these issues.

Participants were very active and asked clever questions that made the whole experience much more inclusive and enjoyable!

When in Italy, just pizza 🍕

After the first talk, it's time for a break! We offered pizzas, chips and beverage to all attendees and had a fun time networking, discussing and generally have a good time. It was really fun and I'm very proud of how the evening turned out.

The Glorious Frontend

Then it was time for the second half of the event. My colleague Mattia Panzeri, one of our finest frontend engineers, talked about the challenges of modern frontend development, from multiple devices and screen resolutions to finicky mobile networks and voice assistants, and how GraphQL help clients tailor API requests exactly to their needs. He provided examples of similar and related technologies like Netflix Falcor and why they failed to achieve what GraphQL does.

All of this explained with his witty and funny style that made the whole speech very lighthearted and enjoyable.

We close the evening with a live demonstration using GitLab's GraphQL Explorer to showcase how queries work in the wild, and of course, this being a live demo it went horribly wrong with authentication problems and the window manager of my laptop ultimately deciding that it didn't like our projector so much, but all in all we had good fun.

We gave out stickers to everyone and announced a followup of this event, that we absolutely want to make again and possibly on a regular basis. We are thinking one per month or every two months, depending on speakers' availability.

I really want to thank the DEV Team for launching IRL and supporting this kind of initiatives. There is still work to be done on the logistic side of things, but in the end, the event was a success and everyone enjoyed it.

Thank you for reading and I hope to see you next time!

You can find the slides here.

// Detect dark theme var iframe = document.getElementById('tweet-1176193547768389633-453'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1176193547768389633&theme=dark" }

I don't like remote working

Matteo Joliveau — Tue, 09 Oct 2018 07:49:36 +0000

I feel like I'm the only developer in the world that truly enjoy going to the office every morning.

I like my colleagues, sometimes we have sushi for lunch, we play boardgames once a week after work and is generally a fun experience to work at the same desk. Being able to share knowledge and discuss with other colleagues face-to-face is really important to me, and the light chitchat at the coffee machine is invaluable.

So while I do understand the benefits of remote working (to be fair, we can do remote at will, and I did a couple of times) I really don't see the value in terms of productivity and social engagement over working physically all together.

What are your thoughts on the matter? Do you work remotely or on site? Are you happy about your setup?

I built a custom CMS because why not

Matteo Joliveau — Mon, 13 Aug 2018 20:07:33 +0000

My personal website was a static site generated with Hugo, and it worked very well. It was fast to serve, lightweight and the Elate theme I used was simply gorgeous.
However, I had to edit the config file manually every time I wanted to add or remove a project from my portfolio, or a job description from my resume. And then I had to recompile the website and push it back to GitHub, which would trigger a pipeline to build a Docker container to be deployed on my server and yadda yadda. Boring and unpractical.
So I started looking for some cool and easy to maintain CMSs. WordPress was obviously the first one that came to mind but I can't really stand it after having worked for 4 years as an editor for various websites, plus it's the opposite of maintainable and I don't like PHP.
October seemed a cool alternative, even if it is written in PHP at least it ses Laravel which is a well-designed framework, but I had troubles deploying it with Docker. Same for Wagtail, which was in Python, a language I like.

So I gave up and started working on my own CMS, tailored around my personal needs.
I figured that if a good looking, well animated and stylish front page is crucial for frontend developers to showcase their skills, a robust and well-built web application works the same way for a backend developer.

Round of applause for DevCMS and its very original name

It's a very simple app, built with Ruby on Rails 5.2 and Bulma for the CSS part (I'm a backend developer, I suck at styles).
It uses Administrate as its back-office interface and ActiveStorage to upload and store images in S3 (actually, I host a private, S3-compatible Minio server, but it's the same). Bear in mind that this app was built in two days, so rough edges are still there.

DevCMS is built around three core concepts.

Projects

A portfolio is crucial for a software developer, it helps to show off what you can do to potential recruiters and peers. So projects are first class citizens here. They have a name, a short description, a longer description (formatted in HTML and written with a WYSIWYG editor in the admin panel), some tags and a cover image. They can be "starred" so that more significant projects appear on homepage, and they have a concept of draft/published, so they can be created and later compiled before being published.

Jobs

We are one of the most mobile and dynamic professional figures, so our career timeline is very important to demonstrate the positions we held in the past. Jobs are very similar to their LinkedIn counterparts, a position at a company during a set period of time. Like projects, they have short and long descriptions, a cover image and can be drafted and published. They are presented in a nice little timeline created with a Bulma extension, from newest to oldest.

~~### Copies~~
I wanted to have some static pages like about me and contacts but didn't want to hardcode them in HTML. So, while I work on implementing proper page generation, I added the concept of a Copy, a piece of formatted text stored in Postgres that is served inside the HTML. Pages are still hardcoded for the moment, but at least I can edit their content directly from the admin panel.

Pages

UPDATE 18/08/2018
I added basic support for dynamic pages so that they can be created and edited without having to access the source code and perform redeploy.
Templates are written in Liquid language, so they are safe and controlled, and can access the published project and job lists. Pages can be automatically added to the navbar and their order controlled by a weight system.

Also, some smaller things like copyright year and author for the footer can be customized via Config objects, key-value pairs stored in the database.

The Need for Speed

Inspired by The Practical Dev and this very website, I wanted mine to be equally fast and performant. Beside standard Rails solutions like the provided Turbolinks installation, I kept the JavaScript presence very small (only Turbolinks, ActiveStorage, and the mobile burger button handler are present). I also enabled AWS CloudFront as its CDN to cache static assets and enabled CloudFlare as an extra caching layer and DDoS protection (I had a couple in the past, no idea why).
I have it monitored by Sentry so that exceptions are automatically handled and reported. It's a very nice service, if I may.

At the moment it is not a general purpose framework or application that anyone can install, but it is easily customizable if you want to try it on your own. It is open sourced at GitHub.
I'd like to make it more generic and tunable in the future, maybe by adding themes support and real admin-defined static pages.
I also want to add the concept of skills, to show how confident or expert I am with various technologies and languages, maybe with some fancy percentage bars or something along those lines.
So, what do you think? How did you put together your personal developer website?

Effective Service Objects in Ruby

Matteo Joliveau — Mon, 04 Jun 2018 19:47:48 +0000

As a Java developer that recently transitioned to a Ruby on Rails company, I felt kinda lost when I discovered that the use of models directly inside of controllers was a common practice.

I have always followed the good practices of Domain Driven Design and encapsulated my business logic inside special classes called service objects, so in Java (with Spring) a controller would look like this:

@Controller
@RequestMapping("/api/users")
public class UserController {

    private final UserService userService;

    public UserController(UserService userService) {
        this.userService = userService;
    }

    @GetMapping
    public ResponseEntity<Iterable<User>> getAllUsers() {
        return ResponseEntity.ok(userService.getUsers());
    }
}

Verbosity aside, this is nice and clean with good separation of concerns. The actual business logic that retrieves the user list is delegated to the UserService implementation and can be swapped out at any time.

However, in Rails we would write this controller as such:

class Api::UserController < ApplicationController
    def index
        @users = User.all
        render json: @users
    end
end

Yes, this is indeed shorter and even cleaner than the Java example, but it has a major flaw. User is an ActiveRecord model, and by doing this we are tightly coupling our controller to our persistence layer, breaking one of the key aspects of DDD. Moreover, if we wanted to add authorization checks to our requests, maybe only returning a subgroup of users based on the current user's role, we would have to refactor our controller and putting it in charge of something that is not part of the presentation logic. By using a service object, we can add more logic to it while being transparent to the rest of the world.

Let's build a service object

In Java this is simple. It's a singleton class that is injected into other classes by our IoC container (Spring DI in our example).
In Ruby, and Rails especially, this is not quite the same, since we can't really inject anything in our controller constructor. What we can do, however, is taking inspiration by another programming language: Elixir.
In Elixir, a functional language, there are no classes nor objects, only functions and structs. Functions are grouped into modules and have no side effects, a great feature to ensure immutability and stability in our code.
Since Ruby too has modules, we can use them to implement our service object as stateless collections of methods.

Our UserService can look something like this:

module UserService
    class << self
        def all_users
            User.all
        end
    end
end

And then will be used like this:

class Api::UserController < ApplicationController
    def index
        @users = UserService.all_users
        render json: @users
    end
end

This doesn't sound like a smart move, does it? We just moved the User.all call in another class. And that's true, but now, as our application grow we can add more logic to it without breaking other code or refactoring, as long as we keep our API stable.
One small change I'll make before proceding. Since we may want to inject some data into our service on every call, we'll define our methods with a first parameter named ctx, which will contain the current execution context. Stuff like the current user and such will be contained there.

module UserService
    class << self
        def all_users _ctx # we'll ignore it for now
            User.all
        end
    end
end

class Api::UserController < ApplicationController
    def index
        @users = UserService.all_users { current_user: current_user }
        render json: @users
    end
end

Applying business logic

Now let's build a more complex case, and let's use a user story to describe it first. Let's imagine we're building a ToDo app (Wow, how revolutionary!).
The story would be:

As a normal user I want to be able to see all my todos for the next month.

The RESTful HTTP call will be something like:
GET /api/todos?from=${today}&to=${today + 1 month}

Our controller will be:

class Api::TodoController < ApplicationController
    def index
        @ctx = { current_user: current_user }
        @todos = TodoService.all_todos_by_interval @ctx, permitted_params
        render json: @todos
    end

    private

    def permitted_params
        params.require(:todo).permit(:from, :to)
    end
end

And our service:

module TodoService
    class << self
        def all_todos_by_interval ctx, params
            Todos.where(user: ctx[:current_user]).by_interval params
        end
    end
end

As you can see we are still delegating the heavy database lifting to the model (throught the scope by_interval) but the service is actually in control of filtering only for the current user. Our controller stays skinny, our model is used only for persistence access, and our business logic doesn't leak in every corner of our source code. Yay!

Service Composition

Another very useful OOP pattern we can use to enhance our business layer is the composite pattern. With it, we can segregate common logic into dedicated, opaque services and call them from other services. For example we might want to send a notification to the user when a todo is updated (for instance because it expired). We can put the notification logic into another service and call it from the previous one.

module TodoService
    class << self
        def update_todo ctx, params
            updated_todo = Todos.find ctx[:todo_id]
            updated_todo.update! params # raise exception if unable to update
            notify_expiration ctx[:current_user], updated_todo if todo.expired?
        end

        private

        def notify_expiration user, todo # put in a private method for convenience
            NotificationService.notify_of_expiration { current_user: user }, todo
        end
    end
end

Commands for repetitive tasks

As the Gang of Four gave us a huge amount of great OOP patterns, I'm going to borrow one last concepts from them and greatly increase our code segregation. You see, our services could act as coordinators instead of executors, delegating the actual work to other classes and only caring about calling the right ones. Those smaller, "worker-style" classes can be implemented as commands. This has the biggest advantage of enhancing composition by using smaller execution units (single commands instead of complex services) and separating concerns even more. Now services act as action coordinators, orchestrating how logic is executed, while the actual execution is run inside simple, testable and reusable components.

Side Note: I'm going to use the gem simple_command to implement the command pattern, but you are free to use anything you want

Let's refactor the update logic to use the command pattern:

class UpdateTodo
    prepend SimpleCommand

    def initialize todo_id, params
        @todo_id = todo_id
        @params = params
    end

    def call
        todo = Todos.find @todo_id

        # gather errors instead of throwing exception
        errors.add_multiple_errors todo.errors unless todo.update @params
        todo
    end
end

module TodoService
    class << self
        def update_todo ctx, params
            cmd = UpdateTodo.call ctx[:todo_id], params

            if cmd.success?
                todo = cmd.result
                notify_expiration ctx[:current_user], todo if todo.expired?
            end

            # let's return the command result so that the controller can
            # access the errors if any
            cmd
        end

        private

        def notify_expiration user, todo # put in a private method for convenience
            NotificationService.notify_of_expiration { current_user: user }, todo if todo.expired?
        end
    end
end

Beautiful. Now every class has one job (Controllers receive requests and return responses, Commands execute small tasks and Services wire everything together), our business logic is easily testable without needing any supporting infrastructure (just mock everything. Mocks are nice.) and we have smaller and more reusable methods. We just have a slightly bigger codebase, but it's still nothing compared to a Java project and it's worth the effort on the long run.
Also, our services are no longer coupled to any Rails (or other frameworks) specific class. If for instance we wanted to change the persistence library, or migrate one business domain to an external microservice, we just have to refactor the related commands without having to touch our services.

Are you using service objects in your Ruby projects? How did you implement the pattern and what challenges did you solved that my approach does not?

PlugFace Reborn - the Java Plugin System

Matteo Joliveau — Mon, 09 Apr 2018 08:53:44 +0000

Visit https://plugface.matteojoliveau.com for the full documentation

One year ago I came across the need to dynamically load bits of code into my Java applications without having to recompile or repackage my software every time. Basically, I was looking for a plugin system.

There was only one open source plugin system worth noting at the time, PF4J, but I felt it was a bit clunky and cumbersome. I wanted something dead simple, that I could use to be productive in minutes. So I decided to write my own.

Enter PlugFace.
And the first iteration was sh*tty as hell, because at the time I was just an intern at my company, still learning real-world Java and didn't have the knowledge to write a good, usable framework.

The Ugly

The first implementation was not so simple as I wanted it to be.
There were interfaces to implement, reflection to use, configuration files to write, a sandboxed permission system, a half-working/half-broken attempt at dependency injection between plugins, and a lot of sweat and tears from me to both write it and use it.
Not the greatest software in the world.
Here is an example of an old PlugFace plugin:

package org.plugface.demo.plugins.greet;

import org.plugface.Plugin;
import org.plugface.PluginConfiguration;
import org.plugface.PluginStatus;
import org.plugface.demo.app.sdk.Greeter;
import org.plugface.impl.DefaultPluginConfiguration;

import java.util.Collections;

public class GreeterPlugin implements Plugin<String[], String>, Greeter{

    private final String name;
    private PluginConfiguration configuration;
    private PluginStatus status;
    private boolean enabled;

    public GreeterPlugin () {
        name = "greeter";
        configuration = new DefaultPluginConfiguration();
        status = PluginStatus.READY;
        enabled = false;
    }

    @Override
    public void start() {
        throw new UnsupportedOperationException("This plugin operates in single mode only");
    }

    @Override
    public void stop() {
        throw new UnsupportedOperationException("This plugin operates in single mode only");
    }

    @Override
    public String execute(String[] parameters) {
        return greet();
    }

    @Override
    public PluginConfiguration getPluginConfiguration() {
        return (PluginConfiguration) Collections.unmodifiableMap(configuration);
    }

    @Override
    public void setPluginConfiguration(PluginConfiguration configuration) {
        this.configuration = configuration;
    }

    @Override
    public String getName() {
        return name;
    }

    @Override
    public PluginStatus getStatus() {
        return status;
    }

    @Override
    public void setStatus(PluginStatus pluginStatus) {
        status = pluginStatus;
    }

    @Override
    public void enable() {
        this.enabled = true;
    }

    @Override
    public void disable() {
        this.enabled = false;
    }

    @Override
    public boolean isEnabled() {
        return enabled ;
    }

    @Override
    public String greet() {
        return "Hello PlugFace!":
    }

}

These things were monsters, the interface was huge and full of debatably useful features (like status and name) and they were not practical to implement.
If you wanted more complex behaviors than 'start', 'stop' and 'execute', you had to add your own interfaces and cast your plugin to that type in order to use it or access the methods via reflection.

Then an idea stroke me. Why bother restricting the way a user interacts with a plugin when all the framework could deal with was the dynamic class loading?

The Good

With version 0.6, the framework revived. It was an (almost) complete rewrite. I ditched the whole interface story for a much cleaner @Plugin("name") annotation and ZERO opinions on how your plugin shall behave. Simply make it implement some interface your application is aware of and then access it through it.

Here is the same plugin but with the new system:

package org.plugface.demo.plugins.greet;

import org.plugface.core.annotations.Plugin;
import org.plugface.demo.app.sdk.Greeter;

@Plugin("greeter")
public class GreeterPlugin implements Greeter {

    @Override
    public String greet() {
        return "Hello PlugFace!";
    }
}

It implements the Greeter interface provided by my application. Now my code can simply retrieve the plugin an put it in a variable of type Greeter without ever have to be aware of the GreeterPlugin class. Neat!

Here is my application code.

final PluginManager manager = PluginManagers.defaultPluginManager();

manager.loadPlugins(PluginSources.jarSource("path/to/my/plugin/jars"));

final Greeter greeter = manager.getPlugin(Greeter.class);

greeter.greet(); // => "Hello PlugFace!"

The PluginManager is a utility class to load plugins from various sources, another improvement over the old version which only featured hard-coded JAR loading, and accesses the PluginContext to add/get/remove plugins both by name and by type.

I also entirely ditched the Sandbox functionalities, which leveraged the Java SecurityContext features to restrict plugins in what they were allowed to do. As it was a huge pain to deal with, I preferred to remove it entirely to focus on the core features for this new version. I might reintroduce it in a future version if I find a way to implement it in a convenient fashion.

Dependency Injection done right

A feature the old system tried to introduce was the possibility of having plugins injected into each other. It achieved this by putting plugins into the PluginConfiguration object (which was basically a glorified Map<String, Object>, it worked but was fairly unpractical to use.

PlugFace 0.6 introduced a full-fledged dependency injection system via constructors, using the standard @javax.inject.Inject annotation (which will be made optional in the future in case of a single constructor being present) so that plugin classes can be used with any standard DI framework such as Spring or Guice.
It supports complicated dependency graphs using topological sorting, with circular dependencies detection and all that good stuff.

package org.plugface.demo.plugins.math;

import org.plugface.core.annotations.Plugin;
import org.plugface.demo.app.sdk.Mathematics;

import javax.inject.Inject;

@Plugin("math")
public class MathPlugin implements Mathematics {

    private final SumPlugin sum;
    private final MultPlugin mult;

    @Inject
    public MathPlugin(SumPlugin sum, MultPlugin mult) {
        this.sum = sum;
        this.mult = mult;
    }

    @Override
    public Integer sum(int a, int b) {
        return sum.sum(a, b);
    }

    @Override
    public Integer mult(int a, int b) {
        return mult.mult(a, b);
    }
}

Spring Integration

The Spring Framework is a tremendous toolkit in Java. It features pretty much anything, but its core feature is clearly the dependency injection system. Since I use Spring a lot in nearly all my projects, I wanted PlugFace to be well integrated with it.
If you want to use plain vanilla PlugFace you can import the plugface-core module, but if using Spring you may want to switch to the special plugface-spring module.
Not only it sports auto-configured beans for both the PluginManager and the PluginContext, but it also adds support for plugin retrieval from the ApplicationContext (meaning if an object is not found as a plugin, it will be looked up amongst Spring beans) and, perhaps more importantly, it makes Spring beans viable targets for dependency injection in plugins.

That's right, with plugface-spring you can not only inject other plugins inside your plugins, but also any Spring bean you have registered in your application.

Your plugins can access service classes, repositories, utility singletons, basically anything lives inside the Spring context.

package org.plugface.demo.plugins.user;

import org.plugface.core.annotations.Plugin;
import org.plugface.demo.app.sdk.Greeter;
import org.plugface.demo.app.sdk.TestService;

import javax.inject.Inject;

@Plugin("greeter")
public class UserPlugin implements UserDetails {

    private final UserService userService; //this is a Spring service

    @Inject
    public GreeterPlugin(UserService userService) {
        this.userService = userService;
    }

    @Override
    public String getUsername() {
        final User user = userService.getUserById(0L);
        return user.getUsername();
    }
}

From Now On

In the future, I want to integrate with other DI frameworks as well, such as Guice, and expand upon the current feature set if any new use case for plugins in Java comes up.
Until then, thanks for the attention, and happy coding!

Microservices communications. Why you should switch to message queues.

Matteo Joliveau — Fri, 23 Feb 2018 16:59:52 +0000

This is actually a more elaborate version of an old comment I wrote here

When doing microservices, a fairly crucial design point is: how should my services communicate with each other?
Many people usually choose to design some RESTful HTTP API that each service expose and then have the other services invoke it with a normal HTTP client.
This has some advantages, primarily making it easy to discover services by using DNS resolution and API Gateways, but it has many drawbacks too.
For example, what if the called service has crashed down and cannot respond? Your client service has to implement some kind of reconnection or failover logic, otherwise, you risk to lose requests and pieces of information. A cloud architecture should be resilient and recover gracefully from failures. Also, an HTTP request is a blocking API, making it asynchronous implies some tricky shenanigans on the client side.

Entering message queues

Using message queues is actually a fairly old solution when dealing with multiple intercommunicating services. Think about old approaches like Enterprise Service Buses or the Java JMS specification.

Let's start by introducing some terminology:

Message: a package for information, usually composed of two parts. Some headers, key-value pairs containing metadata, and a body, a binary package containing the actual message itself.
Producer: whoever creates and sends a message
Consumer: whoever receives and reads a message
Queue: a communication channel that enqueues messages for later retrieval by one or more consumers
Exchange: a queue aggregator that abstract away message queues and routes messages to the appropriate queue based on some predefined logic.

A message queue architecture requires an additional service called a message broker that is tasked with gathering, routing and distributing your messages from senders to the right receivers.

Think of it as a mail service. You (the producer) are sending a letter (your message) to someone (the consumer), and you do this by specifying the address (the routing logic for the message, such as the topic on which it is published) and by giving the letter to the local Post office (the message broker). After you deliver the letter, it's no longer your business to ensure that it actually arrives at your friend. The postal service will take care of that.

In a microservice environment, it's actually a really solid solution because it adds a layer of abstraction (the message broker itself) between loosely coupled services and allows for fully asynchronous communications.

Errors will occur. Let's deal with them instead of simply avoiding them

Remember, a cloud environment should be error-resilient and services must fail gracefully and not take down the whole application if one of them dies. With message queues if a microservice dies unexpectedly the broker can still receive incoming messages, store them for later (so that the dead service can recover them when it comes back online) and optionally send back a Last Will and Testament message to the other service saying that the receiver has died. Also, Push/Pull, Publish/Subscribe and Topic queue mechanisms give more flexibility and efficiency when designing inter-service communications, and the ability to send binary payloads allow for easy use of formats like Google ProtoBuff or MessagePack instead of JSON, which are much more efficient in terms of bandwidth usage.

Security is key

Advanced broker servers like RabbitMQ support multiple message protocols (AMQP, MQTT, STOMP...), have flexible authorization mechanism (access control, virtual hosts and even third party/custom auth services via many different transports such as HTTP or AMQP) and will basically remove the burden of orchestrating requests authorization from your service. Plug a custom microservice into Rabbit's auth backend and code your policies in there. The rest will be taken care of by the broker.

Infrastructure at scale

Using message queues with brokers like Rabbit helps a lot with scalability too, which is another crucial aspect of microservices.
If one of the services is struggling under load we want to be able to spin up more instances quickly and without having to reconfigure stuff around. With HTTP as our communication method we would normally have a self-registering Service Discovery Server (like Netflix Eureka or a container orchestration system like Kubernetes or Rancher) and some kind of load balancer integrated with it so that our traffic is distributed and routed to the various instances. With message queues, our broker IS the load balancer. If multiple consumers are listening to a topic at the same time messages will be delivered following a configured strategy (more on RabbitMQ's QoS here).

So, for example, we can have four instances of our service processing messages in a round robin fashion, but since the QoS parameter is customizable at runtime we could also switch to a fair dispatching strategy if one of the consumers is taking too long to complete its work and we need to balance the load out. Also, note that the client configuration is entirely done at runtime (and so is exchanges, queues and topics declarations by the way) so there is no need to tinker with the broker's configuration or restart anything.

Async == Efficiency

We said at the beginning that one of the key features of messages compared to HTTP requests is that they allow for fully async communications. If I need to make a request to another service I can send a message on it's topic and then move on with my work. I can deal with the response when it comes in on my own topic, without having to wait for it. Correlation IDs are used to keep track of which response message refers to which request. This works in concert with what we said about resiliency.
In an HTTP scenario, if I make a request and the callee is down, the connection would be refused and I would have to try again and again until it comes back up.
With message systems, I can send my request and forget about it. If the callee cannot receive it the broker will keep the message in the queue, and then deliver it when the consumer connects back. My response will come back when it can and I won't have to block or wait for it.

In conclusion, messages are great. Show them some love

This is by no mean a rant against HTTP (which is still great, especially for public front facing APIs) but rather a try to guide you towards a more efficient and proficient strategy to coordinate microservice communications and all of this can be achieved with open source software, open protocols such as AMQP that prevent vendor lock-in and with a low (and scalable) infrastructure cost (see RabbitMQ's requirements).

I used RabbitMQ for all my microservices project and it has been a great experience. Let me know in the comments what are your thoughts! Have you used message queues before? What is your experience with it?

Moving to OpenShift!

Matteo Joliveau — Tue, 29 Aug 2017 21:57:06 +0000

I finally made it!

I dreamed of installing OpenShift Origin on my server since a collegue showed me our installation at work. The idea of having a way to automate builds, deploys, orchestration and scaling of the applications that I develop was just amazing for me.

BUT , big problems arise.

The old server

A couple of friends and I one day rented a server together. Tired of having small, crappy VPSs we rented a massive dedicated server from Hetzner, something like a quadcore Xeon processor and 32 GB of RAM. It was all good and wonderful, we ran a Docker engine to run our applications, databases and whatnot inside containers and an NGINX reverse proxy in front of them to route the traffic coming from the outside world.

Since OpenShift nearly cannibalize a server when it's installed, since it wants to handle all the traffic coming from ports 80 and 443 with it's own internal proxy, it was out of question to uninstall everything and migrate to the Origin platform. End of story.

Then we realized we needed nearly half of this compute power and paying 40€ a month for the dedicated server seemed a bit too much for a few guys playing the sysadmin game.

So we decided to migrate our server to a much cheaper VPS from Contabo (still a very resonable machine, with 6 cores, 24 GB of RAM and a whopping 600 GB of SSD storage) AAAAAAND this allowed me to install my so long awaited OpenShift platform :D

It was an excruciating pain to set up, let me tell you. First the Ansible installer didn't manage to complete its job, but since it seemed to be a bug in the installer itself using a newer patch seemed to fix it.

Then the integrated Docker registry and HAProxy router couldn't deploy because I didn't labeled the node (aka the master aka the only server in my cluster) as "infra", or "infrastructure".

Then the Docker registry couldn't write to its internal directory, mounted as a NFS share, because of some wrong POSIX permissions.

BUT NOW EVERYTHING IS COMPLETED

If you are reading this post in the recent future, and we didn't already migrate yet another time, then this very blog is actually running as an OpenShift service inside a docker container, using yet another service holding a MariaDB engine as the storage backend.

If needed (and it won't be needed) I can scale this container a hundred of times and I will be able to handle the combined traffic of the Washington Post and Wikipedia during peak hours.

Or maybe not, I suppose the VPS would die way before the blog.

But still, you get the idea.

Also I'm running a couple of useful services on the platform, such as a private Gogs repository server and a Sonatype Nexus repository manager for my Maven projects.

My friend has developed some Telegram bots in the past and OpenShift makes him able to simply push new code to Git to have the changes immediately deployed and live. (BTW go check his awsome TagAlertBot out, if you use Telegram it's a huge improvement)

Long Live OpenShift

So I'm pretty happy with the deployment. 99% of the stuff we need runs without an issue, and the 1% that doesn't have some kind of workaround we can try to make it work. So yay, welcome to the PaaS world!

GNU Public License is all but free and you should never use it

Matteo Joliveau — Wed, 31 May 2017 15:17:26 +0000

The Free Software Foundation and its sprout the GNU Project are a core element of the free and open source software movement.

They led to a new era of software development where sharing and improving as a community is more valuable than selling proprietary software (maybe because we stopped selling software altogether and started selling services).

But there is a key thing in the most popular free licence of the world, a key problem that let me question if this license is truly free, or if it brings a distorted concept of freedom.

And it's a problems affecting both version 2.0 and 3.0 of the GNU Public License.

Is a license that sets limits to developers really "free"?

Software freedom as intended by Richard Stallman and the FSF is not real freedom, just like communism is not really about sharing and loving each other.

Communism is more like being at the bottom of the pit, and forcing everyone to stay down there just because you don't wanna be left behind by those who manage to climb.

Software freedom in the stallmanian way of thinking is not about being free, is about not doing anything on your own.

No commercial software, no code that you keep for yourself, no way to use it in a manner that makes you keep it private. Which is good, right?

Well, not really. Not at all.

Free not to be free

A key point of this whole text is about the definition of freedom. What follows is MY POINT OF VIEW and does not reflect any "official" definition of freedom.

In my vision of software freedom, free software means that I'm allowed to use it in my daily work, to make use of libraries and programs inside my projects. I mean, that is the whole reason I do open source in the first place. When I started the PlugFace Framework I decided to release it under the MIT License because I thought that it could be useful to other people in need of using plugins inside their software. I thought of other developers in my same situation, in need of modularizing their monolitic application at work with a simple and robust solution. I don't care if you want to use it in proprietary software, on the contrary I would be flattered by this.

GPL wouldn't allow that to happen.

A PlugFace Framework released under GPL would have never been used in enterprise environment because it would require to the user to release their software under the same license. Even with LGPL, which allows software linking to proprietary software, would not have been ideal since any customization (i.e. proprietary security features to validate plugins?) could not have been kept private.

So if I'm developing a library, why should I want to use a GPL-like license?

Why should I choose a free license in order to limit other developers' freedom?

But Matt, without a strong copyleft license $InsertNefariousSoftwareCompanyHere could steal your work and sell it with their name on it, and you wouldn't see a dime of it.

I can hear you reply like that already.

And my answer would be: THAT'S EXACTLY WHAT I WANT.

If you are reading this and you are a software company, please TAKE MY CHILD SOFTWARE AND DO WHAT YOU WISH WITH IT.

If my goal was to make money I wouldn't be doing open source, I would try with something easier. Like forex trading, ice-cream making or underground drug dealing.

Instead, I'm doing open source software because I want people to use it, to solve problems with it. I imagine professional software developers like myself being stuck with a problem, finding out my GitHub account and thinking "Well damn, this guy has made a library that exactly do what I need".

And it would download it, and link it into his software and proceding working, and I would send him a spiritual "You're welcome buddy" through the Force.

That's why, if you write software libraries, you should NEVER EVER USE GPL LICENSES. The best enterprise software are released under weak copyleft licenses, like Kubernetes, Docker, Google Guava, FreeBSD and many others.

cLinux doesn't count. Linux is a final product, not a middleware. But still, remember that without weak licensing and with Linux as the only open source OS, products like the PlayStation 4 and the Nintendo Switch would be very very different.

Choose another license, please

There are many many great licenses out there that are truly free. They will make Stallman angry (and that's probably something that I would truly enjoy), but they will also make many fellow developers happy and thankful.

My favourite ones are the the Apache License 2.0 and MIT License. I release all of my works under those two licenses (and also the MPL 2.0 for complete products, aka not libraries). If you want a complete list, check out this awesome site from GitHub and look for licenses that does not feature the Same license tag in the Conditions column. They are the weak-copyleft ones.

That's all for now. Thank you all for your kind attention, and happy coding! <3

From product makers to service providers. Thoughts on the evolution of software development

Matteo Joliveau — Fri, 31 Mar 2017 13:14:25 +0000

We used to ship products.

We were craftsmen, artisans as Laravel calls us.

We spent months planning a project, writing the code and polishing rough edges (well, not everyone always did this part, to be honest...) before releasing it, so that our users don't find any problem.

When users wanted to use our software, they had to buy it, and then we shipped it to them either as a download or on a physical media.

They owned the software , and even if they were not allowed to do everything they wanted with it, they didn't have to talk to us longer that needed.

This is no longer the case.

Gone are the days of software as products, gone are the days of downloads and installations and updates and...

Well, ok not yet. But I can see it coming. The Software as a Service paradigm is starting to take root in Software Land and users are already accostumed to it. Take Google's cloud services like Gmail or Inbox, Google Docs and Drive.

Take YouTube, or Skype Web.

Even take the whole concept of smartphones, the most common and diffused personal devices of the entire computer history.

People are starting to see the Cloud as a growing and powerful computing platform, and I'm not only talking about students writing notes on Google Docs. I'm talking about big companies moving away their local infrastructure and start using IaaS like Amazon AWS or MS Azure, and developers like me who don't need to send their software to a sysadmin with a full deploying manual to see their applications running.

For instance, at my current workplace we use a private instance of Red Hat OpenShift to deploy dev versions of our solutions. It's easy to use and does a great job at keeping the complexity away from developers so that they can focus on coding full speed ahead.

We no longer have to buy machines to run our business^[1]. We just use a web browser and deploy our software.

You can hide all of the hardware-related issues and keep your dev team up and running working on new features without sacrificing the user experience.

And it's even better on the user side: hardware cost is going down.

There is no more need for powerful computers or portable devices, cheap, high energy efficient hardware can run all the modern computing tasks leveraging the power of a remote machine.

Some took this idea and brought it at an insane level, with cloud IDEs and developing tools like Eclipse Che, Codenvy(which uses the aforementioned Eclipse) and Cloud9, which let you code and develop software as you were on your computer, but from a web browser.

Chromebooks are a great example of this shift in mentality. Laptops that exist with the sole purpose of running the Chrome web browser and entirely rely on the power of the cloud to provide mainstream users with the light productivity and content consumption environment they need, sold at low cost and with very low specs.

They are the quintessence of the as a Service philosophy, of the distributed world we are building right now in the software industry.

It's easier and more secure

When a new version or patch of a software came out, we had to inform our customers that they could update their apps. They didn't always listen. Even with some forms of auto-update, security risks still existed and legacy versions maintenance still had to be done. LTS, Release Candidates, patch, hotfixes, too much complicated stuff to handle for a poor user that just wanted to write a spreadsheet.

With a cloud-based approach , by distributing your application as a Software as a Service, you move all of that burden away from the user's machine. Security updates and new versions can be deployed directly on your infrastructure and be immediately available to your users. Forget about updater and download links, users will have the latest versions without even noticing it.

But there are drawbacks too

Of course nothing is perfect in our world, and so it is in the software world too.

By moving all the computation to a cloud infrastructure you are increasing the computational needs on your side. Higher needs means higher costs and a more complicated backends, possibly fault tolerant and redundant so that your customers don't have to experience any downtime or data loss.

There is still a lot to do

We'll get there someday. A world of shared, distributed computing with close-to-zero complexity for the end user, high abstraction for the developers and easy, standard platform maintenance for sysadmins.

We're not quite there yet, but with the right effort Service Land can become a reality in no time.

Well, we still had to rent a server on which to run OpenShift Origin. But if we used the commercial version, it would run on Red Hat's infrastructure. ↩︎