DEV Community: Jeferson Moura

LDAP Auth in Buildly Core

Jeferson Moura — Wed, 05 Feb 2020 09:26:10 +0000

Several enterprise companies ask engineers to integrate their application authentication with LDAP (Lightweight Directory Access Protocol) or Active Directory (AD), Microsoft’s proprietary solution for storing user or employee information. The focus of this article is to explain how to integrate Buildly with an LDAP server.

LDAP

LDAP, the Lightweight Directory Access Protocol, is a mature, flexible, and well supported standards-based mechanism for interacting with directory servers. It’s often used for authentication and storing information about users, groups, and applications, but an LDAP directory server is a fairly general-purpose data store and can be used in a wide variety of applications.[1]

LDAP can define permissions, set by the admin to permit just certain individuals to access the LDAP database, and schema, to describe the format and properties of data in the server.

Active Directory

Active Directory is used to store user information. It helps to work with interconnected, complex and different network resources.

A directory service, such as Active Directory Domain Services (AD DS), provides the methods for storing directory data and making this data available to network users and administrators. For example, AD DS stores information about user accounts, such as names, passwords, phone numbers, and so on, and enables other authorized users on the same network to access this information.[2]

Configure Buildly LDAP connection

Before start
In our example, we’re using OpenLDAP as the LDAP server, so we also need an instance of it up and running. There are a couple of tutorials on the internet of how to start a LDAP server, so we’re not going to explain how to do it here but share some links that should help you. You can also use Azure Active Directory.

OpenLDAP

Active Directory

Configure Buildly

First of all, we need to have a Buildly Core instance up and running locally, so we can safely expose our services and use an LDAP server to retrieve users’ data for authentication. To know how to quickly have a local instance of Buildly Core, go through the Quickstart documentation.

Now, we need to configure Buildly Core to connect to our LDAP server, so we have to have the URI, username, password and base domain name for the user search in hands. Buildly Core uses specific environment variables to configure its LDAP connection. These are the environment variables with our LDAP server values:

We can specify each parameter using -e, –env, and/or –env-file flags to set simple (non-array) environment variables when running docker containers but if you are running Buildly Core with docker compose, you can update the docker-compose.yml file with the new environment variables.

Let’s run Buildly Core with the new configuration and test the authentication. But first of all, we are going to list all users from our current Buildly Core instance using superuser powers.

As we can see in the response below, currently, there’s no other user besides the superuser.

We have already added a user called Rick Sanchez to our LDAP server with username rick and password admin. We will send the following request to Buildly Core to authenticate with it:

Buildly Core responded with an access token for our user:

Since this user didn’t exist in the Buildly database before, it should be created now. Let’s check that.

Example of request:

Buildly returns all information about my user:

Now, I will change the first and last name of the user in our LDAP server, then authenticate with it again and check if its name was updated in the Buildly Core database.

This is the data of the user in the LDAP server.

Let’s authenticate to Buildly Core again:

Now, let’s retrieve the user data:

Everything worked and the user data was updated in the Buildly Core database based on the information we have in our LDAP server. Buildly authenticates users against LDAP server, creates and updates users based on the data retrieved from the server.

Conclusion

We connected Buildly Core to an OpenLDAP server in this tutorial but you can connect to Active Directory as well. To connect to an Active Directory Domain Service (AD DS) you would need to update the user search filter in the config, to the AD field that you would define as the username in your AD user data.

Example:
LDAP_USERNAME_FIELD_SEARCH: “sAMAccountName”

Reference

LDAP – https://ldap.com/
Active Directory Domain Service – https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/get-started/virtual-dc/active-directory-domain-services-overview

How to connect IoT devices to Buildly

Jeferson Moura — Tue, 14 Jan 2020 22:52:34 +0000

As we have talked in one of our previous blog posts, most applications on the internet work synchronously but this has been changing in the last few years, especially because of IoT. In this post, we’re going to show you how to create a “micro”service responsible for publishing messages to an MQTT broker and connect it to Buildly Core. This service will receive command specifications from Buildly Core and send it to the broker, so IoT devices can subscribe to them. The main idea is that we have an event producer, a message broker, and a consumer. If you want to know more about this type of architecture, have a look at the Event-driven Architecture post.

Before starting

First of all, we need to have a Buildly Core instance up and running, so we can safely expose our IoT service and possibly connect other services to publish commands to other brokers our trigger different actions, processes, etc. To know how to quickly have an instance of Buildly Core, go through the Quickstart documentation.

In our example, we’re using Mosquitto as the MQTT broker, so we also need an instance of it up and running. There are a couple of tutorials on the internet of how to start a Mosquitto server, so we’re not going to explain how to do it here. Eclipse Mosquitto is an open-source (EPL/EDL licensed) message broker that implements the MQTT protocol versions 5.0, 3.1.1 and 3.1. Mosquitto is lightweight and is suitable for use on all devices from low power single board computers to full servers. [1]

You can find the code of the service we’re going to create in this tutorial on GitHub. https://github.com/Buildly-Marketplace/iot_service

Create the IoT Service

Now, the fun part will start. We need to know which programming languages have an official Mosquitto client, so we can choose which one we’ll pick to build our service. In our case, we chose Python because we have a Buildly CLI tool that generates a simple and configured Django application ready to be used with Buildly Core.

To connect to the Mosquitto server we need to install the Python library called paho-mqtt, so we added it to the application requirements. The Eclipse Paho project provides open-source client implementations of MQTT and MQTT-SN messaging protocols aimed at new, existing, and emerging applications for the Internet of Things (IoT). [2]

Connect to a Mosquitto server

After running the service and verifying that everything is working we need to implement the logic that will connect our IoT service to the broker and publish commands sent to it. We have to define a topic in the broker, so messages(commands) can be published. Therefore, we decided to call our topic: hello/world.

To test our implementation above we need to have a Mosquitto server running and subscribe to the same topic that we’re going to publish messages. Therefore, we added another function to the MQTTClient class to perform the topic subscription.

Now, we have the IoT service and a broker running, we need to test the connection and see if we can send and receive messages via our Mosquitto server.

Implement an endpoint

Last but not least, we need to implement an endpoint to receive command specifications in HTTP requests and send them to the broker. Our implementation will be super simple, so we aren’t going to process the body of incoming requests but just send them directly to the Mosquitto server. But you can add more logic to your service, e.g, structure messages before publishing them, define a list of commands that can be published to the broker, etc.

Connect the new service to Buildly Core

Now that we have our IoT Service connected to a Mosquitto server and ready to be connected to Buildly Core, we’re going to it then. First, we need to have an instance of Buildly Core and our IoT service, then we send the following POST request to Buildly Core with our Admin credentials, so the service is registered to it.

Now, let’s check the Buildly Core Swagger specification, so we can see if the IoT service is accessible via Buildly Core.

Testing everything together

We connected all the pieces and everything is running, so we need to test it now. We are going to send a request to Buildly Core with a command specification to be redirected to our IoT service. Therefore, the Mosquitto server should have the specified command queued. To retrieve the message from the Mosquitto server we are using the Mosquitto Client to subscribe to the hello/world topic.

Awesome! Everything is working and we can send command specifications to Buildly and they will be published to a Mosquitto server.

And now what?

Now, we can create consumers and connect them to the Mosquito server to consume messages(commands), in other words, we can send HTTP requests to Buildly Core specifying which command should be performed by an IoT device and these requests will be redirected to a service responsible for handling and publishing them to a broker using MQTT protocol.

There’s a diagram at the beginning of the post showing a database connected to and an event subscription done by the IoT Service. It means that the IoT Service could also subscribe to the broker to receive information from IoT devices, and use a database to store this kind of data.

Resource

Eclipse Mosquitto — https://mosquitto.org/
Eclipse Paho — https://www.eclipse.org/paho/
Buildly Core Documentation — https://buildly-core.readthedocs.io/en/latest/

Event-driven Architecture Explained

Jeferson Moura — Tue, 17 Sep 2019 17:35:26 +0000

Most applications on the internet work synchronously but this has been changing quickly in the last few years, especially because of IoT, self driving cars, etc. Synchronous applications receive requests to perform an action and the actor waits until everything is done, he/she waits for a response. However, there are situations when we don't need to wait for a response but only send a request. It’s very useful when you just want to inform something or trigger a process and maybe check it later if everything went well.

Concept

Event-Driven Architectures (EDAs) are usually broker-centric, so there’s a producer that generates events/sends messages, a broker that receives the events and redirect to consumers, and a consumer that connects to the broker and waits for events.

Producer & Consumer

As explained in the first sentence above, a producer (aka publisher) is an application that sends messages to the broker and a consumer (aka listener) is an application that connects to the broker, manifests interest in type of messages, and leaves the connection open so the broker can push messages to them.

Message broker

A message broker (aka integration broker) is an application/software that translates a message from the formal messaging protocol of producers to the formal messaging protocol of consumers. There are a couple of famous brokers out there, like Redis, RabbitMQ, Kafka, etc.

Message

As mentioned above, publishers send messages to a broker, and this message will be redirected and received by all the interested consumers. The message payload can be anything but they are frequently cataloged as events and commands. To clarify, events are used to communicate things to consumers and commands are used to ask consumers to do things.

Protocols

As we have HTTP and HTTPS, for example, used by REST APIs, there are also some protocols used by Async APIs. And these are some of them:

AMQP

Advanced Message Queuing Protocol (aka AMQP) is an open standard for passing business messages between applications or organizations. It connects systems, feeds business processes with the information they need and reliably transmits onward the instructions that achieve their goals. [1]

MQTT

MQTT is a machine-to-machine/"Internet of Things" connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium. [2]

Websocket

The WebSocket Protocol enables two-way communication between a client running untrusted code in a controlled environment to a remote host that has opted-in to communications from that code. The security model used for this is the origin-based security model commonly used by web browsers. [3]

Conclusion

Now, we know a bit about event-driven architectures (EDAs) and that they promote the production, detection, consumption of, and reaction to events. A piece of external information about them is that they are also extremely loosely coupled and well distributed.
We also know that there's a big variety of protocols and to decide which one we should use is highly dependent on our specific use case and codebase.

References

1 - https://www.amqp.org/about/what
2 - http://mqtt.org/
3 - https://tools.ietf.org/html/rfc6455

Developing APIs based on OpenAPI Specification

Jeferson Moura — Mon, 16 Sep 2019 09:14:19 +0000

APIs have been around for a really long time connecting different logical units together but they were not meant to be used by a lot of people in the past. Nowadays, APIs are not constrained to the Backend side of applications anymore and are more consumer-driven.
The universe of APIs is regularly one of contending standards, interests, and arrangements. Standards help give a common framework of communication and development and ground us in picking the correct tools dependent on a particular need.

What is OpenAPI?

I really like the explanation from OpenAPI website: "The OpenAPI Specification (OAS) defines a standard, language-agnostic interface to RESTful APIs which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation, or through network traffic inspection.".[1]
Summarizing, OpenAPI Specification is a solution that produces machine-readable documentation for REST APIs.

Developing APIs based on OpenAPI spec

As Test-driven development (TDD) that combines test-first development where you write a test before you write a solution code to fulfill that test and refactor it, we can also specify endpoints of an API using the OpenAPI framework before starting coding them and refactor. So, prior to beginning to build the business logic of an API, test it, or any other lifecycle, we create the API's interface, specifying the accurate requests and responses endpoints will display.

Why OpenAPI spec-driven development?

A significant advantage of developing APIs based on OpenAPI specifications is that it enables independence between teams, e.g, Frontend, Backend and QA Engineers know how the API is supposed to do, so they are all aligned on it. Removing these dependencies will also speed up the release process because different teams can do their tasks at a much faster pace.
Another advantage of using this approach is the improvement of Developer experience. Developer experience is about delivering robust functionality that is stable, speedy, and visually intuitive.[2] If you're thinking about how it can improve the developer experience, you just need to remember that developers will consume your API and having a well defined/documented API will have a great impact on it. You can focus on the API consumer's needs beforehand.

Conclusion

As I said at the beginning APIs were not meant to be used by a lot of people in the past but nowadays, they are not limited to the Backend side of applications anymore and are more consumer-driven. So, we need people to comprehend what APIs do, which is the reason a consumer-driven perspective on APIs is quick getting on and the OpenAPI Specification helps us to build APIs while guaranteeing a good developer experience through a definition driven methodology.

Reference

1 - https://swagger.io/specification/
2 - https://hackernoon.com/developer-experience-dx-devs-are-people-too-6590d6577afe

Microservice Architecture in a Nutshell

Jeferson Moura — Sun, 08 Sep 2019 16:57:28 +0000

We have been building applications and getting better at it for years and a variety of technologies, architectural patterns, and best practices have been developed along this period. Microservices is an architectural pattern that has emerged during those years, especially because of containers. Docker is the main actor of all these changes because it has popularized containers and helped us containerizing our applications.

Concept

Microservice architecture is a variant of the service-oriented architecture that organizes an application as a collection of loosely coupled services. These services are responsible for specific tasks and can communicate with each other through simple APIs to solve a larger complex business issue. Summarizing, microservices are a particular technique for developing software systems that attempt to concentrate on structuring single-function modules with well-defined interfaces and operations.

Pros & Cons

Microservices are used to speed up development, feature release, ease scalability; they offer improved fault isolation, etc. If you are developing an application and it has to support a variety of different clients including desktop, mobile browsers, and native mobile applications, you must consider implementing your application using microservice architecture.

Microservices also bring complexity to network, testing, deploying, monitoring, etc. Team communication can be hard without using automation and Agile methodology. You also might have to introduce CI/CD tool, configuration management, and application-deployment tool, service mesh, and API gateway to help you with testing, building, shipping, discovering and connecting all services.

Conclusion

Along all these years we have improved the way we develop systems and we’ll keep doing it. If you automate and do everything in the right way, you probably won’t face issues and will love microservices. All software architectural patterns and styles have their pros and cons and microservices architecture isn’t different but it helps us to be more agile.