DEV Community: Volkan Alkılıç

Benchmarking MQTT Brokers with ThingsOn MQTT Bench

Volkan Alkılıç — Fri, 17 Feb 2023 08:33:07 +0000

Are you looking to evaluate the performance of your MQTT broker? Do you want to know the maximum number of messages that can be sent by the broker in a specific amount of time? If so, ThingsOn MQTT Bench might be the tool you need!

In this article, I will introduce ThingsOn MQTT Bench, a benchmark tool for MQTT brokers. I will explain why I created the tool and how it works.

Github repo: https://github.com/volkanalkilic/ThingsOn.MQTT.Bench

Why I Created ThingsOn MQTT Bench

As an IoT and messaging systems developer, I struggled to find a benchmark tool that met my needs when testing the performance of MQTT brokers for my IIoT Platform. Most tools I found were either too complicated or too basic, and lacked the level of customization and control I desired. This motivated me to create my own benchmark tool that is simple, lightweight, and flexible, allowing for easy adaptation to various scenarios and requirements. The end result is ThingsOn MQTT Bench.

How ThingsOn MQTT Bench Works

ThingsOn MQTT Bench is a command-line tool that measures the maximum number of messages that can be sent by an MQTT broker in a specific amount of time. The tool is built on top of the MQTTnet library, a high-performance .NET library for MQTT communication.

The heart of ThingsOn MQTT Bench is a simple C# code that sends a large number of messages to an MQTT broker and measures the time it takes to send them. The code creates a specified number of MQTT clients and generates a specified number of messages for each client.

The code then connects the clients to the broker, sends the messages, and disconnects the clients from the broker. While sending the messages, the code measures the time it takes to send each message and reports the progress in real-time.

After the messages are sent, the code calculates the number of messages sent, the total time taken to send the messages, the message throughput, the connect time, the disconnect time, the success rate, and the loss rate.

Here is a summary of the C# code used in ThingsOn MQTT Bench to measure the maximum number of messages that an MQTT broker can handle:

// Create MQTT clients
var clients = new List<IMqttClient>();
for (var i = 0; i < clientCount; i++) {
    var client = factory.CreateMqttClient();
    clients.Add(client);
}

// Connect to MQTT broker and measure connect time
var stopwatchConnect = new Stopwatch();
stopwatchConnect.Start();
var connectTasks = clients.Select(c => c.ConnectAsync(mqttOptions)).ToList();
await Task.WhenAll(connectTasks);
stopwatchConnect.Stop();
var elapsedConnect = stopwatchConnect.Elapsed;

// Generate messages
var messages = new List<MqttApplicationMessage>();
for (int i = 0; i < messageCount; i++) {
    var message = new MqttApplicationMessageBuilder()
        .WithTopic($"test/{i}")
        .WithPayload(new byte[messageSize])
        .WithQualityOfServiceLevel((MqttQualityOfServiceLevel) qos)
        .WithRetainFlag(retain)
        .Build();
       messages.Add(message);
}

After the clients are connected to the broker, the application generates messages that will be sent by each client during the benchmark. The messages are created using the MqttApplicationMessageBuilder class from the MQTTnet library. The builder allows us to specify the topic, payload, quality of service level, and retain flag of each message.

In this implementation, we create a list of messages with the number of messages specified in the configuration file. The payload of each message is a byte array with the size specified in the configuration file.

The next step is to send the messages to the MQTT broker. We use the PublishAsync method of the MQTTnet library to send each message to the broker. We create a list of tasks to send the messages using the SendMessagesAsync method:

// Send messages
    private var sendTasks = clients.Select(client => Task.Run(async () =>
    {
        var results = new List<MqttClientPublishResult>();
        var progressReport = 0;
        foreach (var message in messages)
        {
            var result = await client.PublishAsync(message);
            results.Add(result);
            var newProgressReport = results.Count * 100 / messages.Count;
            if (newProgressReport > progressReport)
            {
                progressReport = newProgressReport;
                Console.WriteLine($"Progress: {progressReport}%");
            }
        }

        return results;
    }})).ToList();

The SendMessagesAsync method sends a list of messages to an MQTT client and returns a list of results indicating whether each message was successfully sent.

We create a list of tasks to send messages for each client using the Task.Run method. The tasks are created using the SendMessagesAsync method, which sends a list of messages to an MQTT client.

In the SendMessagesAsync method, we iterate over the list of messages and call the PublishAsync method for each message. The method returns a MqttClientPublishResult that indicates whether the message was successfully sent to the MQTT broker.

We keep track of the progress of the benchmark by calculating the percentage of messages that have been sent and printing it to the console. We use the progressReport variable to keep track of the last reported progress percentage and only print a new progress message if the progress percentage has changed:

if (newProgressReport > progressReport)
{
    progressReport = newProgressReport;
    Console.WriteLine($"Progress: {progressReport}%");
}

Once all messages have been sent, the benchmark is complete, and we can calculate the results. We do this by measuring the time it took to send all the messages, the time it took to connect to and disconnect from the MQTT broker, and the number of messages that were successfully sent. We print the results to the console using the following code:

stopwatch.Stop();
var elapsed = stopwatch.Elapsed;
var messagesSent = clientCount * messageCount;
var throughput = messagesSent / elapsed.TotalSeconds;
var lossRate = (double) sendResults.SelectMany(r => r).Count(r => r.ReasonCode != MqttClientPublishReasonCode.Success) / messagesSent;
var successRate = 1.0 - lossRate;// Disconnect from MQTT broker and measure disconnect time
var stopwatchDisconnect = new Stopwatch();
stopwatchDisconnect.Start();var disconnectTasks = clients.Select(c => c.DisconnectAsync()).ToList();
await Task.WhenAll(disconnectTasks);stopwatchDisconnect.Stop();
var elapsedDisconnect = stopwatchDisconnect.Elapsed;

// Print results
Console.WriteLine($"Sent {messagesSent:N0} messages in {elapsed.TotalSeconds:F5} seconds. Throughput: {throughput:N1} messages/second.");
Console.WriteLine($"Connect time: {elapsedConnect.TotalSeconds:F5} seconds. Disconnect time: {elapsedDisconnect.TotalSeconds:F5} seconds.");
Console.WriteLine($"Success rate: {successRate:F3}. Loss rate: {lossRate:F3}.");

In the output, we print the following information:

Number of messages sent
Total time taken to send messages
Message throughput (messages per second)
Connect time (time taken to connect to the MQTT broker)
Disconnect time (time taken to disconnect from the MQTT broker)
Success rate (percentage of messages that were successfully sent)
Loss rate (percentage of messages that were not successfully sent)

To generate the messages that will be sent to the MQTT broker, we create a list of MqttApplicationMessage objects using the specified messageCount and messageSize. Each message is given a unique topic that includes the message index (e.g. “test/0”, “test/1”, etc.), and the payload is a byte array of the specified size filled with zeros.

Once the messages have been generated, we create a list of tasks to send the messages using the SendMessagesAsync method. Each task is responsible for sending messages for a single client. The SendMessagesAsync method iterates over the messages and publishes each one using the client’s PublishAsync method. After all messages have been sent, the task returns a list of MqttClientPublishResult objects, which contains information about the publication of each message.

After all send tasks have completed, we calculate the benchmark results and print them to the console using the information that was collected during the benchmark.

Features

Measures the maximum number of messages that can be sent to an MQTT broker in a specified amount of time.
Supports MQTT brokers running MQTT protocol versions 3.1.1 (MQTTv3.1.1) and 5.0 (MQTTv5.0).
Allows customization of the following benchmark settings:
Number of MQTT clients to use for benchmarking
Number of messages to send per client
Message size in bytes
Quality of Service (QoS) level for messages (0, 1, or 2)
Whether messages should be retained by the broker
Outputs progress information to the console during benchmarking.
Outputs the following information upon benchmark completion:
Number of messages sent
Total time taken to send messages
Message throughput (messages per second)
Time taken to connect to the MQTT broker
Time taken to disconnect from the MQTT broker
Success rate (percentage of messages that were successfully sent)
Loss rate (percentage of messages that were not successfully sent)

How to Use

To use ThingsOn MQTT Bench, first make sure that you have .NET 7 installed on your system. Then, download the latest release of ThingsOn MQTT Bench for your operating system from the releases page.

Once you have downloaded the tool, you can run it from the command line by navigating to the directory where the tool is located and running the following command:

dotnet ThingsOn.MQTT.Bench.dll

By default, the tool will read its settings from a TOML file named config.toml in the same directory as the tool. You can customize the settings by editing this file.

Contributing

Contributions to ThingsOn MQTT Bench are welcome! If you find a bug or would like to suggest a new feature, please open an issue on the GitHub repository.

If you would like to contribute code to ThingsOn MQTT Bench, please fork the repository and submit a pull request.

ThingsOn MQTT Bench is a powerful and customizable benchmark tool for MQTT brokers. It provides real-time progress and comprehensive performance metrics that give you a complete view of the performance of your MQTT broker. If you need to evaluate the performance of an MQTT broker, give ThingsOn MQTT Bench a try!

MQTT File Uploader: Automate Your File Transfers with MQTT and .NET

Volkan Alkılıç — Mon, 13 Feb 2023 07:39:35 +0000

Motivation

As an IoT developer, you may need to transfer files from your devices or servers to a remote location for analysis, storage, or sharing. One popular protocol for this purpose is the Message Queuing Telemetry Transport (MQTT), which is lightweight, efficient, and reliable. However, MQTT is primarily designed for messaging and may not provide native support for file transfer. Therefore, you need to develop a custom application that converts your files into MQTT messages and publishes them to a broker.

To help you with this task, I have created the MQTT File Uploader, a .NET Core console application that monitors a list of directories and uploads any created, changed, or deleted files to an MQTT broker. The program supports various configuration options, such as the file types, the broker settings, the compression, and the encryption.

Source Code

You can access code from Github Repository

Source Code

Features

Here are the main features of the MQTT File Uploader:

Supports multiple directories to monitor.
Supports multiple file types to upload.
Supports optional compression of the file contents.
Supports optional encryption of the MQTT communication.
Supports various MQTT brokers, such as Mosquitto, HiveMQ, and AWS IoT.
Supports various MQTT protocol versions, such as 3.1.1, 5.0, and 5.0 enhanced authentication.
Supports various event types, such as created, changed, and deleted files.

Use Case

The MQTT File Uploader is useful in many scenarios, such as:

Uploading sensor data from embedded devices to cloud services for analysis and visualization.
Uploading log files from servers to remote storages for backup and sharing.
Uploading media files from cameras to multimedia servers for streaming and processing.
Uploading documents from scanners to document management systems for archiving and indexing.
Uploading firmware files from developers to OTA servers for updates and testing.

Usage

Here are the steps to use the MQTT File Uploader:

Install .NET Core Runtime or SDK on your device or server.
Download the MQTT File Uploader from the GitHub repository or build it from the source code.
Edit configuration file in TOML format and specify the directories to monitor, the file types to upload, the broker settings, and other options.
Run the MQTT File Uploader.
Create, change, or delete files in the monitored directories and observe the upload status in the console.

Here is an example of a configuration file for the MQTT File Uploader:

directoryPaths = ["/path/to/directory1", "/path/to/directory2"]
includeSubdirectories = true
fileTypes = ["txt", "csv", "pdf"]
brokerHostname = "example.com"
brokerPort = 8883
brokerUsername = "username"
brokerPassword = "password"
topic = "uploads"
protocolVersion = 5
compress = true
sslEnabled = true
sslCertificatePath = "/path/to/certificate.pem"
createdEventEnabled = true
changedEventEnabled = true
deletedEventEnabled = true

Conclusion

The MQTT File Uploader is open-source software released under the MIT License. You can download it, use it, modify it, and distribute it freely, as long as you preserve the copyright notice and the license terms. You can also contribute to the MQTT File Uploader by reporting issues, suggesting features, submitting pull requests, or giving feedback. Your contribution can help improve the MQTT File Uploader and make it more useful for the IoT community.

Maximizing PostgreSQL Performance: Advanced Features and Techniques

Volkan Alkılıç — Thu, 26 Jan 2023 16:30:56 +0000

PostgreSQL is a powerful open-source relational database management system that is widely used for a variety of applications. It is known for its stability, performance, and scalability. However, many users may not be aware of the full range of performance-related features that PostgreSQL has to offer. In this article, we will explore some of the advanced features and techniques that can help you unlock the hidden performance potential of your PostgreSQL database.

Parallel

PostgreSQL can divide a query into multiple parallel tasks, which allows it to use multiple CPU cores to process the query. This can greatly speed up the execution of complex queries.

SET max_parallel_workers_per_gather = 4;

Partitioning:

PostgreSQL allows you to divide a table into smaller, more manageable chunks called partitions. This can greatly improve query performance by reducing the amount of data that needs to be scanned.

CREATE TABLE sales ( sales_id SERIAL PRIMARY KEY, sale_date DATE NOT NULL, customer_id INTEGER NOT NULL, amount NUMERIC(10,2) NOT NULL ) PARTITION BY RANGE (sale_date);

Index-Only Scans:

PostgreSQL can use a technique called an index-only scan to retrieve data from an index without ever having to access the table itself. This can greatly improve query performance, especially for large tables.

CREATE INDEX sales_amount_idx ON sales (amount);

Materialized Views:

PostgreSQL allows you to create a materialized view, which is a pre-computed table based on the result of a SELECT statement. This can greatly improve query performance for frequently-run queries.

CREATE MATERIALIZED VIEW sales_summary AS SELECT customer_id, SUM(amount) AS total_sales FROM sales GROUP BY customer_id;

Caching:

PostgreSQL can cache frequently-used data in memory, which can greatly improve query performance for frequently-run queries.

SHOW shared_buffers;

Explain Analyze:

PostgreSQL provides a way to analyze the execution plan of a query, which can be used to identify performance bottlenecks and optimize the query.

Multi-Version Concurrency Control (MVCC):

PostgreSQL uses MVCC to provide concurrent access to the data without the need for locks. This can greatly improve the scalability of the system.

Hot Standby:

PostgreSQL allows you to create a hot standby replica of a primary server, which can be used for high availability and disaster recovery.

Statistics and VACUUM:

PostgreSQL keeps track of statistics about the distribution of data in tables and indexes, which the query planner uses to create efficient execution plans. It also provides the VACUUM command which can reclaim space and improve performance by removing dead rows.

TimescaleDB Extension:

One feature that can greatly improve the performance of time-series data in PostgreSQL is the use of TimescaleDB. TimescaleDB is an open-source time-series database that is built on top of PostgreSQL. It provides a number of performance-related features specifically designed for handling time-series data, such as automatic partitioning and indexing, which can greatly improve query performance.

All these performance optimization techniques can be combined to achieve the best performance for your specific use case. It is important to monitor the performance and fine-tune the settings as needed. Additionally, it is always a good practice to test your performance optimization before deploying them to production.

Join Me

Follow me on Twitter and Linkedin for more content.

Using Named Pipes in C# for Interprocess Communication

Volkan Alkılıç — Sat, 07 Jan 2023 18:10:51 +0000

Named pipes are a type of interprocess communication (IPC) mechanism that allow data to be exchanged between two or more processes on a computer. They are called "named" pipes because they are identified by a unique name, rather than by a file descriptor like other types of pipes in Unix-like systems.

Named pipes are useful for a variety of purposes, including:

Enabling processes that run on different computers to communicate with each other over a network
Allowing processes to communicate with each other even if they don't have permission to access each other's memory or files
Allowing multiple processes to read and write to the same pipe concurrently
In C#, named pipes can be created and used through the System.IO.Pipes namespace. To create a named pipe, you can use the NamedPipeServerStream or NamedPipeClientStream class, depending on whether you want to create a server or client pipe.

Here is an example of how to create a named pipe server in C#:

using (var server = new NamedPipeServerStream("my_pipe_name"))
{
    Console.WriteLine("Waiting for client connection...");
    server.WaitForConnection();
    Console.WriteLine("Client connected.");

    // Read and write data through the pipe
}

And here is an example of how to create a named pipe client in C#:

using (var client = new NamedPipeClientStream(".", "my_pipe_name", PipeDirection.InOut))
{
    Console.WriteLine("Connecting to server...");
    client.Connect();
    Console.WriteLine("Connected to server.");

    // Read and write data through the pipe
}

Once a named pipe has been created, you can use the Read() and Write() methods of the NamedPipeServerStream or NamedPipeClientStream class to read and write data to the pipe.

Join Me

Follow me on Twitter and Linkedin for more content.

A small benchmark to compare concurrent data structures in C#

Volkan Alkılıç — Fri, 30 Dec 2022 09:04:37 +0000

The advantage of using concurrent data structures is that they allow you to write highly concurrent code that can take advantage of multiple CPU cores and run faster on multi-core systems. By using these data structures, you can avoid the overhead and complexity of manually implementing synchronization in your code, which can be error-prone and lead to race conditions and other concurrency issues.

    static void Main(string[] args)
    {
        // Set the number of iterations and threads to use in the benchmark
        const int numIterations = 1000000;
        const int numThreads = 8;

        // Create a stopwatch to measure the time it takes to complete the benchmarks
        Stopwatch stopwatch = new Stopwatch();

        // Benchmark the concurrent queue
        stopwatch.Start();
        ConcurrentQueue<int> queue = new ConcurrentQueue<int>();
        Parallel.For(0, numThreads, i =>
        {
            for (int j = 0; j < numIterations; j++)
            {
                queue.Enqueue(j);
                queue.TryDequeue(out _);
            }
        });
        stopwatch.Stop();
        Console.WriteLine($"ConcurrentQueue: {stopwatch.ElapsedMilliseconds} ms");

        // Benchmark the concurrent stack
        stopwatch.Reset();
        stopwatch.Start();
        ConcurrentStack<int> stack = new ConcurrentStack<int>();
        Parallel.For(0, numThreads, i =>
        {
            for (int j = 0; j < numIterations; j++)
            {
                stack.Push(j);
                stack.TryPop(out _);
            }
        });
        stopwatch.Stop();
        Console.WriteLine($"ConcurrentStack: {stopwatch.ElapsedMilliseconds} ms");

        // Benchmark the concurrent dictionary
        stopwatch.Reset();
        stopwatch.Start();
        ConcurrentDictionary<int, int> dictionary = new ConcurrentDictionary<int, int>();
        Parallel.For(0, numThreads, i =>
        {
            for (int j = 0; j < numIterations; j++)
            {
                dictionary.TryAdd(j, j);
                dictionary.TryRemove(j, out _);
            }
        });
        stopwatch.Stop();
        Console.WriteLine($"ConcurrentDictionary: {stopwatch.ElapsedMilliseconds} ms");

        Console.ReadKey();
    }

Join Me

Follow me on Twitter and Linkedin for more content.

Working with Memory Efficiently in C#: An Introduction to Span

Volkan Alkılıç — Sat, 17 Dec 2022 20:09:07 +0000

C# developers are often faced with the challenge of working with large arrays and data structures in a memory-efficient way. One tool that can help with this task is the Span type, which was introduced in C# 7.2. In this article, we'll take a look at what Span is and how it can be used to improve the performance of your C# code.

What is Span?

Span is a value type that represents a contiguous region of memory. It is similar to an array in many ways, but there are a few key differences.

One of the main benefits of using Span is that it allows you to avoid creating unnecessary copies of data. When you pass an array to a method, for example, a new copy of the array is created on the heap. This can be inefficient, especially when working with large arrays. With Span, you can pass a reference to the original data, rather than a copy, which can improve performance.

How to Use Span

Using Span is straightforward. Here is an example of how to use it to reverse the elements in an array:

using System;
using System.Diagnostics;

namespace ConsoleApp
{
    class Program
    {
        static void Main(string[] args)
        {
            int[] numbers = { 1, 2, 3, 4, 5 };

            Reverse(numbers);

            foreach (int number in numbers)
            {
                Console.WriteLine(number);
            }
        }

        static void Reverse(Span<int> span)
        {
            int length = span.Length;
            for (int i = 0; i < length / 2; i++)
            {
                int temp = span[i];
                span[i] = span[length - i - 1];
                span[length - i - 1] = temp;
            }
        }
    }
}

In this example, the Reverse method takes a Span as its argument. The method then uses a loop to reverse the elements in the span. Since the Span type is a reference type, any changes made to the span in the method are reflected in the original array.

Performance Benefits of Using Span

There are several benefits to using Span:

Reduced memory allocations:

Because Span does not allocate its own memory, it can help reduce the number of memory allocations in your program. This can be especially beneficial in high-performance or resource-constrained scenarios.

Improved performance:

Because Span allows you to operate directly on memory without the overhead of additional allocations or object references, it can often lead to improved performance compared to using Array or string.

Greater flexibility:

Span can be used to represent any contiguous region of memory, whether it is the contents of an array, a portion of a string, or memory allocated on the stack. This makes it a versatile tool for working with memory in a variety of scenarios.

Easier to use with APIs that accept raw pointers:

Many operating system and platform APIs accept pointers as arguments, and Span provides an easy way to work with these APIs in a type-safe and memory-safe way.

In summary, using Span can help improve the performance and flexibility of your code by allowing you to work with contiguous regions of memory more efficiently.

Join Me

Follow me on Twitter and Linkedin for more content.

Hidden Gems of C#: Exploring lesser known C# Language Features Part I

Volkan Alkılıç — Sat, 17 Dec 2022 19:03:30 +0000

C# is a powerful and popular programming language, with a rich set of features that allow developers to build a wide range of applications. However, like any language, C# has a number of lesser-known features that can be useful in certain situations. In this article, we'll explore some of these hidden gems and see how they can be used in practice.

Volatile

The volatile keyword is used to indicate that a field can be modified by multiple threads simultaneously. When a field is marked as volatile, the compiler and runtime will take extra steps to ensure that its value is always up-to-date, even in the presence of concurrent access.

Here's an example of how to use volatile:

public class Counter
{
    private volatile int _count;

    public int Count
    {
        get { return _count; }
        set { _count = value; }
    }
}

extern alias

The extern alias keyword is used to specify an alias for a namespace when using a type from another assembly. This can be useful when you have multiple assemblies that contain types with the same names, and you want to disambiguate them.

Here's an example of how to use extern alias:

extern alias MyLibrary;

using System;
using MyLibrary::MyNamespace;

public class Program
{
    static void Main()
    {
        Console.WriteLine(typeof(MyNamespace.MyType));
    }
}

__arglist

The __arglist keyword is used to specify a variable number of arguments in a method declaration. It allows you to pass a variable number of arguments to a method, without using the params keyword.

Here's an example of how to use __arglist:

public static void PrintNumbers(__arglist)
{
    var args = new ArgIterator(__arglist);
    while (args.GetRemainingCount() > 0)
    {
        Console.WriteLine(__refvalue(args.GetNextArg(), int));
    }
}

makeref and reftype

The __makeref and __reftype keywords are used to work with managed object references in an unsafe context. The __makeref keyword creates a reference to an object, while the __reftype keyword returns the type of an object that is being referenced.

Here's an example of how to use __makeref and __reftype:


unsafe void ModifyString(string s)
{
    TypedReference tr = __makeref(s);
    if (__reftype(tr) == typeof(string))
    {
        // Modify the string through the typed reference
    }
}

as and is keywords

The as and is keywords are used to perform type conversions and type checks, respectively. The as operator performs a type conversion, and returns null if the conversion fails. The is operator returns a boolean indicating whether an object is of a specific type.

Here's an example of how to use as and is:

object obj = "hello";
string s = obj as string;
if (s != null)
{
    Console.WriteLine(s.Length); // 5
}

if (obj is string)
{
    Console.WriteLine(((string)obj).Length); // 5
}

Coalesce Nulls

The coalesce nulls operator, represented by the ??= operator, is used to assign a value to a nullable type only if it is currently null. This can be useful when you want to set a default value for a nullable type, without overwriting any existing values.

Here's an example of how to use the coalesce nulls operator:

int? x = null;
x ??= 42; // x is now 42
x ??= 24; // x is still 42

where T: new

The where T: new constraint is used to specify that a type parameter must have a public parameterless constructor. This can be useful when you want to ensure that a type can be instantiated using the default constructor.

Here's an example of how to use the where T: new constraint:

public class MyClass<T> where T: new()
{
    public T CreateInstance()
    {
        return new T();
    }
}

global::

The global:: prefix is used to specify that a type or member refers to the global namespace, rather than a namespace with the same name in the current context. This can be useful when you want to disambiguate types or members that have the same name as a namespace.

Here's an example of how to use global::

namespace MyNamespace
{
    class MyClass
    {
        global::System.Int32 x; // refers to the Int32 type in the global namespace
    }
}

Conclusion

In this article, we explored some of the lesser-known features of C#, including volatile, extern alias, __arglist, __makeref and __reftype, as and is, coalesce nulls, where T: new, and global::. While these features may not be used frequently, they can be useful in certain situations and are worth knowing about. Understanding these features can help you write more efficient and effective code in C#, and can also deepen your understanding of the language as a whole.

Join Me

Follow me on Twitter and Linkedin for more content.