DEV Community: Ron

Brief Introduction to Microservices

Ron — Thu, 29 Apr 2021 07:53:17 +0000

Almost certainly, in this day and age you have heard of Microservices architectures. The term gets thrown around very often when talking about scaling and handling growth within your system.

But what are, really, Microservices?

The fact is, this type of architecture is nowhere near new. Airbnb was building Microservices at scale back in 2012 just to give you an example. Distributed Systems were around many years before that.

Microservices are small services [micro], that have one purpose or serve a specific domain only. By having many of these, you're able to scale horizontally instead of keep growing a big machine.

The idea of Microservices is actually better explained when put against traditional monolith applications, and that's probably the best example out there.

In the earlier days, you'd have a single codebase for your app, using a single programming language, and you'd scale the resources it needed as required given your growth. In hindsight you can clearly see the problem with that, at some point you'll need huge machines to support scale. There were workarounds to manage this issue, but even then you'll hit a cap at some point.

The logical next step was to start separating these applications into different systems that communicate passing messages between each other. These systems could be split by domain for example, say accounting vs inventory. Still, these splits were also somewhat inflexible and would grow in the same way as their strict monolithic peers.

At some point, what Engineers figured out was that making these services smaller you gain flexibility, to the point that when needing to add functionality you could just build a new service, and if needing more power in a particular area of the system, you could spin many of these services to cope with the load or scale them independently.

That was sort of the birth of Microservices.

The end.

...

Jokes, this is when things got exciting!

Now armed with this knowledge, Software Engineers and Architects could build all sorts of different topologies to handle their specific needs.

However, Microservices did come with some challenges of its own.

Communication between services

There are many ways to communicate across services, and that has also evolved over time. The traditional way is to pass messages via HTTP or SOAP in the same fashion other Service-Oriented Architectures did.

Another option is to use shared data stores like databases to write data on one place and read it in another place.

Each of these are not exclusive to Microservices in fact, they were adopted from previous service-oriented iterations. The novelty now is that messages can be smaller and serve a single purpose as well, for example a service could accept only messages related to updating a user's profile information, or making a booking and that decreases complexity overall.

Given the scale at what modern applications grew, even these ways of communication weren't enough, so the current standard of using message queues was born. Companies like Google, Amazon, Facebook built their own implementations of event queues but nowadays we can find more widely adopted options like Kafka or RabbitMQ.

Event-Driven architectures are the modern standard at the time of writing, and they're a whole new level of complexity and awesomeness, which I will write more about in the future.

Hybrid Architectures

The practice of having hybrid architectures, running different services on multiple programming languages and setups is widespread now. You can easily have multiple programming languages running within the same system now that small services are running somewhat independently from each other.

This allows you to maximise the benefits of each technology you pick, as long as you find a good way to communicate between them.

How small do the Microservice needs to be?

This is where it starts getting tricky. There isn't a single rule to define what is Microservice or how micro is micro. There's a guideline that says it needs to be small enough to serve a single purpose and be contained within a single domain but that's vague, probably on purpose.

In practice, when you create Microservices what tends to happen is that you start with something that resembles a Microservice given that definition, but over time the service grows and you need to reevaluate whether it needs to be split as it doesn't serve a single purpose anymore.

You need to find a balance between having too many services that are too small and make it harder to maintain, versus having services too big that are not Microservices anymore and become harder to scale.

You can call it bad practice, bad design, band planning, or whatever, but that's what usually happens, and it's natural to only notice in hindsight. The good news is that if the services are "small enough", splitting or rewriting them is often not much of a problem. Even replacing them completely in a new programming language is also on the table.

Transactions on Distributed Systems

One of the main issues in Microservice architectures is the concept of transactions. If multiple services execute different parts of a transaction, how do you rollback when the process fails half-way?

This is a very complex problem and can be solved in different ways depending on your architecture. Most certainly you are better off using an existing design pattern to solve this problem.

Two common patterns to solve this issue are the 2-phase commit pattern (2pc), which uses an orchestrator service to rollback transactions if something goes wrong. This is a synchronous approach to solving the issue; The other pattern is the Saga pattern, which aims to produce events of the different stages of a transaction and compensating the failed transactions when something goes wrong. This is the asynchronous approach.

It's out of the scope of this article to visit those in detail, but it's a good idea to research these patterns in depth as you are likely to encounter these issues if working on Microservice architectures.

Microservice versioning

Another common problem is having to update the event structure that Microservices use to communicate with each other.

Given that services need to know how to parse a message from other services, when changes in that event are needed you need to find a way to keep the communication working until such services are back in sync.

To solve this issue you probably need to use some form of versioning. In the case you can shutdown the Microservices completely, update them and put them back on, this is not an issue at all.

When that's not possible the next best case scenario is adding new data to an event without affecting the older structure. This way you can upgrade your sender first to add the new field(s) and work on supporting the listeners later on.

If that's not possible either, solving the issue becomes a tad more complicated. A good solution for this is to add versions on your listeners while you transition to the newest events. Then depending on the version of the message received you can parse it one way or another.

There are multiple ways to solve these types of problems, and the solution will be different depending on the message restrictions; For example, having a hard type schema of your event will make it more complex.

Maintenance

Large Microservice architectures are harder to maintain and to update. You can probably Dockerize the instances that are running the services and update all of the machinery at the same time, but if you have multiple services running different programming languages on different versions upgrading at different speeds, then keeping everything up to day becomes a repetitive and time consuming task.

With this in mind, it's important to take as many steps as possible to ensure that domain functionality is not too tangled so services can be upgraded independently from each other as much as possible.

Conclusion

Microservices architectures are a fantastic way to scale your system horizontally and to manage the growth of your business but it doesn't come cheap. There are tradeoffs to be aware of and having a solid understanding of the principles, patterns and benefits is important.

There are also many different topologies to consider, and what works for me may not work for you, so make sure that you understand your business requirements when designing your system.

Another thing to keep in mind is that this is a complex architecture, even though it sounds simple. You don't need to jump on it right away, especially if you're only getting started. This is a complexity trap. If you don't have to handle a high amount of traffic, it's probably best to start with a single monolithic app and slice and dice it as you grow. It will save you time and probably money.

For more topics, check my profile or visit rarias.dev.

Happy ~~coding~~ designing? 🤔

Not writing automated tests? You have a problem waiting to happen

Ron — Thu, 29 Apr 2021 07:51:48 +0000

Unit testing, Integration testing, A/B testing, stress testing... you name it. There are many ways to ensure your software is working correctly in an automated way.

The times of manually checking your functionality "runs fine" are long gone. In any Software development team, the testing part is (or should be) as integral as writing the code itself.

I personally advocate testing so heavily that I reject PRs on the base of missing or having wrongly written tests. There's certain anxiety on the back of my neck when having to deploy code that's not through-fully tested. I just can't help it.

For a better Software development world, you shouldn't tolerate it either.

As applications and teams grow bigger, the needs for automatic testing suites grow with them.

You may say "But Ron, we have a QA team to do the testing for us".

Sure, having a QA team is a nice thing, and surely they will be able to test your features in due time, but the ownership of code should come from development. It is your responsibility as the person who writes the code to ensure it's working as best as possible, and to reduce the load on the QA team.

Additionally, relying on your QA team promotes a "throwing over the wall" behaviour, in which developers don't follow up to make sure their code is production ready the moment it leaves their machines.

The less tests you write, the more potential issues you're introducing. I've learnt this the hard way. It's almost like divine justice being served. Every time I've decided to skip testing certain features, I've introduced an error in production. Fortunately other measures like heavily alerting and small deploys usually had helped me find the issue quickly and roll it back, but it's still painful.

How to best write automated tests

There's this concept of a testing pyramid. It looks something like this:

The idea is that the majority of your tests are going to be unit tests. They're easy to write, they are fast to run and in general they should even help you clarify your functionalities.

Afterwards you have integration tests, which are more complex and make sure the different parts of your system are interacting correctly.

At last you have UI and potentially manual tests.

There are many variations of this pyramid, with more detail or steps in between but the concept is the same. You go from less to more complex tests, moving up the pyramid.

Your application should at the very least have the bottom of the pyramid covered. Unit tests are essential for every system.

If you engage in practices like TDD or BDD, it's even better. The practice of writing your tests first helps you find functionality gaps and guide you through your feature building. You can consider the different scenarios your app needs to handle before you even start writing your first line of code.

Additionally, you should make sure to pick the right testing framework for you. Pick something that makes your life easy. You don't want to be spending all your time figuring out how your tests are being set up, but writing them.

Do you need 100% test coverage?

Simple answer is no. Test coverage is somewhat a useless metric.

There it is, I said it!

More often than not, test coverage can give you a false sense of security and make you sloppy when considering edge case scenarios.

Entertain my idea with the following example

def debit(account, amount)
    account -= amount
end

def credit(account, amount)
    account += amount
end

Two over-simplistic functions to debit and credit a given amount from an account, similar to a banking transaction.

Logically It is expected that you add the amount to the account in the credit scenario, and the opposite on the debit scenario, right?

There are a lot of validations to be performed, like making sure the account is higher than 0 on debit for example.

Now, consider we write the following tests

it "debits the amount" do
    expect(debit(100, 20)).to eq(80)
end

it "credits the amount" do
    expect(credit(100, 20)).to eq(120)
end

Our tests have now 100% coverage!

See the problem here?

Coverage in itself is a horrible metric, it doesn't consider edge cases, just that you've "covered" all parts of the code.

The following logically incorrect situations won't be considered at all:

debit(100, 500) == -400
credit(100, -200) == -100

When considering the use of test coverage, bear in mind that it's not a substitute for logically and through-fully thinking about all your test cases, edge cases and impossibly looking cases.

You can just as easily have an application with merely 70% test coverage but having a strong set of tests that make sure the code is handling all expected and most unexpected scenarios.

Drawbacks of writing tests

Writing tests is time consuming. It doesn't move your application forward like, say, a new feature. When your line manager asks why that feature is not ready yet, and you say you've been spending a lot of time writing tests it's sometimes not pretty.

It's a necessary time consuming task that needs to be included as an essential part of your development process. You and your team need to understand that if writing the code for a feature takes 1hr and your automated tests take 2hrs, that feature took 3hrs until done, not 1hr.

Additionally, test scenarios need to be rewritten every time a feature is updated, and new scenarios need to be added every time a bug that wasn't considered in the tests is discovered. It's an ongoing time consuming process.

That being said, do not think for a second that it's unnecessary.

What test suite should you use?

Use something that makes it extremely easy, even enjoyable to write tests.

Sometimes writing automated tests is dragging and plainly boring. When pressed for time, you will hate having to take care of the tests, but with the correct testing framework, you'll make the process overall better.

What if I'm the only one writing tests on my team?

Hold your teammates accountable, advocate the benefits. Be repetitive and annoying if necessary.

The cost of letting untested issues into production is just too high. Once that broken feature reaches your users, it's going to negatively impact the business. It may even mean losing users altogether.

When it comes to making sure everyone is pulling their weight and making sure all systems are operational, writing tests should be among your top priorities.

An additional benefit is that you avoid regressions when collaborating on features. If a great testing suite is in place, you can guarantee that whatever you do is not going to affect your teammate's previous work if all tests are ok after you're done.

Conclusion

Testing is as important as writing the features itself when it comes to Software development. Do not ignore them, or you have a problem waiting to happen. No matter how good you think your coding skills are, you increase your probabilities of making mistakes if there isn't an automated way to check everything is working correctly.

In big and interconnected systems, the need is even bigger. As your platform grows in size and complexity, you need a mechanism that ensures you can safely build on top of what you already have, and that's automated testing.

For more topics, check my profile or visit rarias.dev.

Happy coding!

Kanban for Software Engineering

Ron — Thu, 04 Mar 2021 11:19:40 +0000

Hi everyone,
I'm in the process of refining our Kanban methodology and I thought I'd try to reach to some of you in the community for help on best practices and what to look for.

My team and I have been using Kanban for our Software development process for about a couple of years now, with some level of success but I'd like to know what's required to take it to the next level.

If you or your team are using Kanban, what worked for you?

PyEnv && Pipenv

Ron — Fri, 26 Feb 2021 17:58:19 +0000

If there's something I've learnt with experience is that there's almost always a simple way to do things. There are millions of people dealing with development problems the same way you are, so chances are someone already stumble with a way to simplify things.

One of those simple things is putting together a Python development environments.

I'm wouldn't say I'm a Python developer. I only use it for a specific subset of cases, mostly requiring large amounts of data for analysis, Machine Learning, etc.

So intuitively, when setting up my development environment I try to make it as easy as possible, and coming from other programming languages and frameworks such as Ruby, Elixir, Node, etc where there's clear separation between version language and libraries almost by default, I'd expect Python to be no different.

Wrong! 😭

Python's setup is a mess. Python 2 is still around and it has no compatibility with Python 3. Python 3 at the same time has so many ways to be setup that it makes it so confusing.

If you pick the wrong way to setup and tie too many things to the system, then you end up with incompatibilities across projects and all other sorts of issues.

However, there's one type of setup that, in my opinion, seems to be the most similar to what I'm used to. And it's using Pyenv along with Pipenv to setup projects individually and isolated within their own Python and library versions.

Let's go ahead and dig deeper into this.

Pyenv

Pyenv is a Python version manager tool. It was based on the popular Ruby version managers rbenv and ruby-build.

Its functionality is quite simple. It lets you have different versions of Python in your environment, which are not dependent on each other or the system.

There are many other tools that sort of do something similar, but I find Pyenv to be the most unintrusive of all. It does not require Python itself as it's built with shell scripts, and it does not need to be loaded into your shell.

One additional thing to remember is that you don't want your system Python running the show. So the more you can stay away from it and use isolated environments the better.

Installation

There are many ways to install Pyenv.

By far the easiest if you're on MacOS or Linux is via Homebrew

❯ brew install pyenv

After this, you need to let your bash initiate pyenv on start. For this you can add the following init script to whatever shell you're using.

In my case for zsh I need to add it to my .zshrc but for you it could be .bash_profile or .bashrc.

if command -v pyenv >/dev/null 2>&1; then
  eval "$(pyenv init -)"
fi

This will allow Pyenv to manipulate your path and give itself priority inside pyenv environments.

And it's done!

For more installation instructions you can check their Github page.

Using Pyenv

Next thing to do is install a new version of Python. You can install as many as you need, even from different distributions.

To see what versions are available, use

❯ pyenv install --list

This will give you a full list of available versions.

Let's go ahead and install Python 3.9.1

❯ pyenv install 3.9.1

Just like that, you're rolling on Python 3.9.1, yay!

Last step is to tell your system the global version you want to use. You don't necessarily have to do this, but it makes things a bit easier down the line

❯ pyenv global 3.9.1

Congratulations, you have Python in your system 🙂

Pipenv

Pipenv works at a lower level than Pyenv, and they can work together very well.

Pipenv is a tool, again sort of inspired in Bundler, Npm, Yarn etc, that aims to keep your project environment nice and tidy.

It might be counter-intuitive and you may be tempted to just install everything at the global scale so it's available for every project you use, but don't! This is an easy way to depend on libraries that may not be available when using somewhere else.

Additionally Pipenv makes collaboration really easy. Stakeholders all use the environment setup in the Pipfile and this guarantees everyone is on the same page.

So what does Pipenv do exactly?

It uses a Pipfile and Pipfile.lock to sort of snapshot the dependencies you use in a project. It allows you to work in an enclosed environment, without the fear of your work slipping out to your other project environments.

Installation

Once you have Pyenv running and are on your preferred version of Python, run

❯ pip install -U pipenv

Pyenv sort of hijacks the pip command, to make sure you're working in the correct environment.

After this you're ready to start using pipenv.

The library is meant to be used within a project or project folder/context, so go ahead and navigate to your preferred project directory and initialise pipenv in there

❯ pipenv install

This will create a Pipfile containing your dependencies, and a Pipfile.lock containing the versions and dependencies of those dependencies.

Traditionally, another way to manage dependencies in Python projects is using a requirements.txt file. If you already have one of those on an existing project, pipenv will transform it to a Pipfile with the previous step and attempt to migrate your existing dependencies to it.

Using Pipenv

From now on, you're good to start using pipenv to manage your dependencies.

To add a new one you can let pipenv know what you want to install and it will update both the Pipfile and the Pipfile.lock for you

# Installing Jupyter notebooks
❯ pipenv install notebook

Just like that your new dependency has been added your your Pipfile and Pipfile.lock.

Now to "enter" the environment that pipenv has created for you, you need to activate a new shell using

❯ pipenv shell
Launching subshell in virtual environment…

This will load your current bash with the dependencies installed within your pipenv environment. Every command that you run within this shell will execute in the context of your Pipenv.lock.

To exit the shell just type exit.

Conclusion

There's so much to learn about Pyenv and Pipenv, that you're better off just following the documentation and trying it yourself, but I guarantee you it will save you countless hours of configuration and figuring out what's going on with your project dependencies and why some projects are failing after updating other unrelated ones.

If you want to know more about the different environment setups that Python has to offer and its differences, there's this really detailed StackOverflow post that explains it very well. Even though their recommendation is using virtualenv + pip instead, it's worth having a look at the different options summarised so you can make an informed decision of what you want.

For more topics, check my profile or visit rarias.dev.

Happy coding!

Learn how to deploy Elixir apps on Heroku

Ron — Fri, 12 Feb 2021 08:45:37 +0000

With every app we decide to take out to the world, there's one important decision to make when the time comes.

Where is the best place to deploy it?

The short answer is: "It depends, probably there's no one best place actually".

The long answer however, will depend on what your needs are, your budget, the setup of your current applications, company restrictions etc.

Particularly, I think a really convenient way to deploy your apps in no time is using Platform as a Service (PaaS) options like Heroku.

Heroku is a platform that allows you to abstract yourself from all the complexity that comes from deploying and maintaining an app, and to focus your efforts on the app you want to build. Here's a video from Heroku's Greg Nokes explaining the concept behind their platform.

Heroku has this idea of buildpacks, which transform your code into a slug that can be run on their platform and abstract all the complexity.

They support a small range of popular programming languages using these buildpacks, however Elixir is not one of them.

But not all hope is lost, we have the ability to use custom buildpacks to support it ourselves. And there is one already available that Akash Manohar and his contributors built for us.

If this is all new for you, don't worry, we're going to start putting up the pieces together in a bit.

Let's build a very small app using Elixir that queries an open API and displays the data via HTTP in a JSON format.

Before you get started, you'll need a Heroku account. You can create a new one for free at heroku.com, and there's a free tier we can use for our development projects.

If you're not too familiar with Elixir, you can check my hello-world-elixir article to give you an overview.

This article will be divided into 5 parts

Creating your Elixir app skeleton.
Writing the basic functionality.
Testing it works locally.
Creating your Heroku app.
Deploying and testing your app.

1. Creating your Elixir app skeleton

There are multiple ways to create an Elixir app. You could use one of the available frameworks or you could potentially create the files yourself one by one for example. In this exercise, we'll use the convenient mix new helper that will create most of the initial structure for us.

Go to your project folder on your shell, then type mix new simple_app --sup

❯ mix new simple_app --sup
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/simple_app.ex
* creating lib/simple_app/application.ex
* creating test
* creating test/test_helper.exs
* creating test/simple_app_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd simple_app
    mix test

Run "mix help" for more commands.

This will create a basic skeleton for your app, including a module called SimpleApp and a basic supervisor.
Make sure to check your version of Elixir. Syntax and structure might differ slightly on different versions. I'm using 1.11.3 for the following exercise.

Let's go ahead and compile our app first to test the hello method.

❯ cd simple_app
❯ mix compile
Compiling 2 files (.ex)
Generated simple_app app

We can use the interactive console to test manually or just use mix test to use the test suite

❯ iex -S mix
Erlang/OTP 23 [erts-11.1.7]

Interactive Elixir (1.11.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> SimpleApp.hello()
:world

By typing SimpleApp.hello() we are invoking the hello method created by the helper and we should get a symbol :world in return if everything is set up correctly.

2. Writing the basic functionality

For this app, we'll use the public API of Coindesk to query the price of Bitcoin today. If you'd rather prefer to use a different API, you can find a list of public ones on this Github repo. The data that we're getting is really not that important for the purpose of this exercise.

Before continuing however, you may want to check the API you choose is operational and to check the format of the returned data. I'd use curl in my shell, but wget, Postman or any other UI client should do the trick.

❯ curl https://api.coindesk.com/v1/bpi/currentprice.json

To integrate the API via Elixir let's use the HTTP wrapper Tesla. There are many good options out there, such as the good old Httpoison. However, Tesla has some added benefits. I won't go into details as it's not the purpose of this article, but it's worth checking out.

To install Tesla, we need to first add the dependency to our mix.exs file

defp deps do
  [
        {:tesla, "~> 1.4.0"},
        {:hackney, "~> 1.16.0"},
        {:jason, ">= 1.0.0"}
        ...
    ]
end

Notice that I also included hackney and jason. We'll use Hackey as the adapter to actually perform our http calls. Tesla uses httpc as the default adapter, but Hackney is far better in many aspects.

We'll need jason too as it's a required JSON parser for our middleware. It's also more performant than other JSON parsers so it's good to have anyway.

Go back to your shell and install the newly added dependencies

❯ mix deps.get

You may not have hex installed in your system yet, so if you get prompted an option to do it, just say yes.

The last part of the Tesla setup is adding the adapter to our config file. If you haven't created the file yet, just put it under config/config.exs.

We can then add the Hackney adapter so your config file should look like this

use Mix.Config

config :tesla, adapter: Tesla.Adapter.Hackney

And that's it, we're done configuring. Throw in a mix compile for good measure, to make sure we're still good to go and we haven't made any typos.

The API calling class

To query the Coinbase API via Tesla, we're going to create a model.

I like to have business logic inside a models folder because it helps me keep things organised on larger projects. Go ahead and create lib/models/coinbase.ex

Inside it we'll need 3 things, our module definition, our Tesla middleware configuration and the method(s) we want to abstract.

defmodule Coinbase do
  @moduledoc """  
  A module abstraction used to interact with the coinbase http API  
  """
  use Tesla

  plug Tesla.Middleware.BaseUrl, "https://api.coindesk.com"
  plug Tesla.Middleware.Headers, [{"Content-Type", "application/json"}]
  plug Tesla.Middleware.JSON

  @doc """  
  Returns the Bitcoin Price Index from coinbase  
  """
  def bpi_current_price do
    get("/v1/bpi/currentprice.json")
  end
end

We can test this right away in our iex console

❯ iex -S mix
Erlang/OTP 23

Interactive Elixir (1.11.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Coinbase.bpi_current_price
{:ok,
 %Tesla.Env{
   __client__: %Tesla.Client{adapter: nil, fun: nil, post: [], pre: []},
   __module__: Coinbase,
   body: ...,
   headers: [
     ...
   ],
   method: :get,
   opts: [],
   query: [],
   status: 200,
   url: "https://api.coindesk.com/v1/bpi/currentprice.json"
 }}

Displaying the data

With our base model done, the last thing we need is something to display the data in an easy to read format.

We are going to be using the light http server Cowboy and the adapter Plug to return our data.

Exploring the different server options and frameworks along with their pros and cons is out of the scope of this article, so we'll stick with the simplest solution.

Let's go back to our mix.exs file to add Plug and Cowboy to our dependencies

 defp deps do
  [
        {:tesla, "~> 1.4.0"},
        {:hackney, "~> 1.16.0"},
        {:jason, ">= 1.0.0"},
        {:plug_cowboy, "~> 2.0"} # NEW!
        ...
    ]
end

Get those deps, compile the project and we're ready to move forward

❯ mix deps.get
❯ mix compile

Now, without caring for organisation too much, let's add a little router file that will expose our endpoint to the world and call the API we've created previously.

Go ahead and create lib/router.ex and define a basic Plug endpoint

defmodule SimpleAppRouter do
  use Plug.Router

  plug :match
  plug :dispatch

  get "/bpi" do
    send_resp(conn, 200, "BPI price coming soon...")
  end

  match _ do
    send_resp(conn, 404, "Wrong place mate")
  end
end

If you're not familiar with it, feel free to check the inner workings of Plug in their documentation. For now, the code above is fairly self-explanatory I hope. All we need to know is that we're using the Plug.Router capabilities and exposing an endpoint /bpi which we're going to use to retrieve our data and to show it.

We can now proceed to tell our supervision tree that our Router is ready to roll.

Go to the lib/simple_app/application.ex where you'll find the application file created by the mix new helper. Here we will ask our application to start the router under the supervision tree.

Your resulting file will look like this

defmodule SimpleApp.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc """
  Application that starts the http router for `SimpleApp`.
  """

  use Application

  @impl true
  def start(_type, _args) do
    children = [
      {Plug.Cowboy, scheme: :http, plug: SimpleAppRouter, options: [port: 8080]}
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: SimpleApp.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

You can go ahead and test it all works fine by running your server with

❯ mix run --no-halt

To display the Coinbase data, all we need to do is go back to our SimpleAppRouter and add the Coinbase call. Your router function will then look like this

get "/bpi" do
  send_resp(conn, 200, Coinbase.bpi_current_price)
end

If you remember the format of the bpi_current_price function response, you may have noticed that it includes not only the body of the response but also headers, process status etc which we don't necessarily want to display. So to fix this, go back to lib/models/coinbase.ex and modify the function to only return the response body instead

def bpi_current_price do
    {:ok, response} = get("/v1/bpi/currentprice.json")
    response.body
 end

Bear in mind we're not doing any sort of validations or error considerations for the sake of simplicity.

3. Testing it works locally

Now with the app finally ready, we can test it works in our local environment.

Go ahead and run your server again

❯ mix run --no-halt

Then in your browser or curl client, you should be able to obtain the results on http://localhost:8080/bpi

❯ curl http://localhost:8080/bpi
HTTP/1.1 200 OK
...
server: Cowboy

{ ..."chartName":"Bitcoin","bpi":{"USD":{"code":"USD"... }

4. Creating your Heroku app

Before we move entirely into the Heroku app creation, there's one last thing we need to change in the code.

Notice how we used port 8080 as the default, but once in Heroku our app port will be dynamic and determined by them. For this reason, we need to read the port from the environment variables.

To do this, go ahead and update the port 8080 inside the application file with one taken from the system environment variables.

children = [
  {Plug.Cowboy, 
        scheme: :http, 
        plug: SimpleAppRouter, 
        options: [port: (System.get_env("PORT") || "8080") |> String.to_integer()]
  }
]

This has the added benefit of defaulting to 8080 if the PORT environment variable does not exist, which is convenient when using it locally.

To create our Heroku app we're going to be using the heroku-cli. However, if you feel more comfortable using an UI, you can go to your Heroku dashboard and the steps are pretty straightforward.

The first thing you need is a Heroku account, which I'm going to assume you already have. If you don't, just sign up on their website using your preferred email and password.

PRO TIP: If you happen to have an existing Heroku account that you do not want
to use for this exercise, you can use the heroku-accounts plugin to manage
multiple accounts easily.

PRO TIP: If you happen to have an existing Heroku account that you do not want 
to use for this exercise, you can use the [heroku-accounts](https://github.com/heroku/heroku-accounts) plugin to manage
multiple accounts easily.

To create a new Heroku app using the cli, just go to the root directory of your app in the shell and type

❯ heroku create simple-app-elixir
Creating ⬢ simple-app-elixir... done
https://simple-app-elixir.herokuapp.com/ | https://git.heroku.com/simple-app-elixir.git

Where simple-app-elixir is a unique name you want to give to your app.

Because Heroku doesn't have any default buildpack for Elixir at the time of writing, we need to set our app's buildpack to HashNuke's version of it.

❯ heroku buildpacks:set \
  https://github.com/HashNuke/heroku-buildpack-elixir.git -a simple-app-elixir

For this buildpack to work in production, we need to specify a default configuration on our Heroku app. In the root of the project go ahead and create a file named elixir_buildpack.config and fill it with your versions of the OTP and Elixir.

erlang_version=23.0
elixir_version=1.11
always_rebuild=false
runtime_path=/app

The last thing we need to do is to tell Heroku what process to run when deploying the app. For this we can use a Procfile. In our Procfile we will specify that we want mix to run our app which in turn will run the Cowboy server.

Create another file at the root of your project called Procfile (without any file extension), and add the following inside it

web: mix run --no-halt

Notice this is the same command that we use to run our server locally. The buildpack will actually use this same command as default if no Profile is found, but it's good practice to define it ourselves anyway.

5. Deploying and testing your app

Now we're ready to deploy. Heroku has a git based deployment system, which is really convenient because all we need to do to deploy our app is to git push our code.

In your shell, at the root of your project, go ahead and commit your code to Heroku as if it was another regular git repository.
You may need to add your Heroku remote using your own git URL

 ❯ git remote add heroku [https://git.heroku.com/simple-app-elixir.git](https://git.heroku.com/simple-app-elixir.git)

Yours was printed when creating the app, or it can also be found in the settings of your Heroku app.

# If you haven't initialised the repository yet
❯ git init
Initialized empty Git repository

# Commit your code
❯ git add .
❯ git commit -m "SimpleApp initial commit. Coinbase API"
❯ git push heroku master

This should've deployed your app if your setup is correct, SSH keys are set and there aren't any additional configuration issues.

Once your code is deployed, Heroku will assign a subdomain to your app, something like

https://simple-app-elixir.herokuapp.com/

Depending on the time you're reading this article, mine may or may not be active. Either way, once yours is deployed you should be able to check it out.

You can go ahead and test our /bpi endpoint in the browser or via curl.

❯ curl https://simple-app-elixir.herokuapp.com/bpi

This endpoint should return a json object with the Coinbase Bitcoin exchange rates and your deployment would be complete. Congrats! 👏

Conclusion

As you can see, it's fairly simple to deploy Elixir apps in Heroku. The platform allows for scaling as you go and it's able to handle Elixir's performance requirements very well.

Another advantage of Heroku is that you can have apps in multiple languages running all under the same umbrella platform, which lets you have complex microservices architectures servicing different needs without any additional platform knowledge required.

I've put a full working version of the code on this Github repository, in case you want to compare with it or simply clone it and test it.

For more topics, check my profile or visit rarias.dev.
Happy coding!

Why you should learn multiple Programming Languages

Ron — Fri, 05 Feb 2021 16:40:29 +0000

Disclaimer: This is not an article to teach you a quick hack on how to learn multiple programming languages, but an analysis on its advantages. Soz!

If you have a career as a Software Developer, chances are you already know more than one programming language. If you're a specialised Frontend Developer, you may know Javascript and Typescript for example. If you're a specialised Backend Developer, you may prefer Java but know a thing or two about Python or C++. If you're a "Full Stack" Developer your odds of knowing more than one are higher.

My point is that you may already fit the basic definition of knowing more than one programming language.

For the purpose of this article, I want to extend this definition a little bit to focus on not just knowing but actively seeking to solve problems with more than one programming language.

This is what it's commonly known as being "Programming Language Agnostic".

The idea behind it is that as Software Engineers we are meant to solve the problems we encounter using the best tool for the job. It makes sense if you think about it. Why would you limit yourself to only a few available options?

Before I carry on to explain why this approach is better in my opinion, I want to make a few clarifications:

There are plenty of scenarios in which expertise and knowing the intrinsic details of a particular technology is the best approach. If you want to build a career as a consultant and target a very specific market for example, it makes sense for you to not look around much.
Not everyone has the flexibility to choose the programming language they want to work with a lot of times. There are real infrastructure or cost limitations that may prevent you from doing so.

Now, for the rest of you out there, here's the approach I'm talking about.

With the growth in popularity of microservice architectures and cloud hosting, it's easier than ever to have hybrid infrastructures, which allows you to pick different tools to solve the problems at hand.

I'm not entirely sure how many people out there take advantage of this flexibility, but my wild guess would be that most don't fully consider the possibilities of this setup.

What is the solution going to look like if you only know one or two programming languages? As the saying goes "If you only have a hammer, everything looks like a nail".

Let's see how a potential problem solving session might look like in a scenario when we're open minded about picking the best tool for the job, independently of how familiar or comfortable we feel with it.

The problem

Suppose you want to build an algorithmic trading application. This app will need to:

Gather data from different financial APIs in real time.
Parse, process and store this data.
Retrieve this data to apply backtesting strategies.
Run your strategy in real time and trigger alerts for your users to trade.

NOTE: This is just an educational example. The solutions suggested are for illustration purposes and may not be the most optimal.

Approaching the problem

If you want to have a monolithic application, there's less flexibility around the tools you can use, but if you decide to work in a microservices environment, approaching the problem gives you a lot of options and freedom.

If you are a Python developer for example, or Python is your preferred programming language, you can potentially build this whole application in a Python stack.

Similarly if you're a Java developer, a Ruby developer, a Golang developer etc. You get the point.

But approaching the problem without thinking about the language will let you pick the best one that fits the criteria.

Gathering & storing the data

In our little language agnostic world, our problem of gathering the data becomes one that requires a lot of concurrency.

If you hit the APIs sequentially, this will be incredibly slow. So you'll need a tool that's really good at concurrency and parallelism but also allows you to deal with HTTP requests and data manipulation easily.

Perhaps you've heard of Elixir. It is known that this language was built thinking about powerful concurrency, so it makes sense for us to consider it. This is not the only one of course, so keep your options open.

Now, we can perhaps build a small application with Elixir, that leverages the concurrency and hits the APIs with a high number of threads and collects that data for us at regular intervals.

At the same time we're collecting the data, we can either store it right away, or send it down a pipeline for someone else to process it. This will depend on how scalable you want to be.

For now, let's say we send this data down a pipeline using something like Apache Kafka as soon as we've retrieved it. This will reduce the efforts of the API crawler and delegate the storing of the data to someone else.

Backtesting

This is the most passive option of all. You do not need anything with great real time performance, but a language or tool that's really good at data manipulation.

Perhaps Python is a great option here? You can use it to a great extent to read large amounts of data and apply mathematical models and test different strategies with it. Depending on the amount of data you have you can even throw in tools like Apache Spark in the mix to help you out.

Running your strategy in real time

Here's where you need a language that is very good with IO operations. You want to be as real time as you can to let your users make the best decisions.

You then may decide that a combination of Javascript/Nodejs with a tool like Firebase for example might do the trick.

If you design your infrastructure the right way, you can perhaps even read the events from your Kafka queue straight away and send it to the users connecting 2 completely different technologies via the Kafka events queue.

As you can see, at a higher level we can already start including many different programming languages in the mix to build the best possible solution.

How much of each language do you need to know?

You may be wondering now that building that whole stack will require a team of developers, with expertise in different areas and each one specialised in each of those technologies.

But the reality is far from it.

To be able to build hybrid infrastructures like these, you only need to know about the capabilities of a language very well. Of course, you'll have to learn the actual specifics of the language at some point, but for starters the capabilities are more than enough. You can get by with building your solution if you know what's possible and then replace parts conveniently later.

I am particularly bad at remembering things for example, but I routinely work and build apps on top of at least 3 or 4 very different programming languages almost daily. What allows me to do so is a combination of

Knowing what the language is capable of.
Always having the documentation at hand. I don't have to remember the exact syntax and methods libraries as long as I can look it up.
Best practice guides.
Good understanding of the basic paradigm of the language, is it OO or functional? How far can I push it? How performant is it? At what point does it start to break?

Just like Scott Adams, creator of Dilbert says, you don't have to be a world class expert on any particular one skill, but a combination of many average skills can turn you into someone awesome.

Same applies for programming languages, you don't have to be an expert in one or all of them. For the average Software Engineer, being fairly good in a few will give you far more leverage to build great applications that you imagine.

How to know you've made the right choice

After you've picked the programming language that best suits your solution, you need to start thinking about implementation.

If you've never used the language you picked, it's better to start by testing a few of the principles you want to exploit first. Interactive shells are great for this purpose and most languages have them.

Another thing to consider is the way you build your application, how far into the future can you see? Even though you've made (or think you've made) the right choice, it won't necessarily continue to be the right choice as time passes.

Perhaps you picked Ruby as your general purpose language because you only needed to get a small amount of data processed and needed a solution fast and simple. This fits your purpose very well, but if you start seeing heavy growth in traffic and Ruby starts getting memory problems, then it's time to consider a replacement to better fit the new scenario.

Is it difficult to change your mind and pick another language later?

This will depend on how you've built your system. The best way is to have services (or micro-services) as isolated as possible with a common interface between them.

For example, if you use an HTTP API to exchange JSON messages, or a Kafka queue, you should design your service in a way that the data coming in and going out doesn't get influenced by the programming language underneath.

This way, if your service needs replacing with a completely different programming language, you can basically plug and play without disrupting other dependencies.

Even though you may make an effort of reducing dependencies between your services, you can get away with the better approach of abstracting communication instead, and finding a common efficient way to exchange messages that doesn't depend on the underlying language.

Conclusion

There's no one single solution to every problem, and there's no need to use a single programming language to find that solution.

Learning more than one language gives you the tools to think about possibilities you didn't consider previously.

Limitations of a technology are no longer a limitation. This way, you start looking at programming languages as great tools with pros and shortcomings.

There's almost always going to be a preferred language, particularly for general purpose applications, and that's fine as long as you know when it's ok to use it and its shortcomings, and more importantly, you're willing to pick a better one when it's appropriate to do so.
Happy coding!
For more topics, check my profile or visit rarias.dev.

Hello World - Python

Ron — Fri, 29 Jan 2021 11:58:21 +0000

Python is a programming language that has been around for quite a while and that has so much to offer. Since its inception about 30 years ago Python has come a long way, as well as being on and off fashion in many opportunities.

Nowadays, Python has experienced a big boom due to the rising popularity of Machine Learning and the introduction of easy to use tools aimed at exploiting Python’s capabilities in this area.

So, what’s Python about anyway?

Python is actually a general purpose programming language, it’s interpreted and object oriented and its syntax is quite easy to read. It has a huge community behind it, for which it’s easy to find a solution to basically any problem you encounter with it.

You may find that there’s a lot of chatter focused on the specific version you use, whether it’s Python 2 or Python 3. That’s because in the late 2000s Python 3 was introduced, which represented a major jump from its predecessor line Python 2.x and it came with actual incompatibilities.

A lot of things were deprecated, and a bunch of others were introduced. Unfortunately for the Python community this caused a lot of issues. There were already many apps running Python 2, and the transition wasn’t as easy as you may have wanted.

To this day, there are still a lot of apps and libraries running and depending on the previous version, and developers need to keep this in mind when using the language.

Fortunately for us, just like in many other languages like Ruby or Elixir, Python also has version and environment managers nowadays, which helps keeping track of different versions and sets of libraries within our projects.

This article is going to be divided into 3 sections: Installation, Basics and Capabilities. Within the capabilities however, there's an incredibly long list, especially in the Machine Learning and Lambda architecture areas that I will discuss in more detail later in different posts.

Installation

How easy or simple the installation process is will depend on your operating system and what other dependencies you may already have in your system.

If you’re a developer like me who uses other programming languages and tools, or if you’re on an older version of MacOS, you may already have installed dependencies that require either Python 2 or 3.

I will describe the installation process of one of the simplest methods I know, but be aware that you may encounter some resistance on the way. Should you do so, feel free to drop me a line and I'll do my best to help you get up and running.

First thing to keep in mind is that you do not want any of the built-in installations of Python running the show underneath, so the best approach is to use a version manager.

Probably the less intrusive Python version manager out there is Pyenv. Unlike others, Pyenv does not depend on Python itself as it's built entirely with shell-scripts. This is a nice thing to have.

There are multiple ways to install Pyenv, you can check their installation page for more information. If you're on MacOS, the easiest way is to use homebrew.

You can install Pyenv using

$> brew install pyenv

You can verify the installation was done correctly by typing which pyenv in your shell.

After this, you need to let your bash initiate pyenv on start. For this you can add the following init script to whatever shell you're using.

In my case for zsh I need to add it to my .zshrc but for you it could be .bash_profile or .bashrc.

if command -v pyenv >/dev/null 2>&1; then
  eval "$(pyenv init -)"
fi

This will allow Pyenv to manipulate your path and give itself priority inside pyenv environments.

With all these done, we can now set the global pyenv we want to work with.

To see the available versions we have for installation, we can use

$> pyenv install --list
Available versions:
2.1.3
2.2.3
...
3.9.0
3.9-dev
3.9.1

So let's go ahead and install version 3.9.1 and set it globally.

$> pyenv install 3.9.1

Before you set it globally however, have a look at your current installation of python (if any) so you can see how the global version of Python is changed.

$> python --version
Python 2.7.16 # You may see something well different, this is my MacOS system default

$> pyenv global 3.9.1
$> python --version
Python 3.9.1

If you don't see a version change, it may be that the init script hasn't been executed. You can try reloading your bash and checking again.

That's it, now you have a fully running and updated Python interpreter, with the added ability of installing multiple different versions.

Within projects however, it's best to also use environment managers like pipenv. I will write in more detail about it in the future.

Basics

Python is a really intuitive language, where most code (if done right) can be easily read as if you were reading a textbook.

One main thing to consider is that Python does not have opening and closing blocks in the same way other programming languages do. For this, the language relies on indentation, so if a block is meant to be nested within another block, we indent it a tab further than the parent block. An example of this would be a typical function

def say_hello():
    print("Hello World")

Notice the print is a tab space within the function header and there's no closing statement like end or }.

Let’s test a few things in Python’s interactive console. Like other languages, Python offers an interactive console you can use to test things out. I particularly love them, as it saves me time when I’m unsure about a particular syntax or method. Often-time it’s faster to test it in the console rather than run tests, execute the program you’re writing or browse the documentation.

However, when using recent development tools like Jupyter notebooks, which are heavily used in Machine Learning, you get immediate feedback within the cells. This makes it faster than going back and forth from your development environment to the console.

To start the interactive console, go to your shell and just type python

$> python
Python 3.9.1
Type "help", "copyright", "credits" or "license" for more information.

Bear in mind that the console should start in the version of Python that you've previously set as global default via pyenv. If it doesn't, you may want to exit the console using the exit() command and revise the reason why it's not selecting the right version.

The built-in function in Python for console output is print. That makes our first Hello World fairly easy.

>>> print("Hello World")
Hello World

Python data types and operations don't differ much from other languages

# There's simple arithmetic operations
>>> 5 + 5
10

# There are String concatenations
>>> "Hello" + "World"
'HelloWorld'

# And there are of course String-Integer operations
>>> "Hello" * 5
'HelloHelloHelloHelloHello'

# Hold on, what?

That last one wasn't too intuitive, but it's pretty cool. Python is really flexible and allows you to use nifty tricks like multiplying a string against an integer to obtain many of them. There are plenty of cool examples like these, especially when you start using the numpy and pandas libraries, which supercharge Python into greatness.

Functions in Python are also pretty similar to other languages, and if you break them down enough it can make your code even more readable.

>>> def mean(values):
    return sum(values) / float(len(values))
...
>>> mean([1,2,3])
2.0

This is particularly useful when simplifying complex math operations. A type of function called a lambda, which is just an anonymous function, plays a big part as well in simplifying (or complicating if overusing) your code.

>>> def variance(values):
    mean = lambda v: sum(v) / float(len(v))
    return sum([(x - mean(values))**2 for x in values])
...
>>> variance([1,2,3,4,5])
10.0

You may have noticed the way the values are processed in that last variance function, where the for loop seems to have a very readable structure. That's a list comprehension and it's one of the easiest ways to go over lists in Python.

The flexibility comes from being able to do mapping operations like

# The following will generate a new list in one pass of the array
>>> [value + 1 for value in range(1, 5)]
[2, 3, 4, 5]

but also to apply (usually more complex) functions to each value in the collection

>>> addtwo = lambda a, b: a + b
>>> [addtwo(value, value + 1) for value in range(1, 5)]
[3, 5, 7, 9]

Capabilities

I could write 100 articles listing Python capabilities (ok maybe not me, but someone really into the language), and it still won't be enough to cover it. It is such a versatile and powerful programming language that it can probably be used in any context.

The most recent and popular uses for Python that I'm interested and fond of include Machine Learning and Serverless applications.

Machine Learning

There are many reasons why Python has been chosen for Machine Learning. It's simple and easy to learn, which allows practitioners to focus on the problem rather than on the language, but at the same time there aren't many major tradeoffs in terms of performance.

There's also a great amount of tools focused on Python that were recently open sourced, and allows for model building with ease.

For many years as well, the Python community have been building Math oriented really powerful libraries that makes manipulating data, again, much easier than in other languages.

Serverless applications

Serverless gained popularity recently due to the fact that it offers a lot of abstraction and scalability to developers. In Serverless architectures, "servers" are not explicitly defined by the deployers, but rather we focus on particular functions that handle very specific events.

Platforms such as AWS Lambdas and Google Cloud Functions allow the writing and deployment of such functions with ease, and more importantly, they allow the systems to scale at will, providing more functions when needed (when server load increases for example) and being able to handle higher levels of concurrency without increasing code complexity.

Conclusion

Python is a powerful yet simple language. There's a lot to learn within its environment, but having now an overview of the language and its capabilities, you can go ahead and get started experimenting with it.

Whether you're interested in Machine Learning or building scalable applications, or simply scripting tasks with an easy to use programming language, Python has a lot to offer you and whatever path you decide to take, I'm sure it's going to be an enjoyable one.

Happy coding!
For more topics, check my profile or visit rarias.dev.

Hello World - Ruby

Ron — Fri, 22 Jan 2021 12:27:07 +0000

Because everything in the programming world starts with a Hello World, I’ll come back to the very basics with a Ruby Hello World.

Ruby is an Object Oriented programming language that’s been quite popular in the past years, specially within the Rails framework. I use it on my day to day basis for all sorts of scripting, micro-services and different production tasks.

The thing most people like about Ruby is its simplicity. You can write code quickly as if you're writing plain text (almost), and you can use the interactive console to test things out.

Many languages have adopted these practices and now it's even easier to get started with them as well. The likes of JavaScript, Python, PHP, etc have come a long way too.

This article is going to be divided into 3 sections: Installation, Basics and Capabilities. In other posts I'll go deeper into the more complex and fun characteristics of the language.

Installation

Installation is quite simple, nowadays Ruby can be obtained from many different sources and package managers. You can use apt-get, brew, download a binary or exec file etc.

But for me, the best way is to use a version manager. This little tool allows you to have multiple versions of Ruby running seamlessly in the same machine, which is specially useful when having multiple apps, all running different versions (micro-services environment).

My favourite one is [chruby](https://github.com/postmodern/chruby). There's also other popular ones like rvm or rbenv but I found chruby works the best for me.

The easiest way to install it is using brew. You can check other installing options on their website. Depending on your OS, the tools will differ.

Now for chruby, simply going to your shell and type:

$> brew install chruby

After that, you can install any version of Ruby you want by using:

$> ruby-install ruby-2.7.2

This will install the version 2.7.2.

This is the absolute basics. You can test ruby installed correctly by using ruby -v

You can also switch between installed versions using $> chruby 2.7.1 for example.

Basics

Ruby is a really descriptive programming language. You can find an extensive guide in the official ruby documentation. I will only highlight certain basic aspects that I consider relevant or interesting.

The easiest way to start playing around with Ruby is using the interactive console (irb - Interactive rb). To access it simply type irb in your shell. It should already come installed with your ruby version.

$> irb
irb(main):001:0>

Now as promised, the hello world. The puts command is the equivalent of print in other languages. It simply outputs the object that follows.

>> puts "Hello World!"
Hello World!
=> nil

Just like that!

Notice that the method itself executes the print, and then it returns a value nil. nil is the special object reserved for NULL .

Now, everything in ruby is an object, even the less intuitive things. That's something to keep in mind, because this approach will help us accomplish really cool things.

As an example, take the following code:

>> nil.class
=> NilClass

>>nil.nil?
=> true

Pretty straightforward, right?

Now take the following:

>> true.class
=> TrueClass

>> 1.class
=> Integer

>> "hello".class
=> String

What this means is that what we call values in other programming languages, like true, 1 and hello are objects in Ruby. This allow us to call methods directly on them without having to invoke the parent class Integer or String for example.

>> 1.odd?
=> true

>> "hello".size
=> 5

>> true.to_s
=> "true"

You may have noticed already that Ruby is also a loosely typed language, there's no need to declare types on variables; there's also no need to close the line with a semi-colon for example.

Take the following code:

# Ruby will try to infer the data type based on the value (object) passed
>> a = 100
>> b = 200
>> a + b
=> 300

# But it will also let you override these types as you please
# And it will apply validations when appropriate
>> b = "200"
>> a + b
=> TypeError (String can\'t be coerced into Integer)

# Nevertheless, some other times it will try to infer what you want
# Like concatenating Strings
>> a = "100"
>> a + b
=> "100200"

# Or transforming ints into floats
>> a, b = 100, 200.0
>> a + b
=> 300.0

In case you missed it, Ruby is also capable of doing value destructuring. Check how a gets assigned the value 100 and b the value 200.0. Ruby has very powerful one-liners. Which is at times a bless, but also a curse. Sometimes it can obfuscate the code, but others it can save you many lines.

Have a look at a simple variable value swap for example:

>> a = 100
>> b = 200

# A typical value swap may include a third temp variable
>> c = a
>> a = b
>> b = c

# But in Ruby you can just change them in a one-liner
>> a, b = b, a

There are truly a battalion of cool things you can do with the language, and it takes more than a Hello World post to do it, so I'll cut it short here. I'll release a series of basic posts aimed at Ruby and other programming languages to discuss the many cool things I encounter in my day to day and comparisons between the different programming languages I use or learn.

Capabilities

I like to believe that every programming language have a series of tasks that it does better than others. You have languages that are great for Math problems, others for Machine Learning, then there's the ones great to build amazing user interfaces, and the ones to build great distributed systems, etc.

Personally, I put Ruby in the go-to simple category. It's readable enough to make my code super clean, but powerful enough to be used for a wide variety of applications. Hence I use it as my general purpose programming language.

Within the Rails framework, you can build full websites really quickly and with little expertise. Once learning the conventions, it becomes an easy process.

For scalable systems, you can use Ruby apps on top of the Sinatra Framework as part of a micro services architecture. This way you can ensure the memory load on particular apps can be managed easily, and you have a good separation of domains, apart from all other advantages you get from a generic micro services system.

Conclusion

I hope I've interested you enough to do a little bit more reading on your own. Ruby is a great language if you're just getting started with programming or if you're experienced but looking for clean and readable solutions. It does cover the whole spectrum.

Apart from being easy to learn, it has a good deal of use cases and a great community behind it, making it easier to learn and maintain.

That's all for now, happy coding!

For more topics, check my profile or visit rarias.dev.

Hello World - Elixir

Ron — Fri, 22 Jan 2021 12:20:54 +0000

Here's a personal favourite. Elixir is a dynamic, functional programming language built thinking about scalability and parallelism. It is built on top of Erlang, so performance wise it's really powerful.

It is not quite as popular as other programming languages out there and it has a relatively small community due to being still young, but its capabilities are still great.

I think developers tend to favour GoLang for exploiting distributed tasks or Clojure and Scala on the functional aspect, yet Elixir seems to me like a nice place in the middle.

Some of the issues around Elixir revolve around lack of popularity for functional languages, reduced compatibility in some hostings and sometimes poor error readability when things go wrong.

This article is going to be divided into 3 sections: Installation, Basics and Capabilities.

Installation

Elixir can be installed in many different ways, you can find the way that best works for you on their installation page. My preferred way (on MacOS) is to use a package manager like homebrew. That way I can stay on top of updates and releases fairly easily.

The installation using homebrew is fairly simple. You can use

$> brew install elixir

and brew will do all the heavy lifting for you. You can test your installation using elixir --version.

Using a version manager

As with other programming languages, you may find yourself needing to use multiple versions of the same language on different apps, specially on microservice environments. To resolve this problem, you can either use dockerized setups or use a version manager.

There are multiple version managers available for Elixir. I particularly like and use kiex. It's really simple to use and to be fair, it's the one I've had less trouble with.

To install kiex, they provide an installation script you can run from your shell

$> curl -sSL https://raw.githubusercontent.com/taylor/kiex/master/install | bash -s

From there you can follow the instructions if necessary. This script will install kiex in your $HOME/.kiex

Usage is fairly simple as well, you can check the available versions installed on your system with

$> kiex list
=* elixir-1.8.2

# => - current
# =* - current && default
#  * - default

or generate a list of available releases for you to download with

$> kiex list known
Getting the available releases from https://github.com/elixir-lang/elixir/releases
Known Elixir releases:
    0.7.2
    0.8.0
    ...
        1.11.2
    1.11.3

From where you can pick the version you want to install and let kiex do it for you

$> kiex install 1.11.3

When going around project folders, you can select the Elixir version you want to use (provided it's been installed), via kiex with

kiex use 1.8.1

There are other interesting things you can do listed in their documentation. For now, I'll stick to the simple usage commands.

Basics

Elixir is a functional programming language, so writing code in it may not be the most intuitive thing for everyone. There's so much to learn about it, so feel free to check their documentation if you have specific doubts or are interested in learning more.

To make it easy for ourselves to start testing things out, we'll use a really neat tool called the Elixir Interactive Console (iex - Interactive Elixir). Similarly to the ruby irb or Python interactive shell, it will allow us to test simple concepts first.

If Elixir is correctly installed, you should be able to enter the iex from your shell by typing the iex command

$> iex
Erlang/OTP 23

Interactive Elixir (1.8.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>

Once in the console we can start playing around with Elixir.

Elixir has specific data types, but you don't have to explicitly declare them. The basic ones include

iex> 1          # integer
iex> 0x1F       # integer
iex> 1.0        # float
iex> true       # boolean
iex> :atom      # atom / symbol
iex> "elixir"   # string
iex> [1, 2, 3]  # list
iex> {1, 2, 3}  # tuple

These data types are pretty similar to other programming languages. So you can perform operations on them, pass them around in variables etc. There are a few important differences however, which will be mentioned towards the end of this section.

Using these data types is also pretty similar to other programming languages

# Arithmetic operations behave the same
iex> 1 + 2
3
iex> 5 * 5
25

# String interpolation behaves the same
iex(1)> string = :world
iex(2)> "hello #{string}"
"hello world"

# And you can print using the IO module
iex> IO.puts "hello world"
hello world
:ok
# Notice the return value of the method is :ok. More on that later

To perform operations on data types or variables, you will have to use the methods from its class. Take the String class for example

iex> String.upcase("hello world")
"HELLO WORLD"

iex> String.split("foo bar")
["foo", "bar"]

iex> String.reverse("abcd")
"dcba"

There are however, quite a few functions that are available in the global scope

# To check whether a variable belongs to a data type
iex> is_boolean(true)
true

# To check if a variable is NULL
iex> is_nil("hello world")
false

Pattern matching

Now, here's something that might trip many, and it's the = operator. In most languages this operator is used to assign values to a variable, and at first sight in Elixir it may look just the same

iex(1)> hello = "hello world"
"hello world"
iex(2)> hello
"hello world"

But consider the following sequence

iex(1)> hello = "hello world"
"hello world"
iex(2)> "hello world" = hello
"hello world"
iex(3)> "goodbye world" = hello
** (MatchError) no match of right hand side value: "hello world"

What happened?

It turns out that the = is actually called the match operator, or pattern matching operator. Its goal is not to simply assign values but to compare the two sides of the equation.

This is pretty useful in more complex situations, like destructuring expressions for example

iex(1)> {a, b, c} = {:hello, "world", 42}
{:hello, "world", 42}
iex(2)> a
:hello
iex(3)> b
"world"
iex(4)> c
42

But more interestingly when comparing specific types as a result of a particular operation.

Consider the following example

iex(1)> {:ok, result} = {:ok, 200}
{:ok, 200}
iex(2)> result
200

iex(3)> {:ok, result} = {:error, 401}
** (MatchError) no match of right hand side value: {:error, 401}

Notice that :ok and 200 are values being matched. When those values do not match, we have received unexpected data, and the operation will be invalid. Do the codes seem familiar to you? 200 and 401 are common http response codes, and you can use pattern matching to handle http responses in a neat manner

# Simple http request example for pattern matching response
def http_request(url) do
  case HttpRequest.get(url, headers(), []) do
    {:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
      body
        {:ok, %HTTPoison.Response{status_code: 401}} ->
        IO.puts "[ERROR] Authentication failed."
      nil
    {:ok, %HTTPoison.Response{status_code: status_code}} ->
      IO.puts "Unexpected http response: #{status_code}"
      nil
    {:error, %HTTPoison.Error{reason: reason}} ->
      IO.puts "Unexpected error: #{reason}"
      nil
  end
end

Process calls in Elixir typically return a status and the result. In the previous example you can see either the tuple {:ok, response_object} or {:error, error_object} for which we can pattern match accordingly and handle each scenario separately. The Pattern match can be as complex as you'd like.

There's more much to talk about this complex, yet amazing programming language. I will release a series dedicated to Elixir soon.

Capabilities

The real advantages of Elixir come when you're trying to parallelise tasks. Concurrency, scalability and fault tolerance are built into the language.

In Elixir, all code runs inside processes, which are isolated from each other and run concurrently and communicate passing messages between each other.

These processes are quite lightweight, for which at any given time you may be running a great amount of them.

This allows for incredible fault tolerance. Processes can be restarted quite quickly without big performance implications. In fact, Elixir's strategy for error recovery is simply killing the processes and starting it again. This is a "fail fast" strategy. Elixir developers made the design decision that letting a program fail and recover fast was a better strategy than defensive programming and trying to predict any possible error.

Additionally to standard processes, Elixir has GenServers and Supervisors. These are in charge or keeping state and monitoring other processes respectively. GenServers and Supervisors are the building blocks of server-client relationships.

In terms of parallelism, Elixir has built-in modules like the following

results = Task.Supervisor.async_stream_nolink(
  MyApp.TaskSupervisor,
  MyApp.list_items(),
  (fn item -> MyApp.process_item(item) end),
  [max_concurrency: 50]
)

In which we ask a supervisor to spawn 50 processes concurrently to deal with a list of items that need, well, processing. This is particularly useful in scenarios where we need to query external APIs or run particularly slow operations on each item.

If we have a list of 1000 items and 50 concurrent processes working on it, it only takes 20 rounds to get your results, and more importantly you can have them all combined at the end without dealing with any of the concurrency yourself.

Conclusion

There's much more to talk about Elixir, hopefully this brief introduction is enough to keep you interested.

Every programming language has a thing for which they're good at. Elixir is great for concurrent processing and easy parallelism. There are, of course, other solutions like GoLang or if you need greater data scale, tools like Apache Spark are better suited for the job. But Elixir should be considered given its simplicity and functional approach. Remember there's no one way to solve a problem.

For more topics, check my profile or visit rarias.dev.