DEV Community: Arne

How to create a Elixir release and use environment variables configured at runtime.

Arne — Sun, 27 Jan 2019 06:23:29 +0000

First things first, please indulge me, this is not my mother tongue!

One of the problems we are facing with every new application is the proper usage of Elixir with the possibility to use environment variables from a “docker-compose” file to configure the application at runtime.

There a several approaches to do this, but i want to show you my preffered way with Distillery and the “configTuples” Provider.

This combination allows us to build a new release and set configuration parameters like Database credentials at runtime.

I will guide you through one simple example, we will create a new Elixir application, add the nessecary config provider and run/build it with Docker.

Create our example application

Lets create a new application, I will call it “fuchsbau” … don’t ask me why :-)

$ mix new fuchsbau --module Fuchsbau

We will add some stupid code to echo a bunch of example environment variables (just to verify that this approach is working as expected).

## Example code to test environment variables

Create a new file “../lib/starter.ex” with the following content, to echo our configuration.

defmodule Starter do
   use Application

  def start(_type, _args) do
    IO.puts("... hey fuchsbau")
    Starter.echo_variables
  end

  @doc """
    Echo defined environment variables from config.exs
  """
  def echo_variables do 
    owner = Application.get_env(:fuchsbau, :config)[:fuchsbau_owner]
    street =  Application.get_env(:fuchsbau, :config)[:fuchsbau_street]
    phone =  Application.get_env(:fuchsbau, :config)[:fuchsbau_phone]

    IO.inspect(owner)
    IO.inspect(street)
    IO.inspect(phone)

    {:ok, self()}
  end

end

To run this code we have to add it to our “mix.exs”

 # Run "mix help compile.app" to learn about applications.
  def application do
    [  
      mod: {Starter, []},
      extra_applications: [:logger]
    ]
  end

Add examples to config.exs

To have some config variables for our test, please add some config values to your config.exs.

In a “real- world” example we would use different configs like “prod.exs” .. but for this short how to, it should be fine to just use config.exs

config :fuchsbau, :config,
  fuchsbau_owner: {:system, "FUCHSBAU_OWNER", default: "Anonymous"},
  fuchsbau_street: {:system, "FUCHSBAU_STREET", default: "Somewhere..."},
  fuchsbau_phone: {:system, "FUCHSBAU_PHONE", default: 67282929, type: :integer}

Use Distillery and ConfigTuples Provider

Here is the interesting part, you have to add Distillery and configTuples to your application (mix.exs).

defp deps do
    [
      {:distillery, "~> 2.0"},
      {:config_tuples, "~> 0.2.0"}
    ]
end

At your command line, run the following to get the new dependencies and to init the release (this will create a new directory “/rel” with a config.exs file we need to setup configTuples):

$ mix deps.get
$ mix release.init

To make use of the new config provider, add it to “/rel/config.exs”

release :fuchsbau do
  set version: current_version(:fuchsbau)
  set applications: [
    :runtime_tools
  ]
  # ConfigTuples Provider !
  set config_providers: [
    ConfigTuples.Provider
  ]
end

Time for the first Test

Thats all to use Distillery and the ConfigTuples provider (don’t worry we will continue with the Docker stuff in just a couple of minutes..)

Let’s try our release run:

$ mix release

You will see some examples to start the release, just use the one with “foreground”.. and hell yes .. i have to use Windows, forgot my MacBook at work..

After running your release, you should see the defined config values.

To verify that we are able to set environment variables at runtime, try the following:

$ export FUCHSBAU_OWNER=Medium
$ _build/dev/rel/fuchsbau/bin/fuchsbau foreground

This will output “Medium” as owner of the Fuchsbau!

Use Docker to build and run our application

The last step for this how to: We will create a Dockerfile to build our application and to run it.

Create a new file: “Dockerfile” in your project root with the following content:

FROM bitwalker/alpine-elixir:1.7 as build

# Copy the source folder into the Docker image
COPY . .

# Install dependencies and build Release
RUN export MIX_ENV=prod && \
    rm -Rf _build && \
    mix deps.get && \
    mix release

# Extract Release archive to /rel for copying in next stage, please change the application name 
RUN APP_NAME="fuchsbau" && \
    RELEASE_DIR=`ls -d _build/prod/rel/$APP_NAME/releases/*/` && \
    mkdir /export && \
    tar -xf "$RELEASE_DIR/$APP_NAME.tar.gz" -C /export

# Deplyment
FROM pentacent/alpine-erlang-base:latest

# Copy and extract .tar.gz 
COPY --from=build /export/ .


# Set default entrypoint and command
ENTRYPOINT ["/opt/app/bin/fuchsbau"]
CMD ["foreground"]

This Dockerfile will create a new Image with Alpine as base. Let’s test it with a changed value for our “FUCHSBAU_OWNER” environment variable.

docker build -t fuchsbau .
docker run -e FUCHSBAU_OWNER=FridayIsMyDay fuchsbau

If you want to get the sources for this example, feel free: https://github.com/opHASnoNAME/elixir-docker-envs.

To host your application i would suggest DigitalOcean, works like a charm for my projects.

1st Published at Medium: Visit

How to publish UDP ports with Docker-Swarm

Arne — Thu, 26 Oct 2017 16:09:02 +0000

I am sure you know those requests, where you think: Hey no problem, i have done this thousand times, give me 5 minutes and it will work.

The same happened last week, when a teammate from zisops asked me to publish a UDP Port for a already running Docker Swarm Service. I have done this 100 times before for "normal" Docker Containers. But hey.. Docker Swarm can't handle this with:

-p 500:500/udp

Ok next step: RTFM but hey.. nothing about publishing a UDP Port for a service. Next try: Start to google.. Gotcha, found a solutions in an Github Issue from the Moby Project. So if you ever have to use UDP with Docker-Swarm remember this small article, and buy me a beer :-)

Solution

docker service create --name my_service \
    --publish mode=host,published=500,target=5004,protocol=udp \
    my_image

Original post in german: https://www.geekpub.de/2017/10/docker-swarm-udp-port-freigeben-fuer-einen-service/

Auth0 with Laravel 5.4 including saving the user

Arne — Thu, 10 Aug 2017 05:17:41 +0000

Disclaimer: Sorry for my english, not my mother tongue.. but i try my best.

Vacation after I managed to recover from Wacken 2017, the rest of the time is once again put into some fun with PHP and Laravel.

For a small project where a user administration is necessary, I was recently recommended by a colleague the service auth0.

What is auth0? It takes you all the effort to include various authentication providers such as Facebook, Google, Github, and so on. In the free version somewhat limited, but for a beginning more than sufficient (7000 User, 2 Provider).

Well now it was time to connect my Laravel application to auth0. The documentation is quite good (also related to the integration in Laravel) but when i tried to save the users into my own databases, the examples did not work. So if you have to accomplish the same task, or if i have to do it again.. here comes a small tutorial.

As a starting point, let’s assume:

Laravel 5.4 application works, and the migrations for the users (are already there) were executed
Auth0 account is set up

For the most times we will follow their documentation, with some small changes.

Integrate Auth0 in Laravel 5.4

In your project directory, we install us first the right package for Auth0

composer require auth0/login:"~5.0"

Now change your: /config/app.php

<?php
'providers' => array(
    Auth0\Login\LoginServiceProvider::class,
);
'aliases'array( 
    'Auth0' => Auth0\Login\Facade\Auth0::class 
);

Now we differ from the documentation, since we want to save our users in our databases. For this we build a UserRepository: /app/Repository/UserRepository.php with the following content:

<?php
namespace App\Repository;
use Auth0\Login\Contract\Auth0UserRepository as Auth0UserRepository;
use App\User as User;
class UserRepository implements Auth0UserRepository {
    public function getUserByDecodedJWT($jwt)
    {
        $jwt->user_id = $jwt->sub;
        return $this->upsertUser($jwt);
    }
    public function getUserByUserInfo($userInfo)
    {
        return $this->upsertUser((object) $userInfo['profile']);
    }
    /**
     * Check if user is in database, if not create.
     * 
     * @return User 
     */
    protected function upsertUser($profile) {
        $user = User::where("auth0id", $profile->user_id)->first();
        // create user if not in database
        if ($user === null) {
            $user = new User();
            $user->email = $profile->email;
            $user->auth0id = $profile->user_id;
            $user->name = $profile->name;
            // random password, we dont need it 
            $user->password = md5(time());
            $user->save();
        }
        return $user;
    }
    public function getUserByIdentifier($identifier)
    {
        //Get the user info of the user logged in (probably in session)
        $user = \App::make('auth0')->getUser();
        if ($user === null) return null;
        // build the user
        $user = $this->getUserByUserInfo($user);
        // it is not the same user as logged in, it is not valid
        if ($user && $user->id == $identifier) {
            return $user;
        }
    }
}

We do not set a password, the login is managed via auth0. What you still have to do is to insert the column auth0id in the user table, which of course, Laravel does not know before. The ID is used for the assignment.

The repository will now be registered in our AppServiceProvider: /app/Providers/AppServiceProvider.php

<?php
   /**
     * Register any application services.
     *
     * @return void
     */
    public function register()
    {
        $this->app->bind(
          Auth0UserRepositoryContract::class,
          UserRepository::class // das unser Repository - in der Doku wird Auth0Repo.. genutzt
        );
    }

Now it goes to the configuration. For this, we create the necessary configuration files, and also set the necessary routes for the service.

php artisan vendor:publish

File: /routes/web.php

Route::get('/login', ['as' => 'login', 'uses' => 'IndexController@login']);
Route::get('/logout', ['as' => 'logout', 'uses' => 'IndexController@logout']);
Route::get('/callback', '\Auth0\Login\Auth0Controller@callback');

The functions in the IndexController are as follows:

<?php

    public function index() 
    {
        $isLoggedIn = \Auth::check();

        return view('index')
              ->with('isLoggedIn', $isLoggedIn)
              ->with('user', \Auth::user());
    }

    public function login()
    {
        return \App::make('auth0')->login();
    }

    public function logout()
    {
        \Auth::logout();
        return \Redirect::intended('/');
    }

So it is already clear, how you later access the user data: \Auth::user(); Now we need to include auth0 as userprovider: /app/config/auth.php

    'providers' => [
        'users' => [
            'driver' => 'auth0',
        ],

Settings for the application

To get the necessary parameters for our configuration, we visit the website of Auth0. You have to create a basic web application there, and set the following URLs.

-Callback: http://dein-server.de/callback
-Logout: http://dein-server.de/logout

The whole should also work with “localhost if you do not have a domain yet.

For the configuration of the applications we still need:

-Domain
-ClientID
-ClientSecret

In your /app/config/laravel-auth0.php you now either set directly the values or use your “.env file.

Testing auth0

If you now call http://dein-server.de/login, you should seea login and then in your user object the corresponding data as well as an entry in the Users database table.

What is missing in the documentation: How do I actually protect a route? Simply add the Auth-Middleware to your route.

# monitor (protected)
Route::get('/monitor/id/{id?}/{title?}', 'MonitorController@show')
    ->middleware('auth')
    ->name('monitor.show');

Route::get('/monitor/create', 'MonitorController@create')
    ->middleware('auth')
    ->name('monitor.create');

Thats it! — if you prefer a german version of this visit my GeekPub

Adventure with Docker: Conflicts with UIDs of the container and the host

Arne — Sun, 16 Jul 2017 05:19:19 +0000

There are so days when you deal with problems you would never have expected.
We had the goal to operate an ElasticSearch Cluster in our Docker-Swarm. After some time, a finished image was found which makes this halfway possible (problematic is the discovery of other nodes, just as a hint).

To ensure that all containers in the ElasticSearch Clusters have all the data on the different Docker Swarm Nodes, we use GlusterFS as a distributed file system.

The whole construct was running very well, until we have noticed: Hey NTP is not running on our servers. Ok quickly thrown into the Ansible Playbook and run on the hosts. BAM! The ElasticSearch reports: I do not like you any more dude…

Now, of course, you wonder what happened here? A look at the hosts showed that now all the files of ElasticSearch belong to the user “systemd-timesync” .. eh? The first guess was, of course, the somewhat worn GlusterFS.

The actual error is in principle not an error. If you add a host volume to a container, all the files of the container are created with the user running inside the container. Normally many containers run with the user root, which causes no problems, but ElasticSearch after version 5.x does not run as root…

Now comes the chance in the game, the Image for the ElasticSearch is based on Alpine, here the users begin with the UID 100, unfortunately exactly the UID which has now used by our little friend the “systemd-timesync” user.

Unfortunately, a really good solution was not available for us, but as a Workaround, the Dockerfile was changed, and we assigned the ElasticSearch user a UID 1200+. Now the files are created with this UID.

If you find a more clever solution, I’m looking forward to a comment, and if you prefer this text in german, you can find it at the Geek Pub.

How to deploy a Laravel 5.4 (PHP) Application to Microsoft Azure WebApp service

Arne — Sat, 01 Jul 2017 16:41:56 +0000

You’ve certainly noticed it, there was just a little silence.
This is mainly due to the change of my employer, after 20 years in an energy company, it has left me in a smaller german “Startup (ZENNER IoT Solutions), which is costs me some time at the moment (which is completely OK for me!)â€Š–â€Šwhat we do? Building a great IoT Platform!

There we are happy to participate in the Microsoft BizSpark program,
which means some resources, such as testing Microsoft’s Azure Cloud.
Now my task was to bring a PHP application based on the Laravel 5.4 fFramework to a web app from Microsoft. Microsoft WebApps are their PaaS offer when it comes to hosting .. Surprise: Web applications.

The base of the WebApps is an iiS server, which makes the whole story a bit more complex than deploying to a 08/15 Apache with PHP .. the iiS is more suitable for Microsoft’s languages like ASP.NET etc. Well, however, nothing is impossible. Here is a short story on how to deploy a Laravel App to Microsoft Azure WebApp Service (Spoiler: It runs very well, if it runs!).

Let’s assume that your application is finished, and you have already set up
an Azure WebApp (which is trivial). Now we have to do some preparations.

PHP Setting for the Azure WebApp

Let â€˜s start with a new PHP version for our WebApp. Fortunately, Microsoft supports PHP 7.1, which we also now select in the Applicationsâ€Š–â€ŠSettings.

Configure the Azure WebApp for Laravel

Further on the same page, here a few variables we have to set, so that Azure plays well with Laravel.

-SCM_REPOSITORY_PATH to: ..\repository
-SCM_TARGET_PATH to: ..

Now it is time to change the entry point to our website, the default for Azure is “/site/index.html”, but Laravel want’s : “site/public”. But hey we can also do this on the same page.

The matching counterpart to the Apache .htaccess, is the iiS “web.config file, fortunately Laravel ships with a default one, we can use for our application. So no work for us.

Composer extension for WebApp

The settings now fit, what makes life even easier for us is an extension that you can simply activate, so you make sure that “composer is executed during the deployment and you do not have to do this manually.

Deployment “pipeline for our Azure WebApp

The last step: Somehow we must get our application to Azure. Personally, I prefer the deployment via GIT, we clone some kind of a local repository and copy our code into it. After a short “git push we are online, Hooray.

Setting up a VM with PHP 7.0, MySQL, Apache 2 and PhpMyAdmin on Ubuntu

Arne — Sun, 16 Apr 2017 08:46:35 +0000

It is for me once again time to deal with PHP, yes, some may cry out, but I love the language. Since I wanted to use again a “real VM for it, instead of a Docker container, here is a small tutorial for creating a VM using Virtualbox which can do the following.

VM "Features"

PHP 7.x
MySQL 5.x
Apache 2.x
Ubuntu 16.04.x
PHPMyAdmin
Shared Folder mit dem Mac

The first step is to get our Ubuntu 16.04.x image: Download.

So we create a VM in the Virtualbox GUI, which I always like the best is the use of a network bridge for ssh accessâ€Š–â€Šso the VM gets an internal IP from my network and would be in principle also accessible by NAT from the outside.

I think, the installation of the Ubuntu server is nothing I need not describe here, only important: At the end of the installation tick the OpenSSH server so that we can work well with our VM via SSH.

If you’re still not sure how to do this, or just want to be put into a kind of trance, i have uploaded the whole as a screencast (without comments) on Youtube, 6:30 relaxing (is a bit slow, because in the background Elder Scrolls online is doing his install job).

Screencast @ Youtube

To get the private IP of the fresh VM, just login and use “ifconfig as command.

From now on, we are using an SSH connection from our computer to Virtualbox VM.

ssh ssh user@vm-ip

Installation of packages

Continue with the installation of the packages we need for our small server: PHP, Apache and MySQL, PHPMyADMIN (to manage databases from the browser).

sudo apt-get update
sudo apt-get install apache2 libapache2-mod-php7.0 php7.0 php7.0-mysql mysql-server
sudo apt-get install phpmyadmin php-mbstring php-gettext

Please note the “root password, which is set when the MySQL server is installed. So we can connect later to the databasesâ€Š–â€Šthink on a development server we do not need dedicated users.

Some, or most PHP frameworks require mod rewrite to rewrite the URL, so:

sudo a2enmod rewrite

For the VirtualBOX Guest-extension (which we need to share us with the Mac folder) please set up the following packages before:

sudo apt-get install build-essential module-assistant

Next: Mount the “CD into the VirtualBox, there are the guest extensions on it. To do this, open the window with the virtualbox and click “Devicesâ€Š–â€ŠInsert Guest Additions .. at the top of the menu bar.

Back in the terminal these are now installed:

sudo mount /dev/sr0 /media/cdrom/
cd /media/cdrom
sudo ./VBoxLinuxAdditions.run

Time for a reboot of the box! To share a folder with your Mac (you’ll later put your code there), click Devices again, and open Shared Folders. Search for a suitable folder on your computer and a nice name, check options for Auto-Mount and Make Permanent. In my example I want to use the Temp directory in my personal folder.

Back in the terminal, we now only have to include the folder, I always mount it directly into the document root of the Apache web server.

sudo mount -t vboxsf Temp /var/www/html

We don’t want to repeat this on every reboot, so we put this stuff into “/etc/rc.local”:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
mount -t vboxsf Temp /var/www/html

Let’s create an index.php in our shared folder on our computer with the following content:

<?php phpinfo(); ?>

If we now open our Browser with the private ip of our VM, we should the the PHP informations:

To open the PHPMyAdmin just hit the URL: http://IP/phpymadmin

That’s itâ€Š–â€Šsorry for my english, but not my mother tongue. Hope someone finds this short tutorial useful.

If you want to read more from my stuff, just follow me on Twitter: @ophasnoname

In only 2 1/2 days to Docker expert: books, videos and tutorial recommendations

Arne — Fri, 17 Mar 2017 15:39:08 +0000

As you have already noticed, I work in a shop which is using Docker the whole day…

Especially for new colleagues or not quite “docker insiders”, often comes the desire in me to show them a page or whatever to point them to the most important resources about Docker for rookies.

Certainly lists with links are out there a lot, but unfortunately those are kind of: We collect everything which has to do something with Docker.
In the end I am however the opinion that one should rather check a few good articles, videos or books, and study them in detail!

Later if you want to go further, or if you want to look like the Docker Super-Hero, you know where to get more stuff to read!

Basicsâ€Š–â€ŠWhat is docker and what is not

A Beginner-Friendly Introduction to Containers, VMs and Docker: Unfortunately, not 100% up-to-date, but for me, one of the best articles to enter the Docker universe. The Likes speak for themselves. [30 minutes]
Docker in Production - A History of Failure & Docker in Production: A History of Failure and the Update: I also like critical articles, these two have burned into my brain. I am still convinced by Docker and can not share many points, but you should also be aware of the bad things. Please also take a look at the comments! [30 minutes]
The Documentation: Have you worked through the 3 articles? There are still a few questions about Docker?! RTFM. Docker's documentation is fairly up-to-date and comprehensive.

Training

If you now have an approximate plan of what it's all about, you might want to get a little deeper into the matter. This is how I feel when a topic really interests me. Here you can also find 1000+ tutorials, books, videos, courses with which you could deal .. but who wants to do it all? So we restrict ourselves to a link for each category!

The Video - Docker fundamentals: basics, storage, networkingâ€Š–â€ŠIntroduction to Docker: Just 50 minutes packed with information, really also something for beginners. [50 minutes]
The Tutorial - A Practical Guide for Docker: Interactive tutorial, very nicely done and extensive (free!). My recommendation number 1! Unfortunately, a registration is necessary before … nothing is completely free in this world [120 minutes]
The Course - The Complete Docker Course for DevOps and Developers: Offered by udemy for just 25 â‚¬. 4.5 hours of pure knowledge on the subject! [4.5 hours]
The Book - The Docker Book: Why this? It is continuously updatedâ€Š–â€Šwhat is really important. There is, of course, only an e-book, but I guess that should not be a problem. [2 days..?]

If you now understand what it is, one should also stay up to date. So here is my newsletter recommendation.

Docker Weekly: In fact, the only one I read regularly. Why? Well news directly from the makers and a comprehensive list of links with new tutorials and articles (the Captain Corner)

Ask something …

When reading and trying out, you will surely encounter problems and questions, but where should they be placed?

Stackoverflow: Go to Stackoverflow and use the Docker tag!

Thats all, again sorry for my poor english .. i will try to work on it!

ok One more thing!

I would love to set up my own newsletter, around the topics of DevOps.
Maybe you are so fond of supporting me and just register?
You will get some fresh links and news every 2 Weeks (if we get enough nice people).

Let's call the Newsletter “DevOps Newsletter which has NoName!”

Latest DevOps Blogs, Boards and Books to check out today

Arne — Sat, 11 Mar 2017 06:41:49 +0000

Update 03-16-2017: Add one new Blog and Twitter Accounts, thanks for your support!

I am currently working in the "DevOps" areaâ€Š. Since I originally looked after small cute PHP applications, this was of course, somehow a jump into the cold water.

Thanks to the Internet, there is enough material for reading , to deal with the subject. If you are interested, whats in my opinion are must be read blogs, pages and books on the subject, feel free and read this little post in peace.

I've also taken the trouble for you, and listed only the links, where somebody is still alive. In any case, all this links are followed by myself. Except for the Blog of “otto.de”all are in English language.

Blogs with fresh stuff

Boards

Podcasts

Newsletters

Slack Channels for DevOps

Want to chat to some other Op Folks?

Books to read

Yes if you buy one from those link, i will get some pennies…

Twitter

Adrian Cockcroft Architect @ Netflix
Martin Fowler ThoughtWorks
Gene Kim Co-Authorâ€Š–â€ŠThe Phoenix Project
Adam Jacob Co-Founder from Chef
Andi Mann Blogger devops.com
Andrew Hay OpenDNS Manager
Carl Quinn Software-Architect Riot Games
David Mortman Security-Architect at Dell
Charity Majors Engineer/cofounder at @honeycombio, formerly Parse/Facebook)

You know more good Blogs, Podcasts, Links, Books or whatever .. please feel free to leave a comment or use this form to suggest a Link:

Suggest new Link: Google Form

By the way, from time to time I will create some more of those, so follow me on twitter: @ophasnoname

Originally posted on Medium