DEV Community: Aleksi Holappa

My web development environment (Alacritty, Tmux, Neovim)

Aleksi Holappa — Sat, 02 Jan 2021 19:24:04 +0000

This article is based on Neovim 0.4.4 without the new LSP!

During first years of my software engineering career I noticed that most of my co-workers were using Fedora, Ubuntu, Manjaro etc. distros on their machines. That raised my interest enough to start exploring the opportunities on the Linux side of things, since before this I had only Windows or MacOS machines. I decided to first try out Fedora on my personal home PC and install it alongside of Windows 10. The first impressions were just like "Okay, this looks nice" and installed VSCode along with other tools I used back then. After a while I was intrigued of the productivity side of things. Some folks were using VSCode, Vim, Emacs etc. and I read a lot of articles of how using (Neo)Vim can boost your productivity and be blazing fast when working with code.

That was about 7-8 months ago, and during the last 6 months I've been tweaking my setup to become even more productive. Today I'm sharing my own development environment with you with all the dot-files so you can also try it out! For each tools I have listed the reasons I use that particular tool.

Note: This is the setup what I use everywhere and is my personal configuration

Background and what I do

As a software engineer I do web development for most of my time. This includes front- and backend development, Terraform scripts, CI configuration via yml-files etc. Below is a few key technologies what I use

Ruby, Type-/JavaScript, Elixir languages and related CLI tooling
Terraform (IaC)
Run Docker containers - These also include web applications, DB's, key-value storages etc
AWS and GCP tooling
Firefox and other browsers are under heavy usage

Fedora - OS

This was kind of obvious thing. I really like how every Fedora release brings something new on board and does it very well. So far I have not countered any bugs and the last upgrade from 32 to 33 went smoothly. The actual key things for me in Fedora was that it offers built-in OCI image support, is built for developers in mind and has many open source tools available. The Gnome desktop environment sits like a glove for me. Some say that it is slow and buggy, but personally I like it a lot.

Also, I use Nvidia graphics card and for some distros it is just pain to install any Nvidia related stuff, but for Fedora it just works and there are good community support and guides for it.

And as a Theme for Fedora I use the dark theme which comes along with fresh Fedora installation. I haven't installed any Gnome plugins since personally I'm okay with the default shortcuts for handling windows, moving between workspaces and arranging stuff in general.

Alacritty - Terminal Emulator

Before Alacritty I had only used the default terminal emulators on MacOS and Fedora, and also iTerm on MacOS. I came across to Alacritty on some blog post or article that I read half a year ago and thought this is nice maybe I should try it out, so I tried. The main selling point for me was that Alacritty is GPU accelerated terminal emulator which means it should be blazing fast. It is also cross-platform so I can use it everywhere (Linux, MacOS, Windows etc.).

Below you can find my Alacritty configuration file

`alacritty.yml`



# Configuration for Alacritty, the GPU enhanced terminal emulator.
shell:
  program: /bin/zsh
  args:
    - -l
    - -c
    - "tmux"

window:
  # Window padding (changes require restart)
  #
  # Blank space added around the window in pixels. This padding is scaled
  # by DPI and the specified value is always added at both opposing sides.
  padding:
    x: 0
    y: 0

  # Spread additional padding evenly around the terminal content.
  dynamic_padding: true

  # Window decorations
  #
  # Values for `decorations`:
  #     - full: Borders and title bar
  #     - none: Neither borders nor title bar
  #
  # Values for `decorations` (macOS only):
  #     - transparent: Title bar, transparent background and title bar buttons
  #     - buttonless: Title bar, transparent background, but no title bar buttons
  decorations: full

  # Startup Mode (changes require restart)
  #
  # Values for `startup_mode`:
  #   - Windowed
  #   - Maximized
  #   - Fullscreen
  #
  # Values for `startup_mode` (macOS only):
  #   - SimpleFullscreen
  startup_mode: Windowed

scrolling:
  # Maximum number of lines in the scrollback buffer.
  # Specifying '0' will disable scrolling.
  history: 10000

  # Number of lines the viewport will move for every line scrolled when
  # scrollback is enabled (history > 0).
  multiplier: 3

# Font configuration (changes require restart)
font:
  normal:
    # Font family
    family: "JetBrains Mono"
    style: Regular

  # Point size
  size: 11.0

  # Offset is the extra space around each character. `offset.y` can be thought of
  # as modifying the line spacing, and `offset.x` as modifying the letter spacing.
  offset:
    x: 0
    y: 0

  # Glyph offset determines the locations of the glyphs within their cells with
  # the default being at the bottom. Increasing `x` moves the glyph to the right,
  # increasing `y` moves the glyph upwards.
  glyph_offset:
    x: 0
    y: 0

  # Thin stroke font rendering (macOS only)
  #
  # Thin strokes are suitable for retina displays, but for non-retina screens
  # it is recommended to set `use_thin_strokes` to `false`
  #
  # macOS >= 10.14.x:
  #
  # If the font quality on non-retina display looks bad then set
  # `use_thin_strokes` to `true` and enable font smoothing by running the
  # following command:
  #   `defaults write -g CGFontRenderingFontSmoothingDisabled -bool NO`
  #
  # This is a global setting and will require a log out or restart to take
  # effect.
  use_thin_strokes: true

# If `true`, bold text is drawn using the bright color variants.
draw_bold_text_with_bright_colors: true

# Colors (Gruvbox dark)
colors:
  primary:
    hard contrast: background = '0x1d2021'
    background: "0x282828"
    soft contrast: background = '0x32302f'
    foreground: "0xebdbb2"

  # Normal colors
  normal:
    black: "0x282828"
    red: "0xcc241d"
    green: "0x98971a"
    yellow: "0xd79921"
    blue: "0x458588"
    magenta: "0xb16286"
    cyan: "0x689d6a"
    white: "0xa89984"

  # Bright colors
  bright:
    black: "0x928374"
    red: "0xfb4934"
    green: "0xb8bb26"
    yellow: "0xfabd2f"
    blue: "0x83a598"
    magenta: "0xd3869b"
    cyan: "0x8ec07c"
    white: "0xebdbb2"

mouse:
  # Click settings
  #
  # The `double_click` and `triple_click` settings control the time
  # alacritty should wait for accepting multiple clicks as one double
  # or triple click.
  #double_click: { threshold: 300 }
  #triple_click: { threshold: 300 }
  # If this is `true`, the cursor is temporarily hidden when typing.
  hide_when_typing: false

selection:
  semantic_escape_chars: ',│`|:"'' ()[]{}<>'

  # When set to `true`, selected text will be copied to the primary clipboard.
  save_to_clipboard: false

cursor:
  # Cursor style
  #
  # Values for `style`:
  #   - ▇ Block
  #   - _ Underline
  #   - | Beam
  style: Block
  # If this is `true`, the cursor will be rendered as a hollow box when the
  # window is not focused.
  unfocused_hollow: true

# Live config reload (changes require restart)
live_config_reload: true

# Shell
#
# You can set `shell.program` to the path of your favorite shell, e.g. `/bin/fish`.
# Entries in `shell.args` are passed unmodified as arguments to the shell.
#
# Default:
#   - (macOS) /bin/bash --login
#   - (Linux) user login shell
#   - (Windows) powershell
# shell:
# program: /usr/bin/zsh
#  args:
#    - --login

# Startup directory
#
# Directory the shell is started in. If this is unset, or `None`, the working
# directory of the parent process will be used.
working_directory: None

# Send ESC (\x1b) before characters when alt is pressed.
alt_send_esc: true

debug:
  # Display the time it takes to redraw each frame.
  render_timer: false

  # Keep the log file after quitting Alacritty.
  persistent_logging: false

  # Log level
  #
  # Values for `log_level`:
  #   - None
  #   - Error
  #   - Warn
  #   - Info
  #   - Debug
  #   - Trace
  log_level: Warn

  # Print all received window events.
  print_events: false

  # Record all characters and escape sequences as test data.
  ref_test: false

# Key bindings
#
# Key bindings are specified as a list of objects. Each binding will specify a
# key and modifiers required to trigger it, terminal modes where the binding is
# applicable, and what should be done when the key binding fires. It can either
# send a byte sequence to the running application (`chars`), execute a
# predefined action (`action`) or fork and execute a specified command plus
# arguments (`command`).
#
# Bindings are always filled by default, but will be replaced when a new binding
# with the same triggers is defined. To unset a default binding, it can be
# mapped to the `None` action.
#
# Example:
#   `- { key: V, mods: Control|Shift, action: Paste }`
#
# Available fields:
#   - key
#   - mods (optional)
#   - chars | action | command (exactly one required)
#   - mode (optional)
#
# Values for `key`:
#   - `A` -> `Z`
#   - `F1` -> `F12`
#   - `Key1` -> `Key0`
#
#   A full list with available key codes can be found here:
#   https://docs.rs/glutin/*/glutin/enum.VirtualKeyCode.html#variants
#
#   Instead of using the name of the keys, the `key` field also supports using
#   the scancode of the desired key. Scancodes have to be specified as a
#   decimal number.
#   This command will allow you to display the hex scancodes for certain keys:
#     `showkey --scancodes`
#
# Values for `mods`:
#   - Command
#   - Control
#   - Option
#   - Super
#   - Shift
#   - Alt
#
#   Multiple `mods` can be combined using `|` like this: `mods: Control|Shift`.
#   Whitespace and capitalization is relevant and must match the example.
#
# Values for `chars`:
#   The `chars` field writes the specified string to the terminal. This makes
#   it possible to pass escape sequences.
#   To find escape codes for bindings like `PageUp` ("\x1b[5~"), you can run
#   the command `showkey -a` outside of tmux.
#   Note that applications use terminfo to map escape sequences back to
#   keys. It is therefore required to update the terminfo when
#   changing an escape sequence.
#
# Values for `action`:
#   - Paste
#   - PasteSelection
#   - Copy
#   - IncreaseFontSize
#   - DecreaseFontSize
#   - ResetFontSize
#   - ScrollPageUp
#   - ScrollPageDown
#   - ScrollLineUp
#   - ScrollLineDown
#   - ScrollToTop
#   - ScrollToBottom
#   - ClearHistory
#   - Hide
#   - Quit
#   - ClearLogNotice
#   - SpawnNewInstance
#   - ToggleFullscreen
#   - None
#
# Values for `action` (macOS only):
#   - ToggleSimpleFullscreen: Enters fullscreen without occupying another space
#
# Values for `command`:
#   The `command` field must be a map containing a `program` string and
#   an `args` array of command line parameter strings.
#
#   Example:
#       `command: { program: "alacritty", args: ["-e", "vttest"] }`
#
# Values for `mode`:
#   - ~AppCursor
#   - AppCursor
#   - ~AppKeypad
#   - AppKeypad
key_bindings:
  # (Windows/Linux only)
  #- { key: V,        mods: Control|Shift, action: Paste            }
  #- { key: C,        mods: Control|Shift, action: Copy             }
  #- { key: Insert,   mods: Shift,         action: PasteSelection   }
  #- { key: Key0,     mods: Control,       action: ResetFontSize    }
  #- { key: Equals,   mods: Control,       action: IncreaseFontSize }
  #- { key: Add,      mods: Control,       action: IncreaseFontSize }
  #- { key: Subtract, mods: Control,       action: DecreaseFontSize }
  #- { key: Minus,    mods: Control,       action: DecreaseFontSize }
  #- { key: Return,   mods: Alt,           action: ToggleFullscreen }

  # (macOS only)
  #- { key: Key0,   mods: Command,         action: ResetFontSize    }
  #- { key: Equals, mods: Command,         action: IncreaseFontSize }
  #- { key: Add,    mods: Command,         action: IncreaseFontSize }
  #- { key: Minus,  mods: Command,         action: DecreaseFontSize }
  #- { key: K,      mods: Command,         action: ClearHistory     }
  #- { key: K,      mods: Command,         chars: "\x0c"            }
  #- { key: V,      mods: Command,         action: Paste            }
  #- { key: C,      mods: Command,         action: Copy             }
  #- { key: H,      mods: Command,         action: Hide             }
  #- { key: Q,      mods: Command,         action: Quit             }
  #- { key: W,      mods: Command,         action: Quit             }
  #- { key: F,      mods: Command|Control, action: ToggleFullscreen }

  - { key: Paste, action: Paste }
  - { key: Copy, action: Copy }
  - { key: L, mods: Control, action: ClearLogNotice }
  - { key: L, mods: Control, chars: "\x0c" }
  - { key: Home, mods: Alt, chars: "\x1b[1;3H" }
  - { key: Home, chars: "\x1bOH", mode: AppCursor }
  - { key: Home, chars: "\x1b[H", mode: ~AppCursor }
  - { key: End, mods: Alt, chars: "\x1b[1;3F" }
  - { key: End, chars: "\x1bOF", mode: AppCursor }
  - { key: End, chars: "\x1b[F", mode: ~AppCursor }
  - { key: PageUp, mods: Shift, action: ScrollPageUp, mode: ~Alt }
  - { key: PageUp, mods: Shift, chars: "\x1b[5;2~", mode: Alt }
  - { key: PageUp, mods: Control, chars: "\x1b[5;5~" }
  - { key: PageUp, mods: Alt, chars: "\x1b[5;3~" }
  - { key: PageUp, chars: "\x1b[5~" }
  - { key: PageDown, mods: Shift, action: ScrollPageDown, mode: ~Alt }
  - { key: PageDown, mods: Shift, chars: "\x1b[6;2~", mode: Alt }
  - { key: PageDown, mods: Control, chars: "\x1b[6;5~" }
  - { key: PageDown, mods: Alt, chars: "\x1b[6;3~" }
  - { key: PageDown, chars: "\x1b[6~" }
  - { key: Tab, mods: Shift, chars: "\x1b[Z" }
  - { key: Back, chars: "\x7f" }
  - { key: Back, mods: Alt, chars: "\x1b\x7f" }
  - { key: Insert, chars: "\x1b[2~" }
  - { key: Delete, chars: "\x1b[3~" }
  - { key: Left, mods: Shift, chars: "\x1b[1;2D" }
  - { key: Left, mods: Control, chars: "\x1b[1;5D" }
  - { key: Left, mods: Alt, chars: "\x1b[1;3D" }
  - { key: Left, chars: "\x1b[D", mode: ~AppCursor }
  - { key: Left, chars: "\x1bOD", mode: AppCursor }
  - { key: Right, mods: Shift, chars: "\x1b[1;2C" }
  - { key: Right, mods: Control, chars: "\x1b[1;5C" }
  - { key: Right, mods: Alt, chars: "\x1b[1;3C" }
  - { key: Right, chars: "\x1b[C", mode: ~AppCursor }
  - { key: Right, chars: "\x1bOC", mode: AppCursor }
  - { key: Up, mods: Shift, chars: "\x1b[1;2A" }
  - { key: Up, mods: Control, chars: "\x1b[1;5A" }
  - { key: Up, mods: Alt, chars: "\x1b[1;3A" }
  - { key: Up, chars: "\x1b[A", mode: ~AppCursor }
  - { key: Up, chars: "\x1bOA", mode: AppCursor }
  - { key: Down, mods: Shift, chars: "\x1b[1;2B" }
  - { key: Down, mods: Control, chars: "\x1b[1;5B" }
  - { key: Down, mods: Alt, chars: "\x1b[1;3B" }
  - { key: Down, chars: "\x1b[B", mode: ~AppCursor }
  - { key: Down, chars: "\x1bOB", mode: AppCursor }
  - { key: F1, chars: "\x1bOP" }
  - { key: F2, chars: "\x1bOQ" }
  - { key: F3, chars: "\x1bOR" }
  - { key: F4, chars: "\x1bOS" }
  - { key: F5, chars: "\x1b[15~" }
  - { key: F6, chars: "\x1b[17~" }
  - { key: F7, chars: "\x1b[18~" }
  - { key: F8, chars: "\x1b[19~" }
  - { key: F9, chars: "\x1b[20~" }
  - { key: F10, chars: "\x1b[21~" }
  - { key: F11, chars: "\x1b[23~" }
  - { key: F12, chars: "\x1b[24~" }
  - { key: F1, mods: Shift, chars: "\x1b[1;2P" }
  - { key: F2, mods: Shift, chars: "\x1b[1;2Q" }
  - { key: F3, mods: Shift, chars: "\x1b[1;2R" }
  - { key: F4, mods: Shift, chars: "\x1b[1;2S" }
  - { key: F5, mods: Shift, chars: "\x1b[15;2~" }
  - { key: F6, mods: Shift, chars: "\x1b[17;2~" }
  - { key: F7, mods: Shift, chars: "\x1b[18;2~" }
  - { key: F8, mods: Shift, chars: "\x1b[19;2~" }
  - { key: F9, mods: Shift, chars: "\x1b[20;2~" }
  - { key: F10, mods: Shift, chars: "\x1b[21;2~" }
  - { key: F11, mods: Shift, chars: "\x1b[23;2~" }
  - { key: F12, mods: Shift, chars: "\x1b[24;2~" }
  - { key: F1, mods: Control, chars: "\x1b[1;5P" }
  - { key: F2, mods: Control, chars: "\x1b[1;5Q" }
  - { key: F3, mods: Control, chars: "\x1b[1;5R" }
  - { key: F4, mods: Control, chars: "\x1b[1;5S" }
  - { key: F5, mods: Control, chars: "\x1b[15;5~" }
  - { key: F6, mods: Control, chars: "\x1b[17;5~" }
  - { key: F7, mods: Control, chars: "\x1b[18;5~" }
  - { key: F8, mods: Control, chars: "\x1b[19;5~" }
  - { key: F9, mods: Control, chars: "\x1b[20;5~" }
  - { key: F10, mods: Control, chars: "\x1b[21;5~" }
  - { key: F11, mods: Control, chars: "\x1b[23;5~" }
  - { key: F12, mods: Control, chars: "\x1b[24;5~" }
  - { key: F1, mods: Alt, chars: "\x1b[1;6P" }
  - { key: F2, mods: Alt, chars: "\x1b[1;6Q" }
  - { key: F3, mods: Alt, chars: "\x1b[1;6R" }
  - { key: F4, mods: Alt, chars: "\x1b[1;6S" }
  - { key: F5, mods: Alt, chars: "\x1b[15;6~" }
  - { key: F6, mods: Alt, chars: "\x1b[17;6~" }
  - { key: F7, mods: Alt, chars: "\x1b[18;6~" }
  - { key: F8, mods: Alt, chars: "\x1b[19;6~" }
  - { key: F9, mods: Alt, chars: "\x1b[20;6~" }
  - { key: F10, mods: Alt, chars: "\x1b[21;6~" }
  - { key: F11, mods: Alt, chars: "\x1b[23;6~" }
  - { key: F12, mods: Alt, chars: "\x1b[24;6~" }
  - { key: F1, mods: Super, chars: "\x1b[1;3P" }
  - { key: F2, mods: Super, chars: "\x1b[1;3Q" }
  - { key: F3, mods: Super, chars: "\x1b[1;3R" }
  - { key: F4, mods: Super, chars: "\x1b[1;3S" }
  - { key: F5, mods: Super, chars: "\x1b[15;3~" }
  - { key: F6, mods: Super, chars: "\x1b[17;3~" }
  - { key: F7, mods: Super, chars: "\x1b[18;3~" }
  - { key: F8, mods: Super, chars: "\x1b[19;3~" }
  - { key: F9, mods: Super, chars: "\x1b[20;3~" }
  - { key: F10, mods: Super, chars: "\x1b[21;3~" }
  - { key: F11, mods: Super, chars: "\x1b[23;3~" }
  - { key: F12, mods: Super, chars: "\x1b[24;3~" }
  - { key: NumpadEnter, chars: "\n" }
```

# ZSH and [Oh my Zsh](https://ohmyz.sh/)

For a shell I use ZSH, which I picked up when I was using MacOS and have used it since. It is stable, easy to configure and the Oh My Zsh is wonderful with its themes. Can't really say anything from the other like Fish or Bash, but ZSH has been working fine so I see no reason to change to other!

I'm not going to paste my `.zshrc` here since it has so little modifications it. Basically I have just changed the theme to agnoster, set my default editor to Neovim and put some aliases which you can find below.

```
alias zshconfig="nvim ~/.zshrc"
alias ohmyzsh="nvim ~/.oh-my-zsh"
alias ls="ls -al"
alias up="docker-compose up"
alias run="docker-compose run"
alias n="nvim"
```

# [ASDF](https://asdf-vm.com/#/) - Language management tool

This is relatively new tool for me also, I recently find it out since I had problem installing the newest version of Elixir to my machine and the official Fedora repositories had only up to 1.10 or so.

I use this tool to manage all the versions of the programming languages and runtimes I use. It has a simplistic CLI and has all the configuration options I need. For example if you have installed Nodejs versions 12 and 14, you can temporarily set version 12 to one shell session if you are working with older project which uses version 12. This makes it really easy and handy to use in different situations. It supports all the major programming languages and I have had 0 issues with it. I Definitely recommended to use this piece of software!

# [TMUX](https://github.com/tmux/tmux) - Multiplexer

Tmux is as far as I know the most used terminal multiplexer (I base this assumption to Github stars). It comes pre-installed in Fedora and has handy shortcuts. On the first sight when I started to use Tmux I was kind of confused how this should make my life easier since I can have just two separate terminal windows, but after a while I realised that I can split the terminal window in to multiple terminal windows which you can detach and let run in the background, move between Tmux screens/pages etc.

#### **`.tmux.conf`**
```
# Scroll up with mouse
set-option -g mouse on
set -g set-clipboard on

# Scrollback buffer
set -g history-limit 10000

# tmux display things in 256 colors
set-option -g default-terminal "screen-256color"
set-option -g terminal-overrides "screen-256color"

# allow for navigating between words with option
set-window-option -g xterm-keys on

# command delay? We don't want that, make it short
set -g escape-time 10

# Allow the arrow key to be used immediately after changing windows
set-option -g repeat-time 0

# Set window notifications
set -g monitor-activity on
set -g visual-activity on

# Update files on focus (using for vim)
set -g focus-events on

# Status update interval
set -g status-interval 1

# Reduce time to wait for Escape key. You'll want this for neovim.
set-option escape-time 40

# Option to clear histroy
bind -n C-k clear-history
bind-key -n C-l send-keys 'C-l'
bind-key R switch-client -r

######### DISPLAY ##########
set -g renumber-windows on    # renumber windows when a window is closed

######### THEME  ##########
set -g status-bg colour237
set -g status-fg colour246
set-option -g pane-active-border-style fg=colour239
set-option -g pane-border-style fg=colour237
set -g mode-style fg=colour235,bg=colour66
set-option -g message-style bg=colour66,fg=colour235

set-option -g status-justify "left"
set-option -g status-left-style none
set-option -g status-left-length "80"
set-option -g status-right-style none
set-option -g status-right-length "80"

set-option -g status-right '#(gitmux -cfg ~/.gitmux.conf "#{pane_current_path}") %H:%M:%S '
set-window-option -g window-status-separator " "
set-window-option -g window-status-current-format "#[fg=colour66]#W"
set-window-option -g window-status-format "#W"

# Allow us to reload our Tmux configuration while
# using Tmux
bind r source-file ~/.tmux.conf \; display "Reloaded!"

if "test ! -d ~/.tmux/plugins/tpm" \
   "run 'git clone https://github.com/tmux-plugins/tpm ~/.tmux/plugins/tpm && ~/.tmux/plugins/tpm/bin/install_plugins'"

set -g @plugin 'casonadams/tmux-vi-navigation'
set -g @yank_selection 'clipboard'
set -g @yank_selection_mouse 'clipboard'

run -b '~/.tmux/plugins/tpm/tpm'

 aleksi@localhost  ~  n
 aleksi@localhost  ~  cat .tmux.conf
# Scroll up with mouse
set-option -g mouse on
set -g set-clipboard on

# Scrollback buffer
set -g history-limit 10000

# tmux display things in 256 colors
set-option -g default-terminal "screen-256color"
set-option -g terminal-overrides "screen-256color"

# allow for navigating between words with option
set-window-option -g xterm-keys on

# command delay? We don't want that, make it short
set -g escape-time 10

# Allow the arrow key to be used immediately after changing windows
set-option -g repeat-time 0

# Set window notifications
set -g monitor-activity on
set -g visual-activity on

# Update files on focus (using for neovim)
set -g focus-events on

# Status update interval
set -g status-interval 1

# Reduce time to wait for Escape key. You'll want this for neovim.
set-option escape-time 40

# Option to clear histroy
bind -n C-k clear-history
bind-key -n C-l send-keys 'C-l'
bind-key R switch-client -r

set -g renumber-windows on    # renumber windows when a window is closed

set -g status-bg colour237
set -g status-fg colour246
set-option -g pane-active-border-style fg=colour239
set-option -g pane-border-style fg=colour237
set -g mode-style fg=colour235,bg=colour66
set-option -g message-style bg=colour66,fg=colour235

set-option -g status-justify "left"
set-option -g status-left-style none
set-option -g status-left-length "80"
set-option -g status-right-style none
set-option -g status-right-length "80"

set-option -g status-right '#(gitmux -cfg ~/.gitmux.conf "#{pane_current_path}") %H:%M:%S '
set-window-option -g window-status-separator " "
set-window-option -g window-status-current-format "#[fg=colour66]#W"
set-window-option -g window-status-format "#W"

# Allow us to reload our Tmux configuration while
# using Tmux
bind r source-file ~/.tmux.conf \; display "Reloaded!"

if "test ! -d ~/.tmux/plugins/tpm" \
   "run 'git clone https://github.com/tmux-plugins/tpm ~/.tmux/plugins/tpm && ~/.tmux/plugins/tpm/bin/install_plugins'"

set -g @plugin 'casonadams/tmux-vi-navigation'
set -g @yank_selection 'clipboard'
set -g @yank_selection_mouse 'clipboard'

run -b '~/.tmux/plugins/tpm/tpm'
```

# [Neovim](https://github.com/neovim/neovim) - Editor

Along my studies and career I have used multiple different editors from CodeBlocks(yes, we used this on our first programming course for C) to VSCode and none of those have ever felt so comfortable to use as Neovim. At some point of first thoughts about Vim I decided to try it out and shortly after I realised that the learning curve is steep and went back to VSCode. After reading few articles of Neovim, I decided that now is the time to make the transition from VSCode to Neovim. First I struggled with the different modes, movement etc. but after a while I get used to it and nowadays if I switch back to VSCode or to some other editor, I'm totally lost because they don't have any of the remappings and shortcuts what I have when I use Neovim.

## Plugins

For managing plugins I use the `vim-plug` since it is minimalistic and supported among the plugin developers. Below is a list of plugins from my `init.vim` and every plugin or plugin group has a comment above which explains what it brings to the table.

#### **`init.vim - plugins section`**
```
call plug#begin()
  " Git status icons
  Plug 'airblade/vim-gitgutter'

  " Configure editor
  Plug 'editorconfig/editorconfig-vim'

  " Tab through coc
  Plug 'ervandew/supertab'

  " Fuzzy finder
  Plug 'junegunn/fzf', { 'do': { -> fzf#install() } }
  Plug 'junegunn/fzf.vim'

  " Buffer line on the bottom
  Plug 'itchyny/lightline.vim'
  Plug 'mengelbrecht/lightline-bufferline'

  " Theme and coloring
  Plug 'morhetz/gruvbox'

  " Code completion
  Plug 'neoclide/coc.nvim', {'branch': 'release'}

  " Icons for different file types
  Plug 'ryanoasis/vim-devicons'

  " Comment out lines
  Plug 'tpope/vim-commentary'

  " NERDTree
  Plug 'preservim/nerdtree'

  " Languages
  Plug 'elixir-editors/vim-elixir'
  Plug 'hashivim/vim-terraform'

  " Show error hints and highlights
  Plug 'vim-syntastic/syntastic'

  " Select multiple same items
  Plug 'mg979/vim-visual-multi', {'branch': 'master'}

  " Terraform completion and syntax highlight
  Plug 'juliosueiras/vim-terraform-completion'

  " Git blamer
  Plug 'APZelos/blamer.nvim'

  " Dockerfile syntax
  Plug 'ekalinin/Dockerfile.vim'

  " Markdown
  Plug 'iamcco/markdown-preview.nvim', { 'do': 'cd app && yarn install'  }

  " Cheat.sh integration
  Plug 'RishabhRD/popfix'
  Plug 'RishabhRD/nvim-cheat.sh'
call plug#end()
```

## The rest of the `init.vim`

Here is the rest of my Neovim configuration. There is a lot of remappings and configuration options. Some of them are my own and some of them are copy-pasted from various sources.

#### **`init.vim - rest of the file`**
```
" Settings
let mapleader = "\<Space>"
filetype plugin on
set completeopt=longest,menuone
set mouse=a
set nobackup
set nocompatible
set noswapfile
set nowritebackup
set number
set title
set wrap
setlocal wrap
set encoding=UTF-8

" persist
set undofile " Maintain undo history between sessions
set undodir=~/.vim/undodir

" Persist cursor
autocmd BufReadPost *
  \ if line("'\"") >= 1 && line("'\"") <= line("$") && &ft !~# 'commit'
  \ |   exe "normal! g`\""
  \ | endif

" SuperTab
let g:SuperTabMappingForward = '<S-tab>'
let g:SuperTabMappingBackward = '<tab>'

" Theme
syntax on
colorscheme gruvbox
set background=dark
set cursorline
set hidden
set list
set listchars=tab:»·,trail:·

" let buffers be clickable
let g:lightline#bufferline#clickable=1
let g:lightline#bfferline#shorten_path=1
let g:lightline#bufferline#min_buffer_count=1

" Git blamer
let g:blamer_enabled = 1
let g:blamer_delay = 200

" Remaps
nmap <Leader>p :Rg<CR>
nmap <Leader>h :History<CR>
nmap <Leader>n :NERDTreeToggle<CR>
imap jj <Esc>:w<CR>a
nmap <Leader>r :NERDTreeFocus<cr>R<c-w><c-p>

" Give more space for displaying messages.
set cmdheight=2

" Reduce updatetime
set updatetime=300

" Don't pass messages to |ins-completion-menu|.
set shortmess+=c

" Always show the signcolumn, otherwise it would shift the text each time
" diagnostics appear/become resolved.
set signcolumn=yes

function! s:check_back_space() abort
  let col = col('.') - 1
  return !col || getline('.')[col - 1]  =~# '\s'
endfunction

" Use <c-space> to trigger completion.
inoremap <silent><expr> <c-space> coc#refresh()

" Use <cr> to confirm completion, `<C-g>u` means break undo chain at current
" position. Coc only does snippet and additional edit on confirm.
if exists('*complete_info')
  inoremap <expr> <cr> complete_info()["selected"] != "-1" ? "\<C-y>" : "\<C-g>u\<CR>"
else
  imap <expr> <cr> pumvisible() ? "\<C-y>" : "\<C-g>u\<CR>"
endif

" GoTo code navigation.
nmap <silent> gd <Plug>(coc-definition)
nmap <silent> gy <Plug>(coc-type-definition)
nmap <silent> gi <Plug>(coc-implementation)
nmap <silent> gr <Plug>(coc-references)

" Use K to show documentation in preview window.
nnoremap <silent> K :call <SID>show_documentation()<CR>

function! s:show_documentation()
  if (index(['vim','help'], &filetype) >= 0)
    execute 'h '.expand('<cword>')
  else
    call CocAction('doHover')
  endif
endfunction

" Highlight the symbol and its references when holding the cursor.
autocmd CursorHold * silent call CocActionAsync('highlight')

" Launch NERDTree on start-up
autocmd VimEnter * NERDTree
let g:NERDTreeShowHidden=1

" Symbol renaming.
nmap <leader>rn <Plug>(coc-rename)

augroup mygroup
  autocmd!
  " Setup formatexpr specified filetype(s).
  autocmd FileType typescript,json setl formatexpr=CocAction('formatSelected')
  " Update signature help on jump placeholder.
  autocmd User CocJumpPlaceholder call CocActionAsync('showSignatureHelp')
augroup end

" Apply AutoFix to problem on the current line.
nmap <leader>qf  <Plug>(coc-fix-current)

inoremap " ""<left>
inoremap ' ''<left>
inoremap ( ()<left>
inoremap [ []<left>
inoremap { {}<left>
inoremap {<CR> {<CR>}<ESC>O
inoremap {;<CR> {<CR>};<ESC>O

" Use <TAB> for selections ranges.
" NOTE: Requires 'textDocument/selectionRange' support from the language server.
" coc-tsserver, coc-python are the examples of servers that support it.
nmap <silent> <TAB> <Plug>(coc-range-select)
xmap <silent> <TAB> <Plug>(coc-range-select)

" Add `:Format` command to format current buffer.
command! -nargs=0 Format :call CocAction('format')

" Add `:Fold` command to fold current buffer.
command! -nargs=? Fold :call     CocAction('fold', <f-args>)

" Add `:OR` command for organize imports of the current buffer.
command! -nargs=0 OR   :call     CocAction('runCommand', 'editor.action.organizeImport')

command! -nargs=0 Prettier :CocCommand prettier.formatFile

" Add (Neo)Vim's native statusline support.
" NOTE: Please see `:h coc-status` for integrations with external plugins that
" provide custom statusline: lightline.vim, vim-airline.
set statusline^=%{coc#status()}

CoC - Conquer of Completion

I'm using CoC since I'm still on Neovim 0.4.4, and when Neovim 0.5 gets released I'm going to move from CoC to the built-in LSP. Nevertheless, below is my coc-settings.json since there is some language specific configurations.

`coc-settings.json`



{
  "codeLens.enable": true,
  "elixir.pathToElixirLS": "~/.elixir-ls/release/language_server.sh",
  "languageserver": {
    "terraform": {
      "command": "terraform-lsp",
      "filetypes": [
        "terraform",
        "tf"
      ],
      "initializationOptions": {},
      "settings": {}
    }
  },
  "coc.preferences.formatOnSaveFiletypes": [
    "css",
    "markdown",
    "typescript",
    "javascript",
    "json",
    "elixir"
  ]
}

As seen, I have enabled to codelens, set manually path to ElixirLS since I have built it from source, configurations for Terraform LSP and set format on save for specific languages.

Conclusion

So there it is, my development configuration for general web development and how it looks. It is not complete yet, since I think there is always something you could do better or more efficiently, but till now it's good enough.

Hope this helped you to make your current development environment better, thanks!

Deploy Elixir + Phoenix to AWS ECS with GitLab CI and Terraform

Aleksi Holappa — Sun, 04 Oct 2020 15:57:07 +0000

On my first article we created a nice local development environment for Elixir and Phoenix with Docker and docker-compose. Now it's time to write some CI scripts and Terraform templates for our continuous integration and AWS infrastructure.

We are going to do things in the following order:

Make sure you have all ready and set
Create needed resources to AWS to push deploy-ready images to AWS (ECR, IAM users, policies)
Write .gitlab-ci.yml except deploy stage
Create rest of the resources (ECS, LB, RDS etc.) in order to run our application
Add deploy-stage to .gitlab-ci.yml

Note: This is not production grade infrastructure setup, but rather one which can be used as a base.

Prerequisites

GitLab

Phoenix application in GitLab repository
Pipelines & Container Registry enabled in GitLab project settings
Production ready Dockerfile for the Phoenix application/project

In case you don't have optimized Dockerfile for cloud deployments, read this.

AWS & Terraform

AWS CLI configured with your credentials (~/.aws/credentials)
Terraform CLI tool installed

Note: GitLab offers 400 minutes of CI time and container registry for free users (private and public repositories). (Updated 16.12. free CI minutes reduced from 2000 to 400)

Creating the initial resources

ECR stands for Elastic Container Registry, which will hold our Docker images. These Docker images are built and tagged in our CI script and pushed to this registry. In order to push to this registry we need to setup the registry itself, an IAM user with sufficient permissions and the CI script.

Once you have your AWS CLI and Terraform CLI setup, you will create a new directory for the terraform files.

$ mkdir myapp-terraform
$ cd myapp-terraform

wonderful, now we can start creating and defining our terraform templates.

`terraform.tf`

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

Here we are saying that we want to use AWS as our provider and the version of the AWS provider, pretty simple.

`main.tf`

provider "aws" {
  region = "eu-north-1" # Later please use the region variable
}

In main.tf we actually define the provider which is required by the definition in terraform.tf. Geographically my closest region is eu-north-1 so I'm using it. I advice to choose region based on your resource needs and geographical location. Note that all AWS services/resources my not be available in your region (e.g. CDN).If that's the case, you can define multiple providers with different regions.

`variables.tf`

variable "name" {
  type        = string
  description = "Name of the application"
  default     = "myapp"
}

variable "environment_name" {
  type        = string
  description = "Current environment"
  default     = "development"
}

Here we can define variables for our Terraform templates. These are very handy when you want to have different configurations in dev, staging and production environments. As seen, we define the name of the variable, what type the variable is, description for it and a default value. These variables can be populated or the default values can be replaced with *.tfvars files (e.g. dev.tfvars, stag.tfvars and so on). We will dive in to these later on and how to use environment specific variables.

`deploy_user.tf`

resource "aws_iam_user" "ci_user" {
  name = "ci-deploy-user"
}

resource "aws_iam_user_policy" "ci_ecr_access" {
  user = aws_iam_user.ci_user.name

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "ecr:GetAuthorizationToken"
      ],
      "Effect": "Allow",
      "Resource": "*"
    },
    {
      "Action": [
        "ecr:*"
      ],
      "Effect": "Allow",
      "Resource": "${aws_ecr_repository.myapp_repo.arn}"
    }
  ]
}
EOF

}

In order to push to the upcoming ECR, we need user with sufficient permissions to do that. Here we are defining a IAM user resource named ci_user and below that we are assigning policies to that user. The policy is in JSON-format and holds the versin and actual statements in an array. On the first statement we are giving specific permissions for our ci_user to get the authorization. On the second statement we are giving all the access rights to our ECR resource (write, read etc.). Read more on here.

`ecr.tf`

resource "aws_ecr_repository" "myapp_repo" {
  name = "${var.environment_name}-${var.name}"
}

This is our final resource till now. This may seem relatively simple, and it is, we are defining an ECR repository for our Docker images. The big part here is how we define the name for the repository. The repository name consists our environment and the application name, so the end result now would be development-myapp since those are our default values in variables.tf. Let's say that we want repositories for staging and production also. We can create corresponding .tfvars-files for them and override the default value which was set in variables.tf. Later on, we will be creating environment specific variable files, but for now I'm leaving them out for simplicity.

Now we have all resources defined and ready to be applied to AWS, but first we will take a look to Terraform workspaces.

Terraform workspaces

Workspaces in Terraform are kind of different environments where you want to deploy your infrastructure. They help you to keep track of the state of different workspaces/environments.

As default we only have one workspace named default. Lets create a now one!

$ terraform workspace new dev

Created and switched to workspace "dev"!

You're now on a new, empty workspace. Workspaces isolate their state,
so if you run "terraform plan" Terraform will not see any existing state
for this configuration.

Wonderful, now we have a new workspace named dev. Now we can try to plan our infrastructure. Note: For the next steps your credentials in the ~/.aws/credentials must be in place. Terraform will respect the AWS_PROFILE environment variable if set, otherwise it will use the default credentials from the earlier mentioned file.

$ terraform plan

What this does, it checks if already existing state file is present, compares your terraform template changes against this state file and prints nicely all the resources which are going to be changed, destroyed or created. When you run above command, you should see that we are creating 3 different resources, ECR repository, IAM user and IAM user policy.

That was the initial planning and checking that our properties for the resources are correct, and we didn't do any mistakes. Now it's time to actually apply this to our AWS environment.

$ terraform apply

Terraform will ask for permissions to perform the listed actions in workspace dev. As Terraform instructs, write yes and your resources will be created in a few seconds! During this, Terraform will create a new folder called terraform.tfstate.d. This folder holds subfolders of your applied workspaces. If you go to terraform.tfstate.d/dev/ you will find a .tfstate-file from there. This file holds the current state of the infrastructure for that particular workspace. This is kinda against the good practises since usually state files are stored somewhere else than locally or in git repository. Read more

In case you ran to an error, check your credentials and syntax of the templates.

Nice, now we have our ECR and IAM user for GitLab CI ready. We can start writing the CI script and start pushing some container images to our newly created repo!

GitLab CI

The GitLab CI script will be just basic yaml file with definitions for jobs, which image to use as a base image since it is Docker/container based CI runtime and many other things!

Things to do first. We need to think what to include to our CI pipeline. Below is a list of stages which we are including to our CI script. You can run multiple jobs parallel in each stage, e.g. we can run security scan and tests at the same time.

Test
Build
Release
deploy

At this point of the article, I'm going to cover the first three stages and we will revisit the deploy stage later when we have infrastructure ready for it (ECS, RDS etc). Note: .gitlab-ci.yml goes to the same repository where your Elixir + Phoenix application is, and also the CI environment variables go there.

Configuring CI environment

In order to push images to our ECR repository we need to have access to that repository.

Go to your AWS console, navigate to IAM dashboard, go to users and you should see our previously created CI user there. Click the CI user, select security credentials and create access key. This will create access key ID and secret access key for our CI user. Save these two values from the modal view. Other value you need is the ECR repository URL. You can find it by searching for Elastic Container Registry --> repositories and you should see earlier created repository and its URI, copy it.

Go to your GitLab repository and navigate settings --> CI/CD --> expand variables. We are going to add following variables:

AWS_REGION
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
DEV_AWS_ECR_URI

Note: Nowadays you can mask CI environment variables, I suggest you to do so for the access key ID, secret access key and ECR URI.

.gitlab-ci.yml

Once we have all the variables in place, we can start writing our script.

stages:
  - test
  - build
  - release

variables:
  AWS_DEFAULT_REGION: $AWS_REGION
  CONTAINER_IMAGE: $CI_REGISTRY/$CI_PROJECT_PATH:$CI_COMMIT_SHA
  AWS_CONTAINER_IMAGE_DEV: $DEV_AWS_ECR_URI:latest

First we define our stages and environment variables on top of the file. The variables are defined on top-level, so they are present in all jobs.

test:
  image: elixir:1.10.4-alpine
  services:
    - postgres:11-alpine
  variables:
    DATABASE_URL: postgres://postgres:postgres@postgres/myapp_?
    APP_BASE_URL: http://localhost:4000
    POSTGRES_HOST_AUTH_METHOD: trust
    MIX_ENV: test
  before_script:
    - mix local.rebar --force
    - mix local.hex --force
    - mix deps.get --only test
    - mix ecto.setup
  script:
    - mix test

Here is our test job which is responsible for running our unit tests. We are using Elixir Alpine based image as a runtime image for this job and defined Postgres as our service, since our tests require database connection. Variables are pretty self explanatory, DB accepts all the connections which are coming to it, MIX_ENV is set to test and all the environment variables for the application itself are present. Before running the actual tests, we install rebar and hex, fetch the dependencies and setup the database. After that we run tests!

build:
  stage: build
  image: docker:19.03.13
  services:
    - docker:19.03.13-dind
  before_script:
  - echo -n $CI_JOB_TOKEN | docker login -u gitlab-ci-token --password-stdin $CI_REGISTRY
  script:
    - docker build -f Dockerfile.production -t $CONTAINER_IMAGE .
    - docker push $CONTAINER_IMAGE
  only:
    - master

On this build job we actually build the Docker image which contains our application. We use official Docker image as our job runtime image and defined docker dind image as our service, since we are going to need some tools from it and we can run "docker in docker". The before_script one-liner will log in to GitLab container registry. On the script we are building our Docker image and pushing it to GitLab container registry (username/project-name). We are accessing our variables/environment variables with $-prefix notation. These can be variables we defined at the top of the CI script or environment variables which is placed in GitLab project settings. Find more on here As sidenote: This job is only run on master-branch.

release_aws_dev:
  stage: release
  variables:
    AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
    AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
  dependencies:
    - build
  image: docker:19.03.13
  services:
    - docker:19.03.13-dind
  before_script:
    - apk add --no-cache curl jq python3 py3-pip
    - pip install awscli
    - echo -n $CI_JOB_TOKEN | docker login -u gitlab-ci-token --password-stdin $CI_REGISTRY
    - $(aws ecr get-login --no-include-email)
  script:
    - docker pull $CONTAINER_IMAGE
    - docker tag $CONTAINER_IMAGE $AWS_CONTAINER_IMAGE_DEV
    - docker push $AWS_CONTAINER_IMAGE_DEV
  only:
    - master

In release job we are releasing/pushing the recently built image to our ECR repository. For that, we need to define AWS credentials for the job as environment variables for the job runtime. We are using same runtime image and service for this job as in build job. In before_script we are installing needed tools to run AWSCLI, logging in to the GitLab container registry and AWS ECR repository. Once we have logged in, in script we pull the image which we built in the build job, tag it with AWS ECR repository URL which contains the repository name and :latest-tag. After that we push the image to the ECR.

`.gitlab-ci.yml`

stages:
  - test
  - build
  - release

variables:
  AWS_DEFAULT_REGION: $AWS_REGION
  CONTAINER_IMAGE: $CI_REGISTRY/$CI_PROJECT_PATH:$CI_COMMIT_SHA
  AWS_CONTAINER_IMAGE_DEV: $DEV_AWS_ECR_URI:latest

test:
  stage: test
  image: elixir:1.10.4-alpine
  services:
    - postgres:11-alpine
  variables:
    DATABASE_URL: postgres://postgres:postgres@postgres/yourapp_?
    APP_BASE_URL: http://localhost:4000
    POSTGRES_HOST_AUTH_METHOD: trust
    MIX_ENV: test
  before_script:
    - mix local.rebar --force
    - mix local.hex --force
    - mix deps.get --only test
    - mix ecto.setup
  script:
    - mix test

build:
  stage: build
  image: docker:19.03.13
  services:
    - docker:19.03.13-dind
  before_script:
  - echo -n $CI_JOB_TOKEN | docker login -u gitlab-ci-token --password-stdin $CI_REGISTRY
  script:
    - docker build -f Dockerfile.production -t $CONTAINER_IMAGE .
    - docker push $CONTAINER_IMAGE
  only:
    - master

release_aws_dev:
  stage: release
  variables:
    AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
    AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
  dependencies:
    - build
  image: docker:19.03.13
  services:
    - docker:19.03.13-dind
  before_script:
    - apk add --no-cache curl jq python3 py3-pip
    - pip install awscli
    - echo -n $CI_JOB_TOKEN | docker login -u gitlab-ci-token --password-stdin $CI_REGISTRY
    - $(aws ecr get-login --no-include-email)
  script:
    - docker pull $CONTAINER_IMAGE
    - docker tag $CONTAINER_IMAGE $AWS_CONTAINER_IMAGE_DEV
    - docker push $AWS_CONTAINER_IMAGE_DEV
  only:
    - master

If you did everything correctly, your CI pipeline should pass and your image should show up in the ECR repository! Next up we will actually deploy the application to the cloud!

AWS ECS, RDS, LB and all other folks

Now when we have our images being built and pushed to ECR, it's time to look for the actual deployment. We will defining Terraform resources for database (RDS), container service which is responsible for running the application image (ECS Fargate), load-balancing ((A)LB), networking (subnets, route tables etc.), security groups, some .tfvars-files and output some values once we have applied templates to AWS.

Lets get started!

`main.tf`

...

resource "aws_vpc" "default" {
  cidr_block = "10.0.0.0/16"

  tags = {
    Environment = var.environment_name
  }
}

resource "aws_route_table" "public" {
  vpc_id = aws_vpc.default.id
}

resource "aws_route_table" "private" {
  vpc_id = aws_vpc.default.id
}

resource "aws_route_table_association" "public_subnet" {
  subnet_id      = aws_subnet.public.id
  route_table_id = aws_route_table.public.id
}

resource "aws_route_table_association" "private_subnet" {
  subnet_id      = aws_subnet.private.id
  route_table_id = aws_route_table.private.id
}

resource "aws_eip" "nat_ip" {
  vpc = true
}

resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.default.id
}

resource "aws_nat_gateway" "ngw" {
  subnet_id     = aws_subnet.public.id
  allocation_id = aws_eip.nat_ip.id
}

resource "aws_route" "public_igw" {
  route_table_id         = aws_route_table.public.id
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = aws_internet_gateway.igw.id
}

resource "aws_route" "private_ngw" {
  route_table_id         = aws_route_table.private.id
  destination_cidr_block = "0.0.0.0/0"
  nat_gateway_id         = aws_nat_gateway.ngw.id
}

Add VPC (Virtual Private Cloud) resource below the provider. This will be our cloud environment where all other resources will be in. Below it we are configuring a lot of network resources. We are defining route tables for two different subnets (which we will define next), one is exposed to public internet with internet gateway and the other one is private subnet behind NAT gateway. This way our upcoming ECS service can talk to internet (for pulling images from ECR), but no one can get in to it.

`subnet.tf`

resource "aws_subnet" "public" {
  vpc_id     = aws_vpc.default.id
  cidr_block = "10.0.1.0/24"

  tags = {
    Environment = var.environment_name
  }
}

resource "aws_subnet" "private" {
  vpc_id     = aws_vpc.default.id
  cidr_block = "10.0.2.0/24"

  tags = {
    Environment = var.environment_name
  }
}

resource "aws_subnet" "db_a" {
  vpc_id            = aws_vpc.default.id
  cidr_block        = "10.0.3.0/24"
  availability_zone = "${var.aws_region}a"

  tags = {
    Environment = var.environment_name
  }
}

resource "aws_subnet" "db_b" {
  vpc_id            = aws_vpc.default.id
  cidr_block        = "10.0.4.0/24"
  availability_zone = "${var.aws_region}b"

  tags = {
    Environment = var.environment_name
  }
}


resource "aws_db_subnet_group" "default" {
  name        = "${var.environment_name}-${var.name}-db"
  description = "Subnet group for DB"
  subnet_ids  = [aws_subnet.db_a.id, aws_subnet.db_b.id]

  tags = {
    Environment = var.environment_name
  }
}

Here we have those two subnets, public and private. Also we have defined two subnets for our RDS database instance, which are in different availability zones (AZ). The current upcoming RDS setup requires us to have two availability zones in subnets where it is placed in to. Finally we can create subnet group for the RDS and configure these two DB subnets to it.

`security_groups.tf`

resource "aws_security_group" "http" {
  name        = "http"
  description = "HTTP traffic"
  vpc_id      = aws_vpc.default.id

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "TCP"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_security_group" "https" {
  name        = "https"
  description = "HTTPS traffic"
  vpc_id      = aws_vpc.default.id

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "TCP"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_security_group" "egress-all" {
  name        = "egress_all"
  description = "Allow all outbound traffic"
  vpc_id      = aws_vpc.default.id

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_security_group" "myapp-service" {
  name   = "${var.environment_name}-${var.name}-service"
  vpc_id = aws_vpc.default.id

  ingress {
    from_port   = 4000
    to_port     = 4000
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}


resource "aws_security_group" "db" {
  name        = "${var.environment_name}-${var.name}-db"
  description = "Security group for database"
  vpc_id      = aws_vpc.default.id

  ingress {
    from_port       = 5432
    to_port         = 5432
    protocol        = "tcp"
    security_groups = [aws_security_group.myapp-service.id]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  lifecycle {
    create_before_destroy = true
  }
}

Security groups help you to restrict traffic from your resources. E.g. we are only allowing incoming traffic (ingress) to specific ports. Also we can restric the outgoing traffic (egress) to specific IPs etc. These security groups are applied to other resources in our infrastructure (RDS, ECS, Load balancer etc.).

`rds.tf`

resource "aws_db_instance" "default" {
  allocated_storage = var.db_storage
  engine            = var.db_engine
  engine_version    = var.db_engine_version
  instance_class    = var.db_instance_type
  name              = var.db_name
  username          = var.db_username
  password          = var.db_password

  availability_zone = var.aws_default_zone

  publicly_accessible    = false
  vpc_security_group_ids = [aws_security_group.db.id]
  db_subnet_group_name   = aws_db_subnet_group.default.name

  tags = {
    App         = var.name
    Environment = var.environment_name
  }
}

All the properties are configurable through variables for RDS resource. We are also assigning security group for this resource, and also the subnet group which we defined earlier.

`lb.tf`

resource "aws_lb" "myapp" {
  name = "${var.environment_name}-${var.name}"

  subnets = [
    aws_subnet.public.id,
    aws_subnet.private.id
  ]

  security_groups = [
    aws_security_group.http.id,
    aws_security_group.https.id,
    aws_security_group.egress-all.id
  ]

  tags = {
    Environment = var.environment_name
  }
}

resource "aws_lb_target_group" "myapp" {
  port        = "4000"
  protocol    = "HTTP"
  vpc_id      = aws_vpc.default.id
  target_type = "ip"

  health_check {
    enabled = true
    path = "/health"
    matcher = "200"
    interval = 30
    unhealthy_threshold = 10
    timeout = 25
  }

  tags = {
    Environment = var.environment_name
  }

  depends_on = [aws_lb.myapp]
}

resource "aws_lb_listener" "myapp-http" {
  load_balancer_arn = aws_lb.myapp.arn
  port              = "80"
  protocol          = "HTTP"

  default_action {
    target_group_arn = aws_lb_target_group.myapp.arn
    type             = "forward"
  }
}

This is setup for our application load balancer. It will accept basic HTTP requests to port 80 and forward them to our container. If you want to have HTTPS, you must assign certificate to the aws_lb_target_group. You can get one from AWS ACM, but I'm not covering it in this article. For real life production grade systems you want to have SSL/HTTPS always enabled.

`ecs.tf`

# Role for ECS task
# This is because our Fargate ECS must be able to pull images from ECS
# and put logs from application container to log driver

data "aws_iam_policy_document" "ecs_task_exec_role" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["ecs-tasks.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "ecsTaskExecutionRole" {
  name               = "${var.environment_name}-${var.name}-taskrole-ecs"
  assume_role_policy = data.aws_iam_policy_document.ecs_task_exec_role.json
}

resource "aws_iam_role_policy_attachment" "ecs_task_exec_role" {
  role       = aws_iam_role.ecsTaskExecutionRole.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

# Cloudwatch logs

resource "aws_cloudwatch_log_group" "myapp" {
  name = "/fargate/${var.environment_name}-${var.name}"
}

# Cluster

resource "aws_ecs_cluster" "default" {
  depends_on = [aws_cloudwatch_log_group.myapp]
  name       = "${var.environment_name}-${var.name}"
}

# Task definition for the application

resource "aws_ecs_task_definition" "myapp" {
  family                   = "${var.environment_name}-${var.name}-td"
  requires_compatibilities = ["FARGATE"]
  cpu                      = var.ecs_fargate_application_cpu
  memory                   = var.ecs_fargate_application_mem
  network_mode             = "awsvpc"
  execution_role_arn       = aws_iam_role.ecsTaskExecutionRole.arn
  container_definitions    = <<DEFINITION
[
  {
    "environment": [
      {"name": "SECRET_KEY_BASE", "value": "generate one with mix phx.gen.secret"}
    ],
    "image": "${aws_ecr_repository.myapp_repo.repository_url}:latest",
    "name": "${var.environment_name}-${var.name}",
    "portMappings": [
        {
            "containerPort": 4000
        }
      ],
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "${aws_cloudwatch_log_group.myapp.name}",
        "awslogs-region": "${var.aws_region}",
        "awslogs-stream-prefix": "ecs-fargate"
      }
    }
  }
]
DEFINITION
}


resource "aws_ecs_service" "myapp" {
  name            = "${var.environment_name}-${var.name}-service"
  cluster         = aws_ecs_cluster.default.id
  launch_type     = "FARGATE"
  task_definition = aws_ecs_task_definition.myapp.arn
  desired_count   = var.ecs_application_count

  load_balancer {
    target_group_arn = aws_lb_target_group.myapp.arn
    container_name   = "${var.environment_name}-${var.name}"
    container_port   = 4000
  }

  network_configuration {
    assign_public_ip = false

    security_groups = [
      aws_security_group.egress-all.id,
      aws_security_group.myapp-service.id
    ]
    subnets = [aws_subnet.private.id]
  }

  depends_on = [
    aws_lb_listener.myapp-http,
    aws_ecs_task_definition.myapp
  ]
}

There is a lot happening here, but don't get overwhelmed. We are first creating execution role for the ECS task definition (see the comment in the template). After that we define the lob group and the actual ECS cluster. The aws_ecs_task_definition is where all the important configuration happens to you container and environment in and around it. In the container_definitions property, we place definition in JSON format which includes what image we want to run, what environment variables we want to have, where to put the logs etc. Last but not least, we define service where we are going to run that task definition. This basically glues our task definition, cluster, load balancer etc. together.

`variables.tf`

variable "name" {
  type        = string
  description = "Name of the application"
  default     = "myapp"
}

variable "environment_name" {
  type        = string
  description = "Current environment"
  default     = "development"
}

variable "aws_region" {
  type        = string
  description = "Region of the resources"
}

variable "aws_default_zone" {
  type        = string
  description = "The AWS region where the resources will be created"
}

variable "db_storage" {
  type        = string
  description = "Storage size for DB"
  default     = "20"
}

variable "db_engine" {
  type        = string
  description = "DB Engine"
  default     = "postgres"
}

variable "db_engine_version" {
  type        = string
  description = "Version of the database engine"
  default     = "11"
}

variable "db_instance_type" {
  type        = string
  description = "Type of the DB instance"
  default     = "db.t3.micro"
}

variable "db_name" {
  type        = string
  description = "Name of the db"
}

variable "db_username" {
  type        = string
  description = "Name of the DB user"
}

variable "db_password" {
  type        = string
  description = "Name of the DB user"
}

variable "ecs_fargate_application_cpu" {
  type        = string
  description = "CPU units"
}

variable "ecs_fargate_application_mem" {
  type        = string
  description = "Memory value"
}

variable "ecs_application_count" {
  type        = number
  description = "Container count of the application"
  default     = 1
}

Here are the variables which we are uring across the Terraform templates.

`environment/dev.tfvars`

environment_name = ""

aws_region = ""

db_name = ""

db_username = ""

db_password = ""

ecs_fargate_application_cpu = "256"

ecs_fargate_application_mem = "512"

ecs_application_count = 1

Here you can fill the values for the variables. Note that you can create multiple workspaces e.g. dev, stag and prod, and also create .tfvars-files accordingly dev/stag/prod.tfvars. I'll show you soon how to use them.

`outputs.tf`

output "load_balancer_dns" {
  value = aws_lb.myapp.dns_name
}

Here we are just printing the URI out of our load balancer. We can use this to access our application from browser (or if you application is an API, then in Postman or other equivalent software).

`deploy_user.tf`

resource "aws_iam_user_policy" "ecs-fargate-deploy" {
  user = aws_iam_user.ci_user.name

  policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "ecs:UpdateService",
        "ecs:UpdateTaskDefinition",
        "ecs:DescribeServices",
        "ecs:DescribeTaskDefinition",
        "ecs:DescribeTasks",
        "ecs:RegisterTaskDefinition",
        "ecs:ListTasks"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}
POLICY
}

Last but not least, we assign user policy to our IAM CI user. This way we can deploy a new version of the application to ECS.

Now we can start applying this to AWS.

$ terraform plan --var-file=environment/dev.tfvars

First you want to plan the current changes your infrastructure. This will catch any typos and configuration mistakes in your templates. If it goes through without any problems, you can start applying.

$ terraform apply --var-file=environment/dev.tfvars

Terraform will ask you do you want to apply these changes, go through the changes and if they look good, hit Terraform with a yes answer. Now it will take some time to create all the resources to AWS. Grab a cup of coffee/tea/whatever and watch.

Now we are pretty much done for the infrstructure part. Next we will update our CI script and add the deploy job to it which will update the ECS service with newly built application Docker image from ECR.

Update .gitlab-ci.yml

Below you can see the changes we must do in order to add deploy job to our CI pipeline.

`.gitlab-ci.yml`

stages:
  - test
  - build
  - release
  - deploy

...

...

...

deploy_aws_dev:
  stage: deploy
  dependencies:
    - release_aws_dev
  image: alpine:latest
  variables:
    AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
    AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
  before_script:
    - apk add --no-cache curl jq python3 py3-pip
    - pip install awscli
  script:
    - aws ecs update-service --force-new-deployment --cluster development-myapp --service development-myapp-service
  only:
    - master

First add deployment stage on top of the file and then we add the job on the bottom of the file. In before_script we install needed tools and AWSCLI. On script we make the actual deployment. This tells to update the service from the defined cluster.

Conclusions

If you are developing a small scale application, this kind of infrstructure may be an "overkill" for it, but for medium to large sized applications this would be a good starting point. Elixir + Phoenix itself can handle concurrently large amount of requests, but ability to be able to scale underlaying infrastructure with Terraform is a feature you want to have when your application grows.

Things to do better:

HTTPS/SSL certificate and domain for the application - This is pretty straight forward, but I decided to leave it out for now. It would require ACM resource with output of the domain validation options for setting CNAME record to you domain DNS.
Better Terraform variable usage - We could map multiple subnet AZ to single variable and use Terraform's functions to map those values.
VPC endpoints - Instead of accessing ECR images through NAT from ECS, we could define VPC Endpoints for ECR, S3 and CloudWatch. This way we could keep all the traffic on the private network.

This was a fun little project and gives you a nice starting point for you infrastructure and CI pipelines in GitLab for Elixir + Phoenix. This article came out less "elixiry" than I wanted, this same template can be applied for other languages as well.

Any feedback is appreciated!

URL shortener with Elixir and Phoenix

Aleksi Holappa — Thu, 10 Sep 2020 20:55:57 +0000

One evening I was thinking a topic for a small Elixir + Phoenix project and came up with an idea of implementing URL shorten-er with Elixir and Phoenix. The implementation is quite easy and quick, and overall the application is going to be very small.

The technology stack consists following technologies, Elixir, Phoenix framework and PostgreSQL. This stack is run in docker containers.

If you want to follow through this article I have public GitLab repository which contains the application.

Features

I started defining key user paths:

User is able to create new link
User is able to navigate to created link and redirected to original URL
User is able to review statistics of the created link

These 3 paths are pretty trivial to implement, so lets dive in to it!

Implementation

I'm not going to go through the initial project set-up in this article. You can do so here!

Resource

The only resource we are going to need is Links. We can create one running the following command inside our docker container. If you didn't follow the earlier link for setting up the local docker + docker-compose environment ignore the docker-compose related commands.

$ docker-compose run web mix phx.gen.html Links Link links

This command will create Links and Link modules, Link-controller, migrations and test files.

A lot of files are generated when creating new resource, but take your time to go through the files. You can read more information of the phx.gen.html here.

Migration and schema

In previous step we generated resource for our application. The mix phx.gen.html accepts fields as parameter for the database table etc. You can put them there or define the fields manually in migration and schema file.

Migration

Our migration file looks pretty neat. The only notable change is that we are saying "Don't create primary key" for the links-table, since we are defining the id-field to be the primary key of the links table and setting its length to 8-digits.

defmodule Shorturl.Repo.Migrations.CreateLinks do
  use Ecto.Migration

  def change do
    create table(:links, primary_key: false) do
      add :id, :string, size: 8, primary_key: true
      add :url, :text, null: false
      add :visits, :integer, default: 0

      timestamps()
    end
  end
end

Schema and changeset

Here we define the schema for our model/struct. Pretty basic stuff, but we need to define that we are not using the default primary key.

defmodule Shorturl.Links.Link do
  use Ecto.Schema
  import Ecto.Changeset

  @primary_key {:id, :string, []}
  schema "links" do
    field :url, :string
    field :visits, :integer

    timestamps()
  end

...

end

Below the schema block we have our changeset function with a custom validation for the user defined original URL.

defmodule Shorturl.Links.Link do
  use Ecto.Schema
  import Ecto.Changeset

...

  @doc false
  def changeset(link, attrs) do
    link
    |> cast(attrs, [:id, :url, :visits])
    |> validate_required([:id, :url])
    |> validate_url(:url)
  end

  def validate_url(changeset, field, options \\ %{}) do
    validate_change(changeset, field, fn :url, url ->
      uri = URI.parse(url)

      if uri.scheme == nil do
        [{field, options[:message] || "Please enter valid url!"}]
      else
        []
      end
    end)
  end
end

The validate_url/3 checks if the original URL is an URL with scheme (http, https). If not, error message is attached to the :url field of the changeset.

Routes

If you are not familiar with the Phoenix routing, I suggest you to read this.

When creating a new Phoenix project, there is already some routes defined in routes.ex. The LiveView routes for metrics dashboard are enabled only in dev and test environments, we can leave it there. The initial route for PageController can be removed, since we are only using one controller in this application, LinkController.

At this point we need following routes

Path to show the form of the new shortened URL
Path where to post the form
Path which redirects the user to original URL
Path where the stats of the shortened URL are shown

defmodule ShorturlWeb.Router do
  use ShorturlWeb, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  scope "/", ShorturlWeb do
    pipe_through :browser

    get "/", LinkController, :new
    post "/links", LinkController, :create
    get "/:id", LinkController, :redirect_to
    get "/:id/stats", LinkController, :show
  end

  ...
end

Now we have defined our routes for our application, wonderful! Each atom after LinkController is the action where the request is passed to.

LinkController

New

This is the action which is responsible for rendering the front page of the application. The form helper in the view needs Link changeset to be available. If you look the corresponding template file (new.html.eex), you can see that we also pass @action variable for the form-template. This is used to tell the form helper where to send all the form data.

defmodule ShorturlWeb.LinkController do
  use ShorturlWeb, :controller

  alias Shorturl.Links
  alias Shorturl.Links.Link

  def new(conn, _params) do
    changeset = Links.change_link(%Link{})
    render(conn, "new.html", changeset: changeset)
  end

...

end

Create

All the magic happens here what comes to the creation of the link. First we need to create random 8-digit identifier for the link, this is the earlier mentioned primary key for the database entry. We put the identifier to our parameter map and try to create the link. If the link is successfully created and persisted in database, we redirect the user to :show-action, otherwise we just render the front page (:new-action) with the Link changeset which holds the possible errors and display them in the UI.

defmodule ShorturlWeb.LinkController do
  use ShorturlWeb, :controller

  alias Shorturl.Links
  alias Shorturl.Links.Link

...

  def create(conn, %{"link" => link_params}) do
    case create_link(link_params) do
      {:ok, link} ->
        conn
        |> put_flash(:info, "Link created successfully.")
        |> redirect(to: Routes.link_path(conn, :show, link))

      {:error, changeset} ->
        render(conn, "new.html", changeset: changeset)
    end
  end


...

defp create_link(link_params) do
    key = random_string(8)
    params = Map.put(link_params, "id", key)

    try do
      case Links.create_link(params) do
        {:ok, link} ->
          {:ok, link}

        {:error, %Ecto.Changeset{} = changeset} ->
          {:error, changeset}
      end
    rescue
      Ecto.ConstraintError ->
        create_link(params)
    end
  end

...

  defp random_string(string_length) do
    :crypto.strong_rand_bytes(string_length)
    |> Base.url_encode64()
    |> binary_part(0, string_length)
  end
end

Edit:
I totally forgot to handle situation where the randomly generated ID is already taken by some other URL. I updated the above code to handle also this case with a nice recursive function! I abstracted the actual business logic of this case to it's own function, so the create controller function stays clean and readable. Thank you Roberto Amorim!

Show

This is purely just displaying the information of the shortened link. Original link, shortened URL and visits are shown in the view. In the controller we fetch the Link with the id from our path parameters and pass the link for the view. If the query results in Ecto.NoResultsError the user is redirected to front page with error flash message. Flash message documentation. We also pass domain to the view, this is used for prefixing the shortened URL.

defmodule ShorturlWeb.LinkController do
  use ShorturlWeb, :controller

  alias Shorturl.Links
  alias Shorturl.Links.Link

...

  def show(conn, %{"id" => id}) do
    try do
      link = Links.get_link!(id)
      domain = System.get_env("APP_BASE_URL") || nil
      render(conn, "show.html", link: link, domain: domain)
    rescue
      Ecto.NoResultsError ->
        conn
        |> put_flash(:error, "Invalid link")
        |> redirect(to: Routes.link_path(conn, :new))
    end
  end

...

end

Redirect_to

This is the action which is responsible for redirecting the users to original URL when they click a shortened URL/link somewhere. The structure is pretty much the same as in :show, but we need to update the visits count for the associated link. The Task.start/1 is used for side-effects in Elixir applications and we don't care what the task returns. Also the function call in the task is run on its own process, so it doesn't block the execution of the following code.

defmodule ShorturlWeb.LinkController do
  use ShorturlWeb, :controller

  alias Shorturl.Links
  alias Shorturl.Links.Link

...

  def redirect_to(conn, %{"id" => id}) do
    try do
      link = Links.get_link!(id)
      # Start task for side-effect
      Task.start(fn -> update_visits_for_link(link) end)
      redirect(conn, external: link.url)
    rescue
      Ecto.NoResultsError ->
        conn
        |> put_flash(:error, "Invalid link")
        |> redirect(to: Routes.link_path(conn, :new))
    end
  end

...

  defp update_visits_for_link(link) do
    Links.update_link(link, %{visits: link.visits + 1})
  end

...

end

Templates and styling

I'm not going through styling and templates in this article, but you can look for them in the GitLab repository which were linked earlier in this article.

Testing

There is small portion of tests done in the repository, go look it up! For the large part they are generated by the mix-task and modified to meet the functionalities of our application.

Are we done? No.

Lets say that the application gets huge amount of links daily and you persist all of them in your database. At some point your DB will blow up. We need to clean old URLs away which are not used any more, sounds fair?

To begin with the implementation, I took Quantum library and configured it according the documentation. This will be responsible for scheduling jobs and running them.

We want to delete all the links which are not visited during the last 7 days and we want to run this cleaning job daily.

config :shorturl, Shorturl.Scheduler,
  jobs: [
    {"@daily", {Shorturl.Clean, :delete_old_links, []}}
  ]

defmodule Shorturl.Clean do
  alias Shorturl.Links
  require Logger

  def delete_old_links do
    Logger.info("Running job for old links")
    {count, _} = Links.delete_all_old()
    Logger.info("Removed #{count} old links")
  end
end

In links.ex

  def delete_all_old() do
    from(l in Link, where: l.updated_at < ago(7, "day")) |> Repo.delete_all()
  end

Just like that, with a few lines of code the database is cleaned daily. The Ecto query API is just wonderful and we don't have to delete the links one by one when using delete_all/1.

Conclusion

This was a nice tiny project which was fun and quick to implement, no biggie. I tried to make this article beginner friendly as possible!

The next step is to deploy this application to AWS with some GitLab CI/CD and Terraform magic!

Development environment for Elixir + Phoenix with Docker and Docker-compose

Aleksi Holappa — Sun, 16 Aug 2020 20:09:37 +0000

You can now watch this article as a video:
Video link OUTDATED

Edit 27.11.2021: Since Phoenix 1.6.0 release removed Webpack, Npm etc. and moved to Esbuild, the article is now updated accordingly.

Edit 25.12.2021: Change endpoint ip which works in Docker environment.

I've been following Elixir, Phoenix and Docker for a two years now. I Started with small demos and experiments. Back then configuring Elixir and Phoenix to Docker environment was a pain, especially the configuration for runtime. There was at least a 20 different ways and tutorials how to configure Elixir and Phoenix for Docker and Docker-compose. Each of those tutorials worked out, but some were outdated or too complicated.

Today I had some spare time to deep dive again into this problem and configure proper hot-reloaded local development environment for Elixir + Phoenix using Docker and Docker-compose.

Prerequisite

In this article I'm not going to go through installing these tools, but below is a list of what is needed.

Docker
Docker-compose
Elixir (I used latest 1.10.4)
Phoenix framework (I used latest 1.5.4)

Creating Phoenix application

Once you have everything installed you're ready to generate new Phoenix project by executing:

$ mix phx.new app_name

There are a few flags to pass along this command, but this command covers everything we need (Ecto, webpack etc). More on phx.new documentation.

Deps

During the project bootstrap mix will execute command deps.get & deps.compile to fetch and compile needed dependencies

This is problematic since we want to run our program/web-server in container, not on bare host system. In order to run Elixir application it needs to be compiled in same system architecture where the release binary will be run on. For now, it's safe to remove deps folder from the project structure.

Now lets dive into setting up our Docker configuration for our newly created application!

Docker and Docker-compose

In this section we will be creating following files:

Dockerfile
docker-compose.yml
.env

Dockerfile is used to build Docker image which holds our application, dependencies and needed tools.

docker-compose.yml - yaml-markup file to define our services, volumes etc.

.env - Holds our environment variables for the application.

Dockerfile

Below you can see the contents of our Dockerfile.

FROM bitwalker/alpine-elixir-phoenix:latest

WORKDIR /app

COPY mix.exs .
COPY mix.lock .

CMD mix deps.get && mix phx.server

On the first line we define our base image which is used to build our image on to. As you can see, we're using bitwalker/alpine-elixir-phoenix which holds Elixir, Phoenix and other needed tools for the application.

On the next few lines we're defining our working directory, copying files and making one new directory.

The CMD is the line which is executed when we fire up our container. There is two commands for fetching deps and firing up our server.

docker-compose.yml

version: '3.6'
services:
  db:
    environment:
      PGDATA: /var/lib/postgresql/data/pgdata
      POSTGRES_PASSWORD: postgres
      POSTGRES_USER: postgres
      POSTGRES_HOST_AUTH_METHOD: trust
    image: 'postgres:11-alpine'
    restart: always
    volumes:
      - 'pgdata:/var/lib/postgresql/data'
  web:
    build: .
    depends_on:
      - db
    environment:
      MIX_ENV: dev
    env_file:
      - .env
    ports:
      - '4000:4000'
    volumes:
      - .:/app
volumes:
  pgdata:

Here we defined our docker-compose script version, needed services and volumes to be created.

As you can see, there is two services(containers) being created db and web. db is our database container which is used by web-container. On the image-property we define our image, which is postgres:11-alpine. You could use newer version, for example PostgreSQL 12. It really depends what version you are going to use in production environment. I recommend to use same versions across the environments to minimize infrastructure related problems.

More explanation of the properties can be found from docker-compose documentation

.env

DATABASE_URL=postgres://postgres:postgres@db:5432/myapp_dev

For now there is only our environment variable for database URL. Later on when your application grows you will be storing your application environment variables in this file.

Fire it up

First we need to build our Docker image:

$ docker-compose build

If you did everything correctly, the image should be built relatively quick. Now we have our image ready, but first we need to configure our application to use our environment variable for database URL, so lets do that.

Navigate to config/dev.exs and open it up. Change the content for your database from this:

# Configure your database
config :myapp, Myapp.Repo,
  username: "postgres",
  password: "postgres",
  database: "myapp_dev",
  hostname: "localhost",
  show_sensitive_data_on_connection_error: true,
  pool_size: 10

To this:

# Configure your database
config :myapp, Myapp.Repo,
  url: System.get_env("DATABASE_URL"),
  show_sensitive_data_on_connection_error: true,
  pool_size: 10

NOTE: change endpoint IP to 0.0.0.0!

As you can see, we are fetching our database URL for database connection from the container environment.

Now we should be ready for our application start-up:

$ docker-compose up

It checks the deps, compiles deps & source files, but wait a second...

[error] Postgrex.Protocol (#PID<0.3932.0>) failed to connect: ** (Postgrex.Error) FATAL 3D000 (invalid_catalog_name) database "myapp_dev" does not exist

We don't have a database where to connect! Lets fix that:

$ docker-compose run web mix ecto.create

After running this you should be greeted with a pleasant one line message which tells you that the database for YourApp.Repo has been created! Wonderful!

Note that you can execute any mix commands inside the container since it has mix tooling available. You can run migrations, seeds, destroy the database and set it up again for a clean start etc.

Now execute:

$ docker-compose up

The application should start up with database connection and be ready for development! Navigate to localhost:4000 and now if you make changes to the source files, the changes will be updated to the server, so no need for manual restart of the containers!

Conclusion

This is a pretty simplistic setup with minimal amount of files to support/configure.

I'll do a follow-up post where we will deploy Elixir application to AWS. It will include configuring the application for production environment, Terraform infrastructure and little bit of CI magic.

Thanks for reading, hope you liked it! 🙂