DEV Community: Joseph Kahn

Vagrant Plugin: Back 2 Tha Hood

Joseph Kahn — Fri, 10 Apr 2020 17:38:31 +0000

Migrating my old blog

As part of migrating my older blog posts, this one was posted on January 25th, 2015.

Introduction

Once again, writing a post after a long break from updating my blog. I found my free time began to disappear when I started taking class at night after work and didn't have much time for blogging. When my co-op finished in September, I then worked part time between my classes and finished up the last of my requirements. This has been my first month back to work and getting used to having some free time again.

In school I covered a variety of topics from databases and PHP to the SOLID principles. I might try to make a post about something I covered in school over the next few weeks. I’ve also gotten back into reading and just finished Geek Sublime: The Beauty of Code, the Code of Beauty as it was a gift from a friend. The book was pretty interesting and yet it did seem a bit dry at times and certain passages seemed to go on forever. Anyway, on to the good stuff.

One thing I was attempting to do this week at work was to write a custom Vagrant command. For those of you unfamiliar with Vagrant, it’s a wrapper around tools like VirtualBox. It makes configuring, creating, and managing virtual machines for development a lot easier. It also provides some useful commands to make starting and using VMs easier.

When I first started researching how to make custom commands, it seemed rather daunting. The work to gain ratio seemed to be a lot higher than I'd imagined. I had thought it would be a few lines of configuration to map command X with function Y, but that was just not the case.

Woes With Creating The Plugin

As far as I can tell, in the current version of Vagrant, there is only one way to build these and that is through their plugin system. Once again I was hopeful as I dashed into the documentation and found this:

Warning: Advanced Topic! Developing plugins is an advanced topic that only experienced Vagrant users who are reasonably comfortable with Ruby should approach.

As far as I was aware, I was neither an experienced Vagrant user or reasonably comfortable with Ruby…so I was preparing for the worst. Apparently, all plugins are to defined as gems, regardless of complexity. My only experience with gems was to run bundle install to be able to work on this blog. I figured it couldn't be too hard to do and they must have examples of how to build them. I was sadly mistaken and confused as began researching. I found confusing documentation and half-examples with decent explanation or full examples with none. I was flummoxed.

I thought, how can there not be a complete (and easy to follow) Hello World type example for those who know little of Ruby? The documentation was the closest I could find but instead of giving you the full story, they leave that warning. I hope this will serve as an example to the next person who just wants a simple custom vagrant command.

It Begins

The first thing you need for a gem file is the structure. Apparently you can auto-generate the skeleton, I opted to take a simple gem and make my changes. Here’s the file structure of my gem:

Gemfile                         # The is where I put my dependancies.
Rakefile                        # The is somehow tied to building the gem.
vagrant-custom-command.gemspec  # The information/build data, like a `setup.py` file.
lib/
--- vagrant-custom-command.rb   # This is the top level file of the command.
--- custom-command/
--- --- command.rb              # The logic of the command and the argument parsing.
--- --- plugin.rb               # Defining the command and point it at `command.rb`.

There aren't too many files here , it was reasonably easy to make a custom gem. I'm not making use of any other ruby gems or doing anything complex.

The Files

Gemfile

gemspec

group :development do
  gem 'vagrant',         github: 'mitchellh/vagrant'
end

As you can see, I've just defined vagrant as a requirement for development, that's all. Simple file.

Rakefile

require "bundler/gem_tasks"

I don't know why this is here, at some point I'd like to delve more into Ruby. For the moment I just accept that this works and that that line is suffient to use rake build.

vagrant-custom-command.gemspec

# coding: utf-8
lib = File.expand_path('../lib', __FILE__)
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)

Gem::Specification.new do |spec|
  spec.name          = "vagrant-custom-command"
  spec.version       = 1
  spec.authors       = ["Joseph Kahn"]
  spec.email         = ["JosephBKahn@gmail.com"]
  spec.description   = %q{Awesome custom command to do something}
  spec.summary       = spec.description
  spec.homepage      = "<repo url>"

  spec.files         = `git ls-files`.split($/)
  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
  spec.require_paths = ["lib"]

  spec.add_development_dependency "bundler", "~> 1.3"
  spec.add_development_dependency "rake"
end

That's pretty straight forward. I think I used bundler to install rake and rake to build the module so that's what this gem required for development.

vagrant-project-setup.rb

require_relative 'custom-command/plugin'

This file is just going to use the following plugin.rb file and exists in here to make the directory structure clean.

plugin.rb

module VagrantPlugins
  module CustomCommand
    class Plugin < Vagrant.plugin(2)
      name 'test'
      description 'testing'

      command "hello" do
        require_relative "command"
        Command
      end

    end
  end
end

All this file does it to register a command which can be called using vagrant hello and then will execute the logic in the command.rb file.

command.rb

module VagrantPlugins
  module CustomCommand
    class Command < Vagrant.plugin(2, :command)
      def self.synopsis
        "Prints Hello World"
      end

      def execute
        puts "Hello World!"
        return 0
      end
    end
  end
end

The execute function is the code that is executed when it's called and the synopsis is how the command is described under vagrant list-commands.

The above snippet prints Hello World! to the screen and exits with a 0 status.

More Complex `command.rb`

module VagrantPlugins
  module CustomCommand
    class Command < Vagrant.plugin(2, :command)
      def self.synopsis
        "Calls an Ansible script to do something."
      end

      def execute
        options = {}
        options[:optionA] = false
        options[:optionB] = false

        opts = OptionParser.new do |o|
          o.banner = "Usage: vagrant blah [item]"
          o.separator ""

          o.on("-a", "--optionA", "do something conditional.") do |a|
            options[:optionA] = a
          end
          o.on("-b", "--optionB", "do something conditional.") do |b|
            options[:optionB] = b
          end
        end

        argv = parse_options(opts)

        if argv.length == 0
          puts "You must specify at least one item."
          return 1
        end
        items = argv.join(',')

        host_arg = "-i ./provisioning/hosts_file"
        private_key_arg = "--private-key=./.vagrant/machines/default/virtualbox/private_key"
        user_arg = "-u vagrant"
        playbook_path = "./provisioning/some_playbook.yml"
        extra_vars = "--extra-vars \"items=#{items} option_a=#{options[:optionA]} option_b=#{options[:optionB]}\""

        result = system("ansible-playbook #{host_arg} #{private_key_arg} #{user_arg} #{playbook_path} #{exatra_vars}")

        if result.nil?
          puts "Error was #{$?}"
          return 1
        end
        puts "Success!!"
        return 0
      end
    end
  end
end

In this example, I'm using the built-in option parsing to pass a list of items and two conditionals to an ansible playbook. That snippet is pretty straight forward until we get to the system call. I had to specify the vagrant user and give vagrant’s private key to have sufficient privileges to run this playbook. This makes it easier for the other developers to run these commands. They don’t need to remember the full command or where the private_key file is.

Building, Testing & Usage

In order to build the file, I downloaded ruby (I use rvm) and bundler following their documentation. After installing rvm, I ran rvm all do gem install bundler (I only have one version of Ruby installed, otherwise I would have specified which). Once I had both of those, I ran rake build to build my plugin. That created a .gem file in a pkg directory it created at the root of the gem.

Once I had the gem file, I went to my vagrant folder for the project and ran vagrant plugin install <path to gem file>. Once it's installed, a simple vagrant hello will test whether the plgin works and will return an error if there was an issue. During development I would continuously run vagrant plugin uninstall vagrant-custom-command && vagran plugin install <path to gem>.

Conclusion

I feel that plugin creation is a bit more complicated than it should have been for a Hello World case. It seems a bit like overkill for printing something to the screen. Hiding behind a warning makes the initial plunge into vagrant plugins that much scarier and harder. While I had a few hiccups during development, it was more confusing because I thought it was more difficult than it turned out to be. I didn't see an easier way to define something simple inside the Vagrantfile or anything like that. I hope this post helps the next confused person overcome those hurdles be on their way to make a great plugin.

If you have any suggestions on an easier way to do this, information on the gem that I missed, or any part of this than please leave a comment below.

For a Few Ansible Modules More

Joseph Kahn — Fri, 10 Apr 2020 17:36:41 +0000

Migrating my old blog

As part of migrating my older blog posts, this one was posted on August 17th, 2014.

Introduction

Before I get into the article, I'd like to thank Matt Jaynes for featuring my last article in issue 43 of Anisble Weekly. That was a great surprise and hopefully got me a few more readers. Now back to the post.

Once again, school has kept me busy. I've since finished taking CSC 373 - Algorithm Design, Analysis, and Complexity, completing a final assignment and subsequent exam. I've just gotten back to working on the Ansible script that I started a month ago. In case you missed my post from last time, Ansible is a tool used to deploy and update applications in an easy to use language, using SSH, with no agents to install on remote systems. The specifications for Ansible are all written in YAML, making them well structured and generally easy to follow. Back at Wave, Nathan wrote An Ansible primer blog post on the Wave Engineering blog. A few weeks ago, I wrote a post about using Ansible to setup my local machine: Ansible or: How I Learned to Stop Wasting Time Setting Up My Computer and Script It. Here, I'll be taking you through the creation of custom modules, starting with the why.

While writing the YAML for my task files, I found there were a few common tasks I wanted to do that did not have a module wrapper. The easy way to deal with those tasks is to use the Command module. That way I can tell Ansible to execute an arbitrary command on the machine. There are some downsides to doing this, which depend on the specific commands in question. One thing I like about the output of Ansible is that it can tell me how many commands Ansible executed, previously executed and caused errors (if I allow them). But yet, arbitrary commands are always run because Ansible has no way to know if they have been executed.

There are two options that I saw to address this, as I wanted to lower the number of changed tasks on repeated runs. I also wanted to prevent some commands from running multiple times. The first is to add another task and use conditional execution. I did that for two of the tasks, however it still had to register a changed task in order to do the check. This led me to look into writing my own modules and I wasn't able to find many great examples. I should have started with looking at the modules in the Ansible library, but that did not occur to me. In the end I wrote the modules with the help of the docs, the Ansible IRC channel and Ansible library modules. Below I'll outline writing my own module as well as conditional execution. Note, I'll be providing specific examples, but there's a lot more Ansible can do so please check out the docs if you're interested in using Ansible.

Conditional Execution

There isn't a whole lot to write here, the code is self explanatory. I wrote this to fix a screen dimming issue on my sager laptop. This particular set of tasks is also conditionally executed, covered by my last blog post.

---
- name: screen dimming - alter grub file
  lineinfile:
    dest: /etc/default/grub
    regexp: "^GRUB_CMDLINE_LINUX_DEFAULT="
    line: 'GRUB_CMDLINE_LINUX_DEFAULT="quiet splash video.use_native_backlight=1"'
  sudo: yes
  register: grubfile
  tags: sager

- name: screen dimming - update grub
  command: sudo update-grub
  sudo: yes
  when: grubfile|changed
  tags: sager

# or

- name: keyboard colors - link clevo-wmi to kernel check
  command: cat /etc/modules
  sudo: yes
  register: running_modules
  tags: sager

- name: keyboard colors - link clevo-wmi to kernel
  command: "sudo insmod /home/{{ username }}/setup/clevo-wmi-code/clevo_wmi.ko"
  sudo: yes
  when: running_modules.stdout.find('clevo_wmi') == -1
  tags: sager

Module Writing

The first thing to work out for a new module is what arguments it will take and how to leverage Ansible to do most of the heavily lifting around it. Here's the start of my Ansible module for gconftool-2:

from ansible.constants import mk_boolean
from ansible.module_utils.basic import *


def main():

    module = AnsibleModule(
        argument_spec={
            'key': {'required': True},
            'bool': {'type': 'bool'},
            'int': {'type': 'int'},
            'string': {'type': 'str'},
            'float': {'type': 'float'},
            'list': {'type': 'list'},
            'pair': {'type': 'list'},
            'pair-cdr-type': {'choices': ['int', 'bool', 'float', 'string']},
            'pair-car-type': {'choices': ['int', 'bool', 'float', 'string']},
            'list-type': {'choices': ['int', 'bool', 'float', 'string']}
        },
        mutually_exclusive=[
            ['bool', 'string', 'int', 'float', 'list', 'pair'],
            ['bool', 'string', 'int', 'float', 'list-type', 'pair'],
            ['bool', 'string', 'int', 'float', 'list', 'pair-car-type'],
            ['bool', 'string', 'int', 'float', 'list', 'pair-cdr-type']
        ],
        required_one_of=[['bool', 'string', 'int', 'float', 'list', 'pair']],
        required_together=[
            ['pair', 'pair-car-type', 'pair-cdr-type'],
            ['list', 'list-type']
        ],
        supports_check_mode=True
    )

Most of that is pretty easy to follow and it shows a few of the options Ansible handles. It handles mutual exclusions, requiring one of a set items as well as items which are required together. Additionally, in the argument_spec you can also define defaults (and more, although this was all that I needed). The supports_check_mode is a boolean which represents that the module supports dry runs. Now that I've defined my argument structure, it's time to get onto the next piece of code.

The Getter and Setter

def _set_value(module, key, value, argument_type, additional_args):
    ''' Set value of setting, under `key`, using gconftool-2 to `value` of type `argument_type`'''
    cmd = '/usr/bin/gconftool-2 --set --type {} {} "{}" {}'
    return module.run_command(" ".join([cmd.format(argument_type, key, value, additional_args)]))


def _get_value(module, key):
    ''' Return value of setting, under `key`, from gconftool-2'''
    return module.run_command('gconftool-2 --get {}'.format(key))[1].strip()

The module set in the above code block contains some crucial functions, one of which is run_command. This wraps the native python system library for running commands. It has very good error handling and provides a plethora of input options. Most of those aren't needed for simple tasks like running gconftool-2. All I'm doing in the code block is building the command that I'd previously supplied to the Command Module. I've already shown how to define the arguments, but now I'll show how to access them.

Argument Accessing

key = module.params['key']
boolean_value = module.params['bool']
string_value = module.params['string']
integer_value = module.params['int']
float_value = module.params['float']
list_value = module.params['list']
pair_value = module.params['pair']

Ansible makes arguments extremly easy to access. Now I'll show you a sample of how I worked with the input.

Argument Parsing

additional_args = ''
instance_type_mapping = {'int': int, 'string': str, 'float': float, 'bool': mk_boolean}
if boolean_value is not None:
    argument_type = 'bool'
    value = str(mk_boolean(boolean_value)).lower()
    old_value = str(mk_boolean(old_value)).lower()
...
elif float_value is not None:
    argument_type = 'float'
    value = float_value
...
elif pair_value is not None:
    if len(pair_value) != 2:
        module.fail_json(msg='A pair must be a list of length 2, {} items found.'.format(len(pair_value)))
    argument_type = 'pair'
    try:
        car_value = pair_value[0]
        car_type = module.params['pair-car-type']
        if car_type == 'bool':
            module.boolean(car_value)
            car_value = mk_boolean(car_value)
        elif not str(instance_type_mapping.get(car_type)(car_value)) == car_value:
            raise ValueError

        cdr_value = pair_value[1]
        cdr_type = module.params['pair-cdr-type']
        if cdr_type == 'bool':
            module.boolean(cdr_value)
            cdr_value = mk_boolean(cdr_value)
        elif not str(instance_type_mapping.get(cdr_type)(cdr_value)) == cdr_value:
            raise ValueError
    except ValueError:
        error_msg = 'pair type `{}` or `{}` does not match the type of the contents.'
        module.fail_json(msg=error_msg.format(module.params['pair-car-type'], module.params['pair-cdr-type']))
    additional_args = '--car-type={} --cdr-type={}'.format(car_type, cdr_type)
    value = '({},{})'.format(car_value, cdr_value)

There are a few interesting things to note in this code block. The type specifications works for individual items but there is no equivalent for lists. When I set pair to type list it does not support type checking of particular elements or the length of the list. As such, I have to do my own error checking. Fortunately, Ansible provides a module.fail_json(msg='blah') to use for error reporting back to the console. There is a lot more you can provide than just a message but for my purposes that was perfect. Otherwise I'm just formatting the code to send off to my getter and setter. Now I'll provide you with the skeleton for the rest of the module.

The Skeleton

def main():
    #  module specification
    old_value = _get_value(module, key)
    # argument parsing

    changed = old_value != str(value)

    if changed and not module.check_mode:
        _set_value(module, key, value, argument_type, additional_args)

    module.exit_json(
        changed=changed,
        key=key,
        type=argument_type,
        value=value,
        old_value=old_value
    )

main()

As you can see, there isn't much more to do. That conditional is how I support check mode. All I'm doing here is calling my getter and setter and then using Ansible's module.exit_json() to handle to integration with Ansible. It requires the changed argument and the rest is custom and allows me to decide what gets printed on the screen when things change. It's also important to note that at the bottom of the file I'm calling the main function, which is required to run the module. The only other section of the file, which I've done so that I may contribute to Ansible, is the documentation at the top of the file.

Documentation

The documentation includes ussage information as well as examples, here is an exerpt from the file:


DOCUMENTATION = '''
---
module: gconftool-2
version_added: "post 1.7.1"
author: Joseph Kahn
short_description: alter gconftool-2 controlled settings.
description:
   - Set the value of a gconftool-2 controlled setting using a key and a
     string, an integer, a boolean, a pair or a list.
options:
  key:
    description:
      - The key of the gconftool-2 setting to change.
    required: true
  bool:
    description:
      - The boolean value to set the key to.
    required: false
...
'''

EXAMPLES = '''
# Set string value
- gconftool-2: key=/apps/gnome-terminal/global/default_profile string=base-16-monokai-dark

# Set bool value
- gconftool-2: key=/apps/gnome-terminal/profiles/base-16-monokai-dark/use_system_font bool=false

# Set pair value
- gconftool-2: key=/path/to/something pair-car-type=int pair-cdr-type=string pair=1,'Joseph Kahn'
'''

Usage

Using your custom modules is easy to do. Once you create a custom_modules directory, you can use them within Ansible. All you need to do is provide the module-path argument like so --module-path custom_modules.

Once you've done that, you can use it like any other Ansible module in your task files. i.e.

- name: base16 - set default terminal profile
  gconftool-2: key=/apps/gnome-terminal/global/default_profile string=base-16-monokai-dark

Testing

The Ansible repo provides an executable that allows for easy module testing from the command line. It can be invoked with a command like:

~/ansible/hacking/test-module -m ./gconftool-2 -a "key=/path/to/stuff pair-car-type=int pair-cdr-type=string pair=1,'Joseph Kahn'"

You can also use breakpoints if you provide the debugger used, i.e.

~/ansible/hacking/test-module -m ./gconftool-2 -a "key=2 list=1,1,true,1 list-type=bool" -D ipdb

Running the testing module provides to full generated source file, as well as the standard output information. The output file can be run manually without the use of additional arguments.

Conclusion

Writing a simple Ansible module wasn't very difficult and requires little code to get the basics up and running. Even with all the error handling and verification done above, the module code is only 117 lines. While I did this for convenience, it's easy to see how making modules simple to write can help make great provisioning scripts. These modules are relatively easy to follow and integrate seamlessly with the rest of the modules.

That's It

That's all you need to get a custom module up and running and reporting changes. I'll leave you with a few relevant links:

The Repository (The custom modules are in the ansible modules folder).
Ansible Modules.

Ansible or: How I Learned to Stop Wasting Time Setting Up My Computer and Script It

Joseph Kahn — Fri, 10 Apr 2020 17:33:57 +0000

Migrating my old blog

As part of migrating my older blog posts, this one was posted on July 27th, 2014.

Introduction

I know it has been a while since I put up my last post, but school was kept me busy. I wrote this one in a hurry and ask that you excuse any spelling and grammar mistakes. In particular, I've been taking CSC 373 - Algorithm Design, Analysis, and Complexity and have had several assignments to work on. In my free time, I've been working on my own Ansible script. Ansible is a tool used to deploy and update applications in an easy to use language, using SSH, with no agents to install on remote systems. The specifications for Ansible are all written in YAML, making them well structured and generally easy to follow. Back at Wave, Nathan wrote An Ansible primer blog post on the Wave Engineering blog. Here, I'll be taking you through my Ansible script as a practical example of what you could use Ansible for, outside of setting up remote servers.

Unfortunately, having recently had a hard drive failure on my laptop. I had to manually setup my machine just the way I like it. When setting up, I realized the colossal waste of time it was to lookup the ppa's I needed, the Sager specific driver I'd installed for my keyboard backlight and trying to recall what I used most. This time I'd done the smart thing and simply documented all my steps. There were a few missing things there involving using dconf, gconftool-2 and gsettings and a couple of steps I'd not yet written down. We'd been using Ansible at Wave and my last attempt to learn it met with confusion, ten VMs running in the background and an episode of The Sopranos. I had tried to follow Nathan's example (simplifying it along the way) as well as a bunch of other tutorials relating to Django. They all met with failure to get a successfully provisioned VM. I'd been making small contributions at work to our Ansible scripts in hopes to get the practice I needed for when I decided to try again. That's when Alex Tucker suggested I take my setup instructions and give it another go at Ansible. Now that you know how I got started, I'll get into the project.

The File Structure & Content

This is mostly self explanatory, I'll go into more detail about each section.

setup.yml                 # master playbook
HOSTS                     # HOSTS file which defines where the script will run. This points to localhost.
requirements.txt          # the requierments for running this playbook, including ansible
run.sh                    # one command script for install requirements, cloning the repo and provisioning.

roles/
    common/               # this hierarchy represents a "role"
        tasks/            #
            *.yml         #  <-- tasks files i'll talk about later
        files/            #
            *.*           #  <-- files for use with the copy resource
        vars/             #
            main.yml      #  <-- variables associated with this role

The HOSTS file

This file lets Ansible know where the machines are located, in this case all we'll need is localhost.

[local]
127.0.0.1

The Requirements File

Some on the Ansible modules have requirements, the Ansible docs provide them on each modules page.

python-apt==0.9.3.5
ansible==1.7.1

The Master Playbook

This is also pretty simple. Here we can define what we run where, here I'm just telling it to run the scripts, in my common role, on the local machine.

- name: a playbook to setup my local machine with my basic customizations
  hosts: local
  connection: local
  roles:
    - common

Common Roles: Variables main.yml

Here we can define variables which we can use with {{ variable name }} syntax. I use some of them to conditionally run tasks, to do them for a particular user and for downloading applications which are not available via ppa.

---
username: joseph
sager_laptop: false
vagrant_url: https://dl.bintray.com/mitchellh/vagrant/vagrant_1.6.3_x86_64.deb
virtualbox_url: http://download.virtualbox.org/virtualbox/4.3.14/virtualbox-4.3_4.3.14-95030~Ubuntu~raring_amd64.deb
github_username: JBKahn
bumblebee: false
uname_r: 3.13.0-32-generic

Common Roles: Files

For example, I place the autostart files in there so that I can copy them into the correct directory so applications will launch on boot. I also have configuration files for applications and my terminal.

[Desktop Entry]
Name=Variety
Comment=Variety Wallpaper Changer
Icon=/opt/extras.ubuntu.com/variety/share/variety/media/variety.svg
Exec=/opt/extras.ubuntu.com/variety/bin/variety
Terminal=false
Type=Application
X-GNOME-Autostart-Delay=20

Common Tasks

Here's where most of the coding is and I'll provide five examples, the rest you can checkout in the repository.

1) main.yml

---
  - include: pre.yml
  - include: sublime.yml
  - include: chrome.yml
  - include: numix.yml
  - include: plank.yml
  - include: keepass2.yml
  - include: laptop.yml
    when: sager_laptop == true
  - include: variety.yml
  - include: font.yml
  - include: base16.yml
  - include: work.yml
  - include: scm_breeze.yml
  - include: vim.yml
  - include: other_requirements.yml
  - include: dropbox.yml
  - include: bash_customizations.yml
  - include: bumblebee.yml
    when: bumblebee == true

'main.yml' is the file that the setup.yml common role calls, I just use it to conditionally call the other task files I use.

2) base16.yml

---
- name: base16 - checkout repo
  git:
    repo: https://github.com/chriskempson/base16-gnome-terminal.git
    dest: "/home/{{ username }}/setup/base16-gnome"

- name: base16 - copy monokai-dark
  copy:
    src: "/home/{{ username }}/setup/base16-gnome/base16-monokai.dark.sh"
    dest: "/home/{{ username }}/base16-monokai.dark.sh"
    mode: 0755

- name: base16 - install monokai dark
  command: "/home/{{ username }}/base16-monokai.dark.sh"

- name: base16 - set system font to source code pro
  gconftool-2:
    key: /apps/gnome-terminal/profiles/base-16-monokai-dark/font
    string: "Source Code Pro Semi-Bold 12"

- name: base16 - dont use default system font in terminal
  gconftool-2:
    key: /apps/gnome-terminal/profiles/base-16-monokai-dark/use_system_font
    bool: false

- name: base16 - set default terminal profile
  gconftool-2:
    key: /apps/gnome-terminal/global/default_profile
    string: "base-16-monokai-dark"

- name: base-16 - get base16-shell so that colors show up correctly in vim
  git:
    repo: https://github.com/chriskempson/base16-shell.git
    dest: "/home/{{ username }}/.config/base16-shell"

Here I make use of the git, file and command modules to checkout the amazing base-16 color scheme for my terminal, set it as the default and set the font (which I already setup in another task).

3) vim.yml

---
- name: horse vim - download vim
  apt:
    name: vim
    update_cache: yes
  sudo: yes

- name: horse vim - make sure bundle directory exists
  file:
    path: "/home/{{ username }}/.vim/bundle"
    state: directory
    mode: 0777
  sudo: yes

- name: horse vim - checkout vundle repo
  git:
    repo: https://github.com/gmarik/vundle.vim.git
    dest: "/home/{{ username }}/.vim/bundle/vundle"

- name: horse vim - checkout repo
  git:
    repo: https://github.com/JBKahn/horse.vim.git
    dest: "/home/{{ username }}/horse.vim"

- name: horse vim - copy .vimrc file
  file:
    src: "/home/{{ username }}/horse.vim/.vimrc"
    dest: "/home/{{ username }}/.vimrc"
    mode: 0755
    state: link

- name: horse vim - copy .vimrc.bundles file
  file:
    src: "/home/{{ username }}/horse.vim/.vimrc.bundles"
    dest: "/home/{{ username }}/.vimrc.bundles"
    mode: 0755
    state: link

- name: horse vim - make sure vim colors directory exists
  file:
    path: "/home/{{ username }}/.vim/colors"
    state: directory
    mode: 0777
  sudo: yes

- name: horse vim - copy color to config to avoid error
  copy:
    src: base16-monokai.vim
    dest: "/home/{{ username }}/.vim/colors/base16-monokai.vim"
    mode: 0755

- name: horse vim - run bundle install
  command: vim +BundleInstall +qall

- name: horse vim - add NeoVim ppa
  apt_repository:
    repo: 'ppa:rudenko/neovim'
    state: present
    update_cache: yes
  sudo: yes

- name: horse vim - install neovim
  apt:
    name: neovim
    update_cache: yes
  sudo: yes

- name: horse vim - symlink nvimrc
  file:
    src: "/home/{{ username }}/horse.vim/.vimrc"
    dest: "/home/{{ username }}/.nvimrc"
    mode: 0755
    state: link

- name: horse vim - symlink nvim
  file:
    src: "/home/{{ username }}/.vim"
    dest: "/home/{{ username }}/.nvim"
    mode: 0755
    state: link

Another part of my setup is to put Vim and all of the important packages on my machine. Unfortunately, when I had tried to install the packages it seemed that missing my color scheme file prevented me from downloading it through Vundle. Using Ansible's copy module I'm taking the current version (rather than doing a git pull for a more up to date one) to temporarily place the file there and have it replaced by running the setup.

4) work.yml

---
- name: download vagrant .deb package
  get_url:
    url: "{{ vagrant_url }}"
    dest: "/home/{{ username }}/setup/vagrant.deb"
    mode: 0755

- name: get currently installed vagrant version
  command: "vagrant --version"
  sudo: yes
  register: result
  ignore_errors: True

- name: install vagrant
  apt:
    deb: "/home/{{ username }}/setup/vagrant.deb"
  sudo: yes
  when: result.rc !=0 or result.stderr.find('A later version is already installed') != -1

- name: download virtualbox .deb package
  get_url:
    url: "{{ virtualbox_url }}"
    dest: "/home/{{ username }}/setup/virtualbox.deb"
    mode: 0755

- name: install virtualbox
  apt:
    deb: "/home/{{ username }}/setup/virtualbox.deb"
  sudo: yes

- name: install wave dependencies
  apt:
    name: "{{ item }}"
    state: installed
    update_cache: yes
  with_items:
    - python-pip
    - curl
    - nodejs
    - nfs-kernel-server
    - nfs-common
    - rpcbind
    - tmux
    - npm
    - libjpeg-dev
    - libfreetype6-dev
    - zlib1g-dev
    - libpng12-dev
    - python-dev
    - libpq-dev
    - python-dev
  sudo: yes

- name: get currently installed node version
  command: "node --version"
  sudo: yes
  register: result
  ignore_errors: True
  tags: nodejs

- name: checkout node
  git:
    repo: "git://github.com/ry/node.git"
    dest: "/setup/node"
    force: no
    accept_hostkey: yes
  when: result.rc !=0
  tags: nodejs
  sudo: yes

- name: configure
  sudo: yes
  command: "./configure"
  args:
    chdir: /setup/node
  when: result.rc !=0
  tags: nodejs

- name: make
  command: "make"
  sudo: yes
  args:
    chdir: /setup/node
  when: result.rc !=0
  tags: nodejs

- name: make install
  command: "make install"
  sudo: yes
  args:
    chdir: /setup/node
  when: result.rc !=0
  tags: nodejs

- name: install global nodejs packages
  npm:
    name: "{{ item.name }}"
    state: present
    global: yes
    version: "{{ item.version|default('') }}"
  with_items:
    - {name: 'bower'}
    - {name: 'coffee-script', version: '1.7.1'}
    - {name: 'grunt-cli'}
    - {name: 'less'}
    - {name: 'jshint'}
    - {name: 'coffee-jshint'}
  sudo: yes

- name: install tmuxp
  pip:
    name: tmuxp
  sudo: yes

- name: copy tmuxp config
  copy:
    src: .tmux.conf
    dest: "/home/{{ username }}/.tmux.conf"
    mode: 0755

- name: install ipython
  pip:
    name: ipython
  sudo: yes

There's a fair amount of overlap between things I use for my personal projects and those which I need for work. In here I've put the more development focused requirements making use of some of its wide variety of modules for installing packages through pip, .deb files and apt.

5) laptop.yml

---
- name: screen dimming - alter grub file
  lineinfile:
    dest: /etc/default/grub
    regexp: "^GRUB_CMDLINE_LINUX_DEFAULT="
    line: 'GRUB_CMDLINE_LINUX_DEFAULT="quiet splash video.use_native_backlight=1"'
  sudo: yes
  register: grubfile
  tags: sager

- name: screen dimming - update grub
  command: sudo update-grub
  sudo: yes
  when: grubfile|changed
  tags: sager

- name: keyboard colors - requirements
  apt:
    pkg: "{{ item }}"
    state: installed
  with_items:
    - build-essential
    - linux-source
  sudo: yes
  tags: sager

- name: keyboard colors - Make sure setup directory exists
  file:
    path: "/home/{{ username }}/setup"
    state: directory
  tags: sager

- name: keyboard colors - checkout clevo-wmi-code
  git:
    repo: git://git.code.sf.net/p/clevo-wmi/code
    dest: "/home/{{ username }}/setup/clevo-wmi-code"
    accept_hostkey: yes
  tags: sager

- name: keyboard colors - build clevo-wmi
  command: make
  args:
    chdir: "/home/{{ username }}/setup/clevo-wmi-code"
  tags: sager

- name: keyboard colors - link clevo-wmi to kernel check
  command: cat /etc/modules
  sudo: yes
  register: running_modules
  tags: sager

- name: keyboard colors - link clevo-wmi to kernel
  command: "sudo insmod /home/{{ username }}/setup/clevo-wmi-code/clevo_wmi.ko"
  sudo: yes
  # when: running_modules.stdout.find('clevo_wmi') == -1
  tags: sager
  ignore_errors: True

- name: keyboard colors - copy to kernel drivers
  copy:
    src: "/home/{{ username }}/setup/clevo-wmi-code/clevo_wmi.ko"
    dest: "/lib/modules/{{ uname_r }}/kernel/drivers/platform/x86/"
  sudo: yes
  tags: sager

- name: keyboard colors - handle dependencies
  command: sudo depmod -a
  sudo: yes
  tags: sager

- name: keyboard colors- add to etc/modules
  lineinfile:
    dest: /etc/modules
    line: clevo_wmi
  sudo: yes
  tags: sager

- name: keyboard colors - copy kbbl script
  copy:
    src: keyboard-color.sh
    dest: "/home/{{ username }}/keyboard-color.sh"
    mode: 0755
  tags: sager

One of the disadvantages of using linux can be that some things are hard to make work, although Ubuntu helps with a lot of it. For a long time I was unable to dim my backlight and my keyboard backlight was always on and always blue. Through the use of a small settings tweak, a custom driver and a custom script I wrote for changing the keyboard backlight, I'm not able to do all of these things. The task will only run contingent on the sager_laptop variable being true. One of the truly great modules in called 'lineinfile' and it does is ensure that a line exists in a file. That sounds pretty simple but it offers the ability to use a regular expression to identify the line (so that if it does not exist but it is meant to replace another line, it will) and the ability to create the file if it does not exist (configurable).

The script

sudo apt-get install python-setuptools

sudo easy_install pip

sudo apt-get install aptitude
sudo apt-get install git
sudo apt-get install python-dev libxml2-dev libxslt-dev

cd ~
mkdir -p setup
cd setup
git clone https://github.com/JBKahn/provisioning-local.git
cd provisioning-local

sudo pip install -r requirements.txt

echo -e "please enter your username, followed by [ENTER]" && read PROVISIONING_USER
sudo sed -i "s/^username: .*/username: $PROVISIONING_USER/" roles/common/vars/main.yml

echo -e "please enter your github username, followed by [ENTER]" && read PROVISIONING_GITHUB_USERNAME
sudo sed -i "s/^github_username: .*/github_username: $PROVISIONING_GITHUB_USERNAME/" roles/common/vars/main.yml

sudo sed -i "s/^uname_r: .*/uname_r: `uname -r`/" roles/common/vars/main.yml

ansible-playbook setup.yml -i HOSTS --ask-sudo-pass  --module-path ./ansible_modules

# currently unable to use ansible due to EULA that I can't seem to stub using debconf
sudo apt-get install steam

dropbox start -i &
/opt/extras.ubuntu.com/variety/bin/variety $
plank &

source ~/.bashrc
exit 0

By using:

bash wget -qO- https://github.com/JBKahn/provisioning-local/raw/master/run.sh | sudo bash

you can have this script download and run and have the machine setup in no time. It handles the requirements, and starts up three of the applications after the provisioning is completed.

The Output

joseph@Batcave-Ubuntu:~/dev/provisioning-local$ sudo rm -fr ../../setup/base16-gnome/
[sudo] password for joseph:
joseph@Batcave-Ubuntu:~/dev/provisioning-local$ ansible-playbook setup.yml -i HOSTS --ask-sudo-pass
sudo password:

PLAY [a playbook to setup my local machine with my basic customizations] ******

GATHERING FACTS ***************************************************************
ok: [127.0.0.1]

TASK: [common | base16 - checkout repo] ***************************************
changed: [127.0.0.1]

TASK: [common | set file permissions] *****************************************
changed: [127.0.0.1]

TASK: [common | base16 - install monokai dark] ********************************
changed: [127.0.0.1]

TASK: [common | base16 - set system font to source code pro] ******************
changed: [127.0.0.1]

TASK: [common | base16 - dont use default system font in terminal] ************
changed: [127.0.0.1]

TASK: [common | base16 - set default terminal profile] ************************
changed: [127.0.0.1]

PLAY RECAP ********************************************************************
127.0.0.1                  : ok=7    changed=6    unreachable=0    failed=0

Here after I run it, it will tell me how many of them at ran (changed) or did not require any more work (ok). There are some tasks (i.e. commands) which will always be 'changed' unless they are conditionally executed or a module is written to explain to Ansible how to check and see if something has changed.

That's It

That's all you need to get this up and running and provisioning your own machine. I hope to be able to write a blog post about using it with Vagrant and what the differences are. For now, I'll leave you with a few relevant links:

Pre-commit Hooks: The Good, The Bad & The Ugly

Joseph Kahn — Fri, 10 Apr 2020 17:28:59 +0000

Migrating my old blog

As part of migrating my older blog posts, this one was posted on June 29th, 2014.

Introduction

I know it has been a while since I put the page up but I've been busy. I've been preparing for a big interview, working full time at Wave and taking a class at night twice a week. So I decided to take an hour and write up a quick post about something I've been working on recently (hopefully it isn't too badly written). Since I've been working on my pre-commit hook at work, I'll write a bit about my thoughts on them. I hope by taking the time to write this, I can get in the groove of writing more often.

The Good

The good thing about pre-commit hooks is that you can "stop" yourself, and team members, from pushing code that doesn't pass through various checks. You can't really stop this because they can force push or not use the hook, but you can discourage it. A lot of these checks tend to be running the changed files through a linter and a handful of custom scripts. I've worked quite a bit with Django so I'll be using some examples from my experiences within the post but these could apply to any language.

A simple example of a custom script I plan to write, for working with Django, is to search for any url in any git added html files. Any local links or assets that use direct linking should be caught by the hook. This is bad practice and can lead to a headache later on if the static directory changes name or the URL of a page changes. I've run into this in the past, where I wanted to make the URLs of a website more friendly and uniform. The old templates had direct links to pages throughout and as a result, I had to fix them by hand. I named each url and used Django's url tag instead. This way if we decide to change the url of one of our pages we could simply change the one line in the urls.py file. After doing that, each link to that page will be generated on the fly using the tag. I would also like to add another check that looks at any file name that does not end in 'thirdparty.html' and not allow any direct links at all. The rest could be stored in a context processor so that there is one place for all outside links that don't belong to third party code.

The Bad

So far I've only touched on what I like about pre-commit hooks but there is also a bad side. It's easy to fall into a trap where developers ignore pre-commit hook results or treated them as 'good to merge' test. The former tends to happen when using pre-commit hooks with legacy code which does not meet the requirements of new hooks. In that situation the team has to make an active decision to fix anything found by the script in order for it to be effective. This means making legacy code match the new standards every time you alter an older file. This can be annoying when a one line change nets fixing hundreds of linting, indentation and whitespace errors. This can lead to skipping the pre-commit hook or ignoring its output if not agreed upon by the team. The opposing effect is that a developer may view pre-commit hook results as being much more valuable than it really is. It's possible for there to be holes or missing checks in the hook. This is especially true for code design standards that are not community standards. The take away in this case should be to treat it as one step in a checklist before merging code, not the checklist itself.

The Ugly

Now to get down to an actual example. I'll be including a part of the pre-commit hook I use at work, with anything Wave specific removed:

The above hook can be broken down into several sections:

Check for duplicate south migration files. This is used to avoid merging branches into master with the same migration number as an existing migration. This would make it harder to rollback code and can happen to long running branches.
Run python files through flake8 for style and simple linting checks (i,e. unnecessary imports) as well as print statement and debug statement checkers.
Run coffee script and javascript files through jshint and coffee-jshint linters.
Run JSON files through jsonlint, good for node, grunt and bower.

This is a work in progress as I hope to continuously improve this script as I find more things to lint and improvements for this script.

Things I'd like to do:

Use the cached version of the files added that git diff can output. I would pass these to the linters instead so that I can lint only the copies staged for commit and not the local copy in the folder.
Work on a 'jshintrc' file so that the js linter will not use the default settings and will only do the checks I want it to do.
CSS/LESS linting; I'm uncertain it's of huge value but there are a few things I might add custom scripts to check for.
The URLs and the direct links in the html files of Django projects.

DEV Community: Joseph Kahn

Vagrant Plugin: Back 2 Tha Hood

Migrating my old blog

Introduction

Woes With Creating The Plugin

It Begins

The Files

Gemfile

Rakefile

vagrant-custom-command.gemspec

vagrant-project-setup.rb

plugin.rb

command.rb

More Complex command.rb

Building, Testing & Usage

Conclusion

For a Few Ansible Modules More

Migrating my old blog

Introduction

Conditional Execution

Module Writing

The Getter and Setter

Argument Accessing

Argument Parsing

The Skeleton

Documentation

Usage

Testing

Conclusion

That's It

Ansible or: How I Learned to Stop Wasting Time Setting Up My Computer and Script It

Migrating my old blog

Introduction

The File Structure & Content

The HOSTS file

The Requirements File

The Master Playbook

Common Roles: Variables main.yml

Common Roles: Files

Common Tasks

The script

The Output

That's It

Pre-commit Hooks: The Good, The Bad & The Ugly

Migrating my old blog

Introduction

The Good

The Bad

The Ugly

More Complex `command.rb`