DEV Community: Valentino Stoll

Twilio Powered Minecraft Server on AWS

Valentino Stoll — Tue, 28 Apr 2020 01:33:23 +0000

What I built

A way to control access to a Minecraft server hosted on an EC2 instance via SMS.

Background

This all came about when my kids wanted to play Minecraft with their friends. While Realms seemed like a decent choice, I wanted something with more control and a way to allow other parents to easily add their kid(s) and start/stop the server.

The Full Picture

An administrator sends an SMS to a Twilio number with a command (e.g. status). Twilio then sends an incoming SMS web request to an Amazon API Gateway endpoint. This endpoint hands off the Twilio event and request data to an AWS Lambda function which validates and processes the command to perform one of the following:

1) startup – Starts the EC2 instance.
2) shutdown – Stops the EC2 instance.
3) whitelist add 1.2.3.4 as beetlejuice – Allows a new user to connect to the Minecraft Server.
4) whitelist remove 1.2.3.4 – Dis-allows an existing user from connecting to the Minecraft Server.
5) status – Check the status to see if the instance is running.

Introducing:

codenamev / twilio-aws-minecraft

Control a minecraft server in AWS via Twilio SMS

Setup

Prerequisites

setting up a Minecraft Server on an EC2 instance. I hope to update this at some point with a slimmer set of instructions that I now use, but this got me where I needed.
Set up a Lambda to receive Twilio events

With the above in place, you should have a working Minecraft Server running in EC2, and a simple Lambda setup with the API Gateway to receive events from Twilio.

Lambda Environment Variables

Make sure to add the following environment variables to your Lambda function:

Key	Description
ADMIN_NUMBERS	A comma delimited list of administrator phone numbers (e.g. `+12125552368,+12125551234`)
TWILIO_ACCOUNT_SID	Your primary Twilio account identifier - find this in the Console.
TWILIO_AUTH_TOKEN	Used to authenticate - just like the above, you'll find this here.
EC2_INSTANCE_ID	The ID of the EC2 Instance you created above. You can find it here.
EC2_REGION	The region your EC2 instance is running in (e.g. `us-east-1b`)
EC2_SECURITY_GROUP_ID	The ID of the security group that is assigned to the EC2 Instance (e.g. `sg-1234567ab89cd0987`)
TWILIO_NUMBER	the Twilio phone number you setup to receive SMS (e.g. `+12125552368`, also found in your Twilio Console)
MINECRAFT_HOST	The server domain or IP that points to your Minecraft Server (e.g. `minecraft.example.com`)
MINECRAFT_PORT	e.g. `25565`
REQUEST_URL	The API Gateway endpoint you setup for the Lambda function and added as the SMS incoming webhook (e.g. `https://1somehash23.execute-api.us-east-1.amazonaws.com/prod/message`)

Lambda permissions

You will have to add the following actions to the permissions for your Lambda function:

Action	Description
`ec2:AuthorizeSecurityGroupIngress`	Allows adding new IPs to the server whitelist
`ec2:DescribeInstanceStatus`	Allows checking the status of the instance
`ec2:DescribeSecurityGroups`	Allows looking for existing IPs in the server whitelist
`ec2:RevokeSecurityGroupIngress`	Allows removing a whitelisted IP from the server
`ec2:StartInstances`	Allows starting the instance
`ec2:StopInstances`	Allows stopping the instance
`logs:CreateLogStream`	Allows capturing output to logs
`logs:PutLogEvents`	Allows saving log events

Next, you'll want to checkout my twilio-aws-minecraft repo:

git clone https://github.com/codenamev/twilio-aws-minecraft.git \
cd twilio-aws-minecraft \
zip -r twilio_function.zip ./

That will create a packaged lambda script with all required libraries in .zip form that you can upload to replace the existing Lambda script you created in the previous step.

Once you've saved the newly uploaded Lambda and deployed, you can test out your new EC2 manager via SMS!

Somessues I faced:

Lambda IPs for EC2 whitelist
Minecraft Forge and minecraft.service setup issues
Instance startup not starting minecraft.service
UserData
GameMode(s)
Configuring OPs
Debugging

How to Make Colors With Ruby and Bitwise Operations

Valentino Stoll — Fri, 11 Oct 2019 21:05:00 +0000

In our last lesson, we utilized the badass (P)ulse (W)idth (M)odulation tech to breathe life into a simple LED. Here, we're going to take this to new levels and apply it to multiple channels.

Unfortunately, the PWM hardware included with the Raspberry B+ is only capable of communicating over a single channel. Since we have 3 channels we need to talk to (for each of our three colors), we'll have to go soft and crank this up with software: softPwm.

First: How the RGB LED works

The RGB LED I'm working with has four pins. Three of these pins take input that control the brightness level for (R)ed, (G)reen, and (B)lue. This allows us to take advantage of the additive color system to create all kinds of colors by supplying different proportions of these three types of light (e.g. fully bright red and fully bright blue will appear purple).

Our Problem: How can we easily store color values in RGB format as a percentage of luminance?

Hexadecimal To The Rescue!

I know what you're thinking:

Don't lose me just yet. There's some hidden gold here! The magic of bitwise operations.

The binary 11111111 is equal to 255. Luckily for us, this binary representation maps perfectly to Hexadecimal, as FF == 255!

11111111 00000000 00000000

RED GREEN BLUE

What this means is that we can use Hexadecimal literals in Ruby, and as a result start to see some serious gains. First off, Hexadecimal numbers make a lot of sense to us as colors already. These are the Ruby equivalent of the CSS versions you might be familiar with:

Color	Hexadecimal Representation in Ruby
Red	`0xFF0000`
Green	`0x00FF00`
Blue	`0x0000FF`
Purple	`0xFF00FF`

The real value here comes with the calculations we can make. Using hexadecimal literals makes some things easier. We can extract each red, green, or blue value from any given hexadecimal color by AND'ing the bits of the maxed out version of our target.

Let's say we have the color 0xFF1133 . This has a binary form of:

11111111 00010001 00110011

We can extract the value of just the blue color 0x0000FF by using the bitwise AND (&) operator to cancel out all non-blue bits. So:

0xFF1133 & 0x0000FF == 0x000033

This is because all the 0 bits in our pure blue color 0x0000FF override any bits set in any of the other color positions, and any 0 bits in our 0xFF0033 color override any 1 bits in our pure blue.

  11111111 00010001 00110011
& 00000000 00000000 11111111
               ⇣
  00000000 00000000 00110011

This works great for our blue value, but for the others we have to take this one step further. If we were to perform this same bitwise AND on the pure green color 0x00FF00 then we will get what we want, except with a lot of trailing zeros.

  11111111 00010001 00110011
& 00000000 11111111 00000000
               ⇣
  00000000 00010001 00000000

In order to target that middle section of the resulting bitwise operation, we need to shift the bits to the right to get rid of the extra zeros. To do this, we can use the bitwise right shift operator >> and push them eight places to the right.

00000000 00010001 00000000 >> 8
             ⇣
00000000 00000000 00010001

Bingo! Unity. With this in mind, we can translate it to Ruby and apply these lightning fast calculations to extract percentages of each RGB color value from a hexadecimal numeric:

# Given the folowing color
color = 0x6495ED
percentage_blue = (color & 0x0000FF) * 100 / 255 # => 92%
percentage_green = ((color & 0x00FF00) >> 8) * 100 / 255 # => 58%
percentage_red = ((color & 0xFF0000) >> 16) * 100 / 255 # => 39%

All Together Now

Since the duty cycle value is essential the percentage that the signal will be ON, we can just extract the RGB color percentages from a hexadecimal color value and set these to the duty cycle of each RGB pin of the LED.

Ok, ok, all this binary mumbo jumbo and all we get is a bunch of flashing colors!? Let's turn the dial all the way up to eleven and add a simple twist: have a button trigger a random color.

BOOM! We have made a game! Guess what color the LED will be when you press the button?

Special thank you to Calle Erlandsson's fantastic write-up on bitwise operations in Ruby.

What is your favorite open source project?

Valentino Stoll — Sat, 28 Sep 2019 00:11:43 +0000

Gearing up for Hacktoberfest here. Aren't you!?

I'm sure I'm not alone in just jumping on GitHub come October and plucking the first few projects I find interesting. Instead, I'd love to hear what projects people can't live without and hope I can help this time around.

Personally, there are quite a few I depend on for day-to-day smooth workflows git-reflow, rubocop/standard, rspec, rails, ruby. What about you?

The Great Breathing Ruby LED

Valentino Stoll — Fri, 27 Sep 2019 21:21:23 +0000

This is part 4 in my Sunfounder Superkit series.

In our last experiment, you were probably bedazzled by the alluring display of LEDs. This time around, we'll go from bedazzling to mesmerizing and use sweet technological magic to make an LED breathe.

Admittedly, not that kind of magic. Lucky for you, I'm taking big risks, breaking the Magician's Code and revealing the secret behind the trick: Pulse Width Modulation.

Pulse Width Modulation (PWM)

To get a good understanding of what the heck this bad girl is, let's sprint down history lane and attempt to reproduce our results analog style. Before the wonderful invention of PWM, most electrical signalswere handled continuously by analyzing voltage over time.

via GIPHY

As you might imagine, this can lead to huge wastes in energy while constantly having to check for changes to the output voltage. Most electronic applications will likely not need this continuous stream of signal, and to take advantage of this, the GPIO pins on the Raspberry Pi rely on digital signals. Instead of treating the signal as a continuous stream, digital signals stream a sequence of discrete values.

via GIPHY

Ok, so where the hell does Pulse Width Modulation come in!? PWM takes advantage of these digital signals by rapidly turning the voltage off and then turning them on again to a slightly higher or lower voltage. In our example, this choppy signal will simulate a smooth increase in voltage (LED getting brighter) or decrease in voltage (LED getting dimmer).

Credit: https://www.instructables.com/id/Fade-LED-Arduino-Analog-Output-Tinkercad/

PWM with Ruby!

Ok, enough with the circuits already, and on to the fun part: Ruby! Luckily for us, the rpi_gpio gem comes with PWM built-in. With a little elbow grease we can start the mesmerization process.

Wrap Up

We're really starting to dive into the embedded systems waters here. PWM is a great way to finely control things we want to... modulate. I'm out of ideas for alternatives on this one, but next time we'll take LEDs to new colored dimensions and I'll invent the simplest of games :-)

Ruby Driven LED Arrays

Valentino Stoll — Fri, 20 Sep 2019 21:08:00 +0000

This is part 3 in my Sunfounder Superkit series.

We’ve blinked it, hooked it to a button, and now for some serious fun: lots of them! This exercise really tests your patience with the breadboard.

(1 LED + 1 resistor + 2 wires) * 8 = 24 things to plug in

Once this was all setup (~15min), it looked great! I was really pumped to start playing around with so many twinkling lights!

Fully wired up LED array

The Ruby Journey Begins

So I crack open the python script and start to pipe it into ruby, because Ruby. I get to the line where it sets up the LEDs with rpi_gpio and what do I find? No multi-pin setup support.

I know what you’re thinking, “Just move on and test it with Python or C already”. Helllllll no! I didn’t make it all this way to have to mangle together some half-baked snake-oil code (in all fairness, the C code is quite nice to work with, and I did try it out before converting this **blush**).

It’s time to pony it up and ride the open source winds into the sunset.

Fortunately, the Ruby documentation on creating extension libraries is really spectacular. With a little fiddling, I was able to get a working multi-pin GPIO.setup method, and PR submitted. Yay open source!

Before the PR merged, I had to build the gem locally ( rake compile) and reference it using the gem “rpi_gpio", path: /path/to/local/rpi_gpio in my Gemfile. I'll float by all the gripes I have with developing Ruby extensions and dive right into the goal: a real-life self-made majestic racing marquee!

So, that was a fun ride. The creative corner of my brain came knocking through the wall like the Kool-aid man though. I just have to spin this drab loop into something a little more fun and, dare I say, practical?

A Simple Binary Counter

Ok, not the most amazingly-fantastic-best-idea-ever, but I'm looking for a small twist to my existing example so I can escape the monotony of this textbook tutorial. With our new lovely Ruby script, we can make some small tweaks and have our LEDs count all the way to two hundred and fifty five (code here to save space).

Impressive, I know, but practical? Well my creativity was oozing today (with this many twinkling lights how could it not!?) and discovered how I might be able to turn this into my new favorite desk toy.

To the Future and Beyond

I'm a pretty big fan of the Pomodoro Technique, and since I now made something that can count, I'll likely jack this bad girl up and spin this sweet little counter into a meaningful meter for time spent. A visual Pomodoro Meter if you will.

Or maybe, instead of counting time, it could count the number of PRs I have open on GitHub. So many possibilities!

Well that's all for now folks. I hope you enjoyed the sunset. If you decide to boldly clone this and make something of your own I'd love to hear about it :-)

Tune in next week, where I venture into mutli-colored LED land and a slowly pulsing light show.

Ruby driven Raspberry Pi buttons

Valentino Stoll — Fri, 13 Sep 2019 21:00:00 +0000

This is part 2 in my Sunfounder Superkit series.

Blinking an LED was too easy! Let's make this piece interactive. Buttons are exactly the magic I'm looking for. Push it, and do something. ANYTHING! Let's keep it simple and build off of our first simple LED hookup.

Wiring a button to an LED on a Raspberry Pi B+

With that in place, and a little Ruby script, we can loop in the button to join the LED blinking fun.

The Great Ruby Takeaways

Ok, so maybe the circuit is not that exciting, but this is just the start! With this we've gained some exposure to both listening in on and controlling the state of GPIO pins. Writing this script also introduces us to some great aspects of working with Ruby programs from a lower level.

IO#flush and IO#sync

Firstly, STDOUT.flush. This will clear any buffered data to standard out allowing future calls to puts to replace what was there. Little did I know there is a flag you can set on an IO instance to immediately flush the output after every call with: STDOUT.sync = true

Kernel#at_exit

Ruby's Kernel comes with some great stuff for working with the command line. The fine folks over at HoneyBadger have a great writeup on different ways to exit a Ruby program, and some best practices. In this specific case, we can use the special at_exit callback to perform some code to reset the state of our GPIO pins (so we make sure not to set any fires).

Interrupt and SignalException

Lastly, because we want to ensure that our circuit resets itself whenever the program stops running, we need a way to catch both user-initiated interrupts (a.k.a Interrupt caused by a ctrl+c) and system-initiated interrupts (a.k.a. SignalException). While it may be more beneficial to trap unix signals, I went with the quick and dirty.

Hopefully you learned something today. I know I did! Tune in next time when I get serious with the breadboard and jam as many LEDs in it as I can (or have patience for).

Setting up a Raspberry Pi For Ruby

Valentino Stoll — Wed, 04 Sep 2019 15:01:00 +0000

Updated 9/6/19: This is part 1 of my Sunfounder Superkit Series.

Ruby comes pre-installed on Raspbian, but unfortunately is not the latest version. After a bit of headache, here’s what I learned and how to make it right.

My dream is to make a split-flap display, but since I have near-zero experience working with microcontrollers I decided to pick up Sunfounder’s Superkit (v3). To be honest, it was cheap, but also comes with lots of different components to play with. Let’s see what kind of lit experiments we can build.

Flashing the SD card

I decided to use the Raspbian OS (with desktop client), just to make the learning curve easier. After downloading the latest image, I then installed Etcher (as recommended) and this was incredibly easy to burn the image onto the micro SD.

Booting Up & Configuration

Booting up the Raspberry Pi couldn’t have been easier. Jammed the micro SD card delicately in, plugged the power cable and the HDMI, and ~1minute later I was ready to go.

First thing’s first, change the system password (because security).

passwd

One nice thing about the B+ is that you can remotely control the GPIO (more on this later). Unfortunately, both this feature, and ssh, are turned off by default. So, let’s flip the table on those bad girls.

Launch Raspberry Pi Configuration from the Preferences menu
Navigate to the Interfaces tab
Select Enabled next to SSH

Update : I’ve started a small repo for all my experiments that includes a handy script to automate the remaining setup.

For me, the latest gpio software was out of date and was raising hell. Let’s start this puppy up fresh:

sudo apt-get update && sudo apt-get upgrade

Boom. System configured, dependencies updated, and GPIO configured properly. Check out all the wonderful pins we get access to in the GPIO on a B+!

Raspberry Pi B+ GPIO table

Ok, now Ruby time!

Setting up Ruby

As of today (Jun 2018), Raspbian comes with Ruby pre-installed! In order to get some of the gems with native-extensions to build properly, I just had to get ruby-dev

sudo apt-get install ruby-dev

Finally Blinking a Damn LED

After all that, you should be ready to use that Pi that’s just been collecting dust all these years. Just for fun, let’s perform the staple of what to do with your shiny new micro-controller: blinking an LED.

With a couple jumpers, an LED, and a 220Ω resistor we can safely control the state of the LED:

Raspberry Pi B+ blinking LED with Ruby!

Wrapping It Up

Alright, I get it, not too exciting. But hey, with this same setup I can see altering the code to, say, light up the LED when my tests fail.

Tune in for more of me documenting my dope ass crusade of micro-controlling justice.

The code behind GitHub's new Code Navigation

Valentino Stoll — Thu, 22 Aug 2019 15:00:00 +0000

GitHub recently announced two new code navigation features:

Jump to definition – quickly view the source for any given method.
Find all references – Identify all references of a function within the same repo.

I have been waiting a long time for these features to surface, and best of all, the code behind all this is open source!

From the top-level, they're using a Haskell library called Semantic. Semantic can parse, analyze, and compare source code. As of the last update to this post, it supports Ruby, JavaScript, TypeScript, Python, Go, PHP, Java, JSON, JSX, Haskell, and Markdown.

For a full list of examples, check out the Semantic usage docs. It comes with a great CLI that allows you to:

Generate parse trees

$ semantic parse test.A.py
(Statements
  (Annotation
    (Function
      (Identifier)
      (Identifier)
      (Return
        (Identifier)))
    (Empty))
  (Call
    (Identifier)
    (Call
      (Identifier)
      (TextElement)
      (Empty))
    (Empty)))

Generate an abstract syntax tree (AST)

Generate syntax aware diffs

$ semantic diff test.A.py test.B.py
(Statements
  (Annotation
    (Function
    { (Identifier)
    ->(Identifier) }
      (Identifier)
      (Return
        (Identifier)))
    (Empty))
  (Call
    (Identifier)
    (Call
    { (Identifier)
    ->(Identifier) }
      (TextElement)
      (Empty))
    (Empty)))

Display import call graphs

semantic graph main.py | dot -Tsvg > main.html && open main.html

Display call graphs

semantic graph --calls main.py | dot -Tsvg > main.html && open main.html

Making Sense of Your Code

The code parsing is done with the tree-sitter library which appears to be C with a handful of language bindings (the cli is in Rust!).

[Tree-sitter] can build a concrete syntax tree for a source file and efficiently update the syntax tree as the source file is edited

Languages are supported by writing language-specific parsers. These parsers primarily consist of two parts:

Grammar – contains various rules for how the language is formatted
Lexical Analysis – patterning individual character groups (for lexing).

Example Grammar for parsing simple ruby methods from tree-sitter-ruby:

{
  "name": "ruby",
  "word": "identifier",
  "rules": {
    "method": {
      "type": "SEQ",
      "members": [
        {
          "type": "STRING",
          "value": "def"
        },
        {
          "type": "SYMBOL",
          "name": "_method_rest"
        }
      ]
    }
  }
}

https://github.com/tree-sitter/tree-sitter-ruby/blob/master/src/grammar.json

Tree-sitter has an awesome Playground where you can explore the parse trees it generates for any given code.

When you feed code to the Semantic library, it generates parse trees using tree-sitter, and then uses the language's grammar (e.g. from tree-sitter-ruby) to parse the parse trees and map AST nodes (many of which are already available from the library itself for: literals, expressions, statements, and types).

While they hope to drop this step in the future, once the AST nodes are mapped, a second round of parsing is done to wrap them into data structures the library can comprehend (for more on this, see the "Assignment" documentation).

There are quite a few more things that happen to analyze and compare source code, but I'll leave the explanation it to their great documentation on program analysis.

Thank you GitHub ❤️ Keep up the solid work 😄

How to fix all Ruby "Failed to build gem native extension" Errors in OS X

Valentino Stoll — Thu, 11 Jul 2019 21:00:00 +0000

cp /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg ~/Desktop && open ~/Desktop/macOS_SDK_headers_for_macOS_10.14.pkg

Boom. Now start removing all those pesky overrides you set in ~/.bundle/config

All credit goes to the amazing João Rocha da Silva. Be sure to check out his detailed write-up on why you need to do this. TLDR; Mac OS X 10.14 no longer places the include libraries in the standard /usr/include directory and locked by SIP.

Custom Git Workflows with the latest git-reflow

Valentino Stoll — Sat, 10 Nov 2018 21:16:06 +0000

This release includes the culmination of several years of effort to allow you to customize our process to suit your specific needs. We’ve dropped a heavy dependency (gli) and introduced a pretty slick DSL for creating new git commands or override our existing ones.

I am really excited to announce the released version 0.9.0 of the git_reflow gem!

Custom Workflows

Git-reflow’s default process isn’t meant to fit every team, which is why we’ve introduced Workflows.

We’ve introduced a special file that can contain customizations to the default process. With this, you can:

Add hooks to be run before, or after any command
Use one of our pre-configured workflows as a basis for your own
Override any of the default commands
Create new commands

Here is an example showcasing all the various ways you can customize the process:

# Use a pre-defined workflow
use "FlatMergeWorkflow"

# Do something before the `start` command
before :start do
  say "Booya! Started this bad girl."
end

# Do something after the `deliver` command
after :deliver do
  run "say 'Well done!'"
end

# Override the `start` command to auto-name feature branches
command :start, arguments: { feature_branch: nil }, flags: { base: 'dev' }, switches: { loud: false } do |**params|
  base_branch    = params[:base]
  feature_branch = params[:feature_branch] || "feature-#{Time.now.strftime("%Y-%m-%d")}"

  run_command_with_label "git checkout #{base_branch}"
  run_command_with_label "git pull origin #{base_branch}"
  run_command_with_label "git push origin #{base_branch}:refs/heads/#{feature_branch}"
  run_command_with_label "git checkout --track -b #{feature_branch} origin/#{feature_branch}"
end

# Create a new command
# Makes `git reflow boom` available in the command line.
command :boom do
  GitReflow.say "Boom."
end

You can define a unique workflow for each of your projects simply by placing it in the root of your project, or by setting a special reflow.workflow git-config setting with the full path of the file.

The block that you pass to each of the git-reflow reserved commands (use, before, after, command) are executed in the context of our Core workflow class. This gives you the ability to use any code available to the GitReflow module. We encourage you to use as much of GitReflow’s tooling as possible so that you can take full advantage of getting the current branch name, logging, exception-handling, colorized output, and several other features that are baked into GitReflow’s internals. For a full list of what’s available, see Available Workflow Helpers on our wiki.

Ramping Up To 1.0

With the re-workings of Workflows in place, we’re getting ready for our first major release! Our current plans are to focus on:

Introduce Pivotal Tracker & Trello workflows
Add GitLab integration
Add an Open Source workflow

We’re looking forward to some exciting times! We’d love to hear all the interesting ways you use these new workflows, so please don’t hesitate to let us know.

Happy hacking!