DEV Community: Benjamin Congdon

A Gentle Nudge

Benjamin Congdon — Sun, 03 Nov 2019 07:12:00 +0000

Earlier this year, I was persuaded to start paper journaling. Paper journaling is unlikely to replace digital task management for me, but there definitely are some benefits to an analog, offline journal.

The paper journal that I keep nominally follows the Bullet Journal method, but only loosely. I’ve tried a couple different techniques over the year. Some, like retrospectively writing down what I did in a given day, felt tedious and redundant given that I already capture that type of information in other ways.

One technique that I’ve found to be useful is “habit trackers”. The idea is simple: each month, make a list of 4-5 things that you either want to do or abstain from doing each day. Then, take a page in your notebook and made headers for each of these activities, and a row for each day. At the end of each day, mark which activities you completed. For me, I used this to track habits that didn’t need to get done everyday, but would improve my sense of productivity and personal satisfaction if I did do them.

I’ve written before that some tasks really don’t belong in a task tracker like Todoist. For habits that are already on “autopilot” (for me, running daily), having a daily repeating task just clutters my dashboard. For habits that are less of a slam dunk (for me, meditating daily), I find that I’m actually less likely to do them if they appear as anitem on my todo list. (“Procrastiworking,” anyone?)

Since habits are tracked as completed or uncompleted at the end of the day, there’s less of a sense of guilt for not doing something, but you still get a little dopamine hit for crossing off an accomplished habit. You aren’t shamed into deferring a todo item to tomorrow; you simply didn’t do that habit today. This practice also bakes in a bit of end-of-day reflection, which can be helpful for other reasons.

This was on my mind as I was reading Michael Lopp’s Being Geek. When I read his essay on what he calls “The Trickle List”, I noticed it closely resembled my discovered system of habit tracking:

My first excursion into the word trickle was a productivity article called Trickle Theory. The argument was simple. You can do more than you think with small, consistent investments of your time.

To understand the Trickle List, you need to first look at the headers at the top of the list. These are the heart of the list and how you define them is how you define what you want to do.

The items on your Trickle List don’t need to be huge, in fact, as we’ll learn in a moment, the bigger they are, the less likely you’ll do them. What they need to be is aligned with where you’re headed. However small, they need to be a daily reminder that you’re headed somewhere.

Lopp’s “trickle list” emphasizes a couple points which makes this type of habit tracking successful:

The habit should be useful to you personally, not a vague aspiration.
The habit should be small.
The habit should be measurable as a “yes”/“no”.

At its best, the “trickle list” gives your frantic “daily self” a gentle nudge from your more rational “long-term thinking” self.

Personally, I find reevaluating my habits monthly to be a good cadence. If I’ve accomplished something everyday for the entire month, I probably don’t need to include that on next month’s list. Similarly, if I have a habit that I only managed to do a handful of times during the month, I probably need to evaluate why I wasn’t successful. Is the habit too big? Am I just not motivated enough? What environmental factors prevented me from doing what I’d planned?

The result is two levels of reflection:

The daily reflection of “what could I, given the limits of my schedule, done better today” and…
The monthly reflection of “at a meta level, what types of activities do I want to encourage myself to do”.

I’ve started to think more about this notion of “gentle nudging.” Where else in our environment does it make sense to add moments of pause or slightly change our environment to slowly bend ourselves towards self-improvement?

Finding Flow in Haskell

Benjamin Congdon — Thu, 25 Apr 2019 14:30:10 +0000

For the last couple of months, I’ve been working on-and-off in Haskell for my programming languages class. I’ve written previously about my preference for declarative functional syntax, but I’d never really spent the time to learn a pure Functional Programming™️ language.

tl;dr: I’m glad I learned Haskell. It’s great at expressing some programs that are difficult (or practically impossible) to express in other languages. However, this necessarily makes other types of programs difficult to write in Haskell.

Keep in mind that I’ve learned just enough Haskell to “be dangerous” (both in the sense of being comfortable enough to write moderately complex programs and in the sense that I’m naive enough to be potentially misguided à la Dunning-Kruger). Take this with a grain of salt.

The Entry Point: Higher-Order Functions

I was thinking about the first time I encountered anything that looked remotely like Functional Programming. I think that first interaction was probably when I learned Javascript’s Array.{map, reduce, filter}. These higher-order functions, so-called because they take a function as an argument, make you think about the execution of a program in a radically different way.

Internalizing the concept of a higher-order function requires a similar type of mental leap as that of learning recursion. If you follow the “traditional” imperative path of learning to program, recursion is a common stumbling block because it forces you to think about functions not merely as procedures, but as abstracted procedures that can be called within themselves. Higher-order functions impose a similar stumbling block: you have to think of functions as data. Just as you manipulate and pass variables, you can combine and pass functions.

Once comfortable with map, reduce, filter, and the like, you can express a crazy amount of functionality using HOFs. Some of my favorite parts of Ruby and Rust are in its support for just this small set of HOFs.

Haskell is not `Array.map`, but “more”

From the above section, you can see that my initial mental model of functional programming was fairly narrow. It’s that thing that allows you to move around functions like you would variables… Well, kinda. Of course, thoughts about functional programming also elicited the obligatory thoughts of “there are no side effects” and “everything is a function”

Haskell does make generous use of HOFs. It uses a lot of functions (though not everything is a function) and, indeed, it’s pretty difficult to have unintentional side-effects in Haskell programs. But, to sell Haskell as “what if we took Array.map and made a language around that” would be missing the point.

I’d argue that what makes Haskell unique and useful is an altogether different set of features:

Pattern Matching
Immutability
Algebraic Data Types

These three features alone (as well as some necessaries like functional composition) already make a compelling case for giving Haskell a try.

Pattern Matching

First-class pattern matching might be my favorite thing in Haskell. Take the following example:

mySearch :: Eq t => [t] -> t -> Bool
mySearch (x:xs) target = (x == target) || mySearch xs target
mySearch [] _ = False

>>> mySearch [1,2,3] 2
True
>>> mySearch [1,2,3] 4
False
>>> mySearch [] 10
False

mySearch checks if an element (target) is present in the list (which is the first argument). The first line pattern matches on a non-empty list. If the list is non-empty, x is the first element in the list, and xs is the rest of the list. If the list is empty, the second case will be used ([] pattern matches on the empty list), and we return False.

Pattern matching on lists is awesome in itself. You can also pattern match on specific values and on algebraic data types.

isMeaningOfLife :: Maybe Int -> Bool
isMeaningOfLife (Just 42) = True
isMeaningOfLife _ = False

>>> isMeaningOfLife Nothing
False
>>> isMeaningOfLife (Just 42)
True

This, of course, just scratches the surface of what’s possible. You can pattern match each variable independently and have tons of different cases for a given function. This becomes super powerful in more complicated programs and is a good shorthand for decomposing complicated functions into their simpler cases. (Many recursive functions written in Haskell feel like writing mathematical proofs; “there are 3 cases to consider…”)

My pet theory on why pattern matching is so great in Haskell is that for purely functional programs, a function’s output is much more tightly bound to its input. As side-effects are prohibited, the only way a function can actually do anything is by returning something. Similarly, functional programming is syntactically more declarative. Pattern matching simply makes it a lot easier to write declarative pure functions. Patterns allow you to describe behaviors of a function under different forms of input without much syntactic boilerplate.

Immutability

Haskell expressions are immutable. Once you construct a piece of data, it cannot be changed. To change data (i.e. appending to a list, adding a key to a dictionary, etc.) a new data structure is created. This prevents large classes of bugs. You never have to care about some other thread or function changing the internal state of some data structure because, well, they can’t.

Other language communities have picked up on immutability too, either as libraries (notably Google’s Guava for Java and immutable-js for Javascript) or as language defaults (like in Rust).

Immutability can be a real pain to deal with. (i.e. “I just want to do in-place array operations!”) However, I think Haskell mostly gets away with it because it uses immutability everywhere. When mutating internal state isn’t an option, it becomes less of a temptation — you start to think in ways that don’t necessitate mutability.

Algebraic Data Types

“Algebraic Data Type” is one of those scary-sounding phrases that end up being pretty harmless when explained.

Put simply, there are product types (which are analogous to tuples or structs) and sum types (which are kinda like enums with an associated value.

-- Example Product Type
data Point = Point Int Int

-- Example Sum Type
data MyMaybe a = Nothing
               | Just a

The product type Point is a type that contains 2 integers. For example, we could represent the origin as (Point 0 0).

Sum types are like enums, in that there is a set of valid symbolic forms for the given type. MyMaybe is parametrized over type a (like List<T> in an OOP language). It can have the form Nothing or Just a. This is a useful way to describe optional values. For example, if we wanted to describe an optional integer, we could have a function that returns Just 42 if the operation is successful and Nothing otherwise.

Sum types are especially powerful, as they can easily be used to represent syntax trees:

data Exp = IntExp Int
         | PlusExp Expression Expression
         | IfExp Exp Exp Exp
         | GreaterThanExp Exp Exp

>>> parse "if 1 > 0 then 2 else 3"
(IfExp (GreaterThanExp (IntExp 1) (IntExp 0)) (IntExp 2) (IntExp 3))

Note that sum types can be recursive. In PlusExp there are 2 subexpressions, each of which have their own internal syntax tree.

Recall that we can pattern match on algebraic data types, which means that if we wanted to write an evaluator for a syntax tree like the one presented above, we could do so pretty easily:

eval :: Expression -> Int
eval (IntExp i) = i
eval (PlusExp e1 e2) = v1 + v2 where
    e1 = eval e1
    e2 = eval e2
...

I really enjoy using ADTs (especially in combination with pattern matching) in Haskell. They’re a surprisingly good way of representing certain types of data, and it allows the type-checker to do a lot of reasoning on your behalf about the correctness of programs.

I’m encouraged by Rust’s typed enums and Swift’s associated value enums, both of which give similar functionality in a non-functional setting.

`<*>`, `<$>`, `>>=`, Oh My!

So with all that gushing about Haskell out of the way, I should mention some areas that frustrated me.

Haskell syntax is definitively not C-derived, so there’s a bit of a burden to relearn basic syntax. However, the real weirdness comes in when you start going a bit deeper into Haskell’s language features.

The basics of Haskell’s type system will be familiar for programmers of other languages. Functions take typed inputs and produce typed outputs. However, there are also type classes, which are kinda like protocols or traits for types.

Common type classes include Functor/Applicative, Foldable, and Monad. As these type classes are used so often (and have associated operations that are used frequently), Haskell has a number of syntactic operators that allow you to invoke type class functions easily. <$> is identical to Functor’s fmap, >>= is Monad’s bind. There are a bunch more operators, including a few variations of each of the ones I’ve mentioned so far. The result can be a bit of symbol soup, which is difficult to read at a glance.

I mean, there’s the very real question (if posed somewhat in jest) of “how do you pronounce <*>?”

To its credit, Haskell has a really excellent REPL, and you can ask it the type signature of an operator when you get confused.

>>> :t (>>=)
(>>=) :: Monad m => m a -> (a -> m b) -> m b

There’s also the concern that some types of programs just don’t lend themselves to being written in a functional way. For anything involving super heavy system programming, file IO, or GUI interactions, I don’t think I’d want to use Haskell.

I was also a bit surprised about the story for debugging in Haskell. It’s… not amazing (in my opinion). You can still do printf debugging with Debug.Trace, but putting trace as a shim in between function calls feels more obtrusive than a printf statement. To its credit, Haskell’s REPL is great, so you can debug smaller components there easily. GHCI (Haskell’s REPL) also has breakpoint support. Safe to say though, just as you’ll need to relearn parts of programming when picking up FP, you also need to relearn parts of debugging.

Aside: Category/Type Theory

As an aside, one can’t help but notice that the average Haskell programmer seems to be more interested in Category and/or Type theory (also formal methods) than, say, the average Python developer.

Type theory has always been something that I “feel” like I should be interested in, but more from an intellectual perspective than for any practical reason.

At the risk of sounding dismissive, I think you can probably ignore most of the discussions about “a monad being just a monoid in the category of endofunctors” and still get along just fine in Haskell.

Conclusions

logether, I find the programming experience in Haskell to be really enjoyable. When working in a domain well-suited to Haskell’s strengths, I find it pretty easy to get into a flow state. I don’t feel like I’m fighting thetype-checker; rather, it feels like it’s keeping me honest.

I’m sure this doesn’t generalize, but when working in Haskell I get the feeling that if my code passes the type-checker, I’m reasonably confident that I’ve written a “correct” program. Many people also feel this about Rust’s type-checker. It’s a refreshing feeling. When working in Go or Python (or Javascript, for that matter), writing code feels like just the first half of the battle — you need to exercise it at runtime to build confidence in its correctness. Of course, semantic bugs are easy to commit in Haskell. But, just as a baseline, I feel more confident in “onced-over” Haskell code than I would code written in other languages.

I’m glad I took the time to learn Haskell. It comes at writing programs from a very different perspective than languages I’ve used before, and it’s made me more attuned to aspects of type systems that aren’t as easily explored in non-functional languages.

Live Refreshing Cargo Docs

Benjamin Congdon — Wed, 22 Aug 2018 21:03:28 +0000

rustdoc is a great tool, but as of now there isn’t an official way to have its generated docs refresh as you make edits. Running cargo doc with the --open argument will open the generated docs in browser window. If you make changes to your source code, you’ll need to re-run cargo doc to have the changes reflected in your browser. By chaining together a few other Rust tools, we can pretty easily get the functionality of live-reloading docs.

We’ll be 2 tools:

cargo-watch to watch when our project’s source has changed and trigger the docs rebuild.
http as a static server for the generated docs.
- This isn’t strictly necessary: As mentioned earlier, you can use the --open argument with cargo doc to view docs using the file:// protocol, but I like having a localhost address to visit.

I’ll assume that you already have cargo installed. If you don’t, then you can use rustup to install it.

Step 1: Install `cargo-watch` and `https`

First, install both commands we’ll need:

cargo install cargo-watch https

Step 2: Run Watch Command

Now, navigate to the project directory that contains your Cargo.toml file. Then, run the following:

cargo watch -s 'cargo doc && http target/doc'

You should now see something like the following:

[Running cargo doc && http target/doc]
 Documenting docs-example v0.1.0 (file:///path/to/project/docs-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.43s
Hosting "target/doc" on port 8000 without TLS...
Ctrl-C to stop.

You can visit localhost:8000 in your browser to view the generated docs. (Note: you’ll initially see a directory page, so you’ll have to navigate to one of the package listings to see actual docs)

If you make any changes to any source file in your project, you’ll see that cargo-watch will automatically re-run cargo doc and restart the file server. You should be able to refresh your browser and see the documentation changes live.

Bonus: Auto-refresh

One annoyance with this approach is that editing a source file does not trigger a refresh in the browser (i.e. you still have to refresh the page manually).

If you’re fine with using a node-based tool, browser-sync fixes this problem.

Install with:

npm install -g browser-sync

… then edit your cargo-watch command to use browser-sync instead of http:

cargo watch -s 'cargo doc && browser-sync start --ss target/doc -s target/doc --directory --no-open'

Now, visit localhost:3000 (or whichever port is listed on startup). Any subsequent changes to your Rust source code will now trigger any open browsers to refresh automatically with the updated documentation.

I’ve found that using this tweak to have auto-refreshing docs makes it much more pleasant to document my Rust crates. I’d love to see if there’s a more streamlined way to have auto-refreshing docs – be sure to let me know! 😄

More Information

To learn the basics of rustdoc (which powers cargo doc), check this chapter in the rust book.
These issues give context as to why this functionality isn’t officially supported (as of now): cargo#1472, cargo#4966

Lossy Numeric Casting in Rust

Benjamin Congdon — Mon, 06 Aug 2018 20:28:01 +0000

Recently, I was trying out clippy — a new linting and static analysis tool for Rust, when I ran into a lint warning that wasn’t immediately clear to me: warning: casting u8 to u16 may become silently lossy if types change.

For context, this is the snippet of code I ran through clippy:

fn main() {
    let a: u8 = 0x12;
    let b: u8 = 0x34;
    assert_eq!(silly_max(a, b), b as u16);
}

fn silly_max(a: u8, b: u8) -> u16 {
    let x = a as u16;
    let y = b as u16;

    if x > y {
        x
    } else {
        y
    }
}

Essentially, we take two 8-bit unsigned integers, cast them to 16-bit unsigned integers, and compare the casted results, returning the greater of the two.

Running clippy on this code, this is the lint error I received:

warning: casting u8 to u16 may become silently lossy if types change
 --> src/main.rs:8:13
  |
8 | let x = a as u16;
  | ^^^^^^^^ help: try: `u16::from(a)`
  |
  = note: #[warn(cast_lossless)] on by default
  = help: for further information visit https://rust-lang-nursery.github.io/rust-clippy/v0.0.212/index.html#cast_lossless

Now, the issue with this code isn’t immediately obvious. And that’s because this code is ”correct” in that, at present, it does what it should do.

The potential error, as clippy suggests, is what could happen if I later change this code. Suppose in the future, the design of the program changes and I actually want to find the max of two u32s. Look what would happen:

fn main() {
    let a: u32 = 0x00000002;
    let b: u32 = 0x10000001;
    assert_eq!(silly_max(a, b), b as u16);
}

fn silly_max(a: u32, b: u32) -> u16 {
    let x = a as u16;
    let y = b as u16;

    if x > y {
        x
    } else {
        y
    }
}

When we try to run this code, we get:

Running `target/debug/silly_max`
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `2`,
 right: `1`', src/main.rs:4:5

Clearly something went wrong. This is a classic truncation issue. When we cast u32->u16, we lose the top 16 most significant bits of our integers. 0x10000001 gets truncated to 0x0001 and 0x00000002 becomes 0x0002. Under truncation, our comparison operation doesn’t always work how we’d expect.

Luckily in this case, the truncation was rather clear. However, our code still compiled. We only noticed at runtime that we’d silently introduced a bug. (Just as clippy warned)

Let’s go back to the first version and implement the fix clippy suggested, using u16::from instead of casting via as.

fn silly_max(a: u8, b: u8) -> u16 {
    let x = u16::from(a);
    let y = u16::from(b);

    if x > y {
        x
    } else {
        y
    }
}

The above code compiles and works as expected. Now, lets make the same code change (switching the types of a and b to u32).

fn silly_max(a: u32, b: u32) -> u16 {
    let x = u16::from(a);
    let y = u16::from(b);

    if x > y {
        x
    } else {
        y
    }
}

Now, we get a friendly compiler error:

 --> src/main.rs:8:13
  |
8 | let x = u16::from(a);
  | ^^^^^^^^^ the trait `std::convert::From<u32>` is not implemented for `u16`
  |
  = help: the following implementations were found:
            <u16 as std::convert::From<bool>>
            <u16 as std::convert::From<u8>>

This is what we want! Instead of silently introducing the possibility of truncation into our program, the compiler lets us know that we’re trying to do an unsafe translation.

But what if we want this possible truncation behavior? Well, you can still use as and you’ll get truncated casts. Interestingly, clippy doesn’t complain if you do u32 -> u16 via as — it assumes you know what you’re doing.

So, what’s the moral of the story?

Prefer using the std::convert::From trait for doing type conversions. The as keyword performs “safe” casts (casting won’t cause a panic), but you can still run into classes of errors that can be avoided by using designated converter methods.

Here are some resources if you’re interested in learning more:

Switching from Jekyll to Hugo

Benjamin Congdon — Wed, 06 Jun 2018 07:22:07 +0000

I recently went through the process of switching my blog’s static site generator from Jekyll to Hugo. Both are great tools, but as I’ve updated my website over time, I began to feel constrained by Jekyll — and wanted to switch to something more stable and robust.

Context

I really like Jekyll. It’s fast to setup, comes with an extremely expressive templating language (sometimes dangerously so), and has the full support of Github — all of which lowers the barrier to deploying and maintaining a personal site.

That being said, Jekyll’s generation times start to skyrocket when you develop a site with meaningful complexity. I frequently had site generation times of 7+ seconds, which made working on formatting fixes on my website a complete slog.

Additionally, I’m not the biggest fan of the Ruby ecosystem — mostly due to the trouble I seem to have when installing dependencies. I’ve learned to accept Python’s dependency management warts — they’re generally ameliorated by sticking with a single tool and using it as intended. Ruby… I’ve never gotten to behave well. Without fail, I run into some byzantine installation bug with Nokogiri or Bundler when setting up a completely clean environment. I inevitably fall down some rabbit-hole of desperate StackOverflow questions, costing me an afternoon of annoyance.

Jekyll’s Moat

Despite this, I stuck with Jekyll for several years — I had a large switching cost in moving away from it: it’s the SSG that I have the most experience with and have used with my site since I started college. However, I reached a tipping point.

After using Netlify for a bit, the “moat” that Github Pages’ blessed support afforded Jekyll fell away. It’s just as easy to deploy a Hugo-, Gatsby-, or Pelican-generated site on Netlify as it is a Jekyll-generated one (and with HTTPS, no less!).

In fact, the experience of deploying using Netlify is (in my opinion) superior to Github Pages because you can actually see deploy logs and build failure messages instead of getting a generic email from Github saying “Something went wrong”.

Why Hugo?

Just as much as I wanted to switch away from Jekyll, I was intrigued in switching to Hugo.

Hugo is Fast My biggest complaint with Jekyll was the page generation times. Hugo boasts that it can have sub-millisecond page generation times — I’ve found that to be a slightly over-optimistic. However, my site’s total generation time dropped from 7+ seconds to ~300ms, so I’d score that as a win.

Hugo is written in Go This is a pretty superficial reason to change SSGs, but… I like Go. It’s a verbose language, but that makes projects written in it easy to read. Its templating system is really good, and Hugo makes smart additions to get similar expressiveness to Liquid (Jekyll’s templating system) with notable performance improvements.

Hugo is a Single Binary The installation and dependency management process for maintaining a Hugo site is trivial compared to Jekyll — you just have to download the single hugo binary. That’s it. No messing with package installers or system libraries (cough Nokogiri’s love-hate relationship with lxml cough).

Hiccups During Conversion

To get a grasp on Hugo’s conventions, I followed Sara Soueidan’s excellent “Migrating from Jekyll+Github Pages to Hugo+Netlify” guide. There are many Hugo introductions out there, but I found this one to be comprehensive.

Essentially the process for switching from Hugo to Jekyll was this:

Rewrite my Jekyll page templates in Go template language.
Restructure my site’s templates and content to fit Hugo’s directory structure.
Restructure my config.yaml file to conform to Hugo’s expectations.
Add some asset pipeline magic with gulp to minify assets and compile SASS.
Fix any miscellaneous errors that I didn’t catch in steps 1-4.

Most of this was fairly self-explanatory — once you understand Hugo’s “philosophy.” However, I still encountered a couple hiccups that are worth mentioning:

Hugo is much more sensitive to directory structure than Jekyll. This was the single most annoying thing about Hugo. Jekyll will render just about anything. Except for a few special directories (i.e. _partials, _includes, etc.) Jekyll renders essentially every file in your site source to the build directory.

Hugo is selective. If a page doesn’t have a valid template or there’s something wrong with a template, the page simply doesn’t get built — often quietly, without error.

You really do have to follow the template lookup orders that Hugo lists in the documentation, else you risk severe frustration. I suggest making default layouts (i.e. in the _default folder) for every page type so you don’t run into the issue of “quiet non-rendering”.

Hugo is much less flexible in templating than Jekyll. In Jekyll, anything goes. Want to put some Liquid interpolation in your front-matter? Sure why not. Pull in all your SCSS and run a SCSS compiler in a template? Yes, you can.

Hugo is much more conservative. It draws a much clearer distinction between what is a template and what is “content”. This was a bit of an annoyance for some custom pages I have (i.e. my projects page) that are highly custom — it doesn’t “feel” right to have these pages be templated, but Hugo requires that they be templates anyway.

However, I find that in the long run, I spend much less time writing and editing templates than adding Markdown-formatted content. As long as I could get a comfortable set of Hugo templates that span the distinction between content and template, Hugo is really not any more restrictive.

Hugo is more of a CMS than Jekyll is. Many times in the conversion process, I felt like Hugo was over-engineered for my use case. I don’t have a site with multiple authors/feeds or produce enough content that first-class support for categories and tags is crucial.

Yet Hugo includes all of this and more. It’s a bit overwhelming. Hugo has the marks of a top-down engineered product — “prescriptive to what it thinks users need” — in comparison to Jekyll’s bottom-up approach — “you build only the features you actually use”.

However, by having someone else make all these project layout decisions for you, the result is a fairly clean and consistent project directory.

I still found myself having to turn off many of Hugo’s more advanced features to avoid needless page generations (such as a separate page for each blog post tag).

Hugo’s Plugin Ecosystem is Less Rich Want a SASS compiler or HTML minifier plugin for your SSG? Those exist for Jekyll, but not for Hugo. Fortunately, many have encountered this problem and so the support for Hugo in the gulp ecosystem is really good. Gulp has top-notch asset pipeline management plugins, so this becomes a non-issue if you’re willing to sprinkle a little Node.js in with your SSG.

My Final Setup

In the end, my asset pipeline looks a lot like Netlify’s victor-hugo template. I use gulp to compile my SASS and minify my assets (HTML, images, etc). The “victor-hugo” template also brings in BrowserSync, which adds automatic refresh on rebuilds, among other niceties.

You may be saying that I’ve effectively traded the Ruby ecosystem for the Golang and Node.js ecosystems — and you’d be right. However, I’m significantly more confident in my ability to fix issues in this environment than I am in Ruby’s ecosystem, so I count that as a win.

I’m hoping that I won’t have to mess with the setup too much. Stability is valuable for a project like this where what I actually care about is writing, and all the SSG scaffolding is a distant second priority.

Conclusion

Ultimately, this can be read as just another “Why I moved from $X to $Y” post. I’ve given Hugo a spin and I like it so far. It’s fast, expressive where it should be, and conservative where it needs to be.

I try, as a rule, to not switch tech stacks on a whim. I’ve convinced myself that there are significant benefits in changing to Hugo, and so I’m hopeful that I’ll stick with this new setup for the next several years.

Corral: A Serverless MapReduce Framework

Benjamin Congdon — Wed, 02 May 2018 09:09:54 +0000

I’ve been working recently with Hadoop and Spark for a class I help teach. PySpark is fantastic, but Hadoop MapReduce never really “clicked” for me until I found mrjob. The concept of MapReduce is incredibly powerful, but the amount of boilerplate needed to write even a simple Hadoop job in Java is, in my opinion, rather off-putting.

Hadoop and Spark also require at least some infrastructure knowledge. Services like EMR and Dataproc make this easier, but at a hefty cost.

There have been past rumblings about using Lambda as a MapReduce platform. AWS published a (limited) reference architecture and there are some enterprise solutions that seem to be taking this approach as well. However, I couldn’t find a fully developed open source project that took this approach.

Around the same time, AWS announced native Go support for Lambda. Go’s short startup time, ease of deployment (i.e. single-binary packages), and general speed made it a great candidate for this project.

My idea was this: use Lambda as an execution environment, much like Hadoop MapReduce uses YARN. A local driver coordinates function invocation, and S3 is used for data storage.

The result is corral, a framework for writing arbitrary MapReduce applications that can be executed in AWS Lambda.

A Golang Interface for MapReduce

As Go famously doesn’t have generics, I had to scratch my head a bit to think of a compelling interface for mappers and reducers. Hadoop MapReduce has a large degree of flexibility in specifying input/output formats, the way records are split, and so on.

I considered using interface{} values for keys and values, but in the words of Rob Pike, “interface{} says nothing”. I decided on a minimalist interface: keys and values are strings. Input files are split by newlines. These simplifying assumptions made the implementation of the whole system simpler and cleaner. Hadoop MapReduce wins as far as customizability, so I decided to go for ease-of-use.

I’m happy with the final interface for Map and Reduce (some of which was inspired by Damian Gryski’s dmrgo):

type Mapper interface {
    Map(key, value string, emitter Emitter)
}

type Reducer interface {
    Reduce(key string, values ValueIterator, emitter Emitter)
}

type Emitter interface {
    Emit(key, value string) error
}

ValueIterator has just one method: Iter(), which iterates over a range of strings.

Emitter and ValueIterator hide a lot of internal framework implementation (shuffling, partitioning, filesystem interactions, etc). I’m also pleased that I decided on using an iterator for values instead of a normal slice (which would perhaps be more idiomatic), as an iterator allows for more flexibility on the framework’s side (i.e. to lazily stream values instead of having them all in memory).

Serverless MapReduce

From the framework side of things, it took me a while to determine a performant way to implement MapReduce as a completely stateless system.

Hadoop MapReduce’s architecture gives it the benefits of…

Persistent, long-running worker nodes
Data locality at worker nodes
Abstracted, fault-tolerant master/worker containers via YARN/Mesos/etc.

The last 2 aspects of this are fairly easy to replicate with the AWS stack. Bandwidth between S3 and Lambda tends to be pretty good (at least for my purposes), and Lambda is built so that developers “don’t have to think about servers”.

The trickiest thing to replicate on Lambda is persistent worker nodes. Lambda has a maximum timeout of 5 minutes. So, a lot of the way that Hadoop approaches MapReduce no longer work.

For example, directly transferring data between a mapper worker and a reducer worker is infeasible because the mapper needs to be done “as quickly as possible”. Otherwise, you risk the mapper timing-out while the reducer is still working.

This limitation manifests itself most during the shuffle/partition phase. Ideally, mappers would “live” long enough to transfer data to reducers on-demand (even during the map phase), and reducers would “live” long enough to do a full secondary sort using their disk as spill-over for a large merge sort. The 5-minute cap makes these approaches tricky to infeasible.

In the end, I decided to use S3 as the backend for a stateless partition/shuffle.

Semantic Intermediate Filenames Used for Partition/Shuffle

By using “prefix-friendly” names for mapper output, reducers can easily select which files they need to read.

Dealing with input data was significantly more straightforward. Like Hadoop MapReduce, input files are split into chunks. Corral groups these file chunks into “input bins”, and each mapper reads/processes an input bin. The size of input splits and bins is configurable as needed.

Self-deploying Applications

One of the bits of corral that I was most excited about was that it’s able to self-deploy to AWS Lambda. I wanted it to be quick to deploy corral jobs to Lambda — having to manually re-upload deployment packages to Lambda via the web interface is a drag, and frameworks like Serverless rely on non-Go tools, which would be cumbersome to include.

My initial thought was that the built corral binary would upload itself to Lambda as a deployment package. This idea actually works… until you deal with cross-platform build targets. Lambda expects a binary compiled with GOOS=linux , so any binary compiled on macOS or Windows wouldn’t work.

I almost abandoned the idea at this point, but then I stumbled upon Kelsey Hightower’s Self Deploying Kubernetes Applications from GopherCon 2017. Kelsey described a similar approach, though his code ran on Kubernetes instead of Lambda. However, he described the “missing link” that I needed: Have the platform-specific binary recompile itself to target GOOS=linux.

So, in sum, the process that corral uses to deploy to Lambda is as follows:

The user compiles their corral application targeting their platform of choice.
Upon execution, the corral app recompiles itself for GOOS=linux, and compresses that generated binary into a zip file.
Corral then uploads that zip file to Lambda, creating a Lambda function.
Corral invokes this Lambda function as an executor for map/reduce tasks.

Corral is able to use the exact same source code as both a driver and remote executor by doing some clever inspection of the environment at runtime. If the binary detects that it’s in a Lambda environment it listens for an invocation request; otherwise, it behaves normally.

Liquid error: internal

As an aside, the idea of self-uploading or self-recompiling applications is rather thrilling to me. I remember when I was taking a theory of computation class the concept of “self-embedding” programs (often referenced in the context of Undecidability proofs) were interesting, but I couldn’t think of a case in which you’d actually want a program with that level of internal reflection.

Self-deploying apps are, in a way, a pragmatic example of this idea. It’s a program that recompiles itself, uploads itself to the cloud, and invokes itself remotely (albeit through distinct code paths). Neat stuff!

Once deployed, the binary that corral uploads to Lambda conditionally behaves as a Mapper or Reducer depending on the input its invoked with. The binary that you execute locally keeps running and invokes Lambda functions during the Map/Reduce phases.

Corral Job Timeline

Each component in the system is running the same source, but there are many parallel copies running in Lambda (as coordinated by the driver). This results in the parallelism that makes MapReduce fast.

Treating S3 Like a File System

Corral, like mrjob, tries to be agnostic to the filesystem it runs on. This allows it to transparently switch between local and Lambda execution (and allows room for extension, such as if GCP begins to support Go in cloud functions).

However, S3 isn’t really a filesystem; it’s an object store. Using S3 like a filesystem requires a bit of cleverness. For example, when reading input splits corral needs to seek to a certain portion of a file and begin reading. By default, a GET request to S3 returns the entire object. This isn’t great when your objects are potentially tens of gigabytes.

Fortunately, you can set a Range header in S3 GET requests to receive chunks of objects. Corral’s S3FileSystem leverages this, downloading chunks of an object at a time as necessary.

Writing to S3 also requires a bit of thought. For small uploads, a standard “PUT Object” request is fine. For larger uploads, multipart uploads become more appealing. Multipart uploads allow for the functional equivalent of writing to a local file; you “stream” data to a file, instead of retaining it in memory to write all at once.

To my surprise, there wasn’t a great S3 client that provided io.Reader and io.Writer interfaces. s3gof3r is the closest thing I could find; it’s pretty good, but (in my experience) leaks so much memory that I couldn’t use it in the memory-limited Lambda environment.

Memory Management in Lambda

While AWS Lambda has been gaining steam for the past few years, it feels like the tooling for profiling Lambda functions is lacking. Lambda does, by default, log to Cloudwatch. If your function panics, a stack trace is logged. As such, “crashing” errors are relatively straightforward to debug.

However, if your function runs out of memory or time, all you’ll see is something like this:

REPORT RequestId: 16e55aa5-4a87-11e8-9c63-3f70efb9da7e Duration: 1059.94 ms Billed Duration: 1100 ms Memory Size: 1500 MB Max Memory Used: 1500 MB

Locally, tools like pprof are fantastic for getting a sense of where memory leaks come from. In Lambda, you have no such luck.

In an early version of corral, I spent hours tracking down a memory leak that ultimately was caused by s3gof3r. Since Lambda containers are reused, even small memory leaks will result in eventual failure. In other words, memory usage persists across invocations — a leaky abstraction (no pun intended).

It’d be great to see better profiling tools for AWS Lambda, especially since Golang is a notably easy runtime to profile.

When Lambda Becomes Expensive

Ostensibly, the goal of corral is to provide a cheap, fast alternative to Hadoop MapReduce. AWS Lambda is cheap, so this should be a slam dunk, right?

Yes and no. The free tier of Lambda gives you 400,000 GB-seconds per month. This sounds like a lot but gets used up quickly by long-running applications.

Lambda Pricing Scheme as of 4/29/2018

Ultimately, corral can still end up being very inexpensive. However, you’ll want to tune your application to use as little memory as possible. Set the maximum memory in corral as low as possible to reduce cost.

Time is an as-you-use-it resource in AWS Lambda — you’re charged for as many milliseconds as you use. Memory is billed by use-it-or-lose-it. If you set the max memory to 3GB but only use 500MB, you’re still paying for the full 3GB.

Performance

While not a primary design consideration, corral’s performance is fairly respectable. Much of this is due to the nearly infinite parallelism that Lambda offers. I used the Amplab “Big Data Benchmark” to get a sense of corral’s performance. This benchmark tests basic filters, aggregations, and joins.

As I anticipated, corral does quite well on filtering and aggregation. However, it falls flat on joins. Without a secondary sort, joins become expensive.

The Amplab benchmark tests up to ~125GB of input data. I’d be curious to do more benchmarking with around ~1TB of data to see if performance continues to scale more-or-less linearly.

More information and benchmark statistics can be found in the corral examples folder.

Conclusion

So that’s it: corral let’s you write a simple MR job, frictionlessly deploy it to Lambda, and run jobs on datasets in S3.

Running a word count job in Lambda

It’s also worth noting that I’m not wedded to the AWS ecosystem. Corral is mostly agnostic to Lambda and S3 so that, in the future, connectors to GCP’s Cloud Functions and Datastore could be added (if/when GCP adds Go support to CF).

On a personal note, I found this project to be quite rewarding to work on. There were many fewer resources online for building a MapReduce system than for using one, so there were some implementation details that I got to work out.

Feel free to make issues in the corral repository. I’m curious to see if there’s a big enough niche that this project fills to justify continued development. 😁

Counting Scanned Bytes in Go

Benjamin Congdon — Tue, 10 Apr 2018 11:30:59 +0000

For a recent project, I needed to read data from a specific chunk of a file. The data was a sequence of serialized records, so I used a bufio Scanner for splitting. Scanners are great, but they obscure the exact number of bytes read. In working through the problem, I found a solution that worked quite nicely.

Essentially, the problem boiled down to this: I wanted to use bufio’s Scanner to cleanly split data records, but I also needed fine-grain control over exactly how many bytes were being read. Simply counting the bytes in the input record is insufficient, because data records may be split using variable-width delimiters¹. (i.e. \n vs \r\n).

Problem: Read & Split Records until Chunk Boundary Is Reached

My initial thought was to wrap the io.Reader that I was reading records from. The result looked something like this:

type CountingReader struct {
    reader io.Reader
    BytesRead int
}

func (r *CountingReader) Read(p []byte) (n int, err error) {
    n, err = r.reader.Read(p)
    r.BytesRead += n
    return n, err
}

This method does work as advertised, in that it counts the number of bytes read from the reader. However, it doesn’t actually solve my problem – counting the number of bytes read as records. For example:

func main() {
    buf := bytes.NewBuffer([]byte("foo bar baz"))
    reader := &CountingReader{reader: buf}

    scanner := bufio.NewScanner(reader)
    scanner.Split(bufio.ScanWords)
    scanner.Scan()

    fmt.Println(scanner.Text()) // "foo"
    fmt.Println(reader.BytesRead) // 11 :(
}

(You can try this on the Go Playground)

The reason for this behavior, as I found out, is that bufio.Scanner maintains an internal buffer which it reads to in large chunks. My CountingReader could (correctly) say that 11 bytes had been read from it, even though only, say, 4 bytes of data ("foo ") had been yielded from the Scanner.

My next thought was to implement a CountingScanner by wrapping or reimplementing buffio.Scanner to count processed bytes. Neither of these options was appealing. The io module also has LimitReader which enforces a maximum number of bytes read. For my problem, this wouldn’t work; my reader needs to completely read the last record in the chunk of the file, even it if spans the cutoff point.

io.LimitReader would not allow Logical Splitting

In the end, I tried wrapping the scanner’s SplitFunc. For reference, SplitFunc has the following signature:

type SplitFunc func(data []byte, atEOF bool) (advance int, token []byte,
    err error)

Essentially, SplitFunc takes a slice of bytes and returns how many bytes forward the Scanner should advance along with the “token” extracted from the byte slice.

This advance return value is exactly what I want: how many bytes were truly processed by the Scanner.

From here, I put together a simple struct that wraps a SplitFunc, and increments an internal counter each time a split is performed:

type ScanByteCounter struct {
    BytesRead int
}

func (s *ScanByteCounter) Wrap(split bufio.SplitFunc) bufio.SplitFunc {
    return func(data []byte, atEOF bool) (int, []byte, error) {
        adv, tok, err := split(data, atEOF)
        s.BytesRead += adv
        return adv, tok, err
    }
}

func main() {
    data := []byte("foo bar baz")
    counter := ScanByteCounter{}

    scanner := bufio.NewScanner(bytes.NewBuffer(data))
    splitFunc := counter.Wrap(bufio.ScanWords)
    scanner.Split(splitFunc)

    for scanner.Scan() {
        fmt.Printf("Split Text: %s\n", scanner.Text())
        fmt.Printf("Bytes Read: %d\n\n", counter.BytesRead)
    }
    // Split Text: foo
    // Bytes Read: 4

    // Split Text: bar
    // Bytes Read: 8

    // Split Text: baz
    // Bytes Read: 11
}

And the results were successful! Try this on the Go Playground

Here are some lessons learned:

Function wrapping is a good way to extend the usefulness of standard library features
Beware of making assumptions about how much data is being read from/to buffers (especially when Readers/Writers are consumed by other interfaces)

I’m also curious if there’s an official way for solving this problem. If there’s a cleaner solution, let me know! 😄

Cover: Pixabay

Footnotes

1. I also wanted to support the general case of an arbitrary SplitFunc, so anything based on delimiter size was out.

Python Idioms in Rust

Benjamin Congdon — Fri, 23 Mar 2018 12:23:22 +0000

I’ve been going through a period of programming language wanderlust over the past couple months. Recently, I’ve been quite interested in Rust. Coming from Python, I’ve found a lot of Rust’s language features to be quite powerful.

Earlier this year, I dug deep into Go. For the most part, I enjoy Go: It’s a simple language that can be learned in a few hours (literally). Its structure promotes good programming practices and it has a wonderful ecosystem.

However, in Go, I felt limited by the expressiveness of the language. Its much-bemoaned lack of generics is less disruptive than one would expect but makes clever functional-programming tricks essentially impossible. That’s fine — it’s not what Go was designed for — but as a result, I found Go a bit less “fun” to work in.

While Rust does have additional complications coming from it’s infamous borrow checker and the complexity of generics, I’ve found that what you gain in expressiveness is worth the tradeoff.

In fact, I found Rust to be rather Pythonic¹. ✨ Sure, you have to be careful about types and ownership, but there is a strong case to be made that Rust shares many of the strengths of Python’s syntax.

After doing a bit of work in Rust, I identified some idioms² common to Rust and Python. These common features are all things that I enjoy using in Python, and I’ve found to be useful while coming up-to-speed on Rust.

Enumerate

A common pattern in when iterating over lists in Python is to use enumerate to have get both the index and value of list elements:

arr = ['a', 'b', 'c']
for idx, val in enumerate(arr):
    print("{}, {}".format(idx, val))
# 0, a
# 1, b
# 2, c

Rust iterators also support enumerate! (And you don’t even have to learn a new name for the same functionality)

let arr = vec!['a', 'b', 'c'];
for (idx, val) in arr.iter().enumerate() {
    println!("{}, {}", idx, val);
}

Zip

Another function on iterators that you see pretty frequently in Python is zip, which combines 2 iterators element-wise, creating an iterator of tuples with an element of each list.

letters = ['a', 'b', 'c']
numbers = [1, 2, 3]
for l, n in zip(letters, numbers):
    print("{}, {}".format(l, n))
# a, 1
# b, 2
# c, 3

Again, Rust iterators also support zip:

let letters = vec!['a', 'b', 'c'];
let numbers = vec![1, 2, 3];
for zipped in numbers.iter().zip(letters) {
    println!("{:?}", zipped);
}
// (1, 'a')
// (2, 'b')
// (3, 'c')

Like Python’s zip, Rust’s zip halts when either of the iterators is exhausted. This allows you to do cool tricks with infinite iterators:

let start_at_10 = 10..;
let letters = vec!['a', 'b', 'c'];
for zipped in start_at_10.zip(letters) {
    println!("{:?}", zipped);
}
// (10, 'a')
// (11, 'b')
// (12, 'c')

Tuples

Rust has tuples! This is a feature that I sorely missed in Go. Yes, structs work just as well, but for “scripting” or prototype work, tuples are a great language feature.

Python’s tuple system is pretty flexible. Functionally, Python tuples act like immutable lists. Tuples can be aliased into separate variables via unpacking, and are indexed-into like lists.

foo = ('foo', 1, None)
a, b, c = foo
assert a == 'foo'
assert foo[0] == 'foo'

# Tuples are immutable in Python
tup = (1, 2, 3)
tup[0] = 4
# TypeError: 'tuple' object does not support item assignment

Rust’s tuple feature is also quite powerful. Similar to Python, you can alias tuple values into separate variables using syntactic sugar, although in Rust this is called “destructuring” instead of “unpacking”.

let foo = ("foo", 1, None::<i32>);
let (a, b, c) = foo;
println!("{}", a); // "foo"
println!("{}", foo.0); // "foo"

Also, Rust tuples aren’t indexed using square brackets (foo[0]), but rather by the dot operator (foo.0). This is a byproduct of the fact that Rust tuples are sized at compile-time, unlike Python tuples which are dynamic.

# Dynamically sized tuples in Python
tup = tuple(['foo' for _ in range(randint(1, 10))])

Unlike in Python, Rust tuples can be made mutable:

let mut tup = (1, 2, 3);
tup.0 = 4;

List Comprehensions (kind of)

List comprehensions are the bread-and-butter of Python syntactic sugar. It’s a super expressive way of generating/filtering lists.

evens_squared = [x**2 for x in range(10) if x % 2 == 0]
# [0, 4, 16, 36, 64]

Rust doesn’t have list comprehensions per se, but Rust iterators do have map and filter, which allow you to perform similarly expressive “list” generation:

let evens_squared = (0..10)
    .filter(|x| x % 2 == 0)
    .map(|x| x * x)
    .collect();

println!("{:?}", evens_squared);
// [0, 4, 16, 36, 64]

Personally, I like Rust’s extensive iterator functions even better than Python’s. I found functions like flat_map and scan (among others) to be quite useful. These iterator functions bring aspects of functional programming to Rust that I really enjoy.

Single-Line If

Neither Python nor Rust has ternary operators (which may be for the best). However, they both support single-line conditional expressions. This can be useful in certain conditions, but it’s really just syntactic sugar.

result = expensive_function() if some_condition else None

let result = if some_condition { expensive_function() } else { 0 }

Lambda Functions

An oft complained-about language feature of Python is its lambda functions. They’re useful, but the syntax isn’t great. Other languages like Ruby and Javascript have more terse syntax for specifying closures or inline functions.

Python lambdas take a set of arguments and return the evaluation of the expression that follows the :

arr = [3, 10, 15]
sorted(arr, key=lambda x: abs(10-x))
# [10, 15, 3]

Rust closures are similarly expressive. They use pipes (|) to hold arguments and the expression that follows is used as the return value.

let mut arr: Vec<i32> = vec![10, 15, 3];
arr.sort_by_key(|a| (10 - a).abs());

Rust closures can also be multiline functions using brackets, which is quite handy:

let mut arr = vec![10, 15, 3];
arr.sort_by_key(|a| {
    let intermediate_value = some_function(a);
    intermediate_value + 10
});

Both Python lambdas and Rust closures perform variable capture, so you can have access to variables that are in-scope where the lambda/closure is created:

a = 10
plus_a = lambda x: x + a
plus_a(5)
# 15

let a = 10;
let plus_a = |x| x + a;
plus_a(5); // 15

First-Class Functions

In Python, functions are essentially objects that are invoked with the __call__ magic function. They can be passed around like any other object and can even have attributes assigned to them. This can be useful for passing callback functions. While idiomatic Python doesn’t make extensive use of functions-as-variables, this feature comes in handy pretty often.

def plus_one(x):
    return x + 1

def map_on_list(arr, func):
    return [func(x) for x in arr]

arr = [1, 2, 3]
map_on_list(arr, plus_one)
# [2, 3, 4]

Rust also has first-class functions. Functions and closures can be used as argument to other functions:

fn plus_one(x: i32) -> i32 {
    x + 1
}

fn map_on_vec(vec: &Vec<i32>, func: fn(i32) -> i32) -> Vec<i32>
{
    let mut new_vec = Vec::new();
    for i in 0..vec.len() {
        new_vec.push(func(vec[i]));
    }
    new_vec
}

let vec = vec![1, 2, 3];
println!("{:?}", map_on_vec(&vec, plus_one));
// [2, 3, 4]

// Equivalently:
println!("{:?}", map_on_vec(&vec, |x| x + 1));

Interestingly, there are some subtle differences between Rust functions and closures. ³ In general, they can be treated as the same thing. Of course, functions are not objects in Rust, but a Rust struct can implement the Fn, FnMut, or FnOnce traits to similar effect of Python’s __call__ (though this is not recommended).

Conclusion

Hopefully, I’ve given a bit of a hint as to why a Python dev might be interested in picking up Rust (for aesthetic reasons, if nothing else!). There’s a definite learning curve to Rust, but it’s a speedy language that still remains quite expressive.

From a syntactic sense, I found Rust to be more similar to Python than Python is to Go. Using generics and type inference, you can end up writing Rust code that has only minimal “type boilerplate” and feels pretty similar to Python.

If you’re interested in looking into Rust, I’d highly recommend the following resources:

The Rust Programming Language - Official (free) book that walks you through the Rust language
Rust Language Documentation - The API docs of the Rust language — it’s search functionality is pretty great
/r/rust - The Rust Subreddit

Cover: Pixabay

Footnotes

1. Armin Ronacher did a great talk about pairing Rust and Python that expands upon this notion of Rust being “Pythonic”

2. Some of these are probably better defined as syntactic sugar or language features. 😛

3. Ricardo Martins wrote a blog post about this distinction between functions and closures

DEV Community: Benjamin Congdon

A Gentle Nudge

Finding Flow in Haskell

The Entry Point: Higher-Order Functions

Haskell is not Array.map, but “more”

Pattern Matching

Immutability

Algebraic Data Types

<*>, <$>, >>=, Oh My!

Aside: Category/Type Theory

Conclusions

Further Reading

Live Refreshing Cargo Docs

Step 1: Install cargo-watch and https

Step 2: Run Watch Command

Bonus: Auto-refresh

More Information

Lossy Numeric Casting in Rust

Switching from Jekyll to Hugo

Context

Jekyll’s Moat

Why Hugo?

Hiccups During Conversion

My Final Setup

Conclusion

Corral: A Serverless MapReduce Framework

A Golang Interface for MapReduce

Serverless MapReduce

Self-deploying Applications

Treating S3 Like a File System

Memory Management in Lambda

When Lambda Becomes Expensive

Performance

Conclusion

Counting Scanned Bytes in Go

Footnotes

Python Idioms in Rust

Enumerate

Zip

Tuples

List Comprehensions (kind of)

Single-Line If

Lambda Functions

First-Class Functions

Conclusion

Footnotes

Haskell is not `Array.map`, but “more”

`<*>`, `<$>`, `>>=`, Oh My!

Step 1: Install `cargo-watch` and `https`