DEV Community: Alex Kotliarskyi

Moving my serverless project to Ruby on Rails

Alex Kotliarskyi — Sat, 14 Nov 2020 00:00:00 +0000

I have a small side project: digital gift cards for hackers. It uses Shopify for all the store-related stuff: frontend, payments, refunds, reports, etc.

But unlike regular digital products (ebooks, videos) I wanted each card that the user purchases from the store to be unique. So I made a script that generate personalized images and ran it manually for every order.

The next logical step was automating this process. I started with serverless AWS Lambda. At the time it was a hot new tech and I wanted to learn more. It seemed very fitting for my use-case: single-responsibility functions that can run at any time and don’t require server maintenance.

It was super easy to get started. I built a JavaScript function and deployed it to AWS Lambda, added Shopify web hook and it all worked!

Early benefits of serverless (for hobby projects):

Easy to get started
Don’t have to configure or maintain servers
Free for small loads

In reality, writing the simple Lambda functions turned out to be only 10% of the work.

The time passed and my backend started getting more complex: I needed to store some state for each puzzle, send confirmation emails, show order details page. What started as a simple function, grew into a bunch of serverless functions, SNS topics, S3 buckets, DynamoDB tables. All bound together with plenty of YAML glue, schema-less JSON objects passed around and random hardcoded configs in AWS console.

I think it’s just a typical software development life-cycle: things grow organically, become a mess, and require some refactoring. Make it run first (discover market fit), then make it right (refactor to integrate the new discoveries).

But this time it was different. I couldn’t refactor things as easily as I used to in the traditional monolithic apps. Here’s why:

When the building blocks are too simple, the complexity moves into the interaction between the blocks.

And the interactions between the serverless blocks happen outside my application. A lambda publishes a message to SNS, another one picks it up and writes something to DynamoDB, the third one takes that new record and sends an email…

I could test every single block in that flow, but I didn’t have the confidence in the overall process. What if publishing fails, how would I know that? How would system recover? Can I rollback and try again? Where do the logs go?

Another swarm of problems was hiding in my configuration: bad Route 53 record, typos in SNS topics, wrong S3 bucket region. Tracing errors was a challenge, there’s no single log output I can look into.

With serverless, I was no longer dealing with my project's domain, I was dealing with the distributed system's domain.

At this point I felt fooled.

I came for the easy way to deploy code and not think about servers, but in the end had to design my system around the platform’s limitations.

Drawbacks of serverless (for hobby projects):

Hard to follow information flow
Impossible to replicate production environment
Slow iteration speed
Lack of end-to-end testing
Immature documentation (dominated by often outdated Medium posts)
No conventions (have to make hundreds of unessential decisions)

—

I was clearly not enjoying the serverless. So I decided to rewrite it. After all, it is a side project I’m doing for fun. The tech stack of choice — Ruby on Rails.

I haven’t used Rails since 2013, and for the last 8 years at Facebook I’ve been mostly doing JavaScript.

The experience of picking Rails back up was really nice but… uneventful. Nothing really changed that much. Few things got added, a few small things moved around.

Of course I did hit some magical Ruby issues. But unlike my typical experience with JavaScript, I was quickly able to find the solution.

Rails comes with so many things built-in and configured. Over the years without Rails I used to gluing random JavaScript libraries together to roll my own routing, file storage wrappers, email preview pipeline, managing secrets, test setup with fixtures, database migrations, logging, performance reporting, deployment scripts. With Rails I didn’t have to think about all these details and could simply focus on making product-visible changes.

It was as like driving a Tesla after years of making my own scrappy cars. Similar components, but all configured and aligned to work well together.

Benefits of Rails (for hobby projects):

Conventions
Tooling, libraries
Documentation
Monolith is easy to understand and test

Drawbacks of Rails (for hobby projects):

Feels heavyweight in the beginning
Hurts if your opinions differ from Rails conventions
Have to host on a server
Doesn’t sound cool in 2020 (anymore and maybe yet)

—

Serverless is like a black hole. It promised exciting adventures, but the gravity sucked me in and I spend most of my efforts dealing with its complexity, instead of focusing on my product.

Things that are not strings

Alex Kotliarskyi — Wed, 19 Aug 2020 00:00:00 +0000

As programmers, we have a collective delusion that anything that can be represented as a string, is a string. This thinking causes a whole bunch of problems.

Let’s take SQL for example. Every API in every programming language that I’ve seen considers SQL statement a string.

function execute(sql: string): Promise<Result>

The problem with this API is that not every string is a valid SQL (nor sometimes it is the SQL you actually want to run).

Here’s a classic example of the misuse:

const query = 'SELECT * FROM posts WHERE id = '
  + params.id;

In this example params.id can be anything, including invalid or malicious SQL.

The root problem here is not the lack of sanitization. The problem is that SQL is treated as a string.

Think about JSON for another example. You could certainly implement adding an item to a hash by doing this (I hope this code makes you cringe):

function addKeyValue(json, key, value) {
  return json.substr(0, json.length - 1)
    + ', "' + key + '": "' + value + '"}';
}

As with the SQL example, you could add escaping and sanitization, but it’s just hacks hiding the real problem:

A string can be a representation of a thing, but it’s not the thing itself.

And it’s not only about concatenating strings. Can you spot the problem with this function? (hover to see the answer)

function isSafeDomain(url: string): boolean {
  return url.includes('https://safe.com/');
}

Or in this one? (hover to see the answer)

function checkPassword(pass: string, hash: string): boolean {
  return bcrypt(pass) === hash;
}

Strings are lower level, and thus are much more flexible than they need to be to properly implement valid operations.

Incomplete list of things that are not strings:

SQL
HTML
JSON
URL
File path
Password

Things are… things

You can save yourself a lot of headaches if you stop treating everything that can be represented as a string, as a string.

Both OO and FP styles allow for abstracting away something as a type or a class. You can make a closed opaque structure for the thing and limit the ways it can be constructed.

For example, for SQL, you might want to make sure it’s only created from static string literals.

Of course, at some point, you will need to serialize the thing into a string to pass it further, and that fine, as long as you never carelessly take a string and unserialize it yourself.

Here’s a few libraries for inspiration of how to treat things as… things:

SQL: Slonik, Knex
HTML: React, Elm, rum
JSON: a dictionary in any programming language
URL: url - Rust
File paths: std::path::Path - Rust

How not to use Redux: SET actions

Alex Kotliarskyi — Sun, 15 Dec 2019 00:00:00 +0000

TL;DR: Instead of thinking about your actions as something that changes application state, consider them events instead.

In this article by Flux I mean one way data flow pattern implementations like Redux, useReducer or Elm Architecture

Every time I see an action with a name that starts with SET_*, I know there’s a problem.

Consider this example:

function Counter(props) {
  function onClick() {
    props.dispatch({
      type: "SET_VALUE",
      value: props.value + 1
    });
  }
  return (
    <div>
      <div>Value: {props.value}</div>
      <button onClick={onClick}>+</button>
    </div>
  );
}

function reducer(state, action) {
  if (action.type === "SET_VALUE") {
    return { value: action.value };
  }
}

Problem #1: Business logic inside the component

In this example the bussness logic is incrementing the value, i.e. value = value + 1. If we use SET_* action, the logic ends up inside the component. It’s not great because the React component should not care about these details, its main purpose is rendering and dispatching actions.

As the application and the team grows, it’s going to be very tempting to put more stuff into the component. E.g. if we wanted to have a max value for the counter, the most intuitive thing in this example would be to put it into the onClick handler.

Problem #2: Not leaveraging reducer

Reducer is a beautiful pattern that’s older than JavaScript itself. Given a state and an action reducer returns a new state. It’s easy to type check, easy to test, easy to reason about.

When we put the business logic outside of reducer, we miss out on all these advantages. The SET_* actions essentially turn the predictable store into a global variable.

Problem #3: Hard to debug and test

Imagine our counter component isn’t behaving right: after pressing the + button the value is not what we expect. Let’s check Redux devtools:

Old State: {value: 3}
Action: {type: "SET_VALUE", value: 42}
New State: {value: 42}

This is correct, but not helpful. How did we compute 42 as the new value to be set? 🤷‍♂️

Writing tests for SET_* actions is also not great. The tests end up very dumb and the aren’t really testing anything useful:

test("SET_VALUE action", () => {
  store.dispatch({ type: "SET_VALUE", value: 3 });
  expect(store.getState().value).toEqual(3);
});

To test the business logic that lives inside React component we’d have to spend a lot of effort setting up mock renderer, simulating click event, etc.

The Better Way

Let’s think about our actions as objects describing user’s intent. In the simple Counter component example from above, when the user clicks + the user wants the app to increment the counter.

function Counter(props) {
  function onClick() {
    props.dispatch({ type: "INCREMENT" });
  }
  return (
    <div>
      <div>Value: {props.value}</div>
      <button onClick={onClick}>+</button>
    </div>
  );
}

function reducer(state, action) {
  if (action.type === "INCREMENT") {
    return { value: state.value + 1 };
  }
}

Note that the React component in this case doesn’t care how this event is handled, it just tells the system about what happened.

The reducer tests in this case are more meaningful, capturing the essense of the business logic:

test("INCREMENT action", () => {
  const store = createStore({ value: 2 });
  store.dispatch({ type: "INCREMENT" });
  expect(store.getState().value).toEqual(3);
});

Actions are events

The secret here is in the mindset. Instead of thinking about your actions as something that changes application state, consider them events instead. These events could represent user intent (clicking a button, typing, etc.) or a notification from external system (timers, HTTP response, etc.)

Examples:

SET_LOGGED_IN → GOT_ACCESS_TOKEN
SET_TODOS_LIST → LOADED_TODOS_FROM_SERVER

P.S. The “no SET_* actions” rule can be generalized: actions should not be derrived from state. If you need some information from state to construct an action, it’s a sign that the action should be simpler and the computation you are trying to perform should probably live in the reducer.

Digital Typewriter

Alex Kotliarskyi — Sun, 09 Jun 2019 00:00:00 +0000

What if you could use your favorite keyboard away from your computer?

This is what it looks like for now. Check out GIF demo further down.

A long time ago I learned about an interesting “text editor”. Its main feature was, uhh, the lack of text editing. That’s right, you can’t change anything you’ve typed.

The idea behind it is very interesting. It’s about two modes of writing. One where you produce as much text as possible, without trying to organize or correct it in any way. The other one is where you edit and restructure the text until it’s consistent and easy to read.

Doing the first part on modern computers is a challenge for me. I get distracted very easily. I’m also tempted to start editing right away, often times ending up in a loop rewriting a sentence over and over again.

Old typewriters don’t have this problem: there’s no distractions and no way to edit the text.

Doesn't support twitter, doesn't have the backspace key.

Implementation

On my last birthday I got a tiny device, Onion Omega 2+. It’s like Rapsberry Pi, runs Linux, but built on different architecture and has its own set of accessories. It came with 21x7 chars OLED display extension.

Onion Omega

The idea was to connect my favorite keyboard (HHKBPro2 type-S) via USB to the Omega2. Anything I type there should immediately show up on the built-in OLED display and sync to “the cloud”.

I bet I can hack this together with 1 line of bash.

– me, two weeks ago

I thought it would be trivial, just read the keystrokes, pipe them to OLED and a text file, sync the text file over Dropbox.

However, it turned out to be much more complicated than that :)

First, I couldn’t find a trivial way to read the text off of the keyboard. I was looking for something like getc that can get data from a connected USB device, but no luck. Instead I found out that Linux has /dev/input/event* files where I could read raw events. That lead me to the ancient input.h, which I used to hack together a simple parser.

Next, printing this stuff to OLED also had gotchas. My initial version would just repaint the screen on every keystroke, which turned out to be very inefficient. Full display repaint using Omega’s OLED APIs takes as much as 400ms, totally unacceptable for a good experience.

So I changed the code to keep track of cursor position and advanced it as more characters were being printed on the screen.

This got more complicated once I added support for backspace. And then even more complicated with paging. When I got to implementing word wrapping, it was too much — there are so many different states to transition between!

Ideally, I don’t want to track the current screen position and code the drawing logic for each editing command. I want a model where I can define piece of state (the text) and a function that defines what this should look like on the screen, without worrying about the underlying OLED API performance.

Does it remind you of anything? :)

React. I love this model and I’m sure there are tons of applications for it outside the web development. Here’s what my code looks like now:

def render(text)
  # Return the text I want to be on the screen
end

next_screen = render(message)
reconcile(prev_screen, next_screen) do |row, col, text|
  oled_command "cursor", "#{row},#{col}", "write", text
end
prev_screen = next_screen

The latency between pressing a button and seeing it on the screen is amazing, I’ll look into measuring it.

Demo

Sync

Everything I type on this device is stored in text files. It’s not very easy to pull these files from the device, so I also added some code to upload each line to the server.

On the backend I used Firebase. I have a simple “cloud function” that takes text as input and stores it in Firestore. I also have a very simple frontend that renders the text from the Firestore.

The tricky part is that I can’t completely rely on available internet connection. I plan to take this device to coffee shops and places outside of WiFi reach. The script stores a queue of lines to upload in a file, so even after device restarts it should be able to resume uploads.

Ideally I’d love to use Dropbox, but unfortunately I couldn’t find linux MIPS build of their client.

Testing

In retrospective I should have started with proper Ruby tests, but yak shaving and curiosity brought me to cram. It’s a simple CLI tool that takes *.t files as inputs. These files contain shell commands and the expected output.

The development experience of cram is pretty good. You don’t need to set the “expected output” beforehand. You can just run the tool and it will show the difference between expected and actual output, with an option to update the *.t file.

Very similar to Jest snapshot tests.

Deployment

Omega’s WiFi has been terrible in my experience, lots of random disconnects. I couldn’t rely on SSH because of that and had to use serial port with screen. This made the deployment process more complicated, there’s no scp for terminal connection.

Here’s a clever trick I learned:

screen -S $SESSION -X stuff 'cat > /root/onityper.rb <<\EOF'$'\015'
screen -S $SESSION -X readreg p "$ROOT/onityper.rb"
screen -S $SESSION -X paste p
screen -S $SESSION -X stuff 'EOF'$'\015'

I can run screen in one tab and while the connection is active I can send commands to it. stuff sends keystrokes and readreg/paste can simulate typing large chunks of text.

To run the script on every boot, I used the official instructions which recommend putting stuff into /etc/rc.local.

ruby /root/onityper.rb >> /tmp/onityper.log 2>&1 &

I don’t like this at all. There must be a proper way of defining a service that can run in background, with policies about logs and log rotation, when to restart it, etc. Looks like OpenWRT has procd but at that point I was too lazy to look into.

Code

Warning: it’s a bag of hacks. See it on GitHub.

I’m very happy with this setup so far. I bought the cheapest powerbank I could find on Amazon, so the device is pretty much independent from my laptop. 70% of this post was created on the Onityper from a coffee shop.

Now I want to use a 3D printer to build a case for it.

How to convince your boss to use React Native

Alex Kotliarskyi — Sat, 09 Mar 2019 00:00:00 +0000

Likely you landed on this article because you are excited about React Native. Awesome! If you have a team of web engineers and a brand new app, it’s easy to make a case for starting with React Native.

However, most companies already have apps in the stores. They used native stack to build their apps: Xcode or Android Studio. React Native can add value for these companies too, but making a hybrid app comes with some challenges.

During the last 4 years I helped integrate React Native into Facebook and Oculus apps. Here are a few lessons I learned.

Start with Empathy

Wait, start with what?

Understand the people who work on your project—that’s the first step to successful integration.

Your boss (project manager, director, CTO, etc.) is responsible for the app hitting the market with little risk, short timeline and maximum impact. Companies already operate in very competitive environment. To them any new technology is a risk.

Your colleagues have a different perspective as well. They have experience building apps with the “native” technology stack. They know how to format strings, build UIs, access network, write and debug code in the IDE. Switching to a completely different ecosystem is very uncomfortable. There’s natural resistance to getting out of the comfort zone.

Does React Native have layout inflaters? How do I change the text of this button?

Engineers might have existing opinions about React, React Native and JavaScript ecosystem. Or they’ve seen a React Native app somewhere and found it of bad quality. There are many common myths, e.g. “JS is slow”, or “frameworks come and go too fast”.

These myths are not specific to JavaScript. As humans we can’t experience every single tech stack and have a well-formed opinion. We have to rely on things we’ve heard from others.

This might not sound very encouraging, but be willing to accept a defeat. There are so many scenarios where React Native can be very beneficial. But there are also valid situations where it doesn’t make sense at a particular time. And it’s okay. Maybe in a few months you can revisit this decision. The last thing you want is to polarize your team and ruin constructive work relationships.

To fight existing misconceptions, prove them wrong on a real feature inside your app (without making other people feel stupid).

Find value

This guide assumes you are integrating React Native into an existing application and already have a big chunk of functionality written using platform-dependent code. Rewriting it all is a huge risk with very little benefit to your users and business. That’s why it will be extremely hard to convince your team and people responsible for the project to do so.

But you don’t have to rewrite your application! React Native can be adopted incrementally.

Find an area in your application that will immediately get benefits from being implemented in React Native. For example, it could be a screen that is useful but but doesn’t get much development attention or a surface that’s implemented as a WebView. For example, Facebook started with Pokes, Instagram rebuilt “liked photos”, etc.

It has to be with a scope that you can build yourself within few days. It will be a great campground to test different aspects of integration: build system, session sharing, logging, crash reporting, etc.

Bonus points for making the new feature run on both iOS and Android. Suddenly, something that used to take a lot of engineering time and coordination, was created by one person in a short time frame.

Show, don’t tell.

Be a champion

You can’t just say “Let’s use React Native” and hide in the bushes.

TC39, a committee that evolves JavaScript the language, has the following process: Changes to the language are called “proposals”. A member that leads a proposal is called a “champion”. Their mission is to create spec drafts, work with the community and push the spec forward.

Become the React Native champion at your company. Take the responsibility to see this idea through.

The first step to becoming a champion is to get more familiar with React Native. Create a few prototypes (by rebuilding some parts of your app), learn architecture, be ready to give demos and help people get started.

Engineers who are coming from native will most likely feel uncomfortable and overwhelmed by the new ecosystem. Create an environment where getting started with React Native is super easy:

Have an internal wiki page with step-by-step instructions on how to get started. Think of it as a “landing page” of React Native in your organization.
Build a script that automates environment setup: installs NodeJS and other dependencies, runs yarn install, etc.
Setup good IDE defaults. You can’t ask IntelliJ users to configure and use VIM. Document or commit configs that make it much easier to get started with React Native. For example: editor recommendations, plugins, syntax schemes, keyboard shortcuts, etc.
Organize a tech talk or a small hackathon where your team can learn about React Native and try it together on a small project.
Create a place to ask questions, have discussions and hang out. Maybe a #react_native channel on Slack, group, email list, etc. Be there to answer questions.

Super important: if there are people who don’t want to deal with React Native, make sure their development experience is not compromised. They should be able to work the same way as before without having to know or do anything about React Native.

Closing thoughts

If you want to convince somebody to use React Native, ship a feature in your app that wouldn’t have been possible without it.

I hope this small guide gives you few ideas on how to get started. It’s only the beginning.

Lots of big and small companies use React Native in their hybrid apps. Unfortunately, very few of them share the details. I think it’s mostly because some aspects of the integration are specific to their infrastructure and it doesn’t feel useful to talk about these.