DEV Community: YNAB

How we use OpenAPI / Swagger for the YNAB API

Brady Holt — Mon, 16 Mar 2020 21:27:46 +0000

OpenAPI Specification	➡	✅ Tests ✅ Documentation ✅ Client Libraries

YNAB combines software with 4 simple rules to help our users gain control their money. Back in 2018 we released an API to help our community build things to connect their budget to other apps and services.

We built the YNAB API using the OpenAPI Specification and Swagger tooling. In retrospect we think it was a good decision and has provided many benefits. This post gives a landscape of our setup and how the tooling works.

The specification file

Everything starts with our OpenAPI Specification file: https://api.youneedabudget.com/papi/spec-v1-swagger.json. This is a fairly simple JSON file that we edit when anything changes on the API.
This file defines the endpoints and the shape of request and response data.

With this spec file in place, we get access to some great tooling.

Tests

We use a Ruby gem called Apivore to test our actual API implementation against our OpenAPI spec. When our tests run, they exercise all the spec defined endpoints and ensure they are present and work as expected. With Apivore, you have a validate_all_paths method, which does a simple validation to ensure endpoints are available and return the expected statuses. But, you can also exercise the API by passing certain data and Apivore will ensure the shape of the data responses match your spec. So, if your spec says an endpoint returns an array of sharks: [{"id": 1, "name": "megamouth"},{"id": 2, "name": "hammerhead"}] but a particular request returns a single shark: {"id": 1, "name": "megamouth"} a test will fail. Another example: If you were to change the name of a field in the JSON response, a test would fail.

Examples

Here, one of our tests ensures that requesting a non-existent budget at /budgets/:id returns a 404.

it 'response with 404 for a non-existent budget' do
  expect(subject).to validate(
    :get, '/budgets/{budget_id}', 404, params.merge({
      'budget_id' => SecureRandom.uuid
    })
  )
end

Here, we ensure the /budgets/:id/payees/:id endpoint returns a payee, when requested, and also that the shape of that payee conforms to our spec:

it 'returns a single payee' do
  expect(subject).to validate(
    :get, '/budgets/{budget_id}/payees/{payee_id}', 200, params
  )
end

The ability to use a tool like Apivore against our spec is a huge win because we get automatic testing. Having this alone would be a case for using an OpenAPI spec.

Documentation

Using Swagger UI we are able to automatically generate our documentation page just by pointing to our spec. We did make some customizations to suit our preferences but any changes to our spec file are automatically reflected on that page.

For example, the documentation for GET /budgets/:id shows the query parameters that can be passed, expected response status code, and the shape of the response data.

Swagger UI also has the ability to actually use the API on the page itself which is a great help for developers wanting to test something or quickly see the API responses.

For example, this is what the page looks like when requesting a specific payee:

Client Libraries

Arguably, Swagger tooling is most known for Swagger Codegen, which generates client libraries for interfacing with an API.

We use Codegen to generate our JavaScript client and our Ruby client. And others in our community use it to build clients for other languages.

One of the nice things about Codegen is the ability to customize the generated client with the use of templates. For example, we specify a number of templates to override the defaults on our JavaScript client. This allows us to customize things to our suit our preferences.

TypeScript Definitions

Our JavaScript client uses the typescript-fetch generator which, along with generating a JavaScript client usable from both Node.js and the browser, generates TypeScript definition files. This is really useful because developers who are using TypeScript tooling (like VS Code) can get develop-time support.

Examples

IntelliSense support so a developer can clearly see the available fields:

Develop-time errors (a.k.a. red squiggles) so a developer can see when they have accessed a field that does not exist:

Enums selection so a developer can easily select from a list of supported values:

All of this support is coming directly from the original OpenAPI specification file!

Overall Experience

We've been pleased with our usage of an OpenAPI specification and Swagger tooling to build out our API and the ecosystem around it. Of course, there have been a few bumps along the way and we still would like to tweak some things but the amount of benefit this tooling brings is significant and allows us to ship things more rapidly.

The Practice: Semantic Colors for Developers

Emily Carlin — Wed, 12 Feb 2020 20:15:09 +0000

We've already covered the theory behind our semantic color system as well as how it works for designers. This post covers how it works when the rubber hits the road: In code!

We believe that a design system isn’t worth much if it’s only used by designers. So when we started working on this new approach to colors, we knew we wanted to make it simple and usable for developers, too.

So we built a Figma integration that takes all of the color information from our color file and outputs usable code (special thanks/shoutout to Kyle Robinson Young on the YNAB development team, who helped build the integration).

YNAB offers an iOS app, an Android app, and a web app. We rely on our design system to keep things consistent between each platform, so the integration outputs code for each platform.

Each platform has some idiosyncrasies as far as how colors are defined and how light/dark mode switching happens, so we’ll run through each of them.

iOS

For iOS, the integration outputs a color file that is broken into two sections. This is where all of our iOS app's colors are defined.

The first section of the file defines the palette colors. These are private because we never refer to them directly — instead, we refer to semantic colors. This keeps things consistent and guarantees that they look good in light and dark mode.

In the next section of the file, we have our semantic color definitions. These are what actually we use to apply color. Each and every semantic color points at a palette color for light mode and for dark mode.

One of our iOS developers wrote a simple function, that looks at the user’s display mode and determines which color variant to display. That way, things will “just work” between modes. We also built in a safe fallback, in case users aren’t running iOS 13 yet.

We use the description field of the semantic colors from Figma to populate the proper references to the base palette for each semantic color.

Android

For Android, the integration outputs to three color-related files.

First, there’s palette.xml. This is the base color palette where all possible colors are defined.

Next, there’s colors.xml and colors.xml (night). These each have each semantic color listed, with the proper reference to the base palette for each mode — light mode definitions go in palette.xml and dark mode definitions go in colors.xml (night).

When we’re applying colors, we simply refer to colors defined in colors.xml and the right variant renders based on the user’s mode.

Web

We use Stylus for our web app, so we output a palette.styl, colors.styl, and dark-theme.styl file.

As you can probably anticipate from the structure of iOS and Android, palette.styl contains the base palette colors. ‘Colors.styland ‘dark-mode.styl contain CSS variable definitions that reference the proper base palette colors.

So when we are applying colors, we use the CSS variables we define in colors.styl and dark-mode.styl and render the right variant based on the mode the user has selected.

We haven’t released dark mode for the web, so this approach is still experimental, but we’re confident it will work just as well as the system has been working for iOS and Android.

Communication between designers and developers

The fact that we output color definitions directly from Figma means that when a developer looks at a design, they can use the exact same name as the designer and the code will “just work.”

The Figma integration makes it easy and lightweight to keep the system up-to-date; relying on manual updates and communication for a system like this would be a surefire way for it to quickly get out-of-date, messy, and to start to diverge between different platforms.

With this system, we simply run the integration any time colors change in Figma (which is our “source of truth” for all things color). Then developers immediately get access to the latest and greatest colors.

But the technical specificities here are less important than the fact that the philosophy and the language around colors are shared between designers and developers. This system gives us a shared language for talking about colors, and empowers everyone on the team to pick the right color for the job.

Questions? Comments? Get in touch!

That’s all for our series on how we built a semantic color system for designers and developers at YNAB! Feel free to reach out if you have any questions or suggestions. Our hope is that this system can be as useful to other teams as it has been to us!

The Practice: Semantic Colors for Designers

Emily Carlin — Wed, 12 Feb 2020 19:58:24 +0000

In this post, we’ll cover how the semantic color approach we described in the first post works for designers, in practice.

Setting colors up in the design system

Our team’s design tool of choice is Figma, but most of what we describe here could also be accomplished in Sketch (and perhaps other design tools).

Out of the box, Figma only supports one layer of abstraction for colors. You define a style, and that’s that. Here’s how we set things up to support our additional layer of abstraction:

First, we create a file in our Design System project called Colors.

On one page, we set up our base palette. These base palette colors constitute each and every color that can possibly appear in our apps. We make these colors private to the file by prepending a .to the style name. That way, the base palette colors won’t even appear when we publish the color file as a library.

On a new page, we define our semantic colors.

For each semantic color style, we define one light mode version and one dark mode version. In theory, we could define as many as we’d like (maybe we want a “super dark” mode, etc.)

We name these colors with the following convention:

Group Mode/Specific

Groups are things like “Label” “Background” “Header” etc. Mode corresponds to whether the given color is for light or dark mode. And Specific corresponds to the particular usage of the color within the group. This produces colors like:

The semantic color for primary actions, Primary Action is defined with the styles Primary Light/Action and Primary Dark/Action
The semantic color for accents within headers, Header Accent is defined with the styles Header Light/Accent and Header Dark/Accent
The semantic color for our default labels, Label Default is defined as Label Light/Default and Label Dark/Default

As you can see, the light/dark style variants of each of our semantic colors is where we have to hack our way past Figma’s single-layer concept of color.

We further push Figma’s approach to color by denoting the base palette color that each semantic color references in the style’s description field. In the example below, you can see that Primary Action points to primary600 in light mode and primary500 in dark mode.

This description is what connects the semantic color to the base palette color, via a custom plugin that we wrote. Whenever we change our base palette colors, our plugin will automatically update the semantic colors for us, so that we never have to do anything manually.

This description also comes in handy later when we export these colors from Figma to usable code — more on how that works in part 3, where we discuss how this system works for developers.

Using the colors file as a library in other files

When we publish the color file, designers have access to all of the semantic colors.

Each and every color has a light mode and a dark mode variant that are in the same position and have the same name within the color inspector.

So to get a dark mode version of a screen, you can simply toggle each color to use the dark mode variant. This means we don’t have to maintain different components for different modes — everything happens at the level of individual colors.

To make this even easier for designers to use, we plan to build a plugin that automatically flips colors from light to dark (Brian Lovin at Github already built something that does exactly this, though it isn’t available for public use at this point).

We also plan to make a plugin (or use one that someone else has made!) that allows for type-ahead search to make color selection more ergonomic. But for now, we’ve organized semantic colors alphabetically so that designers can quickly find what they’re looking for.

A flexible and efficient system

That’s all there is to it — we build our semantic color file, include it in other files, and go forth and design!

Questions? Want to try out our plugins?

If you have questions about how to set this system up for designers, please feel free to ask! There’s nothing “secret” about this and we’d love for something here to be useful to other designers.

We plan to publish the “semantic color updater” plugin as well as the Figma integration that outputs code (more on that in the next section) — let us know if your team is interested in trying out early versions of either one of these.

The next post covers how this system works for developers — both in the code and in terms of collaboration with designers.

The Theory: A Semantic Color System

Emily Carlin — Wed, 12 Feb 2020 19:35:01 +0000

This post is the first in a three-part series about how our team at You Need a Budget (YNAB) thinks about color in our design system. We'll go over the principles of how our system works, what inspired us to build it, and why we're excited about it.

The other posts cover how the system works for designers and for developers.

Introduction

Over the past few months, the team at YNAB revamped how we approach colors in our design system.

Colors are deceptively simple. They are a foundational element of interfaces but all-too-frequently end up becoming a sprawling mess of inline definitions, different naming conventions across different platforms, confusion around when to use what color, and issues with updates when things change. And dark mode support, which is becoming the norm, introduces yet another dimension of complexity to colors.

Our new system addresses these challenges head-on by using a combination of semantic naming and an additional “layer of abstraction” (more on what exactly this means later) in both our designs and in code.

This series is all about what we might call the “form” of colors in our design system: How we structure the system and how it works for designers and developers.

This series won’t cover the “content” of colors in our design system: Things like how we selected the actual color values we use, how we ensure adequate contrast, our color decisions in designing for dark mode, etc.

Okay, let’s get into it!

What’s in a name?

If you’re a designer or a developer, chances are you’ve seen the words “semantic” and “color” in the same sentence before. It usually refers to a way of naming colors based on how they are used as opposed to their hue. This is one of many theories on the best way to name color.

With a semantic name, you might name a color “destructive” or “negative” — something that connotes what the color means in your interface, as opposed to something like “red” (or even something more fun, like “tomato”).

Naming colors semantically has two benefits:

It helps designers and developers decide what color to use. Instead of needing to memorize or check documentation to decide what color to make a “delete” button, you simply grab the color that relates to destruction.
It makes your color system more efficient and flexible. If you decide you want to update your primary color to be purple instead of blue, you simply update the value for the primary color (assuming you’re using color styles in a tool like Figma or Sketch and color variables in code). If the colors were named based on hue, you’d need to go change every single place the color is used from “blue” to “purple.”

Our new approach to color introduces an additional layer of abstraction to address some of the problems that are left unsolved by simply naming each of your colors semantically.

The base palette

A color palette is, of course, an integral piece of any design system. The palette represents the universe of possibility for color in an interface.

A palette is a great start for building visually consistent apps — a defined set of colors mitigates the chance that a rogue hex value will sneak its way in.

Here’s what our palette looks like at YNAB:

Each of the colors has a semantic name (e.g. primary, accent, etc.) as well as a number that corresponds to its lightness (e.g. 900 for the darkest variants; 100 for the lightest variants). But if we stopped here, a lot would still remain up for interpretation as far as when to use what color. And not in a good “this leaves space for creativity!” way; more in a “What primary color variant do we use for buttons again?” way.

The semantic colors

Our semantic colors go further than simply naming colors based on the general realm of their usage. They are a second layer of abstraction that sits on top of the base palette.

Our color system consists of two layers:

The base palette, which defines every possible color value in our app.
The semantic palette, which defines colors based on how they are used. Each and every semantic color points to a color from the base palette.

For example, we have a semantic color for our primary action color and another one for the background of headers. Each of those semantic colors references a color from the base palette.

Now, you may be thinking, “This seems like a bunch of extra work for something that we get for free with symbols/components.” And in the case of something like a button or a header, that may very well be true — when you design the primary button component, you pick a color from the palette and then others simply use that component in their designs without needing to think about the specific color.

But as things get more complex, the benefits of this two-layer system become clear.

Visual Consistency

YNAB is budgeting software, which means that we frequently use color as a visual signal for positive and negative account balances (we never rely only on color, but color and accessibility would be a whole separate post).

For example, the budget view features “pills” that display the amount remaining in a given category of a user’s budget. There is also a bar at the top of the screen that displays overall budget status.

To keep things consistent, we want both of these elements to use the exact same color. That way, users can start to learn that when they see this particular red color, it always means the same thing — a negative balance!

In our semantic color system, we name this color statusNegative and set it to reference a color from our base palette. So if we want to update this color, we simply point it at a different palette color and the change propagates to all elements that are meant to convey a negative status. And if we update a base palette color, the change will ripple through all of the semantic colors that reference that color (and then all of the elements that use those semantic colors).

This is what we mean by introducing an additional layer of abstraction. Semantic colors act as an intermediary level of specificity, between the raw value of colors in the base palette and the usage of those colors in specific components.

And here’s the best part: Semantic colors make it easy for designers to create new elements that are visually consistent with the rest of the app. If we wanted to create a graph on a dashboard that is in a negative state, we’d simply drop in statusNegative instead of arbitrarily searching through our red hues.

Dark Mode

Beyond visual consistency, a major benefit of this system is how easily it supports dark mode. In fact, adapting our apps to dark mode is what inspired this approach to color in the first place.

Let’s consider the color we use for primary elements — primaryAction. This color references primary600 from the base palette.

Great. But what about dark mode?

Well, all we have to do is create an additional mapping for primaryAction.

For dark mode, we map it to primary500, a lighter blue, because we want primary elements to “pop” more on dark backgrounds (like accessibility, choosing colors for dark mode could be a whole separate article. We won’t get into the specifics here, but check out this article by the team at Superhuman for a great primer).

Each of our semantic colors points to two base palette colors: one for light mode and one for dark mode.

That way, we can just apply a primaryAction style to our buttons and the right palette variant will render based on the user’s mode.

You may be thinking: “Why can’t you just define a dark mode variant of primary600? This doesn’t work because there are some colors that should get darker in dark mode (think backgrounds) and some colors that should get lighter in dark mode (think labels and accents).

Long story short ...

The two-layer semantic approach to colors allows us to flexibly update colors, keep our app visually consistent, and efficiently design for dark mode.

The next two posts in this series will cover the ins and outs of how this system actually works — for designers and for developers.

Thanks to Alan Dennis (who came up with the structure of this approach) and the rest of the YNAB design and development teams for helping to bring this system to life.

Additional thanks to Alan Dennis, Dylan Mason, and Tristan Harward for giving feedback on the first draft of this post.

YNAB at KotlinConf 2019

Brady Holt — Mon, 06 Jan 2020 14:38:07 +0000

A few weeks ago some of the YNAB team attended KotlinConf 2019.

We have been using Kotlin in our Android app for some time and are particularly interested in its multiplatform capability as a way to share code across Android, iOS and the Web.

After attending quite a few multiplatform focused sessions, chatting with folks at the conference, and discussing things as a team, we're planning to start experimenting with multiplatform usage across Android and iOS and then eventually on the Web. There is significant potential here that we want to explore!

Left to Right: Kevin, Graham, Taylor, Jeff, Brady

Elegant memory management with Kotlin and J2V8

Graham Borland — Tue, 29 Oct 2019 17:22:52 +0000

At YNAB we have a cross-platform library written in TypeScript (compiled to JavaScript) which contains all of our shared business logic for Android, iOS and Web.

On Android we use J2V8 as our bridge into the JavaScript world; it's a nice Java wrapper around Google's V8 JavaScript engine. It works beautifully, but one of the challenges it brings is memory management. It's so tricky that the J2V8 maintainers wrote a blog post about it.

Because J2V8 bridges V8 and Java, three different memory models are in play. Both Java and JavaScript provide a managed memory model with their own GC. JNI / C++ which sits in the middle is completely unmanaged. This leads to a complex situation...

To cut a long story short, we have to explicitly release any JS objects we create in our Java/Kotlin code.

Remember to close the door

We can release these objects manually:

// create a JS object
val obj = V8Object(v8)

// do something with the object in JS land
obj.add("someProperty", 54321)
obj.executeJSFunction("someJSFunction", 42)

// now release it
obj.close()

But it's a bit of a pain having to remember to call close() for every object. V8Object implements the Closeable interface which means we can use Java's try-with-resources or Kotlin's use { } to take care of cleanup where we only have a single object to deal with.

V8Object(v8).use { obj ->
    obj.add("someProperty", 54321)
    obj.executeJSFunction("someJSFunction", 42)
}

It gets hairy when we need to track multiple JS objects, though. To help, J2V8 provides a MemoryManager. When we create one of these it starts tracking V8Object allocations while it is open, and releasing the MemoryManager causes all of those objects which were allocated during its lifetime to be released in turn.

val manager = MemoryManager(v8)

val obj1 = V8Object(v8)
val obj2 = V8Object(v8)

obj1.add("someProperty", 54321)
obj2.executeJSFunction("someJSFunction", 42)

manager.release()   // obj1 and obj2 are both released

It would be nice if we could use try-with-resources or use { } again here, to avoid the explicit call to manager.release(), but MemoryManager doesn't implement Closeable so we can't.

An elephant elegant solution

What we can do, though, is add a helper function which wraps all the MemoryManager stuff and provides a scope for allocating and safely cleaning up as many V8Objects as we like.

inline fun <T> V8.scope(body: () -> T) : T {
    val scope = MemoryManager(this)
    try {
        return body()
    } finally {
        scope.release()
    }
}

It has to be inline so that we don't interfere with any return value from the body lambda. And making it an extension function on V8 gives us this concise and elegant syntax.

v8.scope {
    val obj1 = V8Object(v8)
    val obj2 = V8Object(v8)

    obj1.add("someProperty", 54321)
    obj2.executeJSFunction("someJSFunction", 42)
}   // obj1 and obj2 are both released

Elephants never forget... and now, neither do we! This approach helps us to solve some of the memory-related pain points when mixing JavaScript and good old Java/Kotlin code, without too much boilerplate. We do have a keen eye on Kotlin multiplatform for the future, but our JavaScript shared library is serving us very nicely in the meantime.

The code is available on GitHub.

Progressive Conversion of TypeScript Namespaces to Modules

Brady Holt — Mon, 16 Sep 2019 00:00:00 +0000

We love TypeScript at YNAB. One of our main modules is something we call the “Shared Library” and it is a quite large TypeScript project. Actually, it’s comprised of 3 library projects and 3 test projects. It’s big. And, it was initially written using TypeScript namespaces, before TypeScript had support for ES modules.

We wanted to start converting this library over to using ES modules for the various benefits that gives including the ability to tree-shake and better development tooling. But, it became obvious we needed to find a progressive way to do this because a few attempts at an all-or-nothing approach proved daunting. Thousands of errors and no simple or obvious way to automate the conversion.

Guidance for progressively converting a project from namespaces to modules is slim. There is this GitHub issue where some discuss approaches but there are still some gaps the approaches.

To setup progressive migration, we ended up doing the following which has been working well for us.

Add "exclude": ["**/*.m.ts"] to the tsconfig.json file in the original global / namespace project. This allows you to create modules with .m.ts extention and keep the outFile config.
Create a tsconfig.module.json file adjacent to the main tsconfig.json file.
Use a project reference in the tsconfig.module.json file pointing to the namespace project: "references": [{ "path": "./tsconfig.json" }]. This makes the global / namespace types available from within the modules project. It doesn’t emit the code for this project but it tells TypeScript to assume these types will be available at runtime.

It looks like this:

tsconfig.json

{
  "compilerOptions": {
    "outFile": "../dist/package.js",
    "composite": true
  },
  "include": ["**/*.ts"],
  "exclude": ["**/*.m.ts"]
}

tsconfig.module.json

{
  "compilerOptions": {
    "module": "esnext"
  }
  "files": ["./MyClass.m.ts"],
  "references": [{ "path": "./tsconfig.json" }]
}

With the above setup, you can create module files with the extension .m.ts adjacent to the existing namespace files like this:

- MyClass.ts
- MyClass.m.ts

Then, as @danielrosenwasser pointed out, you can use existing code from the namespaced code and wrap it in the module file. You can also just export it as is:

export import MyClass = my_namespace.MyClass;

Since this setup is using ES modules, you can now use a bundler like webpack to prepare it for web consumption. In the following example, webpackwill be used with ts-loader. Notice ts-loader is configured to use the tsconfig.module.json config file.

webpack.config.js

module.exports = {
  entry: './src/MyClass.m.ts',
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: [
          {
            loader: 'ts-loader',
            options: {
              configFile: "./tsconfig.module.json" // Use the module project config!
            }
          }
        ]
      }
    ]
  },
  output: {
    filename: 'bundle.js',
    path: path.resolve(__dirname, '../dist')
  }
};

Finally, in your entry webpage, you can include a reference to the original namespace outFile and also the webpack bundle.

index.html

<head>
  <script src="dist/package.js"></script> <!-- outFile output from tsc -->
  <script src="dist/bundle.js"></script> <!-- webpack bundle -->
</head>

Once you’ve converted over all the namespaced source files to modules, you can remove the original namespace project and stop loading it.

A more exhaustive example can be found in this repository: https://github.com/bradymholt/ts-progressive-convert-namespace-modules.