DEV Community: Kenneth Skovhus

Addressing Client/Server Compatibility in tRPC

Kenneth Skovhus — Tue, 11 Jul 2023 22:00:00 +0000

I'm a huge fan of removing the boundary between the frontend and backend environment, as seen in libraries like tRPC and web application frameworks like Remix and Next.js.

One of the main selling points of tRPC is "End-to-end typesafe APIs made easy", but this doesn't hold if you are not mindful about keeping your client/server versions in sync. Stale web or mobile applications will eventually make calls to a newer version of the server, leading to potential compatibility issues. This is also known as version skew.

The magnitude of the issue can vary from detectable API failures to subtle UI problems or even data inconsistencies.

In this blog post, I'll share the solution we came up with when addressing this issue for the tRPC-powered Backend-For-Frontend setup at Pleo.io.

Library Guidelines & Best Practices

Frameworks and tools tackle the compatibility challenge differently. For example, GraphQL recommends making all changes backward compatible. Next.js only recently introduced a beta of a new skew protection feature, but also automatically update to the latest version in the background when routing, but API routes (used for actions/forms) are not automatically versioned. Remix does not handle automatic upgrades – any action on the page will break if the client is outdated and the loader is incompatible. For tRPC, it is suggested by the community to keep the client and server in sync.

General Strategies

There are several general strategies to resolve the client/server version compatibility issue:

Enforce clients to be in sync with the latest server. We cannot guarantee this, but we can nudge the user to reload their browser, update when the user navigates, and add a reload call to action in case of API failures for forms/actions in the application. For mobile applications using React Native we can use CodePush, although it does introduce some lag and doesn't ensure all clients to upgrade.
Maintain backward compatibility for a grace period until we expect the clients to be updated.
Manually or automatically version the API endpoints and gracefully keep old versions around until clients have migrated to the new endpoints. Manual versioning is standard practice but isn't suitable for tRPC, Next.js, or Remix, where the actual endpoints are abstracted away.

Each strategy has tradeoffs regarding infrastructure complexity, ease of rollback, and friction for developers and end-users.

Let us explore two of the most promising strategies in more detail.

Solution: Backward Compatibility

Maintaining backward compatibility is a fairly standard solution and doesn't require any special infrastructure setup (i.e. multiple server versions).

However, without any custom build time tooling it introduces some friction for the developer that needs to:

maintain backward compatibility
clean up old procedures or fields in input/output DTOs

This can be mitigated by building a custom validation script that would compare the latest server code with an older version (e.g. from a week ago depending on the grace period). The script would ensure:

the latest server code compatible with the older version (e.g. input and output DTOs are a superset of previous versions, and existing procedures are not removed)
inform the developer if some deprecated code should be removed

Even though this approach is viable, it introduces some friction for the developer and requires custom tooling.

Solution: Auto-Versioning

To avoid any friction for the developer, we opted for making our tRPC server code immutable and auto-versioned. It requires some custom infrastructure setup, but it allows for a great developer experience and a seamless end-user experience – especially for web applications.

This can be implemented using serverless services like Cloudflare Workers or AWS Lambda. The solution shares similarities with Next.js' new skew protection feature, but we recommend keeping the version active for longer than 24 hours to ensure compatibility with running applications.

For every deployment we:
1) Calculate a stable hash (e.g. version) of the server source code, configuration, deploy script, and relevant environment variables
2) Verify if any deployed function matches the hash (if so, then skip to step 4)
3) Create a new function with a unique URL
4) Bake the function's URL into your web or mobile application

That's all. 🥂

Want to see some code? Here is an early prototype using the node.js AWS SDK.

A Word of Caution for Mobile Applications

The design choice of making the server code immutable and auto-versioned has some tradeoffs. What if server business logic needs to be updated? What if the database schema or the underlying API services changes? The turnaround time for these changes depends on the time it takes for clients to upgrade. For web applications, it's usually a matter of days, but for mobile applications, it can be months or worse (even when using CodePush).

I would still recommend using auto-versioned servers for tRPC-enabled mobile applications, but I suggest relaxing the immutability requirement by making it easy to update the server code of existing server versions.

Beyond Bug-Squashing: Start Improving Your System

Kenneth Skovhus — Fri, 30 Apr 2021 10:00:00 +0000

If debugging is the process of removing software bugs, then programming must be the process of putting them in.

– Edsger Dijkstra

I really enjoy fixing software bugs! The detective work, building up a hypothesis about the underlying root cause, digging through paper trails and turning stones to find the culprit. But what I enjoy the most is the opportunity to improve the system. Unfortunately, developers sometimes skip this last step by quickly squashing a bug and then carrying on with other activities.

While a quick fix seems efficient you miss important learnings and a perfect opportunity to:

clean up technical debt and making your system easier to maintain
helping co-workers and future you to not commit the same mistake again
understand the quality of your system’s safety nets
lay the groundwork for spending less time fire fighting and more time developing.

I tried to generalize my mental model for fixing bugs. It boils down to a 4 step process:

determine the severity: do we need to do anything about the issue right now?
improve safety nets: how can we catch the bug now and avoid similar bugs in the future?
the actual fix
post-mortem analysis (if applicable)

Step 1: Determine the severity 🐛

When you discover a bug (or incident), the first step is to determine the severity. If you are working on fixing an issue that is already triaged or prioritized, you can skip this step and go straight to step 2.

Some questions to consider:

if the issue revealed itself after a release, should we just roll back or revert?
is there anything else we can do right now to restore the service(s)?
do we need to inform anyone about the issue (e.g. product owner, customers or your co-workers)?
when should the issue be fixed? We should always be mindful to avoid context switching.

Step 2: Improve safety nets 🚨

How can we enable our system to catch or highlight the issue – now and in the future?

You might answer this question by adding one or more of the following safety nets:

improve the linting setup
introduce or refine a static type
automated tests (unit, integration, end-to-end)
add support for feature flags for faster rollbacks
improve monitoring, alerting or observability
manual test protocol (should be a last resort).

Word of caution: establish these safety nets by working within the system and avoid extensive refactoring that might introduce new regressions.

At the end of this step, you should have a red lamp.

If you feel you don't have time to improve your safety nets before fixing the issue then take a step back. How can you get more time to fix this properly? If the production system is in a really bad state, can you revert to a previous version? Could you disable the broken feature by flipping a feature flag? Buy yourself some time to improve the system (hopefully it will pay off).

Step 3: The fix ✅

Now is the time to fix the code, infrastructure configuration, or what caused the bug or regression. Turning the lamp green is often the easy part.

Step 4: Debrief

You should consider if the severity of the bug merits an incident debrief (traditionally called post-mortem).

A debrief is a blame-free analysis and discussion soon after an incident or event has taken place to learn from the experience. It ensures that an incident is documented, that all contributing root cause(s) are well understood, and, especially, that we learn from the incident and that preventive actions are put in place to reduce the likelihood and/or impact of recurrence.

I recommend that you trigger a debrief in case of data loss of any kind, user-visible downtime or degradation, manual developer intervention (e.g. release rollback) or when we lost a substantial amount of time firefighting.

This guide by Atlassian or this one by incident.io seems like a great resources if you want to learn more about incident management and debriefs.

Happy debugging!

On September 9, 1947, the world's first computer bug was recorded by Grace Hopper. It was a real-life moth that was causing issues with the computer’s hardware.

Guidelines for Better Pull Requests

Kenneth Skovhus — Wed, 12 Aug 2020 10:00:00 +0000

Pull requests play a crucial role in ensuring quality code and efficient collaboration within software development teams. In this blog post, we'll explore 7 guidelines to help you create better pull requests that will enhance your workflow, streamline feedback, and elevate the quality of your projects.

Background

During my work to align our development efforts at Leo Innovation Lab I stumbled upon 10 tips for better Pull Requests by Mark Seemann (2015). The blog post aligned well with our perception of a good pull request (PR) and was a great starting point for discussions.

Below is my iteration of the original post. This is a condensed version with a focus on automation, tooling, context, and my personal learnings.

The goal of these guidelines is to make the pull request review process easier and will likely:

improve the feedback you get on your work
reduce the risk that your pull request will be rejected or become stale
elevate overall the quality of the code and product.

Guidelines

1. A single concern per pull request

The more concerns you address in a single Pull Request, the bigger the risk that at least one of them will block acceptance of your contribution. Do only one thing per Pull Request. It also helps you make each Pull Request smaller.

– Mark Seemann

If you often clutter PRs with multiple concerns then ask yourself why. Maybe your CI setup is slow? Maybe merging is cumbersome? In any case, start by fixing the underlying problems to enable you to do only one thing and make your PRs small.

What is a small PR? A good rule of thumb is to limit the scope of your PR to a maximum of a few days of work, preferably less. For instance, if you're fixing a bug, make sure the PR only addresses that specific issue, rather than including unrelated improvements or refactoring. This also gives a great sense of progress. And usually you can break work up into smaller daily deliverables.

2. Short-lived feature branches

Avoid any long-lived feature branches. Getting your changes into the main branch as quickly as possible is often the most productive. It might require that you invest in setting up a system for supporting feature toggles.

3. Utilize proper tooling

A reviewer should not spend any time nitpicking over style, ensuring that tests pass or figuring out if the code coverage dropped. Instead you should invest in tooling and automate any verification steps. Block any merging until your CI server is happy and let your reviewers focus on more interesting topics than nitpicking over line width or spinning up the project on their local machine.

4. Add appropriate context

Adding context for the suggested change is important for the review process. This also serves for future reference when someone is trying to understand why a change was made.

Include the following in your PR description:

link to bug ticket or issue (if a link is missing this is usually an indicator that the work is not planned)
a description of the suggested change
relevant design considerations and alternative solutions considered
checklist with TODOs if the pull request is a draft
if the change can be represented visually, then add a screenshots/gif/video showing how the state before and after the change. Note that this also applies for a backend change (e.g. diff the GraphQL endpoint, show a database optimization using EXPLAIN or similar)

I recommend using a pull request template to guide and ensuring that the relevant fields are filled out.

5. Document your code

Strive for self-documenting code (e.g. using clear and consistent naming, types and function signatures). But if you need to explain the code logic in the PR description, then it probably calls for better code comments.

Code comments should document the why, not the how. Commit messages is also a great place for additional context.

6. Write well

Use correct spelling, grammar, and punctuation in code, code comments, commit messages, and pull requests. If you don’t, your prose (and code) is harder to understand for the reviewers and future maintainers. Note that most code editors can be configured to help you with these concerns.

7. Avoid commit thrashing

When a PR is in review, please avoid rewriting the history on the branch (e.g. amending, squashing, or rebasing in general) and address any review comments in new commits.

If you need to update the branch then avoid trashing the commit history with Merge branch 'master' into MY-BRANCH commits. This can be done by using rebase instead of merging (e.g. git pull --rebase origin master if you are using git).

Summary

In summary, creating better pull requests involves focusing on a single concern, keeping feature branches short-lived, utilizing proper tooling, providing context in your PR, and documenting your code effectively. By following these guidelines, you'll improve your collaboration with your team and elevate the overall quality of your projects.

Migrating from Flow to TypeScript using flow-to-ts

Kenneth Skovhus — Mon, 10 Feb 2020 12:00:00 +0000

Over the last couple of years, I have been migrating several codebases from Flow to TypeScript with minimal effort. In this post, I will describe my motivation and the approach I would recommend.

TL;DR I recommend migrating using the flow-to-ts codemod by Khan Academy.

Motivation

I started using Flow shortly after it was open sourced by Facebook in 2014. It supports many of the same features as TypeScript (open sourced by Microsoft 2012) but focuses on being a static type checker where TypeScript is also a compiler. I originally picked it over TypeScript due to the momentum in the React community and the easy gradual migration path for existing code.

But as the TypeScript community and tooling have been rapidly improving, I decided to abandon the Flow ship. This also seems to be a trend in the open source community (example: Jest, Yarn, and Expo).

My primary motivation for moving to TypeScript:

TypeScript seems more well-maintained and more people working on it: In January 2020 TypeScript had 29 contributors, Flow had 14 contributors, and there were 156 pull requests merged compared to no Flow pull requests merged
in my experience, it is easier to onboard new people to a TypeScript codebase as it is more widely used and the online material is vast
in most editors (notably Microsoft' Visual Studio Code) TypeScript shines: auto import, refactoring, quick error reporting out of the box
more third-party types are available (estimated to roughly 8X)
more libraries are written in TypeScript (naturally improving the quality of the interface types compared to reverse engineering the types)
too often I've discovered Flow silently bailing out type checking, I have not experienced this with TypeScript
although Flow should be more sound by design, I uncovered type-related bugs after migrating a codebase from Flow (maybe related to Flow silently bailing out type checking)
installing types using npm / DefinitelyTyped makes a lot of sense compared to flow-typed where type definitions are checked into your repository (and you forget to update them)
the feature sets and syntax are very comparable, so migration is easy (see this great comparison)
I have experienced Flow running out of memory when type checking a React Native projects. It was a real defeat that I needed to disable type-checking on CI as we ran out of memory. 🤦🏼‍♂️

Approach

I tried out a bunch of different tools (codemods), that automates the refactoring from Flow to TypeScript. If the concept of automated refactoring with codemods is new to you, you might find a talk I gave on the topic interesting.

The best migration tool I have found is Khan Academy's flow-to-ts. From their README:

The goal of this project is to provide a tool that can translate 95% of Flow to TypeScript while maintaining a high percentage of the existing type information.

⚠️ A word on code reviews: You will end up changing most files in your project, so it will be difficult to review. To make this process as easy as possible, I recommend that you make a separate commit for each of the following steps.

Step 1: Prepare your codebase

Before running flow-to-ts I recommend preparing your codebase for type-checking using TypeScript.

1) Remove any checked-in types (typically in a flow-typed folder) and config:

rm -rf flow-typed .flowconfig

2) Install the TypeScript package:

yarn add --dev typescript
# or
npm install --save-dev typescript

3) Remove the Flow package:

yarn remove flow-bin
# or
npm remove flow-bin

4) Setup a tsconfig.json file for configuring TypeScript. There are many ways to do this, but running npx tsc --init is a good start. Read more about the configuration file.

5) Update your linter to support TypeScript. I would recommend using TypeScript ESlint

6) Update your build setup to support TypeScript. Here are a few pointers for some popular setups:
- Create React App
- Babel setup
- Webpack

Step 2: Migrate all the things!

1) Run the flow-to-ts code transformation:

# Replace src with your source folder or a glob
npx @khanacademy/flow-to-ts --write --delete-source src

2) [Optional] If you use a code formatter (like Prettier) then format your new codebase

3) Commit the changes (e.g. "Migrate to TypeScript using flow-to-ts codemod")

4) This is the most time-consuming part: try compiling the project with TypeScript and fix each problem you encounter:

npx tsc --noEmit -p .

Start from the top and move your way through... It will take some time. You probably need to install a bunch of types for your libraries (e.g. @types/react).

If you find yourself doing a lot of repetitive fixes, please share these in an issue over at the flow-to-ts repository and help improve the codemod.

5) Finally, when everything compiles again, check that you can lint and build your project

6) 🥂

7) [Optional] You can now remove any Flow suppression comments (e.g. $FlowFixMe)

8) [Optional] Some Flow utility types are replaced by the utility-types package. You can get rid of some of them if you like (e.g. $Keys<T> can be replaced by keyof T).

How much effort does it take?

It all depends on the quality of your Flow types, the libraries you use, and the size of your codebase.

With the flow-to-ts codemod, you can quickly try out TypeScript on an existing codebase and assess how much work the manual part of the migration would take.

I converted a 30K lines React Native app in a few days. While doing this, I also spend time fixing a few potential bugs that TypeScript uncovered. 🎁

So long Flow! 👋 And thanks for all the checking + healthy competition in the static type space.

Software Engineering Principles

Kenneth Skovhus — Mon, 13 Jan 2020 20:00:00 +0000

After reading the principles of the React team I got inspired to write this post on the high-level technical principles and beliefs that I operate by.

These 8 principles, gathered over 20 years, are not exhaustive and may change as I (hopefully) gain new perspectives and wisdom. I have added a bunch of links and recommendations, that I hope you find inspiring. ☕️

People first, technology second

Technology’s ultimate purpose is to advance humanity. If we do not put the human first in everything we do, then what is the point?

Before I studied computer science, I worked as a technical illustrator and later as a freelance web designer. I was always fascinated by human-computer interaction. Reading Designing Web Usability (1999) by Jakob Nielsen was a real eye-opener. Later in life, working together with really skilled UX designers and product owners changed how I approach problems.

We can easily forget about users while getting carried away solving problems. I believe all technologists will benefit from understanding user-centered and human-centered design.

Recommended material:

Offscreen is my favorite print magazine on how we shape technology and how technology shapes us
Donald Norman’s classic best-selling book The Design of Everyday Things studies fundamental principles of great and meaningful design.
Steve Jobs, just returned to Apple in 1997, as a response to a tech-focused insult: “You’ve got to start with the customer experience and work backward to the technology. You can’t start with the technology and try to figure out where you’re going to sell it.”.

The only constant is change

The greek philosopher Heraclitus knew it 2500 years ago: “Everything changes and nothing stands still”.

Software differs greatly from construction engineering or architecture, despite the common metaphor. Code is organic, it evolves, and needs constant maintenance. I love the analogy that software development resembles gardening. We are gardeners constantly dealing with things changing. And most changes do not turn out the way you expected.

To optimize a codebase for change:

Keep things simple
Prefer boring, explicit code
Invest in documentation, automation, and tests
Encapsulate code that is likely to change (e.g. vendor-dependent modules, business rules, and implementation details)
Make ownership clear, and consider product-mode over projects
Write code that is easy to delete, not easy to extend (recommended reading).

Start with why before jumping into solution mode

I find it so rewarding to solve the right problem in a lean and simple way, and to avoid wasting time on the wrong path.

Solving problems is fun—usually more fun than planning. But you might find yourself jumping into solution-mode before understanding the problem. This can also happen when leadership above you commands you or your team to do a certain task without a clear why. Instead, incorporate asking questions:

What problem are we trying to solve?
What ideas do we have to solve it?
What are the positive and negative implications?
Are there simpler solutions to the problem?
What are our core assumptions?
What are the technical details of the technologies used?

Be curious and come up with more open questions!

Build-Measure-Learn

Requirements and your perspective will change as soon as you ship your solution (or simply show it to users). So avoid gold-plating your design or code, and get your solution in front of real users as fast as possible. It's better to fail fast than to waste time.

In most projects, the first system built is barely usable… Hence plan to throw one away; you will, anyhow.

– Fred Brooks, The Mythical Man-Month (1975)

Build quick prototypes, cut down on scope, and plan to throw code and designs away. Shorten the build-measure-learn cycle.

Prefer duplication over the wrong abstraction

“Don’t Repeat Yourself” (DRY) is often misunderstood. Although programming is often about being lazy and avoiding repetition, prematurely sharing code can be really expensive. Even if the code and functionality look the same at the time, the requirements or direction might differ down the line.

Recommended material:

Sandi Metz The Wrong Abstraction
Joe Spolsky The Law of Leaky Abstractions. “All non-trivial abstractions, to some degree, are leaky”
Dan Abramov Goodbye Clean Code.

Invest in safety nets

“We don’t have time to write tests” means you spend too much time bug fixing because you don’t have the necessary safety nets. Have enough safety nets to not be afraid to deploy to production when merging your main branch. In case you end up breaking production, make sure to identify root causes, learn from your mistakes, and figure out new safety nets to implement.

Safety nets checklist:

A bunch of automated unit and integration tests
A few automated end-to-end system tests
Documented manual procedures and test protocols (if needed)
Infrastructure as code
Ensure proper monitoring and alarms
Process for incident postmortems.

Cross-functional teams

Splitting teams based on competence, or where in the stack their code runs, is easy, but often counter-productive. It introduces an overhead in terms of communication and requirements specifications. The most effective teams I have worked in, where all diverse and cross-functional (design, UX, frontend, backend, product, domain experts).

Give back to the OSS community

Most software companies rely heavily on Open Source Software (OSS). Just enumerate all the different projects you depend on!

In 2017 my new year’s resolution was to give back to the OSS community. It has been a great ride so far. Heck, one might say that this blog post is a continuation of that.

I do not expect anyone to spend their weekends and late evenings on open source. But why not convince your employer to contribute? Get time reserved for reporting bugs, triaging issues, doing pull requests, sharing knowledge in blog posts and talks, open-sourcing some useful homegrown system, or simply sponsor projects you depend upon.

What are your engineering principles?

These are some of the high-level engineering principles I operate by when building software – I hope you found them useful.

Do you, or your team have a set of principles? I would love to hear about them.

This blog post was cross-posted from https://skovhus.github.io/software-engineering-principles/

Thanks to Maciek Pekala for reviewing this post.

Converting an app to React Native — How to get started

Kenneth Skovhus — Fri, 13 Apr 2018 15:05:00 +0000

This is the second part of a series on migrating existing native apps to React Native. Check out the first blog post if you want to know why we went for React Native and how we approached it.

React Native under the hood, a crash course

To understand how React Native works on mobile, you first need to look at the virtual DOM (Document Object Model) that is used in both React and React Native. If you’ve ever done web development, you know how the DOM works in the browser. If not, you might be interested in reading up on it. In short: The DOM represents a page as nodes and objects. In plain React, the virtual DOM is an in-memory representation of the UI on top of the actual DOM. When a node or object changes, React will diff the virtual DOM and the actual DOM, and only update those nodes or objects that have changed. In React Native, there is no browser DOM. Instead, the virtual DOM is rendered into native iOS or Android views.

Virtual DOM in React and React Native

While you write plain JavaScript and JSX (tag syntax that is an extension to JavaScript) to define how your React Native app should look and behave, the UI is still rendered natively. Whenever there is a change in one of the nodes or objects, React Native will update the relevant parts of the UI automatically.

This is what sets React Native apart from other cross-platform solutions. Rendering native components significantly speeds up interaction with the app, and while this also introduces some complexities (sometimes you will need to create a custom UI component), for us, it was definitely a good trade-off.

Navigation is hard

Navigation has always been a bit of a problem on React Native, but the vast amount of packages that are available show that this area is under development. Ideally, Facebook would create or, at least, recommend one solution. But for now, that’s not the case, and we all have to choose a third party library and stick with it. For us, the main contenders were react-navigation and wix/react-native-navigation. The first is very flexible and runs everything purely in JavaScript, while the second one is based on native navigation on both platforms. We decided to go with react-native-navigation because we felt like native components would make the app feel more familiar to our users.

EDIT FROM JANUARY 2019: the native feel of react-native-navigation didn’t make up for the amount of Android bugs, tight coupling with screen components, and non-declarative API. We migrated to react-navigation.

Getting used to JavaScript

JavaScript? No, thanks

In mobile development circles, most people cringe at the thought of writing an entire app in JavaScript. Why would you even do such a thing? JavaScript has a lousy reputation, and rightfully so. If you worked with JavaScript a decade ago, you likely had a horrible experience. Suitable for small scripts on websites, but not ideal for developing an application of any significant size.

However, things have changed, and over the last ten years there have been significant development efforts in tooling, frameworks, and engines. We saw the creation of jQuery, V8, Node, Backbone, … All of these efforts propelled JavaScript into one of the most used languages in the world. Today, Google’s Angular and Facebook’s React are the two most popular JavaScript frameworks. They are used by Google and Facebook themselves and provide a solid foundation for app development.

Modern JavaScript

JavaScript itself also saw significant advances in the last few years. When EcmaScript 6 came out, developers finally had access to features that were already common in most modern programming languages, such as classes, arrow functions (aka lambdas), string interpolation, let and const, and many more. Meanwhile, CoffeeScript and Babel pioneered transpilation so everyone could start using new language features that were not yet implemented by all browsers or engines. ES 7 and 8 kept on advancing the language significantly, and by now we can say that JavaScript can be a very nice language to work with.

Of course, it’s not all sunshine and rainbows. While the language itself is getting better, it is still difficult to set up a good development environment. Because of the dynamic nature of the language, doing something as simple as renaming a variable can still be a challenge. Coming from Android, you might find JetBrains’ IntelliJ useful, because of its familiarity. Web developers tend to stick with editors like VSCode or Atom when they go to React Native. As long as the plugins are there to support what you need, you can use any editor you want.

We found that a lot of JavaScript’s shortcomings can be countered with internal coding conventions and a good tooling setup to enforce them. Once you get into the habit of writing good, idiomatic JavaScript in a proper architecture, it becomes quite nice, even when you come from Swift or Kotlin in native land.

Tooling

After we understood how React Native works and decided to make peace with JavaScript, we wanted to make sure we had the right tooling set up to enforce best practices, and that the native developers on our team are made aware when they write non-idiomatic JavaScript. The setup, with a variety of tools from the JavaScript and React ecosystem, also helps us write more maintainable code that is easier to read.

Static analysis and code consistency

The dynamic and loosely-typed nature of JavaScript makes it especially prone to runtime errors. To help us find these errors before running the app, we use ESlint. ESlint also helps us see dead code and detect problematic patterns that sneak into the codebase. Our configuration is based on the widely used eslint-config-airbnb.

Although ESlint can also check that a codebase adheres to a specific style guideline, we strongly believe that a code style should be enforced by a tool. Instead of debating over coding style, we use Prettier to format our code. It integrates with ESlint, so when hitting save in our editor the code is formatted and statically analyzed.

State management

For state management, we enjoy the simplicity and testability of Redux. We use the redux-persist middleware to read and write parts of our Redux store to disk.

Static type checking

We started rebuilding the application in React Native without types. But as the application grew, it became clear that a static type checking tool like Flow or TypeScript would help us refactor and discover bugs. The more of the codebase we covered with types, the more bugs we uncovered.

TypeScript by Microsoft and Flow by Facebook are similar tools, providing gradual static typing capabilities, a similar syntax, and widespread usage.

For React Native, Flow was a natural choice. It integrates nicely with the build tool, and most third-party libraries already provide Flow types.

EDIT FROM JANUARY 2019: After TypeScript gained more momentum, we decided to give TypeScript another change. We migrated most of our projects to TypeScript and didn’t look back. The editor support and the library support is 👌

A type checker is not a silver bullet, though. The learning curve is steep, and you find yourself fighting the type system. But we are happy that a lot of development is happening in this area. One of the most promising options for the future is Reason (also by Facebook), a type-safe language build on top of OCaml which compiles to very readable JavaScript.

Storybook as a productivity booster

Storybook is a UI development environment for your UI components. With it, you can visualize different states of your UI components and develop them interactively.

If we were to dream up a productive setup for developing UI components and screens, it would:

Work on components and screens in isolation without starting the entire application
Be able to describe and quickly switch between components and screens in different states
Support hot reloading when styles and markup changes
Have multiple simulators and cross-platform devices connected and see them all update when updating the code

We are happy to see that Storybook delivers all of these. It is a major productivity booster when developing UIs.

Automated testing

For unit and integration testing, we are using Jest—another great open source tool by Facebook. It provides a testing framework with excellent watch mode, coverage support, fairly simple mocking and swift feedback when writing tests. As it runs in Node, you mock out all native components (though it requires some setup).

We have experimented with Appium and Amazon Device Farm for cross-platform UI automation tests. But currently, we’re focusing on a solid and fast unit testing setup that helps us catch bugs and documents the expected behavior of our code.

Editor support

Everyone on the team uses their preferred editor, whether it be Visual Studio Code, Atom, or IntelliJ IDEA. To have a good and consistent development experience we ensure that all of our editors:

Show ESlint errors
Call eslint -- fix on file save (we never do manual formatting, we have Prettier for that)
Understand Flow declarations, so we get autocompletion and type errors in the editor

What’s next?

While we are quite happy with the current setup, there is room still for improvement. One thing we want to do is to have a big set of UI tests. We’re not entirely sure yet what the best option for that would be. But in general, we now have a solid foundation on which we can build more features, and we have great checks in place that make sure our code adheres to best practices and our internal style. The development cycle is also a lot faster because of the unit tests and Storybook.

There is one more thing we feel is important when converting to React Native, and that is native modules and UI components. We will cover that in our next blog post.

Co-authored by Kevin Pelgrims.

This blog post was cross-posted from https://skovhus.github.io/converting-an-app-to-react-native/ and originally posted on Leo Innovation Labs’ medium.

Type safe CSS Modules with Flow

Kenneth Skovhus — Wed, 21 Jun 2017 15:00:00 +0000

CSS Modules + Flow = type safety and editor autocompletion

I’ve been dreaming about type safety and editor autocompletion when using CSS Modules. There are a few TypeScript tools for this (see this and this), but I didn’t find any solid tools for Flow.

TL;DR I’ve made some new tools, and when trying them on a codebase I’m working on, Flow found a lot of dead code (and potential bugs) 😬

The problems

It is quite easy to misspell a class name or forget to update consumers after removing a class in a .css file. As an example, the class foo might not be defined in Button.css:

/* @flow */
import styles from './Button.css';
const Button = () => <button className={styles.foo} />;

Solutions

To teach Flow about CSS Modules file, we can create a definition file Button.css.flow containing the class names exposed by Button.css. By doing so we can get:

static type checks showing usage of non-existing classes
editor autocompletion of CSS classes (for editors supporting Flow)

To generate these .flow files I was thinking of two use cases. One using a simple CLI and another using webpack.

Solution: CLI

css-modules-flow-types-cli is a CLI that quickly generate .flow files.

Let us install it:

npm install --save-dev css-modules-flow-types-cli

# or

yarn install -D css-modules-flow-types-cli

And then just run the CLI on your source directory:

css-modules-flow-types src

I recommend using the CLI on your CI system (like Travis or Circle) to ensure that all .flow files are up-to-date before running Flow. This will catch potential styling errors before deploying.

Another use case is quick feedback loop when developing and changing CSS Modules files. The CLI includes a watch mode for this, but I myself would like to avoid having yet another tool that needs to be running while developing. As a lot of people already have webpack running, I did a small loader consuming the tokens from style-loader.

Solution: webpack loader

css-modules-flow-types-loader is a webpack loader keeping .flow files updated by consuming the tokens from style-loader. I recommend using this when developing as part of a webpack-dev-server setup. It will give a small slowdown, as the loader potentially needs write a lot of files.

To get started:

npm install --save-dev css-modules-flow-types-loader

Then update your webpack config:

{
  test: /\.css$/, // or the file format you are using
  use: [
    'style-loader',
    'css-modules-flow-types-loader', // right after style-loader
    // Other loaders like css-loader after this:
    ...
  ]
}

And then sit back and enjoy CSS Modules being type checked by Flow. 🍺

Please let me know what you think… And give a little ⭐️ over at GitHub.

This blog post was cross-posted from https://skovhus.github.io/type-safe-css-modules-with-flow/ and originally posted on hackernoon.

React PropTypes to Flow codemod

Kenneth Skovhus — Tue, 11 Apr 2017 15:00:00 +0000

I’m presenting how to automatically convert your existing codebase using React PropTypes to use more powerful Flow annotations. If you already seen why this is valuable, you can skip down to the next section. 👇🏻

Eye candy, because you deserve it (unsplash.com/@thekorus)

If React were Facebook’s gateway drug to declarative and composable UI, React PropTypes introduced a lot of people to type checking.

PropTypes documents a given component’s props and gives runtime validation of props during development. I find them really helpful, but they also have several shortcomings. Most notably we should aim to catch errors before opening our browser. Say hello to static type checking.

To statically type-check an application, you can use JavaScript extensions like Flow or TypeScript. In this blog post I’m focusing on Flow, but the same approach can be applied to TypeScript.

No doubt (to me): Static type checking (using Flow or TypeScript) at build time is vastly superior to runtime validation.

If you are new to Flow, the new documentation is really good. After setting up Flow you can now start type annotating your components state and props. For a simple component it might look like:

/* 1) Using good old PropTypes */
function Button({ message }) =>
  <button>{message}</button>;

Button.propTypes = {
  message: PropTypes.oneOfType([
    PropTypes.string,
    PropTypes.number,
    PropTypes.instanceOf(Message)
  ]).isRequired,
};

/* 2) Using Flow annotations */
type Props = {
  message: string | number | Message,
};

function Button({ message }: Props) =>
  <button>{message}</button>;

So far so good. But transitioning an existing codebase to use Flow annotations instead of PropTypes can be tedious… However this is a perfect task for a codemod!

Time to mod the code

To automate the conversion of React PropTypes to Flow annotations, we are using another tool popularized by Facebook: jscodeshift (JavaScript codemod toolkit).

If the concept of automated refactoring with codemods is new to you, I’ll promote a meetup talk I did: “Introduction to automated refactoring with JS codemods (Copenhagen.js Meetup, December 2016)”:

Enough talk. Time to get rid of the PropTypes! Turns out Billy Vong already did a lot of the hard work for us with codemod-proptypes-to-flow.

So to automatically convert all components in a folder my-components:

git clone https://github.com/billyvg/codemod-proptypes-to-flow
jscodeshift -t codemod-proptypes-to-flow/src/index.js my-components
Look no PropTypes! (Although createClass, imported and custom PropTypes validators are not supported — yet!)

I’ve successfully used this codemod for multiple projects, and it eased the transition remarkably. Now Flow starts finding errors while you write code instead of when the code is executed! 🍷 🙌🏻

Notice that there are cases where you want to keep your PropTypes (possible alongside Flow annotations):

PropTypes in library code can help consumers and document an interface
If you have no automated tests validating your flow definitions of any external resources (like an API), PropTypes can be really helpful. For this I would recommend using babel-plugin-flow-react-proptypes to add PropTypes at build time
PropTypes are still great for learning material on React (no need to burden new people with Flow/TypeScript)
PropTypes can be more flexible than what the Flow type-checker currently supports (e.g. validating a number is in a particular range)

This blog post was cross-posted from https://skovhus.github.io/react-prop-types-to-flow-codemod/

Thanks to Maciek Pekala and Mads Hartmann for reviewing this post.

Abandoning the mothership

Kenneth Skovhus — Wed, 16 Dec 2015 12:00:00 +0000

The time had come to pour into our front-end code the same poison we had given our back-end systems.

Abandoning the mothership

COMMENT FROM DECEMBER 2019: This blog post was released when Star Wars: Episode VII – The Force Awakens came out… 🎬😄

For many years, issuu’s engineering group was organized around component teams. Our late great Team Monster handled the front-end code and the rest of the teams provided APIs and back-end infrastructure.

As we scaled the engineering organization, this classical split became counterproductive. A lot of cross team coordination was required for releases. Business logic and remappings were spread across multiple layers of the application. Dependencies and ownership of features were unclear. Finally, the Monsters had way too much on their plate!

So in 2013, several of our teams transitioned from component teams to customer-centric, cross-component feature teams. One focused on the end-to-end reading experience, another team handled all publisher-related features and so on. Next was taking ownership of the entire code stack, from database to front end. Our back-end system quickly transitioned into smaller microservices written in technologies the team deemed best (OCaml, Erlang, Python, node.js, MySQL, Postgres, Redis, AMQP).

Until recently, though, the front-end code running issuu.com was still one big, monolithic codebase. It had turned into such an unmanageable beast that we were all fighting with each other to get features out to users. We identified several blockers:

With one big pile of code, team ownership and dependencies were unclear;
we stepped on other people’s toes all the time;
and as a result the rate of feature deployment decreased. (Oh yes, big-bang releases!)

Besides, we had a major technology lockdown due to a very customized build system. This alone was around 3.000 lines of code, including a home-built Browserify and a lot of magical grunt tasks. As nobody dared to touch that part of the system, we were locked to our own require system, ES5 and Ruby SASS. Also the size of the codebase made builds painfully slow.

It was time to pour into our front-end code the same poison we gave our back-end systems, splitting it all apart into maintainable chunks. Our vision was to make each feature team own, build and deploy its front-end code autonomously – and to make things that ought to be trivial actually trivial!

To the lifeboats!

We started by dividing the front-end codebase into four chunks:

shared serving infrastructure;
pages;
widgets;
and UI components.

Partitioning our frontend-end code base

The shared serving infrastructure is a thin horizontal node.js application responsible for routing and serving pages. It holds the configuration for which version of a page to be served.

A feature team delivers products consisting of the following:

pages (node.js handlers, templates and assets like compiled JavaScript/CSS);
and some widgets (shared functionality used on multiple pages, like a message hub, streams or drag-and-drop functionality to initiate a publication upload).

Our UI component package provides high-level building blocks and styling for the site. It is our shared horizontal toolkit for keeping a consistent look and not reinventing buttons, form elements, modals and colors for each page and widget.

To handle internal code dependencies, we decided to use the same tool we use for all other dependencies in JavaScript: npm. So we set up a private npm repository using the open-source Sinopia tool. When pushing code on master for one of our front-end repositories, our CI server will test and publish a new version of the package to Sinopia.

To release a change to a page, you simply bump the product version in the shared webserver package. The package.json for our webserver looks like this:

{
  "dependencies": {
    "issuu-productA": "1.0.2",
    "issuu-productB": "2.0.292",
    "issuu-productC": "1.0.x"
  }
}

To follow the principle of least astonishment, most of our top-level packages are locked to a specific version of a page, but some packages like Marketing pages will always take the newest version.

How did we migrate to this? After the initial serving infrastructure was ready, we made it up to the feature teams to take ownership of the old mothership codebase and turn it into a product/widget in the new world. Of course, the mothership still holds tight to some old products we never touch and that no team really owns. In the end, it is all about ownership.

After a few months, the old monolithic codebase is split into 45 packages. Oh, 46 now!

A new hope

Instead of the old, frustrating technology lockdown, over the last six months a lot our feature teams have been innovating on the tools they have for building products. Most of the teams use the following:

Makefiles for orchestrating everything why reinvent the wheel?;
Webpack for building assets;
Babel for allowing us to write slick ES6 code that transpiles to ES5;
a move to React and Redux;
strict linting using (ESLint)[http://eslint.org]
and more as we go. Nothing stops a team from trying out TypeScript, CSS Modules, Elm, etc.

We are now able to work on isolated, vertical features without worrying about stepping on toes. Deployment time went from 30 minutes to just a few minutes, as we are rebuilding only what have changed and not the entire codebase. We went from a few weekly deploys to multiple times daily — continuous delivery for the win!

The one constraint for a product is that it can be served by node. Heck, we are already talking about breaking the horizontal node server into smaller servers and letting the top-level routing logic reside in a proxy configuration. Ultimate autonomy to the team.

A note on sharing code, and lessons learned

We want to make fast iterations on products and sustain team autonomy. Our new system is optimized for changes in issuu’s vertical products, not for making horizontal changes easier. It highlights clearly that we still have a lot of horizontal dependencies.

In our old mothership repository, it is very easy to make a horizontal change and share a lot of code between products. One thing the code split has highlighted is that developers intuitively love to share code. Instead of copying and pasting small pieces, we would rather generalize it into shared components. This comes at a high cost: You now have yet another dependency, and dependencies slow you down. You lose autonomy and make simple things complex.

On the scale of a single product or feature, team code reuse and the principle of DRY still make sense. But one should be careful about sharing code between products as shared dependencies are cumbersome. Of course, you never want to duplicate business logic, but I would much rather duplicate simple components than share them between verticals.

Still, redundancy sucks, right? I’m going to end with a quote from the excellent post by Yossi Kreinin:

Redundancy always means duplicated efforts, and sometimes interoperability problems. But dependencies are worse. The only reasonable thing to depend on is a full-fledged, real module, not an amorphous bunch of code. You can usually look at a problem and guess quite well if its solution has good chances to become a real module, with a real owner, with a stable interface making all its users happy enough. If these chances are low, pick redundancy. And estimate those chances conservatively, too. Redundancy is bad, but dependencies can actually paralyze you. I say kill dependencies first.

This blog post was cross-posted from https://skovhus.github.io/abandoning-the-mothership/

It was originally posted at issuu’s engineering blog and on medium. This is the second part of our series “Toward a maintainable front-end”, the first part is here.