DEV Community: Tommy Brunn

Is commercial support a viable way to open-source sustainability?

Tommy Brunn — Tue, 06 Oct 2020 13:34:02 +0000

In the middle of the COVID-19 pandemic, I quit my job as a lead engineer at a successful startup and dedicated myself to open-source. As a generally risk-aware person, this was certainly a move that made me uncomfortable, but coming off of a successful career at Europe's largest fintech unicorn I felt like I had the leeway to take some risks.

For the past 3 or so years, I have been developing KafkaJS, a dependency-free, pure Javascript Apache Kafka client for NodeJS. Over that time, I have seen it grow from a small project mostly internal to the company we were working at, to now powering hundreds of businesses and helping thousands of developers.

Tommy Brunn

@tommybrunn

Really cool to see that KafkaJS, the project I've been working on with @tulios for the past 3 years, has finally passed kafka-node to become the #1 most downloaded #kafka client for #nodejs 🎉

kafka.js.org

11:09 AM - 06 Oct 2020

1 2

Scaling open-source

Like most open-source projects, one challenge that we always had was finding time to not only develop it further with new features or improving existing ones and handling valuable community contributions, but also providing support to our growing community of users.

At the time of writing, we have approximately 500 developers in our Slack community and there's no shortage of issues being opened on Github that require triage. Over the years, I've been in numerous Slack conversations and video calls with developers debugging whatever issues they are having. This has been enormously helpful in understanding where the rough edges are and where people tend to get tripped up when building NodeJS applications with Kafka. On a personal level, it has also been very satisfying to get to see how KafkaJS is helping power systems in everything from commodities trading to medtech to commercial retail.

However, there are only so many hours in a week that I can spend. Although open-source clearly brings value to thousands of companies, it's rare for any of that value to find its way back to the people that create it, and KafkaJS is no exception in that regard. In order to further scale the project and keep a healthy community going, there needs to be a mechanism in place to fund maintenance and development.

While Github sponsors and other fundraising tools are laudable efforts, in reality they don't provide any meaningful income except to a rare few extremely widely used projects. Even hugely impactful developers like TJ Holowaychuk are barely making anything.

Commercial support

A time-tested way of funding open-source development is providing commercial support. It allows businesses to accelerate development while simultaneously funding the development of the software they rely on - a win-win.

While this is typically used to fund larger projects, I don't see any inherent reason why it couldn't work for a medium-sized project such as ours. There are hundreds of businesses using it that could surely benefit from working with one of the few people that know it inside and out - I know because I have already done it on a volunteer basis for years.

So, given that I'm finding myself willing to take on some risk, I'm giving it a shot!

Whether you are using KafkaJS or are thinking of leveraging Kafka in NodeJS, get in touch and save yourself time and effort by working with an expert. Whether you are just starting out and want to validate your architectural choices, or you are already established and want to solve some gnarly problems together, I'm sure that my experience building secure, performant and maintainable systems in NodeJS can be of help to you!

If you aren't in need of support, but just want to support my work anyway, any contribution via Github Sponsors is greatly appreciated.

No Dependency is the Best Dependency

Tommy Brunn — Wed, 15 Jul 2020 15:10:26 +0000

Over the past few years, I have been interviewing hundreds of software developers for various positions in tech companies. One question that tends to come up rather frequently is how you pick which package you should depend on. Given that NPM alone currently has around 1 400 000 public packages, chances are that whatever your problem, there are multiple packages that all claim to solve it - leaving you to figure out the best way forward.

Let's dive into how seemingly most people claim to choose packages, then I will share my alternatives and why any of this matters at all.

Github ⭐️ stars and other vanity metrics

Taking the expression "100 Billion Flies Can't Be Wrong" to heart, by far the most common answer I've gotten is that they simply look at the number of stars the repository has on Github.

Stars, for those who are unaware, is the version control platform equivalent of a Facebook "like". Clicking the ⭐️ Star button on a repository adds it to your list of starred repository and potentially shares that activity with your followers.

The theory goes that if other people have deemed this package worthy of a star, it must be good enough for you as well. The benefit of this approach is naturally that it takes very little effort to simply compare a few numbers and pick the highest.

There are of course other similar metrics that people use. Npm Trends primarily uses number of downloads to compare package popularity over time.

The downside is that these numbers really says nothing about the quality of this package, whether it is still actively maintained (who goes back and removes stars?) or if it has bugs and edge cases that might not affect the majority but could affect you.

In short, these types of metrics aim to measure the popularity of a package.

Blogs, articles and tweets 🐦

Another common criteria that I have heard is to look on Twitter or to read blogs to see what other people are recommending.

This is again a form of external validation of a package's popularity, but this time rather than relying on the wisdom of the crowd, we are choosing to rely on "expert opinion".

At a surface level, this may perhaps sound fine. We expect that these experts on the internet have done a proper evaluation so that we don't have to. I do the same when choosing which barbecue to buy - I don't try out every one of them, I look at reviews.

The issue here is that as evidenced by this very article, anyone can put their opinion on the internet - there's no peer review to ensure that my methodology was sound or that I don't have a vested interest in any particular product.

However, even when an endorsement is done in good faith and a thorough evaluation was done and the facts properly presented, that person simply does not know the context of your project. Just because something was a good fit for them, that doesn't necessarily mean that it's a good fit for you.

Activity metrics - the BPM of the maintainer

The MBAs of the development world shun popularity metrics like downloads and stars, and instead looks at "activity metrics", such as number of pull requests and issues, number of commits over time, number of releases and such.

At the time of writing, Openbase.io was just released to the public, which aims to use these metrics to compare how well-maintained different packages are. For example, it can tell you that redis (the NodeJS client) has on average 1 commit every 6 days, that the average PR takes 20 days to close and that the average issue stays open for 8 hours.

These types of metrics all serve to measure activity. No one wants to deal with a project where the maintainers are impossible to reach or where bugs stay open for years.

However, without context, these metrics are not terribly informative.

Some packages are simply done and need no further activity. Simple packages that perform their task well and don't need to be kept up to date with moving targets very rarely need any maintenance - so why would a lack of commits be a negative?
Depending on the target audience of your package, the issues you receive will be of varying quality. Even the most dedicated maintainer will struggle to close issues that require extended back-and-forths with unresponsive reporters that don't provide repro cases or failing tests.
A simple way for a package author to improve their metrics is to simply close issues as "wontfix" as soon as possible. Or to integrate services like Dependabot, which keeps your dependencies up to date by creating pull requests that are very easily merged.
Constant change isn't necessarily a good thing. For complex projects, landing a significant change may require careful thought and discussion over many months. Rushing to release something that's not quite thought through may just introduce bugs or churn as public APIs change.

As an open-source maintainer, this sort of evaluation criteria, while not without its merits, also has a tinge of entitlement. The vast majority of small (and not so small) open source projects are maintained without compensation by a single author. Expecting that person to be on call and to resolve every issue promptly is an awful attitude that is shockingly common. Later we will come back to what approach we can take to mitigate this risk in a responsible way.

Determining quality with this one weird trick...

All of the criteria we have looked at above -popularity and activity- have all been about looking at abstractions or high-level metrics to indicate something else - quality.

Determining quality requires you to do something that seemingly very few people actually resort to doing - reading code.

Depending on the size and complexity of the project, as well as your familiarity with the subject, you may or may not be able to judge the technical merit of every single piece of software that you interact with. For example, I may be able to make use of Tensorflow to detect faces in an image, but I couldn't without some serious time investment compare the quality of Tensorflow to other competing solutions.

However, I think people overestimate how complex most software really is. I bet your average developer could come to an informed decision on the quality of for example express or pg in just a few hours if they weren't so reluctant to even try.

Seriously, this is my secret. Try it next time. Actually sit down and read through the code and understand at least at a high level what it's doing and why. If you don't understand why it's doing something, read up on the subject until you get a decent understanding. Pretty soon you will notice similarities and differences between projects, understand what tradeoffs they made and how easy it will be to evolve the project.

Be aware, however, that you are looking at a snapshot, not necessarily trajectory. When I first read through pg I saw that the protocol code was intermingled within all the other code, and thought that this would make it really hard for the project to evolve. If there was a competing postgres driver at the time, I would at least have had a serious look at that as an alternative. However, nowadays the protocol has been refactored and separated out into a separate package and the remaining codebase looks a lot better. If I had relied only on the quality metric at the time, I would most likely have made a poor choice.

Picking a dependency, the hard way

When you choose to take on a new dependency in a project, you are marrying that dependency until death do you part, or you go through a messy and expensive divorce. If the dependency is central enough to your project (ORM, server or DI framework), replacing it may be almost equivalent to a full rewrite. Especially if the project has not been very well structured to isolate different layers, as is often the case. Therefore, it makes sense to invest time in understanding your dependencies and learning about them before it's too late.

Does the dependency have a sustainable development model (sponsorships, part-/full-time developer, commercial value added services or components)?
Do I feel confident that if the maintainer stops maintaining the dependency, I could maintain it myself?
Does an entity with resources have a vested interest in the future of the dependency, and could I or my company be that entity?
Does my use-case fit with the goals of the dependency?
If there are significant bugs that affect me, do I feel confident that I could fix them myself?

The above points are primarily concerned with sustainability. As we saw from the pg example, a project being sustainable and having a positive trajectory is more important in the long run than quality at any particular snapshot in time. Or in the words of John Ousterhout:

A little bit of slope makes up for a lot of y-intercept

Promoting sustainability

Sustainable projects don't happen without either corporate or community support. If a project provides value for you or your company, and you want that project to continue to be supported, you have to contribute with your support.

Contribute to open-source projects on OpenCollective
Become a Github Sponsor of projects you depend on (shill alert: Sponsor my work on KafkaJS)

You can also contribute your time by providing issue triage and pull requests to address bugs, however, be aware that code contributions also take the maintainer's time, not just when reviewing your contribution, but also because they have to continue to maintain your code for the lifetime of the project. Contributing financially to allow the maintainers to spend more of their time on the project is in many cases more impactful for the long-term health of the project.

How about no dependency?

So far we have learned that tying the fate of your project to a dependency is risky business, and mitigating that risk by careful evaluation not just of popularity and activity, but also of quality and sustainability, can be a lot of work. And even when you do put in the work, there is always the risk that you make the wrong choice anyway.

With this in mind, I propose that we shift our mindset from "which dependencies should I choose" to "should I use a dependency".

One of the core design choices we made when building KafkaJS was that we would have little to no runtime dependencies. We chose that because we knew that KafkaJS would be a core part of our users' applications, and as such any dependency that we take on, our users would also have to take on as a transitive dependency. We also knew that these applications would live for quite a long time and power important business processes, so if we chose to take on a dependency that would no longer be maintained, it would be up to us to make sure that our users would not be exposed to new security issues.

As such, we adopted our "no dependencies" approach - with one small exception, long.js. That exception we chose to allow for because the scope of the dependency was small, we knew that native 64-bit integers for NodeJS was on the horizon, at which point we could get rid of it, and we were confident that we could maintain the library or our own fork ourselves if need be.

Shifting the mindset from "which dependency should I choose" to "should I use a dependency" meant that we don't need to worry about any of the issues listed above. The downside is of course that we have to maintain more code ourselves - but a lot of the time the code that we actually need is so much less than what dependencies provide. Since we know the context of our project, we can implement just the bit that we need, not cater for the needs of the many.

Conclusion

In short, taking on a dependency has a bigger impact than most people tend to consider. If you are working on a long-term project with actual consequences in case of failure, this is a responsibility you should take seriously. Evaluate your options carefully and weigh the risk vs reward for each dependency.

And ⭐️s don't matter.

Cover image from Russ Cox's excellent article on the same subject

(Sort of) fixing autofocus in iOS Safari

Tommy Brunn — Tue, 22 Nov 2016 18:05:06 +0000

This article was originally posted on Medium in 2016. It may be out of date at this point.

(Sort of) Fixing autofocus in iOS Safari

One of my colleagues is transitioning to the front-end team that I used to be a part of. To prepare him mentally for his journey into front-end development, I’ve been sending him a newsletter I call Front-End Hack of the Day. I’m posting them to Medium now for the world to enjoy

Imagine that you are building a form where you would like to help the user out by automatically focusing on the first input field.

<form>
    <input type="email" name="email" placeholder="foo@example.com" autofocus />
    <input type="password" name="password" />
</form>

You fire it up and try it out, and it works great. Ship it!

Sometime later, someone comes to you and says that it’s not working in iOS Safari. Off you go to caniuse.com and see that it is indeed not supported in that browser. Oh well, no big deal, we can fix that with a little bit of Javascript.

document.addEventListener('DOMContentLoaded', () => {
    Array.prototype.slice.call(document.querySelectorAll('input'))
        .filter((el) => el.hasAttribute('autofocus'))[0]
        .focus()
})

To your great surprise, you discover that this is not working either!

Turns out, Apple really doesn’t want you to focus input fields that the user hasn’t tapped on. Not only is the autofocus attribute not supported, but you have in fact made the situation worse!

What I feel like doing any time I’m developing for Safari

See, even manually calling focus on the element won’t work until the user has interacted with the page. If the input is inside an iframe and you try to call focus before the user has interacted, the keyboard opens, the input does not get focus, and typing on the keyboard does absolutely nothing. As an added bonus, if the viewport scrolled at all, the useless, blinking cursor will be displayed somewhere outside the input.

I haven’t been able to find any official resource explaining this decision, but I have to assume that it’s because focusing a field pops up the keyboard, which can be annoying if you didn’t have any intention of filling out the field.

Faking focus

We can’t fully emulate the autofocus behavior, but we can get pretty close.

Focusing a field does three things:

Set focus styles
Scroll the page so the field is somewhere in the middle of the viewport
Open the keyboard

3 is the only thing that Apple has something against, but the other two can be implemented rather easily. I’m going to show you a very specific example, but for your own sanity, I suggest you come up with ways to abstract over this so that you don’t need to worry about whether you’re really focusing the field or if you’re just faking it.

Check out @klarna/ui on Github to see how we are doing it (programmaticFocus.js).

The first part is simple, to set the focus styles, just add a class with the same styling:

input:focus,
input.has-focus {
    border: green;
    color: black:
}

Scrolling the input into view is surprisingly simple, thanks to Element.scrollIntoView.

If we put it all together, we get something like:

const isIos = () => !!window.navigator.userAgent.match(/iPad|iPhone/i)

const hasInteracted = (() => {
    let interacted = false

    const onTouchStart = {
        interacted = true
        document.removeEventListener(onTouchStart)
    }
    document.addEventListener('touchstart', 'onTouchStart')

    return () => interacted
})()

const FOCUS_TYPES = {
    REAL: 'real',
    FAKE: 'fake'
}

const getFocusType = () => (hasInteracted() || !isIos())
    ? FOCUS_TYPES.REAL
    : FOCUS_TYPES.FAKE

const focus = (input) => {
    switch getFocusType() {
        case FOCUS_TYPES.REAL:
            return input.focus()
        case FOCUS_TYPES.FAKE:
            input.classList.add('has-focus')
            const onBlur = (input) => {
                input.classList.remove('has-focus')
                document.removeEventListener(onBlur)
            }
            input.addEventListener('blur', onBlur)
            input.scrollIntoView()
    }
}

document.addEventListener('DOMContentLoaded', () => {
    const autofocusedInput = Array.prototype.slice.call(
            document.querySelectorAll('input')
        ).filter((el) => el.hasAttribute('autofocus'))[0]

    focus(autofocusedInput)
})

What we end up with is a field that looks like it has focus and that is centered in the viewport. The keyboard won’t pop up, but that’s as close as we can get.

Hopefully this has been useful to you. The intention of these posts is not to show you some groundbreaking new front-end technique, but just to share some of the hacks my colleagues and I have had to implement over the years to deal with various browser quirks.

This was written mostly from memory, with some input from Xavier Via, so there may be some inaccuracies. Please leave a comment if I missed something.

Detecting autofilled fields in Javascript

Tommy Brunn — Mon, 21 Nov 2016 11:46:47 +0000

This article was originally posted on Medium in 2016. It may be out of date at this point.

When the browser autofills a form field, it adds some styling to highlight which fields it has edited. In Chrome, as you can see above, it adds a lovely yellow background color.

In principle, I think this is a good idea, as it shows the user which fields they should pay attention to in order to see if everything was correctly filled out. However, it will most likely clash with the rest of your sites’ styling, so it would be nice if we could have some more control over it.

input:-webkit-autofill {
    // Much nicer
    background-color: bisque;
}

You can add styles by using the vendor prefixed pseudo-class -webkit-autofill, but if you need to run some Javascript when a field gets autofilled, it gets a lot trickier.

In @klarna/ui, Klarna’s open-source UI component toolkit, we use the floating label pattern. The label is initially displayed like a placeholder, and when you start typing it transitions to a small label at the top of the field.

This is done by listening for a change event and adding the class is-filled to the field, which then applies the appropriate zooming and positioning.

However, no event whatsoever is fired when a field is autofilled, so there’s no way to know when to add the class.

And this is where the hack comes in…

Change event through animation

We can’t know when the field value changes through autofill, but that’s not the only thing that happens when it gets autofilled — some styles are also applied!

Unfortunately, we can’t really listen for a style change, but we can listen for the start of an animation, and we can start an animation in response to autofilling using the previously mentioned -webkit-autofill pseudo-class.

@keyframes onAutoFillStart { from {/\*\*/} to {/\*\*/}}
@keyframes onAutoFillCancel { from {/\*\*/} to {/\*\*/}}

input:-webkit-autofill {
    // Expose a hook for JavaScript when autofill is shown
    // JavaScript can capture 'animationstart' events
    animation-name: onAutoFillStart;

    // Make the background color become yellow really slowly
    transition: background-color 50000s ease-in-out 0s;
}

input:not(:-webkit-autofill) {
    // Expose a hook for JS onAutoFillCancel
    // JavaScript can capture 'animationstart' events
    animation-name: onAutoFillCancel;
}

To clarify what’s going on here, we are running the onAutoFillStart animation when the psuedo class -webkit-autofill is active, which it will be when Chrome autofills the input and changes the background color.

Now we can listen for the start of that animation from within our Javascript.

const AUTOFILLED = 'is-autofilled'

const onAutoFillStart = (el) => el.classList.add(AUTOFILLED)

const onAutoFillCancel = (el) => el.classList.remove(AUTOFILLED)

const onAnimationStart = ({ target, animationName }) => {
    switch (animationName) {
        case 'onAutoFillStart':
            return onAutoFillStart(target)
        case 'onAutoFillCancel':
            return onAutoFillCancel(target)
    }
}

document.querySelector('input').addEventListener('animationstart', onAnimationStart, false)

Now whenever either the onAutoFillStart or onAutoFillCancel animations start playing, our corresponding functions will run, where we will add or remove the is-autofilled class, or do whatever else we want to do.

Perfectly aligned labels with orange color to show that the field has been autofilled

I hope this hack will be useful to you. If you want to take a peek at how this is used within @klarna/ui, you can find the implementation of the Field on Github.