Web technology is not a constant

Web technology is not a constant, it constantly evolves and morphs, this is one thing that keeps web development interesting for developers, but at the same time can become fatiguing.

The social pressure to stay on top of the latest shiny new things, comes from influence from HN posts, Twitter and educational bootcamps and some commercial online courses that peddle what they believe to be new industry standards; for self-sustenance and profit.

When established tools receive a modicum of negativity - "It doesn't scale", "5ms render time is not as fast a shiny new JS framework", "I don't understand the conventions", it causes doubt, which leads to a grass is greener on the other side outlook, and inexperienced dev teams enter a perpetual loop of
Research -> Social influence -> Confirmation bias -> Excitement -> Learning -> Tooling -> Building -> Complexity! -> Doubt -> repeat.

Don't get me wrong, I believe every development team should question their technology choices at any stage in a products lifecycle, but it must be done with the context of the problem, understanding of what each tool aims to solve and the abilities of the developers using them, and a 8oz cup of cynicism.

This post attempts to portray my experience, I hope the takeaway resembles some sage advice about how simplifying a stack is not a step backwards or defeat.

An anecdotal example...

When I joined my current company, I read through the codebase trying to get a picture of the engine room.
I was excited at the prospect of using a lot cutting edge tech that I had used independently elsewhere, it felt fresh, "modern".

It wasn't until a year after fully investing in SPA technology that prior decisions made were questioned, with confidence.

Great investment went into building out a design system, core components, a tightly coupled test suite, storybooks, conventions and tools on top of tools to patch the holes in what had become a huge vessel of a ship that was unmaintainable by a small team with predominant skills in more familiar technologies.

While engineers were required to spend time addressing performance, finding solutions to problems created by the technology and not the users - it seemed there was no time or maybe inclination to assume the bridge and watch the compass.

The initial shine was quickly dulled, and after some time of getting over an amount of imposter syndrome I realized - there was more code than product, and not a small amount.
But, I was new to the team, I wanted to make an impression, and prove to the team and myself that my existence was justified.
Having come from an agency where most of the projects were in familiar technologies like Rails, I just assumed that this is where we product companies are at(as an industry) and to dig in and level up.
I rolled up my sleeves, put in the long hours needed to move forward.

At this point the app is a SPA and the stack looked like

• Typescript
• Mobx
• Apollo client
• React - including a design system, hooks, abstractions of abstractions
• Graphql
• React-on-rails
• Rails - with all the trimmings
• ES/Searchkick
• Graphql Ruby
• RSpec, Jest, Cypress

Each tool above independently assessed brings a great amount of value to a tech stack, but did not seem like a tech stack selected by design but built ad-hoc to solve many specific problems, some of which self-inflicted.
Performance and productivity benefits promised by each were not being realized - page load times were around 7s, page size 3.6mb, 76 requests, and a google performance grade of 69, pages were not responsive, accessibility wasn't attainable and development times were beyond sustainable of a team of 5.

Building a new feature had an overwhelming amount of cognitive overhead just to get the tooling work done to start building the feature.

A simplified example:

• Create the Model and business logic - unit test
• Create a GraphQL ruby endpoint, maybe new types, generate schema - unit test
• Create query/mutation JS files, deal with Apollo fragment abstractions - unit test
• Wrangle Typescript
• Create a React hook - jest test
• Hook up Apollo to the query/mutation - jest test
• Wrangle Typescript
• Create a page, maybe a react route, maybe a new component, deal with the design system abstraction.
• Create a form - what is typically a few lines of html, now requires intimate understanding of Formik, Yup and the bespoke component structure.
• Wrangle Typescript
• End to end testing with Cypress using JS
• Determine state management, caching, data binding urgh.
• Add searchkick index configuration

The keen eye might notice a pattern emerging and a prime example of where Typescript was used to solve a specific problem, and while Typescript and types have a massive amount of power and value, it felt unnecessary for app and team of our size.
We had a strong typed frontend, an adapter to interface GraphQL with ActiveRecord in Ruby a dynamically/duck typed language and an abstract presentation layer in between for... reasons unknown.

Also worth noting was the amount of weight the frontend architecture has over the backend, creating well formed server side domain models and clean architecture didnt get the time it deserved due to the mass of what lay ahead to expose any of it to the user.

Eventually, frustration outpaced my ego, I felt we had gone overboard and drowning, and it was time to speak up, and try to invoke change. There's more nautical analogies folks, stay tuned!

It meant making some outlandish suggestions to the stack, which would change the engineering focus and incidentally a change in roles for some.
It didn't take much to bring out sentiments from other members of the team of feeling held back by the technology, solving tooling problems and not product problems.
Fortunately, the team agreed to hold the line and assess where these emotions originate.

The team went through weeks of experimentation of front end frameworks like Antd, simplifying our React components and hooks, building abstractions (ultimately with 30k loc), finding backend solutions that interface with GraphQL smoother than Graphql-ruby such as Hasura, and simplifying the SPA stack.

It was a necessary evil, even if it meant a perceived step back both in effort and modernization, even if it meant rocking the boat, abandoning the teams hard efforts.
Even if it meant potentially losing talent that had invested a lot into the product on many fronts, in fact this became an unfortunate reality when an engineer left due to complexity.
After several weeks, frustrations grew that there was not lighthouse to guide our way.

Iceberg ahead

Understandably, there's attachment to work that exists, to tech you are more comfortable and familiar with, and a huge investment of time and emotion, and job security, which can bias any human into making self-sustaining decisions, as engineers we tried to be objective about such things, but as humans, emotion always has a big influence on our outcome.

While some engineers proposed jumping ship, others with the Curse of knowledge tossed more lifejackets.
Meetings, meetings and more meetings were had, arguments ensued, respect was faltering, it got heated.
There was a significant sunk cost fallacy holding us back, but we are now well into the doubt phase and turning around now could mean we sink before realizing the products potential.

Abandon ship

After each developer weighed in with their research and opinion, there was not a consensus, but we had a majority vote and CEO buy in to move forward with a new stack. This meant some sacrifices, some roles were changed and some developers were forced to take on skills not part of their initial job description.

Technical Requirements

On being reactive - A big question was how do we maintain fluid and reactive UX to the user with a reactive and declarative programming style, which was inherently afforded by React.

On being performant - With an SPA, HTML/JS are shipped ahead of the data, the data is asynchronously served by the means of Apollo and GraphQL, this mean potential for fast initial page load and precise DOM manipulation when state changes.

On time to retool - A big consideration is that we have huge design system, a lot of which needs to be recreated.

On community support - Inarguably, React has huge community adoption and support, but it's not the only one. A consideration is picking a tool that suites our needs while being able to quickly find solved problems.

On developer happiness - We've established that our stack is making us unhappy, it can only get better... or can it. Not everything should be seen with rose tinted glasses. When coming from an unhappy place anything can seem like an improvement, but let's not allow that to bias our decisions.

Full Steam ahead

During our experimentations a colleague Michael and I came across Stimulus and Stimulus Reflex.
We we're amazed at the simplicity, the community excitement, but there wasn't a great deal of prior art to compare it to.
I promptly arranged a brief chat with Nathan Hopkins(the Original Author of Stimulus Reflex), and Eric Berry co-founders of the CodeFund, who were kind enough to give me time to ask questions about CodeFund, Stimulus Reflex and how it fit in their product.

stimulus
Stimulus
An underrated, and relatively tiny JS library from Basecamp with bite size API, with enough power to cater for many event driven UI concerns.
We're still becoming acquainted with Stimulus, and its best practices, we are applying many techniques we learnt from our React system with Mixins, developing custom Stimulus Reflex interfaces and clean abstractions.

Stimulus Reflex

Provides a framework on top of Stimulus, and CableReady which directly exposes server side functionality over Websockets with Ruby classes to provide an event driven imperative update cycle.
Native languages, small footprint, little convention or opinion and no boilerplate or supporting frameworks required with the exception of Rails which was already a given choice considering the teams experience.

Testing Reflexes is a pain point both unit tests and integration, the nature of the websocket connection and doesn't fit well with Cypress's (arguably recently dated) opinions on how pages are rendered.
We have made some breakthroughs using stimulus_reflex_testing, and contributing to Stimulus Reflex with solutions for supporting Cypress adding some new events emitted by the initialization of SR.
We're finding new ways to optimize pages with selector morphs and selectively using its powerful data-reflex-morphs to self-reference anchor points in which morphs affect the DOM re-rendering

View Components
While I feel View Components give us some of the modularity React provides I feel there's still a way to go before it reaches it's compositional abilities.
That being said when well utilized and paired with Stimulus, Stimulus Reflex and Tailwind you have a powerful stack that can get you most of the way there (feature parity relative to what we have with a current stack).
From the very first components, I architected our components to lean on inheritance, and a uniform API using dry-initializer.
A step further - I built a prop checking mixin using dry-schema to provide functionality similar to that of PropTypes and defaultProps
While there's a level of complexity introduced here, it's still just Ruby and erb, native languages to the majority of the team.
We can conditionally toggle certain UI elements thanks to Stimulus Reflex, create powerful interfaces for external libraries and with Dry-*, provide an API that we're proud to call our own.
A custom view component helper helps us avoid the verbosity of render DropdownMenuComponent.new(param) {|yielded| ...} into a distilled component :dropdown_menu, param, {|yielded| ...}. It's a minute change, but small changes make for improved developer happiness over time.

Tailwind
Love it or hate it; given the above tools its a no-brainer that Tailwind paired with components yield a huge amount of modularity and isolation.
The Utility driven approach allows us to style components with self-contained concerns similar to css-in-js, CSS modules.
Our Design Engineer Matthew proposed Tailwind as a competitor to an out the box design system like Bootstrap.
There were tradeoffs, Tailwind doesn't provide any ready to use components that Bootstrap does, or included JS to make elements interactive. Batteries are definitely not included.
We decided that the nature of our bespoke designs determined that many of the benefits of Bootstrap would not apply to us, and the flexibility of Tailwind was a perfect fit.

Personally, I love being able to memorize the utilities and quickly shape views up into a reasonable state and tweak the finer details later. It's provides much of the instant gratification from seeing a feature take shape early for design review.

We decided over-the-wire html could maintain a lot of our UX requirements, while leaning on a strongest skills and tooling for productivity.
These choices tick many of the boxes, it's not all rainbows and unicorns, but thats software engineering, it's about solving these problems.

While the community for StimulusReflex Join the Discord is relatively small, it's growing at a healthy pace, fuelled by the desire for simplification, and often frustration from fatigue. I feel it's open and un-opinionated enough that we can actually contribute and have greater control over our own destiny.

I hold a great amount of gratitude for the authors and maintainers of these tools, who are surprisingly small in numbers that take time to support adopters and answer questions to sometimes mundane questions.
In particular Nate Hopkins, Leastbad, julianrubisch, Joel Hawksley have all personally responded to cry's of anxiety and uncertainty with compassionate support.

This post might seem contradictory, I encourage looking for better solutions while avoiding shiny new things syndrome, I should point out, this is the stack that works for us right now, our team size and abilities, our product goals and desired developer happiness.

It may not be considered "modern", which I feel is a well abused word in the tech industry, but I think its ahead of the curve in terms of solving real problems for developers that want to focus on solving real world problems and not self gratifying tooling problems.

I believe React is now a Golden hammer for some people, who have spent significant effort investing in it and its supporting ecosystem, so much so that other often more suitable solutions are overlooked.
While that opinion might not suite your confirmation bias (maybe it does if you read this far!), it's just that - an opinion, but know that its one that aligns with much of the reason why many developers are looking to move away into smaller more focused libraries like Vue, Svelte, Stimulus, Alpine that are more appropriate for their use case.

Hopefully we have broken the cycle...for now.

research -> social influence -> confirmation bias -> excitement -> learning -> tooling -> building -> Profit!

In retrospect

Developers are not completely to blame, when given limited instruction and an unknown future of a product roadmap imagination wonders. Areas of which, that we've since made huge strides in improving alongside engineering.
Stability of a development team and its chosen technology stack isn't limited to the tools themselves, the product direction, team size and skillset, community support and confidence, buy in from executives all play a part in what we use day to day. It's important to consider the whole picture when deciding to make decisions that affect the makeup of the stack.
This can apply from everything - from how form validation is handled right up to what frontend library handles state.

Had we decided not to take a different course with a ship built for the Drake passage, we might never have reached our destination.

If that's too much analogy - There's a point in which its tempting to build a stack ready for anything, using the best of the best because of the safety that by doing so you won't have to worry about it in the future. But reducing tooling, simplifying the tech allows us to stay agile for the ever changing future.

Often, the opposite is true and developers make decisions without consideration of the team size, abilities and product roadmap.
Build smaller ships, improve incrementally; as the story evolves. Evaluate, 1 step back is worth 2 steps forward.
I encourage you, that when things get challenging, stay the course, at least until you are confident that going into a doubt phase is justified and valuable.

• Scott