DEV Community: Horia Varlan

From Code Lone Wolf to Team Player: Confessions of a Former Pair Programming Skeptic (with a Side of AI)

Horia Varlan — Thu, 31 Oct 2024 08:25:27 +0000

A few years ago, within a couple of months, I experienced the two extremes of pair programming. On one hand, I still had a fresh memory of a highly collaborative environment which encouraged engaging in mixed coding and ideation sessions that often led to making rapid headway. It had a healthy focus on results without an obsession for rigid processes and oversight. On the other hand, I was suddenly facing a substantially different one, dominated by performative actions, and where constant negotiation and appeasement of existing hierarchies were the daily norm.

It's no wonder that for the following year, I dreaded the experience, finding it intrusive, highly uncomfortable, and unnecessary. Making matters worse, the social contract of the latter space promoted mutual fear as a prerequisite for success. It had a somewhat dismissive view of direct collaboration, often regarding it as wasteful, best reserved for cases indicating individual failure.

Such situations make it intriguing to explore some of the factors that influence company cultures' stances on pair programming along with the potential benefits they overlook when dismissing it based on a superficial, one-size-fits-all assessment.
Reluctance and Resistance

In highly structured, metric-driven environments which lack extensive experience with this, pushback is likely coming from both the organization and individuals. For contexts where numbers are the be-all and end-all, this change requires a leap of faith, potentially obscuring valuable but less visible work and causing fear that a significant portion of impact may become invisible. If things fail, somebody will have to take responsibility and there aren’t normally many volunteers brave enough to do that. This fear, while valid, can be dangerous.

Companies often view intangible aspects as liabilities, regardless of their intuitive sense. Lack of visibility is linked to wasted resources and people exploiting the system, while also being perceived as a missed opportunity for additional cost-cutting. In itself this is potentially indicative of larger issues like unreliable recruitment and misguided culture, which might warrant a separate discussion.

However, distrust within organizations forms at the confluence of individual fears. After all, no matter how great the “family”, when things go south, you’re on your own, kid. That’s even more so, after a couple of years experiencing lay-off season every other quarter. So it’s unsurprising to notice pushback when you combine the fear of ridicule and retaliation with the perception that sharing your knowledge is giving away the little competitive advantage you might have left. A mix made in heaven, not very conducive to building a robust, innovation friendly culture just ripe for creating that highly elusive stakeholder value. More is always more.

Now let’s take a brief look at what’s to gain. This is one of those activities that happens to be both intrinsically personal yet collectively beneficial, but you’re likely to get the best of it if you understand and respect it for what it is. As with everything human related, there’s a spectrum of perspectives, which means you’ll have to trust the process first.

In return, when done for the right reasons, you get to enhance your team dynamics by unlocking latent interaction patterns that can support your internal processes from the very early stages of recruiting up to delivery and maintenance. It can have a tremendous effect during onboarding by allowing existing members to get a fresh angle on their work, should they care to hear it, while simultaneously allow new members to fast track their introduction to the new system and what it’s like to produce quality work. A stark contrast to going alone through, sometimes, vast amounts of occasionally outdated documentation.

The natural direct consequence is that it also helps build trust and familiarity faster, which, in turn, can have a positive impact in terms of work life balance. For day to day activities, it helps to provide a more even spread of information, avoid blind spots and reduces the likelihood of information loss when team members depart for greener pastures. It can also help speed up things by averting costly refactoring sessions due to undetected suboptimal early design choices or solutions that only emerge during code reviews.

That doesn’t come without risks, but these are more a consequence of how you use this tool rather than its inherent flaws. First of all, this is a double edge sword in the sense that it requires a certain level of openness. As much as it is an opportunity for honest conversations, it can also bring light on less palatable aspects of working together. Then again, those pain points are still going to surface, just at a slower pace. Without a proper set of healthy guidelines and support it can also easily backfire, negating the intended outcomes.

If the internal culture highly prioritizes conformity and social harmony, then it’s likely to encourage strong consensus which in turn negates some of the benefits of having opposing views and exploring ideas through constructive friendly debate. There’s also an equally high risk of losing focus and developing pair programming fatigue when its being overused, especially in high pressure projects with tight deadlines and high feature churn.

Making it work

What led me to reconsider my stance and embrace pair programming as a valuable development approach even in spaces which discourage it? This transformation certainly didn’t happen overnight. In fact, the whole transition unfolded over the course of a couple of years. A new team and a gradual release from imposter syndrome significantly influenced my openness to engage more deeply in this collaborative work style.

It took me a few sessions to understand that this environment was genuinely safe, a results-oriented culture that valued each member's contributions rather than watching out for the things they lacked and centering mostly on this part of the process. Once again, things were starting to feel right.

Let’s see what was different. First of all psychological safety as part of a blameless culture was more than just a simple click bait headline. A reliable indicator of its enforcement is observing behavior during substantive disagreements. And there were a few big ones. This is especially true when power imbalances do not enforce superficial politeness. Additionally, the manner of feedback is crucial. While what is addressed is important, how and where it is delivered is more indicative of a how healthy a culture communicates. This aligns with effective conflict resolution skills that avoid hostility and aggression.

Notwithstanding these factors, it would be remiss not to acknowledge the role of luck in successful pair programming. While one can cultivate sociability and agreeableness, personal chemistry is largely a matter of chance. If it’s there, it can really make things so much better.

How has this played out in practice? While every session is different, there are a few things that stand out. Firstly, it's essential to consider the roles of seniority and skill sets within pair programming and their impact on particular scenarios. Pairing developers of similar seniority levels serves different objectives than pairing those with varying levels of experience. Similarly, tenure matters as it relates to the depth of knowledge each developer brings.

It can also have an interesting side effect on managing imposter syndrome by showcasing a more honest, unfiltered take on each one’s abilities and where they fall short. After all, it’s always mindful to think twice before labelling anyone as either a 10x developer or a rookie. Instead, you’re more likely to simply experience differences in masking professional inadequacies and insecurities.

Selecting skill overlaps or complementary skill sets benefits in distinct ways. For instance, varying seniority or tenure facilitates knowledge transfer and redundancy. Conversely, developers with a diverse combined skill set can tackle complex issues requiring broad contextual awareness. Moreover, this can serve as trial runs when choosing long-term pairing partners.

There are also several common variations in pair programming dynamics, which can sometimes be influenced by the differences we've discussed. At any given moment, one developer may take the lead, while the other provides more passive, thoughtful input. This ranges from one developer shadowing and learning to providing context-aware input.

Development primarily involves thinking, designing, and decision-making rather than merely translating those concepts into a language the computer understands. With a focus on functionality rather than form, a second set of eyes can prevent time-consuming pitfalls such as analysis paralysis, nitpicking and excessive focus on coding details that add little to the conversation. Additionally, splitting research tasks between the pair can avoid downtime.

Async pair programming is also a viable option that can be highly effective when used across significant time zone differences. If your daily synchronization window is limited to 1-2 hours, you may encounter situations where decision-making drags on for days or even weeks. Through collaborative efforts, each pair member can cover gaps, enabling the other to catch up and resume work. This approach also offers the advantage of identifying issues early, reducing lengthy code review sessions, and minimizing upfront design work, particularly for proof of concepts.

Rotational pair programming offers its own benefits, which vary based on scope and area size. For an internal team, it increases resilience during crises by mitigating blind spots, thereby reducing the stress commonly associated with on-call duties. It also serves as a knowledge dissemination mechanism, distributing expertise so that most, if not all, team members can provide well-informed input, enhancing the team's appeal to external stakeholders.

When applied to a broader area, it can serve various purposes. It can help familiarize the team with external processes they rely upon or redistribute internal talent to areas within the organization where they may excel. Shadowing can also prevent expensive internal transfers by providing potential candidates with insights into the everyday tasks they would be expected to handle. If it’s not a good fit, it avoids setting an entire sequence of HR processes in motion.
As with all things in life, making well-considered decisions and applying them in moderation are key to success. Don't simply adopt this solely because it's trendy and everyone else is doing it, or view it as the new go-to pathway towards becoming SuperAgile.

It's beneficial to establish clear pairing objectives and outcomes from the start. However, it's crucial to recognize that some intangible effects may not be immediately noticeable or measurable: increased morale, burnout resistance, and enhanced problem-solving, to name just a few. Encouraging structured feedback and integrating it into retrospectives can enhance team awareness. Indirectly, you can quantify its impact by observing its effects on long-term code maintainability, code review, and quality control.

Unlock the Power of Different Perspectives

I finally discovered the true value of pair programming and its effectiveness in a recent, long-term project filled with uncertainty. Through fortunate circumstances we settled on a process that greatly improved efficiency when working collaboratively. Several substantial task overlaps made pairing seem like a natural choice, in part due to how impractical it would have been for multiple people to cover the same, vast amount of prerequisite material within a reasonable timeframe. On top of that, there were quite a few situations when subtle bugs or nuances in decision making would have been hard or nearly impossible to address in a timely manner when handled independently.

This highlights the importance of considering each case individually rather than applying prescriptive recipes blindly. At best, they serve as broad guidelines rather than rigid rules. It also becomes particularly evident when working with multi-cultural teams, dealing with a variety of differences. It's not sufficient to reduce them to a short list of checkboxes and call it a day.

It's not unusual for a single project to include teams located in various parts of Europe, both coasts of the United States, Asia, and sometimes Australia. Team location and members' cultural backgrounds impact various aspects such as openness to engagement and information sharing, feedback styles, work approaches and hierarchy views, values and priorities, self-perception, perceived stressors, and responses to them, among others. Seamless communication is always great, but most often than not, you should expect and account for a certain level of friction, regardless of how well intended everyone may be.

But wait, there’s more. The other significant factor is neurodiversity, with even conservative figures placing prevalence rates well into double digits, meaning you’re likely to encounter it, especially in self-selecting environments. Once again, it's crucial to remember that each individual is different, regardless of the broader groups they may belong to.

As a manager, you’re more likely to yield better results by inviting the possibility of pair programming than forcing it upon your team members. Otherwise, there's a real risk of performative actions and awkward displays, followed by concluding in disappointment that the new favorite toy is broken. This works because it helps to differentiate between cases where the limitations and reluctance is strongly ingrained in one’s personality or when it’s rather a result of unpleasant past experiences.

While some find pair-programming helpful and liberating, others may find it inhibiting. Just because it sounds and feels intuitive (a subjective notion), it doesn't mean it's suitable for everyone. Furthermore, while there's some correlation, don't expect an obvious split between extroverts and introverts. Again, I argue that preferences for or against are more related to perceived safety and personal experiences with vulnerability in the workplace than to robust neurological makeup.

This is particularly relevant during actual interactions, affecting aspects such as chemistry, flow, energy management, and compatible communication styles. Setting and maintaining reasonable boundaries and adapting the best tools for the job can significantly influence outcomes.

Take interviews centered around this type of interaction as they’re a distilled, high intensity representation of this work paradigm. By overlapping a clear power imbalance over an already high stress situation, the process becomes far more reliant on how invested each party is and how much of their personality and past they decide to bring into it. Things like a condescending tone, a lack of interest or attention, failures to notice non verbal cues or a susceptibility for going into one of the four major trauma responses can make or break the session.

The AI in pair programming

This leads us to consider the future where you may spend a significant amount of time collaborating with AI assistants through pair or mob programming. While you'll receive greater flexibility, many lessons transfer well, and there's a healthy balance of give and take.

It's a viable alternative in many ways, but it's far from perfect. Even disregarding glaring privacy and copyright concerns around black box solutions, you must still consider their limited capabilities, uncertainty, and unreliability in introducing subtle bugs, the risk of investing time in creating and refining prompts, and the possibility of not obtaining satisfactory results for further development. Especially when dealing with complex problems that extend beyond the AI's working context or training data.

Keeping expectations realistic is essential. Thinking AI assistants will perform complex reasoning or deep creative exploration may lead to disappointment for a few more years. They can mimic some abilities, particularly in commercial offerings and open-source models with complex enough internal architectures, especially when integrated into agentic systems simulating chain of thought and similar approaches. However, for smaller models, extensive hand-holding is often required, eventually relegating them to less demanding tasks like code completion.

At this point, don't expect an AI assistant to perform your job, as you likely wouldn't have one if it could. You can and should use AI assistants to quickly learn practical skills, particularly when experienced, and as a creative stimulus for bouncing ideas and sparking your own creativity. You can think of them as a journal who gives feedback, but closely monitor their tendency to mirror and reinforce your own thoughts.

This approach offers a "no wrong questions" mindset, which can sometimes help or hinder your ability to move quickly and experiment. Without judgment, you may feel liberated to ask many wrong questions before eventually asking the right ones and learn from both.

Whether you engage in pair-programming with a fellow team member or with a data system capable of acting like one, it still requires you to step outside of your comfort zone. It’s a good opportunity to reflect that sometimes growth happens when you embrace vulnerability no matter how uncomfortable it might seem. Nevertheless, your openness to collaborate and share space may positively influence more than just a fleeting moment of your professional journey.

Why your brain isn't broken (but documentation might be) - a tab hoarder's musings on learning

Horia Varlan — Fri, 25 Oct 2024 07:28:58 +0000

About a year ago, right after I quit my big tech role to manage my burnout, I had what seemed like a brilliant idea at a time. In the months prior to that moment, I had grown more interested in AI so it made sense to go through several online specializations. A couple of months and a few dozen courses later, I was receiving my last certificate. I don’t regret it but I’d be hard pressed to claim it has helped me all that much. Instead, it was an almost chaotic mix of blogs, books, videos and working on an actual product that made a real difference.

No surprise why I wasn’t impressed, a few days ago, when I came across a post listing nearly a thousand technical certifications. On the surface, it might even sound like a conservative figure, except the post highlighted only prominent institutions. Yet, behind every major source, lies an abundance of lesser-known alternatives. Once you also factor in casual content creators, the numbers add up quickly. Yet, each day is still 24 hours long.

This begs the question why would you need to revisit the same exact content so many times, usually presented in almost the same exact ways? Wasn’t the original treatment supposed to cover everything you needed to be productive? Why introduce incremental changes that give you a false hope of novelty, if the primary resources already meet the audience’s needs? Perhaps they don’t.

If you’ve been in tech related spaces for more than a minute, you should be well acquainted with information overload. Not a novel problem by any means, yet one that has grown in recent years, in part due to misplaced incentives. So you doubt or blame yourself wondering if you’re smart enough or have put in the right amount of hours. You open yet another tutorial, go halfway through it, only to face similar issues: either it stops right after covering the very basics, or simply repeats some more advanced ideas along with a quick copy-paste of some code or diagrams. And there you are, yet again at the end, encouraged to like and subscribe for more of the same. Why settle for dozens of redundant Medium articles when a single Reddit or X thread explains everything? There’s no need for things to be so complicated.

You could also play devil’s advocate and argue that curiosity is a virtue and it drives personal growth. Fair enough. Except, as you grow older, exploring these topics competes with some of your other priorities for your time. The straight out of college luxury to spend tens of hours each week filtering through random materials dwindles a couple of decades into your career. However, in a field where significant shifts occur every five to ten years and the fundamentals will only get you so far, the expectations may stay constant or increase. That cost of lost time has to fall on someone: either you or your employer.

What causes this disparity?

After all, most people are perfectly capable to learn basic concepts and most things can be broken down into relatively simple building blocks, with a reasonable low effort.

While your experience may vary significantly, some things tend to come up over and over again. For example, the strong bias against generalist knowledge tends to favor specialization. Hence the frequently cited, yet often misquoted saying ending in “...master of none”. That isn’t inherently problematic, especially when depth and complexity are more relevant over broad general knowledge.

Nevertheless, some of the same mechanics and motivations can lead to terminology inflation, particularly when the ownership of concepts boosts one’s professional image and ultimately their bottom line. As competition intensifies and spaces become more crowded, it starts to clash with the choice for more banal, yet intuitive names. It’s no surprise when you go through entire books of “lore” only to realize there’s little new insight. Another couple of days down the drain.

Background knowledge is certainly useful. Multiple terms for similar concepts, not so much, as it only adds more noise and confusion resulting in a cacophony of jargon. Conversely, an excessive reliance on tradition and assuming what’s familiar has to also be intuitive, is equally detrimental. An established practice that has been followed for generations is not an inherent proof of its validity just like tenure and years of experience do not automatically guarantee anyone’s performance.

Which naturally brings us to another major aspect impacting the accessibility of a piece of information for the average consumer. There’s frequently insufficient interest in explaining the “why” behind choosing a particular technical solution or doing things in a certain way. Moreover, it’s almost an open secret that simply inquiring about the “backstory” can get you in hot waters due to being perceived as challenging authority and a sign of a confrontational personality. Similarly, it’s often assumed that “how” can serve as a substitute. That’s seldom the case. The former can act like a retrospective of your journey while the latter is just the destination you settled for.

From there to flame wars around the effectiveness of code comments is just a small step. There’s little value in adding comments that simply repeat what the code is already doing. However, when they capture the reasoning or underlying business logic, the situation changes. You’re providing essential context that can’t be easily inferred otherwise: the thing that everyone conveniently forgets to write down and it slowly gets lost as people leave the team. “Why don’t we delete that piece of code that hasn’t been used anywhere else in years? No idea, but let’s play it safe and leave it there, just in case.”

“That’s just how we do it”, “deal with it”, “take it or leave it, plenty of others waiting in line for this opportunity” can certainly end a conversation but probably not for the right reasons. Yet, being able to anchor a piece of information in sound logic yields more enduring benefits.

Now's the point where you could easily label this as just another snowflake rant or weak stream of consciousness puff piece built around anecdotal evidence and scroll down to the next content item in your feed. After all, if you really want something, you have to make some sacrifices, right?

This reminds me of an internal DEI training from a few years ago. At the time, I was still naive enough to think there were sincere intentions beyond merely protecting against legal risks, but that’s a story for another time. What particularly stuck with me was the findings from several studies about the business impact of neglecting a diverse audience. Once you strip the veneer of false concern about equity, the studies made it as clear as possible how exclusionary practices often failed to tap into lucrative sources of revenue.

What's the impact?

Education and upskilling are often seen as secondary aspects of business. Yet, this doesn’t always limit the negative effect that counterproductive entrenched approaches have on outcomes. Even a cursory glance reveals the ways in which it impacts individuals, businesses and researchers alike. Let’s take a closer look, shall we?

From a personal perspective, it’s crucial to recognize that different people learn in different ways. What feels and sounds intuitive to someone, might be completely obscure to someone else, even when engaging on a level playing field. Once you factor in learning disabilities, things become even more complicated.

Take, for instance, the well-known fact that individuals with ADHD strongly favor non-linear information processing while those with high-functioning autism require explicit context. Meanwhile, individuals with dyslexia may struggle with consuming text but can handle audio and video much better. As with fast fashion, one size fits all might be cost conscious but it’s seldom sustainable or beneficial.

No matter what, when looking at this through the lens of practical results, there’s a strong case against unchecked complexity and the compulsive obsession with completeness. Just because these things serve a genuine role for research and archival purposes, it doesn’t mean they translate equally well into a real world focused mostly on achieving fast results.

Having worked in tech for enough time, and provided your choices have not been fully subsidized by ZIRP free funding, you should know fairly well that any new piece you introduce has an added weight to it and it’s only justified if it serves a clear purpose. Otherwise, why commit to anything bound to increase complexity, slow things down and potentially generate tech debt?

You could argue the same principle applies to creating any kind of documentation, formal or not. Unless it makes things simpler, memorable or contextually relevant, it’s just another energy drain. Each new one chipping away at someone’s will power and competing for what is arguably a limited conscious context. Good luck managing this without a solid foundation of meaningful abstractions meant to anchor what you’re learning.

As with anything in life, as much as we fail to acknowledge or hate to admit it, we are opportunistic creatures. At any given time, we engage in weighing how profitable or feasible an endeavor is. When your intuition tells you there’s no point in trying, you’re less likely to start. Even if you pass this initial challenge and take a chance, you still may quit halfway through if one too many hurdles mount along the way. You conclude you might just not be suited for it and forfeit once again to imposter syndrome.

Still, I can only assume the worst of all is when you manage to trick yourself into a false sense of understanding and self-confidence by equating memorization with comprehension. This can lead to mimicking proficiency without truly being able to generalize. Similar to basic AI systems.

Things steer well into toxic, counterproductive territory once you notice the otherwise obvious imbalances. We often place blame on knowledge consumers by implying a lack of initiative, laziness, superficiality or even missing sufficient cognitive abilities. A dissonant stance from the same science that has publicized how differences in neurotypes influence how we process, model and recall information.

Whether that’s an unhappy artefact of academic elitism, an overly pedantic attitude or a certain level of scientific machismo, that’s still up for debate. One thing is clear though: bad traditions die hard, especially in the absence of any awareness or interest to even acknowledge it. Worse yet, when it comes as a result of: “I went through this and it made me stronger, so you should too”.

But wait, there’s obviously more, given how these things permeate well outside of personal spaces. Since tech strongly relies on constant learning, there’s no surprise why inefficiencies in one space bleed well into other interconnected spaces.

Let’s consider the industry’s infatuation with extensively using platforms like Leetcode for recruiting. At first glance, relying on theory and artificial problems masquerading as practical challenges can seem like a quick and painless solution. The results are measurable and this approach works fairly well for a wide range of potential candidates. Thus, it’s frequently hailed as the most effective method available.

After all what kind of professional are you without a good grasp on fundamentals: frog jump, house robber and best time to buy ice cream.
Upon closer inspection, it starts to look more like a cop out. You realize this kind of assessment offers no guarantees about practical knowledge or a candidate’s ability to perform well in a fast changing environment. It strongly favors new grads or anyone who is willing to devote copious amounts of time gaming the system instead of widening their hands-on skills.

It often relies simply on good memory and limited pattern recognition that fades unless exercised. Shouldn’t core knowledge be more enduring? Additionally, is a party trick all that valuable if an AI system can replicate it for just $9.99, in a couple of minutes, and with just a few retries, give or take?

Still uncertain about this issue? It ultimately places an extra burden on your existing roadmap. Then it slowly shifts the culture. In just a few generations, teams learn to rely heavily on a limited set of recipes for success which dissuades individuals from innovating or acting independently.

Are you sure your team affords implementing a big tech architecture when you lack either the funding or the business case to warrant such a “robust” solution? Once that's in place, you finally get to rest and vest. When things go south, you just pass the hot potato somewhere else. Change companies and let others deal with it. Rinse and repeat.

This results in the wrong set of incentives taking hold of recruitment followed closely by a thriving parasitic interview prep industry. We’ve seen this before, one too many times. For example, in QA teams driven to prioritize reporting as many bugs as plausible, rather than finding real-world issues. All as a result of misaligned success metrics.

For the sake of the argument, let’s assume you’re thriving despite relying on these aspects. More power to you. Think the effects stop there? Think again.

The same incentives that encourage sloppy or overly dogmatic processes, disconnected from the needs of their intended audience also impact a team’s daily operations. For example, once they establish a bias against proper technical writing or doing it just for the sake of it, things become dangerous. This leads to a gradual loss of knowledge and syncopated onboarding that invisibly reverberate into other activities, wasting precious resources.

It’s not uncommon, and I’ve certainly been through examples of documentation that looked like a goldmine on first inspection. Yet, it was nearly impossible to find solutions to specific problems. You’d be forced to simply go through tens of pages and try to distil that into a working example. This can add up significantly when multiplied by thousands of developers over the course of a whole year. Was it worth the added cost just so a select few can feel good with their overly academic treatment? I’d argue probably not.

As a direct consequence, it’s also only logical to expect an over reliance on well intended, albeit over prescriptive, set of “clean” patterns. Nothing wrong unless they become the ever present hammer of choice used to transform any problem into a nail, thus contributing to bloated codebases and processes.

This highlights the need for introspection around why we do things in the first place. Once you’ve reached a critical mass around a set of biases that favor performative completeness over pragmatic selectiveness, there’s no easy way back. A more complex solution is not always a guarantee you’ll escape tech debt. Sometimes it’s quite the opposite, especially if your system is changing fast.

Similarly, an overly academic internal culture can contribute to a strong preference for ceremony and the documentation equivalent of play pretend and make work. Which brings us back to one of the original ideas: just because something is comprehensive doesn’t mean it’s effective.

The hard truth is clients seldom care about what happens behind the scenes. They have their own problems and goals and usually only care if your product works well enough to make their lives a little or a lot easier. It makes little difference if every invisible thing is just right or if your product has a full collection of peer review stamps on it. In fact, it’s unsurprising when a culture with this kind of characteristics also showcases pushback against customer feedback. After all, “what do the customers know, we’re the professionals”, right? Sometimes it’s justified and you certainly shouldn’t bend to every request. Other times, it’s just a dangerous, lazy excuse.

Conversely, an excessive focus on scientific rigor, when left unchecked, can negatively impact research just as well. Obviously there’s nuance. However, a good example of things going south is once you start to equate a paper’s scientific value with the density of formulas and expressions.
As AI fills more of these gaps, intuition and connecting diverse ideas become equally important. Sharing a common language with your peers certainly helps. However, an overly dogmatic preference for jargon can end up excluding talent that might have otherwise contributed equally well. After all, the history of science is filled with stories of providential encounters and flashes of inspiration. Are they all be made up?

How do we address these issues?

While there’s no definitive solution, attempting to resolve them is still worthwhile. Addressing the root cause is a fairly easy optimization and a good starting point. You’re essentially taking in a finite amount of initial work, which for all intents and purposes can be outsourced or crowdsourced. This saves your audience from engaging in the same amount of work by simply making it good enough from the get go.

In other words, if your documentation's accessibility depends on more than a couple courses or tutorials, maybe it's time to reconsider either the audience or the presentation. Very much aligned with this is the over reliance on “the curious reader” to fill in the blanks. Providing necessary context and references, especially when consumed online, can remove a lot of friction.

Fewer obstacles remain for switching to a more interactive, real-time approach rather than the cold monologues we’ve been used to so far.
While information heavy manuals and references can still act as a fallback source of truth, maybe we should favor just in time documentation, customized to specific scenarios that the user encounters daily. That’s where AI powered solutions combining LLMs, RAG and knowledge graphs enter the picture, lowering hallucinations to a much more tolerable level while managing to quickly sift away contextual noise. Things become even more engaging once an AI system is able to understand and adapt to your own learning style.

However, there are two more benefits that often get lost in the hype. For one, an automated system provides a non-judgmental and unbiased set of interactions. There are either no wrong questions or if they are you won’t get penalized for asking them. A safe space without raised eyebrows or rolled eyes. Then there’s the added bonus of similarity search allowing you to correct and enhance your original requests when you’re not particularly sure what you’re looking for. Once again, little to no judgement attached to this, so more incentives to explore freely, course correct and make progress fast while staying focused.

As AI becomes more prevalent, providing a wider range of different formats catering to different audiences end up being increasingly straightforward. Also, leaning into multi-modal learning helps tap into different senses and ways to model information. By casting a wider net in terms of presentation, you’re more likely to hook into an effective personal mechanism that improves the chances for success. Do not just blindly assume things about your audience.

On the surface, it might look like just another way to package the same thing but there’s nuance to that too. A new presentation often ends up subtly restructuring the narrative, sometimes relying on different triggers and stimuli and eventually forcing your brain to anchor the information from different perspectives. No wonder, immersive experiences have been touted as the next best thing in areas like language learning.

Beyond focusing on the tools themselves, let’s consider other less tangible changes that can enhance the learning experience. Firstly, keep in mind that at the end of the day, even in the most technical context, you’re creating for people on areas that either impact or involve other people. This means you should consider some messiness and certainly a constant degree of change.

As an author, there’s little value in being the only one able to decipher the knowledge you’re sharing. Much like that US compiler course infamous for having a surprisingly high failure rate among some of the brightest students in the world. Is everyone really that unprepared or maybe the course is not doing a great job making the information accessible? That’s why it pays to stop and ask yourself "Does anyone actually get this?".

Circling back to one of my previous points, if you make it right the first time, you won’t force people to go through the same hoops over and over again. This goes hand in hand with understanding and adapting your delivery to the context in which you function. If you’re acting in a practical, commercial space, then prioritizing full scientific rigor over business impact is probably not a very good approach.

At the end of the day, the main takeaway is that learning is a deeply personal journey. While it often starts with paying attention and taking notes, it can easily turn into a pointless time sink and a hoarding exercise. Instead you’re probably far better off by simply experimenting and combining different tools until you figure out what works best for you. And if you end up on the other side of the table, you might find more fulfilment in facilitating access to knowledge than the fleeting high of shouting “I am the documentation”.

From Burnout to AI Integration: A Prologue to a Year-Long Development Journey

Horia Varlan — Thu, 17 Oct 2024 07:53:16 +0000

Just days before last Christmas, I found myself spacing out during a Leetcode interview gone wrong. The two lads on the other side of the screen where running in circles trying to explain a poorly worded yet simple heap problem while I was once again acknowledging the perils of going against my gut feeling. About a month prior, I had reluctantly agreed to this, despite being still on sabbatical and ignoring a few clear red flags.

Up to that point, I had taken the process seriously, but was far from invested in the outcome. No wonder, my first instinct was to cut the interview short, once I realized it was going nowhere. Yet, I was curious enough to continue. As expected, it went downhill fast, for all the wrong reasons, so afterwards I politely asked them not to contact me for future opportunities. After all, finding humor in watching projects fail is not particularly a value I subscribe to. Coincidentally or not, that team went on to bigger and better things like carelessly paralyzing a decent chunk of the world’s transport infrastructure for about a week.

Still, there was a major silver-lining. It helped jump-start an honest introspection process around my goals and priorities. Was I going to continue to invest weeks at a time mindlessly memorizing theory and patterns that proved to be largely pointless for day to day tasks? Were there instead better ways to further my knowledge and career? Plenty of time over the winter break to think this through.

Fast forward to early February, I was finally aching to start building again. An idea had grown organically after months away from coding while engaging in a couple of creative pursuits and closely following several related online spaces.

Piece by piece, the opportunity to build a solution for bridging those gaps started to solidify. It came from watching random users bring up the same problems over and over again, some of which I was facing myself. The following couple of weeks were all about exploring and assessing if the project was both feasible and worth the otherwise large time investment.

It soon became clear that even in the context of a failure to launch, there were still plenty of benefits. You’d get the chance to develop a project in a holistic manner, while being completely self reliant every step of the way. It would force you to step out of your comfort zone and act as a versatile vehicle to tinker with new tech and gain practical insights on AI integration. Worst case scenario, you’d end up with a multi-purpose sandbox for quickly developing new proof of concepts, all wrapped in a tailor-made tool.

Then again, it was a potentially expensive leap of faith. Not only were you forfeiting a steady stream of income, but you could also end up with little to show for in terms of things that translate well during normal tech interviews. The risk was not unexpected, given the sharp market downturn, but it still required careful consideration.

To better understand the context and why it made sense, let’s rewind to last August. Back then, I had made the tough choice of going on sabbatical as a way to pause and rethink how I wanted my next decade to unfold, while also recovering from burnout. I knew I couldn’t stand still for more than a few months, but I had nothing lined up after that.

For several years prior, I went all in on a few intense projects, without paying too much attention to building a longer term career strategy. To the point where a former manager friendly reminded me good work is far from enough to guarantee momentum. If you’re lucky it gets rewarded, if not it only gets you more work for the same price. For example, you can rake in years of shadowing your managers and being fully responsible for the outcome of your projects, yet that only really matters if you’re able to secure a title to back it up. Especially in an industry haunted by the ghosts of ageism and typecasting. Despite all that, I was still fortunate to grow in many ways, yet the lack of intentionality in a shaky, shifting market was shaping up as a liability.

Switching from golden shackles to the great unknown was a leap of faith just as scary and isolating as it was liberating and transformative. Even more so, when bombarded with a daily dose of FOMO that can throw you in a spiral of second guessing every choice. There’s always a larger RSU package, the next, far better title or a project with enough leadership visibility to set you up for life. Yet, if you take on the challenge for the right reasons, it acts as an excellent proving ground for establishing healthy boundaries, character development and recognizing the environments you thrive in best.

Back to the project, I started it with absolutely no expectations. A way to test if I still wanted to stay in tech and going back to the core of why I had chosen this field over two decades prior. Just a mix of pure curiosity about how it could evolve, while refusing to cut corners and make my life harder few months down the road. Before long, I found an almost therapeutic effect in watching a complex piece of software come to life before my eyes.

I had purposely opted to start out with a very naive solution and iterate on that. It echoed similar patterns from my previous big tech roles where understanding the problem and putting the conceptual pieces in place were paramount aspects. Thus, I got better hands-on experience in product development, explored in-depth refactoring techniques, learned to adapt to naturally shifting architectures, and gained more practical insights into how early mistakes impact future development. A very interesting game of trade-offs centered around balancing complexity, velocity, and resisting the urge for premature optimization. A challenge that surely deserves its own separate treatment.

Come early May, I paused for a second to look back on the progress and check the status. As expected, the project had grown and it was in need of its first serious architectural refinement. In short, moving from the initial naive implementation to a more structured and reusable set of building blocks, modeled on recurring usage patterns.

Around the same time, I was also approached for another lead position. The team was well aware I was highly familiar with the questions they planned on asking and they could have also checked a great deal of my past work if they wanted to. That’s why I made a conscious choice not to cram beforehand as a means to double check team culture fit.

The interview went well, but ended up on an unexpected sour note with a request to recite the definition of a hash map, after a dozen other questions straight out of interview prep sites. I knew in that moment I value building things that solve real problems more than I do conforming to artificial expectations. They withdrew the role shortly after.

With this experience fresh in my mind, I decided then and there to commit myself to the project until development reached a more stable, autopilot mode. While I didn’t anticipate some of the more challenging steps that followed, each hurdle was well worth the effort. Gradually, the proof of concept morphed into something much more versatile and allowed me to integrate new use cases with substantially less effort. Sometimes within hours instead of days.

If you’ve never done this kind of sport before, you need to realize there are cases where progress can no longer happen through incremental changes. Which means you’ll have to sit through seeing your project break and just push through with the conviction that you can eventually put the pieces back together. It’s a daunting task but equally rewarding, once you overcome it.

This process also forced me to reason about feature prioritization and preparing an application for production when you lack corporate or venture capital resources. It’s essential knowledge for bootstrapped products like this, as it helps maintain a strong awareness of the impact on future marketing and early monetization.

Come late summer, I was eager to dive back into AI development after laying a solid foundation, so I had to reassess my understanding of the current ML ecosystem. Fortunately, much of early conceptual work transferred easily to this stage too. It ranged from finding ways to seamlessly combine different paradigms to reasoning about AI agent systems, while being mindful of compute limitations. It also helped cut through the noise and approach scientific literature from a more specialized and discerning perspective. In stark contrast with the confusing experience of aimlessly going through all AI/ML Coursera specializations, a year prior.

Back to the present, my focus has shifted to constructing an architecture that offers users a choice between fast inference using off-the-shelf AI API providers and local-first / edge privacy at the expense of slower speeds.

Looking back on this journey, I believe there’s value in documenting certain aspects of how it came to be. Not only because of the learning value and the potential business impact, but also due to its relevance to a couple of social aspects where there’s a lot of well manicured lip service and far less meaningful actions.

As I worked through the recent stages of this project, I realized there are ideas worth sharing which might help others navigate similar challenges. Over the next few months I intend to do just that, going into further detail on a few main topics.

Firstly, capturing the challenges of transforming a working prototype into a commercially viable solution. More precisely, the attention to detail in areas like infrastructure, security, privacy, performance, usability, and accessibility, to name just a few of the things that can make or break a successful landing. It’s an opportunity to document the lessons learned along the way, including examples of things that might not go according to plan and how to avoid that from happening again.
Another crucial aspect I look forward to exploring in depth is AI/ML — a field still sadly shrouded in hype and proselytism. Instead, my goal is to provide a more grounded and practical approach. One focused on solving real problems with minimal impact on development and maintenance, while keeping projects as future-proof as possible.

Additionally, I intend to write about managing burnout and work-life balance while maintaining high throughput and how that’s essential for long-term success. I’d also like to touch on different learning styles and gaining confidence to explore intimidating knowledge areas — ones that might seem daunting but can be conquered with the right tools and mindset.

As a side benefit of this exercise, I hope to hone my professional writing skills by providing earnest content rather than relying on cookie-cutter output generated at a growing number of tokens per second.

I’d love to hear your thoughts — whether it’s about similar challenges, insights on AI and development, or topics you’d like to see me dive into next. Feel free to share your experiences or ask any questions in the comments, or reach out if something resonates with you.