DEV Community: Gustav Wengel

Azure Event Hubs Are Not Queues

Gustav Wengel — Tue, 12 Jan 2021 00:00:00 +0000

Unfortunately, I’ve seen quite a few people use Azure Event Hubs, when what they really wanted was a queue.

Event Hubs are great for large-scale data ingestion, but if you just need to pass messages between your services - use something else. Here’s a few reasons why.

Messages Block The Queue

Event Hubs uses a partitioning model where messages sent to it are distributed among partitions.

Each partition has one reader that can read from concurrently, and the messages are always processed in order.

This means that if you have a message that’s taking a while to process, the rest of the messages on that partition are delayed.

It means that if you have a message that’s causing your application to crash, you can’t process anything else on that partition until you somehow deal with that message. Either by waiting and retrying, dropping the message or perhaps saving it somewhere else.

Contrast this with something like the Azure Storage Queue where you can lock a message and attempt to process it.

If you can’t process it or it takes a while you’re not holding anyone up. Other consumers don’t need to wait for you to process your message before they can continue.

Checkpointing Makes Error Handling Hard(er)

Event Hubs are batch-oriented. You receive a batch of messages and then when you’ve processed enough you create what’s called a checkpoint.

A checkpoint is a way of writing down: This is how many messages I’ve processed. Checkpointing is something you do occasionally - it isn’t meant to be something you do every message.

If your application crashes between checkpoints it’ll start off at the last checkpoint and you’ll reprocess each message until you’re up-to-date again.

That means your systems have to be pretty durable to messages that are delivered multiple times.

While this is a problem with most queues (applications can always crash between receiving a message and processing it), often you’ll only process one message at a time, so there’s less harm done if something fails.

Event Hubs Require An Extra Storage Account

The partitioned readers use an Azure Storage account to coordinate checkpointing and partition ownership between them.

This means that you can’t read from an Event Hub without also having a storage account. It’s not a big deal, but it does add an extra connection string to manage, and an extra Azure resource to deploy.

Local Development

There is no way of running an Event Hub locally.

Contrast this with e.g. an Azure Storage Queue which you can run locally with Azurite, or many other queues which you can run in Docker.

Being able to run your dependencies locally is really nice. It means you don’t have to deal with spinning up Event Hubs for each developer.

Reasons to use Event Hub

I don’t hate Event Hubs. They’re a good tool if you need what it offers, such as the Event Capture Feature, or you have large amounts of data.

But if you just need to send messages between services? Use something else.

How To Ask Good Questions When Working Remote

Gustav Wengel — Fri, 27 Nov 2020 00:00:00 +0000

At my current job with SCADA MINDS I work with several different teams at the same time. That means I get to answer many questions. I’ve noticed that some teams are much, much better at asking questions and communicating than others.

Let’s take a look at two real interactions I’ve had that effectively illustrate the difference between good and bad communication in a remote setting. First, the example of bad communication. This was a question asked to me in private:

Next up - the better one. This was a question asked by an intern in a public development channel:

These two interactions are as different as night and day!

Let’s try to consider a couple of factors that will lead to asking good questions - and in turn getting good answers.

Write One Self-Contained Message

One of the most important rules when asking the question is to write one, self-contained, message.

While saying hello before asking your question might seem polite in real-life, it’s awful etiquette when asking questions online.

Don’t write “Hello” in one message and then write your question. Whoever you write “Hello” to is going to get a notification disturbing them, and then sit and wait for you to formulate your question.

And definitely don’t wait for them to say “Hello” back before asking your question. That’s just going to delay the whole process even further.

Compose one message, that contains both the formalities and the question:

Ask Questions In Public

A lot of teams have a tendency to ask questions in private, because they know which person usually has the answers.

This will get the question answered - but it’s not the best way. Here’s why:

It leads to more work for the person who’s answering Someone else might have asked the exact same question last week, and maybe someone will next week. This person will end up answering the same question over and over.
Your team doesn’t get smarter Anything that goes on in private chats or mails is invisible to the rest of the team who might be struggling with similar questions.
Someone else might know better Even if you think you’re asking someone who you think knows the answer, they might not be the best person to answer it. Someone else might have much more expertise with a particular topic, but if you ask in private you’ll never find out.

So instead of asking in private - here’s what you should do: Ask your question in the most public forum that the question is relevant to.

This could be StackOverflow if you have questions that aren’t specific to your particular project or company, or it could be a company issue tracker or Q&A platform. If you can ask it somewhere where it’s easy to find it later that’s best, but if not, the appropriate Teams or Slack channel is also fine.

But if I do that, people don’t answer my question!
Some teams have cultures where people are slower to help out if not explicitly asked, and some teams have cultures where people are eager.

You can still ask questions in public - just make sure to tag the people who you would normally write to in private.

But I’m afraid to look stupid if everyone can see I don’t know something?
That’s very human. But not knowing stuff stuff is part of the game. If you ask good questions, no one will judge you for them.

Share The Answer

Sometimes you can’t get your answer in a public forum. Maybe you’ll have to explain the problem in-person or do some extra troubleshooting with someone. That happens. When it does, you should always write down the answer where you asked the question.

Perhaps your colleagues who tried to help you are curious, or perhaps they will struggle with similar questions. Asking questions in public doesn’t help much if you don’t also share the answers in public.

Ask Good Questions

A lot of stuff has been written about asking good questions. This guide is excellent and friendly, and this guide is also excellent - but much less friendly.

You should read those guides - but quickly summing up there’s two things you should consider when you ask someone a question:

Did you give them enough information to understand your question?

Your message should contain enough information that anyone reading doesn’t need to ask any follow-up questions to understand what your problem is. So read your message before sending it, and consider whether or not you’ve provided enough information.

Have you given them enough information to answer your question?

Most problems can have many potential issues. You need to explain whether or not you have any theories as to what’s going on, and what you have already tried to solve the problem.

If someone is going to answer your question, they’ll need this information. So either you’ll write it down at the start or they’re going to ask you for it. Let’s save them some time and give them all the information right off the bat.

Asking good questions is hard and a skill that takes a while to master. But we can always try to improve our communications. Try noticing if you often have exchanges where someone misunderstands your question or asks for clarification - and try to see if you can do better.

How Microsoft Teams Kills Knowledge Management

Gustav Wengel — Thu, 19 Nov 2020 00:00:00 +0000

Microsoft Teams seems like it’s trying to be the one-stop tool for all your organisational needs. I think it works fine for chat and calls, but for knowledge management it is extremely unimpressive, often making things much harder than they need to be. Here’s four ways Teams is killing your knowledge management.

Important and Irrelevant files are mixed

Teams has a “Files” tab where you can save files that are relevant for a channel.

The concept is good, but unfortunately it’s plagued by an unfortunate decision: If anyone posts a file inside a conversation it’s uploaded to the Files tab.

This makes it impossible to distinguish between these two scenarios:

This file is here because someone consciously chose to save it so you could look at it
This file is here because someone needed to share it that one time

Which means that the top level folder quickly becomes an interspersed mess of really important stuff™ and stuff that doesn’t matter at all.

The Wiki Is Unsearchable

Teams also has a wiki functionality where each channel gets its own wiki tab.

That’s wonderful - you and your team can share information and collaboratively edit it!

Except the information can be almost impossible to find again, because there’s no way to search for a specific wiki page or the content inside it.

This means that if you want to find something you have to either know exactly what channel and wiki page it’s located at, or go meandering through all the wikis of all your channels. Writing down stuff isn’t worth anything if you don’t easily let people find it again.

You Can’t Search For Directories

But for files, at least you can group them inside directories and then easily find those again right?

For example, we have a directory inside one of our channels called “Books” for, well, books. And if I want to find the books, I can just search for “Books” and Teams will surely get me the directory right?

Not a chance buddy. Teams only shows you files, not directories.

This means that to figure out what’s in a directory you either have to remember the channel it was in and find it that way, or go search through SharePoint, which luckily manages to get it right.

You Can’t Archive Channels

Often you create channels for projects or events, and those channels at some point become irrelevant.

Teams makes it very easy to be stuck with those channels forever. You can delete them - but what if there’s information in there you might need later?

There’s no way to mark that a channel isn’t active anymore. The best thing you can do is rename it and have everyone hide it so it doesn’t show up all the time - but that’s a real band-aid solution.

While Teams has come a long way for chat, it’s still lagging behind in regards to all the auxiliary functionality it offers.

It seems like it wants to do everything. Unfortunately it doesn’t do any of the things particularly well.

Many of these issues have been open on Microsoft UserVoice for years, with tens of thousand of votes, and no response from Microsoft.

So if you haven’t moved your wikis and file management to Teams yet - I’d suggest you wait a year or two. But if you have, like us, and the cost of switching away is too high, there’s nothing to do but arm yourself with a truckload of patience.

Being In A Hurry Makes You Slower

Gustav Wengel — Sun, 01 Nov 2020 00:00:00 +0000

Not too long ago I was working on an unfamiliar project, with a technology stack I didn’t really know. I had a short deadline, and (what I thought was) a pretty short assignment. Turns out it wasn’t. Every time I thought I was done the requirements shifted slightly and every time I thought I had solved the last bug a new one appeared.

Because I was pressed for time, I did two things:

I cut corners. I didn’t refactor things I should have and I should have tested things I didn’t. I didn’t feel like I had time to do those things - but I also didn’t always know exactly how. This leads to my next point:
I neglected to learn the things I needed. I felt like I couldn’t justify the time spent to learn how to do X properly, when I didn’t have very much time to do X in the first place.

In another world where the task had been as small as we thought, these could have been the right decisions.

However as soon as the task became just a little bigger than we thought, neglecting to learn my tools properly made me much less effective. I traded a short-term benefit for a long-term loss.

I’m not saying you should always spend the time to learn how to do everything properly. But if you’re always in a hurry, and you never get the time to slow down, you might have an issue. Likewise, if you have a short deadline for something and you decide that it’s not worth the time, but then the deadline shifts - it’s time to re-evaluate. A deadline and scope that shifts once, will probably shift again.

How To Not Get Blocked While You’re Waiting For A Code Review

Gustav Wengel — Wed, 23 Sep 2020 00:00:00 +0000

You’ve made your awesome new Pull Request, but the person who has to review it won’t have time for several hours.

While we all strive to review code as fast as possible, there’s always going to be delays.

So what do you do while you’re waiting for that review?

There’s always the coffee machine, but if you’re going to be hanging out there for hours, people might start asking questions like “Why are you here all the time?”, “Is that much coffee healthy for you” and “Please stop staring at me”

Here’s three ways to avoid those comments and keep working, while you’re waiting for a code review

Work On Something Else In The Codebase

A good strategy is to always have at least two tasks you can work on - preferably tasks that have as little as possible to do with each other.

That way, when you’re done with your first task, you can branch out from your main branch again to do some completely unrelated work.

Examples of this:

If your first PR was on the backend, start working on something in the frontend or vice versa.
After your PR, find a juicy bug to hunt down and fix. Generally switching between developing new features and bug-fixing works well.
Or just find another issue to solve that doesn’t have any overlap in the codebase.

Here we see two branches branch out from main. When your first branch is ready for review, start the second branch and keep working.

To me this is one of the most effective ways to keep your velocity up, but it requires that you can juggle multiple tasks (and that there are multiple tasks to juggle)

Do Something Non Code-Related

There’s a lot to do in software development that isn’t just software development. There’s a lot of possible tasks we can do that don’t have anything to do with the code. These tasks are perfect to spend time on while waiting for review. Some examples:

Answer emails
Review a pull request from someone else
Write documentation
Write or break down tasks
Help a co-worker with something

Or just take a break so you’re ready to start working again afterwards.

Continue Working From The Same Code

Unfortunately it’s not always feasible to do something else.

Sometimes there’s only one thing to work on - and you have to base your work on the stuff you’ve just submitted in your PR.

You can branch out from the branch you’re looking to merge into master.

This means that while you’re waiting for the branch, you can simply continue working. You can then merge in the first branch, and later on the second one.

An example of this workflow. When you're done with your first branch, you create a PR, but also a new branch, based on the first branch.Then you can keep working, while keeping all your original changes.

Some caveats:

If you’re expecting to get a lot of feedback in your original Pull Request, you will probably have to deal with some merge conflicts when merging your second branch in.
If you squash your commits when merging PR’s in - this becomes quite a bit more problematic. You’re going to have to do a few merges or rebases to make your second branch merge cleanly. However usually for large branches, this doesn’t take more than a few minutes.

4 Ways To Make Your Pull Requests Faster To Review

Gustav Wengel — Tue, 22 Sep 2020 00:00:00 +0000

It can be frustrating to wait for someone to review your pull request.

While everyone is interested in pull requests getting reviewed as fast as possible, reviewing code takes time.

Here’s 4 ways to make your pull request as easy to read, and as fast to review as possible.

1. Keep It Short

Pull Requests that are shorter, are much faster to review.

The shorter your pull request is - the better:

It’s easier to think through each change and how it interacts with the other changes. This means the review process will take less time.
It takes less time to review. While a short PR can be done when you have 20 minutes of downtime, a longer one might require finding several hours in the calendar.\ Which one do you think will be looked at faster?
Studies indicate that shorter PR’s lead to finding more issues - if your PR has 300 lines, the reviewer will read them carefully, but if it has 2000 they will skim it.\ Shorter PR’s make the reviews more impactful

2. Split Your Changes Up

To keep pull requests short, you’ll often need to split the work you’re doing up into multiple pull requests.\

While working, you might spot something that needs to be refactored, or a rename of a field that will touch many files.

Don’t make the changes directly in your branch - instead:

Branch out from main again and do the refactoring
Create a PR with the refactoring into main
Continue work on your original branch

Instead of having one PR that touches many files, you’ll have two isolated PR’s. Each which does only one thing.

3 Write A Good Description

The reviewer has to understand what was changed in the PR and why. The easier you make this - the faster the review.

You should add a clear description that tells the reviewers what changes the PR contains and the purpose of them - sort of like an introduction to the code.

This description can also help future-you. When you’re looking at that code in six months, good commit messages and pull request descriptions are invaluable.

If anything needs to be manually tested by the reviewer to ensure that it works, make sure to also include a short description on how the reviewer can test this. In complex cases, it might also be beneficial to describe what result the reviewer should expect from running it.

4. Leave Comments For The Reviewer

It’s okay to write comments in the PR for the reviewer.

These comments can be used to proactively describe file moves or other things, you might expect the reviewer to ask questions about.

Note that these comments are different from comments you’d normally leave in the code. They’re codes that might not be relevant in six months, but which are relevant when the reviewer is looking at the code now. Examples could be:

When you’ve moved part of a file to somewhere else, leave a comment. This way the reviewer doesn’t have to wonder why git has marked something as deleted, when in reality it has just moved.
When choosing a solution that doesn’t seem obvious at first glance, you might want to note what other solutions you’ve tried and why they didn’t work.

Doing all of these things might seem like extra work, but most times the 10 minutes you spend on creating a good, reviewable PR, are saved in terms of less back and forth with questions and discussions between you and the reviewer

How To Work Efficiently With Pull Requests

Gustav Wengel — Mon, 21 Sep 2020 00:00:00 +0000

I’ve experimented a bit with different kinds of pull requests during the years - and I think this way of doing them is the Two Pass Pull Request. I like to name it that - because there’s one pass made by you, and one pass made by the reviewer. Let’s take a look at how, and why, it works.

First Pass

You’ve branched out, made some changes and you’re ready to get them merged in.

The first thing you do is to create a Pull Request just for your eyes - a Draft PR.

You can do this by marking the PR as a draft or prefixing the title with [WIP].

Make sure to write a good, but short description for the PR. Explain both what changes have been made and why. This will help both the reviewer look at the changes, but will also help future-you when you look back through the history.

The description should also describe how the reviewer can validate that the code works as it should. This can be as complex as “Do X and Y, and then validate that Z happens”, or it can be as simple as “See the tests”

After you’ve created the draft PR, this is your time to go through it and make sure your code is up to snuff.

Go through each file. If something needs changing, change it immediately before moving onto the next file.

Things you might want to look for / change in the code

Remove any code you’ve commented out instead of deleting
Change names so they’re descriptive
Refactor code if necessary
Add comments to code that are not self-explanatory, such as complex workarounds or reasons WHY some code does what it does

While reviewing the code, there’s a few other tasks you might want to do:

Update the PR description if you’ve made more changes than you first thought
Leave thoughtful comments for your reviewer. If you’ve moved some stuff from one file to another, git will mark it as added/deleted. If you leave a comment saying “Moved to X” you save the reviewer from wondering why something was deleted and then having to review the moved code.

Unless you’ve made large changes, you don’t need to check the PR again.

You might have missed a few things, but that’s fine. It’s hard to keep re-reading the same PR without just skimming over it, so it’s not worth the time. Anything you’ve missed will be caught in the next pass.

After this first pass, you’ve looked at the Pull Request, and made any changes you needed. This Pull Request is now no longer a draft. Remove the [WIP] or draft status.

Second Pass

This is the second pass - by someone else than you.
Inform the reviewer about the PR, either through a direct message or through tagging them as a reviewer.

The reviewer will look at the description, and validate that the code works as it should, based on the description.

After that, the reviewer looks at the code, to find errors or possible refactorings. There are many articles describing what a good review is, so I’m not going to recap them here.

The reviewer should leave comments/suggestions that are detailed enough, that you can resolve the issues without asking them for further clarification.

The reviewer should write what is needed for the pull request to be approved. This can range from: “You can merge this in once you’ve fixed X” to “I’d like to discuss X with you before we merge this in”

Azure DevOps has a handy “Approve with suggestions” feature, that essentially makes the reviewer say “I can approve everything here as is, but I have some suggestions I believe you can handle yourself”.

Sometimes the reviewer might suggest changes that mainly differ in style, rather than substance, often phrased as “This is good, but if it was me, I would change it to X instead, because Y.”

In these situations, you decide whether or not to go with the suggestion - it is an optional suggestion. It can be beneficial to prepend such comments with a pre-approved-upon word that clearly states that it is optional, such as “optional” or “nitpick”.

After the second pass, you can go through your PR and fix the issues that the reviewer has flagged. If the issues are small enough and the comments are good enough, you should be able to do this without asking the reviewer again.

However, if there are comments that are hard to understand, you disagree about something, or there’s something you need to discuss about how to best solve - you should answer the comments, or reach out to the reviewer so you can find a proper conclusion.

When you think the issues have been resolved, you merge the PR into the main branch yourself. The reviewer never does it for you.

You Aren’t Going To Need It - Yet

Gustav Wengel — Mon, 07 Sep 2020 00:00:00 +0000

You’re on a project. You’re developing an API to count the amount of apples in a cart. You expect to have it done, and people using it in 6 months.

Obviously this API needs to be documented, after all you don’t want your consumers to have to guess how it works.

You all agree, that when writing a new endpoint you’ll document it properly.

Four months later it’s looking good - you have a bunch of endpoints that are beautifully documented.

Oops, turns out someone had misunderstood something. You actually need to count occurrences of different fruits from the cart. And there’s multiple carts.

Well, that’s life. We’re agile. Requirements change.

So now it’s time to create some new endpoints. And document them. All over again.

In hindsight, the right call here would be to write the documentation at the end, when the API had stabilized. Could we have known this in advance?

When deciding when to perform tasks, there’s a few things we might want to take into consideration:

Can the task be delayed?
If we have someone who needs this done in six months, and it’ll take us six months to create, we obviously can’t delay it. In our example, the documentation probably wouldn’t take more than a few weeks at most - meaning that we could have delayed it until then.
Does completing the task create value right away, or only in the future?
Some tasks, like tests, create value the minute they’re created. Documentation for external consumers doesn’t. It only creates value when the external users start using the API.
What is the price to maintain the task, when requirements change?
I like to call this the upkeep. Different tasks have different upkeeps. The upkeep for maintaining API documentation for five endpoints is low. The upkeep for maintaining pages of tutorials, quickstart examples etc, is much higher. The higher the upkeep, the riskier it is to do too early.
How uncertain are the requirements?
Are we in a mature stable project, or in the wild west, in regards to requirements? The more mature and sure we are about the requirements, the less we need to be afraid of tasks with high upkeep.

The Right Task For The Wrong Time

Let’s look at a real-life example where I wasted quite a bit of time, doing a task at the wrong time.

I used to work at a project with reasonably complex cloud infrastructure.

At the start of the project, I spent a week or two writing scripts to automatically create the whole cloud environment.

Six months later as the project evolved, many of the cloud components changed.

Every time a cloud component changed, I had to change the scripts. Doing that could easily take a few hours, and at this point in time, we didn’t really use them very much.

Let’s try to use the questions to see whether or not we could’ve been smarter:

Can the task be delayed?
Yes. We did this because we figured it would pay off in the long run. We figured we would save time each time we had to provision a new environment.
Does completing the task create value right away, or only in the future?
It only creates value when we have to provision new environments, and as mentioned before - we rarely did that.
What is the price to maintain the task when requirements change?
It was very high. Provisioning scripts are impossible to test, and can take quite a while to run. Using a few hours every time a service changed was not unusual.
How uncertain are the requirements?
We made this decision in the start of the project, when we were very uncertain about the requirements and the cloud services we needed. If I’d been smarter, I would have realized that this meant I should have postponed this work.

In this case, we should have realized that the value gain wasn’t worth the cost. Particularly not when we were this early in the project. The right call would have been to build the scripts, when provisioning environments by hand happened so often, it became a major pain point. This might never have happened, and we never would have written the scripts. We would have maximized the amount of work not done.

So next time you’re about to start a large task, ask yourself “Is this the right time to do this?” - just maybe it’ll save you from doing it multiple times.

My publishing workflow

Gustav Wengel — Tue, 18 Aug 2020 00:00:00 +0000

I’m trying to get a little more structured when it comes to how I publish and syndicate the content I write.These are primarily my notes to myself about my process, but maybe you’ll find them interesting.

1. Write in Google Docs

I’ve tried a couple of different ways to write, including plain text editors, using markdown and using a collaborative editor like Google Docs.I used to author my posts in markdown, but I’ve since gone away from that for the following reasons

It’s much easier to collaborate and get feedback and reviews. A markdown file published to github somewhere is much less accessible than Google Docs, where reviewers can leave actual comments.
It’s just a little easier. In markdown you’re free to add spaces to the lines, but you also have to manual do line breaking, and you need a separate window open to check the output of your work.

2. Convert it into markdown

When you’re done writing in Google Docs, you can (reasonably) easily convert it into markdown, by using something like clipboard2markdown.It’s not perfect, and e.g. it doesn’t convert headings automatically. But it will get you 95% of the way there, and the rest doesn’t take that long.It’s primarily about adding heading, checking everything through once, and adding some images if needed.

3. Publish to own website

When I’ve then made my post into markdown, I publish it here at www.gustavwengel.com.It’s a Jekyll based blog which uses markdown, so when I have the markdown from above, I’m basically good to go.I use the jekyll-feed plugin to automatically create an RSS feed based on my content as well.

4. Syndicate To Other Platforms

The main reason I write, is because it helps me to learn and that I enjoy it. With that being said - I get immense joyfrom someone telling me that they enjoyed something I’ve written.Because of that, I also try to syndicate the content to some different platforms to get a bunch of different eyes on it.

I syndicate content to:

MediumSay what you will about the UX these days, but the Import Feature of Medium is very nice to use. Apart from code blockswhich the auto-importer doesn’t get, syndicating to medium is pretty frictionless.
Dev.toDev.to is a nice developer community. It gives you the ability to automatically import your stories from your RSS feeds as drafts.This functionality works extremely well, and I almost don’t have to do anything to make the stories work on dev.to

I’m also considered syndicating to Hackernoon, but I never really got around to switching over after they moved off, of Medium.I also don’t know how big their reach is anymore.

5. Post on social media

I’m part of a couple of communities on social media, primarily twitter, hacker news and the subreddits /r/csharp and /r/programming.If I think the post might speak to an audience in any of these groups, I also try to share them there.

On Inclusive Language In Tech

Gustav Wengel — Sun, 05 Jul 2020 00:00:00 +0000

Twitter recently announced that they’re beginning to change many of the words they use, to “more inclusive” variants.They announced this on twitter. Which of course meant everyone wanted to fight about it.

How much of this is just white silicon valley folks changing words they think are offensive? Is this cancel culture just with words? Does this actually matter or is it just virtue signalling?

The words twitter are changing, and what they're changing them to.

Honestly I’m not sure how much the inclusive language matters in regards to minorities. I haven’t heard many members of actual minority groups say that this matters to them.But considering that tech has a huge diversity problem, perhaps this isn’t so surprising - there aren’t that many people to speak up.

So let’s sidestep that discussion for a moment, and talk about another argument for making these changes. Most of the new terms are better!

At least if you don’t know the old ones already. People naturally prefer the terms they’re used to, because things we’re used to seem natural and right.

But I think if you’re just getting into tech, many of the new terms are better. They’re more self-explanatory and descriptive:

Blacklists and Denylists

Using blacklist and whitelist have the advantages of piggybacking on “regular” english words.But we’re creating technical terms here, so we don’t have to use existing words, if a better alternative exist.Using allowlist and denylist have some wonderful advantages. They’re self-contained and very explicit. The first time you read about an allowlist, you know exactly what it does.

Masters or Leaders

I particularly like changing the master/slave terminology to something that’s more nuanced. Often it seems like every relationshipwhere there’s a primary (or master) instance, we describe it as a master/slave relationship.But we can do better than that. If we describe it as primary/standby, we get extra information. There’s a primary instance, and the role of the follower instance is to be a standby in case of errors.Changing master/slave means that we get to use much more expressive terminology.

Dummies vs. Placeholders

I think placeholder value is a much better term than dummy value. It’s much more explicit.

You might think dummy value is pretty explicit already - but I’d argue it’s not.As an international developer, I had no idea that you could “dummy up” something fake.I knew that dummy could mean something that resembled the real thing, but I’d never made the connection betweenthat and dummy value.

It was just one of those terms I used because that’s what we call it._Contrast that with placeholder value - which says _exactly what it is.

A lot of these new words meet resistance. Partly because they’re new, and partly because people disagree on whether or not they’re racist, and if it matters.

I think we should try to keep the broader picture in mind.Generations of people are going to have to learn these terms and use these words in the future. We owe it to them to pick the best damn ones.

Even if it does inconvenience us slightly.

Death Spirals and Working Too Damn Hard

Gustav Wengel — Mon, 08 Jun 2020 00:00:00 +0000

Recently I’ve been thinking a lot about what I call Death Spirals. The name might be a bit much but I like the dramatic flair.

A Death Spiral is a special kind of positive feedback loop.A positive feedback loop is something that’s self-reinforcing.Examples of positive feedback loops:

When you have a lot of followers on social media, more people view and share your content. When more people share it, more people see it and follow you. The more followers you have, the easier it is to get new ones.
When you’re out of shape, exercise is harder so you’re less likely to do it. The less likely you are to exercise, the more you become out of shape, the less likely you are to exercise, and so it goes.

The Death Spiral

A Death Spiral is special kind of positive feedback with some unique characteristics:

Breaking out of it requires more resources than you currently have
The longer you’re in it, the harder it is to break out of it

Death Spirals can happen for many reasons and hit both companies and individuals. Let’s take a look at two examples.

Jane And Her Diabetes

Jane has diabetes. She hasn’t responded well to any medicine¹ and she’s feeling very sick.Her doctor tells her that exercise will help her.

Jane’s symptoms are so severe that after coming home from work she has no energy left to exercise.

Jane is in a Death Spiral. Her diabetes will continually get worse. As it gets worse, she gets and less energy for exercising.The longer she waits, the harder it becomes to break out.

JojaCo: The Burnout Factory

Death Spirals happen to companies too. One of my good friends used to work at a company JojaCo.JojaCo helped their customers manage technical infrastructure.

JojaCo employees were a little overworked. Then someone became long-term sick. Fewer people had to do the same work.To get all of the new work done on time, the employees had to cut some corners.

The company used to have documentation for each customer describing how to make changes in their systems.You usually updated this after interacting with a customer.This documentation slowly stopped being up-to-date as no one had time to update it anymore.

While not updating it saved time in the short run, helping the customer the next time took much longer.This meant that the employees got further behind, and had to cut even more corners.

In a stressful environment like this, people started falling ill due to stress. This left even more work for fewer people.At the end my friend was working 70+ hours a week.He was miserable. Everyone were sick or quitting, and new hires didn’t stick around for long.

Breaking Out of Death Spirals

So how do you break out of a Death Spiral? You somehow have to find energy and resources you don’t have.This can be extremely difficult, and if there is a step-by-step guide I haven’t found it.

These are my best bets:

Do Less

If you don’t have to do as much, you’ll have more resources to break free of the Death Spiral.

If you’re JojaCo you can call up some of your customers, and tell them that you can’t currently help them.They probably won’t be your customers in the future. But it might save you from losing other customers because your employees are burned out.

If you’re Jane, try to take some days off from your job and use them to start exercising.Or maybe you can do find a routine that requires less chores, or try some healthy meal box services.

“Do less” is advice that’s easy to give, but really hard to follow.It’s not always possible to do less. Your boss might not let you take days off. You might not be able to pay yoursalaries if you turn away some of your customers.

Get help

If you can’t reduce the amount of work, you might be able to share it among more people.More people to do the same work means there’s less work for everyone.

JojaCo could hire new employees or temps.

Jane could get someone to run with, or help her with some of the other responsibilities she has.

Getting help isn’t always easy though. Perhaps you don’t have the social support to find someone to run with.Maybe you can’t afford or find qualified people to work at JojaCo.

Death Spirals are particularly tricky because there’s often no easy way out once you’re in them.Often we keep struggling along, while hoping that something will change.

Perhaps you’ll wake up one day, invigorated with new energy to start exercising.
Perhaps your employee will stop getting sick from stress soon.
Perhaps things will improve somehow, even though you’re not really sure how.

I don’t think Death Spirals usually correct themselves until you hit rock bottom.We just stay in that slow, steady decline, because all the ways to break free are either impossible or requireschoices we can’t get ourselves to make.

So here’s my plea to you: If you observe someone in a Death Spiral - say something.Breaking free without help can be extremely difficult. It’s okay to ask if someone needs help.

Do you have any experiences with death spirals? How did you get out of it? I’d really like to hear some other opinions or stories!

I’m not actually sure if there are people who don’t respond well to diabetes medicine. However I have seen the situation play out in real life with people who were unwilling to take it. ↩

5 Reasons To Not Use Observables in C#

Gustav Wengel — Wed, 29 Apr 2020 00:00:00 +0000

When I was first introduced to Observables in C#, they sounded pretty damn good.“They just model streams of data”, “It’s just data over time” and “It’s just the push equivalent of an IEnumerable”.After working with them for a little while, I don’t think they’re as good as I was told.

The statements above might be true. Observables might be simple from a birds-eye perspective. Unfortunately simple doesn’t always mean easy and there’s are some things that will end up biting you in the ass.Here’s five of them.

The Framework Has No Consistent Name

When searching for documentation and solutions to problems with Observables, I always felt like I didn’t know what to search for.Previously I’ve always known the terms to use, django {your-problem-here}, serilog {how-to-do-x} {framework}-{problem description}

However with Observables I had a lot of problems finding people encountering similar issues, and I was never really sure what to put for the {framework} partof my queries.

I spent a little time trying to figure out what the right term is. Turns out - nobody seems to know.

On Stack Overflow the most used tag is system.reactive, but looking official documentation and Stack Overflow questions, I’ve seen it referred to as:

Reactive Extensions [1] [2]
Rx (short for Reactive Extensions) [1] [2]
Rx.Net [1] [2]
.NET Reactive Framework [1]
ReactiveX [1]

With all of these names that are subtly different - what am I supposed to search for?

We’ll call it Rx.Net in the rest of the post - at least it’s short

The Documentation Is Hard To Find

Finding documentation online for Rx.Net is pretty close to impossible.While figuring out the search terms to use is hard, finding comprehensive documentation is much, much harder.

Rx.Net is the .NET flavour of ReactiveX, which luckily seems to have quite a bit of documentation!Unfortunately you pretty much have to know what you’re looking for already.If I want to read the documentation for how the C# Select works, I need to know that in ReactiveX, Select is called Map

When I know that, in the ReactiveX documentation I can find a language-agnostic explanation of what a Map/Select does.There’s also language-specific documentation for each of the different flavours, including Rx.Net!

Just kidding

There’s also no API reference online anywhere. I can find the documentation in the editor, so I know it’s been written - but it isn’t hosted anywhere.The closest we get is this API reference from MSDN. From 2011.

Observables are hard to debug

Observables can be a pain to debug. Most debuggers aren’t particularly suited for tracing streams of data,and the stack traces you get are unimpressive. Let’s dive a little deeper into the stack traces. Take this pieceof code which throws an error after a few integers have passed through the stream.

[Fact]
public void ObservableTest()
{
   IObservable<int> observable = Observable.Range(0, 5)
       .Select(i => i * 2)
       .Do(i =>
       {
           if (i > 5)
           {
               throw new Exception("That's an illegally large number");
           }
       });

   observable.Subscribe(
       onNext: (i) => Console.WriteLine(i),
       onError: (err) =>
   {
       throw err;
   });
}

The humongous Stacktrace is as follows:

Error Message:
   System.Exception : That's an illegally large number
  Stack Trace:
     at MyProject.Test.ObservableTest.<>c.<Foo1>b__0_3(Exception err) in /home/geewee/programming/MyProject.Test/ObservableTest.cs:line 30
   at System.Reactive.AnonymousSafeObserver`1.OnError(Exception error) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\AnonymousSafeObserver.cs:line 62
   at System.Reactive.Sink`1.ForwardOnError(Exception error) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\Sink.cs:line 61
   at System.Reactive.Linq.ObservableImpl.Do`1.OnNext._.OnNext(TSource value) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Linq\Observable\Do.cs:line 42
   at System.Reactive.Sink`1.ForwardOnNext(TTarget value) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\Sink.cs:line 50
   at System.Reactive.Linq.ObservableImpl.Select`2.Selector._.OnNext(TSource value) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Linq\Observable\Select.cs:line 48
   at System.Reactive.Sink`1.ForwardOnNext(TTarget value) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\Sink.cs:line 50
   at System.Reactive.Linq.ObservableImpl.RangeRecursive.RangeSink.LoopRec(IScheduler scheduler) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Linq\Observable\Range.cs:line 62
   at System.Reactive.Linq.ObservableImpl.RangeRecursive.RangeSink.<>c.<LoopRec>b__6_0(IScheduler innerScheduler, RangeSink this) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Linq\Observable\Range.cs:line 62
   at System.Reactive.Concurrency.ScheduledItem`2.InvokeCore() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Concurrency\ScheduledItem.cs:line 208
   at System.Reactive.Concurrency.CurrentThreadScheduler.Trampoline.Run(SchedulerQueue`1 queue) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Concurrency\CurrentThreadScheduler.cs:line 168
   at System.Reactive.Concurrency.CurrentThreadScheduler.Schedule[TState](TState state, TimeSpan dueTime, Func`3 action) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Concurrency\CurrentThreadScheduler.cs:line 118
   at System.Reactive.Concurrency.LocalScheduler.Schedule[TState](TState state, Func`3 action) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Concurrency\LocalScheduler.cs:line 32
   at System.Reactive.Concurrency.Scheduler.ScheduleAction[TState](IScheduler scheduler, TState state, Action`1 action) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Concurrency\Scheduler.Simple.cs:line 61
   at System.Reactive.Producer`2.SubscribeRaw(IObserver`1 observer, Boolean enableSafeguard) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\Producer.cs:line 119
   at System.Reactive.Producer`2.Subscribe(IObserver`1 observer) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\Producer.cs:line 97
   at System.ObservableExtensions.Subscribe[T](IObservable`1 source, Action`1 onNext, Action`1 onError) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Observable.Extensions.cs:line 95
   at MyProject.Test.ObservableTest.ObservableTest() in /home/geewee/programming/OAI/MyProject.Test/ObservableTest.cs:line 25

Removing all the Rx.Net internal stuff, this is the only information in the stack trace we care about:

Error Message:
   System.Exception : That's an illegally large number
  Stack Trace:
     at MyProject.Test.ObservableTest.<>c.<ObservableTest>b__0_3(Exception err) in /home/geewee/programming/MyProject.Test/ObservableTests.cs:line 30
     at MyProject.Test.ObservableTest.ObservableTest() in /home/geewee/programming/OAI/MyProject.Test/ObservableTests.cs:line 25

We get the line number where we subscribed to the observable, and the function inside the chain where the error was thrown.I have no idea what transformations the data has gone through.If the stream has been dynamically constructed I might not even know what it looks like.

These issues aren’t unique to observables. Composing long LINQ-expressions suffer from the same issues.When composing several different functions together, the stack traces very quickly stop becoming meaningful.The below is the complete stack trace the Enumerable/LINQ version of the Observable code.

  Error Message:
   System.Exception : That's an illegally large number
  Stack Trace:
     at MyProject.Tests.ObservableTests.<>c.<TestLinq_StackTraces>b__2_1(Int32 i) in /home/geewee/programming/MyProject.Test/ObservableTests.cs:line 55
   at System.Linq.Utilities.<>c __DisplayClass2_0`3.<CombineSelectors>b__ 0(TSource x)
   at System.Linq.Enumerable.SelectRangeIterator`1.MoveNext()
   at MyProject.Tests.ObservableTests.TestLinq_StackTraces() in /home/geewee/programming/MyProject.Test/ObservableTests.cs:line 60

It’s much cleaner, but it doesn’t give us any more information than the observable version.

The poor debugging and stack-traces are a perpetual thorn in my side when working with Observables.But if the same issues exist with long chains of Enumerable, why is that less of an issue? The short answer - I don’t know.

My best guess is that when using Observables there’s a strong tendency to keep everything as an Observable stream.The longer your streams are the harder debugging with a minuscule stack trace becomes.

It’s Hard To Use Scopes

A very common thing I need to do is attach some sort of context to a request or a series of events.

var id = "myCoolId"
using (LogContext.PushProperty("id", id))
{
    // Every log statement in here will have the `id=myCoolId` attached 
    await DoSomething(id);
}

This is a very common usage patterns for logging, for example in Web Apps if you want to attach a specific GUID to a Request, soyou can later correlate all logs for that specific request.

Something like this isn’t possible when using Rx.Net, as it’s threading model doesn’t carry over the ExecutionContext. [1] [2]

This means that we can’t use something like an AsyncLocal or a ThreadLocal to keep context for a specific piece of data or stream.If we want the myCoolId to be attached to all of the logs, we’ll need to pass it into every step of our observable chain, and manually pass it to every logging call.I know there’s a value in being explicit, but this is a time where choosing Rx.Net locks you out of some pretty handy language features.

It never really took off

Looking at the timelines, it seems like Rx.net in the hype cycled peaked a few years ago. If it ever really took off.

Google Trends for system.reactive compared to the Javascript version of the ReactiveX framework.

Comparing it in google searches to RxJS it seems much less widely used. Looking at the Stack Overflow trendsreveals a little more nuanced picture however:

Stack Overflow trends for system.reactive compared to the Javascript and Java versions of the ReactiveX framework.

While still much less popular than RxJS and the Java ReactiveX version, Rx.Net does seem to predate their popularityby quite a few years. Microsoft was out early with the reactive paradigm but never really seemed to manage to make it take off.

Now while your technology choices shouldn’t be about how hyped something is - the popularity is always worth taking into account.Some old technologies are sturdy, battle-tested and well-documented. Then they don’t have to be shiny.

However with some of the pitfalls, particularly around the documentation issues, Rx.Net feels neither robust or shiny to me.

While this is a negative article, I’m not saying Rx.Net or the Reactive paradigm is always bad.Some of the issues are about the tooling and documentation.I mention that most debuggers aren’t suited for debugging streams, but this is a tooling problem and not inherent to the reactive paradigm.E.g. Jetbrains has an excellent Java Stream Debugger.

Observables seems like a very natural fit for some languages and domains, like reacting to user interactions in JavaScript, as the large adoption of RxJs suggests.I’m just saying that it doesn’t feel like a particularly natural fit for C#, and if you have some data that can be modelled as either Enumerables or Observables,I would think twice about using Observables.