loading...
Cover image for Why you should not use (long-lived) feature branches

Why you should not use (long-lived) feature branches

jpdelimat profile image Jean-Paul Delimat Updated on ・7 min read

Isn't the git history in the picture above nice to work with? There are probably many problems in that repository, but one of them is most definitely the use of feature branches. Let's see how such a bad thing became such a common practice.

When git came to replace SVN it brought a ridiculously easy way to create branches.

The idea behing git is to ease your pain as a developer working on many features at a time. Not to push branches around and tie them to the whole development process of your team.

When JIRA and others came along , companies like Atlassian started to heavily promote the "git workflow" and feature branches. The "Create branch" button appeared in your JIRA tasks and boom, feature branches were born! Atlassian tells you all about it in this interesting article. I like Atlassian's products very much. Keep in mind though that their core business is tasks management for development teams. The more tangled it gets with branches and code, the better.

Ten years later, feature branching is a standard in most teams, when in fact it doesn't bring any benefits to your bottom line: release quality software to production. Not only feature branches provide zero benefit, they actually slow you down!

For the sake of clarity: this article assumes a feature branch will carry the whole feature you are developing and is a so called 'long-lived' feature branch that will last 1 week or more. It's not a "no branches at all" mantra.

"The feature is ready. I just need to merge it!"

I've heard this way too many times. This falls in the same category as statements like "It compiles so it works".

In practice the merging leads to the "unpredictable release syndrome". It can be quick or evince a major incompatibility, which needs fixing in a rush. You are either lucky or … your timeline shifts and code quality drops.

The real problem with feature branches is the reason they are so popular: they pump a developer's pride and make you feel good about your work.

Your feature branch is your own perfect garden and you can keep it clean and shiny. But it is separated from the other gardens of your team. The more you work apart, the harder it is to reconcile.

I am a big fan of the management book "The goal". It shows how over time people tend to use metrics that highlight local optimums of a process because it is more comfortable. They just lose focus on their global bottom line. The book is about a production plant, but the analogy stands. Your feature branch is a local optimum with high quality code. It may also be so far off the main branch that it is of no use for the upcoming release.

Trunk based development to the rescue

As the name suggests, in trunk based development the whole team pushes continuously to the main branch or use very short lived (1 or 2 days max) branches.

Here is a detailed description of the idea. I have no affiliation with the linked website. It is just a great overview of the concept.

When you push your work to the main branch often, the amount of code to merge is way smaller and it becomes trivial. There is a far greater benefit though: you and your team can spot problems before they become painful. It might be that your refactoring clashes with another feature. Or you are drifting off from the project conventions or architecture patterns. This is the real value of the process. As I preach in any place I find myself in:

It is teamwork that makes or breaks software projects.

Working days on code that will never get to the release on time is the biggest failure there is for a team.

Another upside of pushing or merging often to the main branch is that your changes will run live in some environment. It is always good to deploy and battle test your code, even in progress, in some real deploy.

"WIP" in the main branch?

If you read so far you are probably thinking "This is crazy, how can I push my half done work to the main branch when it will probably get deployed to production very soon!?!".

Here are the common objections one might have and a tentative solution.

How can I expose unfinished work?

Use feature toggles. They can be environment variables or whatever suits you best to turn on and off your work in progress. Make it defensive of course so your half finished code does not get active in production by mistake.

Your whole team will love this: you can activate code on any environment at any time to see how it looks or performs. Testers can prepare testing early on. Product owners can comment on your work along the way. It is all live and easy to access for everyone! If your work is like just started this provide little value. But evil lies in the details. It usually takes half the time to get to 90% completion and another half to finish the remaining 10%. Sharing your work in this state of 90% completion is always a good idea ;)

Another thing that comes for free: you can turn the feature off in production if a problem arise after deployment. After a few days or weeks, once the feature runs smoothly, just remove the toggle from the code.

What if I break the main branch for everyone?

It is 2019. If you don't have a continuous integration setup that builds and runs tests automatically … then set it up yesterday. If you break anything you'll be notified before it becomes a problem for the whole team. In pure Trunk Based Development the feedback will come after the merge and has to be fixed right away. If you are using short lived branches the merge should be blocked by your CI tool. A short lived branch is something that should last 1 or 2 days max and carry a consistent piece of code that contribute to the feature you are building.

There must be a code review before we merge anything in!

That's a valid point. Code reviews do not need feature branches though. If the code review culture is strong in your team then it can very well be done on the commit to the main branch. The reviewer would stop by the author of the commit and discuss what needs to be fixed. The fix would come in another commit. Even better, have the code review together before pushing the commit in the first place.

If it is not acceptable to your team to have post merge code reviews (because let's face it it is less handy as tools do not really support that), use short lived branches and apply your code review process there.

I want to see the code that is related to a task

If you have a given branch per feature then it is easy to track code back to your agile board. You can navigate from a task to the branch that implements it.

It sounds cool, while in reality this is useless! How many people can you have on an agile team? Up to 5? Up to 10? How hard is it to ask the guy running a task or story what commits you need to look at to deep dive into the implementation?

After some time, once tasks are completed for a long time, linking tasks to code does not make sense anymore. Developers rely on git blame to know the who, on code content to know the how, and hopefully on comments to know the why.

The cherry on the cake: opt-in on new features

It became common to see major UI features or updates released using an "opt in" approach. Github, Bitbucket, Gmail, … to name a few.

The concept is that major changes are introduced using a banner "Hey we have this new feature / improved dashboard / whatever. Click here to try it out". You can opt in, and usually opt out as easily if you don't like the change. This is a very good adoption testing strategy as it involves the end users in the decision process. If people opt in and stay there it means you are improving the experience. If they opt in and out … you know you've changed things for worse.

If you are using feature toggles from the start, exposing these on a per user basis at run time becomes very easy.

Conclusion

If you never thought of trunk based development as an alternative to the feature branch mantra, I hope this article gave you some perspective and will to try it out. 

The best thing is that to get there you'll need to setup or improve every other aspect of your process (CI, automated tests, code reviews). This is a good path to take. We obviously recommend Fire CI as a continuous integration tool.

Remember that your bottom line is to put quality software in front of your users. The closer your code is from the production environment at all times, the better.

Now, although the article is very much "trunk based development" oriented, note that it might not be a valid approach for your team.
If your team is highly distributed, in different time zones, has a lot of junior developers who need to learn the project conventions and architecture, using longer lived feature branches might work better.

The main idea in this article is:

The faster you integrate the different pieces together and check that things are working, the safer you are to have a working product at the end.

A good common ground is to use short lived branches that last 1 or 2 days max and merge them to the main branch. This way you can human control what gets in and still integrate code fast.

Originally published at Fire CI Blog on March 30, 2019.

Posted on by:

jpdelimat profile

Jean-Paul Delimat

@jpdelimat

Husband, Dad, Developer

Discussion

markdown guide
 

Hi Jean-Paul, you make a good case but you can still use feature toggles even with feature branches and also all devs working on the master branch means that the build might be broken for a long time in some cases.

The biggest drawback though (which is mostly the reason GitHub and the others encourage feature branching) is having to deal with the increasing async nature of our work. More and more companies are distributed and some are even async.

It's doable to work on the master branch if all devs can talk to each other in a room or in a chat room in real time. "Sorry, I broke the build, I'm going to fix it" but what if you have multiple set of developers working on different timezones that may or may not overlap?

The build could be broken for hours if someone is unavailable or asleep.

This already can happen with important bugs that need to be fixed ASAP, adding the load of fixing a feature written by someone else because it impedes the work of those devs that are "online" seems an unnecessary burden.

It'll lead to commit messages like "commenting out feature X because I have no idea how it works but it's breaking the build". :D

I think it's a valid idea, I'm just not sure it works best with fully distributed and remote teams.

 

Hi Rhymes,

First of all thanks for reading!

You make a valid point that "all devs working on the master branch" might no apply to all setups. You could be remote, async or even have newcomers in the team who need more structure and guidance before starting up and being up to the standard.

The way to mitigate that is to use branches, but not feature branches a.k.a "long lived branches". You would create a branch per logical piece of code that you want to merge in. Then merging goes through a continuous integration tool that lets the code in only if it does not break the main branch. You could also open a PR and have this go through a code review if needed.

The main idea remains: you diverge from the master branch for a short period of time: say 1 day max and then sync back in. This way you eliminate big merge issues and avoid the "unpredictable release syndrome".

Cheers,
JP

 

The main idea remains: you diverge from the master branch for a short period of time: say 1 day max and then sync back in. This way you eliminate big merge issues and avoid the "unpredictable release syndrome".

Agreed! Short lived branches are what I interpret feature branches for. For a feature, as small as possible. I once worked in a place where every dev had their own long lived branch and multiple feature branches and it was a bit of a nightmare as you wrote about :D

Yeah when I think of feature branches I think of something that's around for a week or two at most. Committing stuff to trunk is hard because you can't make small, incremental commits that might be a bit sloppy. Having a feature branch can make saving your state less stressful because you don't have to worry about others seeing your commit that says "oops I broke this now it's good".

Yep, also confused here somewhat, always thought of "feature branches" as being around for anywhere from minutes to days at worst. GitHub has no mechanism for code-reviewing individual commits, so I don't know how you'd do reviews on the main branch anyways.

(For longer-term branches, I had thought the term was "project branch", which is a collection of feature branches.)

Thanks for the comments!

I have updated the title and added this to the introduction: "For the sake of clarity: this article assumes a feature branch will carry the whole feature you are developing and is a so called 'long-lived' feature branch that will last 1 week or more. It's not a "no branches at all" mantra."

 

You seem to be referring to people who use feature branches incorrectly.

Some notes:

  • Feature branches, in general, should be small and short-lived.
  • you should regularly merge main into your feature branch, to not have a surprise at the end
  • You should merge main into your feature branch, and run all automated tests, before merging into main

Monster branches are discouraged in all pipelines. But they are not a reason to avoid feature branches.

Doing code reviews after a branch has been merged seems odd. What if the review says the entire approach is wrong, or requires other significant changes?

Your approach muddies the master branch, and increases the instability of it.

Feature toggles are also not incompatible with feature branches. These are orthogonal concepts.

 

I've updated the title and intro to specify I am talking about "feature branches" as "long lived feature branches". By long lived I mean more than a day or two. Usually when I see "feature branches" used, "feature" is a feature of the project or the product. It is rarely developed "shortly" and sometimes require a week or more to get in. Working on a branch like this is not efficient in my opinion.

I don't see how feature toggles can be used with feature branches though. If your feature is only on the branch and this branch is not deployed to production, how do you use feature toggles to turn your feature on and off?

 

I don't think I'd limit it to a day or two, sometimes a small feature can take longer. It's somehow related to the size of code, and certainly, something taking weeks, has likely gone off into la-la land.

Feature toggles can still be used on immature features that need real-world testing. You don't get to toggle features which aren't in the master branch. I've used these before. Part of the feature includes a toggle mechanism. These should be accompanied with a clear plan of when it becomes the default and the old system is deprecated.

However, if you follow the advice of regularly merging main into the feature branches, you should be able to selectively merge several branches to get a working version. I've done this on several projects as well. Sort of a mix-and-match of branches instead of the feature toggles. This definitely requires regular main-to-branch merging though, otherwise you'll hit all sorts of errors.

There is something that always bothered me with feature branching and seeing that you seem pretty convinced of them maybe you can solve this for me.

Developer A and Developer B branch out of master at the start of the sprint to do their job and both pull and merge from master frequently but no one is pushing to master so, how do they know they are not breaking each other stuff?

Keep the branches short-lived and avoid working on the same parts of the code. Avoiding the same parts of the code applies to whatever branching strategy you are using.

There's rarely a need for two people to work on the same bits of code in the same period. This can usually be recognized during triage and planning.

In the unusual case where two people are working on the same code, it should be for the same feature, in which case, they can share a branch. And they should probably sit close to each other in the office and communicate.

The moment one of them feels the need to pull from the other's branch, the warning sirens should go off. It sounds like a monster branch is growing.

 

Hi, thanks for the article.

In my opinion, there is a huge advantage to the feature branches - and that's the visibility in the history. There are reasons to look in the history (like a year ago) of the cvs, when trying to explain why something is implemented that way or the other. And then, if you see the work on a branch, you know the whole context.

Also, if you're worried about the changes on the trunk that might affect the feature branch work, just pull the develop branch and rebase on top of it - as frequently, as you're suggesting to merge. I'm of the opinion that on your branch you can modify that history however you want.

Moreover, if you think you're able to commit every time unfinished work to the develop - maybe the scope of the "features" is just too large. What about splitting it up, and then calling these chunks a feature?

Thanks again,
Krzysiek.

 

and that's the visibility in the history.

If you're not using something like GitHub where this is no notion of pull requests then I completely agree that this is a valid point that need addressing. If you're using GH however, you have the complete history of the branch in a Pull Request.

I suggest taking a look at slides.com/lirantal/effective-git-... to see if you get convinced with trunk based development :)

 

Looks like you worked with feature branches incorrectly perhaps. I prefer them (at least for small distributed teams) 100%. If nothing else than for the reviews. In my experience if everybody commits it becomes a huge mess soon, and the "master" don't work most of the time...

With trunk based devel you need a new look on how to do reviews effectively, and I don't know yet good and available tools for small teams to do that. (I am aware of big companies overgrowing feature branches and having good tooling for trunk based.) Suggestions?

 

Long lived branches miss the lessons which led to continuous integration. The rebase workflow provides a means to do continuous integration on long lived branches as long as all work isn't long lived at the same time. It also puts insentive on the branch maintainer to find a point to merge in their work early.

 

Your feature branch is your own perfect garden and you can keep it clean and shiny. But it is separated from the other gardens of your team. The more you work apart, the harder it is to reconcile. ... Your feature branch is a local optimum with high quality code. It may also be so far off the main branch that it is of no use for the upcoming release.

I don't quite understand what's going on here. Ideally, work in small chunks on feature branches that have short lives. Rebase early and often. Never submit a pull request that hasn't been rebased against master. Then rebase (and squash) it again on approval. There shouldn't be a significant span of time where you're off ignoring what's going on in the rest of the project.

Granted, I've been guilty of monster submissions that add 1000s of lines and need a week to review. But that should be a wild exception, and I kept on top of rebasing and conflict resolution every day.

IMO, the problem with committing straight to master and then reviewing it and then cleaning up after is that it causes a mess. While it's on a feature branch, you can edit history, squash, revise, amend - everything's hypothetical until you finally land it. But, once it's landed, it's real and everyone has to deal with it - including the after-hours breakages to build & test systems that might come up.

 

"work in small chunks on feature branches that have short lives"

Agree on some part: I have updated the title and article to mention short lived branches are OK.

Disagree on the other :): as the name suggests, "feature branch" are supposed to carry a whole feature. Unless you are making a very small change or fixing a bug, this is probably work for 1 week or more. Working on your own that long is a risk.

"But, once it's landed, it's real and everyone has to deal with it"

Agree. And this is true whether you write your code in 15 minutes push and go on vacations, or spend 3 months revising and amending all over until it's perfect. And the more you wait, the bigger the risk. While if you think about "what are the contact points between the existing code and my new code" in a feature toggles kind of way, you will build an abstraction layer around your feature that will make it easier to integrate every step of the way.

What do you think?

 

Great writing and appreciate you advocating for TBD.
I've been running that for several years and have been able to successfully introduce it on several engineering teams I was leading.

If you fancy some slides on the subject to introduce this to new teams I have given this one on several meetups and conferences: slides.com/lirantal/effective-git-...

 

Usually people comment because they disagree. In this case I'm posting because this is heterodoxy while also the way I feel about feature branches. The whole idea of a version control system is to allow the team to work together. This means being synchronized in terms of the code base. Having people run off on their own branches and mark stories as complete based on a branch creates a problem with knowing what is actually included in the main code base. I understand why Git and feature branches might make sense in highly distributed or open source teams where features are highly atomic and developers work more independently, but in co-located teams working closely with one another feature branches lead to endless discussions about which features are actually merged, and conflicts over merges. Evolution means small regular change. Branching is essentially a fork which will require healing.

 

I think the title is a bit misleading. With Trunk Based Development you still have branches but they are short-lived and intended to be integrated very quickly into the main line of development, as opposed to the traditional feature branches that usual span for weeks or months until a feature is "complete".

 

"I think the title is a bit misleading" Indeed. I've updated the title and added mentions of short lived branches where appropriate. Hopefully it's clearer now.

At least the original title brought in a lot of interesting discussion :)

 

Marking an issue/story/req complete in a branch is also a bad workflow. They should not be marked as complete until they are merged into the master branch, or whatever the developing branch is.

GitHub even has this feature automated. If you mark branches fixing issues, those issues will be closed once they are merged.

 

the idea of feature toggles never clicked with me for some reason. maybe because I'd have to test that my code doesn't work unless I enable it. it's trivial if I change the old behavior with a shiny new one but how do I test that my buggy new feature is completely unavailable and cannot be accidentally called by something else?

 

Feature flags (or toggles) are essential for continuous deployment and quick delivery because they allow you to manage things like canary releases and the ability to "test in production" ( 🤯). Obviously you should actual test and choose a well designed feature flags solution but at the end of the day and for the sake of simplicity you can think of them like a code branch:

if (ff.shiny_new_thing === 'on') {
  doThisNewThing();
} else {
  doThisOldThing();
}

You could also argue how those would be tested and while that's not always easy it's definitely doable. Another way to think of feature flags is that they are merely configuration items.

 

Interesting article, I just think the 1 or 2 day limit is a bit tight. I love to get back to master as soon as possible, but at the same time I don't think uncompleted code should be merged to master. Feature toggles can help with that, but only if it is a new pathway. If you are refactoring or modifying existing code it may not be possible to feature toggle it.

We run a simple git flow with a master branch and a branch for each work item or bug we work on. The code merges to master as soon as the work item is code completed and dev tested. The intention is that no work item should take longer than a week.

You're right about any branch that sits around for too long is never going to make it to master.

 

You mentioned "The Goal" and WIP as reasons to go to the Trunk based. If you haven't already, check out "The Phoenix Project", it draws a lot from "The Goal" based on a DevOps environment.

 

Thanks Josh. Downloaded on my Kindle and on to start reading it tonight!

 

Awesome I will be interested to hear the perspective of reading "The Goal" first as I am reading them the opposite way. 👍

 

As a fan of continuous integration I am obviously not a fan of feature branches, and thus agree with your article

Dave Farley wrote a similar article on this:
davefarley.net/?p=247