As more companies strive to deliver software faster it becomes clear what legacy processes are slowing them down, one of these is Feature Branching.
The use of Feature Branches separates work, increases the size of code changes, and prevents Continuous Integration. You may need to reconsider the use of feature branches if you:
- Keep getting stuck with large merge conflicts;
- Are racing other developers to merge code; or
- Are not committing to your main branch in git.
Why are they Incompatible?
Feature Branching is the process of creating a branch per feature and is used in Gitflow https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow. The process of using a feature branch is:
- Create a Feature Branch for a new feature.
- Make changes until the feature is complete.
- Merge the feature branch into the main branch.
Continuous Integration is the process of merging all developers’ code changes frequently https://www.thoughtworks.com/continuous-integration. It can be followed by committing changes to a single branch. This process is:
- Make a code change.
- Commit the change to the main branch.
These two processes are incompatible due to the difference in change duration. Continuous Integration is expected to be done at least daily, whilst a “Feature” is usually expected to take many days and sometimes weeks to complete.
On one hand, if you were to create Feature Branches daily then the developer’s work to constantly create branches and merge them is a waste.
On the other, if you were to leave Feature Branches separate for multiple days then you cannot follow Continuous Integration.
This is why Feature Branches are not compatible with Continuous Integration.
So …
What is the Alternative?
Trunk Based Development is the process of using only a single version of the code. All developers can follow this process by not using branches.
To embrace Continuous Integration and take steps toward Continuous Delivery the Feature Branch habit must be stopped.
Next time you git checkout -b feature-xyz consider breaking that habit and allowing yourself to deliver better software faster by Continuously Integrating your code instead.
If you would like to discuss this further please contact me on Twitter @BenTorvo or Email ben@torvo.com.au
FAQ
Q. What are branches for if not used in the regular development lifecycle?
A. Branches can be used for “untrusted contributors” that want to suggest changes to the code base.
Q. How can you make changes without breaking the whole codebase?
A. One recommendation is to use Feature Flags to allow half-implemented features to be accessible when required or only by authorized users.
Top comments (42)
While I understand the benefits of trunk-based development, I'm still waiting for someone to explain how it's supposed to work in larger organizations. And how code review should look like.
Also, regarding the argument about continuous integration - you are supposed to rebase main branch often, so you are actually continuously integrating changes into a feature branch. And as such, it can be considered as short-living.
Trunk based development works in large organizations by splitting teams code bases by the services they manage and allowing teams to manage their own code review. There isn't a huge difference between a tiny organization and a single team within a a well structured large organisation.
I wouldn't recommend rebasing as it modifies history to be inaccurate, merging is generally a better approach. If you are continuously integrating enough that is always consistent then there is no point having separate branches.
I think you misunderstood what continuous integration is:
Nowhere does it say "branch". CI is infact very compatible with gitflow and infact gitflow is entirely designed towards CI. Feature branches are part of the shared repository and developers are encouraged to make regular commits (check-ins) to their feature branch. Your pipeline can be setup to verify and automatically build each feature branch, so you can detect problems early. When the feature branch is ready it gets merged into a developer branch. This dev branch is then also verified and built, if the build fails, it is easy to revert the change. Gitflow allows you to detect merge conflicts early.
With trunk based development if there's a conflict or a build error, your pipeline fails and someone has to undo the commit, assuming you can figure out which commit is the actual problem... What happens if two developers make commits to your trunk one second after the other and the pipeline wasnt finished yet? How do you figure out which commit is broken? How are you going to fix it? All the time you spend figuring this out, nobody can make commits because the pipeline fails... In gitflow if the dev pipeline fails, you simply revert the merge and everybody can continue happily working on different features. Or am I missing something and do code conflicts magically not happen in trunk based development?
Hold on, the definition we are working from says
"developers to integrate code into a shared repository"
Just having a branch in a shared repository does not mean it is 'integrated'. If you have three people push their work branch into a shared repo, the work was not integrated. That is the point of this discussion on Trunk based flow.
There are issues with branches, and feature flags, and rebaseing, and any other collaborative strategy. What we don't want to do is claim our side is doing something it is not.
That entirely depends on where and when you run your integration tests. If you run integration tests for each pull request, the work will be integrated with whatever is the current status of the branch they're integrating into.
I definitely don't want to claim my side is doing something it is not. Ofcourse it sucks when your pull requests integration tests fail because another feature was merged into the source branch first and the changes somehow affect your work (usually an indicator someone went out of scope). As far as I understand trunk based development has this same issue. My issue with trunk based development is that when this happens you cannot release or continue development on other features.
But it is not integrated with other people's work continuously. That is the point. I don't agree that we need to go to that extreme, but if you have long lived feature branches you aren't doing continuous integration (only with one long lived feature branch at a time)
Trunk based dev reduces conflicts because you're more likely working in the latest modified code. With branches people can go days without pulling in mainline. The issue is less with the branching model and more to do with the people.
Even git flow does not endorse long lived branches, but that still happens.
But why do you want to integrate with other peoples work if their work isnt even ready to be integrated into the source branch yet? That doesnt make sense to me.
Trunk based development wants to integrate with each and every commit... won't that discourage people from making regular commits? They'll feel forced to only commit when they're completely finished with a task and are already pretty sure that their code wont break the pipeline.
I'd prefer if people made commits to the feature branch as often as possible even with "unfinished" code and pull in the source branch into the feature branch as often as possible in order to avoid conflicts, rather than postponing their commit because they're scared it won't integrate with the trunk yet.
Because adding new functionality to the code base that isn't used in production isn't that hard and it reduces the time to integrate which reduces complexity and wasted work.
No, it is just a different style of development, accepting that there is no such thing as a complete start to end finished task in one commit. There is also nothing wrong with tests failing in the pipeline. It is more important to improve quickly than guarantee green tests (which can't be done anyway).
That doesn't help integrate their branch into the mainline which is what continuous delivery is. That fear and worry goes away if you try it and have a reasonable work environment.
I don't see where you think I said CI requires a branch either? I recommended a way to follow Continuous Integration after providing the definition and a link to additional material.
Branch builds doesn't work well in practice since it either results in one environment per build which is impractical or no actual deployment which means you are only testing a local copy of the code.
Unfortunately undoing commits is another bad practice that should be avoided. When a pipeline fails you should fix the code, not undo the work that was already done.
"What happens if two developers make commits to your trunk one second after the other and the pipeline wasnt finished yet?" The pipeline runs once for the first commit and once for the second in order. If the first fails then it was the first, if the second fails then it was the second. You fix it by looking at the change, fixing the code, and integrating the fix into the code base.
The whole premise of Continuous Integration is that when a developer breaks a build they and anyone else who wants to fix it do so immediately. It started with developers using a single shared machine to run the builds, only one person could use it at a time, and it had to be working when they were done.
Code conflicts are so tiny and dealt with so quickly in trunk development that it seems like magic.
There is a reason that the Gitflow document (atlassian.com/git/tutorials/compar...) explicitly states that:
"Gitflow has fallen in popularity in favor of trunk-based workflows, which are now considered best practices for modern continuous software development and DevOps practices. Gitflow also can be challenging to use with CI/CD."
No? I mean... how you setup your build pipeline is entirely up to you... Atleast for web development its pretty easy to add a deployment step to your build pipeline that puts the compiled code in an environment that's a reflection of production (e.g. running the same services (or copies of them) that are running in production). You can control which environment you deploy in by adding environment flags in your code, like dev, staging, prod or whatever works for you.
Yes, thats nice in theory, but in practice I've seen many occassions where the dev team was not able to fix the code in a timely fashion and had to revert, in order to be able to continue other work.
That sounds like a blocking proces to me... so now feature 1 cannot be released because feature 2 has broken the pipeline and it needs to be fixed first... In a gitflow-like process that is non-blocking, if the pipeline for feature 1 succeeds it can be released, regardless of whether feature 2 pipeline fails. The way you're describing trunk based development to me right now, sounds like feature 2 would prevent feature 1 from being released, meaning failing to meet deadlines on ALL the work the team was working on.
I'll have to take your word for it... I don't see any proof of why that would be the case though...
You either have 1 environment per branch, shared environments (which is terrible for knowing what is deployed if you are using multiple branches), or aren't deploying your code. Which of these are you proposing?
I've done it in practice but I'm sure there are always teams that can't just fix their code when it breaks. Normally this happens when teams are making changes that are too big due to lack of automation and continuous delivery.
Yes it is intentionally blocking, and once that is accepted we can talk about how to continuously develop features that don't break pipelines using feature flags and other solutions.
Fair enough, I only write these articles to point at what I think we should be doing. Feel free to give it a try and let me know how it goes.
No? Your options are limitless when it comes to building and deploying your code... You're not limited to deploying into a shared environment, each branch can have its own space even though it interacts with other services that might be running in develop/staging/production... that entirely depends on how you set it up yourself...
I don't believe in perfect humans... Or perfect devs for that matter. I've seen pipelines fail for the silliest of typos... Everybody makes mistakes, if the whole dev team is blocked and has to fix the pipeline, everytime someone makes a mistake... I just don't see it ending well.
If you want to convince me of why trunk based development is better and why intentionally blocking development is better, you're going to have to explain how feature flags and "other solutions" are going to magically fix that.
I havent heard anything yet thats different from the horrible period that existed before gitflow :P
They aren't limitless, it is one of the three I've mentioned and I was asking so I know what you think the solution is.
I never said people are perfect but ignoring pipeline failures and delaying fixes or integration is worse than fixing them immediately. A typo is easy to fix since the person that made the change can just make another change.
Sure, we can get to that. This article was just about identifying that feature branching isn't compatible with continuous integration.
I think it is important to note that the people that wrote the gitflow document don't recommend it and propose trunk development as a better practice. Either they are saying gitflow was a misstep or trunk development isn't the same as what was before both.
The solution is to stop thinking there are only three options of deployment, in order to justify dismissing gitflow. I can deploy my feature branch just fine, infact I can deploy each seperate pull request for a feature branch and show the result to my team members or clients and it wont interfere at all with other branches or pull requests that are deployed. But most importantly I can run integration tests on each pull request seperately, without them interfering.
I agree it's sad that they dismissed gitflow and mention a replacement, without explaining how it is actually better. I don't really care what anyone says if they dont offer a better solution and nobody has explained to me yet how trunk development is better than gitflow. If something better comes along that doesn't block the dev process and the release cycle, I'd be happy to switch.
This is simply not true.
In my experience, this is the problem with async in general - a price to pay to have other benefits of async workflows.
Also, iIf we have timezones in play and two people have one hour of overlap, they can go through the code, one of them later applies the changes, but they have to wait almost full 24 hours to sync on the changes. Alternatively, with async PR workflow - they discuss in on the overlap, person A asynchronously makes changes while person B is off, when person B wakes up, they review the changes made (because they are available on the branch, not sitting in person A's computer) and if they are ok, they can merge it 12 hours earlier that in a synchronous trunk-based flow.
And just to clarify, I'm not against TBD in general. I just see a lot of problems wrongly attributed to using feature branches when it is discussed.
I'm not sure what you are saying? That the reviewer doesn't need to approve the PR before it is merged or the creator doesn't need to create the PR before it is merged?
The PR approver shouldn't be merging or deploying code changes in either case, they would need to fix any errors in the pipeline that they didn't write the code for.
I'm saying they don't have to look at the PR at the same time.
Why not? It's a common misconception about development. Codebase is our shared responsibility. I don't see any objective obstacles for me to merge a PR created by someone else (and other way around). I need to understand the change in order to review it anyway.
They both have to look at the PR before the code is integrated. This means that the process of modifying the code is blocked until that is done and hence is synchronous.
Because you as an approver have spent less time and effort on the code and therefore aren't the best person to respond to failures of the tests. You can see the code at the point of PR but cannot understand all the development work that went into the code unless you were pairing the whole time which isn't the suggested solution here.
That's not what "synchronous" means.
I also disagree with your second statement. This might be a rule if you want to make it so, but there is not a universal law. And definitely is not a native feature of branch-based development.
The literal dictionary definition is ambiguous and in software is used to mean sequential vs parallel tasks. A pull code change doesn't become asynchronous just because there is huge latency in waiting for approval.
Work has occurred and you weren't there for it. I'm not sure how you can disagree with that since the research done during work, design changes, and local code changes aren't all visible in the code change.
If you are familiar with Javascript and async programming, you likely noticed that code was processed serial, asynchronous, parallel, and not concurrently. It is similar in development.
You'll have serial process to develop a change, with concurrent development happening asynchronously in parallel. Wait, that does not help clear up the terms.
No, it doesn't become asynchronous because of that. It is asynchronous because it doesn't require attention of all interested parties at the same time - instead they can do their part in the fitting time slow. The term is not ambiguous, in case of async/remote work it always means that.
Well, as you see - I can. I mean, I don't disagree with what you said - of course these things are not visible in the code. But that doesn't mean I can't merge the PR.
I'm kind of tired with all this inventing new meanings of the words, just to bash branch-flow. It's not the kind of discussion I hoped for.
(Replying here, because replies to previous comments are closed - is it engine feature or deliberate action?)
The rule that, by default, merge of pull request should be done by a person who created it, is proper because that person has been most thinking on code and so sees all its peculiar: decisions made, peculiars, tradeoffs, etc. Reviewers, normally, shall approve or disapprove, but don't make final decision to merge. Anyway, the system keeps proposals and allows to get them later on.
There should be cases somebody other merges a proposal (pull request, etc.) but under special circumstances (author is on vacation / gone / sick, or there is a flaming need to merge for a release and team lead takes ownership on the merge).
For desynchronization issue (typical e.g. between USA, Europe and India team), well, in a normal situation, 1-day delay is not critical under well-established processes. But what I consider more important that
1) There shall be strict requirement on regular time band dedicated by each developer for review process. For example, up to 2 hours each working day when there are pending reviews. Only critical issues should be more important than peer review.
2) The need to review shall be easily checked by a developer. (Mail, dashboard, etc.)
3) Developer tools shall be adapted to alleviate review process as much as possible, including such cases as commit sequences, proposal versions, CI assistance.
To say that creating and merging a branch is "work" completely misses where time is spent.
Naming the branch - what are you working on?
Code review - review takes no time if you already pair programmed. A Pull Request/Merg commit provides a collection of related work.
Long lived branches create additional maintenance, and sometimes that is reasonable for risk management.
Git Branch Strategy
Jesse Phillips ・ Jan 13 '20 ・ 2 min read
Naming each change of code is wasted effort and the more frequently you commit changes the worse it gets. This becomes apparent when you are changing a couple of lines at a time and pushing multiple times a minute.
If you are pair programming the review is already done and the PR isn't necessary at all. Also, true pairing on one pc doesn't make it easy to have both people logged in for creating/approving PRs.
As the size of changes increases so does risk which is why long lived branches are not a good solution for risk management.
I sure hope you provide a commit message which names your changes, no matter how frequently you commit. I also hope those changes are complete (compile, expected to pass testing) especially when being pushed.
I'm going to combine your claim of 'multiple times a minute' with other statements of 300 devs. If 300 devs are pushing changes multiple times a minute, the amount of time rebaseing to get the latest so you can push seems like a waste and is exactly what "racing other developers to merge code" is.
Here is the thing about git, you can't do any work without making a branch. A repo clone finishes by landing you in a branch off the remote repository.
The issues you discrbe with branches I find happen without. Rather then committing code, people will sit on a working directory. The real hard part is getting people to define a completed change and committing it.
Is this just an "other" people issue? No, as I work on one thing I'll usually come across a tweak, or be unsure if I should do x or y.
Then you bring up feature toggle, so rather than mitigating risk by not changing the code in the first place you hide it in an if condition and pretend that you have "integrated" with other people's work.
I'm quality assurance by trade and while I support this approach in reasonable conditions, I'm well aware you can't isolate all changes in this manner. I believe that keeping a codebase dependacy updated and removing cruft is crucial to a quality release. Feature flags get you only so far.
Yes, every commit has a relevant message. Only so far as can be done locally with unit testing, even then this isn't that important.
No reasonable team contains 300 devs and if all of your devs are working on one repository then that is an organisational issue. There is 0 extra work rebasing as it is just done the same time as pushing.
Nobody sits on working directories, they commit many times a day every day and we don't define completed changes.
That isn't pretending, that is actually integrating since it is in a shared state with the latest version all in one place.
It is not integrated until all the features are turned on, with feature flags you always need to keep in mind you have an additional layer of logic which control the functionality of your application.
If the development team is already committing and pushing often that is a great accomplishment. Though I'm concerned that you aren't completing changes before pushing.
Your push does not perform a rebase, git will reject if you are not up to date. If you can't get up-to-date because other devs are pushing their single line change git will continue rejecting your push.
It is great that everyone gets involved in your team to fix things even if they didn't cause it. I'm sure most try to address issues they cause rather than leaving it for another. I just personally know I don't trust myself to put others members through my mistakes.
Integration of code isn't related to configuration and every piece of software has features turned on and off.
That's why you pull changes in as part of pushing changes. Rebasing never makes sense unless you want to rewrite history.
You wouldn't have to. But if you were a junior and a senior wanted to help to speed things up it makes sense.
If I'm understanding this correctly, trunk-based development means no code reviews. That is unacceptable as professional developers. Am I missing something?
Trunk based development doesn't mean no code reviews. Code review can either happen at the time of writing through pair programming or after deploying and testing the code in a development environment. Normally it makes more sense to review just before going into production as part of the release step instead of before the code has even been properly tested.
If you haven't deployed and tested it then the code being reviewed might not even work which is a waste of time for everyone. If it is just reviewing formatting then it should be automated and not require human intervention.
This presumes that testing the code in a development environment is the same as a code review. Or, to put it another way, a code review just checks that code works and is formatted to correctly.
I'd argue that code reviews are much more than that.
Code reviews:
1) Check not only that the code works, but that the code has the proper threshold of quality, following existing design patterns, clean code standards, and refactoring patterns that ensure that the code is not overly complex and is maintainable and consistent.
2) Work as a means of transferring knowledge and collaborating.
3) Work as a means of documenting the "story" of how the code reached its final state.
4) Work as a means of aggregating documentation. I have often taken longer pull request comments and extracted them into documentation.
Sure. Automate using code formatting and linting tools to shorten the scope of code reviews--but not everything can be covered by automation.
As for pair programming, it is a good exercise but what about other developers who may want to review the code who didn't pair on it? What about when you want to go back and see the historical context of a change? What if both people pairing want to review the code in GitHub before merging, to be sure?
Trunk-based development seems like a case of "the cure is worse than the disease." Just as there are ways to still code review if committing straight to main, there are ways to work out the kinks of feature branches.
I'm not saying tests are review. I'm saying that you do review after deploying and testing. Normally this just looks like a diff of the two commits rendered in the pipeline showing the changes next to the button that deploys to the next environment.
Is there a tool for doing this sort of review? This would seem to break the ability to use GitHub and all of its capability in exchange for resolving some mild occurrences with feature branches.
The diff functionality to review changes is built into git. If you want the GUI stuff then just get your pipeline to send a link to the diff on GitHub (docs.github.com/en/pull-requests/c...) you can put the two commits in the url.
Yeah, it seems like a mix of the Atlassian Gitflow document and large organisations thinking that pull requests are the only way to do code review compliance. Though those organisations tend to have no actual preventative controls.
How are they synchronous?
Both the creator of the PR and the reviewer need to look at the change at the same time for any code change to get integrated in.
Thanks. This is very interesting. However, at the same time, sounds very synchronous. Were you able to efficiently pair and/or do code walkthroughs in an async/remote env?
Exactly this ^^. Pull request code reviews are just as synchronous with an increased delay due to developer response times. Generally pairing or trust are more effective solutions imo.
The complexity tends to remain, the question is where you want to put it.
Analogy of that:
Monoliths end up with a single complex codebase and an easy-to-setup, easy-to-maintain pipeline/infrastructure.
Microservices end up with a lesser complex codebase each but the complexity is now into the infrastructure and the pipelines.
Services is the natural evolution for a monolyth when it needs to escalate from the architectural point of view and are also a middle point between the other two.
On the topic of the OP, you pretend to move the complexity to the management of tags, flags and so in a hope that it will go well without taking into account that branches are just what solve that need/issue.
That post may be originated by a missunderstanding on what continuous integration is. You added the link to thoughworks that show us what thoughworks understand and apply as continuous integration, but you don't work in thoughworks neither do I and each project has it's needs and it's budget*.
A more agnostic description of CI would be:
The keys here are "automatic integrations" and "as often as possible".
*When I say it's budget is because a nice thing to have is to deploy each branch when pushing on them so you can depoy feature branches in feature environments. This of course is not cheap, you pay for more instances in the cloud, the complexity of your infrastructure grows and so on. Maybe is what thoughworks use but I usually deal with clients that even earning billions, ask us to down the Dev and QA instances during the non-labour time to reduce costs, so they can invest that money in objectively better things, which I understand and I'd probably do the same in the same situation. Imagine asking them for that 😂
Of course you can raise that in-house with iron and the costs will be assimilated sooner or later as amortized, but this also means you need to deal with the availability and maintenance of that iron (i.e. having a SysAdmin in-house and not being able to remote working most of the time, good luck finding someone).
What a process allows this case? As far as I understand, a feature branch can stall for a while, but, when it is prepared to review/merge, it shall be actualized, and that is responsibility of the branch maintainer, not his colleagues from another branches.
I guess there is a misnomer between two concepts: feature branch in its typical meaning, and "train" (there could be synonyms). Feature branch is usually a short-term beast; a few commits are made for a task, they are merged, and then, even if development continues under the same branch name, it is principally a new development. But, if trains are used (for example: "stable" train for customers; "testing" for QA games and "devel" as the current developers achievements, and changes are migrating over the way devel -> testing -> stable), yes, there will be lots of conflicts when changes are ported over older codebase. I've seen examples of train-based development, but, in general, it is too specific and hardly used. (Is there statistics?) People prefer release branching to it, because it's just simple.