DEV Community: OssiDev

Even our development process practices are trade-offs

OssiDev — Sat, 01 Apr 2023 17:57:59 +0000

This article was inspired by a tweet I saw a couple of days ago that quoted Dave Farley and Kent Beck both stating that:

Pull requests are incompatible with CI/CD

I might be paraphrasing, so here is a video where Dave Farley explains his stance on pull requests in more detail. You can also find an interview with Kent Beck on the same channel.

The contents of that quote are not the focus of this post. However, that tweet got the wheels rolling in my brain, so I asked ChatGPT to list software development process practices and tools that have trade-offs. Pull requests, of course, were one of those tools (or mechanisms).

I’m here to write about some of those and have you think if these are really needed in your development work!

Pull requests

Pull requests are a mechanism of control that were invented to manage the flow of changes in open source projects. It was adopted widely by the organizations who work with proprietary software, even though it isn’t needed there.

++ Pull requests can improve code quality and help to catch bugs early, as well as facilitate knowledge sharing within the team.

-- However, they can also slow down the development process, as developers must wait for their peers to review and approve their changes.

To put that last part a bit differently: if your company’s review train is not moving at a continuous pace, you will be limiting your throughput. Developers might take on more work while waiting, which will lead to context switching. Context switching can lead to mistakes. PRs can also lead to developers just sitting on their hands while waiting for others to approve their work.

Pull requests are a clear trade-off. They are not the only form of code review, however.

Pair programming

Pair programming is the act of two developers working together on the same task, with one writing the code and the other reviewing it in real-time. You keep switching roles depending on the style you want to work with.

++ This approach can lead to higher-quality code, faster problem-solving, and better knowledge transfer. You also shorten the feedback loop and review your code as you work.

-- However, it can also be less efficient, as it requires two developers to be dedicated to a single task.

Some experiments and experts say that the claim about losing speed is false. They say that pair programming, while at start can be slower, will lead to the exact same results in terms of speed at the end.

Often the things you can ‘skip’ with pair programming is pull requests and perhaps even manual testing with QA. You already have two pairs of eyes looking at your code, how many does it need for your code to be approved for merging? You can then deploy your code faster and move on to the next item. This can at best save you day(s) in wait time.

You also now have two developers who’ve worked on that same piece of code and know it intimately. Any time there is a request to change it, fix it or extend it, they can both pick it up much faster. I would say knowledge sharing with pair programming is much better than pull requests.

Pair programming doesn’t work everywhere

There might be organizational decisions that can prevent you from doing pair programming. One of these is having developers work on only one vertical.

What this means is you have front-end developers, back-end developers, QA engineers, UI/UX designers and they all work in their own little area of comfort. Typically a team only has one person in each role which means you don’t have a chance to pair up with anyone in your team. If your back-end engineer is not comfortable with working on the React app with you, you’re out of luck.

In the perfect world we’d have autonomous teams who can hire the developers they need and form the team that they need, but most often teams are subject to management by the organization and can’t do their own hiring.

This is why I often promote that developers should work in more than one discipline in a team. This will not only enable pair programming, it will enable one developer to tackle any story that requires them to work on multiple verticals, and it will grow each team member’s T or M-shaped skills as well.

Feature branches

Feature branches allow developers to work on individual features in isolation before merging them back into the main branch.

++ This can make it easier to manage complex projects with multiple developers

-- It can also lead to merge conflicts and integration challenges.

Do you need feature branches? I say it depends. You can have them as an optional tool in any repo.

Again, this is a trade-off that should be evaluated in the environment it's meant to be applied to. I like to look at at least three things in any project for this:

The number of developers working on the app
The skill level of the developers
The amount of automated tests

When you have a low number of developers working on the app, that often enables you to communicate between each other more effectively on the changes you make even on a daily basis. You trust each other to make changes that are of good quality and don’t harm the application.

You also need automated tests. These are in place to make sure you don’t break the application with your changes, but also bring you confidence in those changes. You need to run these with each commit in your pipeline if you don’t want to use feature branches.

The last one is the skill level of the developers. When I said that you can have feature branches as an optional tool, here is where they become effective. You can assign junior developers to work on the app and have them use feature branches, along with pull requests, to provide feedback on their work and to have better quality control.

Feature branches, or lack of them, can also be linked to the idea of continuous delivery. The concept of continuous delivery is best described by its inventor, Dave Farley:

Continuous delivery means always having your software in a releasable state.

When you skip feature branches, work on the trunk directly and embrace the idea of continuous delivery, you will work on keeping your software in a releasable state at all times. This means when you commit your changes, your CI pipeline validates your changes by building and testing those changes, and that provides you with feedback saying it’s ready for deployment. You work in such small increments that any possible mistakes are easy to fix and the blast radius is limited.

This is also called trunk-based development. You can use feature branches with trunk-based development, but more often I see them being paired with pull requests and the code review train making too many stops. Your branches end up living for multiple days, sitting in that queue, maybe hitting a conflict as other work gets merged into trunk. Also, any feedback you get will delay merging even more, making your branch live out even longer.

Scrum sprints

Sprints were introduced to us by Scrum. They are a commonly used methodology in software development that involves dividing the development process into short, iterative cycles of work.

They are a time box for development, where we should complete a certain amount of work that we’ve agreed to as a team. We try to plan the work to that small time frame before we commit to it.

++ Sprints can help you be more organized. They offer a sense of urgency and focus on delivering work (which can be beneficial for some). They offer management a way to see what the team will be working on during this and the next iteration. Sprints also offer a path to provide more structure to the development process by adapting more of Scrum to it.

-- Sprints create additional pressure on team members to deliver work quickly to keep promises and complete their sprint goals (which might affect quality). They introduce accountability and limit your scope by delivering a specific set of features within a fixed period which can result in incomplete or wrong features as often you only receive feedback at the end of the sprint.

What baffles me with sprints is how widely they are used and nobody really even stops to ask why. They are not actually needed. You can just as well do software development without this time box.

Often what I see is this time box is being mandated by the company, because they treat software development like any project they need to manage. It’s not actually used by the team out of necessity to make the development process smoother.

If you like structure and organizing the work into smaller buckets, Scrum as a framework can suit your needs better where you can also continuously improve the process.

On the other hand something like Kanban offers more flexibility with no fixed iterations or timeboxing, and also allows for more continuous delivery of work, with smaller batches of work being delivered more frequently. This can provide faster feedback loops and reduce the risk of long delays or unexpected problems.

Best development practices are about speed

Accelerate - The Science of DevOps: Building and Scaling High Performance Technology Organizations book, which summarized the State of DevOps reports, has told us that high performance teams have/use:

Automated deployment process’
Continuous integration
Trunk-based development
Test automation
Continuous delivery
Limit WIP

To put this in another way: to have good quality, you need to go fast. Automate as much as you can so you can deploy as quickly as you can without manual stops for quality control, with good focus on the work that you’re doing at that given moment.

While pull requests might offer you better quality control, they will hinder your performance. While feature branches make it easier to manage development with multiple developers, it can lead to merge conflicts and integration challenges that might hinder performance. While sprints offer you better organization and structure, Kanban can give you faster feedback and more flexibility.

Again, it’s all trade-offs. That’s the space we work in. Know what you’re giving up when you adapt one practice and don’t just be a mindless zombie that follows the mainstream because that’s how everyone else is doing it.

Your legacy software might not the problem. Maybe it’s the developers.

OssiDev — Sat, 03 Sep 2022 19:55:11 +0000

Some of us have been there. Working on a software that dates back, 5, 7 or maybe even 10 years. It’s been given the unofficial label of being a “legacy” software application. But it still runs most of your business, so it's forbidden to use that word.

Those years of development have left it’s mark. It’s now a mix and mush of different architectural styles, big classes, some security issues and it's all coupled together.

It’s turned into a big ball of mud. An application that architects shun and try to avoid. Is it hopeless to untangle this mess? Maybe not. But maybe that’s not your biggest problem.

Maybe the problem you should be focusing on is the developers

They’re the ones who’ve been developing the application all these years. Development that's mostly guided by senior developers, because no architect wants to touch the application anymore.

Some have been there for many, many years now. They have a huge amount of domain knowledge and are invaluable as employees. Yet, they never saw the application turn into the ball of mud it is now.

Why? Did they not care what was happening? Did they not attempt to correct it before it was too late?

Maybe they just liked their position and pay checks too much to just let it run it’s course, because it was not their job to care. They were there to build features and put out fires when needed.

Management surely didn't care, because no one made an effort to bring this situation to light.

My experience

I’ve worked as an architect at a company like this. At the time I was too inexperienced in that role. But I had the passion for it, so I learned what it meant to work as a software architect. I studied for it for months. I learned a ton of new things. My learning potential skyrocketed.

But I failed to untangle that big ball of mud even a little bit. Why?

Because I was focusing on the wrong problem. I focused on the technical problems, instead of the people. The developers were the most important ball I should’ve helped untangle here.

I wish I would've realized that..

Still, let's not blame anyone

It is not my intention to point my fingers at anyone. This is more self reflection. I wish I would have been wiser at the time to identify the problem and approach management with what I had realized, so we could work the problem together.

Reflecting on my time there and continuing on my path to learning I’ve come to realize that there are standards of skill in development at some companies. Maybe more so at companies that have a separate technology department, and haven’t realized that the technology is at the core of that company. It’s just a separate department that's maybe 15-20% of the entire workforce.

The developers there might be on the older side of the age spectrum. More people are over 40 years old, than under. They seem to have lost their passion to learn and just want to maintain the status quo. So they’ve been continuing with their usual style of working for the past 3-6 years.

Creating a class after another in one single folder, that's now bloated to a massive scale. Not knowing about dependency injection which exists to introduce loose coupling into the system. Not focusing on modularity, but instead just refer to big classes that do all that work. Everything gets squeezed in nice and tight, into one big package that's deployed at once.

In any case, when the standard of development is low you need to focus on first uplifting the quality in every day development. Introduce people into the basic concepts, such as dependency injection, decoupling, modularity, SOLID principles. Teach them what software architecture is, what architectural qualities are, how to identify them and what to focus on when building new things, what modern things are out there.

As an architect, you are also a mentor. A teacher. And you must produce good quality code to set an example! Get your hands dirty like everyone else.

Also, teach people what business value it brings when you can build a system with good architectural qualities, modularity and good decoupling. That value doesn't have to be instantaneous. Most value is generated later down the line when talking about architectural decisions.

Once you have done this, and actually improved people's skill level, you can start introducing architectural decisions and changes.

Get the leadership on board

However, before you do anything, have your leadership or management sponsor your efforts. You will be shifting the focus away from technical problems, into solving the skill problems with people.

Otherwise you will end up like me. Having to face the push back of senior developers who don't want anything to change. Developers, who'd rather continue to rule over the application in the shadows, than actually claim accountability which results in decisions happening behind your back even though you're in charge of making those decisions. Working to tear down the changes you've made to improve the every day development quality because they don't understand them, instead of just saying that they don't so you can help them see the reasons.

When that happened I simply walked away. I didn’t have the leadership’s backing at that point because I was too inexperienced in office politics to realize I had to identify this issue and work on it, before I got to work on the technical stuff.

Final mentions

Many of us have a story like this. I consider myself lucky that this happened so early in my career and at a point in my career where I continue to learn at a somewhat extraordinary pace so now I see it as a learning opportunity. My skills skyrocketed, and that in turn helped find a new job. One that I'm very happy in.

This experience helped me grow a mile taller.

The inspiration to write this post came to me after I completed reading the "Fundamentals of Software Architecture, An Engineering Approach" by Mark Richards and Neal Ford. This book validated a lot my work as an architect. I was more interested in the definitions and expectations from a software architect, as well as the soft skills portion of the book, rather than the technical portions.

In any case, if you're an aspiring software architect, or one who still questions some of the work you do, I truly recommend you read it.

Some notes:

As an architect, soft skills and interpersonal skills are vital for you to succeed. All of your decisions are going to be challenged. People need to be convinced that your decisions are the right ones. However, if you have a cohort of people almost rebelling against your decisions and trying to tear them down one by one well after they've already been established, you need to have the management's support to enforce those decisions. Sometimes that cohort is led by one developer who's turning other people to his side. It sounds really childish, but that's how we human beings behave.
If you have a lot of people trying to push back on your decision after they've been made, you can always supersede an earlier decision, but at that point you might want to circle back into asking the developers if they understood the decision and maybe in general, if they actually want a change. Management might be imposing that something needs to change, but developers haven't accepted that fact, so you should maybe hold a workshop to figure out if people want a change, what the pain points are, what we should focus on etc. This way you involve the people in the decision making and make sure they understand them. Maybe take one or two hours to educate people on the importance of software architecture and what it contains. This is useful if you're planning on making major changes and essentially trying to steer the ship into another direction.
I personally don't like the idea of an "Architectural Review Board", or any type of oversight committee who determines what's gonna be added to the system and where. That's a bottleneck. I prefer an approach where I work with the teams (or just one team). If you have multiple teams working on one product and you're supposed to be the software architect of that product, it's not a good idea to just be tied to one team because your influence will be limited. You can have a board or a team to set the general rules and direction of the system, but don't have it be this entity that slows down development, because you feel you need that control.
Office politics play their part in everything you do, because some people might have seniority in terms of experience at that company, while you may posses more technical skill. They can easily gain influence to challenge your decisions, even though you've proven your point from a technical and business points of view.
There is another, slower way, to grow the technical skill level of the people in the technology department: hire skilled developers. Most likely though, high level developers will shy away from working at that company if they realize the product you're building is a big ball of mud, and sometimes you'll hit budgetary constraints that prevent you from paying what they're asking. So you're stuck with the skill you got. At that point you need to focus on uplifting the skill level of everyone around you.

You think you’re working in an agile way? Probably not.

OssiDev — Tue, 02 Aug 2022 19:45:00 +0000

Inspired by the recent conversation in YouTube between Allen Holub and Dave Farley, I started to contemplate about all the different events that occur during the development process in companies that claim that “they’re agile”. Even the claim of “working in an agile way” can be brought to question.

Do you work for such a company? Maybe you’ll find yourself in this post.

Let me be clear, that I am in no way some agile expert or a coach or anything. I’m simply a developer who’s interested in agile and wants to discover better ways of working, having seen different work process’ in different companies.

Agile

Some of us have been introduced to agile at some point by a manager who says:

We do Scrum here. We’re agile.

That’s it.

Nobody in the organization asks or tells you to read the actual agile manifesto, located here. Why? Because most of those managers or supervisors don’t know what agile really means. They probably haven’t even read the manifesto themselves. They’d rather just give you the Scrum guide and order you to follow it. (I’ve actually never been advised to read the manifesto by anyone in organizations that I’ve worked with or for.)

By definition agile is very simple. It is what is sounds like. Agility in development. You embrace and adapt to change. You talk to people instead of relying on a project management system. You collaborate with your customers to deliver software early, instead of agreeing to a contract defining what to deliver with a fixed cost in a fixed timeline.

Most of use have come to know agile by using Scrum. But Scrum in itself is not agile. Not, if you follow its guide to the letter.

Scrum is not agile

As Allen Holub has mentioned many times, nowhere in the manifesto does it say you have to have sprints, refinement or planning sessions, or reviews. That’s just the way we’ve been taught to work due to the high adoption rate of Scrum in different organizations. The only thing the manifesto says is to hold some type of a reflection session at regular intervals. In Scrum, a retrospective is held on a fixed date after a fixed time (sprint length).

The Scrum guide in itself, however, is directly opposed to the agile manifesto. It is actually quite rigid. Here’s a couple of quotes from the guide:

Changing the core design or ideas of Scrum, leaving out elements, or not following the rules of Scrum, covers up problems and limits the benefits of Scrum, potentially even rendering it useless.

Essentially, follow Scrum to the letter or you’re not doing Scrum or even creating hidden problems for yourself. That’s not very agile.

Sprint Planning initiates the Sprint by laying out the work to be performed for the Sprint.

Create a plan for the work to be completed during a fixed time period and stick to that plan. What happens to that plan if there’s a change here? This sounds a lot like you’re doing mini waterfall development.

During the Sprint: No changes are made that would endanger the Sprint Goal;

So, your team has agreed upon a sprint goal (to deliver something of value) for the next few weeks, and no changes can be made to the sprint to endanger that goal. You must complete that goal by any means necessary during a set timeframe, even though you’re supposed to stay agile and adjust if you learn something new. And something new comes up during that fixed period, believe me.

So Scrum is not agile. It’s not a software development method. It’s simply about project management. And don't all managers love management?

What is agile, is taking ideas from Scrum and creating your own development process so you can work in an agile way and continuing to work on it. You can call it your process that takes ideas from Scrum, but it’s not Scrum.

“Responding to change over following a plan”

We’re supposed to be continuously improving, so that we can deliver value faster. Still, there are too many things that slow you down. Most of these are things that your company has set up because the people there do not understand agile. They want fixed dates, they want to know estimates, the cost.. That’s just how they operate. Unfortunately, that’s not agile. They are not thinking in an agile way. They are thinking in an industrial way.

Scrum is essentially a mini waterfall delivery process. It’s creating a plan for a fixed timeframe. I’m not even gonna talk about SAFe which has an even bigger waterfall. However, with Scrum or SAFe your company gets the certainty that in a specific timeframe (2-4 weeks) developers will complete certain items, they know the estimates, the delivery dates and can calculate the cost, even though it's for a smaller timeframe.

This also brings accountability into the mix. While most of the companies I’ve seen don’t go blaming people directly for missing a sprint goal (or a product increment goal), I’ve often heard criticism when a team has committed to a goal bigger than they can deliver. That’s the blame game, but in a smaller scale. In their mind, someone has to be held accountable when something doesn’t get delivered.

Things never go as you plan. We can’t complete some things because we learn new things, but we can’t adjust. We hit a roadblock. We have a dependency somewhere that slows us down and we miss the deadline. We stumbled upon a bug that we have to fix before we can deliver. We had to refactor some poorly implemented structure to support our work and we didn’t have time to look at the code before the planning session.

The reason for a planning session is to estimate the fixed amount of work you can do on a fixed timeframe.

No matter how many times I’ve tried to be prepared for the planning session to cover everything, something has always come up. I’ve never thought of every possible scenario during a planning session. And I myself have gone against the value of the manifesto back in the day.

Roadblocks and speed bumps

Your development process might not be the only thing slowing you down. In a normal company setting, there are typically numerous different obstacles in our way. They may stop our development or simply slow it down.

Over the years I’ve encountered multiple different events and organizational challenges that have ended up slowing down my work. You may recognize some of them.

You have to think about the dependencies with other teams before you can work on the item and perhaps need their input before starting the work.
You have to attend retrospectives and other ceremonies "just because" even though no one has contributed to them in years.
You spend days worth of time doing PI planning for the next three months because some genius had the brilliant idea of using SAFe to bring 'agility at scale to their enterprise'. You forget that plan completely the next week when you start working on the items for your next sprint.
You discover that dependency when developing which can cause you to stop your work because you have to go ask another team their input.
You have a separate IT department that provides you with resources you need. Such as access, credentials, cloud resources..
You have a separate OPS team which have their own backlog of items they work on and you have to ask them to allocate to you the resources you need, be it a member of their team or just time from their schedule.
You have an architect/senior developer who has to tell you how to do the work because they tell you what to do and how to do it, so that it gets done right.
You use feature branching where your work goes to a development / quality assurance environment to be tested before it gets to production ‘some time later’. Those tests are run manually for each work item.
Before your work gets merged to a development branch, it has to be reviewed by another developer, perhaps from another team, who has no knowledge of the problem that you have been solving
Your manager hands in your work load for the next sprint, because he's the one talking to customers and knows how to prioritize the work.
You’ve actually never met a customer, or a user, while working there and your manager/product owner acts as a middle man, when you want to have a conversation with this entity. This causes delays because messages get relayed. It can also cause you to waste the time of a sprint doing the wrong things only to be discovered during the review that happens 2-4 weeks later because there was a mistake when relaying the wishes of the customer/user.
You have to use Jira, so that your company has everything in one place and can monitor you work and move the discussion to a tool instead of people talking to each other.

These are but a few of the things I’ve seen slowing me down during development. But how do you solve them?

Have your teams create their own development process

A majority of the speed bumps mentioned above can be solved with one thing: team autonomy.

Teams should be the ones creating their own development process. Not companies. Not some senior developers that want to impose it to all the teams because it worked someplace else. The team itself should discover the best way to work.

As it says in the agile manifesto:

Build projects around motivated individuals. Give them the environment and support their needs, and trust them to get the job done.

Teams should have autonomy in this. Autonomy to discover the best way to work. Autonomy to build the solution the way they like it. They should also interact with each other to continuously evolve the way they work.

You should build cross-functional teams that contain the expertise needed to do this. Often this means everyone should know what software or infrastructure architecture is. They should know how to embed security into their development process. They should talk to stakeholders directly and deliver valuable software early. They should be able to deploy the resources they need to the environment of their choosing without having to ask someone for permission or access.

However, most companies are fixated in believing people can’t do the work unsupervised and without being monitored. Sure, there are people like that, but teams themselves should have a set of standards they uphold and make sure it doesn’t happen.

I refer you to read The Five Dysfunctions of a Team, by Patrick Lencioni, which is a team building strategy. One of the dysfunctions is low standards and avoiding accountability.

Teams should hold themselves to a specific level of quality and work, making sure every member works to complete the goals the team has set for itself. Before you can have a set of standards you need to build a foundation of trust and make sure everyone is committed.

Throw Scrum away

Be bold! Create your own development process. Here’s a few ideas from the top of my head:

Stop using fixed sprints with fixed work loads and instead opt to a more continuous Kanban style flow, where you have a list of work items that you keep prioritizing constantly.
Work on the items that are most important until that list of items is gone, together as a team.
Maybe look at that backlog for 5-10 minutes each morning with the team to think about the priority but leave it at that. If something new comes up that causes you to change priority, you can easily do it.
Never have a backlog that’s longer than three months worth of stories. Everything beyond that is outdated knowledge.
Set up refinement sessions when ever some of those items seem like they’re too big and need chopping down.
Don’t do retrospectives after each sprint. Set up an Andon Cord. A mechanism for team members to alert each other when ever they feel like there’s a problem that needs to be addressed immediately. Don’t mull on that problem for two weeks. This way you can learn and change course immediately.
Share your successes during daily stand ups. Don't just be a robot and list the things you did yesterday, what are you doing today and do you have any blockers.
Improve on a daily basis. Not just at the end of a sprint.

And just have fun! No one wants to work in a way that makes them miserable.

JAMStack deployment with Azure DevOps Pipeline

OssiDev — Sun, 18 Jul 2021 10:59:26 +0000

I often do some freelance work for customers who have their own servers. These servers are usually the type of webhosting solutions where you don't have root access. Just Apache or Nginx running and pointing to a specific folder and serves PHP files. Often enough customers have a website running on Wordpress there.

Not having Node.js available in a server limits a web developer's capabilities, when nowadays a big portion of us work with Node.js apps. I do too, and I often build separate front-end applications with JavaScript frameworks, while having a headless CMS provide the site with content. With this I can often do Jamstack builds.

Since Jamstack requires you to re-build your website each time your content changes, I need Node.js to run on the server. This isn't happening, so I had to figure out an alternative way to build front-end applications, away from the server. Good thing is I have access to Azure meaning I could use it's DevOps Pipelines to have it build the application for me, and deploy the static output to Blob Storage, and then dispatch an event back to the server. The server then would fetch the built website for me and extract it to be hosted.

The thing was, I wasn't sure if I could have external events from a platform like Wordpress or Strapi trigger the build. Luckily, it could, but let me walk you through the process.

Requirements

I will walk you through what I did and provide links to resources (docs) that I read to get everything done. I use Azure CLI to create resources as it's much faster and cleaner for the purposes of this tutorial as instead of providing screenshots of Portal.

Azure provides a lot of documentation regarding their services, but in my experience Microsoft has a habit of sprinkling things here and there so you have to know where to look.

You need the following things to build this pipeline:

An Azure subscription
A repository which can be integrated with Azure DevOps
Azure DevOps organization
Enable Event Grid resource provider
A storage account and a container
An API with an endpoint which can validate the event subscription and then receive events when blobs are created in Blob Storage. I will refer you to this how-to guide on how to validate your subscription.

I will assume you can set up the first four, and last one, yourself and let's simply look at setting up a storage account and the pipeline.

Setting up a storage account

We need to set up a storage account and a container, or use an existing one, to hold our finalized build.

Use the Azure CLI to create a storage account.
```
 az storage account create --name <storage-account-name> --resource-group <resource-group-name>
```
Grab the "id" property value (fully qualified identifier) from the response json in your CLI, which is something like /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Storage/storageAccounts/{storageAccountName}
Create a container
```
 az storage container create --name deployment --account-name <storage-account-name> --public-access blob
```
If you wish for stricter security, please consider dropping the --public-access flag and going the private route.

Create an event subscription

Before you create an event subscription make sure your API endpoint is capable of subscribing to receive events.

Now use the "id" from your storage container creation to create an event subscription.

 az eventgrid event-subscription create --source-resource-id <storage-account-fully-qualified-identifier> --name nodeAppDeployment --endpoint <your-api-endpoint> --included-event-types Microsoft.Storage.BlobCreated Microsoft.Storage.BlobCreated

DevOps

Before we get into setting up a pipeline, we first have to visit Project settings and set up a Service Connection (under Pipelines section.) Select Incoming WebHook as the connection type and add a WebHook name, and Service connection name. Don't grant access to all pipelines unless you want to!

Now, once that's done we can create a pipeline. Simply click on New pipeline from the Pipelines dashboard and add your repository configurations. Select a basic Node.js type task and once you're in the pipeline YAML configuration page, let's add the following there:

# Use our Service Connection WebHook (change)
resources:
  webhooks:
    - webhook: <webhook-name>
      connection: <service-connection-name>

# CI trigger
trigger:
- master

pool:
  vmImage: ubuntu-latest

steps:
# Install Node.js
- task: NodeTool@0
  inputs:
    versionSpec: '14.x'
  displayName: 'Install Node.js'

# Install packages and run build command
- script: |
    npm install
    npm run build
  displayName: 'npm install and build'

# Archive the /out folder
- task: ArchiveFiles@2
  inputs:
    rootFolderOrFile: '$(System.DefaultWorkingDirectory)/out'
    includeRootFolder: false
    archiveType: 'zip'
    archiveFile: '$(Build.ArtifactStagingDirectory)/release.zip'
    replaceExistingArchive: true

# Upload the output zip to blob storage
- task: AzureCLI@2
  inputs:
    azureSubscription: <azure-subscription>
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: 'az storage blob upload --account-name <storage-account-name> -c <container-name> -f $(Build.ArtifactStagingDirectory)/release.zip -n release.zip'

The YAML configuration file is commented, but let me just point a few things here.

First you need to set your webhook settings in the configuration file
The app being built is a Next.js framework app, so npm run build executes next build && next export, that's why the output folder is out. If you use a different framework, you have to adjust this configuration.
You need to authorize your azure subscription so you can use it in your upload task. You need to input your subscription name in the azureSubscription field. What I did was I just grabbed a task like Azure file copy and inputed the configuration via the UI, and from there I got the correct name and ID for the YAML file. Since this is a linux agent, we have to use Azure CLI to upload the file. The Azure file copy only works for PowerShell ie. Windows agents.
I didn't take into consideration any environment variables here. You will have to deal with those yourself.

That's pretty much it.

Triggering the pipeline.

That's simple. You just make a POST request to the following URL:

https://dev.azure.com/<organization-name>/_apis/public/distributedtask/webhooks/<webhook-name>?api-version=6.0-preview

The problem here is that whoever finds out this URL can cause you harm and potentially make your pipeline run constantly by pinging that URL. That's why you can set up a Secret and an HTTP Header in the Service Connection. You can generate a HMAC SHA1 checksum of your payload (not sure if other message digest algorithms are supported), and add it to the specified header. The secret will of course be the key to generate that checksum.

Azure will compare the checksums together and block any request where the payload is empty, or where the checksum does not match.

Wrapping up

This tutorial does not tell you what to do with the archived output file in your API. You're going to have to figure it out yourself. What I've done is simply copy the file from storage (url provided by the BlobCreated event payload) and extracted it to the Apache hosted folder.

As always, there are other alternatives besides Azure and any type of pipeline. You can use services like Vercel or Netlify to host your front-end application and you can skip building anything. I belive both contain some type of webhook trigger mechanism to initiate a re-build. You can also host the static website in Azure Storage, and set up a CDN and everything in Azure. This is propably cheaper, and definitely more scalable, than some non-cloud based virtual machine.

The pros of this approach are that I don't have to run Git or Node.js on the server just to fetch the latest code changes, or build the app. I can fetch the repo and install packages in an isolated container. If it fails it doesn't affect the website in production. You can also run your tests in the same pipeline too to make sure you haven't broken anything. Using TypeScript can give you some confidence for the compilation.

Resources

Below are some resources that helped me figure out what's needed. I've been playing around with Azure DevOps a lot lately, so that experience helped me do this in an hour.

Installing Azure CLI:
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli

Jamstack:
https://jamstack.org/

Azure CLI - az storage account:
https://docs.microsoft.com/en-us/cli/azure/storage/account?view=azure-cli-latest

Azure CLI - az eventgrid event-subscription:
https://docs.microsoft.com/en-us/cli/azure/eventgrid/event-subscription?view=azure-cli-latest

Receive events from Azure Event Grid to an HTTP endpoint: https://docs.microsoft.com/en-us/azure/event-grid/receive-events#endpoint-validation

Generic webhook based triggers for YAML pipelines: https://docs.microsoft.com/en-us/azure/devops/release-notes/2020/pipelines/sprint-172-update?WT.mc_id=DOP-MVP-21138#generic-webhook-based-triggers-for-yaml-pipelines

Enable Event Grid resource provider:
https://docs.microsoft.com/en-us/azure/event-grid/custom-event-quickstart-portal#enable-event-grid-resource-provider