DEV Community: Alessandro Diaferia

Presumed technical debt: how to recognise it and avoid it

Alessandro Diaferia — Thu, 26 Oct 2023 15:01:43 +0000

The programming community unanimously considers technical debt an aspect of our work to keep under control and reduce.

Personally, I’ve been vocal about the perils of technical debt in one of my early blog posts about the organizational issues it can cause.

While I still stand by the majority of what I described in that article, I want to clarify my take on what I call the presumed technical debt.

What is technical debt?

First of all, though, let’s agree on what the term means. Technical debt is a broad term that encompasses many aspects of building and delivering software.

Martin Fowler refers to it as the cruft that builds up over time and that makes it harder to modify and extend the system further. He then references the likely original author of the metaphor Technical Debt in the person of Ward Cunningham.

What Ward Cunningham wrote first about technical debt , as follows:

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.

Ward Cunningham, 1992

Here I interpret not-quite-right code as a way of referring to the design of code based on assumptions and understanding before it gets actually validated by the end users. But I’ll clarify this further later.

Finally, I would add my own definition of what technical debt is:

Technical debt is the gap between the current state of the software non-functional characteristics and the desired state they should be in, provided you had infinite time, money and absolute understanding of the problem at hand that your software is meant to solve.

Different forms of technical debt

There is a substantial natural cruft that builds up in software because of the different forces that affect it: dependencies getting stale, requirements or user needs changing, more code being added, unused code paths, and so on.

This type of natural technical debt should be continuously monitored and kept under control. You won’t ever be able to get rid of it completely but there should be a continuous tension to keep it under control.

In my experience, though, there is an additional form of technical debt that is less tangible, while representing a considerable influencing factor when developers design software: the presumed technical debt.

Presumed technical debt is a form of debt that developers think they will incur into soon if certain non-functional characteristics do not get implemented. Let’s explore together this form of technical debt.

Non-negotiable characteristics

First of all I want to clarify that there are surely certain baseline non-negotiable characteristics that the organization might require before any code gets into production. These characteristics are in the form of, for example:

observability and operational envelopes
security and compliance requirements
data retention and governance constraints

But there are other characteristics, like scalability, availability or resilience, that depend heavily on the specific use you are building to solve for.

Technical debt or just lack of aspirational characteristics?

On the contrary, in my experience, it’s not unusual for developers to design new functionality targeting infinite scalability , eternal availability , absolute resilience.

At times it feels that those characteristics are the default starting point. Developers aim at those but then compromise back as they face business constraints like time and money.

The MVP ends up not being shipped on a container-based cloud infrastructure that auto-scales. Turns out, the same could be delivered via a static HTML page hosted on S3 and served via CloudFront.

Nevertheless, many teams will still aim at container-based cloud infrastructure as the next iteration for what they shipped. They add that aspiration to their backlog, regardless of the feedback they receive from the MVP.

It’s not unusual to think that what they have shipped is not fit-for-success.

When success comes, it’ll bring troubles to this architecture

Anonymous developer

Having compromised on some ideal characteristics, some developers might not feel happy about the MVP. They might think it won’t certainly scale when success brings troubles. They will consider the current architecture as technical debt, necessarily, and aim for getting rid of it.

I would argue, instead, that those characteristics are aspirational and need to be validated. I’ll tell you more: their lack thereof might never turn into debt, effectively. This is especially true when working on new functionality.

This is why I call it Presumed Technical Debt. It’s coming from the non-validated expectation that the software will definitely need those non-functional characteristics to deliver value effectively.

In this case the technical debt is represented by the gap between the current state of the non-functional characteristics and the ideal version of them as desired by the developer.

Non-validated assumptions are a form of technical debt

Say the Product Organization has identified a new customer problem to solve. In an agile context you would work with your team to identify the minimum viable iteration that your customer could consume so that you can, to name a few:

validate your understanding of the problem
understand how your customer would use your software to solve the problem
understand what is critical minimum functionality for your customer
understand traffic patterns, auditing requirements, scalability needs, and so on

You are making a bet on what you think is the right way of solving a problem and you want to validate this assumption early.

You have an understanding based on your experience and ideas and you need to get something out there quickly before you go down a path that might take you far away from the real customer needs.

Similarly, you might have assumptions regarding what the non-functional characteristic of your software should look like: does it really needs three nines of minimum availability? The reality might be that no one is expecting any traffic during weekends so an 80% SLA is more than enough!

There is a great article Kurt Bittner and Pierre Pureur about continuously balancing a Minimum Viable Architecture with the Minimum Viable Product.

Investing in more than is needed to support the necessary level of success of the MVP would be wasteful since that level of investment (i.e. solving those problems) may never be needed; if the MVP fails, no more investment is needed in the MVA.

Kurt Bittner and Pierre Pureur – Agility and Architecture: Balancing Minimum Viable Product and Minimum Viable Architecture

Building expensive non-functional characteristics is a form of debt that you might have to carry throughout the whole lifecycle of the software you are building.

If you start with infinite horizontal scalability you are deliberately adding cost, overhead and constraints to keep up with this non-functional characteristic that you haven’t validated yet.

Don’t be DRY at all costs

Another common fallacy I see is the application of the DRY principle as an end goal. DRY stands for Don’t Repeat Yourself and it is often the reason why incredibly complicated abstractions exist.

I attribute many erroneous implementation of the DRY principle to what I call Presumed Technical Debt.

It’s not rare for developers to deem two use cases similar enough that they require a common abstraction implementation. In my experience it’s unsurprisingly frequent to end up with an abstraction that hardly solves either use cases.

Often, developers build the abstraction out of the conviction that repeated code is a form of technical debt to avoid.

In my experience the reality is that by the time you come up with the idea that two use cases might be similar enough to require a common abstraction, you actually don’t have enough data points to be able to make that decision.

Therefore, developing an abstraction out of a non-validated idea regarding commonality among two or more use cases results in an artificial constraint. Such constraint will impact your pace and your ability to iterate swiftly as you receive feedback signals from what you have put in the hands of the customers.

Only then you will actually have data to be able to decide whether you need an abstraction or not. Maybe you need a third use case first!

There is a great talk by Dan Abramov regarding The WET Codebase that highlights the dangers of always striving for abstracting out code too early.

Presumed technical debt makes your job frustrating

Building software chasing an aspirational architecture is a recipe for frustration.

This is why one should not use the term “technical debt” lightly. Labeling the lack of an arbitrarily ideal set of non-functional characteristics as technical debt will always create tension towards achieving an ideal architecture, regardless of whether it is necessary or not.

In my experience the reality is different: most of the times the functionality you have built will need far-from-ideal non-functional characteristics: as developers, sometimes we hope we are going to be given the chance to work on complex architectures , so we start with that upfront.

A truly agile approach would leave many of the made up non-functional requirements to be identified later, as the functionality starts getting used. Will it actually need to scale that much or is it rather delivering great value to my users while being interacted with sporadically? Will my users ever notice the system becoming unavailable at weekends?

As professionals it is our job to build for sustainability. If we design arbitrary, non-validated, non-functional characteristics we are actually building technical debt into our software unnecessarily. An overly complex architecture that doesn’t serve any valuable purpose for the end user might just add development overhead and slow down your organization.

Closing thoughts

In this post I have highlighted what I find being an emerging trend in software engineering: designing non-functional characteristics upfront, based on non-validated assumptions, hoping to be preparing software for success and avoiding technical debt.

In reality, non-validated non-functional characteristics can turn into technical debt if they don’t end up supporting the real needs of the customers.

While it is important to acknowledge the existence and impact of technical debt, it is equally crucial to differentiate between necessary technical debt and idealized technical debt.

If baseline non-negotiable characteristics such as observability, security, and compliance requirements must be met, it is essential to validate and prioritize other characteristics such as scalability, availability, and resilience based on customer needs and feedback.

Developers often fall into the trap of aiming for an idealized architecture before fully understanding the problem and user requirements. This leads to the development of code and architectures that may not align with the actual needs of the software. Making assumptions and designing for non-validated technical requirements can introduce unnecessary complexity and hinder the agility and speed of the development process.

Rather than striving for an abstract ideal, it is important to approach software development with a mindset of sustainability and continuous reassessment. This means continuously monitoring and addressing technical debt that naturally accumulates over time, while avoiding the introduction of additional debt through arbitrary and non-validated technical requirements.

I hope you enjoyed this post. If you like my writing you can follow me on LinkedIn to stay up-to-date on my thoughts on software engineering.

Photo by Martijn Baudoin on Unsplash

The post Presumed technical debt: how to recognise it and avoid it appeared first on Alessandro Diaferia.

How to communicate efficiently as software engineers

Alessandro Diaferia — Mon, 25 Sep 2023 06:27:28 +0000

Reading Time: 5 minutes

Conversation is the basis of collaboration. And yet, communication for software engineers is often such an overlooked soft skill. As software engineers we understand that different programming languages serve different purposes to describe to machines what we want to build. At the same time, different audiences and contexts call for different natural language subsets.

I have observed many times inefficient communication happen between the people building the software and the ones consuming it in some form or another.

Most of the times the source of the inefficiency comes from the inability to understand what each other are looking for: for example engineers explaining the technical challenges they are facing while a customer service representative might only looking for a delivery date.

Different audiences call for different languages

When talking to other people we most likely use a specific subset of our dictionary to explain our thoughts, to make our message clear and to maximise the chances that our interlocutor understands what we are saying.

The dictionary we use when we speak to our fellow engineers is quite specific and optimised to get technical concepts over. There is probably a huge amount of technical details that we give for granted and that come from, to name a few, the inherent technical expertise we believe our colleagues have, the specific technical context we are building within, recent conversations, the agreed technological direction of our organization and so on.

From time to time, though, we are required to interact with stakeholders from different departments or external to the company altogether. In this case the conversation will need to move to a non-technical level.

Does my audience even care?

Building software products that are actually useful requires software engineers to spend time with collaborators, users and colleagues, share perspectives and exchange feedback.

Collaboration happening with different functions will require different problems to be discussed. To each its own preferred language.

A product manager will be interested in your complexity assessment, but will not care about what Node module you are going to end up depending on to deliver the functionality.

Getting deep into the details of the implementation you already have in mind will also bring the conversation down to a level that is not helpful yet and with this audience. I have discussed this issue in the past when I talked about the business outcome language.

Similarly, if you are explaining to a customer service representative why the software is behaving the way it is, going into the details of your current architecture will hardly help.

As technologists we often miss the opportunity to communicate efficiently because we tend to pollute the conversation with unhelpful technical details. It is most likely due to our forma mentis and the fact that we spend the majority of our time at the technical level.

If we are explaining why we’re not hitting the originally forecasted date to a team of sales reps, what’s the point in telling them that you need to migrate your current webpack version to the latest one but breaking changes are slowing you down? This level of detail is not helpful nor actionable for them. What is the new delivery date you have confidence in? This is all it matters in this conversation.

When what you’re saying does not make sense to your audience you are contributing to making the communication inefficient. I’ve seen unnecessary technical details pollute the conversation, making it even harder for messages to get across. At times, an answer filled with irrelevant technical details results in the interlocutor shutting down completely, unsatisfied with the answer, and unwilling to ask again.

But how can you make the communication more efficient?

Know who you are communicating with

In my opinion, the first step towards efficient communication is having clear who your audience is. What is their day job, what do they care about? How does what you do impact them? What are they currently struggling with and how is your work going to make their life easier? What can they do about what you are saying?

If you keep these questions in mind when interacting with the your audience it’s going to be easier to refrain from including details or digress into irrelevant explanations.

I know I framed this issue as a dichotomy between technical and non-technical audiences so far, but clear communication might also be lacking within the same department.

Bonus: you might think you know your audience

Even your CTO might be interested in a level of detail that is different from what you normally share with the rest of your team.

Say you are having troubles upgrading to a newer version of a library because it’s introducing backwards incompatible changes. You might feel it would be easy to get down to the technical level to explain why your team is late (again?). After all you are talking to a C T O. They must get it.

I’m sure they do get it but that’s not what they are interested in. If your team is facing unforeseen challenges all is interesting to them is what your team is going to do to overcome them. Have you worked on an alternative plan? Do you have confidence on a different date you are going to be able to deliver the change?

Take the time to prepare for an efficient conversation

Communicating outside the comfort zone of the technical level might require a bit of preparation.

Take the time to prepare. If there is a meeting coming up where you are going to share about the progress of your team focus on what outcome the people in the meeting are going to look for.

Try to understand if you will have an answer for them. Sometimes we default to a technical explanation when we haven’t prepared a more useful and focused answer.

Reflecting on this ahead of the meeting also gives you a better chance to find the answer. As a leader, do not feel you must know the answer. Your job is actually to lead your team to find the answer rather than finding one on your own.

Practice makes perfect. Take the time to think about who your audience is and start practising what you are going to say. Reflect as to whether what you are saying is valuable for your audience or not and if not try again. Try to focus on the details that matter to your audience.

Focus on what’s actionable

Another important aspect for efficient cross-functional communication, in my experience, is how actionable what you are saying is.

For example, it’s not unusual for software teams to have to communicate a date is slipping (I won’t get into the debate on deadlines here, but if you are interested I got into it in this other post): how are you communicating this?

My recommendation is to focus on what’s actionable: are you communicating this to the customer success department? Well, then maybe they don’t care as to why the date is slipping but rather on how they are going to manage this communication with the customer. Therefore, focus on giving a new time frame your team has confidence in.

Focusing on what’s actionable is also a good forcing function to come prepared to the conversation. Don’t just communicate you are late, make the effort to adjust the original estimation and give more actionable information to your audience.

Reflect on how you communicate: are you conveying the information that the audience is looking for? Are you making it easy for them to extract the important bits or are you including distracting details?

Closing thoughts

Communication is an often overlooked aspect of the work of software engineers but, in my opinion, it’s the most important one to achieve efficient and high performing organisations. As individuals, working on perfecting our communication skills is a great investment to maximise the effectiveness of the teams we are part of.

Efficient communication enables teams to iterate more quickly, raise issues soon and course-correct fast because information flows swiftly if the people are deliberately taking care of the right level of detail in the conversation.

Take time to invest in this skill by preparing the conversation ahead, be mindful of your audience and deliberate about the information you are going to share.

The post How to communicate efficiently as software engineers appeared first on Alessandro Diaferia.

Yes, TDD slows you down

Alessandro Diaferia — Wed, 26 Aug 2020 13:03:33 +0000

I recently had the opportunity to reflect about testing practices. In particular, the different layers of testing: unit testing, integration testing, acceptance testing.

The test pyramid suggests that each level has an associated cost. The cost of your tests translates then into the pace at which you can make changes to your system.

Quite frequently, early-stage organisations tend to focus less on unit tests and more on acceptance testing. This might be the case at the early days of a new product. There’s a high fluctuation in requirements and you might feel that unit tests will slow you down when it comes to making radical changes to the implementation.

By not investing in unit testing, though, you are giving up on the opportunity to do TDD. Yes, you will still be able to write acceptance tests before writing any code. But having so coarse-grained tests will not constrain you in the way you structure your code, your components and how they communicate with each other.

Your acceptance tests will guarantee that your system meets the requirements, now.

TDD is not just about testing

TDD is a software development methodology that strictly dictates the order you do things in. And it’s not dictating that order for the sake of it. Writing tests first and letting that drive the functional code you write after is going to impact the way you design and architect the system.

This is going to have numerous benefits: your code will be modular, loosely coupled and with high cohesion, as well as being clearly documented and easy to extend.

Unit testing is one aspect of TDD. It helps you at a fundamental level of your system. The implementation level, that is ultimately going to decide how agile you will be in the future at making changes.

If you develop a system without a TDD approach you won’t necessarily have a system without tests. But you will most likely have a system that is hard to extend and hard to understand.

This is the reason why I’m more and more convinced that you can’t really be that liberal about your testing layers. If you’re building new functionality and decide it is going to solely be tested through acceptance testing, you’re most likely going to miss out on the added benefits of TDD at your service layer.

When you mix TDD with non-TDD you’re not just compromising on the tests at certain layers of your test stack. You’re compromising on the architecture of your system.

Do you really need to go faster?

Of course you do! Who thinks that going slower is better?

☕ J. B. Rainsberger

@jbrains

Worried that TDD will slow down your programmers? Don't. They probably need slowing down.

17:23 PM - 08 Feb 2012

1437 1002

Too bad the question should usually be rephrased to:

Do you really need to go faster now, and go slower later?

I think this way of phrasing it helps making the right decision. As I said at the beginning of this post, going faster now might actually be the right decision. TDD might be what slows you down now, and you might not be able to afford slowing down now. But my advice is not to stop asking that question as you keep making compromises.

When you defer the adoption of TDD you’re not just skipping on unit tests. You’re developing code unconstrained, increasing the opportunities of high coupling and low cohesion. This inevitably makes you slower as you go. And it will keep making you slower until you will inevitably incur in such a high cost of change that implementing even the most trivial feature will seem like an insurmountable challenge. Not to mention this will impact your team morale, as well as the confidence that the rest of the organisation will have for your team.

Be conscious

As I said at the beginning of this post, giving up on unit tests might actually be the right thing to do for you, now. What’s important to realise, though, is that giving up on TDD at a certain layer doesn’t just mean giving up on tests. It means giving up on a framework that guides you through specific principles. If you are not going to do TDD, do you have an alternative set of principles to stick to? Do you have an alternative way that helps you structure your system effectively, to help you scale? Are you going to have the confidence to make changes? Are new team members going to be able to easily on-board themselves and make changes without requiring additional contextual knowledge?

If you enjoyed this post, chances are you’ll enjoy what I post on Twitter. Thank you for getting to the end of this article.

How to keep your Amazon MQ queues clean

Alessandro Diaferia — Thu, 30 Jul 2020 19:56:42 +0000

Amazon MQ queues might fill up if you use them in your tests but don’t take care of cleaning them up. Let’s explore together a way of addressing this issue.

I was hoping to avoid writing dedicated code to just consume all the messages enqueued during tests so I started looking around for some tool I could integrate in my Continuous Integration pipeline.

I found amazonmq-cli. I have to say, it’s not the most straightforward tool when used in a CI pipeline. When it comes to command line options it’s not very flexible and it favours the use of command files to read from. It also only allows for file-based configuration.

Nevertheless, I downloaded and configured it.

Configuring amazonmq-cli in your CI pipeline

Amazon MQ does not support JMX but if your broker does, you could give activemq-cli a try.

For the above reason, this tool needs Web Console access, so make sure that’s configured.

Since I’m running this command as part of a clean-up job in my CI pipeline, I have to automate its configuration.

We need to produce two files:

the configuration file to be able to access the broker (and the web console)
and the list of commands to execute

Once you download and extract the tool you’ll see its directory structure is as follows.

➜ tree -L 1
.
├── bin
├── conf
├── lib
└── output

4 directories, 0 files

There’s not much opportunity for customization, so we will have to produce the relevant configuration file and put it in the conf folder (where you can find a sample one).

First thing, then, is to create the broker configuration.

echo -e "broker {\n aws-broker {\n web-console = \"$ACTIVEMQ\_WEB\_CONSOLE\_URL\"\n amqurl = \"$ACTIVEMQ\_BROKER\_URL\"\n username = \"$ACTIVE\_MQ\_USER\"\n password = \"$ACTIVE\_MQ\_PASSWORD\"\n prompt-color = \"light-blue\"\n }\n}\n\nweb-console {\n pause = 100\n timeout = 300000\n}" > /home/ciuser/amazonmq-cli/conf/amazonmq-cli.config

As you can see, I’ve also included a web-console part that is expected by the tool when performing certain commands like the purge-all-queue one that I needed in this case.

Now, let’s configure the list of commands to run.

In my case all the queues generated in the CI environment during the test share the same prefix. This makes it easier for me to purge them all or delete them later.

echo -e "connect --broker aws-broker\npurge-all-queues --force --filter $MY\_SPECIAL\_PREFIX\n" > purge-commands.txt

Drain them!

Finally, we can perform the command.

/home/ciuser/amazonmq-cli/bin/amazonmq-cli --cmdfile $(pwd)/purge-commands.txt

That’s it! Let me know your experiences with cleaning up queues on Amazon MQ!

The post How to keep your Amazon MQ queues clean appeared first on Alessandro Diaferia.

The Mythical DevOps Engineer

Alessandro Diaferia — Mon, 27 Jul 2020 13:50:04 +0000

I'm always a little suspicious of job specs looking for the so-called DevOps Engineer role. Most of the times, they mention a variety of duties and responsibilities that make you wonder: "Are they hiring for a single role or a whole team?".

If you look around, it's easy to see that roles containing the term DevOps don't share the same meaning across different companies. Often, though, they stress the importance of being able to cover for what traditionally would have been the specialization of different people.

Don't get me wrong: cross-functional expertise is definitely important. But I don't think DevOps means replacing a multitude of specialization with a single role. Different specializations like Operations, Security, Testing, Development, Product Management and so on, are vast and require specific knowledge.

This is why I think the key differentiator of successful DevOps-enabled companies is that they enable them to collaborate effectively, having clear in mind that the goal is to deliver value to the end user.

Overall, I don't think we should be talking about a DevOps Engineer, but rather about DevOps culture in organizations.

But let's take a step back first.

What does DevOps mean, really?

My personal take on the term is as follows.

// Detect dark theme var iframe = document.getElementById('tweet-1279460637341560833-104'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1279460637341560833&theme=dark" }

What I mean by this is that in a DevOps organization the different specialities are incentivised to collaborate. The intrinsic existing tension between Dev making changes to the system and Ops wanting to keep the system stable dissolves in favour of a greater good: the value stream.

A stable system that delivers nothing is of no use to the company the same way an unstable system that keeps offering new functionality is of no use to the user.

The greater good, in this case, becomes the flow of value between the organization and its users. Dev and Ops are incentivised to work together to maximise this flow, understanding which bets worked out and which ones didn't. Being able to easily activate and deactivate functionality, resolve issues and, more generally, adapt and evolve, is what really makes adopting a DevOps mindset worth it.

Overall, I think there shouldn't be a single DevOps role but, rather, a set of specific specialities collaborating effectively.

This ideal view of the terminology might sometimes clash with the reality of the job market. Companies willing to attract the best talent with the most current skills may end up advertising for roles that are counterproductive in the context of DevOps principles.

But let's have a look at a few interesting job specs.

Photo by Jordan Whitfield on Unsplash

What are companies looking for?

Let's read through a few excerpts from job specs I found out there in the wild.

The flexible problem solver

[...] Devops Engineers are IT professionals who collaborate with software developers, system operators and other IT staff members to manage code releases. They cross and merge the barriers that exist between software development, testing and operations teams and keep existing networks in mind as they design, plan and test. Responsible for multitasking and dealing with multiple urgent situations at a time, Devops Engineers must be extremely flexible. [...]

A job spec on the internet

This is one of those classic examples where the organization believes that the DevOps principles should be delegated to a single team.

The spec mentions the myriad of duties that are responsibily of the Devops Engineers in the company. A Devops Engineer is expected to "multi-task and deal with multiple urgent situations at a time". Therefore, they "must be extremely flexible".

Multitasking and dealing with multiple urgent situations at a time is, for sure, likely to happen anywhere: I don't think this should be a peculiarity of a role in an organization. I think the aspiration should be that every engineer in the team is enabled to handle urgent situations and learn from them so that the organization can improve and mitigate possibilities of service disruption.

Urgent situations shouldn't be the norm but, rather, the organization should empower its people to be able to adapt to and learn from such situations so that they can be kept an exception.

The way this role is advertised leads me to think that there is no effort to really adopt DevOps practices: teams are not really incentivised to collaborate and improve but rather the expectation is that there is a dedicated team to throw issues and urgent situations at. This job spec would be a big red flag for me.

The productivity booster

A DevOps Engineer combines an understanding of both engineering and coding. A DevOps Engineer works with various departments to create and develop systems within a company. From creating and implementing software systems to analysing data to improve existing ones, a DevOps Engineer increases productivity in the workplace.

Another job spec on the internet

In a DevOps-_enabled organization engineers _do work with various departments. But what's the point then of having a dedicated DevOps Engineer role? Do the other type of engineers not work with the various departments of the organization? Do non-DevOps Engineers not analyse data and improve existing systems? Additionally, the job spec claims that a DevOps Engineer increases productivity in the workplace. How? Does it radiate productivity?

The Release Manager... but DevOps!

A DevOps Engineer works with developers and the IT staff to oversee the code releases. [...] Ultimately, you will execute and automate operational processes fast, accurately and securely.

My favourite so far

This is quite a condensed one but what strikes me as interesting is the release aspect mentioned in it.

It is quite a complex aspect of the DevOps culture in my opinion. I tend to separate the concept of deployment from the one of release. Product updates as experienced by the user are governed by a release policy that may or may not be the same as the deployment policy. This really depends on the strategy of the organization.

Regardless of this distinction, though, I believe that constraining the capability of delivering value to the end user to a specific role undermines the agility of an organization.

The teams should be able to continuously release code into production. The release of functionality should be controlled through mechanisms such as feature flags so that the code that reaches production does not necessarily activate. This makes it possible for the organization to control when the functionality actually reaches the user.

In general, a deployment should be a non-event: nothing special, just another merge into the main branch that causes code to end up in production. Moreover, releasing the actual functionality to the user should not require a dedicated engineer to be performed: the relevant stakeholders in the company, usually departments other than engineering, should be able to enable the functionality to the user, perform experiments and autonomously decide when it is appropriate to release new functionality.

Job specs like this one feel like they're trying to repurpose the role of the Release Manager to keep up with the latest trends by just changing a few words.

I don't think release management goes away in a DevOps-enabled organization. What changes is that the concept of deployment gets decoupled from the one of release. This is done to enhance the agility of the engineering organization and reduce the risk that's intrinsic of the changes reaching production environments.

At the same time, the relevant technological changes are implemented so that releasing new functionality to the users can become a non-event, in the hands of the strategic initiatives of the organizations.

A Platform Engineer. But cooler!

The DevOps Engineer will be a key leader in shaping processes and tools that enable cross-functional collaboration and drive CI/CD transformation. The DevOps Engineer will work closely with product owners, developers, and external development teams to build and configure a high performing, scalable, cloud-based platform that can be leveraged by other product teams.

This job spec is one of those that I consider the least bad. It describes a set of responsibilities that usually pertain to a Platform or Infrastructure Team. Most of these teams often get renamed to DevOps Team and their members become DevOps Engineers for fashion reasons.

The Platform Engineering team becomes the key enabler for organizations that want to embrace the DevOps principles. But thinking that such principles will only be embraced by that team will hardly result in a successful journey.

The Platform Engineering team will surely be responsible to build the relevant infrastructure that enables the other teams to build on top but they can't be left alone in the understanding and application of those principles.

Developer teams will need to become autonomous in adopting and making changes to those systems; they will need to understand the implications of their code running in production; understand how to recognize if the system is not behaving as expected and be able to react to it.

At the same time, even the product team should spend time understanding what new important capabilities derive from successfully adopting DevOps practices. Code continuously flowing into production behind feature flags, containerization technologies, improved monitoring and alerting, and so on, open endless opportunities of improved user experience and experimentation that should be leveraged to maximise the company competitiveness.

Photo by Matteo Vistocco on Unsplash

What should companies be looking for?

We've just gone through a few job specs that look for variations of a DevOps Engineer role and I've outlined what aspects I think are flawed in those roles. But what should companies look for, then?

Before blindly starting to hire for roles driven by industry fashion trends, organizations should rather invest in understanding what's holding them back from being DevOps.

In the Unicorn Project, Gene Kim mentions the Five Ideals of successful DevOps organizations. I think they're an effective set of principles to take the temperature of your organization in terms of DevOps practices. Those ideals are as follows:

Locality and Simplicity
Focus, Flow and Joy
Improvement of Daily Work
Psychological Safety
Customer Focus

Locality and Simplicity

Making changes to the system, in order to deliver greater value to the end user, should be easy: easy in terms of team's autonomy to make changes to an area of the product as well as being easy in terms of friction that the technology in use imposes on the changes.

Focus, Flow and Joy

Developers should be able to focus on their work and be able to make software with minimum impediments. This is facilitated by making sure that the software development lifecycle infrastructure is working for the benefit of the engineering organization.

Improvement of Daily Work

Continuously learning and improving the conditions in which the work gets done is the key to maximise the flow of value and the happiness of the people doing the work. Successful organizations facilitate a continuously improving environment by enabling engineers to build tools and practices that improve their daily operations.

Psychological Safety

An organization will hardly be able to improve if the people that are part of it are not incentivised to raise issues and discuss them. This is not something you solve for by hiring a specific role. It the organization's responsibility to facilitate an environment where constructive feedback is the norm.

Customer Focus

Last but not least, the engineering organization, just like any other department in the company, should be sharply focused on the customer. All the efforts should be balanced against what's best for the customer and, ultimately, for the company.

What should companies be looking for then? I think the priority should be on understanding what's blocking the organization from fully embracing a DevOps mindset. Once that's established, the expertise needed to get there is probably going to be easier to identify.

The most important aspect for me, though, is understanding the importance of specialities. Every role will have incredible value to add to the journey towards DevOps. What's fundamental is understanding the importance of collaboration between the different roles. The organization will have to put the relevant practice changes in place to facilitate collaboration. Specific DevOps knowledge in terms of technology, tools and best practices, will be required, for sure, but it won't be something a single role should be responsible of.

Photo by Roi Dimor on Unsplash

A mythical role

It feels like the DevOps Engineer is a mythical figure that certain organizations pursue in the hope of finding the holy grail of a Software Engineer capable of doing anything.

This, of course, will hardly be the case. Recognizing the importance of the single specializations is what makes organization successful and capable of maximising the expertise of the people that they are made of.

What happens in a DevOps organization is that responsibilities are redistributed: developers are empowered to make changes to production environments because organizations recognize the importance of moving fast. This means opportunities for success increase together with the opportunities of failure.

Eliminating barriers and creating a safe space for collaboration helps Devs and Ops work together to resolve issues when they occur. This is what ultimately leads to high performing teams that are incentivised to follow the North Star of the continuous value stream to the end user.

Instead of pursuing a mythical role then, let's go after the much more plausible alternative of creating a well oiled machine where all the people are incentivised to work together in harmony with the clear goal of maximising the value to the end user.

Thanks for getting to the end of this article. I sincerely hope you've enjoyed it. Follow me on Twitter if you want to stay up-to-date with all my articles and the software I work on.

Cover photo by Rhii Photography on Unsplash

How I enabled CORS for any API on my Single Page App

Alessandro Diaferia — Tue, 07 Jul 2020 17:14:08 +0000

In this blog post I’ll show you how I used free services available to anyone to build a little proxy server for my app to overcome certain CORS limitations for my Single Page App.

I built Chisel to help with some repetitive API responses composition and manipulation that I was doing at work.

It’s a single page app that allows you to perform requests against any API endpoint and compose results to extract only what you need. It also allows for CSV exports. Pretty straightforward.

Being it still in its earliest days I decided that I wanted to build it with the simplest architecture in order for me to be able to iterate quickly. I went for the JAMstack, built it in React and deployed on Netlify.

Since it doesn’t have a back-end server it talks to, anything you do stays on your machine. Unfortunately, not all APIs allow for cross-origin requests so, in certain cases, you won’t be able to perform any request from your browser unless you enable the proxy functionality.

Proxy feature on chisel.cloud

What happens if you don’t is that your browser will attempt a CORS preflight request which will fail if the API doesn’t respond with the expected headers.

CORS preflight request failure

What is CORS and when is it a problem for your Single Page App?

From the MDN documentation:

Cross-Origin Resource Sharing (CORS) is a mechanism that uses additional HTTP headers to tell browsers to give a web application running at one origin, access to selected resources from a different origin. A web application executes a cross-origin HTTP request when it requests a resource that has a different origin (domain, protocol, or port) from its own.

https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

Now, there are certain requests, called Simple Requests, that don’t trigger CORS checks. Unfortunately, these type of requests are quite limited and don’t allow to pass certain headers like the Authorization one (e.g. a basic-auth request). You can read more about these type of requests here.

For this reason, we’re going to allow a good set of HTTP methods and headers to pass through our proxy and return back the response as unchanged as possible.

The bulk of the work will be configuring the right set of Access-Control-Allow-* headers to be returned back to the browser when CORS preflighted checks are performed. I recommend you have a look at the MDN Documentation to learn more about CORS as it is quite comprehensive.

The proxy

In order to allow any request to pass the CORS preflight checks I built a simple proxy server that returns the expected headers to the browser and passes through the requests to the destination server.

You can find the source code for it on Github, but let’s go through the steps to build your own for free.

Setting up NGINX

The proxy itself is a simple instance of NGINX configured with a server to allow for proxied request to a dynamic destination.

In order to be able to run NGINX on Heroku we have to make some changes to run it as non-privileged user.

We’re basically making sure that NGINX will try to write to unprivileged writeable locations: this is because Heroku enforces that our container runs as non-root. You can read more about it here.

Accounting for any URL

The second aspect of this configuration is actually defining our dynamic proxy: we will translate requests to any URL so that they will expose the right CORS information.

The main complexity of the Chisel case resides in the fact that we want to allow any URL to be proxied. This is because we won’t know in advance what URL the user will type in, of course.

The way NGINX allows for setting up the proxy functionality is through the proxy_pass directive:

Sets the protocol and address of a proxied server and an optional URI to which a location should be mapped. As a protocol, “http” or “https” can be specified.

The NGINX documentation

In order to be able to specify the URL to pass to dynamically I decided to go with a custom header: X-Chisel-Proxied-Url. This way Chisel will use that header to tell the proxy which destination to proxy through to.

proxy_pass $http_x_chisel_proxied_url;

The $ symbol in NGINX is used to reference variables and the HTTP headers get automatically converted to $http_ prefixed variables using the above syntax.

There’s quite a bit of things to go through in this NGINX server configuration. Let’s start with the location / block first.

The first bit in there is the if statement: it handles the CORS preflighted requests case and it basically allows for a bunch of HTTP methods and headers by default. It restricts everything to the https://chisel.cloud Origin, just because I don’t want my proxy to be used by other applications.

proxy_redirect off: I disabled redirects for now. I’m still not sure how I’m going to handle them so I decided to turn them off until I can find a use case for them.
proxy_set_header Host $proxy_host: this is simply forwarding the destination host as the Host header. This is a requirement for valid HTTP requests through browsers. This value will be exactly the same as the one being set for proxy_pass.
proxy_set_header X-Real-IP $remote_addr: here we’re simply taking care of forwarding the client IP through to the destination.
proxy_pass $http_x_chisel_proxied_url: this is the real important bit of the whole configuration. We’re taking the header coming in from the Chisel client application and setting it as the URL to pass through to. This is effectively making the dynamic proxy possible.
proxy_hide_header 'access-control-allow-origin': this, together with the following add_header 'access-control-allow-origin' 'https://chisel.cloud' is basically making sure to override whatever Access-Control-Allow-Origin header is coming back from the destination server with one that only allows requests from our Chisel application.

Finally, the top two directives.

resolver: this is needed so that NGINX knows how to resolve the names of the upstream servers to proxy through to. In my case I picked a public free DNS. You can pick yours from here.
listen $ __PORT__ $ default_server: this one, instead, is the directive that makes everything possible using Docker on Heroku. We will have a look at it later in this blog post, so keep reading!

Building the container image

As mentioned above, I’m going to use NGINX’s base image.

Dockerfile

The Dockerfile is pretty simple. We’re replacing the default nginx.conf with our own to make sure that NGINX can run unprivileged. We’re also copying our proxy server configuration.

As you can see I have named the file as proxy.conf.tpl. I’ve done this to be explicit about the fact that the file is not ready to be used as is. We will have to dynamically edit the port it is going to listen on at runtime before starting NGINX.

As clarified in the documentation, Heroku expects the containers to be able to listen on the value specified within the $PORT environment variable. The solution we’re using here, then, is making sure to replace the $ __PORT__ $ placeholder I have included in the configuration with the actual content of the $PORT environment variable.

Setting up Heroku

We’re almost there. Now we need to configure our application so that we can deploy our container straight from our repository.

Create a new lovely app on Heroku so that we can prepare it to work with containers.

Next, let’s configure the app to work with container images. I haven’t found a way to do it through the dashboard so let’s go ahead with the command line.

Now add a simple heroku.yml file to your repository so that Heroku knows what to do to build the image.

build: docker: web: Dockerfile

Simple as that.

Now, in the Deploy tab of your application dashboard, make sure you connect your repository to the app: this way you’ll be able to deploy automatically.

Heroku Deploy section – Github connection

Your proxy is finally ready to go. Once you kick off the deploy you’ll be able to see it start up in the application logs as follows.

Startup application logs

As you can see, the process is being started using the command we have specified through the CMD directive and the PORT value is being injected by Heroku.

With the proxy up you’ll now be able to forward your requests through the proxy. As mentioned above, you will need to use the custom X-Chisel-Proxied-Url header (or whatever header you decide to configure for your proxy) to specify the original URL the user intended to hit.

As you can see from the animated gif below, the proxy feature allows to overcome the CORS limitation when hitting the Nager.Date API from Chisel.

The Chisel proxy in action

Conclusion

We have just built a proxy server reusing open-source technology. This allows us to keep our Singe Page App separate from the server logic that’s needed to overcome the CORS limitations.

In general, CORS is one of the security measures your browser employs to mitigate certain opportunity for hijacking your website to perform unintended activity. Even if we have just examined an opportunity for bypassing this limitation, always think twice about whether it is appropriate or not for your use case.

I hope you enjoyed this quick walk-through to build your own free proxy server. Don’t forget to follow me on Twitter for more content like this.

This post appeared first on Alessandro Diaferia.

How I used GCP to create the transcripts for my Podcast

Alessandro Diaferia — Thu, 14 May 2020 16:07:54 +0000

I’m currently working on a series of episodes for a Podcast I’ll be publishing soon. The Podcast will be in Italian and I wanted to make sure to publish the episode transcripts together with the audio episodes.

The idea of manually typing all the episodes text wasn’t really appealing to me so I started looking around.

What are the tools out there?

From a quick Google search, it seems that some companies are offering a mix of automated and human-driven transcription services.

I wasn’t really interested in that for now. I was, of course, just interested in consuming an API I could push my audio to and get back some text in a reasonable amount of time.

For this reason, I started looking for speech-to-text APIs and, of course, the usual suspects figured among the first results.

To be quite honest, I didn’t spend too much time investigating the solutions above. I probably spent more time reading about them to write this blog post.

I decided to go with Google Cloud because I’ve never used GCP before and wanted to give it a try. Additionally, the documentation for it seemed quite straightforward, as well as the support for Italian as language to transcribe from (the podcast is in Italian). I also had a few free credits available because I’ve never used GCP for personal use before.

Setting up

If you want to try transcribing your episodes too, follow this quick setup guide to get started.

Head over to Google Cloud and set up an account. Make sure you create a project and enable the Speech-to-Text API. If you forget to do so gcloud will be able to take care of that for you, later.

Google Cloud Speech-to-Text

Second thing I did was installing gcloud, the CLI Google Cloud provides for interacting with the APIs. This time I was only interested in testing the API so it seemed to me that this tool was the only way to get started quickly.

Additionally, there’s not much you can do from the Google Cloud Web Console if you want to deal with Speech-to-Text APIs.

Get your file ready for transcription

Sampling rate for your audio file should be at least 16 kHz for better results. Additionally, GCP recommends a lossless codec. I only had an mp3 of my episode handy at the time so I gave it a try anyway and it worked well enough.

Make sure you know the sample rate of your file, though, because specifying a wrong one might lead to poor results.

You can usually verify the sample rate by getting info on your file from your Mac’s Finder:

Just click Get Info on your file’s context menu

There’s the sample rate

You can read more about the recommended settings on the Best Practices section.

Upload your episode to the bucket

GCP needs your file to be available from a Storage Bucket so, go ahead and create one.

Storage Bucket creation example

You’ll be able to upload your episode from there.

Time to transcribe

Once you have your episode file up there in the cloud go back to your local machine terminal were you have configured the gcloud tool.

Gcloud used to trigger the speech-to-text transcription

If your episode lasts longer than 60 seconds (😬) you’ll want to use recognize-long-running and most likely specify --async.

As I said before, make sure you specify the right --sample-rate: in my case 44100. This will help GCP transcribe your file with better results.

The --async switch creates a long-running asynchronous operation. It took around 5 minutes for me to have the operation complete.

Oddly, I wasn’t able to find any reference to the asynchronous operation from my Google Cloud Console. So, if you want to be able to know what happened to your transcription job, make sure you take note of the operation identifier. You’ll need it to query the speech operations API for information about your transcription job.

The speech operation metadata

The transcribed data

Once your transcription operation is complete the describe command will return the transcript excerpts, together with the confidence rate.

The speech transcript excerpt

I wasn’t particularly interested in the confidence rate, I only wanted a big blob of text to be able to review and use for SEO purposes as well as to be able to include it with the episode. For this reason, jq to the resque!

I love jq, you can achieve so much with when it comes to manipulate JSON.

In my case, I only wanted to concatenate all the transcript fields and save them to a file. Here’s how I did:

$ ./bin/gcloud ml speech operations describe <your-transcription-operation-id> | jq '.response.results[].alternatives[].transcript' > my-transcript.txt

And that’s it!

Conclusion

I thought of sharing the steps above because they’ve been useful to me in producing the transcripts. I think GCP Speech-to-Text works quite well with Italian but, of course, the transcript is not suitable to be used as it is, unless your accent is perfect. Mine wasn’t (😅).

If you want to know more about my journey towards publishing my first podcast follow me on Twitter were I’ll be sharing more about it.

This post appeared first on Alessandro Diaferia's blog.

Is Why more important than How?

Alessandro Diaferia — Tue, 12 May 2020 11:38:34 +0000

As developers, we're often eager to start coding the solution. The problem is, sometimes, we start with the solution in mind, but we're not clear as to what the problem is.

I often see myself wondering what's the cleanest way of implementing what I have in mind: but I soon realize I'm focusing so much on the implementation details that I've almost forgotten why it is that I want to implement a certain functionality.

When I focus too much on the implementation, the purpose of the functionality starts to become less concrete and I almost forget what the value I wanted to deliver to my users was.

I've seen this happen in many situations. User stories specified starting with the How rather than the Why. They, deliberately or inadvertently, mandate a specific implementation. But, in doing so, I think they dramatically reduce the possibilities of really delivering something valuable to the user.

Try start with the Why, instead. Why do you want to build this feature? What is it that you want your users to be able to achieve? By doing this exercise I often find that the implementation that was originally suggested wasn't really the best way to help the user achieve the intended outcome.

**What's your take on this philosophical matter?**

Also, if you want to hear more philosophical questions from me, follow me on Twitter

How I used Chisel to pull stats on Gitlab pipelines

Alessandro Diaferia — Sat, 09 May 2020 13:44:53 +0000

I built chisel.cloud in my spare time to automate something I did to derive insights about my Gitlab pipeline times.

In this blog post I’m going to show you how I did it in the hope that it might be useful to you too.

As you can see from the picture above, Chisel is still pretty early stage. I decided to publish it anyway because I’m curious to know whether something like this can be useful to you too or not.

Understanding deployment time

The goal of this exercise was for me to better understand the deployment time (from build to being live in production) of my project and have a data-driven approach as to what to do next.

Since the project in question uses Gitlab CI/CD, I thought of taking advantage of its API to pull down this kind of information.

Gitlab Pipelines API

The Gitlab pipelines API is pretty straightforward but a few differences between the /pipelines and the /pipelines/:id APIs means that you have to do a little composition work to pull down interesting data.

Here’s how I did it.

1. Pull down your successful pipelines

First thing I did was fetching the successful pipelines for my project.

As you can see, this API returns minimal information about each pipeline. What I needed to do next in order to understand pipeline times was to fetch further details for each pipeline.

Chisel – Transform

Chisel provides a handy transformation tool that uses JMESPath to help you manipulate the JSON returned by the API you are working with. I used it to extract the pipeline IDs from the returned response.

Chisel shows you an live preview of your transformation. Something as simple as [*].id is enough for now. The result is an array of pipeline IDs.

Right after obtaining all the IDs I need I can apply another transformation to turn those IDs into pipeline objects with all the relevant information I need for my stats.

Chisel has another kind of transformation type called Fetch that helps you transform the selected values into the result of something fetched from a URL.

In particular, you can use the ${1} placeholder to pass in the mapped value. In my case, each ID is being mapped to the /pipelines/${1} API.

The result is pretty straightforward.

2. Filter out what you don’t need

As you can see, some of the returned pipelines have a before_shaof value 0000000000000000000000000000000000000000. Those are pipelines triggered outside of merges into master so I’m not interested in them.

Filtering those out is as simple as [?before_sha != '0000000000000000000000000000000000000000]

The transformation history

As you can see, on the right of the screen there’s a little widget that shows you the transformations you have applied. You can use it to go back and forth in the transformation history and rollback/reapply the modifications to your data.

3. The last transformation

The last transformation I need to be able to start pulling out useful information has to turn my output into a set of records.

I’m selecting only a few fields and turning the result into an array of array. This is the right format to be able to export it as a CSV.

Google Sheets

Finally, I can upload my CSV export to Google Sheets and plot the information I need.

Conclusion

Chisel is still at its earliest stage of development and it is pretty much tailored on my specific use case but if you see this tool can be useful to you too, please head to the Github repo and suggest the improvements you’d like to see.

If you liked this post and want to know more about Chisel, follow me on Twitter!

Featured image by Dominik Scythe on Unsplash

The post How I used Chisel to pull Gitlab pipelines stats appeared first on Alessandro Diaferia.

chiselcloud / chisel

The Chisel online app

How do people package simple backend-less utility apps made in JavaScript these days?

Alessandro Diaferia — Tue, 21 Apr 2020 13:33:04 +0000

I'm considering releasing a small utility application that helps me pull and compose data from HTTP APIs. What's the best way these days to package such an app that doesn't require any backend but needs to perform network requests?

Any good resources for distributed transactions?

Alessandro Diaferia — Tue, 07 Jan 2020 10:55:41 +0000

I'm looking to read on distributed transaction best practices and tools. Any good resources out there?

Continuous deployment with GitLab, Docker and Heroku

Alessandro Diaferia — Sun, 29 Dec 2019 18:07:24 +0000

Continuous Deployment refers to the capability of your organisation to produce and release software changes in short and frequent cycles.

Pain vs Frequency relationship – https://www.martinfowler.com/bliki/FrequencyReducesDifficulty.html

One of the ideas behind Continuous Deployment is that increasing the frequency of deployment of your changes to production will reduce the friction associated with it. On the contrary, deployment is often an activity that gets neglected until the last minute: it is perceived more as a necessary evil rather than an inherent part of a software engineer’s job. However, shifting deployment left, as early as possible in the development life cycle, will help surfacing issues, dependencies and unexpected constraints sooner rather than later.

For instance, continuously deploying will make it easier to understand which change caused issues, if any, as well as making it easier to recover. Imagine having to scan through hundreds of commit messages in your version control system history to find the change that introduced the issue…

Automatism is key to achieve continuous deployment.

The project

In this article we’re gonna explore how to leverage tools like GitLab Pipeline, Heroku and Docker to achieve a simple continuous deployment pipeline.

Let’s start by creating a simple Hello World application. For the purpose of this article I’m gonna use Create React App:

$ npx create-react-app continuous-deployment
$ cd continuous-deployment
$ npm start

Now that we have a running application, let’s build a Docker image to be able to deploy it to Heroku.

The container image

We’re going to write a simple Dockerfile to build our app:

FROM node:10.17-alpine
COPY . .
RUN sh -c 'yarn global add serve && yarn && yarn build'
CMD serve -l $PORT -s build

First of all, two things to keep in mind when building images for Heroku:

Containers are not run with root privileges
The port to listen on is fed by Heroku into the container and needs to be consumed from an environment variable

As you can see from the Dockerfile definition, we are starting the app by passing the PORT environment variable. We can now test the image locally.

$ docker build . -t continuous-deployment:latest
$ docker run -e PORT=4444 -p4444:4444

The -e PORT=4444 specifies which port we’re going to listen to. You can now try your application at http://localhost:4444.

Additionally, I’ve added a myuser user at the end of the Dockerfile, just to make sure everything still works with a non-root user.

Deploy to Heroku

Before building our continuous deployment pipeline, let’s deploy manually to make sure our image is good. Create a new application on Heroku and give it a name. In my case it’s gonna be cd-alediaferia.

Now let’s tag and push our image to the Heroku Registry after logging in.

$ heroku container:login
$ docker tag <image> registry.heroku.com/<app-name>/web
$ docker push registry.heroku.com/<app-name>/web

And release it straight to Heroku:

$ heroku container:release -a web

You should now have your app successfully up and running on Heroku at this point.

The GitLab Pipeline

In this paragraph, we’re going to configure the pipeline piece on GitLab so that we can continuously deploy our app. Here follows the .gitlab-ci.yml file that I have configured for my repository.

In the above snippet we have defined two jobs: build_image and release.

`build_image`

This job specifies how to build our Docker image. If you look closely, you’re actually going to notice that I’m not using Docker specifically but Buildah. Buildah is an OCI-compliant container building tool that is capable of producing Docker image with some minor configuration.

`release`

This job performs the actual release by pushing to your Heroku app.

Additional configuration

Before trying our pipeline out, let’s configure the HEROKU_API_KEY so that it can get picked up by the herokucli that we’re going to use in the pipeline definition.

GitLab pipeline variable setting

Pushing to GitLab

Now that we have set everything up we are ready to push our code to the deployment pipeline.

GitLab pipeline in action

Let’s have a look at the build step that GitLab successfully executed.

GitLab pushing to the Heroku Registry

The first line uses buildah to build the image. It works pretty much like docker and I’ve used --iidfile to export the Image ID to a file that I then read from the command-line in the subsequent invocation.

The second line simply pushes to the Heroku Registry. Notice how easily I can log in by doing --creds=_:$(heroku auth:token): this tells buildah to use the token provided by Heroku to log into the registry.

The deployment job, finally, is as easy as:

$ heroku container:release -a cd-alediaferia web

Conclusion

My app is finally deployed, and everything happened automatically after my push to master. This is awesome because I can now continuously deliver my changes to production in a pain-free fashion.

My successfully deployed app

I hope you enjoyed this post. Let me know in the comments and follow me on Twitter if you want to stay up-to-date about DevOps and Software Engineering practices.

This post appeared first on Ale's main thread.