WTF even is Software Engineering?

While figuring out why coding is so overcomplicated it occurred to me that the majority of it isn't due to the coding itself. But rather the whole SDLC is to blame. But what actually is that? And what should we be doing? And how do we keep it as simple as possible?

Waterfall

Back in the day we used to do waterfall, which has its pro's and cons... The projects we used to run were quite ok, at least we managed to meet the deadline with all required features and the quality was good enough. Now I'm not here to promote waterfall (although Iguazu is absolutely stunning) but it does nicely illustrate what is needed for software engineering:

Why is this relevant? Whatever your methodology, spend the fucking time! Don't skimp on analysis, design, dev, QA or ops... it will bite you in the arse!

Agile

Now fortunately we moved to a more iterative approach where we can evaluate what we've built more frequently. As said, you still need to spend the time but the big difference is that it's a lot more intertwined. Especially QA, deploy and run have become an integral part of the cycle... and honestly it's fucking awesome!

This is an indication of how it could look... it kinda implies Sprints but a more Kanban style approach would work equally well. Most importantly ensure everyone is involved on a continuous basis...

Team work

For me the best part about agile (aside from it being called ASD) is that it forces you to work as a team. Everybody has to be communicating with each other otherwise it's not gonna fly. I can't stress this enough:

• PO, Architecture, Dev, UX, QA & DevOps all need to be fully aligned!

Any meeting you have on a feature, from conception to deployment, make sure everybody is in it. And for the love of god, take the time to discus... assumption is the mother of all fuck-ups. Even if you think everything is clear, it isn't!

Requirements

I like requirements analysis as a means to determine what is needed as it ignores the question of how something gets implemented. Don't let a PO come up with a solution, that's what UX, Software & DevOps engineers are for. Sorry PO's, no it's not your job... by all means raise suggestions and examples of how things can be solved but leave it to the team to come up with the how and jointly validate it meets the requirements.

Also please ignore the customers' cries that "it has to work like this otherwise it's useless" because that ruins the product... think about all the other customers and really distill what the underlying problem is... what is required?

Reporting

And the first requirement to consider is:

• As a Product Owner, I must receive usage information for feature X, so that I can validate it's useful to our customers.

Normally we leave this till the very last moment and consequently don't implement it... but please please please do this first, you want to prove something is useful or not... cuz if it isn't... BIN IT! There's nothing worse than useless crap in your product... not only is it cumbersome for your users but it also has to be maintained.

Architecture

This may very well be the most important piece to allow the whole thing to work... but it deserves a whole separate post... or even a series... or rather go get a proper architect and consider what is needed for your organisation, cuz there is no one-size-fits-all.

That said, there are some pointers:

• Like software your architecture should be flexible... it's a balancing act between solid guidelines and change to facilitate the ever changing requirements
• Reducing dependencies and compartmentalization helps to create a more sustainable environment
• Hire an architect, someone who knows its shit and actually has time to solve things in the long term... when you build a house you don't just throw builders at it, there is a structure

DevOps

You'll eventually need to be able to enable a feature in production, but there are a whole host of things that need to be sorted before that:

• Logging for monitoring & incident management
  • Like reporting this is always considered last, don't make that mistake!
• Automatic provisioning of your infrastructure
  • Doesn't matter whether you do cloud or on premise, automate it
• DX... Developer eXperience is a thing
  • Enable your developers to focus on their core task
• Automatic validation of code
  • Code scans, QA, linting, the whole nine yards
• Automatic deployment to various environments
  • If you need to manually login, you're doing it wrong
• Feature flags for manual enabling of features
  • This is your actual go-live, not the deployment to production

In case this isn't clear yet, DevOps is the fundamental backbone of everything the teams does, not just ops. The more you can automate the better... There's a reason we're moving to Platform engineering.

Design

Less is more, KISS... we as users already got too much shit being thrown at us, we don't need more. So keep the UX as simple as possible, distill till you nailed it.

Also from a technical/architectural perspective try to keep it as simple as possible. Design it so it's easy to implement and have as few dependencies as possible. This does however imply an architecture that facilitates compartmentalization so features don't bleed over into one another.

There's a lot more to be said about design but that's defo a topic for another day!

Coding & Validation

These 2 things have become so intertwined (at least in my twisted mind) that I think we shouldn't be considering them as separate disciplines. When you've built a brick wall, you don't just walk away... you check whether it's according to spec. Do the same with the code.

I'm still not sure what the best approach is when it comes to validation but a substantial part involves the dev... or inversely the QA engineer is also to a large extent a dev. That person may not per se write the unit tests but it makes sense to have tests written in the same way so they can be created, used and improved together. Keep in mind that tests are a ridiculously valuable part of development, it can make your life so much easier!

But don't make the mistake of assuming Dev & QA are the same person! They have a distinctly different focus and you don't want to have devs grade their own homework.

So how do we KISS?

You will need to do all this, however begin as simple as possible and build from there:

• Define & implement a base architecture
  • Spend time on this please and ask for help
• Define how you analyse & design feature with the team
  • Tickets, whiteboards, design tools, tech specs, whatever floats your boat as long as it's clear to the whole team
• Define your reporting requirements and implement basics
• Setup basic automations for coding & validation
  • Simple things like checking in your .vscode directory is already super useful
• Setup basic automatic infrastructure provisioning & deployment scripts

Stick to the basics, don't make it too big... just make sure you at least start.

Rinse and repeat

Use frequent retrospectives to adjust course and do bigger sessions every couple months to look at the fundamentals.

• Continuously evaluate how well you're doing
• Find standardisation opportunities in everything you do (design, coding, QA, DevOps, architecture, everything!)
• Automate the fuck out of your SDLC