DEV Community

loading...

What are good ways to learn software architecture and systems design?

mikkpr profile image Mikk Pristavka ・2 min read

I get anxious when working on back-end code. Data loss, security breaches and scalability/performance issues seem to be lurking around every corner if I don't pay attention. Sure, I can come up with a solution to most problems thrown my way, but with every design choice there's always the thought that maybe I didn't think it through and I forgot about a critical part of the system and it will have serious ramifications in the future.

This feeling is not only at my day job where I work on a fairly complex Rails app, but also in my hobby of game design and development — sketching out ideas and hacking together a prototype is one thing, but building it so that the code is maintainable, scalable, secure and easily understood from the start feels like black magic.

Of course, architecture and design patterns are not only a back-end thing. I've learned to write more or less maintainable code over the years and always seek ways to make it more flexible and concise, but the end result is just cosmetics anyway and the real magic happens elsewhere, so it doesn't feel as important.

Aside from just reading and understanding the source code I'm working on and familiarizing myself with source code of other, open-source applications, what are other good ways to learn the "correct mindset" of building systems? I'm most interested in how to think about the system as a whole and how to divide it into parts that make the most sense, given the overall problem the system is trying to solve; How to think about data flow, single responsibility, patterns that can be applied etc. How and where should I even begin?

I have no idea what I'm doing

Discussion (22)

Collapse
bhilburn profile image
Ben Hilburn • Edited

I've spent a significant portion of my professional career working on exactly these sorts of architectural design decisions. At this point, I don't think I could point you to that "one resource" that I consider a definitive guide. Books by Scott Meyers are gold (non-affiliate Amazon link). I also recommend this book on C++ Design Patters, if that's a language you use.

I think the most influential and instructive thing I ever did was writing a small kernel. In school, I took an OS class where we spent the semester implementing a kernel (specifically, a Pinto kernel). I had already been programming for ~8 years when I took the class, but that experience fundamentally changed the way I approach code and design. If you aren't a student or your school doesn't offer something like this, there are a number of projects that guide you through doing this kind of thing on your own as a self-guided learning experience.

Writing your own kernel might sound a bit terrifying, but looking back at, I really had no idea what I was doing when I started. I never would have dreamed that I could write a thread scheduler, a filesystem, a memory management system, etc., - those all seemed like things that could only be done by CS experts contributing to Linux. Once you've gone through something like that, though, thinking about things like dataflow, scalability, and architecture feels much more natural.

It's not quick & easy answer, but in terms of my skill and comfort level with the kinds of topics you're talking about, nothing has been more important. I'm sure there are other projects like that which would be equally as instructive if that kind of thing isn't of interest to you :)

Collapse
mikkpr profile image
Mikk Pristavka Author

I'm very slowly working on a roguelike game, with the intent of using the Entity-Component-System pattern myself instead of letting a game engine handle it for me. This has already taught me many useful lessons on algorithmic complexity with many moving parts, composition of smaller parts to create a larger whole, and separation of concerns and decoupling. Everything very useful on the front-end as well and as a bonus, I can work on something I really enjoy creating. Not as complex as writing a kernel from scratch, though :)

Collapse
cjbrooks12 profile image
Casey Brooks

It doesn't need to be a kernel you write, I believe that if you write any sufficiently complex piece of software from scratch with the intent of making it scalable and extensible, you will come out on the other side with a much better idea of design patterns and clean code architecture. While I have not built a kernel, my baby is a static site generator, Orchid, and a complex game can also be a great example.

Projects with a clear focus that is comprised of many disparate pieces are the best, as you have to find ways to manage each system independently while also having them play nicely together as an entire system. A kernel runs processes, but that requires filesystems, networking, and memory management, which are all very different. Orchid builds large static websites, but requires knowledge of parsers, filesystem abstraction, and I even implemented a simple HTTP server for an admin panel. A game has a pretty clear goal of playing the game, but involves a rendering/graphics engine, a game logic engine, data storage and state management, maybe even AI.

To do this, you'll have to give yourself strict constraints and promise to stick by them, which is much easier said than done. You have to go slow and constantly refactor, and find that single code path which everything else is attached to. If you don't explicitly find this path, then you run the risk of doing too much with too little governance, which is where code rot starts to creep in. In strongly-typed languages like Java, you can have the compiler enforce this with package boundaries and multi-module Gradle projects. In other languages, you'll have to be the one enforcing these rules of which methods are allows to be called, which fields are allowed to be changed, etc.

In building Orchid, I actually kind-of "rediscovered" several GoF design patterns, just because I got backed into a corner and needed to really think about how to find my way out. And when you start working on a way to get out of the corner, actually take the time to go and apply that same solution to other areas in the app, and see if it stands the test of scaleability. If it doesn't, refactor and find a better way. If it does, you've now got a new tool in your toolbelt to help you solve even harder problems.

Collapse
bhilburn profile image
Ben Hilburn

That's fantastic!

Also, you might be surprised. Since school, I couldn't tell you how many times I've looked at a "not as complex" problem and thought, "Oh, this is basically just a kernel component but in a different context." It could very well be the same for what you are writing, now :)

Collapse
vbjelak profile image
Vladi Beeblebrox

Can you point to some resources that would guide me through journey of implementing Kernel or link to Pinto should be good enough? Thanks for sharing

Collapse
rhnonose profile image
Rodrigo Nonose

For more back-end stuff, here's a few links I enjoyed (and still studying):
The System Design Primer
Awesome Scalability
Backend Best Practices

Collapse
theodesp profile image
Collapse
migueloop profile image
Miguel Ruiz

Awesome Scalability --> Top

Thanks Rodrigo!

Collapse
bgadrian profile image
Adrian B.G.

I learn from examples, in the last few months I gathered an youtube playlist with real "big" examples architectures.

If you want to go deep and also train for an interview, the System design from the interviewBit is the best resource I could find (as of quality).

As for the books, if you are interested how the distributed systems work (databases, cloud services, back-back end stuff), this is the "bible"

Unfortunately most of the "architecture" books are old, built with the Java/C++ monolith set of mind (outdated), so there aren't so many good ones left out there, but a few were mentioned in the other comments. There are some micro-services dedicated ones, maybe a few for Lambda, but I haven't found one that presented all types of "current" types of architectures, with examples.

If you are into gaming systems, a new book was launched, I haven't read it but it seems very promising and has good reviews: Game Programming Patterns

Collapse
phlash909 profile image
Phil Ashby

Lots of good reading material here, but nobody has mentioned Mr Fowler yet, martinfowler.com/design.html :)

If you are thinking about security (and who isn't?) while feeling a bit like an imposter, then I would suggest a rummage round Pluralsight to find an overview and maybe some deep dives in areas of interest (such as web application security - at work we use Troy Hunt's course as formal accreditation for peer reviewers). I also heartily recommend talks by Kelly Shortridge such as this on resilience: youtube.com/watch?v=ux--pHFpeac she has an ability to keep things understandable while covering a subject very clearly.

Collapse
ghost profile image
Ghost

Like Ben, I have been at this quite a while and completed many projects in my life. I can't really nail one single thing down that made me become confidant in my architectural abilities but I believe it just came down to the experience of doing it more than several times. I think it really comes down to confidence in your abilities from experience.

The few tips I can offer from experience are...

  1. Get a whiteboard if you don't have one already. This is where I draw out my initial thoughts about a project and it's processes and architecture then later use it to visualize flows and relationships.

  2. Read, Read, Read... Pick up some good books like Eloquent Ruby, Design Patterns: Elements of Reusable Object-Oriented Software, and other books that help you build your skills in a thoughtful way.

  3. Don't get overly anxious about your code or architecture. As you complete more projects through the years your confidence will grow as will your knowledge. Go with what you know NOW.

Good luck to you!

Collapse
emkographics profile image
Emko • Edited

Most of the resources on here are links to half baked material.
But check this: educative.io/collection/5668639101...

It is not free, but will teach you a crazy amount by learning from the big players and their architectural patterns. There are a few free chaptere to give you an idea of the flow of the course. Hope that helps.

Collapse
mikkpr profile image
Mikk Pristavka Author

This is amazing - I read through the first few preview chapters and really liked how the information is presented. Thanks!

Collapse
val_baca profile image
Valentin Baca

Check The Architecture of Open Source Applications out!

It's wonderfully well written, though each one is in a different style.

Collapse
mikkpr profile image
Mikk Pristavka Author

Yes, this is one of the things I'm currently reading. Good stuff!

Collapse
imben1109 profile image
Ben • Edited

After a few years of working in software industry, I agree that software architecture or system design areextremely important in the beginning of a project. It definitely determines the success of a project.

Actually, there are many design pattern available for us to study. The below is my list of Design Pattern for study. If you have list, please tell me.

Collapse
mpermar profile image
Martín Pérez

I've read tons of books about the topic but if I have to pick one it would be Release it from Michael Nygard who has also tons of good material about the topic .

Collapse
abbottjam profile image
James Abbott

Developing a functional programming mindset will go a long way to making you a better engineer, since software architecture and engineering are extensions of programming. I'd say, study how functional languages (Clojure, Elixir, Haskell) solve the classic problems of statelessness, concurrency, and distributed architecture.

Here are a couple more resources:

pragprog.com/book/swdddf/domain-mo...
pragprog.com/book/mkdsa/design-it

Best of luck.

Collapse
antonfrattaroli profile image
Anton Frattaroli • Edited

Being a good architect is about awareness. And for that we turn to Donald Rumsfeld: ‘There are known knowns. There are things we know that we know. There are known unknowns. That is to say, there are things that we now know we don't know. But there are also unknown unknowns. There are things we do not know we don't know.’

And so you'd focus on eliminating the unknowns by identifying the paths to discoverability.

There's the hands on approach, actively seek out the problems and solutions of different fields - such as taking on a game project as a web developer.

Working with experts in adjacent fields can help discovering problems you didn't know existed - I wouldn't have known about firewall microsegmentation if it weren't for some proactive network security engineers.

Keeping up on current events can be illuminating - I just saw a tweet about blockchain tokens: a solution for a problem I was unaware of.

The architects I know like conventions. I think it's important to research counterpoints to what's presented afterwards.

Vendor presentations can be enlightening. They're catering to many more companies than just your own.

I'm sure there are more ways to increase awareness. Known unknowns I guess.

Collapse
angelinsharmila profile image
Angelinsharmila

You can visit Sanfoundry website to find the Best Books available for Software Design and Architecture. You can click the link below to find the Books sanfoundry.com/best-reference-book.... These books are used by students of top universities, institutes, and colleges.

If you consider learning online you can visit Sanfoundry website where you can find 1000+ Software Design and Architecture questions and answers from sanfoundry.com/software-architectu...

You can register in Sanfoundry Rank website and can take the test from rank.sanfoundry.com/software-archi... to test your knowledge on the latest trends on Software Design and Architecture.

Collapse
sidd081 profile image
Siddharth Agarwal

Found one android app which curates system design question and different approaches about how to solve them. It also covers many other CS Fundamentals and other material

play.google.com/store/apps/details...

Collapse
kspeakman profile image
Kasey Speakman

I haven't had a structured path while learning architecture. I voraciously read (and re-read) articles on lots of different aspects or architecture and then try to piece it together in my mind into something implementable. There are so many implicit assumptions in arch articles, that they often merit going back to re-read later.

For example, you often hear load balancers mentioned for scalability. But that doesn't really "solve" any scalability issue... it just shifts the burden of scalability from your services to your database. And databases which are scalable (i.e. generally not SQL) have different constraints and problems in their usage for dev and ops. So you really have to take things one at a time and address the constraints important to your business (all the -ilities). I'll include one of my favorite arch videos below. There is a bit of philosophy at the front, but he gets into it later on.

Forem Open with the Forem app