DEV Community: Justin 👨🏾‍💻

How do you begin architecting a solution in AWS? Part 1

Justin 👨🏾‍💻 — Mon, 10 Jan 2022 12:16:30 +0000

Every so often we may be presented with an idea or business problem that needs to be solved. Often as engineers or architects we are the ones that are presented with the challenge of coming up with a way to solve it. Do you ever wonder "Where do I start?". In this series of blog posts, I will break down how I like to approach these to deliver a solution that works.

Use case

Let's say we wanted to create a shorturl service that anyone on the internet could use hosting it in AWS. For this use case, we will keep the feature set fairly basic. We want to be able to:

Create a short URL

We want a user to be able to go to our website http://urlshort.com/ for example, input a long URL that they want shortened and have this optionally emailed to the for their records.

The short URL will look something like http://urlshort.com/XmPLe which would be in place of a longer URL such as http://mywebsite.mydomain.com/downloads/2021/april/files/private/shared/trailer.mov

Access the real URL using the short url

We want a user to be able to enter the short URL http://urlshort.com/XmPLe into their web browser and be redirected to the real URL such as http://mywebsite.mydomain.com/downloads/2021/april/files/private/shared/trailer.mov

Beginning the process

The stages of my process usually follows a pattern similar to this:

1. If I was to do this as a human, what does the process look like from a high level or how would I do this physically?

Lets say the long url is someone's name and address (real URL / value) that they have asked me to store. I could write these details in a green notebook (website/frontend and datastore). Each name and address would appear on its own page in the notebook starting from 1. Let's say we have 10 different entries across 10 pages numbered 1 – 10 (keys). With this we have a way to translate a short url (key) to a real url (value).
If I or the user wanted to add a new entry, I would open the green notebook, go to an empty page, say page 11, check if it is empty. If it is, I would enter new details. If it was not empty, I would go to the next page (creating a different key) until I found an empty one to create the entry (API / backend logic).
After creating a new entry, I might write down the key on a piece of paper saying "Dave's details are on page 11 in the green notebook" and hand this back to the requester to share with anyone they wanted to (notifying a user to remind them where the details are).
Someone might come to me and say "I want to see the details at number 7 as I need to retrieve/use this address". I would go to the green notebook, find the key or page "7" and get the details. From there I would return the details to the requester (API / backend logic).

This analogy and human process translates to what our shorturl service would be doing.

2. "How would I mirror that human process and do this using a single machine/server ONCE?"

One way could be to create a text file in a folder with a unique name and add the information I need stored inside that text file. If we follow the number pattern from 0, our file name would be 0.txt and our real url would be stored inside that file. To retrieve the details I'd open the file and copy / paste the url into the browser.

The goal here isn't to work out the most efficient way to do this once. It's just to do it once. Working out the most efficient way will come later. We generally refer to that type of thing as an "implementation detail" which is something to be solved later.

3. "How would I do this TWICE? How would I do this X times?"

If we refer back to point 2, we have a very basic approach to doing this once. Doing multiple times could range from a number of approaches.

One way would be a program/script with a menu. One option would prompt the user for a value to be stored, the script generates a unique file to store the value and gives the user the filename as a key. Another option on the menu would prompt the user for the key. Once input, opens the file, reads the value and returns this to the user.
You could also use the previous approach but store the data differently. You may choose to store all the data in one file, like a json file called urls.json and append new entries to the file.
You may even also install a small database on the local server to store your data there.

Again, the goal here aren't to work out the most efficient way to do this; it's just to get a feel for our options.

4. What are the main components we have identified?

For example "Do we need a webpage? Do we need to store any data? How / where do we store the data? etc". So far it would look like we need the following to meet our use case:

A website / frontend (for users to input the real url details)
A datastore (to store details of the short / real url)
A backend API or program (to create, add and retrieve values from the datastore)
A way to send notifications (to let a user know the details of the short url)

Once I have covered these main areas to form a basic outline of the main components, I would then be able to evaluate which AWS services fit the use case.

Important considerations

It is important to remember this is one of many approaches when trying to come up with a solution to implement. As there is some ambiguity in the use case in what exactly is required (5 Pillars of the AWS Well Architected Framework anyone?), we would likely have more questions to clarify the requirements and not do more (or less) than is required. This approach can help us to avoid skipping steps and optimise for a better solution. However in the "real world" there are a number of other important considerations that you should factor in with your approach. Some of these include but not limited to:

How much time and resource do we have to finish the project?
What are the cost constraints?
What are the absolute "red lines" or boundaries that cannot be crossed? For example stakeholder / business deadline, must meet a regulatory standard or data must be kept for minimum of 5 years for reporting etc
Do we have to learn new technology? Can this be done without affecting a currently deployed application?
What sort of performance do we need? Do we need it to be highly available, fault tolerant, redundant, secure and scalable?
Do you want a monolith solution or make use of a microservices architecture?

Coming up in Part 2

In the next blog post we will begin to look at each of the main components and evaluate which AWS Service or combination of services would help us achieve an end to end solution.

I wish I knew this when I was younger...

Justin 👨🏾‍💻 — Sat, 02 Jan 2021 16:49:03 +0000

Do you ever look back back and think, "I wish that I knew what I know now when I was younger?" I often do this as I discover something that was right there in my face my whole life. Usually, it would be a case where someone older or with more experience would be giving advice and I would listen, but not take the right action on it. Years later I would then come to a place and think "if I had just done X like so and so said..."

If you've ever felt like that, this post is for you. You'll be able to see what you should do along with why you should be doing it.

TL;DR

You don't need to know everything about everything
Focus on the basics, learn the advanced things as you need them
Be vocal about what you are learning
Be as helpful as possible while maintaining your priorities
Teach / share the knowledge you have with others
Know what you bring to the table
Small consistent improvements = big returns later
Learn how to disagree in a good way
Don't be shy to ask for help

You don't need to know everything about everything

Honestly, you don't. Working in tech there is always something new to use/learn - a new programming language, a new framework, a new monitoring tool... While it is always beneficial to know a wide scope of what is available to fit your use case, you can't possibly know it all and you don't even need to in order to be effective at what you do. You can learn what you need to know when you need to know it.

Focus on the basics, learn the advanced things as you need them

Following on from the previous point, what you do end up getting to know, focus on the basics instead of everything about said tech/subject. You'll go on a course/study for an exam or go through tutorials and they will often cover beginner to advanced material. A lot of the intermediate/advanced parts are for specific use cases which you may never encounter. Not to mention the more advanced topics are often just layers and layers of the basics.

This is why focusing on the basics is important - it will allow you to understand more complex matters as well as filter out what you might not need to solve a problem.

Be vocal about what you are learning

This lets others know that you are learning a particular skill, especially people with influence. You don't need to mention it 600 times a day but in the odd conversation find a way to bring it up.

"How was your weekend?"

"My weekend was good, I spent a little time reading up/practicing on X. I think it is something that can be useful for us so I'll be making my suggestions soon!"

This is important because it gets people used to you being associated with what you are learning. Moreover, it holds you accountable to yourself to see it through. In the work place when there is a problem or a new project on the horizon, people will seek you out/give you a chance once they are used to you becoming confident on the subject. This will help you stand out from the crowd and get an early advantage.

Be as helpful as possible while maintaining your priorities

People tend to remember those that have helped/tried to help them, especially if the help came with a positive attitude. We have all met that person at work that you you only ask for help when there is nobody else to help...

"This isn't my job, why are you asking me?"

"If they listened to me you wouldn't be having this problem"

"I'm only doing this because I don't want your boss to moan at my boss about me"

Don't be that person. Be more like:

"This isn't my area of expertise, but have you tried X? If that doesn't work, I know that David knows more about this. Let's see what he thinks"

"Sure, I can do this for you in about 30 mins as I'm just finishing X. Send me the details and message me again in an hour if I haven't done it by then."

"Normally the Tech Support team will be this but it won't take long to do this quickly, as I know about it also"

The reason why you should be doing this when it doesn't get in the way of your main priorities, is you build social capital and reputation. People are more likely to return the favour and help those they like. They trust you more. It increases the value of your work network. When it is time for pay reviews, promotions or new projects, management will ask others how they feel about you and positive feedback will only improve the rewards you get.

Teach/share the knowledge with others

Giving others some insight into the things you have learned improves your work life for a number of reasons:

There will be less demands on your time to do things they could do themselves without much effort.
You build a good reputation for giving guidance.
In the event you're on holiday and there is a problem which is relatively easy to fix, but the knowledge hasn't been shared - this can turn into a much bigger problem where they will call you to fix.
You can evaluate where you are, and where you need to improve.

This is especially important if you want to advance into management/senior positions. People tend to look up the management chain for direction when a decision needs to be made.

This leads me on to the next point:

Know your worth

Be brutally honest with yourself about what you bring to the table - what you can/can't do, what you are willing to do, etc. You have to know your strengths and weaknesses in order to show a company why you should be employed/paid more. You can work this out in a number of ways:

What skills do I have & how competent am I?
Do I need help when using these skills or am I pretty much self sufficient?
What seems to be the salary range for jobs asking for these skills?
What is my reputation/social capital like?
What are my weaknesses?

Having an idea of where you currently are will help you evaluate what/where you can improve, if you are on track for your own career goals and if you should be paid more. If you decide you should be paid more, you'll need to have a good idea in order to give your salary negotiation the best chance (more on this in another post).

Small consistent improvements = big returns later

Spending a little time inside or outside of work learning things to help you during work is very beneficial. Just 15 mins a day watching a YouTube tutorial/playing around with what you've learned at a basic level, listening to a podcast about something useful, will help more than you realise.

Small gains add up to big returns. If someone said they would give you £1 today if you took 5 mins to take some shopping up some stairs for someone, you might think,

"it's just £1, I won't do it".

If they came back and said they would give you £1 for every day you spend 5 mins taking shopping upstairs to that same person, you might think,

"5 mins every day for £1 would be £365 by the end of the year!" and say yes.

Think about investing in yourself in this way.

1% better every day for a year = 37% improvement overall

That's just with a small time investment. You can do this on your lunch break, or travelling to and from work if you really can't afford to give up any time in the evenings. You can also do it while cooking, tidying up or even at the gym walking on a treadmill!

This type of improvement is also a lot easier to manage day to day vs trying to learn a lot in a short space of time. Think about it, it took us a long time of learning every day to be able to do maths or write sentences. So, why do you think you'll be able to learn and be competent in complex, modern day systems in a 2-3 day course? Our brains need time to process the information we receive - make it easier on yourself.

Learn how to disagree in a good way

Luckily for me, I learned this very early on! We've all been in a situation where our manager tells us "We're going to do this because X, Y, Z". The team who does the work knows it is a terrible idea, and is obligated to being the bearer of bad news.

How do you respond?

"That isn't a good idea. It won't work. I'm not doing that as it will make things worse, and then I'll get shouted at when it breaks. If we have to do it then don't blame me when things go wrong."
"I don't think that will work as planned because of X, Y and Z. A better solution might be something like this as you gain this and limit the negative effects of that. I think this is a much better solution, but if we stick with the original plan I'll do the best I can with it"

In both you are disagreeing with the plan. The first response will leave people with the view that you're being unhelpful and don't really care (remember social capital?). In the second response, you're giving a reason why it won't work, offering an alternative solution, in addition to remaining positive in the disagreement.

It is important to master this as there will always be a time when you have to disagree in the workplace - better that you can do this without leaving a bad feeling in the air.

Don't be shy to ask for help

It's a great feeling when you get around a hard problem all by yourself. But sometimes, you have to ask yourself, "Is the time spent getting there worth it?". I could spend a day researching error codes, looking on StackOverFlow etc trying to fix a bug custom Python Script... or, I could ask someone if they have encountered this before on this specific script.

You have to know when it is a good time to ask for help AND how to ask for help. These are some rules I try to follow:

If you encounter something you might need some help on, set a time limit to try to resolve it on your own. You could do 30 mins of research and if you haven't found a solution, then move on to the next stage and ask for help.
When you need to ask for help, give detail of the problem and what you've tried. "Can you help with this Kubernetes problem? The pod isn't starting and the error says X. I've had a google and it says the fix is Y which hasn't worked. Have you come across this before?"

This shows that you've put the effort in yourself to solve the problem, but are coming to someone for their expertise.

Conclusion

When I reflect back on my life and career, these are just some areas that I think about. In closing, I ask you this question:

What advice would you give your younger self?

Nice to meet you

Justin 👨🏾‍💻 — Sun, 15 Nov 2020 17:51:42 +0000

print("Hello World!")

I bet you've seen that one 1000 times... well make that 1001! I just wanted to introduce myself into the blogging world, let you know what to expect from my future posts and just get started as this isn't really something I've done before.

TL;DR

Who am I? What do I do? My name is Justin and I'm a Devops Engineer in London.
Career - Spanning over 10 years starting from Desktop Support -> SysAdmin -> DevOps Engineer
What to expect - Some of the things I've had the chance to experience / work with presented in the most simple way (tech jargon isn't my strong point and for me felt more like a barrier to learning). Things I wish I knew before I needed to know them and hopefully give the insight to things that I didn't have to others.

Who am I? What do I do?

My name is Justin, a DevOps Engineer working for a Sportsbook Management company in London. I've been working here maybe 8 & half years now because of all the great things I get to do and learn here. Some of the tech stack I get to touch as part of my role are:

Gitlab (for source control + Continuous Integration / Continuous Delivery aka CI/CD)
Jenkins (building packages)
Amazon Web Services (AWS)
Docker (Containers)
Kubernetes (Container orchestration - think what / when / where / how to run a container and supporting infrastructure)
Python (part of scripting deployments / custom programs)
Powershell (deployments / monitoring)
Linux & Windows Servers (hosting our services)
CheckMK (one part of our monitoring solution)
Slack (was it even worth listing this?)
More that I think I'll add in a later iteration

Career background

The start

I started my career as Frontline Desktop Support / Support Analyst on a trading floor (think Wolf of Wall Street but not as crazy) after deciding that if I was going to potentially work for the rest of my life, might as well be with something I know a little about and enjoy - fixing computers! I then took the Comptia A+ so that I could solidify what I already knew as well as get the chance to even get an interview to show my ability & drive. Here I earned a good working knowledge of:

Basic networking skills - VLANs / Firewalls & switches / Network patching / Troubleshooting
Basic troubleshooting skills - Is it even on? Does it have power? Is it connected? What does the error message actually say? Have you googled that error message?
VMWare - Virtualisation - Think another laptop within your laptop to do things on without physically having another laptop.
How applications actually fit together - very important as you'll rarely be using something in isolation.
Soft skills - Talking to people. Translating tech things into "everyday language". Managing deadlines / expectations. Being accountable for your actions / inactions.

These things would become fundamental to the work I do now.

The next step

From here I moved to the company I am with now - I don't actually remember the job title but essentially it was a cross between Desktop Support / Junior SysAdmin in the IT Operations team. Before too long I was promoted to a Windows Systems Administrator. Some of the things I learned in this role:

Solidified VMWare knowledge - setup from start to finish, preparing / racking / network patching the server, installing the software, creating virtual machines & troubleshooting various things
Involved in a Data Centre migration - A lot of network troubleshooting here! Not to mention really learning about the monitoring systems and how crucial they were to knowing what is going on in your environment.
Some of the more advanced Infrastructure concepts & principles - Clustering / Active Directory management / DNS / Config management / Backups / Recovery from problems / Load-balancing / Network storage management / provisioning machine images / introduction to scripted deployments etc
Working with developers - brand new world here being at the end of the pipeline for getting their software into production.
Security "best practices" - Nobody wants to get hacked, so think about some level of security at least in all the work you do.

We used to work via Kanban where the developers worked in an Agile way. This often caused bottlenecks in getting things into production because of differing priorities and ways of working. The company decided to restructure which would take me to the next stage of my career.

Enter DevOps

You could say I entered the world of DevOps by complete chance. You could argue that I was always doing DevOps. However we are NOT going to have that argument on whether DevOps is a job role or a culture or both - you can read up about that if are interested otherwise we might be here all day!

Either way I was here, suddenly placed in a new role, in a new team and called a DevOps Engineer and expected to know EVERYTHING from the jump. When I say EVERYTHING I mean EVERYTHING; Docker, Jenkins, querying metrics & log files, build pipelines, source control, Gitlab CI pipelines, all those AWS services like CloudFormation, EC2, ELB, Security Groups etc. I mean if you've ever touched AWS you'll know those things in their own right take time to get up to speed - throw everything else in the mix as well as needing to support a team of 6 developers? lol.

It was the most challenging thing I've ever had to handle at work but the key thing I kept telling myself was "Rome wasn't built in a day. Skill up day by day, brick by brick." With that I then focused on learning maybe one small thing a day until I knew enough to be comfortable being somewhat "self sufficient". That took about a year and a half for me to feel that way. The most difficult part was detangling all the moving parts that make up the DevOps role. In a future post I'll probably try to help with that.

Anyway, recently I've become AWS Certified with the AWS Solutions Architect Associate which I am very proud of. I'll detail that experience in another post. I'm mentioning this at every opportunity I get. I already do this at work when speaking to any decision makers or even those asking for my advice on what we should do!

"As you know I've been studying for the AWS Solutions Architect and I came across a service that would be useful for us to use / optimise what we have..."

What to expect from future posts

In my future posts I'm going to do the following:

Breakdown the knowledge I've built up over the years in an easy to consume format.
Explain tech and concepts in ways I wish someone would have just told me as if they were my mentor. So many times reading documentation trying to work out where it fits in. It can seem impossible and once I did it for the first time I was like "is that it?"
Walkthrough my thought processes when problem solving
How to's regarding things I use that could be useful for you or match a use case you might have.

Well that's it for now - until next time...
Yours truly,
Justin.