DEV Community: Simon Sugob

Serverless Architecture – Complete Reference Guide 2019

Simon Sugob — Wed, 22 May 2019 20:12:21 +0000

Serverless architecture (also known as serverless computing or function as a service, FaaS), gets us a step closer to independence from humans and brings us much closer to the machine dependency. For many people, serverless is their first exposure to the public cloud.

What Is Serverless Architecture?

Despite the name, serverless architecture doesn’t mean that it runs its code without servers. In this solution there is no need for a company to buy, rent or provision servers/virtual machines for the back-end code to run on. This is a way of building and running applications and services without infrastructure management.

Since the term ‘serverless’ is confusing and there’s no one clear view of what serverless is, I encourage you to read the article by Mike Roberts where he gives a deep insight into this term.

Evolution Of Serverless Architecture

‘Serverless is the evolution of cloud platforms in the direction of pure cloud-native code’.

It is the culmination of several iterations of cloud platforms. The evolution began with physical machine in the data center and progressed through Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Container as a Service (CaaS) to Function as a Service (FaaS). Best explanation you will find in this short, 4-minute video by Jason McGee (IBM’s CTO):

Why Choose Serverless Architecture?

Serverless architecture significantly helps developers focus more on their core product, not the infrastructure. Otherwise, developers would still be worrying about implementing, maintaining, debugging, and monitoring the infrastructure, whether managing them in the cloud or on-site. Using serverless services like AWS Lambda, Google Cloud Functions, Firebase or Microsoft Azure Functions they will take care of the physical hardware, virtual machine operating system as well as the web server, while you – as a developer – would only need to worry about one thing — your code.

Is It Cheaper To Go Serverless?

It depends. If you have a small number of functions to be hosted, you should consider switching to serverless. Having a more complex application, a serverless architecture can still be profitable, but you will need to architect your application differently. This may turn out to be impracticable if you have an existing application. The best solution, in this case, would be migrating smaller pieces of the application into serverless functions over time. I’ve encountered a clever car analogy by [Luciano Mammino (https://www.slideshare.net/loige/building-a-serverless-company-with-nodejs-react-and-the-serverless-framework-jsday-2017-verona) explaining whether to go serverless or no.

Cars are parked 95% of the time (Fortune). How much do you use the car?

What Is Function as a Service (FaaS)?

FaaS is an implementation of serverless architectures where engineers can deploy an individual function or a piece of business logic. It allows developers write and update a piece of code ‘on the fly’, which can then be executed in reply to an event, such as a user clicking on an element in a web application. This facilitates scaling code and is a cost-efficient way to implement microservices.

In other words, instead of scaling a monolithic REST server to handle the potential load, a developer can now split the server into a bunch of functions which can be scaled automatically and independently. If you’re familiar with microservices, this image might be helpful:

Some people argue that serverless should be referred to as function as a service (FaaS), and while it’s not a bad name, serverless is a much broader concept than just an ephemeral function running in the cloud. Learn why?:

The key properties of FaaS are:

Independent, server-side, logical functions
Ephemeral (may only last for one invocation)
Stateless
Event-triggered
Scalable by default
Fully managed by a Cloud vendor (a third party)

What Is The Difference Between FaaS And PaaS?

PaaS makes it easy to deploy an entire application. PaaS providers handle provisioning servers and deploying your application to the servers.

Function as a Service (FaaS) gives you the ability to deploy what is essentially a single function, or part of an application. Instead of the monolith app that you’d run on a PaaS, you break down your app into small self-contained programs, or functions.

Serverless Infrastructure Providers & How-To Guides’ Links

After you have installed the serverless framework on your machine next thing is to choose where you’d like your serverless service to run. Choose your compute provider from below list and follow awesome guides by Serverless Inc. at https://serverless.com/framework/docs/getting-started/.

Top 5 Principles Of Serverless Design

Below principles apply if you decide to create an entire system (back-end & front-end) using a serverless approach. If you are building other types of systems (a pipeline for transforming a file), the principle relevant to the front-end will not apply.

Use a compute service to execute code on demand
Write single-purpose, stateless functions
Design push-based, event-driven pipelines
Create thicker, more powerful front ends
Embrace third-party services

Learn more at https://techbeacon.com/enterprise-it/essential-guide-serverless-technologies-architectures.

The Best Use Cases For Serverless

According to the survey by Serverless, Inc. here is a list of cases when serverless is a good fit:

But When Not To Use Serverless?

Here is the answer on Quora: https://www.quora.com/What-is-the-best-use-cases-for-serverless/answer/Max-Katz

Advantages And Disadvantages Of Serverless Architecture

download PDF: https://www.dropbox.com/s/0wumyqm3oadlww6/Serverless-architecture-PROS-and-CONS.pdf?dl=0

Summary

Serverless is growing fast. Several key adoption metrics are 2x what they were last year. And not just with smaller companies – the enterprise is adopting serverless technologies for critical workloads just as rapidly. And there is clear evidence in the mentioned survey by Serverless, Inc.:

Best Cloud Computing Conferences In 2019

In comparison to previous years, more people are using multiple cloud providers. This fact increases the importance of projects such as CloudEvents and other initiatives that enable vendor choice. As for the best cloud computing conferences happening in 2019, a curated list you will find here.

What Is The Biggest Obstacle To Wider Serverless Adoption Right Now?

Operationalizing serverless (debugging, monitoring, and testing) is the biggest issue at the moment and clearly, indicate a lack of tooling.

DevOps Maturity Assessment — PDF Checklist To Print

Simon Sugob — Wed, 27 Mar 2019 11:54:30 +0000

How well are you doing DevOps?

This is going to be short.

When you mean DevOps — you should not ask ‘IF’, better ask ‘HOW’. Doing DevOps is no longer a differentiator in itself. You have to do it better than your competitors which requires improving key capabilities. This is an incredibly complex and case-specific task.

To make it easier, together with my colleagues I have prepared a list of 29 KEY QUESTIONS you can use to gauge the maturity of your company's software delivery competency.

You can PRINT it and USE at your DevOps team meeting to better understand your needs.

Click the link to grab your checklist: https://bit.ly/2JJYtyd

And feel free to share it.

How To Become DevOps Engineer

Simon Sugob — Tue, 12 Mar 2019 15:48:56 +0000

A DevOps engineer is a valuable part of a company. Kelsey Hightower – head of operations at Puppet Labs – called them “Special Forces”. He said that ‘DevOps engineer encapsulates depth of knowledge and years of hands-on experience’.

If you want to get a couple of directions on How To Become DevOps Engineer – without beating about the bush – have a look at this article but first, watch this video with sound!

How To Become DevOps Engineer – Why Is It Such A HOT Topic Now?

DevOps is revolutionizing the workplace. According to IDC estimates, the worldwide DevOps software market will achieve $6.6 billion in 2022 in comparison to $2.9 billion in 2017. That is an impressive number.

// Detect dark theme var iframe = document.getElementById('tweet-1056992524328427520-604'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1056992524328427520&theme=dark" }

According to Indeed, the number of DevOps jobs offers increased from less than 1% in 2012 to more than 24% in 2017. DevOps engineer was named the no. 3 most in-demand tech job in a 2016 Indeed report, and the no. 2 most difficult tech job to fill.

What trends in DevOps world can we expect in incoming 2019? Here is a list of ‘DevOps: Five Trends For 2019 And Beyond‘ by Greg Baker from Pythian:

• DevOps Will Finally Become Mainstream

• Executives Are Getting On Board

• Security Is Becoming A Priority

• Kubernetes Is King

• The Pipeline Is Maturing

——————————————-

Click here to view the full article.

I think that to close this paragraph you definitely should read Roushan Kumar‘s answer to What is “DevOps” and how does “DevOps” work? on Quora. It is a really nice read.

How To Become DevOps Engineer – One-Page Hard Skills Guide

Since you already know why so many engineers strive to know How To Become DevOps Engineer it is time to share with you a One-Page Guide on How To Become DevOps Engineer. We have elaborated it on the basis of two articles:

• A comprehensive Developer Roadmap by Kamran Ahmed

• ‘An illustrated guide to becoming a DevOps Engineer with links to relevant courses’ by javinpaul.

Click the link to download PDF.

How To Become DevOps Engineer – Soft Skills Also Matter

Apart from hard, tech skills, DevOps Engineer should also possess the ability to collaborate, multi-task, and always put the customer first.
Here they come:

• Curiosity & empathy: According to Sigler, curiosity, and empathy are perhaps two of the most valuable soft-skill traits a potential DevOps engineer can demonstrate.

• Customer-first mindset: a great DevOps engineer always cares about delivering value to customers and wants to understand and measure how their work is moving the needle, said Omri Gazitt, chief product officer at Puppet.

• Communication & collaboration: being able to communicate effectively with different stakeholders and understand the current challenges and the ‘big picture’ in the organization is the first step to being able to build a strategy to implement culture shifts.

• Flexibility: keeping a cool head, juggling fluid workloads and remaining flexible is one of the keys to success in the DevOps Engineer role.

• Big picture thinking: because DevOps combines the needs and wants of multidisciplinary teams and applications, a successful DevOps engineer can’t lose sight of the big picture for DevOps and digital transformation at the enterprise level—especially in a large organization, said Gidron.

• Decision making: in a busy environment, the ability to make the right decision quickly, while keeping in mind the big picture goals, is extremely desirable trait employers are looking for in DevOps engineers.

————————-
source:

https://www.techrepublic.com/article/10-critical-skills-that-every-devops-engineer-needs-for-success/

https://techbeacon.com/5-ways-make-yourself-marketable-devops-engineer

https://www.networkerstechnology.com/traits-successful-devops-engineer>

How To Become DevOps Engineer – Short Recap

How to be a great DevOps Engineer? The best summary has been captured by PavanBelagatti in his article ‘How To Be a Successful DevOps Engineer’.

He lists 5 Key focus areas to be a great DevOps Engineer:

Knowing the basic concepts on DevOps and getting into the mindset of automating almost everything
Knowing different DevOps tools like AWS, GitHub, Puppet, Kubernetes, Docker, Chef, Ansible, Terraform, JIRA, Slack etc.
Org-wide Ops mindset
End-to-end collaboration and helping others solve the issues
Being a scripting guru: Bash, Powershell, Perl, Ruby, JavaScript, Python, etc. in order to write code to automate repeatable processes. ———————————

Additional Resources:

How to build a successful career as a DevOps engineer - free PDF

How to become a DevOps engineer: A cheat sheet

10 bad habits DevOps admins must break

Find more DevOps related stuff here.

Kafka on Kubernetes? Confluent Has Made It

Simon Sugob — Tue, 12 Mar 2019 13:33:50 +0000

Deploying Kafka On Kubernetes NOW Easier With Confluent Operator

The Confluent Operator

The widespread use of Kubernetes by the innumerable number of platforms in the company gave rise to Confluent Operator – says Neha Narkhede, Co-founder and CTO at Confluent – “the best way to deploy Apache Kafka and Confluent Platform on Kubernetes“. Alex Woodie rightly noticed that “many organizations have attempted to wire Kafka to run on the Kubernetes cluster manager, but it has been a challenging technical feat that Confluent has succeeded at!”

“Enabling everyone to run Apache Kafka® on Kubernetes is an important part of our mission to put a streaming platform at the heart of every company. This is why we look forward to releasing an implementation of the Kubernetes Operator API for automated provisioning, management, and operations of Kafka on Kubernetes” says Neha Narkhede.

“This includes a rollout of pre-built Docker images for the entire Confluent Platform, Kubernetes deployment templates, a reference architecture that enables users to deploy Kafka and Confluent Platform on Kubernetes and an implementation of the Kubernetes Operator API for automated provisioning, management, and operations of Apache Kafka on Kubernetes” – adds Confluent’s CTO.

Kafka + Kubernetes expertise = The Confluent Operator

Rohit Bakhshi, the author of the article “Getting Started with Apache Kafka and Kubernetes” best sums up this solution:

“Our goal is to make streaming data ubiquitous: Kubernetes lets you run your apps and services anywhere, Kafka enables you to make your data accessible instantaneously, anywhere”.

Despite the fact that the link between Apache Kafka on Kubernetes may seem like a ‘match in big data heaven’, the devil is in the details. What were the biggest obstacles to overcome? Read more.

Why Should Your Deploy Kafka On Kubernetes With Confluent Operator?

Neha Narkhede explains that “the Confluent Operator directly addresses the challenges of running Apache Kafka on Kubernetes, and will offer the following features across all Confluent Platform components”:

Automated Provisioning
Cluster Management and Operations
Resiliency
Monitoring Read more

Getting Started – How To Easily Run Apache Kafka on Kubernetes

Confluent provides two resources to install Kafka on Kubernetes:

Helm Charts – a starting point for deployment of Confluent Platform components
Recommendations for Deploying Apache Kafka on Kubernetes – a white paper

TIP: Not only we advise you to get familiar with these tools but also recommend you to have a look at two below resources to fully understand deployment part:

Stateful, Stateless, and Serverless – Running Apache Kafka on Kubernetes – online talk featuring Joe Beda, Heptio CTO and Kubernetes Co-creator, and Gwen Shapira, Principal Data Architect at Confluent.
Apache Kafka on Kubernetes – Could you? Should you? – this article gives the first reflection after the launch of Helm Charts.

You can already Sign up to Orchestrate Apache Kafka®on Kubernetes.

also posted here

7 Reasons For Adopting DevOps Culture

Simon Sugob — Thu, 28 Feb 2019 14:02:45 +0000

How To Become DevOps Engineer

Simon Sugob — Thu, 28 Feb 2019 13:57:41 +0000

Why Terraform? - Quick Reference Guide

Simon Sugob — Thu, 28 Feb 2019 13:54:20 +0000

What Should The 12 Factor App Be?

Simon Sugob — Thu, 28 Feb 2019 13:50:58 +0000

The Twelve-Factor App — A Successful Microservices Guideline

Simon Sugob — Thu, 28 Feb 2019 13:46:59 +0000

Why The Twelve-Factor App?

Nowadays everyone is building microservices. No matter where you work, either in a startup or a big company you will encounter microservices architecture. Heroku did a great job at defining what makes a solid baseline for standing up your architecture.

In 2011 Adam Wiggins [Heroku co-founder] published The Twelve-Factor App methodology for building software-as-a-service based on their own experiences. It is technology and language agnostic but successfully compatible with Microservices, Containers and CI/CD Pipelines with a focus on DevOps, hence, this topic is on my radar.

The 12-factor app is based on a few parameters for deployment of cloud-native applications:

Matt Ellis from Tibco said “the 12-factor apps check-list is really just a set of guidelines that dictate how a microservice should be built to properly support the concept of independently managed and iterated services. These factors are important when building decoupled, stateless microservices”. So what should the 12Factor app be? Watch the 28 sec. video: https://www.youtube.com/watch?v=7PH3AOePAXc

Who is this document addressed to?

“Any developer building applications which run as a service. Ops engineers who deploy or manage such applications”, says the author.

Let’s explore the 12-Factors now:

The full content is out there, so there is no need to repeat it. I will only outline the core message of each of the following points:

Factor 1: Codebase

Build on top of one codebase, fully tracked by a Version Control System (VCS). Deployments should be automatic, so everything can run in different environments without work. You should always have one repository for an individual application to ease CI/CD pipelines.

Factor 2: Dependencies

Do not copy any dependencies to the project codebase, instead use a package manager. Always remember to use the correct versions of dependencies so that all environments are in sync and reproduce the same behavior.

Factor 3: Config

Store the config in Environment Variable. There should be a strict separation between config and code. The code should remain the same irrespective of where the application is being deployed, but configurations can vary.

Factor 4: Backing Services

Treat backing services as attached resources as your services should be easily interchangeable. You must be able to easily swap the backing service from one provider to another without code changes. This will ensure good portability and help maintain your system.

Factor 5: Build, Run, Release

A twelve-factor application requires a strict separation between Build, Release and Run stages. Every release should always have a unique release ID and releases should allow rollback. Automation and maintaining the system should be as easy as possible. Then you put everything together in something that can be released and installed in the environment and then be able to run it.

Factor 6: Stateless Processes

You should not be introducing state into your services, applications should execute as a single, stateless process. The Twelve-factor processes are stateless and share-nothing. This factor lies at the core of microservices architecture.

Factor 7: Port Binding

Your service should be visible to others via port binding. If you built a service, make sure that other services can treat this as a resource if they wish. The twelve-factor app is completely self-contained.

Factor 8: Concurrency

Break your app into much smaller pieces rather than trying to make your application larger (by running a single instance on the most powerful machine available). Small, defined apps allow scaling out as needed to handle the varying loads. Each process should be individually scaled, with Factor 6 (Stateless), it is easy to scale the services.

Factor 9: Disposability

Processes should be less time-consuming. Make sure you can run and stop fast. And that you can handle failure. Without this, automatic scaling and ease of deployment, development- are being diminished. You can achieve this with containers.

Factor 10: Dev-Prod Parity

Keep development, staging, and production as similar as possible so anyone can understand it and release it. Continuous deployment needs continuous integration based on matching environments to limit deviation and errors. This also implicitly encourages a DevOps culture where Software Development and Operations are unified. Containerization is a huge help here.

Factor 11: Logs

Treat logs as event streams. Logging is important for debugging and checking up on the general health of your application. At the same time, your application shouldn’t concern itself with the storage of this information. Instead, these logs should be treated as a continuous stream that is captured and stored by a separate service.

Factor 12: Admin Processes

Run admin/management tasks as one-off processes — tasks like database migration or executing one-off scripts in the environment. To avoid messing with the database, use the tooling you built alongside your app to go and check the database.

Twelve-Factor App in Practice

Greg Pryzby in his talk “You must be 12-factors high to ride” discusses this methodology and explains how the 12 Factor App and OpenStack complement each other: https://youtu.be/GZ4sAEUMOnM

There are also two articles worth reading: “Why 12 Factor Application Patterns, Microservices, and CloudFoundry Matter” by Tim Spann and “How I Use the Twelve-Factor App Methodology for Building SaaS Applications with Java & Scala” by Jay Kapadnis.

Conclusions

Microservices are still relatively new architecture pattern, therefore I really recommend you to study the Twelve-Factor App methodology in details. Across my experience, most microservices projects failed at the start. It was not due to deep design issues, or coding difficulties, but in fact, because of getting the basic things wrong.

Some points in 12-Factor may seem trivial, but when running 20+ services via few environments they may be of great importance.

Let me finish with Tim Spann’s distinct explanation of why he decided to combine 12-factor application patterns, microservices, and Cloud Foundry:

“Good code fails when you don’t have good process and a platform to help you. Good teams fail when you don’t have a good culture that embraces DevOps, microservices and not giant monolith.”

also posted here

Terraform Quick Reference Guide

Simon Sugob — Fri, 01 Feb 2019 11:06:10 +0000

Terraform remains one of my favorite DevOps tools. It is not only because of the ease of modularity and working against multi-cloud. Terraform enables its users to manage infrastructure as code and this is a key and foundation for DevOps practices.

Why do I Love Terraform? – For 5 Reasons:

✔It is Open Source with a massive active community developing this tool
✔It is Multi-Cloud provider
✔It is Multi-Platforms
✔Its plan command allows you to see the changes that it will apply before its application
✔It runs in command line enabling automated DevOps pipeline integration

Q1. Why Terraform?

Learn how popular Terraform became since 2016 and consider 6 reasons Why should Terraform be one of your DevOps tools?

Q2. How Terraform Compares to Other Infrastructure as Code Tools?

The answer to this question appears on Quora. The respondents give us either common features as well as differences between Terraform and CloudFormation. What are the pros and cons between using Terraform vs. CloudFormation?. If you want to dive deeper go for: Terraform vs CloudFormation – A Pragmatic Comparison.

Q3. Where Can I Find Terraform Guide, Best Practices and Tips?

Yevgeniy Brikman, Co-founder of Gruntwork, author of “Hello, Startup” and a popular book “Terraform: Up & Running” releases a series of posts that will teach you best practices for using Terraform in the real world. Here is his Comprehensive Guide to Terraform.

A great place not only to find but also to share your Terraform best practices and custom modules with the community is nothing but GitHub. Visit GitHub – Terraform Best Practices and take advantage of it!

Use modules, separate your environments, keep it DRY. These and 4 other useful Tips you will find in an article by Simon So: 7 Tips to Start Your Terraform Project the Right Way. While Terraform Recommended Practices guide is rather meant for enterprise users looking to advance their Terraform usage from a few individuals to a full organization.

Q4. Why Should You Contribute To Terraform?

If you are considering to let your Devs help out with operations work it is now possible. Read this post and learn how. Putting The Dev Into DevOps: Why Your Developers Should Write Terraform Too.

Q5. Why Terraform Modules and How to Build Them?

What to do when infrastructure management extends beyond simple configuration? If you want help to reduce your maintenance and make managing your infrastructure more efficient take advantage of modules. They provide an easy way to abstract common blocks of configuration into reusable infrastructure elements. Below two articles worth studying:

✔HashiCorp Terraform: Modules as Building Blocks for Infrastructure.

✔How to create reusable infrastructure with Terraform modules.

Q6. What about Repository Structure?

If you are familiar with Terraform Enterprise (TFE) you should know that a proper repository structure is crucial. It determines which files Terraform has access to when it is executed within Terraform Enterprise, and when Terraform plans will run. Dig deeper ➛ Repository Structure.

Q7. How To Provide Zero Downtime Updates with Terraform?

Learn how to ensure new infrastructure is created before the old is destroyed. Bear it in mind that a running VM does not necessarily mean a working application. Read more: Zero Downtime Updates with HashiCorp Terraform.

Q8. How to Get Started with Terraform Automation?

Whether you use Terraform regularly in production or by a larger team it can be desirable to orchestrate Terraform runs in some sort of automation in order to ensure consistency between runs, and provide other interesting features such as integration with version control hooks. Sounds interesting? Explore this guide: Running Terraform in Automation.