DEV Community: Nextlink Labs

Beta APIs Disappearing from Kubernetes v1.22

Nextlink Labs — Mon, 19 Jul 2021 14:43:38 +0000

Thinking about upgrading to Kubernetes v1.22? There’s a number of beta APIs that will be making their exit in the new version that could have a big impact on how your code works going forward.

It bears repeating that these beta APIs -- some of which may have been enabled by default -- won’t just be deprecated in the new version of Kubernetes. They will be removed entirely.

Check out the list below and make a plan before making your move to v1.22.

What Beta APIs are Removed in Kubernetes v1.22

If you are using any of the beta API versions listed below, they will immediately stop serving in version 1.22. These APIs were previously deprecated in “favor of newer and more stable API versions,” per the release notes.

Beta versions of the ValidatingWebhookConfiguration and MutatingWebhookConfiguration API (the admissionregistration.k8s.io/v1beta1 API versions)
The beta CustomResourceDefinition API (apiextensions.k8s.io/v1beta1)
The beta APIService API (apiregistration.k8s.io/v1beta1)
The beta TokenReview API (authentication.k8s.io/v1beta1)
Beta API versions of SubjectAccessReview, LocalSubjectAccessReview, SelfSubjectAccessReview (API versions from authorization.k8s.io/v1beta1)
The beta CertificateSigningRequest API (certificates.k8s.io/v1beta1)
The beta Lease API (coordination.k8s.io/v1beta1)
All beta Ingress APIs (the extensions/v1beta1 and networking.k8s.io/v1beta1 API versions)

If you are using any of these and need some help, check out this link for documentation covering migration options, specific nuances, and in some cases, paths forward and tips about switching to the stable APIs.

Why Is It Important (and What Can I Do)?

In short, if your code relies on any of the beta APIs above, which were enabled by default, and upgrade to v1.22, your application will probably not work as it did before.

For software developers, specifically authors of 3rd party integrations, the removal of these APIs is critically important because they could potentially break the applications. Any developer that has created an Ingress controller, webhook authenticator, API aggregators, or another tool that relies on the APIs listed above will probably be negatively impacted.

The release notes for v1.22 includes a guide for testing out your upgrade so you can run your Kubernetes cluster in an environment that only uses the new APIs to make sure your code works as it should.

To try out the API removals prior to upgrading to Kubernetes v1.22, add the following to the kube-apiserver command list arguments:

--runtime-config=admissionregistration.k8s.io/v1beta1=false,apiextensions.k8s.io/v1beta1=false,apiregistration.k8s.io/v1beta1=false,authentication.k8s.io/v1beta1=false,authorization.k9s.io/v1=false,certificates.k8s.io/v1beta=false,coordination.k8s.io/v1beta1=false,extensions/v1beta1/ingresses=false,networking.k8s.io/v1beta1=false

The release notes warn that this command will turn off v1beta1 of EndpointSlice, which is important to remember for testing.

Once you've switched all the kube-apiservers in your cluster to use that setting, the beta APIs are removed. You can test that API clients (kubectl, deployment tools, custom controllers etc) still work how you expect, and you can revert if you need to without having to plan a more disruptive downgrade.

Why Does Kubernetes Remove Beta APIs?

Kubernetes has a defined deprecation policy for features which includes its own API. According to the policy, stable API are only deprecated when a newer, equally stable version of the API becomes available.

For those that are using a stable version of the Kubernetes API, “there won't ever be a new version released that forces you to switch to an alpha or beta feature,” according to the documentation. The big differentiator here is how stable versions are treated in comparison to alpha or beta versions.

Alpha versions of the API are for testing and since they may be incomplete to begin with, are nearly always disabled by default. Sometimes alpha features don’t work out and are removed from Kubernetes releases.

Features that survive alpha testing move to beta and are typically enabled by default, which is an important distinction from their alpha counterparts.

Once an API reaches beta, a countdown begins. Within roughly a year’s time, or three release cycles, per the documentation, the beta API will either reach General Availability (GA) or a new beta version be created and released. In either case, whether it reaches GA or it’s back to the drawing board, the previous beta version will be deprecated.

What Beta APIs are Going Away in the Future

More APIs will be removed with the release of Kubernetes 1.25, when it is released sometime next year. Specifically, PodSecurityPolicy won’t make the jump to stable and will be removed altogether. To learn more about how to move forward with PodSecurityPolicy, read the PodSecurityPolicy Deprecation: Past, Present, and Future post here.

Here’s the list of beta APIs planned for removal as it stands now:

The beta CronJob API (batch/v1beta1)
The beta EndpointSlice API (networking.k8s.io/v1beta1)
The beta PodDisruptionBudget API (policy/v1beta1)
The beta PodSecurityPolicy API (policy/v1beta1)

Click this link for the official list of API removals for Kubernetes 1.25.

This post originally appeared at the Insights blog at NextLink Labs.

Setting Up Kubernetes User Access Using AWS IAM

Nextlink Labs — Mon, 28 Jun 2021 18:56:59 +0000

Here’s the scenario: You want to deploy a fancy new AWS Elastic Kubernetes Service, or EKS cluster as it’s commonly called, running the latest version of Kubernetes.

You have your Terraform all set to run terraform apply but then you think to yourself… "What about user authentication? How’s that going to work?"

Congratulations, you're off to a good start by asking one of the the most important questions. Like it or not, people other than yourself are going to have to access the cluster. This blog post will walk you through how to get authentication working correctly from the beginning!

One important point before we get started, make sure to deploy your cluster with a role that others can assume. AWS EKS maps some integral permissions to the user/role that is used to create the cluster. This will save you from future headaches.

Legend:

Part I: IAM Users and Groups

Role and Trust Relationship
Group and Policy
User and Group

Part II: IAM Roles and EKS

Terraform
eksctl (CloudFormation)
AWS UI (Manual)

Part III: EKS User and Namespace Permissions

Cluster Role/RoleBinding

Part IV: Review and Testing

Preface

For those without single sign-on (SSO), leveraging IAM Users and Groups is the recommended way of handling authentication. Part I will explain how to manage auth solely via AWS IAM. If you have single sign-on (SSO) and the ability for users to authenticate via the steps found here, then you can skip to Part II.

Part I: IAM Users and Groups

To start with the basics, IAM is defined as Identity and Access Management within AWS. From Amazon's documentation, IAM users represent the person or service who uses the IAM user to interact with AWS. IAM user groups are simply collections of IAM users. You can use user groups to specify permissions for a collection of users, which can make those permissions easier to manage for those users.

For an exhaustive view into IAM users and groups, check out the full documentation from Amazon.

Now let’s get into the specifics of setting this part up.

1. Create IAM Roles

First, start by creating two IAM roles named eks-admin-role and eks-developer-role with the following CLI commands:

aws iam create-role --role-name eks-admin-role --output text --query 'Role.Arn'
aws iam create-role --role-name eks-developer-role --output text --query 'Role.Arn'

Next, you'll need to edit the trust relationship on each of the roles. Here's the JSON you'll need to do that.

{
  "Version":"2012-10-17",
  "Statement": [
    {
      "Effect":"Allow",
      "Principal":{
        "AWS": [
          "arn:aws:iam::<ACCOUNT_ID>:user/<USER>"
        ]
      },
      "Action":"sts:AssumeRole",
      "Condition":{}
    }
  ]
}

2. Create User Groups and Attach Policy

In order to give the users the ability to assume the roles we created above, we must first create 2 IAM assume role policies. Save the following JSON to a file in your current directory and name it eks-admin-assume-role-policy.json and change the relative variables to match your AWS account number.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowAssumeOrganizationAccountRole",
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": "arn:aws:iam::<ACCOUNT_ID>:role/eks-admin-role"
    }
  ]
}

Run the following command to create the admin policy:

aws iam create-policy --policy-name eks-admin-assume-role-policy --policy-document file://eks-admin-assume-role-policy

Now we need to create the developer assume role policy. Go ahead and create another JSON file named eks-developer-assume-role-policy.json with the following. Again, make sure you change the ACCOUNT_ID variable to your AWS account id.

{
  "Version": "2012-10-17",
  "Statement": [
    {
        "Sid": "AllowAssumeOrganizationAccountRole",
        "Effect": "Allow",
        "Action": "sts:AssumeRole",
        "Resource": "arn:aws:iam::<ACCOUNT_ID>:role/eks-developer-role"
    }
  ]
}

Run the following command to create the developer policy:

aws iam create-policy --policy-name eks-developer-assume-role-policy --policy-document file://eks-developer-assume-role-policy

Now that we have the assume role policies created, we're going to need to create 2 IAM user groups called eks-admin-group and eks-developer-group and attach those policies to grant each user within the groups the ability to assume their relative role.

Here's the command line inputs for the groups:

aws iam create-group eks-admin-group</p>
aws iam put-group-policy --group-name eks-admin-group --policy-name eks-admin-assume-role-policy
aws iam create-group eks-developer-group
aws iam put-group-policy --group-name eks-developer-group --policy-name eks-developer-assume-role-policy

3. Add Users to Created Groups

Finally, add the IAM users to the newly created groups with this command line input:

aws iam add-user-to-group --group-name <GROUP> --user-name <USER<

Part II: IAM Roles and EKS

AWS defines an IAM role as an IAM identity that you can create in your account with specific permissions. Per AWS’s documentation, "an IAM role is similar to an IAM user, in that it is an AWS identity with permission policies that determine what the identity can and cannot do in AWS. However, instead of being uniquely associated with one person, a role is intended to be assumable by anyone who needs it."

From Part I, you should now be able to assume a role in AWS IAM via either SSO or AWS IAM Groups. Now you can move on to configuring your aws-auth config map in EKS.

There are a few options here, depending on how you deployed your cluster:

Option 1: Terraform

Terraform is a software based infrastructure as code tool developed by Hashicorp that we highly recommend for all projects to help retain clear and concise definitions for cloud infrastructure.

One of the nicest things about Terraform is that there are a plethora of public modules available. Specifically, there is a public eks module we will be using with all of the boilerplate code already written, so all that's left is defining some variables for your specific setup. The various inputs for the eks module we will be using can be found here.

If you deploy your cluster via Terraform, leverage your Terraform configuration to add the map_roles=[] variable inside the module "eks" { … } section.

map_roles = [
    {
    “groups”:[ “system:bootstrappers”, “system:nodes”],
    "rolearn":“arn:aws:iam::<ACCOUNT_ID>:role/<EKS_NODE_ROLE>”,
    “username”: “system:node:{{EC2PrivateDNSName}”
    },
  {
    "groups": [ "system:masters" ],
    "rolearn": "arn:aws:iam::<ACCOUNT_ID>:role/eks-admin-role",
    "username": "eks-admin"
  },
  {
    "groups": [ "" ],
    "rolearn": "arn:aws:iam::<ACCOUNT_ID>:role/eks-developer-role",
    "username": "eks-developer"
  }
]

You can see an example terraform setup using this method inside the repository below:

https://github.com/terraform-aws-modules/terraform-aws-eks/tree/v16.0.1/examples/basic

Note: The "username" variable above is what you will use in Part III to associate the role to a role binding internally to Kubernetes. If you’d like to read more about Kubernetes RBAC, take a look at this documentation explaining user auth within the cluster.

Option 2: eksctl

For those that are unfamiliar, eksctl is a simple CLI tool for creating and managing clusters on EKS - Amazon's managed Kubernetes service for EC2. You can learn more about it here.

If you created your cluster via eksctl then use the eksctl create iamidentitymapping cli function.

    eksctl create iamidentitymapping --cluster  <CLUSTER_NAME> --arn arn:aws:iam::<ACCOUNT_ID>:role/eks-admin-role --group system:masters --username eks-admin`
    eksctl create iamidentitymapping --cluster  <CLUSTER_NAME> --arn arn:aws:iam::<ACCOUNT_ID>:role/eks-developer-role --username eks-developer
    eksctl create iamidentitymapping --cluster  <CLUSTER_NAME> --arn arn:aws:iam::<ACCOUNT_ID>:role/eks-developer-role --username eks-developer

Option 3: AWS Console (UI)

Lastly, if you created your cluster via the AWS Console/UI, then use the kubectl CLI to edit the config map via vim.

kubectl edit cm/aws-auth -n kube-system

apiVersion: v1
data:
  mapRoles: |
    - groups:
      - system:bootstrappers
      - system:nodes
      rolearn: arn:aws:iam::<ACCOUNT_ID>:role/<EKS_NODE_ROLE>
      username: system:node:{{EC2PrivateDNSName}}
    - groups:
      - system:masters
      rolearn: arn:aws:iam::<ACCOUNT_ID>:role/eks-admin-role
      username: eks-admin
    - groups: []
      rolearn: arn:aws:iam::<ACCOUNT_ID>:role/eks-developer-role
      username: eks-developer
  mapUsers: |
    []
kind: ConfigMap

Note: Do not add the developer role to any pre-created groups in Kubernetes. This way you can manage their permissions via a role and role-binding within Kubernetes.

Part III: EKS User and Namespace Permissions

Once your AWS IAM Roles are mapped to a Kubernetes user, you can create Kubernetes roles and role bindings to give permissions to various users and the AWS IAM roles they are attached to.

Since we already mapped the AWS role "eks-developer-role" to an internal Kubernetes user called "eks-developer" in Part II, we just need to grant that user permissions within the cluster. In order to do this, we need to create a role within the namespace, so that the user will have permissions. In this example we will grant all permissions to eks-developer in a namespace.

Save the following to a file and name it something along the lines of dev-role-cfg.yml, then go ahead and apply it to a namespace of your choosing.

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: <NAMESPACE>
  name: eks-developer-role #not to be confused with the AWS IAM role
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]

Next, we have to tell Kubernetes that we want to map the "eks-developer" user to the "eks-developer-role" we just created above. We do that by creating a "RoleBinding".

Save the following to a file called dev-rolebinding-cfg.yml and go ahead and apply it.

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: eks-developer-role-binding
  namespace: <NAMESPACE>
subjects:
- kind: ServiceAccount
  name: eks-developer
  apiGroup: ""
roleRef:
  kind: Role
  name: eks-developer-role
  apiGroup: ""

Part IV: Review and Testing

Let's review what should be in place by this point in the setup:

IAM Roles (Developer and Admin)
IAM Groups and Assume Role Policies (Developer and Admin)
EKS aws-auth ConfigMap (mapRoles) update
EKS Role and RoleBinding (Developer Role)

After you've confirmed each item in the list above, go ahead and assume the role using the AWS CLI command aws sts assume-role. Once assumed from a user in either the admin or developer group, go ahead and configure your .kube/config file locally by running:

aws eks --region <region> update-kubeconfig --name <cluster_name>

Congratulations on making it this far! You are now ready to run commands against the cluster. At this point you should be able to run any commands as admin and only against a specific namespace if you're logging in using the developer group. To avoid losing access to the cluster, it's critical you deploy the cluster with another admin user or role (not your own).

This post appeared on our blog at NextLink Labs where we write about devops and devops consulting services.

Node 15 vs. Node 16

Nextlink Labs — Tue, 22 Jun 2021 20:32:01 +0000

On April 20, 2021, OpenJS released the latest major version of Node.js, Node 16.

With this even-numbered release, we see a few changes to the V8 engine, support for the new Apple Silicon processors, and a stable Timers/Promises API. We can also expect to see long-term support for this version.

High Level Changes

"Even number" vs. "Odd number" releases

As an even-numbered release, we can expect to see Node 16 move to long-term support status, which guarantees that critical bugs will be fixed for a total of 30 months. Also, because Node 16 is an even-numbered release, Node 15 has officially become unsupported.

ONe of the quirks of Node worth remembering is that odd-numbered releases – those literally ending in an odd number -- will go End-of-Life as soon as the next Semantic Version release line ships. According to NodeSource, a blog covering all things Node, Semantic Versions, or SemVers as they are called, include:

Major Release: Major Releases are for incompatible API changes, from version to version. Major releases can also include changes that would normally be included as Minor or Patch releases.
Minor Release: Minor Releases include backward compatible functionality changes. Minor releases can also include changes that would normally be included as Patch releases.
Patch Release: Patch releases include non-breaking bug fixes and security patches.

Conversely though, even-numbered releases become Long-Term Support, which is say they will be supported and maintained by the Node.js project for an extended period of time.

From a support standpoint, it's worth understanding the implications which version of Node version you're business is running.

timers/promises API

Starting with version 16.0, the timers/promises API officially moves from experimental to stable. This API simply gives gives a way to define a timer as a Promise object.

import { setTimeout } from 'timers/promises';

const main = async () => {
  console.log(new Date().toTimeString());
  await setTimeout(1000);
  console.log(new Date().toTimeString());
}

main();

Running the following example correctly await's the Promise returned from setTimeout, and prints out the following:

16:45:41 GMT-0700 (Pacific Daylight Time)
16:45:42 GMT-0700 (Pacific Daylight Time)

V8 Upgraded to V8 9.0

The V8 JavaScript engine is updated to V8 9.0, up from 8.6 in Node 15. With this upgrade comes some performance improvements, and the ECMAScript RegExp Match Indices, which provide the start and end indices of a string through a new .indices property on certain match objects.

const matchObj = /(Node) (16)/d.exec('Node 16');

Given a match object defined above, we get the following when logging out matchObj.indices, outlining the start and end indices of each match.

[[0, 7], [0, 4], [5, 7], groups: undefined]

Apple Silicon support

In order to support Apple's new Apple Silicon M1 chip, the macOS installer (.pkg) will be shipped as a multi-architecture binary. This means that Node 16 will work on the new M1 Macs.

Breaking Changes

None coming from Node 15.

Conclusion

Node 16 will be supported next 30 months, while simultaneously discontinuing support for Node 15 immediately. This makes this a "must" upgrade for those currently on Node 15, and definitely something to consider coming from Node 14 in order to stay on the current LTS version for the longest.

This post originally appeared this blog where we cover devops consulting services.

The Critical Role of Cybersecurity in the Digital Transformation

Nextlink Labs — Thu, 10 Jun 2021 19:16:46 +0000

Cybersecurity and privacy should be priorities for every company when pursuing a digital transformation to maintain security and privacy in the modern digital marketplace.

As DevOps teams develop software solutions and reorganize operations", the focus is often on speed, automation, and business optimization. Security and regulatory compliance can be overlooked as overhead costs when those initiatives are in fact critical, fundamental necessities.

That kind of oversight -- both in practice and in process -- can be costly.

A Costly Oversight with Serious Implications

As far back as 2012, then FBI Director Robert Mueller commented on our vulnerability to cyberattacks, saying that there “are only two types of companies: those that have been hacked, and those that will be.”

While that statement was made nearly a decade ago, it’s meaning hasn’t lost any of it’s value. Development teams that don’t take seriously the place of security in their systems are at risk.

Today, cyberattacks lead to tangible, costly disruptions in our everyday lives.

On May 7, hackers conducted a ransomware cyberattack on the Colonial Pipeline, shutting down computerized equipment that managed the pipeline and halting the delivery of 45% of the gasoline, diesel and jet fuel consumed on the East Coast. The pipeline resumed delivery on May 12.

JBS Holdings, the world’s largest meat company in terms of sales, experienced widespread service interruptions after a ransomware attack halted production at the end of May 2021.

In both cases, the attacks led to panic buying, widespread uncertainty and increased prices for the public. The companies paid a high price as well. Colonial Pipeline paid a $4.4 million ransom to bring operations back online. JBS paid $11 million to cyber attackers, something CEO Andre Nogueira referred to as a "very difficult decision to make."

"However, we felt this decision had to be made to prevent any potential risk for our customers,” Nogueria told NPR.

In late Sept. 2020, a cybersecurity intrusion shutdown the computer network of Universal Health Services Inc., which operates more than 250 hospitals and other clinical facilities in the U.S. This event hit the healthcare system during the weekend and left nurses and doctors using pens and paper to get by.

In the same month, a cybersecurity attack caused the IT network to fail at a German hospital and a woman who needed urgent admission died after she had to be taken to another city for treatment. It’s speculatively the first known death from ransomware.

In December 2018, the International Telecommunications Union estimated about 3.9 billion people or 51.2 percent of the world’s population have become internet users. That fact reveals just about every individual in the world who can afford electronic devices are using web-based services.

And the security of those services isn’t always certain.

Understanding these cultural shifts helps companies meet growing demands. However, as internet usage grows so do risks.

An Increased Importance of Cybersecurity

Internet usage is ever increasing. Innovations that make life easier are connecting us to a quickly growing number of networks with questionable security.

To this end, cybersecurity measures should be seen as an imperative for corporate responsibility.

Infrastructure costs to achieve specific goals should include cybersecurity solutions and regulatory compliance at the onset of any digital transformation project.

That’s accomplished by bringing together security, development, and operations under a DevSecOps team to ensure all aspects of the transformation are managed. Once the correct strategy is assembled, it will have assessed security vulnerabilities and prioritize an organization’s critical processes and systems.

Once vulnerabilities are identified, the DevSecOps team can then determine the solutions and organizational changes required for the digital transformation. Please keep in mind, the best practices and methods for these kinds of projects must respect user experience, performance, and agile software development.

Achieving this kind of cohesion, collaboration, and fluidity is no small feat. That’s why most companies seek out partners to assist with the process.

While implementing a comprehensive cybersecurity solution is not an easy process, the greater ROI generated from the transformation is safeguarded.

Cybersecurity: Not a One-Size-Fits-All Strategy

Unfortunately, there are no comprehensive off the shelf cybersecurity tools.

Every company requires a custom solution based on industry space, ecommerce platforms, records storage, and other considerations.

For example, a health-related company might need to consider HIPAA laws. An accounting firm must navigate specific tax laws. Energy utilities are overwhelmed with consumer and environmental concerns.

Although many companies rely on internal departments to take on cybersecurity challenges, keeping up with new solutions and laws can prove daunting to even the most knowledgeable professionals.

This post originally appeared this blog where we cover devops consulting services.

Finding The Opportunity in Uncertainty in Software Development

Nextlink Labs — Tue, 01 Jun 2021 02:28:09 +0000

Custom software development is a lot of things, but you’d be hard pressed to find a way to describe it as “easy”.

It takes a lot of planning and resources. Often you need a team to make it happen. Technology is changing at an incredible rate, so you need to get serious about adopting a methodology too.

There’s risks and rewards and, like everything else, you celebrate the wins and learn from the mistakes.

But one of the things experienced in all areas of software development, from business owners to junior developers and everyone in between, is the nagging, gut-wrenching feeling of uncertainty.

Sometimes it hits you in a coding sprint or maybe in the middle of a planning meeting. Maybe it even keeps you up at night as you’re reflecting on the day.

Regardless of when or how it happens, everyone who’s ever written a line of code, planned a project, documented a feature or implemented a process has asked the question:

“Am I doing this right?”

Whether you realize it or not, that moment of clarity is worth some serious reflection. It’s a big deal.

It’s a moment worth unpacking because it’s really three separate, important things all wrapped into one package:

It’s an honest admission of a real concern.
It’s a question that actually answers itself, the answer is 99% of the time being a resounding ‘No’.
It’s a simple realization of an opportunity disguised as a challenge.

So often, an individual or company’s reaction to uncertainty can define the path forward, whether it be in the direction of progress or regression. And it’s in no way a task handled simply.

But the good news is that uncertainty in building software isn’t rare.

In fact, the issues you come across, regardless of how specific they may sound, have more than likely been experienced by countless others before you.

You’re not alone in struggling with code challenges -- far from it.

Countless companies have struggled with updating production as often as they’d like, however frequently. Getting the beta version of an app launched is tough for everyone, and version 1 is probably tougher.

Because after all, custom software development is a battle and uncertainty manifests itself in all kinds of interesting ways.

How Does Uncertainty Manifest in Software Development?

Whether developed by an outside vendor or derived in-house, custom software tools become the weapons businesses use to battle their competition in a daily struggle for revenue and mindshare on a highly contentious field of operations.

And while comparing business to battle might be a little dramatic, it’s not that far of a stretch to draw military comparisons to those involved in custom software development.

Software developers often serve as the first line of defense in the battle for progress, solving problems with an ever increasing arsenal of programming languages that are rapidly evolving. It’s a highly fluid situation with lots of pressure and responsibility because the features you deliver today will make a difference in the product tomorrow.

Working in this fast paced, results driven environment can place engineers in a position of great uncertainty. Sometimes code is written when the problem isn’t fully realized, which can result in technical debt. Or sometimes developers wear different hats and deal with infrastructure issues and maintain security for the network.

The “It’s-Not-My-Job-But-I’ll-Do-It-Anyway” nature of being a developer in a small business environment can lead to all kinds of uncertain decisions that had to be made just to get through the day.

Then there are project owners that are trying to organize the battle and make everything run fluidly, like an eight-armed organization machine built to communicate, plan, execute and succeed. Project managers face uncertainty every day and have to make quick decisions that can lead to unpredictable outcomes.

Finally, there’s business owners who act as the Generals of the battle.

They’re not necessarily concerned with the inner workings of the system, they just want the updates to flow and the features delivered on time. They want things to work like they are supposed to work because progress is tied to revenue. It’s simple. Uncertainty around progress means posing some highest risk questions:

Is the app going to crash? Do I have enough resources to tackle the issue? Am I doing this right?

Reasons People Avoid Dealing with Uncertainty

Time and time again, people facing uncertainty in software development spend a lot of time considering the problem, white-boarding thoughts on the matter and after struggling to frame up the problem, admit defeat and move on.

Again, this makes total sense because cutting through the uncertainty is hard work.

Reaching an epiphany that shines a light through the darkness is tough business. Unfortunately, there are plenty of paths that help people take to avoid dealing with uncertainty.

Let’s Stick With What Works

Ambitious business owners are, by definition, driven to excel and reach new heights. Oftentimes, leveling up a business is tied to enhancing software tools.

The bounds of this improvement process, the point A to point B type stuff, the start- and end-view of things, are probably pretty clear.

The uncertainty lies in how to get from here to there. It can be hard to figure out just how to define the change that needs to happen to reach the goal.

And that’s when people fall into the trap of filling the uncertainty gap with a healthy dose of “what’s always worked”. To succeed, sometimes companies try to continually do the same things over and over and expect different results.

Unfortunately, it often doesn’t work that way.

One of the most valuable lessons learned is the thing that brought forth success in one venture won’t necessarily be the tools to bring success in another. Or, as Marshall Goldsmith says in his popular book, what got you here -- the success you’ve had up until now -- isn’t going to get you there.

Instead of facing uncertainty with old ways, it’s time to acknowledge what you don’t know and find some alternatives.

There’s Not Enough Time to Navigate Uncertainty

Another way people deal with uncertainty is simply ignoring that it exists and forging ahead in the interest of getting things done. Handling uncertainty this way has all types of unintended consequences at all different levels.

Software developers spend a ton of time wading through uncertainty in the search for a solution to a problem. And oftentimes, they implement solutions and get things done just to reach a deadline, all the while carrying around with the nagging suspicion that “ something just doesn’t feel right.”

Without proper guidance, communication and teamwork, developers find themselves awash in uncertainty and, through no fault of their own, deliver a solution to a problem that isn’t quite as elegant as needed.

Technical debt is defined as the accumulation of time and effort needed to fix previous work that was completed to achieve an end, such as release date or hot fix. It’s like real debt in that it exists, must be addressed, and can be negative. According to a recent study, companies spent over \$85 billion dealing with technical debt in 2018 alone.

Tech debt isn’t just accumulated by developers though. Far from it.

Tech debt can be incurred when project requirements shift, scope creeps, and things change, causing the original direction of a venture to be altered and swift adjustments to be implemented to reach a quick deadline.

Technical debt is just one side effect of ignoring uncertainty. Another thing is downtime. Every website on the Internet seemingly has a story of going down at one point or another because someone just “tried to do something,” despite a healthy dose of uncertainty.

Progressing forward without addressing uncertainty compounds issues and leads to impacts feature releases. Product upgrades become harder and disaster recovery takes longer.

Ignoring Uncertainty Because it’s Difficult

To restate the original point, custom software development is tough. When a company reaches a point where the next level up is in sight but the path forward is unclear, it’s easy to choose the worst option, which is to do nothing.

And again, this makes perfect sense because it’s not in the nature of all companies to be good at developing software.

Said another way, it’s absolutely true that companies need software to thrive and survive in the modern world. But that doesn’t mean you have to be an expert at making software.

It would be like requiring you to map out every part of an automobile’s engine as a requirement for a driver’s license. The idea is absurd -- that’s the very reason why auto mechanics exist. The same argument applies to networking, websites, and custom software development.

Companies struggle with software development because software development is hard.

Large organizations have a tough time maintaining the safety of their networks because cybersecurity is an inherently difficult task to figure out.

Development teams falter in implementing DevOps strategies because the concept is difficult to handle without years of experience.

The uncertainty around complex ideas in software development is inherently difficult but the rewards associated with overcoming the unknown can be great.

How Addressing Uncertainty Leads to Progress

One of the benefits derived from dealing with uncertainty is it naturally causes companies to admit it’s unclear on how to deal with something. The act of acknowledging uncertainty in any activity kicks off a domino effect that inevitably leads to reflection.

Posing the question “Are we doing this correctly?” naturally leads to the analysis of the process. And more often than not, you’re going to find that the act of asking the question itself is all the evidence needed to provide the answer that things could in fact be better.

One option to dealing uncertainty is to tackle the challenge head on, similar to how streaming content provider Netflix approached the situation.

Netflix handled the uncertainty of faults in the system by creating a tool that would purposefully create an environment of chaos for it’s developers. The Netflix-created tool, named Chaos Monkey, operates in all their systems' environments, causing chaos by randomly shutting down server instances and triggering errors, thus forcing developers to anticipate outages as they constantly work with unreliable services.

This environment, coupled with Netflix’s adoption of a strict DevOps development protocol, produces a seamless end user experience and provides “developers a unique opportunity to test their software in unexpected failure conditions, but incentivizes them to build fault-tolerant systems to make their day-to-day job as developers less frustrating.”

Another option is to pair with a trusted advisor in your space to handle the situation. Many partnerships formed in uncertain situations have led to successful transitions to the next level. In this case, do some homework, ask a lot of questions, and find a way through the fog.

This post originally appeared this blog where we cover devops consulting services.

5 Tips for an Effective Cybersecurity Program

Nextlink Labs — Wed, 26 May 2021 13:18:45 +0000

There’s a monumental cybersecurity shift happening in organizations across the country. Every day, companies are transforming their operations, adopting new technologies and pushing the boundaries of what’s possible with technology to keep up in a highly competitive digital landscape.

And with that change comes risk.

There’s danger involved in dealing with data and peril in opening systems to the world. Access is risky too and everyday seems to bring more news of some kind of cyberattack. Implementing a cybersecurity program in your organization is an important first step in not only staying competitive but also combating cybersecurity threats, securing your organization’s data, and protecting profits.

According to the 2020 IDG Security Priorities Study, 49% of participants indicated that improving the protection of confidential and sensitive data was their top priority. While it’s encouraging to see organizations put security at the forefront, it’s vital that these efforts are not wasted.

So how can you ensure the program you are developing is effective? Let’s consider five tips and their related activities you can use to strengthen your cybersecurity program and improve its effectiveness.

Tip #1: Build a Strong Foundation

Perform an Initial Assessment

Performing an initial assessment of the current state of your cybersecurity program will help guide your gap remediation activities and inform your baseline. Your initial assessment should take a look at your programmatic, or policy and documentation endeavors, as well the technical practices you currently have in place.

If your organization has never created a comprehensive cybersecurity program plan, it can be beneficial to partner with an experienced cybersecurity firm to perform an initial assessment and make recommendations based on the findings.

Whether you perform the assessment yourself or you hire an experienced third-party assessor, be sure you’re asking the right questions, interviewing the correct personnel, and reviewing the relevant documents in order to gain a good understanding of your security strengths and weaknesses.

Create Your Baseline

A cybersecurity baseline is a set of minimum security controls for your organization. The baseline you implement for your organization should reflect your business goals, your compliance requirements, and your accepted risk.

Your cybersecurity baseline should consider what assets you have identified as critical. These assets may be proprietary data, protected client information, critical business systems, or other assets that would cause significant impact to your organization if they were compromised or disrupted. The baseline controls should protect those assets and any connected pathways leading to those critical components.

When establishing baseline controls, it is also important to identify the roles specifically responsible for various portions of your cybersecurity program. Generally, the Chief Information Security Officer (CISO) is responsible for the cybersecurity program. Depending on the size of your organization, you may have different roles overseeing different portions, or a handful of roles splitting up the duties.

No matter how you assign responsibilities within your organization, ensure that your employees are properly trained and supported as they implement the baseline controls.

Tip #2: Document, Document, Document

Most compliance standards that organizations deal with today have guidelines about what information should be documented and how those documents should be maintained. Frameworks such as NIST require auditors to examine documentation, conduct interviews, and test controls.

If your organization is not subject to such a compliance standard or is not using a standard industry framework, it is still important to document. Stanford University researchers and cybersecurity firm Tessian reported that 88% of all data breaches are a result of human error. Documenting your policies and procedures can help improve organizational security awareness, demonstrate organizational accountability, and reduce cost and risks associated with new employee onboarding.

The documents your organization can most benefit from will depend on your industry, your business goals, and the type of data you handle. Some common and impactful documents to develop and maintain include:

Acceptable Use Policy
Interconnection Agreements
Disaster Recovery Plan
Internet Usage Policy
Secure Password Policy
Remote Access Policy
Asset Configuration Documentation
Asset Inventory
Network Diagrams
Incident Logs

Proper documentation can provide internal and external auditors with important information about your security posture. It can also provide your teams with a clear reference for what policies are in place, what procedures have worked or failed in the past, what training is required of them, and what assets are available to them.

Tip #3: Support Continuous Improvement

The cybersecurity landscape is constantly evolving as new technologies and methodologies develop. An effective cybersecurity program strives for continuous improvement to combat the new challenges they face.

Continuous improvement has tremendous benefits for your organization’s security, but it does require a commitment of support from management and decision makers. Here are some of the key continuous improvement activities that can benefit your organization.

Training your Security Team

Security training is becoming readily available as cybersecurity efforts are more widely recognized and supported. Training sessions are available as remote courses, hands-on classes, talks, seminars, conferences, and more, and cover a wide range of topics. Some describe how to configure and use security tools, how to combat common attacks, and how to write effective policies and procedures, while others may provide hands-on practice in simulated environments. The type of training you make available to your security team can depend on your organizational maturity, budget, goals, and data.

Investing in your Program

It shouldn’t be a surprise that implementing a cybersecurity program has some upfront costs, but organizations often drastically reduce the security budget once the program is up and running. When considering how to allocate your funds, remember that the rapidly changing technology landscape means your team may need ongoing training, additional tools, external audits, compliance certifications, or extra personnel, even in a mature program. Those costs may seem well worth it when you consider the $3.86 million dollar worldwide average cost of a data breach, or the $8.64 million dollar cost for US-based companies.

Developing Repeatable Processes

All the hard work you and your team put into your cybersecurity program can mean nothing if the processes are not repeatable. When an employee leaves the organization, takes time off, is promoted or transferred away from the site, or is otherwise separated from the security team, other employees should be poised to step into the role and carry on their tasks. If a repeatable process has not been developed, your team could lose valuable time recreating the tasks previously performed. Additionally, when ad hoc methods are employed the organization is often operating from a reactive position where a breach has already occurred. Developing a repeatable process helps the organization stay proactive, or to shift left of bang.

Tip #4: Audit Your Program

Regular audits of your program can go a long way in ensuring that your program is efficient and effective. Audit frequency can depend on your organization and framework, or may be specified by compliance standards.

To get the most comprehensive picture of your program’s effectiveness, take a holistic look at both your policies and your technical implementations. Don’t just take the policies at face value; interview your personnel to ensure daily activities match up with what your policies and approved procedures stipulate. Make note of any deviations so you can identify the cause and find a solution that fits your security goals as well as the flow of your daily operations.

The Verizon 2021 Data Breach Investigations Report (DBIR) found that of the small businesses who experienced breaches, 57% faced external threat actors and 44% faced internal threat actors. When testing your technical control implementations, it’s important to simulate both types of potential threats. Ensure your internal controls can protect your critical assets from accidental or intentional compromise.

You may choose to have your own security team audit your program, or you may opt for a trusted cybersecurity partner to take the lead. Either way, document the findings and any remediation strategies that can guide your program to a more secure posture.

Tip #5: Solicit Organization-Wide Support

When we think of cybersecurity responsibilities within an organization, we often think of the security team, the SOC, or the IT department. Realistically however, effective security programs are an organizational effort with support across departments.

Who you involve in your efforts may depend on the size and structure of your organization, but common examples of interdepartmental contributions include:

Legal:

Legal teams can offer support by reviewing cybersecurity policies, interconnection agreements, and can, in some cases, offer insight into compliance requirements.

Human Resources:

HR and security teams can coordinate on secure employee onboarding and termination procedures, policy enforcement, and cybersecurity training efforts.
This collaboration can help reduce insider threats, improve security awareness, and ensure policy compliance. Human resources employees are also a common target for phishing and information gathering efforts from malicious actors, so a close relationship with the security team can lead to quick identification and thwarting such attempts.

Physical Security:

Most cybersecurity consulting services know that physical access is king, and even the most secure network cannot protect data from unauthorized access on the physical level. A well trained physical security team can help identify and address malicious actors attempting to tailgate, impersonate, or otherwise bypass security measures in order to gain physical access to your data and systems.

Senior Management:

Senior management’s attitude about security and the need to support security efforts can have a strong influence on the overall effectiveness of a cybersecurity program. When senior management budgets for continual training and effective tools, acknowledges legitimate concerns, and generally supports the security team, the program has a higher chance of success.

Senior management is also at a high risk for spear phishing attacks from malicious actors. They should abide by the same security policies as other employees, and report any suspicious emails, phone calls, or texts.

All other departments: Every department can contribute to cybersecurity efforts. Employees across the organization should be familiar with the policies in place for securing the sensitive systems and data they utilize, reporting procedures for communicating suspicious activity, and training activities to improve their cybersecurity awareness.

The Bottom Line

Establishing a cybersecurity program takes hard work and organizational support. While there is no one size fits all solution, the tips discussed in this article can help you build and maintain a robust, holistic, and effective cybersecurity program.

This post originally appeared on our blog and was written by Jeremy Dodson, CISO at NextLink Labs. Jeremy writes about cybersecurity and cybersecurity consulting services for NextLink Labs.

Shifting Left of Bang to Improve Your DevOps Security

Nextlink Labs — Fri, 14 May 2021 14:37:49 +0000

In the early 2000s, the US Marine Corps' Combat Hunter training program introduced, among many other concepts, the ‘left of bang’ idea as a way to improve situational awareness and save lives.

The way ‘left of bang’ works is simple.

Any critical event that causes significant disruption, loss, or other negative outcome -- an attack, injury, or breach of security -- can be mapped out and analyzed on a timeline. The actual occurrence of the event -- the thing that happened -- falls directly in the middle of the line, equidistant from both sides, at a point labeled “bang”.

To the left of the bang are the things that happened before the event. Everything that was done in the past, all proactive measures, the planning and preparation -- or lack thereof -- live left of bang.

On the right of the bang is everything that followed the event. It’s the time for cleanup, remediation, and reactive measures. It’s the reflection period or in much more serious cases, the aftermath.

Patrick Van Horne and Jason A. Riley, two former active-duty Marines, took the concept further in their 2014 book Left of Bang: How the Marine Corps' Combat Hunter Program Can Save Your Life. The book dives into the idea, exploring the left side of the timeline. The authors dig far into the concept, illustrating how detecting early warning signs and recognizing oncoming danger left of bang can help disrupt, prevent, or avoid attacks altogether.

Horne and Riley show how shifting left isn’t simply about ‘an ounce of prevention’ but how the concept can literally save lives.

But what does this have to do with your DevOps program, and can the fundamental ‘left of bang’ principles improve your security?

In software development, the concept of ‘shifting left’ is not new. In its simplest form, ‘shifting left’ refers to testing earlier in the life cycle. This tactic helps prevent costly delays and improve product quality.

When applied in DevSecOps environments, earlier testing can identify potential vulnerabilities and improve security.

While shifting left is a great starting point, it isn’t enough to guarantee your program is left of bang.

Consider your SDLC is moving along smoothly, you have shifted your testing left, and you have implemented security tests earlier and more often. Suddenly, you’re seeing reports of a data breach and your application has been compromised.

In this scenario, you have encountered a “bang” event.

Fortunately, there are some simple proactive steps you can take to improve your cyber situational awareness and keep your program left of bang.

Knowing the Threat: Identify What “Bang” Looks Like

Identifying bang events is an important step that is often taken for granted. If you were put on the spot and asked to identify what critical events could disrupt your service or cause significant loss to your organization or customers, could you answer?

Let’s take a look at some of the more common bang events that could affect your organization and why they are important to identify.

Data Breaches

You don’t have to look very deep in the news to find reports of significant data breaches. Take for example the 2020 Marriott breach that exposed personal data for over 5 million guests.
It’s safe to say that this breach was a bang event for the Marriott team. But not all data breaches are the same.

When considering what a data breach would mean for your organization, it is important to consider what kind of information you are responsible for and how that information could be used.

Does your application hold financial data? Personally Identifiable Information (PII)? Protected health data? If you were to experience a data breach, what would the result be?

Answering these and similar questions can start you off on the right path to better protecting your organization and your customers.

Phishing Attacks

Successful phishing attacks are a unique type of bang event in that they are almost always a stepping stone to a much larger data breach. Phishing attacks are often an attempt to obtain user credentials, financial information, or other sensitive data. The attacker then uses that information to pivot and elevate the attack.

According to Verizon’s 2020 Data Breach Investigations Report, 22% of data breaches in 2019 involved phishing attacks, while 37% of breaches stole or used credentials.

It’s important to identify phishing attacks as a bang event separate from the larger data breach. Even if a phishing attack is successful, there are ways to mitigate the damage and reduce the risk of a more serious compromise.

Staying left of bang for phishing attacks takes unique measures, starting with targeted training for employees who do not deal directly with security, ensuring strong credential protections and limiting credentialed access to data.

Disasters

Some bang events may seem completely out of your control, like natural or geopolitical disasters. Even so, it’s important to identify reasonable risks and potential outcomes for these events while you’re left of bang so that your team can recover quickly if you encounter them.

Take for example, the recent pandemic that forced many organizations to have their employees work remotely. Most were significantly underprepared for this bang event, having never considered the possibility.

Security threats boomed during the lockdown period of the pandemic and organizations such as the World Health Organization (WHO) experienced dramatic increases in cyber attacks. Despite this, according to a recent report from Tanium, 93% of surveyed organizations said they delayed security priorities during lockdown.

It may be hard to predict what disasters your organization may face. Consider what disasters are most likely to happen in the areas where your employees, servers, and other critical infrastructure are located. Knowing the bang events that are possible will help you craft solid recovery strategies that will allow your organization to remain operational.

Know Your Assets: Understanding What You Have

Another important step in ensuring your DevOps program stays left of bang is to identify and inventory your assets. Different types of assets can pose different types of risks, and must be managed accordingly.

Hardware Assets

Knowing the type and location of hardware used within your organization will help shape your left of bang strategies. Remote laptops will require different security than in-house PCs, and a receptionist’s system poses different access threats than one in a locked office.

Third-Party Software

DevOps environments generally rely heavily on third-party software, from IDEs to third-party testing tools, communications software, deployment applications, code repositories, and more. Having a complete catalog of all third-party software will inform your decisions on acceptable risk. It will also help you understand potential threats that may come from a source outside of your control, and how to respond if the vendor is compromised.

Consider what happened to General Electric in 2020. GE used Canon Business Process Servers for HR purposes. Canon experienced a data breach that exposed PII for GE’s employees and their beneficiaries. Although GE’s systems were unaffected, the breach posed a significant risk to their own organization.

Knowing the third-party software within your organization and understanding their security posture will help improve your own security. It will also help you identify legitimate vendor communications as well as malicious communications masked as coming from your vendor.

Human Assets

While technology is an essential component to any DevOps program, it’s the people that make it all work. It’s those same people who pose a significant risk to the organization. A 2019 report from Kaspersky shows the most frequent incident targeting enterprise and SMBs is inappropriate IT resource use by employees, followed by malware infection of company owned devices.

Understanding where people sit in the organization structure, what data they have access to, and how they use that data is key to building your left of bang strategies. Segmenting data access by job requirement, ensuring secure use of company devices, and providing efficient methods of reporting suspicious emails are some simple steps to help keep your essential human assets from becoming your biggest liability.

Knowing Your Setup: Properly Manage Your Tools

Shifting your testing left might feel like a solid proactive step, but it means nothing if your tools are not selected, distributed, and configured with security in mind.

Tools are an important part of any DevOps program. They can make the SDLC run infinitely smoother, but they can also cause problems.

When selecting tools for your team, it’s easy to simply consider functionality. But applying left of bang fundamentals means also considering security risks associated with the tool itself, how it’s used within your environment, and how the vendor addresses security.

Once you are confident with the tools you have selected, consider who needs to have access to them. Ensure that anyone who is given access is either familiar with the tool already or is given proper training on how to securely use the tool.

The next step is to ensure all tools are properly configured with security in mind. Ensure any default credentials are changed immediately and that the tool is tailored to your specific environment.

It can be beneficial to use automated security testing tools, but ensure the results are reviewed and verified. Some configurations may dismiss low or medium threat findings. Depending on how your environment is connected, however, those lower threats could be pivot points for attackers to gain deeper access into your systems.

What’s Next? Make a Plan

Armed with all this knowledge, you can collect your current policies and procedures and assess if they match up to your left of bang goals. There’s a number of things you’ll need for this process.

Our list of starters include:

Cyber security
Strong Credential Policy
Third-Party Software Policy
Security Testing Policy
Social Media Policy

This post appeared on the Insights blog at NextLink Labs. Check out that blog for more cybersecurity posts.

Terraform Infrastructure as Code Basics

Nextlink Labs — Thu, 13 May 2021 12:39:00 +0000

Overview

During the course of this article we will cover some basic concepts in the Terraform application to create AWS resources. This article assumes you have the following:

Terraform configured on your local machine.
An AWS Account.
A VPC (the default VPC will suffice).
A Subnet that has the ability to allocate Public IP's within your VPC.
An SSH key configured in your AWS Account
An AWS Access and Secret Key

Terraform Providers

In our previous article, we discussed Terraform Providers. Expanding upon the last article; Terraform has the ability to manage not only Public Cloud resources but other resources including but not limited to Grafana, VMWare vSphere, and Docker. In this article we will discuss two providers in particular: the AWS and Template providers. Using these, we will create an EC2 Instance and apply a template created user-data file to change the hostname and reboot.

Our File Structure

While you can contain all your code in a single .tf file, it's best practice to separate your code into multiple files for easier organization and reading. Since Terraform runs in the current directory unless the -target flag is specified it will apply all code in the current directory (and sub directories if specified in the HCL). We will also include a template file for our user data.

main.tf
vars.tf
outputs.tf
main.auto.tfvars
user-data.tpl

You may also separate your code into individually named files such as ec2.tf and sg.tf for easier organization and reading.

Provider Setup

Like the AWS CLI, the AWS Provider can authenticate to AWS leveraging the 4 standard authentication methods provided natively.

Static Credentials
Environment Variables
Shared Credentials File
EC2 Roles

For our purposes, we will leverage the Environment Variables method with an AWS Access and Secret Key.

shell script
export AWS_ACCESS_KEY_ID=my-access-key-id
export AWS_SECRET_ACCESS_KEY=my-secret-access-key
export AWS_DEFAULT_REGION=us-east-1

Terraform Code

We will first start off by creating our variables by making our vars.tf

hcl
variable "vpc_id" {
  type = string
}

variable "subnet_id" {
  type = string
}

variable "instance_name" {
  type = string
}

variable "keypair" {
  type = string
}

variable "instance_type" {
  type = string
}

We can now populate our variables with values from our main.auto.tfvars.

hcl
vpc_id="vpc-myvpcid"
subnet_id="subnet-mysubnetid"
instance_name="myhost"
keypair="mykeyname"
instance_type="t3.small"

We will create our template file user-data.tpl user-data script to be used later by our main.tf

#!/usr/bin/env bash

hostnamectl set-hostname ${instance_name}

reboot

Now that we have our variables and our template, we can continue by creating our resources in our main.tf

hcl
provider "aws" {
    region      = "us-east-1"
}

## This grabs our own external IP address to use later when creating our security groups ##
data "external" "myipaddr" {
  program = ["bash", "-c", "curl -s 'https://api.ipify.org?format=json'"]
}

## This grabs the latest Ubuntu 18.04 Bionic Beaver AMI ##
data aws_ami "bionic" {
  owners = ["099720109477"]
  filter {
    name = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-*"]
  }
  most_recent = true

}

## Create a security group with your local public IP to grant SSH access ##
resource aws_security_group "my_security_group" {
  name = "my_security_group"
  vpc_id = var.vpc_id
  ingress {
    from_port = 22
    protocol = "tcp"
    to_port = 22
    cidr_blocks = [
      "${data.external.myipaddr.result.ip}/32"
    ]
  }
  egress {
    from_port = 0
    protocol = "-1"
    to_port = 0
  }
}

## We can create our templated file that will be rendered into our user-data ##

data template_file "user_data" {
  template = file("./user-data.tpl")
  vars = {
    instance_name = var.instance_name
  }
}

## Now we will create our instance with our created templated user-data and security group id's ##

resource aws_instance "my_ec2" {
  ami = data.aws_ami.bionic.id
  subnet_id = var.subnet_id
  instance_type = var.instance_type
  key_name = var.keypair
  associate_public_ip_address = true
  vpc_security_group_ids = [aws_security_group.my_security_group.id]
  user_data_base64 = base64encode(data.template_file.user_data.rendered)
  ebs_optimized = true
  root_block_device {
    volume_type = "gp2"
    volume_size = 20
    delete_on_termination = true
  }
  tags = {
    Name = var.instance_name
  }
}

Now that we've created our main.tf, next we will finally create our outputs.tf so we can acquire our Public IP address once the instance is created.

output "public_ip" {
  value = aws_instance.my_ec2.public_ip
}

One thing you may have noticed is how I call results from one resource into another. I use the id of the created security group by calling aws_security_group.my_security_group.id in the aws_instance resource. This gives a level of native ordering within Terraform. This means the aws_instance has an implicit dependency on the aws_security_group resource before it can be created. If you want to employ an explicit dependency or ordering on resources that don't natively need something from another resource you can leverage the depends_on attribute of that resource.

Lets Run That Terraform!

Phew...alright, we've got all our Terraform written. Now its time to initialize our Terraform State and create our infrastructure!

But wait...what's Terraform State? Terraform State is the result of the declared resources defined in our .tf files. This is where all of our infrastructure definition will be stored. Our example is leveraging a local state file.

Alright, enough of that informative mumbo-jumbo. Let's get Terraforming!

Terraform Init

The Terraform Init command creates our localized .terraform directory, downloads our modules and providers, and creates and helps manage our state.

Alright, with that out of the way, we can run our Terraform init command, run the terraform init on our command line.

Here we can see our backend being initialized, our plugins being downloaded, and other setup commands Terraform needs to run.

Terraform Plan

The Terraform Plan command does not create Infrastructure or add anything that is not immutable to our existing Terraform State. It simply checks what changes will be applied against the already created resources within the state and refreshes things like data items based on their current result.

Let's run our plan command and see what comes back.

Wunderbar! We have a plan. Our next step is to run an apply.

Terraform Apply

A Terraform Apply will be the command we use to build our Infrastructure. Although there is a flag to -auto-approve changes, we will be prompted (with the same result as our Terraform Plan command) to apply our changes. Once we accept them, our output will show the resources being created.

Here we can see our resources have been created, and our outputs have provided us a Public IP (redacted).

SSH Time!

Now that all of our resources are created, we can access our instance via SSH since our Public IP was whitelisted as part of our Terraform.

Here we can see our User Data has applied and renamed our host to our variable value myhost.

Terraform Destroy

Finally, now that our EC2 Instance is up and running, we are going to blow it and the associated Security Group sky high. Not really, but who doesn't like a bit of drama.

To do this it operates very similar to how a terraform apply works, we just need to run a terraform destroy and accept it. Once that's done our state should be mostly empty and our instance terminated in AWS.

Some final items

You may have noticed that in your local directory structure there are now two additional files, the terraform.tfstate and the terraform.tfstate.backup. These are parts of what I had mentioned earlier where Terraform creates its declarative definition of your Infrastructure. In our next article, we will discuss new secure ways to manage our state so it is not part of our code base but remotely accessed during our Plans, Applies, and Destroys.

Whats Next?

I hope you've enjoyed this little foray into the world of Infrastructure as Code basics with Terraform. In our next article we will dive into more advanced concepts like Remote State Management, Modules for repeatable code, and other providers, such as Kubernetes.

This post appeared on the Insights blog at NextLink Labs. Check out that blog for more devops and devops consulting services posts.

An Introduction to Terraform and Infrastructure as Code

Nextlink Labs — Wed, 12 May 2021 20:45:18 +0000

Introduction to Terraform and Infrastructure as Code

One of the most important tools we use is Hashicorp Terraform. This post is the first of our multi-part series featuring Terraform. In this series we will go through:

Overview of Terraform and Benefits of Infrastructure as Code
Getting started guide for Terraform
Best Practices for Managing Terraform

What is Terraform and how does it enable DevOps

DevOps practices are adopted in order to increase the velocity that code is delivered through making it simple and automated to deploy your applications/systems, regardless if enhancements to those systems require new infrastructure pieces to be added. Terraform fills an important gap here for DevOps in making provisioning new cloud infrastructure simple and repeatable.

Hashicorp Terraform is an open source Infrastructure as Code tool which can be used to provision infrastructure across multiple clouds.

While individual cloud providers such as AWS and Azure have tools like Cloudformation and Azure Resource Manager respectively, we prefer to use cloud agnostic tools like Hashicorp Terraform.

We will discuss the benefits of using a generic tool like Terraform below, but first let’s talk a little about why Infrastructure as Code is so important to a company's cloud operation strategy.

Benefits of Infrastructure as Code

Before Infrastructure as Code tools, managing IT infrastructure in the cloud was very similar to managing on premise infrastructure: very manual and unwieldy.

For companies who have very large workloads on AWS or other public cloud providers the number of resources in their cloud becomes very difficult to manage manually. Using Infrastructure as Code like Terraform provides a number of benefits to software organizations:

Consistency across environments: Since you are defining all of your infrastructure in terraform modules or templates, it is easy to reapply this same template with slightly different variables. This is ideal for a proper DevOps environment and following 12 factor app best practices which call for consistency across development/staging/production environments.
Configuration Consistency: Since modules are versioned and reused, you can continually improve these modules over time. Terraform modules can be used to define typical architecture for an organization’s preferred application stacks. For example if they commonly use AWS and write code in Django or NodeJS they can have a module which defines an RDS DB, a load balancer, launch configuration, etc. Improving these modules as you continue to work with them provides great efficiency
Self Documenting Infrastructure: As organizations grow, it can become difficult to spread the knowledge of the architecture in use. Often times this information is in the minds of a few key engineers on the team. When using an Infrastructure as Code tool like Hashicorp Terraform, the code itself can serve as a clear description of the desired state of the architecture/infrastructure. This can make it easier to bring new engineers into an organization and quickly understand how things are managed and deployed
Developer Efficiency: With an Infrastructure as Code tool, common tasks are easily repeatable and making changes can be completed, tested, and deployed very quickly. Tasks such as adding an Elasticache instance or making a change to a VPC network can be done with just a quick code update

Intro to Terraform

Terraform was created by a company called Hashicorp and originally launched in 2014. Terraform is written in Go and it is used to build, change, and version infrastructure across many different cloud providers.

It is compatible with many different providers, from the major ones such as Amazon AWS, Azure, and GCP to other tools like Gitlab, Github, and Digital Ocean. A full list of providers can be found at: https://www.terraform.io/docs/providers/index.html

Terraform comes in both open source and enterprise flavors. In this article, we will focus mainly on the open source version although many principles apply to both.

Terraform “code” is a series of configuration files written in HCL (Hashicorp Configuration Language) which describe a desired state of provisioned infrastructure (aka a declarative infrastructure tool). As an example of what this means: we may have a terraform file which describes creating a VPC in AWS as well as attaching an internet gateway to it.

This configuration file is descriptive of the desired state and when terraform code runs it automatically will compare the current state vs the desired state to determine what operations need to be run. This is different than many other tools which describe those steps specifically. In our opinion, this is one of the features that makes Terraform so great, you can just describe how you’d like things to be and not have to worry about different environments/applications needing a different set of operations to be run.

The HCL language is meant to be interoperable with json and both human and machine friendly. We feel that Terraform is also very strong in this area, as when used correctly it is simple enough for novices to follow what is happening but is capable of being used to manage fleets of servers

With just some basic configuration files written in HCL we have been able to manage thousands of cloud resources with a very small number of operators.

While there are many things that must be learned to use Terraform in a secure, mature, and scalable manner we hope this guide will get you started.

Next Steps

The next article in this series will cover some basic examples of Terraform in AWS for your consumption. We hope by discussing the patterns we've adopted that you will be able to build on our knowledge and previous mistakes.

This post was written by Jordan Saunders and appeared on the Insights blog at NextLink Labs. Check out that blog for more devops and devops consulting services posts.