DEV Community: Azeez Lukman

Leveraging Deployment Inputs in VMware Orchestrator (VRO) for Enhanced Workflow Customization

Azeez Lukman — Sat, 07 Oct 2023 01:05:46 +0000

VMware Orchestrator (VRO) is a powerful tool that offers a wide array of functionalities for managing and orchestrating deployments. One crucial aspect of these deployments is the ability to customize them according to specific requirements. VRO allows users to declare deployment inputs, which can be subsequently employed to tailor the deployment process to meet specific needs.

In this article, we'll delve into the intricacies of obtaining deployment inputs in VRO and how to effectively utilize them to enhance the customization of your workflows. We'll explore a scenario where these inputs are invaluable, such as allowing users to specify security groups or instance profiles during the deployment process. We will also discuss the role of a VMware vRealize Automation (vRA) endpoint in accessing deployment events, which serve as a repository for these essential inputs.

Understanding Deployment Inputs in VRO

Before we dive into the technical details, let's first grasp the concept of deployment inputs in VRO. Deployment inputs are essentially parameters or variables that allow users to influence the behaviour of their deployments. These inputs can range from simple values, like specifying the number of instances, to more complex configurations, such as security group associations or instance profile assignments. They provide a way to fine-tune deployments without the need for manual intervention at every step.

Imagine a scenario where you are orchestrating the deployment of virtual machines within a cloud environment. You might want to allow users to define the security groups and instance profiles that should be associated with these virtual machines. This is where deployment inputs come into play. Users can provide these inputs via an input form or an API call, allowing for a high degree of customization.

Accessing Deployment Events via vRA Endpoint

To effectively harness deployment inputs, VRO relies on a connection to VMware's vRealize Automation (vRA). vRA serves as a comprehensive cloud automation platform that can be integrated with VRO to extend its capabilities. It's through this integration that VRO gains access to deployment events, which are pivotal for obtaining the deployment inputs.

Deployment events are essentially records or logs of what transpires during a deployment process. They contain a wealth of information, including the deployment inputs provided by users. These events are invaluable for troubleshooting, auditing, and most importantly, for customizing workflows.

To access these deployment events, you need to make a request to the endpoint below:



/deployment/api/deployments/" + deploymentId + "/events"

Remember to replace deploymentId with the id of your deployment. This endpoint can query and retrieve deployment events, making the deployment inputs available for further processing.

Creating a Custom Action to Manipulate Deployment Inputs

Let's move to the practical implementation of obtaining and utilizing deployment inputs. The first step is to create a custom action within VRO. This action will be designed to interact with the vRA endpoint and retrieve the deployment inputs associated with a specific deployment.

Create New Action

Create an action by navigating to Orchestrator and on the actions page you can find a button to create a new action at the top of the page, use the screenshots below for reference:

You can name it “getDeploymentInputs” and select a module to store it.

Declare Inputs

Now, it’s time to build out the action. This action should take the deployment ID as input. With the deployment ID, it can query the vRA endpoint to fetch the corresponding deployment events.

Find Inputs and create a new one, specify deploymentId as the name, as for the type, the returned events are typically represented as an array of events, which includes the deployment inputs, this can be represented as Properties, enter Properties for the type.

Write Custom Script

Declare a path as the endpoint specified earlier, and the action should be GET:



var action = "GET";
var path = "/deployment/api/deployments/" + deploymentId + "/events";

Then, add the script below:



deploymentEventsResponse = System.getModule("com.vra.rest").vraRestCall(action, path, null, null);
var deploymentInputs = deploymentEventsResponse['content'][0]['inputs'];

System.log("deploymentInputs: ");
System.log(deploymentInputs);

return deploymentInputs;

This script sends a request to retrieve the deployment events. Let's break down its components:

Making the API Call: The vraRestCall method is used to make an HTTP GET request to the API path. This function is part of the "com.vra.rest" module, which must have been properly configured in your VRO environment.
Processing the Response: We store the response from the API call in the deploymentEventsResponse variable. This response typically contains all the deployment events.
Accessing Deployment Inputs: The script extracts the inputs array from the content of the first event in the response. This is where the deployment inputs are stored.
Logging and Returning the Inputs: You can then log the retrieved deployment inputs using System.log for debugging and visibility. Finally, it returns the deployment inputs for further use in your workflow.

Save your action and you should be all good 🙂

Integrating the Custom Action into Workflows

With the custom action in place, you can seamlessly integrate it into your VRO workflows. For instance, in the scenario mentioned earlier, where users specify security groups and instance profiles, you can use the custom action within the deployment workflow. This ensures that the desired customization occurs as part of the deployment process.

The flexibility of this approach allows for a wide range of use cases. You can use deployment inputs to automate various aspects of your cloud infrastructure, from network configurations to application installations, all while providing users with the ability to tailor these deployments to their specific needs.

Conclusion

In conclusion, VRO provides a robust framework for orchestrating and automating deployments, and the ability to leverage deployment inputs adds an extra layer of customization and flexibility to this process. By establishing a connection with vRA, accessing deployment events, and creating custom actions to manipulate deployment inputs, you can tailor your workflows to meet your organization's unique requirements. This not only streamlines your cloud management but also empowers your users to actively participate in the customization of their deployments. The synergy between VRO and vRA, in combination with custom actions, enables you to unlock the full potential of cloud orchestration and automation.

My name is Azeez Lukman, and I'm passionate about simplifying complex technical processes and sharing insights into the world of software engineering and cloud automation. Find me everywhere @robogeek95 for more articles like this.

Comprehensive Guide to Infrastructure Continuous Integration and Continuous Deployment with Terraform and GitLab CI/CD

Azeez Lukman — Thu, 21 Sep 2023 08:25:29 +0000

The practice of Infrastructure as Code (IAC) has witnessed an exponential surge in popularity in recent years. DevOps engineers, in particular, have found IAC to be a transformative approach, enabling them to automate infrastructure provisioning and management, resulting in more efficient and scalable processes.

In this comprehensive guide, we will delve into the world of Infrastructure Continuous Integration (CI) and Continuous Deployment (CD) with GitLab, tailored for advanced DevOps professionals aiming to optimize their workflows.

Getting to Know Terraform

Before we embark on our journey into CI/CD with GitLab, it's crucial to have a solid understanding of Terraform. Terraform, developed by HashiCorp, is a powerful open-source Infrastructure as a Code tool. It empowers users to define their infrastructure as code, offering a clear and repeatable method for provisioning and managing resources across diverse cloud providers and on-premises environments. Some key advantages of Terraform include:

Declarative Configuration: Terraform employs a declarative syntax, allowing you to describe the desired end state of your infrastructure, while Terraform determines the steps to achieve it.
Multi-Cloud Support: Terraform boasts compatibility with a multitude of cloud providers, facilitating the implementation of a multi-cloud strategy.
Version Control: Terraform configurations are stored in version control repositories, fostering collaboration and change tracking.

Overview of GitLab CI/CD

GitLab encompasses a comprehensive DevOps platform with robust CI/CD capabilities. It empowers DevOps engineers to automate processes such as code testing, building, and deployment, including Terraform configurations. The fundamental components of GitLab CI/CD include:

Runners: GitLab Runners serve as agents responsible for executing CI/CD jobs. They are adaptable and can be installed on various platforms, ensuring that your pipelines can run tasks across diverse environments.
Pipelines: Pipelines define the complete process of code building, testing, and deployment.
Jobs: Each job within a pipeline represents an individual task, such as executing tests or applying Terraform configurations. We will also explore how to use GitLab's artifact feature to store files and data generated by a job, which is invaluable for transferring data between jobs or stages within a pipeline.

Setting Up GitLab for Terraform

To kickstart your GitLab CI/CD journey for Terraform, the initial step is to create a GitLab project or repository to host your Terraform code. Properly structuring your Terraform project within a Git repository is important for effective version control and collaboration. Consider organizing your repository with directories dedicated to different environments (e.g., development, staging, production) and adopt Git branching strategies.

Creating CI/CD Pipelines for Terraform

GitLab CI/CD pipelines are defined using a .gitlab-ci.yml file situated within your repository. This YAML file outlines the stages, jobs, and dependencies for your CI/CD process. For Terraform, your CI/CD pipeline may encompass stages such as:

Linting: The process of examining Terraform code for syntax and style-related issues.
Validation: Ensuring that Terraform configurations are both valid and safe to apply.
Planning and Applying: Creating an execution plan and implementing changes to the infrastructure.
Testing: Running infrastructure tests to validate that the changes meet the specified requirements.

Create the Pipeline File

Start by creating a file named .gitlab-ci.yml in your GitHub project. This YAML file will define your CI/CD pipeline configuration. The file should be created in the root directory to ensure that GitLab can automatically detect and use it to set up and run your CI/CD pipeline for that project. If you place it in a subdirectory, GitLab will not recognize it as the configuration file for the project's pipeline.

touch ****.gitlab-ci.yml****

Define Stages and Jobs

Inside your .gitlab-ci.yml file, define the stages and jobs for your Terraform CI/CD pipeline. Here's an example configuration:

stages:
  - plan
  - apply

variables:
  TF_VAR_AWS_ACCESS_KEY_ID: "<AWS_ACCESS_KEY_ID>"
  TF_VAR_AWS_SECRET_ACCESS_KEY: "<AWS_SECRET_ACCESS_KEY>"

plan:
  stage: plan
  script:
    - terraform init
    - terraform plan -out=terraform.tfplan

apply:
  stage: apply
  script:
    - terraform apply -auto-approve terraform.tfplan
  only:
    - master # You can adjust the branch where you want to apply changes

In this example, we have two stages: plan and apply. The plan stage initializes Terraform and creates a plan file, while the apply stage applies the changes using the generated plan file. We also set two variables, TF_VAR_AWS_ACCESS_KEY_ID and TF_VAR_AWS_SECRET_ACCESS_KEY, which allow you to specify your AWS credentials, remember to replace them with your actual values. More information on securing the use of these credentials will be provided later in this article.

ok… let’s break things down a little more

In GitLab CI/CD, a pipeline is divided into stages, and each stage contains one or more jobs. Stages represent different phases of your CI/CD process, and jobs within each stage define specific tasks to be executed. In the example above, we've defined two stages: plan and apply. Here's what each stage represents:

Plan Stage: This is the first stage in our CI/CD pipeline. In this stage, we typically perform tasks related to Terraform planning. Planning involves checking the current state of your infrastructure, comparing it to your Terraform code, and generating an execution plan that describes what changes will be made when you apply your Terraform configuration.

Within this stage, a job named plan is defined. This job belongs to the plan stage and contains the terraform commands to init and plan the terraform configuration.
Apply Stage: The second stage in our pipeline is the apply stage. This is where we actually apply the changes to our infrastructure based on the execution plan generated in the previous stage. Applying Terraform changes may involve creating, updating, or deleting resources in your cloud infrastructure.

The job within this stage apply stage specifies the script that defines the commands to be executed as part of the job. Here's what the command does:
- terraform apply -auto-approve terraform.tfplan: Applies the Terraform changes using the execution plan generated in the previous stage. The auto-approve flag automatically approves and applies the changes without requiring manual confirmation.
- only: This section specifies the branch where this job should be executed. In this example, the apply job will only run when changes are pushed to the main branch. You can adjust this branch name to match your specific deployment strategy.

Trigger the pipeline run

The most common and fundamental way to trigger a GitLab CI/CD pipeline is by pushing code changes to your GitLab repository. GitLab is designed to automatically detect changes in your codebase and initiate the CI/CD pipeline accordingly. This seamless integration ensures that your pipeline runs whenever there are new commits or merged branches in your repository. You can also manually trigger a pipeline via the GitLab UI.

Save your .gitlab-ci.yml file and commit it to your GitLab repository. Push the changes to trigger your first CI/CD pipeline.

Once you've pushed your changes, visit your GitLab repository's CI/CD section to monitor the progress of your pipeline. GitLab will automatically detect the .gitlab-ci.yml file and start executing the defined stages and jobs.

Review Pipeline Output

As the pipeline runs, you can review the output of each job and check for any errors or issues. If everything is successful, your Terraform changes will be applied to the specified environment.

Coffee break…

You made it this far Congratulations on your first successful pipeline deployment.

Grab some coffee before we continue to reviewing some more concepts you can put in place to ensure you are securely deploying these changes into your environment.

Manual Approvals

In the previous example, you may have noticed that the pipeline run deployed changes directly into the environment. However, this is usually not desired in production environments.

Manual approvals in GitLab CI/CD are a critical feature that allows you to introduce a human validation step into your automated pipeline. This validation ensures that certain stages or jobs in your CI/CD process won't proceed without manual confirmation, typically by authorized team members. This concept helps maintain control, security, and accuracy in your deployment process, particularly when deploying to production environments or making significant changes to your infrastructure.

To make a job dependent on manual approval, you include the when: manual attribute in the job definition. This attribute tells GitLab that the job should not automatically start but should wait for manual intervention.

Update the apply stage in your configuration to reflect the attribute:

apply:
  stage: apply
  script:
    - terraform apply -auto-approve terraform.tfplan
  only:
    - master # You can adjust the branch where you want to apply changes
  when: manual

How does the manual approval work?

Manual Trigger: When the pipeline reaches a job that has when: manual, GitLab will display a "Play" button next to that job in the pipeline view. Only users with appropriate permissions can click this button to manually trigger the job.
Authorization: GitLab ensures that only authorized users, such as team leads or administrators, can approve and trigger manual jobs. Access control and permissions are managed through GitLab's role-based access control system.
Manual Intervention: When a team member clicks the "Play" button to trigger a manual job, they may be prompted to provide additional information or confirm the action. This step adds a layer of accountability and documentation to the process.
Job Execution: Once the manual job is triggered, it proceeds as usual, executing the defined script or commands.
Subsequent Jobs: After the manual job has been executed and approved, the pipeline continues with any subsequent jobs that are not set to when: manual.

Securely Storing Secrets

To manage sensitive information like API keys and credentials, utilize GitLab's built-in secret management. Store these secrets as CI/CD environment variables to keep them secure.

Securing secrets in GitLab CI/CD is a critical practice for several important reasons. It is not just a best practice; it's a fundamental aspect of maintaining the security, compliance, and reliability of your automation processes. By implementing robust secret management practices, you reduce the risk of data breaches, protect sensitive information, and enhance the overall security posture of your projects.

GitLab provides a robust mechanism for managing and securely accessing secrets. Let's explore this concept step by step:

Access Project Settings

Log in to your GitLab account.
Navigate to the project where you want to store secrets.
Click on "Settings" in the project's sidebar.

Configure CI/CD Variables

In the project's "Settings" page, click on "CI/CD" in the left sidebar.
Scroll down to the "Secret variables" section and click "Expand."
Here, you can add and manage secret variables specific to your project. These variables are securely encrypted and can be accessed by CI/CD jobs.

Add Secret Variables

Click on the "Add variable" button.
Fill in the variable details:
- Key: This is the name of the secret variable, e.g., API_KEY.
- Value: Enter the actual secret value, such as an API key or password.

Add secret variables for both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY these would be available in the environment variables of your pipeline run and are automatically passed to the terraform.

Protect Secret Variables

GitLab provides an option to protect secret variables. When a variable is protected, it cannot be accessed by unprotected branches or tags. This is useful for ensuring that sensitive secrets are only available in authorized environments.

Accessing Secret Variables in CI/CD Jobs

Now you can securely utilize the secret variables in your pipeline. Update the variables section of your YAML file as shown below:

variables:
  TF_VAR_AWS_ACCESS_KEY_ID: "$AWS_ACCESS_KEY_ID"
  TF_VAR_AWS_SECRET_ACCESS_KEY: "$AWS_SECRET_ACCESS_KEY"

These variables are used to pass the AWS access key and secret access key to the Terraform commands. They are denoted by the $ symbol.

Conclusion

The implementation of Infrastructure Continuous Integration and Continuous Deployment with GitLab for Terraform equips DevOps engineers with the means to streamline their workflows, ensure infrastructure reliability, and expedite the delivery of new features and updates. By embracing IAC principles and leveraging GitLab's CI/CD capabilities, you are well-prepared to meet the demands of modern infrastructure management.

Begin by establishing your GitLab project, defining CI/CD pipelines, and integrating Terraform. As you gain experience, delve into advanced concepts to further enhance your journey in infrastructure automation. The synergy between GitLab and Terraform provides a formidable toolkit for achieving excellence in Infrastructure as Code.

Additional Resources

A Comprehensive Guide on Migrating OCI Terraform State from Local to Remote Storage

Azeez Lukman — Thu, 14 Sep 2023 23:39:12 +0000

Introduction

Infrastructure as Code (IaC) has revolutionized how organizations manage their cloud resources. Terraform, a popular IaC tool, enables developers and operations teams to define and provision infrastructure using declarative configuration files. When working with Oracle Cloud Infrastructure (OCI) and Terraform, managing the state file is a crucial aspect of your infrastructure workflow.

In this guide, we will explore the process of migrating your Terraform state from a local file system to an OCI Object Storage bucket. This migration enhances the security and scalability of your state management and allows for better collaboration among team members.

Why Migrate Terraform State to an Object Storage?

To begin with, it is important to thoroughly understand the differences between storing your Terraform state on a local file system versus on OCI Object Storage. While a local file system may seem convenient at first, it can lead to potential data loss, security breaches, and difficulty in scaling your infrastructure. Object Storage, on the other hand, offers a highly secure and scalable solution to store your Terraform state, ensuring that your data is always accessible and protected.

Migrating Terraform state from your local file system to an Object Storage generally offers several compelling benefits:

Enhanced Security: Storing your state in OCI Object Storage ensures that it's stored in a secure and redundant manner. You can take advantage of OCI's robust security features, including encryption and IAM (Identity and Access Management) policies, to control access.
Improved Collaboration: Object Storage enables seamless collaboration among team members by providing a centralized location for storing and sharing Terraform state. Multiple team members can access and modify the state file without the risk of conflicts.
Scalability and Reliability: OCI Object Storage is designed to scale automatically to accommodate your growing infrastructure. It offers high availability and durability, ensuring your Terraform state is always accessible.
Versioning and Auditing: Object Storage can be configured to enable versioning, allowing you to track changes to your Terraform state over time. This feature is invaluable for auditing and troubleshooting.

Prerequisites

Before you begin the migration process, ensure you have the following prerequisites in place:

OCI Account: You should have an active OCI account with the necessary permissions to create and manage Object Storage buckets.
Terraform Installed: Ensure that Terraform is installed on your local machine. You can download it from the official Terraform website (https://www.terraform.io/downloads.html).
OCI CLI: Install the Oracle Cloud Infrastructure Command Line Interface (OCI CLI) and configure it with the necessary credentials. You can follow the official documentation for installation instructions (https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/cliinstall.htm).

Prepare for the Migration Process

The migration process involves creating an Object Storage bucket, updating the Terraform configuration to specify the new backend configuration for storing the state in OCI Object Storage, initializing Terraform with the new backend configuration and migrating the existing local Terraform state to OCI Object Storage.

Create an Object Storage Bucket

The first step is to identify your namespace. In OCI, a namespace is a unique identifier that is used to create a unique name for each object in Object Storage. The namespace is automatically generated when you sign up for an OCI account and cannot be changed.

Retrieve your OCI Object Storage namespace using the following command:

namespace = oci os ns get

This will return a string that represents your namespace and set it as an environment variable in your CLI for easy reference in subsequent commands.

Next, create an Object Storage bucket in your OCI console, ensuring you choose a unique and meaningful name for your bucket. You can also create the bucket using the OCI CLI with the following command:

oci os bucket create --name <bucket-name> --namespace $namespace

Replace "bucket-name" with the actual bucket name and the namespace would be the one set from the previous command.

Use the following commands to migrate your existing local Terraform state to OCI Object Storage, including the additional commands for setting up authentication:

Create a Customer Secret Key

The customer secret key is a key generated for your user in OCI that you can use for authentication when interacting with OCI Object Storage. It consists of an access key ID and secret access key that you can use to authenticate and connect with your Object Storage bucket.

First export your email address as a variable:

export email=<your_email>

This command exports your console email address as a variable in the Command Line Interface (CLI) environment. It sets the variable email to your email address.

Verify the Exported Email:

echo $email

Then run the OCI CLI command to Query Your User Data from OCI IAM service to retrieve user data:

oci iam user list --query "data[?\"name\"=='$USERNAME']".{"name:\"name\",id:\"id\""} --output table

The -query parameter filters the data to find a user with a name matching the value in the $USERNAME variable and the resulting table includes the user's name and OCI User ID (OCIUID).

Export the Returned OCIUID as a Variable:

export OCIUID=<YOUR_OCIUID>

This command exports your OCI User ID (OCIUID) as a variable in the CLI environment. Replace the provided value with the OCIUID obtained from the previous command.

Finally, we can now create a customer secret key for your user using the OCI CLI. Create a customer secret key using the command below

oci iam customer-secret-key create --user-id $OCIUID --display-name 'key-tf-test' --query "data".{"AWS_ACCESS_KEY_ID:\"id\",AWS_SECRET_ACCESS_KEY:\"key\""} --output=table

Create a customer secret key for your user and display it in a tabular format. The -user-id parameter specifies your OCI User ID, and the -display-name parameter sets a name for the secret key. The -query parameter extracts the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY fields from the command's output and formats the output as a table.

Note down the result, we are going to use this for authentication when interacting with OCI blob storage.

Now, export AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as variables:

export AWS_ACCESS_KEY_ID=<YOUR_AWS_ACCESS_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<YOUR_AWS_SECRET_ACCESS_KEY>

These commands export the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY obtained from the previous OCI CLI command as environment variables.

Update Terraform Configuration

In order to establish the connection between your Terraform workspace and the remote state stored in Object Storage, Terraform needs to be aware of the new backend configuration, which directs it to use OCI Object Storage for storing and managing the state of your infrastructure.

Inside your Terraform configuration file that defines the provider, typically a provider.tf file, you would locate or add a terraform block. This block is used to configure various aspects of Terraform's behaviour, including the backend configuration for storing the state. Here's a basic example of how you can configure the block:

terraform {
  backend "s3" {
    bucket         = "bucket-name"
    key            = "terraform.tfstate"
    region         = "your-region"
        endpoint       = "https://<bucket-name>.compat.objectstorage.<region>.oraclecloud.com"

    skip_region_validation      = true
    skip_credentials_validation = true
    skip_metadata_api_check     = true
    force_path_style            = true
  }
}

Let's break down what each of these settings means:

backend: Specifies the backend type. In this case, we are using "s3" because OCI Object Storage is S3-compatible.
bucket: The name of the OCI Object Storage bucket where your Terraform state will be stored. Replace "-bucket-name" with the actual name of the bucket you created earlier.
key: The name of the state file within the bucket. It's typically named "terraform.tfstate" but can be customized if needed.
region: The OCI region where your Object Storage bucket is located. Replace "your-region" with the appropriate region identifier.

skip_region_validation, skip_credentials_validation, skip_metadata_api_check, and force_path_style: These settings control various validation and behaviour options for the backend configuration. Here's what each setting does:

skip_region_validation: Skips validation of the region name, which is useful when using custom endpoints.
skip_credentials_validation: Skips validation of credentials. Be cautious when using this option, as it can potentially lead to authentication issues.
skip_metadata_api_check: Skips checking the metadata API for credentials. This can be useful when running Terraform in environments where the metadata API is not available, such as some development environments.
force_path_style: Forces the use of path-style URLs when interacting with the Object Storage service. This is commonly used when working with S3-compatible storage services.

Migrate Existing State

Now that the configuration has been completed, we can proceed to migrate the state data. This process involves transferring the data from the current system to the new one in a secure and efficient manner. It is important to ensure that all data is accurately transferred and that there are no discrepancies or errors during the migration process.

Initialize Terraform

Start by running the following command to initialize Terraform with the new backend configuration:

terraform init
Initializing modules...

Initializing the backend...
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "s3" backend. No existing state was found in the newly
  configured "s3" backend. Do you want to copy this state to the new "s3"
  backend? Enter "yes" to copy and "no" to start with an empty state.

  Enter a value:

This command will prompt you to migrate your state. Confirm the migration by entering “yes” to proceed.

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Reusing previous version of hashicorp/oci from the dependency lock file
- Using previously-installed hashicorp/oci v5.6.0

╷
│ Warning: Additional provider information from registry
│ 
│ The remote registry returned warnings for registry.terraform.io/hashicorp/oci:
│ - For users on Terraform 0.13 or greater, this provider has moved to oracle/oci. Please update
│ your source in required_providers.
╵

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

💡 Using the -migrate-state option in terraform init command can perform automatic migration of the state. This is useful if you already a different backend configured for your state.

Verify and Test

After the migration, you can verify that the state is now stored in OCI Object Storage by running terraform state list or finding the object in the blob storage in the console. Remember to Test your infrastructure deployments to ensure that everything is functioning as expected.

Conclusion

Migrating your Terraform state from a local file system to OCI Object Storage is a strategic move that enhances security, collaboration, scalability, and reliability in your infrastructure management process. By following the steps outlined in this guide, you can seamlessly transition to a more robust state management solution and take full advantage of Oracle Cloud Infrastructure's capabilities. Happy Terraforming!

VRA API Authentication: Ensuring Secure Access and Understanding Authentication Methods

Azeez Lukman — Mon, 11 Sep 2023 00:52:00 +0000

Using APIs is an effective way to programmatically access vRealize Automation (vRA). However, secure access to your API is essential when managing and automating your infrastructure. This article explores VRA API authentication using API tokens, a secure and efficient way to programmatically access vRA resources.

Understanding vRealize Automation

VMware vRealize Automation is a robust cloud management platform that empowers orga nizations to automate and orchestrate the provisioning and management of various cloud resources, including virtual machines and containers. This platform aids businesses in delivering IT services rapidly, efficiently, and securely. A key feature of vRealize Automation is its robust API infrastructure, which enables developers to seamlessly integrate vRA into their applications and workflows.

Key Concepts of VRA API Authentication

Before delving into the authentication methods, it's essential to establish some fundamental concepts:

1. Authentication: Authentication is the process of confirming the identity of a user or application. It typically involves the use of credentials such as usernames and passwords.

2. Authorization: After successful authentication, authorization determines what actions a user or application is allowed to perform within the VRA environment. It defines access permissions and rights.

3. Tokens: Tokens serve as proof of authentication in API requests. Once a user or application is authenticated, they receive a token that must be included in subsequent API requests to prove their identity.

VRA API Authentication Methods

vRealize Automation provides various authentication methods to cater to diverse use cases and security requirements. The choice of method depends on your specific needs and existing infrastructure. Here are some of the most commonly used VRA API authentication methods:

1. Basic Authentication: This is the simplest form of authentication, involving a username and password. While easy to implement, it is generally considered less secure, as credentials are transmitted in plain text. It's recommended to use this method over secure channels like HTTPS.

2. API Tokens: API tokens are a more secure alternative to basic authentication. Users generate tokens with limited access rights and use them to authenticate API requests. Tokens can be revoked or refreshed as needed, enhancing security.

3. OAuth 2.0: OAuth 2.0 is a widely adopted authentication framework that provides a standardized way to grant limited access to resources without exposing credentials. VRA supports OAuth 2.0, making it a robust choice for integration with third-party applications.

4. SSO Integration: VRA can be integrated with Single Sign-On (SSO) systems like VMware Identity Manager or external identity providers. This streamlines authentication processes and centralizes user management.

Using API Tokens for Authentication

Now, let's focus on API tokens, a secure and versatile authentication method. But why should you choose API tokens for authentication in the first place?

1. Security: API tokens are typically long, randomly generated strings that are difficult to guess. They do not contain sensitive information like passwords, making them more secure for authentication.

2. Scalability: API tokens can be easily generated and revoked as needed. This makes them ideal for managing access to your vRA resources, especially in dynamic environments.

3. Expiration: Tokens can have expiration dates, reducing the risk of unauthorized access if a token is leaked or compromised.

4. Granular Access Control: With API tokens, you can control the level of access granted to a user. For example, you can create tokens with read-only access or restrict access to specific resources.

How to Obtain an API Token for Authentication

To interact with the vRA API using API tokens, follow these steps:

Export Variables

Start by exporting the values on the command line, including the vRA hostname, your username, and password. This makes it easier to reference these values in the other steps.

service_url='https://<vRA_HOST>'
username='<your_username>'
password='<your_password>'

Request an API Token

VRA API authentication uses an API token to authenticate the user or client and request an access token. The API token is obtained through an initial authentication process where credentials are provided, and it does not grant access to resources directly. Instead, it is used to acquire an access token, which determines the scope of access. The API token may have a longer lifetime than the access token but is still limited and used to request access tokens periodically.

Make a request to the Identity Service API to obtain the API token. The token will be attached to your request headers for authentication:

api_token=`curl -X POST \
 "$service_url/csp/gateway/am/api/login?access_token" \
 -H 'Content-Type: application/json' \
 -d '{
 "username": "'"$username"'",
 "password": "'"$password"'"
}' | jq -r .refresh_token`

Retrieve the Access Token

Now that you have the API token, make another request to get and assign the access token. The access token is the JSON web token that grants access to specific vRA resources and defines the scope of actions the client can perform.

access_token=`curl -X POST \
 "$service_url/iaas/api/login" \
 -H 'Content-Type: application/json' \
 -s \
 -d '{
 "refreshToken": "'"$api_token"'"
}' | jq -r .token`

Testing Your Access

To ensure that you have successfully authenticated and can interact with the VRA API, you can perform actions like listing all cloud accounts or workflows. Here's an example using the access token:

curl -X GET $url/iaas/api/cloud-accounts?apiVersion=$api_version -H 'Content-Type: application/json' -H "Authorization: Bearer $access_token" | jq "."

Similarly, you can list workflows or perform other actions based on your requirements.

Conclusion

vRA API authentication is a critical component of securing your infrastructure management and automation tasks. By understanding the available authentication methods and choosing the right one for your needs, such as API tokens for enhanced security and control, you can ensure the integrity and security of your vRealize Automation environment.

Rerefences

https://dokumen.tips/download/link/vrealize-automation-api-programming-guide-vrealize-to-make-api-service-calls.html

Back to the blogosphere (Sharing My Journey in the World of Software Engineering)

Azeez Lukman — Thu, 07 Sep 2023 02:43:43 +0000

Introduction

Greetings, dear readers! It's been a while since I last shared my thoughts, learnings, and discoveries with you all. Life has taken me on a whirlwind journey filled with professional commitments, career growth, and personal responsibilities. As I sit down to reconnect with you through this blog post, I can't help but feel an exhilarating rush of excitement. I'm back, and I'm more pumped than ever to dive into the world of software engineering and DevOps, sharing insightful articles that offer solutions and fresh perspectives.

The Intermission

You might be wondering why I seemingly vanished from the blogging scene. Well, life happened. New projects, challenging assignments, and career milestones demanded my attention. Balancing these commitments alongside my personal life often left me with little time to put pen to paper, or rather, fingers to keyboard. But let me assure you, during this period, my passion for learning and growing in the tech world never wavered. If anything, it grew stronger with each experience.

The Journey

During my time away, I delved into uncharted territories of the software engineering and DevOps landscape. I embraced new tools, languages, and methodologies, and I tackled complex challenges that pushed the boundaries of my knowledge. There were moments of frustration and triumph, but each step further fueled my desire to share these experiences with you. The journey was a testament to the fact that learning never stops, even when our lives get busier.

The Return

And now, here I am, back in the blogosphere, ready to pick up where we left off. The excitement of sharing my learnings, the joy of discovering innovative solutions, and the sense of community we've built together beckoned me back. With a renewed sense of purpose and a commitment to delivering value, I am eager to embark on this next chapter of our journey.

What to Expect

In the coming weeks and months, expect a variety of articles that delve into the heart of software engineering and DevOps. I'll be sharing insights into the latest trends, demystifying complex concepts, and providing practical solutions to the challenges we encounter in our everyday tech lives. From coding tips that streamline your workflow to DevOps strategies that enhance collaboration and efficiency, we'll cover it all.

Conclusion

The tech community we've built is a source of inspiration aiding contributions and learning together and I am genuinely thrilled to be a part of it.

Thank you for joining me on this journey. Let's dive back into the world of software engineering and DevOps with vigour, curiosity, and a commitment to continuous growth. Stay tuned for the upcoming articles – I promise they'll be worth the wait.

I'm Azeez Lukman, a software engineer with a deep passion for unravelling the complexities of code and systems. You can always reach out to me at @robogeek95 to connect, share your thoughts, or simply geek out about all things tech.

Creating a Chrooted Environment

Azeez Lukman — Tue, 27 Sep 2022 10:01:49 +0000

Introduction

Every process in Linux has a base working directory called the root directory. The root directory designated by the slash sign is where every other directory in your system begins. Although you cannot go above this directory, you can change it.

The chroot command changes the root directory for any currently running process as well as its child processes. This article will walk you through the process of creating a chrooted environment, adding access to Linux commands, along with including shared libraries for these commands.

What is chroot

So what does it mean to change the root directory? According to the official Debian documentation:

A chroot is an operation that changes the apparent root directory for the current running process and their children.

Hmm, maybe that isn’t clear enough still… When you change the root directory, commands run from the new environment are locked down to that environment, they are not able to access any files or even commands outside of that directory. When you use the chroot command, you get this completely isolated environment also referred to as a chroot jail.

The “*chroot” command can be very useful in situations when you need t*o create a test environment, or you need to recover the system or password, or even when you need to reinstall the bootloader. There are several other reasons you might want to create a chrooted environment and it’s completely safe to try it out for the sake of learning something new.

Setup jailed root environment

Let’s walk through the process of creating a chrooted environment, and create a new user for that environment.

Performing a chroot requires that you are a super user. So you need to first become the root user by running the command below:

sudo su -

Create the directory we would chroot to within the /home directory named /home/bailey. Although you can chroot into any directory.

Create a directory for bailey with the command below:

mkdir /home/bailey

Create a new user

Create a new user for your environment named bailey.

useradd bailey

Provision commands to run

When we chroot into a directory, we are in a completely isolated environment, you need to provide everything you need afresh including the files and commands so we are limited to the commands and files within this chroot jail directory, in order to enable us to run some commands in a jailed directory we need to make them available there either by installing the commands to that directory or by moving them.

Start by creating the bin and lib64 directories in /home/bailey then move in the needed commands into these directories.

Run the command below to Create two new directories; bin, and lib64, within the /home/bailey directory:

mkdir /home/elba/{bin,lib64}

We need bash, ls, and cat commands

Start by copying /bin/bash on the local computer into the chroot jail at /home/bailey/bin/bash:

cp /bin/bash /home/bailey/bin/bash

Then copy /bin/ls into /home/bailey/bin/ls:

cp /bin/ls /home/bailey/bin/ls

Finally, copy /bin/cat into /home/bailey/bin/cat:

cp /bin/cat /home/bailey/bin/cat

The next step is to find and Copy the libraries needed for bash, ls, and cat over to /home/bailey/lib64

Find the required libraries using ldd:

ldd /bin/bash /bin/ls /bin/cat 
cp /lib64/libtinfo.so.5 \ /lib64/libdl.so.2 \ /lib64/ld-linux-x86-64.so.2 \ /lib64/libselinux.so.1 \ /lib64/librt.so.1 \ /lib64/libcap.so.2 \ /lib64/libacl.so.1 \ /lib64/libc.so.6 \ /lib64/libpthread.so.0 \ /lib64/libattr.so.1 \ /lib64/libpcre.so.1 /home/elba/lib64

Now we have the commands in place, but there’s one more thing to do before creating the chrooted environment.

Create a text file with some sample text in it so we are able to run the commands against it.

Use nano to create the hello.txt file in the /home/bailey and add some text to it:

nano /home/bailey/hello.txt

# Write the following in the file:
Welcome to chroot jail

# Write out and close the file:
^O & ^X

Change the root environment

Finally, create a chrooted environment in /home/bailey with a Bash shell using the chroot command:

chroot /home/bailey /bin/bash

This has now created a chrooted enviroment at /home/bailey

Check the commands

Using the command pwd, confirm the present working directory and then confirm that you can use the ls command to list files in the directory.

pwd
ls

You should see the hello.txt file that was created previously, now confirm you can view the contents of hello.txt

cat hello.txt

Limitations

It's really important to mention that running programs in a chrooted environment is not entirely secure as it is easy for processes to break out of this environment if they are run with root privileges, it cannot handle tampering by privileged users.

It's advised to revoke root privileges after chrooting in order to reduce these security risks. A better option would be to enforce a real jail With other mechanisms like freeBSD jails and more.

Conclusion

You have successfully changed your root environment hereby creating a chrooted environment otherwise known as a chroot jail environment. I hope you enjoyed reading this piece as much as i enjoyed writing it.

Update existing security list ingress rules in OCI with the python SDK

Azeez Lukman — Fri, 19 Aug 2022 22:31:15 +0000

Prerequisites

Some knowledge about python programming language
OCI SDK for python. please check here to install it, if you do not have it.

💡 Feel free to follow along with from this GitHub repository

Introduction

Security lists are components of your VCN that are responsible for allowing traffic in and out of the VCN, they act like firewalls and are applied to the VNICs of your subnets or VCN. You specify the types of traffic allowed in and out with ingress and egress rules respectively.

As your VCN grows in complexity, there could be a need to update the security rules of your security list, this is fairly easy from the console but comes with a gotcha when using the API or SDK as stated in the documentation that egressSecurityRulesor ingressSecurityRules objects you provide replace the entire existing objects. This means you risk replacing the entire rules in an existing security list when you are making an update to an existing security list.

In this article, you would learn how to use the python SDK to update the rules of an existing security list in OCI without losing the existing rules of course.

Import oracle SDK

Let’s get started by preparing our environment and ensuring we have all that’s required to follow along this article.

The oracle python SDK provides the tools you need to interact with the Oracle Cloud Infrastructure using python, so let’s import the SDK module on the command line.

Run the command below in your terminal to install the oracle SDK:

pip3 install oci

Next, create the file to write the python script using the command below:

touch oci-ingress-sl-update.py

This would create a file named [oci-ingress-sl-update.py](http://oci-ingress-sl-update.py), feel free to update the file name.

Next, open the file you created in your favourite code editor, and the code below to import the oci SDK package:

import oci

Authentication

You need to grant access to your tenancy, i would be authenticating using the instance principal, you can use other authentication methods as well:

config = oci.config.from_file()
identity_client = oci.identity.IdentityClient(config)
core_client = oci.core.VirtualNetworkClient(config)

This gets the credentials from the identity file and store in config, oci.config.from_file() takes as arguments, one is for the config file location and the other tells it it to use the default config profile. passing empty arguments falls back to default values with the config file at ~/.oci/identity_file and the profile would be default.

We then use the config to authenticate the identity_client as well as the core_client which we would be using in a minute.

Next, we define the constants:

network_compartment_name = "network_compartment"
vcn_cidr_range = "10.104.0.0/16"
vcn_subnet_cidr_range = "10.103.225.0/28"

network_compartment_name : The compartment that holds your network resources
vcn_cidr_range: The virtual network’s CIDR range
vcn_subnet_cidr_range: The Subnet’s CIDR range

Get compartment data

We would be making use of the compartment id later on. In order to get this, we filter through the list of all compartments and match the compartment that matches the network compartment declared earlier.

Add the following code to get your compartment id:

list_compartments_response = identity_client.list_compartments(
    compartment_id=config['tenancy'],
    lifecycle_state="ACTIVE")

for i in range(len(list_compartments_response.data)):
   if list_compartments_response.data[i].name == network_compartment_name:
     network_compartment_id = list_compartments_response.data[i].id
     break

Now we have the compartment id network_compartment_id, the next step is to identify the VCN we would be working with within the compartment.

Add the following code to get the VCN data below your script:

list_vcns_response = core_client.list_vcns(
    compartment_id=network_compartment_id)

for vcn in range(len(list_vcns_response.data)):
    if list_vcns_response.data[vcn].cidr_block == vcn_cidr_range[0]:
        vcn_id = list_vcns_response.data[vcn].id
        vcn_name = list_vcns_response.data[vcn].display_name

Here, we get the VCN name and id by filtering through the list of VCNs with the CIDR range declared earlier. You can also filter using the VCN name just like we did for the compartment, but i would be doing this way.

Now we have the compartment id and the VCN id, we use those details to get the VCNs security list. You would also need to specify the name of the security list since it’s possible to have multiple security lists in a VCN.

Go ahead and update your script with the following code to get your security list details:

list_security_lists_response = core_client.list_security_lists(
    compartment_id=network_compartment_id,
    vcn_id=vcn_id,
    display_name=sl_name,
 )

vcn_sl_id = list_security_lists_response.data[0].id

get_security_list_response = core_client.get_security_list(
    security_list_id=vcn_sl_id)

# get the current security list data
current_vcn_sl=json.loads(str(get_security_list_response.data.ingress_security_rules))

We simply get the first security list that matches the display name we specified to list_security_lists. Using the id we get the ingress rules for the security list and store it as a json object in current_vcn_sl. The reason we’re getting the existing rules is so that we are able to include the existing rules along with the updates and not replace them.

💡 Remember, the idea is to update the ingress security rules

It would also be nice to keep track of what the security rules are before making the update, the last thing you want to do is mess up the entire control plane without being able to revert the changes.

The following code would write the security rules to a file named vcn_ingress_list.json file. Again, feel free to change the file name to whatever suits your use case. The file would be available on the same directory as your script file as soon as you run the script.

with open('vcn_ingress_list.json', "w") as f:
   json.dump(current_vcn_sl, f, ensure_ascii=False, indent=4)

Create the security list rules

We have declared all the data we need and we’re ready to start effecting the updates.

The first thing you want to do is to declare the ingress security rules you want to update as a python object. i only have two rules to update to the VCN’s ingress security rules, these would tcp and imp connections on the VCN from the subnet.

Update your file with the following code, replacing the rules with yours. You can create another file to hold the rules if you have a lot more.

sl_update_rules=[{
        'description': 'lcm_scan subnet cidr',
        'source': vcn_subnet_cidr_range,
        'source_type': 'CIDR_BLOCK',
        'icmp_options': None,
                'tcp_options': None,
        'is_stateless': False,
        'protocol': '6',
        'udp_options': None
      },{
        'description': 'lcm_scan subnet cidr',
        'source': vcn_subnet_cidr_range,
        'source_type': 'CIDR_BLOCK',
        'icmp_options': None,
                'tcp_options': None,
        'is_stateless': False,
        'protocol': '1',
        'udp_options': None
      }
]

sl_update_rules = current_vcn_sl_rules + sl_update_rules

Notice how we concatenate the current security rules with the update, that way we include the existing rules with the rules to deploy.

We also write out the new security list rules in another file named sl_update_rules.json:

# write out the new security list data
with open('sl_update_rules.json', "w") as f:
   json.dump(sl_update_rules, f, ensure_ascii=False, indent=4)

The API to update the security list doesn’t expect a JSON object, let’s create a function to update the rules into something the API expects using IngressSecurityRule, this function would be called for every one of the rules.

Add the makeIngressRules function below to your file:

def makeIngressRules(t):
    oci.core.models.IngressSecurityRule(
        source=t['source'],
        protocol=t['protocol'],
        source_type=t['source_type'],
        is_stateless=False,
        description=t['description'])

Update Security Rules

Finally, we’re ready to deploy the new rules live. Update your code to reflect the method to make the call to update the security list:

print("Updating "+vcn_sl_id)
core_client.update_security_list(
    security_list_id=vcn_sl_id,
    update_security_list_details=oci.core.models.UpdateSecurityListDetails(
        display_name=sl_display_name,
        ingress_security_rules=[makeIngressRules(t) for t in sl_update_rules]
))

Basically, this says; hey here’s my security list id, i would like to update it’s ingress security rules to now reflect the following rules. This would then convert the JSON security rules update the ingress security rules on the control plane.

Go ahead and run the scripts with the following command:

python3 oci-ingress-sl-update.py

Don’t forget to replace oci-ingress-sl-update.py with your filename including the file path. If everything goes well, the script should now update your ingress security rules and produce two JSON files, one containing the previous rules and the other one containing both the previous rules and the updates.

💡 Remember, the script for this article is available here

Conclusion

Congratulations, you have learnt how to programatically update your ingress security rules without replacing the entire control plane. You can also update the egress rules using the same approach.

Thanks for reading, don’t forget to like and share this article. Follow me on twitter and everywhere else @robogeek95.

A visual giude on Quicksort algorithm

Azeez Lukman — Mon, 31 Jan 2022 07:52:49 +0000

An algorithm is a finite series of steps that a computer would take to implement a task in an efficient manner.

Sorting algorithms are generally used to rearrange elements of an array to follow a certain order. Popular examples of sorting algorithms include Heapsort, Merge sort and Quicksort.

In this article, you would learn the techniques of sorting using a Quicksort algorithm as well as how to implement one in the JavaScript programming language. You would also learn about the algorithmic complexity of this sorting algorithm.

What is Quicksort

Quicksort is a divide-and-conquer algorithm. It works by selecting a 'pivot' element from the array. Every other element of the array is swapped such that elements greater than the pivot elements are on one side and the lesser elements are on the other side of the array and the array ends up being partitioned into sub-arrays. The sub-arrays are then sorted recursively until the array is sorted. This is done in place, so a little additional memory space is required to implement the sorting.

Steps in implementing Quicksort

Just like every other algorithm, the Quicksort algorithm has a few techniques that bound its implementation. Let’s dig into the steps involved to correctly implement a Quicksort algorithm.

Selecting a pivot element

The first step is to pick an element from the array as the pivot element. There are different methods applied to select the pivot element but it is worth mentioning that the method used in sleeting the pivot element might affect the resulting speed of your implementation, you would talk more about this in the complexity analysis section of this article.

The pivot element is selected by either of the following technique:

The first element of the array
The last element of the array
A random element selected from the array

This simply means the pivot element could be any element from the array but it’s a better approach to stick to one method for selecting the for this process.

Compare and swap elements with respect to the pivot element

The selected pivot element is then used as a reference to swapping each element of the array in such a way that all elements of the array that are lesser than the pivot element are placed on the left-hand side and all other items greater than the pivot are placed on the right-hand side. This process generates two partitions, a partition with elements that are greater than the pivot element and the other partition that holds elements lesser than the partition. Finally, the pivot element is placed in between the two partitions generated which can be said to now be in its correct index.

Repeat 🔁

The partitioning process is repeated for every resulting partition of the array until every single item is ultimately in the right place, the array is then said to be fully sorted.

An example of implementing Quicksort

Let’s see a quick example of the process of implementing a Quicksort algorithm by sorting an unordered array. We would look at how to implement this algorithm using a programming language later in this article.

There would have two pointers; the right pointer and the left pointer pointing to both extreme indexes of the array. By comparing the elements at both pointers to that of the pivot element, we decide which element to swap and to which side it should be swiped.

Next, the elements are swapped so that only the items that are lesser than the pivot element are on the left side and the larger elements are swapped to the left. We determine elements to swap by advancing the pointers and comparing each element till we find an element that doesn't match the requirement for the side it resides. The comparison starts from the left pointer, after we have an element that’s lesser than the pivot element, we then find the next element that’s larger than the pivot element from the right pointer to swap out with.

The elements of the unsorted array are listed in the image below:

Images and Indicators

In order to make it easier for you to grasp the concepts, this article provides illustrated images in the examples. The several colours used to indicate the different states and interactions of the elements and the pointers during sorting are described below.

Gray - Indicates the element’s default state
Yellow - Indicates a pivot element
Green - Indicates a sorted element

And the following colours are used for the pointers:

Green - Indicates the pointer is active
Pink - Indicates the pointer is inactive

First Iteration

The first step is to select a pivot element. Using the first method for pivot element selection as described above, we select 5 as the pivot element, since it’s the first element of the array.

In this case, the right and left pointers initially point to 5 and 8 respectively. Element 4 is less than the pivot element, we swap it with element 7. The same goes for elements 1 and 6, they are swapped so they are placed on their appropriate sides of the array.

Finally, we move the pivot element 5 to the centre, it is then marked as sorted.

Second Iteration

This first iteration has placed it’s pivot element in its correct position. The iteration also partitioned the array into two sub-arrays each of which also need to be sorted.

We are sorting the left sub-array completely first, then we proceed to the right sub-array. It’s also okay to go the other way round.

Following the same method as in the first iteration, we select the pivot element using the first method so we pick 1.

Scanning through the left pointer, there are no elements greater than the pivot element found so this sub-array is sorted and there would be no swaps on this iteration.

Third Iteration

The second index has placed element 1 in its correct position, we also indicate that with a green background in the image below. It generates another sub-array with two elements.

We select the element on the first index of the third sub-array 2 as the pivot element.

We place the pointers and compare the two elements, all elements of this sub-array also happen to be in place and there are no swaps for this iteration.

We mark the pivot element sorted. Since there is only one item left unmarked from this sub-array, there is no use partitioning it into another subarray since there would be no comparisons or swaps. Instead, the element is also marked as sorted.

Fourth Iteration

The Third iteration has fully sorted every item on the left side. We proceed to the left subarray next.

Select 6 as the pivot element. Comparing the other elements, element 7 is greater than the pivot element 6 so it would not be swapped, the same is true for 9. There would be no swaps in the partition.

We then mark the pivot element for this sub-array 6 as sorted. This is indicated with the green background in the image below.

Fifth Iteration

We’re almost done sorting with just one more partition to go. The last partition contains two elements.

All of the steps are still required, we select the pivot element by picking the first element of the current sub-array 9. 7 is less than 9. We swap both elements places them in order.

We mark the pivot element 9 as sorted, we also mark the last element as sorted since it cannot be broken into a sub-array.

We now have a fully sorted array using the Quicksort algorithm.

Implementing a Quicksort algorithm in javascript

Let’s now take a look at the programmatic method of implementing the Quicksort algorithm in the JavaScript programming language.

open your code editor and create a new file called quicksort.js. This file would hold all the code for the Quicksort algorithm we are implementing.

First, create a function to perform the swap

Code to swap

First paste the code below in your file to create the function swap() :



function swap(unsortedArray, leftIndex, rightIndex){
    const temp = unsortedArray[leftIndex];
    unsortedArray[leftIndex] = unsortedArray[rightIndex];
    unsortedArray[rightIndex] = temp;
}

swap() as its name implies, basically performs a swap of the indexes of the provided elements and replaces the new value into the original array.

Code to perform the partition

Paste the following code for function partition in your quicksort.js file:



const unsortedArray = [];

function partition(unsortedArray, left, right) {
    let pivot   = unsortedArray[Math.floor((right + left) / 2)], //middle element
        i       = left, //left pointer
        j       = right; //right pointer
    while (i <= j) {
        while (unsortedArray[i] < pivot) {
            i++;
        }
        while (unsortedArray[j] > pivot) {
            j--;
        }
        if (i <= j) {
            swap(unsortedArray, i, j); //sawpping two elements
            i++;
            j--;
        }
    }
    return i;
}

The partition function accepts the unsorted array, starting index and ending index as its arguments. It uses the middle element as the pivot element. The function then swaps the elements of the unsorted array such that all the elements to the left of the pivot are lesser than the pivot and all the elements to the right of the pivot are greater than it. It returns the index of the pivot element in the sorted array after positioning it in its correct position.

unsortedArray would be replaced with the actual values of the array that needs to be sorted.

Implement the sorting

This is the main function that implements quick sort. Update your file to reflect the function:



function quickSort(unsortedArray, left, right) {
    let index;
    if (unsortedArray.length > 1) {
        index = partition(unsortedArray, left, right); //index returned from partition
        if (left < index - 1) { //more elements on the left side of the pivot
            quickSort(unsortedArray, left, index - 1);
        }
        if (index < right) { //more elements on the right side of the pivot
            quickSort(unsortedArray, index, right);
        }
    }
    return unsortedArray;
}

The quicksort function executes recursively, invoking itself several times until all of the unsorted arrays are completely sorted. It also expects the unsorted array, a starting index and an ending index of the array to be sorted.

It uses the index returned from the function partition to divide the array and perform the Quicksort.

💡 Quicksort is performed on the same array and no new arrays are created in the process.

Time to test your algorithm

unsortedArray currently holds an empty array, update the unsortedArray constant in quicksort.js to now hold the actual values from the example earlier:



const unsortedArray = [5,2,7,6,1,9,4];

Finally, add the code to invoke the function and log the output to the console:



const sortedArray = quickSort(unsortedArray, 0, unsortedArray.length - 1);
console.log(sortedArray);

Now, open up your terminal, locate the file and run the file using node. You can run it by running the command below:



node quicksort.js

Complexity Analysis of Quicksort

The best-case complexity of the quick sort algorithm is O(n logn) also, the average case time complexity is O(*n logn)*.

The worst-case time complexity of Quick Sort is O(n2). The worst-case is usually avoided by using a randomized method for selecting the pivot element.

For the space complexity, Quicksort in-place algorithm, meaning it doesn't take any extra memory space for its implementation because it performs all of the operations on the same array. It takes O(log n) stack space.

Conclusion

This article has walked you through the techniques behind the Quicksort algorithm, an illustrated example as well as how to implement one in javascript.

Thank you for reading and I hope you learned a lot. My name is Azeez Lukman, please stay tuned for more if you would like to keep adding to your algorithmic toolbox 🧰

References

https://www.geeksforgeeks.org/sorting-algorithms/

https://www.interviewbit.com/tutorial/quicksort-algorithm/

https://www.guru99.com/quicksort-in-javascript.html

https://en.wikipedia.org/wiki/Quicksort

An ultimte guide to building an order tracking dashboard with Appsmith

Azeez Lukman — Sat, 29 Jan 2022 13:09:38 +0000

Appsmith is an open-source framework for developing low code dashboards, workflows, pages, and CRUD apps very quickly. Appsmith allows you to build tools with little or no code. Appsmith has more flexibility than off-the-shelf options and it’s fast to build and easy to maintain.

In this tutorial, you are going to learn how to use Appsmith to build an order tracking dashboard for an e-commerce store. The store’s order data is stored on a Firestore database, you would also see how to quickly integrate Firestore into your Appsmith dashboard and run queries from Appsmith.

Before we get started let’s go over some requirements needed to complete this tutorial:

A Firebase account with Firestore setup
Experience with Javascript is a plus but not required
Finally, you can view a demo of what you would be building here

Alright, that’s all you need. So let’s jump right in.

Setup

Before you can start using Appsmith, you need to create an account, so head over to Appsmth to create an account if you have not already done that. Alternatively, you can also deploy Appsmith on your local machine or private server instance since it’s open-source software but that’s beyond the scope of this tutorial.

You should be redirected to the dashboard after signing up. The next step is to create a new application on Appsmith.

Click the Create New button to create a new application and name it Acme Store. This takes you to an empty workspace with a sidebar. On the sidebar, you should see a navigation menu with a page. Pages are synonymous with web pages. Each page on Appsmith includes the widget, the datasources and the JS objects. in the following content, we would build the app by digging deeper into each of these items.

The Datasource

Appsmith allows you to easily integrate APIs or databases such as MongoDB, Firestore and more out of the box. You can run queries to read and write data directly from the Appsmith editor.

This tutorial integrates with a Firestore database.

💡 It is assumed that you already have an existing Firestore database set up, you can follow this guide to quickly get one running.

Configure the Datasource

On the Appsmith dashboard, click on + next to Datasources and find Firebase on the list.

The Firestore plugin requires three pieces of information to establish a connection; the database URL, the project id and the service account credentials. Follow the below instructions to set it up:

Project ID: Navigate to "Project Settings" in your Firebase console and ensure you're in the "General" tab. You'll find a "Project ID" here, copy this value into the "Project ID" field in the Firestore configuration.
Database URL: The database URL need to be formed using the Firebase project ID you found above. Enter the database URL in the following manner your-project-id.firebaseio.com
Fetching Service Account Credentials: Navigate to the “service accounts” tab in your firebase console and find a "Generate new private key" button below the code snippet. Click the button to generate a new private key to access this Firestore instance and open the JSON file downloaded. Paste the content of this file into the "Service Account Credentials" field.

After filling up the three fields as described above, click on the "Test" button to verify the configuration and click "Save".

Data Schema

Remember we’re building an application to track a record of orders. It would be nice to populate the database with some documents at this point so we can focus on integrating these records into the app moving on.

Collections hold documents. Since Firestore is a no-SQL database, it doesn't expect your documents to be fixed to a certain schema. You can structure every object of your documents as you deem fit but you still try as much as possible to keep your data uniform.

Create a new collection and call it orders to record the orders. Then you need to place some documents in the collection. Here’s the orders schema this tutorial is working with:

date: A timestamp value for when the order was placed.
shipment: A map/object that holds the shipping details for the order. it contains the address, email and phone of string types.
**items:** An array of objects/maps of the items bought. Each item contains its price, quantity and productName
**customer**: info about the customer that placed the order. Contains the name and email of the customer
status: a string label to track the status of the order

Go ahead and populate the orders collection with some document records following the above convention.

you should have something similar to this:

You also need a collection to hold the list of status you track orders with. It follows the same process as for the orders.

Create a new collection and call it labels add a document to the collection, name the document status_labels and populate it with the following fields:

**values:** an array of objects/map containing label and value properties which are both string types.

you should also have something similar to this:

Execute Queries

Now that you have fully integrated the database, you are ready to start adding some actual integrations. Appsmith allows you to create queries to fetch and update data from your datasource using the Appsmith query editor.

You can create and manage queries from the sidebar on the Appsmith dashboard. Click on the datasource you integrated earlier, on the next page, click on the “new query” button to create a new query.

Get orders

This query would fetch all the records from the orders collection.

Create a new query and name it get_orders. Click the commands dropdown and select ‘Get Documents in Collection’ since we are getting the record of documents in the orders collection. The “Collection/Document Path” should be orders. Leave the remaining fields as their default.

Run the query and you should have something similar to this in the response field:

{
  "date": {
    "seconds": 1642460400,
    "nanos": 0
  },
  "shipment": {
    "address": "lagos, nigeria",
    "phone": "+2346994948383",
    "email": "taylorswift@gmail.com"
  },
  "_ref": {
    "id": "1g3VP3gNePsYGZWNaC4k",
    "path": "orders/1g3VP3gNePsYGZWNaC4k"
  },
  "items": [
    {
      "quantity": "4",
      "price": "25000",
      "productName": "Stallion Rice"
    }
  ],
  "status": "draft",
  "customer": {
    "name": "Taylor Swift",
    "email": "taylorswift@gmail.com"
  }
}

Get status labels

This query returns the record of status labels in the database. It follows a similar process as getting all orders except this reads a document and not a collection.

Create a new query, set commands to “Get Single Document” and the path as “labels/status”.

Run the query and you should get a response similar to this:

{
  "values": [
    {
      "label": "pending",
      "value": "pending"
    },
    {
      "label": "draft",
      "value": "draft"
    }
  ],
  "_ref": {
    "id": "status",
    "path": "labels/status"
  }
}

Update order

The final query allows us to update the order directly from the dashboard, to keep this tutorial simple we’re only focusing on updating the order’s status.

Create a new query and set the command to “Update Document”. Next is the “Collection/Document Path”, set it to orders/{{orders.selectedRow._ref["id"]}}. Notice how the query extracts the current row data id from the table widget, this way, you’re able to update the selected document. But we don’t have any widgets yet, so let’s go ahead and build that next.

Run the query and firestore should return a timestamp value denoting the last update time:

{
  "lastUpdateTime": {
    "seconds": 1642981099,
    "nanos": 88277000
  }
}

The Widgets

Widgets act as the building blocks for Appsmith UI. They can be dragged from the widget pane, positioned on the canvas, and resized to fit the data they need to display. They also come with properties that can be visually edited to set their data, change their styles, and trigger actions from them.

Widget properties can be edited via the property pane which is opened using the top-right icon (Edit Widget Properties). Data from a Query can be set in a widget property by referencing the name (unique identifier) of the Query.

We’ve been going over some boring stuff all day, it’s about time we did something fun. Let’s start building out the UI for the app.

Heading

It would be nice to place a heading on the top of the app. On the sidebar toggle the widgets panel and drag the text widget onto the canvas. Then from the properties panel, change the text to “Acme Store Dashboard”. Next drag in a divider widget right under the heading.

Every other widget would be placed under the header.

Table

This would be used to display the orders in tabular form. On the sidebar toggle the widgets panel and drag the table widget onto the canvas. Then from the properties panel, delete the default columns and the default table data. Then create the date, order Id, customer and status columns.

Details panel

On the selection of each table row, we want to show more data about the order on that row in the details panel. Drag in a “container” widget and logically divide the screen portion under the heading such that the table takes up 60% and the details panel takes up 40% of the screen width.

Add another container widget in the details panel and place a text widget in it, rename the text to “Track Order”. Next, drag in a text widget for order Id and place another text widget next to it to hold the value, do the same for the customer, address, email and phone.

Place a divider before the address, email and phone fields and place a heading “Shipment” using the text widget above them.

Add another divider, an heading “Items” and drag in another table widget into the panel.

💡 You might need to resize the table in an open area of the canvas so that it fits the details panel before you can place it there. learn more about resizing a widget here

Finally, add a button to update the database with the edits. Update the button text to “Update” from the properties panel.

You’re almost done with the UI, but first, take some time to update the UI to match the image below:

Integrate Datasource

It's time to bring your implementation to life by making everything you have been implementing communicate with each other.

Display Data in Tables

The table widget can automatically loop through an array of items provided as the table data.

Set the table data from the properties pane by referencing the get all orders query.

{{get_orders.data}}

Let’s take a moment to talk about the brackets. Appsmith supports writing single line javascript from any field between the template syntax {{}}, the content within it can then point to a variable or be an evaluation.

Appsmith is reactive, that way, the get_orders value is stored immediately you tested it earlier so you don’t have to run the query for every field you need it. In the example above, the syntax refers to the data returned from the get_orders query.

💡 widgets are automatically updated whenever the data in the Query changes

After adding the table data, the columns and values are automatically populated for you with the top-level properties from the response data, but we don’t want to show that much information on the table. On the right of each icon use the eye icon to hide each column except the date customer and status.

You also need to add another column for the order id by clicking the “ADD A NEW COLUMN” button and naming it to order Id. Configure the value by clicking the “Setting” icon on the column and place the following syntax in the computed value field:

#{{currentRow._ref["id"]}}

The {{currentRow._ref["id"]}} is used to select the id from the table’s current selected row. We then place a # character is placed before each value.

Display values in the detail panel

The values displayed here would be inferred from the selected row, so it’s dynamic and shows detail for each order.

For the Order id, replace its value field with the following template syntax:

#{{orders.selectedRow._ref["id"]}}

Replace the customer value field with the following template syntax:

{{orders.selectedRow.customer}}

Replace the label text for the status with the following template syntax:

{{orders.selectedRow.status}}

For the options, you need to get the value from the query to get status labels, replace the options field of the status dropdown with the syntax below:

{{get_status_labels.data.values}}

Next is the update button. We want to keep track of the orders by updating their status using the labels you have just configured. But when you change the label from the dashboard, it doesn't reflect on the database yet.

Button widgets in Appsmith have an event handler property which can be found on the properties pane. The idea is to run the update record query each time the button is clicked, persisting the updated status in the Firestore. Select execute a query as the event select the update_order query you created earlier. Now change the label and click the update button, notice the change from your database.

Keep your data up to date

You’re almost done except that you don’t want your table to have redundant fields. Appsmith allows you to also specify callback actions to handle both success and error events. For now, we are concerned with the success callback, we can use this to fetch the current data from Firestore and Appsmith would automatically propagate the data into the widgets.

Click on the update button, then select the onSuccess callback from the onClick event handler, select execute a query as the action the select the get_orders query.

You should now have a similar dashboard to the one in the image below:

Publish your Application

The last step is to make it available for use, there are a couple of ways to do this; you can either add people to the organization where the project sits or make it generally available to the public.

Select the share button from the far right of the navbar and you should see a toggle button labelled Make the application public in the modal that pops up, toggle it and copy the generated link to start sharing your progress.

💡 Remember, the demo for this tutorial is available here

Conclusion

Congratulations! you have seen how easy it is to build a custom dashboard with little code using a WYSWYG editor. You can fully utilize this approach to build a fully functional dashboard for your enterprise and small scale e-commerce applications.

Thanks for reading, I’m Azeez Lukman and here’s a developer’s journey to building something awesome every day. Please let's meet on Twitter, LinkedIn, GitHub and anywhere else @robogeeek95.

How to access string values in Go

Azeez Lukman — Tue, 26 Oct 2021 22:42:22 +0000

GO represents strings as byte slices using under the hood. This means you can access different indexes of a string like you would for a slice variable.

A byte slice is a slice whose underlying type is a slice. Byte slices are more like lists of bytes that represent UTF-8 encodings of Unicode code points.

Strings are immutable, unicode compliant and are UTF-8 encoded

Accessing the individual bytes of a string

I mentioned above that a string is a slice of bytes. We can access every individual byte in a string

package main


import (
    "fmt"
)


func printBytes(s string) {


    fmt.Printf("Bytes: ")


    for i := 0; i < len(s); i++ {


        fmt.Printf("%x ", s[i])


    }


}


func main() {


    string := "Hello String"


    printBytes(string)


}

outputs:

String: Hello World  
Bytes: 48 65 6c 6c 6f 20 57 6f 72 6c 64

We print the bytes in the string 'Hello String' by looping through the string using len() method. the len() method returns the number of bytes in the string, we then use the returned number to loop through the string and access the bytes at each index. The bytes are printed in hexadecimal formats using %x format.

Accessing individual characters of a string

Let's modify the above program a little bit to print the characters of the string.

package main


import (
    "fmt"
)


func printBytes(s string) {


    fmt.Printf("Bytes: ")


    for i := 0; i < len(s); i++ {


        fmt.Printf("%x ", s[i])


    }


}


func printChars(s string) {


    fmt.Printf("Characters: ")


    for i := 0; i < len(s); i++ {


        fmt.Printf("%c ", s[i])


    }


}


func main() {


    name := "Hello World"


    fmt.Printf("String: %s\n", name)


    printChars(name)


    fmt.Printf("\n")


    printBytes(name)

}

String: Hello World
Characters: H e l l o   W o r l d
Bytes: 48 65 6c 6c 6f 20 57 6f 72 6c 64

The logic remains the same as above, but this time, you would notice the use of %c format specifier, which is used to to print the characters of the string in the method.

In UTF-8 encoding a code point can occupy more than 1 byte, so this method of accessing the characters is not well suited since we are only assuming that each code point occupies one byte. A better approach is to use runes

package main


import (
    "fmt"
)


func printBytes(s string) {


    fmt.Printf("Bytes: ")


    for i := 0; i < len(s); i++ {


        fmt.Printf("%x ", s[i])


    }


}


func printChars(s string) {


    fmt.Printf("Characters: ")


    for i := 0; i < len(s); i++ {


        fmt.Printf("%c ", s[i])


    }


}


func main() {


    testSring := "Señor"


    fmt.Printf("String: %s\n", testSring)


    printChars(testSring)


    fmt.Printf("\n")


    printBytes(testSring)


}

This outputs:

String: Señor
Characters: S e Ã ± o r
Bytes: 53 65 c3 b1 6f 72

Notice that the program breaks, the characters returns Ã ± instead for ñ. The reason is that the Unicode code point of ñ is U+00F1 and its UTF-8 encoding occupies 2 bytes c3 and b1. We are trying to print characters assuming that each code point will be one byte long which is wrong.

Rune

A rune is simply a character. It is a builtin data type in Go. Rune literals are 32-bit integer values that represents a Unicode Codepoint.

package main


import (
    "fmt"
)


func printBytes(s string) {


    fmt.Printf("Bytes: ")


    for i := 0; i < len(s); i++ {


        fmt.Printf("%x ", s[i])


    }


}


func printChars(s string) {


    fmt.Printf("Characters: ")


    runes := []rune(s)


    for i := 0; i < len(runes); i++ {


        fmt.Printf("%c ", runes[i])


    }


}


func main() {


    testString := "Señor"


    fmt.Printf("String: %s\n", testString)


    printChars(testString)


    fmt.Printf("\n")


    printBytes(testString)


}

String: Señor
Characters: S e ñ o r
Bytes: 53 65 c3 b1 6f 72

In this example, the string is converted to a slice of runes using []rune. We then loop over it and display the characters. This works because a rune can represent any number of bytes the code point has.

Accessing specific characters in a string

now we have seen how to access all the characters of a string. Let's see how we can access the individual indexes of the string. Remember that a string in Go is a slice of bytes so we can easily access the character at a specific index like we would for a slice, or an array without needing to loop through the string or convert it to a rune.

package main


import (
    "fmt"
)


func main() {
    testString := "Hello String"


    fmt.Println(testString[2])


    fmt.Println(testString[1])


    fmt.Println(testString[4])
}

108
101
111

This returns the unicode code points for the specified indexes

Trying to access an index that is larger than your string's length throws an index out of range error, since the index specified exceeds the available range in your declared string

That was swift, all we did was declare the string and specify the index we would like to access. This is actually not our intended purpose, we still need to be able to access the actual character and not it's unicode value.

To access the character, we convert the Unicode code point using the builtin string method string()

package main


import (
    "fmt"
)


func main() {
    testString := "Hello String"


    fmt.Println(string(testString[2]))


    fmt.Println(string(testString[1]))


    fmt.Println(string(testString[4]))
}

l
e
o

A simple program to check if a string begins with a lower case letter or an upper case letter

Using our knowledge on accessing string values, we are going to write a small Go program that reports if a string passed in begins with a lower-case or upper-case letter

Declare package and and a function that checks if the whether the string has a lower-case letter at the begining.

There is no perform any checks if the parameter is an empty string, so the function checks for that first and returns false is it's an empty string

Next is the actual work, Go comparisons can automatically compare values within a range, in this case, we are checking if the first slice index of the string parameter exists within the range of lower-case letters.

package main


// startsWithLowerCase reports whether the string has a lower-case letter at the beginning.
func startsWithLowerCase(str string) bool {


    if len(str) == 0 {


        return false


    }


    c := str[0]


    return 'a' <= c && c <= 'z'


}

startsWithUpperCase function also compares the first letter of the string parameter across a range, but this time, it compares across a range of capital letters. add this function to your program

// startsWithUpperCase reports whether the string has an upper-case letter at the beginning.
func startsWithUpperCase(str string) bool {


    if len(str) == 0 {


        return false


    }


    c := str[0]


    return 'A' <= c && c <= 'Z'


}

It's time to wrap up and test out program, declare the main function. Inside the main function, you would declare your test string and call the functions passing the testString as parameter. We want to properly report our results so we use fmt.Printf to format our report and print to the console

func main() {

    testString := "Hello String"


    fmt.Printf("'%s' begins with upper-case letter? %t \n",


        testString,


        startsWithUpperCase(testString))


    fmt.Printf("'%s' begins with lower-case letter? %t \n",


        testString,


        startsWithLowerCase(testString))


}

'Hello String' begins with upper-case letter? true 
'Hello String' begins with lower-case letter? false

Cooool right? You have just created an enterprise grade program. Yes, startsWithLowerCase is the same logic used to in Go time package for the purpose of preventing matching strings like "Month" when looking for "Mon"

Conclusion

With this deep dive on accessing held in Go strings, you're ready to take over the world. But before that, There’s only one way to learn to develop Go programs: write a lot of code. Keep coding and taking over the world is only a matter of time.

Thank you for reading, I'm Azeez Lukman and here's a developer's journey building something awesome every day. Please let's meet on Twitter, LinkedIn and GitHub and anywhere else @robogeeek95

Slicing Slices in GO

Azeez Lukman — Fri, 08 Oct 2021 04:28:47 +0000

Slices wrap arrays in Go and provide a more general, powerful, and convenient interface to data sequences. In this article, you will cover slice basics such as creating, initializing, and iteration. You will also learn how to grow a slice, work with subsets of slices, and slice tricks.

Slices contain a list of elements of a particular type, but unlike arrays, there are methods available to add or remove elements from the slice. This is because as the name implies, slices don't hold any actual data themselves. A slice provides a view into the elements of the underlying array and provides extra methods for interacting with the data.

slices are much more common than arrays

Slice Type

The type []T denotes a slice of type T for example:

[]string

This says hey, this is a slice of strings or this slice contains string values. The values would be comma separated. This is similar to how you would declare the type for an array variable, except the length is omitted here.

For the most part, functions for working with slices is identical to functions for working with arrays. These functions includes ones for accessing elements, using zero values, passing slices to the len function, and for...range loops.

Slice Values

Like i mentioned above, slices themselves don't store any values at all. You may be wondering how it then holds data. Slices are designed to work as a convenience wrapper for arrays, allowing you to easily perform functions that would have been difficult to manually perform directly on an array.

I like to think of a slice as a peek into some section of the underlying array. That peek is where you interact with the array from and the data you peek into is what you get to operate on.

Now, you should note that modifying the peek elements makes the actual changes into the underlying array directly. This could be a blessing and a curse at the same time since the other slices with the same underlying array would see the changes.

Declaring slices

To declare the type for a variable that holds a slice, you use an empty pair of square brackets, followed by the type of elements the slice will hold.

var intSlice []int

This only declares a slice type that holds integer values as intSlice. Declaring a slice variable doesn’t automatically create a slice. For that, you call the built-in make function. You pass make the type of the slice you want to create and the length of slice it should create and assign it to your slice variable.

var intSlice []int
intSlice = make([]int, 5)

After declaring the slice variable intSlice . The next line creates the actual slice using the data type and the length. Notice the data type passed into make function is the same as that of the slice variable.

So the make method created the slice, but hold, what about the values? By default the make method would fill the slice with the underlying data type's zero value of the slice we created for the length we passed in. In our example the slice intSlice now contains 5 values as 0

You can now use the builtin slice methods, let's run a few on the newly created slice.

len(intSlice) //4
intSlice[3] // 0

Next, we want to fill those spots up with our actual data, you would assign its elements using the same syntax you used for accessing it.

intSlice[0] = 10 
intSlice[1] = 8
intSlice[2] = 9 
intSlice[3] = 12

fmt.Println(inSlice[0])  // 10
fmt.Println(inSlice[1])  // 8
fmt.Println(inSlice)  // 10, 8, 9, 12, 0, 0

Slice literal

When you know the values that goes into your slice before declaring, you can directly pass them in using slice literal. With slice literal, you do not need to use the make function, your slice gets declared and assigned its values in one step

A slice literal looks just like an array literal, except the length is omitted:

stringSlice := []string{"hello","world"}

You can even use a multi line slice literal to make your more organized

stringSlice := []string{
         "hello",
         "world",
             "once",
         "more",
}

When you use a slice literal, an array is created and the slice is created to reference it, the slice gets a peek of the whole value inserted by the slice literal. The only way to access the array created is through the slice.

Slicing arrays

Using the slice operator, you can create a slice directly from the array without using the make function, the operator is similar to the one used for accessing and assigning values in slices, but it has two indexes instead, the first denotes the start of the slice and the other denotes the end, excluding that index. The elements peeked into would span from the first index up to but not including the second index.

arr = [4]{1, 2, 4, 8}
arrSlice = arr[1:3]
fmt.Println(arrSlice) // 2,4

omitting the last index causes the slice to include vlalues through to the last index of the array

arr = [4]{1, 2, 4, 8}
arrSlice = arr[1:]
fmt.Println(arrSlice) // 2,4,

The os.Args package variable contains a slice of strings with the command-line arguments the current program was run with

Add items onto a slice

Go provides a built-in append function that takes a slice, and one or more values you want to append to the end of that slice and returns a new, larger slice with all the same elements as the original slice, with the new element(s) added onto the end.

slice = []int{10, 5, 8}
append(slice, 20)
append(slice, 20)

A slice’s underlying array can’t grow in size. If there isn’t room in the array to add elements, A new, larger array would be created and all its elements will be copied into it, then the slice will be updated to refer to this new array. But since all this happens behind the scenes in the append function, there’s no easy way to tell whether the slice returned from append has the same underlying array as the slice you passed in, or a different underlying array. If you keep both slices, this can lead to some unpredictable behavior such as inconsistencies.

To avoid this, Always make sure to assign the return value of append function back to the same slice variable we passed to append

slice = []int{10, 5, 8}
slice = append(slice, 20)
slice = append(slice, 20)

Summary

Slices are a convenience wrapper around arrays, this is why they are very similar to arrays even with the methods used in manipulating them. Slices don't store value but operates on the underlying array. They can only hold one data type just like arrays too.

Other things to explore includes converting strings to slices and vice versa, you should also look into how slices are used in place of variadic arguments in variadic functions.

Thank you for reading, i'm Azeez Lukman, should you need any help or questions please feel free to reach out @robogeek95 everywhere

Ansible File Structure, destructured

Azeez Lukman — Sun, 03 Oct 2021 18:22:50 +0000

The big question, Is there a recommended way to structure Ansible projects?

Well, Ansible doesn’t have enforce patterns on how you put files into folders. But there are a few common approaches popular in the ecosystem you may want to consider. This guides walks you through.

Keeping it simple 😉

A pattern recommended though is to keep things as simple as you can, do things simply. Use advanced features only when necessary, if it begins to feel complex for the task, then it's probably complex. The documentation asks you to take the time to look for a simpler solution.

Look no further, one solution is to start with the minimum pattern that is easy to build upon when you need to, and that is what this article is all about.

A typical Ansible file-structure

Keep your files organized in your working Ansible project directory like this:

...
ansible
    ├── ansible.cfg
    ├── first_playbook.yml
    ├── inventory
    ├── group_vars
    │   ├── vyos.yml
    │   └── eos.yml
    ├── roles
    │   ├── static_route
    │   └── system
    ├── second_playbook.yml
    └── third_playbook.yml

With this you can expand your inventory and create and run more network playbooks, let's destructure all these and what goes into them.

Ansible.cfg

This is the ansible config file, commonly referred to as the brain and the heart of Ansible. It is file that defines Ansible config, and is responsible for how all processes in Ansible functions.

By default Ansible reads its configuration file in /etc/ansible/ansible.cfg, however this behavior can be altered. The recommended practice is either to have an ansible.cfg in your current project working directory or to set it as an environment variable.

Playbooks

Ansible Playbooks are way to automate deployment tasks by using a repeatable, re-usable, simple configuration management and multi-platform deployment system.

Each playbook contains one or more plays and each play defines the managed nodes to target and runs one or tasks on it. Each task calls an Ansible module. When you run a playbook, the plays are executed sequentially, play by play and each play also executes it's tasks sequentially from top to bottom.

Ansible lets you define more than one playbook with each paybook focused on one function, you can have one playbook to create your deployment and another one to destroy it. Our example structure show first_playbook.yml, second_playbook.yml and third_playbook.yml.

Inventory

The Ansible inventory file defines the groups of hosts, commands, modules, and tasks a playbook can operate on. It functions as a list of managed nodes. An inventory can specify information like IP address for each managed node, you can designate different servers into groups like webservers, or databases

Group_vars

Ansible uses a combination of a hosts file and a group_vars directory to pull variables per host group and run Ansible plays/tasks against hosts. group_vars/all is used to set variables that will be used for every host that Ansible is ran against.

The group_vars in Ansible are a convenient way to apply variables to multiple hosts at once, they also provide a pattern for the reusability of group variables, saving you efforts to sort the hosts and also enhance the flexibility in your code.

Roles

Roles allows to easily reuse your grouped content, One way to use roles is with the roles option for a play, then ansible follow your file structure and the order listed below to execute the role and replace the play task with its values.

Any pre_tasks defined in the play.
Any handlers triggered by pre_tasks.
Each role listed in roles:, in the order listed. Any role dependencies defined in the role’s meta/main.yml run first, subject to tag filtering and conditionals.
Any tasks defined in the play.
Any handlers triggered by the roles or tasks.
Any post_tasks defined in the play.
Any handlers triggered by post_tasks.

Time for an example

In addition to the above recommendations, i also recommend that you practalize your knowledge, Rafael has written an amazing article on how to automate the setup of a Kubernetes cluster on GCP following a similar structure.

This is the format he implemented

...
└── ansible                    # Ansible top-level folder
    ├── ansible.cfg            # Ansible config file
    ├── create-k8s.yml         # Ansible playbook to provision env
    ├── destroy-k8s.yml        # Ansible playbook to destroy env
    ├── inventory             
    │   └── gcp.yml            # Ansible inventory file
    └── roles
        ├── destroy_k8s        # Ansible role to remove k8s cluster        
        │   └── tasks
        │       └── main.yml
        ├── destroy_network    # Ansible role to remove VPC
        │   └── tasks
        │       └── main.yml
        ├── k8s                # Ansible role to create k8s cluster
        │   └── tasks
        │       └── main.yml
        └── network            # Ansible role to create VPC
            └── tasks
                └── main.yml

Conclusion

Ansible makes deploying your infrastructure easier and it's best when used as intended, Don't over complicate it, start simple and with time you would discover what structure works best for your organization.

Thank you for reading, I hope you learned a lot already. I'm Azeez Lukman and here's a developer's journey building something awesome every day. Please let's meet on Twitter, LinkedIn, GitHub and anywhere else @robogeeek95

DEV Community: Azeez Lukman

Leveraging Deployment Inputs in VMware Orchestrator (VRO) for Enhanced Workflow Customization

Understanding Deployment Inputs in VRO

Accessing Deployment Events via vRA Endpoint

Creating a Custom Action to Manipulate Deployment Inputs

Create New Action

Declare Inputs

Write Custom Script

Integrating the Custom Action into Workflows

Conclusion

Comprehensive Guide to Infrastructure Continuous Integration and Continuous Deployment with Terraform and GitLab CI/CD

Getting to Know Terraform

Overview of GitLab CI/CD

Setting Up GitLab for Terraform

Creating CI/CD Pipelines for Terraform

*Create the Pipeline File*

Define Stages and Jobs

Trigger the pipeline run

Review Pipeline Output

Coffee break…

Manual Approvals

How does the manual approval work?

Securely Storing Secrets

Access Project Settings

Configure CI/CD Variables

Add Secret Variables

Protect Secret Variables

Accessing Secret Variables in CI/CD Jobs

Conclusion

A Comprehensive Guide on Migrating OCI Terraform State from Local to Remote Storage

Introduction

Why Migrate Terraform State to an Object Storage?

Prerequisites

Prepare for the Migration Process

Create an Object Storage Bucket

Create a Customer Secret Key

Update Terraform Configuration

Migrate Existing State

Initialize Terraform

Verify and Test

Conclusion

VRA API Authentication: Ensuring Secure Access and Understanding Authentication Methods

Understanding vRealize Automation

Key Concepts of VRA API Authentication

VRA API Authentication Methods

Using API Tokens for Authentication

How to Obtain an API Token for Authentication

Export Variables

Request an API Token

Retrieve the Access Token

Testing Your Access

Conclusion

Rerefences

Back to the blogosphere (Sharing My Journey in the World of Software Engineering)

Introduction

The Intermission

The Journey

The Return

What to Expect

Conclusion

Creating a Chrooted Environment

Introduction

What is chroot

Setup jailed root environment

Create a new user

Provision commands to run

Change the root environment

Check the commands

Limitations

Conclusion

Update existing security list ingress rules in OCI with the python SDK

Prerequisites

Introduction

Import oracle SDK

Authentication

Get compartment data

Create the security list rules

Update Security Rules

Conclusion

A visual giude on Quicksort algorithm

Create the Pipeline File