DEV Community: Nishkarsh Raj

Popular Terraform Commands for Cloud Provisioning

Nishkarsh Raj — Sat, 21 Nov 2020 04:20:00 +0000

Let's use the following HCL script for provisioning multiple EC2 instances, creating a VPC with a VPN and also launching an S3 bucket on AWS Cloud.

Although the following script is meant specifically for AWS cloud, the terraform commands have the same usage for all cloud platforms.

provider "aws" {
  region = "ap-south-1"
}

resource "aws_instance" "nish" {
  ami           = "ami-02b5fbc2cb28b77b8"
  count         = 2
  instance_type = "t2.micro"
  tags = {
    Name = "noicecurse"
  }
}

resource "aws_s3_bucket" "nish" {
  bucket = "shiftnoicecurse" # name must be universally unique for all AWS users
  acl    = "private"
}

resource "aws_vpc" "vpc" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_vpn_gateway" "vpn_gateway" {
  vpc_id = "vpc-0a5a4d4f01bc6769e" #hardcoding default vpc id
}

resource "aws_customer_gateway" "customer_gateway" {
  bgp_asn    = 65000
  ip_address = "172.0.0.1"
  type       = "ipsec.1"
}

resource "aws_vpn_connection" "main" {
  vpn_gateway_id      = aws_vpn_gateway.vpn_gateway.id
  customer_gateway_id = aws_customer_gateway.customer_gateway.id
  type                = "ipsec.1"
  static_routes_only  = true
}

It is assumed that the AWS credentials are stored locally by using AWS-CLI.

Commands

1. Fmt

$ terraform fmt

Format the HCL script into canonical form for increased readability.

Before format

After format

2. Init

$ terraform init

Install the plugins and dependencies mentioned for the provider.

3. Validate

$ terraform validate

Validate syntax and find errors in the HCL script.

4. Plan

$ terraform plan

See the changes that would occur on provisioning the infrastructure

5. Graph

$ terraform graph -h # help
$ terraform graph -draw-cycles # visual help for debugging
$ terraform graph -type=plan # type of graph ranging from plan, destroy, apply etc.

6. Apply

$ terraform apply
$ terraform apply -auto-approve # to skip user interaction

7. Destroy

$ terraform destroy
$ terraform destroy -auto-approve # to skip user interaction

Note: If you are getting started with your journey in the world of Terraform, I highly recommend you to focus on destroying resources. In my experience, I have left resources after creating them, and ended up spending a lot of capital on cloud resources.

For an in-depth coverage around Terraform destroy, I highly recommend reading this blog by Spacelift.

Hacktoberfest 2020 Beginner's Guide

Nishkarsh Raj — Fri, 02 Oct 2020 06:20:02 +0000

StatusNeo have created an open source guide to help beginners and newbies in open source get started with their journeys in this year's edition of Hacktoberfest.

Introduce yourself to Open Source community and write technical blogs on relevant technical and programming topics.

StatusNeo / Hacktoberfest_2020

An initiative for Code Newbies in Open Source to participate in Hacktoberfest 2020.

#100DaysofMLCode Challenge

Nishkarsh Raj — Fri, 25 Sep 2020 11:10:55 +0000

100DaysofCode challenge is the best way to start building consistency in your Dev journey and share your experience in the community.

I performed the #100DaysofMLCode challenge earlier this year and it became my highest starred repository.

Since joining the Dev community earlier this month, I thought of sharing this repository with this amazing community and take their feedback.

Please review it, share feedback and contribute because there are many improvements that can be made to this. Cheers ❤️

NishkarshRaj / 100DaysofMLCode

My journey to learn and grow in the domain of Machine Learning and Artificial Intelligence by performing the #100DaysofMLCode Challenge.

CodeQL Analysis on GitHub

Nishkarsh Raj — Wed, 23 Sep 2020 05:40:36 +0000

Enabling Code Scanning - CodeQL Analysis using GitHub Actions

Note: Code Scanning is currently in Beta - Join the waitlist using this link

For demonstration, I will use a repository with known security vulnerabilities and coding errors, former of which were also tacked using GitHub Native Dependabot Application, to see that blog, please visit the link below.

Shifting From Dependabot Preview to GitHub Native Dependabot App | by Nishkarsh Raj | Medium

Nishkarsh Raj ・ Aug 2, 2020 ・
Medium

NishkarshRaj / Maven-Using-CMD

Create a project in Java using Apache Maven Build tool via Command Line and use GitHub Actions to build it on remote Docker Engine.

Step 1) Click on Security Tab on your Repository

Step 2) Under Code Scanning Alerts - Click on Code Scanning

Step 3) Setup CodeQL Analysis GitHub Actions

Let's take a look at the GitHub Actions Job YML script and understand few important points:

When will the Action perform the analysis:

1. Event Driven:

i. Push - Push event on master branch

ii. Pull_request - Pull Request on master or any sub branch.

2. Time Driven: Cron Schedules can be created. Cron is an open source package used to schedule tasks and events. To learn more about Cron Jobs, use this link.

It will take few minutes for the CodeQL Engine to analyze your repository for the first time. Wait till the Action jobs have successfully finished.

Step 4) View the GitHub Native Alerts Dashboard

Hurray! 💗

There were no errors or security issues in this repository.

In Part 3, I will create a demo repository and add intentional code and security errors and let's see how CodeQL Engine would help us with it. 😄

References

https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/enabling-code-scanning-for-a-repository

Adios

The posts are meant to spread awareness about the latest tips and tricks for upcoming and trending technologies in the software world.

If you like this work, please support me by following me on Dev, GitHub and Twitter. Cheers! ❤️

The Ultimate Hadoop Installation Cheat Sheet

Nishkarsh Raj — Wed, 23 Sep 2020 03:31:47 +0000

1. Install Java

$ apt-get -y upgrade && apt-get -y update
$ apt install -y default-jdk
$ java --version

2. Create Dedicated Hadoop user

$ sudo addgroup [group name]
$ sudo adduser --ingroup [group name] [user name] 
$ sudo adduser [username] sudo # Add to sudoers group

3. Setup Local and HDFS network connection using SSH

$ sudo apt-get install openssh-client openssh-server
$ su - [username]
$ ssh-keygen -t rsa -P ""
$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

4. Download Hadoop Tar file from official registry

Link to Hadoop Registry.

$ cd [to hadoop folder]
$ sudo tar xvzf [folder name]
$ sudo mv [extracted folder] /usr/local/hadoop
$ sudo chown -R [username] /usr/local/hadoop

5. Perform configurations

1. ~/.bashrc

Add following lines at End of file

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/native"

Execute the file to modify changes.

$ source ~/.bashrc

2. /usr/local/hadoop/etc/hadoop/hadoop-env.sh

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64

3. nano /usr/local/hadoop/etc/hadoop/core-site.xml

<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>

4. /usr/local/hadoop/etc/hadoop/hdfs-site.xml

<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/usr/local/hadoop_tmp/hdfs/namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/usr/local/hadoop_tmp/hdfs/datanode</value>
</property>

5. /usr/local/hadoop/etc/hadoop/yarn-site.xml

<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>

6. /usr/local/hadoop/etc/hadoop/mapred-site.xml

<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>

6. Create directories for data node and name node

$ sudo mkdir -p /usr/local/hadoop_space
$ sudo mkdir -p /usr/local/hadoop_space/hdfs/namenode
$ sudo mkdir -p /usr/local/hadoop_space/hdfs/datanode
$ sudo chown -R nish /usr/local/hadoop_space

7. Running Hadoop in Action

i. Format Name node

$ hdfs namenode -format

ii. Start All hadoop components

$ start-dfs.sh

iii. Start YARN

$ start-yarn.sh

iv. Check which components are up

$ jps

Explain the need of Containerization like I am five?

Nishkarsh Raj — Wed, 23 Sep 2020 02:54:43 +0000

I am a beginner in the field of containerization and tools like Docker, Kubernetes give me a hard time.
Can someone please explain the concepts in a simple manner?

Thread - DevSecOps Automation on GitHub Thread - Part 1

Nishkarsh Raj — Tue, 22 Sep 2020 08:16:16 +0000

Code Scanning

GitHub is working immensely to improve security issues in the open source repositories hosted on its platform, further evolving from its traditional Source Code Management approach to DevSecOps using its in-house Continuous Integration platform, GitHub Actions.

Securing Software, Together

With above motto in mind, GitHub has released its new feature called Code Scanning in Beta - releasing publicly soon after taking feedback from those who are using it in early access.

What is Code Scanning?

Code Scanning is an upcoming GitHub Native tool to find security vulnerabilities and coding errors in the repositories.

For any error/vulnerability found, GitHub creates an alert in the repository which can be removed only after the triggering cause has been fixed.

Code Scanning is automated using GitHub Actions and can be setup using time-driven triggers or event-driven triggers

The CodeQL Engine

Code Scanning uses CodeQL Engine for semantic code analysis.

QL is an object-oriented programming and query language that powers CodeQL.

CodeQL supports both compiled and interpreted languages including - C/C++, C#, Golang etc.

Third-Party Support

GitHub's Code Scanning is compatible with third-party tools given that they follow the open standard SARIF protocol (Static Analysis Results Interchange Format)

References / Further Readings

https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/about-code-scanning

https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/sarif-support-for-code-scanning

Adios

The posts are meant to spread awareness about the latest tips and tricks for upcoming and trending technologies in the software world.

If you like this work, please support me by following me on Dev, GitHub and Twitter. Cheers! ❤️

Hacktoberfest 2020 - GitHub 101 Virtual Meetup

Nishkarsh Raj — Sat, 19 Sep 2020 06:57:34 +0000

DigitalOcean's Hacktoberfest is one of the most popular open source events that helps kick start the journey of many aspiring talented developers on various open source repositories maintained on GitHub.

StatusNeo is conducting a virtual meetup in association with Major League Hacking to help aspiring developers get started in their open source journey by introducing them to the concepts of source code management and version control using Git VCS and the most popular Git hosting service, GitHub.

The knowledge and experience you would take away from this meetup would definitely help the participants in successfully participating in Hacktoberfest 2020.

To know more about the event or for registration, refer this link

Do register and share this event for maximum reach. Thanks ❤️

About the speaker

NishkarshRaj / NishkarshRaj

GitHub Profile Readme

Hello World 💖 👋🏽

I am an enthusiastic person with hands-on experience in software development and frameworks. Well-versed in software development methodologies, including DevOps and Agile.

📖 📚 Skills 📕 📗

Programming

DevOps

Continuous Integration and Continuous Delivery

Cloud Computing & IaaS

Web Development

I ❤️ GitHub

Achievements and Milestones 👑

🥇 University Top Ranker, School of Computer Science, University of Petroleum and Energy Studies

🏆 Intern of the Month (August 2019) @OpenGenus

🎉 Top 100 in GitHub Actions Hackathon 2020

✨ Completed #100DaysofCode #100DaysofMLCode and #301DaysofCode challenges.

Work Experience 🖱️ 🖥️ ⌨️

Position	Organization	Timeline
Developer Evangelist	@statusneo	June 2020 - July 2020
DevOps Intern	@xebia	April 2020 - June 2020
Mentor for RGSoC 2020	@OpenGenus	Feb 2020 - June 2020
Software Developer Intern	@OpenGenus	July 2019 - Sept 2019
Project Development Lead	@upes-open	Jan 2019 - June 2019

Let's Connect 🧑‍🤝‍🧑

View on GitHub

Dockerize Actions

Nishkarsh Raj — Wed, 16 Sep 2020 14:07:13 +0000

My Workflow

Dockerize-Actions

Open source projects contain source code which must be built into end-applications.

These must be deployed on servers and made available to the end-users.

To make this process easy, this GitHub Actions can be used.

It builds the source code fetched from open source repositories remotely in Docker containers using Build tools (Maven is used here) and deploys the cross-platform Docker Image on GitHub Packages publically.

Submission Category:

DIY Deployments

Yaml File or Link to Code

NishkarshRaj / Dockerize-Actions

#ActionsHackathon submission for Dev: GitHub Actions Hackathon - Dockerize your source code repository on GitHub, build it remotely via GitHub Actions and deploy it on GitHub Packages.

Dockerize-Actions

ActionsHackathon submission for Dev: GitHub Actions Hackathon - Dockerize your source code repository on GitHub, build it remotely via GitHub Actions and deploy it on GitHub Packages.

About

Open source projects contain source code which must be built into end-applications.

These must be deployed on servers and made available to the end-users.

To make this process easy, this GitHub Actions can be used.

It builds the source code fetched from open source repositories remotely in Docker containers using Build tools (Maven is used here) and deploys the cross-platform Docker Image on GitHub Packages publically.

How to Collaborate

This is an open source project. Contributions in all forms welcomed.

If you want to collaborate, follow these guidelines.

License

MIT License is used.

View on GitHub

https://github.com/NishkarshRaj/Dockerize-Actions/blob/master/.github/workflows/main.yml

DEV Community: Nishkarsh Raj

Popular Terraform Commands for Cloud Provisioning

Commands

1. Fmt

2. Init

3. Validate

4. Plan

5. Graph

6. Apply

7. Destroy

Hacktoberfest 2020 Beginner's Guide

StatusNeo / Hacktoberfest_2020

An initiative for Code Newbies in Open Source to participate in Hacktoberfest 2020.

#100DaysofMLCode Challenge

NishkarshRaj / 100DaysofMLCode

My journey to learn and grow in the domain of Machine Learning and Artificial Intelligence by performing the #100DaysofMLCode Challenge.

CodeQL Analysis on GitHub

Enabling Code Scanning - CodeQL Analysis using GitHub Actions

Shifting From Dependabot Preview to GitHub Native Dependabot App | by Nishkarsh Raj | Medium

Nishkarsh Raj ・ Aug 2, 2020 ・ Medium

NishkarshRaj / Maven-Using-CMD

Create a project in Java using Apache Maven Build tool via Command Line and use GitHub Actions to build it on remote Docker Engine.

Step 1) Click on Security Tab on your Repository

Step 2) Under Code Scanning Alerts - Click on Code Scanning

Step 3) Setup CodeQL Analysis GitHub Actions

Step 4) View the GitHub Native Alerts Dashboard

References

Adios

The Ultimate Hadoop Installation Cheat Sheet

1. Install Java

2. Create Dedicated Hadoop user

3. Setup Local and HDFS network connection using SSH

4. Download Hadoop Tar file from official registry

5. Perform configurations

1. ~/.bashrc

2. /usr/local/hadoop/etc/hadoop/hadoop-env.sh

3. nano /usr/local/hadoop/etc/hadoop/core-site.xml

4. /usr/local/hadoop/etc/hadoop/hdfs-site.xml

5. /usr/local/hadoop/etc/hadoop/yarn-site.xml

6. /usr/local/hadoop/etc/hadoop/mapred-site.xml

6. Create directories for data node and name node

7. Running Hadoop in Action

i. Format Name node

ii. Start All hadoop components

iii. Start YARN

iv. Check which components are up

Explain the need of Containerization like I am five?

Thread - DevSecOps Automation on GitHub Thread - Part 1

Code Scanning

What is Code Scanning?

The CodeQL Engine

Third-Party Support

References / Further Readings

Adios

Hacktoberfest 2020 - GitHub 101 Virtual Meetup

About the speaker

NishkarshRaj / NishkarshRaj

GitHub Profile Readme

Hello World 💖 👋🏽

📖 📚 Skills 📕 📗

Programming

DevOps

Continuous Integration and Continuous Delivery

Cloud Computing & IaaS

Web Development

I ❤️ GitHub

Achievements and Milestones 👑

Work Experience 🖱️ 🖥️ ⌨️

Let's Connect 🧑‍🤝‍🧑

Dockerize Actions

My Workflow

Submission Category:

Yaml File or Link to Code

NishkarshRaj / Dockerize-Actions

#ActionsHackathon submission for Dev: GitHub Actions Hackathon - Dockerize your source code repository on GitHub, build it remotely via GitHub Actions and deploy it on GitHub Packages.

Dockerize-Actions

About

How to Collaborate

License

Nishkarsh Raj ・ Aug 2, 2020 ・
Medium