DEV Community: Shivam Jain

Creating and Deploying a Google Cloud Run Service Using Artifact Registry and GitHub Actions

Shivam Jain — Sun, 01 Dec 2024 17:14:24 +0000

In this guide, we’ll walk through the process of creating a Google Cloud Run (GCR) service, building and pushing a Docker image to Artifact Registry, and configuring the necessary resources using Terraform. The steps will also cover setting up a service account and IAM roles to ensure proper access control.

Step 1: Create a Service Account

Navigate to IAM & Admin > Service Accounts in the GCP console.
Fill in the required details, ensuring meaningful names are chosen for easier debugging and logging.
Skip step 3 and click on Done.

Manage keys by clicking on the three dots under Actions, and select Manage keys.

You will be redirected to a page like this:

Click on Add Key > Create New Key > JSON.
A JSON file will be downloaded — keep it safe and do not share it.

Step 2: Set Up Project Structure

|
| ~~~~~~
| ~~~~~~~
| Infra 
    > main.tf
    > providers.tf
    > variables.tf
    > registry.tf
    | environments
        > dev.tfvars
        > prod.tfvars
    | Credentials
        > gcp.json

Step 3: Initializing the Provider

First, create two files: providers.tf and variables.tf.

`variables.tf`

variable "project" {
    type        = string
    description = "GCP project name"
}

variable "region" {
    type    = map(string)
    default = {
        "us-central" = "us-central1",
        "us-east1"   = "us-east1"
    }
    description = "Region mapping"
}

variable "zone" {
    type    = map(string)
    default = {
        "us-central1-a" = "us-central1-a",
        "us-central1-b" = "us-central1-b",
        "us-central1-c" = "us-central1-c",
        "us-east1-a"    = "us-east1-a",
    }
    description = "Zone mapping"
}

`dev.tfvars`

project = "your-app-name"

`providers.tf`

provider "google" {
    project     = var.project
    region      = var.region["us-central"]
    zone        = var.zone["us-central1-c"]
    credentials = "./credentials/gcp.json"
}

Make sure to add a .gitignore to avoid committing sensitive data.

# .gitignore
credentials
environments
*.tfstate

Step 4: Create an Artifact Registry

`registry.tf`

resource "google_artifact_registry_repository" "executor-repo" {
  location      = var.region["us-east1"]
  repository_id = var.artifact_repository_id
  description   = "Executor Docker Repository"
  format        = "DOCKER"

  docker_config {
    immutable_tags = true
  }
}

Step 5: Build and Upload Image via GitHub Actions

In your repo, create the .github/workflows directory.
Inside, create a file called push_image.yml.

`push_image.yml`

name: Deploy to GCP

on:
  push:
    branches:
      - master
    paths:
      - images/*

jobs:
  build-and-push:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Docker
        uses: docker/setup-buildx-action@v2

      - name: Authenticate with GCP
        uses: google-github-actions/auth@v2
        with:
          credentials_json: ${{ secrets.GCP_CREDENTIALS }}
          token_format: 'access_token'

      - name: Set up Cloud SDK
        uses: google-github-actions/setup-gcloud@v1

      - name: Configure Docker for Artifact Registry
        run: |
          gcloud auth configure-docker ${{secrets.gcp_region}}-docker.pkg.dev --quiet

      - name: Build and push Docker image
        working-directory: ./images
        env:
          IMAGE_NAME: ${{ secrets.gcp_region }}-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/${{ secrets.ARTIFACT_REPOSITORY_ID }}/executor-image:latest
        run: |
          # Verify repository exists
          gcloud artifacts repositories describe ${{ secrets.ARTIFACT_REPOSITORY_ID }} \
            --project=${{ secrets.GCP_PROJECT_ID }} \
            --location=${{ secrets.gcp_region }} || \
          gcloud artifacts repositories create ${{ secrets.ARTIFACT_REPOSITORY_ID }} \
            --project=${{ secrets.GCP_PROJECT_ID }} \
            --location=${{ secrets.gcp_region }} \
            --repository-format=docker

          docker build -t ${IMAGE_NAME} .
          docker push ${IMAGE_NAME}

Step 6: Create Cloud Run Service

Now, let’s set up the Cloud Run service using the uploaded Docker image.

`gcr.tf`

resource "google_cloud_run_v2_service" "default" {
  name     = "cloudrun-service"
  location = "us-central1"
  deletion_protection = false
  ingress = "INGRESS_TRAFFIC_ALL"

  template {
    containers {
      image = "${var.location["us-east1"]}-docker.pkg.dev/${var.project}/${var.artifact_repository_id}/${var.exec_image_name}:${var.image_tag["latest"]}"
      resources {
        limits = {
          cpu    = "1"
          memory = "512Mi"
        }
      }
    }
  }
}

Here, we reference the Docker image URI created in the GitHub Actions workflow.

Step 7: Configure Variables

Add the following variables to variables.tf:

variable "artifact_repository_id" {
  type        = string
  description = "Artifact repository ID"
}

variable "exec_image_name" {
  type        = string
  description = "Docker image name"
}

variable "image_tag" {
  type        = map(string)
  description = "Tag for executor image"
  default     = {
    "latest" = "latest"
  }
}

variable "cloud_run_service_name" {
  type        = string
  description = "Cloud Run service name"
}

Step 8: Add Environment-Specific Configurations

In dev.tfvars, set the values for the variables:

artifact_repository_id = "arti-repo"
exec_image_name = "cool-image-name"

Step 9: IAM Policy for Cloud Run Service

To protect the Cloud Run service from unauthorized access, configure IAM roles for invocation.

`gcr.tf` (IAM Configuration)

resource "google_cloud_run_v2_service_iam_binding" "binding" {
  project  = google_cloud_run_v2_service.executor.project
  location = google_cloud_run_v2_service.executor.location
  name     = google_cloud_run_v2_service.executor.name
  role     = "roles/run.invoker"
  members  = [
    "serviceAccount:${var.account-email-invoker}",
  ]
}

This IAM binding ensures that only the designated service account can invoke the Cloud Run service.

Step 10: Conclusion and Extra Steps

Now that everything is set up, you can deploy your service with the image from Artifact Registry. You should also consider setting up Virtual Private Cloud (VPC) networks to mitigate potential DDoS risks and reduce costs.

Final Thoughts

This guide walks you through setting up a complete pipeline for deploying Dockerized applications to Google Cloud Run with Terraform and GitHub Actions. Keep your credentials and sensitive data secure, and use IAM policies to limit access as needed. If you have any questions or need further help, feel free to reach out to me on LinkedIn.

How to find missing values in dataset using pandas

Shivam Jain — Thu, 21 Sep 2023 02:00:06 +0000

1st method

If you have a dataset that is uniform , you can use .info() to figure out , which column has missing values.

housingdata.info()

RangeIndex: 20640 entries, 0 to 20639
Data columns (total 10 columns):
# Column Non-Null Count Dtype

0 longitude 20640 non-null float64
1 latitude 20640 non-null float64
2 housing_median_age 20640 non-null float64
3 total_rooms 20640 non-null float64
4 total_bedrooms 20433 non-null float64
5 population 20640 non-null float64
6 households 20640 non-null float64
7 median_income 20640 non-null float64
8 median_house_value 20640 non-null float64
9 ocean_proximity 20640 non-null object
dtypes: float64(9), object(1)
memory usage: 1.6+ MB

Look at 4 total_bedrooms , it has less non-null values , hence it has missing data

Second method

housingdata.isnull().sum()

longitude 0
latitude 0
housing_median_age 0
total_rooms 0
total_bedrooms 207
population 0
households 0
median_income 0
median_house_value 0
ocean_proximity 0
dtype: int64

How to find number of columns of dataset using pandas

Shivam Jain — Thu, 21 Sep 2023 01:51:55 +0000

housingdata = pd.read_csv('oneDataset/housing.csv')
len(housingdata.columns)

Running MongoDB Locally with Podman: Quick Guide

Shivam Jain — Thu, 17 Aug 2023 18:05:56 +0000

Welcome to our guide on setting up a local MongoDB server using Podman.

Step 1: Install Podman (if needed)

Check Podman installation:

podman --version

If not installed, get Podman from official instructions

Step 2: Pull MongoDB Image

Fetch MongoDB Community Server image:

podman pull docker.io/mongodb/mongodb-community-server:latest

Step 3: Verify Image

Confirm MongoDB image:

podman images

Step 4: Run MongoDB Container

Choose:

Option 1: No Volume (Data won't persist)

podman run --detach --name todoDB -p HOST_PORT:27017 docker.io/mongodb/mongodb-community-server:latest

Replace HOST_PORT with your preferred port number (e.g., 3000).

Option 2: With Volume (Data persists)

podman run --detach --name todoDB -p HOST_PORT:27017 -v /path/to/host/data:/data/db docker.io/mongodb/mongodb-community-server:latest

Replace HOST_PORT with your preferred port number (e.g., 3000).

Understanding Port Mapping:

Use -p HOST_PORT:27017 to map a port on your machine to port 27017 in the container.
For example, -p 3000:27017 maps port 3000 on your machine to port 27017 in the container.

Your MongoDB server is accessible at mongodb://localhost:HOST_PORT from your Node.js application using Mongoose or any MongoDB client.

Volume vs. Non-Volume in MongoDB with Podman

Choose wisely:

Non-Volume Approach:

Data stored in container.
Lost if container removed.
For quick tasks, testing, or learning.

Volume Approach:

Data stored outside container.
Persists even if container removed.
Recommended for development, production-like scenarios, or staging.

Choosing:

Use Non-Volume: Quick tasks or learning.
Use Volume: Persistent data needs.

Your choice depends on your use case. Your data, your decision.

How to setup tailwind CSS with vite

Shivam Jain — Thu, 17 Aug 2023 15:21:56 +0000

Prerequisites:

Node.js and npm installed on your machine

Step 1: Set Up Your Project
Open your terminal and execute the following commands:

# Create a new Vite project with React template
$ npm create vite@latest my-project -- --template react
$ cd my-project

Step 2: Install Tailwind CSS and PostCSS
Next, install Tailwind CSS, PostCSS, and Autoprefixer as development dependencies:

$ npm install -D tailwindcss postcss autoprefixer

Step 3: Initialize Tailwind CSS
Generate the Tailwind CSS configuration file and initialize it:

$ npx tailwindcss init -p

Step 4: Configure Tailwind
Open the tailwind.config.js file and modify it as follows:

module.exports = {
  content: [
    "./index.html",
    "./src/**/*.{js,ts,jsx,tsx}",
  ],
  theme: {
    extend: {},
  },
  plugins: [],
};

Step 5: Create Your Styles
In the src directory, create a new file named index.css. Add the following content to it:

@tailwind base;
@tailwind components;
@tailwind utilities;

Step 6: Style Your React Component
In the src directory, open the App.jsx file and update its contents as follows:

import React from 'react';

export default function App() {
  return (
    <h1 className="text-3xl font-bold underline">
      Hello world!
    </h1>
  );
}

Step 7: Run Your Development Server
Start the development server to see your app in action:

$ npm run dev

Open your browser and navigate to http://localhost:3000. You should see the "Hello world!" text styled according to the Tailwind CSS classes you applied.

PostCSS

PostCSS is used in your project to enhance and optimize your CSS. It enables you to:

Apply modern CSS features and proposals.
Automatically add vendor prefixes using Autoprefixer.
Optimize and minimize your CSS for better performance.
Use a modular, plugin-based approach for customized transformations.
Ensure cross-browser compatibility for your styles.

Recursion : pow(x,n)

Shivam Jain — Mon, 14 Aug 2023 17:33:27 +0000

x^n = x*x*x*x.....(n times)

class Solution:
    def powiFunc(self,x:float, n:int):
        if(n==0):
            return 1

        ans = self.powiFunc(x,n-1)

        return x*ans


    def myPow(self, x: float, n: int) -> float:

       return self.powiFunc(x,n)

Recursion tree for 3^3

            powiFunc(3, 3)
               /     \
      powiFunc(3, 2)  * 3
         /     \
powiFunc(3, 1)  * 3
   /     \

powiFunc(3, 0) * 3

Now, let's evaluate the tree step by step:

powiFunc(3, 0) returns 1 (base case).
powiFunc(3, 1) returns 3 * 1 (3).
powiFunc(3, 2) returns 3 * (3 * 1) (9).
powiFunc(3, 3) returns 3 * (3 * (3 * 1)) (27).

So, 3^3 is indeed equal to 27

Learning Linux on the Go : CLI

Shivam Jain — Sun, 13 Aug 2023 13:16:28 +0000

As a beginner stepping into the world of Linux, the command-line interface (CLI) might seem like a daunting and mysterious realm. However, let me assure you that the CLI is not only a powerful tool but also remarkably fast and productive once you become acquainted with its capabilities. In this blog post, we'll take a dive into the basics of the Linux CLI, shedding light on fundamental commands that will set you on a path to navigating, manipulating files, and performing tasks with confidence.

Unveiling the Command Line

Linux command syntax: command options arguments

Gliding Through Folders

The heart of file management lies in moving through directories and manipulating them.

cd (Change Directory): Navigate between directories effortlessly.
- Example: cd Documents - Moves to the "Documents" directory.
rmdir (Remove Directory): Delete an empty directory.
- Example: rmdir EmptyFolder - Deletes the empty folder named "EmptyFolder."

Mastering File Operations

Working with files is a breeze once you know the basics.

touch: Create empty files.
- Example: touch file.txt - Creates an empty file named "file.txt."
rm: Remove files or directories (use with caution!).
- Example: rm file.txt - Deletes the file named "file.txt."
- Example: rm -r Folder - Recursively deletes the "Folder" directory and its contents.
mv: Move or rename files and directories.
- Example: mv old.txt new.txt - Renames "old.txt" to "new.txt."
- Example: mv file.txt /new/location/ - Moves "file.txt" to a different location.
cp: Copy files and directories.
- Example: cp file.txt copy.txt - Copies "file.txt" to a new file named "copy.txt."
- Example: cp -r Folder /backup/ - Copies the "Folder" directory and its contents to the "/backup/" directory.

Exploring General Commands

Discover essential commands that provide useful system information.

pwd (Print Working Directory): Display the current directory's path.
- Example: pwd - Prints the path of the current directory.
date: Show the current date and time.
- Example: date - Displays the current date and time.
cal: Display a calendar for the current month.
- Example: cal - Shows a calendar for the current month.
ls: List files and directories in the current location.
- Example: ls - Lists files and directories in the current directory.
- Example: ls -l - Lists files in long format, showing detailed information.

A challenge :)

Try deleting a folder with some files (make a dummy folder , don't blame me for deleting something important :P )

try using rmdir . What does it do?

Well , that's the thing about linux , you need to be the google/gpt genius .

heed this sage advice: in the Linux realm, be the inquisitive genius, not the reckless fool. Each command wields power; wield it wisely and embrace the journey of discovery. The Linux realm beckons, dear comrade. Welcome!

Majority Element I and II : Boyer-Moore Majority Vote algorithm

Shivam Jain — Thu, 10 Aug 2023 16:15:39 +0000

These are the questions touched :

https://leetcode.com/problems/majority-element/

https://leetcode.com/problems/majority-element-ii/description/

History and Intent

Introduced by Robert S. Boyer and J Strother Moore in their paper titled "MJRTY - A Fast Majority Vote Algorithm."
Solves the problem of finding a majority element in an array (> n/2 times).
Shows the innovation and ingenuity in tackling computational problems.

Background and Motivation

Boyer and Moore were known for their work in computer science and formal methods.
Developed the Boyer-Moore theorem prover, bridging computer science and mathematics.

Majority Element Problem

Problem: Find an element that appears > n/2 times in an array.
Applications in voting systems, data analysis, and distributed algorithms.
Existing algorithms had higher complexities than desired.

Algorithm Innovation

Developed the Boyer-Moore Majority Vote algorithm to efficiently solve the problem.
Based on observation and mathematical reasoning.
Eliminates pairs of different elements, preserving the majority element.

Publication and Impact

Published in 1981 paper "MJRTY - A Fast Majority Vote Algorithm."
Presented principles, correctness proofs, and applications.
Efficient linear time complexity (O(n)) and constant space complexity (O(1)).

Legacy and Continued Use

Recognized for its elegance and efficiency.
Extended to various domains beyond computer science.
Variations used to solve related problems (> n/3 times).

Boyer-Moore Majority Vote Algorithm Overview

Purpose: Efficiently identifies the majority element in an array (> n/2 times).
Origin: Introduced in 1981 by Robert S. Boyer and J Strother Moore.
Observation: Majority element's count will surpass others during traversal.

Algorithm Steps

Initialization: Start with candidate and count variables.
Traverse the Array:
- If count is 0, set candidate as current element and increment count.
- If current element matches candidate, increment count.
- If different, decrement count.
Validation: candidate holds potential majority element. Validate if it appears > n/2 times.

Algorithm Efficiency

Single traversal of the array.
Utilizes constant space for candidate and count variables.

Application: Majority Element I

https://leetcode.com/problems/majority-element/

Here's an example of how the Boyer-Moore Majority Vote algorithm is applied to solve the Majority Element I problem:

class Solution:
    def majorityElement(self, nums: List[int]) -> int:
        candidate = None
        count = 0

        for vote in nums:
            if count == 0:
                candidate = vote
                count = 1
            elif candidate == vote:
                count += 1
            else:
                count -= 1

        result = 0
        for check in nums:
            if check == candidate:
                result += 1

        majority = len(nums) // 2

        if result > majority:
            return candidate

Application: Majority Element II

https://leetcode.com/problems/majority-element-ii/description/

For the Majority Element II problem, the Boyer-Moore Majority Vote algorithm can be adjusted to find elements that appear more than n/3 times in an array:

class Solution:
    def majorityElement(self, nums: List[int]) -> List[int]:
        mapi = {}
        for i in range(len(nums)):
            if nums[i] not in mapi:
                mapi[nums[i]] = 0

            mapi[nums[i]] += 1

        n, list1 = len(nums), []
        for i, v in mapi.items():
            if v > n // 3:
                list1.append(i)
        return list1

DSA + real life . Intent and goals.

Shivam Jain — Thu, 10 Aug 2023 14:33:38 +0000

Hey there, coding enthusiasts! 🌟 Are you ready to level up your Data Structures and Algorithms (DSA) skills while exploring their real-life applications? Join me on this exciting journey as we dive into the world of problem-solving, creativity, and practical coding.

** These posts will also be hosted on Github , if anyone wants to add code of missing languages , you can just give a pull request **

shivam-jainn / DSArl

dev.to first real life DSA series

DSArl

dev.to first real life DSA series

View on GitHub

Clear Objectives and Goals

My goal is crystal clear: over the next 4 months, I'm taking on the Striver a2z DSA course. But it's not just about acing the course – I'm infusing every challenge with a real-life application twist to make this journey both educational and intriguing.

Structured Learning Path

To stay on track, I've meticulously divided the Striver a2z DSA course into smaller, manageable sections. Each topic will be conquered within a set timeframe, giving us a sense of achievement as we progress.

Interactive Learning

Learning should be fun and engaging! Along the way, I'll be throwing in quizzes, challenges, and polls related to the DSA concepts we're exploring. Get ready to test your skills and learn through active participation.

Real-Life Applications

Here's where the magic happens – I'm not just solving problems, I'm giving each solution a real-world twist. We'll explore how these algorithms can be applied to actual scenarios, from optimizing travel routes to managing inventory systems.

Code Walkthroughs

Solutions are great, but understanding the why and how is even better. With detailed code explanations, I'll break down each step, unravel the thought process, and show you how the solution aligns with the problem's real-life context.

Showcase Progress

I'm not just sharing the destination, but the journey as well. Expect regular updates on the challenges I faced, the "aha" moments I experienced, and how my DSA skills are evolving over time.

Community Interaction

Let's learn together and foster a supportive community. Share your solutions, insights, and your own real-life applications of the problems. Collaboration makes the learning experience richer.

Multi-Platform Approach

From in-depth blog posts to bite-sized updates on Instagram, Twitter, and LinkedIn, I'll be sharing content across various platforms to ensure you don't miss a thing.

Collaborations

Bringing in fresh perspectives is essential. I'll be teaming up with fellow learners and experienced developers to explore new angles and deepen our understanding.

Feedback and Improvement

Your feedback is gold! I'm eager to hear your thoughts and suggestions so I can enhance the content and ensure it's serving you the best it can.

Consistency is Key

Expect regular updates and a consistent posting schedule. Together, we'll maintain the momentum and keep the learning fire burning.

Personal Touch

Coding is personal, and I'll be sharing anecdotes and stories from my journey. Let's connect on a deeper level and inspire each other.

Are you as excited as I am? If you're ready to master DSA, explore real-life applications, and join a vibrant coding community, hit that follow button and let's embark on this adventure together! Stay tuned for my first post on dev.to where we'll dive into the first topic of the Striver a2z DSA course.

Happy coding, and remember: it's not just about the code; it's about the impact we create with it. Let's make a difference, one algorithm at a time!

DEV Community: Shivam Jain

Creating and Deploying a Google Cloud Run Service Using Artifact Registry and GitHub Actions

Step 1: Create a Service Account

Step 2: Set Up Project Structure

Step 3: Initializing the Provider

variables.tf

dev.tfvars

providers.tf

Step 4: Create an Artifact Registry

registry.tf

Step 5: Build and Upload Image via GitHub Actions

push_image.yml

Step 6: Create Cloud Run Service

gcr.tf

Step 7: Configure Variables

Step 8: Add Environment-Specific Configurations

Step 9: IAM Policy for Cloud Run Service

gcr.tf (IAM Configuration)

Step 10: Conclusion and Extra Steps

Final Thoughts

How to find missing values in dataset using pandas

1st method

Second method

How to find number of columns of dataset using pandas

Running MongoDB Locally with Podman: Quick Guide

How to setup tailwind CSS with vite

PostCSS

Recursion : pow(x,n)

Learning Linux on the Go : CLI

Unveiling the Command Line

Gliding Through Folders

Mastering File Operations

Exploring General Commands

A challenge :)

Majority Element I and II : Boyer-Moore Majority Vote algorithm

History and Intent

Background and Motivation

Majority Element Problem

Algorithm Innovation

Publication and Impact

Legacy and Continued Use

Boyer-Moore Majority Vote Algorithm Overview

Algorithm Steps

Algorithm Efficiency

Application: Majority Element I

Application: Majority Element II

DSA + real life . Intent and goals.

shivam-jainn / DSArl

dev.to first real life DSA series

DSArl

Clear Objectives and Goals

Structured Learning Path

Interactive Learning

Real-Life Applications

Code Walkthroughs

Showcase Progress

Community Interaction

Multi-Platform Approach

Collaborations

Feedback and Improvement

Consistency is Key

Personal Touch

`variables.tf`

`dev.tfvars`

`providers.tf`

`registry.tf`

`push_image.yml`

`gcr.tf`

`gcr.tf` (IAM Configuration)