DEV Community: Deniss T.

Granting Access to Cloud Build - Impersonating a Service Account

Deniss T. — Sat, 05 Jun 2021 00:05:23 +0000

Another option to allow your team members to interact with the Cloud Build in your project is to impersonate a service account.

How it Works:

Create a Service account giving it the Predefined roles or a Custom one (preferred) to grant it the required permissions.
Your users will (only) need to have the following roles:
- roles/iam.serviceAccountTokenCreator
- roles/serviceusage.serviceUsageConsumer

This will allow your team members to submit builds using the impersonation flag:

gcloud builds submit \
  --config cloudbuild.yaml \
  --impersonate-service-account=<SERVICE_ACCOUNT>
  --gcs-log-dir=gs://<BUCKET_NAME>/<SUBDIRECTORY>

The Flaws of this Option

Allowing the users to impersonate service accounts like that will provide them with a lot of possibilities within the project as they will technically be able to list the service accounts within the project and impersonate any of them, thus having access not only to Cloud Build but other project resources as well.

Therefore, you should never grant the Service Account Token Creator role to a user this way.

Solution for the Service Account Token Creator role

Instead of giving users the project-wide Service Account Token Creator role for the account impersonation, you should make that role service account-specific.

Here is how you can do that via Cloud Console or CLI:

Cloud Console solution

Navigate to IAM & Admin -> Service Accounts.
Click 'SHOW INFO PANEL'.
Select the relevant Service Account.
Click 'ADD MEMBER'.
Specify the user account granting it Service Account Token Creator role.
Click 'SAVE'.

CLI solution

Using the gcloud tool, add an IAM policy binding for the service account:

gcloud iam service-accounts add-iam-policy-binding <SERVICE_ACCOUNT> \
  --member="user:<USER_ACCOUNT>" \
  --role="roles/iam.serviceAccountTokenCreator"

To see the current IAM policy bindings run the following gcloud command:
```
gcloud iam service-accounts get-iam-policy <SERVICE_ACCOUNT>
```

In this case, your team members (group) will only need to have the Service Usage Consumer role, while the Service Account Token Creator role will be bound only to the specified service account.

Granting Access to Cloud Build - Custom Roles

Deniss T. — Sat, 05 Jun 2021 00:05:04 +0000

In this article, I will describe how to take advantage of the Custom roles to allow your team to use the Cloud Build in your project.

This is a better solution than using the Predefined roles as it gives you more control over the permissions you give to your team members.

Create a Custom Role

Create a Custom role that contains all the required permissions. Later, you can assign it to the group with the relevant team members.

Here are the minimum permissions that your Custom role will need to have:

cloudbuild.builds.create
cloudbuild.builds.get
cloudbuild.builds.list
cloudbuild.builds.update
storage.buckets.get - Grants permission to read bucket metadata
storage.buckets.list - Grants permission to list buckets in the project
storage.objects.get - Grants permission to view objects
storage.objects.create - Grants permission to create objects
serviceusage.services.use - Required to use the project for quota and billing purposes

Note that the storage.objects.get permission is needed for accessing the build logs, if you are storing the logs in a non-default bucket (see the "The Access Denied Error" section in my previous article).

Sample command for submitting a build:

gcloud builds submit \
  --config cloudbuild.yaml \
  --gcs-log-dir=gs://<BUCKET_NAME>/<SUBDIRECTORY>

Granting Access to Cloud Build - Predefined Roles

Deniss T. — Sat, 05 Jun 2021 00:04:58 +0000

In this article, I will go through the permissions required for your team members to use the Cloud Build in your project, and which Predefined roles you could use for that.

Required Permissions

First of all, let’s review which permissions your team members will need.

The users who access the Cloud Build API will need to have the following permissions:

storage.buckets.get
storage.buckets.list
storage.objects.create
serviceusage.services.use

The storage permissions are required as Cloud Build pushes its logs into the Google Cloud Storage, while serviceusage.services.use is needed since it enables you to consume quota and billing for the project.

Besides the permissions mentioned above, your team will also need the obvious ones like:

cloudbuild.builds.create
cloudbuild.builds.update
cloudbuild.builds.get
cloudbuild.builds.list

Predefined Roles

The following roles can grant your team the permissions mentioned above:

(storage.buckets.get and storage.buckets.list)

roles/storage.admin
roles/viewer

(storage.objects.create)

roles/storage.objectCreator

(serviceusage.services.use)

roles/serviceusage.serviceUsageConsumer

And, of course, roles/cloudbuild.builds.editor to create and cancel builds.

The Access Denied Error

Due to the known issue with the Cloud Build logs, your team members will still be experiencing the Access Denied error unless they have one of the Basic roles (Viewer/Editor/Owner).

This error is driven by not having access to the Cloud Build logs stored in GCS - they are stored in a Google-managed bucket that is available for Basic roles only.

This doesn't affect the builds themselves but their logs - your team will still be able to use the Cloud Build without having a Viewer/Editor/Owner role.

The Access Denied Error: Solution

Let's see how to avoid the Access Denied error without giving the users a Basic role.

As mentioned previously, the root cause of that error is that Cloud Build stores logs in a Google-managed bucket by default.

However, we are not obliged to use that particular
bucket and can specify one of our own with the following flag:

--gcs-log-dir=gs://<BUCKET_NAME>/<SUBDIRECTORY>

Note that you will need to specify the subdirectory for the bucket of your choice. Otherwise you will run into the following error:

ERROR: (gcloud.builds.submit) wrong collection: expected [storage.buckets], got [storage.objects]

Granting Access to Cloud Build - Intro

Deniss T. — Sat, 05 Jun 2021 00:04:39 +0000

In this series, I will review a situation when you own a GCP project and your development team needs to use the Cloud Build in it. Your team members will need certain permissions for that but there are a few details which you will need to consider when making a decision.

Available Options

There are three options that I will cover. Not to make it a long read, I have divided them into the separate articles:

A simple way to automate On-demand Backups for Cloud SQL

Deniss T. — Sat, 15 May 2021 21:40:11 +0000

Cloud SQL offers two types of backups: on-demand and automated. While enabled automated backups occur once a day and cannot be scheduled for shorter intervals, you can create the on-demand ones at any time. This article will show you how to automate the on-demand backups to be created every X hours in the simplest way possible. For that, I will be using the Cloud Scheduler.

I: Create a Custom Role that will allow creating Cloud SQL backups

You need the following permission to create Cloud SQL backups:

cloudsql.backupRuns.create

This permission should be granted to the service account that will be used by Cloud Scheduler as identity.

You could simply grant the service account one of the predefined roles, such as Cloud SQL Editor or Cloud SQL Admin , however, following the principle of least privilege, I encourage you to create a custom role instead. Here is how you can do it:

In Cloud Console, navigate to IAM & Admin -> Roles and click “ CREATE ROLE ”.
Fill in the required fields and add the cloudsql.backupRuns.create permission.

In the Cloud Console it will look like this:

II: Create a Service Account for your Cloud Scheduler job

Create a service account giving it the role from the previous step:

III: Create a Cloud Scheduler job

1. Define the job and its frequency. I have configured it to run every 6 hours:

2. Select HTTP as the target type and provide the following URL:

https://sqladmin.googleapis.com/sql/v1beta4/projects/{project}/instances/{instance}/backupRuns

Remember to replace {project} with your project ID, and {instance} with your Cloud SQL instance ID. Make sure that the selected HTTP method is POST.

3. For Auth header select the “Add OAuth token” option.

4. Specify the service account you have created in Step II.

5. Click “ CREATE ”.

Conclusion

That’s it! Now your Cloud SQL instance will have the backups created every X hours (in my particular example — every 6 hours), and you can modify the frequency easily.

Nevertheless, you should keep in mind the following:

On-demand backups are not automatically deleted the way automated backups are. They persist until you delete them or until their instance is deleted. Because they are not automatically deleted, on-demand backups can have a long-term effect on your billing charges if you do not delete them.

Reference: https://cloud.google.com/sql/docs/mysql/backup-recovery/backups#on-demand-backups

However, you can address this issue by configuring Object Lifecycle Management on the GCS bucket storing your backups.

Automated deployments for GCP Cloud Functions

Deniss T. — Sun, 06 Dec 2020 15:28:17 +0000

In this article, I will describe a scenario, in which you have two projects: dev and prod. Upon the successful deployment of your Cloud Function in the development project, it will also be deployed to the production.

For that, I will use a GitHub repository, to which I will be pushing the files for my function. In turn, this will be triggering the Cloud Build to deploy the functions.

I will first configure it for the development project only (steps 1–7), but then I will describe how you can automate the deployments to the production as well (steps 8–10).

GCP documentation reference — link.

Step 1

Create a GitHub repository for your Cloud Function.

Step 2

Create the files for your function. For simplicity purposes, I will grab the sample from the Node.js Quickstart for Cloud Functions.

index.js:

/**
* Responds to any HTTP request.
*
* @param {!express:Request} req HTTP request context.
* @param {!express:Response} res HTTP response context.
*/
exports.helloWorld = (req, res) => {
  let message = req.query.message || req.body.message || ‘Hello World!’;
  res.status(200).send(message);
};

package.json:

{
  “name”: “sample-http”,
  “version”: “0.0.1”
}

Step 3

Push your function files to the GitHub repository you created in Step 1. I will also create a branch called dev to later use it for the trigger.

git init
git add .
git commit -m “First commit”
git branch dev
git remote add origin _[LINK-TO-THE-GITHUB-REPO]_
git push -u origin dev

Step 4

Go to the Cloud Build settings in your development project and enable the suggested roles for Service Accounts and Cloud Functions.

Step 5

Navigate to the Cloud Build Triggers page to create a trigger:

Enter the name for your trigger.
Make sure that “ Push to a branch ” is selected for the event.
Under “ Source ” click “ CONNECT NEW REPOSITORY ” and select “GitHub (Cloud Build GitHub App)”.
Authorize your GitHub and select the repository you created in Step 1.
Select the branch , pushing to which you want the build to be triggered.
Click CREATE.

Step 6

Create a config file for your build. I have created the following cloudbuild.yaml for that:

steps:
- name: ‘gcr.io/cloud-builders/npm’
  args: [‘install’]
- name: ‘gcr.io/cloud-builders/npm’
  args: [‘test’]
- name: ‘gcr.io/cloud-builders/gcloud’
  args: [‘functions’, ‘deploy’, ‘cicdFunction_dev’, ‘ — trigger-http’, ‘ — runtime’, ‘nodejs10’, ‘ — entry-point’, ‘helloWorld’]

GCP documentation reference — link

Note: “helloWorld” is the name of the function in my index.js file, while “cicdFunction_dev” will be the name of the deployed Cloud Function.

Step 7

Push the config file to the GitHub repository and wait for the Cloud Build to deploy the function.

git add cloudbuild.yaml
git commit -m “Added cloudbuild.yaml”
git push -u origin dev

You can check the build status and details by visiting the Cloud Build History page.

Check the Cloud Functions page in Cloud Console and you will see that your function has been deployed:

You will notice that it also contains the Cloud Build config file:

Step 8

Now set up the prod project.

Add the Cloud Build service account from the dev project as an IAM member (Cloud Console -> IAM & Admin -> IAM), giving it the Cloud Functions Deployer and Service Account User roles in your production project.

Make sure to give the new roles a few minutes to propagate.

Step 9

Add another deployment step in your Cloud Build config file, specifying the production project.

- name: ‘gcr.io/cloud-builders/gcloud’
  args: [‘functions’, ‘deploy’, ‘cicdFunction_prod’, ‘ — trigger-http’, ‘ — runtime’, ‘nodejs10’, ‘ — entry-point’, ‘helloWorld’, ‘ — project’, ‘PRODUCTION_PROJECT’]

Step 10

Push the updated config file to the GitHub repository.

git add cloudbuild.yaml
git commit -m “Added production deployment step”
git push -u origin dev

This will trigger the build that will also deploy the function to the production project.

From now on, this function will be getting deployed in production project as well (after pushing to the GitHub repository or triggering the build manually) if the deployment to development is successful.

Note: Successful deployment of a function still doesn’t guarantee that there are no logical errors within your code. Therefore, you may want to include additional testing before deploying your function to the production environment.