DEV Community: Henrik Hedlund

Google Cloud Functions with private Go dependencies

Henrik Hedlund — Wed, 26 May 2021 19:01:47 +0000

Anyone who has deployed a Google Cloud Function written in Go knows that there are a number of restrictions involved. For example, the highest Go version supported is 1.13. Another example is the lack of built-in support for dependencies when using Go modules. This post covers an approach to handling the latter using Terraform.

I'm not going into details about Go modules, or how to configure your environment to use private modules. Basically, I am assuming you have a cloud function with private dependencies, and a working local environment where you are able to fetch those dependencies.

The principles in this post can be applied to a CI/CD pipeline as well. The only thing that would be different is how you authenticate with the upstream Git repository, but the details for that could be the topic for another post.

The function module

So let's start with a quick overview of what we're really talking about here. A hypothetical scenario could be a Git repository where you have defined a Go module that has a go.mod that looks a little something like this:

module github.com/hedlund/my-cloud-function

go 1.13

require (
    cloud.google.com/go v0.81.0
    github.com/hedlund/private-module v0.1.0
)

Basically, you have a module that may, or may not, have a number of public dependencies (the cloud-google.com/go module in this example). Those are not an issue, as if you deploy the function and simply include the go.mod and go.sum files, the deployment pipeline will automatically download and use those dependencies for you.

The issue arise when it tries to download any private modules, such as the github.com/hedlund/private-module, as Google Cloud Build (which is used under the hood), and it's service accounts, do not have access to download your private repositories.

If you have worked with Cloud Build before, you know that there are ways to authenticate the builds to actually get access (this is how you get a CI/CD pipeline working), but unfortunately Google does not expose, or give us control, over the actual build tasks involved when deploying a Cloud Function, so that route is not available to us.

The solution is to move the parts that require authentication to outside of Google's automated build process - to before we actually upload any code to the Cloud Functions. As mentioned earlier, this can either be on your local development machine, or it can be part of a CI/CD pipeline, even one running on Google Cloud Build. It does not matter from where you do it, as long as the environment running it can authenticate with whatever upstream Git repository you are using.

A bit of Terraform

Let's start with the basic Terraform setup to actually deploy a function. The way I typically do it is to zip the whole folder containing the function code (you may need to exclude some files though), upload it to a storage bucket (not included in this example), and then deploy that zip file:

data "archive_file" "function_archive" {
  type        = "zip"
  source_dir  = "${path.module}/.."
  output_path = "${path.module}/../my-function.zip"
  excludes    = ["terraform", "my-function.zip"]
}

In this hypothetical example the Terraform code is in a sub-folder, terraform, so it zips everything from it's parent folder, excluding the Terraform code and the zip-file itself. Then we upload that file to a storage bucket:

resource "google_storage_bucket_object" "function_archive" {
  bucket = var.bucket
  name   = "my-function-${data.archive_file.function_archive.output_md5}.zip"
  source = data.archive_file.function_archive.output_path
}

The bucket is not defined in this code, but just passed as a variable. Finally, we create the Cloud Function itself using the uploaded archive:

resource "google_cloudfunctions_function" "my_function" {
  name         = "my-function"
  entry_point  = "MyFunction"
  runtime      = "go113"
  trigger_http = true

  source_archive_bucket = google_storage_bucket_object.function_archive.bucket
  source_archive_object = google_storage_bucket_object.function_archive.name
}

I've simplified the code a bit, and only included the most relevant parts, but you can always check the documentation for more thorough examples.

If we didn't have private dependencies in our example, that would be everything we need to do in order to deploy the function. But as it we do, this will fail during the deployment, as it can't download hedlund/private-module.

Vendoring

The simplest solution is really simple, and all we need to do is to wrap all our Terraform commands in a script that vendors our dependencies before creating the archive:

#!/bin/bash -e

go mod vendor
(cd terraform && terraform apply)
rm -rf vendor

Basically, what we do is tell Go to vendor all dependencies, meaning it copies the dependencies from the cache in your local GOROOT into a vendor folder created within your local project. Then we run terraform apply (remember, our Terraform code is in a sub-folder), which will automatically incude the new folder in the zip-file. Finally we need to remove the vendor folder again, as it's in the way of our normal development.

This is actually quite fragile, as if something goes wrong when running the Terraform command, or you terminate the script with Ctrl + C, it will most likely not remove the vendor folder. You can write some scripts to trap any termination signal, and do the cleanup that way, but we will improve the script in another way, so don't worry about it.

Another, bigger issue, is that you need to do this for every Terraform command you need to run. Granted, plan and apply is likely the most important ones, but it still adds a bit of overhead.

Since we have vendored your dependencies, we must also make sure to not include the go.mod and go.sum files in the archive file, otherwise the deployment process will still try to download the dependencies:

data "archive_file" "function_archive" {
  type        = "zip"
  source_dir  = "${path.module}/.."
  output_path = "${path.module}/../my-function.zip"
  excludes    = [
    "terraform", "my-function.zip",
    "go.mod", "go.sum", # This line is the only change
  ]
}

At this stage, you are pretty much good to go and should be able to deploy your Cloud Function using the script. That is, as long as all your local function code is in the root directory! If there are any other packages, the deployment will still fail.

Local packages

If any of your code is organised into packages (i.e. sub-folders), the Cloud Function runtime will lose the context of how to import that code the moment we remove the go.mod file, as we are technically no longer deploying a Go module.

What we need to do is to extend our script to copy the local packages into the vendor folder, following the same structure as the package would have on the GOPATH. Let's say that we have two packages sub1 and sub2 that we need to copy; our script would then look something like:

#!/bin/bash -e

go mod vendor

# New: copy local packages
mkdir -p vendor/github.com/hedlund/my-cloud-function
cp -r sub1 vendor/github.com/hedlund/my-cloud-function
cp -r sub2 vendor/github.com/hedlund/my-cloud-function

(cd terraform && terraform apply)
rm -rf vendor

First we create a folder matching the fully qualified name of our module, then we simply copy the local packages into that folder. The Terraform script will handle the rest, and cleanup works the same as before.

This is still quite verbose, and we have a lot of hard-coded (and duplicated) strings in here. If we change the module name, we'd have to remember to also change it in our scripts, and if we add another package we'd have to add code for that as well. Let's improve the script, making it a bit more generic:

#!/bin/bash -e

go mod vendor

# New: extract module name...
module=$(< go.mod grep "^module .*")
module=$${module#"module "}

# ...and copy any folders
mkdir -p "vendor/$module"
for f in * ; do [ -d "$f" ] && cp -r $f "vendor/$module" ; done

(cd terraform && terraform apply)
rm -rf vendor

There're two changes made to the script. First, we extract the name of the module from the go.mod file itself. That way we only have to define it in a single location. I'm using grep and parameter expansion to accomplish the task, but there are probably better ways of doing it - I'm no bash ninja.

Secondly, we loop over all files in the root directory and automatically copy those that are folders. That way we don't have to remember to update the script if we add a new package, it will always be copied automatically.

This solution actually works quite well, but there are still some problems with it. First of all it's an additional layer on top of Terraform that we always have to run and maintain. More importantly, due to the vendoring and copying of the files, it will always mark the archive as changed, and thus as in need of redeployment, even though no code has actually been changed. Let's fix that with a bit of Terraform hackery!

"Plain" Terraform

Let's remove the need for en external script, by actually making it internal instead. I tried a bunch of different aproaches and resources types before actually reaching this solution. While it's not perfect, it does work - at least on my machine!

The first thing we're going to do is to change the archive code back to the original:


data "archive_file" "function_archive" {
  type        = "zip"
  source_dir  = "${path.module}/.."
  output_path = "${path.module}/../my-function.zip"
  excludes    = ["terraform", "my-function.zip"]
}

In order for this to work, we need to include the go.mod and go.sumfiles in the zip.

Then we are going to use a null_resource and a local-exec provisioner to run our script from before, but as part of the normal Terraform process:

locals {
  function_vendor_path = replace(data.archive_file.function.output_path, ".zip", "-vendor.zip")
}

resource "null_resource" "function_vendor" {
  triggers = {
    archive_md5 = data.archive_file.function_archive.output_md5
  }

  provisioner "local-exec" {
    interpreter = ["/bin/bash", "-c"]
    command = <<-EOT
      cd "$(mktemp -d)"
      unzip ${abspath(data.atchive_file.function_archive.output_path)}

      module=$(< go.mod grep "^module .*")
      module=${module#"module "}

      mkdir -p "vendor/$module"
      for f in * ; do [ -d "$f" ] && mv $f "vendor/$module" ; done
      rm go.mod && rm go.sum

      rm -f ${abspath(local.function_vendor_path)}
      zip -r ${abspath(local.function_vendor_path)} .
    EOT
  }
}

There's lots of things going on in these few lines. The whole purpose of the code is to take the zip-file Terraform creates of our function, and automatically create a new one that also contains our vendored dependencies. So we first declare a local variable that creates the path to our new zip-file based on the original one, but with a -vendor suffix.

Then we create a null_resource with a trigger that depends on the MD5 hash of the original zip-file. That means that any time the zip-file is updated, the null_resource will also be updated. Thus, if we make any changes to the code, or the dependencies, the null_resource will be updated, but otherwise nothing will happen - exactly what we are after.

On the resource we run a local script, which is quite similar to the one we used earlier. One change is that instead of running the script in the local folder, we extract the zip-file to a temporary folder and make our changes there instead. This also means that instead of copying our local packages, we can instead move them into place (and save some space). As this is a temporary copy of the original code, we can also simply delete the go.mod and go.sum files before we zip everything back together into our new zip-file, which will be placed alongside the original one.

The final thing we need to do, is to upload the new vendored file instead of the original zip-file:

resource "google_storage_bucket_object" "function_archive" {
  bucket = var.bucket
  name   = "my-function-${data.archive_file.function_archive.output_md5}.zip"

  # These are the changes:
  source     = local.function_vendor_path
  depends_on = {
    null_resource.function_vendor,
  }
}

As you can see, we only change the source declaration, meaning it will upload the vendored zip-file, and actually keep the original name, using the MD5 of the original zip-file. This way we are actually lying a bit, but there is a good reason for doing so. Because our vendored zip is actually created outside of the Terraform state, there is no way (that I found at least) to trigger the upload based on the changes made the "external" vendored file. So what I ended up doing is piggy-backing on the same mechanism we use to actually trigger creating the file in the first place, and that is changes made to the original zip-file, using the MD5 hash in the name as a trigger signal.

Since we also declare that the storage bucket object depends_on the null_resource, we make sure that our custom script is run before Terraform actually uploads the new zip-file to the storage bucket.

Final words

Is this a perfect solution? Nope, not at all. Is this even a good solution? Borderline, but it works (again, on my machine at least) and it can be nicely bundled into a Terraform module to hide its ugliness.

The best solution would be for Google to up their game and actually support the features of their own language when used together with their cloud solutions... But that's a topic for another day.

Scheduled Google Cloud Functions using Terraform and HTTP triggers

Henrik Hedlund — Tue, 19 Jan 2021 19:22:03 +0000

This is the second part of a three part series about scheduling Google Cloud Functions, using technologies such as Terraform and Cloud Scheduler. If you haven't read the first part yet, I suggest you have a glance at it, because it will be the foundation for the things I'm setting out to do here.

In the last article we deployed, and scheduled, a Cloud Function using Pub/Sub, which is the quickest, easiest and recommend way of getting it done. In this article, we're going to remove the Pub/Sub topic and simply have Cloud Scheduler invoke the Cloud Function directly over HTTP.

As I mentioned above, I would argue that using Pub/Sub for scheduling your Cloud Functions is the preferred way. You get security by default (as nothing is exposed directly to the internet), and the general infrastructure and configuration is much easier (as you will see).

So why would you bother using HTTP triggers? I actually don't have a good answer for that. Perhaps if you needed to be able to invoke the same function both via HTTP and using a schedule? The reason why I'm covering it here, is because it is a necessary ingredient in achieving what I'm setting out to do in the next part of the series - which is triggering Cloud Functions using Pub/Sub across project borders. By breaking it up into three distinct steps, my goal is to make all the information a bit more digestible.

As usual, all source code for this project is available at github.com/hedlund/scheduled-cloud-functions.

HTTP triggered function

If you look at the code in the Github repository, I'm just going to leave the old resources as-is, and create the new infrastructure along side it. Thus I'm reusing the definitions and resources we created in the last article, such as the project, services, storage bucket, etc.

As I mentioned in part 1, the type of trigger that we choose to use defines what the signature of our (Go) function will look like. As we are going to invoke this function using HTTP, we need to write a new function with an HTTP signature:

package hello

import (
    "encoding/json"
    "log"
    "net/http"
)

type HTTPMessage struct {
    Name string `json:"name"`
}

func HelloHTTP(w http.ResponseWriter, r *http.Request) {
    if r.Method != "POST" {
        http.Error(w, "Not allowed", http.StatusMethodNotAllowed)
        return
    }

    var m HTTPMessage
    if err := json.NewDecoder(r.Body).Decode(&m); err != nil {
        http.Error(w, "Bad request", http.StatusBadRequest)
        return
    }

    log.Printf("Hello, %s!", m.Name)
    w.WriteHeader(http.StatusNoContent)
}

Again, the purpose of this article is not to cover the Go specifics (use whatever language you are comfortable with), but basically what we do is check that we get a POST request, parse the request body as JSON, and then log the name within it. Save the file as hello_http.go.

We're going to deploy this function alongside the Pub/Sub one, so we need to create a new ZIP archive, and upload it to the function bucket:

data "archive_file" "http_trigger" {
  type        = "zip"
  source_file = "${path.module}/hello_http.go"
  output_path = "${path.module}/http_trigger.zip"
}

resource "google_storage_bucket_object" "http_trigger" {
  bucket = google_storage_bucket.functions.name
  name   = "http_trigger-${data.archive_file.http_trigger.output_md5}.zip"
  source = data.archive_file.http_trigger.output_path
}

Deploying the function is equally straight-forward:

resource "google_cloudfunctions_function" "http_trigger" {
  project = google_project.project.project_id
  name    = "hello-http"
  region  = "us-central1"

  entry_point  = "HelloHTTP"
  runtime      = "go113"
  trigger_http = true

  source_archive_bucket = google_storage_bucket.functions.name
  source_archive_object = google_storage_bucket_object.http_trigger.name


  depends_on = [
    google_project_service.cloudbuild,
    google_project_service.cloudfunctions,
  ]
}

Compared to the Pub/Sub function we deployed earlier, there is basically only one change (besides referencing a completely different function that is). The event_trigger block has been replaced with trigger_http = true, and that is all that's needed to make the function an HTTP triggered one.

A few words about access

At this point you may have an HTTP triggered Cloud Function, but you're not allowed to actually trigger it, as we haven't configured access yet. This is actually were it gets a bit complicated (not much though), but we'll manage.

The absolutely easiest way of "solving" the access to the function, is to make it publicly accessible. That means that anyone with the URL to the function can invoke it. If you were writing part of an API of a web service, this may be exactly what you want, but as this is a scheduled function, my bet is that you'd want to limit access to it.

In case you want to make the function publicly accessible, the following code will do the trick:

resource "google_cloudfunctions_function_iam_member" "public" {
  project        = google_cloudfunctions_function.http_trigger.project
  region         = google_cloudfunctions_function.http_trigger.region
  cloud_function = google_cloudfunctions_function.http_trigger.name

  role   = "roles/cloudfunctions.invoker"
  member = "allUsers"
}

Unless you are absolutely certain that this is what you want to do, I really recommend against making the function public.

So, I'll assume that we're not taking the easy (i.e. public) route here, so there are a few things we need to cover. First, whenever you deploy a Cloud Function with the default settings (like we have), your function will actually run as the Google App Engine service account. You can see this by inspecting the details of your function in the console - the account will have an email address looking like YOUR_PROJECT_ID@appspot.gserviceaccount.com.

While it is perfectly possible to configure access using the default service account, I would argue that it is both cleaner and better practice to use a separate service account instead (separation of concern, and principles of least privilege). So let's create a new service account:

resource "google_service_account" "user" {
  project      = google_project.project.project_id
  account_id   = "invocation-user"
  display_name = "Invocation Service Account"
}

Then we're going to configure the Cloud Function to run using the new service account. All you need to do is add a service_account_email parameter, turning the whole block into:

resource "google_cloudfunctions_function" "http_trigger" {
  project = google_project.project.project_id
  name    = "hello-http"
  region  = "us-central1"

  service_account_email = google_service_account.user.email

  entry_point  = "HelloHTTP"
  runtime      = "go113"
  trigger_http = true

  source_archive_bucket = google_storage_bucket.functions.name
  source_archive_object = google_storage_bucket_object.http_trigger.name


  depends_on = [
    google_project_service.cloudbuild,
    google_project_service.cloudfunctions,
  ]
}

Creating, and running with, an explicit service account like this is much better from a security stand-point, as the GAE service account has a scary amount of permissions by default.

If you're following along and re-apply the Terraform rules, you can see that the function must be completely replaced.

Creating the schedule

Now that we have the new function, we can create a scheduler job to invoke it. In order to do so, we are going to use the same service account as we created above. But even if the function itself is running as the service account, we still need to grant it invoker permissions before we can use it with the scheduler:

resource "google_cloudfunctions_function_iam_member" "invoker" {
  project        = google_cloudfunctions_function.http_trigger.project
  region         = google_cloudfunctions_function.http_trigger.region
  cloud_function = google_cloudfunctions_function.http_trigger.name

  role   = "roles/cloudfunctions.invoker"
  member = "serviceAccount:${google_service_account.user.email}"
}

As you can see it is almost identical to the public access block earlier, but instead of granting access to everyone we target a specific user.

Creating the scheduler job is then just a matter of:

resource "google_cloud_scheduler_job" "hello_http_job" {
  project  = google_project.project.project_id
  region   = google_cloudfunctions_function.http_trigger.region
  name     = "hello-http-job"
  schedule = "every 10 minutes"

  http_target {
    uri         = google_cloudfunctions_function.http_trigger.https_trigger_url
    http_method = "POST"
    body        = base64encode("{\"name\":\"HTTP\"}")
    oidc_token {
      service_account_email = google_service_account.user.email
    }
  }
}

Again, this is very similar to the scheduler job we created with Pub/Sub, but the target declaration is different. We tell it to use the HTTPS trigger URL of our function, use a POST request, and encode a small JSON object matching the format our function expects. Finally, we tell it to make authenticated calls using the OIDC token of our service account.

If we open the Logs Explorer and wait until the scheduler kicks in, we an see that our function is invoked as expected:

Wrapping up

As usual, if you've followed along the guide, remember to destroy any resources you have created, so you don't waste any money unnecessarily.

The next, and final, article in the series will cover invoking Cloud Functions using Pub/Sub across project boundaries.

Scheduled Google Cloud Functions using Terraform and Pub/Sub

Henrik Hedlund — Mon, 18 Jan 2021 16:25:24 +0000

This is the first part of a planned three part series, covering using Terraform to deploy Google Cloud Functions and schedule invoking them using the Cloud Scheduler.

This first article will cover a normal, and simple, setup using Pub/Sub for the scheduling. This is actually already quite well documented in the official documentation, but it is the foundation for the following articles, so let's start simple, shall we?

I will try to keep all examples as short and to the point as possible. That means I will skip configuring some things, such as regions and versions, where possible, leaving it up to Terraform and Google Cloud to pick the defaults for me. This is just to keep the examples focused on the subject at hand, but means that they are not production ready in any way.

All source code for this project is available at github.com/hedlund/scheduled-cloud-functions.

Prerequisites

In order to follow along with the examples, you'll need a working Terraform installation. I'm using Terraform 0.14.4 when writing this, but anything newer should also work as well.

You also need to authenticate the Google Cloud provider somehow. One way of accomplishing this is to install the gcloud command-line tool, and then authenticating it by running:

gcloud auth application-default login

Creating a project

A lot of tutorials expect you to create a project first, using the Google Cloud Platform console, and then connecting your Terraform scripts to that project. I'm going to create the project as part of the script itself, because down the line we are going to address some peculiarities of doing inter-project configuration, so I want to have everything in my Terraform code.

In order to work with Google Cloud, we need to add the Google provider to our code, and then we can create a brand new project:

google {
}

resource "google_project" "project" {
  name       = "Scheduled Cloud Functions"
  project_id = "hedlund-scheduled-functions"
}

There's a number of things to note here...

First, as I mentioned in the introduction, I'll be leaving out a lot of specifics in the examples, such as not defining a default region and similar. In your production code, you probably want to be more in control, so study the provisioner documentation thoroughly.

Second, for the same reason I am not specifying any kind of remote state backend (such as a Storage bucket), which I really recommend that you use in your projects.

Third, project IDs are globally unique, so if you're following along, you'll probably need to change this to your own ID.

Creating a Pub/Sub topic

When scheduling a Cloud Function, we basically have two options:

Using a Pub/Sub topic
Using an HTTP trigger

You can connect your functions to other type of events as well, but when it comes to scheduling these are the two main alternatives. In this first part of the series, we're going to focus on the Pub/Sub trigger, as that is the easiest way to configure scheduling.

In order to trigger the function, we need to enable the Pub/Sub service and create a topic that our function can listen to:

resource "google_project_service" "pubsub" {
  project = google_project.project.project_id
  service = "pubsub.googleapis.com"
}

resource "google_pubsub_topic" "hello" {
  project = google_project.project.project_id
  name    = "hello-topic"

  depends_on = [
    google_project_service.pubsub,
  ]
}

We add the depends_on coupling to help Terraform understand that the Pub/Sub service must be created before it creates the topic.

Once the topic is created, we can deploy the function.

Creating a Cloud Function

As the function itself is not the focus of this article, I'm just going to deploy a simple Hello World. I'm going to use Go, but of course you can write your function in any of the supported languages.

The type of trigger defines the signature of your cloud function. In the case of a Pub/Sub trigger, it should look something along the lines of:

package hello

import (
    "context"
    "log"
)

type PubSubMessage struct {
    Data []byte `json:"data"`
}

func HelloPubSub(ctx context.Context, m PubSubMessage) error {
    name := string(m.Data)
    log.Printf("Hello, %s!", name)
    return nil
}

I'm not going to cover much of what we're doing here as it's basically straight out of the reference documentation.

I'm going to use Terraform to deploy the function as well, but you could deploy the function separately (using some kind of CI/CD pipeline perhaps), and then configure only the scheduling parts using Terraform.

Deploying Cloud Functions using Terraform actually requires quite a bit of boilerplate infrastructure. Luckily, it is quite easy to setup. We start by saving the file above as hello_pubsub.go. Then, we create a Storage bucket:

resource "google_storage_bucket" "functions" {
  project = google_project.project.project_id
  name    = "${google_project.project.project_id}-functions"
}

Storage bucket names are, like project IDs, globally unique, so a little trick I'm using is to always use the project ID as a prefix to the bucket name. That way it's easy to find the project where the bucket actually exists.

At this point, you may need to associate a billing account with your project in order to continue. After you've done so, you also need to add billing_account = "<BILLING ACCOUNT ID>" to your google_project configuration, otherwise Terraform will de-associate your project the next time you apply the rules.

We create the deployable function, by first zipping it locally on your machine, and then automatically uploading it to the Storage bucket:

data "archive_file" "pubsub_trigger" {
  type        = "zip"
  source_file = "${path.module}/hello_pubsub.go"
  output_path = "${path.module}/pubsub_trigger.zip"
}

resource "google_storage_bucket_object" "pubsub_trigger" {
  bucket = google_storage_bucket.functions.name
  name   = "pubsub_trigger-${data.archive_file.pubsub_trigger.output_md5}.zip"
  source = data.archive_file.pubsub_trigger.output_path
}

You probably need to run terraform init again, as we are introducing a new provider to create the ZIP-file (hashicorp/archive).

The archive containing the function source code is now available in the Storage bucket, and we can finally deploy the function itself:

resource "google_project_service" "cloudbuild" {
  project = google_project.project.project_id
  service = "cloudbuild.googleapis.com"
}

resource "google_project_service" "cloudfunctions" {
  project = google_project.project.project_id
  service = "cloudfunctions.googleapis.com"
}

resource "google_cloudfunctions_function" "pubsub_trigger" {
  project = google_project.project.project_id
  name    = "hello-pubsub"
  region  = "us-central1"

  entry_point = "HelloPubSub"
  runtime     = "go113"

  source_archive_bucket = google_storage_bucket.functions.name
  source_archive_object = google_storage_bucket_object.pubsub_trigger.name

  event_trigger {
    event_type = "google.pubsub.topic.publish"
    resource   = google_pubsub_topic.hello.name
  }

  depends_on = [
    google_project_service.cloudbuild,
    google_project_service.cloudfunctions,
  ]
}

Again, we need to enable a couple of services in order to deploy the function. Not only the obvious Cloud Functions, but we also need Cloud Build in order to build the function code. As for the function itself, there are a numbers of things we need to define, and I've tried to group them into logical blocks.

First, we define which project the function belongs to, its name, and, as Cloud Functions are regional resources, we also need to specify the region where it will run. If you had defined a default region for your project, the function could've inferred it from that, but I prefer to be explicit about it (as you may need to deploy your functions in multiple regions).

We also need to specify the runtime (Go 1.13) and the entrypoint, i.e. which (Go) function to invoke. The entrypoint can actually be inferred as long as you follow some naming rules, but again, this is something that I prefer is explicit.

The source archive properties tells Terraform to use the ZIP file that we uploaded to the Storage bucket earlier. This also means that if we change the source code of the function, re-applying the Terraform rules will automatically re-deploy the function as well.

The event_trigger block is were we configure the function to be triggered by Pub/Sub, and we link the topic we created earlier. If we change the type of trigger here, we change the whole behavior of the Cloud Function, and the expected structure of the code, so this is a very important part of the configuration.

Finally, we help Terraform again by informing it that it needs to enable the services before it tries to deploy the Cloud Function.

At this point, we have a working Cloud Function deployed, but the only way of triggering it, is to publish something to the Pub/Sub topic, which is what we'll use the Cloud Scheduler for.

Creating a scheduler job

There's a curious prerequisite before we can use the Cloud Scheduler - we actually have to enable Google App Engine for our project:

resource "google_app_engine_application" "app" {
  project     = google_project.project.project_id
  location_id = "us-central"
}

Then, in a similar fashion as before, we can enable the scheduler service and create the scheduler job itself:

resource "google_project_service" "cloudscheduler" {
  project = google_project.project.project_id
  service = "cloudscheduler.googleapis.com"
}

resource "google_cloud_scheduler_job" "hello_pubsub_job" {
  project  = google_project.project.project_id
  region   = google_cloudfunctions_function.pubsub_trigger.region
  name     = "hello-pubsub-job"
  schedule = "every 10 minutes"

  pubsub_target {
    topic_name = google_pubsub_topic.hello.id
    data       = base64encode("Pub/Sub")
  }

  depends_on = [
    google_app_engine_application.app,
    google_project_service.cloudscheduler,
  ]
}

We define the scheduler job in the same region as the Cloud Function, meaning we can easily change the region in one place and simply re-apply the rules without having to search and replace.

We also configure it to run every 10 minutes. For this part you can either use traditional crontab syntax, or you can use the simpler App Engine cron syntax, which we do in our example.

The pubsub_target informs Scheduler that our job should publish to the topic we created, and that the data payload should be the string Pub/Sub. Here you can send pretty much anything you want, as long as your base64 encode it, and your function knows how to parse it.

Making sure it works

By opening the Cloud Scheduler in the GCP console, we can check that the job has been created, and we can even trigger it manually if we like:

Switching to the Cloud Functions instead, we can check the logs of the function and see that it performs the action we told it to - printing Hello, Pub/Sub!.

Wrapping up

If you've followed along the guide, and especially if you've connected a billing account, remember that you probably should tear down and delete all resources once you are finished with the project. Otherwise you risk having stuff lying around that may end up costing you money.

The next article in the series will cover scheduling the Cloud Function using
HTTP triggers instead.