Automating a conference submission workflow: integrating the solution

In the previous post, I described a poster child for automation: managing the repetitive tasks around a conference submission workflow. I wrote about the setup, and how to use ngrok to redirect webhooks to my computer.

This week, I'd like to write about some of the hiccups I encountered along the way.

Registering a Trello webhook

Most SaaS providers allow to register webhooks through their GUI. This is the case of GitHub for example: it's located in the Settings > Webhooks menu.

For better or worse, this is not the case of Trello. To register a webhook, one needs to execute an HTTP request. This requires:

1. An API key, which is unique per account. The fun part is, it cannot be reset. Don't leak it, or you'll be sorry!

2. A token, which is should be generated for each application. To generate a token, follow instructions on the relevant page. Required parameters are the following:

   • The scopes: read, write, etc.
   • The application name
   • An expiration date

   • The board's ID. In order to get it, append .json at the end of the board's URL to display its JSON representation. The ID is among the first few parameters:
     

     {
     "id": "5d383691a3aeaa3b5741953d",
     "name": "My board",
     "desc": "",
     "descData": null,
     "closed": false,
     "idOrganization": null,
     "idEnterprise": null
     }
     

3. Finally, send a POST request similar to the following:
   

   curl -X POST -H "Content-Type: application/json" \
   https://api.trello.com/1/tokens/{token}/webhooks/ \
   -d '{
   "key": "{key}",
   "callbackURL": "http://www.mywebsite.com/trelloCallback",
   "idModel":"{boardID}",
   "description": "Name of the webhook"
   }'
   

Preparing for the webhook

To check the webhook is valid, Trello will immediately send a HEAD request to the just-registered callback. If this request is not successful, then registration will fail. For example, this will happen if the app is not up, or if the URL is not reachable.

The HEAD HTTP method is rarely used. For example, when using Spring Boot, it requires explicitly configuring such an endpoint:

fun routes(runtimeService: RuntimeService) = router {
  val handler = TriggerHandler(runtimeService)
  POST("/trigger", handler::post)
  HEAD("/trigger", handler::head)
}

class TriggerHandler(private val runtimeService: RuntimeService) {

  fun post(request: ServerRequest): ServerResponse {
    // Starts the workflow instance
  }

  fun head(request: ServerRequest) = ServerResponse.ok().build()
}

The above snippet isn't very compliant with the HTTP specifications as the controller doesn't offer any GET endpoint':

The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request. This method can be used for obtaining metainformation about the entity implied by the request without transferring the entity-body itself. This method is often used for testing hypertext links for validity, accessibility, and recent modification.

-- Hypertext Transfer Protocol -- HTTP/1.1,https://ietf.org/rfc/rfc2616.txt

In all cases, it gets the job done. Besides, I didn't find any way to easily create a HEAD endpoint that delegates to a GET endpoint. Hints are more than welcome.

Google permissions

Google requires permission to manage objects: read, write, etc. The problem with the most recent documentation, whether on Calendar or Sheets (or any other Google API), is that it's skewed toward Android e.g. Calendar Java Quickstart. In such a use-case, the request for permission is interactive: the user is first presented with a authenticate screen, and then asked to give the required permission.

Workflows in general allow for interactive or automated steps. An interactive step allows to enter one's credentials and authorize the app, but the previously-defined workflow only has automated steps. For a truly automated workflow, interactive steps should be avoided if possible.
However, Google prevents to use one's account credentials - login and password - for security reasons. But it provides an alternative: one can create a so-called service account, and give it write permissions to the relevant calendar and documents.

Creating a Google service account

To create a service account:

1. Go to the service account page in the Google Cloud console
2. Create or select a project
3. Fill in the required fields
4. Select the newly-created account, click on Edit and then on Create Key. Download the key and store it in an safe place, as it contains the credentials necessary to authenticate oneself as the created service account.

Configuring permissions

To give the service account access to one's calendar (or documents) requires two steps:

1. First, one should give the project permissions to use the API. In the context of the project the service account was created in, open the left menu. Go to APIs & Services > Library. Select Google Calendar API, and in the opening page, click on Enable. Repeat for Google Calendar API.
2. Then, one should give delegation access to one's own calendar. In Calendar, locate the My calendars list, then click on the associated menu, and on Settings and sharing. Find the Share with specific people section, and add the service account with access level Manage changes to events. For Google Sheets, share the document with the service account.
3. If the option is grayed out, it means the domain administrator didn't allow it. If you're the administrator of your own domain, go to the Google administration console. Then, click on Apps > G Suite > Calendar. Click on Sharing settings. Select External sharing options for primary calendars, and Share all information, and outsiders can change calendars. This allows to give accounts outside the organization - such as service accounts - write access. Repeat for Google Sheets, if necessary.

Conclusion

While the previous post described the context behind automating a workflow, this post lists the necessary configuration steps: the Trello webhook and the Google service accounts. While nothing out-of-the-ordinary, it takes time to research the above facts if one doesn't know about them previously.

Originally published at A Java Geek on May 17th, 2020