One of the issues developers can encounter when developing in Cloud Functions is the time taken to deploy changes. You can help reduce this time by dynamically loading some of your Python classes. This allows you to make iterative changes to just the area of your application that you’re working on.
For example, if you’re creating code that requires lots of small changes and want to see your results quickly, without needing to redeploy the Cloud Function each time, then this solution is for you. Put the code you want to change in a class you can dynamically load, and redeploy the Cloud Function once calling this new class. You can now copy the new class to your chosen storage and test iteratively by just changing the file in storage without having to redeploy the Cloud Function.
Once you’re happy with the result, simply move the code into your main application and redeploy the Cloud Function just once more!
I use this extensively in developing Google Chat Apps, where I have to create display cards and want to check for wording, spacing etc., so I shall use this as an example in the remainder of this post. If you’re curious about developing Apps for Google Chat, the documentation is linked in the references at the end.
How it works
There is a small Python class which leverages the standard Python classloader to load source code from a text file, currently stored in a Google Cloud Storage bucket or a Secret Manager secret. If the loader discovers that the class you’re adding has already been loaded then it performs a reload, ensuring that the latest version from the storage area is used.
[Subheading] Example: Developing a Google Chat App
The Google Chat Samples GitHub repository includes a simple Chat App that will load commands on demand from Google Cloud Storage or Secret Manager.
To try this, first grab the code from the GitHub repository. Then install the sample following the directives in the README.md file. There’s an installer provided for the bulk of the automatable work, or there are manual instructions if you’d rather see what the installer is actually doing.
Once you are setup and ready to run the code, the sample version of hello.py loaded in the Google Cloud Storage will look like this:
and produce the following output in Google Chat when executed
If you change the DecoratedText in hello.py from Hello world! to Greetings from the Dynamic Bot!, and then copy hello.py to the same location in Google Cloud Storage (the command is listed in README.md or you can use drag-and-drop in the Cloud Console) and then repeat the command /hello, you’ll see the change in your message immediately!
The way the code is written in the repository, the Google Chat App is set to use Google Cloud Storage - but all that’s necessary to switch to Secret Manager is to change line 82 in classes/dynamic_command.py to read SecretManager instead of CloudStorage (the import is left in place in the code for simplicity) and to ensure that a secret called hello is loaded in SecretManager with the code uploaded as the latest version.
Security
There is obviously a security implication here to do with loading arbitrary classes from a Google Cloud Storage bucket at runtime so care should be taken to only use this solution in a secure environment. Mitigating solutions used in the current sample are:
The bucket is hard-coded in the Python code so that only a single bucket can be used. Changes to the Cloud Function itself would be needed to alter this.
The bucket is restricted access to only project administrators and the service account under which the Cloud Function runs.
Only classes that extend a specific Python object will be loaded, and only a specific function within that is called.
There is no requirement for the code to be loaded from Google Cloud Storage, it was simply chosen as an easy way to demonstrate the use of this system and the ease by which Cloud Function development and testing can be sped up.
Should this be intended to be used in a more ‘production’ style environment, Secret Manager would be an excellent choice. It has benefits such as:
Data once loaded is immutable and can only be changed by creating a new version, leaving a version history of changes.
Secret Manager versions can be up to 64kb, which should be more than enough for this purpose.
Both Google Cloud Storage and Secret Manager implementations are provided.
Clearly, these are not the only choices. You could use any text-storage system as long as you create a new SourceGrabber to fetch the text from the desired storage location. Example SourceGrabbers for Google Cloud Storage and Secret Manager can be found in the source repository linked below, in the source_grabbers.py file.
Known Restrictions
- No net-new third-party libraries can be used. If your Cloud Function does not already have Big Query (for example) in its
requirements.txtfile, then no dynamic command can use Big Query. - All the code must be in a single file - the system will NOT attempt to load other files from storage.
Top comments (0)