Setting config vars/secret API keys in Heroku is usually pretty straightforward. Here's a great post I refer to a lot on hiding your API keys in general. But things get tricky when you have anything more than the standard KEY=value setup.
When you sign up for something like the Google Maps API, you get a standard API key - a single string of characters you can plug into a .env and then Heroku config vars. But with Google Cloud APIs, you need a whole 'service account,' and .json file of various keys that looks like this:
{
"type": "service_account",
"project_id": "project-00000000",
"private_key_id": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"private_key": "-----BEGIN PRIVATE KEY-----xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-----END PRIVATE KEY-----\n",
"client_email": "xxxxxxxxxxxxxxxxx@xxxxxxxx.iam.gserviceaccount.com",
"client_id": "xxxxxxxxxxxxxxxxx",
"auth_uri": "xxxxxxxxxxxxxxxxx",
"token_uri": "xxxxxxxxxxxxxxxxx",
"auth_provider_cert_url": "xxxxxxxxxxxxxxxxx",
"client_cert_url": "xxxxxxxxxxxxxxxxx"
}
And it doesn't seem like you could just plug each of those into config vars individually. (But I didn't actually try it - so, lol if that does work! But the docs pretty clearly say to use the whole file).
It took me a while to figure out even the basic set up for local testing (I didn't install the SDK), but the process of hosting was even more stressful, because of the risk of getting credentials stolen, so assuming you've made it past all these steps, I thought I'd write a blog post about safely setting Heroku config vars when you have a .json credentials file.
This might be kind of niche, but I'm writing this based on a bootcamp project that was built with a Rails frontend and backend - so no React/JavaScript here. It should be applicable regardless - the Heroku google-cloud buildpacks here are language non-specific - but a lot of my struggles revolved around the Ruby buildpack, so just be careful if you're using React, etc., that you have the appropriate React buildpack in addition to the Google buildpack. Are you sick of the word buildpack yet?
heroku create
So now let's assume you have an app set up and ready to go for Heroku. For Rails, this might involve some extra steps with postgres. This might go without saying, but pretty much every step here can be done either from the command line or in browser. I liked to have both open, and refresh my Heroku settings page after command line changes just to sanity check things were working. But I'll show command line here, because I love code blocks!
Make sure you're logged into Heroku (heroku login) and in your main project directory, and run:
heroku create my-app
heroku buildpacks:set heroku/ruby
heroku buildpacks:add --index 1 https://github.com/buyersight/heroku-google-application-credentials-buildpack.git
Note that the --index 1 and the order here are important. Heroku docs re: multiple buildpacks: "The buildpack for the primary language of your app should be the last buildpack added."
Terminal should return, or you can verify with heroku buildpacks at any time, the following:
Buildpack added. Next release on example-app-test-123 will use:
1. https://github.com/buyersight/heroku-google-application-credentials-buildpack.git
2. heroku/ruby
Run git push heroku master to create a new release using these buildpacks.
But at this point we haven't staged anything so don't forget to run:
git add .
git commit -m "first heroku commit"
git push heroku master
General reminder, before you run git push heroku master after a buildpack change, you'll probably need to change something in the code (like even just adding a space in any file) so you can add/commit changes to be pushed and the app will be "re" built.
At this point you should notice the long chain of ruby gems installing. If the build is fast, short, and has no gems, it probably means your Ruby buildpack didn't get through and this is bad news bears! Again, check heroku buildpacks and/or settings in browser to verify it looks like this:
heroku buildpacks
=== my-app Buildpack URLs
1. https://github.com/buyersight/heroku-google-application-credentials-buildpack.git
2. heroku/ruby
Now for the real config var stuff.
As always, either in command line or browser, set the vars. In terminal:
heroku config:set GOOGLE_CREDENTIALS='<copy and paste contents of your .json file>'
and don't forget to wrap in quotes. But if you do it in browser, no quotes:
and now, same thing, but:
heroku config:set GOOGLE_APPLICATION_CREDENTIALS='google-credentials.json'
and in browser to confirm (also a larger image so you can kind of see what the settings/config vars page should look like at this point):
Note that it's important that you use exactly google-credentials.json in this step. At first I thought I could/should use whatever name I had been using in my local directory, i.e. my-config.json, but it didn't work, and I'm pretty sure it's part of how the buildpack works that this is named exactly like this. And it has nothing to do with your local directory (kind of the whole point I realized) so you don't need to make the names match or anything.
Finally, if everything has gone according to plan, you should be able to run:
heroku run rake db:migrate
heroku run rake db:seed
heroku open
and for me, all I needed to confirm the credentials worked was if the db:seed was successful, because I used the API to get the list of languages and language codes in my seeds.
Small note here - I actually call on the API in my code like this:
translate = Google::Cloud::Translate.new version: :v2, project_id: ENV["CLOUD_PROJECT_ID"]
but I don't have CLOUD_PROJECT_ID anywhere in Heroku config vars. It seems that I only needed this for local env/testing, and you could just take out the whole project_id: ENV["CLOUD_PROJECT_ID"] part here if you were just using Heroku, because of the buildpack.
Mistakes I made
I thought, with a Rails app, the Ruby buildpack gets added automatically when you run heroku create. It doesn't seem that it does, but it also seems like if you aren't using additional buildpacks, you wouldn't need to explicitly set the Ruby one. But throughout my process of trying two different google-buildpacks, I kept getting this error without realizing it was because I did not have the Ruby buildpack specified:
heroku run rake db:migrate
Running rake db:migrate on ⬢ translation-chat-app... up, run.7099 (Free)
rake aborted!
LoadError: cannot load such file -- bundler/setup
/app/config/boot.rb:3:in `<top (required)>'
/app/config/application.rb:1:in `require_relative'
/app/config/application.rb:1:in `<top (required)>'
/app/Rakefile:4:in `require_relative'
/app/Rakefile:4:in `<top (required)>'
(See full trace by running task with --trace)
So I recommend frequently running heroku buildpacks and/or checking Settings in your browser. After these bundler errors, and some stack overflowing, I realized I somehow now only had two different Google buildpacks, and no Ruby buildpack:
heroku buildpacks
=== translation-chat-app Buildpack URLs
1. https://github.com/buyersight/heroku-google-application-credentials-buildpack.git
2. https://github.com/elishaterada/heroku-google-application-credentials-buildpack
This second one - from elishaterada - did not work for me, although looking back now, this may have been because of the Ruby buildpack. So I can't entirely say it doesn't work, but it didn't for me, so I'm going to say follow this buildpack strictly: buyersight. It is a fork of the elishaterada one. Either way, you're probably not going to want two of the same thing in your buildpacks!
I found it easier to just click the 'x' button on browser to remove, but you can also do:
heroku buildpacks:remove https://github.com/elishaterada/heroku-google-application-credentials-buildpack
and then to get my Ruby back I did:
heroku buildpacks:set heroku/ruby
Conclusion
Again, there were a ton of related stack overflow posts about this out there, but nothing exactly followed/explained this in a satisfying way. It's kind of a niche problem, and weird that Google can't just give a simple API key here, but there seem to be a lot of amazing Google Cloud services that might require this. I was almost discouraged from hosting my project because I was so scared of exposing my info. But I'm glad I got past that! Hope this helped you too.
Latest comments (3)
Thanks - can you clarify how you are loading these credentials in your application? For example, I need to use text-to-speech client library - locally I can just use application default credentials, but in Heroku how do I write the code to load this credentials file?
I tried using the file method but it considers that file name as a string and doesnot read that file
thanks for this !!!