DEV Community

Cover image for Configuring Google Cloud Storage for Mage

Configuring Google Cloud Storage for Mage

Cris Crawford on January 27, 2024

In this post I'll talk about how I configured Google Cloud Storage so that Mage could transfer data to a Google Cloud Storage Bucket. First I creat...
Collapse
 
twissi84 profile image
twissi84

Hi, I am basically stuck exaxtly at that point where I need to authenticate myself and I always get a connection error that the file can't be found.
I also have my service account json file saved in "/.gc". Actually I have two service accounts now. Anyway I want to point to the newly created mage-serviceaccount but it seems not possible as I am not able to write down the right directory in mage, in the "io_config_yaml". It seems impossible to write down the right json-file.

I also don't understand why you use "/home/src/keys.json" if you have the key saved in ".gc/". I tried the same but it can't find the json-file neither. Any idea what I might be doing wrong when writing down where my key-file is?

Collapse
 
cmcrawford2 profile image
Cris Crawford

Hi! I found it easier to just copy my key file to the directory where he has it. I copied it to the directory "mage-zoomcamp". That is, to ~/data-engineering-zoomcamp/02-workflow-orchestration/mage-zoomcamp. Then you have to edit the io_config.yaml and add the path "/home/src/keyname.json" for GOOGLE_SERVICE_ACC_KEY_FILEPATH. This was not intuitive because I have no directory named /home/src. But that's what he used, that's what I did, and it worked.

Collapse
 
twissi84 profile image
twissi84

Thank you for the insights. I did it as well. Sadly I didnt really get how this path links to it but it works :)