This is a common problem with client applications. The same problem also affects mobile apps. Consider API keys and other secrets not to be secret anymore when you decide to bundle them with your client apps. See also: developer.okta.com/blog/2019/01/22...
However, when you set up Google Drive API credentials for the first time Google guides you through a small questionnaire and also asks you how and from where you want to access their APIs:
Have you tried that instead? Does it work in your case?
Other than that, just in principle when you're dealing with 3rd party API keys and secrets, your best bet is to run your own web server and web API, and let that talk to 3rd party APIs on behalf of your client app.
That way you don't expose your secrets. However, it's up to you to implement your own auth schemes in order to limit and guard access to your own web API.
In case of Google Drive access, you're basically asking your end users to trust your app with their confidential documents. Here it's a bit more reassuring to let your client app do the talking directly with Google's API, and not through your service as a man in the middle.
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.