DEV Community

Cover image for Config Secret Value Databricks Python SDK Windows
chuongmep
chuongmep

Posted on • Edited on

Config Secret Value Databricks Python SDK Windows

Introduction

The Databricks SDK for Python simplifies interaction with Databricks services, enabling developers to programmatically manage resources, execute workflows, and handle data securely. Setting up the SDK on a Windows environment involves straightforward steps to install, configure, and utilize its features effectively.

This guide provides an overview of how to install the Databricks SDK for Python, followed by an example to retrieve the list of secret scopes and keys. These secret scopes are crucial for securely managing credentials and sensitive data within your Databricks environment.

By the end of this guide, you'll be equipped with the foundational steps to interact with Databricks securely and efficiently in a Windows environment.

Install Databricks Cli

To install fastest version of Databricks-CLI, you need to download from https://github.com/databricks/cli/releases

Image description

After download, you need to setup the enviroment path in Windows to able to use command databricks

  1. Choose edit Path in Environment Variables
  2. Add your zip file downloaded and extracted in Databricks GitHub

After complete download, you need to do login to authentication

databricks auth login --host <workspace-url>
Enter fullscreen mode Exit fullscreen mode

Image description

When you get the information auth is successfully saved, you can check the config at %USERPROFILE%\.databrickscfg

Now you can go to next part to be able to use Databricks python SDK

Install Databricks SDK Python

pip install databricks-sdk --upgrade
Enter fullscreen mode Exit fullscreen mode

Get the list of scopes

from databricks.sdk.runtime import dbutils
for secret_scope in dbutils.secrets.listScopes():
    for secret_metadata in dbutils.secrets.list(secret_scope.name):
        print(f'found {secret_metadata.key} secret in {secret_scope.name} scope')
Enter fullscreen mode Exit fullscreen mode

Usage Databricks Python SDK

Create a scope

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
scope_name = 'aps-dev'
w.secrets.create_scope(scope=scope_name)
Enter fullscreen mode Exit fullscreen mode

Put a secret key and value

w.secrets.put_secret("aps-dev","<key>",string_value="<value>")
Enter fullscreen mode Exit fullscreen mode

Get a secret by key and value

dbutils.secrets.get(scope = "<scope name>", key = "<key>")
Enter fullscreen mode Exit fullscreen mode

Delete a secret by key

w.secrets.delete_secret(scope_name, "<key name>")
Enter fullscreen mode Exit fullscreen mode

Get all scopes

dbutils.secrets.listScopes()
Enter fullscreen mode Exit fullscreen mode

Get all keys from scope name

dbutils.secrets.list("<key name>")
Enter fullscreen mode Exit fullscreen mode

How to use in Databricks

To use in Databricks notebook, you don't need to import the library to use

Get List Scopes

dbutils.secrets.listScopes()
Enter fullscreen mode Exit fullscreen mode

Get a secret by key and value

dbutils.secrets.get(scope = "<scope name>", key = "<key>")
Enter fullscreen mode Exit fullscreen mode

When you try to see the value secret of key, you will see it REDACTED
Image description

Done, now you use your scope secret in Databricks notebook !

Reference :

Image of AssemblyAI tool

Transforming Interviews into Publishable Stories with AssemblyAI

Insightview is a modern web application that streamlines the interview workflow for journalists. By leveraging AssemblyAI's LeMUR and Universal-2 technology, it transforms raw interview recordings into structured, actionable content, dramatically reducing the time from recording to publication.

Key Features:
🎥 Audio/video file upload with real-time preview
🗣️ Advanced transcription with speaker identification
⭐ Automatic highlight extraction of key moments
✍️ AI-powered article draft generation
📤 Export interview's subtitles in VTT format

Read full post

Top comments (0)

Image of AssemblyAI

Automatic Speech Recognition with AssemblyAI

Experience near-human accuracy, low-latency performance, and advanced Speech AI capabilities with AssemblyAI's Speech-to-Text API. Sign up today and get $50 in API credit. No credit card required.

Try the API

👋 Kindness is contagious

Immerse yourself in a wealth of knowledge with this piece, supported by the inclusive DEV Community—every developer, no matter where they are in their journey, is invited to contribute to our collective wisdom.

A simple “thank you” goes a long way—express your gratitude below in the comments!

Gathering insights enriches our journey on DEV and fortifies our community ties. Did you find this article valuable? Taking a moment to thank the author can have a significant impact.

Okay