Databricks Install & Auth
Overview
Set up Databricks CLI and SDK with authentication credentials.
Prerequisites
-
Python 3.8+ with pip
-
Databricks workspace URL (e.g., https://adb-1234567890.1.azuredatabricks.net )
-
Personal Access Token or OAuth credentials
-
Admin access for Unity Catalog setup
Instructions
Step 1: Install Databricks CLI and SDK
set -euo pipefail
Install CLI v2 (recommended)
pip install databricks-cli
Install Python SDK
pip install databricks-sdk
Install dbx for local development
pip install dbx
Step 2: Configure Authentication
Option A: Personal Access Token (Simplest)
Interactive configuration
databricks configure --token
Enter workspace URL: https://adb-1234567890.1.azuredatabricks.net
Enter token: dapi1234567890abcdef...
Option B: Environment Variables
Set environment variables
export DATABRICKS_HOST="https://adb-1234567890.1.azuredatabricks.net" export DATABRICKS_TOKEN="dapi1234567890abcdef..."
Or create .env file
cat > .env << 'EOF' DATABRICKS_HOST=https://adb-1234567890.1.azuredatabricks.net DATABRICKS_TOKEN=dapi1234567890abcdef... EOF
Option C: OAuth (Recommended for Production)
Using OAuth U2M (user-to-machine)
databricks configure --oauth
Using OAuth M2M (service principal)
export DATABRICKS_HOST="https://adb-1234567890.1.azuredatabricks.net" export DATABRICKS_CLIENT_ID="your-client-id" export DATABRICKS_CLIENT_SECRET="your-client-secret"
Step 3: Configure Profiles (Multi-Workspace)
~/.databrickscfg
[DEFAULT] host = https://adb-dev-1234567890.1.azuredatabricks.net token = dapi_dev_token...
[staging] host = https://adb-staging-1234567890.1.azuredatabricks.net token = dapi_staging_token...
[production] host = https://adb-prod-1234567890.1.azuredatabricks.net token = dapi_prod_token...
Step 4: Verify Connection
Test CLI connection
databricks workspace list /
Test with specific profile
databricks workspace list / --profile staging
Test SDK connection
python -c " from databricks.sdk import WorkspaceClient w = WorkspaceClient() print(f'Connected to: {w.config.host}') print(f'Clusters: {len(list(w.clusters.list()))}') "
Output
-
Databricks CLI installed and configured
-
Python SDK installed
-
Authentication credentials stored
-
Connection verified to workspace
Error Handling
Error Cause Solution
Invalid token
Expired or incorrect token Generate new PAT in User Settings
Workspace not found
Wrong host URL Verify workspace URL format
Permission denied
Token lacks permissions Request admin access or correct scopes
Connection refused
Network/firewall issue Check VPN, firewall rules
SSL certificate error
Corporate proxy Set --insecure or configure proxy
Examples
Python SDK Initialization
from databricks.sdk import WorkspaceClient
Auto-detect from environment/config file
w = WorkspaceClient()
Explicit configuration
w = WorkspaceClient( host="https://adb-1234567890.1.azuredatabricks.net", token="dapi1234567890abcdef..." )
With profile
w = WorkspaceClient(profile="production")
Service Principal Authentication
from databricks.sdk import WorkspaceClient from databricks.sdk.config import Config
config = Config( host="https://adb-1234567890.1.azuredatabricks.net", client_id="your-client-id", client_secret="your-client-secret" ) w = WorkspaceClient(config=config)
Azure AD Authentication
from databricks.sdk import WorkspaceClient from azure.identity import DefaultAzureCredential
Uses Azure AD managed identity
w = WorkspaceClient( host="https://adb-1234567890.1.azuredatabricks.net", credentials_provider=DefaultAzureCredential() )
Resources
-
Databricks CLI Documentation
-
Databricks SDK for Python
-
Authentication Methods
-
Personal Access Tokens
Next Steps
After successful auth, proceed to databricks-hello-world for your first cluster and notebook.