Jupyter Notebook Expert Skill

This skill provides a guide for Jupyter Notebook execution.

1. Databricks Jupyter Kernel

https://github.com/i9wa4/jupyter-databricks-kernel

uv pip install jupyter-databricks-kernel uv run python -m jupyter_databricks_kernel.install

2. Default Execution Method

When instructed to execute an entire notebook, use this command:

uv run jupyter execute <notebook_path> --inplace --timeout=300

3. Execute with Databricks Kernel

When running notebook on Databricks cluster:

uv run jupyter execute <notebook_path> --inplace --kernel_name=databricks --timeout=300

Required environment variables:

• DATABRICKS_HOST : Databricks workspace URL

• DATABRICKS_TOKEN : Personal Access Token

• DATABRICKS_CLUSTER_ID : Cluster ID

4. Usage Examples

Execute with local Python kernel

uv run jupyter execute /workspace/notebooks/sample.ipynb --inplace --timeout=300

Execute with Databricks kernel

uv run jupyter execute /workspace/notebooks/databricks-sample.ipynb --inplace --kernel_name=databricks --timeout=300

5. Option Descriptions

• --inplace : Overwrite original file with execution results

• --kernel_name=<name> : Specify kernel to use (databricks, python3, etc.)

• --timeout=<seconds> : Set timeout in seconds (-1 for unlimited)

• --startup_timeout=<seconds> : Kernel startup timeout (default 60 seconds)

• --allow-errors : Continue execution to end even with errors

6. Notes

• Verify required environment variables are properly set before execution

• Adjust --timeout value for long-running cells

• If open in VS Code, verify file updates after execution

• For Databricks kernel, cluster startup takes 5-6 minutes if stopped

7. Reference Links

• jupyter-databricks-kernel: https://github.com/i9wa4/jupyter-databricks-kernel

• Jupyter nbclient: https://nbclient.readthedocs.io/