Databricks Notebook Framework: Project Scaffold — Quick Start Guide

Project Scaffold — Quick Start Guide Databricks Notebook Framework by Datanest Digital Overview This guide walks you through setting up a new Databricks data pipeline project using the Notebook Framework. In 15 minutes you'll have a working project structure with Bronze, Silver, and Gold layers ready for development. Step 1: Create Your Project Structure # Create project root mkdir my-data-project && cd my-data-project git init # Copy framework files cp -r /path/to/databricks-notebook-framework/* . # Create additional directories mkdir -p notebooks/{bronze,silver,gold} mkdir -p tests mkdir -p config Your project should look like this: my-data-project/ ├── .pre-commit-config.yaml # Copy from cicd/pre-commit-config.yaml ├── databricks.yml # Copy from cicd/databricks.yml ├── notebooks/ │ ├── bronze/ │ │ └── bronze_my_source.py │ ├── silver/ │ │ └── silver_my_entity.py │ └── gold/ │ └── gold_my_summary.py ├── utils/ │ ├── logging_utils.py │ ├── config_manager.py │ ├── quality_check