Ollama Python Tutorial: Run Local LLMs with Chat, Streaming, Structured Outputs, Tools and Embeddings (2026) Python sqlite3 Tutorial: Tables, Transactions, JSON and Upserts with the Built-in Database (2026) Convert Python to EXE with PyInstaller: One-File Apps, Data Files and Hidden Imports (2026) Two-Factor Authentication in Python with pyotp: TOTP, QR Codes and FastAPI (2026) Hash Passwords in Python with Argon2: pwdlib, argon2-cffi and bcrypt Migration (2026) Encrypt and Decrypt in Python with Fernet: Strings, Files and Passwords (2026) psutil Tutorial: Monitor CPU, Memory, Disk and Processes in Python (2026) PyMuPDF Tutorial: PDF to Image, Extract Text and Images in Python (2026) MoviePy 2 Tutorial: Edit Videos with Python (Trim, Text, Merge, Audio) 2026 FastAPI Dependency Injection with Depends: Annotated, Yield and Overrides (2026) HTML to PDF in Python with WeasyPrint and Jinja2: Invoices and Reports (2026) Extract Tables from PDF in Python with pdfplumber: CSV, Excel and pandas (2026) OCR in Python: Extract Text from Images and Scanned PDFs with RapidOCR (2026) Text to Speech in Python with Piper: Offline Natural Voices, WAV Files and a TTS API (2026) Speech to Text in Python with faster-whisper: Transcribe Audio and Make Subtitles (2026) Send Email with Python: smtplib, HTML, Attachments and Bulk Mail (2026) Python QR Code Generator: Styled QR Codes with qrcode (2026) Pillow 13: Resize, Crop and Watermark Images in Python (2026) Parallel Programming with Dask in Python 2026: Intro Guide MarkItDown: Convert Files to Markdown for LLMs in Python (2026) Plotly Express: Interactive Charts in Python (2026) pandera: Validate pandas DataFrames in Python (2026) jsonschema: Validate JSON Data in Python (2026) Faker: Generate Fake Test Data in Python (2026) RapidFuzz: Fuzzy String Matching in Python (2026) sqlglot: Parse and Transpile SQL in Python (2026) structlog: Structured Logging in Python (2026) python-pptx in 2026: Automate PowerPoint Reports with Python python-docx: Create and Edit Word Documents with Python (2026) pypdf: Merge, Split and Edit PDFs with Python (2026) View all in this category →

dbt for Data Engineering - Full Course (dbt Core, DuckDB, Tests, Snapshots, Airflow Project) – Source Code

Learn dbt for data engineering from zero, locally: dbt Core with DuckDB, sources and freshness, staging, intermediate and mart models with ref(), seeds, materializations, Jinja macros, data tests, unit tests, docs, SCD type 2 snapshots, incremental models, dbt_utils, selectors and state, a report-ready revenue mart, an Airflow DAG, model contracts, a Python model, hooks and dbt retry.

Duration
2:02:00 (29 chapters)
Tools
dbt Core 1.12.6, dbt-duckdb 1.11.1, DuckDB 2.0.0, dbt_utils 1.4.1, Python 3.13, Apache Airflow 3.4.0, VS Code, bash

Watch the course

Watch on YouTube

Chapters

  1. 0:00:0001. Welcome: why dbt, ELT and the ShopFlow plan
  2. 0:04:1402. Setup: Python, dbt Core and the DuckDB adapter
  3. 0:08:1803. Generating the ShopFlow raw data
  4. 0:13:5304. Extract and load: raw data into DuckDB
  5. 0:17:1805. A dbt project from scratch: dbt_project.yml and folders
  6. 0:20:3906. profiles.yml, targets and dbt debug
  7. 0:23:3807. Sources and source freshness
  8. 0:26:3108. Your first models: staging with source()
  9. 0:30:5409. Seeds and custom schemas
  10. 0:34:5510. ref() and the DAG: intermediate models
  11. 0:38:5311. Materializations: view, table, ephemeral, incremental
  12. 0:44:3512. Jinja and macros: reusable SQL
  13. 0:48:1313. Data tests: unique, not_null, accepted_values, relationships
  14. 0:52:5214. Singular tests and your own generic test
  15. 0:56:2015. Unit tests: test model logic with fake rows
  16. 1:00:1816. Documentation and dbt docs
  17. 1:03:3817. Snapshots: keeping history with SCD type 2
  18. 1:08:0418. Incremental models for big, growing tables
  19. 1:13:2619. Packages: dbt_utils
  20. 1:16:5820. Selectors, tags, state and dev vs prod
  21. 1:21:1021. A report-ready mart and exporting the report
  22. 1:25:0622. Orchestrating dbt with Airflow on a schedule
  23. 1:30:2523. Model contracts and constraints
  24. 1:34:5924. Python models in dbt
  25. 1:38:2525. Hooks and run-operation macros
  26. 1:42:2026. Analyses, exposures and fixing failures with dbt retry
  27. 1:46:4227. A production checklist for dbt projects
  28. 1:50:1428. dbt interview questions
  29. 1:56:4929. Recap and next steps

What's in the zip

  • dbt-for-data-engineering/.gitignore
  • dbt-for-data-engineering/HOW_TO_RUN.txt
  • dbt-for-data-engineering/LICENSE
  • dbt-for-data-engineering/MANIFEST.sha256
  • dbt-for-data-engineering/README.md
  • dbt-for-data-engineering/airflow_env.sh
  • dbt-for-data-engineering/analyses/.gitkeep
  • dbt-for-data-engineering/analyses/top_customers.sql
  • dbt-for-data-engineering/dags/shopflow_dbt.py
  • dbt-for-data-engineering/dbt_project.yml
  • dbt-for-data-engineering/export_report.py
  • dbt-for-data-engineering/generate_data.py
  • dbt-for-data-engineering/interview.md
  • dbt-for-data-engineering/load_raw.py
  • dbt-for-data-engineering/macros/cents_to_dollars.sql
  • dbt-for-data-engineering/macros/generate_schema_name.sql
  • dbt-for-data-engineering/macros/row_counts.sql
  • dbt-for-data-engineering/models/docs.md
  • dbt-for-data-engineering/models/intermediate/int_order_items_enriched.sql
  • dbt-for-data-engineering/models/intermediate/int_payments_pivoted.sql
  • dbt-for-data-engineering/models/marts/_exposures.yml
  • dbt-for-data-engineering/models/marts/_marts.yml
  • dbt-for-data-engineering/models/marts/dim_customers.sql
  • dbt-for-data-engineering/models/marts/fct_order_events.sql
  • dbt-for-data-engineering/models/marts/fct_order_items.sql
  • dbt-for-data-engineering/models/marts/fct_orders.sql
  • dbt-for-data-engineering/models/marts/mart_monthly_revenue.sql
  • dbt-for-data-engineering/models/marts/status_summary.py
  • dbt-for-data-engineering/models/staging/_sources.yml
  • dbt-for-data-engineering/models/staging/_staging.yml
  • dbt-for-data-engineering/models/staging/stg_customers.sql
  • dbt-for-data-engineering/models/staging/stg_order_items.sql
  • dbt-for-data-engineering/models/staging/stg_orders.sql
  • dbt-for-data-engineering/models/staging/stg_payments.sql
  • dbt-for-data-engineering/models/staging/stg_products.sql
  • dbt-for-data-engineering/new_events.py
  • dbt-for-data-engineering/notes/materializations.md
  • dbt-for-data-engineering/notes/production_checklist.md
  • dbt-for-data-engineering/package-lock.yml
  • dbt-for-data-engineering/packages.yml
  • dbt-for-data-engineering/plan.md
  • dbt-for-data-engineering/profiles.yml
  • dbt-for-data-engineering/seeds/country_regions.csv
  • dbt-for-data-engineering/seeds/payment_methods.csv
  • dbt-for-data-engineering/selectors.yml
  • dbt-for-data-engineering/snapshots/customers_snapshot.yml
  • dbt-for-data-engineering/tests/assert_payments_match_order_totals.sql
  • dbt-for-data-engineering/tests/generic/positive_value.sql
  • dbt-for-data-engineering/update_customers.py

How to run

Python 3.10+ (tested with 3.13). Linux or macOS; on Windows use WSL. Run every command from the course folder in a bash terminal.

python3 -m venv .venv && source .venv/bin/activate
pip install dbt-core==1.12.6 dbt-duckdb==1.11.1
python generate_data.py
python load_raw.py
dbt debug
dbt deps
dbt build
python export_report.py

See HOW_TO_RUN.txt for every step from the video: snapshots, incremental runs, the prod target and state, and the Airflow DAG.

Data

No datasets are bundled. generate_data.py generates fake ShopFlow data (seed 42, with a few messy rows on purpose) and load_raw.py loads it into a local DuckDB file. dbt packages are downloaded by dbt deps, not bundled. No cloud account and no API key.

Download source code

File: dbt-for-data-engineering-code.zip

Size: 25.4 KB (26,004 bytes)

SHA-256:
29d38dfec90dc6294cfd2490f2921019e0454525d156dc7a832ee5f252cfcee9

Licence: code is released under the MIT licence, copyright PyInns. See LICENSE and README inside the zip. Free learning code, not production code.