Portable DataFrames with Ibis and DuckDB in Python (2026) — express analytics once, run them on DuckDB (the default) or swap backends later.
After DuckDB + Polars and Narwhals, Ibis is the deferred-expression layer: lazy tables, SQL-shaped verbs, DuckDB under the hood until you to_pandas().
TL;DR
pip install 'ibis-framework[duckdb]'ibis.memtable(df)builds a lazy table (DuckDB default)- Chain filters/aggs; materialize with
.to_pandas()/.execute() - Swap backends with
ibis.set_backend(...)when ready
Install
pip install 'ibis-framework[duckdb]' pandas
# or: uv add 'ibis-framework[duckdb]' pandas
Example 1 — memtable + filter
import ibis
import pandas as pd
df = pd.DataFrame({
"region": ["N", "S", "N", "E"],
"sales": [120, 80, 40, 60],
})
t = ibis.memtable(df)
expr = t.filter(t.sales >= 60).select("region", "sales")
print(expr) # lazy plan
print(expr.to_pandas())
Example 2 — aggregate on DuckDB
summary = (
t.group_by("region")
.agg(sales_sum=t.sales.sum(), n=t.count())
.order_by("region")
)
print(summary.to_pandas())
print(ibis.get_backend(t)) # DuckDB by default
Compare engine tradeoffs in Polars vs Pandas when choosing what to feed Ibis.
Production tips
- Keep expressions lazy until the edge of the API
- Persist large sets with
ibis.duckdb.connect(...).create_table(...) - Test critical queries on the backend you ship with
- Prefer Parquet / Arrow transfer when crossing library boundaries
Wrap-up
In 2026, portable analytics means one expression tree and pluggable engines. Ibis + DuckDB is the default starting point for that habit.