PyInns - Home

Computing the Fraction of Long Trips with Dask in Python 2026 – Best Practices

Calculating fractions or percentages (e.g., "what fraction of trips were longer than 30 minutes?") is a common analytical task. When working with large trip datasets (taxis, rideshares, deliveries, etc.), Dask allows you to compute these fractions efficiently in parallel without loading the entire dataset into memory.

TL;DR — Efficient Pattern

Filter with Dask (lazy)
Use boolean masking or .mean() for fraction calculation
Call .compute() only on the final scalar result
Combine with grouping for more insightful analysis

1. Basic Fraction Calculation


import dask.dataframe as dd

# Load large trip dataset
trips = dd.read_parquet("trips/year=2025/*.parquet")

# Define "long trip" condition
long_trips = trips[trips["trip_duration_minutes"] > 30]

# Compute fraction of long trips
fraction_long = long_trips.shape[0] / trips.shape[0]

print("Fraction of long trips:", fraction_long.compute())

2. More Efficient & Readable Way (Recommended)


# Create a boolean column (lazy)
trips = trips.assign(
    is_long_trip = trips["trip_duration_minutes"] > 30
)

# Compute fraction using mean() - very efficient
fraction_long = trips["is_long_trip"].mean().compute()

print(f"Fraction of trips longer than 30 minutes: {fraction_long:.4f} ({fraction_long*100:.2f}%)")

3. Grouped Fraction Analysis (Real-World Example)


# Fraction of long trips by region and hour
result = (
    trips.assign(
        is_long = trips["trip_duration_minutes"] > 30,
        hour = trips["pickup_datetime"].dt.hour
    )
    .groupby(["region", "hour"])
    .agg({
        "is_long": "mean",           # fraction of long trips
        "trip_id": "count"           # total trips
    })
    .rename(columns={"is_long": "fraction_long", "trip_id": "total_trips"})
    .compute()
)

print(result.sort_values("fraction_long", ascending=False).head(10))

4. Best Practices in 2026

Use boolean columns + .mean() for clean, efficient fraction calculations
Filter early to reduce data volume before aggregation
Use .assign() to create temporary columns instead of complex expressions
Always call .compute() only on the final aggregated result
After heavy filtering, consider .repartition() before grouping
Monitor memory usage in the Dask Dashboard when working with very large trip datasets

Conclusion

Computing fractions like "percentage of long trips" is straightforward with Dask when you follow the pattern: **filter early → create boolean column → use .mean() → compute only the final result**. In 2026, this approach scales beautifully from millions to billions of records while keeping your code clean and memory-efficient.

Next steps: