Replacing Missing Values in Pandas – Imputation Techniques 2026

Replacing Missing Values in Pandas – Imputation Techniques 2026

Replacing (imputing) missing values is often preferable to simply dropping them, especially when data is limited or missingness is high. In 2026, Pandas offers several smart and context-aware ways to fill missing values while preserving the integrity of your dataset.

TL;DR — Most Common Imputation Methods

• fillna(0) – For counts and amounts when missing means zero
• fillna(df.mean()) – Mean imputation
• fillna(df.median()) – Median imputation (more robust to outliers)
• Group-wise imputation using groupby().transform()

1. Basic Replacement Techniques

import pandas as pd

# Create a small demo file so this example runs as-is
# (in your project, sales_data.csv is your real export)
import numpy as np
rng = np.random.default_rng(42)
n = 60
pd.DataFrame({
    "order_date": pd.date_range("2026-01-01", periods=n, freq="3D"),
    "customer_id": rng.integers(101, 121, n),
    "customer_name": rng.choice([" ana", "ben ", "chen", "dana"], n),
    "product_id": rng.integers(1, 9, n),
    "region": rng.choice(["North", "South", "East", "West"], n),
    "category": rng.choice(["Electronics", "Computers", "Books", "Home"], n),
    "payment_method": rng.choice(["card", "cash", "wallet"], n),
    "segment": rng.choice(["consumer", "business"], n),
    "quantity": rng.integers(1, 6, n),
    "amount": rng.uniform(50, 2500, n).round(2),
    "discount": rng.choice([0.0, 0.05, 0.1, 0.2], n),
}).assign(
    price=lambda d: (d["amount"] / d["quantity"]).round(2),
    profit=lambda d: (d["amount"] * rng.uniform(0.05, 0.3, n)).round(2),
).assign(
    ship_date=lambda d: d["order_date"] + pd.Timedelta(days=2),
    status=lambda d: np.where(d["amount"] > 2000, "priority", "standard"),
).to_csv("sales_data.csv", index=False)

df = pd.read_csv("sales_data.csv", parse_dates=["order_date"])

# Replace with constant (useful for counts)
df["quantity"] = df["quantity"].fillna(0)

# Replace with mean
df["amount"] = df["amount"].fillna(df["amount"].mean())

# Replace with median (better for skewed data)
df["profit"] = df["profit"].fillna(df["profit"].median())

2. Smart Group-wise Imputation (Best Practice)

# Fill missing amounts with the mean of their respective region
df["amount"] = df.groupby("region")["amount"].transform(lambda x: x.fillna(x.mean()))

# Fill missing values with the median of their category
df["price"] = df.groupby("category")["price"].transform(lambda x: x.fillna(x.median()))

3. Advanced Imputation Strategies

# Forward fill for time series data
df["amount"] = df["amount"].ffill()

# Backward fill as fallback
df["amount"] = df["amount"].bfill()

# Fill with different values per column
values = {
    "amount": df["amount"].median(),
    "quantity": 0,
    "region": df["region"].mode()[0]
}
df = df.fillna(value=values)

4. Best Practices in 2026

• Use group-wise imputation (groupby().transform()) when missingness depends on categories
• Use median instead of mean for skewed numerical columns
• Use fillna(0) only when missing truly means zero
• For time series data, consider ffill or interpolation
• Always document your imputation strategy and compare results before/after imputation
• Consider advanced methods (KNN, MICE) for complex cases with many missing values

Imputation Choices

Imputation replaces missing values with estimates. Constants suit categories, the median suits skewed numbers, group-wise means use related rows, and forward fill or interpolation suit ordered time series.

Every imputed value is a guess, so add a flag column marking which values were filled. Downstream models and readers can then tell real values from estimates.

Worked Example: Group Means with a Flag

Prices are filled with the mean of their own category, and the fill is recorded.

import pandas as pd
import numpy as np

df = pd.DataFrame({
    "category": ["Books", "Books", "Home", "Home", "Home"],
    "price": [12.0, np.nan, 40.0, 44.0, np.nan],
})

df["price_imputed"] = df["price"].isna()
df["price"] = df["price"].fillna(df.groupby("category")["price"].transform("mean"))
print(df)

ts = pd.Series([1.0, np.nan, np.nan, 4.0])
print(ts.interpolate().tolist())

Each missing price takes the mean of its own category, so the Home gap becomes 42.0 rather than an overall average. Linear interpolation fills the series gaps with 2.0 and 3.0.

Practical Tips

Always compare summary statistics before and after imputation. If the mean, spread or group differences change noticeably, the chosen method is shaping your results, and a simpler or more cautious approach may be safer.

Common Mistakes

• Filling skewed data with the mean instead of the median.
• Ignoring group structure when filling.
• Not flagging which values were imputed.
• Filling before splitting training and test data.

Quick Summary

• Choose the fill that matches the data type.
• Use transform for group-wise fills.
• Use interpolate() or ffill() for ordered data.
• Flag imputed values in a separate column.

Conclusion

Replacing missing values intelligently is often better than dropping them. In 2026, the most effective approach is group-wise imputation using groupby().transform(), combined with thoughtful choice of mean vs median based on data distribution. Always document your strategy and validate that imputation doesn't introduce unwanted bias into your analysis.

Next steps:

• Analyze the missing values in your dataset and apply appropriate imputation techniques (constant, mean, median, or group-wise) based on each column's characteristics

Related Tutorials

• Removing Missing Values in Pandas – When and How to Use dropna() 2026
• Handling Missing Values in Pandas