What is Iteration in Python – Understanding Iterables and Iterators for Data Science 2026

What is Iteration in Python – Understanding Iterables and Iterators for Data Science 2026

Iteration is one of the most fundamental concepts in Python and is used constantly in data science. Understanding the difference between iterables and iterators helps you write more efficient, memory-friendly, and Pythonic code when working with large datasets.

TL;DR — Key Concepts

• Iterable: Any object you can loop over (lists, tuples, strings, DataFrames, dictionaries, etc.)
• Iterator: An object that remembers its position and returns one value at a time using next()
• for loops work on iterables by automatically creating an iterator

1. Iterables vs Iterators – Simple Explanation

numbers = [1, 2, 3, 4, 5]          # This is an iterable

# You can loop over it multiple times
for n in numbers:
    print(n)

# Creating an iterator from an iterable
iterator = iter(numbers)           # Now it's an iterator

print(next(iterator))   # 1
print(next(iterator))   # 2
print(next(iterator))   # 3

# Once exhausted, it raises StopIteration

2. Real-World Data Science Examples

import pandas as pd

# Create a small demo file so this example runs as-is
# (in your project, sales_data.csv is your real export)
import numpy as np
rng = np.random.default_rng(42)
n = 60
pd.DataFrame({
    "order_id": range(1001, 1001 + n),
    "order_date": pd.date_range("2026-01-01", periods=n, freq="3D"),
    "customer_id": rng.integers(101, 121, n),
    "customer_name": rng.choice([" ana", "ben ", "chen", "dana"], n),
    "product_id": rng.integers(1, 9, n),
    "region": rng.choice(["North", "South", "East", "West"], n),
    "category": rng.choice(["Electronics", "Computers", "Books", "Home"], n),
    "payment_method": rng.choice(["card", "cash", "wallet"], n),
    "segment": rng.choice(["consumer", "business"], n),
    "quantity": rng.integers(1, 6, n),
    "amount": rng.uniform(50, 2500, n).round(2),
    "discount": rng.choice([0.0, 0.05, 0.1, 0.2], n),
}).assign(
    price=lambda d: (d["amount"] / d["quantity"]).round(2),
    profit=lambda d: (d["amount"] * rng.uniform(0.05, 0.3, n)).round(2),
).assign(
    ship_date=lambda d: d["order_date"] + pd.Timedelta(days=2),
    status=lambda d: np.where(d["amount"] > 2000, "priority", "standard"),
).to_csv("sales_data.csv", index=False)

df = pd.read_csv("sales_data.csv")

# 1. Iterating over DataFrame rows (not recommended for large data)
for index, row in df.iterrows():           # iterrows() returns an iterator
    if row["amount"] > 1000:
        print(f"High value order: {row['customer_id']}")

# 2. Better: Using itertuples() - faster iterator
for row in df.itertuples():
    if row.amount > 1000:
        print(f"High value: {row.customer_id}")

# 3. Iterating over column names (iterable)
for col in df.columns:
    print(f"Processing column: {col}")

3. Best Practices for Iteration in Data Science 2026

• Avoid iterrows() on large DataFrames — use itertuples() or vectorized operations instead
• Use for loops with iterables for readability
• Use generators (yield) when working with very large datasets to save memory
• Prefer built-in functions like enumerate(), zip(), and map() over manual iteration
• Understand that many Pandas methods (groupby, resample, etc.) return iterators internally

Iterables and Iterators

An iterable is any object you can loop over, such as a list, string, dictionary, file or DataFrame column. An iterator is the object that actually produces the values one at a time; iter() creates it from an iterable and next() asks it for the next value.

A for loop does this automatically: it calls iter() once and next() repeatedly until a StopIteration exception signals the end. Iterators are consumed as they go, so a second loop over the same iterator produces nothing.

This model explains memory-efficient tools such as file objects, generators and pandas chunked readers, which create values only when asked.

Worked Example: A for Loop by Hand

We reproduce what a for loop does internally and show that an iterator is used up.

values = [3, 5, 8]

it = iter(values)
while True:
    try:
        item = next(it)
    except StopIteration:
        break
    print("got", item)

again = iter(values)
print(list(again))
print(list(again))

print(hasattr(values, "__next__"), hasattr(iter(values), "__next__"))

The manual loop prints the same values a for loop would. The second list(again) is empty because the iterator was already consumed, while the original list can be iterated as often as needed. Only the iterator has a __next__ method.

Common Mistakes

• Looping twice over an iterator and expecting the same values.
• Calling next() on a list instead of on iter(list).
• Converting large iterators to lists without need.
• Forgetting that StopIteration marks the end.

Quick Summary

• Iterables can be looped over; iterators produce values.
• iter() creates an iterator; next() advances it.
• Iterators are consumed once.
• Lazy iterators save memory on large data.

Conclusion

Iteration is everywhere in data science — from looping through rows and columns to processing large files and model training. In 2026, the key is to understand the difference between iterables and iterators and to choose the most efficient method for each situation. Writing clean, memory-efficient iteration code is a hallmark of experienced data scientists.

Next steps:

• Review your current loops over DataFrames and replace slow iterrows() with faster alternatives like itertuples() or vectorized operations

Related Tutorials

• Iterators vs Iterables in Python – Essential Concepts for Data Science 2026
• Iterating Over Iterables with next() – Understanding Iterators in Data Science 2026