Problem Solved

Python · yield

Home › Courses › Python › Problems › yield
🐍 Concept · 4 min read

What does yield do?
Not return. Pause.

yield turns a function into a generator — a function that produces a sequence of values one at a time, pausing between each. It's not return. return exits the function and destroys its state. yield pauses the function and preserves its state, so the next call can resume from exactly where it left off. This page covers what a generator is, how yield differs from return, and the cases where generators actually matter.

Short answer: yield pauses a function and returns a value, keeping its state alive. The next call to next() resumes from that point. A function with yield is a generator function — calling it returns a generator, not a value. Generators are lazy — they produce one item at a time, using O(1) memory regardless of sequence length. Use them for large datasets, infinite sequences, and pipelines. They can only be iterated once.

✅ The difference from return

One exits. One pauses.

yield-vs-return.py
# Function with return — exits at the first return
def returns():
print('start')
return 1
print('never printed') # unreachable

returns()
start
(returns 1)

Generator with yield — pauses at each yield
def generates():
print('start')
yield 1
print('middle')
yield 2
print('end')
yield 3

gen = generates()
print(type(gen)) # 

Calling the function does NOT run the body.
The body runs when you call next().
print(next(gen))
start
1

print(next(gen))
middle
2

print(next(gen))
end
3

Fourth call raises StopIteration
next(gen) # StopIteration
Step by step:

gen = generates()
→ The body does NOT run.
→ Returns a generator object.
→ No 'start' printed yet.

print(next(gen))
→ Runs until the first yield.
→ Prints 'start'.
→ Yields 1.
→ Pauses. Saves state.
→ next() returns 1.

print(next(gen))
→ Resumes from after the first yield.
→ Prints 'middle'.
→ Yields 2.
→ Pauses.
→ next() returns 2.

print(next(gen))
→ Resumes.
→ Prints 'end'.
→ Yields 3.
→ Pauses.
→ next() returns 3.

next(gen) (4th time)
→ Resumes from after the third yield.
→ Function body ends.
→ Raises StopIteration.

Key insight:
Calling generates() does not run the body.
Calling next(gen) does, one step at a time.

This is why generators are called "lazy."
They don't compute anything until you ask.

Compare to return:
returns() runs the whole body.
Only the first return runs.
The rest is unreachable.

A function with any yield becomes a generator
function. Calling it returns a generator,
not the yielded value. Even if the yield is
inside an if branch that never executes.

def f():
if False:
yield 1
return 42

f() # returns , not 42

To get 42, you'd have to iterate the generator
or access its StopIteration.value.

✅ Why use generators

Memory, laziness, and streaming.

why-generators.py
# ❌ List — builds everything in memory
def first_n_list(n):
result = []
for i in range(n):
result.append(i * i)
return result

10 million integers: ~400 MB of memory
numbers = first_n_list(10_000_000)

✅ Generator — one item at a time
def first_n_gen(n):
for i in range(n):
yield i * i

10 million integers: ~constant memory
numbers = first_n_gen(10_000_000)

Memory: O(1) regardless of n.
Time to first value: instant.

Lazy processing — take only what you need
def read_large_file(path):
with open(path) as f:
for line in f:
yield line.strip()

Process 10 GB file without loading it
for line in read_large_file('huge.log'):
if 'ERROR' in line:
print(line)
break # only read until the first error
# The rest of the file is never read.

Infinite sequences — impossible with a list
def naturals():
n = 0
while True:
yield n
n += 1

This would crash a list. A generator is fine.
for n in naturals():
if n > 100:
break
print(n)
Three cases where generators win:

1. Large datasets
❌ List: builds the whole thing in memory.
✅ Generator: one item at a time.

For 10 million integers:
List: ~400 MB
Generator: a few hundred bytes

The generator is O(1) in memory.
The list is O(n).

2. Lazy evaluation — partial consumption
❌ List: computes every item even if you
only need the first 3.
✅ Generator: computes only what you consume.

for item in gen:
if condition:
break
The rest of the sequence is never computed.

3. Streaming data
❌ List: must load the whole file / stream
before processing.
✅ Generator: process line by line, chunk by
chunk, as data arrives.

Useful for: log files, CSV parsing, network
responses, database cursors.

When NOT to use generators:

❌ You need random access (gen[5])
❌ You need len() (no len on generators)
❌ You need to iterate multiple times
❌ The sequence is small (list is simpler)
❌ You need to slice (gen[1:3] doesn't work)

For small data, a list is clearer.
For large or streaming data, a generator.
Use the right tool for the size.

⚠️ Generators can be iterated only once:
gen = (x for x in range(3))
list(gen) # [0, 1, 2]
list(gen) # [] — exhausted

To iterate again: gen = (x for x in range(3))
(create a new generator) or list(gen) to
cache the values.

✅ Generator expressions

One-line generators.

gen-expressions.py
# List comprehension — brackets
squares_list = [x * x for x in range(10)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Generator expression — parentheses
squares_gen = (x * x for x in range(10))


Same values, but:
- list builds all at once
- generator is lazy

When to use which:
✅ Use a list if you need to:
- iterate multiple times
- access by index
- check length
- pass to a function that needs a list
✅ Use a generator if:
- you're passing to sum, any, all, max, min
- the sequence is large
- you only need to iterate once

Common patterns:
total = sum(x * x for x in range(1000)) # generator
any_negative = any(x < 0 for x in nums) # generator
all_positive = all(x > 0 for x in nums) # generator
longest = max(len(s) for s in strings) # generator

The generator is faster here — no intermediate
list is built. Each function consumes the
generator one item at a time.

Nested generator expression
matrix = [[1, 2], [3, 4], [5, 6]]
flat = (x for row in matrix for x in row)
print(list(flat)) # [1, 2, 3, 4, 5, 6]
The syntax difference:

List: [expr for x in iterable]
Generator: (expr for x in iterable)

Same expression. Brackets vs parentheses.
Completely different behavior:

List: eager, builds all, can reuse
Generator: lazy, one at a time, one-shot

The performance win:

sum([x * x for x in range(10000)])
→ Builds a 10,000-item list
→ Sums it
→ Discards the list

sum(x * x for x in range(10000))
→ Iterates lazily
→ Sums each item as it's produced
→ No list is ever built

For 10,000 items, the difference is small.
For 10 million items, the generator uses
KB and the list uses 100+ MB.

Rule of thumb:
Use a generator expression when passing
directly to sum, any, all, max, min,
or a for loop that breaks.
Use a list when you need the list.

The parentheses rule:

Single argument — parens optional
sum(x for x in nums) # OK
sum((x for x in nums)) # OK, explicit

Multiple arguments — parens required
sum((x for x in nums), 0) # parens needed

No other argument — parens optional
for x in (x for x in nums): # parenthesized
...
for x in x for x in nums: # SyntaxError

Use parentheses when in doubt.
They're never wrong.

⚠️ Generator expressions are one-shot too:
gen = (x for x in range(3))
list(gen) # [0, 1, 2]
list(gen) # []

Same exhausted behavior as generator
functions.

Don't use a generator when you need to iterate twice. This is the most common generator bug. gen = (x for x in data), then sum(gen), then max(gen) — the second call returns 0 or raises ValueError, because the generator is already exhausted. A list doesn't have this problem: lst = [x for x in data], sum(lst), max(lst) — both work. If you need to iterate multiple times, convert to a list at the point of creation, or re-call the generator function. The tee function from itertools can duplicate a generator, but it buffers consumed items in memory — which defeats the purpose.

⚠️ Common mistakes

The generator mistakes that cost real time.

1. Forgetting the function is a generator

Calling generates() returns a generator, not a value. Fix: iterate it or use next().

2. Iterating a generator twice

It's exhausted after the first pass. Fix: convert to a list, or recreate.

3. Calling len() on a generator

No len — TypeError. Fix: sum(1 for _ in gen), or use a list.

4. Indexing a generator

gen[5] — TypeError. Fix: itertools.islice, or use a list.

5. Slicing a generator

gen[1:3] — TypeError. Fix: itertools.islice(gen, 1, 3).

6. Mixing yield and return value

Technically valid but rarely useful. Fix: use a class or a different design.

7. Using yield in async functions

Different construct — async for and async generators. Fix: use async def with yield, iterate with async for.

8. Generators for small data

Over-engineering. Fix: use a list for small sequences; generators are for large or streaming data.

❓ Frequently asked

What does yield do in Python?

yield turns a function into a generator — a function that produces a sequence of values one at a time, pausing between each. When you call a generator function, the body doesn't run immediately. Instead, you get a generator object. Each time you call next() on it, the function runs until it hits a yield, produces that value, and pauses. The next call resumes from exactly that point.

What is the difference between yield and return?

Three differences. (1) return exits the function permanently. yield pauses the function, which can be resumed later. (2) return produces one value and ends. yield can produce many values over time. (3) return destroys local state. yield preserves local state between calls. A function with any yield in its body becomes a generator function.

Why use generators instead of returning a list?

Memory efficiency and laziness. If you return a list of 10 million items, Python builds the whole list in memory. A generator produces one item at a time, using O(1) memory. Generators also start work immediately — the first item is available without waiting. The trade-off: generators can only be iterated once.

Why does my generator return nothing the second time?

Because generators are one-shot iterators. Once a generator has been exhausted, it's done — iterating it again produces nothing. Example: gen = (x for x in range(3)); list(gen) gives [0, 1, 2]; list(gen) gives []. If you need to iterate multiple times, either convert to a list, or call the generator function again.

What is a generator expression?

A one-line generator, with the same syntax as a list comprehension but using parentheses: (x * 2 for x in range(10)). It produces values lazily, same as a generator function. Use it when you're passing a sequence to a function that consumes it once — like sum(x for x in nums), any(x > 0 for x in nums), or max(len(s) for s in strings).

Can a generator return a value?

Technically yes, since Python 3.3 — a generator can have return value, and it becomes the StopIteration.value. Example: def gen(): yield 1; return 'done'. Calling next() twice: first gives 1, second raises StopIteration('done'). In practice, this is rarely useful. Generators are for producing values, not for returning a final result.

🔗 Continue learning

🗺️ Python learning path
The full roadmap from install to first project — every concept in order

Next in this series:

📦 List vs tuple
Mutable vs immutable, and why tuples can be dict keys
← All Python problems
Problem solved · Python