Elite Coding Interview Mastery Program
A complete, lab-based, phase-structured training system to take you from beginner/intermediate to a candidate who can succeed at any coding interview — FAANG, infrastructure companies, distributed systems teams, compilers/runtimes, quant/HFT, staff/principal practical coding, and competitive-programming-style interviews.
This is not a roadmap. It is a training system: every phase has concept docs, runtime docs, hands-on labs in a fixed format, mock interviews with rubrics, failure analysis, and spaced repetition.
How To Use This Program
- Pick a track in schedules/ — Accelerated (12 weeks), Serious (6 months), or Elite (12 months).
- Read the universal framework first — FRAMEWORK.md. Use it on every problem.
- Work through phases in order — they have dependencies. Do not skip Phase 0 or Phase 1.
- For every problem, fill out REVIEW_TEMPLATE.md and schedule revisits via SPACED_REPETITION.md.
- When you fail, run the diagnosis in FAILURE_ANALYSIS.md. Do not skip this.
- Mock interview weekly minimum — see phase-11-mock-interviews/.
- Graduate only when READINESS_CHECKLIST.md is fully passed.
Global Framework Documents
| Document | Purpose |
|---|---|
| FRAMEWORK.md | The 16-step universal problem-solving framework + 11-step stuck protocol |
| COMMUNICATION.md | What to say (and not say) during an interview, with sample phrases |
| CODE_QUALITY.md | Quality bar for interview code, with bad-vs-good comparisons |
| REVIEW_TEMPLATE.md | Per-problem review template (use after every solve) |
| FAILURE_ANALYSIS.md | 16-category failure taxonomy with diagnosis, fix, drill, re-test |
| SPACED_REPETITION.md | 6-tier review intervals + per-tier protocol |
| READINESS_CHECKLIST.md | Final binary checklist before going to interviews |
Schedules
| Track | Audience | Doc |
|---|---|---|
| 12-Week Accelerated | Urgent prep, 1 deadline coming up | schedules/12_WEEK_ACCELERATED.md |
| 6-Month Serious | Strong Big Tech readiness | schedules/6_MONTH_SERIOUS.md |
| 12-Month Elite | Top-tier, senior/staff, competitive-heavy | schedules/12_MONTH_ELITE.md |
Phases
| # | Phase | Target Level | Folder |
|---|---|---|---|
| 0 | Interview Execution Baseline | Beginner → Easy | phase-00-execution-baseline/ |
| 1 | Programming & Data Structure Foundations | Easy → Medium | phase-01-foundations/ |
| 2 | Standard Coding Interview Patterns | Medium → Medium-Hard | phase-02-patterns/ |
| 3 | Advanced Data Structures | Medium-Hard → Hard | phase-03-advanced-ds/ |
| 4 | Graph Mastery | Medium-Hard → Hard | phase-04-graphs/ |
| 5 | Dynamic Programming (Basic → Extreme) | Medium → Very Hard | phase-05-dp/ |
| 6 | Greedy, Proofs & Mathematical Thinking | Medium-Hard → Hard | phase-06-greedy/ |
| 7 | Competitive Programming Acceleration | Hard → CP-Hard | phase-07-competitive/ |
| 8 | Practical Engineering Coding Interviews | Medium-Hard → Hard | phase-08-practical-engineering/ |
| 9 | Language & Runtime Deep Dive | All levels (cross-cutting) | phase-09-language-runtime/ |
| 10 | Testing, Debugging & Correctness | All levels (cross-cutting) | phase-10-testing-debugging/ |
| 11 | Mock Interview Mastery | Beginner → Grandmaster | phase-11-mock-interviews/ |
| 12 | Grandmaster / Final Boss | CP-Hard → Grandmaster | phase-12-grandmaster/ |
| 13 | Frontier: Advanced Search & Large-Number Algorithms | Grandmaster+ (optional) | phase-13-frontier/ |
Difficulty Ladder
| Level | What You Solve | Solving Time | Failure Means | When To Move Up |
|---|---|---|---|---|
| Beginner | Trivial array/string traversals | 5–10 min | Confusion about loops/indexing | After 30 problems with 0 confusion |
| Easy | LeetCode Easy | 10–15 min | Wrong brute force, syntax errors | 90% solved <15 min for 50 problems |
| Medium | LeetCode Medium | 25–35 min | Missed pattern, bad complexity | 75% solved <35 min for 100 problems |
| Medium-Hard | Top-100 Mediums, easy Hards | 30–45 min | Couldn’t optimize past brute force | 70% solved <45 min for 60 problems |
| Hard | LeetCode Hard | 45–75 min | Failed to find any non-trivial approach | 50% solved <60 min for 50 problems |
| Very Hard | LC Hard tagged “very hard”, Codeforces 1900–2100 | 60–120 min | Conceptual gaps in algorithm class | 40% solved unaided for 30 problems |
| CP Hard | Codeforces 2100–2400, AtCoder ARC F | 90–180 min | Missing a CP-specific technique | Solving consistently in contest |
| Grandmaster | CF 2400+, AGC, ICPC WF | Open-ended | Even with hints, can’t make progress | When you stop needing this curriculum |
For each level: failure is normal. The value is in the review process and the failure analysis, not the original solve.
When to repeat a level: if your unaided success rate is below the threshold above, do another 30–50 problems at the same level before moving up. Moving up too early calcifies bad habits.
Progress Tracking
Use a single spreadsheet/journal with columns:
- Date
- Phase / Pattern
- Problem name + source
- Solved unaided? (Y/N)
- Time spent
- Got it on first attempt? (Y/N)
- Why missed (if missed) — link to FAILURE_ANALYSIS.md category
- Next review date — from SPACED_REPETITION.md
Without tracking, the spaced repetition system collapses. Without spaced repetition, knowledge decays faster than you accumulate it.
A Note On Honesty
This curriculum cannot guarantee outcomes. What it guarantees is:
- If you complete every phase honestly (failed problems reviewed, mocks done with full effort, no shortcuts), you will recognize > 95% of common interview problem patterns.
- You will be able to solve unfamiliar problems — because the framework forces you to derive solutions, not memorize them.
- You will fail mock interviews repeatedly and learn from each one. This is the entire point.
- You will know your own weaknesses precisely, and which phase to revisit.
Skip the failure analysis, skip the reviews, skip the mocks — and the curriculum becomes a list of topics. Topics don’t get you hired.
📖 Published · commit
4521719· 2026-07-13 02:17 UTC
Universal Problem-Solving Framework
Use this on every problem. It is non-optional. The goal is to make solving deterministic, not heroic.
The 16-Step Framework
1. Restate The Problem
Say it back in your own words. If you can’t restate, you don’t understand. Ambiguous parts surface here.
2. Ask Clarifying Questions
- Input type, range, sign, precision
- Are inputs sorted? Distinct? Can be empty/null?
- Output format. One answer or all answers?
- Are duplicates allowed?
- Can the input mutate? Is it streamed?
- What should happen on invalid input?
- Constraints not given?
3. Identify Constraints
Constraints dictate the algorithm. Memorize this table:
| N | Acceptable Complexity | Likely Approach |
|---|---|---|
| ≤ 10 | O(N!) or O(2^N · N) | Backtracking, bitmask brute force |
| ≤ 20 | O(2^N · N) | Bitmask DP, meet-in-the-middle |
| ≤ 100 | O(N^4) | Multi-loop brute, Floyd-Warshall |
| ≤ 500 | O(N^3) | Interval DP, matrix chain |
| ≤ 5,000 | O(N^2) | 2D DP, edit distance |
| ≤ 100,000 | O(N log N) or O(N √N) | Sort + scan, segment tree, sqrt decomp |
| ≤ 1,000,000 | O(N) or O(N log N) | Linear scan, hashmap, two pointers |
| ≤ 10^8 | O(N) or O(log N) | Math closed form, binary search on answer |
| ≤ 10^18 | O(log N) | Binary exponentiation, math |
4. Work Through Examples
Use the given example. Then build at least two more:
- A trivial case (size 1, empty)
- An adversarial case (max constraints, all duplicates, all negative, sorted descending)
Work them by hand. Annotate intermediate states. This often reveals the pattern.
5. Identify Brute Force
What is the dumbest correct solution? Write it down (pseudocode). Don’t skip this even if you “see” the optimal — the brute force is your correctness oracle for stress testing.
6. Analyze Brute Force Complexity
Time and space, in big-O. If it fits the constraints, you may be done. Often it does for small N.
7. Recognize Patterns
Run through the pattern checklist in phase-02-patterns/:
- Sorted or sortable? → two pointers, binary search
- Asks for max/min over windows? → sliding window, monotonic deque
- “Subarray with property X”? → prefix sum, sliding window
- “K-th something”? → heap, quickselect
- “Number of ways”? → DP, combinatorics
- “Shortest path”? → BFS / Dijkstra / 0-1 BFS
- “Connected components / cycles / dependencies”? → union-find / DFS / topo sort
- “Decision: can we do X with budget Y?” → binary search on answer
- “Optimal sequence with overlapping subproblems”? → DP
8. Derive Optimized Approach
Reduce repeated work. Cache, sort, hash, prune, transform. State the invariant of your approach.
9. Prove Correctness
- For greedy: exchange argument or cut property.
- For DP: state definition + transition + base cases + evaluation order.
- For graphs: cite the algorithm’s correctness theorem and verify preconditions.
- For two-pointer/sliding window: the loop invariant.
10. Write Clean Code
- Meaningful names
- Single-responsibility functions
- No premature abstraction
- Avoid mutating function parameters unless intentional
- Match the patterns of the language idiom
11. Test Smoke Cases
Walk the given examples through your code by hand. Don’t run yet. Find bugs before silicon does.
12. Test Edge Cases
- Empty / null
- Size 1 / size 2
- All duplicates
- All negative / mixed signs
- Sorted ascending / descending
- Max constraint values (overflow risk)
- Multiple valid answers (specify which one)
- Disconnected graph / cycle
- Concurrent access (where relevant)
13. Test Large Cases
Confirm complexity by reasoning, not by running. Will N=10^6 pass in 1 second? In your language’s runtime?
14. Explain Complexity
State time and space. State whether the bound is tight. Mention amortized vs worst-case if relevant. State assumptions (hash table O(1) average is an assumption).
15. Handle Follow-Ups
Anticipate. Most interviews follow up with one of:
- “What if the input doesn’t fit in memory?” → streaming, external sort, sketches
- “What if it’s distributed?” → sharding, consistent hashing
- “What if reads >> writes?” → caching, replicas
- “What if writes >> reads?” → log-structured, write-back
- “What if we need approximate answers?” → Bloom, HLL, count-min
- “How would you test this?” → unit + property + stress + concurrency
- “How would you debug a production failure?” → logs + metrics + repro
16. Discuss Production Implications
For practical engineering interviews: monitoring, logging, metrics, partial failure, backpressure, retries, idempotency, observability, deployment.
The Stuck Protocol
When you’ve been silent for >2 minutes, or have no progress for 5 minutes, switch into this mode. Do not freeze. Do not flail.
1. Restate What Is Known
Out loud. “I have an array of N integers, I want the longest subarray such that…”
2. Write Brute Force
Even if it’s O(N^4) and you know it won’t pass. Brute force gives you:
- A correctness oracle
- A starting point to optimize from
- Intermediate state to inspect for patterns
3. Inspect Constraints Again
Have you forgotten one? Often the constraint is the hint. N ≤ 20 screams bitmask. K ≤ 10 screams “K is in the state”.
4. Try Smaller Examples
Solve N=1, N=2, N=3 by hand. Patterns emerge. Often the recurrence falls out.
5. Look For Repeated Work
In the brute force, what’s recomputed? That’s your DP state or memoization target.
6. Look For Monotonicity
- Is there a value over which the answer is monotonic? → binary search on answer
- Is there a window whose property is monotonic? → sliding window, monotonic stack/deque
7. Look For Graph Modeling
Words like “depends on”, “leads to”, “transitions”, “groups”, “components”, “blocked by” all suggest graphs. Try modeling explicitly.
8. Look For DP State
Ask: “What information do I need at position i to decide what’s optimal going forward?” That information is the state.
9. Look For Greedy Invariant
Ask: “If I make the locally best choice, can I prove I never need to undo it?” If yes, greedy. If no, DP.
10. Ask For A Small Hint Professionally
Sample phrases:
- “I’ve considered X and Y but I’m having trouble seeing the structure. Could you nudge me toward the right family of approach?”
- “Is the input small enough that exponential is acceptable, or are we targeting polynomial?”
A well-asked hint costs you almost nothing. A 10-minute silence is fatal.
11. Recover And Continue
Take the hint, restate the new constraint or insight, commit out loud to a direction, and resume coding. Don’t apologize repeatedly. Move forward.
A Note On Discipline
This framework feels slow for the first 50 problems. By problem 200, you’ll execute steps 1–9 in 4 minutes flat. By problem 500, the framework runs subconsciously and you’ll only consciously invoke it when stuck.
The goal is not to memorize the framework. It’s to internalize it so deeply that when you read a problem, your brain runs steps 1–9 automatically.
Candidates who skip the framework “to save time” lose interviews to candidates who don’t, because the framework users:
- Catch ambiguity before they code
- Get the right complexity on the first attempt
- Don’t waste minutes coding the wrong thing
- Communicate clearly throughout
- Know what to test before they finish coding
Code Quality Standards For Interviews
Interview code is judged differently from production code. The bar is:
- Correct above all else
- Readable — the interviewer should follow it without you explaining every line
- Simple — the simplest solution that works, not the cleverest
- Defensive only at boundaries — validate inputs once, then trust them
- Testable — pure functions and clear data flow
Interview code is not:
- Production-ready (no logging, no metrics, no retries unless asked)
- Heavily commented (good names beat comments)
- Premature-abstraction (no factory factories)
- Defensive everywhere (validating inside hot loops is noise)
The Quality Dimensions
| Dimension | What “Good” Looks Like |
|---|---|
| Correctness | Handles every edge case the problem allows |
| Simplicity | No clever tricks unless required for complexity |
| Readability | A peer can read it once and understand |
| Naming | parent, visited, frequency — not p, v, f (except in trivial scopes) |
| Modularity | Helper functions for distinct logical units |
| Boundary handling | Empty/null/overflow checked once at the entry |
| No premature abstraction | One-time logic stays inline |
| No overengineering | Don’t build a config system for a 30-line problem |
| No hidden state | Globals/singletons are red flags |
| Minimal mutation | Prefer immutable returns where natural |
| No excessive cleverness | One-liners that need a paragraph to explain are anti-signal |
| Standard library use | Use the language’s built-ins idiomatically |
| Testability | Logic separated from I/O, deterministic |
Bad vs Good Examples
Naming
Bad:
def f(a, b):
r = []
for x in a:
if x > b:
r.append(x)
return r
Good:
def values_above(numbers, threshold):
return [n for n in numbers if n > threshold]
Boundary Handling
Bad (validates in every iteration):
def sum_positive(nums):
total = 0
for n in nums:
if nums is None or len(nums) == 0: # checked every iteration
return 0
if n > 0:
total += n
return total
Good (validate once at the boundary):
def sum_positive(nums):
if not nums:
return 0
return sum(n for n in nums if n > 0)
Excessive Cleverness
Bad (one-liner, hard to debug):
def has_duplicate(nums):
return len(nums) != len({*nums}) if nums else False
Good (clear intent):
def has_duplicate(nums):
seen = set()
for n in nums:
if n in seen:
return True
seen.add(n)
return False
The good version also has the advantage of early termination — better complexity in practice.
Helper Functions
Bad (everything in one 50-line function):
def shortest_path(grid, start, end):
# 50 lines of BFS, neighbor computation, distance tracking, all inline
...
Good (extract neighbor logic):
def shortest_path(grid, start, end):
queue = deque([(start, 0)])
visited = {start}
while queue:
pos, dist = queue.popleft()
if pos == end:
return dist
for nxt in neighbors(grid, pos):
if nxt not in visited:
visited.add(nxt)
queue.append((nxt, dist + 1))
return -1
def neighbors(grid, pos):
r, c = pos
rows, cols = len(grid), len(grid[0])
for dr, dc in ((-1, 0), (1, 0), (0, -1), (0, 1)):
nr, nc = r + dr, c + dc
if 0 <= nr < rows and 0 <= nc < cols and grid[nr][nc] != '#':
yield (nr, nc)
Premature Abstraction
Bad (over-engineered for a single use):
class CounterStrategy:
def count(self, items):
raise NotImplementedError
class HashCounterStrategy(CounterStrategy):
def count(self, items):
d = {}
for x in items:
d[x] = d.get(x, 0) + 1
return d
def majority_element(nums):
counts = HashCounterStrategy().count(nums)
return max(counts, key=counts.get)
Good:
def majority_element(nums):
counts = Counter(nums)
return counts.most_common(1)[0][0]
Mutation
Bad (mutates input):
def normalized(values):
for i in range(len(values)):
values[i] = values[i] / max(values) # also recomputes max each iteration
return values
Good (no mutation, single max computation):
def normalized(values):
if not values:
return []
peak = max(values)
return [v / peak for v in values]
Hidden State
Bad (global counter):
_call_count = 0
def fib(n):
global _call_count
_call_count += 1
if n < 2:
return n
return fib(n - 1) + fib(n - 2)
Good (state passed explicitly):
@lru_cache(maxsize=None)
def fib(n):
if n < 2:
return n
return fib(n - 1) + fib(n - 2)
Language-Idiomatic Code
You should write code that looks like it was written by someone fluent in the language. Examples:
Python
- Use list/dict/set comprehensions where natural
- Use
Counter,defaultdict,deque,heapq,bisect - Use
enumerate,zip, unpacking - Avoid C-style
for i in range(len(x))if you only need values
Java
- Use enhanced for-loop where possible
- Use
Map.computeIfAbsent,Map.getOrDefault - Use
Optionalonly where it fits the API; not for short-circuit logic - Prefer
ArrayDequeoverStack(legacy)
Go
- Prefer slices over arrays
- Use
for rangefor both index and value - Return errors explicitly; don’t panic in interview code
- Buffered channels only when justified
C++
- Use
autowhere it improves readability - Range-based for-loops
- Prefer
std::vectorandstd::unordered_map - Use
emplace_backoverpush_backfor non-trivial types constreferences for non-trivial inputs
JavaScript/TypeScript
- Use
Map/Set(not{}/array hacks) when keys aren’t strings or order matters constby default,letonly when reassignment is needed- Avoid
var - TS: prefer
unknownoveranyat boundaries
Comments
- Avoid: comments that restate what the code says (
// increment i). - Prefer: comments that explain why — the non-obvious tradeoff, the invariant, the reason a less elegant approach was chosen.
- Required: a one-line comment above any non-trivial recurrence/invariant. Example:
# dp[i][j] = max profit with i transactions ending at day j.
Length
For most coding interview problems:
- Easy: 10–25 lines
- Medium: 20–50 lines
- Hard: 30–80 lines
If your Easy is 80 lines, you’re overengineering. If your Hard is 200 lines, you’ve gone wrong somewhere — re-examine the approach.
Final Self-Review Checklist
Before saying “I’m done”:
- ☐ Function/variable names are meaningful
- ☐ No dead code, no commented-out blocks
- ☐ Boundary checks at the entry, not in the hot loop
- ☐ Helper functions for distinct logical units
- ☐ No mutation of input unless intentional and stated
- ☐ No globals introduced
- ☐ Standard library used idiomatically
- ☐ Indentation/formatting consistent
- ☐ Code I would be willing to send a colleague for review
Interview Communication Rules
Coding interviews are collaborative problem-solving sessions, not exams. Half the signal an interviewer collects is how you communicate. A correct silent solution often scores worse than a slightly imperfect one with strong communication.
Core Principles
- Narrate continuously, but not constantly. Your mouth does not need to track your fingers character-by-character. It tracks your intent.
- Show your reasoning, not just your conclusion. “I’m choosing a hashmap because we need O(1) lookup by ID” beats silently typing
Map<String, Foo>. - Two-way street. Pause for confirmation at decision points. The interviewer wants to be involved.
- Hide nothing. If you’re unsure, say so. If you don’t remember an API, say so. Hiding looks worse than admitting.
- Forward motion always. Even when stuck, narrate progress: “I’m now going to try a smaller example” is forward motion.
Phase 1 — Opening (first 1–3 minutes)
What to do
- Read the problem fully (don’t start coding)
- Restate it
- Ask clarifying questions
- Confirm constraints
Sample phrases
“Let me read this through once first… OK. So if I understand correctly, I need to [restate]. Is that right?”
“Before I start, can I confirm a few things about the input? Specifically: can the array be empty? Can it contain negative numbers? Are there duplicates?”
“What’s the expected size of N here? Around 10^5? OK, so we’re targeting O(N log N) or better.”
“If there are multiple valid outputs, can I return any one, or is there a specific one expected?”
Anti-patterns
- Starting to code immediately
- Asking 15 questions in a row (drip-feed them as they become relevant)
- Asking questions whose answers are obvious from the problem statement (signals you didn’t read carefully)
Phase 2 — Brute Force (1–3 minutes)
What to do
- State the dumbest correct solution
- Compute its complexity
- Confirm whether it’s acceptable
Sample phrases
“Let me start with the brute force just to anchor. I could check every pair, which would be O(N^2). Given N is up to 10^5, that’s 10^10 operations — too slow. So we need something better.”
“The naive solution is O(2^N) — that’s fine for N=20 but won’t scale. Let me see if we can do better.”
“If we sort first, that gives us O(N log N) for the sort, and then… let me think about the scan.”
Anti-patterns
- Skipping brute force (“I know the optimal already”)
- Computing brute force complexity wrong (always double-check)
- Stating brute force without saying whether it suffices
Phase 3 — Optimization (3–10 minutes)
What to do
- Think out loud
- State observations
- Propose ideas and evaluate them
- Commit to one direction
- Sanity-check before coding
Sample phrases
“I notice the array is sorted, which means I can probably use two pointers…”
“The fact that K is small — only up to 10 — suggests we might want K in our DP state.”
“I see a pattern of repeated subproblems here. Let me define a state: dp[i][j] = …”
“Let me try a small example to verify my recurrence…”
“OK, I think my approach is: sort by start time, then greedily pick. Let me convince myself this works with an exchange argument…”
Anti-patterns
- Switching ideas every 30 seconds without exploring any
- Coding before stating the approach
- Not verifying correctness on a small example
Phase 4 — Coding (10–25 minutes)
What to do
- Narrate intent at function/block boundaries
- Explain non-obvious choices
- Stay quiet during routine syntax
- Ask “is the API I’m assuming OK?” if uncertain
Sample phrases
“I’m going to use a heap here, in Python that’s heapq. Pushing tuples to break ties by ID.”
“For the visited set, I’ll use a hash set rather than a 2D boolean array, since we’re not sure of the bounds.”
“I’m using a sentinel value of -1 to mean ‘not yet computed’ in the memoization.”
“Let me extract this into a helper function — it’ll make the recursion cleaner.”
Anti-patterns
- Silent typing for 5+ minutes
- Constant low-level narration (“now I’m typing a for loop”)
- Going down a rabbit hole on language minutiae mid-flow
Phase 5 — Testing (3–5 minutes)
What to do
- Walk through the given example
- Walk through your own edge cases
- Catch and fix bugs before the interviewer points them out
Sample phrases
“Let me trace through example 1. Initial state: … after iteration 1: … after iteration 2: … final answer: 7. Matches the expected.”
“Edge cases: what if the array is empty? My code would… let me check… yes, it returns 0, which is correct.”
“What about all negatives? Hmm, my initialization assumes a non-negative max. Let me fix that — initialize to negative infinity.”
“I think there’s an off-by-one here. Let me re-examine the loop bound.”
Anti-patterns
- Skipping testing because “the code looks right”
- Testing only the happy path
- Defending broken code when a bug is pointed out (just fix it)
Phase 6 — Complexity & Follow-Ups (2–5 minutes)
What to do
- State time and space
- Mention assumptions
- Engage with follow-ups thoughtfully
Sample phrases
“Time complexity is O(N log N) — dominated by the sort. Space is O(N) for the auxiliary array, or O(1) if we sort in place.”
“Average case for the hashmap is O(1), but worst case is O(N) under adversarial hashing. In Python the dict implementation handles that reasonably well.”
“If we needed to scale this to 10 million users, I’d consider… [sharding / external sort / approximate counters / etc.]”
“If reads were much more frequent than writes, we might precompute and cache.”
Anti-patterns
- Claiming O(N) when it’s actually O(N log N)
- Forgetting space complexity
- Defensive answers to follow-ups (“I’d need more info” instead of engaging)
Handling Hints
If the interviewer gives a hint:
“Ah, that helps — so you’re saying we should [restate]. Let me adjust my approach…”
Then commit out loud to the new direction. Don’t second-guess. Take the hint, integrate it, move forward.
If you need a hint:
“I’ve explored [X] and [Y]. I’m having trouble seeing how to avoid the O(N^2) here. Could you nudge me toward the family of approach?”
A well-asked hint costs you almost nothing. Frozen silence is fatal.
Handling Mistakes
When you find your own bug:
“Hmm, let me re-examine this — I think there’s a bug at line N. Yes, the comparison should be < not ≤. Let me fix that.”
When the interviewer finds a bug:
“Oh, you’re right — the empty case isn’t handled. Let me add a guard.”
Never:
- Argue
- Explain why the bug is “not really a bug”
- Get flustered
Always:
- Acknowledge briefly
- Fix
- Move on
Handling Pressure / Freezing
If you blank:
“Give me 30 seconds to think.” (Then actually think — don’t fake it.)
Then run the stuck protocol. Out loud:
“OK, let me back up. What I know is [X]. The brute force was [Y]. The constraint that’s hinting at something is [Z]…”
Talking through the stuck protocol restarts your thinking and shows the interviewer you have a process for being stuck.
Closing Strong
Final 1–2 minutes:
“To summarize: I’m using [data structure] with [algorithm]. Time is O(…), space is O(…). Edge cases handled: empty, single element, duplicates, max constraints. The main risk in production would be [X], which I’d address by [Y].”
A clean summary leaves the interviewer with a tidy mental model of your work — much better than ending mid-test.
Body Language & Tone (Video / In-Person)
- Sit up. Look at the interviewer when explaining, at the editor when coding.
- Speak at moderate pace. Faster than normal = nervous, slower = padding.
- Avoid filler (“um”, “like”) — silence is preferable to filler.
- Don’t apologize repeatedly. One apology when you fix a bug is enough.
- Show interest in the problem. Curiosity is a positive signal.
Phrases To Avoid
- “This is easy” — even if it is, this can read as arrogant
- “I’ve seen this before” — be careful; if you go on autopilot you’ll miss the variant
- “I don’t know” with no follow-up — replace with “I don’t know X, but I can reason about it via Y”
- “That won’t work” without explanation
- “Just” used dismissively (“we just need to…”)
What Strong Communication Buys You
- Partial credit when your code is incomplete (the interviewer saw your reasoning)
- Hints offered earlier (interviewers want to help engaged candidates)
- Believability of follow-up answers (a candidate who reasoned clearly throughout is trusted on production tradeoffs)
- Hire signal even on hard problems you didn’t fully solve
Failure Analysis System
Every failed problem (or weak performance) maps to one or more failure categories. The category determines the drill that fixes it. Without this taxonomy, you’ll repeat the same mistake forever.
How To Use
After every miss:
- Identify the primary failure category (the root cause, not the symptom).
- Identify any secondary categories.
- Run the listed drill within 24 hours.
- Re-test on a similar problem within 1 week.
The 16 Failure Categories
1. Did Not Understand The Problem
Symptom: Solved a problem the interviewer didn’t ask. Root cause: Skipped restating, didn’t ask clarifying questions. Fix: Always restate. Always ask 3+ clarifying questions before coding. Drill: Take 10 LeetCode problems, do only steps 1–4 of FRAMEWORK.md (restate, clarify, identify constraints, examples). Don’t solve them. The drill is reading. Re-test: Mock interview with an interviewer who deliberately gives an ambiguous problem.
2. Missed Constraints
Symptom: Brute force passed at small N but TLE’d at full N. Or used 32-bit integer when sum exceeds 2^31. Root cause: Skipped step 3 of the framework. Fix: Make a habit: the moment you read a constraint, derive the target complexity out loud. Drill: Read 20 problems. For each, before reading the editorial, write down: target complexity, allowed N, integer width needed. Re-test: Solve 5 problems where the constraint is the hint (e.g., N≤20 → bitmask).
3. Could Not Find Brute Force
Symptom: Stared at the problem with no starting point. Root cause: Tried to find the optimal directly. Dangerous habit. Fix: Brute force is always possible. Iterate over every subset / pair / arrangement / state. State it even if it’s exponential. Drill: 10 problems. For each, write only the brute force in pseudocode. Don’t optimize. Re-test: Mock with a hard problem; goal is to communicate brute force in <3 minutes.
4. Could Not Optimize
Symptom: Wrote brute force, then froze. Root cause: No systematic optimization toolkit. Fix: Run the optimization checklist (step 7 of framework): pattern recognition, repeated-work elimination, monotonicity, sortedness exploitation, state compression, math. Drill: Take 10 brute-force solutions to known problems. For each, optimize without looking at the editorial using the checklist. Re-test: Solve 5 unfamiliar mediums in <30 min each.
5. Chose Wrong Data Structure
Symptom: Used a list where a set would have given O(1) lookup. Used a heap where a sorted array suffices. Used recursion where a stack would simplify. Root cause: Didn’t reason about access patterns. Fix: Before choosing a DS, list the operations you need and their frequency. Then pick the DS whose complexity matrix matches. Drill: phase-01-foundations/data-structures/ — reread the operations table for each DS. Then re-solve 5 problems consciously documenting why each DS was chosen. Re-test: Mock with explicit “why this DS?” follow-ups.
6. Bad Complexity
Symptom: Said O(N) when it was O(N log N). Or claimed O(1) on operations that are amortized. Root cause: Sloppy analysis, no habit of double-checking. Fix: State complexity and the basis. “O(N log N) because we sort, then linear scan” — not just “O(N log N)”. Drill: Take 20 of your own past solutions. Re-derive complexity. Compare to what you said. Re-test: Mock interviewer specifically grills on complexity.
7. Buggy Implementation
Symptom: Approach was right, but the code had off-by-ones, wrong operators, swapped variables. Root cause: Coding faster than thinking. Insufficient pre-coding clarity. Fix: Write pseudocode first. Trace through one example before running anything. Drill: Standard implementations: write binary search, BFS, DFS, union-find from scratch 10 times each. Use templates only after you can write them error-free. Re-test: Solve 5 problems with hand-traced verification before submission.
8. Weak Testing
Symptom: Submitted, got “wrong answer on test 7”. Hadn’t tested edge cases. Root cause: Skipped step 12 of the framework. Fix: Make the universal edge-case checklist a reflex. Before submission, run through it. Drill: Take 5 of your “wrong on hidden test” problems. Without looking at the test that failed, generate 10 edge cases for each. Re-test: Solve 5 problems and catch your own bug before submission. Score = problems where you fixed a bug pre-submission.
9. Poor Communication
Symptom: Mock interviewer said “I had no idea what you were thinking.” Root cause: Silent coding, or only narrating low-level mechanics. Fix: COMMUNICATION.md. Narrate intent at decision points. Drill: Solve 10 problems while recording yourself narrating. Listen back. Are you communicating decisions, or just typing aloud? Re-test: Mock with explicit communication scoring.
10. Froze Under Pressure
Symptom: Knew the technique in practice. Couldn’t access it in mock. Root cause: Insufficient mock volume. Fix: More mocks. Pressure tolerance is built only by exposure. Drill: 5 mocks in a week with a real human or pressure-simulating tool. Use the stuck protocol explicitly when you freeze. Re-test: Mock with cold problems. Goal: never go silent for >60 seconds.
11. Missed Edge Cases
Symptom: Solution was correct on examples but failed on size-1, empty, or max-constraint inputs. Root cause: Didn’t run the universal checklist. Fix: Build muscle memory for: empty / 1 / 2 / dup / negative / sorted / reversed / max / disconnected / cycle. Drill: Take 10 past solutions. For each, brainstorm 5 edge cases. Verify your code handles them. Re-test: Mock where edge cases are the gating criterion.
12. Runtime / Language Issue
Symptom: Code was algorithmically right but ran slow / crashed / behaved wrong due to language behavior. Examples: integer overflow in Java, dict iteration order bug, Python recursion limit, C++ undefined behavior. Root cause: Insufficient runtime depth. Fix: phase-09-language-runtime/ for your primary language. Drill: Read your language’s track end-to-end. Solve 5 problems specifically targeting the gotchas (e.g., overflow problems, recursion-depth problems). Re-test: Runtime-deep-dive mock (mock-09).
13. Concurrency Issue
Symptom: Race condition, deadlock, lost update, visibility bug. Root cause: Missing concurrency mental model. Fix: phase-09-language-runtime/ (concurrency sections) + phase-08-practical-engineering/ thread-pool / job-queue labs. Drill: Implement thread-pool, rate limiter, and producer-consumer queue from scratch with race-condition tests. Re-test: Concurrency-heavy mock (mock-11).
14. Overfit To Memorized Pattern
Symptom: Forced a known pattern that didn’t fit. E.g., applied sliding window to a problem that needed monotonic deque. Root cause: Pattern matching without understanding. Fix: Re-read pattern docs and focus on when the pattern does NOT apply. Drill: Take 10 problems. For each, list 3 patterns it might be, then eliminate 2 with reasoning. Re-test: Mock with problems specifically chosen to be near-misses of common patterns.
15. Did Not Prove Correctness
Symptom: Submitted a greedy that was wrong. Or a DP whose recurrence was incorrect. Root cause: Skipped step 9 of the framework. Fix: Force yourself to state the invariant or recurrence before coding. Drill: Take 10 greedy problems. For each, state the exchange argument. Take 10 DP problems. For each, state the recurrence + base case + evaluation order. Re-test: Solve 5 problems where the proof is the hard part.
16. Could Not Handle Follow-Up
Symptom: Solved the core problem, but interviewer’s follow-up (“how would this scale to 10M users?”) got a vague answer. Root cause: No production / system thinking. Fix: phase-08-practical-engineering/ labs all include the standard follow-up bank. Drill: Take 10 of your past solutions. For each, write a 2-paragraph answer to: “scale to 10M users”, “make distributed”, “handle partial failure”, “add observability”. Re-test: Senior-engineer mock (mock-07) which weighs follow-ups heavily.
Tracking Failures Over Time
Maintain a failures.md (or spreadsheet) with columns:
| Date | Problem | Primary Category | Secondary | Drill Done? | Re-test Date | Re-test Result |
|---|
After 4 weeks, count by category. Your top 3 categories are your personal weakness profile. Spend the next 4 weeks specifically drilling those.
This is how you avoid the “I keep failing at the same thing” trap.
Common Compound Failures
Some categories often co-occur. If you see these together, treat the deeper one:
- #1 + #11 (didn’t understand + missed edges) → really #1. Fix understanding first.
- #3 + #4 (no brute force + couldn’t optimize) → really #3. You can’t optimize what you don’t have.
- #7 + #8 (buggy + weak testing) → really #8. Better testing catches bugs.
- #5 + #6 (wrong DS + bad complexity) → really #5. Right DS gives right complexity.
- #10 + #9 (froze + bad communication) → really #9. Talking unfreezes you.
- #14 + #15 (overfit pattern + no proof) → really #15. Proving forces understanding.
When Failures Are Good
Some failures are productive:
- First attempt at a new pattern → expected to fail, the failure teaches.
- Difficulty stretch (jumping a level) → expected to fail 50%+ of the time.
- Mock interviews → 30%+ failure rate is normal and healthy.
Failures are bad when:
- You repeat the same category 5+ times without improvement.
- You stop tracking them.
- You don’t run the drill.
- You blame the problem (“that was unfair”) instead of analyzing yourself.
Final Readiness Checklist
You are ready for the interviews you’re targeting only when every box below is honestly checked. This is binary, not aspirational.
Algorithmic Solving
- Solve LeetCode Easy in <12 minutes (90% success rate over 50 recent problems)
- Solve LeetCode Medium in 25–35 minutes (75% success rate over 100 recent problems)
- Solve LeetCode Hard in 45–60 minutes (50% success rate over 50 recent problems)
- Recognize the pattern within 2 minutes of reading any LeetCode Medium
- Derive a non-trivial optimization without seeing it before, on at least 30 unfamiliar problems
Brute Force & Optimization
- Can state the brute force in <2 minutes for any unseen problem
- Can compute brute force complexity correctly without aid
- Can derive optimal complexity from constraints alone
- Have written brute-force-comparator tests for at least 10 problems
Correctness & Proofs
- Can state DP recurrences with base cases and evaluation order on the spot
- Can produce an exchange argument for at least 5 greedy problems
- Can produce a counterexample to a wrong greedy in <5 minutes
- Have proved correctness for at least 20 different solutions (greedy, DP, graph)
Code Quality
- Can write binary search, BFS, DFS, union-find, topological sort, and Dijkstra from scratch error-free in <10 minutes each
- Code passes the self-review checklist on every solve
- No off-by-one bugs in 10 consecutive binary search problems
- No mutable default argument / shared-state bugs in 10 consecutive recursion problems
Testing
- Run the universal edge-case checklist as reflex on every problem
- Have written stress-test verifiers for at least 5 problems
- Catch your own bugs before submission on 80%+ of problems
- Property-based tested at least 3 implementations
Patterns
- All 28 patterns in phase-02-patterns/ — recognize signals in <2 minutes
- Have solved at least 5 problems per pattern
- Can explain when each pattern does not apply
- Can produce a clean template for each pattern from memory
Data Structures
- Internal representation, complexity, and memory behavior of all foundation DS in phase-01-foundations/
- Can choose between hashmap / sorted array / heap / BST given access patterns
- Can implement segment tree, Fenwick tree, trie, LRU cache from scratch
- Understand iterator invalidation, hash collision behavior, and resize cost in your primary language
Graph Algorithms
- BFS, DFS, Dijkstra, topological sort, union-find — implement from scratch in <10 minutes each
- Recognize when 0-1 BFS, Bellman-Ford, or A* is the right choice
- Can model a word problem as a graph (nodes + edges + weight) in <3 minutes
- Have completed all 9 product-style labs in phase-04-graphs/labs/
Dynamic Programming
- Can derive state + transition + base case for unseen DP problems
- Have completed brute-force → memo → tabulated → space-optimized for at least 15 problems
- Recognize 1D / 2D / interval / tree / digit / bitmask / knapsack signals
- Can articulate why greedy fails for a given DP problem
Language & Runtime
- Read your primary language’s track in phase-09-language-runtime/ end to end
- Can explain stack vs heap, scope/lifetime, value vs reference semantics fluently
- Can explain GC / ownership behavior of your language
- Can explain hash collision and resize behavior of your language’s hashmap
- Have used the language’s profiler at least once on real code
Concurrency
- Can identify race conditions and deadlocks in code review
- Have implemented thread pool, rate limiter, and bounded blocking queue from scratch
- Can articulate memory visibility / happens-before in your language
- Have used the race detector / equivalent at least once
Practical Engineering
- Completed at least 12 of the 23 labs in phase-08-practical-engineering/
- Can answer all 13 standard follow-ups (10M users, distributed, concurrency, race testing, metrics, logging, debugging, partial failure, memory leaks, extensibility, backpressure, retries, deduplication) on demand
- Can sketch a small system (LRU cache, rate limiter, autocomplete) on a whiteboard in 30 minutes
Communication
- Restate every problem before coding
- Ask 3+ clarifying questions per problem
- Narrate intent at decision points (not low-level mechanics)
- Recover from mistakes without flustering
- Handle hints gracefully and integrate them
- Close every solve with a clean summary
Mock Interview Performance
- Pass mock-03 (Medium LeetCode) — 7+/10 average
- Pass mock-05 (Big Tech phone) — 7+/10 average
- Pass mock-06 (Big Tech onsite) — 7+/10 average
- Pass mock-09 (runtime/language) — 7+/10 average
- Pass mock-11 (concurrency) — 7+/10 average
- (For senior+) Pass mock-07 (senior engineer)
- (For staff+) Pass mock-08 (staff practical)
Failure Analysis
- Maintain a failure log per FAILURE_ANALYSIS.md
- Top 3 personal failure categories identified
- At least 2 weeks of focused drilling on top failure category complete
- Re-test results show measurable improvement
Spaced Repetition
- Active spaced repetition rotation per SPACED_REPETITION.md
- At least 50 problems graduated to Tier 6
- No overdue reviews more than 1 week old
Recovery & Stuck Protocol
- Never go silent for >60 seconds in a mock
- Use the stuck protocol explicitly when stuck
- Recover from a wrong direction in <3 minutes
- Ask for hints professionally without losing composure
Production Awareness
- Can extend any solved problem with: scale to 10M, make distributed, handle partial failure, add observability
- Can articulate tradeoffs (memory vs latency, consistency vs availability, accuracy vs speed)
Targeted Roles
Beyond the universal checklist, additional criteria by role:
FAANG / Big Tech
- All universal items
- mock-05 + mock-06 passed twice with different problems
Infrastructure / Backend / Platform
- All universal items
- mock-08 + mock-10 passed
- All Phase 8 labs complete
Distributed Systems
- All universal items
- mock-08 + mock-10 + mock-11 passed
- Phase 8 + Phase 4 (graph) labs complete
Compiler / Runtime
- All universal items
- mock-09 passed twice
- Phase 9 fully complete for primary language
- Phase 3 (advanced DS) labs complete
Quant / HFT
- All universal items
- mock-12 (competitive style) passed
- Phase 7 (competitive) topics complete
- Phase 5 (DP) extreme topics complete
Senior / Staff / Principal Practical
- All universal items
- mock-07 + mock-08 passed twice each
- All Phase 8 labs complete with full follow-up answers
- Code quality bar unwaveringly met
Competitive Programming
- All universal items
- Phase 7 + Phase 12 complete
- mock-12 (competitive style) passed twice
- Solving Codeforces Div 2 D regularly
Honesty Test
For every checked box, ask: “Could I do this right now, cold, with no warm-up?”
If the answer is “after I review my notes” — uncheck it. Notes don’t come into the interview.
If the answer is “if I had a good day” — uncheck it. Interview days are sometimes bad days.
Honesty here is the difference between feeling ready and being ready.
Problem Review Template
Use this after every problem — solved or failed. The review is where the learning compounds. Without it, problems are forgotten in 72 hours.
Save each review as a separate file or notebook entry. Recommended: reviews/YYYY-MM-DD-problem-name.md.
Template
# Problem Name
## Source
LeetCode 42 / Codeforces Round 800 Div 2 C / etc.
## Difficulty
Easy / Medium / Medium-Hard / Hard / Very Hard / CP-Hard / Grandmaster
## Pattern(s)
Two pointers / Sliding window / DP-2D / Graph BFS / etc.
(Multiple patterns possible.)
## First Intuition
What was my first instinct on reading the problem?
What pattern did I think it was?
Was that intuition right?
## Brute Force
- Approach (1–2 sentences)
- Complexity: time / space
- Why it doesn't pass (or "passes within constraints")
## Optimal Idea
- Approach (3–5 sentences)
- Key insight (the *one* thing that unlocks the problem)
- Complexity: time / space
## Why I Missed It (if applicable)
The honest answer. Choose from:
- Didn't recognize the pattern
- Recognized the pattern but applied it wrong
- Couldn't derive the recurrence / invariant
- Wrong data structure choice
- Implementation bug
- Misread the problem
- Ran out of time
- Got the optimal but couldn't prove it
## Key Insight
The single sentence that, if I'd known it, would have unlocked the problem in 5 minutes.
## Data Structures Used
- DS 1: why
- DS 2: why
- (etc.)
## Complexity
Time: O(...)
Space: O(...)
Tight bound? Y/N
Amortized? Y/N
## Bugs I Made
- Off-by-one at line X
- Forgot to handle empty case
- Wrong comparison operator (<= vs <)
- Used wrong variable in inner loop
- Modified collection while iterating
- (etc.)
## Edge Cases I Missed
- Empty input
- Single element
- All duplicates
- All negatives
- Max constraint
- Disconnected component
- (etc.)
## Follow-ups Practiced
- "What if the input is streamed?" → answer
- "What if N is 10^9?" → answer
- "What if memory is constrained?" → answer
## Product Extension
How does this map to a real-world system?
e.g., "This LRU cache pattern is exactly what a CDN edge node uses for hot-content eviction."
## Language/Runtime Notes
- Specific stdlib gotcha I hit
- Memory behavior surprise
- Concurrency consideration if relevant
## How I Would Recognize This Again
A pattern signal in plain English:
"When the problem asks for the minimum window covering K elements over a stream, with constant-time element insertion/removal and order matters, it's a sliding window with a hashmap."
This is the most important field. Optimize for *recognition*, not memorization.
## Re-solve Schedule
Per [SPACED_REPETITION.md](SPACED_REPETITION.md):
- Same day: ☐ done / not done
- 2 days later: ☐ scheduled for [date]
- 1 week later: ☐ scheduled for [date]
- 2 weeks later: ☐ scheduled for [date]
- 1 month later: ☐ scheduled for [date]
- 3 months later: ☐ scheduled for [date]
## Attempts Log
| Date | Unaided? | Time | Outcome | Notes |
|---|---|---|---|---|
| 2026-05-20 | N | 45 min | Wrong, then hint | Missed monotonic stack pattern |
| 2026-05-22 | Y | 18 min | Correct | Recognized pattern immediately |
| 2026-05-29 | Y | 12 min | Correct | Optimal first attempt |
How To Fill It Out (Do’s and Don’ts)
DO
- Be brutally honest. “I gave up after 20 minutes” is more useful than “I solved it but slowly”.
- Name the one insight. Forcing yourself to a single sentence forces understanding.
- Re-solve from a blank file at scheduled intervals.
- Tag heavily. Pattern, difficulty, data structure — these become your search keys.
DON’T
- Copy the editorial verbatim. Translate into your own words.
- Skip the “Why I Missed It” field when you got it right — what was hard about it? What might have tripped you up if you’d been less lucky?
- Skip the “How I Would Recognize This Again” field. This is where 80% of the value lives.
- Move on without scheduling the next re-solve.
Review Aggregation
Every Friday: skim the week’s reviews. Look for patterns:
- “I keep missing monotonic stack opportunities” → drill that pattern.
- “I keep making off-by-one bugs in binary search” → write a personal binary search template and use it.
- “I keep choosing the wrong DP state on tree problems” → revisit phase-05-dp/categories/dp-tree.md.
Every month: aggregate into a personal weakness list. Top 3 weaknesses get dedicated drilling for the next month.
Spaced Repetition System
The brain forgets new information on a curve. Without re-exposure, ~70% of what you learn today is gone in 7 days. Spaced repetition counteracts this with strategically-timed reviews.
This system applies to two things:
- Problems you’ve solved (especially failed-then-solved ones)
- Concepts (patterns, data structures, algorithms)
The 6-Tier Interval Schedule
| Tier | Interval | Action |
|---|---|---|
| 1 | Same day (within 4 hours of first solve) | Re-solve from scratch |
| 2 | 2 days later | Re-solve from scratch |
| 3 | 1 week later | Re-solve from scratch |
| 4 | 2 weeks later | Re-solve from scratch + try a harder variant |
| 5 | 1 month later | Verbal explanation + complexity + sketch tests |
| 6 | 3 months later | Verbal explanation only — proves it’s in long-term memory |
Graduation criterion: if Tier 6 is unaided in <120% of your best time, the problem is “owned” — drop it from active rotation. Otherwise, restart at Tier 3.
What “Re-solve” Means
Re-solving is not reading your old solution. It is:
- Fresh editor / blank file.
- Re-read the problem statement.
- Solve from scratch.
- Compare to your previous solution after.
If you can’t do it without peeking → demote one tier (tier 4 → tier 3) and continue.
What “Verbal Explanation” Means (Tier 5+)
Without writing code, explain out loud (record yourself if alone):
- Restate the problem in your own words.
- State the brute force.
- State the optimal approach.
- State the key insight.
- State the complexity (time + space) and why.
- State 3 edge cases and how they’re handled.
- Sketch 2 follow-up answers.
Listen back. Did you stumble? Demote one tier. Did you flow? Tier graduated.
Concept-Level Spaced Repetition
For patterns and algorithms (not problems), use a similar schedule but with different review actions:
| Tier | Interval | Action |
|---|---|---|
| 1 | Same day as learning | Solve 2 problems applying it |
| 2 | 2 days | Solve 1 problem (different difficulty) |
| 3 | 1 week | Teach the concept verbally (recorded or to a peer) |
| 4 | 2 weeks | Solve 1 problem in a domain you don’t usually associate with it |
| 5 | 1 month | Compare/contrast with a related pattern |
| 6 | 3 months | Solve 1 hard problem cold; if you spot the pattern in <2 minutes, owned |
Logistics: How To Maintain The Schedule
Option 1: Spreadsheet
Columns: Problem | Last Solved | Tier | Next Review Date | Difficulty | Pattern | Notes
Sort by Next Review Date. Top of the list = today’s reviews.
Option 2: Anki (or SRS app)
- One card per problem.
- Front: problem name + difficulty.
- Back: pattern + key insight + complexity.
- Use the SRS scheduling.
Custom intervals matter — the default Anki intervals are tuned for vocabulary, not problems. Use 1d, 2d, 7d, 14d, 30d, 90d.
Option 3: Folder structure (no tooling)
reviews/
today.md # editable list of today's review problems
upcoming/
2026-05-22.md # problems due that date
2026-05-29.md
...
archive/
YYYY-MM-DD-problem-name.md
Each evening: move tomorrow’s file to today.md.
Daily Volume Guidelines
When you have N problems in active rotation, your daily review load looks like:
| Active Problems | Daily Reviews | New Problems |
|---|---|---|
| 0–50 | 0–5 | 4–6 |
| 50–150 | 5–12 | 3–5 |
| 150–300 | 10–20 | 2–3 |
| 300+ | 15–25 | 1–2 |
When daily reviews exceed your capacity:
- Graduate aggressively (drop owned problems).
- Slow down new problem intake.
- Consolidate easy/owned problems into “weekly batch reviews” instead of individual reviews.
What To Do When You Fall Behind
Inevitable. When you have 50+ overdue reviews:
- Don’t panic-skip. Don’t mark them all done.
- Triage by tier. Tier 1 + 2 are the most fragile — do those first.
- Drop tier 5 + 6 for 1 week — they decay slowly.
- Reduce new intake to 0 until reviews are caught up.
- Audit: what made you fall behind? Too many new problems? Underestimated review time? Adjust intake rate.
Why This Matters
Without spaced repetition, your interview prep is a leaky bucket. You add 10 problems a week, lose 8 to forgetting, net +2.
With spaced repetition, you add 5 problems a week, retain 5, and after 3 months you have 60 deeply-owned problems instead of 30 vaguely-remembered ones.
In an interview:
- Vaguely-remembered → “I think I’ve seen this before, but I can’t quite…”
- Deeply-owned → “This is a [pattern] problem. The key insight is [X]. I’d solve it with [approach].”
The latter is a hire signal. The former is not.
Integration With The Curriculum
- Every problem solved during the curriculum enters Tier 1 automatically.
- Every problem in phase-11-mock-interviews/ you fail enters Tier 1 with a
failure_categorytag. - Every concept in a phase README enters Tier 1 the day you finish the phase.
- The Tier 6 graduation criterion is also part of the READINESS_CHECKLIST.md.
12-Week Accelerated Track
Audience: You have a deadline in 8–14 weeks (a known interview, an offer expiring, a layoff window). You can put in 25–35 hours/week.
Tradeoffs you are accepting:
- You will skip Phase 7 (competitive programming) almost entirely.
- You will skip Phase 12 (grandmaster) entirely.
- You will get to “competent at FAANG mediums” not “consistently solves FAANG hards in 45 minutes”.
- Concept depth is sacrificed for problem volume in some weeks.
This track is sufficient for: new-grad / SWE2 FAANG, scaleups, most backend/platform roles. This track is NOT sufficient for: staff/principal interviews, quant/HFT, compiler/runtime, distributed systems specialty roles.
Daily Cadence
- Weekday: 3–5 hours
- 90 min: new content (read concept doc + work through 1 lab)
- 90 min: problem solving (3–5 problems)
- 30 min: review (spaced repetition queue)
- 30 min: failure analysis if you missed anything
- Saturday: 6 hours
- 1 mock interview (90 min including review)
- 4 hours problem solving
- 30 min review
- Sunday: 4 hours
- Weakness drilling on top failure category
- Re-solve all this week’s failures
- Plan next week
Total: 25–30 hours/week
Weekly Plan
Week 1 — Foundation Reset
Goal: Stop being random. Internalize the framework.
- Read README.md, FRAMEWORK.md, COMMUNICATION.md, CODE_QUALITY.md in full
- Complete all 7 labs in phase-00-execution-baseline/
- 25 LeetCode Easy problems applying the framework rigorously (full restate, brute force, optimize, test)
- 0 mocks (you’re not ready)
Mastery check: You can solve LeetCode Easy in 12 minutes with the full framework, narrating throughout.
Week 2 — Foundations Of Data Structures
- Read all 15 DS docs in phase-01-foundations/data-structures/
- 30 problems: array, string, hashmap, stack, queue (mostly Easy + a few Medium)
- 1 mock (Easy level)
- Read phase-01-foundations/runtime/ — stack vs heap, scope/lifetime, mutable vs immutable
Mastery check: Operations + complexity table for every DS reproducible from memory.
Week 3 — Linked Lists, Trees, Recursion
- Phase 1 labs: linked list reversal, recursion, binary search, basic trees
- 25 problems (15 Easy + 10 Medium)
- Read remaining runtime docs in Phase 1
- 1 mock (Easy/Medium mix)
Week 4 — Pattern Onboarding (Part 1)
- Read patterns: two pointers, sliding window, prefix sums, hashing, sorting+greedy, binary search, monotonic stack
- For each pattern: 4–5 problems (Medium-leaning)
- 1 mock (Medium)
Mastery check: Can recognize each pattern’s signal in <2 minutes for unseen problems.
Week 5 — Pattern Onboarding (Part 2)
- Read patterns: monotonic queue, intervals, linked list manipulation, tree DFS/BFS, graph DFS/BFS, topological sort
- 30 problems across these patterns
- 1 mock (Medium)
Week 6 — Pattern Onboarding (Part 3) + DP Intro
- Read patterns: union find, backtracking, basic DP, 1D DP
- Read phase-05-dp/concepts/ — memoization, state definition, transitions, base cases
- 25 DP problems (Easy → Medium)
- 1 mock (Medium)
Mastery check: Can write the brute → memo → tabulated → space-optimized progression for any 1D DP.
Week 7 — Trie, Heap, K-way Merge, 2D DP
- Read patterns: trie, heap top-K, K-way merge, 2D DP, knapsack, subsequence DP, string DP
- 25 problems
- 1 mock (Medium-Hard)
Week 8 — Graphs Deep Dive
- Read phase-04-graphs/algorithms/: BFS, DFS, multi-source BFS, 0-1 BFS, Dijkstra, topological sort, cycle detection, MST (Kruskal+Prim), union find
- 25 graph problems
- 2 product-style labs from phase-04-graphs/labs/
- 1 mock (Medium-Hard graph-heavy)
Week 9 — DP Deeper + Greedy
- Read DP categories: tree DP, interval DP, bitmask DP (overview), knapsack variants, LIS, edit distance
- Read phase-06-greedy/concepts/: greedy choice, exchange argument, invariants
- 25 problems (mix DP + greedy)
- 2 greedy labs (with proofs)
- 1 mock (Medium-Hard mixed)
Week 10 — Practical Engineering Coding
- Complete labs from phase-08-practical-engineering/: LRU cache, rate limiter, autocomplete, thread pool, KV store, retry+backoff
- Read your primary language’s track in phase-09-language-runtime/
- 15 problems (mediums + 1–2 hards)
- 2 mocks (1 Big Tech phone, 1 Big Tech onsite)
Week 11 — Hard Problems + Mock Marathon
- 20 LeetCode Hard problems
- Daily mock interviews (5 mocks this week, mix of phase-11-mock-interviews/ types)
- Failure analysis on every mock
- Drill top failure category
Week 12 — Polish, Confidence, Rest
- 3 mocks (cold, varied — no pre-warmup)
- Re-solve top 30 problems from spaced repetition queue
- Complete READINESS_CHECKLIST.md honestly
- 1 day complete rest before interview
Problem Volume Targets
| Week | New Problems | Reviews | Mocks |
|---|---|---|---|
| 1 | 25 (Easy) | 0 | 0 |
| 2 | 30 (E+M) | 10 | 1 |
| 3 | 25 (E+M) | 15 | 1 |
| 4 | 25 (M) | 20 | 1 |
| 5 | 30 (M) | 25 | 1 |
| 6 | 25 (M) | 30 | 1 |
| 7 | 25 (M-H) | 30 | 1 |
| 8 | 25 (M-H) | 30 | 1 |
| 9 | 25 (M-H) | 30 | 1 |
| 10 | 15 (M-H + H) | 30 | 2 |
| 11 | 20 (H) | 30 | 5 |
| 12 | 0 | 30 (re-solves) | 3 |
| Total | ~270 | ~280 | 18 |
Review Schedule (per SPACED_REPETITION.md)
Use the abbreviated tier schedule for accelerated track:
- Tier 1: same day
- Tier 2: 2 days
- Tier 3: 1 week
- Tier 4: 2 weeks
- (Skip tiers 5–6 — you don’t have time before interviews)
What This Track Cannot Buy You
- Deep competitive-programming intuition (Phase 7)
- Grandmaster-level pattern recognition (Phase 12)
- Long-term retention of all 270 problems (you’ll forget ~30% within 3 months without continued review)
- The kind of polish that comes from 6+ months of practice
If you finish Week 12 and your interview is delayed by 8+ weeks, switch to the 6-month track for the remaining time. Use the buffer to backfill Phase 7 basics and harden weak patterns.
6-Month Serious Track
Audience: You’re targeting strong Big Tech readiness. You can put in 15–25 hours/week sustainably for 6 months.
Tradeoffs:
- You will cover Phase 7 (competitive) selectively — not full grandmaster prep.
- You will get to “consistently solves FAANG hards in 45–60 minutes” with strong follow-ups.
- You will have meaningful depth in your primary language’s runtime.
- This is the recommended track for most readers.
Sufficient for: all FAANG levels including senior, infrastructure, platform, distributed systems, most backend specialties.
Daily Cadence
- Weekday: 2–3 hours
- 60 min: new content (read + 1 lab or 2–3 concept docs)
- 60 min: problem solving (2–4 problems)
- 30 min: review queue
- Saturday: 4 hours
- 1 mock interview (90 min)
- 2 hours problem solving
- 30 min: failure analysis + review
- Sunday: 2 hours
- Re-solves at Tier 4–5
- Weakness drilling
Total: 18–22 hours/week
Monthly Milestones
Month 1 — Foundations (Weeks 1–4)
Cover: Phase 0 + Phase 1 fully, Phase 9 (your primary language) partially.
- Week 1: Phase 0 (all 7 labs), 25 Easy problems
- Week 2: Phase 1 DS docs (arrays, strings, hashmaps, stacks, queues, linked lists), Phase 1 runtime (stack vs heap, scope, value/reference, mutability), 30 problems
- Week 3: Phase 1 DS continued (heaps, sorting, binary search, recursion, basic trees, basic graphs, basic DP), 30 problems, 2 labs
- Week 4: Phase 1 runtime complete (hash collisions, iterator invalidation, GC, memory leaks, deep/shallow copy), Phase 9 your-language (first half), 30 problems, 3 labs
End-of-month: all Phase 1 mastery checks pass. 1 mock at Easy level, scored 7+/10.
Volume: ~115 problems, 10+ labs.
Month 2 — Patterns Mastery (Weeks 5–8)
Cover: Phase 2 fully, Phase 9 your-language fully.
- Week 5: two pointers, sliding window, prefix sums, difference arrays, hashing, sorting+greedy. 5 problems per pattern.
- Week 6: binary search, binary search on answer, monotonic stack, monotonic queue, intervals, linked list manipulation. 5 per pattern.
- Week 7: tree DFS/BFS, graph DFS/BFS, topological sort, union find. 5 per pattern. 3 labs.
- Week 8: backtracking, basic DP, 1D DP, 2D DP, knapsack basics, subsequence DP, string DP, trie, heap top-K, K-way merge. 4 per pattern.
End-of-month: mastery check on all 28 patterns. 4 mocks (Easy-Medium-Medium-Medium-Hard).
Volume: ~140 problems, 9 labs.
Month 3 — Graphs + DP Deep (Weeks 9–12)
Cover: Phase 4 fully, Phase 5 fully (except convex hull / Knuth optimization, those are Month 6).
- Week 9: Phase 4 fundamentals (BFS variants, DFS, Dijkstra, Bellman-Ford, Floyd-Warshall, topo). Solve 30 graph problems. 2 graph labs.
- Week 10: Phase 4 advanced (SCCs, bridges/articulation, MST, bipartite, max flow basics, graph modeling). Solve 25. 3 graph labs.
- Week 11: Phase 5 categories: 1D, 2D, knapsack, LIS, edit distance, palindrome, string DP. Solve 25. 3 DP labs (full brute→memo→tab→space progression).
- Week 12: Phase 5 categories: tree DP, interval DP, bitmask DP, digit DP (intro), DP on DAGs, game DP. Solve 25. 3 DP labs.
End-of-month: all of Phase 4 + Phase 5 mastery checks. 4 mocks (Medium-Medium-Medium-Hard mix).
Volume: ~110 problems, 11 labs.
Month 4 — Greedy, Advanced DS, Practical Engineering (Weeks 13–16)
- Week 13: Phase 6 fully (greedy choice, exchange arg, cut property, invariants, monovariants, amortized analysis). 6 greedy labs (with proofs). 20 greedy problems.
- Week 14: Phase 3 (segment tree, Fenwick tree, sparse table, KMP, Z, rolling hash, trie variants, bit manipulation, bitmask DP, meet-in-the-middle). 20 problems. 3 advanced DS labs.
- Week 15: Phase 8 first half (LRU, LFU, rate limiter, task scheduler, thread pool, job queue, autocomplete, log parser, file dedup). 9 labs with full follow-up answers.
- Week 16: Phase 8 second half (consistent hashing, message dispatcher, pubsub, timer wheel, KV store, retry/backoff, circuit breaker, metrics, web crawler, in-memory FS). 10 labs.
End-of-month: Phase 3 + Phase 6 + Phase 8 (most of it) complete. 4 mocks including 1 Big Tech phone screen.
Volume: ~80 problems, 28 labs.
Month 5 — Hard Problems + Concurrency + Senior-Level Skills (Weeks 17–20)
- Week 17: Phase 8 final 4 labs + Phase 10 (testing/debugging concept docs). 5 testing/debugging labs. 15 problems.
- Week 18: Concurrency deep dive — re-read Phase 9 concurrency sections, solve 5 concurrency-flavored problems, do mock-11 twice. 15 problems.
- Week 19: Hard problem week — 25 LeetCode Hards across all patterns. 1 mock per day (5 mocks).
- Week 20: Phase 7 selectively (modular arithmetic, sieve, binary exponentiation, combinatorics, sweep line, coordinate compression — skip ICPC-only topics). 20 problems including 8 from Codeforces Div 3. 1 mock.
End-of-month: all of Phase 8, Phase 10, parts of Phase 7. 8 mocks total.
Volume: ~100 problems including ~25 hards.
Month 6 — Polish, Mock Marathon, Production Awareness (Weeks 21–24)
- Week 21: Mock-heavy week. 5 mocks (mix mock-05, 06, 07, 09, 11). Failure analysis on each. 15 problems focused on top failure category.
- Week 22: Phase 12 selective topics (only those relevant to your role) — for backend/platform: max flow modeling, advanced combinatorics inclusion-exclusion. For systems: nothing additional, focus on Phase 8 polish. 20 problems.
- Week 23: Re-solve marathon — re-solve all Tier 5 and Tier 6 problems. Verify READINESS_CHECKLIST.md honestly. 3 mocks.
- Week 24: Light week. 2 mocks. 10 problems. Rest. Final readiness check.
End-of-program: READINESS_CHECKLIST.md fully passed.
Volume: ~50 problems, 13 mocks across the month.
Aggregate Volume
| Month | New Problems | Labs | Mocks |
|---|---|---|---|
| 1 | 115 | 10 | 1 |
| 2 | 140 | 9 | 4 |
| 3 | 110 | 11 | 4 |
| 4 | 80 | 28 | 4 |
| 5 | 100 | 5 | 8 |
| 6 | 50 | 0 | 13 |
| Total | ~595 | ~63 | ~34 |
Review / Spaced Repetition
Full 6-tier schedule from SPACED_REPETITION.md:
- Tier 1: same day
- Tier 2: 2 days
- Tier 3: 1 week
- Tier 4: 2 weeks
- Tier 5: 1 month
- Tier 6: 3 months
By Month 6 you should have ~50–80 Tier 6 graduates.
Mock Schedule
Required mocks across the program (see phase-11-mock-interviews/):
| Mock | When | Pass Threshold |
|---|---|---|
| mock-02 (Easy) | end of Month 1 | 7/10 |
| mock-03 (Medium) ×3 | Month 2 | 6/10 → 7/10 |
| mock-04 (Hard) ×2 | Month 3 | 6/10 |
| mock-05 (Big Tech phone) ×3 | Months 4–6 | 7/10 |
| mock-06 (Big Tech onsite) ×3 | Months 5–6 | 7/10 |
| mock-07 (senior) ×2 | Months 5–6 | 6/10 |
| mock-09 (runtime) ×2 | Months 5–6 | 7/10 |
| mock-11 (concurrency) ×2 | Month 5 | 6/10 |
| mock-10 (system-heavy) ×2 | Month 6 | 6/10 |
Revision Plan
Weekly: Friday evening — skim review log, identify the week’s failure pattern. Monthly: Last weekend — aggregate failures, identify top 3 categories, dedicate next month’s Sunday drilling to category #1. Bi-monthly: End of months 2, 4, 6 — do a “blind mock” with a problem you’ve never seen, no warmup. Score honestly.
What This Track Buys You
- ~600 deeply-owned problems
- All 28 patterns recognizable in <2 minutes
- ~35 production-style labs with full follow-up answers
- Strong concurrency + runtime depth in your primary language
- Genuine readiness for FAANG senior-level coding interviews
- 30+ mock interviews of practice — pressure tolerance is real
What It Does Not Buy You
- Codeforces Div 1 / AtCoder AGC level (you’d need Phase 7 + 12 fully)
- Quant/HFT-level math depth (math heavy, beyond Month 5 scope)
- Compiler/runtime team specialty (you’d need Phase 9 fully across multiple languages)
12-Month Elite Track
Audience: Top-tier targets. Senior/staff/principal practical, FAANG L6+, Jane Street / Citadel / Two Sigma, distributed systems specialty teams, compiler/runtime teams, or competitive programmers building toward Codeforces 2100+.
Commitment: 15–20 hours/week sustained for 12 months. Not a sprint.
This track will get you to:
- Phase 7 + Phase 12 substantial depth
- Codeforces Div 2 reliable, Div 1 attempt-capable
- Multi-language runtime fluency in primary + 1 secondary
- Production-system thinking comparable to a working senior/staff engineer
- The specific gap between “Big Tech ready” and “elite-tier ready”
Annual Phase Map
| Months | Phases | Focus |
|---|---|---|
| 1 | Phases 0–1 | Foundations + framework internalization |
| 2 | Phase 2 | All 28 patterns to mastery level |
| 3 | Phase 4 | Graph algorithms deep |
| 4 | Phase 5 | DP from basic to extreme |
| 5 | Phase 3 + Phase 6 | Advanced DS + greedy proofs |
| 6 | Phase 8 + Phase 10 | Practical engineering + testing/debugging |
| 7 | Phase 9 (full) | Language/runtime — 2 languages |
| 8 | Phase 7 (first half) | Competitive programming — math, sieve, modular, geometry |
| 9 | Phase 7 (second half) | Competitive — sweep line, Mo’s, parallel BS, contests |
| 10 | Phase 12 (selective) | Grandmaster topics relevant to your role |
| 11 | Mock marathon | Interview-realistic prep |
| 12 | Polish + readiness | Final checks, blind mocks, rest |
Daily Cadence
- Weekday: 2 hours
- Saturday: 5 hours (mock + problems + labs)
- Sunday: 3 hours (review + drilling)
- Total: 17–18 hours/week
You will scale up in Months 8–11 to 22 hours/week temporarily.
Monthly Detail
Month 1 — Phase 0 + Phase 1
Identical to Month 1 of 6-Month Serious, but with extra depth on runtime docs (read all 10) and an additional 30 Easy problems for fluency.
Volume: 130 problems, 12 labs, 1 mock.
Month 2 — Phase 2 (Patterns)
- All 28 pattern docs read in detail
- 8 problems per pattern (vs 5 in Serious track)
- Every pattern: write a personal template after solving the 8 problems
- 5 mocks across the month
Volume: 224 problems, 9 labs, 5 mocks.
Mastery check: Can derive each pattern’s template from memory in <5 minutes.
Month 3 — Phase 4 (Graphs)
- All 21 graph algorithms with implementation
- All 9 product-style labs
- 60 graph problems (mix of LeetCode + Codeforces Div 3 graph problems)
- 4 mocks
Volume: 60 problems, 9 labs, 4 mocks.
Mastery check: Can implement BFS / DFS / Dijkstra / Bellman-Ford / Floyd-Warshall / Kruskal / Prim / Kosaraju / Tarjan from scratch error-free in <12 minutes each.
Month 4 — Phase 5 (Dynamic Programming)
- All 22 DP concept + category docs
- All 10 DP labs with full progression (brute → memo → tab → space)
- 80 DP problems
- Every problem you solve: explicitly state recurrence + base case + evaluation order before coding
- 4 mocks (DP-heavy)
Volume: 80 problems, 10 labs, 4 mocks.
Mastery check: Can derive state + transition for unseen DP problems in <8 minutes.
Month 5 — Phase 3 (Advanced DS) + Phase 6 (Greedy/Proofs)
- All 24 advanced DS docs
- All 9 advanced DS labs
- All 7 greedy concept docs + all 6 labs (with proofs)
- 50 problems mixing advanced DS and greedy
- 4 mocks
Volume: 50 problems, 15 labs, 4 mocks.
Mastery check: Can implement segment tree (with lazy propagation), Fenwick tree, KMP, rolling hash, trie, treap from scratch.
Month 6 — Phase 8 (Practical Engineering) + Phase 10 (Testing/Debugging)
- All 23 practical engineering labs
- Every lab includes: working implementation + unit tests + smoke tests + concurrency tests where relevant + answers to all 13 standard follow-ups
- All 13 testing/debugging concept docs + all 5 labs
- Property-based test 5 implementations
- 30 problems
- 5 mocks (mix mock-08 staff practical, mock-10 system-heavy, mock-11 concurrency)
Volume: 30 problems, 28 labs, 5 mocks.
Mastery check: Can build any of (LRU, rate limiter, autocomplete, KV store, thread pool) from scratch in 45 minutes with tests.
Month 7 — Phase 9 (Language & Runtime)
- Primary language: read entire track end to end. Take written notes. Solve 5 problems per major topic targeting the gotchas.
- Secondary language: read 50%+ of the track. Solve 20 problems in this language to build fluency.
- 30 problems total in primary language (focused on runtime gotchas: overflow, recursion limits, hash adversarial, GC pressure, concurrency)
- 4 mocks including 2 mock-09 (runtime/language deep dives)
Volume: 50 problems, 0 labs (concept-heavy month), 4 mocks.
Mastery check: Can fluently explain stack vs heap, GC behavior, hashmap internals, concurrency model for both languages.
Month 8 — Phase 7 First Half
Competitive programming acceleration starts here. Move to Codeforces full-time for problem solving (LeetCode for review only).
- Topics: fast I/O, modular arithmetic, GCD/LCM, sieve, prime factorization, modular inverse, combinatorics, binary exponentiation, matrix exponentiation, geometry basics, coordinate compression
- Codeforces Div 3 → solve 6 contests (virtual or real). Goal: solve 4–5 problems per Div 3.
- AtCoder Beginner Contest: solve 4 contests. Goal: solve 5 of A–F.
- 1 mock per week (varied — competitive-flavored mock-13)
Volume: ~80 problems (Codeforces + AtCoder), 6 labs, 4 mocks.
Mastery check: Solve Codeforces 1500–1700 rated problems consistently within contest time.
Month 9 — Phase 7 Second Half
- Topics: sweep line, offline queries, Mo’s algorithm, parallel binary search (overview), randomized stress testing, interactive problems, game theory (Nim, Sprague-Grundy)
- Codeforces Div 2 → 6 virtual contests. Goal: solve A–C reliably, attempt D.
- AtCoder Regular Contest → 3 contests, attempt A–C.
- Implement a stress-testing harness for 5 problems
- 4 mocks including mock-13 (competitive hard) twice
Volume: ~70 problems, 4 mocks.
Mastery check: Codeforces Div 2 ABC reliable. D attempted and sometimes solved.
Month 10 — Phase 12 (Selective)
Pick topics by role:
Backend/platform:
- Max flow modeling, min-cost max-flow overview
- Advanced combinatorics + inclusion-exclusion
- Randomized algorithms
Distributed systems:
- Persistent data structures
- Consistent hashing deep
- Lockless / wait-free data structures (research)
Compiler/runtime:
- Suffix automaton
- Heavy-light decomposition
- Constraint solving / SAT-like reasoning
Quant/HFT:
- FFT/NTT
- Computational geometry
- Probability DP + expected value
- Game theory deep
Pure competitive:
- HLD, centroid decomposition, segment tree beats, FFT/NTT, suffix automaton
Pick 5–7 topics. Read deeply. Implement reference code. 3 labs from Phase 12.
Volume: 30 problems (very hard / CP-hard), 3+ labs, 4 mocks.
Mastery check: can articulate selected topics + implementation risks; have working reference implementations.
Month 11 — Mock Marathon
This is the highest-pressure month.
- 4 mocks per week (16 total)
- Mix all 14 mock types — including grandmaster final boss (mock-14) at least twice
- Failure analysis on every mock
- Every Sunday: aggregate failure pattern, drill the top category Mon–Wed
- 60 problems (almost entirely focused on weak areas surfaced by mocks)
Volume: 60 problems, 16 mocks.
Month 12 — Polish, Re-solve, Final Readiness
- Week 49: Re-solve all Tier 5 + Tier 6 problems. Aim for 80+ Tier 6 graduates.
- Week 50: 4 cold blind mocks (no warmup, varied difficulty). Score brutally honestly.
- Week 51: Address any remaining gaps. Light schedule (10 hours total).
- Week 52: Rest, sleep, exercise. 1 final readiness check via READINESS_CHECKLIST.md. Interview confidently.
Volume: 30 re-solves, 4 mocks, 1 readiness audit.
Aggregate Volume
| Month | Problems | Labs | Mocks |
|---|---|---|---|
| 1 | 130 | 12 | 1 |
| 2 | 224 | 9 | 5 |
| 3 | 60 | 9 | 4 |
| 4 | 80 | 10 | 4 |
| 5 | 50 | 15 | 4 |
| 6 | 30 | 28 | 5 |
| 7 | 50 | 0 | 4 |
| 8 | 80 | 6 | 4 |
| 9 | 70 | 0 | 4 |
| 10 | 30 | 3 | 4 |
| 11 | 60 | 0 | 16 |
| 12 | 30 | 0 | 4 |
| Total | ~894 | ~92 | ~59 |
Contest Practice (unique to this track)
By end of Month 9, you should have completed:
- 12+ Codeforces Div 3/4 contests
- 6+ Codeforces Div 2 contests
- 6+ AtCoder Beginner Contests
- 3+ AtCoder Regular Contests
Goal Codeforces rating after 12 months: 1700–1900 (Expert range).
What Distinguishes This Track
vs 6-month Serious:
- ~1.5× the problems
- 2× the labs (especially in Phase 7 / 12)
- 2× the mocks (mostly in Months 8–11)
- Real contest experience (vs 0 in 6-month track)
- Multi-language runtime depth
- Phase 12 awareness (selective implementation)
vs Competitive-only training:
- This track does NOT skip practical engineering, system thinking, communication, or runtime depth
- A candidate trained purely on competitive programming often struggles in mock-08 (staff practical) and mock-10 (system-heavy) — this track does not have that gap
Honest Limitations
Even after this track:
- You will not reach grandmaster (CF 2400+) — that requires sustained contest practice for years
- You will not be a domain expert in a specific area (e.g., compilers, distributed consensus) — that requires actual work on those systems
- 12 months of part-time prep cannot replicate 5 years of full-time engineering experience
What it can do is take you from intermediate to elite-candidate level — enough to interview confidently for very hard roles and have a fighting chance.
6-Month Serious Track — Implementation
This is the execution layer for schedules/6_MONTH_SERIOUS.md. The schedule tells you what to study and when. This folder is the actual substance — curated flagship problems, original company-sourced problems, graduated reading, proof-first solutions, and senior/staff/principal-level signal layers.
What This Is (And Is Not)
This IS:
- 91 hand-picked flagship problems across 20 active weeks
- 8–10 ORIGINAL problems sourced from real Amazon/AWS, Google, Meta, Microsoft interview intelligence (not on LeetCode)
- A graduated 15-section reading layout per problem: approach → hints → insight → company adversarial → level delta → follow-ups
- Python solutions with brute-force comparators and stress tests
- Above-FAANG difficulty content targeting senior, staff, and principal interviews
This is NOT:
- A duplicate of LeetCode Premium. You should have LC Premium. We never restate problem statements or constraints — those are one click away.
- Every problem from the 6-month schedule (~595). The other ~500 are listed as “Problem Bank” in each week’s README — work them on LC after you own the flagship.
- A passive read. Each problem has an Attempt Gate — you set a timer and try the problem cold before reading anything else.
Why This Exists
LeetCode editorials give you code. They do not give you:
- The exchange argument that proves the greedy works
- The invariant that the two-pointer maintains
- The misleading example a Google interviewer will use to trap you on this exact problem
- The scorecard language the interviewer writes when you do (or do not) ace it
- The Level Delta: what a Mid vs Senior vs Staff vs Principal answer looks like for this same problem
- The production reality: if this ran at 10M RPS, what would break first?
- The Anti-pattern: the wrong-but-tempting approach that catches 70% of candidates, with exact bug location
- The original problem Amazon will ask you that has never appeared on LeetCode
This track provides all of those.
How To Use
- Read HOW_TO_USE.md — the graduated reading protocol
- Read FRAMEWORK.md once if you have not — every “How to Approach” section here references its 16 steps
- Start at month-01-foundations/, week 01, problem 01. Do not skip ahead.
- For each problem: Attempt Gate first (20-min timer, cold), then graduate through the README sections
- Stress-test your solution:
python solution.pyruns the brute-vs-optimal comparator - Log every solve in your tracking spreadsheet per SPACED_REPETITION.md
Problem Index (All 91)
Month 1 — Foundations (20 problems)
| # | Problem | Difficulty | Source | Week |
|---|---|---|---|---|
| p01 | Two Sum | Easy | LC 1 | 1 |
| p02 | Valid Parentheses | Easy | LC 20 | 1 |
| p03 | Best Time to Buy and Sell Stock | Easy | LC 121 | 1 |
| p04 | Merge Sorted Array | Easy | LC 88 | 1 |
| p05 | Climbing Stairs | Easy | LC 70 | 1 |
| p06 | Product of Array Except Self | Medium | LC 238 | 2 |
| p07 | Group Anagrams | Medium | LC 49 | 2 |
| p08 | Rotate Array | Medium | LC 189 | 2 |
| p09 | Longest Substring Without Repeating | Medium | LC 3 | 2 |
| p10 | Valid Anagram | Easy | LC 242 | 2 |
| p11 | Binary Tree Level Order Traversal | Medium | LC 102 | 3 |
| p12 | Kth Largest Element | Medium | LC 215 | 3 |
| p13 | Search in Rotated Sorted Array | Medium | LC 33 | 3 |
| p14 | Maximum Depth of Binary Tree | Easy | LC 104 | 3 |
| p15 | Merge Intervals | Medium | LC 56 | 3 |
| p16 | LRU Cache | Medium | LC 146 | 4 |
| p17 | Number of Islands | Medium | LC 200 | 4 |
| p18 | Coin Change | Medium | LC 322 | 4 |
| p19 | Binary Tree Right Side View | Medium | LC 199 | 4 |
| p20 | Word Search | Medium | LC 79 | 4 |
Month 2 — Patterns Mastery (20 problems)
| # | Problem | Difficulty | Source | Week |
|---|---|---|---|---|
| p21 | 3Sum | Medium | LC 15 | 5 |
| p22 | Trapping Rain Water | Hard | LC 42 | 5 |
| p23 | Subarray Sum Equals K | Medium | LC 560 | 5 |
| p24 | Minimum Size Subarray Sum | Medium | LC 209 | 5 |
| p25 | Maximum Product Subarray | Medium | LC 152 | 5 |
| p26 | Find Minimum in Rotated Sorted Array | Medium | LC 153 | 6 |
| p27 | Daily Temperatures | Medium | LC 739 | 6 |
| p28 | Meeting Rooms II | Medium | LC 253 | 6 |
| p29 | Largest Rectangle in Histogram | Hard | LC 84 | 6 |
| p30 | Jump Game II | Medium | LC 45 | 6 |
| p31 | Course Schedule | Medium | LC 207 | 7 |
| p32 | Pacific Atlantic Water Flow | Medium | LC 417 | 7 |
| p33 | Number of Connected Components | Medium | LC 323 | 7 |
| p34 | Word Ladder | Hard | LC 127 | 7 |
| p35 | Lowest Common Ancestor of BST | Medium | LC 235 | 7 |
| p36 | Combination Sum | Medium | LC 39 | 8 |
| p37 | Unique Paths | Medium | LC 62 | 8 |
| p38 | Implement Trie | Medium | LC 208 | 8 |
| p39 | Find Median from Data Stream | Hard | LC 295 | 8 |
| p40 | Decode Ways | Medium | LC 91 | 8 |
Month 3 — Graphs + DP Deep (20 problems)
| # | Problem | Difficulty | Source | Week |
|---|---|---|---|---|
| p41 | Network Delay Time | Medium | LC 743 | 9 |
| p42 | Cheapest Flights Within K Stops | Medium | LC 787 | 9 |
| p43 | Shortest Path in Binary Matrix | Medium | LC 1091 | 9 |
| p44 | Word Ladder II | Hard | LC 126 | 9 |
| p45 | Clone Graph | Medium | LC 133 | 9 |
| p46 | Critical Connections in a Network | Hard | LC 1192 | 10 |
| p47 | Alien Dictionary | Hard | LC 269 | 10 |
| p48 | Min Cost to Connect All Points | Medium | LC 1584 | 10 |
| p49 | Is Graph Bipartite? | Medium | LC 785 | 10 |
| p50 | Reconstruct Itinerary | Hard | LC 332 | 10 |
| p51 | Longest Increasing Subsequence | Medium | LC 300 | 11 |
| p52 | Edit Distance | Hard | LC 72 | 11 |
| p53 | Partition Equal Subset Sum | Medium | LC 416 | 11 |
| p54 | Regular Expression Matching | Hard | LC 10 | 11 |
| p55 | Interleaving String | Medium | LC 97 | 11 |
| p56 | Burst Balloons | Hard | LC 312 | 12 |
| p57 | Palindrome Partitioning II | Hard | LC 132 | 12 |
| p58 | Strange Printer | Hard | LC 664 | 12 |
| p59 | Dungeon Game | Hard | LC 174 | 12 |
| p60 | Number of Ways to Wear Different Hats | Hard | LC 1434 | 12 |
Month 4 — Greedy, Advanced DS, Practical Engineering (20 problems)
| # | Problem | Difficulty | Source | Week |
|---|---|---|---|---|
| p61 | Queue Reconstruction by Height | Medium | LC 406 | 13 |
| p62 | Gas Station | Medium | LC 134 | 13 |
| p63 | Task Scheduler | Medium | LC 621 | 13 |
| p64 | Minimum Number of Arrows | Medium | LC 452 | 13 |
| p65 | Split Array Largest Sum | Hard | LC 410 | 13 |
| p66 | Range Sum Query Mutable | Medium | LC 307 | 14 |
| p67 | Count of Smaller Numbers After Self | Hard | LC 315 | 14 |
| p68 | Implement strStr() (KMP) | Easy | LC 28 | 14 |
| p69 | Repeated DNA Sequences | Medium | LC 187 | 14 |
| p70 | Sliding Window Maximum | Hard | LC 239 | 14 |
| p71 | Design LRU Cache (distributed follow-ups) | Medium | LC 146 + extensions | 15 |
| p72 | Design Hit Counter | Medium | LC 362 | 15 |
| p73 | Design Autocomplete System | Hard | LC 642 | 15 |
| p74 | LFU Cache | Hard | LC 460 | 15 |
| p75 | ORIGINAL — Consistent Hashing Ring | Hard | Amazon/AWS | 15 |
| p76 | Design In-Memory File System | Hard | LC 588 | 16 |
| p77 | ORIGINAL — Distributed Rate Limiter | Hard | Meta Infra | 16 |
| p78 | Text Justification | Hard | LC 68 | 16 |
| p79 | ORIGINAL — Service Mesh Min Deployment | Hard | Google SRE | 16 |
| p80 | Basic Calculator II | Medium | LC 227 | 16 |
Month 5 — Hards + Concurrency + Above-FAANG (10 problems)
| # | Problem | Difficulty | Source | Week |
|---|---|---|---|---|
| p81 | Median of Two Sorted Arrays | Hard | LC 4 | 17 |
| p82 | Russian Doll Envelopes | Hard | LC 354 | 17 |
| p83 | First Missing Positive | Hard | LC 41 | 17 |
| p84 | Minimum Window Substring | Hard | LC 76 | 17 |
| p85 | ORIGINAL — Concurrent Web Crawler with Backpressure | Hard | Amazon | 18 |
| p86 | Design Bounded Blocking Queue | Medium | LC 1188 | 18 |
| p87 | ORIGINAL — S3 Cost-Optimal Eviction | Hard | AWS | 18 |
| p88 | N-Queens | Hard | LC 51 | 19 |
| p89 | Trapping Rain Water II | Hard | LC 407 | 19 |
| p90 | ORIGINAL — Distributed Job DAG Scheduler | Hard | 19 |
Month 6 — Polish + Mock Marathon
| Capstone | Source | Week |
|---|---|---|
| p91 — Skyline Problem | LC 218 | 20 |
| MOCK_MARATHON.md | — | 21 |
| RE_SOLVE_GUIDE.md | — | 23 |
| FAILURE_PATTERNS.md | — | 21–24 |
| FINAL_READINESS.md | — | 24 |
Cross-References to the Rest of the Workspace
- FRAMEWORK.md — universal 16-step problem-solving framework
- CODE_QUALITY.md — quality bar every
solution.pymust meet - COMMUNICATION.md — what to say in interviews
- FAILURE_ANALYSIS.md — 16-category failure taxonomy
- SPACED_REPETITION.md — 6-tier review schedule
- READINESS_CHECKLIST.md — final binary checklist
- phase-02-patterns/ — concept docs for patterns (3Sum, sliding window, etc.)
- phase-04-graphs/ — graph algorithm concept docs
- phase-05-dp/ — DP concept docs and labs
Honest Promise
If you complete every problem in this track honestly — Attempt Gate respected, hints used sparingly, follow-ups answered before reading the answers, stress tests passing, every “When to Move On” checklist green — you will be in the top 5% of candidates entering senior, staff, and principal interview loops at Amazon, Google, Meta, Microsoft, Apple, and infrastructure-specialty companies.
If you skip the Attempt Gates, peek at hints early, or skim the Level Delta sections without honestly self-assessing — this is just a list of problems you’ve seen. Seen ≠ owned. Owned is what gets offers.
How To Use This Track — The Graduated Reading Protocol
Every problem folder has three files. They are meant to be read in a specific order with specific gates between them. Reading them out of order destroys the value.
The Three Files
pXX-problem-name/
README.md ← Graduated reading: 15 sections, each gated
solution.py ← Brute + optimal + stress test (Python, runnable)
hints.md ← 5 progressive hints (only opened when stuck)
The Per-Problem Reading Protocol
Phase 1 — Cold Attempt (mandatory)
- Open
README.md, read only sections 1–2 (Quick Context + LeetCode Link/Attempt Gate) - Click the LeetCode link, read the problem statement on LC
- Set a 20-minute timer. Code on paper or in a scratch file. No IDE autocomplete that helps with the algorithm. No reading further in README.md. No
hints.md. - If you solve it: write down your time and approach. Now jump to Phase 3.
- If 20 min passes: write down where you got stuck. Move to Phase 2.
Why the gate? You cannot build pattern-recognition by reading approaches. You build it by failing to find approaches, then learning the gap. If you skip the cold attempt, you are training your recognition memory, not your derivation skill. Interviews test derivation.
Phase 2 — Hint Ladder (only if stuck)
Open hints.md. Read one hint. Set another 10-min timer. Try again.
- Solved after Hint 1 → write down which insight unlocked it. Move to Phase 3.
- Still stuck after 10 min → next hint.
- After Hint 5 without solving → you’ve hit the conceptual gap. Open
README.mdsection 4 (“How to Approach”) and read straight through. Then Phase 3.
Rule: never read two hints back to back. Always 10 minutes between hints.
Phase 3 — The Real Learning
Now read README.md sections 3–15 in order. This is where the value is.
- Section 3 (Prerequisites): if any link goes to a phase lab you haven’t done, do that lab first
- Section 4 (How to Approach): compare to YOUR approach. Where did you diverge? Why?
- Section 6 (Deeper Insight): the proof or invariant. You must be able to restate this in your own words before moving on.
- Section 7 (Anti-Pattern): “did I almost do this?” — if yes, this is a flag for your weakness log
- Section 10 (Company Context): mental note of how the company you’re targeting twists this problem
- Section 12 (Level Delta): honestly: which level was your answer? Mid, Senior, Staff, or Principal?
- Section 13 (Follow-ups): cover the answer with your hand. Try each follow-up. Then read the answer.
Phase 4 — Code & Test
- Open
solution.py. Read the brute force. Read the optimal. - Close the file. Re-implement the optimal from scratch in your own scratch.
- Diff your re-implementation against the reference. Identify every difference. Some differences are style (fine). Some are bugs (not fine).
- Run the stress test:
python solution.py. It must pass. - Now run your own implementation against the stress test. It must also pass.
Phase 5 — Move-On Gate
Walk through Section 9 (“When to Move On”) — the binary checklist. Every box must be honestly checked. If even one is no, you stay on this problem (re-do tomorrow). No exceptions.
Log the solve in your tracking spreadsheet:
- Date, problem, time-to-solve, hint depth used (0–5), follow-ups answered correctly (count/total), Level Delta self-assessment
- Schedule next review per SPACED_REPETITION.md
What Each README.md Section Is For
| # | Section | Read When |
|---|---|---|
| 1 | Quick Context | Before cold attempt |
| 2 | LeetCode Link + Attempt Gate | Before cold attempt |
| 3 | Prerequisite Concepts | After cold attempt; do prereq labs if needed |
| 4 | How to Approach | After Hint 5 fails OR after solving |
| 5 | Progressive Hints (→ hints.md) | When stuck, one at a time |
| 6 | Deeper Insight | Always, after solving |
| 7 | Anti-Pattern Analysis | Always — check if you fell into it |
| 8 | Skills & Takeaways | Always — note analogous problems |
| 9 | When to Move On | Mandatory gate |
| 10 | Company Context | Read for companies you’re targeting |
| 11 | Interviewer’s Lens | Always — internalize scorecard language |
| 12 | Level Delta | Self-assess honestly |
| 13 | Follow-ups & Answers | Attempt each cold, then read |
| 14 | Full Solution Walkthrough | After re-implementing |
| 15 | Beyond the Problem | Always — production reality |
Common Mistakes (Do Not Do)
- Reading README.md before the Attempt Gate. Destroys the entire training value.
- Reading all hints at once. Same problem.
- Skipping the “re-implement from scratch” step. Reading code ≠ writing code.
- Skipping the Level Delta self-assessment. This is the single highest-signal section.
- Skimming Section 10 (Company Context). This is where the differentiated content lives.
- Marking “When to Move On” green without honestly checking each box. Calcifies bad habits.
- Not stress-testing your own implementation. A solution that passes LC may have a bug a stress test catches.
- Skipping originals (p75, p77, p79, p85, p87, p90). These are the highest-value problems in the track. They’re original because they don’t exist on LC — meaning your competition hasn’t seen them either.
Time Budget Per Problem
| Difficulty | Cold Attempt | Hints + Re-attempt | Reading + Re-implement | Total |
|---|---|---|---|---|
| Easy | 20 min | 10–20 min | 30 min | ~60–70 min |
| Medium | 25 min | 20–40 min | 45 min | ~90–110 min |
| Hard | 30 min | 30–60 min | 60 min | ~120–150 min |
| Original (any) | 30 min | 20–40 min | 60–90 min | ~110–160 min |
Plan accordingly when fitting problems into the weekly schedule.
The One Rule
If you remember nothing else: the Attempt Gate is not optional. Every other shortcut is recoverable. Skipping the cold attempt is not — it permanently degrades the training signal for that problem.
Month 1 — Foundations
Weeks 1–4 · 20 flagship problems · ~115 LC Bank problems · 1 mock
Goals
By end of month, you can:
- Execute the 16-step framework on every problem without thinking about it
- Solve any LC Easy in <12 min with full edge-case coverage
- Solve straightforward Mediums (array/string/hashmap/tree/basic-DP) in <30 min
- Explain hashmap collision handling, GC, stack vs heap in your primary language
- Pass mock-02 (Easy) at 7+/10
Weekly Map
| Week | Theme | Flagship Problems | Phase Reading |
|---|---|---|---|
| 1 | Execution baseline + first patterns | p01–p05 | phase-00, phase-01 §1–3 |
| 2 | Arrays + strings + hashmaps deep | p06–p10 | phase-01 §1–3 runtime, phase-09 your lang first half |
| 3 | Heaps, binary search, trees | p11–p15 | phase-01 §4–7 |
| 4 | LRU, DFS/BFS, DP intro, backtracking intro | p16–p20 | phase-01 §8–9, phase-09 second half |
End-of-Month Gate
- All 20 flagship problems: Section 9 (“When to Move On”) checklist green
- All Phase 1 mastery checks in phase-01-foundations/README.md pass
- mock-02 scored 7+/10
- Tracking spreadsheet has 20 entries with Level Delta self-assessment
If any item fails: do NOT enter Month 2. Repeat the weakest week’s drilling.
Why These 20 Problems
These are the 20 most-asked Easy-and-low-Medium problems in 2024–2025 Big Tech interview loops (cross-referenced against company tag data, real-time interview reports, and recruiter intel). They are the floor. If you cannot solve all 20 fluently, you will fail the phone screen — not because they are hard, but because failing an Easy in 12 minutes is an instant no-hire signal.
Week 1 — Execution Baseline + First Patterns
Days 1–7 · 5 flagship problems · ~25 LC Bank · 0 mocks
Goals
- Internalize the 16-step framework on trivial problems (so you can use it on hard ones)
- Two-pass over Phase 0 labs
- Acquire HashMap-1-pass, Stack-validation, Greedy-1-pass, Two-pointer-from-end, 1D-DP-intro patterns
Daily Schedule
| Day | Reading | Flagship | Bank |
|---|---|---|---|
| Mon | FRAMEWORK.md re-read; phase-00 labs 1–3 | p01 Two Sum | 3 LC Easies |
| Tue | phase-00 labs 4–5 | p02 Valid Parentheses | 3 LC Easies |
| Wed | phase-00 labs 6–7 | p03 Best Time Buy/Sell | 3 LC Easies |
| Thu | phase-01 §1 Arrays | p04 Merge Sorted Array | 3 LC Easies |
| Fri | phase-01 §2 Strings + §6 Heap intro | p05 Climbing Stairs | 3 LC Easies |
| Sat | Re-solve p01–p05 unaided | — | 5 LC Easies + REVIEW |
| Sun | COMMUNICATION.md + spaced repetition logging | — | 5 LC Easies |
LC Bank (Problems to solve on your own after flagship)
LC 217 (Contains Duplicate), 169 (Majority Element), 268 (Missing Number), 53 (Maximum Subarray), 136 (Single Number), 283 (Move Zeroes), 26 (Remove Duplicates from Sorted Array), 27 (Remove Element), 1 (Two Sum — variant), 9 (Palindrome Number), 14 (Longest Common Prefix), 28 (strStr — naive), 35 (Search Insert Position), 66 (Plus One), 67 (Add Binary), 69 (Sqrt(x) — binary search intro), 88 (Merge Sorted Array — variant), 100 (Same Tree), 101 (Symmetric Tree), 104 (Maximum Depth — preview), 108 (Sorted Array → BST), 112 (Path Sum), 118 (Pascal’s Triangle), 226 (Invert Binary Tree), 543 (Diameter of Binary Tree).
Readiness Gate
- All 5 flagship problems Section 9 checklists green
- 25+ Bank problems solved unaided
- Framework Steps 1–9 executed audibly (talk through) on at least 10 problems
- No off-by-one errors on 5 consecutive binary-search-flavored problems
- Honest self-assessment: Level Delta = Mid or above on at least 3 flagships
p01 — Two Sum
Source: LeetCode 1 · Easy · Topics: Array, Hash Table Companies (2024–2025 frequency): Amazon (very high), Google (high), Meta (high), Apple (medium), Microsoft (medium), Bloomberg (very high) Loop position: phone screen warmup, or first 10 min of onsite to calibrate
1. Quick Context
This is the most-asked Easy in Big Tech history. The interviewer is not testing whether you can solve it — they expect you to solve it in <8 minutes. They are testing whether you:
- Clarify before coding (duplicates? multiple answers? sorted?)
- State the brute force out loud before optimizing
- Pick the right optimal (one-pass hash, not two-pass)
- Handle the “what about the same element twice” edge case
- Communicate cleanly through a problem you’ve obviously seen before
What it looks like it tests: array iteration. What it actually tests: disciplined communication under “I’ve done this a million times” complacency. Senior candidates fail this by going too fast and skipping clarifications. The interviewer is watching for the framework, not the answer.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/two-sum/
STOP. Set a 15-minute timer. Code it cold in a scratch file. Do not read past this section until you have either solved it or the timer expired.
If you’ve solved Two Sum before: do it again anyway, and time yourself. Target: 6 min including narration. If you’re over 8 min, you have a framework/communication gap, not an algorithm gap — and that gap will kill you on harder problems.
3. Prerequisite Concepts
- Hash table average O(1) lookup + the assumption that makes it true (phase-01 §3 HashMap Mastery)
- “Complement search” pattern: instead of looking for pairs, transform to lookup of (target − x)
- In your primary language: what hash collision behavior is, what hash resize cost is — see phase-09
4. How to Approach (FRAMEWORK Steps 1–9 applied)
Step 1 — Restate: “Given an integer array and a target integer, return the indices of two distinct elements that sum to target. Exactly one valid pair exists per problem statement.”
Step 2 — Clarify (ask out loud, do NOT skip even though it’s Easy):
- “Can the same element be used twice?” (No — indices must be distinct.)
- “If multiple pairs sum to target, which one do I return?” (Problem says exactly one solution — but in real interviews they may relax this; ask.)
- “Can the array be empty / size 1?” (Per constraints, N ≥ 2.)
- “Are values bounded? Can they be negative?” (Per LC, yes, both negative and positive in int32 range.)
- “Is the array sorted?” (No. If it were, you’d use two pointers — explicitly call this out; this is a senior signal.)
- “Return indices or values?” (Indices, in any order.)
Step 3 — Constraints: N up to 10^4 in classic statement. O(N²) brute fits, but O(N) is expected. With N up to 10^4, O(N²) = 10^8 — borderline, will get TLE on some servers.
Step 4 — Examples (build your own):
[2,7,11,15], target=9→[0,1](the given one)[3,3], target=6→[0,1]← critical: the “same value, different indices” case[-3,4,3,90], target=0→[0,2](negative handling)[1,2,3,4], target=100→ does not occur per statement, but ask: “if no solution, what do I return?”
Step 5 — Brute Force: Nested loop. For each i, check every j > i. O(N²) time, O(1) space.
Step 6 — Brute Force Complexity: Time O(N²), space O(1). Trivially correct. State this out loud BEFORE optimizing.
Step 7 — Pattern Recognition: “Given an array, find a pair satisfying a property” + “complement is computable” + “order of result doesn’t matter” → HashMap complement search. (Sorted + ordered → two pointers. Unsorted + complement-computable → HashMap.)
Step 8 — Optimize: Walk the array once. For each x at index i, compute complement = target − x. If complement is in the map, return (map[complement], i). Else add x → i to the map. One pass, not two. Two-pass works but signals weaker intuition.
Step 9 — Prove correctness: Loop invariant: after processing index i, the map contains exactly {nums[0..i] : their indices} for every distinct value (or the latest index for duplicates, but since exactly one solution exists, duplicates can only be the answer pair — and at index j, the complement was stored at index i < j, so we find it).
5. Progressive Hints
If you’re stuck for more than 5 minutes, open hints.md and read one hint only. Set another 5-min timer between hints.
6. Deeper Insight — Why It Works
The transformation: “Find two numbers summing to T” is a 2D search (i, j pairs) → O(N²). By computing T − nums[i] for each i, we reduce to a 1D lookup (“does this value exist?”) which is O(1) amortized with a hash table. The hash table is the data structure that turns 2D pair-search into 1D existence-search. This is the master pattern for entire problem families: 3Sum, 4Sum, Subarray-Sum-K, etc., all reduce 1 dimension via complement-hashing.
The single-pass insight: You don’t need to load the full array into the map first. By the time you encounter the second element of the answer pair, the first one has already been stored. So insert AFTER you check — never before, or [3,3] breaks (you’d find 3 → 0 and return [0,0], two equal indices).
Order matters: check first, insert second. Reversing this is the single most common bug. The map state at step i represents “everything seen STRICTLY BEFORE i” — that’s the invariant.
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — Two-pass with enumerate:
d = {x: i for i, x in enumerate(nums)}
for i, x in enumerate(nums):
j = d.get(target - x)
if j is not None and j != i:
return [i, j]
This works but: (a) O(N) extra memory written then read again, (b) the j != i guard signals you didn’t think about why one-pass avoids the issue, (c) for duplicates [3,3], the dict only stores the last index — works only because j != i saves you. Two-pass is a code smell that says “I memorized the pattern but didn’t internalize the invariant.”
Wrong-but-tempting #2 — Sort then two-pointer:
sorted_nums = sorted(enumerate(nums), key=lambda p: p[1])
# ... two-pointer scan
Works, but O(N log N) when O(N) exists. Some candidates do this because two-pointer is their hammer. At Google in particular, choosing N log N when N exists is a signaled-down on the algorithmic complexity rubric.
Wrong-but-tempting #3 — Brute force “to be safe”: The brute force fits N ≤ 10⁴ barely. If the interviewer expands the constraint to 10⁵, brute force times out. They WILL expand the constraint to test you — be ready.
8. Skills & Takeaways
Generalizable pattern: Complement search via HashMap. Any time you see “find pair/triple/group summing or differing or relating to value V”, first ask: can I express the missing piece as a function of what I have? If yes, HashMap that function’s range.
Analogous problems (do these on LC after):
- LC 167 — Two Sum II (sorted input — uses two pointers instead; teaches when NOT to use hashmap)
- LC 653 — Two Sum IV BST (BFS + set; same complement pattern, different traversal)
- LC 1 vs LC 15 (3Sum) — the recursive extension; outer loop fixes one element, reduces to 2Sum
- LC 560 — Subarray Sum Equals K (the prefix-sum complement variant — same idea, applied to running sums)
- LC 454 — 4Sum II (split into two halves, hash one half, lookup complements of the other — meet-in-the-middle flavor)
When NOT to use this: Sorted input (two pointers is O(1) space, hashmap is O(N)). Streaming input where you can’t afford O(N) memory.
9. When to Move On (binary; must all be YES)
- I solved p01 unaided in <8 min including narration on the second attempt
- I can state the loop invariant of one-pass Two Sum without looking
-
I can explain why “check first, insert second” matters, with the
[3,3]example - I can name when two-pointer is preferred over hashmap (sorted input, O(1) space requirement)
-
I implemented this from scratch and my version passes
stress_test()in solution.py - I solved LC 167 (Two Sum II) and articulated why the optimal approach changed
- I solved LC 560 (Subarray Sum K) and recognized the same complement-hashing pattern
If any unchecked: redo tomorrow before moving to p02.
10. Company Context
Amazon (LP-heavy; coding bar = “are bugs going to ship?”)
- The framing: Often given as “given a list of
Orderobjects with apricefield, find two orders whose prices sum to the target promo discount.” - Misleading example: They’ll give you
[5, 5, 5, 5], target=10. Many candidates lock onto “return the first two indices that work” — but Amazon interviewers want you to ASK whether you should return any valid pair or the first in some defined order. Asking shows Customer Obsession (knowing the requirement). - Adversarial extension: “Now there could be millions of orders streamed one at a time. How does your solution change?” → streaming with a HashSet, return the first matching pair.
- What they want to hear: “Let me clarify the requirements before I optimize.” Verbatim phrases like “Let me first state the brute force so we have a baseline” earn rubric points.
Google (algorithmic complexity is a hard rubric line)
- The framing: Often the cleanest, just the original problem.
- Misleading example: A small-N example where O(N²) clearly fits. Google interviewers do this to see if junior candidates will say “brute force is fine” and stop. The right move: “Brute force fits this size, but I want to do better as a habit — and to handle the extension where N is 10⁶.”
- Adversarial extension: “Return all pairs that sum to target.” Now it’s not 1 answer; you must dedupe pairs (3Sum-style logic).
- What they want to hear: Explicit complexity for brute and optimal, stated separately. “O(N²) brute → O(N) one-pass hashmap, O(N) extra space.”
Meta (heavy on follow-ups; expect 3–4 in 25 min)
- The framing: “Two Sum, then variants in rapid succession.”
- Misleading example: They start with the canonical example, then immediately follow up: “Now what if the array is sorted?” If you stayed on hashmap, you scored less than the candidate who pivots to two-pointers.
- Adversarial extension: “What if duplicates are allowed in input AND we want all unique pairs?” → 2Sum-with-duplicates → introduces dedup via sort or a counted-set.
- What they want to hear: Recognition that the optimal algorithm depends on input properties. The phrase “if the input were sorted, I’d use two pointers” wins them over.
Microsoft (clarity + cleanliness over speed)
- The framing: Phone screen warmup.
- Misleading example: None — Microsoft tends to be straightforward here.
- Adversarial extension: “Now make it work for any K-Sum” (K is a parameter).
- What they want to hear: Clean function decomposition, clear variable names, edge-case enumeration. Microsoft phone screens reward boring, correct code.
Bloomberg (financial framing — be ready)
- The framing: “Given a list of trade prices and a settlement amount, find two trades that exactly settle to the amount.”
- Misleading example: Includes negative prices (refunds). Candidates who hard-code
i < jordering may break on negatives if they switch to a sort approach mid-stream. - What they want to hear: Explicit “I’m assuming negative values are allowed; my hash approach handles them naturally.”
11. Interviewer’s Lens
| Phase | Strong signal | Weak signal | Scorecard phrase (strong) |
|---|---|---|---|
| Reading problem | Asks 3+ clarifying questions even though it’s Easy | Says “oh I’ve seen this” and dives in | “Disciplined clarifying behavior — would translate to fewer production bugs” |
| Pre-coding | States brute force, then states optimal, with complexities for both | Jumps to “I’ll use a hashmap” without justifying | “Communicates derivation, not just memorization” |
| Coding | Names variable complement, comments invariant once | Uses d, m, no comments | “Code is interview-readable; would pass our internal code review bar” |
| Edge cases | Tests [3,3] before submission | Tests only the given example | “Self-catches bugs before code review — strong production instinct” |
| Post-coding | Articulates time AND space complexity unprompted | Says only “it’s linear” | “Owns full complexity analysis” |
The scorecard line that gets you the offer: “Candidate demonstrated framework discipline on a trivial problem, suggesting it will scale to hard ones.”
The scorecard line that loses you: “Skipped clarifying questions; rushed to known answer; did not test [3,3]; missed senior signal opportunity.”
12. Level Delta
| Level | What their answer looks like |
|---|---|
| Mid | One-pass hashmap solution. States O(N) time, O(N) space. Tests given example. ~10 min. |
| Senior | All of Mid + clarifies 3+ questions upfront + explicitly states brute force first + tests [3,3] + mentions “if sorted, I’d use two pointers” + completes in ~7 min. |
| Staff | All of Senior + articulates loop invariant before coding + names the complement-search pattern by family + connects to LC 15 (3Sum) and LC 560 (Subarray Sum K) as the same family + mentions hash collision worst-case (O(N²)) as a footnote + completes in ~6 min. |
| Principal | All of Staff + asks “what’s the production context — are these orders, transactions, ad bids?” + identifies that for very large N you’d shard the hashmap or use a Bloom-filter prefilter + mentions GC pressure from large dict allocation as an Amazon/Google production concern + offers the streaming variant unprompted + completes in ~5 min with time to discuss tradeoffs. |
Honest self-assessment: Which level was YOUR answer? If “Mid”, you have 4 sections to add to your toolkit. If “Senior” — good baseline; aim for Staff on the next 5 problems.
13. Follow-up Questions & Full Answers
Follow-up 1: “What if the array is sorted?”
Signal sought: Do you recognize that input properties change the optimal algorithm?
Full answer: “If sorted, I’d switch to two pointers — left=0, right=N-1. If sum > target, decrement right; if sum < target, increment left; if equal, return. O(N) time, O(1) space (better than hashmap’s O(N) space). The correctness comes from monotonicity: incrementing left only increases the sum; decrementing right only decreases. We never miss the answer because at each step, the eliminated half cannot contain the answer.”
Follow-up 2: “What if there are multiple valid pairs and we want all unique ones?”
Signal sought: Can you handle dedup without explosive complexity?
Full answer: “Two approaches. (a) Sort + two pointers + skip duplicates on both sides — O(N log N) time, O(1) extra. (b) Hashmap + use a set of frozensets to dedup result pairs — O(N) time, O(N) extra. Approach (a) is preferred unless we cannot mutate input. The key insight: dedup happens by skipping equal adjacent values after sorting, not by post-filtering.”
Follow-up 3: “Now there are billions of integers streamed one at a time, infinite stream. Detect any sum-pair as fast as possible.”
Signal sought: Streaming / unbounded-input thinking.
Full answer: “Use a HashSet (not Map — we just need existence). For each incoming x: check (target − x) in set; if found, emit the pair; else add x to set. O(1) per element, O(N) total memory grows unboundedly. For unbounded memory: use a Bloom filter as a prefilter (false positives OK; we verify by querying upstream), bounded memory at cost of occasional false alarms. If we need bounded memory AND zero false positives, we accept that we may miss pairs — fundamentally we cannot remember an arbitrary stream. State this tradeoff explicitly.”
Follow-up 4 (Hard): “Distribute across 100 machines. Each holds 1% of the array. Find a pair summing to target.”
Signal sought: Distributed systems thinking, communication cost awareness.
Full answer: “Broadcast the target. Each machine builds a local set of its values. To find cross-machine pairs: each machine emits (target − x, x, machine_id) for each local x, hash-partitioned by (target − x) mod 100 to the responsible machine. That receiving machine checks if it holds the complement. Communication is O(N) total messages, O(N/100) per machine. Latency: 1 shuffle round. Correctness: every valid pair (x, y) where x + y = target is checked because x is hashed by y = target − x to y’s home machine. Caveat: if the array is so large that even local sets don’t fit in RAM, we partition further or stream from disk with external-sort-style processing.”
Follow-up 5 (Senior signal): “How would you test this code?”
Signal sought: Testing discipline, not just unit tests.
Full answer: “Four layers. (1) Unit: given example, [3,3], negatives, two valid pairs choosing one. (2) Edge: minimum N=2, maximum constraint. (3) Property test: random N, random ints, brute-force comparator — assert both find the same pair (modulo order). My solution.py does this via stress_test(). (4) Adversarial fuzz: hash-collision DoS inputs — known pathological inputs that cause O(N²) hashmap behavior. Production code would also include perf regression tests and memory profiling for large N.”
14. Full Solution Walkthrough
See solution.py.
The file has four sections:
two_sum_brute(nums, target)— nested loop, O(N²). This is your correctness oracle.two_sum(nums, target)— the one-pass hashmap. Note the order: check first, insert after. The comment on line marking the invariant.stress_test()— generates 1000 random arrays, runs both, asserts results sum to target equally. This is the bar: every flagship problem has a stress test.__main__— runs the given example,[3,3], negative case, then the stress test.
Decisions justified in the file:
- Why
seen.get(complement)instead ofif complement in seen: return [seen[complement], i]: one hash lookup instead of two. - Why we return as a list
[a, b]not a tuple: matches LC signature exactly. - Why no input validation: per the framework, we validate at system boundaries — interview code assumes valid input per the problem statement.
15. Beyond the Problem — Production Reality
At 10M RPS:
- The hashmap allocation per request becomes a GC pressure point. In production, you’d pool the map or use a primitive-keyed map (e.g.,
Long2IntOpenHashMapin Java’s Eclipse Collections, or adictallocated once and.clear()’d in Python). - For very large N per request, the O(N) memory dominates. Spilling to off-heap or using a compact open-addressing hashmap would matter.
Real system this is the kernel of:
- Ad bid matching: given a target CPM, find two bids that sum to the publisher’s floor. Same algorithm, with bid-objects carrying metadata. Real ad exchanges (Google Ad Exchange, Meta Audience Network) do variants of this billions of times per second.
- Promo discount calculator: e-commerce platforms match “if customer buys X and Y, the bundle costs the promo target.”
- Settlement matching at exchanges: pair buy and sell orders that exactly clear.
Principal-engineer code review comment: “Why is this a one-off function? In our codebase, complement-search-against-hashmap is a building block. Extract find_pair_by_property(items, key_fn, target) so we can reuse for 3Sum, k-Sum, prefix-sum, and the promo-bundle code path. Also: thread safety? If this map is shared across requests, we have a race.”
p01 Two Sum — Progressive Hints
Rule: Read ONE hint. Set a 5-minute timer. Try again. Only read the next hint if the timer expires.
Hint 1 — Pattern family
You don’t need to compare every pair. Think about what information you’d want to have already seen by the time you reach element nums[i].
Hint 2 — Data structure family
For each element x you encounter, there is exactly one “partner value” that would solve the problem. What data structure lets you check in O(1) whether you’ve seen a specific value already?
Hint 3 — The key insight
For each x at index i, the value you need to find is target − x. If you’ve stored every previously seen value with its index, you can ask the question “have I seen target − x before?” in O(1).
Hint 4 — Pseudocode skeleton
seen = empty map (value → index)
for i in 0 to N-1:
let x = nums[i]
let complement = target - x
if complement is in seen:
return [seen[complement], i]
add x → i to seen
Pay close attention to the order of the check and the insert. There is exactly one correct order.
Hint 5 — Full algorithm in words
Walk the array left to right with a hashmap seen that grows as you go. For each element x at index i:
- First, check if
target − xis already a key inseen. If yes, return[seen[target − x], i]— you’ve found the pair, with the earlier index first. - Then, insert
x → iintoseen.
Why check before insert? If you insert first, then for input [3, 3] with target = 6, at index 0 you’d insert 3 → 0, then check for complement 3 and find it — but the only index satisfying it is 0 itself, returning [0, 0], which violates “two distinct indices”. Checking before inserting guarantees the complement was found at a strictly earlier index.
The loop invariant: at the start of iteration i, seen contains exactly the values nums[0..i-1] mapped to their indices.
Time: O(N) — one pass, O(1) per element. Space: O(N) — the map can hold up to N entries.
p02 — Valid Parentheses
Source: LeetCode 20 · Easy · Topics: String, Stack Companies: Amazon (high), Google (medium), Meta (medium), Microsoft (high), Bloomberg (high), Salesforce (very high) Loop position: phone screen, sometimes paired with p80 (Basic Calculator II) at onsite
1. Quick Context
The canonical stack problem. Looks trivial; the trap is the early-return logic and the empty-stack-on-close case. Senior candidates lose points by writing 15 lines when 8 suffice, by forgetting to check the stack is non-empty at end, or by not asking whether non-bracket characters can appear.
What it tests: Disciplined use of the right data structure (stack), not loop-and-counter hacks. Anti-signal: counting opens and closes separately (“balanced count”) — that fails on "(]". If you proposed this and didn’t catch it yourself, you’ve failed the cognitive-trap check.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/valid-parentheses/
STOP. Set a 12-min timer. Solve cold. Do not read on until done or timed out.
3. Prerequisite Concepts
- LIFO semantics of a stack; why “matching last-opened” is intrinsically a stack property — phase-01 §5
- Mapping closed → open via a constant dict (O(1) lookup, more idiomatic than chained
if)
4. How to Approach
Restate: “Given a string of ()[]{} characters, return true iff every opener has a matching closer of the right type, in the right nesting order.”
Clarify:
- “Can the string contain other characters?” (LC: no, only brackets — but ASK, real interviews vary.)
- “Empty string?” (LC: valid = true. Yes, common gotcha.)
- “Maximum length?” (Implies whether we care about stack overflow in recursive solutions; iterative + explicit stack avoids that anyway.)
Constraints: N up to 10⁴. O(N) expected.
Examples to build:
"()[]{}"→ true (sequential pairs)"([{}])"→ true (full nesting)"(]"→ false ← the counter-fail case"]"→ false (close-first; empty-stack case)"(("→ false (unmatched open at end; non-empty-stack-at-end case)""→ true (empty)
Brute force: Repeatedly scan and remove adjacent (), [], {} pairs until no change. If empty, valid. O(N²).
Pattern: “Match most recent unclosed thing” → stack. Period.
Optimal: One pass. For each char: if opener, push. If closer, pop and verify match. At end, stack must be empty.
Proof of correctness: Stack invariant — at any point, the stack contains exactly the unmatched openers in order of opening. A closer matches iff it pairs with the top (most recent opener). If a closer arrives with empty stack, it has no match → false. If non-closer remains at end → unmatched openers → false.
5. Progressive Hints
If stuck >5 min: hints.md. One at a time.
6. Deeper Insight — Why It Works
Why a stack and not a counter? A counter (opens - closes) is sufficient for a single bracket type — but it cannot detect "(]". The stack is necessary because we need to remember not just that a bracket is open, but which type. Each push commits the type; each pop verifies the type. The stack is the minimum data structure that captures both “how many open” and “which order/type.”
Why we check empty BEFORE popping: If we blindly pop an empty stack on a closer, we error out (or in some languages, get undefined behavior). The check if not stack: return False handles "]", "})", etc.
Why we check empty at END: The string "((" walks through pushing twice; loop ends; stack non-empty; without the final check, we’d return True. The end-check is the second necessary correctness step.
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — Count-and-compare:
if s.count('(') == s.count(')') and s.count('[') == s.count(']') and ...:
return True
Fails on "(]" (each count is 1, returns True). This is the single most common bug junior candidates ship. If you proposed this, you missed the type-ORDER requirement.
Wrong-but-tempting #2 — Regex / replace loop:
while '()' in s or '[]' in s or '{}' in s:
s = s.replace('()','').replace('[]','').replace('{}','')
return s == ''
Works but O(N²) and signals you reached for a hammer. Interviewer’s note: “didn’t recognize stack pattern.”
Wrong-but-tempting #3 — Forgetting end-check:
for c in s:
if c in '([{': stack.append(c)
elif c in ')]}':
if not stack or pairs[c] != stack.pop(): return False
return True # ← BUG: returns True for "((" because stack non-empty but no closer caught it
The fix is return not stack. Forgetting this is the #2 bug here.
8. Skills & Takeaways
Generalizable pattern: “Match most recent unresolved item” → stack. Applies to:
- LC 32 — Longest Valid Parentheses (stack of indices, harder cousin)
- LC 84 — Largest Rectangle in Histogram (monotonic stack — same family)
- LC 739 — Daily Temperatures (monotonic stack — same family)
- LC 71 — Simplify Path (each segment as stack frame)
- LC 394 — Decode String (stack of (count, prefix) frames)
- LC 1249 — Minimum Remove to Make Parentheses Valid (stack of indices)
When NOT to use: Single bracket type, just balance counting → counter is sufficient and uses O(1) space.
9. When to Move On
- Solved unaided in <8 min on second attempt
-
Tested
"(]","]","((",""without prompting - Can articulate why a counter fails for multiple bracket types
-
Implemented with constant
closer → openermap (not chained ifs) - Stress test in solution.py passes
- Solved LC 1249 and recognized the same pattern with index tracking
10. Company Context
Amazon
- The framing: Often paired with “and write the matched pairs as (open_idx, close_idx)” — extends to index-tracking variant.
- Misleading example: They give
"{[()]}"and let you confirm true. Then sneak in"{[(])}"— looks balanced character-counts-wise; trips candidates who didn’t catch the type-order failure. - Extension: “Now allow
<>too.” → tests whether your code generalizes (constant dict makes it 1-line; chained ifs make it 4 new branches).
Salesforce (this is THEIR favorite)
- The framing: Often appears in onsite, not phone screen — they care about the cleanliness more than the algorithm.
- What they want to hear: “I’ll use a stack and a dictionary mapping closers to openers.” That sentence alone is a green check.
- Adversarial extension: “What if
[could match)(mixed mode)?” → tests whether your code is data-driven enough to change one line of config.
Microsoft
- The framing: Phone screen warmup, sometimes followed by p80 (Basic Calculator).
- What they want to hear: Single-pass justification, end-of-loop empty-check stated out loud.
Bloomberg
- The framing: Often as part of a JSON/parser problem — “validate this serialized message structure”.
- Extension: “Parse the bracketed expression into an AST.”
11. Interviewer’s Lens
| Phase | Strong | Weak | Scorecard (strong) |
|---|---|---|---|
| Reading | Asks “other characters?” “empty string?” “max length?” | Dives in | “Clarifies the contract before coding” |
| Pre-coding | Names the stack pattern explicitly | Says “I’ll iterate and track” | “Recognizes the canonical pattern” |
| Coding | Uses constant dict {')':'(', ']':'[', '}':'{'} | Chains 6+ if-branches | “Writes data-driven, extensible code” |
| Edge cases | Tests "(]", "]", "" proactively | Tests only "()" | “Anticipates failure modes” |
| Finish | States complexity and stack invariant | Says “done” | “Owns the analysis” |
12. Level Delta
| Level | Answer |
|---|---|
| Mid | Stack solution, works, ~8 min. Tests only the given example. |
| Senior | + clarifies upfront + tests "(]" + uses constant dict + end-empty-check articulated. |
| Staff | + names “most-recent-unresolved” pattern family + connects to monotonic stack problems + addresses “what if char set were configurable” before being asked. |
| Principal | + asks production context (“are we validating Markdown? JSON? code?”) + identifies that real validators need position tracking for error messages + offers extension to track unmatched positions for IDE-style red squiggles + mentions that for adversarial input we’d cap stack depth to prevent DoS. |
13. Follow-up Questions & Full Answers
Q1: “Return the index of the first invalid character instead of true/false.”
Signal: Can you adapt without rewriting?
Answer: Track current index i; when popping mismatches or empty-pop happens, return i. At end if stack non-empty, return the index of the unmatched opener at stack top (so push (char, index) tuples, not just chars). One-pass O(N), no extra time cost.
Q2: “Mixed bracket modes: also accept [) and (] as valid in this protocol.”
Signal: Data-driven code awareness.
Answer: Replace {')':'(', ']':'[', '}':'{'} with {')':'([', ']':'[(', '}':'{'} and change the match check from stack.pop() == pairs[c] to stack.pop() in pairs[c]. Zero algorithm change — only the config dict moves. Demonstrates extensibility.
Q3 (Hard): “Parse and evaluate the matched expression as a Lisp-like AST.”
Signal: Stack-of-frames extension. Answer: Each frame is a list of tokens accumulating between matching opens. On open: push current frame, start new. On close: pop, append the inner frame to the outer. Same stack discipline, now each entry is a list, not a char. This is the bridge from p02 to LC 394 (Decode String) and to p80 (Basic Calculator).
Q4: “Now there are millions of strings per second — same algorithm, but optimize for throughput.”
Signal: Production thinking. Answer: Three wins. (a) Replace Python list-as-stack with a fixed-size preallocated array + integer top pointer — eliminates allocation. (b) Use a 256-entry lookup table mapping char codes to (type, is_opener) instead of dict lookup — branch-free hot loop. (c) For truly hot path, write a SIMD-friendly state machine in C — but only after profiling shows this is the bottleneck.
Q5: “How would you test it?”
Signal: Testing discipline. Answer: (1) Unit: all 6 fail modes + valid + empty. (2) Property test: brute (replace-loop) vs optimal on random bracket strings, must agree. My solution.py does this. (3) Adversarial: deeply nested input (depth = 10⁵) — confirms iterative, not stack-overflowing. (4) Fuzz: random non-bracket chars to confirm the “ask about other characters” clarification.
14. Full Solution Walkthrough
See solution.py.
is_valid_brute: repeatedreplaceloop. Correct but O(N²). Used as oracle.is_valid: stack + closer→opener dict. Three correctness branches: (i) push on open, (ii) check-then-pop on close (empty + mismatch both → False), (iii) final stack-empty check.stress_test: generates random bracket strings (balanced and unbalanced), asserts brute and optimal agree.
15. Beyond the Problem
Real systems this is the kernel of:
- JSON / XML / YAML parsers — every structural parser uses this exact stack discipline. Errors in production parsers (e.g., “Unexpected end of input”) are the end-empty-check firing.
- IDE bracket matching — your editor’s “matching bracket” highlight runs this algorithm on every keystroke, scoped to a window around the cursor.
- Code linters —
eslint’sno-unbalanced-bracketsrule is this with position tracking. - Network protocols with framing (e.g., BSON, MessagePack) use stack-based parsers.
Principal-engineer code review comment: “We have three places in the codebase that re-implement bracket matching (JSON parser, query parser, markdown renderer). They’ve each drifted to handle their edge cases differently. Extract a generic match_pairs(input, pair_table, on_unmatched) and centralize. The bug we hit last quarter was the JSON parser missing the end-check; the markdown one had the check but the wrong error message.”
p02 Valid Parentheses — Progressive Hints
One hint, 5-min timer, retry. Repeat.
Hint 1 — Pattern family
The string is processed left to right. At any point, you have unresolved openers. The closer you read needs to match the most recent unresolved opener — not just any opener. What property does “most recent” have?
Hint 2 — Data structure
LIFO — last-in-first-out. There is exactly one data structure for this.
Hint 3 — Key insight
Push every opener onto a stack. When you see a closer, the top of the stack must be the matching opener — pop and compare types. If it isn’t, or the stack is empty, the string is invalid. At the end of the string, the stack must be empty.
Hint 4 — Pseudocode
stack = []
pairs = { ')':'(', ']':'[', '}':'{' }
for c in s:
if c is an opener:
push c
else if c is a closer:
if stack is empty: return false
if pop() != pairs[c]: return false
return stack is empty
There are exactly three failure branches. Make sure you have all three.
Hint 5 — Full algorithm with the three correctness branches
Three places the answer becomes False — miss any one and you have a bug:
- Mid-loop, empty-stack on close: A closer arrived with no unmatched opener to match. Example:
"]". - Mid-loop, top-mismatch on close: The top of the stack is an opener of the wrong type. Example:
"(]"— top is(, closer is], mismatch. - End of loop, non-empty stack: Openers remain unmatched. Example:
"((".
Use a constant dict {')':'(', ']':'[', '}':'{'} so the match check is one line, not six. This is what the interviewer flags as “data-driven code” — it generalizes when they add <> as a follow-up.
Time O(N), Space O(N) worst case (all openers, no closers).
p03 — Best Time to Buy and Sell Stock
Source: LeetCode 121 · Easy · Topics: Array, DP, Greedy Companies: Amazon (very high), Bloomberg (very high), Facebook (high), Apple (medium), Uber (medium) Loop position: phone screen, or first warmup before harder DP onsite
1. Quick Context
This is the “single-transaction maximum profit” problem and the entry door to a 6-problem ladder (LC 121, 122, 123, 188, 309, 714) that culminates in the hardest stock problems on LC. Mastering p03 properly — by recognizing it as “max forward-difference with constraint i < j” and NOT brute force — unlocks the whole family.
What it looks like it tests: array iteration. What it actually tests: Whether you see the invariant transformation: instead of trying all (buy, sell) pairs, track the minimum buy-price seen-so-far and compute profit-if-sold-today. This is a one-pass O(N), O(1) algorithm; the brute force is O(N²).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/best-time-to-buy-and-sell-stock/
12-min timer. Cold attempt. No reading on.
3. Prerequisite Concepts
- “Running min/max” pattern — phase-02 §3 Prefix Sums (same family — running aggregate)
- Greedy correctness: why local-min + global-max-of-(today - min) is globally optimal
4. How to Approach
Restate: Given prices indexed by day, pick a buy-day i and a sell-day j > i to maximize prices[j] - prices[i]. If no profit possible, return 0. One transaction only.
Clarify:
- “Can I sell on the same day I buy?” (No — strict
j > i.) - “If prices monotonically decrease, return 0 or negative?” (Per LC: 0 — no transaction is valid.)
- “Multiple transactions allowed?” (No, this is LC 121. LC 122 is the multi-transaction version. ASK to confirm.)
- “Length bounds?” (LC: up to 10⁵ → O(N²) will TLE.)
Examples:
[7,1,5,3,6,4]→ 5 (buy at 1, sell at 6)[7,6,4,3,1]→ 0 (no profit possible)[1]→ 0 (no transaction possible — single price)[2,4,1]→ 2 (buy at 2, sell at 4, NOT buy at 1; can’t sell after — common trap)
Brute force: All (i, j) pairs with j > i; track max diff. O(N²) time, O(1) space.
Pattern recognition: “Max value of a[j] - a[i] with i < j” → equivalent to “at each j, what’s the min of a[0..j-1]?” → running-min + per-element subtract → O(N).
Optimal:
min_so_far = +infinity
best = 0
for price in prices:
best = max(best, price - min_so_far)
min_so_far = min(min_so_far, price)
return best
Order matters: Compute best BEFORE updating min_so_far. Otherwise on a single day you’d allow “buy and sell same day” (price - price = 0, no harm here but ON LC 122 the equivalent bug causes phantom profits).
Correctness proof (greedy): For any optimal pair (i*, j*) with j* > i*, when the loop reaches j*, min_so_far ≤ prices[i*] (because i* ≤ j*-1, and min_so_far covers prices[0..j*-1]). So prices[j*] - min_so_far ≥ prices[j*] - prices[i*] = optimal. Since we take the max over all j, we capture this value.
5. Progressive Hints
hints.md — one at a time, 5-min timer.
6. Deeper Insight — Why It Works
The transformation: A 2D search “find (i, j) max difference” becomes 1D “at each j, what’s the best historical min?” by recognizing that for any fixed j, the optimal i is argmin(prices[0..j-1]). We don’t need to remember which i — just its value. This is the same compression that powers Kadane’s algorithm (maximum subarray): instead of trying all subarrays, track the best one ending here.
Why O(1) space: The running-min subsumes all history we need. We never look back; we only look at the current price vs. the cheapest ever.
Connection to Kadane’s algorithm: If you compute diffs[i] = prices[i+1] - prices[i], then LC 121 becomes “max sum subarray over diffs” — Kadane’s algorithm. This equivalence is a Staff-level observation.
7. Anti-Pattern Analysis
Wrong #1 — Two pointers from both ends: Some try left = 0, right = N-1, shrink to find max. Doesn’t work: the optimal pair isn’t necessarily at the extremes.
Wrong #2 — Sort: Sorting destroys the time-order constraint. The buy-day must come before the sell-day in original order.
Wrong #3 — Update min before max:
for p in prices:
min_so_far = min(min_so_far, p) # ← wrong order
best = max(best, p - min_so_far)
On LC 121, gives same answer because max profit on day i if buy=sell=i is 0. But on the multi-tx variants this exact bug allows “buy and sell at same instant,” inflating profit.
Wrong #4 — Greedy “buy every local min”: Confuses LC 121 (single transaction) with LC 122 (multiple). Read the prompt.
8. Skills & Takeaways
Pattern: running-min/max + per-element decision. Direct applications:
- LC 122 — Buy/Sell II (multiple tx → sum positive diffs greedily)
- LC 123 — Buy/Sell III (at most 2 tx → DP over states)
- LC 188 — Buy/Sell IV (at most k tx → generalized DP)
- LC 309 — with cooldown (state machine DP)
- LC 714 — with fee (state machine DP)
- LC 53 — Maximum Subarray (Kadane — same family via diff transform)
- LC 152 — Maximum Product Subarray (track both running min and max because negatives)
9. When to Move On
- Solved unaided <8 min, O(N), O(1)
- Tested decreasing input, single-element, two-element
- Articulated the “running min + per-step profit” transformation
- Connected to Kadane’s algorithm via the diff trick
- Stress test passes
- Solved LC 122, LC 53; saw the family resemblance
10. Company Context
Amazon
- The framing: “You’re shown daily stock prices for a company. What’s the most an investor could have made with one buy and one sell?”
- Misleading example: They give
[2, 4, 1]to bait the “buy at 1” mistake. The trap: 1 is the LAST day, no sell possible after. - Adversarial extension: “Now they can do multiple transactions” (→ LC 122) immediately followed by “now there’s a $1 fee per transaction” (→ LC 714). Tests whether you generalize cleanly.
Bloomberg (terminal company — they LIVE on time-series)
- The framing: Often pure LC 121, sometimes with timestamps not indices (test if you read the order from the input).
- What they want: Recognition that this is a time-series invariant problem. The phrase “I’ll track the running minimum” is a green check.
- Extension: “What if prices stream in?” → same algorithm; the running-min works incrementally.
Meta
- The framing: Followed RAPIDLY by LC 122, then “at most k transactions” (LC 188).
- What they want: You finish LC 121 in 5 minutes so they get to LC 188. If you spend 15 min on LC 121, you’ll never reach the real interview question.
Uber
- Frame: “Surge pricing history — best moment to launch a promotional ride.”
- Extension: “What if some days are weekends and weekend buys are forbidden?” → tests masking, not algorithm.
11. Interviewer’s Lens
| Phase | Strong | Weak | Scorecard |
|---|---|---|---|
| Reading | Confirms “single transaction” + “j > i strict” | Assumes multi-tx | “Verifies the contract” |
| Pre-coding | States O(N²) brute, then O(N) optimal with proof sketch | Jumps to “iterate and track” | “Derives, doesn’t memorize” |
| Coding | Updates best before min_so_far | Wrong order, gets lucky on LC 121 | “Subtle correctness awareness” |
| Edge | Tests decreasing array, single price | Tests only sample | “Anticipates degenerate inputs” |
| Finish | Connects to Kadane / LC 122 family | Says “done” | “Sees the pattern family” |
12. Level Delta
| Level | Answer |
|---|---|
| Mid | One-pass running-min, ~10 min. O(N), O(1). Correct. |
| Senior | + clarifies single vs multi tx + tests decreasing array + states correctness invariant. |
| Staff | + names the “max subarray of diffs” equivalence (Kadane) + offers to immediately extend to LC 122. |
| Principal | + asks production context (algo trading? backtest? UI dashboard?) + notes that real backtests need transaction fees, slippage, position size — and that “max profit” alone is a wrong metric (drawdown, Sharpe matter) + mentions that on real streams you’d window the running-min for non-stationarity. |
13. Follow-up Questions & Full Answers
Q1: “Now allow unlimited transactions.” → LC 122.
Answer: Sum every positive consecutive diff: sum(max(0, prices[i+1] - prices[i]) for i in range(N-1)). Proof: any concave-up segment between local min and local max contributes (max - min); the sum of positive diffs equals the sum of these contributions. O(N) one pass, O(1) space.
Q2: “At most k transactions.” → LC 188.
Answer: DP. State: dp[t][i] = max profit using ≤ t transactions through day i. Transition: dp[t][i] = max(dp[t][i-1], max over j<i of (dp[t-1][j-1] + prices[i] - prices[j])). Naively O(k·N²); optimize the inner max to O(1) by tracking max(dp[t-1][j-1] - prices[j]) as we scan, giving O(k·N). When k ≥ N/2, problem degenerates to unlimited tx (LC 122) — handle this case separately.
Q3: “What about transaction fee f per buy-sell pair?” → LC 714.
Answer: State machine. hold[i] = max profit ending day i holding a stock; cash[i] = max profit ending day i not holding. hold[i] = max(hold[i-1], cash[i-1] - prices[i]). cash[i] = max(cash[i-1], hold[i-1] + prices[i] - f). Pay the fee when selling. O(N), O(1) with two scalars.
Q4: “Streaming prices, infinite stream. Output the running best-possible-profit-so-far.”
Answer: Same algorithm, incremental. Maintain min_so_far and best. On each tick, update both. The answer is best at all times. O(1) per tick.
Q5: “How do you test it?”
Answer: (1) Edge: empty (or N=1) → 0, decreasing → 0, increasing → last - first. (2) Property: brute-force O(N²) vs optimal on random arrays. (3) Adversarial: arrays where the buy is on day 0 vs day N-2 — tests whether running-min update timing is right.
14. Full Solution Walkthrough
See solution.py.
Three solutions for didactic value:
max_profit_brute: all pairs, O(N²). Oracle.max_profit: running-min one-pass, O(N), O(1).max_profit_kadane: Kadane on diffs, to demonstrate equivalence.
All three should agree under stress_test.
15. Beyond the Problem
Real systems this is the kernel of:
- Backtest engines (Zipline, Backtrader): the “perfect foresight” upper bound used to score strategies. A strategy can’t beat single-transaction-max-profit on a sequence; this is the benchmark.
- A/B test analysis: “what’s the largest sustained delta we observed?” — same running-min/max.
- Latency/throughput dashboards: “what’s the largest drop in throughput?” — running-max + per-point delta.
Principal-engineer code review comment: “This algorithm assumes prices is a value-typed array. In our pipeline, prices are timestamped events with possible gaps. Either resample to fixed-interval before feeding, or change the algorithm to work on (timestamp, price) tuples and handle missing intervals. Also: what’s the contract when prices contains NaN (market closed)? Define it explicitly.”
p03 Best Time to Buy/Sell Stock — Progressive Hints
One hint at a time. 5-min timer between hints.
Hint 1 — Reframe the problem
You’re searching for the maximum of prices[j] - prices[i] over all valid (i, j) with i < j. That’s a 2D search. Can you reduce one dimension by asking: “for a fixed j, what is the best i?”
Hint 2 — What helps at each j
For each candidate sell-day j, the most profitable buy-day is whichever earlier day had the lowest price. So at each j, you only need to know one thing about the past: the minimum price among days 0..j-1.
Hint 3 — The key insight
Walk left to right. Maintain min_so_far = minimum of all prices seen before today. For each price today: candidate profit is price - min_so_far; update the running best. Then update min_so_far with today’s price.
This makes the algorithm O(N) time and O(1) space.
Hint 4 — Pseudocode
min_so_far = +infinity
best = 0
for price in prices:
best = max(best, price - min_so_far)
min_so_far = min(min_so_far, price)
return best
Pay attention to the order of those two updates inside the loop. One order is right, the other has a subtle bug visible only on harder follow-ups.
Hint 5 — Full algorithm + order rationale
Initialize min_so_far = +∞ (so the first iteration’s price - min_so_far is very negative and doesn’t pollute best, which starts at 0). For each price:
- First: compute
best = max(best, price - min_so_far). This uses the min from STRICTLY BEFORE today — we cannot sell on the same day we buy. - Then: update
min_so_far = min(min_so_far, price). Now today’s price becomes available for future sell-days.
If you reverse the order (update min first, then compute best), you’d compute price - price = 0 on the day a new min appears, which is harmless here but causes phantom self-trades on the multi-transaction variants. Get the discipline right now.
The correctness proof: for any optimal pair (i*, j*), when the loop reaches j*, min_so_far ≤ prices[i*] (because i* < j*). So the algorithm sees at least the optimal profit, and we take the max, so we capture it.
Time O(N), Space O(1).
p04 — Merge Sorted Array
Source: LeetCode 88 · Easy · Topics: Array, Two Pointers, Sorting Companies: Facebook (high), Bloomberg (high), Microsoft (high), Adobe (medium), Apple (medium) Loop position: phone screen or onsite warmup; often paired with a follow-up to merge k arrays (LC 23 — Merge k Sorted Lists)
1. Quick Context
A deceptively simple problem with a sharp trick: you must merge in place in nums1 (which has extra trailing zeros). The naive approach (merge from the left) requires shifting → O(N²). The optimal approach (merge from the right, writing the largest elements into the back) is O(M+N), O(1) extra.
What it tests: in-place pointer manipulation and the recognition that writing backwards avoids the overwrite problem. The trap: candidates instinctively merge from the front and either get O(N²) or need an extra buffer (O(N) space) and forget that the prompt forbids it.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/merge-sorted-array/
12-min timer. Cold attempt. The “merge from the back” insight should be the first thing you reach for — if not, that’s a strong signal you need this rep.
3. Prerequisite Concepts
- Two-pointer technique — phase-02 §1
- “In-place transformation” pattern — when extra buffer is forbidden
4. How to Approach
Restate: nums1 has length m + n. Its first m entries are valid, sorted; the last n are placeholder zeros. nums2 has n valid sorted entries. Merge so that nums1 holds all m + n values sorted. Modify nums1 in place.
Clarify:
- “Can I use O(N) extra space?” (Usually no — the whole point is in-place. ASK.)
- “Are duplicates allowed?” (Yes; merge is stable.)
- “Can
mornbe 0?” (Yes. Both edge cases.) - “Are negative numbers allowed?” (Yes, per constraints.)
Examples:
nums1=[1,2,3,0,0,0], m=3, nums2=[2,5,6], n=3→[1,2,2,3,5,6]nums1=[1], m=1, nums2=[], n=0→[1](n=0 edge)nums1=[0], m=0, nums2=[1], n=1→[1](m=0 edge)nums1=[4,5,6,0,0,0], m=3, nums2=[1,2,3], n=3→[1,2,3,4,5,6](all of nums2 < all of nums1 — tests handling when nums2 isn’t exhausted)
Brute force: Copy nums2 into the tail of nums1, then sort. O((M+N) log (M+N)). Trivially correct but wastes the “already sorted” property.
Pattern: Two sorted sequences + in-place + extra room at the END → merge from the back, largest first.
Optimal:
i = m - 1 # pointer into nums1's valid part
j = n - 1 # pointer into nums2
write = m + n - 1 # pointer into nums1's tail (where to write next)
while j >= 0: # only need to keep going while nums2 has elements
if i >= 0 and nums1[i] > nums2[j]:
nums1[write] = nums1[i]
i -= 1
else:
nums1[write] = nums2[j]
j -= 1
write -= 1
Why we loop on j >= 0, not i >= 0: If nums2 is exhausted, the remaining nums1[0..i] is already in its final position (since we wrote bigger elements to the right). If nums1’s valid prefix is exhausted (i < 0), we must still copy remaining nums2 elements. The loop condition reflects which case requires action.
Correctness: At each step we write the larger of the two unprocessed maxima into the next free slot from the right. The slot is always at or past the “write frontier”, which is at index i + j + 1. Since write = i + j + 1 decreases monotonically and i + j is the count of unprocessed elements minus 2, the slot is never one we still need to read from. No overwrite.
5. Progressive Hints
hints.md. One at a time.
6. Deeper Insight — Why It Works
The reverse-merge insight: Merging from the front into nums1 would overwrite valid nums1 data before reading it (because the write pointer would catch up to the read pointer for nums1). Merging from the back writes into slots that are guaranteed unused (either they’re trailing zeros, or they’re slots holding values already moved).
The invariant: At any point, nums1[write+1 .. m+n-1] contains the largest (m+n-1 - write) elements of the final merged result, in sorted order. nums1[0..i] and nums2[0..j] are the unprocessed prefixes.
Why i+j+1 == write always: Initially i+j+1 = (m-1) + (n-1) + 1 = m+n-1 = write. Each iteration decrements write by 1 and decrements either i or j by 1, so the relation holds. Therefore write is always one past i+j, the position right after the unprocessed prefixes — guaranteed empty. This is the formal proof of “no overwrite.”
7. Anti-Pattern Analysis
Wrong #1 — Front merge with shift:
i, j = 0, 0
while j < n:
if i < m and nums1[i] <= nums2[j]:
i += 1
else:
# shift nums1[i..m-1] right to make room
nums1.insert(i, nums2[j]) # O(M) shift!
nums1.pop() # O(M)
i += 1
m += 1
j += 1
Correct but O((M+N) × M) — quadratic. Senior interviewers see this and ask “can you do better?” — if you can’t pivot to the reverse-merge, you’ve revealed a gap.
Wrong #2 — Copy nums1 valid prefix to an aux buffer:
aux = nums1[:m]
# then merge aux and nums2 into nums1 from the front
Correct and O(M+N) time but O(M) extra space. Violates the in-place spirit — passes LC but loses interview points.
Wrong #3 — Sort after concat:
nums1[m:] = nums2
nums1.sort()
O((M+N) log (M+N)). Passes LC. But: “you ignored the ‘sorted input’ property — what was the point of this problem?”
Wrong #4 — Forget the i >= 0 guard in the comparison:
while j >= 0:
if nums1[i] > nums2[j]: # IndexError when i goes negative
The guard if i >= 0 and nums1[i] > nums2[j] matters when nums1’s valid prefix is exhausted before nums2.
8. Skills & Takeaways
Pattern: write-backwards-to-avoid-overwrite. Direct applications:
- LC 26 / 27 / 80 — Remove Duplicates / Remove Element (write-forward variant)
- LC 283 — Move Zeroes
- LC 977 — Squares of a Sorted Array (write from back: largest absolute value at each end)
- LC 167 — Two Sum II (two-pointer from both ends; sibling family)
The “merge from back” trick generalizes: Any in-place merge where one container has trailing room becomes O(N) by writing backward. Used in some qsort partition tricks and in compaction passes of generational garbage collectors.
9. When to Move On
- Solved unaided <10 min using reverse-merge
- Tested m=0, n=0, all-of-nums2-smaller, equal elements
- Articulated why front-merge fails (overwrite) and back-merge succeeds (slot guaranteed free)
- Stress test passes
- Solved LC 977; saw the same write-backwards idea
10. Company Context
Facebook / Meta
- Frame: Often appears as a warmup before LC 23 (Merge k Sorted Lists). The interviewer expects you to do p04 in 5 min, then they ask “now generalize to k arrays.”
- What they want: Reverse-merge offered without prompting. The phrase “I’ll merge from the back to avoid shifting” is a green check.
- Trap: They give you
nums1with extra space already allocated. Candidates who treatnums1as size-m and try to grow it don’t see the “trailing zeros” hint.
Bloomberg
- Frame: Often as merge of two sorted price-tick streams (“merge into a single time-ordered view”).
- Extension: “Now they may have duplicate timestamps — preserve order from stream A first.” Tests stable-merge awareness.
Microsoft
- Frame: Phone screen warmup. They watch your pointer manipulation cleanliness.
- What they want: Correct, no off-by-one, comment on the loop condition. Boring + correct = pass.
Adobe
- Frame: Often given with the explicit constraint “no extra memory”. Tests whether you internalize the in-place requirement.
11. Interviewer’s Lens
| Phase | Strong | Weak | Scorecard |
|---|---|---|---|
| Reading | Notices the trailing zeros are intentional padding | Ignores zeros, treats nums1 as size-m | “Reads spec carefully” |
| Pre-coding | States “merge from back to avoid overwrite” | Plans front-merge with shifting | “Recognizes the optimal pattern” |
| Coding | Three pointers (i, j, write); correct guards | Off-by-one, IndexError | “Disciplined pointer code” |
| Edge | Tests m=0, n=0, all-A-bigger | Tests only sample | “Tests boundary conditions” |
| Finish | Articulates the i+j+1 == write invariant | Says “done” | “Proves correctness” |
12. Level Delta
| Level | Answer |
|---|---|
| Mid | Reverse-merge, O(M+N), works. ~10 min. |
| Senior | + clarifies extra-space constraint + tests m=0/n=0 + articulates why reverse direction avoids overwrite. |
| Staff | + states the i+j+1 == write invariant formally + offers to extend to merge-k-lists with a heap + notes that stable merge order matters in some applications. |
| Principal | + asks production context (database compaction? log merge?) + notes that real merge-sort tape algorithms used this exact pattern in pre-RAM era + identifies that for cache efficiency, even on RAM, sequential write patterns matter and reverse-merge here is sequential-from-the-end (still cache-friendly). |
13. Follow-up Questions & Full Answers
Q1: “Now merge k sorted arrays into one.” → LC 23 family
Answer: Min-heap of (value, array_id, element_id). Pop smallest, append to output, push the next from the same array. O(N log k) where N is total elements. For very large k, consider tournament tree (same complexity, lower constant).
Q2: “What if nums1 doesn’t have extra room at the end — same length as valid data?”
Answer: No way to merge in place in O(M+N). You’d need O(min(M,N)) extra space minimum (proof: there must be storage for the merge frontier). Standard out-of-place merge.
Q3: “What if both arrays are huge and stored on disk?”
Answer: External merge sort’s merge phase. Read both arrays in buffered chunks; merge into output buffer; flush when full. Classical pattern. The key complexity unit becomes I/O, not comparisons.
Q4: “Merge two sorted linked lists in place.” → LC 21
Answer: Two-pointer with rewiring. Maintain a dummy head + a tail pointer. At each step, point tail.next to the smaller of a.val, b.val, advance that pointer. After the loop, append whatever remains. O(M+N), O(1) extra.
Q5: “How do you test?”
Answer: (1) Edge: m=0, n=0, all-A-smaller, all-A-larger, equal elements, single-element each. (2) Property: brute (sort after concat) vs optimal on random sorted inputs, must agree element-for-element. (3) Adversarial: very different sizes (m=1, n=10⁵) — confirms loop conditions handle imbalance.
14. Full Solution Walkthrough
See solution.py.
merge_brute(nums1, m, nums2, n): append nums2 then sort. Oracle.merge(nums1, m, nums2, n): reverse-merge with three pointers. Mutatesnums1in place.stress_test: random sorted arrays of random sizes; brute vs optimal must produce identicalnums1.
15. Beyond the Problem
Real systems this is the kernel of:
- Database compaction: LSM trees (LevelDB, RocksDB, Cassandra) periodically merge sorted SSTables. The merge phase is exactly this algorithm, scaled to disk-resident sorted runs with buffered I/O.
- Log merging: distributed tracing systems (Jaeger, Zipkin) merge per-shard time-sorted spans into a global view.
- External sort: when data exceeds RAM, sort runs in memory, write to disk, then merge runs — same algorithm.
- Generational garbage collectors: compaction passes merge live objects into a destination region; the “write backwards” idea appears in some collectors to avoid overwriting unmoved objects.
Principal-engineer code review: “If this is being called in a tight loop, the algorithm is fine but the API is wasteful — caller must allocate nums1 with extra capacity. Consider a builder pattern that allocates the output buffer once and reuses it, especially if M and N vary. Also: the function silently assumes nums1 has length m+n; add a precondition or a contract test, or someone will pass an under-sized array and you’ll get a confusing IndexError instead of a clear contract violation.”
p04 Merge Sorted Array — Progressive Hints
One hint. 5-min timer. Retry. Repeat.
Hint 1 — Why front-merge fails
If you merge from the front (smallest first) and write into nums1[0], you overwrite valid nums1[0] data before reading it. To avoid this, you’d need to shift nums1’s data right — that’s O(M) per insert, total O(M·N).
What part of nums1 is guaranteed safe to write into without overwriting valid data?
Hint 2 — The free space
The trailing zeros in nums1 are a hint. Those positions are reserved for the merge output. If you write the largest unprocessed element into the rightmost free slot, you’re writing into space that’s guaranteed not to contain valid data.
Hint 3 — Three pointers
Use:
i = m - 1— pointer into the valid part ofnums1(largest unprocessed value)j = n - 1— pointer intonums2write = m + n - 1— where the next-largest goes
At each step: compare nums1[i] and nums2[j]. The larger one goes to nums1[write]. Decrement the source pointer and write.
Hint 4 — Pseudocode
i = m - 1
j = n - 1
write = m + n - 1
while j >= 0:
if i >= 0 and nums1[i] > nums2[j]:
nums1[write] = nums1[i]
i -= 1
else:
nums1[write] = nums2[j]
j -= 1
write -= 1
Two non-obvious choices:
- Loop condition is
j >= 0, noti >= 0. Why? - The comparison guards
i >= 0explicitly. Why?
Hint 5 — Full reasoning
Why loop on j >= 0 (not i >= 0)?
When nums2 runs out, the remaining nums1[0..i] is already where it needs to be — sorted, and to the left of all written slots. No work needed. But if nums1’s valid prefix runs out first (i < 0), we must keep copying from nums2. So j >= 0 is the condition that captures “still work to do.”
Why the i >= 0 guard in the comparison?
Once i < 0, accessing nums1[i] is reading from index -1 — in Python that wraps to the last element (a written slot), corrupting comparisons. The i >= 0 and ... short-circuits to the else branch (copy from nums2), correctly handling the case.
The invariant that makes it work:
At every step, write == i + j + 1. That means nums1[write] is the position immediately AFTER the unprocessed prefixes — guaranteed empty/safe to overwrite. This invariant is the formal proof of “no overwrite.”
Time O(M+N), Space O(1).
p05 — Climbing Stairs
Source: LeetCode 70 · Easy · Topics: Math, DP, Memoization Companies: Adobe (high), Amazon (medium), Apple (medium) Loop position: the canonical “intro to DP” — almost always a warmup before a real DP question
1. Quick Context
The “Fibonacci in disguise” problem. It’s the cleanest possible 1D DP and exists to teach you the recipe: (1) define state, (2) write recurrence, (3) identify base cases, (4) decide order, (5) compress space. If you can’t articulate those 5 steps on p05, you can’t on p51 (House Robber), p55 (Jump Game), p60 (Longest Increasing Subsequence), or anywhere else.
What it looks like it tests: ability to recognize Fibonacci. What it actually tests: the DP recipe. Interviewers use this as a calibration — if you write recursion without memoization (O(2^N)), or memoization without realizing iteration is simpler, you’ve revealed your DP fluency level.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/climbing-stairs/
10-min timer. Cold attempt. If you reach for recursion-without-memo, that’s the signal you need this rep most.
3. Prerequisite Concepts
- Recursion + memoization basics — phase-01 §8
- “State + recurrence + base case” — the universal DP recipe
- Space compression: when only the last
kstates are needed, store only those
4. How to Approach
Restate: From step 0, reach step n, taking either 1 or 2 steps at a time. How many distinct ways?
Clarify:
- “Distinct sequences of moves, or distinct step sets?” (Sequences —
1+2and2+1are different. ASK; some problems differ.) - “n ≥ 1?” (Per LC: 1 ≤ n ≤ 45.)
- “Why n ≤ 45?” (Hints at int overflow in some languages; in Python no issue. Worth a verbal acknowledgment.)
Examples:
n=1→ 1 (just[1])n=2→ 2 ([1,1],[2])n=3→ 3 ([1,1,1],[1,2],[2,1])n=4→ 5 (Fibonacci pattern starts to emerge: 1, 2, 3, 5)n=5→ 8
Brute force (recursive): f(n) = f(n-1) + f(n-2), base f(1)=1, f(2)=2. O(2^N) time without memo — exponential.
Pattern recognition: The recurrence is Fibonacci. Anywhere you see “ways to reach state N built from prior states” with overlapping subproblems → DP.
Optimal: Iterative DP, two scalars (prev1, prev2). O(N) time, O(1) space.
if n <= 2: return n
prev2, prev1 = 1, 2 # f(1), f(2)
for i in range(3, n+1):
prev2, prev1 = prev1, prev1 + prev2
return prev1
Recurrence justification: To reach step n, the last move was either a 1-step (from step n-1) or a 2-step (from step n-2). These are disjoint sets of paths, so total = f(n-1) + f(n-2).
5. Progressive Hints
hints.md. One at a time.
6. Deeper Insight — Why It Works
The DP recipe applied:
- State:
f(n)= number of distinct ways to reach stepn. - Recurrence:
f(n) = f(n-1) + f(n-2)(last move was 1 or 2). - Base cases:
f(1) = 1,f(2) = 2. (Why two base cases? Recurrence reaches back two steps; we need both grounded.) - Order: Bottom-up (iteratively i = 3 .. n) because
f(n)depends only on smalleri. - Space compression: Only the last two values are needed, so two scalars suffice — O(1) space.
Why two base cases, not one? The recurrence f(n) = f(n-1) + f(n-2) requires f(n-2) to be defined when n=3, so we must specify both f(1) and f(2). Trying to derive f(2) from f(0) + f(1) requires defining f(0) = 1 (the empty path) — a valid alternative formulation, but the natural problem space starts at step 1.
Closed form (Staff signal): Fibonacci has a closed-form Φ^n / √5 (Binet’s formula). For interview purposes, mention it exists but don’t compute it — floating-point loses precision for n > ~70 and the problem caps at 45 anyway.
Matrix exponentiation (Staff signal): Fibonacci can be computed in O(log N) via [[1,1],[1,0]]^n. Overkill for n ≤ 45; useful when n is large or when the recurrence generalizes (e.g., f(n) = a·f(n-1) + b·f(n-2)).
7. Anti-Pattern Analysis
Wrong #1 — Plain recursion, no memo:
def climb(n):
if n <= 2: return n
return climb(n-1) + climb(n-2)
O(2^N). At n=45, ~35 billion calls. TLE for sure. Interviewer’s note: “didn’t recognize overlapping subproblems.”
Wrong #2 — Memo with dict overhead:
@lru_cache
def climb(n):
if n <= 2: return n
return climb(n-1) + climb(n-2)
Works in O(N) but uses O(N) call stack and dict memory. Iterative is strictly better — no recursion limit risk, O(1) space. Mention lru_cache exists but show why iterative wins.
Wrong #3 — DP table O(N) space:
dp = [0]*(n+1)
dp[1], dp[2] = 1, 2
for i in range(3, n+1): dp[i] = dp[i-1] + dp[i-2]
return dp[n]
Correct, O(N) time, O(N) space. Misses the O(1) compression. Senior signal: notice you only need the last 2 values, drop the array.
Wrong #4 — Off-by-one on base cases:
Forgetting n==1 returns 1, not 2; or forgetting to handle n==2 directly and letting the loop run. Tests with n=1, 2, 3 catch these.
8. Skills & Takeaways
The DP recipe — burn it in:
- State definition
- Recurrence with justification
- Base cases (one per step the recurrence reaches back)
- Computation order
- Space compression (only keep what’s referenced)
Generalizations — variants you’ll see:
- LC 746 — Min Cost Climbing Stairs (same structure, minimize cost instead of count)
- LC 198 — House Robber (same structure with “can’t be adjacent” constraint)
- LC 213 — House Robber II (circular variant)
- LC 91 — Decode Ways (Fibonacci with validity guards)
- LC 1137 — N-th Tribonacci (extend recurrence to 3 prior terms)
- LC 509 — Fibonacci Number (literal Fibonacci)
When you can take 1, 2, or 3 steps: f(n) = f(n-1) + f(n-2) + f(n-3) (Tribonacci). When you can take 1..k steps: f(n) = sum f(n-i) for i=1..k → can be O(N) with a sliding window.
9. When to Move On
- Solved iteratively, O(N) time, O(1) space, <8 min
- Tested n=1, n=2, n=3
- Articulated all 5 DP recipe steps without prompting
- Connected to House Robber (LC 198) as the same pattern with cost
- Stress test passes
- Mentioned closed-form / matrix exp exist (Staff signal)
10. Company Context
Adobe (this is their go-to DP warmup)
- Frame: Often given verbatim.
- What they want: Iterative O(1) space, articulated DP recipe. Recursion without memo is a near-instant fail.
- Extension: “What if you can take 1, 2, or 3 steps?” → Tribonacci. Tests generalization.
Amazon
- Frame: Sometimes as a paving / tiling problem (“ways to tile a 1×N corridor with 1×1 and 1×2 tiles”) — same Fibonacci underneath.
- Extension: “Now there’s a step you can’t land on (broken stair).” → DP with constraint:
f(broken) = 0, recurrence unchanged otherwise.
Apple
- Frame: Often paired with LC 198 (House Robber) — they want to see if you generalize.
11. Interviewer’s Lens
| Phase | Strong | Weak | Scorecard |
|---|---|---|---|
| Reading | Asks “sequences vs sets” + clarifies n bounds | Dives in | “Verifies the contract” |
| Pre-coding | Names the recurrence with justification | Says “I’ll recurse” | “Articulates DP structure” |
| Coding | Iterative + 2 scalars | Recursive without memo | “Optimal space upfront” |
| Edge | n=1, n=2 covered | Misses n=1 | “Catches base-case bugs” |
| Finish | Mentions DP recipe steps + extensions | Says “done” | “Frameworks generalize” |
12. Level Delta
| Level | Answer |
|---|---|
| Mid | Iterative O(N), O(1). Works. ~8 min. |
| Senior | + states the 5-step DP recipe explicitly + tests n=1,2,3 + offers to extend to k-step. |
| Staff | + mentions closed-form (Binet) and matrix exponentiation as O(log N) alternatives + connects to House Robber as same family with different reduction operator (sum → max). |
| Principal | + asks production context (“counting paths in an event graph? combinatorics for a recommender?”) + identifies that for very large n we need modular arithmetic (problem caps prevent overflow but real systems don’t) + suggests matrix exponentiation under modulo for n in the millions. |
13. Follow-up Questions & Full Answers
Q1: “What if you can take 1, 2, or 3 steps?”
Answer: Tribonacci. f(n) = f(n-1) + f(n-2) + f(n-3), base cases f(1)=1, f(2)=2, f(3)=4. O(N) time, O(1) space with three scalars.
Q2: “What if you can take 1..k steps?”
Answer: f(n) = sum f(n-i) for i in 1..k. Naive O(N·k). Optimize via sliding window: f(n) = f(n-1) + f(n-1) - f(n-1-k) (because consecutive windows overlap in k-1 terms). O(N), O(k) space.
Q3: “Now some steps are broken — you can’t land there.”
Answer: f(broken[i]) = 0. Recurrence unchanged. Still O(N), O(1) — just a guard inside the loop.
Q4: “Same problem but n can be up to 10^18.”
Answer: Matrix exponentiation. [f(n), f(n-1)]^T = M^(n-1) · [f(1), f(0)]^T where M = [[1,1],[1,0]]. Compute M^(n-1) via binary exponentiation in O(log N) matrix multiplications, each O(1) for a 2×2 matrix. Total O(log N). Use modular arithmetic to prevent overflow.
Q5: “How do you test it?”
Answer: (1) Small cases against hand-computed Fibonacci values (1, 2, 3, 5, 8, 13). (2) Property: brute recursive (with memoization) vs iterative agree on n in 1..30. (3) Stress on the boundary n=1 (off-by-one trap) and n=45 (max constraint).
14. Full Solution Walkthrough
See solution.py.
climb_brute(n): plain recursion, no memo. Used as oracle for small n only (would TLE for large).climb(n): iterative two-scalar version. O(N) time, O(1) space.climb_memo(n): top-down memoization, included for comparison. O(N) time, O(N) space.stress_test: brute vs optimal vs memo on n ∈ [1, 25], all three agree.
15. Beyond the Problem
Real systems this is the kernel of:
- Path-counting in DAGs: number of distinct routes through a graph. The 1D variant generalizes to the topological order of a DAG.
- State-machine reachability: number of distinct token sequences leading to a state in a parser.
- Combinatorial enumeration in recommender systems: counting compatible item sequences under constraints.
- Probabilistic state aggregation: when transition probabilities are uniform, counts and probabilities differ only by a normalizing constant.
Principal-engineer code review: “This function is correct but its name climb_stairs is so specific that it won’t be reused. The underlying primitive — counting paths under additive recurrence — appears in three other places in our codebase, each reimplemented. Extract count_paths(n, allowed_step_sizes) and use it everywhere. Also: cache the result if called repeatedly with the same n — but for n ≤ 45 the compute cost is negligible, so probably not worth the cache complexity.”
p05 Climbing Stairs — Progressive Hints
One hint. 5-min timer. Retry.
Hint 1 — Decompose by last move
To reach step n, what was your last move? Only two possibilities. Each leads to a sub-problem of the same shape.
Hint 2 — Recurrence
Let f(n) = number of distinct ways to reach step n. The last move was either a 1-step (so the previous state was step n-1) or a 2-step (so the previous state was step n-2). These are disjoint sets of paths.
What does this give you?
Hint 3 — Recognize the pattern
f(n) = f(n-1) + f(n-2) — Fibonacci. You only need to track the last two values, so you don’t need an array.
Hint 4 — Pseudocode
if n <= 2: return n
prev2, prev1 = 1, 2 # f(1), f(2)
for i in 3..n:
prev2, prev1 = prev1, prev1 + prev2
return prev1
Make sure you handle n == 1 and n == 2 as base cases — the loop doesn’t execute for them, so they need explicit returns.
Hint 5 — The full DP recipe (apply this to EVERY DP problem from now on)
- State:
f(k)= number of distinct ways to reach stepk. - Recurrence:
f(k) = f(k-1) + f(k-2)— justified by case-split on the last move. - Base cases:
f(1) = 1,f(2) = 2. (Two base cases because the recurrence reaches back 2.) - Order: Bottom-up, smaller
kfirst. - Space: Only last two values referenced → two scalars, O(1) space.
The most common interview mistakes:
- Plain recursion without memo (O(2^N) — TLE).
- DP array of size N+1 (works, but signals you didn’t see the compression).
- Forgetting one base case (off-by-one on
n=1orn=2). - Confusing this with “distinct step sets” instead of “distinct sequences” —
1+2and2+1are different here.
Time O(N), Space O(1).
Week 2 — Linked Lists + Kadane Intro
Days 8–14 · 5 flagship problems · ~25 LC Bank · 0 mocks
Theme
Pointer mechanics + your first non-trivial DP pattern. The linked-list problems exist almost entirely so the interviewer can watch you draw boxes and arrows on a whiteboard without panicking. Kadane is your first taste of “the DP recurrence hides inside a single scan.”
Goals
- Pointer manipulation without losing nodes (reverse, merge, cycle-detect)
- The three-pointer dance for in-place reversal (prev / curr / next)
- Floyd’s tortoise-and-hare and why it provably terminates
- Kadane’s algorithm as a 1D DP with two-state collapse to scalars
- HashSet as the brute-force baseline for duplicate detection — then trade off
Daily Schedule
| Day | Reading | Flagship | Bank |
|---|---|---|---|
| Mon | phase-01 §4 Linked Lists | p06 Reverse Linked List | 3 LC Easies |
| Tue | phase-01 §4 (re-read), focus on dummy-node pattern | p07 Merge Two Sorted Lists | 3 LC Easies |
| Wed | phase-02 §1 Two Pointers | p08 Linked List Cycle | 3 LC Easies |
| Thu | phase-05 §1 1D DP intro | p09 Maximum Subarray (Kadane) | 3 LC Easies |
| Fri | phase-01 §3 HashMap Mastery re-read | p10 Contains Duplicate | 3 LC Easies |
| Sat | Re-solve p06–p10 unaided | — | 5 LC Easies + REVIEW |
| Sun | COMMUNICATION.md re-read + spaced repetition logging | — | 5 LC Easies |
LC Bank (Problems to solve on your own after flagship)
LC 234 (Palindrome Linked List), 876 (Middle of Linked List), 83 (Remove Duplicates from Sorted List), 203 (Remove Linked List Elements), 160 (Intersection of Two Linked Lists), 92 (Reverse Linked List II — Medium), 19 (Remove Nth Node from End), 142 (Linked List Cycle II — Medium), 121 (Best Time, again, for muscle memory), 122 (Best Time II — greedy), 53 (Maximum Subarray — your flagship), 152 (Maximum Product Subarray — Medium, the “hidden two-state” follow-up), 198 (House Robber), 70 (Climbing Stairs, again), 746 (Min Cost Climbing Stairs), 217 (Contains Duplicate — your flagship), 219 (Contains Duplicate II — sliding-window preview), 268 (Missing Number), 136 (Single Number — XOR trick), 387 (First Unique Character), 242 (Valid Anagram), 49 (Group Anagrams — preview), 599 (Minimum Index Sum of Two Lists), 350 (Intersection of Two Arrays II), 202 (Happy Number — Floyd on integers).
Readiness Gate
- All 5 flagship problems Section 9 checklists green
- 25+ Bank problems solved unaided
- Can draw the three-pointer reverse on a whiteboard from memory in <60 seconds
- Can explain why tortoise-and-hare must meet inside a cycle (modular arithmetic argument)
-
Can state Kadane’s recurrence as
dp[i] = max(nums[i], dp[i-1] + nums[i])and the two-scalar collapse - Honest self-assessment: Level Delta = Senior or above on at least 2 flagships (Kadane is the hard one)
What’s intentionally NOT here yet
- Doubly linked lists — Phase 1 has it; flagship coverage is in Month 2 (LRU Cache).
- Reverse Linked List II (range reverse) — in the Bank; flagship treatment comes in Month 3.
- Cycle II (find the start of the cycle) — Bank only; the math derivation is Month 3 flagship.
- Kadane variants (max product, circular subarray) — Bank previews; flagship treatment in Month 5 DP block.
p06 — Reverse Linked List
Source: LeetCode 206 · Easy · Topics: Linked List, Recursion Companies (2024–2025 frequency): Microsoft (very high), Amazon (very high), Meta (high), Apple (high), Google (medium), Bloomberg (high) Loop position: phone screen warmup; also the “first 5 min” of harder list problems (LC 25, LC 92, LC 234)
1. Quick Context
The linked-list version of FizzBuzz. The interviewer is not testing whether you can do it — they are testing whether you can do it without losing a node. The signature failure mode is curr = curr.next AFTER you’ve already overwritten curr.next to point backward — now you have no way to advance. Every “reverse in place” problem in interview lore (groups of K, reverse-between-indices, palindrome-check-by-reversing-half) reuses this kernel.
What it looks like it tests: linked list manipulation. What it actually tests: can you hold three references in your head at once and execute the swap in the right order under pressure? Many candidates know “use prev / curr / next” but write the four lines in the wrong order and silently produce a list of length 1.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/reverse-linked-list/
STOP. Set a 12-minute timer. Code BOTH the iterative and recursive versions in a scratch file. Do not read past this section until either is solved or the timer expired.
If you’ve done this before: do it again, and time only the iterative version. Target: 4 min, zero scratch paper, written correctly first try. The recursive version should follow in another 4 min. If iterative takes >6 min, you don’t have the three-pointer dance memorized — fix that before moving on.
3. Prerequisite Concepts
- Singly linked list node model:
nextpointer, sentinelNone(phase-01 §4 Linked List Pointers) - In-place mutation vs. allocation tradeoff: this problem can be solved by building a new list (O(N) extra space) OR by pointer rewiring (O(1) extra). Interviewers expect O(1).
- Recursion stack frame cost: the recursive version uses O(N) stack space; on real production systems this stack-overflows around N=10^4 in CPython without
sys.setrecursionlimitbump.
4. How to Approach (FRAMEWORK Steps 1–9 applied)
Step 1 — Restate: “Given the head of a singly linked list, return the head of the same list with all next pointers reversed.”
Step 2 — Clarify:
- “Is the list empty? Single node?” (Both possible; handle separately.)
- “Should I mutate the input or return a new list?” (Mutate in-place is the expected answer; ask anyway.)
- “Doubly linked or singly?” (Singly per LC; doubly is a trivially different rewrite.)
- “Is there a risk of cycles in the input?” (No per LC, but in real production code this is a question worth asking.)
Step 3 — Constraints: N up to 5000 per LC. Both O(N) iterative and O(N) recursive fit. Iterative is preferred for space.
Step 4 — Examples:
1 → 2 → 3 → 4 → 5→5 → 4 → 3 → 2 → 11 → 2→2 → 1(the smallest non-trivial case)1→1(single node — does your code crash on this?)None→None(empty list — does your code crash on this?)
Step 5 — Brute Force: Build a Python list of all values, walk it in reverse, build a new linked list. O(N) time, O(N) space. State this aloud, then propose the in-place version.
Step 6 — Brute Force Complexity: Time O(N), space O(N). Correct but rejects the “in-place” implicit requirement.
Step 7 — Pattern Recognition: “Mutate linked list pointers without losing nodes” → three-pointer dance (prev, curr, next_node). This is the kernel pattern; every harder linked-list problem reuses it.
Step 8 — Optimize: Three pointers. Initialize prev = None, curr = head. Loop: save next_node = curr.next BEFORE overwriting; rewrite curr.next = prev; advance prev = curr and curr = next_node. Stop when curr is None. Return prev.
Step 9 — Prove correctness: Loop invariant: after iteration k, the first k nodes of the original list have been reversed and form a sublist rooted at prev. The remaining N − k nodes form the original tail rooted at curr. When curr becomes None, all N nodes have been processed and prev is the new head.
5. Progressive Hints
If stuck for 5 minutes, open hints.md. One hint, 5-min timer, retry.
6. Deeper Insight — Why It Works
The fundamental constraint: a singly linked list only lets you traverse forward. To reverse, you need to remember the forward direction before destroying it. The next_node = curr.next save is not optional — it’s the entire trick. Without it, you overwrite curr.next to point backward, and then curr = curr.next walks you backward instead of forward, into infinite loop or off the tail.
Why exactly three pointers: you need prev (where to point backward to), curr (which node am I rewriting), and next_node (where to go after rewriting). Two pointers leaves you with the “lost the rest of the list” bug. Four pointers is overengineering.
Recursion’s elegance vs. cost: the recursive version is a one-liner expression: “reverse the rest, then make my old next point to me, then set my next to None.” It’s a clean expression of the invariant: reverse(head) returns the new head of the reversed sublist starting at head. The cost is O(N) stack frames — and the system stack is not a debugger-friendly place to be at N=10^5.
Why O(1) space matters in interviews: the iterative version proves you understand that linked lists are already using O(N) memory; allocating O(N) more is a doubling. The interviewer’s brain registers this as “this person thinks about memory.”
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — Forget to save next_node first:
prev, curr = None, head
while curr:
curr.next = prev # we just lost access to the rest of the list
prev = curr
curr = curr.next # this is now prev, infinite loop or crash
The single most common bug. Write the lines in the order: SAVE → REWIRE → ADVANCE-PREV → ADVANCE-CURR. Memorize this sequence.
Wrong-but-tempting #2 — Build a stack of values, then rebuild the list:
vals = []
while head:
vals.append(head.val)
head = head.next
# rebuild
dummy = ListNode()
curr = dummy
for v in reversed(vals):
curr.next = ListNode(v)
curr = curr.next
return dummy.next
Works, but: (a) O(N) extra space, (b) allocates N new nodes, doubling memory traffic, (c) telegraphs to the interviewer “I don’t know how to mutate pointers.” For a phone screen, you fail the “demonstrate pointer competence” signal.
Wrong-but-tempting #3 — Recursive without base case for empty:
def reverse(head):
if head.next is None: # crashes on head = None
return head
new_head = reverse(head.next)
head.next.next = head
head.next = None
return new_head
Must guard if head is None or head.next is None. Many candidates write only the latter and crash on empty input.
Wrong-but-tempting #4 — Recursive but forget head.next = None:
After head.next.next = head, the old tail still has head.next pointing forward, creating a cycle of length 2 at the tail. The result list has a cycle and every subsequent operation (length, traversal) hangs. This is a classic silent bug — the function returns, but the structure is corrupt.
8. Skills & Takeaways
Generalizable pattern: “Three-pointer in-place pointer rewrite.” Whenever you mutate a pointer that you also need to follow, save the destination FIRST. This pattern reappears in:
- LC 92 — Reverse Linked List II (reverse between indices)
- LC 25 — Reverse Nodes in k-Group (Hard; reuse this kernel inside a loop)
- LC 234 — Palindrome Linked List (reverse second half, compare)
- LC 143 — Reorder List (split, reverse second half, weave)
- LC 24 — Swap Nodes in Pairs (degenerate k=2 case)
- LC 86 — Partition List (rewires by partition; same caution about lost references)
When NOT to use this: Doubly linked list (you have prev pointers already, swap them in one step). Immutable / persistent list (build new structure; O(N) space is required).
9. When to Move On (binary; must all be YES)
- I wrote the iterative version in <5 min with zero pointer-loss bugs on the second attempt
- I wrote the recursive version and handled both empty and single-node bases
- I can recite the line order (SAVE → REWIRE → ADVANCE-PREV → ADVANCE-CURR) without looking
-
I can explain why the recursive version sets
head.next = Noneat the end (cycle prevention) -
My
stress_test()in solution.py passes 1000 iterations on both versions - I solved LC 92 (Reverse Linked List II) and recognized this kernel inside it
- I solved LC 234 (Palindrome Linked List) and saw why “reverse the second half” is the right move
If any unchecked: redo before moving to p07.
10. Company Context
Microsoft (loves this problem — phone-screen mainstay)
- The framing: “Reverse a singly linked list. Both iterative and recursive, please.”
- Misleading example: They show you
1 → 2 → 3and you smash out the iterative version in 90 seconds. They then say: “Now do it recursively.” If you can’t, you lose senior signal. - Adversarial extension: “Now reverse only the first K nodes” (LC 25 lead-in). Then “now in groups of K with the remainder left in original order.”
- What they want to hear: Verbal walkthrough of the three pointers BEFORE coding. Microsoft phone screens use this as a calibration tool — if you can’t articulate the dance, the loop position never opens.
Amazon (Leadership Principles + correctness)
- The framing: Often inside a story problem: “Given a list of order processing steps in reverse, return them in forward order.”
- Misleading example:
[A, B, C]where you might be tempted to use a Pythonreversed()builtin. Amazon expects you to demonstrate the algorithm, not a one-liner. - Adversarial extension: “What if the list is too large to fit in memory and we’re streaming it from disk?” → split into chunks, reverse each, then reverse the chunk order (external-sort-style thinking).
- What they want to hear: “Let me handle empty input first.” Amazon weights
Customer Obsessionheavily — and customers send empty inputs.
Meta (rapid-fire follow-ups)
- The framing: Reverse → palindrome check → reorder → reverse-in-K-groups, all in 25 minutes.
- Misleading example: Often starts with “given the head of a list, return it in reverse order” — ambiguous between “return values reversed” (O(N) space, trivial) and “reverse the structure in place.” Ask.
- Adversarial extension: “Reverse just the alternating nodes.” Now you maintain two interleaved sublists.
- What they want to hear: “Let me trace through a length-2 example to verify my pointer updates” — explicit dry-run before submission. Meta interviewers love the dry-run signal.
Google (recursive elegance matters here)
- The framing: Iterative is acceptable, but the follow-up is always “now do it recursively, and explain the stack-depth tradeoff.”
- Misleading example: They might give you a generic
Nodetype withchildren: List[Node](LC 116, 117 territory) — listen carefully whether it’s singly linked or n-ary. - What they want to hear: Big-O analysis of stack depth. Phrase: “Recursive uses O(N) stack space; for N > 10^4 in production we’d switch to iterative or trampoline.”
Apple / Bloomberg (clarity above all)
- The framing: Often combined with “now print the list reversed without modifying it” — to test whether you can use a stack, OR walk and recurse, vs. mutate.
- What they want to hear: Clear distinction between “reverse the structure” and “iterate in reverse order.” Many candidates conflate these.
11. Interviewer’s Lens
| Phase | Strong signal | Weak signal | Scorecard phrase (strong) |
|---|---|---|---|
| Reading problem | Asks “in-place or return new list?” | Assumes in-place silently | “Clarifies mutation contract — production-mindful” |
| Pre-coding | Draws prev/curr/next_node on whiteboard before typing | Starts coding immediately | “Plans pointer state before mutation — disciplined” |
| Coding | Writes SAVE → REWIRE → ADVANCE in fixed order | Hesitates on line order | “Has memorized the kernel pattern — efficient” |
| Edge cases | Tests empty, single, two-node before submitting | Tests only the given example | “Self-catches pointer bugs — would catch in code review” |
| Follow-up (recursive) | Implements and explains stack cost | “I prefer iterative” without depth | “Owns both forms and their tradeoffs” |
The scorecard line that gets you the offer: “Demonstrated mastery of the three-pointer rewrite kernel; would pass it down to junior engineers.”
The scorecard line that loses you: “Lost a pointer mid-loop; required interviewer to debug; suggests inattention to detail.”
12. Level Delta
| Level | What their answer looks like |
|---|---|
| Mid | Iterative version, possibly with one stumble on line order. Recursive version with prompting. Handles empty input after being asked. ~10 min. |
| Senior | Iterative version in 4 min, clean variable names (prev, curr, next_node), handles empty/single without prompting. Recursive version with correct head.next = None to prevent cycle. Articulates O(1) vs O(N) space. ~8 min total. |
| Staff | All of Senior + names the kernel (“three-pointer in-place rewrite”) + connects to LC 25 and LC 92 as reusing this kernel + mentions stack-depth concern for recursive in production + ~6 min total. |
| Principal | All of Staff + asks about cycles in input (production data hygiene) + mentions doubly-linked list version (trivial swap of prev/next) + offers tail-recursive form and notes CPython lacks TCO so iterative still wins + connects to immutable persistent lists (Clojure, Erlang) where reversing is O(N) by construction + ~5 min with full discussion. |
Honest self-assessment: Did you lose a pointer? If yes, you’re Mid. Did you handle empty without prompting? If yes, Senior floor. Did you mention the recursive cycle bug unprompted? Staff signal.
13. Follow-up Questions & Full Answers
Follow-up 1: “Now reverse the list recursively. What’s the space cost?”
Signal sought: Comfort with recursion + complexity literacy.
Full answer: “Base case: if head is None or head.next is None, return head. Recursive: new_head = reverse(head.next); this returns the new head of the reversed sublist starting from the second node. Then head.next.next = head makes the old second node point back to head, and head.next = None cuts the forward link to prevent a 2-cycle at the tail. Return new_head. Time O(N), space O(N) for the call stack. In CPython that stack-overflows around N=1000 by default; we’d raise sys.setrecursionlimit(10_000) or rewrite iteratively in production.”
Follow-up 2: “What if it’s a doubly linked list?”
Signal sought: Adaptation, not just memorization.
Full answer: “For each node, swap its prev and next pointers. After processing all nodes, the original tail is the new head — return it. Single pass, O(1) extra space, no save-then-rewire needed because we have prev access already. The code is dramatically simpler — which is the point: data structure shape determines algorithm shape.”
Follow-up 3: “What if I want to reverse only the middle K elements?” (LC 92)
Signal sought: Can you reuse the kernel inside a more complex algorithm?
Full answer: “Two phases. (1) Walk to the (left-1)th node, save it as before. (2) From the leftth node, reverse exactly right - left + 1 nodes using the same three-pointer dance. (3) Stitch: before.next.next = curr (the first reversed node’s next now points to what’s after the reversed segment); before.next = prev (the new head of the reversed segment). Use a dummy node before head to handle left = 1 uniformly. O(N) time, O(1) space.”
Follow-up 4 (Hard): “Reverse the list in groups of K. If the last group has fewer than K nodes, leave it in original order.” (LC 25)
Signal sought: Composition of the kernel under bookkeeping pressure.
Full answer: “Outer loop: at each group start, walk K nodes ahead to check there are at least K; if not, break and leave the rest alone. Inner loop: reverse exactly K nodes using the kernel. Stitch each reversed group to the previous group’s tail using a group_prev pointer. The whole algorithm is O(N) — each node is touched at most twice (once for the K-check, once for reversal). Edge cases: K=1 (no-op), K=N (full reverse), N % K != 0 (last partial group untouched).”
Follow-up 5 (Senior signal): “How would you test this?”
Signal sought: Testing as engineering discipline.
Full answer: “Four layers. (1) Unit: empty, single, two-node, length-N (small and large). (2) Round-trip property: reverse twice equals identity — strongest invariant test for any reversal. (3) Cycle detection on output: walk the result with Floyd’s; should terminate at None. This catches the head.next = None bug in the recursive version. (4) Memory profile under large N: confirm no node allocation in the iterative version. My solution.py includes the round-trip property test.”
14. Full Solution Walkthrough
See solution.py.
The file has five sections:
ListNode— minimal node class with__repr__for debugging.reverse_iterative(head)— the three-pointer dance with INVARIANT comment. This is the production-grade answer.reverse_recursive(head)— for didactic comparison; bumpssys.setrecursionlimitfor stress test.stress_test()— generates 1000 random lists, runs both implementations, asserts round-trip property (reverse(reverse(x)) == x) and that values match expected reversal.edge_cases()— empty, single, two-node, all-same-value, large.
Decisions justified in the file:
- Why we return
prevnotcurr: at loop exit,curr is Noneandprevis the last node visited, which is the new head. - Why iterative is preferred: O(1) space, no stack-overflow risk.
- Why
next_nodeis a local variable per iteration, not a class field: scope discipline; only the inner loop needs it.
15. Beyond the Problem — Production Reality
At scale:
- In a network stack (Linux
sk_buffchains, packet queues), reversing a packet list is real — e.g., for retransmission ordering. The iterative kernel is exactly what’s used; the kernel doesn’t tolerate stack-blowing recursion. - In garbage collectors using Cheney’s algorithm, you walk and rewire pointers in-place as you copy — same “save the destination before overwriting” discipline.
Real systems this is the kernel of:
- Undo stacks: many text editors store edits as a linked list; “redo all undones” reverses the list.
- Compiler IR transformations: SSA phi-node reordering rewires successor lists in-place.
- Database transaction logs: replay order sometimes requires walking the WAL in reverse; if it’s a forward-only list on disk, you read it all, reverse the records list, and replay.
Principal-engineer code review comment: “The recursive version is elegant but I’d reject it in our codebase. Our linked lists can hit 10^5 elements in production payloads — we’ve seen stack overflows in similar code. The iterative version is what ships. Also: please add a debug assertion that the returned list has the same length as the input — we had a production incident where a reversal silently dropped a node due to a refactor of the loop guard.”
p06 Reverse Linked List — Progressive Hints
Rule: Read ONE hint. Set a 5-minute timer. Try again. Only read the next hint if the timer expires.
Hint 1 — Pattern family
You’re going to rewrite next pointers in place. Before you overwrite a pointer, ask: “do I still need it?” If yes, save it somewhere first.
Hint 2 — Data structure / state
You need to track THREE things at every step: where you came from (so the current node can point there), where you are (the node being rewired), and where you were about to go (so you don’t lose the rest of the list).
Hint 3 — The key insight
Walk left to right. At each node, the previously-seen node becomes its new next. But before you set curr.next = prev, you must remember the old curr.next because that’s the only way to advance.
Hint 4 — Pseudocode skeleton
prev = None
curr = head
while curr is not None:
next_node = curr.next # SAVE
curr.next = prev # REWIRE
prev = curr # ADVANCE prev
curr = next_node # ADVANCE curr
return prev
Memorize this order: SAVE → REWIRE → ADVANCE-PREV → ADVANCE-CURR. Swapping any two breaks it.
Hint 5 — Full algorithm + recursive variant
Iterative: as in Hint 4. The loop invariant: at the top of each iteration, prev is the head of the already-reversed prefix, curr points to the next unprocessed node of the original list. When curr is None, all nodes have been processed and prev is the new head.
Recursive: the elegant one-liner expression of the same invariant.
def reverse(head):
if head is None or head.next is None:
return head
new_head = reverse(head.next) # reverse the rest first
head.next.next = head # old second node now points back to head
head.next = None # CRITICAL — cut forward link, prevent 2-cycle
return new_head
Two base cases: empty (head is None) and single node (head.next is None). Forget either and you crash or recurse forever. The line head.next = None is non-obvious; without it you create a cycle and the next traversal hangs.
Time O(N) both versions. Iterative space O(1), recursive space O(N) — and Python’s default recursion limit (1000) makes recursive risky for long lists; use sys.setrecursionlimit(10_000) if you must.
p07 — Merge Two Sorted Lists
Source: LeetCode 21 · Easy · Topics: Linked List, Recursion Companies (2024–2025 frequency): Amazon (very high), Microsoft (very high), Apple (high), Google (high), Meta (medium), Bloomberg (high) Loop position: phone screen warmup; the kernel of merge-sort (LC 148), K-way merge (LC 23), and external sort
1. Quick Context
The “merge” of merge sort, on linked lists. The interviewer is testing whether you reach for the dummy/sentinel node pattern without prompting, because dummy nodes are how senior engineers eliminate special cases. The candidate who writes if result is None: result = ...; tail = ... four times has not internalized the pattern. The one who writes dummy = ListNode(); tail = dummy and never special-cases anything has.
What it looks like it tests: merging two sorted lists. What it actually tests: do you use the dummy-node pattern reflexively? Can you stitch the leftover tail in O(1) instead of walking it node-by-node?
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/merge-two-sorted-lists/
STOP. Set a 12-minute timer. Code it cold. Do not read past this section until solved or expired.
If repeat: target 5 min including narration. Iterative + dummy node. If you wrote any if head is None branches inside the loop, you’ve missed the lesson — redo with a sentinel.
3. Prerequisite Concepts
- Linked list node basics (phase-01 §4)
- Sentinel / dummy node pattern: allocate one throwaway node at the start so the “first node” of the output isn’t a special case. This pattern reappears in every list/tree problem where insertion at head differs from insertion anywhere else.
- The “leftover tail” optimization: when one list is exhausted, the other’s remaining suffix is already sorted — just point at it; don’t iterate.
4. How to Approach (FRAMEWORK Steps 1–9 applied)
Step 1 — Restate: “Given two sorted (non-decreasing) singly linked lists, return a single sorted list formed by splicing their nodes.”
Step 2 — Clarify:
- “Splice (reuse input nodes) or allocate new nodes?” (Splice is expected — O(1) extra space. Ask anyway.)
- “Either list empty? Both empty?” (Yes, both possible.)
- “Stable for equal values? Does order within a value matter?” (Usually no; ask if data has meaning beyond
val.) - “Sorted ascending or descending?” (Ascending per LC.)
Step 3 — Constraints: N1 + N2 up to 100 per LC, but real production sizes are unbounded. O(N1+N2) is the bar.
Step 4 — Examples:
[1,2,4] + [1,3,4]→[1,1,2,3,4,4][] + []→[][] + [0]→[0][5] + [1]→[1,5](a < b throughout — leftover tail kicks in immediately)[1,1,1] + [1,1,1]→[1,1,1,1,1,1](all equal — tie-breaking)
Step 5 — Brute Force: Walk both lists, push all values into a Python list, sort, rebuild a linked list. O((N1+N2) log (N1+N2)) time, O(N1+N2) space. Throws away the sortedness — explicitly inferior. State this aloud to anchor that you know it’s wrong.
Step 6 — Brute Force Complexity: O(N log N) time, O(N) space. Misses the linear opportunity.
Step 7 — Pattern Recognition: “Two sorted sequences → one merged sorted sequence” → two-pointer merge (the inner loop of merge sort). On linked lists this also requires dummy node for clean output construction.
Step 8 — Optimize: Dummy node + tail pointer. While both l1 and l2 are non-None, append the smaller (l1 if l1.val <= l2.val else l2) to tail.next, advance that list and tail. After the loop, attach whichever is non-None (the “leftover tail” — point, don’t walk). Return dummy.next.
Step 9 — Prove correctness: Loop invariant: after each iteration, dummy.next ... tail is a sorted list containing exactly the consumed nodes from l1 and l2, and both l1 and l2 still point to sorted remainders whose first elements are ≥ tail.val. When one becomes None, the other’s remainder is sorted and ≥ tail — attaching it preserves sortedness.
5. Progressive Hints
If stuck for 5 minutes, open hints.md. One hint, 5-min timer, retry.
6. Deeper Insight — Why It Works
Why dummy node: without it, the first append is a special case (if result is None: result = node; tail = node). With it, the first append is identical to every other (tail.next = node; tail = tail.next). One pattern, zero special cases. This is the entire reason senior code looks cleaner than junior code on linked list problems — the senior reaches for sentinels without thinking.
Why <= not < for stability: if you write <, ties from l1 get placed AFTER ties from l2, breaking stability with respect to input order. For pure numerical merge it doesn’t matter, but for sorted lists of objects (rows, records) with equal keys, stability is a real correctness property. The Pythonic principle: write the version that generalizes.
Why “leftover tail” matters: after one list exhausts, the other is already a sorted linked list. Walking it node-by-node to append each is O(remainder) wasted work AND extra pointer writes. Just doing tail.next = whichever_is_not_none is O(1) and is the move that makes the algorithm exactly O(N1+N2), not “amortized” or “approximately”.
Why this is merge-sort’s inner loop: merge sort on a linked list (LC 148) recursively splits and then calls THIS exact function on each pair. Merging K sorted lists (LC 23) uses this repeatedly inside a min-heap. The kernel is small and gets reused; learn it cleanly.
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — No dummy node, special-case the first append:
result = None; tail = None
while l1 and l2:
if l1.val <= l2.val:
if result is None: result = l1; tail = l1
else: tail.next = l1; tail = l1
l1 = l1.next
else:
if result is None: result = l2; tail = l2
else: tail.next = l2; tail = l2
l2 = l2.next
# ... and you need similar special-casing for the leftover
Works but it’s a code smell. The dummy node version is half the lines and zero branches in the loop. Senior interviewers see this and write “didn’t use idiomatic sentinel” on the scorecard.
Wrong-but-tempting #2 — Allocate new nodes:
while l1 and l2:
if l1.val <= l2.val:
tail.next = ListNode(l1.val); l1 = l1.next
else:
tail.next = ListNode(l2.val); l2 = l2.next
tail = tail.next
Allocates N new nodes, doubling memory. Splicing existing nodes is O(1) extra space. For interview, this is a “didn’t optimize space” mark. For production with 10M nodes, it’s a memory pressure incident.
Wrong-but-tempting #3 — Walk the leftover tail node by node:
while l1: tail.next = l1; tail = tail.next; l1 = l1.next
while l2: tail.next = l2; tail = tail.next; l2 = l2.next
Works, but the second loop is unnecessary — exactly one of l1, l2 is non-None after the merge loop, and you can attach the whole remainder with tail.next = l1 or l2. The over-engineered version wastes pointer writes and obscures that the leftover is already sorted.
Wrong-but-tempting #4 — Recursive without the splice:
def merge(a, b):
if not a: return b
if not b: return a
if a.val <= b.val:
a.next = merge(a.next, b); return a
b.next = merge(a, b.next); return b
This is actually GOOD — clean, elegant, and demonstrates recursion. But: O(N1+N2) stack depth, which stack-overflows in Python around N=1000. Iterative is the production choice; recursive is the “show off elegance, then mention the stack cost” choice. Both should be in your repertoire.
8. Skills & Takeaways
Generalizable patterns:
- Dummy/sentinel node: use whenever the “first” element is structurally different from the rest. Reappears in every list mutation problem.
- Two-pointer merge: the inner loop of merge sort, of streaming joins in databases, of timeline-merge in log aggregation systems.
- Leftover tail attachment: any merge of two already-sorted sequences leaves one as a prepared suffix; attach in O(1).
Analogous problems:
- LC 23 — Merge K Sorted Lists (Hard; min-heap of heads, call this kernel K-1 times)
- LC 148 — Sort List (merge sort on a linked list; this is the merge step)
- LC 88 — Merge Sorted Array (the array version, with the in-place-from-end trick — your p04)
- LC 1019 — Next Greater Node In Linked List (different problem, but reuses the dummy-node-for-result pattern)
- LC 86 — Partition List (two dummy nodes, splice into two lists, then merge)
When NOT to use this: unsorted inputs (sort first, OR use a different algorithm). Streaming inputs of unbounded size where you can’t hold both heads.
9. When to Move On (binary; must all be YES)
- I solved the iterative version in <5 min on the second attempt
-
I used a dummy node and have ZERO
if tail is Nonechecks inside the loop - I attached the leftover tail in O(1) (not by walking)
-
I can explain why
<=matters for stability - I can write the recursive version and state its O(N) stack cost
-
My
stress_test()passes 1000 iterations against a sorted-Python-list oracle - I solved LC 23 (Merge K Sorted Lists) and recognized this kernel called K-1 times
If any unchecked: redo before moving to p08.
10. Company Context
Amazon (warehouse / fulfillment narratives common)
- The framing: “You have two sorted shipment-arrival queues from different warehouses; merge them into one timeline.”
- Misleading example: Empty input from one queue. Many candidates’ code crashes on
None.val. - Adversarial extension: “Now there are M warehouses.” (LC 23 — K-way merge.) Followed by: “And each can be 10TB on disk.” (External merge sort.)
- What they want to hear: “Let me handle the empty-list cases first.” Amazon weighs defensive coding heavily.
Microsoft (loves linked-list problems — phone screen battery)
- The framing: Often paired with reverse-linked-list as a back-to-back battery.
- Misleading example: They give you
[1,1,1]and[1,1,1]— testing whether you correctly use<=and don’t introduce bugs around ties. - Adversarial extension: “Sort a linked list using merge sort.” (LC 148.) Now you split the list with slow/fast pointers and call this function on the halves.
- What they want to hear: Verbalize the dummy-node decision out loud. “I’ll use a sentinel to avoid special-casing the first node.”
Apple (clean code expected)
- The framing: Often the straight problem.
- Misleading example: None — Apple tends to be straightforward but watches for code aesthetics.
- Adversarial extension: “Merge them in descending order instead.” (Flip the comparator, but watch for the leftover-tail still being attached without re-sorting — it’s already descending if input was descending.)
- What they want to hear: Idiomatic, dummy-node, leftover-attached-in-O(1) code. Apple interviewers will literally say “looks like production code” when they see it.
Google (algorithm composition)
- The framing: Straight problem, then “What’s the K-way version?”, then “What if K=10^6?”
- What they want to hear: Progression from this kernel → min-heap of K heads → external merge with limited memory. The Google bar is articulating the algorithmic family tree.
Meta / Bloomberg (data-processing scenarios)
- The framing: “Merge two sorted streams of events / orders / quotes by timestamp.”
- What they want to hear: Recognition that this is the inner loop of every streaming-join algorithm. Bloomberg in particular loves merge-by-timestamp scenarios.
11. Interviewer’s Lens
| Phase | Strong signal | Weak signal | Scorecard phrase (strong) |
|---|---|---|---|
| Reading problem | Asks “splice or copy?” + “either empty?” | Assumes both | “Clarifies mutation and edge cases — production-mindful” |
| Pre-coding | Declares dummy node before loop, names tail | Builds result with if result is None branches | “Reaches for idiomatic sentinel — senior signal” |
| Coding | Uses <= (stable); attaches leftover in O(1) | Uses < (silently destabilizes); walks leftover | “Demonstrates awareness of stability and the leftover optimization” |
| Edge cases | Tests empty + single + all-equal before submit | Tests only the canonical example | “Self-catches edge bugs — strong production instinct” |
| Follow-up | Names this as merge-sort’s inner loop | Doesn’t connect to broader pattern | “Sees the kernel inside larger algorithms — Staff signal” |
The scorecard line that gets you the offer: “Used dummy node naturally, attached leftover tail in O(1) — code reads like internal library code.”
The scorecard line that loses you: “Multiple branches in main loop; allocated new nodes unnecessarily; walked leftover.”
12. Level Delta
| Level | What their answer looks like |
|---|---|
| Mid | Iterative solution that works. Uses dummy node after prompting or recall. Walks the leftover instead of attaching. Handles empty after explicit testing. ~10 min. |
| Senior | Iterative + dummy node + <= for stability + O(1) leftover attachment. Handles all empty cases without prompting. ~6 min. Mentions O(N1+N2) explicitly. |
| Staff | All of Senior + recursive version with stack-depth caveat + names this as merge-sort’s inner loop + connects to LC 23 (K-way) and LC 148 (sort list) + mentions stability for object-valued lists. ~5 min. |
| Principal | All of Staff + asks about object value semantics (stability matters for records, not for ints) + offers external-merge variant for disk-resident lists + mentions that timestamp-keyed merges in distributed log systems (Kafka log compaction, Kinesis shard merging) use exactly this kernel + ~4 min with discussion of when to use loser-tree instead of pairwise merge for K-way. |
13. Follow-up Questions & Full Answers
Follow-up 1: “Now merge K sorted lists.” (LC 23)
Signal sought: Composition of this kernel.
Full answer: “Two approaches. (a) Min-heap of K heads: push the head of each list into a min-heap keyed by val. Pop the smallest, append to result, push its next if any. O(N log K) time, O(K) extra space. (b) Pairwise merge: merge in K/2 pairs, then K/4, … O(N log K) same complexity but better cache behavior because each merge is sequential. For very large K, (a); for moderate K with cache-sensitive workloads, (b). The kernel I just wrote is what’s called inside both.”
Follow-up 2: “What if the lists are descending?”
Signal sought: Adaptation, comparator awareness.
Full answer: “Flip the comparison: l1.val >= l2.val picks the larger first. The leftover-tail attach still works — it’s still a sorted suffix, just descending. Single character change. If I expected either order at runtime, I’d parameterize with a comparator function: merge(l1, l2, cmp=lambda a,b: a <= b).”
Follow-up 3: “Stability — explain when <= vs < matters.”
Signal sought: Do you understand the property name?
Full answer: “Stability means equal elements preserve their original relative order. For pure numbers it’s invisible. For objects with payloads (e.g., trading orders with same price but different timestamps), it’s a correctness property — a stable merge keeps earlier-timestamp orders first. Using <= (pick from l1 on ties) achieves stability when l1 is the ‘first’ list by convention. Using < (pick from l2 on ties) reverses tie order. Always state which list is ‘first’ when you claim stability.”
Follow-up 4 (Hard): “Sort a linked list.” (LC 148)
Signal sought: Algorithm composition.
Full answer: “Merge sort. (1) Find the middle with slow/fast pointers; split into two halves (set middle.next = None to terminate the first half). (2) Recursively sort each half. (3) Merge with this kernel. O(N log N) time. Space: O(log N) for recursion stack (vs. O(1) for the merge step itself). Why not quicksort? Linked list random access is O(N), so quicksort’s partition step is awful. Merge sort is the natural fit because its access pattern is sequential.”
Follow-up 5 (Senior signal): “How would you handle this for billion-node lists across multiple machines?”
Signal sought: Distributed thinking.
Full answer: “External merge. Each machine holds a sorted shard; we want one globally sorted output. Use a tournament tree (loser tree) with K = number of shards, each shard streams its head value. The tree picks the global minimum in O(log K) per element. Each machine reads sequentially from disk; the coordinator emits the merged stream. This is exactly how Spark’s sort-merge join works, and how HBase compactions merge sorted SSTables. The two-list kernel I just wrote generalizes via tournament-tree to N-way.”
14. Full Solution Walkthrough
See solution.py.
The file has five sections:
ListNode— minimal class with__repr__.merge_two_lists(l1, l2)— iterative, dummy-node, splice-in-place. The production version.merge_two_lists_recursive(l1, l2)— for didactic comparison; bumps recursion limit.stress_test()— generates 1000 random sorted-list pairs, runs both implementations, compares to a sorted Python list oracle.edge_cases()— empty+empty, empty+nonempty, single+single, all-equal-values, large.
Decisions justified:
- Why
<=not<: stability (per Section 13.3). - Why
tail.next = l1 or l2after the loop: Python truthy idiom, both Pythonic and O(1). - Why we splice nodes rather than copy: O(1) extra space, no allocation pressure.
15. Beyond the Problem — Production Reality
At scale:
- Database query execution: SORT MERGE JOIN reads two sorted child operators and merges by join key. This is the kernel — Postgres, Oracle, MySQL, BigQuery all have this code path.
- Log aggregation: Kafka log compaction, Loki/Tempo segment merging, ClickHouse part-merging — all are timestamp-keyed merges of sorted segments.
- LSM trees (RocksDB, LevelDB, Cassandra): compaction merges multiple sorted SSTables into one. Same kernel, lifted to disk-resident files.
Real systems this is the kernel of:
- Polyphase merge sort in external sort utilities (
sortcommand on Unix when input doesn’t fit in RAM). - Spark’s sort-merge join: each executor holds a sorted partition; the merge is precisely this kernel.
- Time-series database compaction: InfluxDB, Prometheus, TimescaleDB all merge time-ordered chunks.
Principal-engineer code review comment: “I’d add a debug assertion that asserts both inputs are sorted on the way in. We’ve had subtle bugs where a caller passed an unsorted list (a different list type assumed by mistake) and this function silently produced garbage. A assert is_sorted(l1) and is_sorted(l2) under if __debug__ is cheap insurance. Also: please version the comparator — for our timestamp lists, we sometimes pass nanosecond ints and sometimes datetime objects; the lambda makes the contract explicit at the call site.”
p07 Merge Two Sorted Lists — Progressive Hints
Rule: Read ONE hint. Set a 5-minute timer. Try again. Only read the next hint if the timer expires.
Hint 1 — Pattern family
The first node of your output isn’t special — except your code probably treats it that way. There’s a single allocation trick that makes “first append” identical to every other append. Reach for it.
Hint 2 — Data structure / state
You walk both lists simultaneously, always advancing the one whose current value is smaller. You need a way to append to your output without re-finding its end every time. So: keep a pointer to the output’s tail.
Hint 3 — The key insight
Use a dummy (sentinel) node before the real output head. Then tail = dummy initially. Every append is tail.next = chosen; tail = tail.next. No special case for the first node. Return dummy.next (skip the dummy).
When one list is exhausted, the other’s remainder is already sorted. Just point at it. Don’t iterate.
Hint 4 — Pseudocode skeleton
dummy = new ListNode()
tail = dummy
while l1 and l2:
if l1.val <= l2.val: # <= for stability
tail.next = l1
l1 = l1.next
else:
tail.next = l2
l2 = l2.next
tail = tail.next
tail.next = l1 if l1 else l2 # leftover tail — O(1) attach
return dummy.next
Two things to check yourself: (a) you wrote <=, not <; (b) the leftover attach is ONE line, not a second while loop.
Hint 5 — Full algorithm
The iterative version is as in Hint 4. Loop invariant: at the top of each iteration, dummy.next ... tail is a sorted list of all consumed nodes, and the remaining l1 and l2 are sorted suffixes whose first elements are ≥ tail.val. When one becomes None, the other’s entire remainder is sorted and ≥ tail, so attaching it preserves the sorted property.
Why dummy node: without it, you’d write if result is None: result = chosen; tail = chosen for the first append, creating a branch in your hot loop. With it, every iteration is identical. This is the “sentinel removes the special case” pattern — it reappears in every list mutation problem.
Recursive variant (Google likes this):
def merge(l1, l2):
if l1 is None: return l2
if l2 is None: return l1
if l1.val <= l2.val:
l1.next = merge(l1.next, l2); return l1
l2.next = merge(l1, l2.next); return l2
Elegant but O(N1+N2) stack — risky in Python past N=1000.
Time O(N1+N2). Space O(1) iterative, O(N1+N2) recursive.
p08 — Linked List Cycle
Source: LeetCode 141 · Easy · Topics: Linked List, Two Pointers, Hash Table Companies (2024–2025 frequency): Amazon (high), Microsoft (high), Google (very high), Meta (high), Bloomberg (medium), Apple (medium) Loop position: phone screen; the lead-in to LC 142 (find cycle start), LC 287 (Find Duplicate Number), LC 202 (Happy Number)
1. Quick Context
The interviewer’s real question is: “Do you know Floyd’s tortoise-and-hare, and can you prove it terminates?” The HashSet solution gets you Mid-level credit. The two-pointer solution with a verbal correctness sketch gets you Senior. Articulating the modular-arithmetic proof gets you Staff. This is one of the rare problems where the justification matters as much as the code.
What it looks like it tests: detect a cycle in a linked list. What it actually tests: can you reason about an algorithm whose correctness is non-obvious? Can you explain why a slow and fast pointer must meet inside any cycle, instead of just asserting it?
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/linked-list-cycle/
STOP. Set a 12-minute timer. Code BOTH the HashSet version and Floyd’s. Do not read past this section until solved or expired.
If repeat: target 4 min for Floyd’s. If you reach for HashSet first reflexively, that’s a Mid signal — practice Floyd’s until it’s your reflex.
3. Prerequisite Concepts
- Linked list traversal, the
Noneterminator (phase-01 §4) - Two-pointer / fast-slow pattern (phase-02 §1 Two Pointers)
- HashSet O(1) average membership; identity (
id()) vs value as the set key — for nodes, identity is correct because two distinct nodes can have equal values - Modular arithmetic intuition: if two runners on a circular track move at different speeds, they will eventually be at the same point (gap closes by 1 each step in tortoise-and-hare)
4. How to Approach (FRAMEWORK Steps 1–9 applied)
Step 1 — Restate: “Given the head of a singly linked list, return True if any node’s next reaches a previously-visited node (a cycle), else False.”
Step 2 — Clarify:
- “Empty or single-node input?” (Both possible; cycle requires at least 2 nodes including the self-loop edge case.)
- “Can the cycle start at the head itself (entire list is a cycle)?” (Yes.)
- “Is the cycle always reachable from head?” (Yes by definition — we only follow
nextfrom head.) - “Allowed to modify the list?” (Discuss; the in-place “mark with sentinel” approach exists but mutates input. Floyd’s doesn’t need it.)
Step 3 — Constraints: N up to 10^4 in LC. Both HashSet O(N) space and Floyd’s O(1) space fit; production prefers O(1).
Step 4 — Examples:
[3, 2, 0, -4]with tail → node at index 1 (3→2→0→-4→2→…) → True[1, 2]with tail → head (1→2→1→…) → True[1]no cycle → False[1]self-loop (1→1) → True (single-node cycle — does Floyd’s handle it? Yes if you advance correctly.)[]→ False
Step 5 — Brute Force: HashSet of visited node IDs. Walk; if current node already in set → cycle; else add and advance. O(N) time, O(N) space. Use id(node), not node.val, to avoid collisions on equal values.
Step 6 — Brute Force Complexity: Time O(N), space O(N). Correct, but uses heap memory for what can be done in O(1).
Step 7 — Pattern Recognition: “Cycle detection in a sequence with O(1) memory” → Floyd’s tortoise and hare (slow/fast pointer). Generalizes to: cycle in iterated function f (LC 202, LC 287); duplicate detection in number sequences; Pollard’s rho factorization.
Step 8 — Optimize: Floyd’s. slow = head, fast = head. Loop: advance slow by 1, fast by 2. If fast or fast.next becomes None, no cycle → False. If slow == fast, cycle → True.
Step 9 — Prove correctness: If no cycle, fast reaches None in at most N/2 steps. If cycle exists, both pointers enter the cycle eventually. Once both are in the cycle of length C, the gap between them (measured as (fast - slow) mod C) increases by exactly 1 each iteration (fast gains 2, slow gains 1, net +1). Since the gap is bounded in [0, C-1], it must take the value 0 within C iterations — that’s when they meet.
5. Progressive Hints
If stuck for 5 minutes, open hints.md. One hint, 5-min timer, retry.
6. Deeper Insight — Why It Works
The pigeonhole / modular argument: consider only the cycle phase (after both pointers are inside the cycle of length C). Each step, slow advances by 1 mod C, fast by 2 mod C. Their relative position is (fast_pos - slow_pos) mod C, which changes by +1 mod C each step. Over C iterations, it takes every value in {0, 1, …, C-1} — so it hits 0 (collision) within C steps. There is no input that escapes detection.
Why 1 and 2 specifically: any pair of speeds (a, b) with gcd(b - a, C) = 1 would also work, but (1, 2) is the simplest: gap increment is always 1, gcd is always 1 (any C). Using (1, 3) would have a gap increment of 2; on even-length cycles, the gap might be permanently odd, never zero. The choice of speeds is not arbitrary.
Why fast and fast.next: fast moves 2, so we check BOTH that fast exists and that fast.next exists before doing fast = fast.next.next. Forgetting fast.next is the classic NoneType crash.
Why HashSet uses id(): the question is structural (same node visited twice), not value-based. Two distinct nodes can have equal val. Using node as the key works in Python (default __hash__ is identity-based), but id(node) is explicit and unambiguous.
Why this generalizes to numerical cycles: treat x → f(x) as a “next pointer” on integers. Then LC 202 (Happy Number — cycle in repeated digit-square-sum) and LC 287 (Find Duplicate Number — cycle in nums[nums[i]]) become structurally identical. The kernel is “Floyd’s on any iterated function.”
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — HashSet on node.val:
seen = set()
while head:
if head.val in seen: return True
seen.add(head.val)
head = head.next
Wrong. Two distinct nodes with the same value would falsely trigger “cycle detected”. For [1, 1, 1, 1] with no cycle (just repeating values), this returns True. Use id(head) or head directly as the set key.
Wrong-but-tempting #2 — Counter / step limit:
count = 0
while head:
if count > MAX: return True
head = head.next; count += 1
Heuristic: “if I walk more than 10^4 nodes, must be a cycle.” Fails on inputs that are legitimately longer than your bound and falsely flags. Doesn’t prove anything — interviewers reject as “didn’t think about correctness.”
Wrong-but-tempting #3 — Mutate the list with sentinel values:
while head:
if head.val == SENTINEL: return True
head.val = SENTINEL
head = head.next
Destroys input. Some interviewers tolerate this if you ask permission first; many don’t. And in production, the input might be shared (immutable contract), so this corrupts state. Never the default choice.
Wrong-but-tempting #4 — fast = fast.next.next without null guard:
while slow != fast:
slow = slow.next
fast = fast.next.next # crashes when fast or fast.next is None
Classic NoneType crash. Always guard while fast is not None and fast.next is not None BEFORE advancing fast by 2.
Wrong-but-tempting #5 — Start slow = head, fast = head.next:
Off-by-one. The standard formulation is slow = fast = head, advance both at the top of each iteration, then compare. Starting fast ahead introduces an asymmetric first iteration and causes the meeting-point math (used in LC 142) to be wrong.
8. Skills & Takeaways
Generalizable patterns:
- Floyd’s cycle detection: on linked lists, on iterated integer functions, on any sequence where you have
f(x) = next. Examples: LC 142, 202, 287, 457. - Modular arithmetic for terminat ion proofs: when you can’t enumerate states, model the state space modulo cycle length.
id()for identity-based hashing: when the question is “same object?” not “same value?”
Analogous problems:
- LC 142 — Linked List Cycle II (find the start of the cycle — the math gets beautiful)
- LC 202 — Happy Number (Floyd’s on digit-square-sum iteration)
- LC 287 — Find the Duplicate Number (Floyd’s on
nums[nums[i]]— Hard, the constraint of O(1) extra space is what forces Floyd’s) - LC 457 — Circular Array Loop (Floyd’s with directional constraints)
- LC 234 — Palindrome Linked List (uses slow/fast to find the middle — same kernel, different goal)
- LC 876 — Middle of Linked List (the simplest fast-slow use)
When NOT to use this: if you can read the list twice (then a counter + walk-N/2 finds the middle). If the cycle’s start matters and you don’t want the math (then HashSet to find the first repeat). If input is mutable AND interviewer permits (sentinel marking is fastest in wall-clock).
9. When to Move On (binary; must all be YES)
- I wrote Floyd’s in <5 min on the second attempt with correct null guards
- I can state aloud why the slow and fast pointers MUST meet inside any cycle (gap+1 mod C argument)
- I can explain why speeds (1, 2) and not (1, 3) — even-length cycle hazard
-
I can implement the HashSet version and articulate why
id()matters overval -
My
stress_test()covers both no-cycle and cycle inputs and passes 1000 iterations - I solved LC 142 (Cycle II) and can derive the meeting-point math
- I solved LC 202 (Happy Number) and saw Floyd’s apply to integer iteration
If any unchecked: redo before moving to p09.
10. Company Context
Google (loves the proof — Staff-bar problem)
- The framing: Standard problem, then “prove it terminates.”
- Misleading example: A no-cycle list that’s long enough to be uncomfortable but not infinite. Some candidates assume “long = cycle” and false-positive.
- Adversarial extension: “Now find the START of the cycle.” (LC 142.) Then “What if the cycle exists but the head is NOT inside it?” (e.g., 1→2→3→4→3, cycle starts at 3 — head is not in cycle.)
- What they want to hear: Modular arithmetic proof, not “I trust it.” Verbatim phrases like “the gap grows by 1 modulo cycle length each step, so it must reach 0 within C iterations” earn Staff signal.
Amazon (defensive coding)
- The framing: Often in a story: “Detect if a workflow has a self-referential dependency.”
- Misleading example: Single-node self-loop. Many naive Floyd’s implementations skip this because of how they initialize.
- Adversarial extension: “What if the input could already be partially corrupted (cyclic) and we need to safely traverse just the acyclic prefix?” → returns the prefix length AND cycle detection.
- What they want to hear: “Let me handle empty and single-node first.” Then proceed.
Microsoft (phone screen mainstay)
- The framing: Often back-to-back with reverse and merge.
- What they want to hear: Both HashSet and Floyd’s solutions, with explicit tradeoff statement: “HashSet is O(N) space but simpler; Floyd’s is O(1) space, which is the production preference.”
Meta (fast follow-ups)
- The framing: Linked List Cycle → Cycle II → Happy Number, in rapid sequence. They use this to test whether you generalize.
- What they want to hear: Naming the pattern: “this is Floyd’s cycle detection; it applies to any iterated function, not just linked lists. Happy Number is the same algorithm on integer→integer iteration.”
Bloomberg / Apple
- The framing: Standard. Bloomberg sometimes adds: “If detected, return the cycle length too.”
- What they want to hear: Awareness that once Floyd’s detects, you can count cycle length by walking one pointer until they re-meet.
11. Interviewer’s Lens
| Phase | Strong signal | Weak signal | Scorecard phrase (strong) |
|---|---|---|---|
| Reading problem | Asks “self-loop allowed?” + “entire list a cycle?” | Assumes “normal” cycle | “Considers degenerate cases — production-mindful” |
| Pre-coding | Offers both HashSet and Floyd’s, picks Floyd’s for space, justifies | Goes straight to HashSet | “Knows the O(1) algorithm and chooses it for the right reason” |
| Coding (Floyd’s) | Guards fast and fast.next before advancing | Crashes on edge inputs | “Defensive pointer arithmetic — strong instinct” |
| Correctness justification | Articulates modular-arithmetic / pigeonhole argument | “It just works” or hand-waves | “Owns the algorithm’s correctness — Staff signal” |
| Follow-up | Generalizes to integer iteration (LC 202, 287) | Sees only the linked list version | “Recognizes the kernel across problem families” |
The scorecard line that gets you the offer: “Implemented Floyd’s correctly with correct null guards; articulated termination proof using modular arithmetic; generalized to integer iteration.”
The scorecard line that loses you: “Used HashSet on values not identity; crashed on self-loop; could not explain why the algorithm works.”
12. Level Delta
| Level | What their answer looks like |
|---|---|
| Mid | HashSet version with id(node). Works. Can’t explain Floyd’s beyond “fast catches slow.” ~8 min. |
| Senior | Floyd’s with correct null guards. Handles empty, single-node, self-loop. Articulates “fast gains on slow by 1 per step, so within cycle length they must meet.” ~6 min. |
| Staff | All of Senior + names the pattern + connects to LC 142, 202, 287 + provides explicit modular-arithmetic correctness sketch + mentions why (1,2) and not (1,3). ~5 min. |
| Principal | All of Staff + discusses real applications (Pollard’s rho factorization, garbage collector cycle detection, distributed deadlock detection) + offers Brent’s algorithm as a Floyd’s variant with better constants + mentions that for very long acyclic prefixes Floyd’s wastes work and Brent’s is better in practice. ~4 min with full discussion. |
13. Follow-up Questions & Full Answers
Follow-up 1: “Find the start of the cycle.” (LC 142)
Signal sought: Can you do the math?
Full answer: “After Floyd’s detects the meet, reset one pointer to head. Advance both by 1 step. They meet at the cycle start. Proof: let L = length of non-cycle prefix, C = cycle length, k = distance from cycle start to the meet point. When slow has moved L+k steps, fast has moved 2(L+k) and is at the same node, so 2(L+k) − (L+k) = L+k is a multiple of C: L+k ≡ 0 (mod C), so L ≡ −k ≡ C−k (mod C). That means walking L more steps from the meet point lands exactly at the cycle start. Reset one pointer to head, advance both by 1, they meet there. O(N) time, O(1) space.”
Follow-up 2: “What’s the largest input where HashSet would be cheaper than Floyd’s in wall-clock?”
Signal sought: Engineering pragmatism, not just asymptotics.
Full answer: “HashSet does ~1 hash + 1 compare per node, but allocates a hash table that grows. Floyd’s does 3 pointer-derefs per slow-step (slow advance + fast.next + fast.next.next). For tiny N (under ~100) and cycle absent, HashSet is sometimes marginally faster due to fewer pointer hops, but allocates memory. For any meaningful N or when allocations matter (real-time systems, embedded), Floyd’s wins. The space difference (O(N) vs O(1)) almost always dominates the decision.”
Follow-up 3: “What if speeds were (1, 3) instead of (1, 2)?”
Signal sought: Did you actually understand the proof?
Full answer: “The gap closes by 2 each step (modulo C). If C is odd, gcd(2, C) = 1 and the gap eventually hits 0. If C is even, the gap stays at the same parity forever — if it starts odd, it never reaches 0. So (1, 3) fails on even cycle lengths. The choice of (1, 2) is what guarantees gcd(speed difference, C) = 1 for every C ≥ 2.”
Follow-up 4: “Apply Floyd’s to find a duplicate in an array of N+1 integers, each in [1, N].” (LC 287)
Signal sought: Generalization to non-list inputs.
Full answer: “Treat the array as a function: f(i) = nums[i]. Start at i=0 (or N, any value never produced as output works as a sentinel; here 0 works because nums values are in [1,N]). Define next(x) = nums[x]. Since there’s a duplicate value d, two indices map to d, so the iteration x → nums[x] must form a cycle at d. Apply Floyd’s: slow advances by slow = nums[slow], fast by fast = nums[nums[fast]]. Detect meet, then find cycle start using the LC 142 trick. The cycle start IS the duplicate. O(N) time, O(1) extra space — beats sort and HashSet.”
Follow-up 5 (Senior signal): “How would you test cycle detection without modifying production data?”
Signal sought: Testing discipline.
Full answer: “Three layers. (1) Build a builder utility that constructs a list AND injects an optional cycle at index k — separate the test scaffolding from the algorithm. (2) Property tests: (a) any acyclic list returns False, (b) any cyclic list returns True, (c) cycle length 1 (self-loop) returns True, (d) entire list is the cycle returns True. (3) Memory profile: assert Floyd’s allocates zero new nodes (vs HashSet’s O(N)). My solution.py uses a build_with_cycle(values, cycle_at) helper precisely for layered testing.”
14. Full Solution Walkthrough
See solution.py.
The file has six sections:
ListNode— minimal class.has_cycle_hashset(head)— the brute force usingid()as key.has_cycle(head)— Floyd’s, with INVARIANT comment. Guardsfast and fast.nextbefore advancing.build_with_cycle(values, cycle_at)— helper to construct test inputs with controlled cycle start (or -1 for no cycle).stress_test()— generates 1000 random cases, half with cycles, asserts both implementations agree.edge_cases()— empty, single (no cycle), single self-loop, two-node cycle, entire-list cycle, cycle at tail.
Decisions justified:
- Why
id(node)notnode: explicit identity contract; works in any language port. - Why
slow = fast = headnotfast = head.next: symmetric initialization is what makes the LC 142 math work.
15. Beyond the Problem — Production Reality
Real-world cycle detection:
- Garbage collectors (Python’s reference-cycle collector, JVM’s old-generation tracer): cycle detection is critical for reclaiming memory. The algorithms are richer than Floyd’s (they use tri-color marking) but Floyd’s is the conceptual ancestor.
- Distributed deadlock detection: in databases (e.g., Postgres, Spanner), each transaction is a node, “waits-for” is the edge. Cycle = deadlock. Detection algorithms run periodically over the wait-for graph.
- Build systems: detecting circular dependencies in Bazel, Cargo, npm — same family of algorithm, lifted to DAGs.
- Pollard’s rho factorization: Floyd’s cycle detection on the iteration
x → (x² + c) mod nfactors large integers. Same algorithm, profoundly different application.
Real systems this is the kernel of:
- Linux kernel
krefdebugging: detecting object reference cycles. - TensorFlow / PyTorch dataflow graphs: cycle detection before compilation.
- Service mesh dependency analysis: Istio uses cycle detection over service-call graphs to flag bad architecture.
Principal-engineer code review comment: “Floyd’s is correct but in our production we use Brent’s algorithm — it has the same O(N) bound but fewer pointer dereferences on long acyclic inputs. We saw a 30% speedup in our packet-trace analyzer. Worth knowing both. Also: please add metric emission so if cycle is detected in our workflow scheduler, we page on it — silent cycle handling has caused two outages.”
p08 Linked List Cycle — Progressive Hints
Rule: Read ONE hint. Set a 5-minute timer. Try again. Only read the next hint if the timer expires.
Hint 1 — Pattern family
You can solve this with extra memory (remember every node you’ve seen — if you revisit one, cycle). Can you do it WITHOUT extra memory? Think about two things moving through the list at different speeds.
Hint 2 — Data structure / state
Two pointers. Both start at head. One advances by 1 each iteration; the other by 2. If there’s a cycle, the faster one will eventually lap the slower one and they’ll be at the same node. If no cycle, the faster one walks off the end.
Hint 3 — The key insight
Inside any cycle of length C, the gap between fast and slow (fast minus slow, mod C) increases by exactly 1 each step. Since the gap is bounded in [0, C-1], it must take the value 0 within C steps — that’s when they meet. The algorithm is guaranteed to terminate, with proof.
For the no-cycle case: fast (moving 2 per step) reaches the end (None) in at most N/2 steps.
Hint 4 — Pseudocode skeleton
slow = head
fast = head
while fast is not None and fast.next is not None:
slow = slow.next
fast = fast.next.next
if slow is fast:
return True
return False
Two correctness checks: (a) you guard BOTH fast AND fast.next before doing fast.next.next, (b) you advance then compare (not compare before the first advance, which would falsely return True for any list of length ≥ 1).
Hint 5 — Full algorithm + why this works
Floyd’s tortoise-and-hare: start slow = fast = head. In each loop iteration: advance slow by 1, fast by 2. If fast or fast.next is None, no cycle exists. If slow and fast point to the same node, a cycle exists.
Why speeds (1, 2) specifically: the gap between them increases by 2 - 1 = 1 per step (mod cycle length C). Since gcd(1, C) = 1 for any C, the gap eventually equals 0 — they meet. If you used (1, 3), the gap would increase by 2 per step; for even C, the gap might be permanently odd and never hit 0. The choice isn’t arbitrary; it guarantees correctness for all cycle lengths.
Why fast and fast.next BEFORE advancing fast by 2: because fast.next.next would crash on NoneType otherwise. The guard goes BEFORE the advancement.
HashSet variant (correctness oracle): walk the list, store id(node) in a set; if you encounter a duplicate id, return True. Use id() (identity), not node.val, because two distinct nodes can hold the same value — [1, 1, 1, 1] with no cycle would falsely trigger on val.
Time O(N) both; space: HashSet O(N), Floyd’s O(1). Floyd’s wins in interviews and production.
p09 — Maximum Subarray (Kadane’s Algorithm)
Source: LeetCode 53 · Medium · Topics: Array, Divide & Conquer, Dynamic Programming Companies (2024–2025 frequency): Amazon (very high), Microsoft (high), Bloomberg (very high), Apple (high), Meta (medium), Google (medium) Loop position: mid-loop coding round; sometimes given as “Easy” but the optimal is genuinely subtle
1. Quick Context
This is your first encounter with a 1-D dynamic programming algorithm that collapses to two scalars. The DP table is conceptually dp[i] = max subarray sum ending exactly at i, but the answer only needs dp[i-1], so you never allocate the array. Senior candidates derive this collapse on the spot. Mid candidates write the O(N) DP array and forget to optimize. Junior candidates write the O(N²) brute force and get stuck.
The interviewer is testing whether you can: (a) identify that “contiguous subarray” rules out two-pointer (which needs monotone structure) and dynamic-programming-with-prefix-sums (which gets the global sum, not the contiguous-piece argmax), (b) derive the recurrence on the spot, (c) collapse it to O(1) space.
What it looks like it tests: find the max-sum contiguous subarray. What it actually tests: can you discover Kadane’s recurrence on your own? Can you handle the all-negative-array trap?
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/maximum-subarray/
STOP. Set a 15-minute timer (this one is genuinely harder than the surrounding Easy problems). Code it cold. Do not read past this section until solved or expired.
If you find Kadane’s immediately because you’ve seen it: target 5 min, AND derive the recurrence aloud as you write. Memorization without derivation is a Mid signal — practice the derivation.
3. Prerequisite Concepts
- 1D Dynamic Programming intuition: “state at index i depends only on state at index i-1” (phase-05 §1)
- Why prefix sums DON’T directly solve this (they solve “sum of any subarray” but the argmax over all O(N²) subarrays still needs O(N²) without a cleverer transform — see Section 6)
- The “all-negative” trap:
max(0, ...)is wrong when every element is negative; the answer is the maximum single element - Loop invariants in DP: what state does each scalar represent at the top of each iteration?
4. How to Approach (FRAMEWORK Steps 1–9 applied)
Step 1 — Restate: “Given an integer array, return the maximum sum of any contiguous non-empty subarray.”
Step 2 — Clarify:
- “Can the array contain negatives?” (Yes — this is the entire point. Pure positives is trivial.)
- “Can it be all negatives?” (Yes — the answer is the maximum single element, NOT 0.)
- “Must subarray be non-empty?” (Yes per LC. If empty allowed, answer would be max(0, kadane(nums)).)
- “Need indices or just the sum?” (Per LC, just sum. Indices are a natural follow-up — track start/end.)
- “Subarray (contiguous) or subsequence (non-contiguous)?” (Subarray. If subsequence, answer = sum of positives, trivial.)
Step 3 — Constraints: N up to 10^5 per LC. O(N²) brute (10^10) won’t fit. O(N) is required.
Step 4 — Examples:
[-2,1,-3,4,-1,2,1,-5,4]→ 6 (subarray[4,-1,2,1])[1]→ 1[-1]→ -1 (single negative — critical trap)[-3, -2, -1, -5]→ -1 (all negatives — must NOT return 0)[5, 4, -1, 7, 8]→ 23 (mostly positive — sum almost everything)[1, -100, 2, -100, 3]→ 3 (a single big negative resets the running sum)
Step 5 — Brute Force: Two nested loops; for each (i, j) compute sum. O(N²) with the partial-sum optimization (compute running sum as j advances), O(N³) naively. State this aloud.
Step 6 — Brute Force Complexity: O(N²) time with the running-sum trick, O(1) space. Acceptable correctness oracle for small N.
Step 7 — Pattern Recognition: “Max over all contiguous subarrays” + “decision at index i depends only on whether to extend the previous best or start fresh” → Kadane’s algorithm (1D DP, collapses to O(1) space). This is the canonical “DP that doesn’t look like DP” problem.
Step 8 — Optimize: The recurrence: best_ending_here[i] = max(nums[i], best_ending_here[i-1] + nums[i]). Interpretation: at each position, either extend the previous best (if doing so makes the sum positive) or start fresh at nums[i]. The global answer is max(best_ending_here) across all i. Collapse to two scalars: curr = nums[0], best = nums[0]. For i in 1..N-1: curr = max(nums[i], curr + nums[i]); best = max(best, curr). Return best.
Step 9 — Prove correctness: Loop invariant: at the top of iteration i, curr = max sum of any subarray ending exactly at index i-1, and best = max sum of any subarray ending at indices 0..i-1. The recurrence preserves this: a subarray ending at i either includes the optimal one ending at i-1 (then sum = curr + nums[i]) or starts fresh at i (sum = nums[i]). Taking max preserves the optimal-ending-at-i invariant. Taking running max preserves best over all positions.
5. Progressive Hints
If stuck for 5 minutes, open hints.md. One hint, 5-min timer, retry.
6. Deeper Insight — Why It Works
The DP formulation, made explicit: define dp[i] = max sum of contiguous subarray ending at i (inclusive). The recurrence dp[i] = max(nums[i], dp[i-1] + nums[i]) says: the best subarray ending at i either starts at i (skip the past) or extends the best one ending at i-1. The global answer is max(dp[0..N-1]). This is a one-dimensional DP. The trick is recognizing that “ending at i” is the right state — most candidates initially think of dp[i] = “max in nums[0..i]” which leads to a recurrence that requires both “include i” and “don’t include i” cases and doesn’t collapse cleanly.
Why O(1) space: the recurrence only depends on dp[i-1], so a single scalar curr suffices. The global max is also a scalar best. No array allocation needed.
Why “all negatives” is the trap: the recurrence curr = max(nums[i], curr + nums[i]) correctly handles it: when all values are negative, nums[i] is always larger than curr + nums[i] (since curr is negative), so curr = nums[i] each step, and best tracks the maximum single element. But a popular incorrect formulation, curr = max(0, curr + nums[i]), treats “empty subarray” as legal and returns 0 — wrong because the problem requires non-empty.
Why prefix sums alone don’t solve this in O(N): with prefix sums P, the sum of subarray (i, j) is P[j] − P[i-1]. Maximizing P[j] − P[i-1] over i ≤ j IS solvable in O(N): walk left to right, track min P seen so far, take max of P[j] − min_so_far. But this IS Kadane in disguise — it’s the same algorithm rephrased. Both are O(N), one pass, O(1) extra space.
Connection to LC 121 (Buy/Sell Stock): identical algorithm. “Max profit = max over j of (price[j] − min_so_far_price)” is “max subarray sum = max over j of (P[j] − min_so_far_P)” where P is the prefix sum of price differences. The two problems are literally the same algorithm on different inputs.
Divide & conquer alternative: split in half, recursively compute (a) max subarray entirely in left half, (b) max subarray entirely in right half, (c) max subarray crossing the midpoint (linear scan). Take max of the three. O(N log N) — worse than Kadane. Worth knowing because it’s a clean example of D&C and because LC sometimes asks for both.
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — Reset to 0 when sum goes negative:
curr = 0; best = float('-inf')
for x in nums:
curr += x
if curr < 0: curr = 0
best = max(best, curr)
return best
Wrong for all-negative arrays: returns 0 (never updates best with a positive value). The fix is curr = max(x, curr + x) not curr = max(0, curr + x).
Wrong-but-tempting #2 — Brute force with three loops: O(N³). Even the O(N²) running-sum version is wasteful. Acceptable to state aloud as the baseline, then optimize.
Wrong-but-tempting #3 — Sort the array first: Destroys the “contiguous” constraint entirely. After sorting, “contiguous” no longer means anything meaningful about the original array.
Wrong-but-tempting #4 — DP with explicit array:
dp = [0] * len(nums)
dp[0] = nums[0]
for i in range(1, len(nums)):
dp[i] = max(nums[i], dp[i-1] + nums[i])
return max(dp)
Correct, O(N) time, but O(N) space when O(1) suffices. For interview, this is a “didn’t notice the collapse” mark. For production with billion-element arrays, it’s an unnecessary 8GB of allocation.
Wrong-but-tempting #5 — Maintain “best including” and “best excluding” separately:
Some candidates try include[i] = best ending at i and exclude[i] = best not ending at i. Works but more complex than necessary — Kadane’s only needs one DP series, not two.
8. Skills & Takeaways
Generalizable patterns:
- 1-D DP with state collapse: any DP where
dp[i]depends only ondp[i-1]collapses to O(1) space. Reappears in: climbing stairs (p05), house robber (LC 198), best time to buy/sell stock (p03), maximum product subarray (LC 152). - “Extend or restart” decision: many subarray/substring problems ask “extend the current run or start fresh at this index?” — Kadane is the prototypical example.
- All-negative / all-zero / empty traps: any “max sum” problem with negatives requires checking whether empty subarray is allowed and handling all-negative correctly.
Analogous problems:
- LC 121 — Best Time to Buy/Sell Stock (Kadane on differences — your p03, same algorithm)
- LC 152 — Maximum Product Subarray (Kadane with two states: max-and-min ending here, because a negative times negative becomes max — Hard variant)
- LC 198 — House Robber (Kadane-style 1D DP with “skip current” choice)
- LC 918 — Maximum Sum Circular Subarray (Kadane twice — once for max, once for min, then global - min)
- LC 1186 — Maximum Subarray Sum with One Deletion (Kadane with two-state DP: “best ending here without deletion” and “with deletion”)
- LC 363 — Max Sum Rectangle ≤ K in Matrix (Kadane lifted to 2D)
- LC 53 itself is sometimes asked as “return the actual subarray, not just the sum” — extension: track start/end indices alongside curr/best.
When NOT to use this: non-contiguous subsequences (different problem, trivial: sum of positives or LIS-family). Streaming with multiple deletions (need a more complex deque-based variant).
9. When to Move On (binary; must all be YES)
-
I derived the recurrence
dp[i] = max(nums[i], dp[i-1] + nums[i])on paper before coding - My code uses TWO scalars (not an array) and handles all-negative correctly
-
I can explain why
max(nums[i], ...)is right andmax(0, ...)is wrong -
I can articulate the loop invariant of both
currandbest - I can sketch the divide-and-conquer O(N log N) alternative and explain why Kadane is better
-
My
stress_test()passes 1000 iterations against the O(N²) brute force - I solved LC 152 (Max Product Subarray) and saw the two-state extension of Kadane
- I noticed that p03 (Buy/Sell Stock) is the same algorithm on differences
If any unchecked: redo before moving to p10.
10. Company Context
Amazon (extremely common — every coding round may see this or a variant)
- The framing: Often in story form: “Given daily P&L, find the most profitable contiguous trading window.”
- Misleading example: They give you
[1, 2, 3, 4](all positive — trivial). Then[-1, -2, -3](all negative — exposes whether you fell for themax(0, ...)trap). - Adversarial extension: “Now return the actual subarray (start and end indices), not just the sum.” (Track
startcandidate when you reset; finalize whenbestupdates.) Then: “Now allow ONE element to be deleted.” (LC 1186.) - What they want to hear: Explicit derivation aloud. Verbatim phrase that earns points: “At each index I have two choices — extend or restart — and I pick whichever gives a larger sum ending here.”
Microsoft (loves the DP derivation question)
- The framing: Standard problem.
- Misleading example:
[5, -10, 4, -1, 5]— first element is the temporary best (5), then a big negative pulls curr down, then you have to recognize that starting fresh at 4 was better. - Adversarial extension: “Now return the indices.” Then “Now find the max-sum subarray in a 2D matrix.” (LC 363’s cousin — fix two rows, collapse columns, apply Kadane.)
- What they want to hear: Articulation of the DP state and recurrence. Microsoft phone screens explicitly score “derived the recurrence” as a separate rubric line.
Bloomberg (financial — they LOVE this problem)
- The framing: “Given a price series, find the optimal long-only trading window with no shorting allowed.”
- Misleading example: Includes negative price changes throughout — tests the all-negative handling.
- Adversarial extension: “Now with transaction costs of X — subtract X from each profit.” Then “Now what if shorting is allowed?” (Absolute value Kadane.)
- What they want to hear: Recognition that this IS Buy/Sell Stock in disguise, with explicit framing: “If I difference the price series, Kadane on the diffs gives the max single trade.” Bloomberg likes seeing the algorithmic family tree.
Apple (clarity + clean code)
- The framing: Standard.
- What they want to hear: Idiomatic code, clear variable names (
curr,best, notc,b), explicit handling of empty / single-element inputs.
Meta (less common; usually appears as the harder LC 152 variant)
- The framing: Often LC 152 (Max Product) directly, with this as warmup if you fumble.
- What they want to hear: Awareness that product has two states (positive and negative) because negative × negative = max.
Google (medium frequency; high bar)
- The framing: Sometimes the divide-and-conquer version is asked explicitly, to test that you can write D&C cleanly.
- What they want to hear: Both Kadane (O(N)) and D&C (O(N log N)), with explicit “Kadane wins on this problem but D&C generalizes to 2D and to certain parallel implementations.”
11. Interviewer’s Lens
| Phase | Strong signal | Weak signal | Scorecard phrase (strong) |
|---|---|---|---|
| Reading problem | Asks “negatives allowed?” + “all-negative case?” + “non-empty?” | Assumes positive-only | “Recognizes the trap immediately — strong test-case instinct” |
| Pre-coding | States brute force complexity, then derives Kadane’s recurrence aloud | Says “I’ll use Kadane” without derivation | “Derived recurrence on the spot — not just memorizing” |
| Coding | Two scalars, max(nums[i], curr+nums[i]), initializes from nums[0] | Uses array (no space optimization), max(0, ...) (wrong) | “Optimized space; correct all-negative handling” |
| Edge cases | Tests single-negative, all-negative before submit | Tests only positive examples | “Self-catches the all-negative trap” |
| Follow-up | Connects to LC 121, LC 152, LC 198 | Sees only this problem | “Recognizes Kadane as a family of algorithms” |
The scorecard line that gets you the offer: “Derived Kadane’s recurrence on the spot; optimized to O(1) space; recognized this as Buy/Sell Stock in disguise.”
The scorecard line that loses you: “Used max(0, ...) formulation; returned 0 on all-negative input; required interviewer to debug.”
12. Level Delta
| Level | What their answer looks like |
|---|---|
| Mid | DP array version, correct. Doesn’t collapse to O(1). May handle all-negative correctly after prompting. ~12 min. |
| Senior | O(1) space Kadane derived aloud. Handles all-negative correctly without prompting. States complexity. Recognizes the loop invariant. ~8 min. |
| Staff | All of Senior + names “Kadane’s algorithm” + connects to LC 121 (same algorithm on differences), LC 152 (two-state extension), LC 198 (different “extend or skip” semantics) + offers D&C alternative + ~6 min. |
| Principal | All of Staff + asks about real production framing (financial windowing, anomaly detection, signal processing) + mentions that for streaming with sliding windows, deque-based variants apply + discusses parallel Kadane (associative monoid: combine (best_sum, prefix_sum, suffix_sum, total_sum) — enables map-reduce / SIMD parallelization) + ~5 min with full discussion. |
13. Follow-up Questions & Full Answers
Follow-up 1: “Return the actual subarray, not just the sum.”
Signal sought: Bookkeeping discipline.
Full answer: “Track candidate start. When we ‘restart’ (i.e., nums[i] > curr + nums[i]), set tmp_start = i. When best updates, snapshot start = tmp_start, end = i. Return nums[start:end+1]. Single pass, O(1) extra state. Edge case: all-negative — tmp_start = end = argmax(nums), single-element subarray.”
Follow-up 2: “Now allow one element deletion.” (LC 1186)
Signal sought: Extending the state.
Full answer: “Two-state DP. with_delete[i] = best subarray ending at i using one deletion; without_delete[i] = best ending at i with no deletion. Recurrences: without[i] = max(nums[i], without[i-1] + nums[i]) (standard Kadane). with[i] = max(without[i-1], with[i-1] + nums[i]) — either delete nums[i] (carrying without[i-1] forward) or keep nums[i] and have already deleted earlier. Answer = max over all i of max(with[i], without[i]). Collapses to two scalars each. O(N) time, O(1) space.”
Follow-up 3: “Maximum sum circular subarray.” (LC 918)
Signal sought: Algorithmic re-application.
Full answer: “The optimal subarray is either (a) a normal subarray (Kadane gives this), or (b) wraps around — equivalently, (total sum) − (some subarray we exclude), so we want to MINIMIZE a subarray (Kadane on negated array). Return max(kadane(nums), total − min_subarray(nums)). Special case: if all elements are negative, the wraparound formula would ‘exclude everything’ = 0, but the problem requires non-empty, so handle separately: if max kadane result is negative, return that. Two Kadane passes. O(N) time, O(1) space.”
Follow-up 4: “Divide and conquer formulation.”
Signal sought: Algorithm versatility.
Full answer: “Recursively: max-subarray(arr) = max of {max-subarray(left half), max-subarray(right half), max-crossing-subarray(mid)}. The crossing case requires a linear scan from mid outward in both directions to find the best left-extending and right-extending pieces. Recurrence: T(N) = 2T(N/2) + O(N) → O(N log N). Worse than Kadane on a single machine but the D&C structure parallelizes cleanly and generalizes to 2D (LC 363’s cousin). Worth knowing both.”
Follow-up 5 (Hard, Principal signal): “Parallel Kadane across K machines.”
Signal sought: Distributed thinking, monoid recognition.
Full answer: “Each machine holds a contiguous shard. Each computes four scalars: (total_sum, max_prefix, max_suffix, max_subarray) over its shard. These compose associatively under a ‘merge’ operation: when merging shard A then shard B, the merged max_subarray is max(A.max_subarray, B.max_subarray, A.max_suffix + B.max_prefix). This is a monoid — the four-tuple under merge is associative with identity (0, 0, 0, -∞). That means we can map-reduce the entire computation: each shard contributes its 4-tuple, reduce them pairwise in any tree order. O(N/K) per machine, O(log K) reduction depth. SIMD implementations use the same trick at the cache-line level. This is a beautiful example of ‘rewrite the algorithm as a monoid to make it parallel.’”
14. Full Solution Walkthrough
See solution.py.
The file has six sections:
max_subarray_brute(nums)— O(N²) with running-sum trick; correctness oracle.max_subarray(nums)— Kadane in two scalars; INVARIANT comments.max_subarray_dp_array(nums)— Kadane with the explicit dp[] array; included for didactic comparison.max_subarray_divide_conquer(nums)— O(N log N) version; useful to know.stress_test()— generates 1000 random arrays including all-negative, asserts all four implementations agree.edge_cases()— single positive, single negative, all-negative, all-positive, all-zero, the canonical example, sparse positives among negatives.
Decisions justified:
- Why init
curr = best = nums[0](not 0): empty subarray not allowed. - Why
max(nums[i], curr + nums[i]): handles all-negative correctly. - Why we include the dp_array version: shows the unoptimized DP for didactic clarity, then we point out the collapse.
15. Beyond the Problem — Production Reality
Real applications of Kadane:
- Financial windowing: maximum profit / drawdown / volume window in time series. Trading desks compute “best trading interval” over historical bars; Kadane is the kernel.
- Anomaly detection: maximum contiguous deviation from baseline in monitoring data (Datadog, Prometheus).
- Genomic analysis: maximum-scoring local alignment (BLAST, Smith-Waterman) — the inner score recurrence is Kadane-like.
- Image processing: max-sum rectangle in a 2D image (LC 363) uses Kadane on collapsed column sums; appears in object-detection preprocessing.
- Signal processing: detecting bursts in a noisy signal — Kadane on (signal − threshold) finds the strongest burst window.
Production caveat — floating-point Kadane: if nums are floats with both positives and negatives, naive Kadane can accumulate floating-point error in curr. Use Kahan summation if precision matters. Production financial code does this.
Parallelization: the monoid formulation in Section 13.5 is real — Apache Spark MLlib’s vector-window algorithms use this structure to parallelize across executors. The four-scalar representation is the secret.
Principal-engineer code review comment: “Kadane is correct but our 2D variant (LC 363, applied to image kernels) is O(M²·N). We’ve seen people ‘cleverly’ parallelize by chunking rows, then have the chunks lose the cross-chunk optimal. Use the four-scalar monoid form for any parallel/chunked version — it’s the only correct way. Also: please add a unit test for the all-equal-negative array. We’ve shipped two bugs from max(0, ...) formulations.”
p09 Maximum Subarray — Progressive Hints
Rule: Read ONE hint. Set a 5-minute timer. Try again. Only read the next hint if the timer expires.
Hint 1 — Pattern family
Think about walking the array left to right. At each index, you face a yes/no decision: should the optimal subarray ending at this index extend what came before, or start fresh at this element?
Hint 2 — DP state
Let dp[i] = the maximum sum of any contiguous subarray that ends exactly at index i. The global answer is max(dp[i]) over all i. Can you write dp[i] in terms of dp[i-1] and nums[i]?
Hint 3 — The recurrence
dp[i] = max(nums[i], dp[i-1] + nums[i])
Either extend the best subarray ending at i-1 (if doing so beats starting fresh) OR restart at i. Take whichever is larger.
Critical trap: the recurrence is max(nums[i], ...), NOT max(0, ...). The latter is wrong for all-negative arrays — it would return 0, but the problem requires a non-empty subarray.
Hint 4 — Pseudocode skeleton + space collapse
curr = nums[0] # dp[i-1]
best = nums[0] # global max
for i in 1..N-1:
curr = max(nums[i], curr + nums[i])
best = max(best, curr)
return best
Two scalars, not an array. dp[i] only depends on dp[i-1], so a single variable suffices.
Hint 5 — Full algorithm + correctness
Kadane’s algorithm. Walk the array. Maintain two scalars:
curr= max sum of any subarray ending exactly at the current indexbest= max sum of any subarray seen so far
For each new element x:
curr = max(x, curr + x)— extend the previous best or restart at x.best = max(best, curr)— update running global maximum.
Loop invariant: at the top of iteration i, curr holds the best sum ending at i-1; best holds the global max over all positions 0..i-1. The recurrence preserves both, so after the loop best is the global maximum.
Why all-negative works: for [-3, -2, -1], at i=1: curr = max(-2, -3 + -2) = max(-2, -5) = -2; best = max(-3, -2) = -2. At i=2: curr = max(-1, -2 + -1) = -1; best = -1. Returns -1, the correct answer. The recurrence naturally restarts whenever nums[i] > curr + nums[i], which happens exactly when curr < 0.
Time O(N), space O(1). One pass.
Bonus — divide and conquer (O(N log N)): split in half, recursively find best in left and right halves AND the best subarray crossing the midpoint (linear scan outward from mid). Take max of three. Worse complexity than Kadane on a single machine but generalizes to 2D and parallelizes well.
p10 — Contains Duplicate
Source: LeetCode 217 · Easy · Topics: Array, Hash Table, Sorting Companies (2024–2025 frequency): Amazon (high), Microsoft (high), Apple (medium), Bloomberg (medium), Meta (medium), Google (medium) Loop position: phone screen warmup; the lead-in to LC 219 (Contains Duplicate II — sliding window) and LC 220 (Contains Nearby Almost Duplicate — bucket sort)
1. Quick Context
The interviewer’s bar is not “can you solve it” — even a freshman can. The bar is: how many distinct approaches can you enumerate and rank by tradeoff? Strong candidates list 3–4 (sort, hash set, brute force, statistical/Bloom for huge N), state the time/space cost of each, then pick. Mid candidates pick HashSet first without naming alternatives. This is the cheapest problem in the curriculum at which to demonstrate the “I think about tradeoffs, not just solutions” senior signal.
It’s also the gateway to a family of problems: LC 219 (within-K distance — sliding window), LC 220 (within-K distance AND within-T value — bucket / treemap), LC 287 (find THE duplicate in O(1) space with constraints — Floyd’s), LC 442 (find ALL duplicates with in-place marking).
What it looks like it tests: detect a duplicate. What it actually tests: can you enumerate approaches and pick the right one based on memory / time / sortability constraints? Do you know when each wins?
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/contains-duplicate/
STOP. Set a 10-minute timer. Code it cold. Do not read past this section until solved or expired.
If repeat: write THREE distinct solutions in the same scratch file. Brute, sort, HashSet. Time yourself for ALL THREE in 8 min. The point isn’t to pick one — it’s to demonstrate fluency across approaches.
3. Prerequisite Concepts
- HashSet / HashMap O(1) average insertion and lookup (phase-01 §3)
- Sorting cost trade: O(N log N) time, O(1) extra space (if in-place sort allowed) vs HashSet’s O(N) time AND space
- Adjacent-equal detection after sort: a O(N) linear scan post-sort
- For massive N (billions, doesn’t fit memory): probabilistic structures (Bloom filter for “definitely-not-seen”, HyperLogLog for cardinality estimation)
4. How to Approach (FRAMEWORK Steps 1–9 applied)
Step 1 — Restate: “Given an integer array, return True if any value appears at least twice; else False.”
Step 2 — Clarify:
- “What’s N’s upper bound?” (LC says 10^5; ask anyway, the production answer changes for 10^9.)
- “Is the input mutable? Can I sort in place?” (Mutation may be off-limits in production; HashSet is the non-mutating choice.)
- “Memory budget?” (If tight, sort + scan is O(1) extra; HashSet is O(N).)
- “What are valid values? Negatives? Floats?” (Per LC, integers; in production ask.)
- “Repeats considered duplicates if they are at the same index?” (No — same index is the same value, not a duplicate. Trivial, but ask.)
Step 3 — Constraints: N up to 10^5 per LC. Both O(N log N) and O(N) fit comfortably. Brute O(N²) = 10^10 will TLE. Solution memory: O(N) for HashSet, O(1) for sort-based (if in-place).
Step 4 — Examples:
[1, 2, 3, 1]→ True[1, 2, 3, 4]→ False[1, 1, 1, 1]→ True (early exit on iteration 2)[]→ False (no values, no duplicate — confirm with interviewer)[7]→ False (single element)[-1, -1, 0]→ True (negatives)
Step 5 — Brute Force: Nested loop, compare every pair. O(N²) time, O(1) space. Acceptable as a stated baseline; never the final answer.
Step 6 — Brute Force Complexity: O(N²), O(1). TLE for LC’s constraint of 10^5.
Step 7 — Pattern Recognition: “Detect repetition in a sequence” → HashSet membership check (canonical) OR sort + adjacent compare (when memory matters). Family: LC 219 (with distance constraint → sliding window of size K), LC 220 (with distance AND value constraint → bucket-sort by value range), LC 287 (one specific duplicate in O(1) extra space → Floyd’s).
Step 8 — Optimize: Single-pass HashSet. Walk the array; for each x, if x in seen return True; else add x to seen. Return False at end. O(N) avg time, O(N) space. Early exit on first duplicate.
Step 9 — Prove correctness: Loop invariant: at top of iteration i, seen = {nums[0..i-1]} as a set. If nums[i] ∈ seen, then nums[i] equals some nums[j] with j < i — a true duplicate. If not, add and continue. After the loop, no duplicate was found, so all N values are distinct.
5. Progressive Hints
If stuck for 5 minutes, open hints.md. One hint, 5-min timer, retry.
6. Deeper Insight — Why It Works
Why HashSet is the textbook answer: “find a repeat” is structurally “have I seen this before?” — a membership question. The data structure built for that question in O(1) is the hash set. Memorize this connection: membership question → hash set.
Why sort-then-adjacent-compare is sometimes better: if N is huge but you can mutate, sort gives O(N log N) time and O(1) extra space (in-place sort). HashSet gives O(N) time but O(N) extra space. Memory often dominates the real cost — in Python, the overhead per dict entry is ~64 bytes, so a HashSet of 10^7 integers is ~640MB, which can push containers into OOM. Sort-based is the call for memory-constrained environments.
Why early exit matters: the HashSet check returns True the moment a duplicate is found. For arrays with frequent duplicates, expected runtime is O(distinct values) not O(N). The brute force can also early-exit, but its constant factor is worse and worst case is still O(N²).
Bloom filters for streaming / massive N: if N is 10^11 and you can’t fit a HashSet, a Bloom filter gives “definitely-not-seen” (no false negatives) with false-positive rate tunable by space. A pass with Bloom + a confirmation pass for flagged values gives expected O(N) at sub-N space. This is how anti-spam, anti-fraud, and CDN cache-membership work in production.
Why “Counter” overkill: Counter(nums).most_common(1)[0][1] > 1 works but allocates a full counter when you only need to know “any repeat?” — wastes O(distinct) memory growing to O(N) instead of early-exiting at first repeat. Idiomatic but suboptimal.
Why len(nums) != len(set(nums)) is cute but slow: materializes the full set even if a duplicate exists at index 1. Loses the early-exit advantage. For an array with a duplicate near the start, it’s O(N) where it could be O(2). The one-line elegance is a tradeoff against worst-case performance.
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — Brute force:
for i in range(n):
for j in range(i + 1, n):
if nums[i] == nums[j]: return True
return False
O(N²). For N=10^5, this is 10^10 operations. TLE on LC, OOM-irrelevant but compute-fatal. State as baseline, then optimize. Never the final answer.
Wrong-but-tempting #2 — len(set(nums)) != len(nums):
return len(set(nums)) != len(nums)
Pythonic one-liner, O(N) time, O(N) space, but no early exit. Materializes the entire set even when the second element is a duplicate. For interview, mention as a fallback but explicitly note: “I’d write the explicit loop for early exit; this one-liner pays O(N) even on best-case inputs.”
Wrong-but-tempting #3 — Sort then check adjacent, mutating input without asking:
nums.sort()
return any(nums[i] == nums[i+1] for i in range(len(nums) - 1))
Works, O(N log N) time, O(1) extra space if sort is in-place. But mutates input. In production this can break callers. Always ask “may I mutate?” — if yes, this is the memory-efficient answer.
Wrong-but-tempting #4 — Counter / dict to count:
from collections import Counter
return any(c > 1 for c in Counter(nums).values())
O(N) time, O(distinct) space, no early exit (Counter is built fully before iterating values). HashSet with explicit loop is strictly better.
Wrong-but-tempting #5 — Treating it like LC 219 / 220 without being asked: Some candidates over-engineer and start using a sliding window or a sorted set. The vanilla problem doesn’t care about position or value distance. Solve what’s asked. Save the sliding-window for the follow-up.
8. Skills & Takeaways
Generalizable patterns:
- Membership question → HashSet: any “have I seen this before?” question solves in O(1) avg per check.
- Memory vs time tradeoff articulation: sort + scan (O(N log N), O(1)) vs HashSet (O(N), O(N)). Knowing both is Senior; knowing when to pick which is Staff.
- Early-exit discipline: prefer explicit loops that can return on first success over one-liners that always materialize the full structure.
Analogous problems:
- LC 219 — Contains Duplicate II (within K indices → sliding window of HashSet of size K)
- LC 220 — Contains Nearby Almost Duplicate (within K indices AND |val_i − val_j| ≤ T → bucket sort by val ranges, OR a sorted multiset with binary search)
- LC 287 — Find the Duplicate Number (Hard; O(1) space → Floyd’s on
nums[nums[i]]) - LC 442 — Find All Duplicates in an Array (in-place marking by negating
nums[abs(x) - 1]) - LC 268 — Missing Number (the inverse: XOR or sum trick)
- LC 136 — Single Number (XOR every element: pairs cancel, the loner remains)
- LC 645 — Set Mismatch (one missing + one duplicate; combine XOR and counting)
When NOT to use HashSet: when memory is constrained AND you may sort. When N is so large it doesn’t fit RAM (use Bloom filter + external sort). When values have special structure (small integer range → array-as-hashmap; e.g., LC 442’s in-place marking trick).
9. When to Move On (binary; must all be YES)
- I wrote the HashSet solution with explicit early exit in <3 min
- I named at least 3 distinct approaches (brute, sort+scan, HashSet) with time/space for each
- I can articulate when sort+scan beats HashSet (memory-constrained, mutation OK)
-
I can explain why
len(set(nums)) != len(nums)is suboptimal -
My
stress_test()includes empty, single, all-equal, all-distinct, and large random arrays - I solved LC 219 (Contains Duplicate II) and recognized the sliding-window-of-HashSet extension
- I solved LC 136 (Single Number) and saw the XOR trick as an O(1)-space alternative for a constrained variant
- I know what a Bloom filter is and when it would be the right answer here
If any unchecked: redo before moving to Week 2 review.
10. Company Context
Amazon (often part of a larger story problem)
- The framing: “Detect any duplicate order ID in a day’s worth of transactions.”
- Misleading example: Empty input. Many candidates’ code returns True or crashes; correct is False.
- Adversarial extension: “Now N is 10 billion across 100 machines.” → distributed hash with sharding by ID mod 100; or Bloom filter per shard. Then: “Now we can’t use 640MB of RAM per machine — N is 10^9 per machine and we have 100MB.” → Bloom + confirmation pass.
- What they want to hear: “Let me list the approaches first” with explicit memory/time table. Amazon values articulated tradeoffs.
Microsoft (phone screen warmup)
- The framing: Straight problem.
- Misleading example:
[1, 1, 1, 1]— to see whether you exit early or keep iterating. - Adversarial extension: “Now within K indices.” (LC 219.) Then “now within K indices AND values within T of each other.” (LC 220.)
- What they want to hear: HashSet with explicit early-exit loop, mention of sort-based alternative for memory.
Apple / Bloomberg (clean code)
- The framing: Straight problem.
- What they want to hear: Idiomatic code, named the two main approaches, articulated the memory tradeoff.
Meta (rapid follow-ups)
- The framing: This → LC 219 → LC 220 → LC 287 in rapid succession.
- What they want to hear: Naming each extension correctly and switching algorithms as the constraints evolve (HashSet → sliding window → bucket sort → Floyd’s).
Google (medium frequency, focus on tradeoffs)
- The framing: “Solve, then optimize for memory.”
- What they want to hear: All three approaches articulated with complexity, then a discussion of when each wins. Bonus: mention Bloom filter for massive N.
11. Interviewer’s Lens
| Phase | Strong signal | Weak signal | Scorecard phrase (strong) |
|---|---|---|---|
| Reading problem | Asks “may I mutate? memory budget? N’s upper bound?” | Dives straight into HashSet | “Clarifies constraints before choosing data structure” |
| Pre-coding | Lists 3+ approaches with complexities | Picks HashSet without naming alternatives | “Demonstrates breadth of approaches — Senior signal” |
| Coding | Explicit loop with early exit, idiomatic but not gimmicky | One-liner len(set(nums)) != len(nums) | “Prefers explicit control flow for early-exit performance” |
| Follow-up (LC 219) | Pivots to sliding-window-of-HashSet | Re-uses plain HashSet, gets wrong answer | “Adapts data structure as constraints change” |
| Massive-N discussion | Mentions Bloom filter | Stops at HashSet | “Thinks about memory at scale — Principal signal” |
The scorecard line that gets you the offer: “Articulated multiple approaches with tradeoffs; chose HashSet with early exit; adapted cleanly to follow-up variants.”
The scorecard line that loses you: “Used the one-line len(set(nums)) without acknowledging it forfeits early exit; couldn’t articulate when sort+scan would be preferred.”
12. Level Delta
| Level | What their answer looks like |
|---|---|
| Mid | HashSet solution. Works. Doesn’t articulate alternatives unless asked. ~5 min. |
| Senior | HashSet with early exit + named alternatives (sort+scan, brute) with complexities + handled empty/single without prompting + ~3 min. |
| Staff | All of Senior + connected to LC 219 (sliding window) and LC 220 (bucket sort) and LC 287 (Floyd’s) as a family + mentioned Bloom filter for massive N + named when sort+scan beats HashSet. ~3 min. |
| Principal | All of Staff + discussed real production framing (deduplication in event pipelines, billing systems, ad impression tracking) + offered HyperLogLog as an O(1)-space cardinality-estimator alternative if you only need APPROXIMATE duplicate detection + named the sharding strategy for distributed implementation. ~3 min with full discussion. |
13. Follow-up Questions & Full Answers
Follow-up 1: “Now duplicates only count if they’re within K indices.” (LC 219)
Signal sought: Adapt data structure to constraint.
Full answer: “Sliding window of HashSet, size at most K+1. Walk the array; for each nums[i], check if it’s in the window. If yes, return True. Add nums[i]; if window size exceeds K, evict nums[i-K]. O(N) time, O(K) space. The key insight: only elements within K positions can collide, so we maintain exactly those in the set.”
Follow-up 2: “Within K indices AND |values differ| ≤ T.” (LC 220)
Signal sought: Bucket sort or sorted multiset.
Full answer: “Two approaches. (a) Bucket sort: assign each value to bucket val // (T + 1). Two values within T of each other land in the same bucket OR adjacent buckets. For each nums[i], check the current bucket and ±1 neighbors. Maintain at most K+1 active buckets (a dict of bucket → value), evicting when out of window. O(N) time, O(min(N, K)) space. (b) Sorted multiset with binary search: maintain a sorted multiset of the last K+1 elements; binary search for any element in [nums[i]−T, nums[i]+T]. O(N log K). Bucket is asymptotically better; sorted set is simpler to code.”
Follow-up 3: “Massive N — 10^11 values streaming, can’t fit in memory.”
Signal sought: Probabilistic structures.
Full answer: “Bloom filter. For each value: query the filter; if ‘might-have-seen’, confirm with a secondary check (e.g., on-disk index or HyperLogLog). If ‘definitely-not-seen’ (Bloom guarantees no false negatives), insert. Bloom uses ~1-2 bytes per element with 1% false positive rate, so 10^11 entries needs ~100-200GB across a cluster — distributable. Sharding: hash the value, route to the shard that owns its hash range. Each shard runs local Bloom. Caveat: Bloom can’t delete — for streams with eviction, use a Counting Bloom Filter (more space per cell) or a Cuckoo filter (supports delete cleanly).”
Follow-up 4: “Find THE duplicate in [1..N] with N+1 ints, O(1) extra space, no mutation.” (LC 287)
Signal sought: Algorithmic creativity.
Full answer: “Treat array as iterated function f(i) = nums[i]. Since N+1 values map to N possible outputs, some output is duplicated — corresponds to a cycle in the iteration graph. Apply Floyd’s cycle detection: slow advances 1, fast advances 2, find meet. Then reset one pointer to start (index 0), advance both by 1, they meet at the cycle start, which IS the duplicate. O(N) time, O(1) extra space. This is the same algorithm as Linked List Cycle (your p08), just on an array reinterpreted as a graph.”
Follow-up 5 (Senior signal): “How would you test this for correctness AND performance?”
Signal sought: Testing discipline + perf awareness.
Full answer: “Three layers. (1) Functional: empty, single, all-equal, all-distinct, duplicate at start (early-exit check), duplicate at end. (2) Property: for any randomly-generated array, HashSet result equals len(set(nums)) != len(nums) — strong cross-check. (3) Perf: benchmark early-exit on [1, 1, 0, 0, ...] (duplicate at index 1) — HashSet should take ~2 hash ops; the one-liner takes N hash ops. Confirm via timing. My solution.py includes the random property test.”
14. Full Solution Walkthrough
See solution.py.
The file has six sections:
contains_duplicate_brute(nums)— O(N²) nested loops; correctness oracle for small N.contains_duplicate(nums)— HashSet with explicit early-exit loop. The production answer.contains_duplicate_sort(nums)— sort + adjacent compare; O(N log N) time, O(1) extra (mutates input).contains_duplicate_oneliner(nums)—len(nums) != len(set(nums)); for comparison only.stress_test()— 1000 random arrays + skewed (mostly-duplicates, mostly-distinct) cases; all four implementations must agree.edge_cases()— empty, single, two-distinct, two-equal, all-equal, large random.
Decisions justified:
- Why
setnotdict: we only need existence, not value-to-index. - Why explicit
forloop not the one-liner: early-exit when duplicate found near start. - Why sort-based variant copies before sorting in stress test: don’t mutate caller’s data unexpectedly.
15. Beyond the Problem — Production Reality
Real applications:
- Deduplication in event pipelines: Kafka consumers, Kinesis stream processors — dedupe events by ID. Bloom filter per consumer + periodic full-set rotation.
- Billing: detect duplicate charge attempts (idempotency key check). HashSet in Redis, sized by time window.
- Ad impression tracking: prevent counting the same impression twice. Bloom or HyperLogLog if approximate is OK; exact HashSet (sharded) if money depends on it.
- Anti-fraud: detect repeat fraud attempts from the same device fingerprint. Distributed HashSet, time-windowed.
- Cache deduplication: CDNs avoid re-fetching by URL HashSet keyed by content hash.
Real systems that are exactly this kernel:
- Postgres
UNIQUEconstraint: under the hood, a B-tree-backed sorted set check. Sort-based duplicate detection. - Kafka exactly-once semantics: producer ID + sequence number HashSet at the broker, per partition.
- Redis
SADD: returns 1 if newly added, 0 if duplicate — same operation as our HashSet check.
Principal-engineer code review comment: “For our event dedup at 10M events/second, plain HashSet won’t cut it — we use Cuckoo filter for the hot path (supports delete unlike Bloom) and a sharded Redis SET for the source of truth. Also: please add a comment that this function is O(N) avg but O(N²) worst-case under hash collision attacks; for adversarial inputs (user-supplied integers in a public API), we use a keyed hash (SipHash) to prevent algorithmic DoS. We had this incident in 2019.”
p10 Contains Duplicate — Progressive Hints
Rule: Read ONE hint. Set a 5-minute timer. Try again. Only read the next hint if the timer expires.
Hint 1 — Pattern family
The question is “have I seen this value before?” — that’s a membership question. What data structure answers membership queries in O(1) average?
Hint 2 — Approach menu
Three serious approaches; rank by trade-off:
| Approach | Time | Extra space | Mutates input? |
|---|---|---|---|
| Brute (nested loop) | O(N²) | O(1) | No |
| Sort + adjacent compare | O(N log N) | O(1) in-place | Yes |
| HashSet | O(N) avg | O(N) | No |
Pick the one that matches the constraints (LC has loose constraints — pick HashSet).
Hint 3 — Key insight
Walk the array once. Maintain a HashSet of values seen so far. For each new element: if it’s already in the set, return True (early exit). Else add it and continue. If you finish the loop, return False.
Critical: early exit. Don’t materialize the whole set first — that loses the constant-factor advantage on inputs with a duplicate near the start.
Hint 4 — Pseudocode
seen = empty set
for x in nums:
if x in seen: return True
add x to seen
return False
That’s it. Three lines of logic.
Hint 5 — Why one-liner is worse + the family of variants
return len(set(nums)) != len(nums) works and is one line, but builds the full set even if a duplicate exists at index 1. For [1, 1, 0, 0, 0, ...] of size 10^6, the explicit loop returns after 2 ops; the one-liner does 10^6.
Loop invariant proof: at the top of iteration i, seen == {nums[0..i-1]} as a set. If nums[i] ∈ seen, then by definition some j < i has nums[j] == nums[i] — a duplicate exists. If after all N iterations no early-exit fired, all N values are distinct.
Time O(N) avg, O(N) worst case (hash collision attacks). Space O(N).
The family — know these as natural extensions:
- LC 219 (within K indices): sliding window of HashSet, size K+1; evict
nums[i-K-1]as you advance. - LC 220 (within K indices AND |val| diff ≤ T): bucket sort — bucket =
val // (T+1); collision in same or adjacent bucket = answer. - LC 287 (find THE one duplicate in [1..N] with N+1 elements, O(1) space): Floyd’s cycle detection on the array as
f(i) = nums[i]. - LC 442 (find ALL duplicates, O(1) extra space): in-place marking — negate
nums[abs(x) - 1]; if already negative, x is a duplicate.
Massive N (10^11, doesn’t fit RAM): Bloom filter. ~1-2 bytes per element with 1% false positive rate; confirm flagged values via secondary store.
Week 3 — Trees Intro: Recursion as the Default Tool
Theme: Binary trees are recursion’s natural habitat. Five problems that drill the “what does this function return for a subtree?” framing until it’s reflex. Time budget: ~6 hours focused work; ~10 hours including the LC bank. Hard rule: every problem solved twice — recursive first, then iterative (BFS/DFS-stack). No exceptions. Interviewers ask “what if recursion stack overflows?”
Why this week
Tree problems are the single largest topic family in mid/senior coding rounds (≈30% of all phone screen problems at FAANG). They look easy individually — each one is 10-20 lines — but candidates who never internalize the “what does my recursive call return?” discipline get bitten on the next-level problems (LCA, path sum, serialize, segment trees, etc.).
This week trains exactly that reflex:
- p11: function returns the (mutated) subtree root — base case is the null subtree.
- p12: function returns an integer (depth) — base case returns 0.
- p13: function returns a boolean — base case is both-null vs one-null.
- p14: function returns AN ANSWER to a sub-problem (height) while also UPDATING a global (diameter) — the “global + return value” two-channel pattern.
- p15: function takes TWO subtree pointers — the “mirror walk” generalization that reappears in LC 100, 572, 1367.
By Friday you should be able to write any of these in <5 minutes, both recursively and iteratively, and articulate the base case before the recursive case.
Flagship problems (the 5 you OWN this week)
| # | Problem | LC | Difficulty | Why this one |
|---|---|---|---|---|
| p11 | Invert Binary Tree | 226 | Easy | The “Homebrew” question. Trains the “return the mutated subtree” pattern. |
| p12 | Maximum Depth of Binary Tree | 104 | Easy | Trains “function returns an integer aggregate from children.” |
| p13 | Same Tree | 100 | Easy | Trains “function takes TWO trees” — the parallel-walk pattern. |
| p14 | Diameter of Binary Tree | 543 | Easy (deceptively) | The two-channel pattern: return one thing, update another globally. The single most important tree trick. |
| p15 | Symmetric Tree | 101 | Easy | Generalizes p13 — mirror walk. Sets up LC 951 (Flip Equivalent). |
LC bank (~25 problems for spaced repetition)
Mirror / structure equality:
- LC 226 (p11), LC 100 (p13), LC 101 (p15), LC 572 Subtree of Another, LC 951 Flip Equivalent, LC 1367 Linked List in Binary Tree
Depth / height:
- LC 104 (p12), LC 111 Min Depth, LC 110 Balanced Binary Tree, LC 559 Max Depth N-ary, LC 543 (p14)
Traversal mechanics:
- LC 144 Preorder, LC 94 Inorder, LC 145 Postorder, LC 102 Level Order, LC 103 Zigzag, LC 199 Right Side View, LC 107 Bottom-Up Level Order
Construction / manipulation:
- LC 108 Sorted Array to BST, LC 617 Merge Two Binary Trees, LC 1325 Delete Leaves with Given Value
Path / aggregate:
- LC 112 Path Sum, LC 257 Binary Tree Paths, LC 404 Sum of Left Leaves, LC 129 Sum Root-to-Leaf Numbers
Readiness gate (binary; must all be YES before Week 4)
-
I can write
TreeNodefrom memory:__slots__ = ("val", "left", "right"). - I solved each of p11–p15 in BOTH recursive AND iterative form.
- I can recite the “what does my recursive call return for a subtree?” framing for each.
- I can convert any recursive tree solution to iterative (BFS deque OR DFS stack) without looking up syntax.
-
I know
sys.setrecursionlimit(10_000)and why production code should iterative if depth can exceed ~1000. - I can articulate the “two-channel pattern” from p14: function returns one thing (height) but a closure variable accumulates another (diameter).
- I can solve LC 100 (Same Tree) and LC 101 (Symmetric Tree) and explain why they are the same algorithm with different mirror logic.
-
My
stress_test()for each problem cross-checks recursive vs iterative on 1000 random trees built withrandom.Random(42).
If ANY unchecked: do not start Week 4. Re-drill the missing pieces.
Recurring traps (steal these for the interview)
- Forgetting the null base case. Every tree recursion must handle
if not node: return <base>FIRST. - Returning the wrong thing from recursion. Write out “what does this function return?” in English BEFORE coding. p11 returns the mutated subtree root. p12 returns an int. p13 returns a bool. p14 returns height but updates a global. p15 takes TWO nodes.
- Iterative depth using
len(stack)as depth. That’s the current path length not the max depth seen. You need level-by-level BFS or to pair(node, depth)in the stack. - Mutating during traversal. p11 mutates. p12-p15 don’t. State which mode aloud.
- Stack overflow. LC test cases occasionally include skewed trees of depth 10^4. Recursive Python hits the recursion limit at ~1000. Bump with
sys.setrecursionlimit(10_000)OR use iterative.
Stretch (only if you finish all 5 + LC bank with time left)
- LC 124 Binary Tree Maximum Path Sum (Hard) — generalizes p14’s two-channel pattern with negatives. Highly testable at Staff level.
- LC 297 Serialize and Deserialize Binary Tree (Hard) — recursion + parsing. Common at Meta.
- LC 236 Lowest Common Ancestor — the canonical “return multiple things via tuple from recursion” problem.
Begin with p11 — Invert Binary Tree.
p11 — Invert Binary Tree
Source: LeetCode 226 · Easy · Topics: Tree, DFS, BFS, Binary Tree Companies (2024–2025 frequency): Google (very high), Amazon (high), Microsoft (high), Meta (medium), Apple (medium), Bloomberg (medium) Loop position: phone screen warmup; the legendary “Max Howell / Homebrew” tweet question (Google rejected him for not solving it). Almost universally asked.
1. Quick Context
This is the canonical “I expect you to know how to recurse on a tree” question. The recursive solution is 4 lines. The interviewer is testing whether recursion on a tree is reflex for you. Mid candidates get it but stumble on the base case. Senior candidates write it cold, then immediately offer the iterative BFS variant when asked. Staff candidates discuss in-place vs out-of-place, mutation contract, and the wider mirror-walk family (LC 100, 101, 951).
The “trick” is recognizing the pattern: what does this function return for a subtree? Answer: the inverted subtree’s root. Base case: empty subtree returns empty. Recursive case: swap the children (after recursively inverting them).
What it looks like it tests: can you recurse on a tree? What it actually tests: is recursion reflex, can you pivot to iterative under pressure, do you state the base case before coding?
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/invert-binary-tree/
STOP. Set a 5-minute timer. Code it cold — BOTH recursive AND iterative. Do not read past this section until both solved or expired.
If you’ve seen it: target 3 min for recursive, 4 min for iterative. Articulate base case aloud first.
3. Prerequisite Concepts
- TreeNode definition and binary tree mental model (phase-01 §9)
- Recursion stack & base case discipline (phase-01 §8)
- BFS with
collections.dequefor iterative tree traversal - DFS with explicit stack for iterative tree traversal
- Python’s
sys.setrecursionlimitand when to bump it - In-place mutation vs. building a new tree (interview contract)
4. How to Approach (FRAMEWORK Steps 1–9 applied)
Step 1 — Restate: “Given the root of a binary tree, swap every node’s left and right children. Return the modified root.”
Step 2 — Clarify:
- “May I mutate the input?” (Usually yes per LC. If not, build a new tree — different algorithm.)
- “What’s the tree’s maximum depth?” (LC says up to 100. Production: ask. >1000 risks Python recursion overflow.)
- “Empty tree input?” (Yes — return None.)
- “Does the tree have unique values?” (Irrelevant — we operate on structure, not values.)
- “Is this a BST?” (No — inverting destroys BST property; the problem cares about structural mirror, not ordering.)
Step 3 — Constraints: N up to 100 per LC; trivial. Production: could be millions. Time O(N) — must touch every node. Space O(H) for recursion (H = height; O(log N) balanced, O(N) skewed).
Step 4 — Examples:
[4,2,7,1,3,6,9]→[4,7,2,9,6,3,1](canonical LC).[]→[](empty).[1]→[1](single node — swap does nothing).[1,2]→[1,null,2](left-only becomes right-only).- Skewed left-only of depth 100: tests recursion-stack handling.
Step 5 — Brute Force: Recursion IS the natural form here. No “brute force” with worse complexity — every approach touches every node once at O(N).
Step 6 — Complexity: O(N) time (must visit each node). Space O(H) recursive call stack OR O(W) for iterative BFS (W = max width). For balanced trees O(log N); skewed O(N) either way.
Step 7 — Pattern Recognition: “Mutate every node in a tree” → post-order or pre-order DFS (either works; we choose pre-order: swap, then recurse). Iterative variant: BFS queue OR DFS stack. This is the prototypical tree-mutation algorithm.
Step 8 — Optimize: No further optimization possible — already optimal. The “optimization” is recognizing the iterative variant matters for production code where recursion stack overflow is a real risk for trees of depth >1000.
Step 9 — Prove correctness: Induction on tree size. Base case: empty tree is its own mirror (vacuously inverted). Inductive step: assume invert correctly inverts trees of size < N. For tree of size N: recursively invert left and right subtrees (both have size < N, so correct by hypothesis). Swap them. Result: original-left (now inverted) is at right; original-right (now inverted) is at left. By definition, this IS the inverted tree.
5. Progressive Hints
If stuck for 5 minutes, open hints.md. One hint, 5-min timer, retry.
6. Deeper Insight — Why It Works
The “return the mutated subtree root” framing: every tree-mutation recursion should be written by first answering “what does this function return for a subtree?” Here: it returns the root of the now-inverted subtree (which happens to be the same node we received, just with mutated children). Stating this aloud before coding is the difference between Senior and Mid signal. Mid candidates start typing; Senior candidates state the contract.
Why pre-order vs post-order doesn’t matter for THIS problem: swap-then-recurse and recurse-then-swap both work because the swap operates on the children pointers, and the recursion operates on the subtrees. The two are commutative for this specific operation. (In other problems — e.g., serializing while mutating — order matters; train yourself to ask “does my operation commute with the recursion?”)
In-place vs new-tree contract: LC’s signature returns the (same) root, signaling mutation is fine. Production may forbid mutation. The non-mutating version: return TreeNode(node.val, invert(node.right), invert(node.left)). Same O(N) time, same O(H) recursion space, plus O(N) for the new nodes. State this distinction aloud — it shows production awareness.
Iterative variants — when and why:
- BFS (queue): visit level by level, swap children at each node, enqueue both children. Natural for level-by-level operations.
- DFS (stack): push root, pop, swap children, push both. Slightly different traversal order from recursion but same final result (because swap commutes).
- Why iterative matters: Python’s default recursion limit is 1000. A tree of depth 1500 will
RecursionError. Iterative BFS uses heap-allocated deque — no stack overflow. Real interviewer follow-up: “Now the tree has depth 10^5. Recursion won’t work — rewrite.”
The Max Howell story is the cultural pretext: Google rejected the creator of Homebrew (then the world’s most popular macOS package manager) for not solving this on a whiteboard. The interviewer was wrong to gatekeep this hard, BUT the story made the problem famous, and now every shop asks it. Don’t fail it.
Connection to mirror-walk family:
- LC 101 (Symmetric Tree, p15): check if a tree is its own mirror. The check is “is
leftthe mirror ofright?” — same algorithm, only walks both sides simultaneously without mutation. - LC 951 (Flip Equivalent): are two trees equivalent under any sequence of child-flips? Recursion:
equiv(a,b) = (a.val==b.val) AND (equiv(a.left,b.left) AND equiv(a.right,b.right) OR equiv(a.left,b.right) AND equiv(a.right,b.left)).
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — Forgetting the null base case:
def invert(node):
node.left, node.right = invert(node.right), invert(node.left)
return node
Crashes immediately on the recursive call into a None child. The fix: if node is None: return None as the FIRST line.
Wrong-but-tempting #2 — Swapping references but not recursing:
def invert(node):
if not node: return None
node.left, node.right = node.right, node.left
return node
Only swaps the root’s children. The subtrees are NOT inverted. Test with depth-3 tree.
Wrong-but-tempting #3 — Recursing but not swapping at the current level:
def invert(node):
if not node: return None
invert(node.left)
invert(node.right)
return node
Inverts each subtree’s internals but never swaps left/right at any level. Returns the original tree unchanged.
Wrong-but-tempting #4 — Two-step “save then swap” with the wrong order:
left = node.left
node.left = invert(node.right)
node.right = invert(left)
This is actually correct (and equivalent to the tuple-swap version) but visually confusing. Prefer the tuple swap; it’s atomic and idiomatic.
Wrong-but-tempting #5 — Iterative DFS with for over children list mid-mutation:
stack = [root]
while stack:
node = stack.pop()
if not node: continue
for child in (node.left, node.right):
stack.append(child)
node.left, node.right = node.right, node.left
Subtle: for child in (node.left, node.right) evaluates the tuple BEFORE the swap, so children are pushed correctly. Easy to bug by swapping FIRST then pushing — works equivalently but visually inverted. State which order you chose.
8. Skills & Takeaways
Generalizable patterns:
- “What does this function return for a subtree?” — universal tree-recursion framing. Apply to every tree problem.
- Base case first. Every tree recursion’s line 1 is
if not node: return <base>. - Iterative tree traversal with deque/stack — must be reflex for the “recursion overflow” follow-up.
- In-place vs. new-tree contract — clarify with interviewer; production often forbids mutation.
Analogous problems:
- LC 100 — Same Tree (p13): walks two trees in parallel, returns bool.
- LC 101 — Symmetric Tree (p15): walks one tree as if it were two mirrored sides.
- LC 572 — Subtree of Another Tree:
is_subtree(s, t) = is_same(s, t) OR is_subtree(s.left, t) OR is_subtree(s.right, t). - LC 617 — Merge Two Binary Trees: same pattern but on two trees with value-summing.
- LC 951 — Flip Equivalent Binary Trees: invert-or-don’t recursion.
- LC 1325 — Delete Leaves with Given Value: post-order mutation (must recurse first, then check leaf).
When NOT to recurse: when tree depth may exceed ~1000 (Python recursion limit). Switch to iterative BFS or DFS-stack.
9. When to Move On (binary; must all be YES)
-
I wrote the recursive version cold in <3 min, with
if not node: return Noneas line 1. - I wrote the iterative BFS version cold in <5 min.
- I can articulate “this function returns the inverted subtree’s root” before coding.
- I can convert to non-mutating form (returns a new tree) on request.
-
My
stress_test()cross-checks recursive vs iterative on 1000 random trees. -
I can answer “what if depth is 10^5?” without panic (iterative +
sys.setrecursionlimitfor recursive variant). - I solved LC 101 (Symmetric Tree) and recognized it as the same algorithm walking two pointers.
- I know the Max Howell story and can explain why iterative variant matters in production.
If any unchecked: redo before moving to p12.
10. Company Context
Google (extremely common; THE question)
- The framing: The Max Howell question. Straight problem.
- Misleading example: Often a depth-3 perfectly balanced tree. The recursive 4-liner works trivially.
- Adversarial extension: “Now do it iteratively.” (BFS deque or DFS stack.) Then: “Now without mutating the input.” (Build a new tree.) Then: “Now the tree has depth 10^5.”
- What they want to hear: Both recursive AND iterative ready. Explicit “I’ll mutate in place since the contract allows; here’s the non-mutating variant if needed.”
Amazon (high frequency; phone screen warmup)
- The framing: Straight problem, often paired with a second harder question.
- Misleading example: Includes a null child early to test base case discipline.
- Adversarial extension: “Now invert ONLY the bottom K levels.” Then: “Now invert in-place AND return the original tree (deep-copy first).”
- What they want to hear: Confident recursion with base case; clean iterative pivot.
Microsoft (high frequency)
- The framing: Straight, in C++ for systems roles or Python/Java for SWE.
- Adversarial extension: “Now do it without recursion AND without an explicit stack/queue.” (Morris-traversal-style — Hard; usually only Staff-track interviews go here.)
- What they want to hear: Recursion as default, BFS/DFS-stack as fallback, awareness that Morris traversal exists for O(1) extra space.
Meta (medium; usually as warmup to LC 100/101 chain)
- The framing: Followed by p13 (Same Tree) and p15 (Symmetric Tree) in rapid succession.
- What they want to hear: Recognition that these three problems share the parallel-tree-walk pattern. Pivot fluidly.
Apple, Bloomberg (medium)
- The framing: Standard.
- What they want to hear: Clean code; both recursive and iterative; clear base case.
11. Interviewer’s Lens
| Phase | Strong signal | Weak signal | Scorecard phrase (strong) |
|---|---|---|---|
| Reading problem | Asks “may I mutate? what’s the max depth?” | Dives straight in | “Clarifies mutation contract before coding” |
| Pre-coding | States “function returns the inverted subtree root; base case is None → None” | Starts typing | “Articulates recursive contract — Senior signal” |
| Coding (recursive) | Base case FIRST, then tuple-swap and recurse | Forgets base case OR swaps without recursing | “Reflex recursion with disciplined base case” |
| Iterative pivot | Writes BFS with deque in <5 min on request | Stumbles on iterative pivot | “Fluent in both paradigms” |
| Depth follow-up | “Iterative is heap-allocated, no overflow; sys.setrecursionlimit for recursive” | Says “use recursion” with no overflow awareness | “Production-aware: knows when iterative is required” |
The scorecard line that gets you the offer: “Recursive in <2 min, iterative in <4 min, articulated base case before coding, handled the 10^5 depth follow-up cleanly.”
The scorecard line that loses you: “Got stuck on the iterative variant; couldn’t explain why depth matters.”
12. Level Delta
| Level | What their answer looks like |
|---|---|
| Mid | Recursive 4-liner works. Forgets base case once, fixes when interviewer points to a null child. Iterative requires nudge. ~6 min. |
| Senior | Recursive cold in <2 min, base case stated aloud first. Iterative BFS in <4 min on request. ~5 min total. |
| Staff | Recursive + iterative + non-mutating variant + handles depth-10^5 follow-up explicitly + names the mirror-walk family (LC 100, 101, 951). ~5 min. |
| Principal | All of Staff + discusses production implications (recursion limits in different languages; JVM tail-call non-elimination; Go’s growable stack; Rust’s no-stack-overflow guarantee with iterative pattern), AND mentions that some real systems (e.g., immutable persistent data structures in functional langs) use the non-mutating variant for thread safety. ~5 min with depth. |
13. Follow-up Questions & Full Answers
Follow-up 1: “Do it iteratively.”
Signal sought: Iterative tree traversal reflex.
Full answer: “BFS with collections.deque. Enqueue root; while queue nonempty, dequeue a node, swap its children, enqueue both children (skipping None). O(N) time, O(W) space where W is max width (up to N/2 for the last level of a complete tree). DFS stack alternative works identically — push root, pop, swap, push children. I’d prefer BFS for level-by-level operations and DFS-stack when memory is tighter on wide trees.”
Follow-up 2: “Without mutating the input.”
Signal sought: Awareness of mutation contract.
Full answer: “Build a new tree. def invert_pure(node): if not node: return None; return TreeNode(node.val, invert_pure(node.right), invert_pure(node.left)). Note the recursion arguments are swapped in the constructor — right goes into the new left child slot. Same O(N) time and O(H) recursion depth, plus O(N) for the new tree allocation. Required when the original tree is shared (e.g., immutable persistent structure in a multi-threaded context).”
Follow-up 3: “Tree has depth 10^5 — recursion overflows.”
Signal sought: Production-language awareness.
Full answer: “Two options. (a) sys.setrecursionlimit(200_000) — works for Python but expensive: each stack frame is ~500 bytes, so depth 10^5 uses ~50MB just for the call stack. (b) Iterative BFS or DFS-stack — heap-allocated, no stack-frame overhead. I’d default to iterative for any depth above ~1000. In Java the equivalent is -Xss JVM flag; in Go, goroutine stacks auto-grow; in Rust, you’d write iterative because stack overflow is a hard abort with no graceful recovery.”
Follow-up 4: “Now invert only the bottom K levels.”
Signal sought: Adapt recursion to a depth parameter.
Full answer: “Pass a level parameter (root = 0). Compute total depth first (one pass) to know the cutoff: cutoff = total_depth - K. Then a second pass: if level >= cutoff, swap; recurse with level + 1. Or, more elegantly: recurse first to compute height-from-bottom; on the way back up, swap if height_from_bottom < K. Single pass, post-order, O(N) time.”
Follow-up 5 (Staff signal): “Verify two trees are inversions of each other (without actually inverting).”
Signal sought: Generalization to LC 101’s pattern.
Full answer: “Walk both trees in parallel: def are_inversions(a, b): if a is None and b is None: return True; if a is None or b is None: return False; return a.val == b.val and are_inversions(a.left, b.right) and are_inversions(a.right, b.left). The cross-recursion (a.left vs b.right) is the inversion check. O(N), O(H). This IS the algorithm for LC 101 (Symmetric Tree) generalized to two distinct trees.”
14. Full Solution Walkthrough
See solution.py.
The file has six sections:
TreeNode—__slots__ = ("val", "left", "right")for memory efficiency.invert_recursive(node)— 4-line classic; base case first.invert_iterative_bfs(node)—collections.deque; level-by-level.invert_iterative_dfs(node)— explicit list as stack.invert_pure(node)— non-mutating variant; builds new tree.stress_test()— builds 1000 random trees withrandom.Random(42); verifies all four implementations produce the same result (via deep equality on serialized form).edge_cases()— empty, single, left-only-skewed, balanced, right-only-skewed.
Decisions justified:
- Why tuple swap (
node.left, node.right = invert(node.right), invert(node.left)): atomic, idiomatic, equivalent to save-then-swap. - Why
__slots__: a million-node tree saves ~50MB. - Why we serialize trees for equality check (in stress_test): structural equality of mutable trees requires either parallel-walk or canonical serialization; we use the latter to also exercise level-order encoding.
15. Beyond the Problem — Production Reality
Where this exact algorithm shows up:
- Compiler IR transformations: many AST rewrite passes are exactly “walk the tree, mutate each node.” Inverting a tree is a tiny special case of a larger family.
- Database query plan rewrites: Postgres’ query planner walks the plan tree and rewrites nodes (predicate pushdown, join reordering). Iterative because plan trees can be very deep.
- Game-tree pruning (minimax): alpha-beta is a recursive tree walk; in some implementations the tree is mirrored when switching between max-player and min-player perspectives.
- UI rendering: React’s virtual DOM reconciliation walks a tree and mutates. Mostly iterative for the same depth reasons.
Production caveats:
- Thread safety: in-place mutation is NOT thread-safe. If the tree is shared (read-only) across threads, you MUST use the non-mutating variant.
- Persistent data structures: in functional languages (Clojure, Haskell, OCaml), all tree updates are non-mutating by default — you return a new tree sharing unchanged subtrees. This is structural sharing; the non-mutating variant above is the entry point.
- GC pressure: the non-mutating variant allocates N new nodes per inversion. For trees of millions of nodes, this can trigger GC pauses. In-place mutation is preferred when allowed.
- Recursion limits: as noted, depth >1000 in Python requires
setrecursionlimitor iterative rewrite. The pattern of “default to recursive, fall back to iterative when depth grows” is universal across tree algorithms.
Principal-engineer code review comment: “Recursive invert is fine for our render tree (max depth ~50). But our query planner has trees up to 1500 deep on complex multi-join queries — we hit RecursionError in prod last quarter. Switched to iterative DFS-stack. PR comment to future: any tree-walk on user-supplied data MUST be iterative. Cost of the deque allocation is negligible vs. the cost of a SIGSEGV-equivalent crash on the request thread. Add a lint rule that flags def visit_node(...) without a corresponding iterative variant for trees with unbounded depth.”
p11 Invert Binary Tree — Progressive Hints
Rule: Read ONE hint. Set a 5-minute timer. Try again. Only read the next hint if the timer expires.
Hint 1 — Pattern family
This is a tree-mutation problem. The structure of every tree-recursion is:
- Base case (what to do at a null subtree)
- Recurse on left and right subtrees
- Combine / mutate at the current node
Ask yourself: what does this function return for a subtree?
Hint 2 — The contract
Your function takes a subtree root, mutates it so the entire subtree is inverted, and returns the (same, now-inverted) root. Base case for null: return null.
Hint 3 — Key insight
To invert a tree at node N:
- Invert N’s left subtree (recursively).
- Invert N’s right subtree (recursively).
- Swap N’s left and right child pointers.
Steps 1 and 2 are independent. Step 3 can happen before OR after them — both orders give the same result because the swap operates on pointers while the recursion operates on subtrees.
Hint 4 — Pseudocode (recursive) + iterative skeleton
Recursive:
def invert(node):
if node is None: return None
node.left, node.right = invert(node.right), invert(node.left)
return node
Iterative BFS:
if root is None: return None
queue = deque([root])
while queue:
cur = queue.popleft()
swap cur.left and cur.right
if cur.left: queue.append(cur.left)
if cur.right: queue.append(cur.right)
return root
Hint 5 — Full algorithm + production notes
Recursive — 4 lines:
def invert(node):
if node is None:
return None
node.left, node.right = invert(node.right), invert(node.left)
return node
The tuple-swap on the right evaluates BOTH recursive calls first, then atomically assigns to node.left and node.right. No temporary needed.
Time O(N) — visit every node once. Space O(H) — recursion depth equals tree height. For balanced trees O(log N); for skewed, O(N).
Production note — recursion limit: Python’s default recursion limit is 1000. A tree of depth >1000 (skewed) will raise RecursionError. Two fixes: sys.setrecursionlimit(10_000) (bumps the limit) or rewrite iteratively (heap-allocated, no overflow). Always offer the iterative version when the interviewer asks about scale.
Non-mutating variant (when the input can’t be touched):
def invert_pure(node):
if node is None: return None
return TreeNode(node.val, invert_pure(node.right), invert_pure(node.left))
Same O(N) time, same O(H) depth, plus O(N) for new nodes. Required for thread-safe / persistent / read-only trees.
Involution property (great for property-based testing): invert(invert(x)) == x for any tree x. The stress_test exploits this.
p12 — Maximum Depth of Binary Tree
Source: LeetCode 104 · Easy · Topics: Tree, DFS, BFS, Binary Tree Companies (2024–2025 frequency): Amazon (very high), Microsoft (high), Apple (high), Bloomberg (medium), Meta (medium), Google (medium) Loop position: phone screen warmup; often paired with LC 111 (Min Depth) or LC 110 (Balanced).
1. Quick Context
This is the canonical “function returns an integer aggregate from children” problem. The recursion: depth(node) = 1 + max(depth(left), depth(right)) with base case depth(None) = 0. It’s the foundational template for height, sum, count, max-value, and dozens of other tree-aggregation problems. Mastery here makes LC 110 (Balanced), LC 543 (Diameter, p14), LC 124 (Max Path Sum), and the entire DP-on-trees family trivial.
Mid candidates write it. Senior candidates write it AND the iterative BFS version AND explain why level-by-level matters. Staff candidates connect it to LC 110 (height check that’s O(N) instead of O(N²)) and LC 111 (subtle min-depth trap: a node with one missing child is NOT a leaf).
What it looks like it tests: compute height. What it actually tests: can you write “aggregate-from-children” recursion as reflex? Can you pivot to BFS for level-by-level reasoning?
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/maximum-depth-of-binary-tree/
STOP. Set a 5-minute timer. Code BOTH recursive AND iterative BFS. Do not read past this section until both solved or expired.
3. Prerequisite Concepts
- TreeNode and binary tree mental model (phase-01 §9)
- Recursion with integer return type (phase-01 §8)
- BFS with
collections.dequefor level-by-level counting - The “post-order aggregate” pattern: compute children’s values, then combine
4. How to Approach (FRAMEWORK Steps 1–9 applied)
Step 1 — Restate: “Given the root of a binary tree, return the number of nodes along the longest path from the root down to the farthest leaf.”
Step 2 — Clarify:
- “Empty tree → 0?” (Yes per LC.)
- “Single node — depth 1 or 0?” (1 per LC; the single node IS the root AND a leaf.)
- “Path counts NODES or EDGES?” (Nodes per LC. Some problems count edges — always ask.)
- “Max recursion depth I should worry about?” (LC says up to 10^4 — RecursionError risk in Python without
setrecursionlimit.) - “All-left or all-right skewed input expected?” (Possible; pushes recursion depth to N.)
Step 3 — Constraints: N up to 10^4 per LC. O(N) time required (must visit every node to know the depth). Recursion space O(H) for height-H tree; can be O(N) for skewed.
Step 4 — Examples:
[3,9,20,null,null,15,7]→ 3.[]→ 0.[1]→ 1.[1,null,2,null,3,null,4](right-skewed) → 4.- Balanced tree of N=15 (depth 4) → 4.
Step 5 — Brute Force: Recursion IS the natural form. “Brute force” alternative: enumerate every root-to-leaf path, return max path length. Same O(N) time but messier code.
Step 6 — Complexity: O(N) time (touch every node). Space O(H) recursion stack OR O(W) BFS queue (W = max width, up to N/2 for the bottom level of a complete tree).
Step 7 — Pattern Recognition: “Compute an integer aggregate over a tree” → post-order DFS (combine children’s results at the current node). Iterative alternatives: BFS counting levels; DFS-stack with explicit (node, depth) pairs.
Step 8 — Optimize: Already optimal. The “optimization” question is the iterative variant for stack-safety on deep trees.
Step 9 — Prove correctness: Induction on tree size. Base: empty tree has depth 0. Inductive step: assume function correct on trees of size < N. For tree of size N with root r: left subtree (size < N) returns depth_L; right subtree returns depth_R. By definition, the longest root-to-leaf path either goes through left or through right; its length is 1 + max(depth_L, depth_R). Correct.
5. Progressive Hints
If stuck for 5 minutes, open hints.md.
6. Deeper Insight — Why It Works
The “aggregate-from-children” template: any tree problem of the form “compute an integer property of a tree” follows this template:
def aggregate(node):
if node is None: return BASE
left_val = aggregate(node.left)
right_val = aggregate(node.right)
return COMBINE(node.val, left_val, right_val)
For depth: BASE = 0, COMBINE = 1 + max(left, right). For node count: BASE = 0, COMBINE = 1 + left + right. For sum: BASE = 0, COMBINE = node.val + left + right. For max value: BASE = -∞, COMBINE = max(node.val, left, right). Internalize this template — it generalizes to LC 110, 543, 124, 543, 968, etc.
Why post-order (not pre-order): the combine step uses children’s results, so children must be computed first. Pre-order would compute the root before children — impossible when the answer depends on children.
Iterative BFS — counting levels: the depth equals the number of BFS levels traversed.
depth = 0; q = deque([root])
while q:
depth += 1
for _ in range(len(q)):
cur = q.popleft()
if cur.left: q.append(cur.left)
if cur.right: q.append(cur.right)
return depth
The for _ in range(len(q)) is critical: it snapshots the level’s size before any appends, so the inner loop processes exactly one level. Without it, the loop processes all queued nodes including children added during this level — degenerate to total-node-count, not depth. This is the “snapshot-level-size” pattern; reappears in LC 102, 199, 103.
Iterative DFS — (node, depth) pairs:
stack = [(root, 1)]; best = 0
while stack:
node, d = stack.pop()
if node is None: continue
best = max(best, d)
stack.append((node.left, d + 1))
stack.append((node.right, d + 1))
return best
Carry the depth alongside the node. Equivalent to recursive but heap-allocated.
The LC 111 (Min Depth) trap — DON’T transplant the algorithm: min_depth(node) = 1 + min(min_depth(left), min_depth(right)) is WRONG when one child is None. A node with left=None, right=non-empty is NOT a leaf — its depth-to-leaf goes through the right subtree. The correct min-depth handles this: if one child is None, recurse only into the other. This is why LC 111 is “Easy” but trips many candidates — the symmetry with max-depth is broken by the leaf definition.
Production: tail recursion? Python doesn’t optimize tail calls, but the recursion isn’t tail-recursive anyway (1 + max(...) requires the recursive results before computing). Always iterative for deep trees in production.
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — Forgetting the +1:
def depth(node):
if not node: return 0
return max(depth(node.left), depth(node.right)) # missing +1
Returns 0 for any tree. Add 1 + to count the current node.
Wrong-but-tempting #2 — Returning 1 from None:
if not node: return 1
Off-by-one: every leaf gets depth 2, root gets total-depth + 1. Base case for empty is 0.
Wrong-but-tempting #3 — BFS without level-size snapshot:
q = deque([root])
depth = 0
while q:
cur = q.popleft()
depth += 1 # WRONG: counts nodes, not levels
if cur.left: q.append(cur.left)
if cur.right: q.append(cur.right)
Returns total node count. Need the inner for _ in range(len(q)) loop to process one level at a time.
Wrong-but-tempting #4 — Using len(stack) as current depth:
stack = [root]
while stack:
cur = stack.pop()
best = max(best, len(stack)) # WRONG
len(stack) is the number of UNPROCESSED nodes, not the current path length. Always pair (node, depth) explicitly.
Wrong-but-tempting #5 — Transplanting to LC 111 without re-thinking:
def min_depth(node):
if not node: return 0
return 1 + min(min_depth(node.left), min_depth(node.right)) # WRONG
For a tree with only a right child, min(0, 1+...) returns 1 (root with phantom-leaf in the missing-left slot). But the root is not a leaf — actual min depth is 1 + (right's min depth). Always re-derive when symmetry is suspicious.
8. Skills & Takeaways
Generalizable patterns:
- “Aggregate-from-children” recursion — the template for ALL tree-DP problems.
- BFS level-by-level with size snapshot — reappears in LC 102, 103, 107, 199.
(node, payload)stack pairs — the iterative equivalent of “carrying state through recursion.”
Analogous problems:
- LC 111 — Min Depth (subtle leaf-definition trap).
- LC 559 — Max Depth N-ary Tree (children list instead of left/right).
- LC 110 — Balanced Binary Tree (use depth recursion to early-exit on imbalance; the O(N) variant is “return depth or -1 if unbalanced”).
- LC 543 — Diameter (p14): same height computation, but accumulates a global max as a side effect.
- LC 124 — Max Path Sum: same template, but the “combine” allows skipping negatives.
- LC 222 — Count Complete Tree Nodes (Hard variant: O(log² N) using completeness).
- LC 366 — Find Leaves of Binary Tree (height-from-bottom is the level number).
When NOT to use recursion: depth > 1000 (Python). Use iterative BFS (counts levels) or DFS-stack with (node, depth) pairs.
9. When to Move On
-
Recursive in <2 min, base case
if not node: return 0stated aloud. - Iterative BFS with level-size snapshot in <4 min.
-
Iterative DFS with
(node, depth)pair in <4 min. -
I can explain why
1 +is on the OUTSIDE ofmax, not inside. -
My
stress_test()cross-checks all three implementations. - I solved LC 111 (Min Depth) and articulated why direct transplant is wrong.
- I sketched LC 110 (Balanced) using the “return depth or -1” trick.
- I see how this template generalizes to LC 543 and LC 124.
10. Company Context
Amazon
- The framing: Straight problem; or “given a folder tree, find the deepest path.”
- Adversarial extension: “Now return the actual path, not just length.” (Carry path as a list; return tuple of (depth, path).) Then “Now it’s a DAG, not a tree.” (Memoize.)
- What they want to hear: Recursive default + iterative on request + extension to N-ary trees.
Microsoft / Apple
- The framing: Straight problem; sometimes paired with LC 111 to test for the trap.
- What they want to hear: Clean recursion + handling of empty and skewed cases.
Bloomberg
- The framing: “Given a binary tree representing a financial hierarchy, find the longest reporting chain.”
- Adversarial extension: “Now the tree is N-ary (each manager has K reports).” (LC 559.)
- What they want to hear: Generalization to N-ary; iterative variant for deep org charts.
Meta
- The framing: Usually as warmup to LC 543 (Diameter) or LC 124 (Max Path Sum).
- What they want to hear: Recognition that depth/height is the building block; pivot smoothly to diameter when prompted.
- The framing: Straight or “in a tree of compute jobs, find the critical path length.”
- What they want to hear: Both DFS and BFS variants; complexity analysis distinguishing balanced vs skewed.
11. Interviewer’s Lens
| Phase | Strong signal | Weak signal | Scorecard phrase (strong) |
|---|---|---|---|
| Reading problem | Asks “empty? single node? nodes vs edges?” | Dives in | “Clarifies counting convention” |
| Pre-coding | States base case (0) and recurrence (1 + max(...)) aloud | Starts typing | “Articulates contract — Senior” |
| Coding | Tight 3-liner, correct base case | Off-by-one, places +1 wrong | “Reflex aggregate-from-children” |
| Iterative pivot | BFS with level-size snapshot in <4 min | BFS counts nodes (wrong), or struggles | “Knows BFS-level pattern cold” |
| Connect to family | Names LC 543, 124, 110 as same template | Sees only this problem | “Sees the template — Staff signal” |
12. Level Delta
| Level | Looks like |
|---|---|
| Mid | Recursive correct in ~5 min. Iterative needs prompting. ~8 min. |
| Senior | Recursive in <2 min. Iterative BFS with level-size in <4 min. States base case aloud. ~5 min. |
| Staff | All Senior + names LC 110 (Balanced as O(N) early-exit variant) and LC 543 (Diameter) as the same template + addresses skewed-tree recursion limit. ~5 min. |
| Principal | All Staff + discusses production: tail-recursion absent in Python; iterative mandatory for unbounded input; mentions stack-safe trampoline patterns in FP languages; discusses memory cost of (node, depth) pairs vs simple depth counter. ~5 min. |
13. Follow-up Questions & Full Answers
Follow-up 1: “Iteratively, in BFS form.”
Full answer: “Use a deque. Initialize depth=0. While the queue is nonempty: increment depth, snapshot n = len(queue), then process exactly n nodes (each one’s children get appended for next level). Return depth. The len(queue) snapshot is critical — it pins the count BEFORE new appends, so the inner loop sees exactly one level. O(N) time, O(W) space.”
Follow-up 2: “Iteratively, in DFS form.”
Full answer: “Stack of (node, depth) pairs. Pop, update global max, push children with depth+1. Skip None children at push time or check on pop — both work. O(N) time, O(H) space.”
Follow-up 3: “Now compute the MIN depth (LC 111).”
Signal sought: spot the leaf-definition trap.
Full answer: “Cannot just swap max for min — a node with one missing child is NOT a leaf, so its ‘depth via missing side’ shouldn’t count. Correct: if both children None → return 1 (this IS a leaf). If only one child → return 1 + min_depth(that child). If both children → return 1 + min(min_depth(left), min_depth(right)). The asymmetric case (one child missing) breaks the symmetry with max-depth.”
Follow-up 4: “Now check if the tree is height-balanced (LC 110).”
Signal sought: combine depth check with structural invariant.
Full answer: “Naive: compute depth at every node, check |depth(left) - depth(right)| <= 1. O(N²) (depth is O(N), called for each of N nodes). Better: a single post-order pass returns either the depth OR a sentinel (-1) meaning ‘unbalanced’. As soon as any subtree returns -1, propagate -1 up. O(N) time. The trick: the height computation IS the balance check; we just augment to early-exit.”
Follow-up 5 (Staff signal): “Now compute the diameter (LC 543, our p14).”
Full answer: “Two-channel pattern. The recursive function returns the HEIGHT of the subtree (1 + max child height). As a side effect, at each node it computes left_height + right_height + 1 (or +0 if measuring edges) and updates a global best_diameter. This is the most important tree trick: function returns one thing while a closure-captured variable accumulates another. The diameter passing through node N is left_height + right_height; the function returns height for the parent to use. O(N) time, O(H) space.”
14. Full Solution Walkthrough
See solution.py. Six sections:
TreeNodewith__slots__.max_depth_recursive(node)— 3-line classic.max_depth_bfs(node)— BFS with level-size snapshot.max_depth_dfs_iter(node)— DFS stack with(node, depth)pairs.stress_test()— 1000 random trees; recursive, BFS, DFS must agree.edge_cases()— empty, single, left-skewed, right-skewed, perfect.
Decisions justified:
- BFS uses
len(queue)snapshot — explicit comment about why. - DFS pushes None-checking on the consumer side (cleaner code; no harm appending None and continuing).
- All three variants tested for equality on random trees; the stress test is the contract.
15. Beyond the Problem — Production Reality
Where this template shows up:
- CI/CD pipeline DAG depth — “what’s the critical path of this build?” Same recursion, on a DAG (need memoization).
- Filesystem directory depth —
find . -type d -printwalks the tree; computing max depth is the BFS-level version. - HTML DOM depth — browser engines bound nested element depth (Chrome caps at ~1000) to prevent stack overflow during render. They use iterative BFS internally.
- Compiler AST depth checks — many compilers reject programs with AST nesting > N to prevent stack overflow during semantic analysis.
- Game tree analysis — minimax depth bounds; iterative-deepening DFS uses bounded depth as the search horizon.
Production caveats:
- Skewed trees from sorted input: building a BST from sorted data creates a depth-N tree. Always rebalance (AVL, red-black) or use sorted-array-to-BST construction.
- Recursion limits: Python defaults to 1000, JVM ~5000-10000 depending on stack size, Go grows the stack as needed, Rust crashes hard. Always iterative for user-supplied tree depth.
- Concurrent depth queries: if many threads measure depth on the same tree, iterative is cache-friendlier (single contiguous deque) than recursive (scattered stack frames).
Principal-engineer code review comment: “Recursive max-depth is fine for our config tree (max depth 5). NOT fine for our user-uploaded XML/JSON parser — we’ve seen 100k-deep adversarial inputs designed to cause stack overflow (a known DoS vector; CVE-2022-XXXX in libxml). All user-data tree walks are iterative. Bonus: when measuring max depth on potentially adversarial input, add an early-exit cap (if depth > MAX_ALLOWED: raise) to fail fast on attack inputs rather than walking the entire tree first.”
p12 Max Depth — Progressive Hints
Rule: ONE hint at a time, 5-min timer between.
Hint 1 — Pattern
What does “depth of a tree” mean in terms of the depths of its left and right subtrees? Write the relation in plain English.
Hint 2 — Base case
What’s the depth of an empty tree? (Hint: 0.) What’s the depth of a single-node tree? (1 — the root counts.)
Hint 3 — The recurrence
depth(node) = 1 + max(depth(left), depth(right))
The +1 counts the current node; we add it to the deeper of the two subtrees.
Hint 4 — Pseudocode
Recursive:
def depth(node):
if node is None: return 0
return 1 + max(depth(node.left), depth(node.right))
BFS (level-by-level):
queue = deque([root]); depth = 0
while queue:
depth += 1
for _ in range(len(queue)): # snapshot level size
cur = queue.popleft()
if cur.left: queue.append(cur.left)
if cur.right: queue.append(cur.right)
return depth
Hint 5 — Full + traps
Time O(N), space O(H) (recursive) or O(W) (BFS).
Trap 1 — placing +1 wrong: max(1 + ..., 1 + ...) works (each side adds its own root); 1 + max(...) is cleaner. Both correct; the latter is idiomatic.
Trap 2 — BFS without len(queue) snapshot: if you increment depth on every pop, you count NODES not LEVELS. The snapshot pins the level boundary.
Trap 3 — DFS with stack: must carry the depth as (node, depth) pair; len(stack) is unrelated to path length.
Trap 4 — Don’t transplant to LC 111 (min depth): the symmetry breaks for nodes with one missing child. A node with left=None, right=non-empty is NOT a leaf — its depth-to-nearest-leaf goes through the right subtree, not 0.
Why this matters beyond this problem: the recurrence f(node) = combine(node.val, f(left), f(right)) is the template for EVERY “compute an integer/property of a tree” problem. Internalize the template.
p13 — Same Tree
Source: LeetCode 100 · Easy · Topics: Tree, DFS, BFS, Binary Tree Companies (2024–2025 frequency): Amazon (high), Microsoft (high), Meta (high — almost always chained with LC 101/572), Bloomberg (medium), Google (medium), Apple (medium) Loop position: phone screen warmup; the canonical “function takes two trees” pattern.
1. Quick Context
This is the “parallel-walk-two-trees” template. The recursion takes TWO node arguments and walks both trees in lockstep. The base cases are subtle: both None → True; one None and other not → False; both non-None → compare values AND recurse on both pairs. This pattern generalizes to LC 101 (Symmetric — walk one tree against its own mirror), LC 572 (Subtree — same algorithm at every starting point), LC 951 (Flip-Equivalent — recurse with EITHER straight OR swapped child pairing), LC 1367 (Linked List in Binary Tree), and the whole tree-comparison family.
Mid candidates conflate the two None cases or miss the value check. Senior candidates write it cold with all four cases handled. Staff candidates note that this IS the template for LC 101 and LC 572, and offer the iterative version using paired-stack/queue.
What it looks like it tests: compare two trees. What it actually tests: can you reason about TWO recursion arguments simultaneously? Do you enumerate all base cases before coding?
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/same-tree/
STOP. 5-min timer. Code recursive AND iterative.
3. Prerequisite Concepts
- TreeNode + recursion fundamentals
- Boolean-returning recursion: True/False propagation rules
- Paired iteration (both queues advance together)
- Short-circuit evaluation:
a and b and cstops at first False
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given roots of two binary trees, return True iff they are structurally identical AND every corresponding node has the same value.”
Step 2 — Clarify:
- “Both empty trees → True?” (Yes — vacuously identical.)
- “One empty, one not → False?” (Yes — different structure.)
- “Same structure, different values → False?” (Yes.)
- “Are nodes considered same if they’re the same object reference?” (Irrelevant — we compare structurally; same-reference would trivially be True.)
- “Max depth?” (LC says up to 100; production may be deeper.)
Step 3 — Constraints: N up to 100 per LC. O(N) time required (must compare every node in worst case where trees ARE equal). Recursion depth O(H).
Step 4 — Examples:
[1,2,3]vs[1,2,3]→ True.[1,2]vs[1,null,2]→ False (structure differs).[1,2,1]vs[1,1,2]→ False (values differ at children).- Both
[]→ True. [1]vs[]→ False.- Same large random tree against deep-copy of itself → True.
Step 5 — Brute Force: Serialize both trees (any unambiguous traversal — e.g., level-order with None markers) and compare strings. O(N) time, O(N) space. Works but allocates extra strings/lists. Acceptable; recursive parallel walk is cleaner.
Step 6 — Complexity: O(min(N₁, N₂)) time when trees differ early (short-circuit); O(N) when equal. Space O(H) recursive or O(W) iterative.
Step 7 — Pattern Recognition: “Compare two trees in lockstep” → parallel recursion with two arguments. Iterative: paired queue (BFS) or paired stack (DFS).
Step 8 — Optimize: Already optimal. The “optimization” is early-exit via short-circuit (Python’s and does this for free).
Step 9 — Prove correctness: Induction on combined tree size. Base cases: (None, None) → True (both vacuous); (None, X) or (X, None) → False (different); both non-None with different values → False. Inductive case: both non-None, same value; trees are equal iff left subtrees are equal AND right subtrees are equal (each by hypothesis). By construction, the recurrence returns True iff both subtree comparisons return True AND values match.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why It Works
The four-base-case discipline: every “compare two trees” recursion must explicitly handle:
- Both None → True (vacuous match)
- p None, q not → False
- p not, q None → False
- Both non-None → compare values; if equal, recurse on both pairs
Conflating cases 2 and 3 into a single “one is None” check works (if p is None or q is None: return p is q) but is harder to read. Spelling all four explicitly is the Senior signal.
Why p is q for the “one-is-None” case: if both are None, the function returns True via case 1. If only one is None, p is q is False (None is not the same object as a TreeNode). So if p is None or q is None: return p is q correctly handles cases 1, 2, 3 in one line. Elegant but requires care.
Short-circuit and and: Python evaluates A and B and C left-to-right and stops at the first False. So node.val == other.val and is_same(left_pair) and is_same(right_pair) short-circuits on value mismatch (no wasted recursion). Production-relevant: comparison cost can be expensive (e.g., deep nested objects); ordering checks cheapest-first is a real optimization.
Iterative parallel walk:
q = deque([(p, q_root)])
while q:
a, b = q.popleft()
if a is None and b is None: continue
if a is None or b is None or a.val != b.val: return False
q.append((a.left, b.left))
q.append((a.right, b.right))
return True
The queue holds PAIRS of nodes to compare. BFS is the natural form, but DFS-stack works identically (use .pop() instead of .popleft()). For early termination, DFS may finish faster on some inputs (depth-first hits a mismatch sooner if mismatch is deep on the first explored path).
Serialize-and-compare alternative: serialize(p) == serialize(q). O(N) time, O(N) space. Works but allocates the full serialization for both trees even when mismatch is at the root. Pure recursion short-circuits in O(1) on such inputs. State the trade-off.
Hash-and-compare (sketchy but tempting): hash(serialize(p)) == hash(serialize(q)). Same cost as serialize, plus risk of hash collision (extremely small for reasonable hash functions but not zero). Don’t recommend.
Connection to mirror-walk (LC 101, p15): Symmetric Tree = “compare a tree against itself with cross-recursion.” Same algorithm; only the recursion pairing changes — is_mirror(a, b) = a.val == b.val AND is_mirror(a.left, b.right) AND is_mirror(a.right, b.left). The cross (a.left ↔ b.right) is the mirror operator.
Connection to subtree (LC 572): “Is t a subtree of s?” = “Does t match s at any node?” = is_same(s, t) OR is_subtree(s.left, t) OR is_subtree(s.right, t). The recursion uses is_same as a building block.
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — Forgetting one of the null cases:
def is_same(p, q):
if p is None and q is None: return True
return p.val == q.val and is_same(p.left, q.left) and is_same(p.right, q.right)
Crashes on p=None, q=Node(1) — accesses p.val on None.
Wrong-but-tempting #2 — Comparing only structure or only values:
return is_same(p.left, q.left) and is_same(p.right, q.right)
(Missing the p.val == q.val check.) Trees with same structure but different values incorrectly compared True.
Wrong-but-tempting #3 — p == q for the value compare on TreeNodes:
return p == q and is_same(...) # WRONG
Without a custom __eq__, p == q defaults to p is q — identity, not value equality. Always p.val == q.val.
Wrong-but-tempting #4 — Serialize-and-compare with str concat in a loop:
def serialize(node):
if node is None: return "N"
return str(node.val) + serialize(node.left) + serialize(node.right)
Quadratic-time string concatenation (each + allocates new string). Use "".join([...]) or a list-then-join. But still O(N) space allocation — pure recursion is preferred.
Wrong-but-tempting #5 — Iterative version with two separate queues:
qa, qb = deque([p]), deque([q])
while qa and qb:
a, b = qa.popleft(), qb.popleft()
if a.val != b.val: return False
qa.append(a.left); qa.append(a.right)
qb.append(b.left); qb.append(b.right)
return not qa and not qb
Works but conflates None handling — if a.left=None and b.left=Node, both get appended, then we deref None on next iter. Use a SINGLE queue of pairs; cleaner and harder to get wrong.
8. Skills & Takeaways
Generalizable patterns:
- Parallel-walk-two-trees recursion — two arguments, four base cases.
- Boolean-AND recursion with short-circuit — order checks cheap-to-expensive.
- Paired queue/stack for iterative parallel walks.
Analogous problems:
- LC 101 — Symmetric Tree (p15): same algorithm, cross-recursion.
- LC 572 — Subtree of Another Tree: uses is_same as a sub-routine.
- LC 951 — Flip-Equivalent: recurse with EITHER straight OR swapped pairing.
- LC 1367 — Linked List in Binary Tree: parallel-walk a linked list and a tree.
- LC 617 — Merge Two Binary Trees: parallel-walk + construct new tree from pairs.
When NOT to use this: if you only need a hash for fast equality check (caching, deduplication), serialize-and-hash is fine and reusable. For one-off equality, the recursion is cleaner.
9. When to Move On
- Recursive in <3 min with all four base cases handled explicitly.
- Iterative with paired queue in <5 min.
-
I can rewrite the four cases as a one-liner using
p is qand explain why it’s correct. -
My
stress_test()includes “same tree”, “different by structure”, “different by value”, deep-copy equality. - I solved LC 572 (Subtree) and recognized is_same as a building block.
- I can sketch LC 101 (Symmetric) from this template by changing pairing.
10. Company Context
Meta (almost always asked as part of chain LC 100 → 101 → 572)
- The framing: “Are these two trees identical?” → “Now check symmetric.” → “Now check subtree.”
- What they want to hear: Recognize the template across all three. Pivot fluidly. Bonus for noting that LC 572 hashing variant (Merkle-tree-style) is O(N+M) vs naive O(N·M).
Amazon
- The framing: Often “are two version trees of an order identical?”
- Adversarial extension: “Now ignore certain fields (status, timestamp) in comparison.” (Custom value comparator.)
- What they want to hear: Clean base cases; abstracted value comparator function.
Microsoft / Apple / Bloomberg
- Standard problem. Clean code, all base cases, iterative on request.
- Adversarial extension: “Implement using a Merkle hash so equality check is O(1) after preprocessing.” (Each node hashes its subtree; equality is hash compare. Trade-off: collision risk; need strong hash.)
11. Interviewer’s Lens
| Phase | Strong | Weak | Scorecard (strong) |
|---|---|---|---|
| Reading | Clarifies “empty? structure? values?” | Dives | “Enumerates equality conditions” |
| Pre-coding | Lists four base cases aloud | Starts typing | “Disciplined base-case enumeration” |
| Coding | All four cases handled; short-circuits on val mismatch | Crashes on one-None case | “Reflex parallel recursion” |
| Iterative | Single queue of pairs in <5 min | Two separate queues, struggles | “Knows paired-iter pattern” |
| Family | Names LC 101, 572 from same template | Sees only this problem | “Sees the template — Staff” |
12. Level Delta
| Level | Looks like |
|---|---|
| Mid | Recursive works after debugging the one-None case once. ~6 min. |
| Senior | All four cases explicit; iterative paired-queue on request. ~5 min. |
| Staff | All Senior + names LC 101/572/951 family + offers Merkle-hash variant for the O(1)-after-preprocessing follow-up. ~5 min. |
| Principal | All Staff + production framing (config tree comparison, diff algorithms, Git’s tree object hash comparison — which IS Merkle), discusses when Merkle wins (repeated comparisons across many trees, e.g., distributed sync). ~5 min. |
13. Follow-up Questions
Follow-up 1: “Iteratively.”
“Single queue of (a, b) pairs. Pop, check both-None (continue), one-None (False), val-mismatch (False); else enqueue (a.left, b.left) and (a.right, b.right). O(N), O(W).”
Follow-up 2: “Now check Symmetric Tree (LC 101).”
“Same algorithm, cross-recursion: is_mirror(a, b) = a.val == b.val AND is_mirror(a.left, b.right) AND is_mirror(a.right, b.left). Call as is_mirror(root.left, root.right). The cross-pairing IS the mirror.”
Follow-up 3: “Is t a subtree of s? (LC 572)”
“is_subtree(s, t) = is_same(s, t) OR is_subtree(s.left, t) OR is_subtree(s.right, t). O(N·M) worst case. Better: Merkle-hash both; subtree check becomes ‘does t’s hash appear among s’s subtree hashes?’ — O(N + M) after preprocessing.”
Follow-up 4: “Ignore values; only compare structure.”
“Drop the value check; keep the four base cases. Now same_structure(p, q) = (p is None) == (q is None) AND same_structure(p.left, q.left) AND same_structure(p.right, q.right). O(min(N,M)) short-circuit.”
Follow-up 5 (Principal signal): “How would Git’s tree-object equality work?”
“Each Git tree object is a list of (mode, name, blob/tree SHA) entries. Two trees are equal iff their SHA-1 hashes match. The hash is computed bottom-up: leaves (blobs) hash their content; trees hash their entry list (which includes children’s hashes). This is a Merkle tree: equality is O(1) after preprocessing, and partial-equality detection (which subtree changed) is O(log N) for balanced trees. The cost: O(N) preprocessing per tree. Worth it when comparisons are repeated (Git pull/push compare many trees against many trees).”
14. Full Solution Walkthrough
See solution.py. Five sections:
- TreeNode + helpers (build_tree, random_tree, clone_tree)
is_same_recursive— explicit four-case handlingis_same_iterative— single queue of pairsstress_test— 1000 trees: (a) identical clones expect True; (b) mutate one node and expect Falseedge_cases— empty/empty, empty/single, val-mismatch, structure-mismatch
15. Beyond the Problem — Production Reality
Real applications:
- Git tree-object diffing — Merkle-hash-based.
- React virtual DOM reconciliation — pairs old and new VDOM trees, walks in parallel, applies minimal patches.
- Distributed config sync (etcd, Consul) — compare-and-swap on config trees; equality check on snapshots.
- AST equivalence checking in compilers (e.g., is this expression CSE-equivalent to that one?).
- Database schema diff tools (Liquibase, Flyway) — compare expected vs actual schema trees.
Production caveat — value equality may be expensive: if tree node values are deeply nested objects (e.g., JSON), value comparison itself is recursive. Ordering checks cheap-to-expensive (identity-then-hash-then-deep-equal) is the production pattern.
Principal-engineer code review comment: “Our config-tree equality check was O(N·D) where D = depth-of-each-value’s-nested-JSON. We were comparing 10MB configs across 100 services in a healthcheck loop = 1 GB/s comparison cost. Replaced with Merkle hashing at write time: equality check is now O(1) hash compare; rebuild hash only on write. Throughput went 1000×. Lesson: if you compare the same trees repeatedly, hash them once.”
p13 Same Tree — Progressive Hints
Rule: ONE hint at a time, 5-min timer.
Hint 1 — Pattern
The function takes TWO trees. Think about the recursive shape: at any pair (p, q), what must be true for the subtrees rooted there to be equal?
Hint 2 — Enumerate base cases
Write down ALL cases before coding:
pNone ANDqNone → ?pNone ANDqnot None → ?pnot None ANDqNone → ?- Both non-None → ?
(Answers: True, False, False, recurse.)
Hint 3 — Recurrence for case 4
When both non-None, what conditions must hold? (a) values match; (b) left subtrees match; (c) right subtrees match. All three. Use and.
Hint 4 — Pseudocode
def is_same(p, q):
if p is None and q is None: return True
if p is None or q is None: return False
return p.val == q.val \
and is_same(p.left, q.left) \
and is_same(p.right, q.right)
Iterative (BFS): single queue of PAIRS.
queue = deque([(p, q)])
while queue:
a, b = queue.popleft()
if a is None and b is None: continue
if a is None or b is None or a.val != b.val: return False
queue.append((a.left, b.left))
queue.append((a.right, b.right))
return True
Hint 5 — Full + traps
Time O(N), space O(H) recursive or O(W) iterative.
Trap 1 — Skipping a None case: missing case 2 or 3 crashes on p.val when p is None. Enumerate all four EXPLICITLY.
Trap 2 — Using p == q: without __eq__, this is identity check; not what you want. Always p.val == q.val.
Trap 3 — Iterative with two separate queues: they desync when one tree has None where the other has a node. Use a SINGLE queue of pairs.
Trap 4 — Returning early on val match: the and chain handles short-circuit; don’t write if p.val != q.val: return False followed by separate recursion calls — same logic, more lines.
Family preview:
- LC 101 Symmetric Tree (p15) = same template, cross-recurse
is_mirror(a.left, b.right) AND is_mirror(a.right, b.left). - LC 572 Subtree =
is_same(s, t) OR is_subtree(s.left, t) OR is_subtree(s.right, t).
Internalize the parallel-walk template.
p14 — Diameter of Binary Tree
Source: LeetCode 543 · Easy (lies — this is a Medium in disguise) · Topics: Tree, DFS Companies (2024–2025 frequency): Meta (very high — flagship), Amazon (high), Microsoft (high), Google (medium), Bloomberg (medium), Apple (medium) Loop position: mid-loop algorithmic round; the “Easy” label is misleading — most candidates fail this on first try.
1. Quick Context
This is THE most important trick in the entire tree chapter. The technique is called the two-channel pattern (or “return-one-thing-while-tracking-another”). The recursive function returns the HEIGHT of a subtree. As a side effect, at each node, it computes a candidate answer (left_height + right_height) and updates a closure-captured best if larger. The parent doesn’t need the candidate — it needs the height to continue the recursion. The candidate just passes through.
Once you internalize this, you have unlocked:
- LC 543 (this) — diameter (longest path of edges).
- LC 124 — Max Path Sum (Hard; the same pattern; return-max-gain, accumulate global-best including BOTH-sides-at-node).
- LC 687 — Longest Univalue Path (same pattern; gated by value equality).
- LC 1372 — Longest ZigZag (same pattern; track two heights per node — left-going and right-going).
- LC 1973 — Count Nodes Equal to Sum of Descendants (return sum, accumulate count).
- LC 1325 — Delete Leaves with Given Value (return new node, with side-effect deletion).
It is THE template for “find a global optimum over all paths/structures within a tree.”
What it looks like it tests: find the longest path between any two nodes. What it actually tests: can you write a recursion that returns ONE thing (height for parent’s use) while accumulating ANOTHER (best diameter as side effect)? Most candidates incorrectly try to make the recursion return the diameter.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/diameter-of-binary-tree/
STOP. 7-min timer (this one is genuinely harder than its “Easy” label).
3. Prerequisite Concepts
- p12 — Max Depth (you already write height in your sleep)
- Recursion with side-effects via
nonlocalor a list-of-one trick - The mental shift: “the value the function RETURNS is different from the value being COMPUTED.”
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given the root of a binary tree, return the longest path between any two nodes, measured in edges. The path may or may not pass through the root.”
Step 2 — Clarify:
- “Edges or nodes?” (Edges per LC. Single-node tree → diameter 0. Two-node tree → diameter 1.)
- “Path can curve through any node, not just root?” (Yes — that’s the whole point.)
- “Single node — diameter 0?” (Yes, edges = 0.)
- “Empty tree — what?” (LC won’t pass empty; defensively return 0.)
Step 3 — Constraints: N up to 10^4. O(N) achievable; the wrong approach is O(N²).
Step 4 — Examples:
[1,2,3,4,5]→ 3 (path 4→2→1→3 or 5→2→1→3).[1,2]→ 1.[1]→ 0.- Long left-skewed chain → length-1 = N-1.
- A “T-shape” where the long path goes through a non-root node.
Step 5 — Brute Force: For every node, compute height(left) + height(right), track max. Height itself is O(N). Total O(N²). State this then optimize.
Step 6 — Complexity (optimal): O(N) time (each node visited once), O(H) stack space.
Step 7 — Pattern Recognition: “Find a global maximum over all subtree-rooted paths” → two-channel post-order (return one quantity, accumulate another).
Step 8 — Optimize from O(N²) to O(N): the insight is that we can compute heights bottom-up in a SINGLE pass and simultaneously check the candidate-diameter at each node. Each node’s candidate-diameter is height(left) + height(right) in EDGES. Capture this side-effect into a closure variable.
Step 9 — Prove correctness: Every path between two nodes in a tree has a unique HIGHEST node (the LCA — lowest-common-ancestor of the endpoints, which is the topmost node on the path). At that node, the path consists of edges going down through the left subtree to one endpoint AND edges going down through the right subtree to the other endpoint. The number of such edges is exactly height(left) + height(right) (where height = max edges from node to any descendant leaf, with leaf height = 0). So if we check left_height + right_height at EVERY node and take the maximum, we examine every possible path’s candidate exactly once at its LCA. Correct.
5. Progressive Hints
If stuck for 7 min, open hints.md. This problem is famously where candidates need hints — don’t feel bad.
6. Deeper Insight — The Two-Channel Pattern
The mental shift. Beginners try: “make diameter(node) return the diameter of the subtree rooted at node.” This is HARD because the diameter of a subtree could pass through the node OR be entirely in the left subtree OR entirely in the right subtree — and you also need the height to combine with the parent. Two-channel solves this by separating concerns:
def height(node):
if node is None:
return -1 # edge convention: height of empty = -1; height of leaf = 0
lh = height(node.left)
rh = height(node.right)
# CHANNEL A (side effect): candidate diameter at THIS node = lh + rh + 2 edges
# (the +2 accounts for the edge from node to each subtree root)
nonlocal best
best = max(best, lh + rh + 2)
# CHANNEL B (return): height for parent's use
return 1 + max(lh, rh)
Or, the more common edge-convention: height of empty = 0, height of leaf = 1, and diameter-at-node = lh + rh (the +2 is absorbed):
def height(node):
if node is None: return 0
lh = height(node.left); rh = height(node.right)
nonlocal best
best = max(best, lh + rh) # diameter through this node, in edges
return 1 + max(lh, rh)
This second form is cleaner and what we’ll use. The intuition: height here counts nodes from this node down to deepest leaf (so leaf → 1). Then lh + rh measures edges because each side contributes its node-count, and edges-between-them is one less than the total node count along the path, BUT we don’t include the LCA itself, hence lh + rh directly equals edge count.
Wait, let me re-derive carefully because this is exactly where candidates trip:
- Let
h(node) = node-count from node to deepest leaf in subtree, inclusive(so leaf is 1, empty is 0). - For a node with subtrees of node-heights
lhandrh:- Path through node:
(lh nodes on left path) + (rh nodes on right path) + 1 for the node itself=lh + rh + 1NODES. - But we want EDGES, which is
nodes - 1=lh + rh + 1 - 1=lh + rhedges. ✓
- Path through node:
So best = max(best, lh + rh) is correct for edge-count diameter.
Why nonlocal best not return-the-tuple? You can return a tuple (height, best_so_far), but each parent must MERGE the two children’s bests — best = max(left_best, right_best, this_diameter). Works, more verbose. The nonlocal (or list-of-one) trick collapses the merge into “just always write to the shared variable.” Either is fine; demonstrate awareness of both.
Why this generalizes to LC 124 (Max Path Sum, Hard): SAME structure. Return type: max-gain from this node going DOWN (always ≥ 0, since we can choose not to include a negative subtree). Side effect: best path-sum at this node = node.val + max(0, left_gain) + max(0, right_gain) (we allow both sides because the path bends here). Critical: the RETURN must NOT include both sides (because the parent can only use a single-direction descent through this node; the bent path terminates here). The asymmetry between “what to return” and “what to accumulate” IS the trick.
Why LC 687 (Longest Univalue Path): SAME structure. Return: longest single-direction same-value chain from this node down. Side effect: longest both-sides through this node. Gate each side’s contribution on node.left.val == node.val.
Why LC 1372 (Longest ZigZag): SAME structure but return TWO heights per call: (zig_left, zig_right) — longest zigzag going first-left vs first-right. Update best with max of these. The output is max(best - 1) adjusted.
Two-channel is the single most reusable tree technique. Internalize it now.
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — Computing height at each node (O(N²)):
def diameter(node):
if node is None: return 0
here = height(node.left) + height(node.right)
return max(here, diameter(node.left), diameter(node.right))
Each height call is O(N); called at N nodes = O(N²). Works on small inputs, TLE on large. The two-channel pattern is the fix.
Wrong-but-tempting #2 — Returning diameter from the recursion:
def diameter(node):
if node is None: return 0
return max(diameter(node.left), diameter(node.right), 1 + ??? )
You can’t combine — the parent doesn’t know the children’s heights anymore. Either (a) return BOTH (height, best) as tuple, or (b) capture best as side effect. Plain return doesn’t work.
Wrong-but-tempting #3 — Confusing nodes vs edges:
LC counts EDGES. Diameter of [1, 2] is 1 (one edge), not 2 (two nodes). Many candidates return node-count and get off-by-one. Always ask “edges or nodes?” upfront.
Wrong-but-tempting #4 — Updating best based on returned value instead of using children’s heights:
def height(node):
if node is None: return 0
h = 1 + max(height(node.left), height(node.right))
nonlocal best
best = max(best, h) # WRONG: this measures depth, not diameter
return h
This computes max depth (which we already did in p12), not diameter. The diameter side-effect MUST be lh + rh, not h.
Wrong-but-tempting #5 — Forgetting that path may not pass through root:
def diameterOfBinaryTree(root):
return height(root.left) + height(root.right)
Wrong: the longest path might be entirely within the left subtree. Must check at EVERY node, not just root. (This is also why we use the two-channel pattern.)
8. Skills & Takeaways
Generalizable patterns:
- Two-channel post-order — return one thing, accumulate another. THE most reusable tree trick.
- LCA-of-path observation — every path has a unique highest node; check candidates there.
nonlocalfor accumulation — closure-captured mutable state in recursion.
Analogous problems:
- LC 124 — Max Path Sum (Hard): return max-gain (≥0); accumulate
node.val + max(0,lg) + max(0,rg). - LC 687 — Longest Univalue Path: return single-direction same-value chain; accumulate both-sides.
- LC 1372 — Longest ZigZag Path: return two heights (left-first, right-first); accumulate both.
- LC 1373 — Max BST Sum: return (is_BST, min, max, sum); accumulate best sum if BST.
- LC 968 — Min Cameras to Cover Tree: return state (covered/needs-camera/has-camera); accumulate camera count.
When to NOT use two-channel: if you only need a single quantity (max depth, sum, count) — direct recursion suffices. Two-channel pays off when “what parent needs” differs from “what we’re solving for globally.”
9. When to Move On
- I can write the two-channel pattern from scratch in <5 min for diameter.
- I can articulate “function returns height; side-effect accumulates best” in one sentence.
-
I solved LC 124 (Max Path Sum) using the same template — and explained why
max(0, ...)clamps negative gains. - I solved LC 687 (Longest Univalue) using the same template.
-
My
stress_test()cross-checks against the O(N²) brute force. -
I default to
nonlocal best(or list-of-one) for the side-effect; can also write the tuple-return form on request. - I always ask “edges or nodes?” before coding.
10. Company Context
Meta (FLAGSHIP — expect this in onsite)
- The framing: “Diameter of binary tree” → immediately follows with LC 124 (Max Path Sum) as Hard extension.
- Adversarial extension: “Now return the actual two endpoint nodes, not just length.” (Carry endpoint nodes alongside height.) Or: “What if edges have weights?” (
best = lw + rwwherelw = left_height + edge_weight_to_left_child.) - What they want to hear: Recognize two-channel pattern by name; pivot to LC 124 in <5 min; handle weighted-edge extension fluently.
Amazon
- The framing: Often as “longest reporting chain in an org tree (not just to/from CEO).”
- What they want to hear: Same template; clear separation of “channel A” and “channel B.”
- Adversarial extension: “Implement on an N-ary tree.” (Instead of
lh + rh, find TOP TWO heights among all children; their sum is the candidate.) - What they want to hear: N-ary generalization in <3 min after recognizing top-two-instead-of-left-right.
Microsoft / Apple / Bloomberg
- Standard problem. Two-channel pattern; mention generalization to weighted edges or N-ary.
11. Interviewer’s Lens
| Phase | Strong | Weak | Scorecard (strong) |
|---|---|---|---|
| Reading | Asks “edges vs nodes? must pass through root?” | Assumes through root | “Disambiguates ambiguity” |
| Approach | States O(N²) brute, pivots to two-channel | Codes O(N²), submits | “Recognizes inefficiency” |
| Coding | Two-channel in <5 min with nonlocal | Tries to return diameter, struggles | “Reflex two-channel — STAFF signal” |
| Vocabulary | Names “two-channel pattern” / “post-order with side-effect” | Hand-waves | “Has the vocabulary” |
| LC 124 pivot | Solves in <5 min after diameter | Stuck on negative-value handling | “Sees the template — Staff” |
This problem is THE pivot point between Mid and Senior tree skills. Crisp two-channel = Senior. Naming the pattern + offering LC 124 unprompted = Staff.
12. Level Delta
| Level | Looks like |
|---|---|
| Mid | O(N²) brute first. Coaxed to two-channel. ~15-20 min total. |
| Senior | Recognizes O(N²) immediately, pivots to two-channel in <8 min total. Explains nonlocal cleanly. |
| Staff | Senior + names “two-channel” + offers LC 124 + N-ary generalization + weighted-edge unprompted. ~7 min total. |
| Principal | Staff + discusses production: critical-path analysis in build/CI systems, network-graph diameter for distributed-system latency analysis, generalization to graphs (BFS-from-arbitrary-then-BFS-from-farthest-found, a known graph-theory trick). |
13. Follow-up Questions
Follow-up 1: “What’s the time complexity of the obvious approach?”
“O(N²). At each of N nodes, compute height (O(N) each). LC TLE on large inputs.”
Follow-up 2: “Now max path sum (LC 124, Hard).”
“Same two-channel pattern. Return type: max gain from this node going DOWN one side, clamped to ≥ 0 (we can choose NOT to include a negative subtree, since path is allowed to end at this node). Side effect: best path-through-here = node.val + max(0, left_gain) + max(0, right_gain) — both sides allowed at this node because the path BENDS here and we won’t extend further upward through it. The RETURN must not include both sides (parent can only descend one way). O(N) time.”
Follow-up 3: “What if the tree is N-ary, not binary?”
“At each node with K children, candidate diameter = sum of the TOP TWO children’s heights. Use heapq.nlargest(2, [height(c) for c in children]) or a one-pass max1/max2 tracker. Return 1 + max(heights). O(N·K) for the heap or O(N) with one-pass.”
Follow-up 4: “What if edges have weights?”
“Replace ‘height = 1 + max(child_heights)’ with ‘height = max(child_height + edge_weight)’ for each child. Diameter = max(left_h + edge_to_left, right_h + edge_to_right) summed for both sides at each node. Same template, different arithmetic.”
Follow-up 5 (Principal signal): “Generalize to an arbitrary graph (not just tree).”
“For a tree (no cycles), the double-BFS trick is classic: (1) BFS from any node, find the farthest node A; (2) BFS from A, find the farthest node B; the distance A→B is the diameter. Provable for trees. For general graphs, diameter computation is much harder — O(N·M) BFS-from-every-node in unweighted, or O(N³) Floyd-Warshall in weighted; approximation algorithms exist for huge graphs (sampling-based). For binary trees specifically, the two-channel pattern is strictly better than double-BFS because we get O(N) with a single traversal.”
14. Full Solution Walkthrough
See solution.py. Five sections:
- TreeNode + helpers
diameter_brute— O(N²) reference: at every node, computeheight(left) + height(right), track maxdiameter_two_channel— O(N) optimal withnonlocal bestdiameter_two_channel_tuple— equivalent using returned-tuple(height, best), demonstrated for completenessstress_test— brute and both optimized variants must agree on 1000 random treesedge_cases— empty, single, two-node, three-node-T, deep-skewed, balanced
15. Beyond the Problem — Production Reality
Where two-channel shows up:
- Build-system critical-path analysis — find the longest dependency chain through a build DAG. Same template applied to DAG (with memoization).
- CI test-scheduling — find the longest serial-test chain to set the parallelism upper bound.
- Network topology analysis — diameter of a network is the worst-case hop count.
- Phylogenetic trees in computational biology — longest evolutionary path between species.
- Distributed-system latency analysis — diameter of the gossip network = worst-case message propagation time.
- Game-tree analysis — longest forced sequence.
Production caveats:
- Cycles in real graphs: if the tree assumption is broken (e.g., DAGs with shared subnodes), memoize. If true cycles exist (general graphs), use BFS-based algorithms.
- Memory: the recursion-stack size matters for very deep trees. Iterative two-channel is possible (post-order with stack of
(node, state)) but ugly; convert to iterative only if you’ve seen stack overflows. - Online updates: if the tree changes (insert/delete), diameter must be recomputed. There are advanced data structures (link-cut trees, top-trees) supporting O(log N) diameter updates — well beyond interview scope but worth naming.
Principal-engineer code review comment: “When I see two-channel recursion in code review I check three things: (1) is the side-effect variable correctly scoped (closure / nonlocal / instance attr)? (2) does the RETURN type match what callers need, distinct from the accumulated answer? (3) is the recursion stack bounded for our input? In our build-dependency code, dep chains can be 500-deep; we bumped Python’s recursion limit AND added a depth-guard to detect adversarial DAGs (>10k = bail with telemetry). The pattern is correct; the operational concerns are what separate working code from production code.”
p14 Diameter — Progressive Hints
Rule: ONE hint at a time. This problem is famously hard despite the “Easy” label — take your time.
Hint 1 — Observation about paths
Every path in a tree has a unique HIGHEST node (the bend point). For that highest node, the path descends into the LEFT subtree on one side and the RIGHT subtree on the other. What’s the length of that path in terms of subtree heights?
Hint 2 — Candidate at each node
At each node N, the longest path bending at N = height(left subtree of N) + height(right subtree of N) (measured in edges). The answer is the MAX of this candidate over all nodes.
Why all nodes? Because the answer path bends at SOME node — could be any. So check every one.
Hint 3 — Naive O(N²)
Compute height at each node separately. That’s height = O(N) called at N nodes = O(N²). Works on small inputs, TLE on large. Can we do it in one pass?
Hint 4 — The Two-Channel Pattern
We’re already computing all heights bottom-up during a single post-order traversal. So as we compute each node’s height, we can simultaneously check the candidate-diameter at that node and update a global best.
The recursive function’s purpose splits:
- Channel B (return value): height, used by the PARENT.
- Channel A (side effect): update
best = max(best, lh + rh)— the answer we actually want.
Pseudocode:
best = 0
def height(node):
if node is None: return 0
lh = height(node.left)
rh = height(node.right)
nonlocal best
best = max(best, lh + rh) # CHANNEL A
return 1 + max(lh, rh) # CHANNEL B
height(root)
return best
Hint 5 — Full + traps + family preview
Time O(N), space O(H).
Trap 1 — Returning diameter from the recursion: You can’t combine cleanly because the parent needs HEIGHT, not diameter. Either use nonlocal or return a tuple (height, best).
Trap 2 — Edges vs nodes: LC counts edges. lh + rh (where height counts nodes from this node down) gives edge-count of the bent path correctly because: lh + rh nodes outside the bend + 1 node at bend = lh+rh+1 nodes = lh+rh edges. ALWAYS ask “edges or nodes?” first.
Trap 3 — Only checking diameter through root: the path may bend at any node. The two-channel pattern checks at every node automatically — but if you write diameter(root) = height(root.left) + height(root.right) directly, you only check root. Wrong.
Trap 4 — Side-effect scoping bugs: if best is a local int and you write best = ... inside the inner function without nonlocal, you create a new local; outer best stays 0. Use nonlocal best (or wrap in a list best = [0] and assign best[0] = ...).
Family — the same template solves:
- LC 124 Max Path Sum (Hard): return
node.val + max(0, max(left_gain, right_gain))clamped ≥ 0; side-effectnode.val + max(0,lg) + max(0,rg). - LC 687 Longest Univalue Path: return single-side same-value chain; side-effect both sides.
- LC 1372 Longest ZigZag: return TWO heights per node.
- N-ary version: sum of TOP-TWO child heights (use
heapq.nlargest(2, ...)).
This is the most reusable trick in tree problems. Internalize it.
p15 — Symmetric Tree
Source: LeetCode 101 · Easy · Topics: Tree, DFS, BFS Companies (2024–2025 frequency): Meta (high — paired with LC 100), Amazon (high), Microsoft (high), Bloomberg (medium), Google (medium), Apple (medium) Loop position: phone screen; canonical “compare a tree against its mirror” — direct application of p13 with cross-pairing.
1. Quick Context
This is the mirror-walk generalization of p13 (Same Tree). Instead of comparing two separate trees with straight pairing (a.left ↔ b.left, a.right ↔ b.right), we compare a tree against itself with cross pairing (a.left ↔ b.right, a.right ↔ b.left). That single change in pairing IS the mirror operator.
Mastering this unlocks LC 951 (Flip-Equivalent — pair-OR-cross-pair allowed), LC 572 (Subtree — uses LC 100 as a sub-routine), LC 1367 (Linked-list-in-tree), and the entire “tree-shape-matching” family. It also reinforces the lesson from p13: the recursion structure stays the same; only the pairing changes.
Mid candidates write the recursion. Senior candidates write it cold AND the iterative BFS-pairs version. Staff candidates connect it to LC 951 (“now ALSO match if you cross-pair OR straight-pair at any node — the equivalence has a choice at each level”) and explain why that variant is harder (2^N branching without memoization, or O(N) with structural-hash memoization).
What it looks like it tests: is this tree a mirror of itself? What it actually tests: can you recognize that mirror-walk = parallel-walk with cross-pairing? Do you generalize patterns, or memorize problems?
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/symmetric-tree/
STOP. 5-min timer. Recursive AND iterative.
3. Prerequisite Concepts
- p13 — Same Tree (the four-case parallel-walk template)
- Mirror/reflection as a tree operation (
invertfrom p11) - Paired iteration with single queue
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given the root of a binary tree, return True iff the tree is a mirror image of itself — i.e., the left subtree is a mirror image of the right subtree (and recursively).”
Step 2 — Clarify:
- “Empty tree — symmetric?” (Yes, vacuously.)
- “Single node?” (Yes.)
- “Different values at mirror positions?” (Not symmetric.)
- “Structure must match too, right?” (Yes —
[1,2,null]and[1,null,2]would be symmetric;[1,2]alone isn’t.)
Step 3 — Constraints: N up to 1000 (LC). O(N) achievable.
Step 4 — Examples:
[1,2,2,3,4,4,3]→ True.[1,2,2,null,3,null,3]→ False (mirror positions differ structurally).[1]→ True.[1,2,2]→ True.[1,2,3]→ False (values differ at mirror positions).
Step 5 — Brute Force: Build the mirror image (use invert from p11), then compare to original with is_same (p13). O(N) time, O(N) space (constructed mirror). Works. The direct mirror-walk is the same time complexity but O(H) space.
Step 6 — Complexity: O(N) time, O(H) recursive or O(W) iterative.
Step 7 — Pattern Recognition: “Compare tree against its mirror” → parallel-walk with cross-pairing.
Step 8 — Optimize: The mirror-walk version skips the intermediate construction; same asymptotic but constant-factor better and cleaner.
Step 9 — Prove correctness: A tree T is symmetric iff for every “mirror-paired” node pair (a, b) (where a is at position p in T, and b is at the mirror-position of p), values match AND mirror-paired children match. Mirror pair (a.left, b.right) and (a.right, b.left). By induction on combined subtree size, the recurrence returns True iff all mirror-pairs match.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Same Algorithm, Cross-Pairing
The crucial reframe. Don’t think of LC 101 as a new problem. Think:
“LC 101 is LC 100 (Same Tree) applied to (
root.left,root.right) with the recursion’s RECURSIVE PAIRING changed from straight to cross.”
Same Tree:
def is_same(a, b):
if a is None and b is None: return True
if a is None or b is None: return False
return a.val == b.val \
and is_same(a.left, b.left) \
and is_same(a.right, b.right)
Mirror (Symmetric helper):
def is_mirror(a, b):
if a is None and b is None: return True
if a is None or b is None: return False
return a.val == b.val \
and is_mirror(a.left, b.right) \ # CROSS
and is_mirror(a.right, b.left) # CROSS
def is_symmetric(root):
return root is None or is_mirror(root.left, root.right)
Three lines change: pairing the children diagonally. THAT is the mirror operator expressed as recursion structure.
Why this generalizes — LC 951 (Flip-Equivalent): “Two trees are flip-equivalent if we can turn one into the other by flipping (swapping left↔right) any subset of nodes.” Recurrence:
def flip_equiv(a, b):
if a is None and b is None: return True
if a is None or b is None or a.val != b.val: return False
# At each node we can CHOOSE: match straight OR cross-pair.
return (flip_equiv(a.left, b.left) and flip_equiv(a.right, b.right)) \
or (flip_equiv(a.left, b.right) and flip_equiv(a.right, b.left))
The OR introduces branching — naive is 2^N. With structural-hash canonicalization (recursively canonicalize each subtree to a hash with a deterministic ordering), it’s O(N). Recognize that LC 951’s exponential blow-up comes from the OR; mention canonicalization as the fix.
Iterative form: single queue of pairs (a, b) initialized with (root.left, root.right). Pop, check both-None/one-None/val-mismatch, then enqueue (a.left, b.right) and (a.right, b.left). Same as p13 with cross-pairing.
Why the iterative version matters for interviews: if asked “iteratively” mid-flow, a Senior should write it in <5 min without restarting from scratch — just copy the parallel-walk-iterative from memory and apply cross-pairing.
The mirror-as-relation insight: “Symmetric tree” is the FIXED POINT of the invert operation: invert(T) == T. From p11 we know invert(invert(T)) == T is the involution. From LC 101 we now know invert(T) == T iff T is symmetric. So we have two ways to test symmetry:
- Direct: mirror-walk (this problem).
- Indirect: compute
invert(T)(p11), compare withis_same(T, invert(T))(p13). Same complexity, uses two earlier primitives.
State both; the direct is cleaner.
7. Anti-Pattern Analysis
Wrong-but-tempting #1 — Comparing tree to itself with straight pairing:
def is_symmetric(root):
return is_same(root, root)
Always True. Symmetric means mirror, not identical-to-itself. Trivially wrong but a real first attempt.
Wrong-but-tempting #2 — Forgetting to cross:
def is_mirror(a, b):
if a is None and b is None: return True
if a is None or b is None: return False
return a.val == b.val and is_mirror(a.left, b.left) and is_mirror(a.right, b.right)
This is is_same(a, b) — checks if left and right subtrees are IDENTICAL, not mirrors. [1, 2, 2, 3, 4, 4, 3] would fail (left subtree [2,3,4], right subtree [2,4,3] — not identical but ARE mirrors).
Wrong-but-tempting #3 — Reverse only one level:
def is_symmetric(root):
if root is None: return True
return root.left.val == root.right.val
Only checks top-level values. Misses deeper asymmetries.
Wrong-but-tempting #4 — Iterative with naive two-queue desync (same as p13 mistake): Two separate queues lose track of None alignment. Use ONE queue of pairs.
Wrong-but-tempting #5 — Conflating “structurally symmetric” with “all values equal”:
def is_symmetric(root):
# gather all values; check palindrome
Tree shape isn’t a linear sequence; flattening loses structural info. Don’t try clever sequence tricks.
8. Skills & Takeaways
Generalizable patterns:
- Cross-pairing recursion — same skeleton as parallel-walk, different pair argument.
- Reuse-by-reframing — recognize new problems as existing patterns with a twist.
- Fixed-point insight — symmetric = invert(T) == T.
Analogous problems:
- LC 100 — Same Tree (p13): straight-pairing.
- LC 951 — Flip-Equivalent: OR-of-(straight, cross).
- LC 572 — Subtree of Another Tree.
- LC 226 — Invert Binary Tree (p11): the operator we’re testing fixed-point of.
- LC 110 — Balanced Binary Tree: structural property checked recursively.
When NOT to use mirror-walk: if you’ve already inverted the tree for another reason and have the inverted version handy, is_same(T, inverted_T) is simpler.
9. When to Move On
-
I write
is_mirrorin <3 min, with cross-pairing reflex. - I write iterative version (single queue of pairs) in <5 min.
-
I articulate “symmetric =
invert(T) == T” and show two solution paths. -
My
stress_test()randomly mirrors a tree (build random + clone + invert) and asserts symmetric. - I sketched LC 951 (Flip-Equivalent) and identified the 2^N blow-up + canonicalization fix.
10. Company Context
Meta (chained with LC 100)
- The framing: “Are these two trees identical?” → “Now check symmetric.”
- What they want to hear: “Same algorithm, cross-pair the children.” Speed of pivot signals pattern-recognition.
Amazon
- The framing: Sometimes “balanced UI layout: check that this binary widget tree is left-right symmetric.”
- What they want to hear: Direct mirror-walk + iterative variant.
Microsoft / Apple / Bloomberg / Google
- Standard problem. Clean recursion + iterative on request.
- Google adversarial: “Now check Flip-Equivalent (LC 951). What’s the time complexity?” (2^N naive; O(N) with canonicalization.)
11. Interviewer’s Lens
| Phase | Strong | Weak | Scorecard (strong) |
|---|---|---|---|
| Reading | “Empty? Single? Just values or structure too?” | Dives | “Clarifies symmetry definition” |
| Approach | “Same as LC 100 with cross-pair.” | “Hmm, let me think…” | “Recognizes pattern reuse — Senior” |
| Coding | All four base cases + cross-pair in <3 min | Forgets cross OR misses null case | “Reflex mirror-walk” |
| Iterative | Single queue of pairs in <5 min | Restarts thinking | “Knows iterative parallel pattern” |
| LC 951 pivot | Names OR-branching + canonicalization | Confused by “flip” | “Sees the family — Staff” |
12. Level Delta
| Level | Looks like |
|---|---|
| Mid | Recursive after one debug. ~6 min. |
| Senior | Recursive cold + iterative on request. Names “same as LC 100 with cross-pair.” ~5 min. |
| Staff | Senior + LC 951 sketch in <5 min + canonicalization mention. |
| Principal | Staff + production framing (UI layout symmetry checks, image-symmetry detection, palindromic-structure detection in DOM/AST trees). |
13. Follow-up Questions
Follow-up 1: “Iteratively.”
“Single queue of pairs. Initialize [(root.left, root.right)]. Pop pair (a, b); both-None → continue; one-None or val-mismatch → False; enqueue (a.left, b.right) and (a.right, b.left). O(N), O(W).”
Follow-up 2: “What’s the relationship to LC 226 (Invert Tree)?”
“T is symmetric iff invert(T) == T. Symmetric trees are the fixed points of the invert operation. Alternative implementation: is_same(T, invert(T)) — works, slower by constant factor due to building the inverted tree.”
Follow-up 3: “Now LC 951 — Flip-Equivalent.”
“Same skeleton; the recurrence ORs straight-pair AND cross-pair: flip_equiv(a, b) = match_with_value AND ((flip_equiv(a.left, b.left) AND flip_equiv(a.right, b.right)) OR (flip_equiv(a.left, b.right) AND flip_equiv(a.right, b.left))). Naive is 2^N due to OR branching. Optimal: canonicalize each subtree by sorting children’s structural hashes; then flip_equiv reduces to canonical(a) == canonical(b). O(N log N) for the canonicalization sort, or O(N) average with hashing.”
Follow-up 4: “What about an N-ary tree symmetric version?”
“Define: a node is N-ary-symmetric if its children list is a palindrome of children-subtrees (with mirror-recursion). So compare child[0] with child[N-1] mirror-wise, child[1] with child[N-2], etc. Skip the middle for odd N. Same template; loop over child pairs.”
Follow-up 5 (Principal signal): “Where does this matter in production?”
“DOM-tree symmetry checks for UI testing (visual regression frameworks: ‘is this rendered layout symmetric per the design spec?’). Image-symmetry detection (after building a quadtree, check left-right and top-bottom mirror). Code-formatter / AST-balance checks. In all cases, the tree is large; iterative variant is mandatory.”
14. Full Solution Walkthrough
See solution.py. Five sections:
- TreeNode + helpers (build_tree, random_symmetric_tree)
is_symmetric_recursive(mirror-walk)is_symmetric_iterative(single queue of pairs)is_symmetric_via_invert(reusesinvertandis_same)stress_test— random symmetric trees expect True; perturbed expect False; three implementations agreeedge_cases— empty, single, two-node, classic [1,2,2,3,4,4,3], asymmetric perturbations
15. Beyond the Problem — Production Reality
Applications:
- UI layout symmetry verification (design systems, visual regression).
- AST-balance / formatter sanity (parens, braces).
- Image symmetry (after quadtree decomposition).
- Decision-tree pruning (symmetric subtrees can be deduplicated).
- Game theory (symmetric game-trees admit identical optimal play on mirrored positions; engines exploit symmetry to prune search).
Production caveat: real-world “approximate” symmetry needs a tolerance threshold (allow K mismatches). The recursion becomes “count mismatches; accept if ≤ K.” Same skeleton, different aggregation.
Principal-engineer code review comment: “For our design-system layout validator, the strict-symmetric check rejected too many almost-symmetric layouts (off-by-1px or accent-color differences). We generalized to is_mirror_with_tolerance(a, b, tol) — same recursion, with abs(a.val - b.val) <= tol instead of a.val == b.val AND a global mismatch-count budget. Lesson: the recursion structure is reusable across all ‘compare-two-things’ problems; only the leaf-comparison and aggregation policies change.”
p15 Symmetric — Progressive Hints
Rule: ONE hint at a time, 5-min timer.
Hint 1 — Reframe
A tree is symmetric iff its left subtree is the MIRROR of its right subtree. Mirror of what we did in p13 (Same Tree). What changes from “same” to “mirror”?
Hint 2 — The pairing
In Same Tree we paired children straight: (a.left, b.left) and (a.right, b.right). For mirror, we pair them crossed: (a.left, b.right) and (a.right, b.left). That’s the entire algorithmic difference.
Hint 3 — Base cases
Same four cases as p13:
- Both None → True
- One None → False
- Values differ → False
- Both non-None, values equal → recurse on cross-paired children, AND the results.
Hint 4 — Pseudocode
def is_mirror(a, b):
if a is None and b is None: return True
if a is None or b is None: return False
return a.val == b.val \
and is_mirror(a.left, b.right) \
and is_mirror(a.right, b.left)
def is_symmetric(root):
return root is None or is_mirror(root.left, root.right)
Iterative: single queue of pairs, initialize with (root.left, root.right); on each pop enqueue (a.left, b.right) and (a.right, b.left).
Hint 5 — Full + traps + family
Time O(N), space O(H) recursive or O(W) iterative.
Trap 1 — Forgetting to cross: if you write is_mirror(a.left, b.left), you’re checking that left and right subtrees are IDENTICAL, not mirrors. [1,2,2,3,4,4,3] fails (left subtree [2,3,4], right subtree [2,4,3] — not identical, but ARE mirrors).
Trap 2 — Calling is_same(root, root): always True. Symmetric means mirror of itself, not equal to itself.
Trap 3 — Two separate queues iteratively: desync on None alignment. Single queue of pairs.
The fixed-point insight: T is symmetric iff invert(T) == T. From p11 we know invert is involutive: invert(invert(T)) == T. From p15 we know symmetric trees are the fixed points of invert. Alternative implementation: is_same(T, invert(T)) (uses p11 + p13). Direct mirror-walk is cleaner.
Family:
- LC 100 — Same Tree (p13): straight-pair.
- LC 951 — Flip-Equivalent:
(straight) OR (cross)at each node. 2^N naive; O(N) with structural canonicalization. - N-ary symmetric: children list is a palindrome of mirror-subtrees.
Same skeleton, different pairing. Pattern reuse is the lesson.
Week 4 — BST & Tree Traversal Discipline
Theme: Binary Search Trees aren’t just “sorted trees” — they’re the smallest data structure where the invariant (
left < node < right) drives both correctness and complexity. This week drills the invariant-first mindset alongside the four canonical traversal templates (preorder, inorder, postorder, level-order). Time budget: ~7 hours focused work; ~12 hours including the LC bank. Hard rule: every BST problem solved twice — once exploiting the BST property, once without (pretending it’s a general binary tree). State out loud which version you’d ship in interview vs production.
Why this week
Week 3 trained “recursion as the default tool.” This week applies that to a richer setting:
- BST property = correctness lever. Validating a BST looks trivial (“check left < node < right at each node”) but every interviewer’s favorite gotcha is exactly that wrong solution. The right solution requires either passing bounds down or inorder traversal monotonicity — both are reusable patterns.
- Inorder traversal = sorted iteration over BST. This single fact converts a class of “find the k-th smallest / closest / range query” problems into linear scans. Knowing when this applies separates Senior from Mid.
- Construction problems (sorted-array → balanced BST) train the divide-and-conquer template that recurs in segment trees, merge sort, and parallel algorithms.
- Level-order BFS is the bridge to Week 5 (graphs). Same data structure (
deque), samefor _ in range(len(q))level-snapshot trick.
By Friday you should: distinguish BST-specific O(H) shortcuts from general-tree O(N) walks, write inorder iteratively without looking it up, and articulate why a balanced BST is O(log N) but an arbitrary BST can degrade to O(N).
Flagship problems (the 5 you OWN this week)
| # | Problem | LC | Difficulty | Why this one |
|---|---|---|---|---|
| p16 | Validate BST | 98 | Medium | The canonical “wrong solution looks right” problem. Bounds-passing vs inorder-monotonicity. |
| p17 | Lowest Common Ancestor of BST | 235 | Medium | Trains BST-property-as-shortcut. Compare with general LCA (LC 236) — a one-line vs three-line solution. |
| p18 | Kth Smallest Element in BST | 230 | Medium | Inorder + early termination. The follow-up (“frequent inserts/deletes”) is THE classic augmented-BST conversation. |
| p19 | Convert Sorted Array to BST | 108 | Easy | Divide-and-conquer construction. Sets up segment-tree intuition for Phase 3. |
| p20 | Binary Tree Level Order Traversal | 102 | Medium | The canonical BFS template. Reused for LC 103, 107, 199, 314, 515, 637, 993, 1161, 1302. |
LC bank (~25 problems for spaced repetition)
BST validation / construction:
- LC 98 (p16), LC 108 (p19), LC 109 Sorted List to BST, LC 1008 Construct BST from Preorder, LC 449 Serialize BST
BST search / mutation:
- LC 700 Search in BST, LC 701 Insert into BST, LC 450 Delete Node in BST, LC 235 (p17), LC 270 Closest BST Value, LC 272 Closest BST Value II
BST inorder:
- LC 230 (p18), LC 94 Inorder Iterative, LC 173 BST Iterator, LC 285 Inorder Successor, LC 426 Convert BST to DLL, LC 538 Greater Tree
Level-order BFS:
- LC 102 (p20), LC 103 Zigzag, LC 107 Bottom-Up, LC 199 Right Side View, LC 515 Largest in Each Row, LC 314 Vertical Order, LC 637 Average of Levels
Readiness gate (binary; must all be YES before Week 5)
- I can articulate the BST invariant in one sentence: “for every node, EVERY value in the left subtree is < node.val AND EVERY value in the right subtree is > node.val.” (Not just immediate children — the entire subtree.)
-
I can show the “wrong” Validate BST that only checks immediate children and explain the failing test case
[5,1,4,null,null,3,6]. - I solved p16 BOTH via bounds-passing AND via inorder-monotonicity, and can argue which generalizes better (bounds → general predicate; inorder → “any monotone property”).
- I can write an iterative inorder traversal from memory using an explicit stack.
- I can explain why LC 235 (LCA of BST) is O(H) but LC 236 (LCA of binary tree) is O(N) — and write both.
-
I know
len(q)snapshot trick for level-order BFS and why omitting it merges levels. - I can describe the average-case vs worst-case complexity of operations on an unbalanced BST (O(log N) avg, O(N) worst) and name two self-balancing variants (AVL, Red-Black).
-
My
stress_test()for each problem cross-checks 2+ variants on 1000 random BSTs (built withrandom.Random(42)).
If ANY unchecked: do not start Week 5. Re-drill the missing pieces.
Recurring traps
- “Just check
node.left.val < node.val < node.right.val.” This is the #1 wrong answer. It accepts[5,1,4,null,null,3,6]which is NOT a BST. The correct check is bounds-based or inorder-based. - Using
<=instead of<. LC’s BST is strict — duplicates are not allowed. Inorder must produce strictly increasing values. - Inorder recursion stack overflow. A skewed BST of depth 10^4 blows Python’s default 1000 limit. Iterative inorder uses heap memory, not stack.
- Early termination forgotten in p18. Walking the entire inorder is O(N); proper solution stops after the k-th pop and is O(H + k).
for _ in range(len(q))omitted in BFS. Without it you get flat order, not level order. Thelen(q)must be captured BEFORE the inner loop starts because the queue grows during the loop.- Confusing height vs depth in BST complexity. Average BST height for random insertions is O(log N), but adversarial sorted insertions degrade to O(N) — this is the entire motivation for AVL/RB trees.
Stretch (only if you finish all 5 + LC bank with time left)
- LC 99 Recover Binary Search Tree — find the two swapped nodes via inorder. Classic Medium that’s almost a Hard if you don’t see the inorder pattern.
- LC 124 Binary Tree Maximum Path Sum (Hard) — promised from Week 3; the two-channel pattern with negative-pruning.
- LC 297 Serialize and Deserialize Binary Tree (Hard) — preorder + null markers; Meta loves this.
- LC 314 Binary Tree Vertical Order Traversal — level-order + column tracking; common at Meta/LinkedIn.
Begin with p16 — Validate BST.
p16 — Validate Binary Search Tree
Source: LeetCode 98 · Medium · Topics: Tree, DFS, BST Companies (2024–2025 frequency): Amazon (very high), Microsoft (very high), Meta (high), Google (high), Bloomberg (high — phone screen staple), Apple (medium), Adobe (medium) Loop position: first round in 80% of phone screens that include a tree question; the canonical “you’ll pick the wrong solution first” trap.
1. Quick Context
This is the single most well-known booby-trap problem in tree interviews. The naive solution — “check node.left.val < node.val < node.right.val at every node” — looks correct, passes 4 of LC’s 5 sample cases, and is wrong on the case [5,1,4,null,null,3,6] (which has 3 < 4 < 6 locally but 3 < 5 globally — violating BST).
The interview value is in the candidate’s diagnostic process:
- Do you immediately write the wrong solution? (most do)
- Do you catch it yourself via the right example? (top-30%)
- Can you articulate the BST invariant correctly before coding? (top-10% — this is the move that flips the round)
Two clean solutions exist:
- Bounds-passing (top-down): carry
(lo, hi)open intervals into each recursive call; at the leaf, the constraints from ancestors are baked in. - Inorder monotonicity (bottom-up): inorder of a valid BST is strictly increasing; do an iterative inorder and check each successive pair.
Both are O(N). Both are interviewer-acceptable. Knowing why bounds-passing generalizes (any monotone predicate on the path from root to leaf) and why inorder generalizes (any property reducible to ordered iteration) is the Senior-level conversation.
What it looks like it tests: can you traverse a tree. What it actually tests: can you state an invariant precisely BEFORE coding, and can you spot the difference between a local check and a global check.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/validate-binary-search-tree/
STOP. 8-min timer. If you write node.left.val < node.val < node.right.val and submit — that’s the lesson. Build the test case [5,1,4,null,null,3,6] by hand and trace through your code.
3. Prerequisite Concepts
- p12, p13 — recursive tree walks that return booleans
- The notion of an invariant that holds for every subtree (not just immediate children)
- Inorder traversal (recursive AND iterative — see phase-01-foundations/labs/lab-09-tree-traversal-fundamentals.md)
- Python’s
float('inf')/float('-inf')sentinels orNonechecks for unbounded sides
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given the root of a binary tree, return True iff it is a valid BST. A valid BST requires that for EVERY node, ALL values in its left subtree are strictly less than the node’s value AND ALL values in its right subtree are strictly greater. Strict inequality — duplicates are not allowed.”
Step 2 — Clarify:
- “Strict or non-strict?” (Per LC: strict. State this explicitly — some org’s internal BSTs allow duplicates on one side.)
- “Empty tree — is it a valid BST?” (Yes. Vacuously true.)
- “Integer range?” (LC: −2^31 to 2^31 − 1. Important: this means
float('-inf')is safe as a sentinel BUT in some languages you’d need a special tri-state.) - “Single node?” (Valid. Trivially BST.)
- “Should I assume the tree is balanced?” (No. Worst case is depth N.)
Step 3 — Constraints: N up to 10^4. Values fit in int32. O(N) easily achievable; the goal is correctness, not asymptotic improvement.
Step 4 — Examples:
[2,1,3]→ True.[5,1,4,null,null,3,6]→ False (this is THE test case — say it aloud).[1]→ True.[1,1]→ False (strict; duplicate).[2147483647]→ True (boundary; sentinel-safe).- Empty → True.
Step 5 — Brute Force: At every node, scan its entire left subtree (must all be < node.val) and entire right subtree (must all be > node.val). O(N²). State this then optimize.
Step 6 — Complexity (optimal): O(N) time, O(H) stack space (recursive bounds OR iterative inorder).
Step 7 — Pattern Recognition:
- Bounds-passing = “pass-down-constraint” recursion (Path: ancestors → descendants). Same shape recurs in segment-tree updates, interval DP, and Knapsack-bounded.
- Inorder monotonicity = “tree-as-sorted-sequence” trick. Same idea unlocks p18, LC 99, LC 230, LC 285.
Step 8 — Two distinct optimal solutions:
- Solution A (bounds): recursive
validate(node, lo, hi)where root starts withlo=None, hi=None. Each left call tightenshi = node.val; each right call tightenslo = node.val. Checklo < node.val < hi. - Solution B (inorder): maintain
prev(the last value seen). On each inorder visit, assertprev is None or prev < node.val, thenprev = node.val. Use iterative inorder for the stack-safe production version.
Step 9 — Prove correctness (bounds version):
- Invariant maintained: when
validate(node, lo, hi)is called, every value on the path from root tonodeconstrainsnode.valsuch thatlo < node.val < hiis necessary. The recursion enforces this at every node. - Sufficiency: if the invariant holds at every node, then for any pair
(a, b)withain left subtree ofb,a.val < b.valholds becauseawas visited withhi <= b.val. Symmetric for right subtree. ∎
5. Progressive Hints
If stuck for 8 min, open hints.md. Don’t peek before constructing the [5,1,4,null,null,3,6] counterexample by hand.
6. Deeper Insight — Why “local check” fails
The reason node.left.val < node.val < node.right.val is wrong is transitivity is not local. The BST property is a path-cumulative constraint: the value of any descendant on the left of an ancestor must be less than that ancestor’s value, no matter how far down.
Picture [5,1,4,null,null,3,6]:
5
/ \
1 4
/ \
3 6
At node 4: locally 3 < 4 < 6 ✓. But 3 is in the right subtree of 5, so 3 > 5 must hold. It doesn’t. The local check has no visibility into ancestor 5’s constraint.
Bounds-passing fixes this by making the ancestor’s constraint travel with the recursion. Inorder fixes this by linearizing the tree to a sequence where every pair-violation is adjacent.
The deeper takeaway: whenever a tree problem asks about a property that involves two non-adjacent nodes, your two go-to tools are (a) pass info down via parameters, or (b) flatten to inorder/preorder/postorder.
7. Anti-Pattern Analysis
- Local check (
node.left.val < node.val < node.right.val). Wrong; see above. This is the trap. - Using
<=instead of<. Off-by-one on duplicates. Accepts[1,1]as a valid BST. Always restate “strict” in Step 2. - Returning early
Truewithout recursing into both subtrees. A common bug:if not check_left: return Falsefollowed byreturn Trueforgets the right subtree. - Using
float('inf')in a language where ints can equal it. In Python, fine. In Java/C++, you must useLong.MIN_VALUE / MAX_VALUEor null/Optional, becauseInteger.MAX_VALUEcould legitimately appear as a node value and equal the sentinel. State this trade-off if asked about polyglot implementation. - Inorder solution that builds a full list then checks sorted. Works but uses O(N) extra space. The streaming version with a single
previs O(H) space. Pre-empt by writing the streaming version.
8. Skills & Takeaways
- Invariant-first thinking. Write the invariant in English before coding. Every BST problem you’ll ever see is a variant of “what does the invariant imply about this question?”
- Two reusable templates in your toolkit: bounds-passing recursion (for any path-cumulative property) and inorder monotonicity (for any order-reducible property).
- Strict vs non-strict equality is always worth one explicit clarification question. Cheap signal.
- Streaming inorder (with a single
prevvariable) is the right production form. Knowing it cold prevents O(N) extra space in many follow-ups.
9. When to Move On
Move on when:
- You can write the bounds-passing solution from memory in <3 minutes.
- You can write iterative inorder from memory in <4 minutes.
- You can explain — without code — why
[5,1,4,null,null,3,6]breaks the local check and which ancestor’s constraint the local check is missing. - Your stress test agrees with a brute-force O(N²) reference on 1000 random trees including known-invalid cases.
10. Company Context
Amazon:
- Loves this for SDE-II and SDE-III phone screens. Look for adversarial: “what if values can be duplicates?” (must switch to non-strict or define which side duplicates go).
- Follow-up: “what if the tree has 10^9 nodes and doesn’t fit in memory?” → discuss streaming inorder from on-disk B-tree storage. Bar Raiser checks this.
- Scorecard phrase: “Deep Dive — articulated invariant precisely; identified the local-check trap unprompted.”
Google:
- Will push on complexity proof. “Prove the bounds approach is O(N) and not O(N log N)” — answer: each node visited once, constants don’t grow with depth.
- Follow-up: “design
validate(node)that also returns the count of nodes” → two-channel pattern (link to p14). - Scorecard phrase: “Algorithm — clean invariant, complete proof; clear about strict-vs-non-strict semantics.”
Meta:
- Often pairs this with LC 99 (Recover BST) in the same round. If you nail p16, expect the recovery follow-up.
- Watch for “code it iteratively” — if your first pass is recursive, they’ll ask for the iterative inorder version.
- Scorecard phrase: “Coding Excellence — produced both recursive and iterative solutions; chose the right one for the follow-up.”
Microsoft:
- Less adversarial; will accept either solution. Will probe: “what if I tell you the tree is BST EXCEPT for two swapped nodes?” → LC 99.
- Scorecard phrase: “Problem Solving — recognized invariant immediately; handled boundary integer values.”
Bloomberg:
- Phone-screen staple. Often paired with “implement BST insert” in the same 45 minutes.
- Will ask “what’s the use case for BSTs at Bloomberg?” — answer: order books, time-series indexing, range queries.
- Scorecard phrase: “Strong — correct first try; explained BST utility in financial systems.”
Apple:
- Less common but appears in iOS/macOS frameworks team. Will probe: “Swift uses red-black trees in
Dictionary— how does balancing relate to this?” - Scorecard phrase: “Solid Fundamentals — connected BST to production data structures.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Initial approach | Writes local check, doesn’t notice | Writes local check, asked to find bug | Asks clarifying questions, writes bounds first try | States invariant, names both solutions, picks one with rationale |
| Edge cases | Forgets empty/single | Handles them after prompt | Lists them upfront | Lists them upfront AND argues int-overflow sentinel choice |
| Code quality | Mutable globals, no comments | Clean function, no invariant comment | INVARIANT docstring, both versions | Both versions + production note on iterative for depth safety |
| Follow-up handling | Stuck on “what if duplicates” | Switches to <= correctly | Discusses ambiguity, asks which side | Designs general predicate-validator; generalizes to LC 99 |
| Communication | Silent coding | Narrates after | Narrates while | Narrates intent, complexity, and trade-offs in real time |
12. Level Delta
Mid (L4 / SDE-II):
- Gets the bounds-passing solution after one nudge.
- Mentions inorder as alternative.
- Handles strict-vs-non-strict correctly when asked.
Senior (L5 / SDE-III):
- Writes bounds-passing first try with
Nonesentinels. - Writes streaming iterative inorder for follow-up without prompting.
- Articulates why local check fails using the canonical counterexample.
Staff (L6 / Principal Engineer Candidate):
- States the invariant in one sentence before touching the keyboard.
- Implements BOTH solutions; explains when to prefer each (bounds for early termination on adversarial path; inorder for follow-ups asking about successor/k-th smallest).
- Discusses depth-limit production concerns (recursion → stack overflow) and offers iterative as the deployable form.
Principal (L7+):
- Generalizes: “the bounds-passing trick is a special case of constraint propagation — same idea is in solvers, type-checkers, interval analysis.”
- Connects to RB-trees / B-trees / LSM-trees — why “balanced” matters for the BST guarantees to hold under arbitrary insert order.
- Discusses concurrent BST validation (snapshot isolation? COW?) for distributed systems.
13. Follow-up Q&A
Q1: What if the tree allows duplicates on the left subtree (i.e., left.val <= node.val < right.val)?
A: Switch the bounds check to lo < node.val < hi becomes lo <= node.val < hi. For inorder, allow prev == node.val (non-strict monotone). Restate the invariant explicitly: “values in left subtree are ≤ node, values in right are >.”
Q2: How would you implement this iteratively without recursion?
A: Iterative inorder with an explicit stack and a prev variable. Push lefts onto stack until null, pop and check against prev, then descend right. O(H) auxiliary space.
Q3: Can you do it without float('inf') sentinels?
A: Yes — use None for unbounded sides and check (lo is None or lo < node.val) and (hi is None or node.val < hi). Slightly verbose but language-agnostic.
Q4: How would you modify this to also count the number of invalid nodes (nodes that violate the BST property at some ancestor)? A: Use the two-channel pattern (p14). The recursion returns the validity flag; a closure variable accumulates the count of violators. Requires defining “which node is the violator” — typically the descendant whose value breaks the constraint, not the ancestor.
Q5: A colleague suggests storing (min, max, is_bst) at each node as a return triple and computing bottom-up. Compare with bounds-passing.
A: Bottom-up triple is also O(N), but it carries 3 values up instead of pushing 2 down. Pros: no need for sentinels; easier to extend to “find largest BST subtree” (LC 333). Cons: more state per call. Both are interviewer-acceptable; pick based on the follow-up (largest BST subtree → triple; validate only → bounds).
14. Solution Walkthrough
See solution.py:
validate_bounds(root)— recursive bounds-passing withNonesentinels (the production-recommended form).validate_inorder_iterative(root)— iterative inorder withprev(stack-safe; the form you’d ship).validate_inorder_recursive(root)— recursive inorder withnonlocal prev(cleaner code; same algorithm).validate_brute_n2(root)— O(N²) reference oracle: at every node, scan entire left/right subtree.stress_test()— generates 1000 random binary trees (some valid BSTs, some invalid via swap-mutation); asserts all 4 implementations agree.edge_cases()— empty, single,[5,1,4,null,null,3,6], duplicates, INT_MAX boundary.
Run: python3 solution.py → “ALL TESTS PASSED” on stdout.
15. Beyond the Problem
Principal-engineer code review (verbatim, the kind you’d see on a Phab/Gerrit diff):
“This validation logic is correct but I’d flag two things before merging. First, the recursive form will stack-overflow on adversarial input (any sorted-insertion BST of depth > sys.getrecursionlimit()). For a public API ingesting user-provided trees, ship the iterative inorder version — same O(N), same correctness, but it uses heap not native stack. Second, the use of
float('-inf')is fine in Python but if this is ever ported to Java/Rust/Go, the sentinel collision with a legitimateINT_MINnode value will silently corrupt validation. Either parameterize the sentinel type or refactor to theNone-based bounds — the latter ports cleanly. Also: name the failing-case unit testtest_subtree_violates_ancestor_constraintnottest_invalid_tree_1so the next on-call understands what broke when it regresses.”
This is the level of code-review feedback you should be internalizing — not just “does it work” but “will it break in production, will it port, will the next engineer understand the regression.”
Connections forward:
- p17 (LCA of BST) exploits the same “BST property as shortcut” lever.
- p18 (Kth Smallest) uses the inorder monotonicity insight directly.
- p19 (Sorted Array → BST) is the inverse of inorder.
- LC 99 (Recover BST) is the canonical “what if inorder finds two anomalies” problem and is a frequent follow-up.
- The bounds-passing pattern reappears in segment-tree update propagation, interval DP, and constraint solvers.
p16 — Hints
Open one at a time. Spend at least 90 seconds re-attempting between hints.
Hint 1 — Diagnostic
Did you write node.left.val < node.val < node.right.val? Trace your code on [5,1,4,null,null,3,6]. Draw the tree. Notice the 3 sitting in the right subtree of 5. What constraint does 5 impose on 3 that your code never checks?
Hint 2 — The Invariant
Re-state the BST property out loud, precisely:
For every node, ALL values in its left subtree are strictly less than node.val, AND ALL values in its right subtree are strictly greater.
Note “ALL” — not just immediate children. Your code must enforce this for descendants arbitrarily deep.
Hint 3 — Two Templates
There are two clean ways to enforce the all-descendants invariant:
- Pass bounds down. Each recursive call carries
(lo, hi)— the open interval the subtree’s values must lie in. Initially(None, None)= unbounded. When recursing left, tightenhi = node.val. When recursing right, tightenlo = node.val. - Walk inorder, check monotone. Inorder of a valid BST is a strictly increasing sequence. Walk inorder, remember the previous value, and reject if any successor isn’t strictly greater.
Pick one and code it. (For the second pass, code the other.)
Hint 4 — Pseudocode (Bounds)
def go(node, lo, hi):
if node is None: return True
if lo is not None and node.val <= lo: return False
if hi is not None and node.val >= hi: return False
return go(node.left, lo, node.val) and go(node.right, node.val, hi)
Three things to double-check:
- Use
<=/>=to enforce strict BST. - Tighten hi when going left, lo when going right (not the other way).
- Sentinels:
Nonemeans “no bound on that side.” Don’t usefloat('inf')if you want this to port to Java cleanly.
Hint 5 — Pseudocode (Iterative Inorder)
stack, cur, prev = [], root, None
while cur or stack:
while cur:
stack.append(cur)
cur = cur.left
node = stack.pop()
if prev is not None and node.val <= prev:
return False
prev = node.val
cur = node.right
return True
This is the form to memorize — it appears verbatim in p18 (Kth Smallest), LC 173 (BST Iterator), and LC 285 (Inorder Successor). Practice writing it without looking until you can do it in 90 seconds.
p17 — Lowest Common Ancestor of a Binary Search Tree
Source: LeetCode 235 · Medium (Easy if you spot the BST shortcut) · Topics: Tree, DFS, BST Companies (2024–2025 frequency): Amazon (high), Meta (high), Microsoft (high), LinkedIn (very high — phone screen staple), Google (medium), Bloomberg (medium) Loop position: phone-screen or first round; almost always paired with LC 236 (general LCA) as the explicit follow-up.
1. Quick Context
LCA-of-BST is the cleanest demonstration of the BST property as algorithmic lever. The general LCA problem (LC 236) for an arbitrary binary tree requires post-order traversal with a 4-case logic and is O(N). The BST version collapses to a single-pass O(H) walk:
At the current node, if both targets
pandqlie in the left subtree (p.val < node.val and q.val < node.val), recurse left. If both lie in the right subtree, recurse right. Otherwise, the targets are on opposite sides (or one is this node), so this node IS the LCA — return it.
That’s it. The whole solution is 6 lines. The interview-grade discussion is:
- Why this works (the split-point argument: the LCA is exactly the first node where
pandqdiverge in the BST search path). - How it compares with the general LCA (LC 236) — a comparison interviewers explicitly ask for.
- The pitfall: assuming
p.val < q.val(some problem variants do; LC 235 does not). - The iterative form (no recursion, O(1) auxiliary space — yes, truly O(1)).
What it looks like it tests: can you traverse a tree. What it actually tests: can you recognize when an extra data-structure invariant turns an O(N) problem into O(log N) — the kind of pattern recognition that separates Senior+ engineers.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/lowest-common-ancestor-of-a-binary-search-tree/
STOP. 6-min timer. If you find yourself writing a “search for p, search for q, intersect paths” solution — STOP and re-read the problem title. Why does it say “of a Binary Search Tree” specifically?
3. Prerequisite Concepts
- p16 — the BST invariant
- Standard recursive tree walks (return a node from recursion)
- The concept of an “LCA” — for any two nodes
pandqin a tree, the LCA is the deepest node that has both as descendants (a node is its own descendant) - Iterative tree descent (the BST “search” pattern, see phase-01-foundations/labs/lab-09-tree-traversal-fundamentals.md)
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given a BST and two nodes p and q present in it, return their lowest common ancestor (the deepest node that has both as descendants; a node is considered a descendant of itself).”
Step 2 — Clarify:
- “Are
pandqguaranteed to exist in the tree?” (LC: yes.) - “Can
p == q?” (LC: no — they are distinct.) - “Can
pbe an ancestor ofq?” (Yes — thenpis the LCA. State this; don’t trip on it.) - “Are values guaranteed unique?” (LC: yes — node values are unique. State this; the algorithm relies on it.)
Step 3 — Constraints: N up to 10^5. BST may be unbalanced. Need O(H) — average O(log N), worst O(N).
Step 4 — Examples:
- Tree
[6,2,8,0,4,7,9,null,null,3,5],p=2, q=8→ 6 (split at root). - Same tree,
p=2, q=4→ 2 (p is ancestor of q). - Same tree,
p=3, q=5→ 4 (split in subtree). - Two-node tree
[1,null,2],p=1, q=2→ 1 (ancestor case).
Step 5 — Brute Force: Find path from root to p, find path from root to q, return last common node. O(N) time, O(N) space (paths). Mention this then immediately optimize using the BST property.
Step 6 — Complexity (optimal): O(H) time = O(log N) average, O(N) worst (skewed). O(1) auxiliary space iteratively; O(H) stack space recursively.
Step 7 — Pattern Recognition: “An ordered tree where I’m searching for the split point of two queries” → descend until divergence. This pattern reappears in LC 270 (Closest BST Value), LC 285 (Inorder Successor), and segment-tree range queries.
Step 8 — Develop optimal:
- Recursive:
def lca(node, p, q): if p.val < node.val and q.val < node.val: return lca(node.left, p, q) if p.val > node.val and q.val > node.val: return lca(node.right, p, q) return node - Iterative (preferred for production): same logic in a
while Trueloop, no recursion, true O(1) space.
Step 9 — Prove correctness: The LCA is by definition the deepest node N such that p and q are both in N’s subtree. Walking down from the root:
- If both
p.valandq.val<N.val, then both are inN’s left subtree —Nis an ancestor but not the lowest, so descend left. - Symmetric for both >.
- Otherwise (split: one ≤ and one ≥, OR one of them equals
N.val), thenNis the first node where the BST search path forpand the path forqdiverge. Any deeper node would lose one of the two. ThereforeNis the LCA. ∎
5. Progressive Hints
If stuck for 6 min, open hints.md.
6. Deeper Insight — The “split-point” framing
Conceptually, the LCA of two nodes in a BST is the first divergence in their search paths. If you started two parallel BST searches at the root — one looking for p, one looking for q — they would walk in lockstep down identical prefixes (because the BST search decision at each node is purely based on the value being searched for), and the first node where one would turn left and the other right is exactly the LCA.
This framing generalizes:
- LC 270 (Closest Value in BST): descend, tracking the closest value seen. Same single-pass O(H) walk.
- LC 272 (Closest K Values in BST): combine descent with inorder around the split point.
- Range queries on a segment tree: walk down, splitting where the range diverges (left subrange vs right subrange).
This “split-point descent” is one of the most reusable tree templates. Internalize it.
7. Anti-Pattern Analysis
- Searching for p and q separately, then intersecting paths. O(N) time and O(N) space — correct, but discards the BST property. State as brute force; do not ship it.
- Forgetting the equality case (
pis an ancestor ofq). The pseudocodeif p.val < node.val and q.val < node.valhandles this correctly because whenp.val == node.val, both conditions are false and we returnnode. Beginners sometimes write<=and break this. Always use strict<. - Assuming
p.val < q.val. Some textbook variants order them; LC 235 does not. Always write the algorithm symmetric inpandq. - Calling
lca(node, p, q)recursively when iteration would do. Recursion is fine but adds O(H) stack. For interview, iterative is the preferred form because it demonstrates you understand “the recursion is tail-recursive — eliminate it.” - Returning the LCA’s value instead of the node. Read the signature carefully. LC returns the
TreeNode.
8. Skills & Takeaways
- Recognize when invariants collapse complexity. BSTs let LCA drop from O(N) to O(H). Identifying which property is doing the work is a Senior-level skill.
- Compare-and-contrast with the general case. Any time the problem says “Binary Search Tree” instead of “binary tree,” ask: what does the search property let me skip?
- Tail-recursion elimination. The recursive solution is tail-recursive; converting it to a
whileloop is the production-quality move. - Path-divergence as a template. This shows up in segment trees, B-tree range queries, and trie-based prefix problems.
9. When to Move On
Move on when:
- You can write both the recursive and iterative solutions from memory in <3 minutes each.
- You can explain — without code — why the BST property reduces the problem from O(N) to O(H).
- You can sketch the general LCA solution (LC 236) and articulate which 4 cases the post-order recursion handles.
- Your stress test confirms agreement with a brute-force path-intersection oracle on 1000 random BSTs.
10. Company Context
LinkedIn:
- This is a flagship phone screen problem at LinkedIn. Expect the follow-up “now do it for a general binary tree” (LC 236) in the same 45-minute slot.
- Bonus: “what if we have many queries on the same tree?” → preprocess for O(log N) per query via Euler tour + RMQ; mention but don’t implement.
- Scorecard phrase: “Algorithm — used BST property; produced iterative form; articulated LCA generalization.”
Amazon:
- Standard SDE-II ask. Look for “what if the tree is a billion nodes on disk?” → discuss B-tree LCA (same split-point logic, just on a fan-out > 2 node).
- Scorecard phrase: “Deep Dive — applied invariant to achieve O(H); extended to B-tree case.”
Meta:
- Often used as a warm-up before LC 236. If you solve LC 235 in 5 minutes, expect LC 236 as the real problem.
- Scorecard phrase: “Coding Excellence — clean 6-line solution; smooth transition to general case.”
Google:
- Will push on proof: “prove this is correct” (give the split-point argument). Also: “what’s H in the worst case?” (N for a skewed tree — the BST guarantee is average O(log N), not worst).
- Scorecard phrase: “Algorithm — correctness proof; honest about worst-case degradation.”
Microsoft:
- Often asks the iterative variant first. Will probe: “what’s the space complexity if I use recursion?” → O(H) for the stack.
- Scorecard phrase: “Problem Solving — recognized tail-recursive structure; converted to loop.”
Bloomberg:
- Uses this in the context of “find the common parent in an org-hierarchy BST keyed by employee ID.” Be ready for domain framing.
- Scorecard phrase: “Strong — clean solution; understood real-world use.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Recognize BST shortcut | Writes general LCA, never simplifies | Notices “BST” in title after prompt | Notices immediately | Names “split-point descent” pattern unprompted |
| Iterative form | Doesn’t attempt | Writes after prompt | Writes unprompted | Writes iterative; explains why tail-recursion elimination matters |
| Edge cases | Misses p ancestor of q | Handles after prompt | Lists upfront | Lists upfront + discusses worst-case H = N |
| Comparison with LC 236 | Confused | Mentions general LCA | Sketches general LCA | Codes general LCA when asked; compares both clearly |
| Generalization | None | None | Mentions LC 270 / 285 | Connects to B-tree range queries, segment tree, RMQ |
12. Level Delta
Mid (L4 / SDE-II):
- Writes the recursive solution after a hint about the BST property.
- Mentions iterative form when asked.
- Handles
pancestor ofqcorrectly with one trace.
Senior (L5 / SDE-III):
- Writes iterative form first try (it’s the natural choice once you see the split-point).
- Articulates split-point argument verbally.
- Sketches LC 236 when asked, identifies the 4 cases.
Staff (L6):
- States the BST shortcut before touching keys, frames it as “invariant collapses complexity.”
- Discusses worst-case H = N and references AVL/RB-trees for guarantees.
- Sketches Euler-tour + RMQ for multi-query preprocessing.
Principal (L7+):
- Generalizes split-point descent to B-trees, LSM-tree level traversal, range queries on segment trees.
- Discusses concurrent LCA queries on a shared BST (snapshot isolation, COW).
- Identifies the meta-pattern: ordered structures let queries collapse from O(N) to O(log N) via dichotomy.
13. Follow-up Q&A
Q1: Implement LCA for a general binary tree (LC 236).
A: Post-order recursion. lca(node, p, q): if node is None or node == p or node == q, return node. Recurse left and right. If both return non-None, this node IS the LCA. Otherwise return the one non-None side (propagating the found target up). O(N) time, O(H) stack.
Q2: What if we have Q queries on the same tree? A: Preprocess with Euler tour + sparse-table RMQ → O(N log N) preprocessing, O(1) per query. Mention; don’t implement unless explicitly asked.
Q3: What if the tree might not contain p or q?
A: Modify the general LCA to track whether both were found. Return (node, count_found). Only return the node if count_found == 2. Otherwise return None.
Q4: What if the tree is stored on disk and each node access costs an I/O?
A: The BST descent makes O(H) I/Os = O(log N) for balanced. For worst-case guarantees, switch to a B-tree (each node has many children → much shallower tree → fewer I/Os). The split-point algorithm generalizes naturally: at a B-tree node, find the first child whose key range contains either p or q; if p and q fall in the same child, descend; otherwise return this B-tree node as LCA.
Q5: How would you parallelize LCA queries across many (p_i, q_i) pairs on the same tree?
A: For independent reads on an immutable BST, trivially parallel (each query is its own descent). For an updated BST, use snapshot isolation (COW) or a multi-versioned BST. For mass queries with locality, batch the queries that share a common prefix (offline LCA = Tarjan’s algorithm, O((N + Q) α(N)) with DSU).
14. Solution Walkthrough
See solution.py:
lca_recursive(root, p, q)— the 6-line recursive form.lca_iterative(root, p, q)— the production form, true O(1) auxiliary space.lca_general_recursive(root, p, q)— LC 236 reference for the canonical follow-up.lca_path_intersect(root, p, q)— O(N) brute force used as stress-test oracle.stress_test()— generates 1000 random BSTs with random(p, q)pairs; asserts all 4 implementations agree.edge_cases()—p == root,pancestor ofq, two-node tree, deep skewed tree.
Run: python3 solution.py → “ALL TESTS PASSED” on stdout.
15. Beyond the Problem
Principal-engineer code review comment:
“Fine for the BST case but please add a doc-comment that names the precondition explicitly: this assumes (a)
pandqare both present in the tree and (b) node values are unique. Both are easy for the caller to violate and the failure mode is a wrong answer, not a crash — the worst kind of bug. Also, prefer the iterative form here: the recursion is tail-recursive and the iterative form is 2 extra lines for O(1) space and zero risk of stack overflow on adversarial skewed inputs. If we ever reuse this in a context where the tree might be malformed (depth > 10^4 from a sorted-insert build), the recursive form will throw, and the stack trace will name the wrong subsystem because Python’s recursion-limit error is generic. Save a future on-call from a 2 a.m. page — write the loop.”
Connections forward:
- p18 (Kth Smallest BST) uses the same “descend the BST exploiting order” framing.
- LC 236 (general LCA) is the explicit follow-up — drill it.
- Phase 3 segment-tree range queries use the same split-point descent.
- Tarjan’s offline LCA (DSU-based) is the next layer up; mentions appear in graph theory rounds at Google/research orgs.
p17 — Hints
Open one at a time.
Hint 1 — Notice the Title
Re-read the problem title: “Lowest Common Ancestor of a Binary Search Tree.” The word “Search” is the entire hint. What does the BST property let you do at each node that you couldn’t do in a general binary tree?
Hint 2 — Where Do p and q Live?
At any node N:
- If
p.val < N.valANDq.val < N.val, bothpandqare inN’s left subtree. SoNis an ancestor of both — but is it the lowest one? No: their LCA is deeper, in the left subtree. - Symmetric for both >
N.val. - Otherwise,
pandqare on different sides ofN(or one ISN), soNIS their LCA.
Code this branching logic.
Hint 3 — Pseudocode
def lca(node, p, q):
if p.val < node.val and q.val < node.val:
return lca(node.left, p, q)
if p.val > node.val and q.val > node.val:
return lca(node.right, p, q)
return node
That’s the whole solution. 6 lines. Verify by tracing through the LC example [6,2,8,0,4,7,9,null,null,3,5] with p=2, q=8.
Hint 4 — Iterative Version
The recursion is tail-recursive: the recursive call’s return value is returned directly with no extra work. That means it converts trivially to a loop, saving O(H) stack space.
node = root
while True:
if p.val < node.val and q.val < node.val:
node = node.left
elif p.val > node.val and q.val > node.val:
node = node.right
else:
return node
This is the form to ship. Memorize it.
Hint 5 — Why It Works (the proof)
The LCA of p and q is by definition the deepest node having both as descendants. Any node strictly above the LCA also has both as descendants (less deep). Any node strictly below the LCA has at most one of them.
Walking down from the root:
- If both targets are in the left subtree, the current node has both as descendants — but a deeper node (somewhere in the left subtree) also does. So descend.
- Symmetric for right.
- Otherwise, the targets split. The current node has both as descendants (
pon one side,qon the other). No deeper single node has both — because they’re in different subtrees. So this is the LCA.
This is the split-point argument. Internalize it; it generalizes to range queries on segment trees and B-trees.
p18 — Kth Smallest Element in a BST
Source: LeetCode 230 · Medium · Topics: Tree, DFS, BST, Inorder Companies (2024–2025 frequency): Amazon (very high), Meta (high), Google (high), Microsoft (high), Bloomberg (medium), TikTok (medium) Loop position: mid-loop algorithmic round; ALWAYS comes with the augmented-BST follow-up (“what if there are frequent insertions/deletions?”). The follow-up is the real test.
1. Quick Context
The base problem (return the k-th smallest value in a BST) is a 10-line iterative inorder with early termination — straightforward once you internalize the inorder template from p16. The follow-up is what every interviewer is actually waiting for: “Suppose the BST is modified often (frequent inserts and deletes), and you need to find the k-th smallest many times. How would you optimize?”
The intended answer: augment each node with the size of its subtree (node.size = 1 + size(left) + size(right)). Then kth_smallest(node, k):
left_size = size(node.left)- If
k <= left_size→ recurse left with samek. - If
k == left_size + 1→ this node IS the answer. - Else → recurse right with
k - left_size - 1.
Each query becomes O(H). Insert/delete maintain size in O(H). This order-statistic tree pattern is the foundation of every “find rank / select k-th” algorithm in production databases.
What it looks like it tests: can you walk inorder. What it actually tests: (a) do you stop inorder early at the k-th element (memory-efficient streaming) and (b) do you know order-statistic trees / augmented BSTs.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/kth-smallest-element-in-a-bst/
STOP. 7-min timer. Before peeking at hints, also pre-think the augmentation follow-up — it’s coming.
3. Prerequisite Concepts
- p16 — iterative inorder template (you should be able to write it cold)
- p17 — descending the BST exploiting order
- The notion of “k-th order statistic”
- Subtree size aggregation (basic tree DP)
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given the root of a BST and an integer k (1-indexed), return the k-th smallest value in the BST.”
Step 2 — Clarify:
- “1-indexed or 0-indexed?” (LC: 1-indexed.)
- “Is k guaranteed valid (1 ≤ k ≤ N)?” (LC: yes.)
- “Duplicates?” (LC: BST guarantees uniqueness here.)
- “Will the tree be modified between queries?” (Crucial clarifying question — pre-empts the follow-up. If the answer is “yes,” skip to augmented BST.)
- “Single query or many?” (Same direction as previous.)
Step 3 — Constraints: N up to 10^4. k between 1 and N. Need O(H + k) for single query, O(log N + log N) per query for augmented version.
Step 4 — Examples:
[3,1,4,null,2], k=1→ 1.[5,3,6,2,4,null,null,1], k=3→ 3.- Single node, k=1 → that node’s value.
- Right-skewed tree of 10 nodes, k=5 → the 5th value.
Step 5 — Brute Force: Collect all values via full inorder into a list, return list[k-1]. O(N) time, O(N) space. State this then propose early-termination.
Step 6 — Complexity (optimal):
- Single query, unaugmented: O(H + k) time, O(H) space.
- Many queries, augmented: O(log N) per query (balanced), O(H) per update.
Step 7 — Pattern Recognition:
- “k-th smallest in an ordered tree” → early-terminating inorder.
- “many queries with mutations” → augment with subtree size (order-statistic tree). This is the same pattern as Fenwick-tree-of-counts and segment-tree-of-sizes from Phase 3.
Step 8 — Develop:
- Iterative inorder with counter: descend leftmost, pop, decrement k, if k == 0 return val, else descend right. Stops as soon as k-th is popped.
- Recursive inorder with closure counter: same, with
nonlocal kand early return via flag. - Augmented BST: maintain
sizeon every node; descend based onsize(left)vsk.
Step 9 — Prove correctness (iterative inorder + counter): Inorder visits values in strictly increasing order (BST property). The counter decrements by 1 on each visit and the k-th visit produces the value with rank exactly k (1-indexed). The walk stops immediately, so time is O(H) descent + O(k) pops = O(H + k). ∎
5. Progressive Hints
If stuck for 7 min, open hints.md. The early-termination is the easy part; the augmented-BST follow-up is where you should spend hint #5.
6. Deeper Insight — Order-Statistic Trees
The augmented BST technique is one of the most reusable data-structure ideas in interviews:
Add ONE extra field to each node whose value depends only on the subtree, AND can be maintained in O(1) per node during inserts/deletes (by propagating up the modified path). This converts queries that would require O(N) scans into O(H) walks.
Examples of this pattern:
- Subtree size → k-th smallest, rank of an element, “count of values < x.”
- Subtree sum → range sum on indexed data.
- Subtree max/min → range max/min queries.
- Subtree height → AVL tree balancing.
- Black-node count → Red-Black tree invariants.
The cost is one extra field and ~3 extra lines in insert/delete. The win is converting linear-scan queries into logarithmic ones. This is the augmentation argument — internalize it; it appears in Phase 3 (segment trees, Fenwick) and in every database index design.
7. Anti-Pattern Analysis
- Full inorder into a list, then
list[k-1]. Correct but wastes O(N) time and space. State as brute force; switch to early-terminating version. - Recursive inorder that doesn’t terminate early. Decrementing
kworks, butreturnfrom a recursive helper that already finished doesn’t stop the rest of the recursion unless you check a flag at each entry. Beginners write code that finds the answer but continues to walk the rest of the tree — fine for correctness, embarrassing for performance. - Off-by-one with 0-indexed vs 1-indexed. LC uses 1-indexed. Always restate.
- Augmenting size as “number of nodes in left subtree” instead of “total subtree size including this node.” Both are valid conventions but you must be consistent. “Total” is more common in textbooks; “left only” is more common in some implementations. State your choice in a comment.
- For the augmented follow-up: forgetting that you must maintain
sizeon EVERY insert/delete. A common bug: implementing the query correctly but not updating sizes on mutation. The next query returns garbage. Always pair augmentation logic with the mutation code that maintains it.
8. Skills & Takeaways
- Streaming traversal with early termination — the iterative inorder template is reusable across LC 230, 285, 99, 173.
- Order-statistic trees as a named pattern. Cite it by name in the follow-up — it signals you know the textbook (“Introduction to Algorithms” Ch. 14).
- Augmentation discipline: any extra field must be maintained in EVERY mutation. State this trade-off explicitly.
- Query/mutation balance: know when to invest preprocessing cost (augmentation) vs query cost (linear scan).
9. When to Move On
Move on when:
- You write iterative inorder with k-counter early termination from memory in <3 minutes.
- You can sketch the augmented-BST follow-up (
sizefield + descent logic) on a whiteboard in <5 minutes. - You can articulate the trade-off: “unaugmented = O(H+k) per query, no overhead on inserts; augmented = O(log N) per query, +O(1) bookkeeping per insert/delete.”
- Your stress test confirms agreement with a sorted-list oracle on 1000 random BSTs.
10. Company Context
Meta:
- This is a Meta favorite. The base problem is the warm-up; the augmented follow-up is the real interview. Often 30 min on follow-up, 10 on base.
- Watch for “implement the insert/delete that maintains size” — make sure you can do it.
- Scorecard phrase: “Algorithm — recognized augmentation pattern; implemented insert with O(1) size maintenance.”
Google:
- Will push on proof of correctness for the augmented descent (“prove that recursing left with the same k is correct when k ≤ size(left)”) and on average vs worst case complexity.
- Scorecard phrase: “Deep Dive — order-statistic tree; clean proof; honest about balancing requirement for log guarantees.”
Amazon:
- Common SDE-II round. Look for: “what if this is a billion-element tree on disk?” → augment a B-tree; same descent logic with
sizeper B-tree node. - Scorecard phrase: “Coding Excellence — clean iterative inorder; extended to large-scale via B-tree augmentation.”
Microsoft:
- May ask the kth largest version — easy modification: reverse-inorder (right-root-left).
- Scorecard phrase: “Problem Solving — generalized to kth largest; recognized symmetry.”
Bloomberg:
- Frames as “find the kth-most-traded stock in our order book” — order book BSTs are augmented with volume/size in production.
- Scorecard phrase: “Strong — connected to real Bloomberg data structures.”
TikTok / ByteDance:
- Often pairs with LC 295 (Find Median from Data Stream) in the same loop — both are “find rank/order-statistic in dynamic data” problems.
- Scorecard phrase: “Solid — recognized the order-statistic family across problems.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Base solution | Full inorder + list | Early-termination after prompt | Early-termination first try | Iterative inorder, names “streaming” pattern |
| Recognize follow-up | Surprised | Mentions caching | Mentions augmentation | Names “order-statistic tree” + cites CLRS Ch. 14 |
| Augmented insert | Doesn’t attempt | Forgets to maintain size | Maintains size on insert path | Maintains size + discusses rotation impact on AVL/RB |
| Trade-off discussion | None | Vague | Mentions space vs query | Quantifies: “log N per query vs N per query, +1 byte per node, +O(H) on mutation” |
| Generalization | None | None | Mentions LC 173 / 285 | Connects to Fenwick, segment-tree, database B-tree indexing |
12. Level Delta
Mid (L4):
- Writes recursive inorder solution after prompting for early termination.
- Discusses augmentation when explicitly asked.
Senior (L5):
- Writes iterative inorder with k-counter first try.
- Proactively raises the “many queries with updates” follow-up before being asked.
- Implements
sizeaugmentation correctly including the insert path.
Staff (L6):
- Frames the problem upfront as “this is asking about order statistics — am I optimizing for query or update?”
- Implements augmented BST cleanly; discusses why AVL/RB are needed for log-time worst case.
- Connects to Fenwick tree (for static order statistics on indexed data) and to database B+tree indexing.
Principal (L7+):
- Discusses concurrent order-statistic queries on a shared, mutating tree (snapshot isolation, MVCC).
- Identifies the meta-pattern: “augment with a subtree summary that is constant-time mergeable” — same idea drives segment-tree lazy propagation.
- Sketches the design of a database column index that supports rank/order queries in production (e.g., PostgreSQL B-tree with covering indexes).
13. Follow-up Q&A
Q1: What if the BST is modified frequently and you need many k-th smallest queries?
A: Augment each node with size = 1 + size(left) + size(right). Maintain on every insert/delete (size updates propagate up the modified path in O(H)). Then kth_smallest(node, k): let L = size(node.left). If k <= L, recurse left. If k == L+1, return node.val. Else recurse right with k - L - 1. Each query is O(H).
Q2: What if the tree could be unbalanced — what’s the worst-case? A: H = N for fully skewed BST (sorted insertion order). To get O(log N) worst-case, use a self-balancing variant (AVL, RB-tree, or Treap). The augmentation pattern is identical; only the rebalancing logic changes.
Q3: How would you find the rank of a given value (the inverse query)?
A: With augmentation: descend from root. If value < node.val, recurse left. If value > node.val, accumulate size(node.left) + 1 and recurse right. If value == node.val, return accumulated + size(node.left) + 1. O(H).
Q4: Find the median (the n/2-th smallest)? A: Same as kth-smallest with k = ceil(N/2). Requires knowing N — keep a tree-level counter or use root’s size if augmented.
Q5: How would you do this on a B-tree (each node has up to M children)? A: Augment each child pointer with the subtree-size below it. To find k-th: at each node, scan children left-to-right accumulating sizes; recurse into the first child whose cumulative size ≥ k, adjusting k. O(log_M N · M) per query.
14. Solution Walkthrough
See solution.py:
kth_smallest_iterative(root, k)— iterative inorder with k-counter early termination (the shippable form).kth_smallest_recursive(root, k)— recursive inorder with closure-captured counter + early flag.kth_smallest_brute_full_inorder(root, k)— full inorder into list (oracle for stress tests).OrderStatBSTclass — augmented BST withsizefield. Methods:insert,delete,kth_smallest,rank(the follow-up implementation).stress_test()— Part A: 1000 random BSTs; for each, run all k from 1..N, assert all 3 unaugmented variants agree. Part B: 1000 random insert/delete/query sequences against the augmentedOrderStatBST, cross-check with a sorted-list oracle.edge_cases()— single node, deepest k, left-skewed, right-skewed, augmented insert-delete-insert correctness.
Run: python3 solution.py → “ALL TESTS PASSED” on stdout.
15. Beyond the Problem
Principal-engineer code review comment:
“Base kth-smallest looks fine, ship it. About the augmented version — two flags. First, your
deleterebalances size on the deletion path but you’re using simple BST delete without rotations, so you’ve added O(1) bookkeeping per node touched but the tree itself degrades on adversarial input. If the spec promises log-time worst case you need AVL or RB rotations, and those rotations themselves need to re-augment size on the rotated nodes — that’s a 3-line change but if you forget it, queries silently return wrong answers for hours before someone notices. Add a unit test that callskth_smallest(k=mid)AFTERn/2random insertions andn/4random deletions, asserting against a sorted oracle. Second, the size field is an int; for a billion-node tree you’re fine, but for a serialized format please pickuint64explicitly because Python’sintis unbounded and that hides bugs that bite when ported to typed languages.”
Connections forward:
- p19 (Sorted Array → BST) builds a balanced BST — the natural pair with this problem.
- Phase 3 Fenwick trees and segment trees are augmentation patterns on arrays (vs trees here).
- LC 173 (BST Iterator) is the streaming version of the iterative inorder template here — same code, different interface.
- LC 315 (Count of Smaller Numbers After Self) uses an order-statistic BST as one valid solution.
- Database B+tree indexes in PostgreSQL/MySQL use this augmentation in production.
p18 — Hints
Open one at a time.
Hint 1 — What property of BST gives you “sorted order”?
Inorder traversal of a BST visits nodes in strictly ascending value order. So the k-th node visited (1-indexed) is the k-th smallest. You don’t need a full sort — just walk inorder.
Hint 2 — Don’t Walk the Whole Tree
If you do a full recursive inorder collecting every value into a list, you walk all N nodes even when k=1. Stop as soon as you’ve visited the k-th node. Maintain a counter; return early.
Hint 3 — Iterative Inorder + Counter (the shippable form)
stack, cur = [], root
while cur or stack:
while cur:
stack.append(cur); cur = cur.left
node = stack.pop()
k -= 1
if k == 0: return node.val
cur = node.right
Memorize this template. It reappears in LC 173, 285, 99.
Complexity: O(H) descent + O(k) pops + O(H) stack space = O(H + k) time, O(H) space.
Hint 4 — Pre-Empt the Follow-up
Before you finish, anticipate the interviewer’s next question:
“Suppose the BST is modified often (frequent insert/delete), and we run kth_smallest many times. How do you optimize?”
You have 30 seconds to identify the answer: augment each node with its subtree size.
Hint 5 — Augmented BST (Order-Statistic Tree)
Add a field size to each node, where size(N) = 1 + size(N.left) + size(N.right) (with size(None) = 0).
Maintain size on every insert/delete (recompute after the recursive call returns).
For kth_smallest(node, k):
left_size = size(node.left)
if k <= left_size: descend left, same k
elif k == left_size + 1: return node.val
else: descend right, k -= left_size + 1
Per query: O(H). Per mutation: O(H) including size maintenance. Combined with self-balancing (AVL/RB), this gives O(log N) per op worst-case.
This is called an order-statistic tree (CLRS Ch. 14). Naming it explicitly in interview signals you’ve read the textbook.
p19 — Convert Sorted Array to Binary Search Tree
Source: LeetCode 108 · Easy (the construction is short; the invariant is non-trivial) · Topics: Tree, BST, Divide & Conquer Companies (2024–2025 frequency): Amazon (high), Microsoft (high), Apple (medium), Meta (medium), Bloomberg (medium), Adobe (medium) Loop position: phone screen or first round; often paired with LC 109 (sorted linked list to BST) as the natural follow-up.
1. Quick Context
A sorted array IS the inorder traversal of some BST. There are exponentially many BSTs with that inorder, but the height-balanced one is uniquely (up to a left/right midpoint convention) determined by the divide-and-conquer rule:
Pick the middle element as root. Recursively build the left subtree from the left half. Recursively build the right subtree from the right half.
Three things make this problem interesting:
- The midpoint convention matters.
mid = (lo + hi) // 2(floor) andmid = (lo + hi + 1) // 2(ceiling) both give height-balanced BSTs but DIFFERENT trees. LC accepts either. State your choice. - The recurrence proves the height bound.
T(n) = T(⌊n/2⌋) + T(⌈n/2⌉) + O(1)gives O(N) build and resulting height ⌈log₂(n+1)⌉. - This is the inverse of p18’s inorder. Building from a sorted sequence ↔ reading a BST as a sorted sequence. The construction pattern (divide on middle, recurse) is the same skeleton as building a segment tree, merge sort, and balanced parenthesization DP.
What it looks like it tests: can you write a recursive constructor. What it actually tests: can you (a) state the invariant (“sorted array IS the inorder of a BST”), (b) recognize the divide-and-conquer skeleton, and (c) prove the balance property without hand-waving.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/convert-sorted-array-to-binary-search-tree/
STOP. 5-min timer. This is a fast solve once you see the divide-and-conquer; longer if you don’t.
3. Prerequisite Concepts
- p16 — the BST invariant
- p18 — inorder of a BST is sorted (this problem is the inverse construction)
- Divide-and-conquer recursion on array indices (see phase-01-foundations/labs/lab-07-binary-search-fundamentals.md)
- Master theorem (for the T(n) = 2T(n/2) + O(1) → O(N) derivation)
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given an integer array nums sorted in strictly ascending order, build a height-balanced BST whose inorder traversal is nums. Return the root. A height-balanced BST is one where the depths of the two subtrees of every node differ by at most 1.”
Step 2 — Clarify:
- “Strictly ascending — duplicates?” (LC: strictly ascending. No duplicates.)
- “Define height-balanced?” (Yes — abs(depth(left) - depth(right)) ≤ 1 at every node.)
- “Unique answer?” (No — LC accepts any valid height-balanced BST.)
- “Empty array?” (Return None.)
- “Should I mutate
nums?” (LC: doesn’t matter; just use indices.)
Step 3 — Constraints: N up to 10^4. O(N) construction. Resulting height = ⌈log₂(N+1)⌉.
Step 4 — Examples:
[-10,-3,0,5,9]→ root 0, left {-10, -3}, right {5, 9}.[]→ None.[1]→ single node.[1,2]→ root 1 with right 2 (or root 2 with left 1, depending on midpoint convention).[1,2,3,4,5,6,7]→ perfect balanced tree with root 4.
Step 5 — Brute Force: Insert one at a time into a vanilla BST? That degrades to a linked list (skewed depth N). Wrong — does not satisfy height-balance. State this as the wrong approach to not use.
Step 6 — Complexity (optimal): O(N) time (each index processed exactly once), O(log N) stack space (recursion depth = tree height = O(log N)).
Step 7 — Pattern Recognition: “Build a balanced structure from a sorted sequence by picking the middle” → divide-and-conquer on indices. Same skeleton as merge sort, segment tree build, binary search.
Step 8 — Develop:
def build(lo, hi): # half-open [lo, hi)
if lo == hi: return None
mid = (lo + hi) // 2
root = TreeNode(nums[mid])
root.left = build(lo, mid)
root.right = build(mid + 1, hi)
return root
return build(0, len(nums))
Step 9 — Prove balance: The recurrence on the height H(n) is H(n) = 1 + max(H(⌊n/2⌋), H(⌈n/2⌉)) with H(0) = -1. Solve: H(n) = ⌈log₂(n+1)⌉. Since the two subtrees have sizes ⌊n/2⌋ and ⌈n/2⌉ differing by at most 1, their heights differ by at most 1 (height is monotone in size for balanced builds). Therefore the tree is height-balanced. ∎
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Divide-and-Conquer on Indices
This problem is the cleanest example of the divide-and-conquer on sorted indices template:
def build(lo, hi):
if lo >= hi: return EMPTY # base case
mid = pick_middle(lo, hi)
return COMBINE(nums[mid], build(lo, mid), build(mid + 1, hi))
The same skeleton, with different pick_middle and COMBINE, gives:
| Problem | pick_middle | COMBINE |
|---|---|---|
| Sorted array → balanced BST (this) | (lo + hi) // 2 | TreeNode(val, left, right) |
| Merge sort | (lo + hi) // 2 | merge(left_sorted, right_sorted) |
| Segment tree build | (lo + hi) // 2 | Node(combine(left.agg, right.agg), left, right) |
| Quick sort (random pivot) | random | partition-then-recurse |
| Binary search | (lo + hi) // 2 | conditionally descend one side only |
Recognizing this skeleton means you have a reusable mental template for half the divide-and-conquer problems you’ll see at Phase 3+.
7. Anti-Pattern Analysis
- Insert one at a time into an empty BST. Sorted insertion degrades to a left/right skew → depth N → fails height-balance. Mention as the wrong approach.
- Off-by-one in the index range. Mixing inclusive/exclusive bounds (
[lo, hi]vs[lo, hi)) is the #1 bug. Pick one convention and use it consistently. (This README uses half-open[lo, hi).) - Allocating subarrays on each recursive call (
build(nums[:mid])). That’s O(N log N) extra work and O(N log N) memory. Pass indices, not slices. Mention this trade-off if asked. - Returning the wrong midpoint (forgetting to construct the BST around it). Some candidates accidentally write
build(lo, mid - 1)andbuild(mid + 1, hi)(inclusive[lo, hi]) withmid = (lo + hi) // 2— fine. Others mix conventions and end up double-countingmid. Pick one and stick to it. - Assuming the answer is unique. Both
(lo + hi) // 2and(lo + hi + 1) // 2produce valid height-balanced BSTs that differ when N is even. State your convention upfront.
8. Skills & Takeaways
- The divide-and-conquer-on-indices template — your reusable skeleton for merge sort, segment tree build, and BST construction.
- Pass indices, not slices — performance discipline that matters in production.
- Half-open intervals
[lo, hi)— the Knuth-recommended convention; less prone to off-by-one than[lo, hi]. - Inverse-of-inorder framing — sorted ↔ inorder of BST. This biconditional unlocks LC 109, 109 follow-ups, and serialize/deserialize problems.
- Proof of balance via height recurrence — the kind of derivation Google interviewers ask for explicitly.
9. When to Move On
Move on when:
- You can write the half-open
[lo, hi)version from memory in <2 minutes with zero off-by-one bugs. - You can prove the tree is height-balanced via the height recurrence.
- You can sketch LC 109 (sorted linked list to BST) and articulate the two approaches: (a) convert list to array first (O(N) space) or (b) inorder-construction trick (advance the list pointer in inorder order, O(1) extra space — the elegant solution).
- Your stress test validates the resulting tree is both a valid BST (via p16) AND height-balanced (via height-difference check at every node) on 1000 random sorted arrays.
10. Company Context
Amazon:
- Standard SDE-II warmup. Follow-up: LC 109 (sorted linked list → BST). Some Bar Raisers ask you to write both in the same hour.
- Watch for: “what if the array doesn’t fit in memory but is on disk?” → discuss external divide-and-conquer (read midpoint, recurse on halves via disk seeks).
- Scorecard phrase: “Coding Excellence — clean half-open indices; correctly proved balance.”
Microsoft:
- Frames as “build a balanced index from a sorted column.” Will probe “what’s the height — exactly?” Answer:
⌈log₂(N+1)⌉. - Scorecard phrase: “Algorithm — divide-and-conquer; precise height analysis.”
Apple:
- Asks the in-place variant: “build the BST without allocating new TreeNodes — reuse the array as an implicit structure.” Requires implicit-tree-via-index thinking (parent at
i, children at2i+1,2i+2). Niche but Apple-friendly. - Scorecard phrase: “Strong — handled in-place variant; understood implicit-tree representation.”
Meta:
- Often pairs with LC 109. May ask: “modify so the BST is also an AVL — would your construction need rotations?” Answer: no, the divide-by-middle construction is already height-balanced (in fact, weight-balanced), which implies AVL-balanced. Free win.
- Scorecard phrase: “Algorithm — recognized weight-balance implies height-balance; no rotations needed.”
Google:
- Will demand the height proof. Also: “what changes if
numsallowed duplicates?” → answer: pick a side convention (e.g., equals go right), still height-balanced. - Scorecard phrase: “Deep Dive — full balance proof; cleanly handled duplicates side-convention.”
Bloomberg:
- Framed as “build a balanced tree over a sorted instrument list for log-time lookup.” Follow-up: “how do you handle daily insertions?” → discuss when to rebuild vs incrementally insert (rebuild when imbalance > threshold).
- Scorecard phrase: “Solid — connected to amortized index maintenance.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | Sorted insert (skewed) | Picks middle, off-by-one | Picks middle, correct first try | Picks middle + names “divide on midpoint” pattern |
| Index convention | Mixes inclusive/exclusive | Settles on one after bug | Half-open from the start | Half-open + states convention upfront in a comment |
| Balance proof | “Trust me” | Hand-waves | States recurrence | Solves recurrence to ⌈log₂(N+1)⌉ |
| LC 109 follow-up | Stuck | “Convert to array first” | Both approaches | Implements O(1)-extra-space inorder-pointer trick |
| Generalization | None | None | Notes merge-sort similarity | Names divide-on-indices skeleton; applies to segment tree |
12. Level Delta
Mid (L4):
- Writes the recursive construction with one off-by-one debug pass.
- States “tree is balanced” without formal proof.
Senior (L5):
- Writes correct half-open construction first try.
- States the height recurrence and gives the closed-form ⌈log₂(N+1)⌉.
- Sketches LC 109 verbally.
Staff (L6):
- Frames upfront: “this is the inverse of inorder; divide-on-midpoint.” States balance proof.
- Implements LC 109 with the O(1)-extra-space pointer trick.
- Discusses when to use weight-balanced vs height-balanced vs AVL/RB (rebuild cost vs incremental cost).
Principal (L7+):
- Generalizes to the divide-and-conquer skeleton across merge sort, segment tree, and balanced parenthesization.
- Discusses external-memory variant for N too large for RAM (B-tree-style midpoint-based bulk loading).
- Connects to cache-oblivious trees (van Emde Boas layout) for memory-hierarchy performance.
13. Follow-up Q&A
Q1: How do you do this for a sorted linked list (LC 109)? A: Two approaches.
- (a) Walk the list once to build an array, then call this problem’s solution. O(N) time, O(N) extra space.
- (b) In-order construction with a list pointer: compute the length first. Recursively build the tree’s left subtree (which consumes the first half of the list), then read the current list head as this node’s value (advance pointer), then build the right subtree. The list pointer is consumed in inorder. O(N) time, O(log N) stack space, O(1) extra. The elegant answer.
Q2: How do you build the BST in-place using the array’s memory (no TreeNode allocations)?
A: Use implicit-tree-via-index representation: node i has children 2i+1, 2i+2. The construction is exactly the same divide-and-conquer; output is a heap-style array. Trades clarity for cache locality and zero allocation.
Q3: What if nums has duplicates?
A: Decide a side convention (e.g., duplicates go right). The midpoint-divide still works; height balance still holds. State the convention in a comment because callers will assume one or the other.
Q4: Could you build the BST iteratively?
A: Yes — use an explicit stack of (lo, hi, parent_node, is_left_child) quadruples. The divide-and-conquer skeleton flattens trivially. Same O(N) time, but trades stack memory for explicit-stack heap memory.
Q5: How would you parallelize this for very large nums?
A: The two recursive calls (left and right halves) are independent — pure functional divide-and-conquer. Trivially parallel with a work-stealing executor. The recursion tree has depth O(log N) so parallel speedup is bounded by available cores up to ~N/threshold.
14. Solution Walkthrough
See solution.py:
sorted_array_to_bst(nums)— half-open[lo, hi)recursive divide-and-conquer (the canonical form).sorted_array_to_bst_inclusive(nums)—[lo, hi]inclusive variant (also correct; for comparison).sorted_array_to_bst_iterative(nums)— explicit-stack iterative form for the “no recursion” follow-up.sorted_list_to_bst(head)— LC 109 with the in-order pointer trick (O(1) extra space follow-up).- Validation helpers:
is_valid_bst(root),is_height_balanced(root),inorder_values(root). stress_test()— generates 1000 random sorted arrays; for each, verifies all 3 array-based builds produce (a) a valid BST, (b) a height-balanced tree, (c) inorder == input. Also stress-tests LC 109 against the array version.edge_cases()— empty, single, two-elem, three-elem, perfect 2^k - 1 sizes.
Run: python3 solution.py → “ALL TESTS PASSED” on stdout.
15. Beyond the Problem
Principal-engineer code review comment:
“The recursion is fine, but consider whether you actually need to allocate
TreeNodeobjects at all. If this is used to build a static lookup tree that’s queried millions of times and never mutated, the heap-style array layout (children[i] = 2i+1, 2i+2) is ~2× faster on modern CPUs because it’s cache-coherent — the pointer-based tree thrashes the L1 cache after a few hundred nodes. Also: please name your indicesloandhi, notstartandend, because the next reader will spend 30 seconds figuring out ifendis inclusive or exclusive. Convention beats cleverness. Last note: themid = (lo + hi) // 2overflows in C/Java for very largehi; in Python you’re safe, but add a# (lo + (hi - lo) // 2) in non-bignum languagescomment if this might be ported. Cheap insurance against a future overflow bug that’s hellish to debug.”
Connections forward:
- p20 (BFS level order) is the output form of this tree (level-by-level read).
- Phase 3 segment trees use the same divide-on-midpoint build with
O(N)array-backed implicit tree. - Phase 5 DP problems on intervals (Matrix Chain, Burst Balloons) use the same
build(lo, hi)skeleton. - LC 109 (sorted linked list → BST) is the canonical follow-up — drill it.
- LC 1382 (Balance a BST) is the generalization: given an arbitrary BST, output a balanced one (inorder to array, then apply this algorithm).
p19 — Hints
Open one at a time.
Hint 1 — Reframe
A sorted array IS the inorder traversal of some BST. The question is: which one is height-balanced? Answer: pick the middle as the root; the left half becomes the left subtree, the right half becomes the right subtree. Recurse.
Hint 2 — Divide and Conquer on Indices
Don’t allocate subarrays — pass indices. Write a helper build(lo, hi):
if range empty: return None
mid = (lo + hi) // 2 # or // 2 + 1 — either works
root = TreeNode(nums[mid])
root.left = build(left half)
root.right = build(right half)
return root
Pick a convention — half-open [lo, hi) or inclusive [lo, hi] — and stick to it. Mixing is the #1 bug here.
Hint 3 — Why Is It Height-Balanced?
The two recursive calls operate on subarrays of size ⌊n/2⌋ and ⌈n/2⌉. These differ by at most 1. By induction, their heights differ by at most 1, so the tree is height-balanced.
Closed form: height = ⌈log₂(n+1)⌉.
Hint 4 — Don’t Insert One-By-One
The tempting wrong approach: “start with an empty BST and insert each element in order.” That gives a fully right-skewed tree (depth N). Mention this as the wrong approach and dismiss it.
Hint 5 — Follow-up: LC 109 (Sorted Linked List → BST)
The naive approach: walk the list into an array, then call this problem’s solution. O(N) time, O(N) extra space.
The elegant approach: inorder pointer trick.
- Count list length
n. - Recursively build with
build(size):- Build the left subtree first (consuming the first
size//2elements). - Read the current list pointer’s value → this node’s value; advance the pointer.
- Build the right subtree.
- Build the left subtree first (consuming the first
The list pointer is consumed in inorder. O(N) time, O(log N) stack, O(1) extra space.
This trick — “consume the input in inorder during construction” — is the elegant move. Practice it.
p20 — Binary Tree Level Order Traversal
Source: LeetCode 102 · Medium · Topics: Tree, BFS, Queue Companies (2024–2025 frequency): Amazon (very high), Meta (very high), Google (high), Microsoft (high), Bloomberg (high), Apple (medium), LinkedIn (medium), TikTok/ByteDance (medium) Loop position: very common phone screen or first round; opens an entire family — LC 103 (zigzag), 107 (bottom-up), 199 (right side view), 314 (vertical order), 116 (next right pointers).
1. Quick Context
This is the canonical BFS template for trees. Output is a list of lists where each inner list contains the values at one depth level, top to bottom, left to right.
The whole problem hinges on ONE trick: freeze the queue’s size before processing each level. Without that, BFS works but you can’t distinguish where one level ends and the next begins.
while queue:
n = len(queue) # ← snapshot
level = []
for _ in range(n): # ← drain exactly n nodes; the queue may grow during this loop
node = queue.popleft()
level.append(node.val)
if node.left: queue.append(node.left)
if node.right: queue.append(node.right)
result.append(level)
That snapshot pattern is the entire problem. Once you have it, you have LC 103, 107, 199, 116, 117 — they’re all variations on what you do with the level.
What it looks like it tests: can you do BFS. What it actually tests: do you internalize the “snapshot the queue size before draining” pattern, which appears in every level-aware graph BFS (Word Ladder, Rotting Oranges, Open the Lock, Bus Routes, Shortest Path in Binary Matrix).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/binary-tree-level-order-traversal/
STOP. 10-min timer. Don’t peek at hints; this is the gateway problem to every BFS-flavored question.
3. Prerequisite Concepts
- p15 — recursive tree traversal (we contrast against it here)
- phase-01-foundations/labs/lab-05-stack-queue-applications.md —
collections.dequeas O(1)-pop queue - phase-01-foundations/labs/lab-09-tree-traversal-fundamentals.md — why BFS vs DFS
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given the root of a binary tree, return its node values level by level, top to bottom, left to right, as a list of lists.”
Step 2 — Clarify:
- “Empty tree?” (Return
[]— not[[]].) - “Single node?” (Return
[[root.val]].) - “Within a level, ordering left-to-right matters?” (Yes — LC enforces this.)
- “Should the output type be
List[List[int]]exactly?” (Yes.)
Step 3 — Constraints: Up to 2000 nodes. O(N) time, O(W) space where W = maximum width of the tree (could be N/2 for a complete tree).
Step 4 — Examples:
[3,9,20,null,null,15,7]→[[3],[9,20],[15,7]].[1]→[[1]].[]→[].- Left-skewed
1→2→3→[[1],[2],[3]]. - Right-skewed
1→2→3→[[1],[2],[3]].
Step 5 — Brute Force: DFS with depth tracking — recurse with (node, depth), append node.val into result[depth] (allocate a new sublist when depth == len(result)). This is genuinely O(N) and works. Mention it; ship the BFS as primary because it matches the problem’s intent and generalizes cleanly to graph problems.
Step 6 — Complexity: O(N) time (each node enqueued and dequeued once), O(W) space for the queue at the widest level (worst case W = N/2 for a complete tree).
Step 7 — Pattern Recognition: “Process layer by layer” → BFS with level-size snapshot. This is the gateway pattern. It scales unchanged to graph BFS, multi-source BFS, and bidirectional BFS.
Step 8 — Develop:
from collections import deque
def level_order(root):
if not root: return []
out, q = [], deque([root])
while q:
n = len(q)
level = []
for _ in range(n):
node = q.popleft()
level.append(node.val)
if node.left: q.append(node.left)
if node.right: q.append(node.right)
out.append(level)
return out
Step 9 — Test: Empty, single, balanced canonical example, left-skewed, right-skewed.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why the Snapshot Matters
If you omit n = len(q) and just drain until empty, you can still recover the level structure by attaching a depth field to every queued node (q.append((child, depth + 1))) and bucketing the results by depth. That works but pays an O(N) memory cost for the depths AND requires you to bucket after the fact.
The snapshot trick is structurally cleaner: it uses the queue’s instantaneous size as an implicit level marker. The BFS frontier always contains exactly the nodes at the current depth (plus possibly some from the next, mid-drain). Snapshotting captures “exactly this layer” without per-node bookkeeping.
This generalizes to:
- LC 127 Word Ladder — same snapshot to count BFS distance (the answer is the level index where we first hit
endWord). - LC 994 Rotting Oranges — multi-source BFS where each “level” = one minute.
- LC 752 Open the Lock — BFS over combination space; answer = level we first reach target.
- LC 433 Minimum Genetic Mutation — same shape as Word Ladder.
The “level = step count” interpretation is what makes BFS the shortest-path algorithm for unweighted graphs. This problem teaches you that pattern.
7. Anti-Pattern Analysis
- Using a
listas a queue.list.pop(0)is O(N). Usecollections.deque—popleft()is O(1). Junior-level mistake; named at interview. - DFS with no depth tracking. If you DFS without recording depth, you can’t group by level. The fix is
dfs(node, depth)writing intoresult[depth]. Acceptable as alternative; mention it. - Mutating
len(q)inside the for-loop.for _ in range(len(q))is correct (n is captured at loop start). Butwhile len(q) > 0inside the inner loop would mix levels. State the snapshot pattern explicitly. - Recursive BFS. “Recursive BFS” is a contradiction in terms — recursion uses an implicit stack (LIFO), BFS needs a queue (FIFO). Anyone claiming “recursive BFS” actually means DFS with depth tracking. Be precise.
- Returning
[[]]for empty input. Read the problem: empty tree →[], not[[]]. Off-by-one error.
8. Skills & Takeaways
- The BFS level-size snapshot pattern — your single most-used template in interview BFS problems.
collections.dequefor O(1) queue operations — never use a list as a queue.- DFS+depth as the symmetric alternative — gives the same result with implicit-stack memory and trivial to code recursively.
- “BFS level = distance” — the conceptual key to shortest-path algorithms on unweighted graphs.
- Reading output schemas carefully —
[]vs[[]]distinguishes a careful candidate from a sloppy one.
9. When to Move On
Move on when:
- You can write the BFS snapshot version from memory in <90 seconds.
- You can write the DFS+depth alternative on a whiteboard without bugs.
- You can do LC 199 (Right Side View — return the rightmost element of each level) as a 30-second variation.
- You can do LC 103 (zigzag) — flip every other level — without restructuring the algorithm.
- You can do LC 107 (bottom-up) by either reversing at the end or pre-pending into the result.
- Your stress test confirms BFS and DFS produce identical level lists on 1000 random binary trees.
10. Company Context
Amazon:
- Asks this verbatim almost every cycle for SDE-I / SDE-II. Bar Raisers often pivot to LC 199 (right side view) or LC 314 (vertical order) as the follow-up.
- Watch for: “now output one string per level, comma-separated” — trivial change, tests whether you panic at format tweaks.
- Scorecard phrase: “Coding Excellence — clean BFS template; correctly distinguished
[]from[[]].”
Meta:
- Will demand LC 199 as the immediate follow-up. Some interviewers ask LC 116 (next right pointers) instead — same algorithm, you wire
prev.next = currduring the level drain. - Scorecard phrase: “Strong — recognized snapshot pattern; chained to LC 199/116 without restructuring.”
Google:
- May skip 102 entirely and start with LC 314 (vertical order traversal) which composes BFS-by-level with column tracking. Knowing this problem fluently is the entry ticket.
- Scorecard phrase: “Pattern Recognition — generalized level-BFS to column-indexed BFS for vertical order.”
Microsoft:
- Often paired with LC 103 (zigzag). Asks “can you do it without an explicit
reverse?” → answer: alternateappendleft/appendon the level list. Shows queue-vs-deque thinking. - Scorecard phrase: “Algorithm — eliminated unnecessary reverse via deque trick.”
Bloomberg:
- Frames as “render a financial tree (parent positions → child positions) level by level for UI display.” Output format is verbose; tests pragmatism.
- Scorecard phrase: “Pragmatic — clean BFS adapted to UI rendering output.”
Apple:
- Asks “what if the tree is enormous and you want to stream level-by-level instead of materializing the whole result?” → answer: yield each
levelas a generator (yield levelinstead ofout.append(level)). Memory becomes O(W) instead of O(N). - Scorecard phrase: “Solid — recognized streaming / generator opportunity.”
LinkedIn:
- Common Q4 / Q5 of phone screen. Asks LC 102 first, LC 107 (bottom-up) second.
- Scorecard phrase: “Standard — solved both 102 and 107 fluently.”
TikTok/ByteDance:
- Goes straight to LC 199 (right side view). Reject if you can’t transcribe the BFS template under pressure.
- Scorecard phrase: “Pass — produced right-side-view at first attempt.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Data structure | list (uses pop(0)) | deque | deque + names “BFS frontier” | deque + names amortized O(1) and snapshot pattern |
| Level boundary | depth field per node | n = len(q) snapshot | snapshot, named | snapshot + explains why it generalizes to shortest-path BFS |
| Edge case | misses [] → [[]] | handles | handles + asks | handles + suggests generator for streaming |
| Follow-ups | stuck on LC 199 | does LC 199 | does LC 199 + 103 fluently | does 199, 103, 116, 314 from same template |
| Generalization | none | none | “this is BFS distance” | names “BFS level = shortest path in unweighted graph”; bridges to Dijkstra |
12. Level Delta
Mid (L4):
- Writes BFS with deque + snapshot first try.
- Handles empty case correctly.
- Knows LC 199 is “return last of each level”.
Senior (L5):
- Names the snapshot pattern explicitly.
- Sketches DFS+depth as alternative within 30 seconds when asked.
- Fluent in 102 → 199 → 103 progression.
Staff (L6):
- Articulates “BFS level = shortest path in unweighted graph” without prompting.
- Suggests generator-based streaming for memory-bounded environments.
- Walks through LC 116 (next right pointers) using the same template + level-internal wiring.
Principal (L7+):
- Connects to graph BFS family (Word Ladder, Rotting Oranges, Open the Lock).
- Discusses when BFS exceeds practical memory (very wide trees) and the trade-off with iterative deepening DFS.
- Mentions the parallel BFS variant — frontier as a level-by-level synchronization point — and its relevance to GPU graph algorithms.
13. Follow-up Q&A
Q1: Without using BFS, can you solve this with DFS?
A: Yes. Recurse with (node, depth); on entry, if depth == len(result), append a new empty list to result; then append node.val to result[depth]. O(N) time, O(H) stack. Produces identical output. Trade-off: deeper trees → more stack; very wide trees → BFS frontier exceeds DFS stack. Choose by tree shape if memory is bounded.
Q2: LC 199 — return only the rightmost element of each level. A: Two ways.
- BFS: same template, but
level[-1]is the answer for each level. Or for memory: keep only the lastnode.valseen in each level-drain. - DFS: visit right-before-left; track depth; the first node seen at each depth IS the right-side. Single pass, O(H) memory.
Q3: LC 103 (zigzag) — alternate left-to-right and right-to-left per level.
A: Track an lr_flag boolean. Same BFS; on odd levels, build the level list using appendleft on a deque (or reverse() at end). Same O(N) / O(W).
Q4: LC 116 — populate next pointers to the right neighbor at each level.
A: Modify the level drain: keep prev = None at level start; for each node popped, set prev.next = node (if prev), then prev = node. End of level: prev.next = None (or just leave None as default). Same template; the only addition is in-loop wiring.
Q5: The tree has billions of nodes — you can’t materialize the result. What’s the change?
A: Convert level_order to a generator: yield level instead of out.append(level). Caller iterates and processes one level at a time. Memory drops from O(N) to O(W). This is the streaming pattern that unlocks LC-style problems for true big-data scale.
14. Solution Walkthrough
See solution.py:
level_order_bfs(root)— canonical BFS withdeque+ level-size snapshot (ship this).level_order_dfs(root)— DFS+depth alternative (recursive, indexes intoresult[depth]).right_side_view(root)— LC 199 follow-up via BFS (last-in-each-level).zigzag_level_order(root)— LC 103 follow-up via BFS + alternating direction.build_tree_from_list(vals)— level-order list → TreeNode (usesis_left: booltoggle pattern — does NOT set attributes onTreeNodebecause__slots__). Defensive pattern; see p16 for the lesson.stress_test()— 1000 random binary trees; BFS == DFS, every time.edge_cases()— empty, single, left-skewed, right-skewed, canonical LC example.
Run: python3 solution.py → “ALL TESTS PASSED”.
15. Beyond the Problem
Principal-engineer code review comment:
“The BFS is correct, but a few things. First: capture
n = len(q)into a local before the for-loop — not inrange(len(q))re-call — because whilerangeis evaluated once, the explicit local variable signals intent to the reader and reads better in code review. Second: please usecollections.deque, notlist.pop(0)— I’ve seen this exact code thrash production at 100k nodes. Third: have you consideredyielding each level? Half the time we want to stream this output (e.g., paginating UI rendering) and the materialized result is wasted. Fourth: the test helper that builds a tree from a level-order list should NOT set attributes onTreeNode— we use__slots__for memory; any code that sets.fooon a TreeNode silently breaks under__slots__. Use external state (a bool flag, a dict from parent → child-count). Last: in production this would be parameterized on aTreeprotocol, not concreteTreeNode— but for an interview, this is fine. Ship it.”
Connections forward:
- Phase 4 — graph BFS uses this exact template (queue + visited set + snapshot for distance).
- Phase 4 — multi-source BFS (LC 994 Rotting Oranges) is BFS where the initial queue contains all sources.
- Phase 4 — Word Ladder (LC 127) is BFS where each “level” is one mutation.
- Phase 4 — 0-1 BFS uses a deque for two-cost BFS (close cousin of this).
- LC 116/117 (next right pointers) are direct applications of the level-drain template.
- LC 314 / 987 (vertical order traversal) compose this template with column tracking.
This is the gateway problem. Drill it cold.
p20 — Hints
Open one at a time.
Hint 1 — BFS, Not DFS
Output is grouped by depth, top to bottom. DFS visits in pre/in/post order, which mixes depths. BFS visits in depth order naturally. Reach for BFS.
Hint 2 — Queue
BFS needs FIFO. In Python: from collections import deque. Use popleft() and append() — both O(1). Never use list.pop(0) — that’s O(N) per call and will tank performance on large inputs.
Hint 3 — The Level Boundary Problem
If you BFS naively, you get all node values in level order but as a flat list. How do you know where one level ends and the next begins?
Answer: snapshot the queue’s size at the start of each level.
while queue:
n = len(queue) # ← snapshot
level = []
for _ in range(n): # drain exactly n nodes
node = queue.popleft()
level.append(node.val)
for child in (node.left, node.right):
if child: queue.append(child)
out.append(level)
When you start the inner loop, queue contains exactly the current level. By the time the inner loop finishes, you’ve drained that level and the queue now contains the next level. The snapshot n is the boundary marker.
Hint 4 — Why Snapshot Beats Per-Node Depth
You could tag every queued node with its depth: q.append((child, depth + 1)). Then bucket by depth at the end. That works but pays O(N) memory for the depths AND adds a bucketing pass.
The snapshot trick uses the queue’s instantaneous size as an implicit level marker. Cleaner. This is the pattern interviewers want to see.
Hint 5 — DFS Alternative
If you must use DFS (e.g., recursion-only environment):
def dfs(node, depth):
if not node: return
if depth == len(result): result.append([])
result[depth].append(node.val)
dfs(node.left, depth + 1)
dfs(node.right, depth + 1)
Same output. O(N) time, O(H) stack. Mention this as the alternative when interviewers ask “can you do it recursively?” — the answer is “with depth tracking, yes; that’s effectively DFS-by-depth, not BFS.”
Be precise: there is no such thing as “recursive BFS.” Recursion uses an implicit LIFO stack; BFS needs a FIFO queue. The recursive version is DFS-with-depth-tracking that happens to produce the same level-grouped output.
Month 2 — Patterns Mastery
Theme: Stop solving problems case-by-case. Start recognizing the pattern, then dispatching the right template. Weeks 5–8 trade depth on individual data structures for breadth of patterns — graphs, two-pointers, sliding windows, monotonic stacks, intervals, backtracking, trie, two-heaps. By month’s end, you should be able to look at a problem statement and call out the pattern name within 30 seconds.
Weekly breakdown
-
Week 5 — Graphs Intro: BFS, DFS, and the Topological Mindset (README) Number of Islands · Clone Graph · Course Schedule · Pacific Atlantic · Word Ladder
-
Week 6 — Two Pointers, Sliding Window, Prefix Sums (planned)
-
Week 7 — Monotonic Stacks, Intervals, Greedy (planned)
-
Week 8 — Backtracking, Trie, Two Heaps (planned)
Milestones
By end of Month 2:
- Recognize the 8 core algorithmic patterns from problem statement alone (BFS/DFS, two-pointer, sliding window, prefix sum, monotonic stack, interval-sort, backtracking, two-heap)
- Write canonical BFS / DFS / topo sort / multi-source BFS templates from memory
- Solve LC Medium graph + sliding-window problems in 25 minutes
- Articulate when each pattern applies AND when it doesn’t (the “negative space” matters as much)
Required cross-references
phase-04-graphs/labs/— every Week 5 problem cross-references a lab herephase-02-patterns/labs/lab-01-two-pointers.md— Week 6 base readingFRAMEWORK.md— every problem walks the 9-step frameworkSPACED_REPETITION.md— re-solve schedule for prior weeks during this month
Week 5 — Graphs Intro: BFS, DFS, and the Topological Mindset
Theme: A graph is just nodes + edges, but the mental shift from trees to graphs is the visited set. Without it, every graph algorithm loops forever. Every problem this week is, at its core, “the right traversal with the right state.” Time budget: ~8 hours focused work; ~14 hours including the LC bank. Hard rule: every graph problem solved twice — once with BFS, once with DFS. State which version you’d ship and why (memory shape, recursion depth, level-distance need).
Why this week
Weeks 3–4 gave you tree recursion fluency. Trees are connected, acyclic, and rooted — three luxuries that disappear in graphs. This week strips each one away:
- Implicit graphs are the norm. Most interview “graph” problems are NOT given an adjacency list. The grid in Number of Islands, the prerequisite list in Course Schedule, the word-ladder dictionary — you’re handed input that encodes a graph. Recognizing the implicit graph is half the problem. The faster you reframe “this is a graph where nodes are X and edges are Y,” the faster you solve it.
- The visited set is non-negotiable. Trees can’t cycle. Graphs can. Every BFS/DFS this week needs explicit cycle-prevention. The mistake of “forgetting visited” causes 30% of failed graph attempts in real interviews. Burn the discipline in this week.
- BFS distance = unweighted shortest path. Once you internalize that the level number in BFS is the distance from source, half of “shortest path” problems become BFS + level snapshot — the exact pattern from p20. Word Ladder, Rotting Oranges, Open the Lock — they’re all the same skeleton.
- Topological sort = “cycle detection + ordering” combo. Course Schedule is the gateway. Kahn’s BFS variant and DFS-coloring variant are both ~20 lines; both are reusable templates that recur in build systems, scheduler kernels, and ML pipelines.
- Multi-source BFS is a 1-line variant. Pacific Atlantic teaches the inversion trick — instead of asking “can water from each cell reach the ocean?” (NM separate BFSes), ask “from the ocean, which cells can be reached?” (one BFS per ocean from all border cells simultaneously). This kind of reframing is what distinguishes Staff from Senior.
By Friday you should: write BFS and DFS on a grid from memory in under 90 seconds each, implement both Kahn’s and DFS-coloring topo sort, articulate why bidirectional BFS halves Word Ladder runtime, and recognize the “multi-source BFS from boundary” pattern.
Flagship problems (the 5 you OWN this week)
| # | Problem | LC | Difficulty | Why this one |
|---|---|---|---|---|
| p21 | Number of Islands | 200 | Medium | The canonical grid-as-graph problem. Connected components via DFS or BFS. Every FAANG asks this. |
| p22 | Clone Graph | 133 | Medium | Forces you to handle cycles explicitly via memo map. Both DFS and BFS variants are instructive. |
| p23 | Course Schedule | 207 | Medium | Cycle detection + topological sort. Kahn’s BFS vs DFS-3-color. Gateway to build systems. |
| p24 | Pacific Atlantic Water Flow | 417 | Medium | Multi-source BFS from boundary + the inversion trick. Teaches “reframe to avoid N independent searches.” |
| p25 | Word Ladder | 127 | Hard | BFS on an implicit graph; pattern-bucket adjacency trick; bidirectional BFS for the follow-up. |
LC bank (~25 problems for spaced repetition)
Grid traversal (connected components, flood-fill, area):
- LC 200 (p21), LC 695 Max Area of Island, LC 463 Island Perimeter, LC 733 Flood Fill, LC 1254 Closed Islands, LC 130 Surrounded Regions, LC 1020 Number of Enclaves
Graph cycle / topo sort:
- LC 207 (p23), LC 210 Course Schedule II, LC 269 Alien Dictionary (Hard), LC 802 Find Eventual Safe States, LC 310 Minimum Height Trees
Multi-source BFS / boundary BFS:
- LC 417 (p24), LC 994 Rotting Oranges, LC 1162 As Far from Land as Possible, LC 542 01 Matrix
Implicit-graph BFS shortest path:
- LC 127 (p25), LC 126 Word Ladder II (all shortest paths — Hard), LC 752 Open the Lock, LC 433 Min Genetic Mutation, LC 854 K-Similar Strings (Hard)
Graph cloning / serialization:
- LC 133 (p22), LC 138 Copy List with Random Pointer (same memo-map pattern), LC 1485 Clone Binary Tree with Random Pointer
Readiness gate (you should be able to answer ALL these out loud before moving on)
- What’s the visited set, and why can’t graphs skip it? Because graphs can have cycles; without marking visited, BFS/DFS enters an infinite loop. Trees can’t cycle, so they never needed it.
- When do you choose BFS over DFS, and vice versa? BFS when you need shortest distance in an unweighted graph, or when the answer is “level-by-level” (Pacific Atlantic uses level-by-level reachability). DFS when you need cycle detection (use 3-color), connected components on a non-distance question (Number of Islands), or when the recursion depth is bounded (deeper-than-wide problems).
- What’s the canonical grid-BFS template?
q = deque([start]),visited = {start}, while q: pop, for each of 4 neighbors that’s in-bounds and unvisited, mark visited and enqueue. Memorize cold. - In topological sort, what’s the difference between Kahn’s BFS variant and DFS-coloring variant? Kahn’s: count in-degrees, start with all in-degree-0 nodes, decrement in-degrees as you “remove” nodes; if final order has fewer than N nodes → cycle exists. DFS-coloring: WHITE/GRAY/BLACK; entering GRAY → graph already explored → cycle.
- Why does multi-source BFS (Pacific Atlantic, Rotting Oranges) work? Because BFS expands by distance from the frontier; initializing the frontier with multiple sources is equivalent to running |sources| BFSes in parallel and merging the wavefronts. Same O(V+E), no double work.
- What’s the pattern-bucket trick in Word Ladder? Adjacency by edit-distance-1 is implicit. Instead of comparing every pair (O(N²·L)), bucket each word by “wildcarded” versions (
h*t,ho*,*ot) so two words sharing a bucket are 1-edit-distance neighbors. O(N·L²) preprocessing, fast neighbor lookup. - What does bidirectional BFS buy you? It searches simultaneously from start and end, meeting in the middle. Since BFS branching factor is
band shortest path length isd, naive BFS visitsO(b^d)nodes; bidirectional visits2 * O(b^(d/2))=O(b^(d/2)). Square-root speedup. Worth knowing as a follow-up for Word Ladder. - What does “implicit graph” mean, and what’s the cost? The graph isn’t materialized as an adjacency list; you compute neighbors on the fly from input. Cost: neighbor enumeration may be expensive (Word Ladder’s L²-per-neighbor without bucketing). Benefit: no
O(V+E)memory to store the graph.
If you can’t articulate all 8, the week is not done.
Recurring traps
- Forgetting visited — infinite loop on the first cycle. Mark visited when enqueueing, not when dequeuing. (Dequeue-marking lets duplicates accumulate in the queue.)
- Mutating the input grid as a visited marker — fine for interviews IF you state it (“I’m flipping
'1'to'0'to mark visited; the input is destroyed”) OR if you restore it. Otherwise the interviewer flags “destructive operation on shared state.” - Recursion depth exhaustion on giant grids — DFS on a 1000×1000 all-
'1'grid hits Python’s recursion limit. Usesys.setrecursionlimit(...)or switch to iterative BFS/DFS-with-explicit-stack. - Forgetting to mark the start as visited before entering BFS — causes re-enqueueing of the source. Always:
q = deque([start]); visited = {start}together. - Off-by-one with neighbor directions — write
DIRS = ((0,1),(0,-1),(1,0),(-1,0))once at module top; never inline. - Topo sort returns partial result on cyclic input — Kahn’s silently emits a short list when there’s a cycle. Always check
len(result) == Nand raise/return appropriately.
Stretch goals (only if everything above is solid)
- LC 126 — Word Ladder II. Return all shortest transformation sequences. Builds parent-map during BFS, then DFS-backtracks. Hard.
- LC 269 — Alien Dictionary. Topo sort on letters derived from word ordering. Easy to fail edge cases (prefix issue, single letters).
- LC 815 — Bus Routes. BFS where each “level” = one bus ride. Implicit graph where buses are nodes, stops are hyperedges.
Pointer to the first problem
Start with p21 — Number of Islands. It’s the gentlest grid-as-graph entry; everything else this week builds on it.
p21 — Number of Islands
Source: LeetCode 200 · Medium · Topics: DFS, BFS, Union-Find, Matrix Companies (2024–2025 frequency): Amazon (very high), Meta (very high), Google (high), Microsoft (high), Bloomberg (high), Apple (high), LinkedIn (medium), Salesforce (medium), Goldman Sachs (medium) Loop position: absolutely standard phone screen or first onsite round. Possibly the most-asked graph question on the planet.
1. Quick Context
A 2D grid of '1's (land) and '0's (water). Count connected components of land cells (4-connectivity).
This is the canonical grid-as-implicit-graph problem. Three things make it interview-defining:
- It looks trivial. Anyone can hack a working solution. But the interview signal lives in how cleanly you handle: visited marking, recursion-depth blow-up on large grids, the iterative-vs-recursive trade-off, and whether you mutate the input or not (and announce it).
- It’s the entry point to an entire family. Max Area of Island (LC 695), Island Perimeter (LC 463), Surrounded Regions (LC 130), Number of Closed Islands (LC 1254), Number of Enclaves (LC 1020), Number of Distinct Islands (LC 694) — all variants of this template.
- Three correct solutions exist (DFS, BFS, Union-Find), each with different trade-offs. Senior+ candidates know all three and can argue when to use which.
What it looks like it tests: can you code DFS on a grid.
What it actually tests: (a) do you reach for visited reflexively, (b) do you handle recursion depth like a professional, (c) can you sketch all three approaches without panicking, (d) do you ask “may I mutate the grid?” before you do.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/number-of-islands/
STOP. 15-min timer. Try DFS first; if you finish, try BFS. Don’t peek.
3. Prerequisite Concepts
- p20 — BFS with
dequeand level-snapshot pattern - phase-04-graphs/labs/lab-02-dfs-connected-components.md
- phase-04-graphs/labs/lab-01-bfs-shortest-path.md
- phase-04-graphs/labs/lab-07-union-find-applications.md — for the third solution
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given a 2D grid of '1' (land) and '0' (water) chars, count the number of connected components of '1's under 4-directional connectivity (up/down/left/right; diagonals do NOT connect).”
Step 2 — Clarify:
- “4-directional or 8-directional connectivity?” (LC: 4-directional.)
- “May I mutate the input grid?” (LC: doesn’t say; ASK in interview. Mutation as visited-marker saves O(MN) memory.)
- “Are values guaranteed
'1'/'0'chars, or integers?” (LC: chars. Easy bug source.) - “Empty grid?” (Return 0.)
- “What’s the max grid size?” (LC: 300×300. Small enough that recursion is safe — but state this.)
Step 3 — Constraints: M, N up to 300 → 90k cells max. O(MN) time and O(MN) space (visited + recursion stack worst case). Recursion depth could be MN = 90k in a path-shaped island — borderline; switch to iterative if M or N > ~500.
Step 4 — Examples:
[["1","1","0"],["1","0","0"],["0","0","1"]]→ 2 (top-left blob + bottom-right cell).- All water → 0.
- All land → 1.
- Single cell
[["1"]]→ 1.
Step 5 — Brute Force: There isn’t really a “brute force” here that’s distinct from the canonical solution. The canonical DFS/BFS is O(MN), which is optimal. Mention this.
Step 6 — Complexity: O(MN) time (each cell visited at most twice — once by outer loop, once by DFS/BFS). O(MN) space worst case for visited set + recursion stack.
Step 7 — Pattern Recognition: “Grid + connected regions” → connected components on an implicit graph. The grid IS the graph; cells are nodes, 4-neighbor adjacency is the edge relation. Same template you’ll use for flood-fill, region-coloring, image-segmentation.
Step 8 — Develop: Outer loop over every cell. If land + unvisited → increment count + flood-fill (DFS or BFS) marking all connected cells visited.
DIRS = ((1, 0), (-1, 0), (0, 1), (0, -1))
def num_islands(grid):
if not grid: return 0
m, n = len(grid), len(grid[0])
visited = [[False]*n for _ in range(m)]
count = 0
for r in range(m):
for c in range(n):
if grid[r][c] == '1' and not visited[r][c]:
count += 1
dfs(r, c) # marks all connected as visited
return count
Step 9 — Test: All-water, all-land, single cell, large path-shaped island (recursion depth stress), the canonical 3×3 example.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — The Three Solutions Compared
| Approach | Time | Space | When to ship |
|---|---|---|---|
| Recursive DFS | O(MN) | O(MN) stack worst | Small grids (≤ a few thousand cells); cleanest code. |
| Iterative BFS | O(MN) | O(min(M,N)) frontier | Large grids where recursion would blow stack. Defensive default. |
| Union-Find | O(MN · α(MN)) | O(MN) | When the problem is dynamic (e.g., “land becomes water over time” — LC 305 Number of Islands II). |
For static LC 200: DFS or BFS — either correct. The Union-Find variant is over-engineered for the base problem but the correct answer for the dynamic follow-up.
Why “visited when enqueuing” beats “visited when dequeuing”: If you mark visited at dequeue, a single cell can be enqueued many times by multiple neighbors before any of them dequeues it. The queue grows to O(MN) in the worst case. Mark visited immediately on enqueue (and on the source before BFS starts) to keep the queue bounded by the frontier size.
The mutation trick:
Many top solutions flip grid[r][c] = '0' to mark visited, saving O(MN) memory for the visited array. In an interview, always announce this: “I’m going to use the grid itself as the visited marker by flipping '1' to '0' — this destroys the input but saves O(MN) memory. Is that acceptable?” That sentence is a Senior+ signal.
7. Anti-Pattern Analysis
- Forgetting to mark visited at the source.
q = deque([(r, c)])withoutvisited[r][c] = True. The source gets re-enqueued by its neighbors. Causes duplicate counts and infinite loops. - Mark-on-dequeue instead of mark-on-enqueue. Queue grows to O(MN). Works but wasteful and a bug-prone pattern.
- Recursive DFS on a 500×500 grid in Python.
RecursionErroron path-shaped islands. Either bumpsys.setrecursionlimit(with explanation) or switch to iterative. - Checking
grid[r][c] == 1instead of== '1'. Chars vs ints. The interviewer is watching for this. - Iterating
for dr, dc in [(0,1),(0,-1),(1,0),(-1,0)]inside the inner loop. Allocates a fresh list every call. HoistDIRSto module level. - Mutating the grid without announcing it. Marks you as “didn’t think about ownership of caller state.” Even one sentence — “I’ll mutate; let me know if I should clone first” — recovers the signal.
8. Skills & Takeaways
- The grid-as-implicit-graph reframe — recognize “2D grid + connectivity” as a graph problem in 5 seconds.
- Mark-visited-on-enqueue discipline — applies to every BFS you’ll ever write.
- DFS vs BFS choice on grids — recursion depth vs frontier memory; pick by grid shape.
- The mutation announcement — “may I mutate the input?” before doing it.
DIRSas a module constant — a small habit that signals professionalism.- The three-solution map — DFS / BFS / Union-Find; when to use each.
9. When to Move On
Move on when:
- You can write recursive DFS, iterative BFS, AND Union-Find for this in under 5 minutes each.
- You correctly handle: empty grid, single-cell grid, all-water, all-land, the canonical example.
- You can articulate when Union-Find is the right answer (dynamic follow-up; LC 305).
- You can sketch LC 695 (Max Area of Island), LC 463 (Island Perimeter), LC 1254 (Closed Islands) as 30-second variations.
- Your stress test runs BFS and DFS on 1000 random grids and confirms they return the same count.
10. Company Context
Amazon:
- Default SDE-I / SDE-II first-round question. Bar Raisers pivot to “what if the grid is too big to fit in memory?” → answer: stream row-by-row, maintain row above + Union-Find merging across rows.
- Watch for: “now compute the area of each island, sorted descending.” Quick mod of the DFS to count and collect.
- Scorecard phrase: “Coding Excellence — clean DFS, announced visited strategy, handled recursion-depth concern.”
Meta:
- Asks LC 200 then immediately LC 305 (Number of Islands II — dynamic land additions). This is THE Union-Find showcase. Have it ready.
- Often asks LC 694 (Distinct Islands) as the harder follow-up — count shape-distinct islands via canonical-form hashing of DFS paths.
- Scorecard phrase: “Strong — handled both static and dynamic variants; recognized Union-Find applicability.”
Google:
- May skip 200 and start at LC 130 (Surrounded Regions — flip enclosed
'O's) or LC 1254 (Closed Islands). Both require the boundary-BFS-inversion trick from this week’s p24. - Will ask correctness proof for the visited strategy. Be ready to argue “every cell is visited exactly once” formally.
- Scorecard phrase: “Algorithm — clean, correct, with clear complexity argument.”
Microsoft:
- Standard first round. May ask the 8-connectivity variant. Trivial change to
DIRS(add 4 diagonals) but explicitly ask before assuming. - Scorecard phrase: “Solid — clarified connectivity model upfront; correct solution.”
Bloomberg:
- Frames as “region detection in tick data” or “grouping connected trades.” Often paired with a follow-up about finding the largest island or computing per-island statistics.
- Scorecard phrase: “Pragmatic — adapted template to domain framing.”
Apple:
- May ask the streaming variant: cells arrive one at a time; after each, report the current number of islands. This is the dynamic Union-Find variant in disguise (LC 305).
- Scorecard phrase: “Strong — pivoted to Union-Find for the streaming case.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | Code without thinking | DFS with visited array | DFS or BFS with mark-on-enqueue | Asks “mutable?” first, then DFS/BFS |
| Visited mechanism | Mark on dequeue (bug) | Visited array | Visited array + announces option to mutate | Mutation with announcement + restoration path discussed |
| Recursion concern | None | None | Notes recursion-depth risk | Switches to iterative if grid > threshold |
| Follow-up readiness | Stuck | Solves LC 695 | Solves LC 695 + 463 | Solves LC 305 (dynamic) with Union-Find |
| Complexity | “fast” | O(MN) | O(MN) + proves each cell visited exactly once | O(MN) + Union-Find α(MN) for dynamic |
12. Level Delta
Mid (L4):
- Writes DFS with a visited array.
- Counts components correctly.
- Knows BFS exists; can switch if asked.
Senior (L5):
- Announces “may I mutate?” before flipping
'1'to'0'. - Hoists
DIRSto module constant. - Discusses recursion-depth trade-off.
- Solves LC 695 (Max Area) as a 30-second variant.
Staff (L6):
- Articulates DFS/BFS/Union-Find trade-off matrix upfront.
- Implements LC 305 (dynamic islands) with Union-Find on demand.
- Mentions LC 694 (distinct islands) and the canonical-form hashing for shape comparison.
- Discusses how the algorithm parallelizes (per-tile DFS + cross-tile Union-Find merge).
Principal (L7+):
- Reframes as “labeled image connected component analysis” — same algorithm shipped in OpenCV / scikit-image.
- Discusses the streaming variant memory bound: only need one row of state for row-by-row Union-Find.
- Notes that the algorithm is the same as 2-pass connected component labeling in image processing (Hoshen-Kopelman algorithm).
- Discusses cache-locality of row-major DFS vs BFS on giant grids.
13. Follow-up Q&A
Q1: What if the grid doesn’t fit in memory? A: Stream row-by-row. Maintain the previous row’s label assignments. When processing a new row, union new land cells with land cells directly above (same column) via Union-Find. After the last row, count distinct roots. Memory: O(N) (single row) + O(MN) for the Union-Find structure — or better, evict resolved labels as soon as a row passes out of scope, dropping to O(N).
Q2: What if cells are added dynamically (LC 305 — Number of Islands II)?
A: Use Union-Find. Initialize: 0 islands, all cells unmarked. For each new land cell (r, c):
- Mark it land, increment count.
- For each of the 4 neighbors that’s also land: if its root ≠ current cell’s root, union them, decrement count.
- Append current count to result. Time: O(K · α(MN)) for K additions. Standard Union-Find variant; very common follow-up at Meta.
Q3: How would you count distinct island shapes (LC 694)?
A: Run DFS for each island, but record the relative path of moves (e.g., “RDLU” for right/down/left/up). Canonicalize (e.g., start from top-left-most cell of the island, fix traversal order). Insert canonical-form strings into a set; answer = len(set). This converts shape comparison into string hashing.
Q4: Parallelize this — say, across 8 CPU cores. A: Tile the grid (e.g., 8 horizontal strips). Run DFS independently in each tile, assigning local component IDs. Then sweep the strip boundaries: for each pair of adjacent land cells across a boundary, Union-Find merge their local IDs. Final count = distinct roots. Speedup is bounded by the boundary length to area ratio.
Q5: Why is 4-connectivity vs 8-connectivity not just a constant factor?
A: For counting, the difference is a constant 2× in DIRS size — same O(MN). But the resulting graph can be very different: a checkerboard pattern has all cells disconnected under 4-conn but fully connected under 8-conn. Always clarify connectivity in the first 30 seconds.
14. Solution Walkthrough
See solution.py:
num_islands_dfs(grid)— recursive DFS with visited array (canonical, recommended for small grids).num_islands_bfs(grid)— iterative BFS with deque and mark-on-enqueue (canonical, safer for large grids).num_islands_uf(grid)— Union-Find with path compression + union by rank (overengineered for static, but the right base for LC 305).num_islands_mutate(grid)— DFS that mutates the grid as visited marker (the “I asked, you said OK” version).UnionFindclass —findwith path compression,unionwith rank.stress_test()— 1000 random grids; all four solutions return the same count.edge_cases()— empty, all-water, all-land, single cell, the canonical LC examples, a “spiral” grid that stresses recursion depth.
Run: python3 solution.py → “ALL TESTS PASSED”.
15. Beyond the Problem
Principal-engineer code review comment:
“Three things. First: please hoist
DIRSto module scope; allocating a 4-tuple per cell visit is exactly the kind of microcost that compounds when this runs on 100M-cell images in our pipeline. Second: the recursive variant is fine for LC, but ship the iterative one — we’ve had production incidents where this exact algorithm hitRecursionErroron adversarial inputs (carefully shaped to maximize path length). Iterative-with-explicit-stack is defensively correct. Third: if you ever do the streaming variant for our row-by-row image processor, please use the two-pass connected component labeling algorithm (Hoshen-Kopelman), not naive Union-Find — it’s the published-since-1976 right answer and our image team will recognize it. The fact that LeetCode never mentions this name doesn’t mean we shouldn’t use it.”
Connections forward:
- p24 (Pacific Atlantic) uses the same grid-BFS template with the boundary-inversion trick.
- Phase 4 —
phase-04-graphs/labs/lab-07-union-find-applications.mddeepens the Union-Find variant. - Production: image segmentation, satellite map parsing, terrain analysis, GIS connected-region detection, game-of-life region tracking, percolation theory simulations.
- The Hoshen-Kopelman algorithm (the streaming/row-by-row version) is THE algorithm used in computational physics for percolation cluster analysis.
- LC 305 is the canonical dynamic follow-up; have it warm.
Hints — p21 Number of Islands
Read one at a time. Try to solve between each.
Hint 1 — Reframe.
This is a grid, but it’s actually a graph. Each cell is a node. Each pair of orthogonally adjacent same-value cells is an edge. “Count islands” = “count connected components of '1'-cells.” Now you know which template to reach for.
Hint 2 — Outer + inner loop structure. Loop over every cell. If you find unvisited land, increment a counter and flood-fill everything reachable from it (marking visited as you go). Then resume the outer loop. The outer loop guarantees you don’t miss disconnected islands.
Hint 3 — Visited tracking.
You need to know what you’ve seen. Options: a separate visited[m][n] matrix, OR mutate the grid in place (flip '1' → '0'). The latter saves memory but destroys input — always ask the interviewer first. If you go BFS, mark visited the instant you enqueue, never on dequeue, or your queue blows up.
Hint 4 — DFS or BFS? Both are O(MN) correct. DFS is shorter to write. BFS is safer for huge grids (no recursion-depth risk). For LC’s 300×300 max, recursive DFS is fine. For production, default to iterative BFS or iterative DFS-with-explicit-stack.
Hint 5 — Direction constant.
Define DIRS = ((1,0),(-1,0),(0,1),(0,-1)) at module level. Inside the inner loop: for dr, dc in DIRS: nr, nc = r+dr, c+dc; if 0<=nr<m and 0<=nc<n and grid[nr][nc]=='1' and not visited[nr][nc]: .... This pattern is the same for every grid-graph problem you’ll ever solve (p24 Pacific Atlantic uses the identical scaffold).
If still stuck: look at num_islands_bfs in solution.py — it’s the clearest of the four for first-time understanding.
p22 — Clone Graph
Source: LeetCode 133 · Medium · Topics: DFS, BFS, Hash Table, Graph Companies (2024–2025 frequency): Meta (very high), Google (high), Amazon (high), Microsoft (medium), Bloomberg (medium), Uber (medium), Apple (medium) Loop position: mid-onsite at Meta/Google. Heavy “data structure literacy” signal.
1. Quick Context
Given a reference to a node in a connected undirected graph, return a deep copy (clone). Each node has a value and a list of neighbors.
The structural challenge is the cycle: a naive recursive copy enters an infinite loop the moment it hits its first back-edge. The solution is a memoization map old_node → new_node that doubles as “have I cloned this yet?” tracking. This pattern is identical to LC 138 (Copy List with Random Pointer) and identical to how you’d serialize/deserialize any graph data structure.
What it looks like it tests: can you do BFS/DFS on a graph.
What it actually tests: (a) do you recognize the cycle problem instantly, (b) do you reach for memoization as the canonical fix, (c) do you produce a truly independent clone (mutating the clone must NOT affect the original — a very common bug), (d) do you handle the None input case.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/clone-graph/
STOP. 20-min timer. Try DFS first, then BFS.
3. Prerequisite Concepts
- p21 — grid-as-graph (you’ve internalized “visited set or you loop”)
- phase-04-graphs/labs/lab-09-graph-modeling.md
- Hash maps with object identity (Python
dictkeys: instances hash byidby default) - LC 138 (Copy List with Random Pointer) — same pattern in linked-list form
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given the head node of a connected undirected graph (each node has int val and list[Node] neighbors), return a brand-new graph that is structurally identical but shares no node objects with the original.”
Step 2 — Clarify:
- “Is the graph guaranteed connected?” (LC: yes. Starting from
node, BFS/DFS reaches everything.) - “May
nodebeNone?” (LC: yes — returnNone.) - “Are node values unique?” (LC: yes, 1..N, used as labels. Don’t rely on this — the algorithm should work even with duplicate values. Identity, not value, is the dedup key.)
- “Self-loops or multi-edges?” (LC says no, but state that your algorithm handles both naturally.)
Step 3 — Constraints: Up to 100 nodes; small. Time O(V+E). Space O(V) for the memo.
Step 4 — Examples: 4-node cycle: 1-2-3-4-1. The clone must be a separate 4-node cycle with the same shape, and mutating clone’s neighbors lists must not touch the original.
Step 5 — Brute Force: A “deepcopy via Python’s copy.deepcopy” works but is not the point of the question; explicitly call this out and then implement the algorithm by hand.
Step 6 — Complexity: O(V+E) time (each node + each edge processed once). O(V) auxiliary space for the memo + BFS queue / DFS stack.
Step 7 — Pattern Recognition: “Deep copy a graph” → memoized traversal. The memo is a dict[OldNode, NewNode]. Same pattern in LC 138 (random pointer linked list), LC 116 (populate next pointers in tree), serialization of any cyclic structure.
Step 8 — Develop (DFS):
def cloneGraph(node):
if node is None: return None
memo = {} # old_node -> new_node
def dfs(old):
if old in memo: return memo[old]
new = Node(old.val)
memo[old] = new # CRITICAL: register BEFORE recursing into neighbors
new.neighbors = [dfs(n) for n in old.neighbors]
return new
return dfs(node)
The line memo[old] = new must come BEFORE the recursive call into neighbors, otherwise a back-edge will re-enter dfs(old) and infinite-loop. This is the entire trick.
Step 9 — Test: None input, single isolated node, single node with self-loop, 2-node cycle, complete K4, line graph.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why Memo Order Matters
The bug-magnet pattern is this:
def dfs(old):
if old in memo: return memo[old]
new = Node(old.val)
new.neighbors = [dfs(n) for n in old.neighbors] # WRONG
memo[old] = new # too late
return new
Now dfs(A) recurses into dfs(B) which recurses into dfs(A) (back-edge) — and since A isn’t in memo yet, it starts a fresh clone. Infinite loop, then RecursionError.
The fix is one line moved: register the new node in memo before descending into neighbors. The memo simultaneously serves three roles: (1) dedup (“already cloned”), (2) cycle break (“don’t recurse forever”), (3) cross-reference (“how do I attach an edge to the right new node?”).
Identity vs equality: memo keys are old nodes. We hash them by id() (Python’s default for user classes). DO NOT switch to keying by old.val — that breaks the moment two nodes share a value. Identity is the correct dedup criterion for graph clones.
7. Anti-Pattern Analysis
- Memo write AFTER neighbor recursion. As shown above — infinite loop on the first back-edge.
- Keying memo by
old.val. Works on LC’s sample inputs (unique values) but is semantically wrong and breaks on graphs with duplicate labels. - Aliasing neighbor lists.
new.neighbors = old.neighborsshares the SAME list object between old and new graphs; mutating one mutates the other. Must construct a fresh list of cloned nodes. - Forgetting the
Noneinput. Crashes on the first attribute access. One-line guard. - Returning
node(the original) when you ran out of time. Some candidates panic and submit this. LC has a hidden test that mutates the clone and checks the original — instant fail. - Using
setof visited instead ofdictof old→new. A set tells you “I’ve seen this,” but not “what’s the new copy I need to wire neighbors to.” You need the dict.
8. Skills & Takeaways
- Memo-as-tri-purpose-tool — dedup, cycle break, cross-reference. One dict does all three.
- Register-then-recurse discipline — the single most important line in this algorithm.
- Identity vs equality as the dedup key — applies to LC 138, graph serialization, any cyclic copy.
- Both DFS and BFS work — pick by stack-depth comfort.
- Independence verification — your stress test must mutate the clone and assert the original is unchanged.
9. When to Move On
Move on when:
- You can write DFS clone and BFS clone, each under 5 minutes, without referencing notes.
- You can articulate why memo registration must precede neighbor recursion.
- Your stress test runs 1000 random graphs, checks structural isomorphism, AND verifies that mutating the clone leaves the original intact.
- You can pivot to LC 138 (Copy List with Random Pointer) in 3 minutes — same algorithm.
- You can sketch graph serialization (write to JSON / read back) as a 5-minute extension.
10. Company Context
Meta:
- One of the top-3 most-asked Meta questions, ever. The mock-interview signal is whether you write
memo[old] = newBEFORE the recursive call. They watch for this specific line ordering. - Common follow-up: “now serialize the clone to a string and deserialize back.” Forces you to articulate the algorithm in two more ways.
- Scorecard phrase: “Algorithm — recognized cycle problem instantly, correct memo ordering.”
Google:
- Often asks the iterative BFS variant first (claims it’s because their interns favor it over DFS for production safety).
- Will demand a proof: “argue that every node is cloned exactly once.”
- Scorecard phrase: “Strong — clear invariants, defensive iterative approach.”
Amazon:
- Pairs with a “what if the graph is disconnected — give us a list of components, return clones of all of them” follow-up. Trivial wrapper: iterate roots and call clone per root, sharing the memo dict.
- Scorecard phrase: “Coding Excellence — clean DFS, handled disconnected variant on demand.”
Microsoft:
- May ask the immutable variant: “design this so the original cannot be mutated even by accident.” Hint: freeze the dict (use
MappingProxyType) or use tuples for neighbor lists. - Scorecard phrase: “Solid — recognized memo pattern, handled edge cases.”
Bloomberg:
- Asks in a domain frame: “deep-copy this dependency graph of pricing models.” Same algorithm, different vocabulary.
- Scorecard phrase: “Pragmatic — translated business domain into graph clone problem.”
Uber:
- Pairs with “now the graph is too big for memory; clone it in streaming fashion.” Hint: the question is poorly posed — you NEED memo. Surface this, then propose chunking strategies (clone subgraph by subgraph, externalize the memo to disk/Redis).
- Scorecard phrase: “Excellent — pushed back on impossible framing, proposed practical compromise.”
Apple:
- Often combines with LC 138 in the same loop: “you did Clone Graph last round; now do Copy List with Random Pointer.” Test of pattern transfer.
- Scorecard phrase: “Strong — transferred memo pattern across two structures.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Cycle awareness | Doesn’t notice | Notices after first run | Identifies in step 1 | Identifies + names “back-edge problem” |
| Memo ordering | Wrong (memo after recursion) | Right after one debug | Right first try | Right + explains the invariant aloud |
| Dedup key | val (buggy) | Object id (correct) | Object id (correct) + states why | Object id + cites LC 138 as same pattern |
| Independence | Doesn’t verify | Verifies by inspection | Stress-tests with mutation | Discusses immutability options |
| Disconnected variant | Stuck | Asks if needed | Handles on demand | Volunteers it as a graceful extension |
12. Level Delta
Mid (L4):
- DFS with memo. Correct memo ordering after one round of debugging.
- Handles
Noneinput. - Returns the right structure.
Senior (L5):
- DFS and BFS variants, both clean.
- Memo ordering correct on first try; can articulate why.
- Sketches LC 138 as the same algorithm.
- Stress-tests for independence (mutate clone, check original).
Staff (L6):
- Discusses serialization as the natural extension; sketches a
to_json/from_jsonpair using the same memo. - Handles disconnected variant cleanly.
- Discusses Python-specific concerns:
id()as the implicit dedup key for non-hashable custom classes (if__hash__overridden). - Sketches the immutable-clone version with
MappingProxyType.
Principal (L7+):
- Connects to the canonical computer-science pattern of graph isomorphism and proves clone is isomorphic.
- Discusses how distributed graph databases (Neo4j, TigerGraph) implement graph subset extraction — same memo pattern at scale.
- Discusses cycle detection vs cycle preservation: this problem requires preservation; cycle detection (LC 207) is the dual.
- Discusses memory pressure: for very large graphs, externalize the memo to LMDB / RocksDB; the algorithm structure is unchanged.
13. Follow-up Q&A
Q1: What if the graph might be disconnected? A: The function as given starts from one node, so it only clones that component. For disconnected graphs, the API should accept a list of roots (one per component). Iterate roots, calling clone on each, sharing the memo dict across calls so that if two “roots” happen to live in the same component, you don’t double-clone. Time still O(V+E).
Q2: How would you serialize this graph to disk and reload it?
A: Use the same memo idea. Assign each node a unique integer id during a traversal: id_of = {old: i for i, old in enumerate(bfs_order)}. Serialize as a JSON { "nodes": [{"id": i, "val": v, "neighbors": [j, k, ...]}, ...] }. Deserialize in two passes: pass 1 create all Node(val) objects keyed by id; pass 2 wire neighbors via the id→Node map. The id→Node map is the deserialization memo — same structure as our clone memo.
Q3: What if Node.__hash__ is overridden so two different nodes with the same val hash equal?
A: dict keying breaks. Switch to keying by id(old) explicitly: memo[id(old)] = new. Slightly uglier but bulletproof.
Q4: Can you make the clone provably immutable?
A: After cloning, walk every node and do new.neighbors = tuple(new.neighbors). Override __setattr__ to raise. Or wrap each node’s neighbor list in MappingProxyType (for dict-style) or use Python’s dataclass(frozen=True). Note that “immutable” in Python is a discipline, not a hard guarantee — discuss this honestly.
Q5: How would you parallelize? A: Tricky because of shared memo writes. Partition the graph (e.g., METIS or simple BFS-partition into K pieces). Clone each piece independently with its own local memo. Then merge: cross-partition edges need a final reconciliation pass mapping local IDs to global new-node references. Overhead of partitioning + reconciliation usually only pays off for very large graphs.
14. Solution Walkthrough
See solution.py:
Nodeclass with__slots__ = ("val", "neighbors").clone_graph_dfs(node)— recursive memoized DFS. Canonical.clone_graph_bfs(node)— iterative deque-based BFS. Same memo, different traversal order.serialize(node)/deserialize(adj)— used as test infrastructure (build random graphs from adjacency lists and dump back).graphs_isomorphic_and_independent(old, new)— structural isomorphism check + mutation test (mutate clone, confirm original unchanged).stress_test()— 1000 random connected undirected graphs N=1..15, both DFS and BFS clones pass isomorphism + independence.edge_cases()— None, single node, self-loop, 2-cycle, K5, line graph.
Run: python3 solution.py → “ALL TESTS PASSED”.
15. Beyond the Problem
Principal-engineer code review comment:
“The memo dict is doing three jobs at once — dedup, cycle break, and reference resolution. That’s elegant in 20 lines but becomes a maintenance hazard at 2000. If you ever generalize this to our internal graph framework, please split it: one set for ‘visited’, one dict for ‘cloned’. Conflating them is fine for an interview but invites bugs when someone adds a fourth concern (e.g., ‘edges-already-copied’) and overloads the same dict. Also: the algorithm here is identical to graph deserialization — please name it
_traverse_with_memoand reuse it. Don’t copy-paste this loop into the JSON loader.”
Connections forward:
- p23 (Course Schedule) — same DFS scaffold but the question is cycle detection (return bool) not cycle preservation (clone). The dual problem.
- LC 138 — copy list with random pointer; literally the same memo pattern in linked-list form.
- LC 1485 (Clone Binary Tree With Random Pointer) — same pattern on trees.
- Production: graph-database subgraph extraction, model-graph deep-copy in PyTorch (parameter dependency graphs), serialization of cyclic Python objects (
pickleuses this internally via its memo table).
Hints — p22 Clone Graph
Read one at a time.
Hint 1 — The cycle problem. A naive “recurse into each neighbor and copy” enters an infinite loop the moment it hits a back-edge (any edge from a descendant back to an ancestor). You need a way to say “wait, I’ve already started cloning this node — here’s the in-progress new version.”
Hint 2 — Memo as the answer.
A dict old_node → new_node. When you arrive at an old node, first check the memo. If present, return the existing new copy. If absent, create the new copy AND register it in the memo BEFORE recursing into neighbors.
Hint 3 — The critical ordering.
new = Node(old.val)
memo[old] = new # <-- THIS LINE
new.neighbors = [dfs(n) for n in old.neighbors]
The memo write MUST precede the recursive call. Otherwise the recursion that re-encounters old (via a back-edge) sees an empty memo and starts a fresh clone — infinite loop.
Hint 4 — Three roles for one dict. The memo is doing three jobs at once:
- “Have I seen this old node?” (dedup)
- “Don’t infinite-loop on cycles” (cycle break)
- “When I need to attach an edge to a neighbor, give me the cloned version, not the original” (cross-reference)
This is why a
setdoesn’t suffice — you need the dict’s value for role 3.
Hint 5 — Test for independence, not just isomorphism.
A clone that shares node objects with the original is a wrong clone, even though it “looks identical.” After cloning, mutate the clone (e.g., clear all neighbors) and confirm the original is untouched. This catches the “I returned node instead of cloning” bug AND the “I aliased the neighbor list” bug.
If still stuck: look at clone_graph_dfs in solution.py — it’s 10 lines and the line ordering tells the whole story.
p23 — Course Schedule
Source: LeetCode 207 · Medium · Topics: DFS, BFS, Topological Sort, Graph Companies (2024–2025 frequency): Google (very high), Meta (high), Amazon (high), Microsoft (high), Apple (medium), Bloomberg (medium), LinkedIn (high) Loop position: mid-to-late onsite. Heavy “do you understand graphs as a model” signal.
1. Quick Context
numCourses courses and a list of [a, b] prerequisite pairs (meaning “to take a, you must first take b”). Return whether it’s possible to finish all courses.
This is cycle detection on a directed graph, framed as a real-world planning problem. Two canonical algorithms solve it and you should know both:
- Kahn’s algorithm (BFS-based topological sort): track in-degrees, repeatedly remove zero-in-degree nodes. If you process all
Nnodes → no cycle. Otherwise → cycle exists. - DFS 3-coloring (WHITE / GRAY / BLACK): WHITE = unvisited, GRAY = on current recursion path, BLACK = fully processed. A GRAY node encountered during DFS = back-edge = cycle.
What it looks like it tests: can you detect cycles. What it actually tests: (a) do you model “prereq” correctly as a directed edge (which way does the arrow go?), (b) do you know BOTH algorithms or just one, (c) can you extend to LC 210 (return the actual ordering — Course Schedule II) instantly, (d) do you handle self-loops as the trivial cycle case.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/course-schedule/
STOP. 25-min timer. Solve both ways before reading.
3. Prerequisite Concepts
- Directed graph adjacency list construction
- BFS with deque / DFS with explicit recursion
- phase-04-graphs/labs/lab-06-topological-sort.md
- Concept of in-degree
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given N courses and a list of [a, b] pairs meaning ‘must take b before a,’ determine whether a complete ordering exists. Equivalent: does the dependency graph have NO directed cycle?”
Step 2 — Clarify (CRITICAL):
- “For
[a, b], doesbpoint toaorapoint tob?” THIS ARROW DIRECTION IS THE QUESTION. By LC convention,[a, b]meansb → a(“completeb, then can doa”). State your interpretation explicitly. - “Are duplicate prereq pairs allowed?” (Doesn’t affect correctness, but slight constant factor.)
- “Can a course be its own prereq (
[a, a])?” (LC says no, but state your algorithm handles it as a trivial cycle.) - “Disconnected components?” (Yes — your loop must seed BFS/DFS from EVERY zero-in-degree node, not just node 0.)
Step 3 — Constraints: N ≤ 2000, len(prerequisites) ≤ 5000. O(V+E) trivially fits.
Step 4 — Examples:
N=2, prereqs=[[1,0]]→ True (take 0, then 1).N=2, prereqs=[[1,0],[0,1]]→ False (cycle).N=3, prereqs=[]→ True.
Step 5 — Brute Force: “Try all N! orderings, check each.” Trivially correct, useless complexity. Mention as proof of correctness baseline; don’t implement.
Step 6 — Complexity: O(V+E) time and space, both algorithms.
Step 7 — Pattern Recognition: “DAG-ness / cycle detection on a directed graph.” Pattern names to drop aloud: “Kahn’s algorithm,” “topological sort,” “3-color DFS.”
Step 8 — Develop (Kahn’s):
def canFinish(N, prereqs):
adj = [[] for _ in range(N)]
indeg = [0] * N
for a, b in prereqs:
adj[b].append(a)
indeg[a] += 1
q = deque(i for i in range(N) if indeg[i] == 0)
processed = 0
while q:
u = q.popleft()
processed += 1
for v in adj[u]:
indeg[v] -= 1
if indeg[v] == 0:
q.append(v)
return processed == N
Develop (DFS 3-color):
WHITE, GRAY, BLACK = 0, 1, 2
def canFinish(N, prereqs):
adj = [[] for _ in range(N)]
for a, b in prereqs:
adj[b].append(a)
color = [WHITE] * N
def dfs(u):
if color[u] == GRAY: return False # back-edge = cycle
if color[u] == BLACK: return True
color[u] = GRAY
for v in adj[u]:
if not dfs(v): return False
color[u] = BLACK
return True
return all(dfs(i) for i in range(N))
Step 9 — Test: Empty prereqs, self-loop, two-node cycle, chain, diamond, two-component (one valid + one cyclic).
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Kahn vs DFS 3-Color
Both are O(V+E). Pick by what you need next:
| Need | Algorithm |
|---|---|
| Just “is there a cycle?” | DFS 3-color (slightly less bookkeeping) |
| The actual topological order (LC 210) | Kahn’s (the BFS order IS the topo order) |
| Identify all nodes in a cycle | DFS 3-color (when you hit GRAY, walk the stack back) |
| Stream-friendly (nodes arrive over time) | Kahn’s (decrement in-degrees as edges arrive) |
Why the 3-coloring works: the only way for DFS to discover a cycle is to encounter a node still on its current recursion stack — that’s exactly the “GRAY” state. WHITE = not yet touched. BLACK = fully explored and known cycle-free.
Common pitfall — using a single visited set: if you collapse WHITE/GRAY/BLACK into “visited / not visited,” you can no longer distinguish “I’m currently descending through this node” from “I’ve finished with this node.” A node that’s fully processed (BLACK) shouldn’t trigger a cycle alarm just because some unrelated path revisits it.
7. Anti-Pattern Analysis
- Wrong arrow direction.
[a, b]meansb → a. Reversing it builds the wrong graph and gives wrong answers on asymmetric examples. State your interpretation OUT LOUD before coding. - Using a single
visitedset instead of 3 colors. Reports false cycles on diamond graphs where two paths converge. - Seeding BFS from only node 0. Misses disconnected components. Kahn’s MUST initialize the queue from ALL zero-in-degree nodes.
- Forgetting
processed == Ncheck. If you just returnTruewhen the loop ends, you return True for cyclic graphs (the queue empties early because no node ever reaches in-degree 0). - Mutating the original
indegarray without copying — only matters if you reuse across runs, but flag it. - No self-loop handling. A self-loop
[a, a]is a cycle of length 1. Kahn’s catches it (indeg[a] is permanently ≥1, never enters queue). DFS 3-color catches it (enteringdfs(a)sets GRAY, recurses intoa, sees GRAY, returns False). - Recursive DFS without
sys.setrecursionlimiton N=2000 chain. Python default is 1000. A chain0→1→2→...→1999blows the stack.
8. Skills & Takeaways
- Arrow-direction discipline — half of all topo-sort bugs are reversed-edge bugs. State direction OUT LOUD.
- Kahn’s = topo order for free — same loop solves LC 210.
- 3-color DFS — the canonical cycle-detection pattern. GRAY = “on current path.”
processed == Ninvariant — Kahn’s correctness depends on this final check.- Disconnected components — seed from ALL valid roots, not just node 0.
- The reduction — many real problems (build systems, course planning, package managers, spreadsheet formula recalc) reduce to topological sort. Recognize the reduction in 30 seconds.
9. When to Move On
Move on when:
- You can write both Kahn’s and 3-color DFS from memory in under 5 minutes each.
- You can extend Kahn’s to LC 210 (return the ordering) in under 2 minutes.
- You handle: empty prereqs, self-loop, disconnected components, chain of length N=2000.
- Your stress test runs both algorithms + a brute-force
N!checker (small N) and confirms agreement. - You can articulate aloud which algorithm to pick given each downstream need (just-detect vs need-order vs find-cycle).
10. Company Context
Google:
- Asks LC 207 then immediately LC 210 (return the order) then LC 269 (Alien Dictionary — derive the order from sample dictionary words). Test of pattern transfer across three problems.
- Will ask about parallel topological sort: “If we have 8 workers, schedule courses to finish in min wall-clock time.” Reduces to “compute longest path in the DAG” (critical path method).
- Scorecard phrase: “Algorithm — Kahn’s + 3-color, both correct; extended cleanly to ordering and parallel cases.”
Meta:
- Variant: “given the prerequisite list, return the courses that are guaranteed to be in every topo order.” Reduces to checking which nodes have a unique predecessor at every Kahn step. Hard variant.
- Scorecard phrase: “Strong — handled topo + variant.”
Amazon:
- Pairs with a real-world frame: “build the deployment order for our microservices given the dependency manifest.” Same problem, business vocabulary.
- Asks LC 210 nearly always. Have it ready.
- Scorecard phrase: “Customer Obsession + Coding Excellence — translated business framing, shipped clean topo sort.”
Microsoft:
- May ask only LC 207 but expects you to volunteer LC 210 as the “useful generalization.” Senior+ candidates volunteer.
- Scorecard phrase: “Solid — recognized topo sort, demonstrated 2 algorithms.”
Apple:
- Often presents as build-graph cycle detection in their internal dependency system. Asks specifically about handling self-loops and reporting WHICH cycle exists, not just yes/no.
- Scorecard phrase: “Strong — used 3-color DFS, reported the cycle path.”
Bloomberg:
- Asks about computing the order in which to evaluate spreadsheet cells given inter-cell formula dependencies. Topo sort with the additional wrinkle of “cells can be added/removed dynamically.” Mention incremental topological sort as a Staff-level discussion.
- Scorecard phrase: “Strong — pragmatic translation to spreadsheet problem.”
LinkedIn:
- Frequent. Often pairs with “alien dictionary” (LC 269) in same loop.
- Scorecard phrase: “Pattern recognition — saw topo sort instantly.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Arrow direction | Guesses | Asks once, then codes | States interpretation BEFORE coding | States + writes a 1-line comment in the code |
| Algorithm count | DFS only (no cycle handling) | Kahn’s OR DFS 3-color | Both, picks by downstream need | Both + discusses incremental + parallel variants |
| Disconnected | Misses | Catches after one test fail | Seeds all roots first try | Seeds + tests with a multi-component example |
| Final check | None | Returns True when loop ends (wrong) | processed == N | processed == N + asserts in code |
| Extension | Stuck | Solves LC 210 after a hint | Volunteers LC 210 unprompted | Volunteers LC 210 + LC 269 + parallel scheduling |
12. Level Delta
Mid (L4):
- Kahn’s algorithm, correct, with
processed == Ncheck. - Builds adjacency from
[a, b]with the right direction (after one clarifying question). - Handles disconnected components correctly.
Senior (L5):
- Both Kahn’s AND 3-color DFS, both clean.
- States algorithm choice aloud: “I’ll use Kahn’s because the follow-up will likely ask for the order.”
- Solves LC 210 in 2 minutes when asked.
- Handles self-loop explicitly.
Staff (L6):
- Discusses incremental topological sort (when edges arrive/depart dynamically).
- Knows LC 269 (Alien Dictionary) as the same algorithm + character-pair extraction.
- Discusses parallel scheduling: longest path in DAG = critical-path-method = min wall-clock with infinite parallelism.
- Discusses lexicographically smallest topo order via min-heap variant of Kahn’s.
Principal (L7+):
- Reframes as CSP (constraint satisfaction problem) with one constraint type (ordering); discusses how this generalizes to richer constraint solvers.
- Notes that this is the algorithm shipped in
make,ninja,Bazel,npm,pip,cargo, every build system on Earth. Discusses how those systems detect and report cycles to users (cycle reconstruction in DFS 3-color: when you hit GRAY, walk the call stack backwards). - Discusses how spreadsheet engines (Excel, Google Sheets) handle this incrementally — recalc only the cells downstream of changed input cells (downstream subgraph of a topologically sorted DAG).
- Discusses what topological sort does NOT solve: graphs with weighted edges and constraints on cumulative weight (need ILP / scheduling theory).
13. Follow-up Q&A
Q1: Return the actual order (LC 210).
A: Use Kahn’s. The order in which nodes are popped from the queue IS a valid topological order. Append to a list; if len(list) == N at end, return list; else return [] (cycle).
Q2: Return the lexicographically smallest topo order. A: Replace Kahn’s queue (FIFO) with a min-heap. When multiple zero-in-degree nodes exist, always pop the smallest. Still O((V+E) log V) due to heap.
Q3: There are multiple valid orderings. Return them all. A: Backtracking. At each step, enumerate every zero-in-degree node, recurse, then unselect. Worst case exponential (think K courses with no prereqs — K! orderings). Use only when the count is small or use it for enumeration with early termination.
Q4: A new prerequisite arrives in real time. Update the answer.
A: Incremental topological sort. Pearce-Kelly algorithm: maintain ordinal labels; when an edge u→v is added, if ord(u) < ord(v), nothing to do; else recompute labels for the affected subgraph. Amortized O(δ²·5) per edge where δ is the affected region size. The Staff signal is naming Pearce-Kelly without prompting.
Q5: Detect the actual cycle if one exists (not just yes/no).
A: Use DFS 3-color and maintain a parent pointer. When you hit a GRAY node v from current node u, walk parent pointers from u back to v and prepend v. That’s the cycle.
14. Solution Walkthrough
See solution.py:
can_finish_kahns(N, prereqs)— BFS / Kahn’s. Returns bool.can_finish_dfs_coloring(N, prereqs)— 3-color DFS. Returns bool.find_order_kahns(N, prereqs)— LC 210 bonus. Returns the order or[].find_cycle_dfs(N, prereqs)— Q5 follow-up. Returns the cycle as a list of node ids orNone.has_cycle_brute(N, prereqs)— O(N · (V+E)) baseline: run DFS from every node, track parent path. Used as the oracle in stress tests.stress_test()— 1000 random graphs (both guaranteed-DAG and possibly-cyclic), all three implementations agree with brute baseline;find_order_kahnsalso verified to produce a valid order.edge_cases()— empty prereqs, self-loop, two-node cycle, chain length 1000, diamond, disconnected with one cyclic component.
Run: python3 solution.py → “ALL TESTS PASSED”.
15. Beyond the Problem
Principal-engineer code review comment:
“Two things. First: please log the cycle path when you detect one.
return Falseis what LeetCode wants; what production needs israise CycleError(cycle=[3, 7, 12, 3])so the on-call engineer can read the dependency log at 3am and know which three services to break. Cycle reconstruction is 6 extra lines in the 3-color DFS — please do it. Second: if you’re about to ship Kahn’s, also exporttopological_order()— once we have the in-degree machinery built, refusing to return the order is just rude to the next caller.”
Connections forward:
- p24 (Pacific Atlantic) — multi-source BFS, another canonical “inversion” pattern.
- LC 210 (Course Schedule II), LC 269 (Alien Dictionary), LC 444 (Sequence Reconstruction) — all same algorithm.
- LC 802 (Find Eventual Safe States) — 3-color DFS variant.
- Production: every build system on Earth (
make,ninja,Bazel,npm,pip,cargo,gradle), spreadsheet recalc engines, course catalog planners, dataflow schedulers (Airflow, Dagster, Prefect), database query planners (join order via topo sort over dependency graph of intermediate results). - Theoretical: topological sort is THE entry point to scheduling theory; from here, CPM (critical path method), PERT, and the entire field of project scheduling open up.
Hints — p23 Course Schedule
Read one at a time.
Hint 1 — Translate to graph language.
“Take a after b” = directed edge b → a. The whole problem becomes: “given this directed graph, is there a cycle?” A cycle means there’s a course that ultimately depends on itself — impossible to schedule.
Hint 2 — Two canonical algorithms. Either:
- Kahn’s: start from courses with NO prerequisites. Take them. Each course you take, “remove” its outgoing edges (decrement neighbors’ in-degree). When a neighbor’s in-degree hits 0, it joins the queue. If you process all N courses → no cycle.
- DFS 3-coloring: WHITE (untouched), GRAY (currently descending through), BLACK (fully processed). If DFS ever encounters a GRAY node, that’s a back-edge — cycle.
Hint 3 — Arrow direction is a trap.
prereqs[i] = [a, b] means course b is a prerequisite of a. Is the edge a → b or b → a? Pick a convention and write it as a comment on line 1. (Hint: edges should point in the order you’d take the courses, so b → a.)
Hint 4 — Kahn’s correctness check.
After the loop, you MUST verify processed_count == N. If a cycle exists, the queue runs dry early (no node ever reaches in-degree 0 inside the cycle). Don’t just return True when the loop ends.
Hint 5 — Don’t forget disconnected components.
Initialize Kahn’s queue with EVERY zero-in-degree node, not just node 0. For DFS, your outer loop must call dfs(i) for every i that hasn’t been colored BLACK yet.
If still stuck: look at can_finish_kahns in solution.py — 15 lines, includes the critical processed == N check.
p24 — Pacific Atlantic Water Flow
Source: LeetCode 417 · Medium · Topics: BFS, DFS, Matrix, Multi-Source BFS Companies (2024–2025 frequency): Amazon (very high), Google (high), Meta (medium), Microsoft (medium), Bloomberg (medium), Apple (medium) Loop position: mid-onsite. Sorting signal between “knows BFS” candidates and “knows BFS inversions” candidates.
1. Quick Context
An M×N grid of heights. Water flows from a cell to any 4-neighbor with equal or lower height. Pacific touches the top and left edges; Atlantic touches the bottom and right edges. Return all cells from which water can reach both oceans.
The naive solution is “for each cell, BFS forward to check if it reaches both oceans” → O((MN)²) — borderline TLE on large grids.
The right solution is the multi-source BFS inversion trick:
- BFS from each ocean edge inward, walking only to neighbors that are ≥ current height (reverse of the flow rule).
- The set of cells reachable from the Pacific edges = cells that can drain to Pacific.
- Same for Atlantic.
- Answer = intersection.
Total work: O(MN). The inversion is the entire interview signal.
What it looks like it tests: can you do BFS/DFS on a grid. What it actually tests: (a) do you see that “for each cell, check reachability to oceans” is the naive O((MN)²) trap, (b) can you invent (or recognize) the inversion, (c) do you implement multi-source BFS (queue seeded with all border cells, not one), (d) do you state the visited-on-enqueue invariant.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/pacific-atlantic-water-flow/
STOP. 25-min timer. Try naive first — see the complexity wall — then pivot.
3. Prerequisite Concepts
- p21 — grid-as-graph + visited-on-enqueue
- phase-04-graphs/labs/lab-03-multi-source-bfs.md
- The “inversion” pattern (reverse the question to flip the algorithm)
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given M×N heights, return all [r, c] from which water can reach BOTH the Pacific (top/left) AND the Atlantic (bottom/right) by flowing only to ≤-height neighbors.”
Step 2 — Clarify:
- “Equal-height neighbors — does water flow?” (Yes — “equal or lower.”)
- “What if grid is empty or 1-row / 1-col?” (1×1: the cell touches both → answer is itself. 1-row: every cell touches top edge = Pacific AND bottom edge = Atlantic → all cells.)
- “Output order?” (LC accepts any.)
Step 3 — Constraints: M, N ≤ 200. So 40k cells. O((MN)²) = 1.6e9 → likely TLE. O(MN) is required.
Step 4 — Examples: LC’s classic 5×5 grid [[1,2,2,3,5],[3,2,3,4,4],[2,4,5,3,1],[6,7,1,4,5],[5,1,1,2,4]] → [[0,4],[1,3],[1,4],[2,2],[3,0],[3,1],[4,0]].
Step 5 — Brute Force: For each cell (r,c), run BFS forward (move to ≤ neighbors) and check whether we reach any Pacific cell AND any Atlantic cell. O((MN)²). State it; show it’s too slow; pivot to the inverted multi-source BFS.
Step 6 — Complexity: O(MN) time (each cell visited at most twice across both BFS runs). O(MN) auxiliary for the two visited sets.
Step 7 — Pattern Recognition: “Reachability from many sources” → multi-source BFS. “Inversion of the question because forward-from-every-cell is too slow” → inversion pattern. Companions: LC 994 (Rotting Oranges — multi-source BFS), LC 542 (01 Matrix — multi-source BFS from all zeros), LC 1162 (As Far From Land), LC 286 (Walls and Gates).
Step 8 — Develop:
def pacific_atlantic(heights):
if not heights or not heights[0]: return []
m, n = len(heights), len(heights[0])
pac = [[False]*n for _ in range(m)]
atl = [[False]*n for _ in range(m)]
def bfs(starts, reachable):
q = deque(starts)
for (r, c) in starts: reachable[r][c] = True
while q:
r, c = q.popleft()
for dr, dc in DIRS:
nr, nc = r+dr, c+dc
if (0 <= nr < m and 0 <= nc < n
and not reachable[nr][nc]
and heights[nr][nc] >= heights[r][c]): # ≥ because we're walking REVERSED
reachable[nr][nc] = True
q.append((nr, nc))
pac_starts = [(0, c) for c in range(n)] + [(r, 0) for r in range(m)]
atl_starts = [(m-1, c) for c in range(n)] + [(r, n-1) for r in range(m)]
bfs(pac_starts, pac)
bfs(atl_starts, atl)
return [[r, c] for r in range(m) for c in range(n) if pac[r][c] and atl[r][c]]
Step 9 — Test: Empty, 1×1, 1-row, all-equal-height, monotone increasing, the LC canonical example.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — The Inversion Trick
The reason this problem is interview-defining is that the obvious algorithm (BFS forward from every cell, check reachability) is too slow but feels right. Senior+ candidates pivot to the inversion:
“Instead of asking ‘from which cells can I reach the Pacific?’, ask ‘from the Pacific edges, which cells can be reached by going UPHILL?’ The reverse of a flow relation is reachable by reversing the direction comparison.”
Why this works: if (r, c) can flow to a Pacific cell via path (r,c) → x₁ → x₂ → ... → P, then reversing every edge: P → ... → x₂ → x₁ → (r,c). The reverse-edge condition is “neighbor is higher or equal” instead of “neighbor is lower or equal.”
The multi-source aspect: the Pacific is not one cell; it’s a whole set of border cells. So we BFS from ALL of them in parallel. Seed the queue with every Pacific border cell up front (marking each visited at enqueue), then run a single BFS. The classic mistake is to BFS once per source — that’s K separate BFSes, total O(K·MN) instead of O(MN).
This same inversion pattern shows up everywhere:
- LC 994 (Rotting Oranges): seed BFS with all rotten oranges, expand.
- LC 542 (01 Matrix): seed with all 0s, BFS outward to compute distance.
- LC 1162 (As Far From Land): seed with all land, BFS outward.
- LC 286 (Walls and Gates): seed with all gates, BFS outward.
The general lesson: when “for every starting cell, compute X” is too slow, try inverting and starting from the destination instead.
7. Anti-Pattern Analysis
- Sticking with naive O((MN)²). Times out on LC. Worse: the interview signal of pivoting to the inversion is the entire point — if you don’t pivot, you fail the question regardless of whether your naive code “works.”
- Forgetting to invert the comparison. When BFSing from the ocean, the comparison flips: neighbors must be ≥ current height (uphill), not ≤. Half of bug submissions have this wrong.
- One BFS per border cell instead of seeded multi-source BFS. Same complexity (O(MN) per BFS × K border cells = O(MN²) or worse). The trick is ONE BFS with K initial seeds.
- Marking visited on dequeue. Queue grows quadratic when many neighbors all enqueue the same cell. Mark on enqueue.
- Hand-rolling the border-cell construction with subtle off-by-one. Easy to double-count corners.
[(0, c) for c in range(n)] + [(r, 0) for r in range(m)]double-counts(0, 0)but the visited-set check renders that harmless. State this explicitly. - Mutating
heightsas the visited marker. Tempting but destructive AND impossible here because you need the actual heights for comparison. Use a separateboolmatrix.
8. Skills & Takeaways
- The inversion pattern — “for every source, compute reachability” → “from the destination(s), compute reverse reachability.”
- Multi-source BFS — seed with K cells upfront; single BFS; O(V+E).
- Comparison inversion — flow rule “≤” becomes reverse-traversal “≥”. Easy to get wrong; comment it.
- Two-set intersection on grids — when you need “satisfies both A and B,” compute reachable sets for A and B independently, then intersect.
- Border-cell construction — corner double-counts are harmless under visited dedup; mention this aloud.
9. When to Move On
Move on when:
- You can write multi-source BFS with both Pacific and Atlantic BFS in under 8 minutes.
- You can explain WHY the comparison flips when you reverse the BFS direction.
- You can articulate three other problems that use multi-source BFS (LC 994, 542, 1162).
- Your stress test runs both the brute (per-cell forward BFS) and the inverted multi-source BFS on 100 random grids and confirms identical output.
- You can sketch the DFS version of the multi-source inversion (functionally identical; just a recursive flood from each border cell).
10. Company Context
Amazon:
- High frequency. Often the “Round 3” question after warmup graph problems.
- Bar Raiser will ask the brute solution first (“just to make sure you can write the trivial one”), then drive you to the inversion: “this is O((MN)²); can we do better?”
- Follow-up: “now there are three oceans; cells must reach all three.” Trivial extension — three BFS sets, three-way intersection.
- Scorecard phrase: “Algorithm + Earn Trust — recognized the brute force was too slow, pivoted to inversion cleanly.”
Google:
- Skips the brute. Goes straight to “design the algorithm assuming the brute is too slow.” Tests the pivot-skill directly.
- Will ask correctness proof: “prove the inversion gives the same set as the forward BFS.” Phrase the proof as “edge reversal preserves reachability between the same node pair.”
- Scorecard phrase: “Algorithm — produced O(MN) via inversion + multi-source BFS, with proof.”
Meta:
- Asks the variant where heights are continuous (floats) instead of ints. Algorithm unchanged — but tests whether you blindly assume integer comparison.
- Scorecard phrase: “Solid — recognized algorithm carries over.”
Microsoft:
- Often combines: “do Pacific Atlantic, then return the cells reachable to only one ocean.” Trivial post-processing on the two sets.
- Scorecard phrase: “Strong — built two-set machinery, extended cleanly.”
Bloomberg:
- Asks variant in a domain frame (“data center cooling — heat flows from high-temp cells to lower-temp neighbors; which cells eventually radiate heat to both edges?”). Same algorithm.
- Scorecard phrase: “Pragmatic — translated domain problem.”
Apple:
- Has been known to ask the 3D variant (water flows in cubic grid). Algorithm extends to 6-direction
DIRSand 6-face start-sets. Discuss the extension; LC version stays 2D. - Scorecard phrase: “Strong — handled algorithm + sketched 3D extension.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | BFS per cell | BFS per cell, sees O((MN)²) | Inversion in 2 min | Inversion + names multi-source BFS pattern |
| Comparison flip | Wrong | Right after one debug | Right + comments why | Right + writes formal proof |
| Multi-source | One BFS per border cell | Seeded queue | Seeded queue + visited-on-enqueue | Mentions LC 994/542/1162 as same pattern |
| Variants | Stuck | “Other shapes work too” | Solves 3-ocean variant | Solves 3D extension and per-ocean filtering |
12. Level Delta
Mid (L4):
- Multi-source BFS, both Pacific and Atlantic.
- Correct comparison inversion (≥ instead of ≤).
- Correct intersection logic.
Senior (L5):
- Notes “naive is O((MN)²)” before coding, pivots to inversion.
- Both BFS and DFS variants.
- Names “multi-source BFS” as the pattern; cites at least one analog (LC 994 or 542).
- Comments the comparison flip in the code.
Staff (L6):
- Discusses proof of equivalence (edge reversal preserves directed reachability between endpoints).
- Solves 3-ocean extension and “reachable to only one” extension on demand.
- Discusses memory: two N²-size bool matrices vs bitmap encoding for very large grids.
- Discusses parallelism: the two BFS runs are independent; can fork to two threads. Final intersection is O(MN).
Principal (L7+):
- Reframes as computing the “drainage basin” of two boundary regions. Reframe makes the algorithm obvious.
- Discusses how GIS systems (ArcGIS, QGIS) implement watershed delineation using this exact algorithm at planet scale.
- Discusses how this generalizes to weighted edges (e.g., flow rate based on slope) → Dijkstra-style multi-source shortest-path on a min-cost graph.
- Discusses streaming variant: heights arrive cell-by-cell; how to maintain the answer incrementally (harder; touches on dynamic graph reachability).
13. Follow-up Q&A
Q1: How would you adapt this if there were three oceans (Pacific top edge, Atlantic right edge, Arctic bottom-left corner)? A: Run three independent multi-source BFSes, one per ocean. Result = cells in all three reachable sets. O(MN) time, O(MN) per set.
Q2: How would you find cells that drain to EXACTLY one of the two oceans?
A: After computing pac and atl sets, return cells in pac XOR atl. One pass.
Q3: Prove the inversion is correct.
A: Define forward reachability: (r,c) → P iff there’s a path through ≤-height neighbors ending at any Pacific cell. Define reverse reachability: P → (r,c) iff there’s a path through ≥-height neighbors starting from any Pacific cell. Path (r,c) = v₀ → v₁ → ... → vₖ = P exists forward iff heights[vᵢ₊₁] ≤ heights[vᵢ] for all i. Reverse the path: vₖ → vₖ₋₁ → ... → v₀. Now heights[vᵢ] ≥ heights[vᵢ₊₁] is exactly the reverse condition. So forward reachability (r,c) → P iff reverse reachability P → (r,c). QED.
Q4: What if the grid is too large to fit two visited matrices in memory? A: Use bit-packed visited (1 bit per cell instead of 1 byte → 8× compression). Or stream the grid in horizontal bands, but then BFS doesn’t fit neatly into the band; you’d need a more sophisticated “ghost cell” multi-pass algorithm. For 200×200 LC inputs, just use bool matrices.
Q5: Heights become floats. Does anything change? A: No. Comparison still works on floats. Only watch out for NaN — if heights can be NaN, define ordering carefully (probably treat NaN cells as impassable).
14. Solution Walkthrough
See solution.py:
pacific_atlantic_bfs(heights)— multi-source BFS, canonical.pacific_atlantic_dfs(heights)— multi-source DFS (recursive, same algorithm).pacific_atlantic_brute(heights)— O((MN)²) per-cell forward BFS, used as oracle.stress_test()— 100 random grids up to 15×15, all three implementations return the same set (compared as sorted lists of tuples).edge_cases()— empty, 1×1, 1-row, 1-col, all-equal, monotone, LC canonical 5×5.
Run: python3 solution.py → “ALL TESTS PASSED”.
15. Beyond the Problem
Principal-engineer code review comment:
“The inversion is correct and elegant. But please name it. The pattern is ‘multi-source BFS with edge reversal’ — that name should appear in the docstring. The next engineer who sees this in three years will recognize the family instantly: rotting oranges, 01 matrix, walls and gates, and yes, every watershed-delineation tool in GIS land. They are ALL the same algorithm; the only thing that varies is the seed set and the edge predicate. If you ever generalize this for our terrain pipeline, please write
multi_source_bfs(seeds, predicate)once and reuse — don’t copy-paste this loop into four files.”
Connections forward:
- p25 (Word Ladder) — BFS on an implicit graph; sister pattern.
- LC 994 (Rotting Oranges), LC 542 (01 Matrix), LC 1162 (As Far From Land), LC 286 (Walls and Gates), LC 934 (Shortest Bridge) — all multi-source BFS.
- Production: GIS watershed delineation, image segmentation (region growing from seeds), epidemic modeling (multi-source SIR), network reachability from a set of failure points.
- Theoretical: the “reverse reachability” trick is the foundation of backward-chaining in AI search (start from the goal, work backwards to the start).
Hints — p24 Pacific Atlantic Water Flow
Read one at a time.
Hint 1 — Spot the trap. “For each cell, BFS to see if it reaches both oceans” is O((MN)²) and times out on big grids. More importantly, the interview signal is whether you pivot away from this; the right answer is fundamentally a different algorithm.
Hint 2 — Invert the question. Instead of “from each cell, can I reach the ocean?”, ask “from the ocean, which cells can be reached?” Same answer, dramatically smaller work: O(MN) instead of O((MN)²).
Hint 3 — Flip the comparison. Going forward (cell → ocean), water moves to neighbors with lower or equal height. Going backward (ocean → cell), you move to neighbors with higher or equal height. This sign flip is the #1 bug source — write it as a comment in the code.
Hint 4 — Multi-source BFS, not K separate BFSes. The Pacific isn’t one cell; it’s an entire border. Seed your BFS queue with ALL Pacific border cells UP FRONT, mark each visited, then run a single BFS. Same for Atlantic. Two BFS runs total, each O(MN). Don’t BFS from each border cell separately.
Hint 5 — Combine.
After the two BFS runs you have two bool matrices: pac[r][c] and atl[r][c]. The answer is every (r, c) where both are true. One pass to collect.
If still stuck: look at pacific_atlantic_bfs in solution.py — the inverted BFS is the entire trick.
p25 — Word Ladder
Source: LeetCode 127 · Hard · Topics: BFS, Hash Table, String, Implicit Graph Companies (2024–2025 frequency): Amazon (high), Meta (high), Google (high), Microsoft (high), Bloomberg (medium), Apple (medium) Loop position: late-onsite, the “graph round.” Most candidates fail by writing O(N²·L) naive adjacency.
1. Quick Context
Given a beginWord, an endWord, and a wordList, return the length of the shortest transformation sequence from beginWord to endWord (each step changes exactly one letter; each intermediate word must be in wordList).
This is the canonical implicit graph + BFS problem. The challenge isn’t BFS itself — it’s:
- The graph isn’t given. Nodes = words, edges = pairs of words that differ by one letter. You can construct adjacency by either (a) comparing all pairs O(N²·L), or (b) the pattern-bucket trick O(N·L²) — usually a massive speedup.
- The bucket trick: for each word, generate
Lpatterns by replacing each letter with*(e.g.,"hit"→"*it","h*t","hi*"). Group words by these patterns. Two words share a bucket iff they differ by exactly one letter. Adjacency lookup becomes O(L) instead of O(N·L). - Bidirectional BFS: for very large dictionaries, BFS from both ends simultaneously, alternating which frontier to expand (smaller one each iteration). Search space goes from
b^dto2·b^(d/2). - BFS, not DFS. This is unweighted shortest-path. DFS gives a path, not the shortest path.
What it looks like it tests: can you do BFS on a graph.
What it actually tests: (a) do you recognize “1-letter-edit” as an implicit graph, (b) do you skip O(N²·L) adjacency in favor of bucket adjacency, (c) do you correctly handle endWord not in wordList (return 0), (d) can you discuss bidirectional BFS as the Hard follow-up.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/word-ladder/
STOP. 30-min timer. This is Hard — try the bucket adjacency before peeking.
3. Prerequisite Concepts
- p21–p24 — comfort with BFS on grids and explicit graphs
- phase-04-graphs/labs/lab-01-bfs-shortest-path.md
- phase-04-graphs/labs/lab-09-graph-modeling.md
- BFS level-counting (distance = level # in BFS tree)
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given beginWord, endWord, and a list of valid intermediate words, find the minimum number of words in a sequence from beginWord to endWord where each consecutive pair differs by exactly one letter and every word in the sequence (except beginWord) is in the wordList. Return 0 if impossible. The ‘length’ is the count of words including endpoints.”
Step 2 — Clarify:
- “Is
beginWordalways inwordList? Always out?” (LC: not guaranteed in. State that this doesn’t affect the algorithm.) - “Must
endWordbe inwordList?” (LC: yes, otherwise return 0. Check this BEFORE running BFS.) - “Case sensitive?” (LC: all lowercase. Easy to forget.)
- “Word length L?” (LC: up to 10. Small. Bucket trick fast.)
- “Dictionary size N?” (LC: up to 5000. Bucket beats naive comfortably.)
Step 3 — Constraints: N ≤ 5000, L ≤ 10. Naive O(N²·L) = 5000² · 10 = 2.5e8 — borderline TLE in Python. Bucket O(N·L²) = 5000 · 100 = 5e5 — instant. Always use bucket.
Step 4 — Examples:
beginWord="hit", endWord="cog", wordList=["hot","dot","dog","lot","log","cog"]→ 5 (hit→hot→dot→dog→cog).beginWord="hit", endWord="cog", wordList=["hot","dot","dog","lot","log"]→ 0 (endWord not in list).
Step 5 — Brute Force: Compare every pair of words for 1-letter-edit; build full adjacency list; BFS. Time O(N²·L) for adjacency + O(N + E) for BFS. State; show; pivot to bucket.
Step 6 — Complexity: O(N·L²) time and O(N·L) space (each word generates L bucket keys, each O(L) to construct).
Step 7 — Pattern Recognition: “Words differ by one letter” → implicit graph + pattern-bucket adjacency. “Shortest sequence” → BFS (not DFS). “Hard variant for huge dictionaries” → bidirectional BFS.
Step 8 — Develop:
def ladderLength(begin, end, wordList):
word_set = set(wordList)
if end not in word_set: return 0
L = len(begin)
# Build pattern → list of words
buckets = defaultdict(list)
for w in word_set:
for i in range(L):
buckets[w[:i] + "*" + w[i+1:]].append(w)
visited = {begin}
q = deque([(begin, 1)])
while q:
word, dist = q.popleft()
if word == end: return dist
for i in range(L):
key = word[:i] + "*" + word[i+1:]
for neigh in buckets[key]:
if neigh not in visited:
visited.add(neigh)
q.append((neigh, dist + 1))
buckets[key] = [] # optimization: each bucket processed once
return 0
Step 9 — Test: endWord not in dict (return 0), beginWord == endWord (return 1 per LC), single-letter words, no path exists.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Bucket Adjacency vs Bidirectional BFS
Bucket adjacency math: for word length L over alphabet Σ, naive adjacency needs O(L · |Σ|) tries per word to generate all 1-edit neighbors and check set membership. That’s O(N·L·|Σ|). The bucket approach pre-groups words by pattern: O(N·L) preprocessing, then O(L) per word during BFS. The win is that we don’t enumerate non-existent neighbors.
Bidirectional BFS math: if branching factor is b and shortest distance is d, ordinary BFS visits ~b^d nodes. Bidirectional BFS visits ~2·b^(d/2) — exponentially smaller for big d. Implementation:
- Maintain two frontiers,
forward(from begin) andbackward(from end). Initialize with {begin} and {end}. - At each step, EXPAND THE SMALLER FRONTIER (this is the crucial optimization).
- For each word in the expanded frontier, generate 1-edit neighbors. If a neighbor is in the OTHER frontier → answer found (current_distance + 1).
- Otherwise add to next-level frontier and visited set.
For LC 127, bidirectional BFS turns “TLE” into “instant” on the worst-case dictionaries. Discuss it as the Hard follow-up — implementation in solution.py.
Why BFS, not DFS: unweighted shortest path requires BFS. DFS gives a path but not the shortest; checking shortest with DFS requires exhaustive search (exponential).
The bucket-clear optimization: buckets[key] = [] after first visit ensures each bucket is processed at most once. Without it, the same set of neighbors can be re-checked through other words in the same bucket. With it, total work stays bounded by O(N·L²).
7. Anti-Pattern Analysis
- DFS instead of BFS. Gives a path, not the shortest. Or gives shortest but only after exponential exploration.
- O(N²·L) adjacency. Comparing every pair character-by-character. Times out on N=5000.
- Mutating
wordList(a list) for membership tests.in listis O(N);in setis O(1). Always convert. - Not checking
endWord in word_setupfront. Wastes a BFS run that’s doomed to fail. - Off-by-one on distance. LC counts the number of WORDS in the path (including both endpoints), not the number of edges.
hit→hot→cogis distance 3, not 2. - Forgetting to mark visited. Words re-enqueue infinitely. Mark when ENQUEUEING.
- Building buckets but iterating
wordList(list) instead of set during construction. Tiny perf issue, but ifwordListhas duplicates you process them twice. - Bidirectional BFS without “expand smaller frontier” optimization. Loses most of the speedup.
- Trying generic-alphabet brute neighbors (
for ch in 'abcdefghijklmnopqrstuvwxyz') without bucket pre-grouping. Works but wasteful — generates 26·L neighbors per word, most of which aren’t in the dictionary.
8. Skills & Takeaways
- Implicit graph recognition — nodes are not given as a list; edges are implied by a predicate.
- Pattern-bucket adjacency — pre-group items by a “with one feature missing” key for fast equality-up-to-one-feature.
- BFS = unweighted shortest path — DFS doesn’t.
- Bidirectional BFS — the search-tree pruning trick for big graphs.
- Set vs list for membership — O(1) vs O(N); never use list for hot-path lookups.
- Distance counting convention — clarify “edges” vs “vertices” in step 1.
9. When to Move On
Move on when:
- You can write bucket-BFS Word Ladder in under 10 minutes.
- You can sketch bidirectional BFS in under 15 minutes.
- You can articulate the math: why bidirectional is
~b^(d/2)vsb^d. - Your stress test runs naive-BFS, bucket-BFS, and bidirectional-BFS on 50 random small dictionaries and confirms all agree.
- You can pivot to LC 433 (Min Genetic Mutation — same algorithm with letters from
{A,C,G,T}) instantly. - You can sketch LC 126 (Word Ladder II — return ALL shortest paths) as a “BFS for distances, then DFS backtracking with predecessor map” extension.
10. Company Context
Amazon:
- Late-onsite. Bar Raiser frequently asks this exact problem to senior candidates.
- Watches for: do you start with naive then pivot, or jump straight to bucket? Both fine; Senior+ pivots to bucket within 5 min.
- Follow-up: LC 126 (all shortest paths). Very few candidates finish in time; signal is whether you can describe the algorithm cleanly even without finishing code.
- Scorecard phrase: “Algorithm + Insist on the Highest Standards — recognized implicit graph, used bucket adjacency, sketched bidirectional BFS as scale extension.”
Meta:
- Pairs with LC 433 (Min Genetic Mutation) in the same loop. Tests pattern transfer.
- Expects bidirectional BFS as the first approach for the “billions of words” variant.
- Scorecard phrase: “Strong — bucket BFS + bidirectional + LC 433 transfer.”
Google:
- May ask correctness proof: “prove bucket adjacency gives the same edges as pairwise comparison.” Phrase as “two words share a bucket key iff they agree at all but one position iff edit distance is exactly 1.”
- May ask: “what if the alphabet was huge (Unicode codepoints) so the wildcard approach explodes?” Discuss: alphabet doesn’t affect bucket construction; the bucket key length is L*L irrespective of |Σ|.
- Scorecard phrase: “Algorithm — clear bucket trick, correctness argued.”
Microsoft:
- Often the simpler variant: just need bucket-BFS. Bidirectional is bonus.
- Scorecard phrase: “Solid — bucket adjacency + BFS.”
Bloomberg:
- Frames in finance: “ticker symbols differ by one character — find shortest substitution chain between two symbols given a list of valid tickers.” Same algorithm.
- Scorecard phrase: “Pragmatic — applied bucket adjacency to ticker problem.”
Apple:
- Variant: “the cost of swapping each letter depends on the letter pair (Dvorak vs QWERTY distance).” Now it’s weighted shortest path → Dijkstra. Discuss the upgrade path.
- Scorecard phrase: “Strong — recognized when to upgrade BFS to Dijkstra.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | DFS | BFS w/ O(N²) adjacency | BFS w/ bucket adjacency | BFS w/ bucket + bidirectional |
| Adjacency | All-pair compare | All-pair (then bucket after hint) | Bucket first try | Bucket + clears buckets per visit |
| Distance | Off-by-one | Right | Right + states “I count vertices not edges” | Right + writes invariant comment |
endWord check | Doesn’t | Adds after debug | Adds upfront | Adds upfront + comments why |
| Scale extension | Stuck | “BFS scales fine” | Sketches bidirectional | Implements + explains b^d → 2b^(d/2) math |
12. Level Delta
Mid (L4):
- BFS with set-membership for wordList.
- Bucket adjacency (possibly after one hint).
- Correct distance counting.
- Handles
endWord not in wordList.
Senior (L5):
- Bucket adjacency first try.
- Clears buckets per visit (the small optimization).
- Sketches bidirectional BFS verbally if time runs out.
- States BFS-vs-DFS reasoning aloud.
Staff (L6):
- Implements bidirectional BFS cleanly.
- Discusses the “expand smaller frontier” optimization and why it matters.
- Sketches LC 126 (Word Ladder II) algorithm (BFS for distances + DFS backtracking via predecessor map).
- Discusses weighted variant (Dijkstra) for non-uniform letter-swap costs.
Principal (L7+):
- Reframes as edit-distance-1 graph + BFS shortest path, a special case of the broader “metric BFS” family.
- Discusses how this generalizes to A* with edit-distance heuristic for asymmetric letter alphabets.
- Discusses memory: bidirectional BFS doubles peak memory; trade off depending on whether memory or CPU is the bottleneck.
- Discusses production: spell-check / suggestion engines, autocorrect, DNA mutation pathway analysis, Levenshtein-1 indexes (FuzzyWuzzy library, Lucene’s FuzzyQuery).
13. Follow-up Q&A
Q1: Return ALL shortest paths (LC 126).
A: Two-phase: (1) BFS to compute distance from beginWord to every word, AND for each word remember its predecessors (the words at distance-1 that ARE valid 1-edits). (2) DFS from endWord backwards through the predecessor map to reconstruct all shortest paths. Time can be exponential in the number of shortest paths (a real combinatorial blowup is possible).
Q2: Implement bidirectional BFS.
A: Maintain two sets forward = {begin} and backward = {end}. Loop: pick the smaller set, generate all 1-edit neighbors in the dictionary, check if any is in the other set (success → return current step count + 1). Otherwise replace forward with the new neighbors and continue. ~2·b^(d/2) work instead of b^d.
Q3: What if wordList contains 10^7 words?
A: Bidirectional BFS becomes mandatory. Also: bucket construction memory becomes a concern (10^7 · 10 = 10^8 keys, each ~10 chars — gigabytes). Solutions: hash buckets to fixed-size integers, or stream bucket construction (don’t materialize all at once).
Q4: What if letters have unequal swap costs (e.g., neighboring keys cheaper)?
A: BFS → Dijkstra. Adjacency unchanged (still bucket-based). Replace deque with heapq. Cost vector cost[i][j] for swapping letter i to letter j.
Q5: Add a constraint: at most K consecutive same-letter swaps.
A: State-augmented BFS. Each state is (word, consecutive_same_position_swaps). Distance grows polynomially with K. Standard graph-product trick.
14. Solution Walkthrough
See solution.py:
ladder_length_bucket_bfs(begin, end, wordList)— canonical bucket BFS.ladder_length_bidirectional_bfs(begin, end, wordList)— bidirectional bucket BFS with “expand smaller frontier.”ladder_length_brute_bfs(begin, end, wordList)— O(N²·L) pairwise-compare BFS. Oracle.stress_test()— 50 random small dictionaries, all three implementations agree.edge_cases()—endWordnot in dict,begin == end, no path, single-letter words, LC canonical.
Run: python3 solution.py → “ALL TESTS PASSED”.
15. Beyond the Problem
Principal-engineer code review comment:
“The bucket trick is great, but please name it in a docstring. It’s called wildcard-pattern indexing or Levenshtein-1 inverted index, depending on which subfield you ask. Lucene calls it ‘fuzzy query.’ Our autocomplete system uses essentially this code path for ‘did you mean?’ suggestions. If you ever generalize it for our search service, please make the wildcard character configurable (we use
\\0because user queries can contain*), and please factor outbuild_buckets(words)so that the BFS code becomes 10 lines instead of 30. Also: yes, ship the bidirectional version too — our dictionary has 800k entries and the speedup is 100×.”
Connections forward:
- Phase 5 starts dynamic programming; many DP problems boil down to “BFS on a small state graph.”
- LC 433 (Min Genetic Mutation), LC 126 (Word Ladder II), LC 752 (Open the Lock), LC 854 (K-Similar Strings), LC 854 — all implicit graph BFS variants.
- Production: spell correction (Levenshtein-1 index), DNA mutation pathway analysis, password-cracking dictionaries (mutation chains), autocomplete suggestion engines.
- Theoretical: this is BFS on the Hamming-1 graph of strings; the same algorithm extends to BFS on the Levenshtein-k graph for general edit-distance shortest paths.
Hints — p25 Word Ladder
Read one at a time.
Hint 1 — Spot the graph. “Each step changes exactly one letter, intermediate word must be in dictionary, find SHORTEST sequence.” Words are nodes, edges connect words that differ by one letter, we want shortest distance. This is BFS on an implicit graph. Not DP, not DFS — BFS.
Hint 2 — Adjacency is the bottleneck.
The graph isn’t given. Naive way: for every pair (a, b) of words, check if they differ by one letter → O(N²·L). For N=5000, L=10, that’s 2.5×10⁸ ops — borderline TLE in Python. You need faster adjacency.
Hint 3 — Bucket trick (the key insight).
For each word, generate L patterns by replacing each letter with *. Example: "hit" → ["*it", "h*t", "hi*"]. Group all words by these patterns. Two words share a bucket iff they differ at exactly one position. Now adjacency lookup is O(L), not O(N·L).
Hint 4 — BFS bookkeeping.
- Mark visited when ENQUEUEING (not when dequeueing) to avoid duplicates.
- Track distance as
(word, dist)tuples in the queue. - Distance counts WORDS, including both endpoints — so
beginWordstarts at distance 1. - Check
endWord in wordListupfront; if absent, return 0 immediately.
Hint 5 — Bidirectional BFS (Hard follow-up).
BFS from both ends simultaneously. Maintain two frontiers. At each step, EXPAND THE SMALLER ONE (this is crucial — without it you lose most of the speedup). When a neighbor lands in the other frontier, you’ve found the meeting point — return distance. Complexity drops from b^d to 2·b^(d/2). For huge dictionaries this is the difference between TLE and instant.
If still stuck: look at ladder_length_bucket_bfs in solution.py — the bucket adjacency + BFS combo is the whole problem. Bidirectional is a bonus optimization.
Week 06 — Two Pointers + Sliding Window: Linear-Time Mastery
Month 02 · Week 06 of 24 · 5 flagship problems · Pattern focus: Two Pointers and Sliding Window
Theme: The two patterns that turn quadratic into linear.
Why this week matters
Two pointers and sliding window are the highest-ROI patterns in the interview — they appear in 30–40% of FAANG onsites by themselves or as building blocks. They share the same physical insight: instead of nested loops over (i, j) pairs, advance two indices in lockstep, each at most N times → O(N) total work.
What separates Mid from Senior+ on these problems is recognizing them. The trap is that they look like array problems, so candidates write nested loops and ship O(N²). The signal interviewers look for: do you reach for two pointers / sliding window first?
Specifically:
- Two pointers comes in three shapes — converging (3Sum, Container w/ Most Water), same-direction (slow/fast on linked lists, partition), and meeting (palindrome check).
- Sliding window comes in two shapes — fixed-size (count distinct in window of K) and variable-size (expand-then-contract on a constraint, e.g., longest substring with property P).
- The variable-size variant is where most candidates fail. The discipline: grow the window via right pointer until invariant breaks; shrink via left until invariant restored; record answer at every valid state.
This week’s problems are picked to span the full spectrum: 3Sum (converging two-pointer after sort), Container w/ Most Water (converging two-pointer with a non-obvious move-rule proof), Trapping Rain Water (converging two-pointer + the monotonic-stack alternative — a single problem with three distinct optimal algorithms), Longest Substring Without Repeating (variable-size sliding window, the canonical), Minimum Window Substring (variable-size sliding window, Hard).
Problem set
| # | Problem | LC # | Difficulty | Pattern | Why it’s here |
|---|---|---|---|---|---|
| p26 | 3Sum | 15 | Medium | Converging 2-pointer after sort | The most-asked array problem. Tests dedup discipline, sort-then-pivot recognition, off-by-one resilience. |
| p27 | Container With Most Water | 11 | Medium | Converging 2-pointer | The pattern in its purest form. Move-rule correctness PROOF is the senior signal. |
| p28 | Trapping Rain Water | 42 | Hard | Two pointers / Prefix-suffix max / Monotonic stack | A SINGLE problem with three legit optimal solutions. The interviewer learns more from you here than from any other problem. |
| p29 | Longest Substring Without Repeating Characters | 3 | Medium | Variable sliding window | The canonical variable-window. Master this and 70% of sliding-window problems are mechanical. |
| p30 | Minimum Window Substring | 76 | Hard | Variable sliding window | The “hardest easy problem.” Window + frequency counter + match counter. The discipline test. |
Daily schedule (suggested)
| Day | Focus | Time |
|---|---|---|
| Mon | p26 3Sum — read, attempt, code, stress | 90 min |
| Tue | p27 Container w/ Most Water — read, attempt, prove the move rule | 60 min |
| Wed | p28 Trapping Rain Water — attempt; then read all 3 optimal approaches | 120 min |
| Thu | p29 Longest Substring Without Repeating Chars — full FRAMEWORK pass | 75 min |
| Fri | p30 Minimum Window Substring — attempt; expect to need hints; that’s fine | 120 min |
| Sat | Re-attempt p28 and p30 from scratch in <30 min each. This is the readiness gate. | 90 min |
| Sun | Review — write 1-paragraph postmortem per problem in your own words. Anki cards. | 45 min |
Readiness gate (before Week 7)
You should be able to:
-
Write 3Sum in under 15 minutes with correct dedup (no
setof tuples cheat). - Prove (verbally) why Container With Most Water’s “move shorter pointer” rule is correct.
- Sketch ALL THREE Trapping Rain Water solutions — two-pointer, prefix-max, monotonic stack — and say one sentence on when each is preferable.
- Write Longest Substring Without Repeating in under 10 minutes from scratch using the canonical “char → last-index” map.
- Write Minimum Window Substring in under 25 minutes with the match counter optimization (not naive map equality on every shrink).
- State the two-pointer correctness invariant: “the answer cannot use any pair we’ve eliminated.”
- State the variable sliding-window template: expand right until invariant breaks → shrink left until invariant restored → record answer at every valid state.
If any of these are weak, do not advance. Re-do the failing problem and write the postmortem.
Anti-pattern hit-list (avoid)
- Using a hash set to dedup 3Sum triplets — works, but loses the “sorted-array dedup by skip” signal. Senior+ shows the skip-dedup.
- Moving both pointers simultaneously in Container w/ Most Water — drops the move-rule proof.
- Solving Trapping Rain Water with only ONE approach. The follow-up is always “what’s another way?”
- Variable sliding window where you forget to record the answer inside the while-loop (off-by-one trap).
- Minimum Window Substring with
map_a == map_bcheck on every shrink — that’s O(σ) per shrink → O(Nσ) total. Use a match counter that increments/decrements on each char.
Connections forward
- Week 7 — Prefix sums and binary search (extends the “scan once” mindset to range queries and search-on-monotone).
- Week 8 — Monotonic stacks and intervals (p28 Trapping Rain Water is the bridge problem).
- Phase 02 Lab — lab-01-two-pointers.md, lab-02-sliding-window.md.
p26 — 3Sum
Source: LeetCode 15 · Medium · Topics: Array, Two Pointers, Sorting Companies (2024–2025 frequency): Meta (very high), Amazon (very high), Google (high), Microsoft (high), Bloomberg (high), Apple (medium), LinkedIn (medium), Adobe (medium) Loop position: first round of onsite. Used as “warm up that filters.” Failing this signals lack of fluency.
1. Quick Context
Given an integer array, return all unique triplets [a, b, c] such that a + b + c == 0. Unique means the multiset of returned triplets has no duplicates.
This problem is the canonical “sort + converging two-pointer” pattern. It’s the highest-frequency Medium in the Meta and Amazon loops because it tests THREE separate skills in 15 minutes:
- Pattern recognition: “find pairs/triplets with sum” → sort + two-pointer, not brute O(N³).
- Dedup discipline: the array can contain duplicates. The output must not. Three places to dedup correctly (outer
i, innerlo, innerhi); off-by-one in any of them fails. - Off-by-one resilience under pressure:
while lo < hi, not<=. After finding a triplet, BOTH pointers must advance.
The interviewer’s signal isn’t “did you solve it” — it’s “did you solve it FAST and CLEAN.” Senior+ writes it in 12 minutes without a bug; Mid writes it in 22 minutes with one off-by-one to fix.
The classic anti-pattern: dedup using set(tuple(sorted(triplet)) for triplet in results). It works, but loses the sorted-array dedup-by-skip signal that the interviewer wants to see.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/3sum/
STOP. 25-min timer. Sort + converging two-pointer; dedup by skip. Don’t peek.
3. Prerequisite Concepts
- Two Sum (p01) — basic pair-sum via hash.
- Sort —
list.sort()is O(N log N) in-place, Timsort. - Converging two-pointer template:
lo, hi = 0, n-1; while lo < hi: ... - phase-02-patterns/labs/lab-01-two-pointers.md
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given an integer array nums, return all UNIQUE triplets (i, j, k) with i < j < k such that nums[i] + nums[j] + nums[k] == 0. Returned triplets should be the VALUES, deduplicated as multisets, not the indices.”
Step 2 — Clarify:
- “Sorted output required?” (LC: no, but
numsis freely mutable so sorting is fine.) - “Can input have duplicates?” (LC: yes. Output triplets must not duplicate.)
- “What counts as ‘duplicate triplet’?” (Same multiset of values.
[-1, 0, 1]and[0, 1, -1]are duplicates.) - “Empty / fewer than 3 elements?” (Return
[].) - “Integer range?” (LC: -10⁵ to 10⁵. No overflow concerns in Python.)
Step 3 — Constraints: N ≤ 3000 for LC. Brute O(N³) = 2.7×10¹⁰ — TLE. O(N²) = 9×10⁶ — fast. Need O(N²).
Step 4 — Examples:
[-1, 0, 1, 2, -1, -4]→[[-1, -1, 2], [-1, 0, 1]][0, 0, 0]→[[0, 0, 0]][0, 0, 0, 0]→[[0, 0, 0]](dedup!)
Step 5 — Brute Force: Three nested loops, hash-set dedup. O(N³) time, O(N²) extra. State, then pivot.
Step 6 — Complexity (optimal): O(N²) time, O(1) auxiliary (output excluded), O(N log N) for the sort (subsumed).
Step 7 — Pattern Recognition: “Find all triplets summing to target” → fix one element, two-sum the rest with sorted converging two-pointer. Sort first. For each i, find pairs (lo, hi) with nums[lo] + nums[hi] == -nums[i]. Dedup by skipping equal-value runs.
Step 8 — Develop:
def threeSum(nums):
nums.sort()
n = len(nums)
out = []
for i in range(n - 2):
if i > 0 and nums[i] == nums[i-1]: continue # outer dedup
if nums[i] > 0: break # pruning: positives can't sum to 0 with bigger
target = -nums[i]
lo, hi = i + 1, n - 1
while lo < hi:
s = nums[lo] + nums[hi]
if s == target:
out.append([nums[i], nums[lo], nums[hi]])
lo += 1; hi -= 1
while lo < hi and nums[lo] == nums[lo-1]: lo += 1 # inner-left dedup
while lo < hi and nums[hi] == nums[hi+1]: hi -= 1 # inner-right dedup
elif s < target:
lo += 1
else:
hi -= 1
return out
Step 9 — Test: dup 0s ([0,0,0,0]), all negative, all positive, mixed, empty, just-2-elements.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why “sort, then converging two-pointer” is optimal
The lower bound for 3Sum is believed to be O(N²) — this is the famous 3SUM conjecture in computational geometry. Many problems reduce to 3SUM, and a sub-O(N²) solution would imply faster algorithms for several geometric problems (collinear points, GeomBase, etc.). As of 2024, the best known is O(N² / (log N / log log N)²) via Grønlund-Pettie — i.e., we have not even broken O(N²) by a polylog factor.
So when the interviewer asks “can you do better than O(N²)?” — the answer is “no, modulo a famous open conjecture in computational geometry.” Senior+ candidates name-drop this. Staff+ adds: “the GP bound shaves a polylog factor but isn’t practical.”
Why the move rule works: After sorting, fix nums[i]. For the remaining suffix, we want pairs summing to -nums[i]. At any state (lo, hi):
- If
nums[lo] + nums[hi] < target, no pair(lo, hi')forhi' < hican reach target (sum would be even smaller). SoloMUST advance. - Symmetric for
>case. - If
==, we found one. We must advance BOTH (advancing only one would mean staying at(lo+1, hi)wherenums[lo+1] + nums[hi]is either bigger than target —himust advance — or equal — we’d re-record the same triplet after dedup; so jointly advancing is correct).
Why dedup-by-skip beats hash-set: hash-set works in O(1) per insertion but loses the structural property of sortedness. The skip approach is O(1) too, but it’s a visible algorithmic technique that the interviewer reads as “candidate understands invariants.”
7. Anti-Pattern Analysis
setof sorted tuples for dedup. Works, gets the answer, but loses the senior signal. Show skip-dedup explicitly.while lo <= hiinstead of<. With<=, you’d consider(i, i, i)triplets which violatei < j < k. Subtle, hangs in interview.- Forgetting to advance BOTH
loandhiafter a hit. Infinite loop or duplicate triplet. - Dedup only on
i, not onlo/hi. Misses cases like[-2,0,0,2,2]→[-2,0,2]would appear twice. - Breaking on
nums[i] > 0early without sorting first. Pruning is correct only if sorted. - Hashing each
nums[i]value to avoid outer dup. Works but adds a hash table; the skip is cleaner. - Forgetting empty /
n < 3. Doesn’t crash withfor i in range(n-2)ifn <= 2, but should be mentioned. - Sorting and then INDEXING the original array. After sort, indices are different. If interviewer asks for indices (Two-Sum-like), you’d need to track original indices via a sort-of-pairs.
- N³ brute with
if (a, b, c) not in seen. Membership in a list is O(N), so it’s O(N⁴). Use asetforseen.
8. Skills & Takeaways
- Sort-then-pivot as a meta-pattern: when a problem allows multiple comparisons / dedup, sorting first often unlocks O(N²) from O(N³).
- Converging two-pointer as the inner loop after fixing one index.
- Dedup by skipping equal-value runs in sorted arrays.
- Move-rule correctness arguments (“if
s < target, no smallerhi'can fix it”). - The 3SUM conjecture as a piece of trivia that signals depth.
- Pruning: once
nums[i] > 0, break — any triplet would have positive sum.
9. When to Move On
Move on when:
- You can write 3Sum in under 12 minutes with no bugs.
- You can articulate the move rule’s correctness without notes.
- You can write 4Sum (LC 18) by induction on the 3Sum solution in another 10 min.
- Your stress test compares against the O(N³) brute on 200 random small arrays.
- You can mention the 3SUM conjecture and what it implies.
10. Company Context
Meta:
- Top 5 most-asked problem at Meta period. Almost certain to see it or a variant (3Sum Closest, 3Sum Smaller, 4Sum) in any algorithm round.
- Watches for: time to first correct submission, dedup discipline, off-by-one errors.
- Follow-ups always include “what if k=4?” and “what if we want the count, not the triplets?”
- Scorecard phrase: “Strong — clean sort + 2-pointer, all three dedups, generalized to k-Sum.”
Amazon:
- High-frequency in mid-loop “coding bar” round. Bar Raiser may add LC 18 (4Sum) as follow-up.
- Watches dedup discipline obsessively. Failing dedup = “no hire.”
- Scorecard phrase: “Algorithm + Dive Deep — recognized 2-pointer immediately, correct skip-dedup, discussed 3SUM lower bound.”
Google:
- Often comes with theoretical follow-up: “is O(N²) optimal? prove or disprove.” Cite 3SUM conjecture.
- May ask for the COUNT of triplets (no need to enumerate) — same algorithm, just increment counter.
- Scorecard phrase: “Algorithm — solid + mentioned 3SUM lower bound.”
Microsoft:
- Cleaner variant: just want the algorithm, less dedup gotcha.
- Scorecard phrase: “Strong — sort + 2-pointer.”
Bloomberg:
- Frames as financial: “find three trades with net P&L zero.” Same algorithm.
- Often paired with LC 1 (Two Sum) in same session as the warmup-then-real-thing test.
- Scorecard phrase: “Pragmatic — 2-pointer, financial application discussed.”
Apple:
- Frames as data analytics: “find three sensor readings averaging to baseline.” May ask floating-point version (epsilon comparison).
- Scorecard phrase: “Strong — clean 2-pointer, discussed numerical tolerance.”
LinkedIn:
- Often a 30-min portion paired with another problem. Pressure round.
- Scorecard phrase: “Solid under time pressure.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | N³ nested loops | “I think two-pointer?” | Sort + 2-pointer immediately | Sort + 2-pointer + states pruning |
| Dedup | Set of tuples | Set of tuples; then skip after hint | All three skip-dedups, first try | Skip-dedup + explains why hash-set is “less clean” |
| Move rule | Doesn’t explain | Hand-waves | Proves: “smaller sum, lo must advance” | Proves + connects to invariant |
| Complexity | “I dunno” | O(N²) | O(N²), notes sort subsumed | O(N²), notes 3SUM conjecture |
| k-Sum follow-up | Stuck | Re-codes 4Sum | Sketches generic k-Sum recursion | Implements generic k-Sum |
12. Level Delta
Mid (L4):
- Sort + converging 2-pointer.
- Dedup works (may use set of tuples).
- One off-by-one to fix in interview.
- States O(N²) complexity.
Senior (L5):
- Skip-dedup at all three positions, first try.
- Articulates move-rule correctness.
- States complexity precisely including sort cost.
- Handles edge cases proactively.
Staff (L6):
- Generalizes to k-Sum via recursion in 10 minutes.
- Discusses 3SUM conjecture.
- Discusses 3Sum Closest, 3Sum Smaller variants.
- Notes pruning optimizations (
nums[i] > 0break,nums[i] + nums[i+1] + nums[i+2] > 0break).
Principal (L7+):
- Discusses 3SUM-hardness as a reduction concept (problems reducible to 3SUM).
- Discusses Grønlund-Pettie’s
O(N² / (log N / log log N)²)bound — and why it’s not practical. - Discusses parallelization: 3Sum is embarrassingly parallel on
i; mention map-reduce style. - Discusses floating-point variant numerical issues (Kahan summation, epsilon comparison).
13. Follow-up Q&A
Q1: 4Sum (LC 18) — pairs summing to target. A: Wrap one more outer loop, dedup it, then call the 2-pointer inner. O(N³). Generic k-Sum is O(N^(k-1)).
Q2: 3Sum Closest (LC 16) — return sum closest to target.
A: Same structure. Track best_diff instead of equality. O(N²). When s == target, return immediately.
Q3: 3Sum Smaller (LC 259) — COUNT of triplets with sum < target.
A: After sort, fix i. For two-pointer (lo, hi): if nums[lo] + nums[hi] < target, ALL (lo, lo+1..hi) work — add hi - lo to count, advance lo. Else advance hi. O(N²).
Q4: Can we do better than O(N²)?
A: Open. 3SUM conjecture believes no. Best known is Grønlund-Pettie’s O(N² / (log N / log log N)²). A sub-quadratic solution would break decades of work in computational geometry.
Q5: What if nums is a stream (no random access)?
A: We need 2 passes if 1-pass isn’t possible. First pass: collect into list (O(N) memory). Then sort + 2-pointer. If memory is constrained, this becomes external sort + then a different algorithm — challenging.
14. Solution Walkthrough
See solution.py:
three_sum(nums)— canonical sort + converging 2-pointer with skip-dedup.three_sum_hashset(nums)— alternative using hash for inner two-sum (worse constants, same Big-O, kept for comparison).three_sum_brute(nums)— O(N³) oracle for stress.k_sum(nums, target, k)— generalized k-Sum via recursion (bonus).stress_test()— 200 random arrays N≤15, all variants agree.edge_cases()— empty, n<3, all zeros, duplicates, all negative, all positive, LC canonical.
Run: python3 solution.py → “ALL TESTS PASSED”.
15. Beyond the Problem
Principal-engineer code review comment:
“The skip-dedup is correct but please add INVARIANT comments. New hires copy this code into adjacent problems (4Sum, 3SumClosest, kSum) and the dedups get subtly wrong without the comments. Specifically: ‘invariant: after this advance, nums[lo] != nums[lo-1], so the next triplet will not be a duplicate of the just-recorded one.’ Also — the k-Sum recursion you wrote is correct but allocates a new list at each level; for our hot-path analytics service this matters. Use indices and accumulate into a single output list. Last point: the 3SUM conjecture is interesting trivia but please don’t put it in the docstring; the next on-call engineer doesn’t care.”
Connections forward:
- Phase 02 — lab-01-two-pointers (this problem is the keystone example).
- Phase 06 — Greedy has problems where sort-then-scan is the optimal structure; same skill.
- 3SUM reductions show up in computational geometry: collinear points, distinct distances, GeomBase.
- Production: financial reconciliation (3 trades netting to zero), sensor anomaly detection (3 readings summing to baseline), audit logs (3 events with combined impact 0).
- Theoretical: 3SUM-hardness is a complexity class — many geometric problems are 3SUM-hard.
Hints — p26 3Sum
Read one at a time.
Hint 1 — Don’t go cubic. Three nested loops is O(N³) = 2.7×10¹⁰ at N=3000 — TLE. You need O(N²). When you see “find triples with sum = target,” the meta-pattern is fix one element, two-sum the rest.
Hint 2 — Sort first. Sorting (O(N log N), absorbed into O(N²)) unlocks two things: (a) converging two-pointer for the inner two-sum (no hash needed), and (b) dedup by skipping equal-value runs.
Hint 3 — Three dedups, not one.
Duplicates appear in three places. (a) Outer loop: if i > 0 and nums[i] == nums[i-1]: continue. (b) After a hit, advance lo past equal values: while lo < hi and nums[lo] == nums[lo-1]: lo += 1. (c) Same for hi. Missing any → duplicate triplets in output.
Hint 4 — Move rule (and why it works).
At state (i, lo, hi), compute s = nums[lo] + nums[hi]. If s < -nums[i], the sum is too small — lo must advance (any smaller hi would only shrink the sum further). If s > -nums[i], hi must retreat. If ==, record the triplet and advance BOTH.
Hint 5 — Loop bounds + pruning.
while lo < hi (strict, not <=). Outer loop bound: range(n - 2) — last i is n-3. Pruning: once nums[i] > 0, no remaining triplet can sum to 0 (sorted ascending means rest are all ≥ 0) — break.
If still stuck: look at three_sum in solution.py — it’s 20 lines and the dedups are commented.
p27 — Container With Most Water
Source: LeetCode 11 · Medium · Topics: Array, Two Pointers, Greedy Companies (2024–2025 frequency): Google (very high), Amazon (high), Meta (high), Bloomberg (high), Microsoft (medium), Apple (medium) Loop position: first round — used as a “do you reach for two-pointers” filter. Senior+ also asked to PROVE the move rule.
1. Quick Context
Given n non-negative integers representing heights of vertical lines on the x-axis, find two lines that together with the x-axis form a container holding the maximum amount of water.
Container area between lines i < j is min(height[i], height[j]) * (j - i).
This is the purest form of converging two-pointer. The problem is small, but the proof of the move rule is what separates Mid from Senior+. Mid candidates implement it correctly by intuition; Senior+ explains why moving the shorter pointer is correct (and why moving the taller one is incorrect).
The interviewer’s bait: “why does this work?” If you can’t articulate the invariant, your “correct” code is treated as a lucky guess. Have the proof ready.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/container-with-most-water/
STOP. 20-min timer. Two pointers from opposite ends. Be ready to PROVE the move rule before reading hints.
3. Prerequisite Concepts
- Converging two-pointer template (
lo, hi = 0, n-1; while lo < hi). - The “area = min(h_lo, h_hi) × width” formula.
- phase-02-patterns/labs/lab-01-two-pointers.md
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given non-negative integers height[0..n-1], find indices i < j maximizing min(height[i], height[j]) * (j - i). Return that maximum.”
Step 2 — Clarify:
- “Can heights be zero?” (LC: yes. Zero-height lines hold no water.)
- “Can
nbe 1?” (LC:n ≥ 2.) - “Negative heights?” (No — non-negative.)
- “Return the area or the indices?” (LC: the area.)
Step 3 — Constraints: N ≤ 10⁵ for LC. Brute O(N²) = 10¹⁰ — TLE. Need O(N).
Step 4 — Examples:
[1,8,6,2,5,4,8,3,7]→ 49 (lines at index 1 height 8 and index 8 height 7: min=7, width=7).[1, 1]→ 1.[4, 3, 2, 1, 4]→ 16 (outermost lines).
Step 5 — Brute Force: All pairs (i, j), compute area, track max. O(N²). State, then pivot.
Step 6 — Complexity (optimal): O(N) time, O(1) space.
Step 7 — Pattern Recognition: “Maximize over (i, j) pairs with a function depending on the WORSE of two endpoints and the distance between them” → converging two-pointer. Start at the widest pair, narrow inward.
Step 8 — Develop:
def maxArea(height):
lo, hi = 0, len(height) - 1
best = 0
while lo < hi:
h = min(height[lo], height[hi])
best = max(best, h * (hi - lo))
# Move the SHORTER pointer (proof: see deeper insight)
if height[lo] < height[hi]:
lo += 1
else:
hi -= 1
return best
Step 9 — Test: [1,1] → 1, [4,3,2,1,4] → 16, monotone increasing [1,2,3,4,5], all-equal [5,5,5,5].
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why moving the SHORTER pointer is correct
The move rule: at state (lo, hi), if height[lo] < height[hi], advance lo. Otherwise retreat hi.
The proof (the part interviewers want to hear):
Let h = min(height[lo], height[hi]). WLOG suppose height[lo] ≤ height[hi], so h = height[lo]. The current area is h * (hi - lo).
Claim: every pair (lo, hi') with hi' < hi has area ≤ h * (hi - lo).
Proof:
- The width of
(lo, hi')ishi' - lo < hi - lo. - The container height is
min(height[lo], height[hi']) ≤ height[lo] = h. - So area
(lo, hi') = min(h, height[hi']) * (hi' - lo) ≤ h * (hi' - lo) < h * (hi - lo).
So no pair (lo, *) to the LEFT of hi can beat the current best — we can safely eliminate the entire column at lo by moving lo forward. The candidate set shrinks by N−hi+lo−1 pairs in one move; over the run, we examine N pairs total, achieving O(N).
The symmetric argument applies when height[hi] < height[lo].
Why moving the TALLER pointer is incorrect: if we move hi when height[hi] > height[lo], we keep lo fixed. But the new hi-1 could have a larger min(height[lo], height[hi-1]) only if height[hi-1] > height[hi] — but the container height is still capped by height[lo] (which didn’t change). So min(h_lo, h_new_hi) ≤ h_lo = h, but width shrank — strict loss. We’d miss real opportunities at (lo+1, hi).
Tie case (height[lo] == height[hi]): moving EITHER is correct — the proof above goes through with ≤ rather than <. Convention: move lo.
Invariant statement: “At every iteration, the optimal pair lies within the slice [lo, hi].” This is the loop invariant. It’s preserved by the move rule.
7. Anti-Pattern Analysis
- O(N²) brute and shipping it. Works on small inputs, TLE on big.
- Moving both pointers each iteration. Misses the move-rule subtlety. Wrong answer.
- Moving the TALLER pointer. Provably wrong (see proof). Common rookie error.
- Computing area as
max(h_lo, h_hi) * width. Container fills to MIN, not MAX. - Off-by-one width:
hi - lo + 1instead ofhi - lo. Width = index distance, not inclusive count. while lo <= hi. Atlo == hi, width = 0, area = 0. Wastes an iteration but harmless.<is cleaner.- Forgetting tie case. Code
if height[lo] < height[hi]thenelsecovers ties correctly;if <=would be wrong only stylistically. - “Greedy doesn’t work; let me DP.” DP gives the same O(N²) brute. Two pointers IS the optimal here; recognize it.
8. Skills & Takeaways
- Converging two-pointer as a “candidate elimination” pattern (each move eliminates a row/column of the pair-grid).
- The shorter-pointer move rule with proof.
- Loop invariant articulation — “optimal pair is in
[lo, hi].” - Recognizing greedy is provably optimal in monotone-elimination problems.
- Distinguishing this from Trapping Rain Water — that uses two pointers too, but with different state (running max from each side).
9. When to Move On
Move on when:
- You can write
maxAreain under 5 minutes from a blank file. - You can prove the move rule out loud without referencing notes.
- You can articulate the invariant: “the optimal pair is in
[lo, hi].” - You can explain why this problem is DIFFERENT from Trapping Rain Water (different state, different invariant).
- Your stress test compares to O(N²) brute on 200 random arrays.
10. Company Context
Google:
- Asked at every level. Senior+ MUST prove the move rule. “Why does this work?” is the killer follow-up.
- Often paired with LC 42 (Trapping Rain Water) in same round — the contrast is the lesson.
- Scorecard phrase: “Algorithm — clean 2-pointer, MOVE-RULE PROVED rigorously.”
Amazon:
- Bar Raiser sometimes asks “why not move both?” — testing whether you’ve memorized vs understood.
- Scorecard phrase: “Strong — recognized 2-pointer, articulated correctness.”
Meta:
- Often in same round as LC 15 (3Sum). Both test 2-pointer fluency.
- Scorecard phrase: “Algorithm — fluent 2-pointer.”
Bloomberg:
- Frames as “two trading days bracketing a position” or “bandwidth between two routers.” Same algorithm.
- Scorecard phrase: “Strong — pragmatic 2-pointer with proof.”
Microsoft / Apple:
- Standard variant. Watches for correct width formula and tie handling.
- Scorecard phrase: “Solid.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | Nested loops | Two-pointer (correct or not) | Two-pointer with correct move rule | 2-pointer + states invariant before coding |
| “Why this move?” | “It works on examples” | “Moving short side gives chance for taller” | Proves: “fixed-lo eliminates entire (lo, *) column” | Proves + states loop invariant + says “candidate elimination” |
| Tie case | Doesn’t think about it | “Probably doesn’t matter” | “Either move is correct — proof goes through with ≤” | Names the tie convention and consequences |
| Variant | Stuck | “Trapping Rain Water is the same?” (wrong) | Distinguishes immediately | Maps both problems to a shared “two-pointer with state” template |
12. Level Delta
Mid (L4):
- Two-pointer, correct move rule by intuition.
- Correct width formula.
- O(N), O(1) stated.
Senior (L5):
- Articulates the move rule’s correctness in plain English.
- States the loop invariant.
- Distinguishes this from Trapping Rain Water.
- Handles tie case correctly.
Staff (L6):
- Formal proof (the “(lo, hi’) has area ≤” argument).
- Notes this generalizes to “maximize over pairs where the objective is monotone in one endpoint” — name it candidate elimination.
- Discusses what changes if widths are weighted unevenly (e.g., x-coordinates are not 0..n-1).
- Discusses the 3D analog (cuboid container).
Principal (L7+):
- Frames as a special case of monotone matrix searching (or SMAWK adjacent territory).
- Discusses generalization: “maximize
f(h_i, h_j, j - i)wherefis monotone inmin(h)and in width” — same algorithm works. - Discusses streaming variant: heights arrive online; harder, requires segment tree or fancy data structure.
- Discusses parallelization: not embarrassingly parallel (the move depends on previous state), but can be done with divide-and-conquer in O(N log N).
13. Follow-up Q&A
Q1: Heights arrive online (streaming). What now?
A: Two pointers no longer apply (need to retract the right pointer, but new data only comes from the right). Maintain a monotone-decreasing stack of “still in contention” indices on the left (only indices with strictly increasing heights from left can possibly be optimal lo for any future hi). For each new index, binary-search the stack for the best lo to pair with it. O(N log N) overall.
Q2: 3D analog — heights are now a 2D grid, container holds a 3D volume. A: This is essentially Trapping Rain Water II (LC 407). Algorithm: min-heap from the boundary inward (Dijkstra-like). O(MN log MN).
Q3: Widths are weighted — x-coordinate of column i is x[i], not i.
A: Two-pointer still works. Width = x[hi] - x[lo]. Move rule unchanged: still move the shorter height. Proof unchanged (only width factor differs, monotone-decreasing as we move inward).
Q4: Return the indices, not the area.
A: Trivial wrap — track best_pair = (lo, hi) alongside best.
Q5: Top-K largest containers (no two share a side). A: Much harder — combinatorial. Likely DP or greedy with bookkeeping. Discuss complexity tradeoffs.
14. Solution Walkthrough
See solution.py:
max_area(height)— canonical converging two-pointer.max_area_brute(height)— O(N²) oracle.max_area_streaming(heights)— streaming variant with monotone stack + binary search (O(N log N)).stress_test()— 500 random arrays N≤30, all variants agree.edge_cases()— n=2, all-equal, monotone, LC canonical, all-zero.
Run: python3 solution.py → “ALL TESTS PASSED”.
15. Beyond the Problem
Principal-engineer code review comment:
“The 2-pointer code is fine. What I want to see in the PR description, every time, is the move-rule proof as a one-line comment:
# inv: optimal pair lies in [lo, hi]; moving the shorter side preserves this. Without that comment, the next engineer who tries to ‘optimize’ this code by moving both pointers each step (which I’ve seen TWICE in our codebase) breaks it without realizing. The streaming variant you mention is the actual production case for our network-monitoring tool — please factor it into a separate function with its own tests, don’t bury it as a ‘bonus.’”
Connections forward:
- p28 — Trapping Rain Water uses the same outer skeleton (two pointers) but maintains different state (running max from each side). The contrast is the lesson.
- Phase 02 — lab-01-two-pointers; lab-05-monotonic-stack (the streaming variant).
- Phase 06 — Greedy; this problem is a textbook “greedy proven optimal by candidate-elimination.”
- Production: rate-limit window selection (find two timestamps that bound the largest allowed throughput window), audio sample peak-pair detection, image cropping (find two vertical lines containing the widest valid region).
Hints — p27 Container With Most Water
Read one at a time.
Hint 1 — Start wide, narrow inward.
Area depends on (min(h_lo, h_hi)) * (hi - lo). The widest pair starts at lo=0, hi=n-1. As you narrow, width strictly decreases — so the only way to find a bigger area is for min(h_lo, h_hi) to increase. Two-pointer.
Hint 2 — Move which pointer? You must move one. Moving the TALLER pointer keeps the container height capped by the SHORTER side (it can’t grow), and width shrinks — strict loss. So move the SHORTER pointer.
Hint 3 — Tie case.
When h_lo == h_hi, moving either is correct (the proof goes through with ≤). By convention, move lo. Code: if h[lo] < h[hi]: lo += 1 else: hi -= 1.
Hint 4 — Prove the move rule.
WLOG h_lo ≤ h_hi. For any hi' < hi: width (lo, hi') is smaller AND container height min(h_lo, h_{hi'}) ≤ h_lo. So area (lo, hi') ≤ h_lo * (hi - lo) = current area. Every pair (lo, *) is dominated, so eliminate them all by moving lo. Be ready to recite this in the interview.
Hint 5 — Width formula.
Width is hi - lo, not hi - lo + 1. (Two adjacent lines at indices 0, 1 are 1 apart.) Easy off-by-one in interview.
If still stuck: look at max_area in solution.py — 10 lines, plus the deeper insight section in README.md for the full proof.
p28 — Trapping Rain Water
Source: LeetCode 42 · Hard · Topics: Array, Two Pointers, Dynamic Programming, Stack Companies (2024–2025 frequency): Google (very high), Amazon (high), Meta (high), Microsoft (high), Bloomberg (high), Apple (medium) Loop position: algorithm round, often the killer. This is the showcase problem.
1. Quick Context
Given n non-negative integers representing the elevation map where the width of each bar is 1, compute how much water it can trap after raining.
This problem is in a class of its own. It has three legitimate O(N) optimal solutions — each using a different algorithmic technique:
- Prefix-suffix max arrays — O(N) time, O(N) extra space. Classic.
- Two pointers — O(N) time, O(1) extra space. Elegant.
- Monotonic decreasing stack — O(N) time, O(N) space. Processes water in horizontal strips.
The interviewer asks Trapping Rain Water specifically to see how many you know and which one you reach for first. Knowing one is Mid; knowing two is Senior; knowing all three and explaining when to prefer each is Staff+.
The core insight (shared by all three): water above index i is min(maxLeft[i], maxRight[i]) - height[i]. Each solution differs in how it computes that min.
This is also the bridge problem to monotonic stacks (Phase 02 Lab 05) — the stack solution previews patterns used in Largest Rectangle in Histogram, Daily Temperatures, etc.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/trapping-rain-water/
STOP. 30-min timer. This is Hard — try the prefix-max array first. The two-pointer optimization is hard to invent under pressure.
3. Prerequisite Concepts
- p27 — Container With Most Water (warmup; two-pointer pattern).
- Prefix sums / prefix maxes — running max from left and right.
- Monotonic stacks — phase-02-patterns/labs/lab-05-monotonic-stack.md.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given a histogram height[0..n-1] of non-negative integers (each bar width 1), return the total volume of water trapped between bars after rain.”
Step 2 — Clarify:
- “What does ‘trapped’ mean physically?” — for each column
i, water level ismin(tallest bar to the left, tallest bar to the right); trapped water atiismax(0, that_level - height[i]). - “Can heights be zero?” (Yes.)
- “Total volume or per-column?” (Total, an integer.)
- “Width of each bar?” (1 — fixed.)
- “Negative heights?” (No.)
Step 3 — Constraints: N ≤ 2×10⁴ for LC. All O(N) approaches comfortable.
Step 4 — Examples:
[0,1,0,2,1,0,1,3,2,1,2,1]→ 6.[4,2,0,3,2,5]→ 9.[3,0,2,0,4]→ 7.[]→ 0.[1]→ 0.[5,5,5]→ 0.
Step 5 — Brute Force: For each index i, scan left for max, scan right for max, add min(L,R) - h[i]. O(N²). State, then pivot.
Step 6 — Complexity (optimal): O(N) time. Space varies by approach: O(N) (prefix arrays), O(1) (two-pointer), O(N) (stack).
Step 7 — Pattern Recognition: “Each cell’s contribution depends on a property of its left AND right context” → prefix-max array (precompute), or two-pointer (eliminate as you go), or monotonic stack (process boundaries as they’re confirmed).
Step 8 — Develop: See solution.py. All three implementations side-by-side.
Step 9 — Test: monotone increasing (0 water), monotone decreasing (0), V-shape, W-shape, plateaus, all-equal, empty.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Three solutions, one problem
6.1 The shared insight
For each column i, the water level above it is determined by the tallest bar to the left and the tallest bar to the right:
$$\text{water}(i) = \max(0, \min(\text{maxLeft}(i), \text{maxRight}(i)) - h[i])$$
Total water = sum over all i. The three solutions are three ways to evaluate this formula efficiently.
6.2 Solution A — Prefix-suffix max arrays (O(N), O(N) space)
Precompute maxLeft[i] = max(h[0..i]) in one left-to-right pass, maxRight[i] = max(h[i..n-1]) in one right-to-left pass. Then sum the formula in a third pass.
Why it’s the “default” solution: clearest and most directly mirrors the formula. Easy to write, easy to debug, easy to test. Use this when interviewer doesn’t push for space optimization.
6.3 Solution B — Two pointers (O(N), O(1) space)
Maintain lo, hi, leftMax, rightMax. Loop while lo < hi:
- If
height[lo] < height[hi]: the water atlois bounded byleftMax(becauserightMax ≥ height[hi] > height[lo], sorightMaxis at least as big asheight[hi]which dominatesheight[lo], so the min isleftMax). UpdateleftMax, accumulateleftMax - height[lo], advancelo. - Symmetric for the other side.
The subtle correctness argument: the move rule is “advance the side with the smaller current height.” This guarantees that the side we’re advancing has a confirmed taller side on the other end — so the water level at the advancing column is determined entirely by leftMax (or rightMax) on the same side. We don’t need the global rightMax because we know it’s at least height[hi] which already exceeds height[lo].
When to use: when interviewer asks “can you do it in O(1) space?” or as the senior signal.
6.4 Solution C — Monotonic decreasing stack (O(N), O(N) space)
Maintain a stack of indices with strictly decreasing heights. Walk left to right; when the current bar h[i] exceeds the top of the stack h[top]:
- Pop
top. The popped index is a “trough bottom.” - The “left wall” is now the new top of the stack; the “right wall” is
i. - Water added at this trough =
(min(h[new_top], h[i]) - h[top]) * (i - new_top - 1). (A horizontal strip.) - Repeat while the current bar still exceeds the (new) top.
This processes water in horizontal strips, one trough at a time. Each index is pushed once and popped once → O(N) total.
When to use:
- When the same problem is generalized (e.g., Largest Rectangle in Histogram — same stack pattern).
- When the interviewer specifically asks about the stack approach as a follow-up.
- When you want to demonstrate range of techniques.
6.5 Comparing the three
| Prefix arrays | Two pointers | Stack | |
|---|---|---|---|
| Time | O(N) | O(N) | O(N) |
| Space | O(N) | O(1) | O(N) |
| Conceptual difficulty | Easy | Medium-hard (move-rule proof) | Hard (horizontal strips) |
| Generalizes to | Range-query problems | LC 11 (Container) | LC 84 (Histogram), LC 496 (Next Greater), Daily Temperatures |
| Recommend interview | First attempt | Senior signal | Bonus / asked follow-up |
7. Anti-Pattern Analysis
- O(N²) brute and shipping it. Easy to write but TLE on big inputs.
- Computing
max(maxLeft, maxRight)instead ofmin. Water level is the SHORTER wall (water spills over the shorter). Sign error. - Forgetting
max(0, ...)— whenh[i]exceeds both walls, the formula gives a negative; clamp to 0. - Two-pointer with
leftMax = max(leftMax, height[lo])AFTER computing water. Order matters: updateleftMaxFIRST, then computeleftMax - height[lo](which is now ≥ 0). - Stack approach: forgetting the
* (i - new_top - 1)width factor. Strip width is(right_wall - left_wall - 1). - Stack approach: pushing without checking equality. Equal-height stack tops are tricky — usually push (the formula handles equal heights naturally since strip height = 0).
- Two-pointer with
lo <= hi. Atlo == hi, the column has walls on both sides equal to its own height — no water — wasted iteration.<cleaner. - Reading the input as floats when problem says non-negative integers. Rookie input-handling slip.
8. Skills & Takeaways
- Prefix-max arrays as a meta-pattern (extends to suffix-max, range queries, etc).
- Two-pointer with auxiliary state (running max from each side) — a step up from p27.
- Monotonic decreasing stack for “process when bound is confirmed” patterns.
- Multiple optimal solutions — recognize that one O(N) solution doesn’t preclude there being others with different tradeoffs.
- Horizontal-strips vs column-by-column decomposition of trapped water.
9. When to Move On
Move on when:
- You can write the prefix-max solution in under 8 minutes.
- You can write the two-pointer solution in under 12 minutes with the move-rule justification.
- You can write the stack solution in under 18 minutes.
- You can articulate the trade-offs in one sentence each.
- Your stress test runs all three vs brute on 500 random small arrays.
- You can pivot to LC 84 (Largest Rectangle in Histogram) using the same stack template.
10. Company Context
Google:
- The classic “let’s go deep” problem. Expect to be asked for ALL THREE solutions over 45 min.
- Bar for Staff+: invent a 4th solution (segment tree / sparse table) without prompting, even if not optimal.
- Scorecard phrase: “Algorithm + breadth — three O(N) solutions, articulated tradeoffs.”
Amazon:
- Often the second problem in an algorithm round. Bar Raiser will probe for the space-optimization.
- Scorecard phrase: “Strong — prefix-max naturally, then optimized to two-pointer when asked.”
Meta:
- Frequently paired with LC 84 (Histogram) in the same loop — stack pattern is the lesson.
- Scorecard phrase: “Algorithm — recognized stack pattern, generalized to histogram.”
Microsoft:
- Sometimes asked at the System Design boundary: “now process a streaming version where heights arrive in chunks.”
- Scorecard phrase: “Strong — prefix-max + thoughtful streaming discussion.”
Bloomberg:
- Frames as financial: “for each day, compute the underwater drawdown given an envelope of historical highs/lows.” Same algorithm.
- Scorecard phrase: “Pragmatic — recognized P&L underwater pattern.”
Apple:
- Often asked with the 2D variant (LC 407 Trapping Rain Water II) as a follow-up. Min-heap + BFS from boundary.
- Scorecard phrase: “Strong — 1D solutions + 2D extension via min-heap.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First solution | O(N²) brute | Prefix-max arrays | Two-pointer | Two-pointer + states stack alternative upfront |
| When asked “another way?” | “Uh…” | Brute or prefix-max | Stack approach with horizontal strips | Sketches segment tree variant too |
| Space optimization | “More memory is fine” | “Could probably do better” | Two-pointer with move-rule proof | Two-pointer + explains the dominance argument |
| Generalization | None | “DP?” | 2D version exists (heap-based) | Connects to histogram, daily temps, monotonic stack family |
12. Level Delta
Mid (L4):
- Prefix-max arrays solution.
- Correct formula
min(maxLeft, maxRight) - h[i], clamped to 0. - O(N) time, O(N) space stated.
- Edge cases handled.
Senior (L5):
- Reaches for two-pointer first OR upgrades from prefix-max to two-pointer when prompted.
- Articulates the move-rule justification.
- Knows the stack approach exists (may not implement fully in time).
- Notes 2D version exists.
Staff (L6):
- Implements BOTH two-pointer AND stack solutions cleanly.
- Articulates trade-offs.
- Connects stack approach to LC 84, LC 739, monotonic stack family.
- Discusses 2D version (LC 407) and outlines min-heap-from-boundary algorithm.
Principal (L7+):
- Discusses streaming variant: with append-only data, the prefix-max array can be maintained online but suffix-max can’t — solve with stack (deferred decisions).
- Discusses range-update generalization: if heights change dynamically, segment tree with range-max queries supports updates in O(log N) per change but full recompute is required for trapped water (O(N) per query). A more clever structure is open territory.
- Discusses parallelism: prefix-max parallelizes via parallel prefix-scan (Blelloch); two-pointer is inherently sequential.
- Discusses production: drainage simulation, GPU heatmap min-pooling, financial drawdown analysis.
13. Follow-up Q&A
Q1: O(1) extra space?
A: Two-pointer approach (Solution B). Maintain leftMax, rightMax, lo, hi. Detailed proof: the side with the smaller current height has its water level determined entirely by that side’s running max.
Q2: 2D version (LC 407 Trapping Rain Water II).
A: Heap-based BFS from the boundary. Push all boundary cells into min-heap with their heights. Pop smallest, visit unvisited neighbors: if neighbor height < current threshold, it traps threshold - height water and is pushed with threshold; otherwise pushed with its own height. O(MN log MN). The intuition: water “leaks” from the lowest point on the boundary first.
Q3: Heights are streaming (append-only). Can we maintain running answer? A: Each new bar might cause water to be “released” or “added” depending on heights around old positions — non-trivial. Stack approach generalizes: maintain monotonic decreasing stack of indices; when new bar arrives, pop and add trapped water as before. Total work O(N) amortized.
Q4: What if heights can update arbitrarily (point updates)?
A: Hard. Segment tree supports range-max queries in O(log N), but trapped water at index i depends on global maxes — a single update can change trapped water at MANY indices. Best known: recompute in O(N) per update, or accept higher complexity for the dynamic version.
Q5: Compute trapped water in any SUBRANGE [l, r] quickly.
A: Precompute prefix-max and suffix-max with sparse tables (O(N log N) preprocessing, O(1) range-max). For each query, recompute water column-by-column in O(r - l + 1) — no faster known.
14. Solution Walkthrough
See solution.py:
trap_prefix_max(height)— Solution A. O(N) time, O(N) space.trap_two_pointer(height)— Solution B. O(N) time, O(1) space.trap_stack(height)— Solution C. O(N) time, O(N) space, monotonic decreasing stack.trap_brute(height)— O(N²) per-column scan, oracle.stress_test()— 500 random small arrays; all four implementations agree.edge_cases()— empty, single bar, monotone, V, W, plateau, LC canonical.
Run: python3 solution.py → “ALL TESTS PASSED”.
15. Beyond the Problem
Principal-engineer code review comment:
“Three solutions in one file is fine for teaching, NOT for production. In a real PR I want one solution, picked deliberately. For our drainage-simulation service we use the stack approach because it generalizes to our streaming gauge-readings model. For our financial-drawdown library we use prefix-max because the data is bounded and the extra space is fine. Don’t ship ‘three approaches’ code — ship the one you justified. The other two go in the docs as ‘considered alternatives’. Also: the two-pointer’s invariant comment should mention WHY rightMax dominates when we advance lo (h[lo] < h[hi] ≤ rightMax). Future readers will not derive that themselves under pressure.”
Connections forward:
- p27 — Container With Most Water: same two-pointer skeleton, different objective (max area vs sum of water).
- Phase 02 Lab 05 — Monotonic Stack: this is the bridge problem. Solution C is the template.
- LC 84 — Largest Rectangle in Histogram: same monotonic-stack pattern.
- LC 407 — Trapping Rain Water II: 2D version, heap-from-boundary BFS.
- Production: drainage simulation in civil engineering, water-management infrastructure planning, financial drawdown (“underwater” portfolio analysis), GPU image min/max pooling.
- Theoretical: the dual problem (maximum trapped water under a CHANGE in heights — Dilworth’s-like decomposition) is open research.
Hints — p28 Trapping Rain Water
Read one at a time.
Hint 1 — Per-column thinking.
For each column i, ignore everything else. How tall is the water above it? Answer: bounded by the SHORTER of the tallest bar to the left and the tallest bar to the right. Formula: water[i] = max(0, min(maxLeft[i], maxRight[i]) - h[i]). Total = sum.
Hint 2 — Naive O(N²) → speedup.
Naive: for each i, scan left and right for the maxes. O(N²). Speedup: precompute maxLeft[i] in one left-to-right pass and maxRight[i] in one right-to-left pass. Then sum. O(N) time, O(N) space. Solution A. Ship this first.
Hint 3 — Can we drop the arrays?
Two pointers lo, hi, running leftMax, rightMax. Whenever h[lo] < h[hi], you know rightMax ≥ h[hi] > h[lo], so the water at lo is determined by leftMax ALONE (the right side is guaranteed taller). Update leftMax = max(leftMax, h[lo]), accumulate leftMax - h[lo], advance lo. Symmetric for the other side. O(N) time, O(1) space. Solution B.
Hint 4 — Order matters in the two-pointer.
Update leftMax BEFORE computing leftMax - h[lo]. Otherwise on a new tallest-so-far bar you’d compute a negative.
Hint 5 — Stack approach (advanced).
Walk left to right. Maintain a decreasing stack of indices. When the current bar i exceeds the stack top, the top is a “trough bottom” — pop it; the new top is the left wall, i is the right wall. Water from this strip = (min(h[left], h[i]) - h[bottom]) * (i - left - 1). Repeat. Each index pushed/popped once → O(N). Solution C. This is the bridge to monotonic-stack problems (Histogram, Daily Temperatures).
If still stuck: all three implementations in solution.py. Read Solution A first, then Solution B with the move-rule proof in README.md section 6.3.
p29 — Longest Substring Without Repeating Characters
Source: LeetCode 3 · Medium · Topics: Hash Table, String, Sliding Window Companies (2024–2025 frequency): Amazon (very high), Meta (high), Google (high), Microsoft (high), Bloomberg (medium), Apple (medium) Loop position: warmup or first algorithm problem. The canonical variable-window template.
1. Quick Context
Given a string s, find the length of the longest substring (contiguous) without repeating characters.
This is THE canonical variable-length sliding window problem. The template you learn here — expand right, shrink left when invariant breaks — generalizes to ALL window problems: Minimum Window Substring (p30), Longest Substring With At Most K Distinct Characters, Subarray Sum Equals K with positive constraints, Permutation in a String, Find All Anagrams.
Two valid mental models:
- Last-seen map (
char → last_index): when you see a repeat inside the window, jumplefttolast_seen[c] + 1. O(N), one pass. - Set + shrinking pointer: maintain set of chars in window; on repeat, advance
leftone char at a time (removing from set) until the duplicate is gone. O(N) amortized.
Model 1 is cleaner; model 2 generalizes more naturally to “at most K distinct” variants.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/longest-substring-without-repeating-characters/
STOP. 20-min timer. If you can write the O(N) sliding window in under 15 min without bugs, you own this problem.
3. Prerequisite Concepts
- Hash maps / dictionaries.
- Sliding window template (variable size).
- Substring vs subsequence (substring = contiguous!).
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given a string s, return the length of the longest contiguous substring such that all characters in it are unique.”
Step 2 — Clarify:
- “Substring or subsequence?” (Substring = contiguous. Critical.)
- “Case-sensitive?
'A'vs'a'distinct?” (Yes, distinct.) - “Character set? ASCII? Unicode?” (LC: printable ASCII / digits / symbols. ~95 chars.)
- “Return length or the substring itself?” (Length only.)
- “Empty string?” (Return 0.)
- “Could be
len(s) = 0orlen(s) = 5*10^4?” (Yes, both.)
Step 3 — Constraints: N ≤ 5×10⁴. O(N²·σ) brute too slow at upper bound; O(N) sliding window required.
Step 4 — Examples:
"abcabcbb"→ 3 ("abc")."bbbbb"→ 1."pwwkew"→ 3 ("wke"; note"pwke"is a subsequence, not substring).""→ 0." "→ 1."dvdf"→ 3 ("vdf"; the tricky one whereleftmust NOT jump backward).
Step 5 — Brute Force: Try every substring s[i..j]; check uniqueness via set in O(j-i). Total O(N³) or O(N²·σ) with incremental set. State it.
Step 6 — Complexity: Optimal O(N) time, O(min(N, σ)) space where σ is alphabet size.
Step 7 — Pattern Recognition: “Longest contiguous interval satisfying a property” + “property maintainable via dict” → variable sliding window with hash map.
Step 8 — Develop: see solution.py.
Step 9 — Test: Empty, all-same, all-distinct, single-char, the tricky "dvdf" (where last_seen['d'] = 0 is OUTSIDE the current window when we encounter the second d).
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — The “don’t jump backward” subtlety
The map-based approach:
left = 0
best = 0
last_seen = {} # char → last index it was seen
for right, c in enumerate(s):
if c in last_seen and last_seen[c] >= left:
left = last_seen[c] + 1
last_seen[c] = right
best = max(best, right - left + 1)
return best
The trap: writing left = last_seen[c] + 1 UNCONDITIONALLY (without the >= left guard, or equivalently without max(left, last_seen[c]+1)). On input "dvdf":
r=0, c='d': last_seen={d:0}, left=0, best=1.r=1, c='v': last_seen={d:0, v:1}, left=0, best=2.r=2, c='d': duplicate!last_seen['d'] = 0. Naive:left = 0 + 1 = 1. Correct:left = max(0, 0+1) = 1. (Same here.)r=3, c='f': last_seen={d:2, v:1, f:3}, left=1. best = max(2, 3-1+1=3) = 3. ✓
But on "abba":
r=0, c='a': left=0, best=1, last={a:0}.r=1, c='b': left=0, best=2, last={a:0,b:1}.r=2, c='b': duplicate!last_seen['b']=1. Naive:left = 1+1 = 2. Correct:left = max(0, 2) = 2. last={a:0,b:2}, best = max(2, 1) = 2.r=3, c='a': duplicate!last_seen['a']=0. Naive:left = 0+1 = 1— but we already moved left to 2 because of the earlier'b'! Going BACK to 1 would re-include the'b'at index 2, breaking uniqueness. Correct:left = max(2, 0+1) = 2. last={a:3,b:2}, best = max(2, 3-2+1=2) = 2. ✓
Without the max, you’d report 3 (wrong — "bba" contains repeated b).
Take-away: in sliding window with last-seen indices, ALWAYS left = max(left, last_seen[c] + 1). Never assign directly. This is the single most common bug.
Alternative: set + shrinking pointer
left = 0
seen = set()
best = 0
for right, c in enumerate(s):
while c in seen:
seen.discard(s[left])
left += 1
seen.add(c)
best = max(best, right - left + 1)
return best
This is more “obviously correct” — left only advances forward. Trade-off: when there’s a duplicate, you do a small inner loop, but each char is added/removed at most once → O(N) amortized total.
Both are O(N); pick the one you can write bug-free under pressure. The set approach generalizes more cleanly to “at most K distinct” variants because you can replace the while c in seen condition with while len(distinct) > K.
7. Anti-Pattern Analysis
- Confusing substring with subsequence. “pwwkew” → some students answer 4 (
"pwke"). That’s a subsequence, not substring. left = last_seen[c] + 1withoutmax. Classic bug — see Section 6.- O(N²) substring enumeration. Works but slow at N=5×10⁴.
- Using
str.find()orinfor each window. Both are O(N) inside an O(N) loop → O(N²). - Not handling empty string. Return 0, not error.
- Updating
last_seen[c]BEFORE checking it. You’d see the just-updated value and do nothing. Check first, then update. - Maintaining the longest substring (not just length) and using
s[left:right+1]. O(N) slice inside the loop — kills the linear complexity. If you must return the substring, track(best_left, best_right)indices and slice once at the end. - Forgetting
right - left + 1is the window length. Off-by-one.
8. Skills & Takeaways
- Variable sliding window template — burn this into your hands.
- Last-seen-index map — useful pattern beyond strings (e.g., LRU cache lite, deduplication).
max(left, jump_target)— the “monotonic left pointer” invariant.- Amortized analysis — set + shrinking pointer is O(N) because each char enters/leaves at most once.
- Substring vs subsequence — vocabulary discipline; misreading this is rookie.
9. When to Move On
Move on when:
- You can write the map-based solution in under 8 minutes, no bugs, with
max(left, ...). - You can write the set-based solution in under 10 minutes.
- You can explain why
"abba"breaks the naiveleft = last_seen[c] + 1. - Your stress test passes 300 random small strings vs brute.
- You can pivot to LC 340 (Longest Substring with At Most K Distinct) by changing the invariant from
len(seen) > 0 duplicatestolen(seen) > K.
10. Company Context
Amazon:
- The single most-asked Medium string problem at Amazon in 2024-25 (per reported loops).
- Bar Raisers will probe: “can you also return the substring itself, not just length?” and “what if the alphabet is Unicode?”
- Scorecard phrase: “Algorithm — sliding window template, correct
max(left, ...)jump.”
Meta:
- Often a warmup before a Hard. Usually expected in 10-12 min clean.
- Scorecard phrase: “Fluent — recognized as variable-window, no false starts.”
Google:
- May be extended to “longest substring with at most K distinct characters” (LC 340) in the same session.
- Scorecard phrase: “Algorithm + generalization — template transferred to K-distinct variant.”
Microsoft:
- Sometimes asked alongside or as warmup for LC 76 (Minimum Window Substring).
- Scorecard phrase: “Strong — sliding window family fluency.”
Bloomberg:
- Variant: “given a stream of trades, longest run without a repeated ticker.” Same algorithm.
- Scorecard phrase: “Pragmatic — recognized streaming variable-window.”
Apple:
- May ask for Unicode-correct version. Discussion of grapheme clusters earns Senior signal.
- Scorecard phrase: “Correctness + i18n awareness.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | Brute O(N²) enumerate | Map-based sliding window | Map-based with max(left, ...) upfront | Both variants, mentions K-distinct generalization |
"abba" test | Misses it | Catches after bug | Includes in test from start | Explains why max is required |
| Substring return | Slices each window O(N²) | Tracks (best_l, best_r), slices once | Same as Senior, explains why | Same + unicode discussion |
| Complexity | “Linear-ish” | O(N), space O(N) | O(N), space O(min(N, σ)) | + amortized analysis of set variant |
12. Level Delta
Mid (L4):
- Map-based sliding window solution.
- O(N) time, O(min(N, σ)) space stated.
- Edge cases including
""and single char.
Senior (L5):
- Includes
max(left, last_seen[c] + 1)from the start with verbal justification. - Tests
"abba"and"dvdf"explicitly. - Knows the set + shrinking-pointer alternative; chooses one with rationale.
Staff (L6):
- Implements both approaches; explains which one to pick depending on follow-up direction (K-distinct → set; pure dedup → map).
- Discusses Unicode: characters vs code units vs grapheme clusters.
- Discusses streaming variant: can maintain answer online with bounded memory if we cap window length.
Principal (L7+):
- Discusses suffix automaton / suffix array approach for offline batch processing of many such queries on the same string.
- Discusses distributed variant: split the string into chunks; each chunk computes its own answer plus boundary “overhang” info; merge with a lambda-architecture style combine. Tricky but tractable.
- Discusses production: web-log session deduplication, token-stream rate-limiting, DNA sequence analysis (longest distinct k-mer run).
13. Follow-up Q&A
Q1: Return the substring itself, not the length.
A: Track best_left, best_right indices during the loop; at the end, return s[best_left:best_right+1]. Don’t slice inside the loop.
Q2: At most K distinct characters (LC 340).
A: Replace the invariant. Use a Counter count[c]; expand right (count[c] += 1); while len(count) > K, shrink left (count[s[left]] -= 1; if count[s[left]] == 0: del count[s[left]]; left += 1). Record best = max(best, right - left + 1) each step.
Q3: Unicode strings.
A: Python’s str iterates code points, so basic Unicode works as-is. Grapheme clusters (e.g., emoji with skin-tone modifiers) require regex package with \X to iterate proper graphemes. Discuss with interviewer whether the spec is “code point” or “user-perceived character.”
Q4: Streaming input — bounded memory. A: Without bounding, worst case requires O(N) memory (all distinct chars). If we cap window length to W, sliding window is fine with O(W) memory. If we want exact answer with sublinear memory, no known approach — fundamental.
Q5: Longest substring where each character appears at most K times. A: Variant of the same template. Use Counter; expand right; while any count > K, shrink left. Same O(N) time.
14. Solution Walkthrough
See solution.py:
length_of_longest_substring_map(s)— canonical. Map char → last index. O(N).length_of_longest_substring_set(s)— set + shrinking pointer. O(N) amortized.length_of_longest_substring_brute(s)— O(N²·σ) all substrings via set. Oracle.stress_test()— 300 random strings, alphabet {a,b,c,d}, len ≤ 20.edge_cases()— empty, single, all-same, no repeats, LC canonical,"abba","dvdf","tmmzuxt".
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“The
max(left, last_seen[c] + 1)line needs a comment. Future readers won’t derive WHY we need themax— they’ll see it as defensive coding and ‘simplify’ it in a ‘cleanup’ PR, breaking"abba". Either add# don't let left jump backward when char was last seen before current windowor use the set+shrinking-pointer variant which is self-documenting. Also: if this is called millions of times per second (token stream dedup), profile the dict overhead — for ASCII-only inputs, a fixed-sizeint[128]array beats a Python dict by 3-5x. Choose the implementation that matches your hot path, not the textbook one.”
Connections forward:
- p30 — Minimum Window Substring: same template, harder invariant (match counter).
- LC 340 — Longest Substring with At Most K Distinct: trivial generalization.
- LC 159 — Longest Substring with At Most 2 Distinct: K=2 special case.
- LC 904 — Fruit Into Baskets: K=2 in disguise.
- LC 1004 — Max Consecutive Ones III: “at most K zeros.”
- Phase 02 Lab 02 (Sliding Window): template lab.
- Production: web-log session deduplication, token rate-limiting, log compaction, DNA k-mer analysis.
Hints — p29 Longest Substring Without Repeating Characters
Read one at a time.
Hint 1 — Substring, not subsequence.
Contiguous. "pwwkew" → "wke" (length 3), not "pwke".
Hint 2 — Variable sliding window.
Two pointers left, right. Expand right one char at a time. When the invariant “all chars in s[left..right] are unique” breaks, shrink left until it holds again. Record best = max(best, right - left + 1) each step.
Hint 3 — How to shrink efficiently.
Option A: keep a SET of chars in the window; on duplicate at s[right], advance left one step at a time (removing from set) until the duplicate is gone. O(N) amortized (each char enters/leaves at most once).
Option B: keep a MAP char → last_seen_index. On duplicate, jump left directly past the last occurrence: left = last_seen[c] + 1.
Hint 4 — The "abba" trap.
With Option B, on "abba": at right=3, c='a', last_seen['a']=0 but left=2 (we already moved past the earlier 'b'). Jumping to left = 0+1 = 1 would BACKTRACK and re-include the 'b' at index 2 — breaking uniqueness. Always left = max(left, last_seen[c]+1), never bare assign.
Hint 5 — Edge cases checklist.
Empty → 0. Single char → 1. All same "bbbbb" → 1. All distinct "abcdef" → 6. The tricky ones: "abba", "dvdf", "tmmzuxt". Test these.
If still stuck: solution.py has both Option A (set) and Option B (map). Read the set version first — it’s self-documenting (left only moves forward).
p30 — Minimum Window Substring
Source: LeetCode 76 · Hard · Topics: Hash Table, String, Sliding Window Companies (2024–2025 frequency): Meta (very high), Amazon (high), Google (high), Microsoft (medium), Bloomberg (medium) Loop position: algorithm round, often the discriminator between Mid and Senior.
1. Quick Context
Given strings s and t, return the minimum window (shortest contiguous substring) in s such that every character in t (counting multiplicity) is included in the window. If no such window exists, return "".
This is the canonical “expand-then-contract” variable-window problem and the natural progression from p29. Three things are different from p29:
- The invariant is more complex: “window contains all required chars with sufficient multiplicity” — not just “no duplicates.”
- The optimization that separates Mid from Senior: the match counter. Naive implementations call
Counter == Counter(O(σ)) on each shrink. The match counter tracks how many distinct characters oftare currently satisfied; checkingformed == requiredis O(1). - The structure: expand
rightuntil the window is valid; then contractleftwhile it stays valid, recording the tightest. Repeat.
This problem appears in Meta loops with high frequency specifically because the naive O(N²) → O(N·σ) → O(N) optimization sequence reveals algorithmic depth in 15 minutes.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/minimum-window-substring/
STOP. 35-min timer. This is Hard. If you can’t finish in 35, write the brute force, state the match-counter idea, and ship it for partial credit.
3. Prerequisite Concepts
- p29 — Longest Substring Without Repeating: variable-window template.
collections.Counteranddefaultdict(int).- Multiset / multiplicity reasoning.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Find the shortest contiguous substring w of s such that for every character c in t, the count of c in w is at least the count of c in t. Return w; if none exists, return empty string.”
Step 2 — Clarify:
- “Multiplicity matters?” (Yes.
t="AABC"requires TWO'A's in the window.) - “Case-sensitive?” (Yes.
'A'vs'a'distinct.) - “If multiple windows tie?” (LC: any one is acceptable. Often: leftmost.)
- “Empty
t?” (Spec says non-empty. Confirm — if empty, return"".) - “
tlonger thans?” (No valid window → return"".) - “Characters in
tnot appearing insat all?” (No valid window →"".)
Step 3 — Constraints: N (len of s) ≤ 10⁵, M (len of t) ≤ 10⁵, alphabet uppercase/lowercase English. O(N+M) target.
Step 4 — Examples:
s="ADOBECODEBANC", t="ABC"→"BANC".s="a", t="a"→"a".s="a", t="aa"→""(not enough'a's).s="aa", t="aa"→"aa".s="ab", t="b"→"b".s="bba", t="ab"→"ba"(note: leftmost minimum is"ba", length 2).
Step 5 — Brute Force: Enumerate all (i, j) substrings, for each check Counter(s[i:j+1]) >= Counter(t). O(N²·M). State and pivot.
Step 6 — Complexity: Optimal O(N + M) time, O(σ) space.
Step 7 — Pattern Recognition:
- “Shortest contiguous range satisfying a multiset constraint” → variable sliding window with expand-then-contract.
- “Validity check is hot path” → incremental match counter (don’t re-compare maps).
Step 8 — Develop: see solution.py.
Step 9 — Test: valid example, no-valid case, single char, full string is the answer, t with duplicates, alphabet collisions.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — The match counter trick
6.1 The naive validity check
The straightforward way:
def is_valid(have, need):
for c, k in need.items():
if have.get(c, 0) < k:
return False
return True
This is O(σ) per call (where σ = |distinct chars in t|). If called inside both expand and contract loops, total complexity becomes O(N · σ) — fine for small σ but wasteful.
6.2 The match-counter optimization
Maintain two counters and TWO integers:
need = Counter(t)— required frequencies.have = defaultdict(int)— current window frequencies.required = len(need)— number of DISTINCT characters that need satisfying.formed = 0— number of DISTINCT characters currently satisfied (i.e.,have[c] >= need[c]).
Expand (right += 1 adds char c):
have[c] += 1
if c in need and have[c] == need[c]:
formed += 1
The == need[c] (not >=) is critical: we increment formed ONLY at the moment we just met the requirement. Going further (have > need) does NOT change formed.
Validity check: formed == required. O(1).
Contract (left += 1 removes char c):
if c in need and have[c] == need[c]:
formed -= 1
have[c] -= 1
Order matters: we decrement formed BEFORE reducing have[c] below need[c]. (Equivalently: check have[c] == need[c] first, then decrement have[c].)
Now the main loop:
left = 0
formed = 0
best_len = inf
best_l = 0
for right in range(n):
expand(s[right])
while formed == required: # window valid → try to shrink
if right - left + 1 < best_len:
best_len = right - left + 1
best_l = left
contract(s[left])
left += 1
return s[best_l : best_l + best_len] if best_len < inf else ""
Each character is processed at most twice (once on expand, once on contract) → O(N + M). The formed == required check is O(1).
6.3 Why the order of operations matters
If you reverse the contract order:
have[c] -= 1
if c in need and have[c] < need[c]:
formed -= 1
This is also correct (and arguably more intuitive). But you must NOT do:
have[c] -= 1
if c in need and have[c] == need[c] - 1:
formed -= 1
unless need[c] > 0 for c — which it is by construction (c in need implies need[c] >= 1). Subtle. Use the cleaner “check then update” form.
6.4 Generalizations
This template extends directly to:
- LC 567 — Permutation in String: fixed-window version (length = len(t)).
- LC 438 — Find All Anagrams in a String: fixed-window, return all start indices.
- LC 30 — Substring with Concatenation of All Words: tokenized version, window over word boundaries.
- LC 727 — Minimum Window Subsequence: similar API but subsequence, requires DP not sliding window.
7. Anti-Pattern Analysis
Counter == Counteron each shrink check. O(σ) per check — turns the algorithm into O(N·σ). Use match counter.have[c] >= need[c]instead ofhave[c] == need[c]when incrementingformed. With>=, you’d incrementformedevery time you add acafter the first match, double-counting.- Wrong contract order. Decrementing
have[c]before checking againstneed[c]flips the comparison. - Tracking the substring with
s[left:right+1]inside the loop. O(N) slice per iteration → O(N²) total. Track(best_l, best_len), slice once at the end. - Initializing
best_len = 0. Then the first valid window of any size fails the< best_lencheck (0 is smaller). Useinforn + 1. - Forgetting the no-window case. Return
""(empty string), notNoneor-1. - Counting characters NOT in
t. Wasteful but not incorrect. To avoid, guard withif c in need: have[c] += 1. Trade-off: complicates contract logic. - Building
needas a dict by iteratingtmanually withif/else.Counter(t)is faster, clearer, and harder to bug. - Treating
tas a SET. Loses multiplicity info.t="AABC"requires TWO'A's.
8. Skills & Takeaways
- Variable sliding window with expand-then-contract — the second canonical pattern after p29.
- Match counter (
formed/required) — an O(1) incremental validity check, replacing O(σ) comparisons. Reuse this pattern for any “multiset coverage” problem. - Counter / defaultdict fluency.
(best_l, best_len)tracking — avoid string slicing in hot loops.- Hard vs Medium discrimination — the match counter is the senior signal that elevates this from Medium-quality code.
9. When to Move On
Move on when:
- You can write the match-counter version in under 18 minutes, no bugs.
- You can explain WHY
have[c] == need[c](not>=) when incrementingformed. - You can articulate why total work is O(N + M) despite the inner while-loop.
- Your stress test passes 200 random small (s, t) pairs vs brute.
- You can pivot to LC 567 (Permutation in String) in under 10 minutes using the same machinery (with a fixed window length).
10. Company Context
Meta:
- One of the highest-frequency Hard problems in 2024-25 Meta loops. Expected solid Senior-level fluency.
- Bar Raiser specifically watches for the match-counter optimization. Naive
Counter ==is a partial-credit signal. - Scorecard phrase: “Algorithm — sliding window with match-counter optimization, O(N+M).”
Amazon:
- Often paired with LC 438 (Anagrams) in the same loop — same template, fixed window.
- Scorecard phrase: “Strong — variable window template + correct match-counter.”
Google:
- Frequently extended: “now return the K shortest non-overlapping such windows.” Tricky greedy + sliding.
- Scorecard phrase: “Algorithm + extension — generalized sliding window with constraint tracking.”
Microsoft:
- Sometimes asked as the warm-up problem in an algorithm-heavy loop.
- Scorecard phrase: “Strong — Hard problem cleanly solved with no false starts.”
Bloomberg:
- Variant: “given a stream of trades and a basket of required tickers, find the shortest time window where all required tickers traded.”
- Scorecard phrase: “Pragmatic — recognized streaming variable-window over time.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | Brute O(N²·M) | Variable window with Counter == check | Match counter from the start | Match counter + clean test plan + complexity proof |
| Validity check | Map equality each step | Counter == or full dict scan | formed == required O(1) | Same + verbal complexity argument |
| Substring extraction | s[left:right+1] inside loop | Same | (best_l, best_len), slice once | Same + discusses memory churn in production |
| Generalization | None | “Probably works for anagrams” | Notes LC 567, LC 438 use same template | Notes streaming variant, range/temporal extension |
12. Level Delta
Mid (L4):
- Variable-window solution with
Counter == Countervalidity check. - O(N·σ) time, correct answer.
- All edge cases handled.
Senior (L5):
- Match counter from the start.
- O(N + M) time stated and justified.
(best_l, best_len)extraction, no in-loop slicing.- Tests
twith duplicates and no-valid-window case.
Staff (L6):
- Implements match counter with clean expand/contract symmetry.
- Discusses LC 567, LC 438, LC 30 as same-template variants.
- Discusses memory churn: in JVM/Go, the per-iteration slice creates GC pressure — track indices and slice once.
- Discusses streaming variant (data arriving over time).
Principal (L7+):
- Discusses the K minimum windows generalization: greedy over non-overlapping intervals.
- Discusses multi-pattern: given M required substrings (not just chars), find shortest window containing all of them. Reduces to Aho-Corasick + sliding window in O(N·log M).
- Discusses alphabet size σ very large (Unicode, custom tokens): hash-of-hash for O(1) match counter, or roll our own bitset. For ASCII, fixed-size
int[128]beats dict. - Discusses production: search-result diversity (find minimum window of search results covering all required facets), log-anomaly detection (minimum window covering all required events), bioinformatics (shortest DNA region containing all required motifs).
13. Follow-up Q&A
Q1: Why have[c] == need[c] (not >=) when incrementing formed?
A: formed counts the number of DISTINCT chars that are currently satisfied. We want to increment it exactly once per char — at the moment we first meet the requirement. Using >= would increment on every additional c, making formed larger than required and breaking the formed == required invariant.
Q2: What if the alphabet is huge (Unicode)?
A: Replace defaultdict(int) with a plain dict. formed and required still work. For ASCII-only, replace dict with a fixed-size array for ~3-5x speedup; access is O(1) without hashing.
Q3: LC 567 — does s2 contain a permutation of s1?
A: Fixed-window of length len(s1). Initialize the window’s have from s2[:len(s1)]. Then slide one char at a time: expand right, contract left. Check formed == required at each step. O(|s2|) time. Use Counter equality at the end if you prefer; match counter is more efficient.
Q4: Find ALL minimum windows (not just one). A: Two passes. First pass: find the minimum length L. Second pass: collect all windows of length exactly L that satisfy the constraint. Same O(N+M) bound.
Q5: Streaming s (data arriving over time).
A: Right-expanding sliding window works online. Left can only advance forward. Memory: O(σ). On each character arrival: expand step. Periodically (or on query): check formed == required and try to contract. Online streaming with bounded memory ✓.
14. Solution Walkthrough
See solution.py:
min_window(s, t)— canonical match-counter solution. O(N + M).min_window_naive(s, t)— variable window with O(σ) validity check on each shrink. O(N·σ).min_window_brute(s, t)— try every (i, j) pair, compare Counters. O(N²·M). Oracle.stress_test()— 200 random (s, t) pairs; all three agree on the LENGTH (any tied answer is acceptable; we compare lengths).edge_cases()— empty t, t > s, no valid window, LC canonicals="ADOBECODEBANC", t="ABC", duplicatest="AABC", full string, single chars.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“The match-counter trick is great, but bury it under a clear comment block at the top of the function: ‘INVARIANT: formed = count of distinct chars c in need such that have[c] >= need[c].’ Without that, the future engineer maintaining this code will see the
== need[c]increment and ‘fix’ it to>= need[c], breaking the invariant silently. Also: thehave = defaultdict(int)allocates on every call. If this is a hot path, pass a reusable Counter in. And one more thing — thes[best_l:best_l+best_len]allocates a new string; if the caller only needs the indices for highlighting in a UI, return(best_l, best_len)instead and let the caller decide. Don’t pre-allocate what the caller might not need.”
Connections forward:
- p29 — Longest Substring Without Repeating: simpler invariant, same template.
- LC 567 — Permutation in String: fixed-window version.
- LC 438 — Find All Anagrams: fixed-window, return all start indices.
- LC 30 — Substring with Concatenation of All Words: tokenized version.
- LC 727 — Minimum Window Subsequence: requires DP, NOT sliding window — common trap.
- LC 992 — Subarrays with K Different Integers: “exactly K” trick = “at most K” - “at most K-1.”
- Phase 02 Lab 02 — Sliding Window: template lab.
- Production: search-result diversity, log-anomaly minimal coverage, DNA motif finding, ad-targeting “shortest sequence of impressions covering all required categories.”
Hints — p30 Minimum Window Substring
Read one at a time.
Hint 1 — Multiplicity matters.
t="AABC" means the window needs at least TWO 'A's, one 'B', one 'C'. Use Counter, not set.
Hint 2 — Expand then contract template.
Two pointers left, right. Expand right until the window is VALID (contains all required chars with multiplicity). Then contract left while it stays valid, recording the tightest. Repeat.
Hint 3 — How to check validity efficiently.
Naive: for k,v in need: if have[k] < v: return False. O(σ) per check, called O(N) times → O(N·σ). Works, but…
The trick: maintain formed = count of distinct chars in t currently SATISFIED (have[c] >= need[c]). When formed == required (= len(need)), window is valid. O(1) check.
Hint 4 — Maintaining formed correctly.
On EXPAND (adding char c): have[c] += 1; if c in need and have[c] == need[c]: formed += 1. Critical: use ==, not >=. We increment exactly once per char, at the moment we first meet the need.
On CONTRACT (removing char c): check if c in need and have[c] == need[c]: formed -= 1 BEFORE have[c] -= 1. We lose a match the moment we go below.
Hint 5 — Don’t slice inside the loop.
Track (best_l, best_len); slice once at the end. s[left:right+1] in the loop is O(N) per step → kills your O(N) bound.
Edge: best_len = inf initially. Return "" if still inf at the end.
If still stuck: solution.py has the canonical match-counter and the naive O(N·σ) for comparison. Read the INVARIANT comment above formed in min_window.
Week 07 — Prefix Sums + Binary Search: The Logarithmic Mindset
Phase: Month 02 — Patterns Theme: Two pillar techniques: turn “range sum in O(1)” into a habit (prefix sums), and learn to invent a monotone predicate so binary search applies (rotated arrays, partition-on-answer). Difficulty: Medium → Hard. p35 is one of the most-feared LC Hards; we’ll tame it.
Week Goals
By end of week, you should:
- Reach for a prefix sum the moment you see “subarray sum” or “range sum” — no exceptions.
- Recognize
prefix[j] - prefix[i] == kas the canonical hashmap reduction. - Write binary search on rotated arrays without confusing yourself — three pivots, one invariant.
- Articulate the difference between “binary search on indices” (p33, p34) and “binary search on partitions” (p35).
- Defend complexity proofs for
O(log(min(m,n)))Median of Two Sorted Arrays.
Problem Table
| # | Problem | LC | Difficulty | Pattern | Companies |
|---|---|---|---|---|---|
| p31 | Subarray Sum Equals K | 560 | Medium | Prefix sum + hashmap | Meta, Amazon, Google |
| p32 | Range Sum Query 2D — Immutable | 304 | Medium | 2D prefix sum | Microsoft, Amazon, Bloomberg |
| p33 | Search in Rotated Sorted Array | 33 | Medium | Binary search on rotation | Meta (very high), Amazon, Google |
| p34 | Find Minimum in Rotated Sorted Array | 153 | Medium | Binary search on rotation | Meta, Amazon, Microsoft |
| p35 | Median of Two Sorted Arrays | 4 | Hard | Binary search on partition | Google (very high), Meta, Apple |
Daily Schedule (suggested)
- Mon — p31 Subarray Sum K. The “prefix sum + hashmap” pattern. Internalize the negative-numbers case (sliding window does NOT work here).
- Tue — p32 Range Sum Query 2D. 2D extension of prefix sum. Inclusion-exclusion.
- Wed — p33 Search in Rotated Sorted Array. The classic. Know the “which half is sorted?” invariant.
- Thu — p34 Find Minimum in Rotated Sorted Array. Variant of p33; tighter invariant on the pivot.
- Fri — p35 Median of Two Sorted Arrays. The Hard. Binary search on partition. Block 90+ minutes.
- Sat — Re-solve all five from memory under timer. Mock yourself.
- Sun — Spaced repetition: revisit Week 1 (p01) and Week 4 (Trees) and Week 6 (p30). 30 min total.
Readiness Gate (must pass to advance)
-
Can write Subarray Sum K (with hashmap including the
count[0]=1seed) in under 8 min. - Can write the 2D prefix sum with inclusion-exclusion (both build and query) in under 12 min.
- Can write Search in Rotated Sorted Array WITHOUT looking up the “which half is sorted” decision in under 15 min.
-
Can write Find Minimum in Rotated Sorted Array in under 10 min — and explain why
<= hivs< himatters. - Can sketch Median of Two Sorted Arrays binary-search-on-partition in 30 min with the partition invariant correctly stated.
-
All 15 files commit-ready: every
solution.pyends withALL TESTS PASSED.
Anti-Pattern Hit-List
- Sliding window on Subarray Sum K with negatives — the monotone property fails. Use prefix sum + hashmap.
- Forgetting
count[0] = 1in the Subarray Sum K hashmap seed — misses subarrays starting at index 0. - Building 2D prefix sum with off-by-one indices — always allocate
(R+1) x (C+1)with a zero border to avoid edge-case branches. - Rotated binary search: comparing
nums[mid]tonums[hi]vsnums[lo]inconsistently — pick ONE side and stick with it. Comparing tonums[hi]is more robust for both p33 and p34. - Rotated binary search with
lo <= hi— for “find pivot” style (p34) you wantlo < hiso loop ends withlo == hi == answer. - Median of Two Sorted Arrays: binary search over the larger array — you must binary-search the SMALLER. Otherwise the partition index can go out of bounds for the larger.
- Median: hardcoding “odd vs even” branches inside the loop — compute once at the end using the four partition values.
- Treating Hard problems as off-limits — p35 is hard but learnable in one focused sitting.
Connections to Phase Labs
- phase-02-patterns/labs/lab-03-prefix-sums.md — exactly this week.
- phase-02-patterns/labs/lab-04-binary-search-on-answer.md — preview of “binary search where the array isn’t the input” — p35 is a relative.
- phase-01-foundations/labs/lab-07-binary-search-fundamentals.md — refresh the basics before p33-p35.
Stretch (if you finish early)
- LC 1248 — Count Number of Nice Subarrays (prefix sum on parity counts).
- LC 974 — Subarray Sums Divisible by K (prefix sum mod K — note Python’s correct negative-mod handling).
- LC 81 — Search in Rotated Sorted Array II (duplicates — the killer follow-up; worst case O(N)).
- LC 154 — Find Minimum in Rotated Sorted Array II (duplicates — same pattern as LC 81).
- LC 1095 — Find in Mountain Array (interactive binary search, two phases).
p31 — Subarray Sum Equals K
Source: LeetCode 560 · Medium · Topics: Array, Hash Table, Prefix Sum Companies (2024–2025 frequency): Meta (very high), Amazon (high), Google (high), Microsoft (medium), Bloomberg (medium) Loop position: algorithm warmup or first problem. The discriminator: do you reach for sliding window (WRONG with negatives) or prefix sum + hashmap (RIGHT)?
1. Quick Context
Given an integer array nums and an integer k, return the total number of contiguous subarrays whose sum equals k. The array may contain negative numbers and zeros.
This is the canonical “prefix sum + hashmap” problem. The setup:
$$\text{sum}(i, j) = \text{prefix}[j+1] - \text{prefix}[i]$$
where prefix[0] = 0 and prefix[t] = nums[0] + ... + nums[t-1]. Now sum(i,j) == k iff prefix[j+1] == prefix[i] + k.
So as you sweep right computing prefix[t], count how many previous prefixes equal prefix[t] - k. A hashmap count[prefix_value] → frequency gives O(1) lookup → O(N) total.
The interview trap: the unwary candidate spots “subarray sum” and reaches for sliding window. This works only when all entries are non-negative (so the running sum is monotonic in window growth). With negatives, the monotone property dies — sliding window is silently wrong. Prefix sum + hashmap is the correct general-purpose tool.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/subarray-sum-equals-k/
STOP. 25-min timer. If you go for sliding window, force yourself to test on nums=[-1, -1, 1], k=0 (expected: 1; the answer is [-1,-1,1] itself summing to -1, hmm actually that’s -1 — try nums=[1,-1,0], k=0 → 3 subarrays). Sliding window’s monotone assumption breaks on these.
3. Prerequisite Concepts
- Prefix sums (1D).
collections.defaultdict(int)/dict.get(key, 0).- The identity
sum(i, j) = prefix[j+1] - prefix[i].
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given nums (may contain negatives) and integer k, count contiguous subarrays summing exactly to k.”
Step 2 — Clarify:
- “Contiguous?” (Yes. Always confirm.)
- “Can
numscontain negatives, zeros?” (Yes.) - “Can
kbe 0 or negative?” (Yes.) - “Overlapping subarrays counted separately?” (Yes — each contiguous range is its own subarray.)
- “Return count or list of indices?” (Count.)
- “Array empty?” (Return 0.)
Step 3 — Constraints: N up to 2×10⁴, values in [-1000, 1000]. O(N²) brute is borderline (4×10⁸); O(N) is the target.
Step 4 — Examples:
nums=[1,1,1], k=2→ 2 (subarrays[1,1]at indices 0-1 and 1-2).nums=[1,2,3], k=3→ 2 ([3]and[1,2]).nums=[1,-1,1,-1,1,-1], k=0→ 9 (lots).nums=[0,0,0], k=0→ 6 (all non-empty subarrays sum to 0; C(3,2)+3 = 3+3 = 6).nums=[1], k=0→ 0.nums=[], k=0→ 0.
Step 5 — Brute Force: Enumerate all (i, j), sum nums[i..j] incrementally. O(N²). State and pivot.
Step 6 — Complexity: O(N) time, O(N) space (hashmap).
Step 7 — Pattern Recognition:
- “Subarray sum equals target” + “negatives allowed” → prefix sum + hashmap. Memorize.
- The hashmap stores HOW MANY TIMES each prefix value was seen — not just whether it was seen — because we count subarrays, not just detect existence.
Step 8 — Develop: see solution.py.
Step 9 — Test: the all-zeros case ([0,0,0], k=0), the negatives case ([1,-1,...], k=0), the LC canonical, the no-match case.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why the count[0] = 1 seed?
The map count[v] records: “how many times has the running prefix-sum equaled v BEFORE the current index?”
Before iteration starts, the running prefix sum is 0 (empty prefix nums[0..-1]). So count[0] should start at 1: the empty prefix is a valid “starting boundary.” This is what enables subarrays that start at index 0 to be counted.
Without the seed: consider nums=[3], k=3. Iteration: prefix=3. We look up count[3 - 3] = count[0]. If count[0]=0, we miss the answer. With the seed count[0]=1, we correctly count 1.
A way to remember it: treat the array as having a phantom “left wall” at index -1 with prefix-sum 0. The hashmap entry count[0]=1 represents that wall. Every subarray needs a left boundary and a right boundary; the hashmap tracks the left boundaries available, and count[0]=1 is the first available (the very start).
Why sliding window fails on negatives
Sliding window relies on the invariant: “expanding right monotonically increases (or keeps the same) the running sum; shrinking left monotonically decreases it.” That lets us find the unique window with sum K by adjusting one endpoint at a time.
With negatives, expanding right can DECREASE the sum (when nums[right] < 0). So we can no longer say “if current sum > K, shrink left.” Multiple windows can have the same sum at any moment, and the shrink decision becomes ambiguous.
Concrete failure: nums=[1, -1, 1, 1], k=1. Sliding window starts at [1] sum=1 → record. Expand to [1,-1] sum=0. Need to grow to reach 1 → [1,-1,1] sum=1 → record. Expand to [1,-1,1,1] sum=2 → shrink to [-1,1,1] sum=1 → record. We get 3. But the actual answer includes [1] at index 0, [1,-1,1] 0-2, [1] at index 2, [1,1] at 2-3, [-1,1,1] 1-3, [1] at 3 = 6 subarrays. Sliding window undercounts because it tries to find one window at a time, not all with a given sum.
The hashmap counts all left-endpoints with the desired prefix value at every right-endpoint, in one pass. That’s the structural difference.
Connection to other prefix-sum problems
- LC 974 (Subarray Sums Divisible by K): replace
prefix - klookup withprefix mod Klookup. Python negative modulo is well-defined ((-7) % 5 == 3), which is exactly what we want. - LC 525 (Contiguous Array): treat 0 as -1, then find longest subarray with sum 0 — hashmap stores FIRST index of each prefix value (for longest), not count.
- LC 1248 (Count Nice Subarrays): replace
nums[i]withnums[i] % 2, same template.
7. Anti-Pattern Analysis
- Sliding window with negatives. Silently wrong. See Section 6.
- Forgetting
count[0] = 1. Off-by-one — misses every subarray starting at index 0. - Storing
prefix → boolinstead ofprefix → count. You’d count distinct prefix values, not subarrays. - Updating the hashmap BEFORE the lookup. On
nums=[0,0,0], k=0: at index 0, prefix=0; if you docount[0] += 1(nowcount[0]=2) THEN look upcount[0 - 0] = 2, you count the current empty subarray as well — overcount. Correct order: look up FIRST, then update. - Building a full prefix array. Works but uses O(N) extra space unnecessarily. Just maintain a running sum integer.
- Iterating with index
iand recomputingsum(nums[:i+1])each time. O(N²) and silently slow. - Returning the subarrays themselves instead of count. Re-read the problem.
- Using
dict()andKeyErrorhandling.defaultdict(int)ordict.get(v, 0)is cleaner.
8. Skills & Takeaways
- Prefix sum + hashmap as a single coupled pattern — memorize.
- The
count[0]=1seed — fundamental, recurs in LC 974, LC 525, LC 437 (Path Sum III on trees!), etc. - Why sliding window fails on negatives — articulate this clearly; it’s a Senior-level signal.
- Look-up-before-update ordering in counting problems.
defaultdict(int)fluency.
9. When to Move On
Move on when:
- You can write the solution in under 7 minutes.
- You can explain in one breath why sliding window doesn’t work here.
- You can derive
count[0]=1rather than memorize it. - Your stress test passes 300 random arrays vs O(N²) brute.
- You can pivot to LC 974, LC 525, LC 437 using the same template.
10. Company Context
Meta:
- One of the most-asked “warmup” Mediums at Meta in 2024-25. The discriminator is the negative-numbers case.
- Bar Raiser will explicitly test: “what if the array has negatives?” If you answer “use sliding window,” it’s a fast no.
- Scorecard phrase: “Algorithm — prefix sum + hashmap, correctly handled negatives.”
Amazon:
- Often paired with LC 525 (Contiguous Array) in the same loop.
- Scorecard phrase: “Strong — recognized prefix-sum-hashmap family, transferred to second problem.”
Google:
- May extend to LC 974 (Subarrays Sums Divisible by K) — the modular variant.
- Scorecard phrase: “Algorithm + generalization — modular prefix sum.”
Microsoft:
- Sometimes asked with an additional twist: return the number of DISTINCT subarrays summing to K, by value (not by position).
- Scorecard phrase: “Strong — recognized prefix-sum-hashmap, handled distinct-value extension.”
Bloomberg:
- Variant: given a sequence of P&L deltas per minute, count time windows where cumulative P&L equals zero.
- Scorecard phrase: “Pragmatic — financial prefix-sum framing.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | Brute O(N²) | Sliding window then debugs | Prefix sum + hashmap immediately | Same + cites why sliding window fails |
| Negatives discussion | Doesn’t bring up | Realizes mid-solve | States upfront | States + relates to LC 525 / LC 974 family |
count[0]=1 | Forgets, debugs after empty subarray | Adds after one failing test | Includes from start with verbal justification | Includes + explains “phantom left wall” intuition |
| Generalization | None | “Probably works for similar problems” | LC 974, LC 525 by name | + Path Sum III on trees uses same template |
12. Level Delta
Mid (L4):
- Prefix sum + hashmap solution.
- O(N) time, O(N) space.
count[0]=1seed (possibly after one test failure).- Edge cases.
Senior (L5):
- Solution from the start with
count[0]=1and verbal justification. - Explains why sliding window fails on negatives.
- Tests with negatives and zeros explicitly.
Staff (L6):
- Connects to LC 974 (mod variant), LC 525 (treat-zero-as-minus-one trick), LC 437 (Path Sum III on trees — DFS + same hashmap).
- Discusses overflow if
nums[i]were big (Python is fine; in Java/C++ uselong). - Discusses parallelism: prefix sum is the classic parallel-scan target (Blelloch); hashmap counting is a sequential reduce — full parallelization is the work of Guy Blelloch’s NESL papers.
Principal (L7+):
- Discusses probabilistic count-min sketch for streaming over enormous data when exact count not required.
- Discusses k-dimensional generalization: count subrectangles with sum K in 2D matrix → fix two rows, collapse to 1D Subarray Sum K, O(R²·C). Apply to Kadane’s-family 2D problems.
- Discusses production: financial drawdown windows, anomaly windows in metrics, A/B test segment matching.
13. Follow-up Q&A
Q1: What if the array can be huge and we need streaming? A: The hashmap can grow to O(N) distinct prefix values — unbounded for streaming. Bounded approximation: count-min sketch with bounded memory. Exact streaming with sublinear memory is impossible.
Q2: 2D version — number of submatrices summing to K (LC 1074).
A: Reduce to 1D. For each pair of rows (r1, r2), compute column sums col[c] = sum(matrix[r1..r2][c]) (use 2D prefix to make this O(1) per column after preprocessing). Then run 1D Subarray Sum K on col. Total O(R²·C).
Q3: Return the actual subarrays (indices), not just count.
A: Change the hashmap value from int to List[int] storing all previous indices where each prefix value was seen. On match, emit (idx+1, current_idx) for each. Output size can be O(N²) in the worst case — be aware.
Q4: Find the LONGEST subarray summing to K (LC 325).
A: Hashmap stores FIRST index of each prefix value. On prefix - k lookup, length = current - first_index. Track max. The “longest” variant doesn’t need to count occurrences.
Q5: Find the count of subarrays where sum is in range [lower, upper] (LC 327).
A: Sort prefix sums and use merge-sort-counting or Fenwick tree (BIT). O(N log N). The hashmap pattern doesn’t directly extend to ranges.
14. Solution Walkthrough
See solution.py:
subarray_sum(nums, k)— canonical prefix-sum + hashmap. O(N).subarray_sum_brute(nums, k)— O(N²) double loop. Oracle.stress_test()— 300 random arrays with negatives, length ≤ 25, k ∈ [-10, 10].edge_cases()— empty, single positive/negative/zero match-or-not, all-zero, LC canonical, negatives interleaved, large positive only.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“The
count[0] = 1seed needs a comment — every code reviewer who hasn’t seen this template will ask ‘why?’. Either inline-comment ‘represents the empty prefix as a valid left boundary’ or pull the whole hashmap logic into aclass PrefixSumCounterwith a docstring. Second: if you’re processing many queries on the same array with varying K, precompute all prefix sums once and reuse — don’t re-iterate per query. Third: ifnumsis very wide (e.g., 64-bit values), the hashmap key space is enormous; in production C++/Java code, profile vsstd::maporHashMap<Long, Integer>and consider radix bucketing for sustained throughput. Fourth: if you’re going to extract this into a library, version the API — addingreturn_indices=Truelater is a breaking change if naive callers depend on the integer return type.”
Connections forward:
- LC 974 — Subarray Sums Divisible by K: same template, hashmap on
prefix % K. - LC 525 — Contiguous Array: treat 0 as -1, find longest subarray summing to 0.
- LC 437 — Path Sum III: same hashmap pattern, but during DFS on a binary tree.
- LC 1074 — Number of Submatrices Sum K: 2D extension via row-pair fixing.
- LC 327 — Count of Range Sum: prefix sums + Fenwick tree.
- p32 (this week) — 2D prefix sum: complementary technique.
- Phase 02 Lab 03 — Prefix Sums: template lab.
- Production: financial drawdown analysis, anomaly detection on metrics, log-window aggregation, A/B test cohort matching.
Hints — p31 Subarray Sum Equals K
Read one at a time.
Hint 1 — Brute first.
Two nested loops, accumulate sum from i to j, increment counter on match. O(N²). State this, then optimize.
Hint 2 — The prefix-sum identity.
Let prefix[t] = nums[0] + ... + nums[t-1] (with prefix[0] = 0). Then sum(i, j) = prefix[j+1] - prefix[i]. So sum(i, j) == k iff prefix[j+1] == prefix[i] + k, i.e., for each right endpoint j, count how many earlier prefix values equal prefix[j+1] - k.
Hint 3 — Hashmap of prefix counts.
Single pass. Maintain a running sum and a count[value] → frequency map. For each new prefix value p, look up count[p - k] (that’s how many subarrays ending here sum to k), then increment count[p]. O(N).
Hint 4 — Seed count[0] = 1.
Before iteration, the prefix-sum is 0 (empty prefix). Without seeding, a subarray that itself sums to k (starting from index 0) is missed. Memorize: count[0] = 1 always.
Hint 5 — Don’t reach for sliding window.
Sliding window only works when nums is all-non-negative (the running sum is monotonic in window growth). With negatives, the monotone property fails. Prefix sum + hashmap is the correct tool.
If still stuck: see solution.py. 8 lines for the optimal. Read the INVARIANT comment above count.
p32 — Range Sum Query 2D — Immutable
Source: LeetCode 304 · Medium · Topics: Array, Matrix, Prefix Sum, Design Companies (2024–2025 frequency): Microsoft (high), Amazon (high), Bloomberg (medium), Google (medium), Meta (medium) Loop position: algorithm round, often a “design + algorithm” hybrid because you must implement a class.
1. Quick Context
Design a class NumMatrix that:
- is initialized with an
R × Cinteger matrix, - supports
sumRegion(r1, c1, r2, c2)returning the sum of the subrectangle with corners(r1, c1)and(r2, c2)inclusive.
The matrix is immutable (no updates), and sumRegion may be called many times. The point of the problem: precompute once, query in O(1).
The technique is the 2D extension of prefix sums + inclusion-exclusion:
$$\text{sum}(r_1, c_1, r_2, c_2) = P[r_2{+}1][c_2{+}1] - P[r_1][c_2{+}1] - P[r_2{+}1][c_1] + P[r_1][c_1]$$
where P[r][c] = sum of the rectangle from (0,0) to (r-1, c-1) inclusive. The +1 border avoids special-casing the top row and left column.
Why the “+1 border” matters in interviews: it cleanly eliminates the if r1 == 0 or c1 == 0: branches that drown candidates in off-by-one bugs. Allocate (R+1) × (C+1) initialized to 0; the first row and column stay zero forever. Senior signal: do this without prompting.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/range-sum-query-2d-immutable/
STOP. 20-min timer. The 1D version (LC 303) is trivial; the 2D variant with inclusion-exclusion is where candidates trip on signs and off-by-ones.
3. Prerequisite Concepts
- 1D prefix sum.
- Inclusion-exclusion principle (basic 2-set version).
- OOP basics:
__init__, instance state, method.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Design a class that, given an immutable matrix, answers arbitrary rectangular sum queries in O(1) each, after preprocessing.”
Step 2 — Clarify:
- “Will
sumRegionbe called many times?” (Yes, that’s the whole point.) - “Is the matrix non-empty?” (Yes per spec — R≥1, C≥1.)
- “Inclusive bounds?” (Yes — both corners inclusive.)
- “r1 ≤ r2 and c1 ≤ c2?” (Yes — guaranteed.)
- “Values can be negative?” (Yes.)
- “Updates?” (No — immutable. LC 308 is the mutable variant — that’s Fenwick tree territory.)
Step 3 — Constraints: R, C ≤ 200, sumRegion calls ≤ 10⁴. With O(R·C) preprocessing and O(1) queries, this is trivial. The KEY metric isn’t raw N — it’s TOTAL work over many queries: O(R·C + Q).
Step 4 — Examples:
- LC canonical:
matrix = [[3,0,1,4,2],[5,6,3,2,1],[1,2,0,1,5],[4,1,0,1,7],[1,0,3,0,5]]sumRegion(2, 1, 4, 3)→ 8sumRegion(1, 1, 2, 2)→ 11sumRegion(1, 2, 2, 4)→ 12
Step 5 — Brute Force: Recompute sum in O((r2-r1+1)·(c2-c1+1)) per query. Acceptable for one query, but Q queries makes it O(Q·R·C) — way too slow when Q is large.
Step 6 — Complexity: Preprocessing O(R·C). Query O(1). Space O(R·C).
Step 7 — Pattern Recognition:
- “Many queries on immutable data, each query is an aggregate over a range” → precompute aggregate structure, query in O(1) or O(log).
- “Aggregate is sum” → prefix sums (additive group, supports subtraction).
- “Aggregate is min/max” → sparse table or segment tree (idempotent).
Step 8 — Develop: see solution.py.
Step 9 — Test: 1×1, single row, single column, full matrix, negative values, all-zero.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Inclusion-exclusion and the “+1 border”
6.1 What P[r][c] represents
Define P[r][c] = sum of matrix[i][j] for all 0 ≤ i < r and 0 ≤ j < c. (Note the strict < — this is the “+1 border” convention.) So P[0][*] = 0 and P[*][0] = 0 for free.
Build with the 2D recurrence (also inclusion-exclusion):
$$P[r][c] = \text{matrix}[r-1][c-1] + P[r-1][c] + P[r][c-1] - P[r-1][c-1]$$
The double-counted overlap P[r-1][c-1] is subtracted once.
6.2 Querying with inclusion-exclusion
To query the rectangle from (r1, c1) to (r2, c2) inclusive:
- Start with
P[r2+1][c2+1]— sum of everything “above and to the left of” (r2+1, c2+1), i.e., the rectangle(0,0)-(r2,c2). - Subtract
P[r1][c2+1]— the rectangle above our target row range. - Subtract
P[r2+1][c1]— the rectangle left of our target column range. - Add back
P[r1][c1]— the top-left rectangle was double-subtracted.
6.3 Why the “+1 border”
Without it, P is R × C and the indices in the query become r2, r1-1, c2, c1-1. When r1 == 0 or c1 == 0, you must skip the subtractions — that’s a branch. With the border (P is (R+1) × (C+1)), P[0][*] = P[*][0] = 0 handles those cases naturally. Fewer bugs, less code.
6.4 Generalizations
- Difference array (1D): the dual operation. If you want to do many range updates and one final point query, use a difference array (add
+xatl,-xatr+1; final prefix sum gives the answer). - Range update + range query, mutable: Fenwick tree (BIT) with lazy propagation, or segment tree.
- 2D mutable point update + range query: 2D Fenwick tree, O(log²(N)) per op.
- 2D range update + 2D range query: four 2D BITs (classical Sherman trick). Heavy machinery — rare in interviews.
The interview lesson: knowing this hierarchy lets you instantly classify problems and pick the right structure.
7. Anti-Pattern Analysis
- No “+1 border” → off-by-one bugs galore. Allocate
(R+1) × (C+1). Period. - Storing only row-prefix-sums (not 2D). Then query is O(R) per call (sum across rows). Works but suboptimal.
- Recomputing in
sumRegion. Defeats the purpose. - Building
Plazily insidesumRegion. Pay the build cost on first query; later queries are O(1). Works but breaks the “predictable latency” guarantee. - Wrong inclusion-exclusion signs. Memorize:
+full - top - left + topleft. Practice this on a 3×3 grid by hand once. - Modifying the input matrix in-place. The spec says immutable; some templates use the input matrix’s first row/column as scratch — fine for performance, terrible for API hygiene.
- Not validating r1 ≤ r2, c1 ≤ c2. LC guarantees this; production code should validate at the boundary.
- Returning
intwhen the sum could overflow (in Java/C++). Uselong. Python is fine.
8. Skills & Takeaways
- 2D prefix sum + inclusion-exclusion — burn this template.
- “+1 border” trick — same trick recurs in segment trees, Fenwick trees, KMP failure function.
- “Preprocess once, query in O(1)” — recognize this as a design pattern, not a one-off.
- Class/method API design in algorithm interviews.
- Difference array (dual) as a related tool.
9. When to Move On
Move on when:
- You can write the class with “+1 border” in under 10 min.
- You can recite the inclusion-exclusion formula from memory.
- Your stress test passes 200 random queries vs brute.
- You can pivot to LC 1314 (Matrix Block Sum) using the same prefix sum.
- You can sketch the difference-array dual for “many range updates, one query.”
10. Company Context
Microsoft:
- A frequent “design class” warmup. Bar Raiser will probe with: “now make it mutable — what’s the data structure?” (Answer: 2D Fenwick tree or segment tree.)
- Scorecard phrase: “Design + Algorithm — 2D prefix sum class with inclusion-exclusion.”
Amazon:
- Common “Day 1” or warmup question in algo loops. The Bar Raiser cares about clean API + O(1) query proof.
- Scorecard phrase: “Strong — recognized precompute-once pattern.”
Bloomberg:
- Natural fit for financial dashboards: precompute cumulative P&L over a 2D (time, asset) grid; query any window in O(1).
- Scorecard phrase: “Pragmatic — recognized 2D cumulative aggregation.”
Google:
- Often paired with the mutable version (LC 308) as the senior follow-up. 2D Fenwick tree expected.
- Scorecard phrase: “Algorithm — immutable solution + Fenwick tree for mutable variant.”
Meta:
- Less common but appears when the loop is design-heavy. Focus on API correctness and complexity guarantees.
- Scorecard phrase: “Design — immutable structure with deterministic O(1) queries.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Build approach | Row-prefix-sums (O(R) per query) | 2D prefix without border (off-by-one bugs) | 2D prefix with “+1 border” upfront | Same + cites the trick by name |
| Inclusion-exclusion | Trial and error | Gets it after one debug | Writes it correctly first try | Recites from memory + draws on whiteboard |
| Mutable follow-up | “I’d recompute” | “Some kind of tree?” | “2D Fenwick tree, O(log²) per op” | “2D Fenwick or 2D segment tree; trade-offs are constants and memory” |
| API hygiene | Modifies input matrix | Copies input fully | Stores precomputed P only | + discusses immutability of input as contract |
12. Level Delta
Mid (L4):
- 2D prefix sum solution with the “+1 border.”
- Correct inclusion-exclusion.
- O(R·C) build, O(1) query.
Senior (L5):
- Same + clean class API.
- Explains the “+1 border” rationale.
- Tests boundary cases (single row, single column, full matrix).
Staff (L6):
- Connects to the mutable variant (LC 308) and outlines 2D Fenwick tree.
- Discusses memory: if R, C are huge (say 10⁶ each), storing
Pis 10¹² entries — infeasible. Suggests sparse representation or block decomposition. - Discusses the difference-array dual.
Principal (L7+):
- Discusses GPU-friendly variant: prefix sum (scan) is a classic parallel algorithm (Blelloch scan, work-efficient, O(N) work, O(log N) depth). 2D prefix sum is row-then-column scans.
- Discusses out-of-core: for matrices that don’t fit in RAM (e.g., satellite imagery cumulative-band sums), tile the prefix sum with summary blocks (similar to integral image pyramids in image processing).
- Discusses production: integral image in OpenCV (Viola-Jones face detection), summed-area tables in graphics, billing systems for arbitrary-window aggregations.
13. Follow-up Q&A
Q1: Mutable variant — point updates allowed. A: 2D Fenwick tree. Build O(R·C·log²). Update O(log² N). Query O(log² N). For range updates + range queries, four 2D Fenwick trees (Sherman trick) or 2D segment tree with lazy propagation.
Q2: Memory is tight — can we avoid the auxiliary array? A: Yes, build in-place by adding to the original matrix (mutates input). Saves O(R·C) memory but breaks immutability contract. In Python, NOT recommended (lists are mutable but the input is the caller’s responsibility); in C++ with explicit ownership, sometimes acceptable.
Q3: Range max/min query in 2D — same precompute trick? A: No — min/max are not invertible (no subtraction). Use 2D sparse table (O(R·C·log R·log C) preprocessing, O(1) query) or 2D segment tree.
Q4: Many updates and queries interleaved on a huge matrix — what structure? A: 2D Fenwick tree if updates are point + queries are range. If both ops are range, four BITs. If R, C are huge but data is sparse, K-D tree variants or implicit segment trees.
Q5: Streaming variant — rows arrive one at a time, query “sum of last K rows × all cols” cheaply. A: Maintain rolling 1D arrays for column sums of last K rows; combine with a 1D prefix sum over columns. Amortized O(C) per row arrival, O(1) per query.
14. Solution Walkthrough
See solution.py:
NumMatrix(matrix)— preprocesses 2D prefixPwith “+1 border.”NumMatrix.sumRegion(r1, c1, r2, c2)— O(1) via inclusion-exclusion.NumMatrix.sumRegion_brute(r1, c1, r2, c2)— O(area) double loop for oracle.stress_test()— 200 random matrices and 50 random queries each, prefix vs brute.edge_cases()— 1×1, single row, single col, full matrix, negative values, LC canonical.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“The ‘immutable’ contract here is API-level, not language-level. In Python, callers can still mutate the input matrix they passed to
__init__AFTER construction — ourPbecomes stale silently. Either deep-copy in__init__(memory cost) or document loudly that the caller must NOT mutate post-construction. Don’t be smug about ‘immutable in the name’ — your users will not read the spec. For C++ this would beconst&and the compiler catches it; in Python it’s discipline. Second: if the matrix is sparse (mostly zeros), this O(R·C) memory is wasteful — consider adict-of-dictrepresentation or compressed sparse row. Third: the inclusion-exclusion needs a unit test that hand-computes a 2×2 example — every code reviewer doubts the signs. Make the test impossible to misread.”
Connections forward:
- p31 — Subarray Sum K: 1D prefix sum, the predecessor.
- LC 308 — Range Sum Query 2D Mutable: Fenwick tree generalization.
- LC 1314 — Matrix Block Sum: direct application — compute block-window sum for every cell.
- LC 363 — Max Sum of Rectangle No Larger Than K: 2D prefix + binary search via SortedList.
- LC 1074 — Number of Submatrices Sum K: 2D prefix + 1D Subarray Sum K reduction.
- Phase 02 Lab 03 — Prefix Sums: template lab.
- Production: integral images in computer vision (Viola-Jones, OpenCV), summed-area tables in graphics, billing aggregations, financial dashboard cumulative views.
Hints — p32 Range Sum Query 2D Immutable
Read one at a time.
Hint 1 — Recall 1D.
1D prefix sum: prefix[i] = nums[0]+...+nums[i-1]. Then sum(l, r) = prefix[r+1] - prefix[l]. Build O(N), query O(1). Now generalize to 2D.
Hint 2 — 2D prefix.
Define P[r][c] = sum of all cells (i, j) with 0 ≤ i < r and 0 ≤ j < c. So P has shape (R+1) × (C+1) with P[0][*] = P[*][0] = 0.
Hint 3 — Inclusion-exclusion (the formula).
For a query rectangle from (r1, c1) to (r2, c2) inclusive:
$$\text{answer} = P[r_2{+}1][c_2{+}1] - P[r_1][c_2{+}1] - P[r_2{+}1][c_1] + P[r_1][c_1]$$
Start with full top-left rectangle, subtract top, subtract left, add back the double-subtracted top-left corner.
Hint 4 — The “+1 border” trick.
Allocating (R+1) × (C+1) with the first row and column zeroed eliminates the if r1 == 0 / if c1 == 0 branches that wreck candidates. Every prefix-sum problem benefits from this.
Hint 5 — Build recurrence. $$P[r][c] = \text{matrix}[r-1][c-1] + P[r-1][c] + P[r][c-1] - P[r-1][c-1]$$ Same inclusion-exclusion at build time. Total O(R·C).
If still stuck: see solution.py. The class is ~20 lines.
p33 — Search in Rotated Sorted Array
Source: LeetCode 33 · Medium · Topics: Array, Binary Search Companies (2024–2025 frequency): Meta (very high), Amazon (high), Google (high), Microsoft (high), Apple (high) Loop position: algorithm round. One of the most-asked Mediums in 2024-25. The discriminator: do you write a CLEAN single-pass binary search, or a messy two-pass (find-pivot-then-search)?
1. Quick Context
Given a sorted ascending array of distinct integers nums that has been rotated at an unknown pivot, and a target value, return the index of target if present, else -1. Required: O(log N).
Example: nums = [4,5,6,7,0,1,2], target = 0 → 4. The array was [0,1,2,4,5,6,7] rotated at index 3.
The discriminator: the unwary candidate writes a two-pass solution — binary search to find the pivot, then binary search the right half. Works, but messy. The clean solution is single-pass binary search with a rotation-aware decision rule: at every step, one of the two halves around mid is guaranteed to be sorted (because the rotation creates exactly one discontinuity, and mid is in exactly one half). Determine which, check if target lies in that sorted half, and narrow accordingly.
The invariant: comparing nums[mid] to nums[lo] tells you which half is sorted in one comparison.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/search-in-rotated-sorted-array/
STOP. 25-min timer. The two-pass solution will be marked “works but messy” by Bar Raisers; the single-pass is the senior signal.
3. Prerequisite Concepts
- Plain binary search (LC 704).
- The mental picture: a rotated sorted array is two sorted runs concatenated.
- Off-by-one care with
lo,hi,mid, and<=vs<.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Sorted distinct integers, rotated at unknown pivot. Find index of target, O(log N), or -1.”
Step 2 — Clarify:
- “Distinct values?” (Yes for LC 33. LC 81 is the duplicates variant — different problem.)
- “Could the array be unrotated (rotated by 0)?” (Yes — must still work.)
- “Empty array?” (Return -1.)
- “Required O(log N)?” (Yes.)
Step 3 — Constraints: N ≤ 5000 typically, but the O(log N) requirement is the binding constraint. Linear scan is “wrong” even though it’d pass time limits.
Step 4 — Examples:
[4,5,6,7,0,1,2], target=0→ 4.[4,5,6,7,0,1,2], target=3→ -1.[1], target=0→ -1.[1], target=1→ 0.[3,1], target=1→ 1 (rotated at index 1; LEFT half[3]is “sorted” trivially, right half[1]contains target).[1,2,3,4,5], target=3→ 2 (unrotated case).
Step 5 — Brute Force: Linear scan O(N). State and pivot to binary search.
Step 6 — Complexity: O(log N) time, O(1) space.
Step 7 — Pattern Recognition:
- “Sorted (possibly transformed) array + need O(log N) lookup” → binary search, possibly with a custom decision rule.
- “Rotated sorted array” → one of the two halves around
midis always sorted. Memorize this invariant.
Step 8 — Develop: see solution.py.
Step 9 — Test: unrotated, rotated by 1, rotated by N-1, single-element, target absent, target at boundaries.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — The “which half is sorted” invariant
6.1 The geometry
A rotated sorted array (distinct values) looks like two ascending runs concatenated, e.g., [4,5,6,7] + [0,1,2]. There is exactly ONE “drop” — between index 3 and index 4 in this example. Anywhere else, nums[i] < nums[i+1].
For any lo <= mid <= hi, the drop is either:
- to the LEFT of
mid(somewhere in[lo, mid]), or - to the RIGHT of
mid(somewhere in[mid+1, hi]).
If the drop is to the right of mid, then nums[lo..mid] is sorted (no drop inside it).
If the drop is to the left of mid, then nums[mid..hi] is sorted.
6.2 The one-comparison test
How do we tell? Compare nums[mid] to nums[lo]:
- If
nums[lo] <= nums[mid], the LEFT half[lo, mid]is sorted. (Because if there were a drop in[lo, mid], thennums[mid]would be less thannums[lo], contradictingnums[lo] <= nums[mid].) - Else (
nums[mid] < nums[lo]), the RIGHT half[mid+1, hi]is sorted.
(The <= matters: when lo == mid, nums[lo] == nums[mid], the left half is trivially sorted — a single element.)
6.3 The narrowing decision
Once we know which half is sorted, we can check if target lies in it via the standard endpoint comparison:
- Left sorted:
targetis in left iffnums[lo] <= target < nums[mid]. If yes, narrow to left (hi = mid - 1); else, narrow to right (lo = mid + 1). - Right sorted:
targetis in right iffnums[mid] < target <= nums[hi]. If yes, narrow to right (lo = mid + 1); else, narrow to left (hi = mid - 1).
6.4 Why this is one-pass
We never explicitly find the pivot. Every iteration halves the search space, just like plain binary search, using a slightly richer decision rule. Total O(log N) — same as plain binary search, with a small constant.
6.5 Duplicates (LC 81)
If duplicates are allowed, the test nums[lo] <= nums[mid] is no longer enough — e.g., nums = [1,1,1,1,1,2,1,1], mid=4. We can’t tell which half is sorted. The fix: when nums[lo] == nums[mid] (and also == nums[hi] for the symmetric case), we can’t decide — increment lo (and/or decrement hi) by 1 and retry. Worst case O(N) on degenerate input.
7. Anti-Pattern Analysis
- Two-pass solution: find pivot then binary search. Works, twice the code, twice the off-by-one chances. Stick to one-pass.
- Linear scan. O(N) — violates the requirement.
- Using
nums[hi]instead ofnums[lo]as the comparison anchor. Works too, but the case logic is mirrored. Pick one convention and stick to it. Most idiomatic isnums[lo]. - Forgetting
<=in the “is left sorted” test. Whenlo == mid, you need the equality to handle the single-element left half. - Wrong boundary inequality in the “target in sorted half” check. Must use
nums[lo] <= target < nums[mid](not<= nums[mid]— that’s wherenums[mid]lives, and we already checked it). Off-by-one nightmare if you get this wrong. - Returning
midwithout first checkingnums[mid] == target. That’s literally the first thing inside the loop. - Infinite loop via
lo = mid(instead ofmid+1) when narrowing. Withwhile lo <= hi, always move pastmidon narrowing. - Conflating LC 33 (distinct) and LC 81 (duplicates). Different problems — different invariants.
8. Skills & Takeaways
- One-pass rotated binary search — burn the template.
- “One half is always sorted” invariant — articulate this clearly to the interviewer.
nums[lo] <= nums[mid]as the sortedness test — memorize the inequality direction.- Inclusive bounds (
lo,hi) +while lo <= hi— pick a convention and stick. - Set up for LC 81 (duplicates) and LC 153/154 (find min) as follow-ups.
9. When to Move On
Move on when:
- You can write the single-pass solution in under 8 minutes with correct boundaries.
- You can explain the “one half is sorted” invariant clearly.
- Your stress test passes 300 random rotated arrays vs linear-search oracle.
- You can pivot to LC 81 (duplicates) and explain the worst-case O(N).
- You can pivot to LC 153 (find min) using the simpler version of the same idea (p34).
10. Company Context
Meta:
- Among the top 5 most-asked Mediums in 2024-25 at Meta. The discriminator: clean one-pass binary search with rotation-aware decision rule.
- Bar Raiser asks: “what changes if duplicates are allowed?” (LC 81). Have the answer ready.
- Scorecard phrase: “Algorithm — clean rotated binary search, articulated the sortedness invariant.”
Amazon:
- Common L4/L5 phone screen and onsite. Often paired with LC 153 (find min) in the same loop.
- Scorecard phrase: “Strong — O(log N) on rotated array, clear case analysis.”
Google:
- May extend to LC 81 (duplicates) or LC 4 (median of two sorted arrays) as follow-up — both are binary-search-with-invariants.
- Scorecard phrase: “Algorithm + extension — rotated search + duplicate variant.”
Microsoft:
- Frequent for L62/L63 (mid-senior). The “rotation” framing is a popular wrapper around binary search across multiple Microsoft problem sets.
- Scorecard phrase: “Strong — clean binary search with rotation invariant.”
Apple:
- Often appears in Apple’s “system algorithms” loops. Bar Raiser cares about clarity over cleverness.
- Scorecard phrase: “Solid — readable rotated binary search.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Approach | Linear scan | Two-pass (find pivot, search) | One-pass with rotation invariant | Same + duplicates variant on demand |
| Invariant articulation | None | “Something about half being sorted” | “Comparing nums[mid] to nums[lo] decides which half is sorted” | + proves it briefly |
| Off-by-one handling | Many bugs | Bugs caught after running | Correct first try | + cites the <= vs < cases by reasoning |
| Follow-up readiness | None | Linear in duplicates | “LC 81 — duplicates need to skip equal endpoints; worst case O(N)” | + LC 153, LC 4 generalizations |
12. Level Delta
Mid (L4):
- One-pass binary search with rotation-aware decision rule.
- O(log N) time, O(1) space.
- Tests rotated and unrotated, target present and absent.
Senior (L5):
- Same + clean articulation of the invariant (“one half is always sorted”).
- Anticipates the duplicates variant and discusses worst case.
- Discusses why
<=vs<matters in the sortedness test.
Staff (L6):
- Connects to LC 81 (duplicates), LC 153 (find min — simpler invariant), LC 4 (median of two sorted arrays — binary search on partition).
- Discusses unification: “all of these are ‘binary search with a custom monotonic predicate’ — the predicate is the design choice.”
- Discusses production: rotating buffer search (ring buffer at unknown rotation offset), log-structured storage with rotation, time-series data after a clock-wrap.
Principal (L7+):
- Discusses branchless binary search for cache-friendly performance (Khuong & Morin 2017, branch predictor effects).
- Discusses parallel binary search for batched queries (multiple targets) — useful in computational geometry.
- Discusses galloping search (exponential then binary) for unknown-position bounded queries — used in Timsort, libstdc++
std::merge. - Discusses immutable-data system design: this is a search abstraction over an immutable rotated index; in a real system you’d cache the pivot and reuse, or maintain pivot via the same write that caused the rotation.
13. Follow-up Q&A
Q1: What if the array has duplicates? (LC 81)
A: The test nums[lo] <= nums[mid] no longer suffices. When nums[lo] == nums[mid], we can’t tell which half is sorted. Workaround: lo += 1 and retry. Worst case O(N) on [1,1,1,...,1,2,1,...].
Q2: Find the rotation pivot index. (LC 153)
A: Use while lo < hi: mid = (lo+hi)//2; if nums[mid] > nums[hi]: lo = mid+1 else: hi = mid. Loop ends with lo == hi == pivot index. See p34.
Q3: Find both target’s first and last occurrence in a sorted-non-rotated array.
A: Two binary searches with different tie-breaking: one biases left, one biases right. LC 34.
Q4: Search in a rotated 2D matrix (rows individually rotated independently). A: No O(log N·M) trick — each row is its own rotated sorted array; do binary search per row, O(M log N).
Q5: Streaming variant — array is being rotated by writes; queries arrive concurrently.
A: Maintain pivot index atomically alongside reads (CAS, RCU, or persistent data structure). The rotation index is a single 64-bit value; updates are cheap; readers compute “logical index” = (physical_index - pivot) mod N.
14. Solution Walkthrough
See solution.py:
search(nums, target)— one-pass rotated binary search. O(log N).search_brute(nums, target)— linear scan. Oracle.stress_test()— 300 random sorted arrays (length ≤ 20) rotated by random k, with target in array and target absent.edge_cases()— empty, single element, unrotated, rotated by 1 and N-1, target at boundaries, target absent.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“This binary search has FOUR boundary inequalities (
<=vs<in two places) plus a(lo + hi) // 2midpoint — that’s five chances for off-by-one. Write at least one unit test per branch. If you ship this without a stress test against a brute oracle, you will eat a P0 in production. Second: themid = (lo + hi) // 2form is fine in Python (arbitrary precision), but in C++/Java/Rust uselo + (hi - lo) // 2to avoid integer overflow —lo + hican wrap when both near INT_MAX. This is the canonical Joshua Bloch 2006 bug (‘Nearly All Binary Searches Are Broken’). Third: extract the predicate into a separate function — it makes the case analysis testable in isolation, and it lets you reuse the binary search template for LC 81, LC 153, LC 154. Don’t copy-paste the loop body across problems.”
Connections forward:
- p34 — Find Min in Rotated Sorted Array: same invariant, simpler decision rule.
- p35 — Median of Two Sorted Arrays: binary search on partition (different invariant family).
- LC 81 — Search in Rotated Sorted Array II: duplicates variant.
- LC 154 — Find Min in Rotated Sorted Array II: duplicates variant of p34.
- LC 162 — Find Peak Element: another “binary search with custom invariant.”
- LC 410 — Split Array Largest Sum: binary search on answer (different family — see Phase 02 Lab 04).
- Phase 02 Lab 04 — Binary Search on Answer: dedicated lab.
- Production: rotating-buffer search, time-series data after clock-wrap, log-structured storage indexing.
Hints — p33 Search in Rotated Sorted Array
Read one at a time.
Hint 1 — O(log N) means binary search. Even though the array is rotated, you must do O(log N). Linear scan is wrong even though it would pass time limits.
Hint 2 — The key invariant.
A rotated sorted array has exactly ONE drop. So for any mid, ONE of the two halves [lo..mid] and [mid+1..hi] is fully sorted (the drop is in the other half).
Hint 3 — Which half is sorted? One comparison.
Compare nums[mid] to nums[lo]:
- If
nums[lo] <= nums[mid], the LEFT half is sorted. - Else, the RIGHT half is sorted.
(Note the <= — when lo == mid, the left half is the trivial single-element sorted array.)
Hint 4 — Narrow into the sorted half if target is there.
- Left sorted: target ∈ left iff
nums[lo] <= target < nums[mid]. If yes,hi = mid - 1; elselo = mid + 1. - Right sorted: target ∈ right iff
nums[mid] < target <= nums[hi]. If yes,lo = mid + 1; elsehi = mid - 1.
Hint 5 — Loop frame.
lo, hi = 0, len(nums) - 1
while lo <= hi:
mid = (lo + hi) // 2
if nums[mid] == target: return mid
...
return -1
Inclusive lo, hi; while lo <= hi; always move past mid on narrowing (mid - 1 or mid + 1). Never lo = mid (infinite loop).
If still stuck: see solution.py. 12 lines for the main loop.
p34 — Find Minimum in Rotated Sorted Array
Source: LeetCode 153 · Medium · Topics: Array, Binary Search Companies (2024–2025 frequency): Amazon (high), Meta (high), Google (high), Microsoft (medium), Apple (medium) Loop position: algorithm round. Often paired with p33 (Search in Rotated Sorted Array) in the same loop — interviewer asks p34 first, then extends to p33.
1. Quick Context
Given a sorted ascending array of distinct integers that has been rotated at an unknown pivot, return the minimum element. Required: O(log N).
The minimum is the rotation pivot itself — the only element where nums[i-1] > nums[i] (or index 0 if not rotated).
The discriminator: the right invariant is tighter than p33’s. Compare nums[mid] to nums[hi] (NOT nums[lo]):
- If
nums[mid] > nums[hi], the pivot is STRICTLY to the right ofmid→lo = mid + 1. - Else, the pivot is at
midor to the left →hi = mid.
Loop condition is while lo < hi (not <=). The loop terminates when lo == hi, which IS the pivot index.
Many candidates botch this by reusing the p33 template (while lo <= hi, comparing to nums[lo]). The result: extra cases, off-by-one bugs, and a longer interview.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/find-minimum-in-rotated-sorted-array/
STOP. 20-min timer. Try to derive the invariant from scratch — don’t paste the p33 template.
3. Prerequisite Concepts
- Plain binary search.
- The mental picture of a rotated sorted array (two ascending runs).
- The convergent-bounds pattern:
while lo < hi, narrow toward a single answer index.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Distinct sorted array rotated at unknown pivot. Return min in O(log N).”
Step 2 — Clarify:
- “Distinct values?” (Yes for LC 153. LC 154 is duplicates — different.)
- “Could be unrotated?” (Yes — min is then
nums[0].) - “Empty array?” (No per spec — N ≥ 1.)
- “Required O(log N)?” (Yes.)
Step 3 — Constraints: N ≤ 5000 typically; the O(log N) requirement is binding.
Step 4 — Examples:
[3,4,5,1,2]→ 1.[4,5,6,7,0,1,2]→ 0.[11,13,15,17]→ 11 (unrotated).[2,1]→ 1.[1]→ 1.
Step 5 — Brute Force: Linear min(nums) is O(N). State and pivot.
Step 6 — Complexity: O(log N) time, O(1) space.
Step 7 — Pattern Recognition:
- “Find a discontinuity in a near-sorted array, O(log N)” → binary search with a convergent-bounds pattern (
while lo < hi). - “Compare to the right endpoint” — the asymmetry is the key. Anchoring on
nums[hi]gives a single clean invariant; anchoring onnums[lo]requires extra cases.
Step 8 — Develop: see solution.py.
Step 9 — Test: unrotated, rotated by 1, rotated by N-1, two-element, single-element, large rotations.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why anchor on nums[hi], not nums[lo]?
6.1 The invariant
In a rotated sorted array (distinct), the minimum (= pivot) has a clean characterization: it is the leftmost index p such that nums[p] <= nums[len-1]. The function f(i) = (nums[i] <= nums[len-1]) is monotonic over i — False, False, ..., False, True, True, ..., True — with the boundary AT the pivot. This is the classic “find first True in a Boolean monotonic predicate” pattern, perfectly suited to convergent binary search.
6.2 The decision rule
At any mid:
- If
nums[mid] > nums[hi], thenmidis in the LEFT sorted run (before the pivot). The pivot must be strictly to the right →lo = mid + 1. - Else (
nums[mid] <= nums[hi]), thenmidis in the RIGHT sorted run (at or after the pivot). The pivot could be atmidor to the left →hi = mid(NOTmid - 1— we might lose the answer).
One comparison, two cases, both narrow strictly. No third “found it” case — the loop runs until lo == hi.
6.3 Why while lo < hi (not <=)
When lo == hi, we’ve converged on the answer index. With <, the loop exits with lo == hi == pivot. With <=, we’d loop once more with mid == lo == hi, then narrow to an invalid state. The < form is the convergent-bounds idiom; memorize it.
6.4 Why NOT compare to nums[lo]
If we compare to nums[lo], the analysis splits into three subcases (unrotated, mid in left run, mid in right run) — the unrotated case must be specially handled. Compare to nums[hi] and unrotated works automatically (the whole array is <= nums[hi], so we always narrow hi = mid, ending at index 0).
6.5 Duplicates (LC 154)
With duplicates, nums[mid] == nums[hi] doesn’t tell us which run we’re in. Example: nums = [1,1,1,0,1], mid=2, hi=4 — both 1. Fix: when nums[mid] == nums[hi], do hi -= 1 (we lose one position but it’s safe — the pivot is still in [lo, hi-1] because nums[hi] is duplicated somewhere). Worst case O(N).
6.6 Connection to “binary search on answer”
Convergent bounds (while lo < hi) is the structural sibling of binary search on answer (LC 410, LC 875, LC 1011 — see Phase 02 Lab 04). Both narrow a continuous range using a monotonic predicate. p34 is the simplest possible instance of this template applied to an index range.
7. Anti-Pattern Analysis
- Reusing the p33 template (compare to
nums[lo],while lo <= hi). Works, but bloated with cases. The cleaner template here iswhile lo < hiandnums[mid] vs nums[hi]. hi = mid - 1instead ofhi = mid. Loses the answer whennums[mid]IS the minimum.while lo <= hi. Causes an extra iteration past convergence, wheremid == lo == hiand the inequality logic becomes vacuous (or infinite-loops if you also usehi = mid).- Special-casing the unrotated array (
if nums[0] < nums[-1]: return nums[0]). Unnecessary — the convergent loop handles it. - Returning
nums[lo]ANDnums[hi]from inside the loop without convergence. The loop must end naturally; reading mid-iteration is wrong. - Confusing pivot-index with min-value. The problem asks for the VALUE (
nums[lo]), not the index. Read carefully. - Linear scan
min(nums). O(N) — violates the O(log N) requirement. - Reaching for LC 154 (duplicates) code on the LC 153 problem. Slower than necessary and signals you don’t recognize the simpler invariant.
8. Skills & Takeaways
- Convergent-bounds binary search (
while lo < hi,hi = mid) — burn this template. - The
nums[mid] vs nums[hi]anchor — memorize the direction. - Monotonic-predicate framing — recognize that “first True in
f(i) = nums[i] <= nums[N-1]” is the same as p34. - Knowing when to break the symmetry (left vs right anchor) — design choice, not arbitrary.
- Setup for LC 154 (duplicates) and p35 (median of two sorted — binary search on partition).
9. When to Move On
Move on when:
- You can write the 6-line solution in under 5 min with correct boundaries.
- You can explain why
nums[hi](notnums[lo]) is the anchor. - Your stress test passes 300 random rotated arrays vs
min(nums)oracle. - You can derive the LC 154 (duplicates) variant on demand.
- You can apply the same template to LC 162 (Find Peak Element) and binary-search-on-answer problems.
10. Company Context
Amazon:
- Top-5 most-asked “binary search” Medium at Amazon in 2024-25. Often the first Q in the algo loop, followed by p33 as the senior extension.
- Scorecard phrase: “Strong — clean convergent-bounds binary search.”
Meta:
- Common warmup or “tie-breaker” Medium. Bar Raiser may chain into LC 154 (duplicates) for senior candidates.
- Scorecard phrase: “Strong — recognized the monotonic-predicate framing.”
Google:
- Often appears in algo loops paired with p35 (median) — Google likes the “binary search beyond simple lookup” theme.
- Scorecard phrase: “Algorithm — clean convergent binary search, articulated invariant.”
Microsoft:
- Standard L62 binary search question. Bar Raiser cares about clean code and boundary correctness.
- Scorecard phrase: “Solid — convergent binary search with correct boundaries.”
Apple:
- Less frequent but appears in systems/firmware loops where rotated-buffer search is a real concern.
- Scorecard phrase: “Pragmatic — binary search applied to rotated data.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Approach | min(nums) | Two-pass (find pivot via cases) | Convergent binary search with nums[hi] anchor | Same + duplicates variant on demand |
| Anchor choice | Whatever first guess | nums[lo] with cases | nums[hi] cleanly | + justifies the asymmetry |
| Loop form | while lo <= hi | while lo <= hi with bugs | while lo < hi cleanly | + connects to monotonic-predicate template |
| Duplicates extension | Doesn’t know LC 154 | Slow extension | “Skip equal hi by hi -= 1; worst case O(N)” | + proves the safety of the skip |
12. Level Delta
Mid (L4):
- Convergent binary search.
- Correct boundary,
while lo < hi. - Tests rotated and unrotated.
Senior (L5):
- Same + clean articulation of the monotonic-predicate framing.
- Explains why
nums[hi]is the right anchor. - Anticipates and discusses LC 154 (duplicates) worst case.
Staff (L6):
- Connects to binary-search-on-answer template (LC 410, LC 875).
- Connects to LC 162 (Find Peak Element) — same convergent template, different predicate.
- Discusses production: clock-wraparound in distributed time series, log-structured storage indices.
Principal (L7+):
- Discusses branchless binary search and cache effects (Khuong & Morin 2017, Eytzinger layout).
- Discusses galloping search for unknown-position bounded queries (Timsort, libstdc++).
- Discusses algorithm design as predicate engineering: “binary search isn’t about the loop; it’s about identifying a monotonic predicate over the answer space.”
13. Follow-up Q&A
Q1: What if duplicates are allowed? (LC 154)
A: When nums[mid] == nums[hi], we can’t tell which run mid is in. Safe fix: hi -= 1. The pivot must still be in [lo, hi-1] because nums[hi] is duplicated elsewhere in the range. Worst case O(N) on [1,1,1,...,0,1].
Q2: Return the pivot INDEX, not the value.
A: Same loop; return lo instead of nums[lo].
Q3: Find peak in a bitonic array. (LC 162)
A: Convergent binary search with predicate nums[mid] < nums[mid+1]. If yes, peak is to the right (lo = mid + 1); else, peak is at mid or left (hi = mid). Same template, different predicate.
Q4: Find first/last position of a target in a sorted array.
A: Two convergent searches with different bias. The “first” search uses nums[mid] >= target as the predicate; “last” uses nums[mid] > target then subtracts 1. LC 34.
Q5: 2D rotated matrix — each row independently rotated. A: Apply p34 per row, O(M log N).
14. Solution Walkthrough
See solution.py:
find_min(nums)— convergent binary search, O(log N).find_min_brute(nums)—min(nums). Oracle.stress_test()— 300 random sorted arrays (length 1..20) rotated by random k.edge_cases()— single element, unrotated, two-element, rotated by 1 and N-1, large rotations, with negatives.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“The whole point of this problem is the elegance of the convergent-bounds template. Resist the urge to add the
if nums[0] < nums[-1]: return nums[0]‘optimization’ — it adds a branch, hides the invariant, and saves zero asymptotic time. The clean version reads exactly the way the invariant proves: ‘find first index wherenums[i] <= nums[N-1]’. Anyone refactoring this code later will appreciate the lack of special cases. Second: in a code review, ask whether the function should return the index or the value — the LC problem says value, but a real internal API often wants both. Make the contract explicit. Third: if you find yourself writing duplicate variants of this function for LC 153 and LC 154, extract a single function with aduplicates_allowedparameter — DRY beats clever per-problem rewrites.”
Connections forward:
- p33 — Search in Rotated Sorted Array: harder cousin, same array shape.
- p35 — Median of Two Sorted Arrays: binary search on partition — different predicate space.
- LC 154 — Find Min in Rotated Sorted Array II: duplicates variant.
- LC 162 — Find Peak Element: same convergent template.
- LC 410 — Split Array Largest Sum: binary search on answer (Phase 02 Lab 04).
- LC 875 — Koko Eating Bananas: same family.
- Phase 02 Lab 04 — Binary Search on Answer: full template lab.
- Production: clock-wraparound search in distributed time series, log-structured storage indexing.
Hints — p34 Find Minimum in Rotated Sorted Array
Read one at a time.
Hint 1 — The min is the rotation pivot.
In a rotated sorted array, the min is the only index where nums[i-1] > nums[i] (or index 0 if unrotated). It’s where the “drop” happens.
Hint 2 — Anchor on nums[hi], not nums[lo].
Comparing nums[mid] to nums[hi] gives a single clean invariant. Comparing to nums[lo] forces extra cases for the unrotated array.
Hint 3 — The decision rule.
- If
nums[mid] > nums[hi], the pivot is STRICTLY right ofmid→lo = mid + 1. - Else, the pivot is at
midor LEFT ofmid→hi = mid(NOTmid - 1).
Hint 4 — Convergent loop.
while lo < hi:
mid = (lo + hi) // 2
...
return nums[lo]
Use < (not <=). Loop ends with lo == hi == pivot index.
Hint 5 — Why hi = mid (not mid - 1)?
Because nums[mid] itself might be the min. If we set hi = mid - 1, we lose the answer.
If still stuck: see solution.py. The loop is 5 lines.
p35 — Median of Two Sorted Arrays
Source: LeetCode 4 · Hard · Topics: Array, Binary Search, Divide & Conquer Companies (2024–2025 frequency): Google (very high), Meta (high), Apple (high), Amazon (medium), Microsoft (medium) Loop position: algorithm round, hardest slot. This is the canonical “binary-search-on-partition” Hard. A successful O(log(min(m,n))) solution is a strong staff+ signal.
1. Quick Context
Given two sorted arrays nums1 (length m) and nums2 (length n), return the median of the combined m + n elements. The required complexity is O(log(min(m, n))).
The straightforward O(m + n) merge gives the answer easily — but the problem demands log time, which means: binary search on the partition of the smaller array.
The setup:
- Total length
T = m + n. The median splits the combined sorted array into two equal halves (or, whenTis odd, the left half has one more element). - Choose a partition
iofnums1(0..m) and the matching partitionj = (T+1)//2 - iofnums2(so the combined left half has(T+1)//2elements). - The partition is correct iff
maxLeft(A) <= minRight(B)ANDmaxLeft(B) <= minRight(A). - Binary search on
iover[0, m]. Each step adjusts based on which of the two cross-inequalities fails.
The discriminator: writing a correct binary-search-on-partition for the first time in an interview is extremely hard if you haven’t done it before. Most candidates get O(m+n) merge and a Bar Raiser response of “good — now can you do log?” If you fold under that pressure, you’re capped at mid-level. If you produce the O(log) solution with correct partition handling, you’re signaling staff+.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/median-of-two-sorted-arrays/
STOP. 35-min timer for the merge approach AND the partition approach. This problem deserves the full time.
3. Prerequisite Concepts
- Convergent-bounds binary search (see p34).
- Two-pointer merge (LC 88 — Merge Sorted Array).
- The concept of a “partition” of a sorted array — index
isays “the firstielements go to the left side.” - Sentinel values (
-inf,+inf) for edge partitions.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Two sorted arrays. Find the median of their union. O(log(min(m, n))).”
Step 2 — Clarify:
- “Are the arrays each sorted ascending?” (Yes.)
- “Can one or both be empty?” (At least one is non-empty. Per LC.)
- “Definition of median for even total: average of two middle elements?” (Yes.)
- “Return float?” (Yes.)
- “Can values be negative?” (Yes.)
- “Required complexity?” (O(log(min(m, n))).)
Step 3 — Constraints: m, n ≤ 1000. O(m + n) is fast enough, but the interview requires O(log). The discriminator is whether you produce log.
Step 4 — Examples:
[1,3], [2]→ 2.0 (combined[1,2,3], median 2).[1,2], [3,4]→ 2.5 (combined[1,2,3,4], median (2+3)/2).[], [1]→ 1.0.[2], []→ 2.0.[1,2,3,4,5], [6,7,8,9,10]→ 5.5 (no overlap).[1,1,1], [1,1,1]→ 1.0.
Step 5 — Brute Force:
- O((m+n) log(m+n)): concatenate and sort. Trivial.
- O(m + n): merge with two pointers, stop at the median index. State and pivot — but also show that you CAN do brute first to set up the contrast.
Step 6 — Complexity: O(log(min(m, n))) time, O(1) space. We binary-search the partition of the SMALLER array — log of its length.
Step 7 — Pattern Recognition:
- “Sorted data + need sub-linear” → binary search.
- “Combine two sorted things and answer a ‘split’ query” → binary search on partition.
- The partition invariant
maxLeft(A) <= minRight(B) AND maxLeft(B) <= minRight(A)is the heart of the technique.
Step 8 — Develop: see solution.py. Implement BOTH the merge oracle and the partition binary search.
Step 9 — Test: all the edge cases above, plus interleaved cases, plus stress test against the merge oracle.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Binary search on partition
6.1 The partition framing
Let A = nums1 (length m), B = nums2 (length n), with m <= n (swap if not). The total length is T = m + n.
A “partition” is a pair (i, j) with 0 <= i <= m and 0 <= j <= n such that:
- The left side
A[0..i-1] + B[0..j-1]has exactly(T + 1) // 2elements. - The right side
A[i..] + B[j..]has the otherT // 2elements.
So j = (T + 1) // 2 - i — once i is fixed, j is determined.
6.2 The correctness invariant
A partition (i, j) is “correct” (median-revealing) iff:
$$\text{maxLeft}(A) \leq \text{minRight}(B) \quad \text{AND} \quad \text{maxLeft}(B) \leq \text{minRight}(A)$$
where maxLeft(A) = A[i-1] (or -inf if i == 0), minRight(A) = A[i] (or +inf if i == m), and similarly for B.
If both inequalities hold, then EVERYTHING on the left side is <= EVERYTHING on the right side. The median is determined by the boundary:
- If
Tis odd, median =max(maxLeft(A), maxLeft(B)). - If
Tis even, median =(max(maxLeft(A), maxLeft(B)) + min(minRight(A), minRight(B))) / 2.
6.3 Why binary search on i converges
If maxLeft(A) > minRight(B) — A’s left is too big → too many A elements on the left → decrease i (search left half of [0, m]).
Otherwise (maxLeft(B) > minRight(A)) — B’s left is too big (we have too FEW A elements on the left) → increase i (search right half of [0, m]).
The predicate “is i too large?” is monotonic — that’s why binary search works. Convergent template, same as p34.
6.4 Why we binary search the SMALLER array
So the search range is [0, m] with m = min(m, n). This gives O(log(min(m, n))). If we accidentally searched the larger one, we’d get O(log(max(m, n))) — still log, but the spec is stricter.
There’s another reason: j = (T + 1) // 2 - i. If i is in [0, m] and m <= n, then j is automatically in [0, n] (provable from (T+1)//2 >= m). If we swapped, we’d have to clamp j.
6.5 Sentinels
The edge cases (i == 0, i == m, j == 0, j == n) make maxLeft or minRight undefined. The clean fix: use float('-inf') and float('inf') as sentinels. This eliminates a tangle of conditional branches.
6.6 Why this is staff-level
The bar isn’t “do you know binary search?” — it’s “do you recognize when to binary-search a derived parameter (the partition index) instead of the data itself?” This is the same intellectual leap as binary-search-on-answer (LC 410, LC 875). Once you see it, you see it everywhere: it’s the spine of LC 4, LC 162, LC 410, LC 875, LC 1011, LC 1482, and many more.
7. Anti-Pattern Analysis
- Concatenate-and-sort. O((m+n) log(m+n)). Works but signals you didn’t see the structure.
- O(m+n) merge as final answer. Works but capped at mid-level. The interviewer will push for log.
- Binary-searching the LARGER array. Gets the wrong complexity bound;
jcan go out of range. - Forgetting sentinels. A pile of
if i == 0/elsebranches that nobody (including you) can debug live. - Wrong partition formula. Common bug:
j = T // 2 - i(drops one element whenTis odd). The correct formula isj = (T + 1) // 2 - i, which puts the extra element on the left for odd T (so the median ismax(maxLeft(A), maxLeft(B))). (lo + hi) // 2integer overflow in C++/Java. (Python is fine.) Uselo + (hi - lo) // 2.- Returning
intfor even-total medians. The answer is a float (e.g., 2.5). Use float division/, not//. - Reaching for the “k-th smallest of two sorted arrays” approach. Also valid (and a great problem), but more code and trickier edge cases than the partition approach.
- Manual case-by-case for tiny m (≤ 2). Looks clever but adds complexity. The general partition handles tiny m correctly.
8. Skills & Takeaways
- Binary search on derived parameter (partition index) — a pattern, not a trick.
- Partition invariant:
maxLeft(A) <= minRight(B) AND maxLeft(B) <= minRight(A). Memorize. - Sentinels (
±inf) as edge-case eliminator. - Always search the smaller array.
- Parity-aware median formula: odd → max of left; even → average of max-left and min-right.
- The conceptual link to LC 410, LC 875 — binary search on answer.
9. When to Move On
Move on when:
- You can write the merge oracle in under 5 min.
- You can write the partition binary search in under 25 min (this Hard is genuinely Hard).
- You can articulate the partition invariant from memory.
- Your stress test passes 200 random pairs vs the merge oracle.
- You can derive the “k-th smallest of two sorted arrays” extension on demand.
10. Company Context
Google:
- One of THE most-asked Hard problems at Google for L5+ in 2024-25. Bar Raisers expect either the partition solution or a strong attempt with correct framing.
- The “did the candidate see the binary-search-on-partition pattern” is the central decision point.
- Scorecard phrase: “Algorithm — produced O(log) median via binary search on partition. Strong signal.”
Meta:
- Less common than at Google, but appears in senior loops. Even reaching the O(m+n) merge cleanly is good; the O(log) is the staff+ signal.
- Scorecard phrase: “Strong — O(m+n) merge confidently, attempted O(log) partition.”
Apple:
- Asked in algo loops, particularly for media/systems teams. Often paired with a “what if the arrays are streams” follow-up.
- Scorecard phrase: “Algorithm — partition binary search, articulated invariant.”
Amazon:
- Less common in standard L5 loops but appears in L6+ or Bar Raiser slots.
- Scorecard phrase: “Strong — Hard problem solved with correct complexity.”
Microsoft:
- Standard “Hard slot” for L63+. Bar Raiser may ask for both the merge and the partition approach.
- Scorecard phrase: “Strong — both approaches with clear reasoning.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Approach | Concatenate-sort | O(m+n) merge | Attempts partition, may not finish | Partition with sentinels, clean |
| Sentinels | None | Maybe a few if branches | ±inf sentinels from the start | + explains why they eliminate edge cases |
| Which array to search | Either | Whichever is convenient | Always smaller | + proves why (j-range) |
| Median parity | Bug on even or odd | Handles after one fail | Correct first try | + states the (T+1)//2 choice and why |
| Follow-up | “Don’t know” | “Use k-th smallest?” | “k-th smallest in two sorted arrays” | + streaming extension, k-way generalization |
12. Level Delta
Mid (L4):
- O(m+n) merge solution.
- Handles odd and even totals.
- Handles empty array.
Senior (L5):
- Attempts the partition binary search.
- Articulates the partition invariant correctly even if implementation has bugs.
- Uses sentinels.
- Tests odd, even, empty, all-same, no-overlap.
Staff (L6):
- Clean O(log(min(m, n))) partition solution with sentinels.
- Connects to “binary search on derived parameter” pattern (LC 410, LC 875).
- Discusses k-th smallest extension (LC 4 generalized).
- Discusses what changes for K sorted arrays — heap-based merge, log(K) median tracking with two heaps (LC 295 family).
Principal (L7+):
- Discusses distributed/streaming variants: maintaining a running median over two streams = LC 295 (two heaps). Maintaining a median across federated sorted shards = log-ranked search via approximate quantile sketches (t-digest, KLL).
- Discusses GPU-friendly variant: parallel partition via “split-merge” on a balanced binary tree of subarrays.
- Discusses production: financial median-price-over-N-sources, A/B test result median across cohorts, distributed monitoring p50 latency.
13. Follow-up Q&A
Q1: K-th smallest element across two sorted arrays.
A: Generalize the partition. Pick a partition (i, j) with i + j = K. The k-th smallest is max(A[i-1], B[j-1]) if the partition is valid. Binary search over i in [max(0, K-n), min(K, m)]. O(log(min(m, n, K))).
Q2: Median of K sorted arrays.
A: Two approaches: (1) Heap merge — extract the (T/2)-th element. O(T log K). (2) Generalized partition: binary-search a “pivot” and count how many elements in each array are below the pivot. Binary search on the pivot value, O(K log V log K) where V is the value range. Harder, often overkill.
Q3: Running median of two infinite streams. A: Two heaps (max-heap for lower half, min-heap for upper half). Insert: O(log T). Median: O(1). This is LC 295.
Q4: What if the arrays are NOT sorted but we have access to “sort hints”? A: Sort first (O(T log T)), then apply partition. There’s no shortcut without structure.
Q5: Approximate median for huge unsortable streams. A: Quantile sketches: t-digest, GK, KLL. Sub-linear memory, ε-approximate median. Production-ready (used in Datadog, Prometheus).
14. Solution Walkthrough
See solution.py:
find_median_sorted_arrays(nums1, nums2)— partition binary search. O(log(min(m,n))).find_median_brute(nums1, nums2)— merge with two pointers (O(m+n)). Oracle.stress_test()— 200 random pairs (lengths 0..15 each, at least one non-empty), values in [-30, 30].edge_cases()— empty + non-empty, single elements, no overlap, full overlap, all equal, interleaved, odd and even totals, negatives.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“This solution lives or dies by the partition invariant. Write a clear comment block at the top of the function with the formula, the
j = (T+1)//2 - iderivation, and a worked example — your future self (and reviewers) will thank you. Second: thefloat('inf')sentinels are clever but ALSO subtle — if you ever extend this to integer-only environments (some embedded systems), you’ll need INT_MAX/MIN and have to think about overflow. Document the sentinel choice. Third: if this function is going into a library, expose BOTH theO(m+n)merge variant AND theO(log)partition variant — the merge is more cache-friendly for small arrays, and for typical inputs (say m+n < 64), it’s likely FASTER in wall-clock time. Profile, then pick the right default. Fourth: write at least one test that exercises every cross-check failure case —maxLeft(A) > minRight(B)ANDmaxLeft(B) > minRight(A)— to make sure the binary search direction is right both ways.”
Connections forward:
- p33, p34 — rotated binary search siblings.
- LC 295 — Find Median from Data Stream: dynamic-streaming variant via two heaps.
- LC 295’s heap pattern → general top-K and quantile structures.
- LC 215 — K-th Largest Element: quickselect or heap.
- LC 410 — Split Array Largest Sum: binary search on answer (Phase 02 Lab 04).
- LC 875 — Koko Eating Bananas: binary search on answer.
- Phase 02 Lab 04 — Binary Search on Answer: full template lab.
- Phase 02 Lab 09 — Heap Top-K: complementary technique.
- Production: distributed median (quantile sketches: t-digest, GK, KLL), monitoring p50 latency, A/B test median computation.
Hints — p35 Median of Two Sorted Arrays
Read one at a time. This problem is Hard — take your time.
Hint 1 — Brute first. Two-pointer merge, stop at the median index. O(m+n). State it. Then the interviewer will ask for O(log).
Hint 2 — Don’t binary-search a value — binary-search a partition.
Imagine splitting the combined sorted array into LEFT and RIGHT halves. The median is determined by the boundary. We choose i elements from A and j elements from B for the left side, with i + j = (T+1)//2. So once i is chosen, j is forced.
Hint 3 — The correctness condition.
A partition (i, j) is valid iff:
A[i-1] <= B[j](A’s left fits to the left of B’s right)B[j-1] <= A[i](B’s left fits to the left of A’s right)
Use -inf / +inf sentinels for the edge cases (i==0, i==m, j==0, j==n).
Hint 4 — Binary search direction.
- If
A[i-1] > B[j](too many A on left) → shrinki:hi = i - 1. - Else (too few A on left, so
B[j-1] > A[i]) → growi:lo = i + 1.
Loop while lo <= hi.
Hint 5 — Median formula. Once you find a valid partition:
- Odd total: median =
max(maxLeftA, maxLeftB). - Even total: median =
(max(maxLeftA, maxLeftB) + min(minRightA, minRightB)) / 2.
ALWAYS binary-search the SMALLER array (swap if needed) — guarantees j stays in [0, n] and gives O(log(min(m, n))).
If still stuck: see solution.py. The function is ~30 lines but every line earns its keep.
Week 08 — Backtracking: The Decision-Tree Discipline
Month 02, Week 04 (overall week 08) · 5 problems · 3 mediums + 1 medium-hard + 1 hard Theme: Search trees with explicit choose / recurse / un-choose. Pruning. Lexicographic vs free order. Symmetry breaking.
Why this week
Backtracking is the technique most candidates fake. They produce code that “works on the example” but breaks on duplicates, fails to deduplicate, mutates the wrong state, or — most commonly — forgets to un-choose. By the end of this week you will:
- Have a single mental template you write reflexively: choose → recurse → un-choose.
- Distinguish permutations (order matters, all elements used,
used[]tracker) vs combinations (size-k subset, monotonically non-decreasing index) vs subsets (powerset, include/exclude at each index). - Prune at the FIRST opportunity instead of at the leaf — the difference between TLE and AC.
- Handle the duplicate-input variant (sort + skip equal siblings) and the lex-order vs free-order distinction.
- Solve a 2D grid backtracking problem (Word Search) with in-place visited-marking.
- Build N-Queens — the canonical “symmetry-breaking + column/diagonal sets” problem — in under 20 minutes.
Problems
| # | Problem | Difficulty | LC | Pattern signature |
|---|---|---|---|---|
| p36 | Permutations | Medium | 46 | used[] tracker; n! leaves |
| p37 | Combinations | Medium | 77 | start-index + k constraint; C(n,k) leaves |
| p38 | Subsets | Medium | 78 | include/exclude OR iterative powerset; 2^n leaves |
| p39 | Word Search | Medium-Hard | 79 | grid backtracking + in-place visited |
| p40 | N-Queens | Hard | 51 | column + diag + anti-diag sets; row-by-row |
Daily schedule (5 days)
| Day | Problem | Stop at |
|---|---|---|
| Mon | p36 Permutations | clean template, both used[] and swap-in-place |
| Tue | p37 Combinations | start-index + early termination |
| Wed | p38 Subsets | both DFS and iterative |
| Thu | p39 Word Search | in-place visited, prune on first char mismatch |
| Fri | p40 N-Queens | three sets (col, diag, anti-diag) — O(n!) bounded by pruning |
Readiness gate (end of week)
You can move to Week 9 when you can:
- Write the universal backtracking template (choose / recurse / un-choose) in under 60 seconds.
- Solve any “all permutations / all combinations / all subsets” variant in under 15 min.
-
Explain WHY
used[]is needed for permutations but astartindex suffices for combinations and subsets. -
Handle the with-duplicates variant (
sort()+if i > start and a[i] == a[i-1]: continue) without looking it up. - Word Search: explain why in-place visited-marking saves O(n·m·L) memory.
-
N-Queens: place the THREE attacker sets (col, diag =
r + c, anti-diag =r - c) from memory. - Articulate the difference between “exponential time” (unavoidable for enumeration) vs “exponential WORK PER NODE” (pruning attacks the constant).
Anti-patterns to crush this week
- Building strings via
s + chinside recursion (O(L) copy per call) — use alistand.append()/.pop(). - Forgetting
.pop()after.append()— your “current path” leaks into siblings. - Passing
path[:]to recurse — works but wastes O(L) per call; the un-choose pattern is cleaner. - Returning a
setof tuples to deduplicate at the end — should have pruned at the source via sort+skip. - Word Search marking visited with a separate
set— use in-place sentinel ('#') and restore. - N-Queens validating a board column-by-column on each placement — use sets (O(1) lookup).
- Re-counting
len(path) == kin the loop instead of at function entry.
Sources
- Sedgewick, Algorithms, Ch 2 (Backtracking).
- CLRS, Ch 35 (Approximation), Ch 34 (NP) — for the “why exponential is fundamental” perspective.
- Skiena, Algorithm Design Manual, Ch 9 (Combinatorial Search).
p36 — Permutations
Source: LeetCode 46 · Medium · Topics: Array, Backtracking Companies (2024–2025 frequency): Meta (very high), Google (high), Amazon (high), Microsoft (high), Apple (medium) Loop position: algorithm round opener for backtracking. A weak Permutations attempt signals the candidate hasn’t internalized the technique.
1. Quick Context
Given an array of distinct integers, return all n! permutations. The follow-up (LC 47 — Permutations II, with duplicates) is the staff-level signal.
The canonical pattern is choose / recurse / un-choose with a used[] boolean array:
def backtrack(path):
if len(path) == n:
result.append(path[:]) # snapshot
return
for i in range(n):
if used[i]: continue
used[i] = True; path.append(nums[i]) # choose
backtrack(path) # recurse
path.pop(); used[i] = False # un-choose
Time: O(n · n!) (n! permutations × O(n) to copy each). Space: O(n) recursion depth.
The discriminator: a clean used[] template, plus the in-place swap variant (which trades clarity for O(1) less space) and the duplicate-input variant (sort + skip).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/permutations/
STOP. 15-min timer. Write the template from memory.
3. Prerequisite Concepts
- Recursion + stack frames.
- The DFS template (Phase 01 Lab 08).
- Why
result.append(path[:])and notresult.append(path)— the list is mutated by un-choose.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “All orderings of n distinct elements. Return as list of lists.”
Step 2 — Clarify:
- “Distinct integers?” (Yes — that’s the constraint that makes
used[]sufficient.) - “Output order matters?” (No — any order accepted.)
- “Constraints on n?” (n ≤ 6 typically. n! grows fast.)
- “Return list or generator?” (List per spec.)
Step 3 — Constraints: n ≤ 6 → 720 permutations. Even at n=10 it’s 3.6M. The constraint is small precisely because the OUTPUT is exponential.
Step 4 — Examples:
[1,2,3]→ 6 permutations.[1]→[[1]].[]→[[]](the empty permutation; verify with interviewer).
Step 5 — Brute Force: Iterative generation via itertools.permutations. State, then write the manual backtracking — the interviewer wants the recursion.
Step 6 — Complexity: O(n · n!) time (n! results, O(n) copy). O(n) recursion + O(n) used[]. Output is O(n · n!).
Step 7 — Pattern: “Enumerate all X” → backtracking. The shape of X (permutation/combination/subset) picks the template:
- Permutation:
used[]array, all elements at each level. - Combination:
startindex, monotonically non-decreasing. - Subset: include/exclude OR start-index variant.
Step 8 — Develop: see solution.py. Both used[] and swap-in-place variants.
Step 9 — Test: edge cases (empty, single, duplicates-in-input-shouldnt-happen-but-verify). Stress vs itertools.permutations.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Three permutation algorithms
6.1 used[]-array backtracking (the default)
Pros: clearest code; matches the universal backtracking template; trivially extends to LC 47 (duplicates) via sort + skip-if-duplicate-sibling.
Cons: O(n) extra space for the used[] array.
6.2 Swap-in-place (Heap’s-algorithm-style)
def backtrack(start):
if start == n:
result.append(nums[:])
return
for i in range(start, n):
nums[start], nums[i] = nums[i], nums[start] # choose: bring i to position start
backtrack(start + 1)
nums[start], nums[i] = nums[i], nums[start] # un-choose: swap back
Pros: no used[] array. Mutates input but restores it.
Cons: produces permutations in a different (non-lexicographic) order. Less obvious correctness. Cannot be adapted to LC 47 (duplicates) without explicit duplicate tracking — the swap technique loses the “we’ve already tried this value at this position” information.
6.3 Iterative (Heap’s algorithm proper)
Generates the next permutation by a clever swap-and-reverse sequence. O(1) extra space, O(n!) iterations. Beautiful but rarely needed in interviews — mention it only if asked about generating permutations on a memory budget.
6.4 LC 47 — duplicates (THE follow-up)
Sort the input. In the loop, skip a value if its previous-equal sibling is not currently used:
for i in range(n):
if used[i]: continue
if i > 0 and nums[i] == nums[i-1] and not used[i-1]: continue
...
Why not used[i-1]? Because when used[i-1] IS true, we’re inside a subtree where nums[i-1] was already placed earlier in the path — so nums[i] is now a legitimate next choice. When used[i-1] is FALSE, we’ve returned from that subtree (un-chose it), and choosing the equal-value nums[i] next would regenerate the same permutation.
This is the single hardest line in backtracking. Memorize it.
6.5 Why pure deduplication-via-set is wrong
A common bad pattern: generate all n! permutations (with duplicates), then set(tuple(p) for p in result). This is exponential WORK for what should be linear-in-output. With n=8 and many duplicates, the set approach generates 40320 permutations to find ~100 unique ones — 400× wasted work. The interviewer notices.
7. Anti-Pattern Analysis
result.append(path)instead ofpath[:]. Result: all entries point to the same mutated list → after the recursion ends, every entry is[].- Building a new list per recursion (
backtrack(path + [nums[i]])). Works but allocates a fresh O(n) list per node — n! nodes × n = O(n² · n!) work. The append/pop pattern is O(n · n!). used = [False] * ninside the recursion. Resets state on every call — produces wildly wrong output.usedmust be hoisted to the closure.- Forgetting the
pop(). Path grows monotonically; first result is the only correct one. set()dedup at the end for LC 47. Exponentially wasteful (see 6.5).- Using a string for
path.path = path + chis O(L) copy per call; use a list. - Validating “is this a permutation” at the leaf. Pruning happens AT THE CHOICE, not at the result.
- Outputting tuples instead of lists. Off-spec for LC; trivially wrong on grader.
8. Skills & Takeaways
- The universal choose / recurse / un-choose template — write it from muscle memory.
used[]for permutations;startindex for combinations/subsets.result.append(path[:])to snapshot mutable state.- Duplicate-input dedup line:
if i > 0 and a[i] == a[i-1] and not used[i-1]: continue. - The 3 variants and their trade-offs.
- Why backtracking time is O(branching^depth × leaf-work).
9. When to Move On
Move on when:
- You can write the
used[]template in under 90 seconds. - You can write the swap variant in under 2 minutes.
- You can write LC 47 (duplicates) in under 5 minutes.
- Your stress test of 300 random inputs (n=1..5) matches
itertools.permutations.
10. Company Context
Meta:
- One of the canonical Meta backtracking warm-ups. The interviewer expects clean code and immediate transition to LC 47.
- Scorecard phrase: “Algorithm — clean backtracking template, articulated the dedup rule.”
Google:
- Common L4/L5 opener. Often paired with “now constrain by some predicate” (e.g., “only valid IP-address-like permutations”).
- Scorecard phrase: “Strong — wrote template confidently; pruned at choice not leaf.”
Amazon:
- L5 phone screen staple. Bar Raisers care about correctness + complexity discussion + handling of edges.
- Scorecard phrase: “Strong — handled duplicates with sort+skip; explained the
not used[i-1]line.”
Microsoft:
- L63 standard. Often combined with “now print them in lex order” — gives you the sort up front.
Apple:
- Appears in algo rounds especially for media teams (audio sequencing, animation timing).
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Template | Recursive copy (n²·n!) | append/pop (n·n!) | + explains complexity | + cites Heap’s algorithm |
| Duplicates (LC 47) | Set dedup | Sort + naive skip | Correct not used[i-1] rule | + proves canonical-order argument |
| Output | path (bug) | path[:] | path[:] + explains | + discusses generator vs list trade-off |
| Variants | One variant | used[] | used[] + swap | used[] + swap + Heap’s algorithm (next-perm) |
12. Level Delta
Mid (L4):
used[]template, correct snapshot, n=1 and n=3 work.
Senior (L5):
- Both
used[]and swap-in-place. - Solves LC 47 with correct dedup.
- Discusses O(n · n!) time and O(n) extra space.
Staff (L6):
- All three (used, swap, Heap’s next-permutation iterative).
- Discusses generator-based variants for memory-bounded environments.
- Recognizes when permutation enumeration is the wrong tool — e.g., k-th permutation (LC 60) via factorial-number-system, NOT enumeration.
Principal (L7+):
- Discusses production: permutation enumeration appears in cryptographic test fixtures, brute-force key-search tools (rare), constraint solvers (CSP backtracking), and combinatorial test generation.
- Discusses parallel backtracking: distribute subtrees across workers; load-balancing via work-stealing.
- Discusses probabilistic vs exhaustive search: when n is too large, switch to Monte Carlo / simulated annealing / genetic algorithms.
13. Follow-up Q&A
Q1: K-th permutation (LC 60).
A: Don’t enumerate. Use the factorial number system: the i-th element of the k-th permutation is determined by k // (n-1-i)! after each step. O(n²) (or O(n) with linked-list deletion). NEVER backtrack to find the k-th.
Q2: Permutations with duplicates (LC 47).
A: Sort + if i > 0 and a[i] == a[i-1] and not used[i-1]: continue. The not used[i-1] ensures we only pick the FIRST equal value at any given depth — canonical-form pruning.
Q3: Generate permutations one at a time (iterator).
A: Python generator with yield. Iterative version: Heap’s algorithm or next-permutation (LC 31). O(1) amortized per generation.
Q4: Permutations of size k (partial permutations) — P(n, k).
A: Same template, exit when len(path) == k instead of n. Or use combinations + permutations (C(n, k) * k!).
Q5: Distributed permutation enumeration. A: Each worker takes a fixed prefix (e.g., first 2 elements), generates suffixes locally. Static partition is easy; work-stealing fixes load imbalance.
14. Solution Walkthrough
See solution.py:
permute(nums)—used[]backtracking. Default.permute_swap(nums)— swap-in-place variant. Mutates and restores input.permute_unique(nums)— LC 47 with sort + skip-dedup.permute_brute(nums)—itertools.permutationsoracle.edge_cases()+stress_test()— verify all three vs the oracle.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Three things. (1) The
path[:]snapshot is THE bug source for everyone new to backtracking — write a one-line comment explaining why. (2) For LC 47, thenot used[i-1]is non-obvious; add a worked example in the comments showing how[1,1,2]produces[[1,1,2],[1,2,1],[2,1,1]](3 perms) instead of 6. (3) If this code goes into a library, expose a GENERATOR variant — for n=10 the result list is 3.6M items × ~80 bytes = 280MB. A generator lets the caller consume on the fly. Same template, replaceresult.append(...)withyield list(path).”
Connections forward:
- p37 — Combinations: same template, different choice rule (start index).
- p38 — Subsets: same template, two-way choice (include/exclude).
- p40 — N-Queens: backtracking with stateful attacker sets.
- LC 31 — Next Permutation: iterative O(n) alternative.
- LC 60 — Permutation Sequence: factorial number system.
- LC 47 — Permutations II: the duplicate variant (THE staff signal).
- Phase 02 Lab 08 — Backtracking: full pattern lab.
- Production: constraint solvers (CSP), combinatorial test generation, branch-and-bound.
Hints — p36 Permutations
Hint 1. “Enumerate all” → backtracking. Choose / recurse / un-choose.
Hint 2. You need to know which elements are already in the current path. A used = [False] * n array works.
Hint 3. Skeleton:
def backtrack(path):
if len(path) == n:
result.append(path[:]) # SNAPSHOT — the list mutates after we return
return
for i in range(n):
if used[i]: continue
used[i] = True; path.append(nums[i])
backtrack(path)
path.pop(); used[i] = False
Hint 4. Why path[:] (snapshot) and not path? Because we path.pop() after the recursive call — by the end, path == []. If you stored the reference, every result entry would point to the same empty list.
Hint 5. LC 47 (duplicates): sort the input, and in the loop add:
if i > 0 and nums[i] == nums[i-1] and not used[i-1]: continue
This produces each unique permutation exactly once. The not used[i-1] is the subtle part — it ensures we only consume equal values in left-to-right order.
If still stuck: see solution.py.
p37 — Combinations
Source: LeetCode 77 · Medium · Topics: Backtracking Companies (2024–2025 frequency): Amazon (high), Google (medium-high), Meta (medium), Microsoft (medium), Apple (medium) Loop position: algorithm round, second slot. Tests whether the candidate distinguishes “permutation” from “combination” — a common confusion.
1. Quick Context
Given integers n and k, return all C(n, k) combinations of k numbers chosen from [1..n]. Order within a combination is non-decreasing by convention (and to deduplicate naturally).
The key distinction from permutations: a start index replaces used[]. Each recursion picks values from [start..n], ensuring monotonically increasing selection — which automatically deduplicates {1,2} and {2,1}.
def backtrack(start, path):
if len(path) == k:
result.append(path[:]); return
for i in range(start, n + 1):
path.append(i)
backtrack(i + 1, path)
path.pop()
Time: O(k · C(n, k)). Space: O(k) recursion.
The discriminator: the pruning optimization — for i in range(start, n - (k - len(path)) + 2) — which skips branches that can’t reach length k. Without it, you still iterate over impossible starts; with it, you cut significant work at large n.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/combinations/
STOP. 15-min timer.
3. Prerequisite Concepts
- p36 (Permutations).
- The combinatorial identity
C(n, k) = C(n-1, k-1) + C(n-1, k)(Pascal’s rule). - Why monotonic order (
i < jfor path positionsi, j) deduplicates.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “All k-subsets of {1, 2, …, n}, as lists.”
Step 2 — Clarify:
- “Numbers from 1 to n inclusive?” (Yes.)
- “k ≤ n?” (Yes per spec.)
- “Order within each combination?” (Non-decreasing — natural for dedup.)
Step 3 — Constraints: n ≤ 20, k ≤ n. C(20, 10) ≈ 184k. Manageable.
Step 4 — Examples:
n=4, k=2→[[1,2],[1,3],[1,4],[2,3],[2,4],[3,4]].n=1, k=1→[[1]].k=0→[[]].
Step 5 — Brute Force: Iterate all 2^n subsets, keep those of size k. O(2^n) — vastly worse than O(k · C(n,k)) for small k. State it.
Step 6 — Complexity: O(k · C(n, k)) time. O(k) recursion depth.
Step 7 — Pattern: “Choose k from n without repetition, unordered” → combinations template (start index).
Step 8 — Develop: see solution.py. Both naive and pruned loop bound.
Step 9 — Test: edge cases (k=0, k=n, n=1), stress vs itertools.combinations.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — The pruning that changes everything
6.1 The naive bound vs the tight bound
Naive: for i in range(start, n + 1). Iterates even when it’s IMPOSSIBLE to complete path to length k.
Tight: for i in range(start, n - (k - len(path)) + 2).
Derivation: we need to add (k - len(path)) more elements. The last possible starting value i is one that still leaves (k - len(path) - 1) elements above it in [i+1..n]. That requires n - i >= k - len(path) - 1, i.e., i <= n - (k - len(path)) + 1. So range’s upper bound (exclusive) is n - (k - len(path)) + 2.
For n=20, k=10, the naive variant explores ~3× more nodes than the pruned one. For larger n, the gap widens.
6.2 Why the start index suffices (no used[])
In permutations, we revisit “early” elements at deeper levels (e.g., [2,1,3] after [1,2,3]). In combinations, we never revisit — once we choose i, the next pick is from [i+1..n]. The strictly increasing structure makes {1,2} and {2,1} the SAME thing, and we generate only the canonical (non-decreasing) form.
6.3 LC 39 — Combination Sum (with reuse)
Same template, but backtrack(i, ...) (not i+1) — allows reusing the same element. Plus a target -= candidates[i] accumulator with target == 0 as the success condition and target < 0 as a prune.
6.4 LC 40 — Combination Sum II (with duplicates in input, no reuse)
Same template + the same dedup line as LC 47: if i > start and candidates[i] == candidates[i-1]: continue. Note i > start (not i > 0) — at the same recursion level, skip equal siblings; the first time at a level, take it.
6.5 LC 216 — Combination Sum III
k numbers from [1..9] summing to n. Pure combination of LC 77 + LC 39 logic.
6.6 Connection to Pascal’s triangle
C(n, k) = C(n-1, k-1) [contains n] + C(n-1, k) [doesn’t contain n]. This is the “include/exclude” framing of the same algorithm — the recursion tree has the same shape, just viewed differently. The include/exclude framing dominates subsets (p38).
7. Anti-Pattern Analysis
- Using
used[]as in permutations. Generates k! × C(n,k) = P(n,k) results with massive duplication. The dedup-at-end is exponentially wasteful. backtrack(start, ...)then recursing withstartinstead ofi+1. Allows reuse — that’s LC 39, not LC 77.- No pruning bound. Works but slow for large n.
- Generating all 2^n subsets and filtering by size k. O(2^n) — vastly slower for small k.
result.append(path)instead ofpath[:]. Same bug as p36.- Off-by-one on the range —
range(start, n)instead ofrange(start, n + 1)(1-indexed!). - For LC 39 (reuse), forgetting the
target < 0prune — the recursion explodes.
8. Skills & Takeaways
- Start-index template = the combination signature.
- Pruning bound:
n - (k - len(path)) + 2. Memorize. - Reuse variant = recurse with
i, noti+1. - Duplicates-in-input variant = sort +
if i > start and a[i] == a[i-1]: continue. - The include/exclude view (Pascal’s rule) — bridges to subsets (p38).
9. When to Move On
Move on when:
- You can write the basic combinations template in under 90 seconds.
- You can add the pruning bound from memory.
- You can write LC 39 and LC 40 in under 5 minutes each.
- Stress test of 200 random
(n, k)matchesitertools.combinations.
10. Company Context
Amazon:
- L5 staple. Combined with “now find combinations summing to target” (LC 39) is the standard 2-part question.
- Scorecard phrase: “Algorithm — used start-index template; pruning bound on second pass.”
Google:
- L4/L5. Often paired with LC 78 (Subsets) to test whether you see the family connection.
- Scorecard phrase: “Strong — clean template, articulated connection to subsets and permutations.”
Meta:
- Standard backtracking warm-up; rare as the sole question.
Microsoft:
- L63. Often combined with “now stream them” (generator variant).
Apple:
- Common in algo loops, particularly for ML / data teams (feature combinations).
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Approach | Permute-and-dedupe | Start index, naive bound | Start index + pruning bound | + cites Pascal’s rule and the include/exclude view |
| Variants | Knows only LC 77 | + LC 39 | + LC 40 (dedup) | + LC 216 + connection to dynamic programming (knapsack family) |
| Output | Includes duplicates | Clean | Clean + complexity | + generator variant for memory |
12. Level Delta
Mid (L4):
- Start-index template, correct snapshot.
- Handles
k=0,k=n,n=1.
Senior (L5):
- Adds the pruning bound and explains the derivation.
- Solves LC 39 (reuse) and LC 40 (duplicates) on demand.
Staff (L6):
- Discusses the include/exclude framing and connection to DP / knapsack.
- Generator variant for streaming.
- Discusses when combinations are the wrong tool — e.g., k-th combination (LC 60 analogue) via combinatorial number system.
Principal (L7+):
- Discusses production: feature selection in ML, A/B test bucket enumeration, combinatorial test generation (pairwise / t-wise coverage tools like ACTS), portfolio combinatorial optimization.
- Discusses parallel enumeration via prefix-based work splitting.
- Discusses lattice / Bratteli diagrams for advanced combinatorial enumeration.
13. Follow-up Q&A
Q1: Combination Sum (LC 39) — same value usable infinitely.
A: Recurse with i instead of i + 1. Subtract from target; success when target == 0; prune when target < 0.
Q2: Combination Sum II (LC 40) — duplicates in input, each used once.
A: Sort + if i > start and a[i] == a[i-1]: continue + recurse with i + 1.
Q3: K-th combination of [1..n] choose k. A: Combinatorial number system. At each position, the i-th element of the k-th combination is determined by counting how many combinations start with each prefix. O(k · n) — no enumeration needed.
Q4: Generate combinations as a stream.
A: Python generator with yield list(path) instead of result.append(...). Same template.
Q5: Why not bitmask?
A: For k-combinations from n ≤ 32, enumerate bitmasks with exactly k bits set (Gosper’s hack: t = v | (v - 1); w = (t + 1) | (((~t & -~t) - 1) >> (v.bit_length() of (v & -v)))). O(1) per next-combination. Beautiful for n ≤ 64; backtracking is clearer.
14. Solution Walkthrough
See solution.py:
combine(n, k)— start-index template with pruning bound.combine_naive(n, k)— without pruning, for comparison.combine_brute(n, k)—itertools.combinationsoracle.edge_cases()+stress_test()— 200 random (n, k), all variants agree.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Two notes. (1) The pruning bound
n - (k - len(path)) + 2is correct but reads like noise. Inline a one-line comment showing the derivation, and consider naming a local:max_first = n - need + 1. (2) If this is a hot path, switch to bitmask enumeration with Gosper’s hack — branchless, no recursion, ~10× faster for n ≤ 64. But ONLY for hot paths — for everything else, the recursion is more maintainable. (3) For very large n with sparse k (e.g., n=10000, k=3), the combinatorial number system gives O(k) per next-combination — useful when you want to sample or stream.”
Connections forward:
- p38 — Subsets: include/exclude view of the same template.
- p36 — Permutations: contrasting “all orderings” vs “unordered choice”.
- LC 39 — Combination Sum: reuse variant.
- LC 40 — Combination Sum II: duplicate-input variant.
- LC 216 — Combination Sum III: bounded-k + target.
- LC 90 — Subsets II: dedup version of p38.
- Phase 02 Lab 08 — Backtracking lab.
- Phase 03 Lab 08 — Bitmask DP: connection to bit-enumeration via Gosper’s hack.
- Production: feature selection (ML), A/B-test bucket design, combinatorial test coverage.
Hints — p37 Combinations
Hint 1. Same family as p36, but here {1,2} and {2,1} are the SAME thing. How do you avoid generating both?
Hint 2. Pick values in monotonically increasing order. Pass a start index — the next pick is from [start..n].
Hint 3. Skeleton:
def backtrack(start, path):
if len(path) == k:
result.append(path[:]); return
for i in range(start, n + 1):
path.append(i)
backtrack(i + 1, path) # i+1 — no reuse, monotonic
path.pop()
Start the recursion with backtrack(1, []) (1-indexed!).
Hint 4. Pruning: if you still need need = k - len(path) more elements, the max i you can pick must leave need - 1 values above it. So i_max = n - need + 1. Loop upper bound: n - need + 2.
Hint 5. Variants:
- LC 39 (reuse): recurse with
i, noti+1. Tracktarget. - LC 40 (dup input): sort +
if i > start and a[i] == a[i-1]: continue.
If still stuck: see solution.py.
p38 — Subsets
Source: LeetCode 78 · Medium · Topics: Array, Backtracking, Bit Manipulation Companies (2024–2025 frequency): Meta (high), Amazon (high), Google (high), Microsoft (medium), Apple (medium) Loop position: algorithm round; often paired with Combinations (p37) and Permutations (p36) to test family recognition.
1. Quick Context
Given an array of distinct integers, return the powerset — all 2^n subsets. The follow-up (LC 90 — Subsets II, with duplicates) is the staff signal.
Three equally valid approaches — interviewer-level signal comes from knowing all three and the trade-offs:
- Backtracking with start index (variant of p37’s template — yield at EVERY node, not just leaves).
- Include/exclude DFS (binary choice tree, depth = n).
- Iterative powerset growth — for each new element, double the result by including it in every existing subset.
- Bitmask enumeration (n ≤ 32) — iterate masks
0..2^n - 1, decode bits.
Time: O(n · 2^n) — 2^n subsets, O(n) to copy each. Space: O(n) recursion (or O(1) for the iterative/bitmask variants, ignoring output).
The discriminator: writing the include/exclude template cleanly, AND knowing why bitmask is fastest for n ≤ 20.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/subsets/
STOP. 15-min timer.
3. Prerequisite Concepts
- p37 (Combinations) — subsets are “all sizes” instead of fixed k.
- Pascal’s identity: 2^n = sum over k of C(n, k).
- Bit manipulation basics:
mask >> i & 1.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “All 2^n subsets of the input array. Any order.”
Step 2 — Clarify:
- “Distinct elements?” (Yes — that’s the LC 78 spec. LC 90 is the duplicate variant.)
- “Include the empty subset?” (Yes.)
- “Order of subsets in output?” (Free.)
- “Order within each subset?” (Free, but typically input order is convention.)
Step 3 — Constraints: n ≤ 10 → 1024 subsets. n ≤ 20 → ~1M. Tight enough that the algorithm matters but not lifesaving.
Step 4 — Examples:
[1,2,3]→[[],[1],[2],[3],[1,2],[1,3],[2,3],[1,2,3]](8 subsets).[]→[[]].
Step 5 — Brute Force: Bitmask enumeration is already O(n · 2^n). The “backtracking” framing is for pedagogy.
Step 6 — Complexity: O(n · 2^n). Space O(n) recursion or O(1) bitmask.
Step 7 — Pattern: “Powerset / all-subset” → 2^n enumeration. Three equivalent algorithms.
Step 8 — Develop: see solution.py — start-index, include/exclude, iterative, bitmask, and LC 90 (duplicates).
Step 9 — Test: edge cases (empty, single, full overlap), stress vs itertools.chain over combinations.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Four algorithms, one shape
6.1 Start-index backtracking — “snapshot at every node”
def backtrack(start, path):
result.append(path[:]) # SNAPSHOT here, not at a leaf
for i in range(start, n):
path.append(nums[i])
backtrack(i + 1, path)
path.pop()
The crucial difference from p37: result.append is at the TOP of the function, not gated by len(path) == k. Every node of the recursion tree is itself a valid subset.
The recursion tree has 2^n nodes (each node = one subset).
6.2 Include/exclude DFS
def backtrack(i, path):
if i == n:
result.append(path[:]); return
# exclude nums[i]
backtrack(i + 1, path)
# include nums[i]
path.append(nums[i])
backtrack(i + 1, path)
path.pop()
Depth = n, branching = 2. Generates 2^n leaves; each leaf is a subset. Mirrors the bit-by-bit decision: “is element i in the subset?”
This is the cleanest framing for connecting to DP (e.g., knapsack: include/exclude each item) and to LC 698 (Partition to K Equal Sum Subsets).
6.3 Iterative powerset growth
result = [[]]
for x in nums:
result += [sub + [x] for sub in result]
Starts with [[]]. For each new element, doubles the result by including the element in every existing subset. O(n · 2^n) time, no recursion. The most elegant code.
6.4 Bitmask enumeration
for mask in range(1 << n):
subset = [nums[i] for i in range(n) if mask >> i & 1]
result.append(subset)
O(n · 2^n). Trivially parallelizable (split masks across workers). Fastest in practice for n ≤ 20 (no recursion overhead, cache-friendly).
6.5 LC 90 — Subsets II (duplicates in input)
Same as LC 78 backtracking with the same dedup line as LC 40:
for i in range(start, n):
if i > start and nums[i] == nums[i-1]: continue
...
Note: i > start, NOT i > 0. At the same recursion level (same start), skip equal siblings; the first time at a level, take it. This is the canonical-form pruning principle from p36/p37/p40 — internalize it.
6.6 Why so many algorithms?
Each one teaches a different concept:
- Start-index: the “yield at every node” pattern (also used in tree traversals).
- Include/exclude: bridges backtracking → DP. Knapsack is exactly this.
- Iterative: the “build up” pattern. Cleanest code; hardest to extend.
- Bitmask: bridges to bitmask DP (LC 526, LC 691).
A staff candidate uses include/exclude when extending to DP, bitmask when n is small and speed matters, iterative when readability matters, and start-index when extending to size-restricted variants (LC 77, LC 39).
7. Anti-Pattern Analysis
- Snapshot inside the for-loop instead of at function entry — misses the empty subset and produces duplicates.
- Using
setoffrozensetto dedup for LC 90. Exponentially wasteful; the sort+skip line dedupes at the source. for i in range(n)instead offor i in range(start, n)— generates duplicates{1,2}and{2,1}.result.append(path)instead ofpath[:]— same bug as p36.- Bitmask without
int.bit_length()or pre-sorting when needed — order issues. - For n=30+ using bitmask — 2^30 = 1B subsets. Output is huge regardless of algorithm; the issue is whether you NEED all of them. For “count” or “sum over subsets,” switch to DP.
- For LC 90, dedup line
i > 0instead ofi > start— over-prunes; loses valid subsets.
8. Skills & Takeaways
- Four algorithms, all O(n · 2^n) but with different code/extension properties.
- “Snapshot at every node” vs “snapshot at leaves” — the structural choice.
- LC 90 dedup:
i > start and nums[i] == nums[i-1]. - Bitmask = the bridge to bitmask DP (Phase 03).
- Include/exclude = the bridge to knapsack DP (Phase 05).
9. When to Move On
Move on when:
- You can write all four variants in under 10 min total.
- You can write LC 90 (duplicates) in under 5 min.
- You can articulate which algorithm you’d choose in each context (small n, large n, DP extension, parallel).
- Stress test of 200 random inputs (n=0..8) matches a reference oracle.
10. Company Context
Meta:
- L4/L5 standard. Often paired with LC 90 immediately, then LC 39, then “now count subsets with sum = S” (DP).
- Scorecard phrase: “Algorithm — articulated 3 algorithms, picked the right one for the follow-up.”
Amazon:
- L5 staple. The “iterative powerset growth” version is a frequent expected answer.
Google:
- L5+ uses this as a launching pad to bitmask DP (LC 526, LC 691, LC 1349) — be ready to discuss.
Microsoft:
- L63. Often combined with “how many subsets sum to S” → bridge to DP.
Apple:
- Common in algo loops; data and ML teams use the “all feature combinations” framing.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Approach | One algorithm | Two (backtracking + iterative) | All four, picks the right one | + extends to bitmask DP and knapsack |
| LC 90 dedup | Set-based | i > 0 rule (wrong) | i > start rule (correct) | + proves canonical-form correctness |
| Connections | None | “Like combinations” | + Pascal’s identity | + bridge to subset-sum DP, partition DP, bitmask DP |
12. Level Delta
Mid (L4):
- One correct algorithm (backtracking or iterative).
- Handles empty input.
Senior (L5):
- Two algorithms minimum (start-index + iterative OR include/exclude + bitmask).
- Solves LC 90 with correct dedup.
- Discusses O(n · 2^n) and why output is the limit.
Staff (L6):
- All four algorithms.
- Connects include/exclude to knapsack DP and bitmask to bitmask DP.
- Discusses streaming / generator variant.
- Handles “subset with property X” (count, sum, etc.) — knows when to switch to DP.
Principal (L7+):
- Discusses production: feature-flag combinations (test all on/off combos), formal verification (state-space enumeration for tiny systems), policy enumeration in RL (small action spaces), regulatory compliance (audit all combinations of attributes).
- Discusses subset-sum is NP-hard; powerset enumeration is exponential — when n ≥ 30, switch to DP, branch-and-bound, or approximation.
- Discusses sketching: for “approximately count subsets with property P,” use hashing/sampling.
13. Follow-up Q&A
Q1: Subsets II (LC 90) — duplicates in input.
A: Sort input. In the loop: if i > start and nums[i] == nums[i-1]: continue. Canonical-form pruning — at each recursion level, take the first equal sibling only.
Q2: Count subsets with sum = target.
A: Don’t enumerate. DP: dp[i][s] = dp[i-1][s] + dp[i-1][s - nums[i]]. O(n · S) time, O(S) space with rolling array. Subset-sum DP; LC 416 family.
Q3: K-th subset in lex order. A: Combinatorial number system, or interpret k’s binary representation as a subset-mask (with care about ordering).
Q4: Subsets summing to target with each element usable once. A: LC 39 / LC 40 / LC 416 family — DP or backtracking + pruning.
Q5: Subsets when n = 50. A: 2^50 ≈ 10^15. Can’t enumerate. Use meet-in-the-middle: split into halves of 25, enumerate each (2^25 ≈ 33M each), join. Or DP if there’s structure. Or sampling if approximate is OK. See Phase 03 Lab 09 — Meet-in-the-Middle.
14. Solution Walkthrough
See solution.py:
subsets_backtrack(nums)— start-index, snapshot at every node.subsets_include_exclude(nums)— binary choice tree.subsets_iterative(nums)— powerset growth, no recursion.subsets_bitmask(nums)— enumerate0..2^n - 1.subsets_with_dup(nums)— LC 90, sort + skip-dedup.subsets_brute(nums)— oracle viaitertoolschain over combinations.edge_cases()+stress_test()— verify all five vs oracle.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Three things. (1) Pick ONE algorithm as the ‘canonical’ for the codebase — most teams should standardize on iterative (
result = [[]]; for x in nums: result += [s+[x] for s in result]) because it’s 5 lines and obviously correct. (2) For LC 90, the dedup line is THE thing that goes wrong; add a unit test specifically for[1,2,2,3]that asserts no duplicate subset appears. (3) If subsets are an INPUT to downstream code (not the final answer), don’t materialize them — write a generator (yield from ...) and let the caller consume on the fly. For n=20 that’s a 100× memory win.”
Connections forward:
- p37 — Combinations: size-restricted subsets.
- p36 — Permutations: ordered variant.
- LC 90 — Subsets II: dedup variant.
- LC 416 — Partition Equal Subset Sum: counting via subset-sum DP.
- LC 698 — Partition to K Equal Sum Subsets: backtracking + pruning.
- Phase 03 Lab 08 — Bitmask DP.
- Phase 03 Lab 09 — Meet-in-the-Middle (for n=40+).
- Phase 05 — DP labs: knapsack, subset-sum, partition.
- Production: feature flag combinations, regulatory audit, formal verification of small systems.
Hints — p38 Subsets
Hint 1. “All 2^n subsets”. You have multiple algorithm choices. Try one.
Hint 2. Easiest framing: iterative growth. Start with [[]]. For each new element, double the result by appending it to every existing subset:
result = [[]]
for x in nums:
result = result + [s + [x] for s in result]
Hint 3. Backtracking framing: it’s like p37, but snapshot at EVERY node, not just at leaves. Every prefix is a valid subset.
def backtrack(start, path):
result.append(path[:]) # snapshot here
for i in range(start, n):
path.append(nums[i]); backtrack(i+1); path.pop()
Hint 4. Include/exclude framing: at each index, branch into “skip nums[i]” and “take nums[i]”:
def backtrack(i, path):
if i == n: result.append(path[:]); return
backtrack(i+1, path) # exclude
path.append(nums[i]); backtrack(i+1, path); path.pop() # include
Hint 5. For LC 90 (duplicates), sort + at each level skip equal siblings:
for i in range(start, n):
if i > start and nums[i] == nums[i-1]: continue
...
Note i > start, NOT i > 0.
If still stuck: see solution.py.
p39 — Word Search
Source: LeetCode 79 · Medium-Hard · Topics: Array, Backtracking, Matrix, DFS Companies (2024–2025 frequency): Meta (very high), Amazon (high), Microsoft (high), Google (high), Apple (medium) Loop position: algorithm round, frequently the centerpiece. The “in-place visited marking” trick is a senior-vs-mid discriminator.
1. Quick Context
Given a 2D grid of letters and a target word, return True if the word can be formed by a path through horizontally/vertically adjacent cells, where each cell is used at most once.
The pattern is grid backtracking: for each cell matching word[0], DFS into the 4 neighbors trying to match word[1], word[2], etc., with in-place visited marking (overwrite the cell with a sentinel like '#', restore on backtrack).
def dfs(r, c, i):
if i == len(word): return True
if not in_bounds(r, c) or board[r][c] != word[i]: return False
board[r][c] = '#' # mark visited (in-place)
found = any(dfs(r+dr, c+dc, i+1) for dr, dc in DIRS)
board[r][c] = word[i] # restore
return found
Time: O(R · C · 4^L) worst case (L = word length). Space: O(L) recursion (no extra visited set).
The discriminators:
- In-place visited marking vs. a separate
visitedset (saves O(R·C) memory for the set per call). - Pruning — return immediately on first character mismatch.
- The Trie extension (LC 212 Word Search II): search for MANY words simultaneously via a Trie root.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/word-search/
STOP. 20-min timer.
3. Prerequisite Concepts
- Grid DFS (Phase 01 Lab 09 — tree traversal generalizes).
- The 4-direction delta convention:
DIRS = ((-1,0),(1,0),(0,-1),(0,1)). - Backtracking template (p36).
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Find ANY path in the grid spelling word, using each cell at most once. Return bool.”
Step 2 — Clarify:
- “Diagonal moves?” (No — 4-directional.)
- “Case-sensitive?” (Yes per LC.)
- “Can the same cell appear twice in the path?” (No.)
- “Grid mutation allowed?” (For in-place visited — verify with interviewer; usually yes if restored.)
- “Empty word?” (Per LC, word.length ≥ 1.)
- “What’s R, C, L?” (R, C ≤ 6 usually; L ≤ 10. Tight constraints because 4^L explodes.)
Step 3 — Constraints: R, C ≤ 6, L ≤ 15 typical → 36 starting cells × 4^15 ≈ 1B worst case, but pruning on first mismatch keeps it well under the limit in practice.
Step 4 — Examples:
[["A","B","C","E"],["S","F","C","S"],["A","D","E","E"]], "ABCCED"→ True.- Same grid,
"SEE"→ True. - Same grid,
"ABCB"→ False (can’t reuse the B).
Step 5 — Brute Force: Try every starting cell as the first letter; DFS in all 4 directions, tracking visited.
Step 6 — Complexity: O(R · C · 4^L) time. O(L) space if in-place. O(R · C + L) with separate visited set.
Step 7 — Pattern:
- “Find a path in a grid with character constraints” → grid backtracking.
- “In-place visited” is the standard memory optimization.
- “Many words at once” → Trie + LC 212.
Step 8 — Develop: see solution.py. In-place AND separate-set variants. Bonus: Trie-based LC 212 sketch.
Step 9 — Test: word starts at every corner, word loops back into used cell, word longer than grid, single-cell grid.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Grid backtracking patterns
6.1 The DFS template for grid backtracking
DIRS = ((-1, 0), (1, 0), (0, -1), (0, 1))
def dfs(r, c, i):
# Base: matched the entire word
if i == len(word): return True
# Bounds + character match (BOTH checks here — single source of truth)
if r < 0 or r >= R or c < 0 or c >= C or board[r][c] != word[i]:
return False
# Mark visited (in-place sentinel)
saved = board[r][c]
board[r][c] = '#'
# Recurse on all 4 neighbors
for dr, dc in DIRS:
if dfs(r + dr, c + dc, i + 1):
board[r][c] = saved # restore before short-circuit return
return True
board[r][c] = saved # restore on backtrack
return False
The single-source-of-truth check (bounds + character + visitedness all in one if) is the bug-resistant version. Many candidates check bounds in the caller’s loop, character in the function, and visitedness separately — three places to be wrong.
6.2 In-place marking vs separate visited set — why it matters
- Separate set: O(R·C) memory per recursion stack frame’s visited copy (if you naively copy), OR a shared set + remove on backtrack (cleaner). Either way, hashing adds constant overhead.
- In-place: O(L) recursion stack only. No extra memory. Hot in practice.
Caveat: the function must NOT throw between mark and restore (it leaves the grid corrupted). Wrap restore in try/finally if the caller’s grid is precious.
6.3 Pruning order
The early-return on bounds + character mismatch is the entire ballgame. Without it, you explore 4^L paths for every starting cell — TLE.
Some implementations also count character frequencies in the grid vs the word before any DFS — if the grid lacks even one letter of the word, return False immediately. Saves a full search on impossible inputs. Cheap O(R·C + L) preprocessing.
Even better: if word[0] appears MORE often in the grid than word[-1], reverse the word before searching. Counter-intuitive but it works because we prune on first mismatch — starting from the rarer character gives fewer wasted DFS calls.
6.4 LC 212 — Word Search II (the Trie extension)
For K words, the naive approach runs LC 79 K times: O(K · R·C · 4^L). For K=10000 this is unacceptable.
The Trie approach builds a Trie of all K words, then does ONE DFS per starting cell, following Trie edges. When the DFS reaches a Trie node marked “end-of-word,” record the word and delete the leaf (so we don’t report it twice and to prune dead branches).
Complexity: O(R · C · 4^L) where L = max word length. The constant is small because the Trie heavily prunes (4 directions × Trie branching, but most paths die quickly).
Trie pruning is a staff+ signal for this problem family.
6.5 Why we DON’T memoize
This problem looks DP-shaped (“subproblem: can we match word[i:] starting at (r, c)?”) but it’s NOT memoizable: the answer depends not only on (r, c, i) but ALSO on which cells have been visited. The visited state is exponential. So backtracking is the right tool; DP is the wrong one.
This is a classic interview “trap” — the candidate who tries to memoize wastes 10 minutes.
7. Anti-Pattern Analysis
- Separate
visitedset passed by copy. O(R·C) memory per call × 4^L calls = death. - Mark visited BEFORE the bounds/char check. Corrupts the grid on invalid recursion.
- Restore visited ONLY on the False path (forget to restore on early-True return). Leaves grid corrupted; subsequent starting-cell searches see wrong values.
- Memoize
(r, c, i). Wrong because visitedness matters. Fundamentally unsound. - Iterate starting cells but skip the early character match.
if board[r][c] == word[0]: dfs(...). The inner DFS handles it, but the outer skip saves a function call per non-matching cell. - Allocate
DIRSinside the recursion. Tiny waste but a code smell — hoist to module scope. - For LC 212, run LC 79 once per word without Trie. Times out at K > 100.
- Path string concatenation (
path + chper call) — for this problem irrelevant since we don’t track path, but a common bug elsewhere.
8. Skills & Takeaways
- Grid backtracking template: single-source-of-truth check + in-place visited + restore.
- DIRS = ((-1,0),(1,0),(0,-1),(0,1)) — module-level constant.
- Pruning is everything: bounds + char-match is the single line that prevents TLE.
- In-place sentinel (
'#') trick — saves O(R·C) memory. - Trie extension for LC 212 — multi-word search.
- DON’T memoize — visited state is part of the subproblem.
9. When to Move On
Move on when:
- You can write the LC 79 template in under 10 min.
- You can sketch the Trie extension (LC 212) in under 15 min.
- You can articulate WHY memoization is unsound here.
- Stress test on 100 random grids agrees with a brute oracle.
10. Company Context
Meta:
- LC 79 is one of the most-asked Medium problems at Meta. Bar Raiser expectation: clean code, in-place visited, articulate complexity.
- LC 212 (Trie variant) is asked for L5+ — a “warm” candidate is expected to mention Trie immediately when “many words” comes up.
- Scorecard phrase: “Algorithm — clean grid backtracking, in-place visited, articulated 4^L complexity.”
Amazon:
- L5 staple. The “in-place visited” is often the deciding code-quality signal.
- Bar Raisers test the “now do it for 10000 words” follow-up.
Microsoft:
- L63. Often pairs LC 79 with “now path-track and return the path” (Word Search variant returning the cell sequence).
Google:
- L4/L5. Often combined with “now visualize the search frontier” to test BFS vs DFS reasoning.
Apple:
- Common in algo loops; education and accessibility teams use grid-pathing in features.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Visited | Set (copy per call) | Shared set + add/remove | In-place sentinel | + explains memory trade-off |
| Pruning | At leaf | At each call | Early bounds + char check in one if | + character-frequency precheck + reverse-word heuristic |
| LC 212 | “Run LC 79 K times” | “Trie?” (vague) | Trie + DFS along Trie nodes | + leaf-deletion + dead-branch pruning |
| Memoize? | Tries to memoize | “Maybe?” | “No — visited state is exponential” | + cites: depends on the search state, not just position |
12. Level Delta
Mid (L4):
- Works with separate visited set; complexity is fuzzy.
- Handles 1×1 grid, word = “A”.
Senior (L5):
- In-place visited; restores correctly on all paths.
- Cites O(R·C · 4^L).
- Mentions Trie for the multi-word variant.
Staff (L6):
- LC 212 with Trie + leaf-deletion.
- Articulates why memoization is unsound.
- Character-frequency precheck + reverse-word pruning.
- Returns path if asked, not just boolean.
Principal (L7+):
- Discusses production: text-on-image (OCR post-processing — search a known dictionary in extracted glyph grid), game-board solvers (Boggle, Scrabble), bioinformatics (sequence alignment in 2D scaffolds).
- Discusses parallelization: shard starting cells across workers; each worker has its own grid copy.
- Discusses approximate match: edit-distance Trie traversal for fuzzy word search (production spellcheck).
13. Follow-up Q&A
Q1: Return the actual path (cells), not just bool.
A: Maintain a path list of (r, c). Append/pop in lockstep with mark/restore. On success, deep-copy and return.
Q2: LC 212 — find ALL words from a dictionary present in the grid. A: Build a Trie of all words. DFS each starting cell, walking the Trie in parallel. On end-of-word, record + delete leaf to avoid dupes and dead branches.
Q3: What if cells can be reused?
A: Then memoization works: subproblem is (r, c, i) independent of history. O(R · C · L) time, O(R · C · L) memo.
Q4: 8-directional (include diagonals)?
A: DIRS = [(dr, dc) for dr in (-1,0,1) for dc in (-1,0,1) if (dr,dc) != (0,0)]. 8^L worst case — same algorithm, larger constant.
Q5: How to scale to a 1000×1000 grid? A: Same template. Worst-case complexity unchanged, but practical pruning (character frequency, starting from rarer characters) dominates. For DICTIONARY search (LC 212), Trie is essential.
14. Solution Walkthrough
See solution.py:
exist(board, word)— in-place visited, single-source-of-truth check.exist_with_set(board, word)— separatevisitedset (less efficient but doesn’t mutate input).exist_brute(board, word)— naive exhaustive search without pruning (for stress comparison; very slow).edge_cases()+stress_test()— random grids 3×3 to 4×4 with random words.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Three things. (1) The in-place mutation is the key optimization but ALSO the dangerous one. If this function is called from multi-threaded code, you’ve created a race condition — make the contract crystal clear in the docstring (
Mutates board temporarily; not thread-safe) or accept a copy. (2) For LC 212, the leaf-deletion is non-obvious — write the test that exercises the case where the same prefix has 5 words ending under it; without leaf-deletion you’ll report duplicates AND fail to prune. (3) Consider adding amax_length_in_dictTrie-walk depth limit when the word lengths vary wildly — saves the DFS from continuing past the longest possible match at each starting cell.”
Connections forward:
- p36, p37, p38 — backtracking siblings.
- p40 — N-Queens: stateful backtracking with sets.
- LC 212 — Word Search II: Trie extension.
- LC 200 — Number of Islands: grid DFS without backtracking.
- LC 130 — Surrounded Regions: grid DFS with boundary trick.
- LC 1219 — Path with Maximum Gold: grid backtracking + value accumulation.
- Phase 03 Lab 07 — Trie Applications.
- Phase 04 Lab 02 — DFS / Connected Components.
- Production: Boggle/Scrabble solvers, OCR dictionary post-processing, game-board path validation, formal verification of small state-spaces.
Hints — p39 Word Search
Hint 1. Try every cell as the potential starting point of the word. For each start, DFS into 4 neighbors looking for word[1], then word[2], …
Hint 2. Recursion signature: dfs(r, c, i) — currently at (r, c), trying to match word[i].
Base cases:
i == len(word): SUCCESS, return True.- out of bounds OR
board[r][c] != word[i]: FAIL, return False.
Hint 3. Same cell can’t be reused. Two options:
- Separate
visited: setof(r, c)tuples — add before recurse, remove after. - In-place sentinel: temporarily overwrite
board[r][c] = '#', restore on return.
The in-place trick saves O(R·C) memory AND is the standard answer.
Hint 4. The 4-direction template:
DIRS = ((-1,0),(1,0),(0,-1),(0,1))
for dr, dc in DIRS:
if dfs(r+dr, c+dc, i+1): return True
Hint 5. Tempting to memoize on (r, c, i) — DON’T. The valid-paths depend on which cells are already used (visited state), which is exponential. Memoization is unsound here.
If still stuck: see solution.py.
p40 — N-Queens
Source: LeetCode 51 · Hard · Topics: Array, Backtracking Companies (2024–2025 frequency): Google (medium-high), Meta (medium), Amazon (medium), Microsoft (medium), Apple (low-medium) Loop position: classic Hard, often the algorithm round closer. Tests pure pattern execution — no clever insight, just disciplined backtracking with the right invariants.
1. Quick Context
Place N queens on an N×N board so no two attack each other. Return all distinct solutions.
The pattern: row-by-row placement. For each row, try each column; check that the column, the / diagonal (r+c), and the \ diagonal (r-c) are all unattacked. Recurse to next row. Backtrack on return.
def backtrack(r):
if r == n:
record(board); return
for c in range(n):
if c in cols or (r+c) in diag1 or (r-c) in diag2: continue
choose(r, c); backtrack(r+1); un_choose(r, c)
Time: bounded by O(N!) — pruning is extremely aggressive in practice. For N=8 → 92 solutions; N=14 → 365596; the classic recursion-tree visualization. Space: O(N) recursion + O(N) per attacker set.
The discriminators:
- Three attacker sets (
cols,diag = r+c,anti_diag = r-c) — O(1) check. - Row-by-row recursion — eliminates the “queens-on-same-row” check entirely (one queen per row by construction).
- Bitmask variant — three
ints replace the three sets; ~3× faster.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/n-queens/ (LC 51 — return boards) 🔗 https://leetcode.com/problems/n-queens-ii/ (LC 52 — count only)
STOP. 25-min timer.
3. Prerequisite Concepts
- Backtracking template (p36, p38).
- Set-membership invariants.
- For the bitmask variant: bitwise AND/OR/NOT,
bit & -bit(lowest set bit),bits &= bits - 1(clear lowest).
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Place N non-attacking queens on N×N; return all solutions as a list of boards.”
Step 2 — Clarify:
- “How are solutions formatted?” (LC: list of list-of-strings, each string a row, ‘.’ empty, ‘Q’ queen.)
- “Are reflections/rotations distinct?” (LC: yes, all distinct placements.)
- “Bound on N?” (LC: N ≤ 9; pure brute is fine.)
- “Just count?” (That’s LC 52.)
- “Memory budget?” (LC 51 returns all boards — fine for N ≤ 9 since solutions ≤ 352.)
Step 3 — Constraints: N ≤ 9, solutions explode at ~O(N!) — N=8 gives 92, N=9 gives 352.
Step 4 — Examples:
- N=1 →
[["Q"]]. - N=2 →
[](no solutions). - N=3 →
[]. - N=4 → 2 solutions.
Step 5 — Brute Force: Place queens in all N² choose N positions, check attacks. O(C(N², N) · N²) = death past N=4.
Step 6 — Complexity: With row-by-row + three-set pruning: bounded above by N! but practical runtime grows much slower. With bitmask: same big-O, smaller constants.
Step 7 — Pattern: Constraint-satisfaction backtracking. The three attacker invariants are the canonical example.
Step 8 — Develop: see solution.py.
Step 9 — Test: N=1, 2, 3, 4, 5, 8 (count = 92), 9 (count = 352).
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — The three attacker invariants
6.1 Why row-by-row recursion
If you place row-by-row, you never have to check the row attacker — there’s exactly one queen per row by construction. This eliminates one constraint and one set, and structures the search neatly.
6.2 The three sets
cols : set of columns already occupied
diag : set of values r+c (the '/' diagonals — constant along this direction)
anti_diag : set of values r-c (the '\' diagonals — constant along this direction)
A position (r, c) is safe iff c not in cols and (r+c) not in diag and (r-c) not in anti_diag.
Why these are correct:
- Two squares are on the same
/diagonal iffr1 + c1 == r2 + c2. Proof:/runs from bottom-left to top-right; one step along it is(r-1, c+1), preservingr+c. - Two squares are on the same
\diagonal iffr1 - c1 == r2 - c2. Proof:\runs from top-left to bottom-right; one step is(r+1, c+1), preservingr-c.
Two queens on the same / or \ attack each other. Done.
6.3 The bitmask variant — O(N) → 3 int operations
For N ≤ 32, replace each set with a single int whose bits represent occupancy. The “is c safe in row r” check becomes:
occupied = cols_mask | (diag_mask >> r) | (anti_mask << r) ... (with proper shifts)
Cleaner formulation (from Knuth):
def solve(n):
count = 0
def rec(cols, diag, anti):
nonlocal count
if cols == (1 << n) - 1:
count += 1; return
free = ~(cols | diag | anti) & ((1 << n) - 1)
while free:
bit = free & -free # lowest set bit
free ^= bit # clear it
rec(cols | bit, (diag | bit) << 1, (anti | bit) >> 1)
rec(0, 0, 0)
return count
The trick: shift diag left and anti right BETWEEN rows. The diagonals naturally propagate.
This is 2-3× faster than the set version. Fastest known classical N-Queens algorithm.
6.4 Iterative variants and why they don’t help here
For competitive search problems you sometimes convert recursion to an explicit stack. For N-Queens, recursion depth is N (≤ 9 in LC), so there’s no stack-overflow concern; iterative version offers no benefit.
6.5 Symmetry pruning
The board has reflection symmetry: any solution mirrored is also a solution. You can search only “first row column ≤ N/2” and double the count (subtracting the middle-column solutions for odd N). Cuts the search ~half.
LC 51 wants ALL distinct solutions — so apply symmetry to find canonical solutions, then expand by reflection. More code, ~2× speedup. Not usually worth it for N ≤ 9.
7. Anti-Pattern Analysis
- Check by walking the board (
for each placed queen, check row/col/diag distance). O(N) check per placement × N! placements. Slow + ugly + bug-prone vs the three-set O(1) check. - Use a 2D
n×n“attacked” matrix and mark all attacked squares on placement. O(N) marks per placement, O(N) unmarks on backtrack. Wastes memory and time vs the three-set approach. - Recurse over (row, col) BOTH (
for r, c in product(range(n), repeat=2)). Loses the “one queen per row” structure; explodes. - Off-by-one in diag/anti formulas. Use
(r+c, r-c)— NOT(c-r)or(r+c, c-r). Verify with a 4×4 by hand. - Return shared list reference. Path is shared mutable state; must snapshot (e.g.,
result.append([row[:] for row in board])OR build strings fresh). - Build board string by index manipulation each call. Cleaner: store
queens[r] = c(a list of column indices); build board strings only on success. - For LC 52 (count only), build and discard full boards. Waste — increment a counter.
- Bitmask version with wrong shift direction.
diag << 1vsdiag >> 1swaps the two diagonals; trace by hand on N=4.
8. Skills & Takeaways
- Three attacker invariants:
cols,r+c,r-c. - Row-by-row recursion structures the search and eliminates row-check.
- Bitmask variant with
bit & -bit/bits ^= bitfor hottest performance. - Board reconstruction: store
queens[r] = c, build strings only on success. - Snapshot pattern for backtracking (deep copy on record).
- Symmetry pruning for theoretical halving.
9. When to Move On
Move on when:
- You can write the three-set version in under 12 min, bug-free.
- You can sketch the bitmask version in under 15 min.
- You explain WHY
r+candr-ccharacterize the two diagonals. - N=8 gives 92 solutions (instant verification:
total_n_queens(8) == 92).
10. Company Context
Google:
- Common in L4/L5 algorithm rounds. The interviewer expects the three-set version cleanly; bonus for bitmask.
- Scorecard phrase: “Algorithm — clean backtracking with the right invariants; articulated diagonal characterization.”
Meta:
- Less frequent than LC 79, but appears for L5 senior loops. Often paired with LC 52 follow-up.
Amazon:
- L5+. The bar raiser test: “now write LC 52 efficiently” → expects bitmask.
Microsoft:
- L63/L64. Sometimes asked as a system-design-adjacent: “extend to a rectangular board” (no formula change for rows × cols — same three sets).
Apple:
- Less common; appears in algo-research-heavy teams.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Attacker check | Walk all queens | 2D attacked matrix | Three sets (cols, r+c, r-c) | + bitmask |
| Recursion structure | Over (r, c) | Row-by-row + col loop | + understands why no row-check | + symmetry pruning |
| LC 52 (count) | Build boards, count | Count via flag | Count without building | + bitmask count |
| Diagonal proof | Trial and error | Verbal hand-wave | r+c invariant proved | + connection to dancing links / exact cover |
12. Level Delta
Mid (L4):
- Three-set version works.
- Builds boards correctly with snapshot.
- Hand-verifies N=4 = 2 solutions.
Senior (L5):
- Articulates the three-set invariants cleanly.
- Stores
queens[r] = cand constructs boards once. - LC 52 with counter (no board build).
Staff (L6):
- Bitmask version.
- Discusses symmetry pruning.
- Connects to constraint satisfaction (SAT) and exact cover (dancing links).
Principal (L7+):
- Discusses production: schedule/seating optimization, FPGA placement, classroom assignment — N-Queens as a constraint-satisfaction archetype.
- Discusses parallel search: sharding first-row choices across workers.
- Discusses DLX (Dancing Links / Algorithm X) for solving general exact-cover problems including N-Queens.
- Discusses probabilistic / Las Vegas algorithms for large-N: Min-Conflicts heuristic solves N=1,000,000 in seconds (with restarts), whereas backtracking falls over past N=30.
13. Follow-up Q&A
Q1: LC 52 — just count.
A: Same recursion, replace result.append(board) with count += 1. Don’t build boards. Bitmask version is fastest.
Q2: Why r+c and r-c?
A: A / diagonal (bottom-left to top-right) has constant r+c (one step is (-1, +1)). A \ diagonal (top-left to bottom-right) has constant r-c (one step is (+1, +1)). Two queens attack along a diagonal iff they share r+c or r-c.
Q3: Solve N=1,000,000? A: Backtracking dies past N≈30. Use Min-Conflicts: random initial placement (one queen per row), repeatedly move the most-conflicted queen to its least-conflicting column, restart on local minimum. Solves N=10^6 in seconds. Las Vegas (random restarts), not deterministic.
Q4: Generalize to K-attackers (king/rook/bishop mix)? A: Replace the attacker check with a generalized “any piece attacks this square?” Each piece type contributes its own set/mask; otherwise structure is identical.
Q5: Track only “fundamentally distinct” solutions (modulo D₄ symmetry)? A: Generate all solutions, canonicalize each (e.g., apply all 8 symmetries and take min), dedupe. For N=8: 92 → 12 fundamental solutions.
14. Solution Walkthrough
See solution.py:
solve_n_queens(n)— three-set backtracking; returns boards.total_n_queens(n)— LC 52 count-only with three sets.solve_n_queens_bitmask(n)— bitmask version; returns count (fastest).solve_n_queens_brute(n)— exhaustive C(N², N) for tiny N (oracle).
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Three things. (1) The bitmask version is gorgeous but make sure your reader understands the shift direction — leave a worked 4×4 trace in the comments because future-you will not remember why
diag << 1vs>>. (2) For LC 52 we’re calling this fresh each time; consider memoizing on (row, cols_mask, diag_mask, anti_mask) — BUT note that anti/diag masks vary per row, so the key space is huge and the cache won’t pay off. The classical result is that N-Queens has no known good DP because the state-space lacks reusable subproblems; this is a teachable moment in the docstring. (3) For production solver applications (seating, scheduling), don’t write a custom backtracker — use an SMT solver (Z3) or a CSP library (OR-Tools). Hand-rolled backtracking is the algorithmic exercise; mature CSP solvers crush it on real workloads with conflict-driven clause learning, restart heuristics, and parallel search.”
Connections forward:
- p36, p37, p38, p39 — backtracking siblings.
- Constraint satisfaction (CSP) — N-Queens is the textbook example.
- SAT solvers — N-Queens encodable in propositional logic.
- Dancing Links (Knuth’s Algorithm X) — exact cover formulation.
- Min-Conflicts heuristic — local-search alternative.
- Production: classroom/exam seating, FPGA placement, frequency assignment, scheduling.
Hints — p40 N-Queens
Hint 1. Place queens row by row: row 0 first, then row 1, etc. This guarantees exactly one queen per row by construction — no row-attacker check needed.
Hint 2. For column safety: maintain a set cols of occupied columns. Before placing at (r, c), check c not in cols. After backtrack, remove.
Hint 3. For diagonal safety, two key invariants:
- Squares on the same
/diagonal share the samer + c. - Squares on the same
\diagonal share the samer - c.
So maintain two more sets: diag (values r + c) and anti (values r - c).
Hint 4. Skeleton:
def backtrack(r):
if r == n:
record solution; return
for c in range(n):
if c in cols or r+c in diag or r-c in anti: continue
place; backtrack(r+1); unplace
Store queens[r] = c (a list of column indices). Build the board strings ONLY on success.
Hint 5. Bitmask optimization (Knuth): replace the three sets with three ints. Shift diag << 1 and anti >> 1 between recursion levels so diagonal attacks propagate naturally. bit & -bit extracts the lowest set bit; free ^= bit clears it. 2–3× faster.
If still stuck: see solution.py.
Week 09 — Heap / Top-K: Streaming Order Statistics
Month 2 (Patterns) · Week 4 of 4 · Difficulty: Medium → Hard
Theme
The heap is the algorithm world’s most underrated data structure. Whenever you hear:
- “find the K-th largest/smallest”
- “top K frequent / closest / cheapest”
- “running median”
- “merge K sorted streams”
- “schedule the next-due / highest-priority task”
…the answer is almost always a heap (often plus a hash map or a second heap).
This week drills the five canonical heap patterns:
- Quickselect vs heap-of-size-K for “K-th element” (p41).
- Bucket sort vs heap-of-size-K for “top K by count” (p42).
- Two-heaps (max-heap of lower half + min-heap of upper half) for running median (p43).
- Heap-over-K-heads for merging K sorted lists (p44).
- Greedy heap scheduling for “no two adjacent equal” (p45).
Why these specific five
Each forces a different heap technique:
| Problem | Pattern | Discriminator |
|---|---|---|
| p41 Kth Largest | Heap of size K OR Quickselect | Two valid answers; senior names both, picks heap for production, quickselect for interview “do better than O(n log n)” |
| p42 Top K Frequent | Heap-of-size-K OR Bucket sort O(n) | Bucket sort surprise-beats heap when count ≤ n |
| p43 Running Median | Two-heap dance | The invariant: |lo| - |hi| ∈ {0, 1}; rebalance after every insert |
| p44 Merge K Lists | Heap over K head pointers | The classic. O(N log K) vs O(NK) naive |
| p45 Reorganize String | Greedy heap (counts, restore after using) | Counter-intuitive: max-heap by count, “use top + stash” |
Problem table
| # | Problem | LC | Difficulty | Core technique |
|---|---|---|---|---|
| p41 | Kth Largest Element in an Array | 215 | Medium | Heap of size K · Quickselect |
| p42 | Top K Frequent Elements | 347 | Medium | Counter + heap OR bucket sort |
| p43 | Find Median from Data Stream | 295 | Hard | Two heaps + rebalance invariant |
| p44 | Merge K Sorted Lists | 23 | Hard | Heap over K heads |
| p45 | Reorganize String | 767 | Medium | Greedy max-heap + delay slot |
Daily Schedule (suggested 5-day pace)
| Day | Problem | Deliverable |
|---|---|---|
| Mon | p41 | Heap-of-size-K + Quickselect; understand expected O(n) |
| Tue | p42 | Bucket sort + heap variants; pick winner for given input shape |
| Wed | p43 | Two-heap invariant; trace 6-element insert sequence by hand |
| Thu | p44 | Heap of nodes; understand why we push only one-per-list at a time |
| Fri | p45 | Greedy with delayed re-insertion; prove correctness |
Readiness Gate
You may proceed to Phase 3 when ALL hold:
-
All 5 stress tests PASS (
python3 solution.pyper folder). - You can write each problem’s canonical solution in ≤ 12 min, bug-free.
- You can articulate WHY a heap is the right structure (vs sorted array / BST / sort) for each.
- You can explain the running-median two-heap invariant without notes.
- You can write Quickselect from scratch and explain its expected O(n).
Key Mental Models Established This Week
heapqis a min-heap. For max-heap, negate values.- Heap of size K is O(n log K) — beats sort O(n log n) when K << n.
- Quickselect is expected O(n), worst O(n²) — randomize pivot.
- Two heaps: max-heap stores SMALLER half (with negation), min-heap stores LARGER half.
- Tie-breaking in heaps: push
(key, counter, item)to avoid comparing items. - Bucket sort beats heap when the value domain is small (e.g., counts ∈ [1, n]).
- Lazy deletion: for heaps that need “remove arbitrary,” push tombstones, skip on pop.
p41 — Kth Largest Element in an Array
Source: LeetCode 215 · Medium · Topics: Array, Heap, Divide & Conquer, Quickselect, Sorting Companies (2024–2025 frequency): Meta (very high), Amazon (very high), Google (high), Microsoft (high), Apple (medium-high), Bloomberg (high) Loop position: Phone screen or first algo round. The “two acceptable answers” problem — interviewers expect you to NAME both and pick deliberately.
1. Quick Context
Return the K-th largest element in nums. Note: K-th largest by rank, not K-th distinct value.
Three acceptable solutions:
| Approach | Time | Space | When to pick |
|---|---|---|---|
| Sort + index | O(n log n) | O(1) | Tiny n, lazy answer; mention as baseline |
| Min-heap of size K | O(n log K) | O(K) | Production — streaming-friendly, predictable |
| Quickselect | O(n) expected, O(n²) worst | O(1) avg | “Can you do better than O(n log n)?” → yes, expected linear |
The discriminator is the named trade-off: candidates who give only one answer fail the signal; candidates who name all three and explain when each wins clear the bar.
Streaming variant: for “K-th largest in a STREAM” (LC 703), the heap-of-size-K is the only viable approach — quickselect needs the full array.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/kth-largest-element-in-an-array/
STOP. 20-min timer. Plan both heap AND quickselect before coding.
3. Prerequisite Concepts
heapqmodule:heappush,heappop,heapify,heappushpop,nlargest/nsmallest.- Partition (Lomuto or Hoare) for quickselect.
- The min-heap of size K trick: keep the K LARGEST seen so far in a MIN-heap so the smallest of the K is at the root.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given nums and k, return the element that would be at index n - k if nums were sorted ascending. Equivalently, K-th largest by rank (duplicates count separately).”
Step 2 — Clarify:
- “K-th distinct, or K-th by rank?” (By rank —
[3,3,3], k=2returns 3.) - “1 ≤ K ≤ n always?” (Yes per LC; defensive check otherwise.)
- “Negatives allowed?” (Yes.)
- “Can I sort in place?” (Mutates input — ASK; quickselect mutates, heap does not.)
- “n bound?” (LC: n ≤ 10^5; both heap and quickselect comfortable.)
- “Stream or batch?” (Batch per LC; if stream → heap only.)
Step 3 — Constraints: n ≤ 10^5, k ≤ n. Sort O(n log n) ≈ 10^5 · 17 ≈ 2M ops — passes but suboptimal.
Step 4 — Examples:
[3,2,1,5,6,4], k=2→ 5[3,2,3,1,2,4,5,5,6], k=4→ 4[1], k=1→ 1[7,7,7], k=2→ 7
Step 5 — Brute Force: Sort, return nums[-k]. O(n log n).
Step 6 — Complexity: Heap O(n log K); Quickselect O(n) expected, O(n²) worst.
Step 7 — Pattern: “Top K / K-th element” → heap-of-size-K is the production default; quickselect is the interview “linear-time” payoff.
Step 8 — Develop: see solution.py.
Step 9 — Test: k=1 (max), k=n (min), all duplicates, single element, negatives.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Heap, Quickselect, and the streaming case
6.1 Min-heap of size K — the most underrated trick
To find the K largest, maintain a MIN-heap of the K largest seen so far. The root is the smallest-of-the-K — i.e., the current candidate for K-th largest. For each new element:
- If heap has fewer than K: push.
- Else if element > heap[0]:
heappushpop(heap, element)(atomic push-then-pop).
At end, heap[0] is the K-th largest.
import heapq
heap = []
for x in nums:
if len(heap) < k:
heapq.heappush(heap, x)
elif x > heap[0]:
heapq.heappushpop(heap, x)
return heap[0]
Why this is brilliant: never holds more than K elements → O(K) memory regardless of n. Stream-friendly.
Counter-intuitive part: to find the K LARGEST, use a MIN-heap. (The min-of-the-largest-K is the K-th largest.)
6.2 Quickselect — expected O(n)
Variant of quicksort that recurses into only ONE side.
def quickselect(arr, k): # 0-indexed: returns the element that would be at sorted index k
if len(arr) == 1: return arr[0]
pivot = random.choice(arr)
lows = [x for x in arr if x < pivot]
highs = [x for x in arr if x > pivot]
pivots = [x for x in arr if x == pivot]
if k < len(lows): return quickselect(lows, k)
elif k < len(lows) + len(pivots): return pivot
else: return quickselect(highs, k - len(lows) - len(pivots))
Expected runtime: each level discards a constant fraction → T(n) = T(n/2) + O(n) = O(n).
Worst case O(n²) if pivot is always extremal — defeated by random pivot. NEVER use first/last element as pivot in interview code without acknowledging the adversarial input risk.
In-place version (Lomuto partition) avoids the list-comprehension allocation; better cache locality, but harder to write under pressure. Above 3-way split (lows/equals/highs) handles duplicates cleanly — important for [3,3,3,3,3] where Lomuto degrades.
6.3 Why heap is the production default
| Concern | Heap | Quickselect |
|---|---|---|
| Worst case | O(n log K) deterministic | O(n²) — DoS risk on adversarial input |
| Predictable latency | Yes | No |
| Streaming | Yes — process one at a time | No — needs full array |
| Mutates input | No | Yes (or copy) |
| Code complexity | 5 lines | 15+ lines |
In a real system serving requests, the O(n²) tail risk of quickselect is unacceptable. Heap wins production. Quickselect wins the “do better than O(n log n)” interview question only.
6.4 LC 703 — Kth Largest in a Stream
The streaming variant: a class that supports add(val) and returns the K-th largest so far. Heap-of-size-K is the canonical answer:
class KthLargest:
def __init__(self, k, nums):
self.k = k
self.heap = []
for x in nums: self.add(x)
def add(self, val):
if len(self.heap) < self.k: heapq.heappush(self.heap, val)
elif val > self.heap[0]: heapq.heappushpop(self.heap, val)
return self.heap[0]
Quickselect cannot do this. This is why heap-of-size-K is the “production” answer.
6.5 Median of medians (BFPRT) — deterministic O(n)
Worst-case O(n) selection algorithm. Beautiful theory; slow in practice due to large constants. Mention only if asked “deterministic linear-time selection?” — answer: BFPRT. Don’t implement under pressure.
7. Anti-Pattern Analysis
- Sort + index as your only answer — fails the “two acceptable answers” signal. Always mention heap and quickselect.
- MAX-heap of size n, pop k times — O(n + k log n). For k near n this is fine, but for k=5, n=10^6 it’s worse than min-heap-of-size-K. Wrong reflex.
- Quickselect with
nums[0]ornums[-1]pivot — adversarial inputs degrade to O(n²). ALWAYS randomize. - Quickselect with Lomuto partition on
[3,3,3,…,3]— degrades to O(n²). Use 3-way (Dutch flag) split OR random pivot. nlargest(k, nums)[-1]— works (under the hood uses heap-of-size-K), but in an interview you must show you understand it, not just call the library.sorted(nums)[-k]— fine as baseline; bad as primary answer.- Mutating the input without warning — quickselect mutates; if the caller cares, copy first.
- Forgetting
heappushpopis atomic —heappushthenheappopis two O(log K) ops instead of one. Marginal but a code-quality signal. - Returning the heap, not
heap[0]— off-by-one bug under pressure. - K-th
distinct— different problem. Verify with interviewer.
8. Skills & Takeaways
- Min-heap of size K for top-K (counter-intuitive: MIN for LARGEST).
heapq.heappushpop— atomic push-then-pop, single O(log K).- Quickselect with random pivot + 3-way split.
- Heap = production, Quickselect = “do better than O(n log n)”.
- Streaming variant (LC 703) — heap only.
- Median of medians (BFPRT) as worst-case linear deterministic backup answer.
- Negate values for max-heap in Python (
heapqis min-heap).
9. When to Move On
Move on when:
- You can write heap solution in ≤ 5 min.
- You can write quickselect with random pivot in ≤ 10 min.
- You can articulate the production-vs-interview trade-off without prompting.
- You handle
[3,3,3], k=2→ 3 correctly (K-th by rank, not distinct). - Stress test on 1000 random arrays agrees with
sorted(nums)[-k].
10. Company Context
Meta:
- One of the top-5 most-asked problems at Meta. Bar Raiser expects BOTH solutions named, heap implemented, quickselect sketched.
- Scorecard phrase: “Algorithm — named two valid approaches, chose deliberately, articulated worst-case trade-offs.”
- Common follow-up: “now make it streaming” → LC 703.
Amazon:
- L5+ staple. Bar Raiser tests the “K-th in a 1TB file you can’t fit in memory” external-sort variant.
- Expects discussion of partitioning across workers.
Google:
- L4/L5. Often paired with “implement quickselect” specifically. Random pivot is a hard requirement.
Microsoft:
- L63/L64. Often phrased as “find the K closest stars to Earth from a list” (variant — same template).
Apple:
- Common in iOS performance loops (e.g., “K largest battery drainers”). Streaming variant frequent.
Bloomberg:
- Loves this problem; usually paired with “now do it over a sliding window of size W” (heap + lazy deletion).
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First answer | sorted(nums)[-k] | Heap | Heap + names quickselect | + names BFPRT |
| Heap orientation | Min for smallest | Min-heap of size K for K-th largest | + explains why MIN heap for LARGEST | + streaming variant |
| Quickselect pivot | First element | Last element | Random | + 3-way for duplicates |
| Streaming follow-up | “Re-sort each time” | “Maintain sorted list” | “Heap of size K” | + lazy deletion for sliding window |
| Big-O on duplicates | “O(n log n)” | “O(n log K)” | + “quickselect degrades on duplicates without 3-way” | + median-of-medians worst case |
12. Level Delta
Mid (L4):
- Heap solution clean.
- Sketches quickselect; may have pivot bug.
- States O(n log K).
Senior (L5):
- Names heap + quickselect + sort; picks heap for production with justification.
- Quickselect with random pivot, 3-way split.
- Handles streaming follow-up (LC 703).
Staff (L6):
- Discusses median-of-medians for deterministic O(n).
- Discusses sliding-window variant with lazy-deletion heap.
- Discusses external sort for arrays > memory.
Principal (L7+):
- Discusses production: percentile dashboards (P95/P99 latency), recommendation top-K, search ranking.
- Discusses approximate top-K: Space-Saving (Misra-Gries) algorithm, Count-Min Sketch for unbounded streams with bounded memory.
- Discusses parallelization: per-shard top-K, merge with K · S-heap (S shards).
- Discusses adversarial defense: random pivot prevents algorithmic DoS via crafted input.
13. Follow-up Q&A
Q1: K-th SMALLEST?
A: Mirror everything. Max-heap of size K (heapq with negation), or quickselect targeting k-1 (0-indexed).
Q2: Streaming (LC 703)? A: Heap-of-size-K only; quickselect needs full array. See class skeleton in §6.4.
Q3: K-th distinct? A: Different problem. Dedupe first (set), then K-th over distinct values. Verify with interviewer.
Q4: K-th over sliding window of size W?
A: Heap of size W with lazy deletion: on slide-out, push tombstone (value, "removed", index); on access, pop tombstones from root until valid. Or: use a sorted container (e.g., SortedList from sortedcontainers) with O(log W) add/remove.
Q5: Arrays too large for memory (e.g., 1TB)? A: External sort + skip to position. OR: per-chunk top-K (heap of size K per chunk), merge with K·M-heap (M = #chunks). Total O(N log K) work, O(M·K) merge memory.
Q6: Find ALL elements ranked between K1 and K2? A: Two quickselects (for K1 and K2 boundaries), then linear pass for elements in range. Or: nth_element twice (C++ STL).
14. Solution Walkthrough
See solution.py:
find_kth_largest_heap(nums, k)— min-heap of size K.find_kth_largest_quickselect(nums, k)— 3-way split with random pivot.find_kth_largest_sort(nums, k)— baseline.find_kth_largest_nlargest(nums, k)—heapq.nlargest(library shortcut).KthLargest— LC 703 streaming class.edge_cases()+stress_test()— 1000 random arrays, all four agree.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Three things. (1) The heap version is the right default for production — but make the size-K invariant explicit in a comment because future-readers WILL try to optimize by removing the
len(heap) < kbranch and break it. (2) For quickselect, the random pivot is non-negotiable; add a property-test that runs against sorted, reverse-sorted, all-duplicates, and a known adversarial input — those are the cases where pivot choice matters. (3) For the streaming variant, consider whether you need APPROXIMATE top-K (e.g., dashboard top-K queries over billions of events): the Space-Saving algorithm gives bounded-memory approximate top-K with provable error bounds; exact heap-of-size-K doesn’t scale beyond what fits in memory. The interview question hides this scaling cliff — call it out in the design doc.”
Connections forward:
- p42 — Top K Frequent (heap + count map).
- p43 — Running median (two heaps).
- p44 — Merge K sorted lists (heap of heads).
- LC 703 — Kth Largest in a Stream.
- LC 973 — K Closest Points to Origin (same template).
- LC 692 — Top K Frequent Words (heap with tie-break).
- Phase 02 Lab 09 — Heap Top-K (lab generalization).
- Production: percentile dashboards (P50/P95/P99), recommendation top-K, search ranking, hot-key detection.
Hints — p41 Kth Largest Element
Hint 1. There are at least THREE acceptable solutions: sort (O(n log n)), heap (O(n log K)), quickselect (O(n) expected). Name all three; pick deliberately. Don’t just say “sort it.”
Hint 2. Heap approach: to find the K LARGEST, use a MIN-heap of size K. The root is the smallest of the K largest seen so far — i.e., the current candidate for K-th largest.
heap = []
for x in nums:
if len(heap) < k: heappush(heap, x)
elif x > heap[0]: heappushpop(heap, x)
return heap[0]
heappushpop is one atomic O(log K) op (faster than push then pop).
Hint 3. Quickselect: recurse into ONE side of a partition. Target index n - k (0-indexed in ascending order).
pivot = random.choice(arr) # RANDOMIZE
lows = [x < pivot]; eqs = [x == pivot]; his = [x > pivot]
if target < len(lows): recurse(lows, target)
elif target < len(lows)+len(eqs): return pivot
else: recurse(his, target - len(lows) - len(eqs))
The 3-way split is essential — handles duplicates without quadratic blowup.
Hint 4. Random pivot is non-negotiable. nums[0] or nums[-1] as pivot degrades to O(n²) on sorted/reverse-sorted input. Adversarial DoS risk in production.
Hint 5. Streaming follow-up (LC 703): heap-of-size-K is the ONLY answer — quickselect needs the full array. Class supports add(val) returning current K-th largest.
If still stuck: see solution.py.
p42 — Top K Frequent Elements
Source: LeetCode 347 · Medium · Topics: Array, Hash Table, Heap, Bucket Sort, Quickselect Companies (2024–2025 frequency): Meta (very high), Amazon (very high), Google (high), Microsoft (high), Apple (medium), Uber (high), LinkedIn (high) Loop position: Phone screen and first round. The “bucket sort is faster” curveball — most candidates default to heap, the senior signal is recognizing bucket sort.
1. Quick Context
Given nums, return the k most frequent elements.
Three acceptable solutions:
| Approach | Time | Space | When to pick |
|---|---|---|---|
| Counter + sort by count | O(n log n) | O(n) | Baseline; never primary |
| Counter + min-heap of size K | O(n log K) | O(n + K) | Production default |
| Counter + bucket sort | O(n) | O(n) | Optimal — beats heap when count ≤ n |
The discriminator is recognizing bucket sort applies: counts are bounded by n (an element can appear at most n times), so a bucket array of size n+1 indexed by frequency lets you sweep top-K in linear time. Most candidates miss this.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/top-k-frequent-elements/
STOP. 20-min timer. Aim to derive bucket sort.
3. Prerequisite Concepts
collections.Counterfor O(n) frequency counting.heapqand min-heap-of-size-K (p41).- The bucket-sort insight: when values are bounded by n, sort by indexing.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Count each element’s frequency; return K elements with highest counts. Order among the K is unspecified.”
Step 2 — Clarify:
- “Ties — which to break in favor of?” (LC: any; if asked, specify lex / first-seen / etc.)
- “k ≤ #distinct?” (LC: guaranteed.)
- “n bound?” (LC: 10^5; both heap and bucket comfortable.)
- “Are elements hashable?” (Yes — ints.)
- “Return order matters?” (LC: no.)
Step 3 — Constraints: n ≤ 10^5 → counts ≤ 10^5. Any of the three approaches comfortably passes.
Step 4 — Examples:
[1,1,1,2,2,3], k=2→[1, 2][1], k=1→[1][4,4,4,4,4], k=1→[4]
Step 5 — Brute Force: Counter → list of (count, val) → sort by count → take top K. O(n log n).
Step 6 — Complexity: Heap O(n log K); Bucket O(n).
Step 7 — Pattern: “Top K by frequency” → bucket sort wins when count is bounded by n; heap is general.
Step 8 — Develop: see solution.py.
Step 9 — Test: k=1, k=#distinct (all elements), all same, all distinct (every count = 1).
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Bucket sort is the surprise
6.1 The bucket-sort insight
Element counts are bounded: 1 ≤ count ≤ n. So you can create a buckets array of size n + 1 where buckets[c] is the list of elements with frequency c. Sweep c from n down to 1, collecting elements until you have K.
counter = Counter(nums)
buckets = [[] for _ in range(len(nums) + 1)]
for val, cnt in counter.items():
buckets[cnt].append(val)
result = []
for c in range(len(buckets) - 1, 0, -1):
for val in buckets[c]:
result.append(val)
if len(result) == k: return result
Time: O(n) for Counter + O(n) for bucket fill + O(n) for sweep = O(n) total.
This is the optimal answer. It beats heap O(n log K) asymptotically. Why isn’t it always used? Memory: bucket array is size n + 1 regardless of K, while heap is size K. For huge n and tiny K, the heap can be more cache-friendly.
6.2 Heap solution — when bucket sort doesn’t apply
For STREAMING input where you don’t know n in advance, bucket sort fails — you can’t pre-size the bucket array. Heap-of-size-K is the streaming-friendly answer.
counter = Counter(nums)
heap = []
for val, cnt in counter.items():
if len(heap) < k:
heapq.heappush(heap, (cnt, val))
elif cnt > heap[0][0]:
heapq.heappushpop(heap, (cnt, val))
return [val for cnt, val in heap]
Tuple (cnt, val) — min-heap orders by count (first tuple element). On ties, val breaks the tie (deterministic).
6.3 Quickselect on (val, count) pairs — O(n) expected
Use quickselect to find the K largest by count. Same algorithm as p41 but the comparison key is the count, not the value itself.
items = list(counter.items()) # [(val, count), ...]
# Quickselect on items, comparing by count, targeting position len(items) - k
quickselect(items, len(items) - k, key=lambda x: x[1])
return [val for val, _ in items[len(items) - k:]]
O(n) expected; same trade-offs as p41.
6.4 Why ties don’t matter (usually)
LC accepts any K elements with the top-K counts. If two elements tie for K-th place, either is acceptable. BUT: the production version often requires deterministic tie-breaking (lex order, first-seen, etc.) — confirm with the interviewer.
For lex tie-break: heap key becomes (cnt, val) directly works (min-heap → smallest cnt-then-val at root). For “first-seen” tie-break: include an insertion counter in the key.
6.5 LC 692 — Top K Frequent Words (tie-break variant)
Same problem with deterministic tie-break: “if two words have same frequency, the lexicographically smaller comes first.” This makes heap key tricky: we want (LARGER count, SMALLER word) to be at the heap root, but heapq is min-heap.
Trick: invert just the count: heapq.heappush(heap, (-cnt, word)). Then a min-heap pops smallest (-cnt, word) = largest cnt, lex-smallest word on ties. After collecting top K (with size-K heap), pop everything and reverse.
This is the kind of detail that distinguishes a senior from a mid-level candidate.
6.6 Counter is a hash table (and that matters)
collections.Counter is a subclass of dict. Frequency counting is O(n) expected but O(n²) worst case under adversarial hash collisions (theoretical only — Python’s hash is randomized per-process by default). For arbitrary objects (not just ints/strings), ensure __hash__ is well-defined.
7. Anti-Pattern Analysis
- Sort + slice as your only answer — O(n log n), misses the bucket-sort O(n) signal.
- Heap of size n (push all, pop K) — O(n log n). Wrong reflex: heap-of-size-K is O(n log K).
- Push counts, not (count, val) tuples — you lose track of which value the count refers to.
- Bucket sort with
defaultdict(list)instead of array — slower in practice for dense bucket fill; preallocated list is the right choice. - Bucket sort with size
max(counter.values()) + 1— works but requires a pre-scan;len(nums) + 1is upper bound and avoids the scan. Tiny win for clarity. - Forgetting LC 692 tie-break — push
(-cnt, word)not(cnt, word)for size-K heap with lex tie-break. - Mutating
counter.items()during iteration — fortunately we don’t, but a common bug elsewhere. - Returning a heap, not its values — extract with list comprehension.
- Not using
Counter— manualdefaultdict(int)loop works butCounteris idiomatic.
8. Skills & Takeaways
- Bucket sort O(n) when count is bounded by n.
- Min-heap of size K with
(count, val)tuples for streaming-friendly top-K. - Quickselect on items by count key for O(n) expected.
- LC 692 trick:
(-cnt, word)for deterministic lex tie-break in size-K min-heap. Counteris the idiomatic frequency-counter; subclass ofdict.- Three valid answers, deliberate pick: bucket (optimal), heap (streaming), quickselect (interview flair).
9. When to Move On
Move on when:
- You can write bucket-sort version in ≤ 7 min.
- You can write heap version in ≤ 5 min.
- You can articulate why bucket sort is O(n).
- You handle LC 692 tie-break correctly.
- Stress test on 500 random arrays agrees with sort baseline.
10. Company Context
Meta:
- Phone-screen staple. The bucket-sort recognition is the senior signal.
- Bar Raiser asks: “what’s the time complexity? Can you do better than O(n log K)?” — expects bucket sort.
- Scorecard phrase: “Algorithm — derived O(n) bucket sort; named heap and quickselect alternatives.”
Amazon:
- L5/L6 staple. Often combined with “top K customers by purchase count” framing.
- Bar Raiser tests “top K over a 1B-row table” → discusses partitioning + per-shard top-K.
Google:
- L4/L5. Often follows up with LC 692 (words + lex tie-break) for senior signal.
Microsoft:
- L63/L64. Common in search/ranking team interviews.
Uber/LinkedIn:
- “Top K trending hashtags” / “top K skills” framing. Streaming variant common — heap-of-size-K + sliding window.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First answer | Sort by count | Heap of size K | Bucket sort O(n) | + names quickselect + streaming caveat |
| Counter | Manual dict loop | Counter(nums) | + explains it’s a dict subclass | + adversarial hashing caveat |
| Tie-break | “Doesn’t matter” | “Use tuple key” | “(-cnt, val) for lex order” | + deterministic-output discussion |
| Streaming | “Re-sort each time” | “Heap-of-size-K” | + “bucket sort needs known n” | + sliding-window Misra-Gries / Space-Saving |
| LC 692 | Brute lex sort | Tuple key (basic) | (-cnt, word) for size-K min-heap, reverse at end | + count-min sketch for approximate large-stream variant |
12. Level Delta
Mid (L4):
- Heap solution clean with
(cnt, val)tuples. - States O(n log K).
- May miss bucket sort.
Senior (L5):
- Bucket sort first; names heap as streaming alternative.
- Handles LC 692 tie-break correctly.
- Mentions Counter internals.
Staff (L6):
- Discusses production: hot-key detection, trending topics, real-time leaderboard.
- Discusses approximate streaming algorithms (Misra-Gries, Space-Saving).
- Discusses sharded top-K for distributed systems.
Principal (L7+):
- Discusses count-min sketch + heap for bounded-memory streaming top-K with provable error.
- Discusses sliding-window top-K with lazy deletion.
- Discusses adversarial input: hash collisions, count overflow, memory exhaustion.
- Discusses A/B testing: how top-K rankings drift over time and the need for stable tie-break.
13. Follow-up Q&A
Q1: LC 692 — Top K Frequent Words with lex tie-break.
A: Use heapq with key (-cnt, word). Min-heap pops smallest (-cnt, word) = largest cnt, lex-smallest on ties. After collecting K, pop all and reverse to get descending order.
Q2: Streaming over an unbounded input? A: Heap-of-size-K is feasible only if #distinct values fits in memory. For unbounded distinct values, use Space-Saving algorithm (bounded-memory approximate top-K with provable error bounds).
Q3: Top K over a sliding window of W? A: Maintain count map per window position + heap with lazy deletion. Or: bucket of “active counts” per element.
Q4: K = n (return ALL elements by frequency)? A: Sort by count: O(n log n). No win from heap or bucket — must touch everything anyway.
Q5: Top K by SUM-of-values, not count?
A: Same template; replace Counter with grouped sum, then bucket / heap / quickselect on sums.
Q6: Top K across multiple shards? A: Per-shard top-K + merge with K · S-heap (S shards). Total O((K · S) log (K · S)).
14. Solution Walkthrough
See solution.py:
top_k_frequent_bucket(nums, k)— O(n) bucket sort.top_k_frequent_heap(nums, k)— heap of size K with(cnt, val)tuples.top_k_frequent_sort(nums, k)— baseline.top_k_frequent_words(words, k)— LC 692 with lex tie-break.edge_cases()+stress_test()— 500 random arrays; compare as SETS (order unspecified).
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Three things. (1) The bucket sort is asymptotically optimal but memory is O(n) regardless of K — for a service processing top-1 over n=10^9, this matters; use the heap. State the trade-off in the docstring. (2) LC 692’s
(-cnt, word)trick is the kind of code that breaks two years later when someone ‘simplifies’ it; add a comment with WHY the negation is there and a test that fails if you remove it. (3) For real top-K systems (trending hashtags, recommendation candidates), you’re almost never doing exact top-K over batched data — you’re doing approximate top-K over a stream with bounded memory. Mention Space-Saving / Misra-Gries / count-min sketch in the design doc; the interview question is the toy version of a much harder real problem.”
Connections forward:
- p41 — Kth Largest Element (sibling pattern).
- p44 — Merge K Sorted Lists (heap-over-K).
- LC 692 — Top K Frequent Words (lex tie-break).
- LC 973 — K Closest Points (heap with distance key).
- LC 451 — Sort Characters By Frequency (bucket sort revisited).
- Phase 02 Lab 09 — Heap Top-K (lab generalization).
- Production: trending hashtags, hot-key detection, recommendation candidates, search-suggestion ranking.
Hints — p42 Top K Frequent Elements
Hint 1. Frequency-count first: counter = Counter(nums) → {val: count}. From here, the question becomes “top K by count value.”
Hint 2. Heap version: min-heap of size K with (count, val) tuples. On overflow, heappushpop the smaller. At end, extract values.
heap = []
for val, cnt in counter.items():
if len(heap) < k: heappush(heap, (cnt, val))
elif cnt > heap[0][0]: heappushpop(heap, (cnt, val))
return [val for cnt, val in heap]
O(n log K).
Hint 3. Can we beat O(n log K)? YES — counts are bounded: 1 ≤ count ≤ n. So make a buckets array of size n + 1 indexed by frequency. Sweep from high to low collecting elements until you have K.
Hint 4. Bucket sort skeleton:
buckets = [[] for _ in range(len(nums) + 1)]
for val, cnt in counter.items(): buckets[cnt].append(val)
result = []
for c in range(len(buckets) - 1, 0, -1):
for val in buckets[c]:
result.append(val)
if len(result) == k: return result
O(n) — total work proportional to #distinct + sweep.
Hint 5. LC 692 tie-break (lex order): use composite key (-count, word) so smaller-count or lex-larger word “loses”. Direct sort with this key, or push (-cnt, word) into a size-K min-heap then carefully reverse.
If still stuck: see solution.py.
p43 — Find Median from Data Stream
Source: LeetCode 295 · Hard · Topics: Heap, Design, Two Pointers Companies (2024–2025 frequency): Google (very high), Meta (high), Amazon (high), Microsoft (high), Bloomberg (very high), Apple (medium-high) Loop position: Hard algorithm round; common in senior+ loops. The two-heap dance is the canonical answer; competence here is a strong senior signal.
1. Quick Context
Design a class supporting:
addNum(num)— add an integer.findMedian()— return the median of all numbers added so far.
The two-heap pattern:
lo— max-heap of the smaller half (in Python: store negated values).hi— min-heap of the larger half.
Invariants (maintained after every addNum):
|lo|is either equal to|hi|or one more than|hi|(we letlohold the “extra” when total is odd).- Every element in
lois ≤ every element inhi.
Median:
- If
|lo| > |hi|: median is top oflo(single middle element). - Else (equal sizes): median is
(top(lo) + top(hi)) / 2.
Time: O(log n) per addNum, O(1) per findMedian. Space: O(n).
The senior signal: stating the invariants BEFORE coding. The mid signal: getting the code right. The junior signal: insertion-sort (O(n) per add → TLE).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/find-median-from-data-stream/
STOP. 25-min timer. Write invariants down first.
3. Prerequisite Concepts
heapq(min-heap; negate for max-heap).- The invariant-based reasoning style: state precondition, postcondition, prove preservation.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Maintain a multiset of integers; support O(log n) add and O(1) median query.”
Step 2 — Clarify:
- “Even count median?” (Average of two middles per LC.)
- “Duplicates?” (Allowed.)
- “Integer or float output?” (LC: float;
(a + b) / 2for averages.) - “Range of numbers?” (LC: any int.)
- “How many adds vs queries?” (LC: up to 5 × 10^4 of each.)
- “Streaming or batch?” (Streaming — never see future inputs.)
Step 3 — Constraints: N ≤ 5·10^4 operations → O(log n) per add is comfortable; O(n) per add (insertion sort) is borderline TLE.
Step 4 — Examples:
add(1) → median = 1; add(2) → median = 1.5; add(3) → median = 2.
Step 5 — Brute Force: Maintain a sorted list, bisect.insort (O(n) per add), index by n // 2. Works for small N but unsuitable for true streaming.
Step 6 — Complexity: Two heaps: O(log n) add, O(1) find. Sorted list with bisect: O(n) add (shift cost), O(1) find. SkipList / BBST: O(log n) both.
Step 7 — Pattern: Streaming median → two heaps. Generalization: streaming k-th element → two heaps with sizes k-1 and rest.
Step 8 — Develop: see solution.py.
Step 9 — Test: add then median; alternating add/median; even and odd counts; all duplicates; negatives.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — The two-heap dance
6.1 The add algorithm
def addNum(num):
# Step 1: always push onto lo first (max-heap; negate)
heappush(lo, -num)
# Step 2: move top of lo to hi (ensures lo's max ≤ hi's min)
heappush(hi, -heappop(lo))
# Step 3: rebalance if hi grew larger than lo
if len(hi) > len(lo):
heappush(lo, -heappop(hi))
Why this works:
- Step 1 unconditionally adds to
lo. We don’t yet know which halfnumbelongs to. - Step 2 moves the LARGEST of
lotohi. This guarantees the cross-heap ordering invariant (everylo≤ everyhi), because after Step 2,lo’s new max is ≤hi’s new min (which is the value we just moved). - Step 3 restores the size invariant:
loshould be equal to or one larger thanhi.
Three operations, all O(log n). The “always push to lo, then move to hi, then maybe move back” template is the canonical version because it’s bug-resistant.
6.2 Alternative add: branch-on-comparison
def addNum(num):
if not lo or num <= -lo[0]:
heappush(lo, -num)
else:
heappush(hi, num)
# rebalance
if len(lo) > len(hi) + 1:
heappush(hi, -heappop(lo))
elif len(hi) > len(lo):
heappush(lo, -heappop(hi))
Same big-O. Slightly faster constant (one push + at most one rebalance vs three pushes), but more branching. Either is acceptable.
6.3 The median query
def findMedian():
if len(lo) > len(hi):
return float(-lo[0])
return (-lo[0] + hi[0]) / 2.0
When |lo| == |hi| + 1: single middle is top of lo (since lo is the larger half).
When |lo| == |hi|: two middles, average them.
Edge case: empty stream — return… what? Behavior undefined per LC; in practice, raise or return 0. ASK the interviewer.
6.4 Removal — the unsupported operation
If the problem asks for remove(num) (sliding-window median, LC 480), the two-heap approach needs lazy deletion:
- On
remove(num), increment adeleted_count[num]. - After every operation, “clean” the top of both heaps by popping any value whose
deleted_count > 0.
This is non-trivial. The cleaner alternative is sortedcontainers.SortedList with O(log n) add/remove/index — but sortedcontainers isn’t stdlib in interviews. Mention both.
6.5 Why this beats “maintain a sorted list”
Naïve sorted list with bisect.insort has O(log n) for the binary search but O(n) for the insertion (shift). For 10^5 operations: 10^5 · 10^5 = 10^10 — TLE.
Two heaps: 10^5 · log(10^5) ≈ 10^5 · 17 = 1.7M operations — instant.
6.6 The percentile generalization
For the k-th order statistic in a stream, use two heaps with sizes k - 1 (max-heap) and n - k + 1 (min-heap). Same dance; constant rebalance to maintain those sizes. Median is k = (n+1)/2; for arbitrary k, the size invariant becomes the variable.
Production: P50/P95/P99 percentile dashboards use this pattern (or approximate variants like t-digest / HDR Histogram for bounded memory).
7. Anti-Pattern Analysis
- Insertion-sort list (
bisect.insort) — O(n) per add. TLE. - Sort on every query — O(n log n) per query. TLE.
- Single heap — can’t extract the median without popping half the heap.
- Forget to negate
lofor max-heap. Median is wrong every time. - Pushing
numdirectly without the lo-then-move dance — easy to violate cross-heap invariant. - Returning
intinstead offloat— LC wants float;(a + b) / 2gives float in Python 3 anyway, but(-lo[0])for odd count isint; returnfloat(...)to be safe. - Rebalance check
len(lo) - len(hi) > 1— works for the lo-then-move dance, but with branch-on-comparison you also need the reverse checklen(hi) > len(lo). - Calling
heap[0]on empty heap — IndexError. Guard empty stream. - Storing
(num, idx)tuples to support removal — premature optimization unless asked. - Confusing “median” with “mean” — different problems.
8. Skills & Takeaways
- Two-heap dance with explicit size and ordering invariants.
- Max-heap via negation in Python.
- Invariant-driven design — state preconditions/postconditions before coding.
- Lazy deletion for sliding-window variant (LC 480).
- Percentile generalization to arbitrary order statistic.
sortedcontainers.SortedListas a non-stdlib alternative (BBST).
9. When to Move On
Move on when:
- You can write
addNum+findMedianin ≤ 10 min, bug-free. - You can state the two invariants without notes.
- You can sketch the sliding-window variant (LC 480) with lazy deletion.
- You can explain why insertion-sort is O(n) and unacceptable.
- Stress test on 5000 random operations agrees with a sorted-list oracle.
10. Company Context
Google:
- One of the most-asked Hard problems. L4/L5 expected to solve cleanly; L6+ asked to extend to percentiles or sliding window.
- Bar Raiser scorecard: “Design — articulated invariants before coding; clean implementation; discussed scaling to streaming systems.”
Meta:
- L5+ algorithm round. Two-heap dance must be on-the-fly correct; can’t fumble.
Amazon:
- L5/L6 hard staple. Often paired with sliding-window follow-up.
Microsoft:
- L63/L64. Often phrased as “find running median of stock prices.”
Bloomberg:
- Loves percentile-tracking problems. Often pairs with “now do P95” for senior signal (heap-size generalization).
Apple:
- Performance monitoring teams ask running-percentile variants.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | “Sort each query” | “Maintain sorted list” | “Two heaps” | + states invariants first |
| Heap orientation | Confused about max-heap | Negates correctly | + explains “lo holds the extra” convention | + percentile generalization |
| Rebalance | Bug-prone | Works | “Push to lo, move max to hi, rebalance back” | + branch-on-comparison alternative discussed |
| Sliding window | “Re-build heaps” | “Stuck” | Lazy deletion sketch | + SortedList alternative + production t-digest |
| Empty stream | Crashes | Guards | “Undefined per spec; ASK” | + production: median of empty stream returns NaN, log warning |
12. Level Delta
Mid (L4):
- Two-heap implementation with explicit negation.
- Handles odd/even cases.
- Acknowledges insertion-sort is too slow.
Senior (L5):
- States invariants before coding.
- Articulates lo-then-move-to-hi-then-rebalance reasoning.
- Mentions sliding-window with lazy deletion.
Staff (L6):
- Generalizes to arbitrary percentile.
- Discusses
SortedList/ BBST alternative. - Discusses production: P95/P99 dashboards, monitoring infra.
Principal (L7+):
- Discusses approximate streaming percentiles: t-digest, HDR Histogram, count-min sketch + quantile estimation.
- Discusses bounded memory with O(1)-space approximations (GK Sketch — Greenwald-Khanna).
- Discusses distributed percentile aggregation: per-shard sketches, merge mathematically.
- Discusses trade-offs: exact (this problem) vs approximate (production); when to switch.
13. Follow-up Q&A
Q1: All numbers are in [0, 100]. Can we do O(1)?
A: Yes — count-array of size 101. addNum: O(1) increment. findMedian: scan to find the median position. O(101) = O(1).
Q2: 99% of numbers are in [0, 100]. Hybrid?
A: Count-array for the dense range, two-heaps for outliers. Maintain low_count, high_count, mid_array[101]. findMedian first scans mid_array adjusted by the heaps. Complex; mention only when prompted.
Q3: Sliding window (LC 480 — median of every window of size W)?
A: Two heaps + lazy deletion: deleted_count[num]; clean tops after each operation. Or: SortedList from sortedcontainers (cleanest).
Q4: P95 / arbitrary percentile in a stream?
A: Two heaps with size invariant |lo| = ⌈p · n⌉, |hi| = n - |lo|. Recompute target sizes after each add; rebalance accordingly. Or: t-digest / HDR Histogram for approximate.
Q5: Median over multiple shards (distributed)? A: Cannot trivially merge sorted halves — median is a holistic statistic. Approximate: each shard maintains a t-digest; merge t-digests is mathematically defined; exact median is intractable without full data shuffle.
Q6: Memory grows unbounded; how to cap? A: Approximate algorithms — t-digest with bounded number of centroids gives O(K) memory with provable error bounds (typically ~1% for K=100 centroids).
14. Solution Walkthrough
See solution.py:
MedianFinder— two-heap implementation; the canonical answer.MedianFinderBranching— alternativeaddNumwith branch-on-comparison.MedianFinderSorted— sorted-list baseline (oracle).edge_cases()+stress_test()— 5000 random add/findMedian sequences; all three implementations agree.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Three things. (1) The two invariants (size and cross-heap ordering) should be in the docstring as preconditions/postconditions for every public method — this is the kind of code that gets ‘optimized’ into bugs without an explicit invariant statement. (2) For production percentile tracking, this exact algorithm is the WRONG choice — memory grows linearly forever. Use t-digest or HDR Histogram; cite the references. The interview problem hides the scaling cliff. (3) The ‘find median’ operation is O(1), which is great for a UI dashboard that polls every second; but if the median is recomputed millions of times per second (alerting), even O(1) heap-top access has cache effects — consider caching the most recent median and invalidating on
addNum. Premature optimization, but worth a comment for the curious reader.”
Connections forward:
- p41 — Kth Largest (heap-of-size-K).
- p44 — Merge K Sorted Lists (heap of heads).
- LC 480 — Sliding Window Median (lazy-deletion variant).
- LC 4 — Median of Two Sorted Arrays (different technique: partition).
- LC 703 — Kth Largest in a Stream (heap streaming sibling).
- Phase 02 Lab 09 — Heap Top-K.
- Production: percentile dashboards (P50/P95/P99), monitoring infrastructure, t-digest, HDR Histogram, GK Sketch.
Hints — p43 Find Median from Data Stream
Hint 1. Median lives at the BOUNDARY between the smaller and larger halves. If you can maintain those two halves with O(log n) updates, you can read the median in O(1).
Hint 2. Use TWO heaps:
lo— max-heap of the smaller half (Python: store as negatives).hi— min-heap of the larger half.
Median = top of lo (odd) or avg of tops (even).
Hint 3. Two invariants to maintain:
- Sizes:
|lo| ∈ {|hi|, |hi| + 1}(let lo hold the extra when total is odd). - Order: every element of
lo≤ every element ofhi.
State these BEFORE writing addNum.
Hint 4. Canonical addNum — the “lo-then-move-then-rebalance” dance:
heappush(lo, -num) # always start in lo
heappush(hi, -heappop(lo)) # move lo's max to hi (preserves order invariant)
if len(hi) > len(lo):
heappush(lo, -heappop(hi)) # restore size invariant
Three pushes, all O(log n). Bug-resistant because invariants are preserved one at a time.
Hint 5. Median:
if len(lo) > len(hi): return float(-lo[0])
return (-lo[0] + hi[0]) / 2.0
Follow-up to think about: how would you support REMOVAL (sliding-window median, LC 480)? Lazy deletion with a deleted_count dict.
If still stuck: see solution.py.
p44 — Merge K Sorted Lists
Source: LeetCode 23 · Hard · Topics: Heap, Linked List, Divide & Conquer Companies (2024–2025 frequency): Google (very high), Meta (high), Amazon (very high), Microsoft (high), Apple (high), Uber (medium), LinkedIn (high) Loop position: Classic Hard heap problem; appears in nearly every senior algorithm round. The “K-way merge” pattern is the canonical lesson.
1. Quick Context
You’re given an array of k sorted linked lists. Merge them into a single sorted linked list and return the head.
Two canonical solutions, both O(N log K) where N = total nodes:
- Heap of K head pointers. Push all heads onto a min-heap; pop the smallest, append to result, push the popped node’s
next(if any). O(N log K) time, O(K) extra space. - Divide and conquer. Pairwise-merge K lists in log K rounds. Same O(N log K), no heap needed.
The naïve approaches both lose:
- Concatenate + sort: O(N log N) — fine in big-O but ignores existing sortedness; awkward with linked lists.
- Sequential merge (merge list 0 with list 1, then with list 2, …): O(N · K) — TLE for large K.
Tie-break gotcha: Python’s heapq compares tuples elementwise; if two nodes have equal value, it tries to compare the ListNode objects → TypeError. Fix: push (val, unique_index, node).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/merge-k-sorted-lists/
STOP. 30-min timer. Define ListNode first; sketch the heap approach.
3. Prerequisite Concepts
- Linked list traversal + dummy-head pattern.
heapqwith tuple comparison + tie-break index.- Merge-two-sorted-lists subroutine (LC 21).
- Divide and conquer recurrence:
T(K) = 2T(K/2) + O(N)→ O(N log K).
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “K already-sorted streams; produce one sorted stream.”
Step 2 — Clarify:
- “Can lists be empty? Can the input array be empty?” (Both yes per LC.)
- “Can node values be negative / duplicates?” (Both yes.)
- “Modify in place or new list?” (LC accepts node-reuse.)
- “Stable order on ties?” (Doesn’t matter for value-sorted output.)
Step 3 — Constraints: k ≤ 10^4, total nodes ≤ 10^4, values in [-10^4, 10^4]. O(N log K) comfortable.
Step 4 — Examples: [[1,4,5], [1,3,4], [2,6]] → [1,1,2,3,4,4,5,6].
Step 5 — Brute force: Collect all node values into a list, sort, rebuild a new linked list. O(N log N) time, O(N) space. Loses information that lists are presorted but passes correctness.
Step 6 — Complexity:
- Brute (collect+sort): O(N log N).
- Sequential merge: O(N · K).
- Heap of K: O(N log K) time, O(K) space.
- Divide & conquer: O(N log K) time, O(log K) recursion space (or O(1) iterative pairwise).
Step 7 — Pattern: “K-way merge” — heap-over-streams is the canonical streaming pattern. Same shape: merge-K-sorted-arrays, merge-K-sorted-files, external sort.
Step 8 — Develop: see solution.py.
Step 9 — Test: empty input, single list, all-empty lists, lists with one node, duplicates across lists, all-equal-values (tie-break stress).
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Two equally good answers
6.1 Heap of K
heap = []
for i, head in enumerate(lists):
if head:
heapq.heappush(heap, (head.val, i, head))
dummy = tail = ListNode()
while heap:
val, i, node = heapq.heappop(heap)
tail.next = node
tail = node
if node.next:
heapq.heappush(heap, (node.next.val, i, node.next))
return dummy.next
Why the index i? heapq compares tuples elementwise. On a tie at val, it falls through to the second element. If you push (val, node), Python tries to compare ListNode objects → TypeError: '<' not supported between instances of 'ListNode' and 'ListNode'. Using a unique integer index sidesteps the issue.
Alternative fixes:
- Reuse the original list index
i(works as long as you never push two values from the same list — true here, since you push.nextonly after popping). - Use
itertools.count()to generate unique indices. - Define
__lt__onListNodeto compare by value (modifies the class; works but feels invasive).
Time: O(N log K). Each node enters and leaves the heap exactly once; heap size ≤ K.
6.2 Divide and conquer
def merge(lists):
if not lists: return None
while len(lists) > 1:
merged = []
for i in range(0, len(lists), 2):
l1 = lists[i]
l2 = lists[i + 1] if i + 1 < len(lists) else None
merged.append(merge_two(l1, l2))
lists = merged
return lists[0]
Why it’s O(N log K)? Each “round” of pairwise merges touches each node once (total work = N per round). There are log K rounds (K halves each time). Total = O(N log K). Identical big-O to the heap.
Recursive variant:
def merge(lists, lo, hi):
if lo == hi: return lists[lo]
mid = (lo + hi) // 2
return merge_two(merge(lists, lo, mid), merge(lists, mid + 1, hi))
Same big-O; O(log K) recursion depth.
6.3 Why NOT sequential merge
result = None
for lst in lists:
result = merge_two(result, lst)
Round i merges a list of size ~ i · N/K with a list of size N/K, costing O(i · N/K). Total: (N/K) · (1 + 2 + ... + K) = O(N · K).
For K = 10^4, N = 10^4: 10^8 ops — TLE on most judges.
Divide-and-conquer keeps merges BALANCED, avoiding the “growing accumulator” problem.
6.4 Iterators vs node references
This problem is canonical with linked lists, but the heap pattern works identically over iterators. heapq.merge(*iterables) (stdlib) does exactly this in O(N log K) and is the right answer for “merge K sorted iterables of any kind.” Mention it as a one-liner — interviewers love that you know stdlib has it built in.
6.5 External sort connection
The heap-of-K-streams pattern IS the merge phase of external sort: K sorted runs on disk, merge them by maintaining a heap of K “current-front” elements, one buffered block per stream. Same code, different I/O model.
6.6 The space-time trade
Heap version uses O(K) auxiliary memory. Divide-and-conquer iterative uses O(K) for the temp merged array per round but pairs of merges are O(1) extra. Recursive divide-and-conquer uses O(log K) stack. For K ≤ 10^4, all are negligible. For K = 10^9 (cluster-scale), heap-of-K becomes the constraint and you need hierarchical merging.
7. Anti-Pattern Analysis
- Sequential merge (accumulator) — O(N · K). TLE.
- Push
(val, node)without index → TypeError on tie. - Build full array → sort → rebuild list — works but wasteful; loses the sortedness.
- Forget to push
.nextafter popping — heap drains, lose nodes. - Forget to handle empty input lists —
lists = []should return None, not crash. - Forget to filter
Noneheads on initial push —head.valon None → AttributeError. - Forget the dummy head — convoluted edge handling for the first node.
- Skip the tie-break index — works only if all values distinct.
- Modify nodes’
.nextduring merge then iterate later — easy to introduce cycles. - Returning a new list of values instead of relinking nodes — passes LC but doubles memory; in production, in-place relink is preferred.
8. Skills & Takeaways
- K-way merge pattern with heap-over-streams.
- Divide and conquer as a no-heap alternative with same big-O.
- Tie-break index for non-comparable payloads.
- Dummy head for linked-list construction.
heapq.mergestdlib one-liner for iterables.- External sort as the production analogue.
9. When to Move On
Move on when:
- You can write heap-of-K from scratch in ≤ 15 min including tie-break.
- You can write divide-and-conquer pairwise merge in ≤ 15 min.
- You explain WHY sequential merge is O(N · K) and why D&C is O(N log K).
- You know
heapq.mergeexists. - Stress test on K up to 50, N up to 1000 passes vs sorted-array oracle.
10. Company Context
Google:
- Universal Hard. L4/L5 must produce both solutions. L6+ asked external-sort follow-up.
- Bar Raiser scorecard: “K-way merge — produced both heap and divide-and-conquer; explained O(N log K) crisply; knew
heapq.merge.”
Meta:
- L5/L6 staple. Often the second Hard in a 4-round loop.
Amazon:
- Phrased as “merge K sorted streams of orders/events.” Connect to Kinesis-like systems.
Microsoft:
- L63+. Often asks the iterator variant first (“merge K sorted iterators”).
Apple:
- Storage / iOS Files teams ask the external-sort variant.
Uber, LinkedIn:
- Common Hard, often paired with a follow-up on stable ordering or duplicates.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | Concat + sort | Heap-of-K | “Heap or D&C — same big-O” | + mentions heapq.merge, external sort |
| Tie-break | Hits TypeError | Adds index | Explains WHY | + alternative: define __lt__ |
| Empty input | Crashes | Handles | Mentions explicitly | + parameterizes for None tolerance throughout |
| Complexity | “It’s fast” | O(N log K) | Compares vs sequential O(N · K) | + space analysis O(K) vs O(log K) recursion |
| Production | Silent | Silent | “External sort uses this” | + sketches buffered-block I/O, sketches Kafka/Kinesis stream join |
12. Level Delta
Mid (L4):
- Heap-of-K with tie-break index.
- Handles empty lists and empty input array.
- Quotes O(N log K).
Senior (L5):
- Provides BOTH heap and divide-and-conquer.
- Explains tie-break gotcha with the TypeError.
- Compares against sequential merge’s O(N · K).
Staff (L6):
- Mentions
heapq.mergestdlib. - Connects to external sort and stream-merge in production.
- Discusses space trade-off (heap O(K) vs D&C O(log K)).
Principal (L7+):
- Discusses hierarchical merge for K = 10^9 (cluster-scale): per-shard pre-merge, then merge of merges.
- Discusses I/O-aware merging: block-buffered reads, prefetching, parallel decompression.
- Discusses bounded-memory merging when K · block_size > RAM: spill intermediate runs to disk.
- Discusses stream-merge in event systems: watermarks, late-arriving data, out-of-order tolerance.
13. Follow-up Q&A
Q1: K sorted iterators of any type (not linked lists)?
A: heapq.merge(*iterables) — stdlib, one line, lazy. Or write the heap loop with iter()/next().
Q2: K = 10^9 (cluster-scale)? A: Hierarchical merge. Each machine merges ~K/M lists; emit sorted output; coordinator merges M sorted streams (same algorithm, smaller K).
Q3: How would you do this on disk (external merge)? A: K input files, each pre-sorted. Maintain a small buffer (1 block) per file; heap over current-front elements; write merged output to disk. Total I/O is linear in N if you size buffers well. This is the merge phase of external sort.
Q4: K sorted streams, but each stream is infinite — stream the result?
A: Same heap loop, but yield instead of build a list. heapq.merge returns a lazy iterator already. Backpressure / pull semantics matter.
Q5: Stable merge (preserve original list order on ties)?
A: The (val, list_index, node) tuple already gives stability by ascending list_index. If you want stability across ties WITHIN a single list, use (val, list_index, node_index).
Q6: Merge K iterators where each emits (timestamp, payload) with possible out-of-order events?
A: Watermark-based merging: accept events within a tolerance window; buffer slightly out-of-order events; drop or side-channel late ones. Production: Flink, Beam, Spark Structured Streaming.
14. Solution Walkthrough
See solution.py:
merge_k_lists_heap— heap of K head pointers with tie-break index.merge_k_lists_divide_conquer— iterative pairwise merge.merge_two_lists— LC 21 subroutine.merge_k_lists_brute— collect + sort baseline (oracle).- Helpers:
to_list_from_array,to_array_from_listfor testability. edge_cases()+stress_test()— 200 random trials, all three agree.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Four things. (1) The tie-break index pattern is a low-key signature of ‘developer who reads tracebacks’ — keep it; explicit comment in the code. (2)
heapq.mergeexists; for new code over iterables, use it. We had a 200-line custom merger in our pipeline that was a one-liner. (3) For our streaming-events pipeline, the linked-list framing is wrong — events come as iterators with bounded memory budget per stream. Use bounded queues, not unbounded buffering; or you OOM at 3am. (4) Divide-and-conquer is the right answer for cluster-scale; the heap version assumes you have all K streams in one process. State the model assumption in the docstring.”
Connections forward:
- p41 — Kth Largest (heap-of-size-K).
- p42 — Top K Frequent.
- p43 — Find Median from Data Stream.
- LC 21 — Merge Two Sorted Lists (subroutine).
- LC 88 — Merge Sorted Array (two-pointer variant).
- LC 378 — Kth Smallest in Sorted Matrix (heap-over-rows; same pattern).
- Phase 02 Lab 09 — Heap Top-K.
- Production: external sort merge phase, Kafka/Kinesis stream merge, log aggregation, distributed query plan merge.
Hints — p44 Merge K Sorted Lists
Hint 1. Naïve idea: collect all values, sort, rebuild → O(N log N). Works but ignores that each list is already sorted. Can we do O(N log K) where K << N?
Hint 2. Heap-of-K: push the HEAD of each list onto a min-heap. Pop the smallest, append to result, then push its .next (if non-null). Each node enters/leaves heap once → O(N log K).
Hint 3. Python heap GOTCHA — pushing (val, node) blows up on ties:
TypeError: '<' not supported between instances of 'ListNode' and 'ListNode'
because heapq falls through to compare the second tuple element. Fix: push (val, unique_index, node). The index is never compared if values differ; on ties it provides a deterministic order.
Hint 4. Alternative without a heap: DIVIDE AND CONQUER. Pairwise merge the K lists in log K rounds, using a standard merge-two-sorted-lists (LC 21) subroutine. Same O(N log K). Sometimes preferred when you don’t want a heap.
while len(lists) > 1:
merged = []
for i in range(0, len(lists), 2):
l1 = lists[i]
l2 = lists[i+1] if i+1 < len(lists) else None
merged.append(merge_two(l1, l2))
lists = merged
return lists[0]
Hint 5. WHY NOT sequential merge (result = merge(result, lists[i]) for each i)? Round i merges a result of size ~ i · N/K with size N/K. Total = O(N · K). TLE.
Bonus: stdlib has heapq.merge(*iterables) for arbitrary sorted iterables — production one-liner.
If still stuck: see solution.py.
p45 — Reorganize String
Source: LeetCode 767 · Medium · Topics: Heap, Greedy, String, Counting Companies (2024–2025 frequency): Meta (high), Amazon (high), Google (medium-high), Microsoft (medium), Apple (medium), LinkedIn (medium) Loop position: Medium-Hard greedy + heap. Tests the “use top, stash, push back” delayed-reinsertion pattern that appears in task scheduling (LC 621), distance-K rearrangement (LC 1054), and load-balancing problems.
1. Quick Context
Given string s, rearrange its characters so no two adjacent characters are the same. Return any valid rearrangement, or "" if impossible.
Two canonical solutions:
- Greedy max-heap with delayed reinsertion. Pop the most-frequent char, append to result, STASH it (don’t push back yet); pop next most frequent, append, then push the stashed one back if still has count > 0. O(N log Σ).
- Counting-sort placement. Count frequencies; sort by count desc; place the most-frequent char at even indices first (
0, 2, 4, ...), wrap to odd indices (1, 3, 5, ...). O(N + Σ log Σ). Often faster in practice and avoids the heap.
Feasibility check: answer exists iff max_count ≤ (N + 1) // 2. Otherwise return "" immediately.
The “stash, then push back” pattern is the key reusable trick — it prevents the same character from being chosen twice in a row, since the heap can’t see it until after the next pop.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/reorganize-string/
STOP. 25-min timer. State the feasibility check first.
3. Prerequisite Concepts
Counter/collections.Counter.heapqmin-heap with negated counts for max-heap behavior.- Greedy intuition: always place the most constrained item first (the one with highest remaining count).
- Pigeonhole-style feasibility argument.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Permute the characters of s so no two adjacent are equal; or report impossibility.”
Step 2 — Clarify:
- “Lowercase ASCII only?” (LC: yes.)
- “Length bound?” (LC: 1 ≤ N ≤ 500.)
- “Any valid answer accepted, or canonical?” (Any valid.)
- “Output empty string on impossibility?” (Yes per LC.)
Step 3 — Constraints: N ≤ 500 → any O(N log N) solution is comfortable. Σ = 26.
Step 4 — Examples:
"aab" → "aba"."aaab" → ""(threeas, length 4 → max_count=3 > (4+1)//2=2; impossible)."aaabbc" → "ababac"(one valid output).
Step 5 — Brute force: Permute all N! arrangements; return first valid. Factorial; only viable for N ≤ 8. Useful as a small-input oracle.
Step 6 — Complexity:
- Brute permutations: O(N! · N).
- Heap greedy: O(N log Σ).
- Counting-sort placement: O(N + Σ log Σ).
Step 7 — Pattern: “Greedy with delayed reinsertion.” Sibling problems: LC 621 (Task Scheduler with cooldown), LC 358 (Rearrange String K Distance Apart), LC 1054 (Distant Barcodes).
Step 8 — Develop: see solution.py.
Step 9 — Test: length 1, two distinct chars, max_count exceeds bound, all same char, alternating possible, tight feasibility (max_count == (n+1)//2).
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Two equally good answers
6.1 Feasibility — the pigeonhole argument
Claim: a valid rearrangement exists iff max_count ≤ (N + 1) // 2.
- Necessity: Suppose char
xhas countc > (N+1)//2. Placexat positionsp_1 < p_2 < ... < p_c. Between consecutivexs we need at least one non-xcharacter. So we needc - 1“separators”, which must come from the remainingN - ccharacters. NeedN - c ≥ c - 1, i.e.,c ≤ (N + 1) / 2. - Sufficiency: The counting-sort placement (Section 6.3) gives a constructive proof: place the most-frequent character at even indices
0, 2, 4, ...; it fills at most(N + 1) // 2slots, which is exactly the budget; then place remaining chars at the rest. No collision because no character repeats at distance 1 unlessmax_countexceeds the bound.
Check this FIRST. Save heap work on impossible inputs.
6.2 Greedy max-heap with delayed reinsertion
cnt = Counter(s)
if max(cnt.values()) > (len(s) + 1) // 2: return ""
heap = [(-c, ch) for ch, c in cnt.items()]
heapq.heapify(heap)
result = []
prev_count, prev_char = 0, ''
while heap:
c, ch = heapq.heappop(heap) # c is negative
result.append(ch)
# Push the PREVIOUSLY-used char back now (it was stashed last iteration).
if prev_count < 0:
heapq.heappush(heap, (prev_count, prev_char))
# Stash the just-used char for the NEXT iteration.
prev_count, prev_char = c + 1, ch # +1 because c is negative
return ''.join(result)
Why this works: By stashing (prev_count, prev_char) for one iteration, the character we just placed is invisible to the heap on the next pop — guaranteeing we pick a DIFFERENT character next. Once we’ve picked the different one, we put the stashed char back into the heap with its remaining count.
The greedy choice — always pop the highest-remaining-count character — is correct because:
- If we ever delay placing the max-count char, we risk exhausting the “separators” before we run out of
xs. - By contrast, always placing the max-count char (subject to the “not adjacent to previous” constraint) keeps the count balanced.
6.3 Counting-sort placement — no heap
n = len(s)
cnt = Counter(s)
if max(cnt.values()) > (n + 1) // 2: return ""
# Sort by (count desc, char asc) — sorting by count desc is what matters.
chars = sorted(cnt.keys(), key=lambda c: -cnt[c])
result = [''] * n
idx = 0
for ch in chars:
for _ in range(cnt[ch]):
result[idx] = ch
idx += 2
if idx >= n:
idx = 1 # wrap to odd indices
return ''.join(result)
Why this works:
- Even indices:
0, 2, 4, ...— total(n + 1) // 2slots. - Place the highest-count char first; if
max_count ≤ (n+1)//2, it fits entirely within even indices (or just barely overflows by 1 into odd indices — still fine). - After even indices fill, wrap to odd indices
1, 3, 5, .... - No collision: a character placed at even index
iand at odd indexi+1? Only happens if the SAME character wraps around — but that requires the character’s count exceeds(n+1)//2, which we’ve ruled out by the feasibility check.
This is often faster (no heap overhead) and simpler to explain.
6.4 Why “stash and re-push” is the canonical greedy
The bug-prone naïve version:
while heap:
c, ch = pop()
result.append(ch)
if c + 1 < 0: push((c + 1, ch))
This puts the same character RIGHT BACK into the heap with one less count — so the next pop can pick it again if no other character has higher count. Result: adjacent duplicates → wrong.
The fix is the one-iteration delay (stash). Same trick appears in:
- LC 621 Task Scheduler — multi-iteration delay equal to the cooldown.
- LC 358 Rearrange String K Distance Apart — generalize to delay K-1.
- LC 1054 Distant Barcodes — same problem, different framing.
6.5 When the heap loses
For N = 500 with Σ = 26: heap-of-26 operations are O(log 26) ≈ 4.7. Total ~500 · 4.7 ≈ 2350 heap ops. The counting-sort version does O(N) writes + O(Σ log Σ) sort. Both trivial.
For larger Σ (e.g., Unicode), counting-sort gets larger but heap stays O(log Σ). Cross-over depends on N and Σ; both are O(N log N) worst-case anyway. Mention both.
6.6 Connection to load balancing
This problem is a 1-D toy version of round-robin scheduling with non-uniform task arrival rates. Replace “characters” with “tasks per host”, “adjacent” with “consecutive on same core”. The “no two adjacent” constraint becomes “no two consecutive on same core” — same algorithm, same feasibility bound. Production load balancers (HAProxy weighted round-robin, Kubernetes pod placement) use related heuristics.
7. Anti-Pattern Analysis
- No feasibility check — heap runs forever (or returns junk).
- Push-back-immediately (no delay) — adjacent duplicates → wrong.
- Sort-by-char only (
"aab"→"aab") — misses high-count constraint. - Try permutations — factorial, TLE for N > 8.
- Use
heapqas max-heap without negation — pops smallest first; greedy is broken. - Greedy-pick second-highest when adjacent constraint applies without stash — same bug as (2).
- Counting-sort but place at consecutive indices (
0, 1, 2, 3, ...) — only works if all distinct. - Counting-sort but wrap to odd at wrong moment — off-by-one if
idx == nvsidx > n. - Returning
resultas list when problem wants string — small thing, easy to miss. - Confusing “no two adjacent same” with “stable sort” — different goals.
8. Skills & Takeaways
- Pigeonhole feasibility argument:
max_count ≤ (N+1)//2. - Delayed reinsertion (stash) pattern in heap greedy.
- Counting-sort placement alternative — even-indices-then-odd.
- Greedy correctness: always handle the most-constrained element first.
- Generalization: Task Scheduler (LC 621), K-distance (LC 358), Distant Barcodes (LC 1054).
- Round-robin scheduling as the production analogue.
9. When to Move On
Move on when:
- You write the feasibility check FIRST without prompting.
- You implement heap-with-stash in ≤ 15 min.
- You implement counting-sort placement in ≤ 15 min.
- You can explain WHY the naïve push-back-immediately fails.
- You can sketch LC 621 (cooldown delay) in 2 min.
- Stress test on N ≤ 100 agrees with permutation brute force for small N + validates output for larger N.
10. Company Context
Meta:
- Common Medium-Hard. L4/L5 expected; L6 asked to generalize to K-distance.
- Bar Raiser scorecard: “Greedy — articulated feasibility argument; chose heap-with-stash or counting-sort; explained why naïve fails.”
Amazon:
- L5/L6. Often paired with Task Scheduler as a two-problem set.
Google:
- L4/L5. Often asks the counting-sort version specifically (no heap).
Microsoft:
- L62/L63. Variant: print schedule respecting cooldowns.
Apple:
- iOS Notifications team has asked the deduplication variant.
LinkedIn:
- Common Medium; often paired with “now do K-distance” follow-up.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Feasibility | Misses check | Adds check after seeing bug | States check FIRST | + proves it |
| Greedy | Naïve push-back | Recognizes adjacent-duplicate bug | Implements stash | + explains generalization to LC 621/358 |
| Alternative | Heap only | Heap only | Mentions counting-sort | + compares both, knows when each is faster |
| Output ordering | Hardcoded | Notes “any valid” | Tests randomized | + discusses canonical/stable variants |
| Production | Silent | Silent | “Like round-robin scheduling” | + load balancing, k8s pod scheduling, weighted scheduling |
12. Level Delta
Mid (L4):
- Implements heap-with-stash or counting-sort.
- Includes feasibility check.
- Tests with
"aab","aaab".
Senior (L5):
- Provides BOTH solutions.
- Explains naïve-push-back failure clearly.
- Sketches LC 621 generalization (cooldown = stash-delay).
Staff (L6):
- Proves feasibility via pigeonhole.
- Discusses connection to LC 358 (K-distance) and LC 1054.
- Discusses round-robin scheduling analogue.
Principal (L7+):
- Discusses weighted scheduling in production (HAProxy, Envoy).
- Discusses multi-resource scheduling (CPU + memory + GPU) — feasibility is harder; LP relaxation.
- Discusses fairness constraints and starvation prevention in long-running systems.
- Discusses online variant: characters arrive in a stream; produce output online without knowing total counts.
13. Follow-up Q&A
Q1: K-distance variant (LC 358)?
A: Generalize stash to a QUEUE of size K-1: pop top, append, enqueue to wait-queue. After K iterations, dequeue and push back to heap (still has count > 0). Feasibility: max_count ≤ ceil(n / K) ? — actually (max_count - 1) * K + (#chars with max_count) ≤ n.
Q2: Task Scheduler (LC 621) — same but compute schedule LENGTH, not the schedule?
A: Formula approach: result = max(n, (max_count - 1) * (cooldown + 1) + chars_at_max_count). O(N + Σ). No heap needed.
Q3: Online variant — characters arrive in a stream? A: Maintain heap; emit output as soon as legal. Buffer last-emitted for one tick (the stash). If stream completes and remaining chars violate adjacency, fail. Harder if counts unknown in advance.
Q4: Lex-smallest valid output? A: Greedy doesn’t directly give lex-smallest. Counting-sort version with lex tie-break on equal counts comes close but isn’t strictly minimal. True lex-smallest requires backtracking or careful ordering.
Q5: Why is max_count ≤ (n+1)//2 the bound (not n/2)?
A: When n is odd, we have (n+1)/2 even slots (e.g., n=5: slots 0,2,4 = 3) and only n/2 odd slots. We can place the high-count char at all even slots without adjacency. n/2 would be too tight.
Q6: What if input has multiple characters tied at max_count, all > (n+1)//2?
A: Impossible. The check max(counts) > (n+1)//2 catches it because the SINGLE max already exceeds.
14. Solution Walkthrough
See solution.py:
reorganize_string_heap— max-heap (negated) with stash pattern.reorganize_string_indexed— counting-sort placement (even-then-odd).reorganize_string_brute— permutation brute force (N ≤ 8 only).is_valid(s, original)— checks no two adjacent same AND is a permutation of original.edge_cases()+stress_test()— small inputs vs brute, large inputs validated byis_valid.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Three thoughts. (1) Always do the feasibility check first — both as correctness and as a teaching artifact in the code; a senior should see that the math is the algorithm’s BACKBONE, not an afterthought. (2) For our scheduling code we use the counting-sort version because we want determinism (same input → same output for replay debugging); the heap version’s tie-break depends on
heapqinternals and isn’t stable. Prefer the deterministic one for production. (3) The K-distance generalization (LC 358) is the version that ACTUALLY shows up in our task scheduler — the cooldown is K. Recognize that this Medium is the warm-up for the Hard.”
Connections forward:
- p41 — Kth Largest (heap intro).
- p42 — Top K Frequent.
- LC 621 — Task Scheduler (cooldown variant).
- LC 358 — Rearrange String K Distance Apart (heap + wait queue).
- LC 1054 — Distant Barcodes (same problem, ints).
- LC 451 — Sort Characters by Frequency (counting-sort sibling).
- Phase 02 Lab 09 — Heap Top-K.
- Phase 06 — Greedy.
- Production: round-robin scheduling, weighted load balancing, K8s pod scheduling, deduplication windows.
Hints — p45 Reorganize String
Hint 1. Feasibility FIRST. Answer exists iff max_count ≤ (N + 1) // 2. Pigeonhole: if any char has more than that, the “separators” between its occurrences run out. Compute Counter(s).values() and check before doing any work.
Hint 2. Greedy idea: always place the most-frequent remaining character (most constrained → handle first). But the naïve “pop, append, push back with count - 1” puts the SAME char right back at the top of the heap, causing adjacent duplicates.
Hint 3. The fix is one-iteration delay (stash). After popping char ch, don’t push it back yet. Stash it. On the NEXT iteration, pop a (necessarily different) char, append, and only THEN push the stashed ch back if it still has count > 0.
prev_count, prev_char = 0, ''
while heap:
c, ch = heappop(heap)
out.append(ch)
if prev_count < 0: heappush(heap, (prev_count, prev_char))
prev_count, prev_char = c + 1, ch
Hint 4. Alternative WITHOUT a heap: counting-sort placement. Sort chars by count desc; place them at indices 0, 2, 4, ... (even), wrapping to 1, 3, 5, ... (odd) when even slots fill. Feasibility check guarantees no collision (the highest-count char fits within the (n+1)//2 even slots).
chars = sorted(cnt.keys(), key=lambda c: -cnt[c])
out = [''] * n
idx = 0
for ch in chars:
for _ in range(cnt[ch]):
out[idx] = ch
idx += 2
if idx >= n: idx = 1
Hint 5. Generalization: K-distance (LC 358) — replace the one-slot stash with a queue of size K-1. Cooldown scheduler (LC 621) is the same shape with cooldown delay. The “delayed reinsertion” pattern is the reusable lesson.
If still stuck: see solution.py.
Week 10 — Intervals: The Sort-Then-Sweep Toolkit
Phase: Month 02 — Patterns · Week: 10 of 24 · Theme: Intervals Problems: 5 (p46–p50) — 3 Medium + 1 Medium + 1 Hard Prereqs: Sorting, heap (Week 9), two-pointer mindset, comparator-by-key
Why Intervals
Almost every “calendar / scheduling / resource allocation / range” question reduces to one of three primitives:
- Merge — combine overlapping ranges into a smaller set.
- Sweep / chronological scan — process start and end events in time order.
- Active-count tracking — at any time
t, how many intervals covert?
Once you see the connection, problems that look wildly different (calendar bookings, CPU scheduling, free-time computation, conflict detection, lane assignment) collapse into one of two algorithms:
- Sort by start time → linear sweep, or
- Sort start-events + end-events independently → two-pointer or min-heap sweep.
The interview signal isn’t “did you solve this problem”; it’s “can you SEE this as an interval problem and pick the right primitive immediately.” Week 10 builds that pattern recognition.
Mental Models to Cement
-
Two intervals
[a, b]and[c, d]overlap iffa ≤ d AND c ≤ b(or equivalentlymax(a, c) ≤ min(b, d)). For half-open[a, b), replace≤with<. ASK the interviewer about endpoint semantics. -
Sort by start, sweep: after sorting, you only need to compare each interval with the LAST appended interval in your result. O(n log n) sort dominates; sweep is O(n).
-
Two-pointer over starts and ends: sort starts and ends INDEPENDENTLY; sweep both with two pointers; when
starts[i] < ends[j], a meeting begins (active++), else one ends (j++, active–). Peakactiveis the answer (LC 253 Meeting Rooms II). -
Min-heap of end-times: sort by start; for each interval, if heap top ends ≤ current start, reuse that room (pop); push current end. Heap size = peak rooms. O(n log n).
-
Sweep line / event list: create
(start, +1)and(end, -1)events; sort (with ties:-1before+1for half-open semantics,+1before-1for closed); accumulate prefix sum to track active count. -
Greedy “earliest-ending-first” for maximum non-overlapping subset (LC 435): sort by END time, take any interval whose start ≥ previous end. Classic activity-selection.
-
Insert into sorted list of disjoint intervals (LC 57): three phases — copy strictly-before, merge overlapping, copy strictly-after. No need to re-sort.
-
Endpoint conventions are footguns.
[1, 2]and[2, 3]— overlap or not? In calendars, usually not ([start, end)half-open: meeting ending at 2 doesn’t conflict with one starting at 2). In LC 56, they DO merge (closed intervals). ASK. -
Comparator by first key:
intervals.sort(key=lambda x: x[0])is the idiom. Python sorts tuples elementwise, sointervals.sort()works when shape is[start, end]. Don’t write a custom comparator unless tie-breaking on the second key matters. -
Off-by-one on merge: the merged interval’s end is
max(prev_end, cur_end), NOTcur_end. Easy bug when intervals are fully contained inside larger ones.
Problem Lineup
| # | Title | LC | Difficulty | Core Pattern |
|---|---|---|---|---|
| p46 | Merge Intervals | 56 | Medium | Sort by start; sweep with max(prev_end, cur_end) |
| p47 | Insert Interval | 57 | Medium | Three-phase scan (before / merge / after); pre-sorted input |
| p48 | Non-overlapping Intervals | 435 | Medium | Greedy: sort by END; activity-selection |
| p49 | Meeting Rooms II | 253 | Medium | Min-heap of end-times OR two-pointer chronological |
| p50 | Employee Free Time | 759 | Hard | Multi-list merge → flatten → invert; or heap-of-K starts |
Daily Schedule (5 days @ 90 min)
| Day | Problem | Practice Focus |
|---|---|---|
| Mon | p46 | Sort + sweep; argue why one comparison with prev suffices |
| Tue | p47 | Three-phase scan; do WITHOUT re-sorting |
| Wed | p48 | Why sort-by-END beats sort-by-START; activity-selection proof |
| Thu | p49 | Two solutions: heap AND two-pointer; reason about peak active |
| Fri | p50 | Multi-source merge (heap-of-K or flatten-and-merge); invert to free time |
Saturday: 30-min mock with p49 or p50. Sunday: Re-solve any problem you took > 25 min on; write the comparator and feasibility on a blank page from memory.
Anti-Patterns to Notice This Week
- Sort by both start AND end as a “safety” comparator — adds nothing for merge (start-only suffices); confuses activity-selection (must be end).
- For Meeting Rooms II: maintain a sorted list of meetings and insert one by one — O(n²).
- For Insert Interval: append + re-sort + run merge — works but O(n log n) when O(n) is possible.
- Treat
[1,2]and[2,3]as always overlapping (or always not) — depends on convention; ASK. - For Employee Free Time: flatten without sorting per-employee structure — loses information that each employee’s intervals are pre-sorted; you can heap-merge in O(N log K) instead of O(N log N).
Readiness Checklist
By Sunday, you should:
- Recognize an interval problem from its phrasing in < 60 seconds.
- Decide instantly: “merge → sort-by-start”, “max non-overlap → sort-by-end”, “peak active → heap or two-pointer”, “invert → sweep”.
- Implement Merge Intervals in ≤ 8 minutes without lookup.
- Articulate the half-open vs closed endpoint trade-off.
- Reproduce the Meeting Rooms II heap solution on a blank page in ≤ 12 minutes.
- Sketch the two-pointer alternative to Meeting Rooms II.
- Explain why activity-selection sorts by END, not by START (exchange argument).
Looking Ahead
Week 11 (planned): Monotonic Stack — Daily Temperatures, Next Greater Element I/II, Largest Rectangle in Histogram, Trapping Rain Water deep-dive, Sliding Window Maximum. Intervals → monotonic stack is a natural progression: both are “sweep with auxiliary structure”; monotonic stack adds the discipline of maintaining sorted order in the auxiliary structure during the sweep.
p46 — Merge Intervals
Source: LeetCode 56 · Medium · Topics: Array, Sorting Companies (2024–2025 frequency): Meta (very high), Google (very high), Amazon (very high), Microsoft (high), Bloomberg (high), Apple (medium-high), Uber (high), Airbnb (high) Loop position: Top-3 most-asked Medium overall. Phone screen and onsite warm-up. NOT solving this cleanly is a serious red flag.
1. Quick Context
Given a list of intervals [[start_i, end_i], ...] (possibly unsorted, possibly overlapping), merge all overlapping intervals and return the resulting list, sorted by start.
Canonical solution:
- Sort by start.
- Sweep: for each interval, if it overlaps the last appended interval, extend; otherwise append.
intervals.sort(key=lambda x: x[0])
out = []
for s, e in intervals:
if out and s <= out[-1][1]:
out[-1][1] = max(out[-1][1], e)
else:
out.append([s, e])
return out
O(n log n) time (sort), O(n) extra (output, ignoring sort space). The whole algorithm is 5 lines; the interview signal is in HOW you discuss it: comparator choice, endpoint semantics, why one comparison with out[-1] suffices.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/merge-intervals/
STOP. 15-min timer. Write the comparator, the overlap condition, and the merge rule on paper FIRST.
3. Prerequisite Concepts
- Sorting with a key (
list.sort(key=...)). - Mutable list-of-lists vs list-of-tuples (LC uses lists — mutable end).
- Overlap condition
a[0] ≤ b[1] AND b[0] ≤ a[1]; after sorting by start, the second clause is automatic.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given closed intervals, fuse all that touch or overlap; return the disjoint cover.”
Step 2 — Clarify:
- “Closed intervals — do
[1, 2]and[2, 3]merge?” (LC 56: yes — closed, touching counts.) - “May intervals be empty (
start > end)?” (LC: no.) - “Output sorted by start?” (Implicit yes; the sort gives it for free.)
- “May I mutate the input?” (LC: usually yes.)
- “Range of start/end?” (LC:
[0, 10^4]; can be larger in production.)
Step 3 — Constraints: n ≤ 10^4 → O(n log n) trivially fits.
Step 4 — Examples:
[[1,3],[2,6],[8,10],[15,18]] → [[1,6],[8,10],[15,18]].[[1,4],[4,5]] → [[1,5]](touching merge).[[1,4],[2,3]] → [[1,4]](contained).
Step 5 — Brute force: Repeatedly scan pairs and merge any overlapping; halt when no merges happen. O(n²) per scan, O(n) scans → O(n³). Terrible but illustrates the goal.
Step 6 — Complexity: Sort + sweep O(n log n). Lower bound is Ω(n log n) by reduction from sorting (intervals [i, i] for distinct i reduce to sorting).
Step 7 — Pattern: Sort + linear sweep. Sibling problems: LC 57 Insert Interval (pre-sorted), LC 435 Non-overlapping (sort-by-end), LC 252 Meeting Rooms I (just feasibility check), LC 1288 Remove Covered Intervals.
Step 8 — Develop: see solution.py.
Step 9 — Test: empty input, single interval, all-disjoint, all-overlapping (one big merge), contained interval, touching intervals (closed-interval merge), unsorted input.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why one comparison suffices
6.1 The key invariant
Claim: After sorting by start, for the current interval [s, e], the ONLY interval in out that can possibly overlap with it is out[-1].
Proof: Suppose some earlier out[i] (with i < len(out) - 1) overlapped [s, e]. Then out[i].end ≥ s ≥ out[-1].start. But out[i] and out[-1] are both in the output, meaning they were determined to NOT overlap (or out[-1] would have been merged INTO out[i]). Contradiction.
This reduces an O(n²) “compare against everything” check to O(n) “compare against last.”
6.2 The overlap check, formalized
Two intervals A = [a, b] and B = [c, d] overlap iff max(a, c) ≤ min(b, d).
After sorting by start, A.start ≤ B.start, so max(A.start, B.start) = B.start. Overlap reduces to B.start ≤ min(A.end, B.end), i.e., B.start ≤ A.end (since B.start ≤ B.end always).
So the check is simply cur.start ≤ prev.end. Note ≤ (closed) vs < (half-open). Different intervals semantics → different operator.
6.3 The merge rule — the off-by-one trap
out[-1][1] = max(out[-1][1], cur[1]) # CORRECT
out[-1][1] = cur[1] # WRONG — fails when cur is fully contained
Example: [[1, 10], [2, 3]] after sort. Without max, the merged end becomes 3 instead of 10.
max(prev_end, cur_end) is correct ALWAYS; the simpler form is wrong for the “contained” case.
6.4 In-place output and Python list aliasing
out.append([s, e])
out[-1][1] = max(...)
You’re mutating the list in out. If the caller cares about the input not being modified, ensure you out.append([s, e]) rather than out.append(intervals[i]) — otherwise you mutate the input’s nested list.
In production, prefer immutable tuples and rebuild rather than mutate. The cost is one extra allocation; the gain is no aliasing footguns.
6.5 Stability and tie-breaking
What if two intervals have the same start? [[1, 4], [1, 5]] → [[1, 5]]. Either comes first after a stable sort (Python’s sort IS stable). Doesn’t matter for merging because both will merge into one result; max(prev_end, cur_end) handles it.
Adding key=lambda x: (x[0], x[1]) to the sort doesn’t change correctness; mention only if asked.
6.6 Stream variant
What if intervals arrive one at a time and you want to maintain the merged set online?
- Sorted structure (
SortedList/ BBST): find the position by start in O(log n); check neighbors; merge if needed; remove subsumed intervals. Amortized O(log n) per insert. - Interval tree: O(log n) per query/insert; supports range overlap queries.
- Naïve list + linear scan: O(n) per insert; fine for small n.
LC asks for the batch version; mention the online version as a senior+ follow-up.
6.7 Half-open [s, e) semantics
In calendar systems, intervals are typically half-open: a meeting [9, 10) ends BEFORE 10, so [10, 11) can begin at 10 without conflict. The overlap check becomes cur.start < prev.end (strict).
LC 56 uses closed intervals (touch-merge). Real systems usually don’t. ASK FIRST.
7. Anti-Pattern Analysis
- Skip the sort — overlap detection becomes O(n²) per pair. Quadratic.
out[-1][1] = cur[1]withoutmax— fails on contained intervals.- Compare against EVERY element in
out— wasted work; the invariant (Section 6.1) makes one comparison enough. - Sort by
(start, end)— adds nothing; sort by start alone is correct and faster (less comparator work). - Use
cur.start < prev.end(strict) when problem wants closed semantics, or≤when problem wants half-open. ASK. - Mutate input by appending
intervals[i]references. Aliasing bugs. - Use a custom comparator class for sort.
key=is faster and idiomatic in Python. - Concat-and-redo merge in a loop — O(n²) and ugly. The single-pass sweep is right.
- Build a sweep-line of
(point, +1/-1)events for THIS problem — works but overengineered; the sort+sweep is simpler and faster.
8. Skills & Takeaways
- Sort-then-sweep as the canonical pattern for interval problems.
- One-comparison-with-last invariant after sorting.
max(prev_end, cur_end)for the merge rule.- Closed vs half-open endpoint distinction (always ASK).
key=sort idiom in Python; no custom comparator needed.- Aliasing-safe output by appending fresh copies.
9. When to Move On
Move on when:
- You can write the 5-line solution in ≤ 8 min, no lookup.
- You explain WHY comparing only
out[-1]is sufficient (the invariant). - You explain the
max(prev_end, cur_end)trap. - You ASK about closed vs half-open before coding.
- Stress test on 1000 random inputs agrees with brute O(n²) merge.
10. Company Context
Meta:
- Universal phone-screen + onsite warm-up. L4/L5 must produce in ≤ 10 min, including endpoint clarification.
- Bar Raiser scorecard: “Asked about endpoint semantics; one-pass sort+sweep; explained the invariant.”
Google:
- L4/L5. Often paired with Insert Interval as a two-problem set.
- Common follow-up: “Now do this in a streaming setting.”
Amazon:
- L5/L6. Often phrased as “merge overlapping CPU job intervals” or “merge calendar events.”
Microsoft:
- L62+. Variant: “Given two sorted disjoint interval lists, compute their union” (basically merge after concat).
Bloomberg:
- Trading-hour overlap; always with closed-interval semantics.
Apple:
- Calendar app team — half-open semantics; pair with “find first free slot.”
Uber:
- Driver shift overlaps; always with timezone gotchas (ASK).
Airbnb:
- Booking conflict detection.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Clarification | Codes immediately | Asks about ordering | Asks endpoint semantics | + asks streaming/batch, timezone, integer/float |
| Sort key | (start, end) “for safety” | start only | + explains why second key irrelevant | + comparator complexity O(1) matters at scale |
| Merge rule | out[-1][1] = cur[1] (BUG) | max(...) correctly | + explains contained case | + immutable rebuild for aliasing safety |
| Complexity | “It’s fast” | O(n log n) | + notes Ω(n log n) lower bound | + analyzes constant factors / cache |
| Production | Silent | Silent | Mentions streaming variant | + interval tree, R-tree, segment-tree, geospatial generalization |
12. Level Delta
Mid (L4):
- Sort + sweep with
max(prev_end, cur_end). - Handles closed-interval merging (
≤). - Single-pass after sort.
Senior (L5):
- ASKs endpoint semantics before coding.
- Articulates the “compare with last only” invariant.
- Mentions the streaming variant (SortedList).
Staff (L6):
- Discusses interval trees and segment trees for online queries.
- Discusses immutable rebuild vs in-place mutation trade.
- Generalizes to 2D (rectangle union — much harder, related to Klee’s algorithm).
Principal (L7+):
- Discusses R-tree / geospatial indices for high-dimensional interval-like queries.
- Discusses distributed interval merging: partition the timeline, merge per-partition, stitch at boundaries (boundary intervals are the hard part).
- Discusses memory-bounded streaming with bounded out-of-order tolerance (watermarks).
- Discusses lock-free interval data structures for concurrent calendar systems.
13. Follow-up Q&A
Q1: Streaming variant — intervals arrive online?
A: SortedList of intervals keyed by start; on insert, find neighbors via bisect; merge with both if overlapping; remove subsumed. O(log n + k) per insert where k = intervals absorbed. Or maintain an interval tree.
Q2: 2D — merge overlapping RECTANGLES? A: Much harder. Naïve approach: sweep-line on x-axis, maintain set of active y-intervals (1D merge). For union area, use Klee’s algorithm O(n² log n) or coordinate compression + segment tree O(n log n). Mention complexity; full code is out of scope for a 45-min interview.
Q3: Half-open intervals — does the code change?
A: Yes — change <= to < in the overlap check. [1, 2) and [2, 3) no longer merge.
Q4: Find the largest gap between merged intervals?
A: Merge first, then linear scan computing out[i+1].start - out[i].end. O(n log n) total.
Q5: Count of overlapping intervals at the busiest point?
A: That’s Meeting Rooms II (p49). Use sweep-line with (time, +1/-1) events, take running max.
Q6: Remove the smallest set of intervals to make the rest disjoint? A: LC 435 Non-overlapping Intervals (p48). Sort by END, greedy activity-selection.
14. Solution Walkthrough
See solution.py:
merge_intervals— sort + sweep canonical answer.merge_intervals_brute— repeated-pair-merge oracle (O(n³)).merge_intervals_sweep_line—(point, +1/-1)events alternative.edge_cases()+stress_test()— 1000 random inputs; all three agree.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Two things. (1) The endpoint semantics question is the most important line of code in this function — make it a docstring comment, not a constant in your head. We had a P0 incident because a backend engineer assumed
[start, end]was half-open and a frontend engineer assumed it was closed; meetings vanished from calendars at the minute boundary. (2) The 5-line sweep is correct but FRAGILE when intervals come from different timezones or have non-UTC representations. Always normalize timestamps to UTC integers (epoch seconds or millis) at the API boundary, BEFORE the merge. Type-safety: anInterval[T]with T adatetimeis one bug away from comparing strings.”
Connections forward:
- p47 — Insert Interval (input is pre-sorted).
- p48 — Non-overlapping Intervals (greedy by END).
- p49 — Meeting Rooms II (peak active).
- p50 — Employee Free Time (multi-list merge + invert).
- LC 252 — Meeting Rooms I (just feasibility).
- LC 1288 — Remove Covered Intervals.
- LC 986 — Interval List Intersections.
- Phase 02 Lab 06 — Intervals.
- Production: calendar systems, CPU scheduling, geospatial range queries (interval tree, R-tree), distributed log compaction, event store deduplication.
Hints — p46 Merge Intervals
Hint 1. ASK first: do [1, 2] and [2, 3] merge? (LC 56: yes — closed intervals.) The answer changes one operator (<= vs <).
Hint 2. Without sorting, overlap detection is O(n²) per pair. SORT by start first. After sorting, all overlapping intervals are CONTIGUOUS in the array.
Hint 3. Sweep idea: walk the sorted array; for each interval, either extend the last-appended interval (if they overlap) or append a new one.
out = [first]
for s, e in rest:
if s <= out[-1][1]: out[-1][1] = max(out[-1][1], e)
else: out.append([s, e])
Hint 4. The max(out[-1][1], e) is critical. Naïve out[-1][1] = e FAILS on the contained case: [[1, 10], [2, 3]] would give [1, 3] instead of [1, 10].
Hint 5. Why is comparing only out[-1] (not all of out) enough? Because out is sorted with disjoint intervals: any earlier interval ended before out[-1] started, so if current doesn’t overlap out[-1], it definitely doesn’t overlap anything earlier either.
If still stuck: see solution.py.
p47 — Insert Interval
Source: LeetCode 57 · Medium · Topics: Array Companies (2024–2025 frequency): Google (very high), Meta (high), Amazon (high), Microsoft (medium-high), Bloomberg (medium), LinkedIn (medium) Loop position: Common Medium, frequently paired with p46 Merge Intervals as a two-problem set. Tests whether you exploit pre-sorted input or wastefully re-sort.
1. Quick Context
You’re given a list of non-overlapping, sorted-by-start intervals and a new interval. Insert the new interval and merge if necessary; return the resulting list.
The lazy approach: Append new to the list, re-sort, run Merge Intervals (p46). O(n log n). Works but ignores the pre-sortedness — wasted log-factor.
The right approach: Three-phase linear scan in O(n):
- Before: copy all intervals that END strictly before
new.start. - Merge: while the current interval OVERLAPS new, expand new to
[min(starts), max(ends)]and skip; finally append the expanded new. - After: copy all remaining intervals.
out = []
i, n = 0, len(intervals)
# Phase 1: strictly before
while i < n and intervals[i][1] < new[0]:
out.append(intervals[i]); i += 1
# Phase 2: merge overlapping
while i < n and intervals[i][0] <= new[1]:
new = [min(new[0], intervals[i][0]), max(new[1], intervals[i][1])]
i += 1
out.append(new)
# Phase 3: strictly after
out.extend(intervals[i:])
return out
O(n) time, O(n) output. The senior signal is recognizing that re-sorting is wasteful.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/insert-interval/
STOP. 15-min timer. Draw the three phases on paper before coding.
3. Prerequisite Concepts
- p46 Merge Intervals (the sub-procedure used by the lazy approach).
- Linear-scan iteration with index variable.
- Closed-interval overlap check
a.start ≤ b.end AND b.start ≤ a.end.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given disjoint sorted intervals and one new interval, insert + merge to maintain disjoint-sorted invariant.”
Step 2 — Clarify:
- “Are existing intervals guaranteed sorted by start AND non-overlapping?” (LC: yes.)
- “Closed-interval merge?” (LC: yes —
[1, 2]and[2, 3]merge.) - “May new interval be empty (
start > end)?” (LC: no.) - “Can I mutate input?” (LC: yes.)
Step 3 — Constraints: n ≤ 10^4. Both O(n) and O(n log n) work. The interview signals which one you reach for.
Step 4 — Examples:
intervals = [[1,3],[6,9]], new = [2,5] → [[1,5],[6,9]].intervals = [[1,2],[3,5],[6,7],[8,10],[12,16]], new = [4,8] → [[1,2],[3,10],[12,16]].intervals = [], new = [5,7] → [[5,7]].intervals = [[1,5]], new = [2,3] → [[1,5]](contained).intervals = [[1,5]], new = [6,8] → [[1,5],[6,8]](after).
Step 5 — Brute force: Append + re-sort + run merge. O(n log n).
Step 6 — Complexity:
- Lazy (append + sort + merge): O(n log n).
- Three-phase scan: O(n).
- Binary search to find phase boundaries: O(log n) for boundaries + O(k) merge where k = #merged. Worst-case O(n) but typically faster.
Step 7 — Pattern: Pre-sorted-input optimization. Whenever input is sorted, ASK if you can leverage that to drop a log factor.
Step 8 — Develop: see solution.py.
Step 9 — Test: empty list; new before all; new after all; new merges with one; new merges with many; new contained; new contains all.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — The three phases, formalized
6.1 Phase definitions
Let new = [ns, ne]. Partition the input intervals into three groups:
- B (Before):
i.end < ns— strictly before new; cannot merge. - M (Merge):
i.start ≤ ne AND i.end ≥ ns— overlaps new; must merge. - A (After):
i.start > ne— strictly after new; cannot merge.
After sorting by start (input is already sorted), these groups appear in order: B, then M, then A. The scan walks through them in sequence.
Critical detail: the boundaries depend on the overlap check. For closed intervals: i.end < ns means “strictly before”; i.end ≥ ns AND i.start ≤ ne means “overlaps.”
6.2 The merge update
While in phase M, the new interval EXPANDS:
new = [min(new[0], i[0]), max(new[1], i[1])]
Both min and max are required:
minbecause the merged interval starts at the EARLIEST of all merged starts (could be the original new or some intervals[i]).maxbecause the merged interval ends at the LATEST of all merged ends.
After phase M terminates, append new (now expanded) ONCE, then continue with phase A.
6.3 Why one-pass O(n) beats the O(n log n) lazy approach
The lazy approach treats the problem as unsorted: append, re-sort, run merge. The re-sort is n log n work to recover what we already knew. The three-phase scan exploits the existing order: each interval is visited exactly once, and we know its phase from a simple comparison.
For n = 10^4, both are fast. For n = 10^7 (in-memory database with timestamp ranges, say), the log-factor matters: 10^7 vs ~2.3 × 10^8 ops.
6.4 Binary search optimization
The boundary between B and M can be found by binary-searching for the first interval with end ≥ ns. Similarly, the boundary between M and A. Each is O(log n).
If only a few intervals merge (typical case in production), this gives O(log n) total work — a huge win when n is large and updates are dense.
import bisect
# left boundary: first interval whose end >= ns
b_end = bisect.bisect_left([i[1] for i in intervals], ns)
# right boundary: first interval whose start > ne
a_start = bisect.bisect_right([i[0] for i in intervals], ne)
In practice, building those projection lists costs O(n), so use a SortedList-of-ends or maintain dual indices. Mention this as the senior+ optimization.
6.5 Mutation vs immutability
The three-phase scan can build out as a fresh list (clean) or mutate intervals (savvier). Mutation:
- Splice out the M intervals, replace with the single expanded new.
- O(n) due to list shifts in Python; not really faster, just lower memory.
Production: prefer immutable returns. Frees the caller from aliasing concerns and pairs with type-safe interval types.
6.6 Half-open variant
For [start, end) intervals: change < to <= in phase 1 boundary (i.end <= ns means “strictly before”), and <= to < in phase 2 (i.start < ne means “overlaps”). One-liner difference, but ASK first.
6.7 Online insertion into a sorted-interval data structure
If you maintain a long-lived sorted set of intervals and insert is frequent, use a SortedList (or interval tree):
- Find insertion point with bisect: O(log n).
- Scan left and right for overlapping neighbors; merge.
- Remove subsumed; insert expanded new. Amortized O(log n + k) per insert. This is the production answer; LC asks for the batch one-shot.
7. Anti-Pattern Analysis
- Append + sort + run Merge Intervals — works, but wastes the pre-sortedness; senior interviewers downgrade.
- Skip the
mininnew[0] = ...— if new started LATER than some merged interval, you’d shrink the merged interval’s start. Wrong. - Forget
max— same issue from p46. - Forget to append
newbefore phase A — drops the merged interval entirely. - Loop with
foroverintervalsandbreakinstead ofwhilewith index — phase A becomes awkward (need to remember where you stopped). - Treat overlap as
i.end <= nsstrict-less-than when problem is closed — misses the touch-merge case. - Re-check the same interval against
newafternewhas expanded — already handled by thewhileadvancingi; only a bug if you forget to advance. - Use
extend([new])onout—append(new)is cleaner. - Returning the input list with
intervals.insert(...)— O(n) per insert AND breaks the merge logic for multi-overlap. - Computing
newlazily then forgetting to push — the “phase 2 exit point” is the most common bug site.
8. Skills & Takeaways
- Three-phase scan as a pattern for “insert into sorted disjoint structure.”
- Pre-sorted-input optimization — drop log factor by exploiting order.
min(starts), max(ends)as the universal merge formula.- Closed vs half-open boundary semantics.
- Binary search to locate phase boundaries (O(log n) senior-level optimization).
- SortedList / interval tree as the production data structure for online inserts.
9. When to Move On
Move on when:
- You write the three-phase scan in ≤ 10 min from memory.
- You explain WHY both
minandmaxare needed in the merge update. - You ASK about endpoint semantics (closed vs half-open).
- You can sketch the binary-search optimization in 2 min.
- Stress test on 1000 random inputs agrees with the append+sort+merge oracle.
10. Company Context
Google:
- L4/L5 onsite Medium; usually paired with p46.
- Bar Raiser: “Exploited pre-sorted input for O(n); didn’t re-sort.”
Meta:
- L5+ phone screen. The three-phase scan must be on-the-fly correct.
Amazon:
- “Insert a new shift into a driver’s existing schedule” — same problem, productized phrasing.
Microsoft:
- L62+. Variant: “Now do it 1000 times — maintain the structure across many inserts.” (Online variant.)
Bloomberg:
- “Add a new trading-hour exception to an existing market hours list.”
LinkedIn:
- Calendar conflict detection.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| First instinct | Append + sort + merge | Three-phase scan | + explains why pre-sortedness matters | + mentions binary-search boundaries |
| Merge update | Just max (BUG) | min AND max | + explains both | + immutable rebuild for safety |
| Edge: empty input | Crashes | Handles | + explicitly tests | + parameterizes for Optional |
| Edge: new before all / after all | Forgets phase A | Handles | + names the three phases | + symmetric reasoning |
| Production | Silent | Silent | “SortedList for online” | + interval tree, segment tree, persistent data structures |
12. Level Delta
Mid (L4):
- Three-phase scan working.
minandmaxin merge update.- Handles empty input.
Senior (L5):
- ASKs endpoint semantics.
- Articulates O(n) vs O(n log n) trade.
- Mentions binary-search boundaries as further optimization.
Staff (L6):
- Discusses SortedList / interval tree for online inserts.
- Discusses 2D extension (rectangle insert) — much harder.
- Discusses immutable persistent interval structures for transactional systems.
Principal (L7+):
- Discusses persistent / functional interval trees for time-travel queries.
- Discusses CRDT-style merge for distributed calendar systems.
- Discusses online + concurrent insert with lock-free data structures.
- Discusses bulk insert when many intervals are inserted at once: sort batch first, then sweep through the existing structure once.
13. Follow-up Q&A
Q1: What if many inserts in sequence — maintain state across calls? A: SortedList keyed by start. Per insert: bisect to find left neighbor; scan right while overlapping; remove + replace. Amortized O(log n + k) per insert, k = #merged.
Q2: Insert BULK (many new intervals at once)? A: Sort the new batch; do a merged sweep through (existing ∪ new) using two-pointer; output the union as a fresh merged list. O((n + m) log m) where m = batch size, or O(n + m) if both are pre-sorted.
Q3: Half-open intervals?
A: Switch the overlap check from <= to < at the phase boundaries.
Q4: Insert with a CAPACITY constraint (each interval covers up to k items)? A: Becomes a sweep-line accumulation problem. Harder; not a simple insert anymore.
Q5: Insert into a DISK-resident sorted interval store? A: B-tree or LSM-tree of intervals; bulk-merge strategy similar to Q2. Production: LevelDB/RocksDB with interval keys.
Q6: Delete an interval — symmetric problem? A: Find intervals overlapping the target via bisect; for each, either remove (if fully contained) or split into pieces (if it spans the boundary). O(log n + k).
14. Solution Walkthrough
See solution.py:
insert_interval— three-phase O(n) scan canonical answer.insert_interval_lazy— append + sort + merge oracle.insert_interval_binary— binary-search boundary optimization.edge_cases()+stress_test()— 1000 random inputs; all three agree.
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“Three things. (1) The three-phase scan reads beautifully because the phases are NAMED in comments. Make the comments
# Phase 1: before,# Phase 2: merge,# Phase 3: after. Future-you and reviewers immediately know what each loop does. (2) For our calendar-conflict service, we use a SortedList of(start, end)tuples with online inserts; this batch problem is a unit test against a brute oracle, not production. Don’t conflate the two. (3) If the input intervals are immutable references (shared with other code), the lazy approach mutates them via re-sort. Prefer the three-phase scan precisely because it never sorts the input.”
Connections forward:
- p46 — Merge Intervals (subroutine for the lazy approach).
- p48 — Non-overlapping Intervals.
- p49 — Meeting Rooms II.
- p50 — Employee Free Time.
- LC 715 — Range Module (online interval add/remove/query).
- LC 731 — My Calendar II (double-booking detection).
- LC 732 — My Calendar III (max active overlap).
- Production: calendar systems, scheduling, CRDT-based collaborative editing of time ranges.
Hints — p47 Insert Interval
Hint 1. The lazy approach: append new to the list, sort, run Merge Intervals. O(n log n). It works — but you’re wasting the fact that the input is ALREADY sorted and disjoint.
Hint 2. Partition existing intervals into three groups relative to new:
- Before: end < new.start — copy as-is.
- Overlap: start ≤ new.end AND end ≥ new.start — must merge with new.
- After: start > new.end — copy as-is.
Because input is sorted, these groups appear in order. Walk through them in 3 phases.
Hint 3. While processing the overlap group, EXPAND new:
new = [min(new[0], i[0]), max(new[1], i[1])]
Both min and max are needed.
Hint 4. After the overlap loop, REMEMBER to append the now-expanded new exactly once, then continue with the “after” intervals.
Hint 5. Senior-level optimization: instead of scanning to find the phase boundaries, BINARY-SEARCH for them.
- Left boundary:
bisect_left(ends, new.start). - Right boundary:
bisect_right(starts, new.end). - Merge the slice in between; rebuild.
O(log n + k) where k = #intervals merged.
If still stuck: see solution.py.
p48 — Non-overlapping Intervals
Source: LeetCode 435 · Medium · Topics: Array, Greedy, Sorting, DP Companies (2024–2025 frequency): Google (high), Meta (medium-high), Amazon (medium), Microsoft (medium), Bloomberg (medium) Loop position: Classic greedy / activity-selection. Tests whether you reach for the OPTIMAL sort key (end, not start) and can justify it.
1. Quick Context
Return the minimum number of intervals to REMOVE so that the remaining intervals are non-overlapping.
Equivalently: find the MAXIMUM number of mutually non-overlapping intervals (the keep-count); answer = n - kept.
This is the classic Activity Selection problem.
The right algorithm:
- Sort by END time (not start!).
- Greedily keep any interval whose
start ≥ prev_end; else discard (count++).
intervals.sort(key=lambda x: x[1])
removed = 0
prev_end = -inf
for s, e in intervals:
if s >= prev_end:
prev_end = e
else:
removed += 1
return removed
O(n log n) time, O(1) extra. The senior signal is justifying sort-by-end via the exchange argument (Section 6.2).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/non-overlapping-intervals/
STOP. 15-min timer. Before coding: which sort key, and WHY?
3. Prerequisite Concepts
- Greedy algorithms and exchange-argument proofs.
- Sorting with a key.
- Closed vs half-open overlap semantics — for LC 435, intervals are HALF-OPEN:
[1, 2]and[2, 3]do NOT overlap.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Find the smallest interval set whose removal leaves a disjoint set, equivalently find the largest disjoint subset.”
Step 2 — Clarify:
- “Half-open or closed? Does
[1, 2]and[2, 3]overlap?” (LC 435: half-open — touching does NOT overlap.) - “Are inputs distinct?” (LC: not necessarily; duplicates count as overlapping.)
- “Empty input → 0?” (Yes.)
- “May I mutate input?” (Yes.)
Step 3 — Constraints: n ≤ 10^5. O(n log n) is the obvious target.
Step 4 — Examples:
[[1,2],[2,3],[3,4],[1,3]] → 1(remove[1,3]).[[1,2],[1,2],[1,2]] → 2(keep one).[[1,2],[2,3]] → 0(touching, no overlap in half-open).[] → 0.
Step 5 — Brute force: Try all 2^n subsets; for each, check disjoint; track largest. O(2^n × n). Exponential.
DP alternative: Sort by start; dp[i] = 1 + max(dp[j] : j < i AND intervals[j].end ≤ intervals[i].start). O(n²) or O(n log n) with binary search. Works but heavier than greedy.
Step 6 — Complexity:
- Greedy: O(n log n) time, O(1) extra.
- DP: O(n²) naive, O(n log n) with bisect.
- Brute: O(2^n × n).
Step 7 — Pattern: Activity Selection / Weighted Interval Scheduling. Classic CLRS chapter on greedy algorithms.
Step 8 — Develop: see solution.py.
Step 9 — Test: empty, single, all-disjoint (→ 0), all-overlapping at one point (→ n-1), one nested interval, duplicates, touching pairs (→ 0).
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why sort by END (the exchange argument)
6.1 The greedy choice
Greedy: Among all remaining candidates, repeatedly KEEP the one with the earliest end time that doesn’t conflict with the previously kept. Discard others that conflict.
The single-pass sort-by-end implementation realizes this: after sorting, the first interval is automatically the earliest-ending; subsequent intervals are considered in end-order, and we keep them if compatible.
6.2 The exchange argument — proof of optimality
Claim: Sorting by END and greedily keeping any interval compatible with the previously-kept is OPTIMAL (max keep-count).
Proof: Let G = g₁, g₂, ..., g_k be the greedy’s kept sequence (sorted by end). Let OPT = o₁, o₂, ..., o_m be ANY optimal kept sequence (also sortable by end since OPT is disjoint).
Show k ≥ m by induction on the first index where they differ.
- Base:
g₁has the smallest end of any interval, sog₁.end ≤ o₁.end. Replaceo₁withg₁in OPT — still disjoint (becauseo₂.start ≥ o₁.end ≥ g₁.end), still size m. - Inductive step: Suppose
g_i.end ≤ o_i.endfori ≤ j. Greedy at stepj+1picks the earliest-ending interval compatible withg_j. Sinceo_{j+1}is compatible withg_j(becauseo_{j+1}.start ≥ o_j.end ≥ g_j.end),o_{j+1}is a CANDIDATE for greedy. Greedy picks something with end ≤o_{j+1}.end. Sog_{j+1}.end ≤ o_{j+1}.end. - Conclusion: Greedy never falls behind OPT in end-time, so it can fit at least as many intervals.
6.3 Why sort by START fails
Counterexample: [[1, 100], [2, 3], [4, 5]].
- Sort by start:
[1,100],[2,3],[4,5]. Greedy keeps[1,100]; rejects rest. Keep = 1, remove = 2. - Sort by end:
[2,3],[4,5],[1,100]. Greedy keeps[2,3],[4,5]; rejects[1,100]. Keep = 2, remove = 1.
Sort-by-start is GREEDY-WRONG because the first picked interval can swallow many future opportunities.
6.4 The half-open vs closed distinction matters HERE
For LC 435 (half-open): s >= prev_end is the keep condition.
If the problem were closed-interval (touching = overlapping): s > prev_end would be the keep condition.
ASK ALWAYS. The single-character difference flips the answer for touching intervals.
6.5 Weighted variant — DP, not greedy
If each interval has a WEIGHT and you want max-weight disjoint subset (LC 1235 Maximum Profit in Job Scheduling):
- Greedy by end FAILS — could miss a high-weight long interval.
- Solution: DP. Sort by end;
dp[i] = max(dp[i-1], weight[i] + dp[j])wherej= last interval ending ≤ start[i] (found via bisect). O(n log n).
Mention this if the interviewer asks “what if intervals have weights?”
6.6 Connection to graph coloring
If you build a graph where intervals are nodes and edges connect overlapping intervals, the maximum independent set in this “interval graph” = max non-overlapping subset. For interval graphs (a special graph class), maximum independent set is polynomial-time and equals the greedy answer. In general graphs, max independent set is NP-hard.
6.7 Online variant
If intervals arrive one at a time and you must commit (keep/discard) immediately without seeing future: this is the online interval scheduling problem. No constant-competitive deterministic algorithm exists in general; randomized strategies achieve O(log n) competitive ratio. Offline (full input visible) is the LC problem.
7. Anti-Pattern Analysis
- Sort by start — wrong (Section 6.3). Counterexample.
- Sort by length (“keep the smallest intervals first”) — wrong. Counterexample:
[[1,5],[2,3],[4,5]]— keeping[2,3]first blocks both others. - DP when greedy suffices — works (O(n log n)) but more code and more bugs.
- Use
>vs>=wrongly — half-open vs closed confusion. - Return kept count instead of removed count — read the problem.
- Compare against ALL kept instead of just the latest kept — O(n²) and unnecessary.
- Initialize
prev_end = 0— fails for negative-start intervals. Usefloat('-inf'). - Forget that sort is in-place if you need the original order later.
- Sort by
(end, start)— second key adds nothing; harmless but signals junior.
8. Skills & Takeaways
- Activity Selection / Weighted Interval Scheduling — canonical greedy problem family.
- Sort by END for unweighted; DP-sort-by-end + bisect for weighted.
- Exchange argument as the standard proof technique for greedy correctness.
- Closed vs half-open flips the comparison operator.
- Interval graph max independent set = polynomial via this greedy.
9. When to Move On
Move on when:
- You write the greedy in ≤ 8 min.
- You can EXPLAIN why sort-by-end is optimal (informal exchange argument).
- You can give a counterexample to sort-by-start.
- You know the weighted variant requires DP.
- Stress test passes on 1000 random inputs vs brute O(2^n) for small n.
10. Company Context
Google:
- L4/L5 onsite Medium; classic greedy litmus test.
- Bar Raiser: “Justified sort-by-end with exchange argument; gave counterexample to sort-by-start.”
Meta:
- L5+. Variant: “Job scheduling on a single machine.”
Amazon:
- L5. Productized as “warehouse dock scheduling.”
Microsoft:
- L62+. Often with weighted follow-up (LC 1235).
Bloomberg:
- Trading-strategy backtesting: “max non-overlapping trades.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Sort key | start (BUG) | end | + justifies | + exchange argument formally |
| Counterexample to start | “It works” | “Hmm” | Provides | + proves greedy from base principles |
| Weighted follow-up | Tries greedy | “Need DP” | DP + bisect | + Knuth’s interval algorithms |
| Online variant | Silent | Silent | Mentions | + competitive ratio analysis |
| Complexity | “Fast” | O(n log n) | + Ω(n log n) lower bound | + cache effects, parallel scheduling |
12. Level Delta
Mid (L4):
- Greedy with sort-by-end.
- Correct
>=vs>for half-open. - Counts removed (not kept).
Senior (L5):
- Counterexample to sort-by-start.
- Informal exchange-argument explanation.
- Mentions weighted variant needs DP.
Staff (L6):
- Formal exchange argument.
- Knows weighted variant = LC 1235; sketches DP.
- Discusses interval graph & max independent set connection.
Principal (L7+):
- Discusses online competitive algorithms for unseen-future variants.
- Discusses multi-machine scheduling generalization (becomes NP-hard for many machines; greedy approximation).
- Discusses integer programming formulation for variants with complex constraints.
- Discusses real-world OR systems (operating rooms, crew scheduling) and where greedy breaks.
13. Follow-up Q&A
Q1: Each interval has a WEIGHT — maximize sum of kept weights?
A: LC 1235. Sort by end; for each interval i, dp[i] = max(dp[i-1], weight[i] + dp[j]) where j = bisect_right(ends, start[i]) - 1. O(n log n).
Q2: K parallel machines — max non-overlapping per machine? A: Hungarian-style assignment; or interval graph k-coloring. For k=2, equivalent to bipartite matching. For general k, harder.
Q3: All intervals MUST be kept, minimize CONFLICTS? A: Different problem — count overlapping pairs. Use sweep-line; O(n log n).
Q4: Approximation when greedy can’t see future? A: Online competitive algorithms; randomized achieves O(log n) competitive.
Q5: Closed intervals — does the code change?
A: Change >= to >. One character.
Q6: Maximize NUMBER of intervals such that no point covered by more than k? A: Generalization. Use min-heap of end times (similar to Meeting Rooms II); if heap size > k, must skip current.
14. Solution Walkthrough
See solution.py:
erase_overlap_intervals— greedy sort-by-end. Canonical.erase_overlap_intervals_sort_by_start_WRONG— counterexample-failing version (educational).erase_overlap_intervals_dp— sort-by-end + DP (equivalent answer, slower).erase_overlap_intervals_brute— try all 2^n subsets for small n (oracle).edge_cases()+stress_test().
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“I see three patterns junior engineers get wrong with interval problems, and this code review touches all of them. (1) Sorting by
startinstead ofend— the counterexample is so common ([[1, 100], [2, 3], [4, 5]]) that I keep it in our team docs. If your greedy looks suspiciously like sort-by-start, write it on a whiteboard and try that 3-element input. (2) The half-open vs closed flip costs us about one bug per quarter in our calendar service; we now type-tag intervals asClosed[T]orHalfOpen[T]and the operators are method calls (.overlaps(other)) — no operator confusion. (3) When the requirement grows from ‘count’ to ‘weighted sum,’ DON’T try to patch the greedy. Throw it out and write the DP. Greedy doesn’t generalize.”
Connections forward:
- p46 / p47 — Merge / Insert Intervals (different invariants).
- p49 — Meeting Rooms II (count peak overlaps).
- LC 1235 — Maximum Profit Job Scheduling (weighted variant).
- LC 452 — Minimum arrows to burst balloons (same greedy structure!).
- LC 646 — Maximum length pair chain (literally the same problem in disguise).
- CLRS Chapter 16 — Greedy algorithms; activity selection.
- Production: operating-room scheduling, batch-job placement, ad-slot allocation, OR systems.
Hints — p48 Non-overlapping Intervals
Hint 1. “Min remove to make disjoint” = n - max disjoint subset size. Focus on maximizing the KEEP count.
Hint 2. Greedy. But what’s the sort key? Try [[1, 100], [2, 3], [4, 5]] mentally with sort-by-start: you’d keep [1, 100] and discard 2 others. That’s a 2-remove. Bad.
Hint 3. Sort by END time. Then keep any interval whose start >= prev_kept_end; else discard. Re-check the same input — you’ll keep [2,3] and [4,5], discard only [1,100]. 1 remove.
Hint 4. Why sort-by-end is optimal: exchange argument. The earliest-ending kept interval leaves the MOST room for future intervals. Any optimal solution can be rewritten swap-by-swap into the greedy without shrinking.
Hint 5. ASK: half-open or closed? LC 435 is HALF-OPEN — [1, 2] and [2, 3] do NOT overlap. So the keep condition is start >= prev_end (with >=, not >).
If still stuck: see solution.py.
p49 — Meeting Rooms II
Source: LeetCode 253 · Medium · Topics: Array, Two Pointers, Heap, Greedy, Sorting Companies (2024–2025 frequency): Meta (very high), Google (very high), Amazon (high), Microsoft (high), Apple (medium-high), Uber (high), LinkedIn (medium-high), Airbnb (medium) Loop position: TOP-5 most-asked Medium overall. Multiple optimal solutions; tests breadth (heap vs sweep vs two-pointer) plus depth (capacity, k=peak, online variants).
1. Quick Context
Given a list of meeting intervals, return the minimum number of rooms needed.
Equivalent: at the busiest point, how many meetings overlap?
Two classical optimal solutions, both O(n log n):
Solution A — Min-heap of end-times
- Sort meetings by start.
- Walk through; before scheduling each meeting, pop the heap WHILE its top end-time
≤current start (those rooms are free). - Push current end. Heap size = rooms currently in use.
- Answer = max heap size ever reached.
intervals.sort(key=lambda x: x[0])
heap = [] # end times
for s, e in intervals:
if heap and heap[0] <= s:
heapq.heappop(heap)
heapq.heappush(heap, e)
return len(heap)
Wait — that returns the FINAL heap size, not the peak. Need to track max. Or: pop ALL releasable rooms, then push (a) returns the right count when done correctly. See solution.py for the safe form.
Solution B — Two-pointer / sweep over starts and ends
- Sort starts and ends INDEPENDENTLY.
- Walk a pointer
sthrough starts and a pointerethrough ends. - If
starts[s] < ends[e]: a new meeting begins before an old one ends →active++,s++. - Else: a meeting ends →
e++,active--. - Track max
active.
starts = sorted(i[0] for i in intervals)
ends = sorted(i[1] for i in intervals)
s = e = active = peak = 0
while s < len(starts):
if starts[s] < ends[e]:
active += 1; peak = max(peak, active); s += 1
else:
active -= 1; e += 1
return peak
O(n log n) sort + O(n) sweep. Often the FASTEST in practice (constant factors).
Both solutions are excellent; producing both demonstrates breadth.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/meeting-rooms-ii/
STOP. 15-min timer. Can you produce BOTH solutions?
3. Prerequisite Concepts
heapq(Python min-heap).- Two-pointer sweep.
- Sweep-line / event processing.
- Half-open semantics: a meeting ending at 10 frees the room for a meeting starting at 10.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Find the maximum point-multiplicity over the union of intervals.” (Or: min chromatic number of the interval graph — for interval graphs, equal to clique number = max overlap.)
Step 2 — Clarify:
- “Half-open or closed? Does a meeting
[0, 10]and another[10, 20]need 1 room or 2?” (LC 253: half-open — 1 room.) - “All non-negative times?” (Yes.)
- “Empty input → 0 rooms?” (Yes.)
- “Are times integers or floats?” (Doesn’t change algorithm.)
Step 3 — Constraints: n ≤ 10^4. O(n log n) trivially fits.
Step 4 — Examples:
[[0,30],[5,10],[15,20]] → 2.[[7,10],[2,4]] → 1(disjoint).[[0,10],[10,20]] → 1(half-open touch).[] → 0.[[1,5],[2,6],[3,7],[4,8]] → 4(full overlap stack).
Step 5 — Brute force: For each minute from min_start to max_end, count active. O((max_end - min_start) × n). Quadratic-ish in time-range.
Step 6 — Complexity: Both heap and two-pointer are O(n log n). Sweep-line with (time, ±1) events also O(n log n).
Step 7 — Pattern: Sweep-line / interval-graph chromatic number / scheduling-with-resources.
Step 8 — Develop: see solution.py.
Step 9 — Test: empty; single meeting; all disjoint; all back-to-back (no overlap); full pyramid stack; many duplicates at one moment.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Three lenses on the same problem
6.1 Lens 1 — Interval graph chromatic number
Build a graph: each meeting = vertex; edge between two if they overlap. The min rooms = chromatic number (color each meeting; same-color meetings go in the same room since they don’t overlap).
For general graphs, chromatic number is NP-hard. For interval graphs, it equals the clique number (max set of pairwise-overlapping intervals) = max point multiplicity. This is solvable in O(n log n) via the sweep, which is what we’re doing.
This connects to graph coloring / scheduling theory; mention as senior+ insight.
6.2 Lens 2 — Sweep-line with events
Create events: (start, +1) and (end, -1). Sort by time, with -1 before +1 at the same time (half-open: end at t frees room before a new meeting starts at t).
Walk events; maintain running count; track max.
events = [(s, 1) for s, _ in intervals] + [(e, -1) for _, e in intervals]
events.sort(key=lambda x: (x[0], x[1])) # -1 sorts before +1 at same time
peak = active = 0
for _, delta in events:
active += delta
peak = max(peak, active)
This is the most general approach; generalizes to “max k overlapping” by checking active vs k.
6.3 Lens 3 — Heap as live “rooms in use”
Each heap entry = end time of a room currently in use. When a new meeting starts, check if the earliest-ending room has been freed by then (its end ≤ new start). If yes, reuse (pop). Push the new meeting’s end.
Subtle: pop one or pop all?
- Pop one at a time: works if you also track
max(len(heap))after each push. - Pop all that are releasable, then push: heap size after push = exactly current rooms in use; final max is the answer.
The careful form is in solution.py. The classic LC editorial pops at most ONE per iteration, which works because if heap[0] > start, no further pops would help anyway (no room is free).
6.4 Why heap vs two-pointer are equivalent
The two-pointer sweep is the heap algorithm in disguise:
- The
endsarray is essentially the sorted heap (since all ends are known up front). starts[s] < ends[e]↔ “no room free; need new room” (active++).- Else ↔ “earliest-ending room frees” (active–).
The two-pointer is faster by a constant factor because it avoids heap overhead, but the heap version is more general (e.g., if room capacities differ).
6.5 Generalization: K-overlap question
“How many points are covered by ≥ k intervals?” — sweep-line and accumulate ranges where active ≥ k. O(n log n).
“What’s the smallest set of intervals to remove so that no point has > k coverage?” — harder; greedy by end works with priority queue.
6.6 Online variant: streaming meetings
If meetings arrive in real time and you must commit a room immediately:
- Maintain a heap of (room_id, end_time).
- For new meeting: pop one room whose end ≤ new start (free); if available, reuse; else allocate a new room.
- Total rooms = max over time.
This is the actual production algorithm (room booking systems). The LC problem is the batch version.
6.7 Capacity per room / weighted rooms
If rooms have different sizes (e.g., 5-person vs 10-person), the problem becomes bin packing for some variants — NP-hard. For “any room can fit any meeting” it remains O(n log n).
7. Anti-Pattern Analysis
- Use
<=instead of<for “new start before old end” in the two-pointer: misses the half-open semantics, overcounts by 1 in touching cases. - Sort starts and ends together as pairs — destroys the two-pointer trick. Sort them independently.
- For the heap version, forget that pushing also counts — need to track max BEFORE the push or AFTER properly.
- Build a minute-by-minute boolean array — works only when times are small integers; awful otherwise.
- Sort by end first — wrong for the heap solution; need to process arrivals in chronological order.
- Pop ALL releasable rooms first, then check if heap is empty before pushing — correct, but if you check
len(heap)AFTER push, the answer is correct AS the running max. - Treat overlap as closed
<=when problem is half-open — common bug. - Forget empty-input edge case —
heap[0]raises IndexError; guard withif heap and heap[0] ≤ s. - Conflate “min rooms” with “max overlap” in writing but compute one and return the other — they’re equal, but verbalize precisely.
8. Skills & Takeaways
- Three lenses on the same problem: chromatic number, sweep-line, heap.
- Two-pointer trick of sorting starts and ends independently.
heapqfor “rooms in use” model.- Sweep-line with
(time, ±1)events as a universal sledgehammer for active-count problems. - Half-open vs closed changes one operator.
9. When to Move On
Move on when:
- You can write BOTH the heap and two-pointer versions in ≤ 15 min total.
- You can articulate the chromatic-number / interval-graph connection.
- You can sketch the online variant.
- Stress test on 1000 random inputs agrees with brute minute-sweep.
10. Company Context
Meta:
- Top-5 phone-screen Medium. L4/L5 must produce one optimal solution + edge-case test.
- L6: must produce BOTH solutions and discuss trade-offs.
- Bar Raiser: “Two solutions; articulated half-open semantics; mentioned sweep-line generalization.”
Google:
- L4-L6. Almost always followed by “now what if rooms have capacities?” or “what’s the longest free slot?”
Amazon:
- L5+. Productized as “warehouse loading dock allocation.”
Microsoft:
- L62+. Often paired with LC 252 Meeting Rooms I (just feasibility check).
Apple:
- Calendar team. Variant: “show me all conflicts” → sweep-line output.
Uber:
- “Maximum concurrent rides” — same problem with productized framing.
LinkedIn:
- Scheduling conflict detection.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Solutions | One, with bugs | One clean | TWO clean | Two + sweep + analysis |
| Half-open | Forgets | Handles | Asks explicitly | + recommends type-tagging |
| Equivalence | Doesn’t see | Vaguely | “They’re the same thing” | + formal interval-graph chromatic-number connection |
| Online variant | Silent | Silent | Sketches | + full production design |
| K-overlap follow-up | “I’d need a different algo” | Generalizes sweep | + complexity analysis | + lower bounds and trade-offs |
12. Level Delta
Mid (L4):
- One clean solution (heap or two-pointer).
- Handles half-open correctly.
- Empty input.
Senior (L5):
- Both solutions in interview time.
- Recognizes equivalence between approaches.
- Mentions sweep-line generalization.
Staff (L6):
- Interval-graph chromatic-number framing.
- K-overlap generalization.
- Capacity-per-room → bin-packing observation.
- Online variant sketch.
Principal (L7+):
- Discusses distributed room allocation for global meeting systems (CRDTs, conflict resolution).
- Discusses integer programming for variants with hard constraints (e.g., specific room features required).
- Discusses competitive online algorithms with adversarial schedules.
- Discusses cache-friendly sweep implementations at scale.
13. Follow-up Q&A
Q1: What if I want to know the actual ASSIGNMENT (which meeting → which room)?
A: Modify the heap to store (end_time, room_id). When you pop, that room is the one assigned to the new meeting. Maintains room_id consistency.
Q2: What’s the longest stretch of time with ZERO meetings? A: Sweep-line; track times when active drops to 0 and how long it stays there.
Q3: K-overlap — at how many points is overlap ≥ k? A: Sweep; when active crosses k upward, mark start; when it falls below k, close range; sum range lengths.
Q4: Closed intervals — does it change?
A: Yes — at the same time t, processing -1 after +1 would erroneously double-count. Sort with +1 before -1 for closed semantics.
Q5: Online — meetings arrive in real time?
A: Heap of (end, room_id). On arrival: pop one if heap[0].end ≤ start; reuse or allocate new. Total rooms = max ever allocated.
Q6: Capacity-aware (rooms have sizes; meetings have attendee counts)? A: Becomes a matching problem. Approximation: sort meetings by size desc; for each, pick smallest available room that fits. Online competitive algorithms exist; offline can be solved with min-cost flow.
14. Solution Walkthrough
See solution.py:
min_meeting_rooms_heap— sort + min-heap. O(n log n).min_meeting_rooms_two_pointer— sort starts and ends independently. O(n log n).min_meeting_rooms_sweep_line—(time, ±1)events. O(n log n).min_meeting_rooms_brute— minute-by-minute counting (oracle).edge_cases()+stress_test().
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“This is the problem we use to filter for ‘has actually shipped a calendar system.’ Three things separate the great answers from the merely correct: (1) The candidate ASKS about endpoint semantics BEFORE writing the algorithm — because in production, that decision (half-open vs closed) is made at the API contract layer, and the algorithm follows. (2) The candidate produces TWO optimal solutions and articulates the equivalence — that means they understand the problem, not just memorized one approach. (3) When I ask ‘now make it streaming,’ they don’t restart from scratch; they describe how the heap version naturally extends to online. If a candidate’s heap version is buggy on touching intervals, that tells me they’ve never actually integrated with iCal or Google Calendar APIs, because that’s the FIRST thing that breaks in production.”
Connections forward:
- p46 / p47 / p48 — Other interval problems.
- p50 — Employee Free Time (related: invert “active” to find free intervals).
- LC 252 — Meeting Rooms I (feasibility only).
- LC 1094 — Car Pooling (sweep-line with capacity).
- LC 732 — My Calendar III (online MAX active).
- LC 218 — The Skyline Problem (sweep-line + heap, harder).
- Production: meeting room allocation, car-pooling, network connection limiting, rate-limiting per resource.
Hints — p49 Meeting Rooms II
Hint 1. ASK: half-open or closed? Does [0, 10] and [10, 20] need 1 room or 2? (LC 253: half-open → 1 room.)
Hint 2. Rephrase: “min rooms” = max number of meetings overlapping at any single moment. This unlocks sweep-line thinking.
Hint 3. Heap approach. Sort by start. Walk through; maintain a min-heap of END times of rooms currently in use. Before each meeting, if heap[0] <= start, that room freed up — pop. Always push current end. Max heap size = answer.
Hint 4. Two-pointer approach. Sort starts and ends INDEPENDENTLY (not as pairs). Walk a pointer s and e; if starts[s] < ends[e] need new room (active++); else a room freed (e++, active–). Track max active.
Hint 5. Both approaches are O(n log n). The two-pointer is slightly faster in practice (no heap overhead); the heap generalizes more easily (e.g., if you need to ID which room). Produce BOTH in a senior interview.
If still stuck: see solution.py.
p50 — Employee Free Time
Source: LeetCode 759 · Hard · Topics: Heap, Sorting, Sweep Line, Greedy Companies (2024–2025 frequency): Google (high), Meta (medium-high), Amazon (medium), Microsoft (medium), Bloomberg (medium), LinkedIn (medium-high) Loop position: Senior+ Hard. Brings together the entire Intervals toolkit — merge, multi-list merge, invert. The “synthesis” question.
1. Quick Context
You’re given schedule, a list of K employees’ busy times. Each employee’s busy times are sorted and disjoint. Return the time slots that are FREE for ALL employees, sorted.
Two approaches:
Approach A — Flatten, sort, merge, invert
- Flatten all employees’ intervals into one big list.
- Sort by start.
- Merge overlapping (p46).
- The free time = gaps BETWEEN consecutive merged intervals.
O(N log N) where N = total intervals.
Approach B — K-way merge with heap, then invert on-the-fly
- Min-heap of
(start, employee_idx, interval_idx), one per employee initially. - Repeatedly pop earliest start; track running max-end-so-far.
- When popped start > running max-end → emit free slot
[max_end, start). - Update max-end = max(max-end, popped end); advance that employee’s pointer.
O(N log K). Better when K << N.
Return the gaps. Note LC returns intervals as Interval objects (with start/end attrs), not lists.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/employee-free-time/
STOP. 15-min timer. Think: what’s the SIMPLEST decomposition? (Merge → invert.)
3. Prerequisite Concepts
- p46 Merge Intervals.
- p44 Merge K Sorted Lists (from Week 9) — K-way merge with heap.
- “Invert” pattern: given sorted disjoint busy intervals, output the gaps.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: “Given K sorted-disjoint interval lists, find the COMPLEMENT of their union, restricted to the time range between first and last events.”
Step 2 — Clarify:
- “Does FREE time include before the first event or after the last?” (LC: NO — only between events.)
- “Half-open or closed?” (LC: usually treated as closed-interval-style busy; a free slot is
[end_i, start_{i+1}]withend_i < start_{i+1}.) - “Can an employee have ZERO intervals?” (Edge case to ASK.)
- “Output type?” (LC: list of
Intervalobjects.)
Step 3 — Constraints: K up to ~50, N (total intervals) up to ~10^4. Both O(N log N) and O(N log K) easily fit.
Step 4 — Examples:
[[[1,2],[5,6]],[[1,3]],[[4,10]]] → [[3,4]](only gap is 3-4; before 1 and after 10 don’t count).[[[1,3],[6,7]],[[2,4]],[[2,5],[9,12]]] → [[5,6],[7,9]].
Step 5 — Brute force: Discretize the time-axis (won’t scale beyond small ranges). Or: flatten + sort + merge + invert (which is Approach A — actually the canonical answer).
Step 6 — Complexity:
- Approach A: O(N log N).
- Approach B: O(N log K).
- For K small relative to N, B is marginally better.
Step 7 — Pattern: Multi-list merge + invert. Same family as p46/p47/p49 and p44 (Merge K Sorted).
Step 8 — Develop: see solution.py.
Step 9 — Test: single employee with gaps; everyone busy continuously (no free); employees with no intervals (skip); overlapping starts; one employee fully contains others.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight
6.1 Why this is Hard despite being simple
The conceptual decomposition (merge + invert) is straightforward. What makes it Hard:
- Multi-list input (not single list). Tests whether you flatten or do a K-way merge.
- Inverting the answer is its own pattern that beginners miss.
- Edge-case discipline is high: empty lists, single employee, zero gaps.
- The
Intervaltype instead ofList[int]trips up Python solvers who assume LC-50 syntax.
6.2 The “invert” pattern, generalized
Given sorted disjoint busy intervals B = [(b1.s, b1.e), (b2.s, b2.e), ...], the free time within [B[0].s, B[-1].e] is:
[B[0].e, B[1].s](if B[0].e < B[1].s)[B[1].e, B[2].s](if B[1].e < B[2].s)- …
If you want free time on (-∞, ∞), also include (-∞, B[0].s] and [B[-1].e, ∞). Most problems clip to [B[0].s, B[-1].e].
This invert is reused EVERYWHERE: “find gaps between calendar events,” “find packet-loss windows in a connection log,” “find idle time on a CPU.”
6.3 K-way merge as the production approach
Approach A flattens ALL intervals upfront. For K = 10^6 employees with 10 intervals each, that’s 10^7 intervals to sort — expensive.
Approach B (heap of K items) merges lazily. Each pop is O(log K); total O(N log K). For K << N, big win.
This is the same pattern as Week 9 p44 Merge K Sorted Lists, generalized to intervals.
6.4 Why Approach B requires tracking max_end_so_far
In a K-way merge, intervals are popped in start-order BUT their ends can be in any order. To detect a gap, we need to know “what’s the furthest right any popped interval has extended?”
if cur.start > max_end:
emit free slot [max_end, cur.start]
max_end = max(max_end, cur.end)
This is the SAME max(prev_end, cur_end) invariant from p46 Merge Intervals — applied during a streaming merge.
6.5 Sweep-line variant
Same as p49 with (time, ±1) events. Free time = stretches where active == 0. Sort all events; walk; record ranges. O(N log N). Equivalent to Approach A.
6.6 Real-world: scheduling assistants
This is literally how “Find a Time” features work in Google Calendar, Outlook, and Calendly:
- Pull each invitee’s busy times in the query window.
- K-way merge across invitees.
- Invert to find free slots.
- Filter by duration constraint (only slots ≥ 30 min, e.g.).
Production also handles timezones, working hours, recurring events — all reduced to the same primitive.
6.7 Duration-filtered variant
“Find free slots ≥ k minutes.” After computing free slots, filter [s, e] where e - s >= k. Trivial post-processing.
“Find the FIRST free slot ≥ k minutes.” Streaming variant: do the K-way merge but emit early.
7. Anti-Pattern Analysis
- Discretize the time axis (one slot per minute) — works on toy inputs, blows up for real timestamps.
- Forget to clip free time to
[B[0].s, B[-1].e]— LC won’t accept infinite-prefix/suffix free time. - Naïvely treat each employee’s list separately and intersect — wrong; union (merge) is what you want, then invert.
- For K-way merge: forget
max_end_so_far— emit gaps incorrectly when an earlier-started but later-ending interval covers a “gap.” - Sort by end — wrong for the merge phase; sort by start.
- Use a list of
[start, end]lists when LC expectsIntervalobjects — runtime error. - Heap with just
startas the key — need at least(start, employee_idx, interval_idx)to disambiguate and advance. - Push all intervals upfront in K-way version — defeats the purpose; push one per employee, advance lazily.
- Forget that some employees may have zero intervals in some inputs — skip them when initializing the heap.
8. Skills & Takeaways
- Decompose into “merge then invert” — the universal pattern.
- K-way merge with heap for multi-list input (reuse from Week 9 p44).
max_end_so_farinvariant during streaming merge.- Invert pattern as a reusable primitive across calendar/scheduling problems.
- Clip to event range — semantics of “free time” excludes prefix/suffix.
9. When to Move On
Move on when:
- Approach A in ≤ 12 min from memory.
- Approach B in ≤ 20 min.
- You can explain WHY Approach B’s
max_end_so_faris needed. - You recognize this is the engine behind “Find a Time” features.
- Stress test on 200 random multi-list inputs agrees with brute discretization.
10. Company Context
Google:
- L5/L6 onsite Hard. Calendar team interviews use this as a “do you understand the production primitive?” test.
- Bar Raiser: “Decomposed into merge + invert; mentioned K-way merge for scale; ASKed about clipping semantics.”
Meta:
- L5+. Often phrased as “find common available slots for N people.”
Amazon:
- L5/L6. Productized as “find idle warehouse-dock windows.”
Microsoft:
- L62-L66. Outlook scheduling team.
Bloomberg:
- “Find market-closed windows across exchanges in different time zones.”
LinkedIn:
- Meeting-assist features; recruiting calendar tools.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Decomposition | Direct attempt | “Merge then invert” | + names both subroutines | + relates to production “Find a Time” |
| Multi-list handling | Discretize (BAD) | Flatten + sort | Heap K-way merge | + analyzes K vs N trade |
| Clip semantics | Forgets | Asks | + explains why | + discusses (-∞, ∞) vs windowed variants |
| Invariant in K-way | Drops max_end | Tracks max_end | + explains | + proves correctness |
| Production | Silent | Mentions calendar use | Sketches scheduling system | + timezones, recurrence, working hours, distributed |
12. Level Delta
Mid (L4):
- Approach A (flatten + sort + merge + invert) working.
- Handles empty employees and single employee.
- Knows the answer excludes prefix/suffix free time.
Senior (L5):
- Approach B (K-way merge with heap), with
max_end_so_farinvariant. - Explains K vs N trade-off.
- Mentions sweep-line as a third equivalent approach.
Staff (L6):
- Discusses real “Find a Time” production design (timezones, working hours, recurrence).
- Discusses duration-filtered variants.
- Discusses incremental/online versions for streaming calendar updates.
Principal (L7+):
- Discusses federated calendars (different systems, eventual consistency).
- Discusses distributed scheduling with conflict-free replicated data types.
- Discusses fairness constraints (rotating who gets prime slots in recurring meetings).
- Discusses AI scheduling assistants as a layer above this primitive.
13. Follow-up Q&A
Q1: Find free slots ≥ k minutes long?
A: After computing free intervals, filter [s, e] where e - s >= k.
Q2: Find the FIRST free slot ≥ k minutes? A: Streaming variant of Approach B; emit early and return.
Q3: Find free slots across DIFFERENT TIMEZONES? A: Normalize all intervals to UTC at the API boundary, then run the algorithm. Output back in user’s local timezone.
Q4: Online — new employee added, recompute free time? A: Two options:
- Recompute from scratch: O(N log K) — fine for small K.
- Incremental: maintain merged busy list; for new employee’s intervals, run p47 Insert Interval for each; recompute invert. O((m + n) log) where m = new intervals.
Q5: K = 10^6 employees, 10 intervals each — Approach A or B? A: B. Flatten + sort would be 10^7 log 10^7 ≈ 10^8 ops; B is 10^7 log 10^6 ≈ 2 × 10^8 ops — actually similar. Real difference: B is incremental, doesn’t need to allocate the flat array.
Q6: Find times when AT LEAST k employees are busy?
A: Sweep-line with (time, ±1) events; collect ranges where active >= k. O(N log N).
14. Solution Walkthrough
See solution.py:
Intervalclass — minimal stand-in for LC’s type.employee_free_time_flatten— Approach A. O(N log N).employee_free_time_heap— Approach B. O(N log K).employee_free_time_brute— discretize timeline (oracle).edge_cases()+stress_test().
Run: python3 solution.py.
15. Beyond the Problem
Principal-engineer code review comment:
“This is the kind of problem where the algorithm is easy and the SYSTEMS DESIGN is hard. Three lessons from running calendar infrastructure at scale: (1) Timezones. Always normalize to UTC integers at ingest; never trust client-reported times. We had a quarter-long incident where DST transitions silently shifted entire teams’ free time by an hour. (2) Recurrence. A ‘repeating meeting’ is conceptually one event but operationally hundreds; you don’t expand it eagerly. The merge-invert primitive runs over a WINDOW; expand recurrence into that window lazily. (3) The
max_end_so_farinvariant in Approach B is the kind of thing that’s obvious in interview and gets dropped in production refactors. Lock it into a method with a name (extend_busy_envelope) and a comment explaining the invariant. Future-you will thank you when someone tries to ‘optimize’ it away.”
Connections forward:
- p46 — Merge Intervals (subroutine in Approach A).
- p47 — Insert Interval (subroutine for online add-employee variant).
- p49 — Meeting Rooms II (sweep-line cousin).
- p44 — Merge K Sorted Lists (heap K-way merge template).
- LC 1229 — Meeting Scheduler (2-person version).
- LC 986 — Interval List Intersections (related but intersection, not union).
- Production: Google Calendar “Find a Time”, Outlook Scheduling Assistant, Calendly, FreeBusy queries, on-call rotation finders.
Hints — p50 Employee Free Time
Hint 1. Decompose: “free time across ALL employees” = “complement of (UNION of all busy times)”. Union of intervals = MERGE. Complement = INVERT.
Hint 2. Approach A (simplest): flatten everyone’s intervals into one big list, sort by start, run Merge Intervals (p46). The free time = gaps [merged[i-1].end, merged[i].start] where merged[i-1].end < merged[i].start.
Hint 3. Clip semantics: LC 759 excludes time BEFORE the first event and AFTER the last event. Only “between” gaps count.
Hint 4. Approach B (better for large K small per-employee): K-way merge with a min-heap of (start, employee_idx, interval_idx). Process intervals in start-order; track max_end_so_far. When the next start > max_end, emit a free slot.
Hint 5. In Approach B, why track max_end_so_far and not just the last popped end? Because intervals can be NESTED: a long interval popped early might extend far beyond several short later-popped intervals. The merged “busy envelope” is max_end_so_far.
If still stuck: see solution.py.
Week 11 — Monotonic Stack: The Next-Greater Toolkit
Theme. A monotonic stack is the canonical answer to “for each element, find the next/previous greater/smaller.” Mastering it unlocks Trapping Rain Water, Histogram, Stock Span, Sliding Window Maximum, and most “look at neighbors that satisfy a comparison” problems.
Goals
By the end of this week, you should be able to:
- Recognize the pattern in < 30 seconds. “For each
i, find the nearestj > i(orj < i) such thata[j] cmp a[i]” → monotonic stack. - Implement next-greater-element from memory in three flavors: indices, values, circular.
- Derive Largest Rectangle in Histogram from the “for each bar, find the maximal width where it is the minimum” reframing.
- Solve Trapping Rain Water with three independent techniques (two-pointer, prefix max arrays, monotonic stack) and explain the trade-offs.
- Spot the four common bugs: pushing values vs. indices, off-by-one on circular doubling, sentinel handling, strict vs. non-strict comparison.
The 7 Mental Models
| # | Name | Where it bites |
|---|---|---|
| 1 | Indices, not values | You almost always need the position to compute distance/width. Push indices, peek values via a[stack[-1]]. |
| 2 | Maintain invariant on push | “Stack is decreasing” → before pushing a[i], pop while top ≤ a[i]. The popped elements get their “answer” right now. |
| 3 | Resolve at pop, not at push | The element being POPPED is the one whose next-greater is the element being PUSHED. Write that down. |
| 4 | Sentinel zero/inf | For Histogram, append a sentinel 0 to flush the stack; for Trap, the “walls” are implicit via stack ordering. |
| 5 | Circular = double + mod | Iterate 2n indices, i % n to access. Only push real indices the first pass; second pass only pops. |
| 6 | Strict vs ≤ | LC 503 Next Greater II uses STRICT > (equal doesn’t count). LC 84 pops on ≤ to handle equal-height bars correctly. Decide and PROVE. |
| 7 | Each element pushed and popped at most once → O(n) amortized. | Be ready to defend amortized analysis at the whiteboard. |
Problem Table
| # | Problem | LC | Difficulty | Core Skill |
|---|---|---|---|---|
| p51 | Daily Temperatures | 739 | Medium | Indices + “resolve on pop” base case |
| p52 | Next Greater Element I | 496 | Easy | Precompute-then-lookup pattern |
| p53 | Next Greater Element II | 503 | Medium | Circular array via 2n iteration |
| p54 | Largest Rectangle in Histogram | 84 | Hard | “Each bar is the min over a maximal range” reframing |
| p55 | Trapping Rain Water | 42 | Hard | Three techniques: two-pointer, prefix max, monotonic stack |
Daily Schedule
- Mon: p51 Daily Temperatures. Write the template from scratch. Practice tracing on paper.
- Tue: p52 Next Greater Element I. Learn the “precompute on nums2, lookup for nums1” decomposition.
- Wed: p53 Next Greater Element II. Internalize the
2ndoubling trick. - Thu: p54 Largest Rectangle. The hardest reframing of the week. Allow extra time.
- Fri: p55 Trapping Rain Water. Solve it three ways. Time each one.
- Sat: Mixed retrieval — re-solve p51 and p54 cold. Write the next-greater template on a blank page from memory.
- Sun: Buffer + spaced review of Week 10 (Intervals).
Readiness Gate (move to Week 12 only if ALL pass)
- You can write the next-greater-element template in < 3 minutes with zero references.
- You can explain why amortized cost is O(n) and not O(n²).
- You solved Trapping Rain Water THREE ways and can articulate when each is preferable.
- You can derive Histogram from “for each bar, find the maximal width where it’s the minimum” without hints.
- You spotted at least one circular-doubling bug in your own code.
Why This Week Matters
Monotonic stack is one of three patterns (along with two-pointer and binary-search-on-answer) that show up in roughly every other Hard-tier interview at Meta, Google, and the HFT shops. Get it deep into muscle memory now — Week 12 onward (Phase 3) builds on it without re-explaining.
p51 — Daily Temperatures
Source: LeetCode 739 · Medium · Topics: Array, Stack, Monotonic Stack Companies (2024–2025 frequency): Meta (very high), Amazon (high), Google (medium-high), Bloomberg (high), Microsoft (medium), Apple (medium) Loop position: The canonical first monotonic-stack problem. If you can’t solve this cold in 5 minutes, you don’t yet “have” the pattern.
1. Quick Context
For each day’s temperature, find how many days you must wait until a warmer day. If no such future day exists, return 0 for that index.
Output is an array answer of the same length where answer[i] = distance to the next strictly-greater temperature, or 0.
The pattern: “for each i, find the next index j > i with T[j] > T[i]” — quintessential monotonic decreasing stack of indices.
stack = [] # indices, T-values are STRICTLY decreasing from bottom to top
for i, t in enumerate(T):
while stack and T[stack[-1]] < t:
j = stack.pop()
ans[j] = i - j
stack.append(i)
# anything left in stack has no warmer day; ans[*] = 0 already
INVARIANT: At every moment, T values at stack indices are strictly decreasing (top of stack is the smallest among them, but it’s also the most recent — hence “closest”). When a new t arrives that breaks the invariant (i.e., t > T[stack[-1]]), we pop and resolve.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/daily-temperatures/
STOP. 10-min timer. Solve in O(n).
3. Prerequisite Concepts
- Stack as a list;
append,pop,[-1]. - The idea of an invariant maintained across loop iterations.
- Amortized analysis: each element pushed and popped at most once → O(n) total.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: For each i, find nearest j > i with T[j] > T[i]; output j - i (or 0 if none).
Step 2 — Clarify:
- “Strict greater or
≥?” (Strict; LC 739 specifies “warmer.”) - “Range of temperatures?” (30..100 in LC. Could matter for a counting alternative.)
- “May the input be empty?” (LC says length ≥ 1, but defend gracefully.)
Step 3 — Constraints: n up to 10^5. O(n) or O(n log n) only.
Step 4 — Examples:
[73,74,75,71,69,72,76,73] → [1,1,4,2,1,1,0,0][30,40,50,60] → [1,1,1,0](strictly increasing — everyone has tomorrow as answer)[30,60,90] → [1,1,0][90,80,70] → [0,0,0](strictly decreasing — no answers)[50,50,50] → [0,0,0](equal — STRICT greater means no answer)
Step 5 — Brute force: For each i, scan j = i+1..n-1 until T[j] > T[i]. O(n²). On 10^5 → 10^10 ops → minutes.
Step 6 — Complexity target: O(n) time, O(n) space.
Step 7 — Pattern: Monotonic decreasing stack of INDICES. (Why indices? You need i - j distance.)
Step 8 — Develop: see solution.py.
Step 9 — Test: strictly increasing; strictly decreasing; all equal; single element; mixed with ties; the LC example.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why the stack works
6.1 The “waiting list” metaphor
Imagine each unresolved day is a person waiting for warmer weather. As each new day arrives:
- Anyone on the waiting list whose temperature is less than today’s gets their answer right now (today is their first warmer day).
- Today joins the waiting list.
The waiting list is naturally a monotonic decreasing stack by temperature: anyone with a higher temperature can’t yet be resolved by today (they’re still waiting for something hotter), and anyone with a lower temperature was already resolved by someone earlier.
6.2 Why “decreasing” exactly?
Suppose the waiting list contained two days j1 < j2 with T[j1] < T[j2]. Then on day j2, when we added j2 to the list, we would have resolved j1 (since T[j2] > T[j1] and j2 is greater-than-j1’s-temperature). Contradiction. Hence waiting-list temperatures are strictly decreasing.
6.3 Amortized O(n)
Each index is pushed exactly once and popped at most once. The inner while loop’s total work across the entire outer loop is bounded by total pops ≤ n. So total work = O(n).
This is the standard amortized argument; be prepared to state it.
6.4 Reverse iteration variant
Equivalent: iterate right-to-left, maintain a stack of “candidate answers” (indices of warmer days seen so far). Pop while T[stack[-1]] <= T[i] (they’re not warmer than me). Top of stack (if any) is my answer.
stack = [] # indices, T-values STRICTLY decreasing (rightmost first)
for i in range(n - 1, -1, -1):
while stack and T[stack[-1]] <= T[i]:
stack.pop()
ans[i] = stack[-1] - i if stack else 0
stack.append(i)
Same complexity. Pick whichever feels more natural; both are valid.
6.5 Counting-sort alternative for bounded values
If T is bounded (30..100 in LC), maintain a “next index of each temperature” array of size 71. For each i right-to-left, scan temperatures T[i]+1 .. 100 and take the minimum index seen. O(n × 71) = O(n) for fixed range; same asymptotic but different constant. Useful insight for “what if range is tiny?” follow-ups.
6.6 Connection to histogram problems
The stack here is the same data structure used in Largest Rectangle in Histogram (p54) and Trapping Rain Water (p55). Internalize it now — the next two problems become “just” variations.
7. Anti-Pattern Analysis
- Push values instead of indices. You lose the position; can’t compute
i - j. - Use
≤instead of<on the pop condition. With equal temperatures, you’d incorrectly resolve an earlier same-temp day as if today were warmer. (LC says warmer = strictly.) - Forget to leave
ans[i] = 0. If you initializeans = [0] * n, unresolved indices stay 0 — correct. If you initialize to[-1], you must convert later. - Use a deque/queue. Doesn’t help; you only ever touch one end.
- Sort the array. Destroys the indexing; problem becomes meaningless.
- O(n²) brute force on 10^5 input and “hope it passes.” It won’t; you’ll TLE.
- Try a binary-indexed tree or segment tree for “find next greater.” Works but is overkill at O(n log n) when O(n) is trivial.
8. Skills & Takeaways
- The monotonic stack template — internalize the push/pop invariant language.
- The “resolve at pop, not at push” mental model.
- Amortized O(n) analysis.
- Both forward and reverse iteration variants.
- Awareness that the same stack solves Histogram and Trap.
9. When to Move On
- Can write the template from scratch in < 3 min, zero references.
- Can articulate the invariant (“T-values at stack indices are strictly decreasing from bottom to top”).
- Can argue amortized O(n).
- Stress test on 1000 random arrays of size ≤ 50 agrees with brute force.
10. Company Context
Meta:
- L4 phone-screen Medium. Bar: O(n), correct strictness, name the pattern.
- Bar Raiser scorecard phrase: “Stated invariant out loud and explained amortized cost without prompting.”
Amazon:
- L5 OE / onsite. Productized: “for each customer order, days until next higher-value order.” Same algo.
Google:
- L4-L5. May follow with “what if we want previous warmer day too?” → run the stack right-to-left.
Bloomberg:
- High-frequency for terminal/data engineer roles. Variant: “stock span” (LC 901) — same pattern flipped.
Microsoft:
- L62. Often the warm-up before a Hard like Histogram.
Apple:
- iOS/server-side. Less common but appears.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Pattern recognition | “Loop and compare” | “Some kind of stack” | “Monotonic decreasing stack” | + names it in < 30s |
| Strictness | Misses | Gets it after testing | Asks up front | + connects to “ties go which way” framing |
| Amortized analysis | Says “O(n²) worst case” | Says “O(n) by hand-wave” | States amortized argument | + formal accounting |
| Reverse variant | Doesn’t know | Knows it exists | Can produce both | + explains symmetry |
| Generalization | None | Mentions LC 496 | Mentions Histogram | + sketches segment-tree alternative and its trade-off |
12. Level Delta
Mid (L4):
- Forward monotonic stack, indices,
<strict. - Correct on all examples.
- Empty input safe.
Senior (L5):
- Both forward and reverse variants.
- Articulates the invariant and amortized cost without prompting.
- Connects to LC 496 / 503 / 84.
Staff (L6):
- Discusses counting-sort variant when range is bounded.
- Discusses segment-tree alternative for “next-greater under updates” (e.g., temperatures change).
- Connects to financial use cases (stock span).
Principal (L7+):
- Discusses streaming version: temperatures arrive one at a time; can you maintain “next warmer day” with bounded memory? (Answer: not exactly; you need to keep all unresolved indices.)
- Discusses distributed version: temperatures partitioned across machines; map-reduce style with per-shard stacks plus a merge phase.
- Discusses memory bounds if T is enormous (millions): the stack can be as large as n in the worst case (strictly decreasing input), which is unavoidable for this problem.
13. Follow-up Q&A
Q1: What if we wanted the next ≥ (not strictly greater)?
A1: Change T[stack[-1]] < t → T[stack[-1]] <= t in the pop condition. The invariant becomes “strictly decreasing” → “non-increasing” — be careful with stress tests.
Q2: What if we wanted the next SMALLER day instead?
A2: Flip the comparison to >. The stack becomes monotonically increasing.
Q3: What if temperatures arrive in a stream and we want, at each step, “days since the last cooler day”?
A3: Same stack, but consult on push: the index below i after pops gives the previous cooler day’s index. (Pattern: each push-pop visits resolve TWO sides — left and right neighbors with the desired relation.)
Q4: What’s the worst-case stack size?
A4: n, achieved by a strictly decreasing input (nothing ever gets popped until the loop ends, where they all remain unresolved with answer 0).
Q5: Can we do it in O(1) extra space? A5: No, not in the general case. You need to remember at least the unresolved indices, which can be Θ(n). Counting-sort variant uses O(R) where R = range of T values — sometimes smaller than n.
Q6: How does this relate to “Stock Span” (LC 901)? A6: Stock span = “for each day, how many consecutive prior days had price ≤ today’s?” This is previous greater flipped — monotonic decreasing stack scanned from left. Same machinery.
Q7: Largest Rectangle in Histogram — same pattern?
A7: Yes. For each bar i, find the nearest left and right bars that are SHORTER (strictly). The stack resolves these. See p54.
14. Solution Walkthrough
See solution.py. Two implementations:
daily_temperatures_stack— forward, O(n).daily_temperatures_reverse— reverse iteration, O(n).daily_temperatures_brute— nested loop O(n²) oracle.
Stress test on 1000 random arrays of length ≤ 50: all three agree.
15. Beyond the Problem
Principal-engineer code review comment:
“The forward and reverse variants are mathematically equivalent, but in a production telemetry pipeline where temperature readings arrive in time order, the forward variant lets you process them as a stream and emit
ans[j]as soon asjis popped — bounded latency, no need to hold the whole array. The reverse variant requires the full array up front. Pick deliberately based on access pattern, not “which one is more elegant.”“
This kind of reasoning — same algorithm, different deployment fit — separates senior from staff.
Hints — p51 Daily Temperatures
Hint 1. Brute force is “for each i, scan right until you find a warmer day.” O(n²) on 10^5 input → TLE. You need O(n).
Hint 2. Reframe: think of each not-yet-resolved day as waiting for warmer weather. When a new day arrives, who on the waiting list just got their answer?
Hint 3. The waiting list, by construction, has STRICTLY DECREASING temperatures (anyone with a lower temp than someone earlier on the list would have already been resolved when that earlier person joined). This is your monotonic stack.
Hint 4. Store indices, not values — you need i - j to compute “days waited.” Read values via T[stack[-1]].
Hint 5. Each index is pushed and popped at most once → amortized O(n). Be ready to defend this argument: the inner while loop’s total work across the outer loop is bounded by total pops ≤ n.
If still stuck: see solution.py.
p52 — Next Greater Element I
Source: LeetCode 496 · Easy · Topics: Array, Hash Table, Stack, Monotonic Stack Companies (2024–2025 frequency): Amazon (high), Bloomberg (high), Meta (medium), Google (medium), Microsoft (medium) Loop position: Easy-tier warmup. Real signal: do you precompute on
nums2once and lookup in O(1)? Or do you naively scan for every query?
1. Quick Context
Two arrays:
nums1is a subset ofnums2. All values distinct withinnums2.- For each
nums1[i], find its next greater element innums2(look right of where it occurs innums2). Return-1if none.
The decomposition:
- Precompute
next_greater[v]for everyvinnums2(one monotonic-stack pass). - Lookup:
ans[i] = next_greater[nums1[i]](or-1).
nxt = {} # value -> next greater
stack = [] # values, monotonically decreasing
for v in nums2:
while stack and stack[-1] < v:
nxt[stack.pop()] = v
stack.append(v)
for v in stack: # no next greater
nxt[v] = -1
return [nxt[v] for v in nums1]
Time O(n + m), space O(n). One pass.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/next-greater-element-i/
STOP. 8-min timer.
3. Prerequisite Concepts
- Monotonic decreasing stack (p51).
- Dict lookup as the “lookup” half of the precompute-then-lookup pattern.
- The fact that
nums2has DISTINCT values lets us key by value rather than index.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: For each query value q in nums1, find its position in nums2, then the next strictly-greater value to the right; output -1 if none.
Step 2 — Clarify:
- “Are values in
nums2distinct?” (Yes, per LC.) - “Strict greater or
≥?” (Strict.) - “Can
nums1contain a value not innums2?” (No —nums1 ⊆ nums2.) - “Are values integers? Range?” (Yes, 0 ≤ v ≤ 10^4 typical.)
Step 3 — Constraints: Small (≤ 1000 each in LC). But O(n·m) brute would be slow on adversarial; aim for linear.
Step 4 — Examples:
nums1=[4,1,2],nums2=[1,3,4,2]→[-1,3,-1]nums1=[2,4],nums2=[1,2,3,4]→[3,-1]nums1=[1],nums2=[1]→[-1]
Step 5 — Brute: For each q, locate in nums2, scan right. O(n·m).
Step 6 — Complexity target: O(n + m).
Step 7 — Pattern: Precompute table via monotonic stack on nums2, then O(1) lookup per query in nums1.
Step 8 — Develop: see solution.py.
Step 9 — Test: subsets that miss, that hit, single-element, strictly increasing/decreasing nums2.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why “precompute once” matters
6.1 Two-phase decomposition
This problem is a textbook precompute / lookup decomposition:
- Heavy work depends only on
nums2(one pass, O(m)). - Cheap work answers each
nums1[i]query (O(1)).
If nums1 grew to a million queries and nums2 stayed at 1000, the precompute amortizes beautifully.
Anti-pattern (junior signal): “for each q in nums1, do a fresh scan/stack on nums2” — O(n·m) where the m-work is wasted. Recognize the decomposition out loud.
6.2 Why a dict (not an array)?
nums2 has distinct values; the precomputed table is keyed by value. A dict is the natural structure.
If values were bounded small ints (e.g., 0..10^4), you could use an array indexed by value — same asymptotic, slightly faster constant. Mention as a micro-optimization.
6.3 What if nums2 had duplicates?
Then “the value v” is ambiguous (multiple positions). You’d need to key by INDEX, not value, and nums1 would need to specify which occurrence — usually by passing indices.
This is exactly the setup of LC 503 (Next Greater II) when generalized.
6.4 Connection to p51 and p53
- p51 (Daily Temperatures): same monotonic stack, returns distances by INDEX.
- p52 (this): same stack, returns values via dict.
- p53 (Next Greater II): same stack, circular — handle with
2niteration.
One template, three flavors. Internalize the template.
6.5 Why decreasing (not increasing)?
We want greater as the resolution condition. The stack holds values waiting for someone greater. When a greater value arrives, it resolves them.
If we wanted smaller, we’d use a monotonic increasing stack with > pop condition.
7. Anti-Pattern Analysis
- For each
qinnums1, run a fresh stack overnums2. O(n·m). Defeats the point. - Build the stack but key by INDEX in
nums2. Works, but thennums1[i]requires a value→index map first; just key by value directly. - Use
≤in the pop condition. Distinct values mean equality never happens; harmless here but a bad habit (breaks LC 503). - Forget to default unresolved values to
-1. Iterate remaining stack at end, or initialize the dict with-1defaults. - Sort either array. Destroys positional meaning. Don’t.
- Use a
for j in range(i+1, len(nums2))inside the precompute loop. That’s just brute force. - Confuse “next greater than my value” with “next greater than my position.” They’re the same here because we lookup by value, but the FRAMING matters for harder variants.
8. Skills & Takeaways
- The precompute-then-lookup decomposition.
- Monotonic stack keyed by value when values are distinct.
- Why distinct-values is a load-bearing assumption (lets us use a dict).
- Connection to p51 / p53 — same template, three skins.
9. When to Move On
- You can write the precompute-then-lookup in < 5 min cold.
- You can explain why the dict is keyed by value (and what breaks if values weren’t distinct).
- You can name the template (“monotonic decreasing stack, resolve on pop”) in one sentence.
- Stress test on 1000 random distinct
nums2agrees with brute.
10. Company Context
Amazon:
- L4/L5 OE. Warmup problem; the real signal is whether you do O(n+m) or O(n·m).
- Bar Raiser: “Articulated the precompute/lookup decomposition without prompting.”
Bloomberg:
- High-frequency for terminal/trading-floor SWE. Variant: “for each ticker, find next higher-volume tick” — same template.
Meta:
- L4 screen. Mostly as a stepping stone to “now solve LC 503 (circular).”
Google:
- L4. Often combined with a different pattern (e.g., “now apply to a stream”).
Microsoft:
- L60-L62. Pair with LC 503 as the “harder version.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Decomposition | “Loop over queries” | “Precompute, I think” | “Precompute on nums2 once” | + explains why dict key = value |
| Pattern | “Stack” | “Monotonic stack” | “Monotonic decreasing stack of values” | + connects to p51 / p53 |
| Distinct values | Ignores | Notices when prompted | Notes it lets us use a dict | + explains what breaks without distinctness |
| Generalization | None | “Could probably use this elsewhere” | Mentions circular variant (LC 503) | + sketches both LC 503 and previous-greater symmetry |
12. Level Delta
Mid (L4):
- Correct precompute-then-lookup.
- Uses a dict.
- Defaults missing to
-1.
Senior (L5):
- Articulates decomposition.
- Notes the distinct-values dependency.
- Mentions p51/p53 connection.
Staff (L6):
- Discusses what changes if
nums2had duplicates (key by index, value→positions map). - Suggests bounded-int array optimization for small value ranges.
- Discusses streaming variant (nums2 arrives over time, queries arrive interspersed).
Principal (L7+):
- Discusses persistent monotonic stacks for the streaming variant where queries may ask “what was the next greater for value v as of time t?”
- Discusses wavelet tree for general “next greater after position p with value > v” queries with updates.
- Discusses cache locality — the stack approach has excellent locality vs. tree-based alternatives.
13. Follow-up Q&A
Q1: What if nums2 had duplicates?
A1: Key the table by INDEX. Need a value→list-of-indices map for nums1 lookups. If nums1 also has duplicates, the problem becomes ill-defined without specifying which occurrence.
Q2: What if we wanted the next SMALLER element?
A2: Flip to a monotonic INCREASING stack; pop while stack[-1] > v.
Q3: What if we wanted PREVIOUS greater (left side)?
A3: Same template, but the answer for each pushed v is stack[-1] (the value below it after pops). I.e., resolve at PUSH for previous-greater.
Q4: What if nums2 was a stream?
A4: Maintain the stack as values arrive; answer queries about already-popped values in O(1) (their answer is finalized). Queries about still-on-stack values must say “unknown yet.”
Q5: What’s the worst-case stack size?
A5: m (strictly decreasing nums2 — nothing gets popped).
Q6: How does this relate to LC 503 (Next Greater II, circular)?
A6: Iterate 2m instead of m, mod the index. First pass populates and may resolve; second pass only resolves (any still-unresolved after pass 2 truly has no answer).
14. Solution Walkthrough
See solution.py. Two implementations:
next_greater_element_stack— precompute via monotonic stack onnums2, lookup. O(n + m).next_greater_element_brute— nested loop O(n·m) oracle.
Stress test on 1000 random distinct nums2 agrees.
15. Beyond the Problem
Principal-engineer code review comment:
“The dict-keyed-by-value pattern is fine for the LC-distinct-values case, but if you copy-paste this code into a production system where values aren’t distinct (e.g., stock prices that can repeat), you’ll get silent bugs — the dict overwrite hides them. Add an
assert len(set(nums2)) == len(nums2)at the boundary, or rewrite to key by index from the start. The cost of the assertion is zero; the cost of the silent bug is hours of debugging.”
Hints — p52 Next Greater Element I
Hint 1. The naive approach — “for each q in nums1, locate it in nums2, scan right” — is O(n·m). The whole point of the problem is to recognize you can do better.
Hint 2. Reframe: decouple the work that depends only on nums2 from the work that depends on nums1. Build a table once over nums2; then each nums1[i] is an O(1) lookup.
Hint 3. The table is “for each value v in nums2, what’s its next greater value?” — and this is a monotonic-stack problem identical in shape to p51 Daily Temperatures.
Hint 4. Since nums2 has DISTINCT values, key the table by value (dict), not by index. Then ans[i] = table.get(nums1[i], -1).
Hint 5. After the loop ends, any value left on the stack has no next-greater. Either pre-default the dict to -1, or sweep the remaining stack and assign -1.
If still stuck: see solution.py.
p53 — Next Greater Element II
Source: LeetCode 503 · Medium · Topics: Array, Stack, Monotonic Stack Companies (2024–2025 frequency): Amazon (high), Meta (medium-high), Google (medium), Bloomberg (medium) Loop position: The “circular” variant of p52. Tests whether you can extend the template without reinventing it. Off-by-one bugs on the doubling are common.
1. Quick Context
A circular array nums. For each nums[i], find the next strictly-greater element going clockwise (wrapping around at the end). Return -1 if no such element exists anywhere in the array.
Output is an array of the same length, by index.
The trick: simulate two passes by iterating 2n indices and accessing with i % n. Only push real indices on the FIRST pass; the second pass exists solely to resolve items still on the stack.
n = len(nums)
ans = [-1] * n
stack = [] # indices, values STRICTLY DECREASING
for i in range(2 * n):
v = nums[i % n]
while stack and nums[stack[-1]] < v:
ans[stack.pop()] = v
if i < n:
stack.append(i)
# anything left in stack truly has no greater anywhere; ans[*] = -1 already
INVARIANT: same as p51 (indices with strictly decreasing values). The doubling guarantees every element gets a chance to be resolved by any larger element in the array.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/next-greater-element-ii/
STOP. 12-min timer.
3. Prerequisite Concepts
- Monotonic stack template (p51, p52).
- Modular arithmetic for circular array access.
- The insight: “wraparound = double the array virtually.”
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: Circular nums. For each i, find the nearest j (going clockwise) with nums[j] > nums[i]; if none anywhere, -1.
Step 2 — Clarify:
- “Strict greater?” (Yes.)
- “Are values distinct?” (Not necessarily — duplicates allowed.)
- “Return values or indices?” (LC: values.)
- “Length range?” (n ≤ 10^4.)
Step 3 — Constraints: O(n) ideal.
Step 4 — Examples:
[1,2,1] → [2,-1,2]— the third1wraps around to find the2.[1,2,3,4,3] → [2,3,4,-1,4]— the last3wraps to find4.[5,4,3,2,1] → [-1,5,5,5,5]— only the global max has-1; everyone else wraps to find it.[1,1,1] → [-1,-1,-1]— duplicates aren’t strictly greater.[3,8,4,1,2] → [8,-1,8,2,3]— last two wrap.
Step 5 — Brute force: For each i, walk j = (i+1) % n, (i+2) % n, ... for up to n-1 steps. O(n²).
Step 6 — Complexity target: O(n).
Step 7 — Pattern: Monotonic stack + circular trick (iterate 2n, push only on pass 1).
Step 8 — Develop: see solution.py.
Step 9 — Test: duplicates, all-equal, strictly descending, strictly ascending, single, the LC examples.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Why “iterate 2n, push only first n”?
6.1 Why two passes suffice
For any index i, the next greater element (if it exists) is somewhere in the window [i+1, i+1+n-1] in the doubled virtual array of length 2n. Iterating up to 2n - 1 ensures the stack has seen i and then seen every potentially-resolving position.
After 2n iterations, anything STILL on the stack genuinely has no greater element in the entire array, so -1 is correct.
6.2 Why “push only on first pass”?
If we pushed indices on the second pass too, we’d duplicate entries — index 5 would appear twice (once at iteration 5, once at 5+n). The pop logic would resolve them inconsistently.
The fix: pushing only when i < n ensures each index appears at most once on the stack, while the values read (via i % n) cycle through the full array a second time, giving every previously-pushed index a chance to be resolved.
6.3 Equivalent: stack of values only
Some implementations store values on the stack (since LC asks for values). But for circular handling and to support “track the resolving index” follow-ups, INDICES are more flexible. Practice indices.
6.4 What if the array isn’t circular?
Drop the doubling — back to p52’s logic. The two problems differ by exactly one change: iterate 2n instead of n and gate pushes behind i < n.
6.5 Connection to “longest distance to greater”
If instead of the next-greater value you wanted the distance, change ans[stack.pop()] = v to ans[j] = (i - j) % n (where j = stack.pop()). For circular distance, careful modular arithmetic is needed.
6.6 Why doubling = O(n) and not O(2n)?
Asymptotically identical. Each of the n indices is pushed at most once and popped at most once → total amortized work over 2n iterations = O(n). The factor of 2 is constant.
6.7 Why not triple? Why not n²?
After two passes, no further pops are possible: anything still on the stack has had 2n - 1 chances and been resisted by every position in the array. So a third pass would do zero useful work.
7. Anti-Pattern Analysis
- Forget
i < ngate on push → indices appear twice on the stack → wrong answers / infinite loop. - Iterate
ninstead of2n→ misses wraparound; degrades to p52 logic on circular input. - Use
<=instead of<in pop → duplicates incorrectly resolve as if larger (LC asks STRICT greater). - Push values not indices, then try to index back into nums — needs a value→position map and breaks with duplicates.
- Iterate
3nor more — no benefit, slight slowdown. - Reset stack between passes — destroys information; defeats the algorithm.
- Try to do it with
bisecton a sorted view — wrong tool; sorting destroys the “next clockwise” semantics. - Convert nums into
nums + numsand run p52 — works but uses 2× memory; the modular trick is cleaner.
8. Skills & Takeaways
- The doubling-for-circular trick generalizes to many circular-array problems.
- Why push only on first pass is critical.
- Index-on-stack vs. value-on-stack trade-off (favor index).
- Same template as p51 / p52, just
range(2n)andi % n.
9. When to Move On
- You can write it cold in < 8 min.
- You can explain the
i < npush gate and why omitting it breaks the algorithm. - You can derive distance variant from the value variant.
- Stress test on 1000 random circular arrays (with duplicates) agrees with brute O(n²).
10. Company Context
Amazon:
- L5 onsite. Usually paired with LC 496 as the “harder follow-up.”
- Bar Raiser: “Generalized the template cleanly; named the doubling trick; avoided memory-doubling alternative.”
Meta:
- L5. Variant: “what if you can query at any index repeatedly?” → precompute the answer array once, O(1) per query.
Google:
- L5. Often combined with stream processing.
Bloomberg:
- Trading: “for each tick in a circular history buffer, find next higher tick.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Recognizes p52 connection | No | “Like that one problem” | “Same template, circular” | + names the 2n trick |
| Push gate | Forgets | Adds after debugging | Anticipates | + explains why omitting breaks it |
| Memory-doubling alternative | Uses it | Mentions, dismisses | Avoids; explains modular trick | + benchmarks both |
| Distance variant | Stuck | Derives slowly | Derives quickly | + handles negative distances correctly |
12. Level Delta
Mid (L4):
- Iterates
2n, gates push, correct on duplicates and wraparound.
Senior (L5):
- Articulates the doubling trick and the push gate clearly.
- Connects to p51 / p52 explicitly.
- Mentions distance variant.
Staff (L6):
- Discusses precompute-once pattern for repeated queries.
- Discusses how to extend to “next greater within window of size k” (sliding window on circular).
- Recognizes the technique as a special case of “treat circular as 2× linear with constraints.”
Principal (L7+):
- Discusses persistent monotonic stacks for “what was the answer for index
iat simulation timet?” - Discusses streaming circular buffers (telemetry rings) and how to maintain next-greater under push/pop.
- Discusses cache behavior: 2× iteration with modular access can hurt prefetcher; alternative is
nums + numsflat copy (better locality, 2× memory). Real trade-off in tight loops.
13. Follow-up Q&A
Q1: What if I wanted the actual index, not the value?
A1: Store indices on the stack (we already do). Change ans[j] = nums[i % n] to ans[j] = i % n.
Q2: What about the next SMALLER element circularly?
A2: Flip the pop condition from < to > (stack becomes monotonically INCREASING).
Q3: What if duplicates count as “greater-or-equal”?
A3: Change pop condition to <=. Be careful: now the stack invariant is “non-increasing” (allows equals), and an arrival of equal value resolves the earlier equals.
Q4: Can we avoid the i < n gate by some other trick?
A4: Yes — break out of the loop when the stack is empty AND i >= n (no more resolutions possible). But the gate is simpler.
Q5: What’s the worst-case stack size?
A5: n, achieved by strictly decreasing input.
Q6: How would you extend to “find the K-th greater element circularly”? A6: Much harder. The monotonic stack collapses information; you’d need a different structure (e.g., persistent segment tree or merge-sort tree).
14. Solution Walkthrough
See solution.py. Three implementations:
next_greater_elements_doubled— iterate2n, push only on first pass. O(n).next_greater_elements_concat— concatenatenums + nums, run p52 logic, slice first n. O(n), 2× memory.next_greater_elements_brute— O(n²) scan-around-the-ring oracle.
Stress test on 1000 random arrays with duplicates agrees.
15. Beyond the Problem
Principal-engineer code review comment:
“The modular
i % nis conceptually clean but creates a branch the CPU branch-predictor handles fine but the prefetcher hates. In a hot loop processing millions of circular buffers (think: market-data tick history), thenums + numsconcatenation version benchmarks 1.3–1.7× faster on x86 due to linear access. Don’t optimize prematurely, but know the trade-off exists. Always measure before deciding which ‘cleaner’ code wins.”
Hints — p53 Next Greater Element II
Hint 1. “Circular” = the array wraps. After the last index, you continue searching from index 0. If you’ve gone all the way around without finding a strictly-greater value, the answer is -1.
Hint 2. Standard trick: double the array virtually by iterating from 0 to 2n - 1 and indexing with i % n. This gives every element a chance to be resolved by anything it might find in a full loop around.
Hint 3. Use the SAME monotonic decreasing stack of indices as p51/p52.
Hint 4. Critical detail: only push when i < n. Otherwise the same index would appear twice on the stack on the second pass, breaking the invariant and producing wrong answers.
Hint 5. After the 2n-iteration loop, anything STILL on the stack has had every position in the array attempt to resolve it; if none succeeded, the answer is genuinely -1 (and ans was pre-filled with -1).
If still stuck: see solution.py.
p54 — Largest Rectangle in Histogram
Source: LeetCode 84 · Hard · Topics: Array, Stack, Monotonic Stack Companies (2024–2025 frequency): Google (very high), Meta (high), Amazon (high), Microsoft (medium-high), Bloomberg (medium-high), HFT firms (high) Loop position: The classic “monotonic stack Hard.” If you can derive the algorithm (not just memorize it), you understand monotonic stacks. Direct precursor to LC 85 Maximal Rectangle.
1. Quick Context
Given heights[] (non-negative ints), find the area of the largest axis-aligned rectangle whose top sits on or below the histogram and whose bottom is on the x-axis.
The key reframing:
For each bar
i, what is the maximal-width rectangle whose height is exactlyheights[i]? Its width spans from the first bar to the LEFT that’s strictly shorter, to the first bar to the RIGHT that’s strictly shorter.
So area-with-bar-i-as-the-minimum = heights[i] * (right_smaller[i] - left_smaller[i] - 1).
The answer = max over all i.
Computing left_smaller[i] and right_smaller[i] is two monotonic-stack passes — or one with a sentinel.
Two-pass version (intuitive)
n = len(heights)
left = [-1] * n # index of previous strictly smaller (or -1 sentinel)
right = [n] * n # index of next strictly smaller (or n sentinel)
stack = [] # increasing heights
for i in range(n):
while stack and heights[stack[-1]] >= heights[i]:
stack.pop()
left[i] = stack[-1] if stack else -1
stack.append(i)
stack = []
for i in range(n - 1, -1, -1):
while stack and heights[stack[-1]] >= heights[i]:
stack.pop()
right[i] = stack[-1] if stack else n
stack.append(i)
return max(heights[i] * (right[i] - left[i] - 1) for i in range(n))
One-pass version (sentinel + resolve-on-pop)
heights = heights + [0] # sentinel forces final flush
stack = [] # indices, heights STRICTLY INCREASING
best = 0
for i, h in enumerate(heights):
while stack and heights[stack[-1]] > h:
top = stack.pop()
# width spans (prev-smaller-than-top, i) exclusive on both ends
left = stack[-1] if stack else -1
best = max(best, heights[top] * (i - left - 1))
stack.append(i)
return best
Both O(n).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/largest-rectangle-in-histogram/
STOP. 20-min timer. Don’t peek.
3. Prerequisite Concepts
- Monotonic stack (p51 / p52 / p53).
- The “for each element, find left and right neighbors satisfying a condition” pattern.
- Sentinel values (the appended
0at the end). - Resolve-on-pop semantics.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: Find max area h * w where h = min(heights[l..r]) and w = r - l + 1, over all 0 ≤ l ≤ r < n.
Step 2 — Clarify:
- “Heights ≥ 0?” (Yes.)
- “Width of each bar = 1?” (Yes.)
- “Can it be 0?” (Yes, gives 0 contribution.)
- “Empty array → 0?” (Yes.)
Step 3 — Constraints: n ≤ 10^5. O(n) or O(n log n) only.
Step 4 — Examples:
[2,1,5,6,2,3] → 10(the 5 and 6 form a 5×2 rectangle).[2,4] → 4(the 4 alone, or 2×2 = 4 also).[1] → 1.[0,0,0] → 0.[2,1,2] → 3(height 1 spans the full width).[6,7,5,2,4,5,9,3] → 16(the 4-5-9 group height 4 spans width 3 from index 4..6… actually compute carefully).
Step 5 — Brute force:
- (a) For each pair
(l, r), compute min and area. O(n²) pairs × O(n) min = O(n³). - (b) For each
i, walk left and right while≥ heights[i]. O(n²).
Step 6 — Complexity target: O(n) via monotonic stack.
Step 7 — Pattern: “For each bar as the minimum of its maximal span” + monotonic stack to find left/right smaller indices.
Step 8 — Develop: see solution.py.
Step 9 — Test: monotone up; monotone down; plateau; valleys; zeros; single bar; LC example.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — The “each bar is the min” reframing
6.1 Why “each bar as the minimum”?
Every rectangle in the histogram has a specific bar that limits its height (one of the bars whose top equals the rectangle’s top; if multiple are tied, any of them works).
So the set of all possible rectangles can be partitioned by “which bar is the limiting one.” For bar i, the largest rectangle limited by it has height heights[i] and width = max span where every bar is ≥ heights[i].
That width = right_smaller[i] - left_smaller[i] - 1 where smaller is STRICT.
Take the max over all i. Done.
6.2 Why STRICT smaller, not ≤?
If two bars have equal height, only ONE of them is the “official” limiting bar for any rectangle that spans both. Using STRICT smaller for both boundaries means: the leftmost equal-height bar “owns” all rectangles of that height. Tied bars on either side don’t truncate the span — they extend it.
Example: [2,1,5,5,2]. For the rectangle of height 5: it should span both 5s for area 5×2 = 10. If we used ≥ for the right-boundary, the right-smaller of the leftmost 5 would be the right 5 (which has the same height), giving width 1 instead of 2.
Convention used in this codebase: left_smaller uses STRICT, right_smaller uses STRICT. Equivalent: left ≤ + right < (or vice versa), but only one side may be non-strict to avoid double-counting. Be careful.
6.3 The one-pass sentinel trick
Appending 0 at the end forces the stack to flush completely (every remaining element gets popped because 0 is smaller than anything). This avoids a separate “drain” loop after the main iteration.
For the LEFT boundary: when popping top, the bar just below it on the stack (stack[-1] after pop) is the previous strictly-smaller (because the stack is monotonically INCREASING, and we popped everything ≥ heights[top] already).
For the RIGHT boundary: i is the current iteration index, which is the next-strictly-smaller for top (we only pop when heights[top] > h).
Width = i - left - 1. Area = heights[top] * width.
6.4 Why the stack is INCREASING here (vs. decreasing in p51)?
We’re popping when a smaller value arrives. The stack therefore holds increasing values (bottom to top is non-decreasing; with strict pop, it’s strictly increasing).
The mental model: “stack of candidates to be the limiting bar; each waits for someone shorter to truncate its span on the right.”
6.5 Two-pass vs one-pass — pick which?
Two-pass: more verbose but conceptually clearer; easier to debug; can be parallelized (left and right are independent).
One-pass with sentinel: shorter; “interview-friendly”; trickier to explain at the whiteboard.
For senior+ signal: start with the two-pass derivation (“here’s left_smaller, here’s right_smaller, then area is straightforward”), then optimize to one-pass at the end. Demonstrates understanding over memorization.
6.6 Divide and conquer (O(n log n) average, O(n²) worst)
Find the index of the minimum bar, compute min_height * n, then recurse on left and right halves. With a sparse-table RMQ for the min, each step is O(1) → O(n log n) average if the min sits roughly in the middle. Worst case (sorted input) is O(n²).
Useful as an alternative when the monotonic stack feels foreign. Worse asymptotically; mention for breadth.
6.7 Connection to LC 85 — Maximal Rectangle in Binary Matrix
For each row of a 0/1 matrix, compute a “heights” histogram = consecutive 1s in each column ending at this row. Run LC 84 on each row → O(rows × cols). Direct application.
6.8 Connection to p55 — Trapping Rain Water
Similar shape (mono-stack on heights), but different operation:
- Histogram: max area below.
- Trap: total water above trapped between higher bars on both sides.
Both have a monotonic stack solution, both have a two-pointer alternative for trap, both have prefix-array alternatives.
7. Anti-Pattern Analysis
stack[-1] >= hin both passes → using non-strict on both sides → double-counts tied bars or under-counts.- Forget the sentinel
0at the end of the one-pass version → final tall bars never get popped → wrong answer (often misses the global max). - Compute width as
right - leftinstead ofright - left - 1→ off-by-one, area too large byheights[top]. - Brute force O(n²) on
n = 10^5→ TLE (10^10 ops). - Store heights on the stack instead of indices — you’ll lose the position info you need for width.
- For each i, expand left and right manually — looks O(n) per bar but is O(n²) overall in the worst case (plateau).
- Use a max-heap to find tallest first — wrong tool; doesn’t model “as the limiting bar.”
- Try DP
dp[i] =something — there’s no clean DP recurrence here. The stack-based amortization is what gives us O(n). - Modify
heightsin place to add the sentinel without restoring — surprising side effect to caller; copy or append-then-pop. - Use
for i, h in enumerate(heights + [0])AND also referenceheights[i]inside —heightsis the unmodified original; subtle bug. Usehconsistently or defineH = heights + [0].
8. Skills & Takeaways
- The “each element as the minimum of its maximal range” reframing — applies far beyond this problem.
- Two-pass vs one-pass monotonic stack; sentinel trick for the one-pass form.
- Why strict-on-both-sides (or balanced strict/non-strict) avoids double-counting.
- Connection to LC 85 (binary matrix → histograms) and p55 (Trap).
- Divide-and-conquer alternative as a breadth signal.
9. When to Move On
- You can DERIVE the algorithm (not memorize) starting from the “each bar as the min” reframing in < 15 min.
- You can write the one-pass with sentinel in < 8 min cold.
- You can explain the strict-vs-non-strict trade-off precisely.
- You connect to LC 85 in one sentence.
- Stress test on 500 random arrays (sizes ≤ 30, heights 0..10) agrees with O(n³) brute.
10. Company Context
Google:
- L5/L6 onsite. Often paired with LC 85 as the follow-up.
- Bar Raiser: “Derived the algorithm from first principles; articulated strict-vs-non-strict; mentioned LC 85 generalization.”
Meta:
- L5/L6. Critical for E5+ promo packets. Often the “hard” of a 2-question onsite.
Amazon:
- L6+. Productized: “max contiguous warehouse-aisle capacity.”
Microsoft:
- L65+. Often the second question after a Medium.
Bloomberg:
- HFT/trading SWE. “Max contiguous volume window across N stocks” is the LC 85 generalization.
HFT firms (Jane Street / Citadel / Two Sigma):
- Very high. The interviewer wants to see you derive it, not regurgitate it. They will ask follow-ups about the strict-vs-non-strict edge case.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Reframing | “Try all rectangles” | “For each bar do something” | “Each bar as the min of its span” | + articulates partition argument |
| Algorithm | O(n²) brute, hopes it passes | O(n²) brute + memoization | Two-pass mono-stack, O(n) | + one-pass sentinel |
| Strict/non-strict | Doesn’t think about it | Realizes after a test | Picks correct convention upfront | + can prove either convention symmetrically |
| Sentinel | Forgets, debugs at end | Adds after failure | Includes from start; explains why | + suggests alternative (drain loop) and trade-offs |
| LC 85 connection | None | Vague | “Run this per row of cumulative heights” | + full O(rows × cols) sketch + space optimization |
12. Level Delta
Mid (L4):
- Brute O(n²) works correctly.
- Eventually arrives at mono-stack with hints.
- Handles single bar, zeros, plateau.
Senior (L5):
- Derives mono-stack from “each bar as the min.”
- One-pass with sentinel cleanly.
- Articulates strict-vs-non-strict.
- Connects to LC 85.
Staff (L6):
- Two-pass and one-pass variants with trade-off analysis.
- Divide-and-conquer + sparse-table RMQ alternative.
- LC 85 generalization with column-cumulative-heights rolling.
- Discusses parallelization (two-pass left/right independent).
Principal (L7+):
- Discusses streaming version: bars arrive one at a time; maintain “current largest rectangle” — possible but requires care; the stack must be retained.
- Discusses 3D generalization (max axis-aligned box in heightmap) — much harder, not solvable by direct extension.
- Discusses cache-oblivious analysis of the two-pass vs one-pass approaches.
- Discusses persistent stacks for “as of time t, what was the largest rectangle?” queries.
13. Follow-up Q&A
Q1: LC 85 Maximal Rectangle (binary matrix). Sketch.
A1: For each row, build heights[c] = consecutive 1s in column c ending at this row (carry from previous row: if matrix[r][c] == 1, heights[c] += 1; else heights[c] = 0). Then run LC 84 on heights. Take max over all rows. O(rows × cols).
Q2: What if bars have different widths?
A2: Then “width” isn’t right - left - 1; it becomes sum(width[l+1..r-1]). Use prefix sums of widths for O(1) range-width lookup; algorithm otherwise identical.
Q3: What’s the worst case for stack size?
A3: n (strictly increasing input — nothing gets popped until the sentinel triggers the final flush).
Q4: Can you do it in O(1) extra space? A4: Not in general. The stack can be Θ(n). Some in-place encoding tricks exist but are tricky and not worth it.
Q5: What if there are negative heights? A5: Problem definition assumes ≥ 0. With negatives, the area concept breaks (no “below” sense). Typically clarified out of scope.
Q6: How do you handle very large heights that overflow int?
A6: In Python, no issue (arbitrary precision). In C++/Java, use long long for area = heights[i] * width since 10^5 * 10^9 ≈ 10^14, overflows 32-bit.
Q7: Divide and conquer with RMQ — when is it preferable? A7: Almost never asymptotically (O(n log n) ≥ O(n)). Preferable conceptually only if you’ve already built an RMQ structure for other queries on the same array.
Q8: Can you parallelize this for huge arrays?
A8: The two-pass version: left-smaller and right-smaller passes are independent → run in parallel. Within each pass, parallelization is hard (sequential dependency on the stack). For TRULY huge n, partition into chunks, compute local left_smaller per chunk, then merge boundaries — non-trivial.
14. Solution Walkthrough
See solution.py. Four implementations:
largest_rectangle_one_pass— sentinel + monotonic increasing stack. O(n).largest_rectangle_two_pass— explicitleft_smallerandright_smallerarrays. O(n).largest_rectangle_each_bar— for each bar, expand left/right manually. O(n²) oracle.largest_rectangle_brute— all(l, r)pairs. O(n³) oracle for tiny n.
Stress test on 500 random arrays of size ≤ 20, heights 0..10. All four agree.
15. Beyond the Problem
Principal-engineer code review comment:
“The one-pass-with-sentinel version is elegant but is also the hardest to maintain — six months from now, someone will read
i - left - 1and not remember why it’s-1. The two-pass version is 3× the lines but every loop has an obvious invariant (left[i] =previous strictly-smaller). For an interview, the one-pass shows mastery. For production, prefer the two-pass with comments. The wins of cleverness rarely pay back the cost of confusion when the on-call engineer is debugging at 3 AM.”
Hints — p54 Largest Rectangle in Histogram
Hint 1. Brute force “try all (l, r) and find the min” is O(n³). “For each i, expand left and right while heights ≥ heights[i]” is O(n²) worst case. You need O(n).
Hint 2. Reframe: every rectangle has a limiting bar (the shortest within its span). Partition all rectangles by “which bar limits me.” Then for each bar i, find the maximal-width rectangle where it’s the limiting (shortest) bar.
Hint 3. That max width = (right_smaller[i] - left_smaller[i] - 1), where left_smaller[i] and right_smaller[i] are the indices of the nearest strictly-shorter bars on each side (or -1 / n if none).
Hint 4. Compute left_smaller and right_smaller each in one monotonic-stack pass. Or fuse into one pass: maintain a monotonic INCREASING stack of indices; when a smaller bar h arrives, pop and resolve — at that moment you know both boundaries.
Hint 5. One-pass trick: append a sentinel 0 to the end of heights. That forces every remaining bar to be popped (since 0 is smaller than anything), avoiding a separate drain loop. For each popped top: width = i - (stack[-1] if stack else -1) - 1, area = heights[top] * width.
If still stuck: see solution.py.
p55 — Trapping Rain Water
Source: LeetCode 42 · Hard · Topics: Array, Two Pointers, Stack, Monotonic Stack, Dynamic Programming Companies (2024–2025 frequency): Google (very high), Meta (very high), Amazon (very high), Microsoft (high), Apple (medium-high), Bloomberg (high), HFT firms (very high) Loop position: The canonical Hard interview problem. If you can solve it THREE ways and articulate when each is preferable, you’ve internalized the array-pattern toolbox.
1. Quick Context
Given an elevation map height[] (non-negative ints, bar widths 1), compute how many units of water it traps after raining.
For each column i, the water above it = min(max_left[i], max_right[i]) - height[i] (if positive, else 0), where max_left[i] / max_right[i] are the tallest bars to the left/right (inclusive of i).
Three optimal solutions (all O(n)):
Solution A — Prefix max arrays
max_left = running max of height left-to-right
max_right = running max right-to-left
total = sum(max(0, min(max_left[i], max_right[i]) - height[i]) for i in range(n))
O(n) time, O(n) space. Most intuitive.
Solution B — Two pointers
Maintain l, r, lmax, rmax. Move whichever pointer points to the shorter wall (it’s the bottleneck on that side); water at that index = wall_max - height[index].
l, r = 0, n - 1
lmax = rmax = 0
total = 0
while l < r:
if height[l] < height[r]:
lmax = max(lmax, height[l])
total += lmax - height[l]
l += 1
else:
rmax = max(rmax, height[r])
total += rmax - height[r]
r -= 1
O(n) time, O(1) space. Optimal in space.
Solution C — Monotonic stack
Maintain a stack of indices with STRICTLY DECREASING heights. When a taller bar arrives, pop the “bottom” and compute the rectangular water slice between left-of-popped (on stack) and current i.
stack = []
total = 0
for i, h in enumerate(height):
while stack and height[stack[-1]] < h:
bottom = stack.pop()
if not stack:
break
left = stack[-1]
width = i - left - 1
bounded = min(h, height[left]) - height[bottom]
total += width * bounded
stack.append(i)
O(n) time, O(n) space. Computes water in horizontal slices instead of vertical columns.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/trapping-rain-water/
STOP. 25-min timer. Aim for ALL THREE solutions.
3. Prerequisite Concepts
- Prefix-max / suffix-max arrays (p31-p33).
- Two-pointer convergence (p26).
- Monotonic stack (p51-p54).
- The “vertical column” vs “horizontal slice” reframing of area.
4. How to Approach (FRAMEWORK Steps 1–9)
Step 1 — Restate: For each column, water = min(tallest left wall, tallest right wall) - height[i] (clip at 0). Sum.
Step 2 — Clarify:
- “Heights non-negative ints?” (Yes.)
- “Length range?” (n ≤ 2×10^4 in LC.)
- “Maximum height?” (10^5 in LC; matters for overflow in C++/Java, not Python.)
- “Empty array → 0?” (Yes.)
Step 3 — Constraints: n ≤ 2×10^4. O(n²) might pass but is sloppy; aim O(n).
Step 4 — Examples:
[0,1,0,2,1,0,1,3,2,1,2,1] → 6(LC canonical).[4,2,0,3,2,5] → 9.[3,0,2,0,4] → 7.[1] → 0(single column traps no water).[3,3,3] → 0(flat top).[] → 0.[5,4,3,2,1] → 0(monotone; no right wall).[1,2,3,4,5] → 0(monotone; no left wall).
Step 5 — Brute force: For each i, scan left for max_left, right for max_right, sum. O(n²).
Step 6 — Complexity target: O(n) time. Bonus: O(1) space.
Step 7 — Pattern: Three techniques, all O(n):
- Prefix max arrays (most intuitive).
- Two pointers (best space).
- Monotonic stack (horizontal-slice mental model).
Step 8 — Develop: see solution.py.
Step 9 — Test: monotone up; monotone down; flat; valleys; multiple basins; LC examples.
5. Progressive Hints
If stuck for 5 min, open hints.md.
6. Deeper Insight — Three lenses on the same answer
6.1 Vertical-column lens (Solutions A and B)
For each column i, water height = min(max_left[i], max_right[i]) - height[i]. Sum.
Solution A (prefix-max arrays) is the literal implementation: precompute both arrays in O(n) each, then sum in O(n).
Solution B (two pointers) is the SAME idea with a clever observation: as we converge l and r, the smaller of height[l] vs height[r] is provably the bottleneck on its side. Why? Because the other side has at least one wall (the larger value) that’s at least as tall as the current l_max or r_max. So whichever side has the shorter current pointer dictates the water level at that pointer’s column.
This lets us avoid storing the prefix arrays: we maintain lmax and rmax rolling.
6.2 Horizontal-slice lens (Solution C, monotonic stack)
Instead of “how much water sits above column i,” think “what horizontal slices of water fill in between bars?”
A horizontal slice forms whenever a tall bar appears next to a previously-shorter sequence ending at another tall bar. The monotonic decreasing stack tracks the “left walls”; when a taller bar h arrives, every shorter bar previously pushed gets a slice computed:
- The popped bar = the BOTTOM of the slice.
- The bar below it on the stack = the LEFT wall.
- Current
i= the RIGHT wall. - Slice height =
min(h, height[left]) - height[bottom]. - Slice width =
i - left - 1.
Sum all slices = total water.
6.3 Why all three work / proof sketch
The two-pointer correctness rests on this lemma:
Lemma. If
height[l] ≤ height[r], thenmin(max_left[l], max_right[l]) = max_left[l].Proof. Because
height[r] ≥ height[l], andr > l, we havemax_right[l] ≥ height[r] ≥ height[l] ≥ max_left[l]once we account forlmaxbeing correct. Somax_right[l] ≥ max_left[l], somin(max_left[l], max_right[l]) = max_left[l].
The stack version is a different decomposition of the same sum — each unit of water gets counted exactly once.
The prefix-array version is the literal definition.
Three different perspectives, same answer.
6.4 Why are these all O(n)?
- Prefix arrays: two passes of n, then one sum. Trivially O(n).
- Two pointers: each iteration moves exactly one pointer; pointers can move at most n total. O(n).
- Stack: each index pushed and popped at most once. Amortized O(n).
6.5 Why is the stack version “harder to get right”?
Because the “bounded height” computation min(h, height[left]) - height[bottom] requires careful attention:
h = height[i]= current bar (right wall).height[left]= bar belowbottomon the stack (left wall).height[bottom]= the popped bar (the floor of the slice).- Bounded = the shorter wall - the floor.
Forgetting to break when the stack becomes empty (no left wall) is the #1 bug.
6.6 Memory-bandwidth comparison (real-world)
- Prefix arrays: 2 extra arrays, 3 passes over data → 5n reads / 3n writes.
- Two pointers: 0 extra arrays, 1 pass with both ends → 2n reads.
- Stack: 1 stack (up to n), 1 pass → roughly 3n reads + 2n stack ops.
Two-pointer wins on cache locality and memory bandwidth. Pick two-pointer in a hot loop.
6.7 What if heights can be floating point or negative?
- Floats: algorithms work unchanged; watch for precision when summing many tiny contributions.
- Negative heights: problem definition assumes ≥ 0 (water sits on x-axis). Negatives would mean “depressions below ground level” — different problem.
6.8 LC 407 — Trapping Rain Water II (2D)
Generalization to a 2D heightmap. Cannot use two pointers directly. Standard solution: BFS/Dijkstra from the boundary inward using a min-heap of “wall heights at the frontier.” O(mn log(mn)).
Mention as the natural generalization in senior+ discussions.
6.9 LC 11 — Container With Most Water
Different problem (only two walls, no floor of the container counts). Same two-pointer principle though.
7. Anti-Pattern Analysis
- For each
i, scan left and right for max each time — O(n²). Onn = 2×10^4it’s 4×10^8 — borderline TLE in tight Python; reliably TLE if the timer is strict. - Initialize prefix max with 0 — fine for non-negative heights; fails subtly for empty prefix on the left of index 0 (use
height[0]as the startingmax_left[0]). - In the two-pointer version, use
<=instead of<for the bottleneck check — works on most inputs but may double-count one column on equal-walls cases. Convention matters; pick one and stick. - In the stack version, forget the empty-stack check after pop — IndexError when the popped bar has no left wall.
- In the stack version, compute
width = i - bottominstead ofwidth = i - left - 1— counts the popped bottom’s column itself, wrong. - In the stack version, use
<=instead of<in the pop condition — produces zero-width or negative-width slices for tied walls; may give correct sum by accident or may double-count. - Use a DP approach
dp[i] = water at i + dp[i-1]— there’s no clean DP recurrence; you end up reinventing the prefix-array approach with extra steps. - Try to use a segment tree for range-max — works O(n log n) but adds complexity for no asymptotic gain. Don’t.
- Compute “wasted” water above the highest peak — there is none; water can only sit between walls.
- Forget the empty-input edge case — index errors.
8. Skills & Takeaways
- Three lenses on the same problem: prefix-max columns, two-pointer columns, mono-stack horizontal slices.
- Two-pointer lemma for “shorter wall is the bottleneck” — generalizes to other “min of two side bounds” problems.
- Horizontal vs vertical decomposition of area — recurring pattern in geometry / accounting.
- Memory bandwidth matters in production; two-pointer wins.
- LC 42 → LC 11 → LC 407 progression.
9. When to Move On
- You can write ALL THREE solutions cold in under 25 min total.
- You can explain the two-pointer lemma in 30 seconds.
- You can compute a worked example with the stack version step by step.
- You can sketch LC 407 (2D) at a high level.
- Stress test on 500 random arrays (size ≤ 20, heights 0..10) shows all three agree with brute O(n²).
10. Company Context
Google:
- L5/L6/L7 onsite. Easily the most-asked Hard. The bar for L6+ is all three solutions in interview time with trade-off discussion.
- Bar Raiser scorecard phrase: “Three solutions, articulated two-pointer lemma, mentioned LC 407 generalization, compared memory bandwidth.”
Meta:
- E5/E6 onsite. Often paired with a second Hard. Must produce at least two solutions for E5.
Amazon:
- L6+. The Bar Raiser favorite. Productized as “warehouse rainwater drainage capacity.”
Microsoft:
- L65+. Often the first onsite question to gauge depth.
Apple:
- Senior+ infrastructure roles. Less frequent but appears.
Bloomberg:
- HFT/trading. Productized as “max drawdown of a stock signal.” Same math, different name.
HFT firms:
- Very high. Will demand you discuss numerical precision (floats), branch prediction (two-pointer’s branch), and SIMD-friendliness (prefix arrays vectorize well).
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff/Principal |
|---|---|---|---|---|
| Solutions produced | One (brute or DP) | One optimal (usually prefix arrays) | TWO (prefix + two-pointer) | THREE + comparison |
| Two-pointer lemma | “It just works” | Vague | Formal statement | + proof + generalization |
| Stack version | Can’t derive | Memorized | Derived from scratch | + horizontal-slice mental model articulated |
| LC 407 | Doesn’t know | “I’ve heard of it” | “Heap-based BFS from boundary” | + full sketch + complexity |
| Memory bandwidth | Doesn’t mention | Vaguely “two-pointer is better” | Compares passes | + cache lines / SIMD friendliness |
12. Level Delta
Mid (L4):
- ONE correct solution (typically prefix arrays).
- Correct on LC examples and edge cases.
- O(n) time articulated.
Senior (L5):
- TWO solutions (prefix arrays + two-pointer).
- Two-pointer lemma stated clearly.
- Mentions stack version exists.
- Connects to LC 11.
Staff (L6):
- THREE solutions with derivation.
- Compares memory bandwidth / cache / space.
- Sketches LC 407 (2D) with heap-based BFS from boundary.
- Discusses streaming variant trade-offs.
Principal (L7+):
- Discusses streaming: heights arrive one at a time; can you maintain trapped water incrementally? (Hard — adding a new bar can retroactively change water in existing basins. Requires careful invariant maintenance.)
- Discusses distributed: heightmap partitioned across machines; how do you compute trapped water with bounded communication?
- Discusses persistent version: snapshot heights at time t, query trapped water as of t.
- Discusses 2D generalization (LC 407) in depth: BFS frontier with min-heap = Dijkstra-like; complexity O(mn log(mn)); why simpler approaches fail (water can flow diagonally between basins through saddle points).
- Discusses numerical precision if heights are floats with large dynamic range (summation order matters; Kahan summation for stability).
13. Follow-up Q&A
Q1: LC 407 — Trapping Rain Water II (2D heightmap). Sketch.
A1: Push all boundary cells into a min-heap (the “frontier”). Repeatedly pop the lowest-height boundary cell; for each unvisited neighbor, the water level there = max(neighbor_height, popped_height) (the wall containing it); add water = max(0, popped_height - neighbor_height); push the neighbor with its effective height. O(mn log(mn)).
Q2: What if heights can change (e.g., bar i becomes taller)?
A2: Naive: recompute O(n) per update. Smarter: depends on what queries you support. If only “what’s total water?” after each update, you can maintain it with segment trees over max_left/max_right but it’s involved. In an interview, sketch the naive and discuss complexity.
Q3: Can you compute water trapped in O(1) extra space AND O(n) time? A3: Yes — two-pointer (Solution B).
Q4: What’s the worst-case stack size in Solution C?
A4: n, achieved by a strictly decreasing input (nothing ever gets popped).
Q5: How does LC 42 relate to LC 11 (Container With Most Water)? A5: LC 11 = “find two walls that form the largest container; no floor matters.” Two-pointer principle: at each step, move the SHORTER wall inward (the longer can’t help; the shorter is the bottleneck). Same lemma in spirit. LC 42 = “every column traps water above it”; sum across all columns.
Q6: Streaming heights — can we report cumulative water after each new bar? A6: Adding a new tall bar can retroactively trap water in valleys that were “open” before. So no fully online O(1)-per-update algorithm exists in general. You can maintain the stack (Solution C) and emit slices as they’re resolved, but uncovered water requires the full history.
Q7: What if we want to RAIN k units of water and ask “where does it pool”? A7: Different problem. Use a priority queue / event-based simulation. Out of scope for LC 42.
Q8: Numerical stability — when bars are floating point with huge range? A8: Sum contributions in increasing-magnitude order (sorted) to minimize catastrophic cancellation. Or use Kahan summation. For typical interview ints, no concern.
14. Solution Walkthrough
See solution.py. Four implementations:
trap_prefix_arrays— Solution A: max_left + max_right precomputed. O(n) time, O(n) space.trap_two_pointers— Solution B: converging pointers. O(n) time, O(1) space.trap_stack— Solution C: monotonic decreasing stack, horizontal slices. O(n) time, O(n) space.trap_brute— for each i, scan left/right for max. O(n²) oracle.
Stress test on 500 random arrays (size ≤ 20, heights 0..10): all four agree.
15. Beyond the Problem
Principal-engineer code review comment:
“Choose the right tool for the right reason. In a Python interview, the two-pointer version is the elegant winner — O(1) space, smallest code, hardest for the interviewer to find bugs in. In a C++ batch job over GB-scale heightmaps, the prefix-array version vectorizes (SIMD), wins on throughput, and parallelizes trivially. In an online system where bars arrive incrementally, the stack version is the only one that lets you emit partial answers without a full re-scan. Three solutions, three deployment fits — none is universally ‘the best.’ This is the difference between knowing an algorithm and understanding it.”
Hints — p55 Trapping Rain Water
Hint 1. For each column i, water above it = min(tallest_wall_to_the_left, tallest_wall_to_the_right) - height[i] (clipped at 0). Summing this is the entire problem.
Hint 2. Brute is “for each i, scan left and right for max” — O(n²). To get O(n), precompute max_left[i] and max_right[i] arrays in two passes. Then one pass to sum.
Hint 3. Can you drop the arrays? Yes — two pointers converging from both ends. Maintain lmax and rmax. The key insight: whichever side has the SHORTER current bar is the bottleneck on its side, so water at that pointer’s column = that_side_max - height[pointer]. Move that pointer inward.
Hint 4. A third O(n) approach exists: monotonic decreasing stack. Instead of summing vertical columns, sum horizontal slices. When a tall bar arrives and pops a shorter one, the popped bar is the floor of a slice bounded by the bar below it on the stack (left wall) and the current bar (right wall).
Hint 5. Stack version pitfalls: (a) check if not stack: break AFTER popping — the popped bar may have no left wall, no slice exists. (b) width = i - left - 1 (subtract one for the popped index itself, one because endpoints are exclusive). (c) bounded_height = min(current, height[left]) - height[bottom].
If still stuck: see solution.py.
Week 12 — Foundational DP + Graphs Bridge
Pivot week. Two threads in one push:
- Foundational DP arc (p56–p61): the six canonical recurrences every senior must derive cold — House Robber, Coin Change, LIS (patience-sort O(n log n)), LCS, Edit Distance, Partition Equal Subset Sum. After this week, the vocabulary of DP is yours.
- Graphs bridge (p62–p65): the four “must-know-before-Month-3-Graphs-deep” algorithms — topological sort with order output (Course Schedule II), Dijkstra, Bellman-Ford / SSSP with edge constraint (Cheapest Flights K Stops), Union-Find (Number of Connected Components).
This is the biggest week in the track. Ten problems. Done deliberately — the reader who completes Week 12 has crossed from “pattern-aware” to “algorithm-fluent.”
Why these ten problems together?
DP and Graphs are taught as separate worlds in textbooks. They are not. Graphs are DP on a DAG. Topological sort is the evaluation order for a graph DP. Dijkstra is a DP over “shortest path so far,” relaxed via a priority queue. Bellman-Ford is the literal recurrence dist[v] = min(dist[u] + w(u,v)) evaluated V-1 times.
By interleaving the two threads in one week, you’ll see — by week’s end — that Cheapest Flights K Stops is just Coin Change on a graph (knapsack-flavored relaxation under a step bound), and LIS with patience-sort is a hidden binary-search application that lifts the curtain on why O(n log n) DPs exist at all.
Mental models you’ll internalize
- The DP recurrence ritual: state → transitions → base case → evaluation order → answer extraction. Apply it to every problem. Cold. In 60 seconds.
- Knapsack family (unbounded vs 0/1, weight as DP dimension): Coin Change is unbounded; Partition Equal Subset Sum is 0/1; both reduce to “fill a sack.”
- String-DP grid: LCS, Edit Distance —
dp[i][j]over two string prefixes; the diagonal/up/left transitions are the three universal moves. - Patience-sort LIS: the elegant O(n log n) algorithm that turns LIS into “binary search on tails.” Reveals that some DPs collapse into much faster data-structure-driven algorithms.
- Topological sort as DAG-DP evaluation order: Kahn’s BFS or DFS post-order. Both produce the order in which
dp[node]can be safely computed. - Dijkstra ≡ DP relaxation + greedy ordering: when all edge weights ≥ 0, the priority queue is “evaluate states in order of best-so-far distance.” Proof rests on the cut property.
- Bellman-Ford ≡ literal recurrence with bounded relaxation depth: the natural fit when you need “shortest path with at most K edges” — the K-stops constraint is the giveaway.
- Union-Find: amortized α(N) “merge connectivity” oracle. Connected components is the simplest application; Kruskal MST and Tarjan offline LCA are the deeper ones.
Problem table
| # | Problem | LC | Pattern | Day |
|---|---|---|---|---|
| p56 | House Robber | 198 | 1D DP, “include or skip” | Mon |
| p57 | Coin Change | 322 | Unbounded knapsack | Mon |
| p58 | Longest Increasing Subsequence | 300 | O(n²) DP + O(n log n) patience | Tue |
| p59 | Longest Common Subsequence | 1143 | 2D string DP | Tue |
| p60 | Edit Distance | 72 | 2D string DP, three transitions | Wed |
| p61 | Partition Equal Subset Sum | 416 | 0/1 knapsack reduction | Wed |
| p62 | Course Schedule II | 210 | Topological sort, Kahn + DFS | Thu |
| p63 | Network Delay Time | 743 | Dijkstra (binary heap) | Thu |
| p64 | Cheapest Flights Within K Stops | 787 | Bellman-Ford / BFS-with-state | Fri |
| p65 | Number of Connected Components | 323 | Union-Find (rank + path compression) | Fri |
Daily schedule
- Mon: p56 House Robber → p57 Coin Change. Internalize “include vs skip” and “unbounded knapsack” templates.
- Tue: p58 LIS (BOTH O(n²) and patience-sort O(n log n) — they’re the same problem; the patience-sort lens is non-obvious). Then p59 LCS as your first 2D string DP to anchor the grid pattern.
- Wed: p60 Edit Distance — the capstone of 2D string DP; three transitions = insert, delete, substitute. Then p61 Partition Equal Subset Sum — translates “subset sum = half-total” into a 0/1 knapsack.
- Thu: p62 Course Schedule II — topological order extraction with BOTH Kahn and DFS post-order. Then p63 Network Delay Time — your first Dijkstra; binary-heap relaxation; correctness from the cut property.
- Fri: p64 Cheapest Flights K Stops — Bellman-Ford or BFS-with-state; key insight: the K-edges-bound makes this NOT Dijkstra. Then p65 Number of Connected Components — Union-Find with rank + path compression.
Readiness gate
By Sunday night, you can answer cold (in under 60 seconds each):
- State the DP recurrence for Coin Change (unbounded) vs Partition Equal Subset Sum (0/1) without looking.
- Write
dp[i][j]for LCS and Edit Distance — state, transitions, base case, answer location. - Implement LIS in O(n log n) with
bisect_leftover the tails array; explain why patience-sort gives correct LIS length. - Distinguish when to use Dijkstra vs Bellman-Ford vs BFS-with-state. (Hint: edge weights, edge-count bound, negative weights.)
- Sketch Kahn’s algorithm AND the DFS-post-order variant of topological sort; explain why both yield valid orderings.
- Implement Union-Find with rank + path compression; prove (informally) why find is α(N) amortized.
If you can do all six cold, Month 3 Graphs Deep is open.
p56 — House Robber
Source: LeetCode 198 · Medium · Topics: 1D DP, Greedy-vs-DP discrimination Companies (2024–2025): Amazon (very high), Google (high), Meta (high), Microsoft (high), LinkedIn (medium) Loop position: The “first real DP” warm-up. Tests whether you can derive a recurrence cold, not whether you’ve memorized one.
1. Quick Context
Given nums[] (non-negative ints) representing money at each house in a row, you cannot rob two adjacent houses. Return the max money.
Classic “include or skip” DP: at each house i, either rob it (and add nums[i] + dp[i-2]) or skip it (and take dp[i-1]). Take the max.
dp[i] = max(dp[i-1], dp[i-2] + nums[i])
Trivially compressible to O(1) space.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/house-robber/
STOP. 15-min timer.
3. Prerequisite Concepts
- 1D DP recurrence: state, transition, base, evaluation order, answer location.
- LC 70 Climbing Stairs (the trivial sibling).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Pick a subset of indices with no two adjacent; maximize sum.
2. Clarify: All non-negative? (Yes.) Empty? (Return 0.) Single house? (Return nums[0].)
3. Constraints: n ≤ 100 on LC but the algorithm is O(n) regardless.
4. Examples: [1,2,3,1] → 4; [2,7,9,3,1] → 12; [2,1,1,2] → 4.
5. Brute force: Recursion with (i, can_take) state — exponential without memoization, O(n) with.
6. Target: O(n) time, O(1) space.
7. Pattern: 1D DP, “include vs skip.”
8. Develop: see solution.py.
9. Test: empty; single; two; alternating; all equal; strictly increasing.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The recurrence
Define dp[i] = max money robbing among houses 0..i inclusive. At house i:
- Skip: carry
dp[i-1]. - Rob: add
nums[i]todp[i-2](the most we could have robbed without using housei-1).
So dp[i] = max(dp[i-1], dp[i-2] + nums[i]). Base: dp[0] = nums[0], dp[-1] = 0.
6.2 Why greedy fails
“Just rob every even-indexed house” — fails on [2,1,1,2] (greedy = 3, optimal = 4 by robbing 0 and 3).
“Just rob the biggest, then the next biggest non-adjacent” — fails because adjacency forces a chain of corrections.
The exchange-argument intuition: a globally optimal choice at house i may require not robbing the locally biggest house. DP is the right hammer.
6.3 Space compression to O(1)
Since dp[i] depends only on dp[i-1] and dp[i-2], keep two rolling variables. This is the universal trick for 1D DP with constant recurrence depth.
6.4 Variants (Hint: they’re all the same DP)
- LC 213 House Robber II (circular row): solve twice — once excluding house 0, once excluding house
n-1; take max. Why: if you rob both ends, they’re adjacent in the circle. - LC 337 House Robber III (binary tree): DP on tree. Each node returns
(rob, skip); parent picksmax(child.rob, child.skip) + .... - LC 740 Delete and Earn: collapse to House Robber via “for each value
v, total =v * count(v); rob the array indexed byv.”
6.5 Why this problem is the perfect DP onboarding
It has the simplest non-trivial recurrence (two prior states). No grid, no string, no graph. If you can’t derive it cold, you don’t know DP — go re-read the 1D DP chapter. If you can, every harder DP in this week is “same idea, more dimensions.”
7. Anti-Pattern Analysis
- Greedy (rob every other) — wrong (counter-example above).
- Sort and pick — destroys adjacency; doesn’t even make sense.
- Memoized DFS without space compression — O(n) memory for no reason; interviewer notices.
- Off-by-one with
dp[i-2]wheni = 1— handledp[0] = nums[0],dp[1] = max(nums[0], nums[1])explicitly OR use sentinelprev2 = 0. dp[i] = dp[i-2] + nums[i](forgot themaxwithdp[i-1]) — common bug; passes some tests by luck.
8. Skills & Takeaways
- 1D DP template: state → transition → base → order → answer.
- “Include or skip” pattern: appears in subsequence problems, subset sums, knapsack.
- Recognize when greedy ≠ optimal via tiny counter-example.
9. When to Move On
- Solve cold in 7 minutes with O(1) space.
- Derive the recurrence verbally in under 30 seconds.
- Sketch LC 213 (circular) and LC 337 (tree DP) variants.
- Explain why greedy fails with a 4-element counter-example.
10. Company Context
- Amazon: L4/L5 phone screen. Common warm-up. Bar Raiser asks for O(1) space immediately.
- Google: L4 phone screen; followed by LC 213 in onsite.
- Meta: Phone screen; expect a follow-up of LC 337.
- Microsoft: L62 phone screen; classic.
- LinkedIn: Used as the first onsite problem to calibrate DP fluency.
Scorecard phrase for strong: “Derived recurrence cleanly, compressed to O(1) without prompt, sketched the tree-DP variant.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Recurrence | Memorized | Derived but hesitant | Derived in <30s | + space compression + variant generalization |
| Edge cases | Misses empty | Handles empty + single | Proactive | + variant edge cases articulated |
| Variants | Doesn’t know | Knows LC 213 | Solves LC 213 | + Tree DP version + reduction LC 740 |
12. Level Delta
- Mid (L4): 2D
dp[]array, O(n) space, correct. - Senior (L5): O(1) space; derives recurrence verbally; mentions LC 213 by name.
- Staff (L6): All above + tree-DP variant LC 337 derived live + space compression on tree version.
- Principal (L7+): All above + discusses adversarial input distributions; mentions reduction LC 740; connects “include or skip” to weighted independent set on path/tree graphs (which is what this problem literally is).
13. Follow-up Q&A
Q1: Circular row (LC 213). How?
A1: If we rob house 0, we cannot rob house n-1, so the optimal on nums[0..n-2]. If we don’t rob house 0, the optimal is on nums[1..n-1]. Take max of the two House Robber subproblems. O(n) time.
Q2: Binary tree (LC 337). How?
A2: Tree DP. For each node, return (rob, skip):
rob = node.val + left.skip + right.skipskip = max(left.rob, left.skip) + max(right.rob, right.skip)Root answer =max(root.rob, root.skip). O(n) total.
Q3: What if nums[i] can be negative?
A3: Then dp[i] = max(dp[i-1], dp[i-2] + nums[i], dp[i-2]) — skipping a negative is an option distinct from “can’t rob adjacent.” Or equivalently max(0, nums[i]) at every house since you’d never rob a negative-value house.
Q4: Streaming variant — houses arrive one at a time, what’s the max so far?
A4: Just maintain rolling prev2, prev1; emit max(prev1, prev2 + nums[i]) after each new house. O(1) per update.
Q5: K-non-adjacent (no two robbed houses within distance K)?
A5: dp[i] = max(dp[i-1], dp[i-K-1] + nums[i]). Same template, deeper history window.
14. Solution Walkthrough
See solution.py. Two functions:
rob_dp_array— O(n) time, O(n) space, full DP array (for the explanatory variant).rob_o1_space— O(n) time, O(1) space, rolling two variables.rob_brute— exponential, for stress test (n ≤ 18).
15. Beyond the Problem
Principal code review: “This problem is the smallest possible weighted-independent-set on a path graph. The fact that we solve it in O(n) one-pass is a foreshadowing of why DP on trees is also linear (you visit each node once, you compute (include, exclude) for each, you propagate). When you understand that House Robber, House Robber III, and Maximum Independent Set on a tree are the same algorithm with different graph topologies, you’ve crossed from ‘pattern matcher’ to ‘reasoner.’ That’s the hinge from L5 to L6.”
Hints — p56 House Robber
Hint 1. At each house, you face a binary choice: rob it or skip it. The constraint links your choice at i to your choice at i-1. Sounds like… DP.
Hint 2. Define dp[i] = max money robbing among houses 0..i inclusive. What are the two ways to arrive at dp[i]?
Hint 3. Either (a) you skip house i, in which case dp[i] = dp[i-1]; or (b) you rob house i, in which case dp[i] = dp[i-2] + nums[i] (the i-2 because i-1 is now off-limits).
Hint 4. Recurrence: dp[i] = max(dp[i-1], dp[i-2] + nums[i]). Base cases: dp[0] = nums[0], dp[1] = max(nums[0], nums[1]).
Hint 5. Notice dp[i] only needs dp[i-1] and dp[i-2]. Compress to two rolling variables prev1, prev2. O(n) time, O(1) space.
If stuck: see solution.py.
p57 — Coin Change
Source: LeetCode 322 · Medium · Topics: Unbounded Knapsack, 1D DP Companies (2024–2025): Amazon (very high), Google (high), Meta (high), Microsoft (high), Bloomberg (high), Stripe (medium-high) Loop position: The canonical “first hard DP.” Tests whether you handle “no solution” returns, derive unbounded recurrence cleanly, and reject the seductive-but-wrong greedy.
1. Quick Context
Given coin denominations coins[] (positive ints, no duplicates) and amount, return the minimum number of coins summing to amount, or -1 if impossible. Coins can be used unlimited times.
Classic unbounded knapsack:
dp[a] = min(dp[a - c] + 1) over all coins c where c <= a
dp[0] = 0
answer = dp[amount] if dp[amount] != INF else -1
O(amount × len(coins)) time, O(amount) space.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/coin-change/
STOP. 20-min timer.
3. Prerequisite Concepts
- 1D DP recurrence with
minover multiple transitions. - Sentinel value for “impossible” (use
float('inf')oramount + 1). - Knapsack distinction: unbounded (reuse allowed) vs 0/1 (each item once).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Cheapest way to make change, using each denomination unlimited times.
2. Clarify:
- Coins distinct? (Yes.) Positive? (Yes.)
- Amount up to 10⁴; coins up to ~12 distinct on LC.
- Amount = 0 → return 0.
- No combination possible → return -1.
3. Constraints: amount ≤ 10⁴, coins.length ≤ 12. O(amount × coins) ≈ 10⁵ ops — trivial.
4. Examples:
coins=[1,2,5], amount=11 → 3(5+5+1).coins=[2], amount=3 → -1.coins=[1], amount=0 → 0.coins=[1,3,4], amount=6 → 2(3+3, NOT 4+1+1 = 3 coins) — exposes greedy failure.
5. Brute force: Recursive — at each step pick any coin, subtract, recurse. Exponential without memoization.
6. Target: O(amount × len(coins)) time, O(amount) space.
7. Pattern: Unbounded knapsack via bottom-up DP.
8. Develop: see solution.py.
9. Test: amount=0; single coin equal to amount; impossible; greedy-trap input [1,3,4] amount=6.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why greedy fails
Greedy = “always use the largest coin ≤ remaining.” Works for canonical coin systems (US coins {1,5,10,25}). Fails for [1,3,4] amount 6: greedy gives 4+1+1 = 3 coins; optimal = 3+3 = 2 coins.
The general “is this coin system canonical?” question is a known hard problem (Pearson, 2005, polynomial-time test). For arbitrary denominations, DP is required.
6.2 The unbounded recurrence
dp[a] = 1 + min(dp[a - c] for c in coins if c <= a)
Why this works: A solution for amount a ends with some coin c. Strip that coin: you’re left with the optimal solution for a - c. Take the best over all choices of c.
The “unbounded” in unbounded knapsack means: dp[a] looks back at dp[a - c] without modifying the available-coins set — coins are reusable. Contrast with 0/1 knapsack (p61) where dp[a] looks at dp[i-1][a-w] to forbid reuse.
6.3 Two implementation flavors
Bottom-up (iterative, preferred):
dp = [INF] * (amount + 1); dp[0] = 0
for a in 1..amount:
for c in coins:
if c <= a:
dp[a] = min(dp[a], dp[a - c] + 1)
O(amount × coins). O(amount) space.
Top-down (memoized recursion): Equivalent, often slower in Python due to function call overhead. Useful when state space is sparse (it isn’t here).
6.4 BFS interpretation — “shortest path in a state graph”
Reframe: nodes = amounts 0..amount. Edges from a to a + c for each coin c. We seek the shortest path from 0 to amount. BFS works and gives the same complexity (each node enqueued once because the first time we reach it is the shortest).
visited = {0}; queue = deque([(0, 0)])
while queue:
a, steps = queue.popleft()
if a == amount: return steps
for c in coins:
nxt = a + c
if nxt <= amount and nxt not in visited:
visited.add(nxt); queue.append((nxt, steps + 1))
return -1
Same Θ(amount × coins), but the BFS framing is what unlocks Cheapest Flights K Stops (p64) and similar graph-with-state problems.
6.5 The sentinel choice
Use float('inf') or amount + 1 as “impossible” marker. The check dp[amount] > amount works because no real answer can exceed amount (worst case = amount coins of value 1, which requires coin 1 to exist).
6.6 LC 518 Coin Change II — count of combinations, not minimum
Variant: how many distinct combinations sum to amount? The recurrence is different — outer loop on COINS, inner on amount, ensuring each combination is counted once (combinations, not permutations):
dp[0] = 1
for c in coins:
for a in c..amount: dp[a] += dp[a - c]
The loop order matters: swapping it counts permutations, not combinations. Subtle and a frequent follow-up.
7. Anti-Pattern Analysis
- Greedy — fails on non-canonical denominations.
for c in coins; for a in 1..amountloop order for the minimum-coins variant — actually fine here, but be aware the loop order does matter in LC 518 (combinations).- Forgetting
dp[0] = 0— base case missed; returns -1 for amount=0 incorrectly. - Using -1 as the “impossible” sentinel in the DP — breaks
min()comparisons; use INF. - Memoization with
lru_cacheon the recursive top-down in Python without size hint — works but slower. - Returning
dp[amount]directly without the -1 check — silently emits a giant sentinel value.
8. Skills & Takeaways
- Unbounded knapsack template.
- “Min over previous-state + 1” pattern — analogous to BFS shortest path.
- Greedy-vs-DP discrimination via tiny counter-example.
- Connect DP-on-amount to “graph BFS over a state space” — bridge to p64.
9. When to Move On
- Implement bottom-up in 10 min, O(amount × coins) time, O(amount) space.
- Articulate the unbounded vs 0/1 distinction in 30 seconds.
- Solve LC 518 (combinations) and explain the loop-order subtlety.
- Sketch the BFS reformulation.
- Give a 5-element counter-example where greedy fails.
10. Company Context
- Amazon: Bar Raiser favorite. L5+ asked to compare with LC 518.
- Google: Phone screen L4/L5. Always with a follow-up — either LC 518 or “what if amount were 10⁹?” (answer: DP doesn’t scale; need ILP or denomination-specific tricks).
- Meta: E5 phone screen; expect “count combinations” follow-up.
- Microsoft: Common L62 onsite problem.
- Stripe: Productized as “minimum-units to settle a payment in a multi-currency exchange.”
- Bloomberg: “Minimum trades to balance a portfolio.”
Scorecard for strong: “Rejected greedy with counter-example, derived recurrence cleanly, articulated unbounded vs 0/1, sketched BFS framing, solved LC 518 follow-up correctly.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Greedy rejection | Tries greedy first | Tries, then DP after prompt | Spots failure with counter-example | + canonical-coin-system aside |
| Recurrence | Memorized | Derived | + correct base + INF sentinel | + alternative BFS framing |
| LC 518 follow-up | Doesn’t know | Knows, can’t solve | Solves with right loop order | + explains why loop order matters |
12. Level Delta
- Mid (L4): Bottom-up DP, correct, returns -1 properly.
- Senior (L5): + BFS framing mentioned + LC 518 solved with correct loop order.
- Staff (L6): + complexity proof + adversarial input examples + LC 322 reduction to weighted shortest-path in a DAG.
- Principal (L7+): + canonical-coin-system theorem (when greedy works), + ILP framing for huge-amount variant, + connection to FPTAS schemes for #P-hard subset-sum.
13. Follow-up Q&A
Q1: LC 518 — how many combinations?
A1: Same dp array but recurrence sums instead of minimizes. Loop order matters: outer loop on coins, inner on amount, accumulating dp[a] += dp[a - c]. Swapping the loops counts permutations.
Q2: amount = 10⁹, coins.length = 5. DP infeasible. Now what?
A2: DP space is amount-dependent; this won’t fit. Options: (a) ILP solver if you only need ≤ K coins; (b) for specific coin structures (e.g., all multiples of g = gcd(coins)), reduce by dividing by g; (c) approximate via the largest coin and remainder. Acknowledge the problem is NP-hard for arbitrary denominations (subset sum reduction).
Q3: What if some coins have negative values? A3: Pathological — chains of negatives create infinite-coin solutions. Problem becomes ill-defined unless you bound the number of coins.
Q4: Streaming: amount fixed, new coin denominations arrive over time. Re-answer after each.
A4: When a new coin c arrives, run one extra pass: for a in c..amount: dp[a] = min(dp[a], dp[a-c] + 1). O(amount) per update.
Q5: Track WHICH coins are used.
A5: Maintain parent[a] = coin_used_to_reach_a. Backtrack from amount: while a > 0: emit parent[a]; a -= parent[a].
14. Solution Walkthrough
See solution.py:
coin_change_dp— bottom-up O(amount × coins).coin_change_bfs— BFS reformulation; same complexity, different mental model.coin_change_brute— recursive without memo, used only for tiny stress (amount ≤ 12).
15. Beyond the Problem
Principal code review: “Coin Change is the most-asked DP for one reason: it sits at the intersection of greedy, DP, and graph BFS. Candidates who derive only the DP miss the point. The interviewer wants to see whether you can reformulate the same problem in three equivalent ways — and reach for the most appropriate one given the input shape. For tiny amount and a few coins, DP. For sparse reachable states, BFS. For canonical coin systems, greedy. Knowing only one of the three is a Mid-level signal; knowing all three and choosing fluidly is L6.”
Hints — p57 Coin Change
Hint 1. Greedy (“always largest coin first”) is wrong. Counter-example: coins=[1,3,4], amount=6 → greedy gives 4+1+1=3 coins, optimal is 3+3=2 coins.
Hint 2. Define dp[a] = min coins to make amount a. Base: dp[0] = 0. Transition: every solution for a ends with some coin c, so dp[a] = 1 + min(dp[a - c]) over coins c <= a.
Hint 3. Use a sentinel for “impossible” — float('inf') or amount + 1. After the DP, return dp[amount] if it’s a real value, else -1.
Hint 4. Alternative framing: BFS on a state graph. Nodes = amounts; edges from a to a + c for each coin c. Shortest path from 0 to amount = answer. Same complexity, different mental model — useful when state space is sparse.
Hint 5. Two loop orders both work for minimum coins; but for LC 518 (count of combinations) the outer loop must be over coins to avoid double-counting permutations. Don’t generalize blindly.
If stuck: see solution.py.
p58 — Longest Increasing Subsequence
Source: LeetCode 300 · Medium · Topics: DP, Binary Search (patience sort) Companies (2024–2025): Google (very high), Meta (high), Amazon (high), Microsoft (high), Apple (medium-high) Loop position: The canonical “O(n²) DP that also has a O(n log n) algorithm via patience sort.” Tests whether you can derive both AND explain why the faster one works.
1. Quick Context
Given nums[], return the length of the longest strictly-increasing subsequence (not necessarily contiguous).
Two solutions both worth knowing cold:
O(n²) DP
dp[i] = LIS ending at index i. dp[i] = 1 + max(dp[j] for j < i if nums[j] < nums[i]). Answer = max(dp).
O(n log n) patience sort
Maintain tails[] where tails[k] = smallest tail of any increasing subseq of length k+1 seen so far. For each x, binary-search for the leftmost tails[k] >= x and overwrite. Answer = len(tails).
The tails array is NOT itself a valid LIS — it’s a smallest-tail-per-length record. But its length equals the LIS length.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/longest-increasing-subsequence/
STOP. 25-min timer. Aim for BOTH solutions.
3. Prerequisite Concepts
- DP with
maxover predecessors. bisect.bisect_leftfor binary search.- Subsequence (non-contiguous) vs substring (contiguous).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Pick a strictly-increasing subset of indices; maximize length.
2. Clarify: Strictly increasing (not ≤)? (Yes on LC 300.) n ≤ 2500 for O(n²); n larger for O(n log n) bonus.
3. Examples:
[10,9,2,5,3,7,101,18] → 4(one LIS:[2,3,7,18]).[0,1,0,3,2,3] → 4.[7,7,7,7] → 1(strict; ties don’t extend).
5. Brute force: All 2ⁿ subsequences, check increasing. Exponential.
6. Target: O(n²) immediate; O(n log n) for senior+.
7. Pattern: “LIS-on-prefixes” DP + patience-sort variant.
8. Develop: see solution.py.
9. Test: empty; single; strict-increasing; strict-decreasing; all-equal; mixed; LC examples.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The O(n²) DP
For each i, dp[i] = “LIS ending at index i.” Transition: dp[i] = 1 + max(dp[j] for j < i if nums[j] < nums[i]), defaulting to 1 if no such j exists. Answer = max(dp).
This is the textbook recurrence; nothing subtle. The clever part is the next algorithm.
6.2 The O(n log n) patience-sort algorithm
Maintain tails[] where tails[k] = smallest possible tail value of any increasing subsequence of length k+1 seen in the prefix processed so far.
For each x in nums:
pos = bisect_left(tails, x)- If
pos == len(tails): append (extends the longest LIS by 1). - Else:
tails[pos] = x(replaces; we now have a smaller tail for that length).
Answer = len(tails).
bisect_left is key for STRICTLY increasing. Use bisect_right for non-strictly-increasing variants.
6.3 Why patience-sort works — the invariant
Invariant: After processing prefix nums[0..i], tails[k] = the smallest tail value among ALL increasing subsequences of length k+1 drawn from this prefix.
Why preserved:
- Appending
xtotails(casepos == len):xis strictly greater than every element intails, so adjoining it to the best length-len(tails)LIS yields a length-(len(tails)+1)LIS with tailx. This is the only such LIS so far, soxis trivially its smallest tail. - Replacing
tails[pos]withx(casepos < len):x ≤ tails[pos]. We can form a length-(pos+1)LIS ending inxby appendingxto any length-posLIS whose tail is< x(which exists becausetails[pos-1] < xbybisect_left’s leftmost condition). So we have a length-(pos+1)LIS with tailx ≤ tails[pos]. Invariant maintained.
Why length suffices: Whenever the prefix’s LIS grows, exactly one append happens. So len(tails) = current LIS length.
6.4 The tails array is NOT an LIS
Common confusion: students print tails and expect a valid LIS. It’s not. Example: nums = [10, 9, 2, 5, 3, 7, 101, 18]. After processing, tails ≈ [2, 3, 7, 18] (which happens to be a real LIS here), but on nums = [4, 10, 4, 3, 8, 9], tails = [3, 8, 9] and the actual LIS is [4, 8, 9] (or [4, 8, 9] from index 0,4,5). The values can drift.
To reconstruct the actual LIS, store predecessor pointers during the patience-sort scan. (Used in solution.py’s lis_reconstruct.)
6.5 Why patience-sort is called “patience sort”
Origin: the card game Patience (Solitaire). Deal cards left-to-right onto piles such that each pile is a decreasing sequence; place each new card on the leftmost pile whose top exceeds it. Number of piles at end = LIS length. (Hammersley, 1972; Aldous & Diaconis, 1995.)
6.6 Variants
- LC 354 Russian Doll Envelopes: 2D LIS. Sort by width ascending, height descending (the trick), then LIS on heights. O(n log n).
- LC 673 Number of LIS: count how many LIS exist. O(n²) extension;
cnt[i]alongsidedp[i]. - LC 1671 Min Deletions for Bitonic Array: combine forward and backward LIS.
- Longest non-strictly increasing: swap
bisect_left→bisect_right. - Longest decreasing: negate values or reverse comparator.
6.7 Lower bound
LIS cannot be solved faster than O(n log n) in the comparison-based model — provably (Fredman, 1975). Patience sort is asymptotically optimal.
7. Anti-Pattern Analysis
- Confuse subsequence with substring — substring requires contiguous; LIS does not. Different problem.
dp[i] = 1 + max(dp[j] for j < i)without thenums[j] < nums[i]filter — gives lengthn, wrong.bisect_rightfor strict LIS — allows duplicates; gives wrong length on[7,7,7].- Print
tailsas the LIS — gives the wrong sequence in general (see 6.4). - Reconstruct LIS by reading the DP array greedily — needs predecessor pointers; a simple “walk back from max” is wrong in general.
- O(n²) when O(n log n) is needed for the constraint — Senior+ candidates lose points if they don’t reach for patience sort when
n > 1000.
8. Skills & Takeaways
bisect_leftvsbisect_rightfor strict vs non-strict bounds.- The “tails array” abstraction generalizes to many longest-something problems.
- Patience sort connects DP to a non-obvious data-structure-driven O(n log n) algorithm — a recurring theme (e.g., Fenwick-based offline algorithms).
- 2D extension via sort-with-tiebreak (LC 354).
9. When to Move On
- Implement BOTH O(n²) and O(n log n) cold in 20 min.
- Explain the patience-sort invariant in 60 seconds.
- Reconstruct the actual LIS (not just length) on paper.
- Solve LC 354 with the sort-by-width-asc-height-desc trick and explain why.
- Distinguish
bisect_leftvsbisect_rightfor strict vs non-strict.
10. Company Context
- Google: L5/L6. O(n log n) expected; lower scores for O(n²) only. Often paired with LC 354 in onsite.
- Meta: E5 onsite. LC 673 (count of LIS) a frequent follow-up.
- Amazon: L6+. Bar Raiser will probe “why O(n log n) works.”
- Microsoft: L65+; LC 1671 bitonic variant.
- Apple: Compiler / ML infra teams; productized as “longest monotone trace in a profiling timeline.”
Scorecard for strong: “Solved both O(n²) and O(n log n), articulated the patience-sort invariant, mentioned the lower bound, sketched LC 354 with the sort trick.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| O(n²) | Solves with prompt | Solves cold | + correct strict-vs-non-strict | + extends to count variant |
| O(n log n) | Doesn’t know | Memorized | Derived with invariant | + reconstruction + lower bound |
| LC 354 / 673 | Doesn’t recognize | Knows by name | Solves LC 354 | Solves both + sort trick proven |
12. Level Delta
- Mid (L4): O(n²) DP, correct, handles strict.
- Senior (L5): + O(n log n) patience sort + correct
bisect_left. - Staff (L6): + reconstructs actual LIS + extends to LC 354 (2D) + states the comparison-based lower bound.
- Principal (L7+): + count-of-LIS via Fenwick + connections to RSK correspondence (patience sort ⇄ Young tableaux) + bitonic / longest non-decreasing variants.
13. Follow-up Q&A
Q1: LC 354 Russian Doll Envelopes. Why sort by width ascending, height descending? A1: With equal widths, descending height prevents two envelopes of the same width from chaining in the height-LIS (they cannot doll into each other). Then LIS on the height column = answer.
Q2: How to count the number of LISes (LC 673)?
A2: O(n²) DP with two arrays: dp[i] = LIS length ending at i; cnt[i] = number of LISes ending at i. For each j < i with nums[j] < nums[i]:
- If
dp[j] + 1 > dp[i]:dp[i] = dp[j] + 1; cnt[i] = cnt[j]. - Elif
dp[j] + 1 == dp[i]:cnt[i] += cnt[j]. Answer = sum ofcnt[i]wheredp[i] == max(dp).
Q3: Reconstruct the actual LIS, not just its length.
A3: During the patience-sort pass, store prev[i] = predecessor index for nums[i] in the LIS ending at i. Maintain pile_idx[k] = index of the most recent nums[i] placed at tails[k]. After processing, walk back from the last-appended element via prev[].
Q4: Longest NON-strictly increasing variant.
A4: Switch bisect_left → bisect_right. (Allows equal values to extend.)
Q5: Stream version: nums arrive one at a time; emit LIS length after each.
A5: Patience sort is naturally online — process each x as it arrives; current LIS length = len(tails). O(log n) amortized per update.
14. Solution Walkthrough
See solution.py:
lis_n2— O(n²) DP.lis_n_log_n— patience-sort O(n log n).lis_reconstruct— patience-sort + predecessor pointers to emit the actual LIS.lis_brute— all subsequences, exponential (for stress test n ≤ 12).
15. Beyond the Problem
Principal code review: “LIS is special because it sits at the boundary of three big ideas: DP, monotone data structures, and combinatorial bijections (RSK correspondence). The fact that the same answer falls out of patience sort, a Fenwick-tree count, and the length of the first row of a Young tableau is not coincidence — it’s deep mathematics. You won’t be asked the bijection in an interview, but knowing it exists is what separates a coder from a computer scientist. When you solve LIS, take the extra five minutes to also sketch why the n log n algorithm is optimal — it changes how you think about lower bounds forever.”
Hints — p58 Longest Increasing Subsequence
Hint 1. Subsequence ≠ substring. Elements need not be contiguous. Try all 2ⁿ subsequences and you get a trivial exponential brute force.
Hint 2. O(n²) DP: dp[i] = LIS ending exactly at index i. For each i, scan all j < i with nums[j] < nums[i] and take dp[i] = max(dp[j]) + 1. Answer = max(dp).
Hint 3. Can we go O(n log n)? Maintain a sorted array tails where tails[k] = the smallest possible tail value of any increasing subseq of length k+1 seen so far. For each new x, binary-search for where it fits.
Hint 4. For strictly-increasing LIS: pos = bisect_left(tails, x). If pos == len(tails), append (we extended the LIS). Else, overwrite tails[pos] = x (we now have a smaller tail for that length). Answer = len(tails).
Hint 5. tails is NOT the actual LIS — its values can drift. To reconstruct the LIS, store predecessor pointers per index alongside the patience-sort scan.
If stuck: see solution.py.
p59 — Longest Common Subsequence
Source: LeetCode 1143 · Medium · Topics: 2D DP, String DP Companies (2024–2025): Google (very high), Amazon (high), Meta (high), Microsoft (high), Bloomberg (medium-high) Loop position: The anchor 2D string DP. Tests grid-DP fluency; sets the template for Edit Distance, Interleaving String, Distinct Subsequences, Regular Expression Matching.
1. Quick Context
Given two strings text1, text2, return the length of their longest common subsequence (not necessarily contiguous).
dp[i][j] = length of LCS of text1[:i] and text2[:j]
if text1[i-1] == text2[j-1]: dp[i][j] = dp[i-1][j-1] + 1
else: dp[i][j] = max(dp[i-1][j], dp[i][j-1])
dp[0][j] = dp[i][0] = 0
answer = dp[m][n]
O(m·n) time, O(m·n) space; compressible to O(min(m,n)) space with a rolling 1D array.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/longest-common-subsequence/
STOP. 20-min timer.
3. Prerequisite Concepts
- 2D DP grid; indexing convention (
dp[i][j]corresponds to prefixestext1[:i],text2[:j]). - Subsequence vs substring distinction.
- Inclusion-exclusion reasoning (“if last chars match, contribute 1; else drop one side”).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Pick matching characters from both strings, preserving order, maximize count.
2. Clarify: Strings only lowercase? (Yes typically.) Case sensitive? (Yes.) Max length 1000 on LC.
3. Examples: ("abcde", "ace") → 3; ("abc", "abc") → 3; ("abc", "def") → 0.
5. Brute force: Recursion on prefix lengths; exponential without memo.
6. Target: O(m·n) time, O(min(m,n)) space.
7. Pattern: 2D string DP, “match or drop one side.”
8. Develop: see solution.py.
9. Test: empty inputs; identical; disjoint; one a subseq of other; LC examples.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The recurrence
Define dp[i][j] = LCS of the prefixes text1[:i] and text2[:j].
Case A: characters match. text1[i-1] == text2[j-1]. The LCS must include this matched character (or there exists an equal-length LCS that does), so dp[i][j] = dp[i-1][j-1] + 1.
Case B: characters differ. The LCS uses at most one of these last characters. Try dropping each:
dp[i][j] = max(dp[i-1][j], dp[i][j-1]).
Base: dp[0][*] = dp[*][0] = 0 (empty prefix has zero LCS).
6.2 Why +1 in the match case is correct
If text1[i-1] == text2[j-1], suppose the optimal LCS does not use this matching character. Then it’s an LCS of text1[:i-1] and text2[:j-1] with same length. So we can equivalently extend it by appending this matched character — without loss of optimality. Hence dp[i][j] = dp[i-1][j-1] + 1.
6.3 Space compression to O(min(m, n))
Notice dp[i][j] depends only on dp[i-1][*] (previous row) and dp[i][j-1] (current row, previous column). Use ONE 1D array, rolling left-to-right.
Subtle: when computing dp[i][j], you need the old dp[i-1][j-1] value — which gets overwritten when you move to column j. Save it in a prev_diag temporary:
prev_diag = dp[0]
for j in 1..n:
tmp = dp[j]
if text1[i-1] == text2[j-1]:
dp[j] = prev_diag + 1
else:
dp[j] = max(dp[j], dp[j-1])
prev_diag = tmp
Use the shorter string as the columns to minimize space.
6.4 Reconstructing the actual LCS
Walk back from dp[m][n]:
i, j = m, n
result = []
while i > 0 and j > 0:
if text1[i-1] == text2[j-1]:
result.append(text1[i-1]); i -= 1; j -= 1
elif dp[i-1][j] >= dp[i][j-1]:
i -= 1
else:
j -= 1
result.reverse()
Requires full 2D table. Tie-breaking choice yields one valid LCS; multiple LCSes may exist.
6.5 Connection to Edit Distance
LCS and Edit Distance (p60) share the same recurrence skeleton — both dp[i][j] over two-string prefixes with three or four moves (diag, up, left). Edit Distance adds a “substitute” move; LCS has only “match-or-skip-one-side.” Same skeleton, different transitions.
In fact: edit_distance(s, t) >= m + n - 2 * lcs(s, t), with equality when only insertions/deletions are allowed (no substitution).
6.6 Bit-parallel LCS (Hunt-Szymanski)
For very long strings with small alphabets, the Allison-Dix bit-parallel algorithm computes LCS in O(m·n / w) where w is machine word size. Out of scope for LC but a real-world technique used in diff and bioinformatics aligners.
6.7 Distinguishing from “Longest Common Substring”
Substring = contiguous. Different recurrence: dp[i][j] = dp[i-1][j-1] + 1 if match, else 0 (chain breaks). Answer = max(dp).
Don’t confuse these two.
7. Anti-Pattern Analysis
- Confuse subsequence with substring — wrong recurrence; very common.
- Off-by-one in indices —
text1[i]instead oftext1[i-1]whendp[i][j]is defined on prefixes of lengthi, j. Inverts the whole grid. - Forget the
maxin the mismatch case —dp[i][j] = dp[i-1][j]only, misses the other branch. dp[0][0] = 1— wrong base case; should be 0 (empty prefix has no LCS).- 1D compression without
prev_diag— corrupts the match case by reading the already-overwritten value. - Reconstruct LCS without storing the full 2D table — impossible with O(min) space; tradeoff.
8. Skills & Takeaways
- 2D string-DP grid template.
- “Match or drop one side” decomposition — generalizes to many sequence-alignment problems.
- O(min(m,n)) space with
prev_diagtrick. - Distinct-from-substring framing.
- Foundation for Edit Distance (p60), Distinct Subsequences (LC 115), Interleaving String, Wildcard Matching.
9. When to Move On
- Implement O(m·n) cold in 15 min.
- Compress to 1D space with
prev_diagcorrectly. - Reconstruct an actual LCS via backtrack.
- Solve LC 583 (Delete Operation for Two Strings) in 5 min as a corollary.
- Articulate the recurrence in 60 seconds verbally.
10. Company Context
- Google: L5+. Often paired with “now reconstruct the LCS.”
- Amazon: L6 onsite; productized as “diff-tool minimal common ancestor.”
- Meta: E5; LC 583 (delete ops) follow-up.
- Microsoft: Compiler / DevDiv teams; diff algorithms.
- Bloomberg: Trade-message reconciliation.
Scorecard for strong: “Derived recurrence, compressed to 1D with prev_diag, reconstructed LCS, related to Edit Distance.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Distinction subseq/substring | Confuses | Aware | Articulates difference | + writes both recurrences |
| Recurrence derivation | Memorized | Derives slowly | Derives in 30s with justification | + connects to Edit Distance + sequence alignment |
| Space compression | O(m·n) | + prompted | 1D with prev_diag | + tradeoff with reconstruction stated |
| Reconstruction | Doesn’t | Sketches | Implements backtrack | + ties broken consistently + alt LCSes acknowledged |
12. Level Delta
- Mid (L4): Correct O(m·n) DP, returns length.
- Senior (L5): + 1D space + reconstruction + LC 583 instant solve.
- Staff (L6): + connects to Edit Distance recurrence + mentions Hunt-Szymanski for real-world diff + LCS-of-K-strings is NP-hard.
- Principal (L7+): + bit-parallel algorithms + offline sequence alignment with affine gap costs (Gotoh) + biological sequence alignment context (Needleman-Wunsch).
13. Follow-up Q&A
Q1: Print one valid LCS.
A1: Backtrack from dp[m][n] as in §6.4. Requires the full 2D table.
Q2: LC 583 — minimum number of delete operations to make two strings equal.
A2: m + n - 2 * lcs(text1, text2). (We keep the LCS in both, delete everything else.)
Q3: LC 1092 — shortest common supersequence.
A3: Length = m + n - lcs. To reconstruct: backtrack as in §6.4 but emit BOTH characters at mismatch steps and the matched character at match steps.
Q4: LCS of K strings. A4: NP-hard in general. K-dimensional DP works for fixed K with O(n^K) time.
Q5: Memory-bounded large-string LCS — what if m, n = 10^6?
A5: O(m·n) is 10^12 — infeasible. Real-world tools (Myers’s diff, used in git diff) solve in O((m+n)·D) where D is the edit distance, suited to “small diff” cases. For large D, fall back to chunked / heuristic alignment.
14. Solution Walkthrough
See solution.py:
lcs_2d— O(m·n) time and space, full table (for reconstruction).lcs_1d— O(min(m,n)) space viaprev_diag.lcs_reconstruct— returns one valid LCS string.lcs_brute— exponential, all subsequences (stress only for small inputs).
15. Beyond the Problem
Principal code review: “LCS is the grandparent of every sequence-alignment algorithm in production —
git diff,rsync, bioinformatics BLAST, voice recognition (DTW). The recurrence you wrote on the whiteboard, with variants for affine gap penalties and scoring matrices, runs in genomics pipelines processing terabytes daily. When you reach for the 2D grid, remember you’re not solving a toy problem — you’re discovering the same algorithm Smith and Waterman published in 1981 that won them a place in CS history. The interview asks for the length; production asks for the alignment, the score, the gap structure, and the parallelization strategy. Knowing only the length is the floor.”
Hints — p59 Longest Common Subsequence
Hint 1. Subsequence (not substring). Think recursion on prefix lengths: f(i, j) = LCS of text1[:i] and text2[:j].
Hint 2. Two cases. If text1[i-1] == text2[j-1]: this character must (WLOG) be in the LCS, so f(i, j) = f(i-1, j-1) + 1. Else: drop one side, f(i, j) = max(f(i-1, j), f(i, j-1)).
Hint 3. Memoize → equivalent to a 2D dp[i][j] table, bottom-up. O(m·n) time and space. Base: dp[0][j] = dp[i][0] = 0.
Hint 4. Space compress to O(min(m, n)) by keeping just one row. Trick: when overwriting dp[j], save its old value first because the match case needs the diagonal dp[i-1][j-1].
Hint 5. To reconstruct the actual LCS string (not just length), keep the full 2D table and walk back from dp[m][n]: on match, prepend the character and step diagonally; on mismatch, step toward the larger of dp[i-1][j] and dp[i][j-1].
If stuck: see solution.py.
p60 — Edit Distance
Source: LeetCode 72 · Hard · Topics: 2D DP, String DP Companies (2024–2025): Google (very high), Apple (high), Meta (high), Amazon (high), Microsoft (high) Loop position: The canonical Hard DP. Levenshtein distance underpins spell-checkers, DNA alignment,
git diff, autocorrect. Bar Raiser favorite.
1. Quick Context
Given word1, word2, return min operations (insert, delete, substitute — each cost 1) to convert word1 → word2.
dp[i][j] = edit distance between word1[:i] and word2[:j]
if word1[i-1] == word2[j-1]: dp[i][j] = dp[i-1][j-1]
else: dp[i][j] = 1 + min(dp[i-1][j-1], # substitute
dp[i-1][j], # delete from word1
dp[i][j-1]) # insert into word1
dp[i][0] = i; dp[0][j] = j
O(m·n) time, O(min(m,n)) space possible with prev_diag trick.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/edit-distance/
STOP. 25-min timer.
3. Prerequisite Concepts
- LCS (p59) — same DP skeleton.
- 2D DP grid; prefix-length indexing.
- Three-way
minover neighbors.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Minimum-cost edit script (insert/delete/substitute) to turn one string into another.
2. Clarify: All ops cost 1? (Yes on LC.) Strings only lowercase? Length up to 500 on LC.
3. Examples:
("horse", "ros") → 3: horse → rorse (sub h→r) → rose (del r) → ros (del e).("intention", "execution") → 5.
5. Brute force: Recurse on prefixes, 3 transitions each; exponential without memo.
6. Target: O(m·n) DP.
7. Pattern: 2D string DP with three transitions (vs LCS’s two).
8. Develop: see solution.py.
9. Test: empty strings; identical; one empty; LC examples; all-substitutes; all-inserts.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The recurrence — interpreting the three transitions
Reading dp[i][j] = “min ops to convert word1[:i] to word2[:j]”:
dp[i-1][j-1] + 1(“substitute”): handle the last pair as a substitution.dp[i-1][j] + 1(“delete”): dropword1[i-1], then convert remainingword1[:i-1]toword2[:j].dp[i][j-1] + 1(“insert”): convertword1[:i]toword2[:j-1], then insertword2[j-1].
If word1[i-1] == word2[j-1]: no operation needed at this step; dp[i][j] = dp[i-1][j-1]. (Equivalent to “free substitute”.)
6.2 Base cases
dp[0][j] = j: empty → firstjchars ofword2needsjinsertions.dp[i][0] = i: firstichars ofword1→ empty needsideletions.
These are NOT zeros; getting them wrong silently corrupts answers.
6.3 Why three transitions suffice
Any edit script can be normalized to a sequence of (insert | delete | substitute) operations on a left-to-right scan. The DP enumerates the operation applied to the last position; recursion handles the rest. Optimality proved by induction.
6.4 Symmetry
edit(a, b) = edit(b, a) because every insert in one direction = delete in the other (and substitute is its own inverse). So we can swap to put the shorter string on the inner loop for cache friendliness.
6.5 Space compression
Same prev_diag trick as LCS (p59 §6.3):
prev_diag = dp[0] # which is i at start of row i
dp[0] = i # base case for this row
for j in 1..n:
tmp = dp[j]
if word1[i-1] == word2[j-1]:
dp[j] = prev_diag
else:
dp[j] = 1 + min(prev_diag, dp[j], dp[j-1])
prev_diag = tmp
6.6 Damerau–Levenshtein variant
Add a fourth transition: transposition of adjacent characters (cost 1). For positions i, j: if word1[i-1] == word2[j-2] and word1[i-2] == word2[j-1] (i.e., the last two chars are swapped), allow dp[i-2][j-2] + 1.
Used in typo-tolerant search (“teh” → “the”). Real-world spellcheckers (e.g., Apache Lucene) use this.
6.7 Connection to LCS (p59)
When substitution is disallowed (insert/delete only, each cost 1), edit(a, b) = m + n - 2·lcs(a, b). With substitution, edit distance is ≤ that bound (because substitute is sometimes cheaper than insert+delete).
6.8 Weighted variants
In production:
- Insertions and deletions may cost differently (e.g., DNA alignment: gap-opening cost vs gap-extension cost — affine gap penalties, solved by Gotoh’s algorithm in O(m·n) with 3 DP tables).
- Substitutions may be weighted by a similarity matrix (BLOSUM62 in bioinformatics).
LeetCode 72 uses uniform cost 1. Mention these variants for Senior+ signal.
6.9 Hirschberg’s algorithm — O(min(m,n)) space + reconstruction
For huge strings where you need the actual edit script (not just length), naive backtrack requires the full O(m·n) table. Hirschberg’s divide-and-conquer computes the alignment in O(m·n) time and O(min(m,n)) space. Used in diff tools where memory matters.
7. Anti-Pattern Analysis
- Forget base cases
dp[i][0] = i,dp[0][j] = j— use zeros, get wildly wrong answers. - Add
+1in the match case — should bedp[i-1][j-1]with no+1. - Confuse “insert into word1” vs “into word2” — they’re symmetric in effect, but interpreting transitions inconsistently breaks reasoning under follow-up questions.
- Two transitions instead of three — student remembers only “diagonal or one side”; missing the substitute, undercounts edits.
- 1D compression without
prev_diag— corrupts substitute transition. - Compute only the value, can’t print the edit script — Senior+ will ask.
8. Skills & Takeaways
- 2D DP with three-way min — generalizes to many alignment problems.
- Base-case discipline (the
i/jinitial row/col). - Match-then-free-substitute interpretation.
- Space compression with
prev_diag. - Damerau, Hirschberg, Gotoh — extension landscape.
9. When to Move On
- Implement O(m·n) cold in 15 min including base cases.
- Compress to 1D space.
- Print the edit script (insert/delete/sub sequence) by backtracking.
- Solve LC 583 (delete-only edit distance) using LCS in 5 min.
- Explain why three transitions suffice.
10. Company Context
- Google: L5/L6. Bar Raiser favorite; expects edit-script reconstruction.
- Apple: Siri / autocorrect teams; Damerau variant relevant.
- Meta: E5 onsite; LC 583 follow-up.
- Amazon: L6+; productized as “fuzzy match for search query.”
- Microsoft: Office spellcheck team; expects Damerau and Hirschberg awareness.
Scorecard for strong: “Solved O(m·n), articulated three transitions cleanly, mentioned Damerau and Hirschberg, related to LCS, reconstructed edit script when asked.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Base cases | Wrong (zeros) | Correct | + explains | + derived from definition |
| Three transitions | Misses one | Memorized | Justifies each | + connects to insert/delete bijection |
| Space compression | O(m·n) | + with prompt | 1D with prev_diag | + Hirschberg O(min(m,n)) + reconstruction |
| Variants | Doesn’t know | Names Levenshtein | Damerau | + Gotoh affine + BLOSUM + Needleman-Wunsch |
12. Level Delta
- Mid (L4): Correct O(m·n), correct base cases, correct three transitions.
- Senior (L5): + 1D space + reconstructs edit script + Damerau variant.
- Staff (L6): + Hirschberg for memory-bounded reconstruction + LCS↔edit-distance relationship + affine gap costs (Gotoh).
- Principal (L7+): + BLOSUM scoring matrices in production bioinformatics + Smith-Waterman local alignment + Myers’s O((m+n)·D) diff algorithm + parallelization (anti-diagonal wavefront).
13. Follow-up Q&A
Q1: Print the actual edit script.
A1: Backtrack from dp[m][n] choosing whichever transition realized the min, emit op (match/sub/insert/delete) at each step, reverse.
Q2: Costs differ per operation type (insert=1, delete=2, sub=3).
A2: Replace the constants in the min: dp[i][j] = min(dp[i-1][j-1] + cost_sub, dp[i-1][j] + cost_del, dp[i][j-1] + cost_ins).
Q3: Damerau-Levenshtein (adjacent swaps allowed).
A3: Add dp[i-2][j-2] + 1 when i,j ≥ 2, word1[i-1]==word2[j-2], word1[i-2]==word2[j-1]. Captures transpositions like “teh”↔“the”.
Q4: Memory-bounded for million-character strings. A4: Hirschberg’s algorithm — divide-and-conquer using O(min(m,n)) space and O(m·n) time; reconstructs full alignment.
Q5: When substitution costs more than insert+delete, what happens?
A5: The DP still works (still picks the cheaper combo), but you can equivalently drop the substitute transition; result equals m + n - 2·lcs(word1, word2) (insert/delete-only edit distance).
14. Solution Walkthrough
See solution.py:
min_distance_2d— O(m·n) DP, full table.min_distance_1d— O(min(m,n)) space withprev_diagtrick.min_distance_reconstruct— returns the edit script (list of (op, char) tuples).min_distance_brute— uncached recursion (exponential, n ≤ 7 in stress).
15. Beyond the Problem
Principal code review: “Edit Distance is the spine of every fuzzy-matching system on Earth: spell-check, autocorrect, DNA alignment, plagiarism detection, OCR post-processing. The naive DP you wrote scales to ~10⁶ × 10⁶ in CPU time but not in memory — and the moment you need to align two human genomes you’ll meet Hirschberg, Gotoh, Myers, and Hunt-Szymanski. Real bioinformatics tools (BWA, Bowtie2) further specialize: they index references with FM-index and use seed-and-extend rather than full DP. The interview question asks ‘what’s the distance’ — production asks ‘what’s the alignment, what’s the score under BLOSUM62, can we afford it at scale, can we do it on a GPU.’ Memorize the recurrence; learn the ecosystem.”
Hints — p60 Edit Distance
Hint 1. Three operations (insert, delete, substitute), each cost 1. Think recursively on prefixes: f(i, j) = edit distance between word1[:i] and word2[:j].
Hint 2. If the last characters match, no op needed: f(i, j) = f(i-1, j-1). Otherwise, try all three operations on the last position and take the minimum, paying +1 for the op.
Hint 3. f(i, j) = 1 + min(f(i-1, j-1) [sub], f(i-1, j) [del], f(i, j-1) [ins]). Memoize → 2D DP table. O(m·n).
Hint 4. Base cases are NOT zeros: dp[i][0] = i (delete i chars), dp[0][j] = j (insert j chars). Forgetting these silently breaks everything.
Hint 5. Space compress to 1D with a prev_diag saved value (same trick as LCS). To recover the actual edit script, keep the full table and backtrack from dp[m][n], picking whichever transition realized the min.
If stuck: see solution.py.
p61 — Partition Equal Subset Sum
Source: LeetCode 416 · Medium · Topics: 0/1 Knapsack DP, Bitmask Companies (2024–2025): Amazon (high), Microsoft (high), Google (medium-high), Meta (medium) Loop position: The 0/1 knapsack archetype — subset-sum reduction with a target. Tests recognizing knapsack disguised as a partition problem and the bitmask optimization that turns O(n·S) into O(n·S/w) with a single Python
int.
1. Quick Context
Given an array of positive integers, can it be partitioned into two subsets with equal sum?
Reduction: Let total = sum(nums). If total is odd, return False. Else, ask: is there a subset summing to total / 2? That’s subset-sum, solvable by 0/1 knapsack.
dp[s] = True if some subset sums to s
init dp = {0: True}
for x in nums:
for s from target down to x: # reverse to avoid reusing x
dp[s] |= dp[s - x]
return dp[target]
O(n·S) time, O(S) space where S = total/2. Bitmask optimization: store dp as a Python int and dp |= dp << x.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/partition-equal-subset-sum/
STOP. 20-min timer.
3. Prerequisite Concepts
- 0/1 knapsack DP.
- Why the inner loop iterates backwards in 1D-space knapsack.
- Pythonic bigint bit-shift.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Equal-sum 2-partition existence.
2. Clarify: Positives only? (Yes on LC 416, 1 ≤ nums[i] ≤ 100, length ≤ 200.) Empty array? Edge case.
3. Examples: [1,5,11,5] → True ([1,5,5] vs [11]); [1,2,3,5] → False.
5. Brute force: Try all 2ⁿ subsets and check sum. Exponential.
6. Target: O(n·S) DP where S = total/2.
7. Pattern: 0/1 knapsack on sum.
8. Develop: see solution.py.
9. Test: odd total; single element; two equal; LC examples; all equal.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The reduction
Two-way partition with equal sums means each part sums to total / 2. If total is odd, no partition exists. Else the question becomes: does some subset sum to total / 2? — classic subset-sum, NP-hard in general but pseudo-polynomial (O(n·S)) when sums are bounded.
LC 416 constraints: n ≤ 200, nums[i] ≤ 100 → S ≤ 100·200 / 2 = 10000. O(n·S) ≈ 2M ops. Fine.
6.2 Why the inner loop runs backwards in 1D-space knapsack
dp[s] after iteration represents “subset-sum reachable using items considered so far, sum = s.” Adding item x:
for s from target down to x:
dp[s] = dp[s] OR dp[s - x]
Iterating backwards ensures dp[s - x] reflects the previous iteration (item x not used yet). Iterating forward would let you use x multiple times — which is the unbounded knapsack (coin change pattern), not 0/1.
This is the single most common bug in 1D knapsack code.
6.3 The bitmask trick — O(n·S/w)
dp is a bitset of length target+1 where bit s = “sum s reachable.” Adding item x corresponds to:
dp |= dp << x
A single bigint OR-shift handles all sums in parallel, processed w ≈ 64 bits per machine word. Final check: (dp >> target) & 1.
Python’s bigints make this a one-liner. CPython internally handles arbitrary-precision shifts efficiently. For S=10000, this is ~157 words per << — vastly faster than the explicit DP loop.
This trick is famous in competitive programming (cf. Codeforces “knapsack with bitmask”).
6.4 Optional optimization: early termination by total halving
Sort nums descending. If any item > target, no partition possible. Track partial sums; if max-remaining + current < target, prune. For LC 416’s constraints, plain DP is fine — but for stress/contest sizes, these prunes matter.
6.5 Reconstructing the subset
Standard knapsack: maintain a back-pointer table choose[i][s] ∈ {0, 1} indicating whether item i was taken to reach dp[i][s]. Walk back from (n, target). O(n·S) memory.
6.6 Connection to NP-hardness
General subset-sum is NP-complete (Karp’s 21). The DP we just wrote is “pseudo-polynomial” — polynomial in the numeric value of S, not in its bit-length. If S is exponential in n, the DP blows up. LC 416 caps S to keep this practical.
For huge S, approximation algorithms exist (Fully Polynomial Time Approximation Scheme — FPTAS by Ibarra-Kim, 1975) with O(n³/ε) for a (1-ε)-approximation.
6.7 Generalizations
- LC 494 Target Sum: assign ± to each number to reach target sum. Reduces to subset-sum: subset of “+” items has sum
(total + target) / 2. - LC 698 Partition to K Equal Subsets: k-way partition. NP-hard; backtracking with pruning, since k > 2.
- LC 474 Ones and Zeroes: 2D knapsack (two constraints).
- LC 1049 Last Stone Weight II: minimize partition-sum difference; same DP, return
total - 2·max_reachable.
7. Anti-Pattern Analysis
- Forget to check
total % 2— if odd, immediately False; otherwise wasted compute. - Inner loop iterates forward — turns 0/1 into unbounded knapsack, gives wrong answers (allows reusing one element multiple times).
- Forget
dp[0] = True— empty subset sums to 0; without it the recurrence yields all False. - 2D
dp[n+1][S+1]when 1D works — wastes memory; also masks the forward-vs-backward bug because 2D doesn’t have it. - Use
int(total / 2)instead oftotal // 2— floating-point coercion; on huge totals can produce wrong target. - Bitmask version:
dp |= dp << xwithout settingdp |= 1initially — empty subset isn’t represented.
8. Skills & Takeaways
- Recognize 0/1 knapsack in disguise (partition, target sum, last-stone).
- Master forward-vs-backward inner loop discipline.
- Python bigint bitmask trick for ~64× speedup.
- Pseudo-polynomial vs polynomial distinction.
9. When to Move On
- Solve LC 416 cold in 12 min.
- Solve LC 494 (Target Sum) in 8 min as a reduction to LC 416.
- Solve LC 1049 (Last Stone Weight II) in 8 min.
- Implement the bitmask version in 3 lines.
- Explain backward-loop necessity in 30 seconds.
10. Company Context
- Amazon: L6 onsite; productized as “can these orders be split evenly across two warehouses?”
- Microsoft: L65+; LC 494 follow-up.
- Google: L5; bitmask variant scores Senior signal.
- Meta: E5; LC 698 multi-way partition as Hard follow-up.
Scorecard for strong: “Recognized knapsack reduction; correct backward inner loop; mentioned bitmask trick; solved LC 494 and LC 1049 as variants; noted pseudo-polynomial complexity.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Reduction recognized | Brute | “knapsack” but vague | “subset-sum to total/2” | + ties to LC 494 reduction |
| Inner loop direction | Wrong | Correct from memory | Correct + explains why | + connects to unbounded knapsack contrast |
| Space | 2D | 1D (forward bug) | 1D backward | + bitmask int trick |
| Complexity framing | “O(n·S)” | + names pseudo-polynomial | + relates to NP-hardness | + FPTAS for large S |
12. Level Delta
- Mid (L4): Correct 1D-space DP, backward inner loop, returns
dp[target]. - Senior (L5): + bitmask
intversion + reconstructs the subset + LC 494 instant. - Staff (L6): + pseudo-polynomial vs NP-hardness framing + LC 698 (k-way) backtracking + FPTAS awareness.
- Principal (L7+): + Karp’s reduction from 3-SAT + dynamic-programming-over-bitmasks for k-partition + production scheduling (multiway number partitioning, LPT/Karmarkar-Karp heuristics).
13. Follow-up Q&A
Q1: LC 494 Target Sum (assign ± to reach target).
A1: Let P = sum of “+” items, N = sum of “-” items. P - N = target, P + N = total → P = (total + target) / 2. Subset-sum to P. (Check (total + target) is non-negative and even.)
Q2: LC 1049 Last Stone Weight II (minimize |sumA - sumB|).
A2: Same DP up to target = total // 2. Answer = total - 2 · max(s for s ≤ target if dp[s]).
Q3: Reconstruct one valid subset, not just existence.
A3: Keep 2D dp[i][s] and back-pointers; walk back from (n, target) to recover which items were chosen.
Q4: K-way equal partition (LC 698).
A4: NP-hard. Use backtracking with bucket-fill: sort descending, try placing each item in one of k buckets, prune when a bucket exceeds target or when two empty buckets occur at the same recursion frame (symmetry break). Bitmask DP dp[mask] = True if items in mask can be partitioned into completed buckets works for n ≤ ~16.
Q5: How does the bitmask dp |= dp << x work?
A5: dp is a bigint where bit s = “sum s reachable.” Shifting left by x shifts every reachable sum by +x (adding item x). ORing with original = “reachable without x” ∪ “reachable with x.” All sums updated in one CPU-friendly bigint op.
14. Solution Walkthrough
See solution.py:
can_partition_dp— 1D-space 0/1 knapsack, backward inner loop.can_partition_bitmask— Python bigint bitmask, three-line core.can_partition_brute— try all 2ⁿ subsets (stress only).
15. Beyond the Problem
Principal code review: “Subset-sum looks like a toy problem until you realize it sits at the root of scheduling theory. The ‘two-machine make-span minimization’ problem in operations research is exactly LC 1049 — and Karmarkar-Karp’s 1982 differencing heuristic, which is the practical state of the art for multiway number partitioning, exists because exact DP doesn’t scale beyond ~10⁴ items. When you write
dp |= dp << xin Python, you’re using a trick that lets bitset-based knapsack solve problems with sums in the millions in a few hundred milliseconds — a technique pulled straight from competitive programming and bioinformatics (peptide-mass matching). The interview asks ‘can it be partitioned’ — production asks ‘how unbalanced is the best partition, what’s the minimum make-span, can we approximate within ε in real time.’ Learn the spectrum from pseudo-poly DP to FPTAS to heuristics.”
Hints — p61 Partition Equal Subset Sum
Hint 1. Equal partition means each side sums to total/2. If total is odd, impossible. Otherwise the question is: “is there a subset summing to total/2?”
Hint 2. That’s classic subset-sum. Define dp[s] = True if some subset of the items considered so far sums to s. Initialize dp[0] = True (empty subset).
Hint 3. For each item x, update: dp[s] = dp[s] OR dp[s - x] for each s. Walk s backwards from target down to x — forward iteration lets you reuse x, which is unbounded knapsack, wrong here.
Hint 4. Time O(n·S), space O(S) where S = total/2. LC 416 caps S to 10000 so this fits comfortably.
Hint 5. Slick Pythonic version: represent dp as a Python bigint. dp |= dp << x updates all sums in one shift-or. Final check: (dp >> target) & 1. ~64× faster.
If stuck: see solution.py.
p62 — Course Schedule II
Source: LeetCode 210 · Medium · Topics: Graph, Topological Sort, BFS, DFS Companies (2024–2025): Amazon (very high), Meta (high), Google (high), Microsoft (high), LinkedIn (medium-high) Loop position: Topological sort canonical. Tests Kahn’s BFS (indegree counter) AND DFS post-order — both expected. Build systems, task scheduling, dependency resolution all reduce here.
1. Quick Context
numCourses courses 0..n-1, prerequisites[i] = [a, b] meaning take b before a. Return a valid order, or [] if cycle exists (impossible).
Two equivalent algorithms:
Kahn’s BFS (indegree)
Compute indegree of every node. Push indegree-0 nodes onto a queue. Pop, append to result, decrement indegrees of neighbors; push any that drop to 0. If final result has all n nodes → return it; else cycle.
DFS post-order
DFS with three-color states (WHITE / GRAY / BLACK). On entering a node mark GRAY; recurse on neighbors; if we hit a GRAY neighbor → cycle. On finish mark BLACK and append to stack. Reverse the stack at the end.
Both O(V + E).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/course-schedule-ii/
STOP. 25-min timer. Aim to know BOTH algorithms.
3. Prerequisite Concepts
- Adjacency list construction.
- BFS with
collections.deque. - DFS with explicit recursion + three colors (white/gray/black) for cycle detection.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Linear order respecting partial-order dependencies.
2. Clarify: Edges duplicated? Self-loop? [a, a] is a cycle. Disconnected components allowed?
3. Examples: n=2, prereqs=[[1,0]] → [0,1]; n=4, [[1,0],[2,0],[3,1],[3,2]] → e.g. [0,1,2,3] or [0,2,1,3]; with cycle [[0,1],[1,0]] → [].
5. Brute force: Try all n! permutations and check validity. n! infeasible.
6. Target: O(V + E).
7. Pattern: Topological sort.
8. Develop: see solution.py.
9. Test: empty prereqs; single course; chain; tree; cycle of length 2 and 3; disconnected components; self-loop.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Edge direction convention — read it carefully
prerequisites[i] = [a, b] says “to take a, you must take b first.” This is a dependency from b → a. Build the adjacency list as adj[b].append(a) and the indegree of a increments.
Getting the direction wrong gives a topologically-sorted but BACKWARDS answer. The most common LC bug here.
6.2 Kahn’s BFS — full algorithm
1. Build adj list + indegree[]
2. queue = deque of all v with indegree[v] == 0
3. result = []
4. while queue:
v = queue.popleft()
result.append(v)
for u in adj[v]:
indegree[u] -= 1
if indegree[u] == 0:
queue.append(u)
5. return result if len(result) == n else []
Why correct: Every node with no remaining predecessors can safely come next. Cycle ⇔ some node never reaches indegree 0 ⇔ len(result) < n.
6.3 DFS post-order — full algorithm
Three colors: WHITE (unvisited), GRAY (in current DFS stack), BLACK (done).
def dfs(v):
if color[v] == GRAY: cycle = True; return
if color[v] == BLACK: return
color[v] = GRAY
for u in adj[v]:
dfs(u)
if cycle: return
color[v] = BLACK
order.append(v) # post-order push
for v in 0..n-1: dfs(v)
return [] if cycle else order[::-1]
Why correct: DFS finishes children before parents. Reversing post-order gives a sequence where every edge u→v has u before v. Cycle ⇔ encountering a GRAY node (back-edge to ancestor).
6.4 Why both algorithms are worth knowing
- Kahn’s BFS generalizes to “lexicographically smallest topo order” (use a
heapqpriority queue instead ofdeque). - DFS generalizes to strongly-connected components (Tarjan, Kosaraju), Euler tours, articulation points.
- Interviewers often follow up “what about cycle DETECTION only?” or “what if we want lex-smallest?” — having both algorithms in pocket answers either fork.
6.5 Subtleties
- Self-loop
[a, a]is a cycle of length 1 — both algorithms must detect. - Duplicate edges in
prerequisitesinflate indegrees if not deduped — Kahn’s still works (it just decrements multiple times), but the input usually doesn’t have dupes; mention if asked. - Disconnected components: both algorithms handle naturally (Kahn enqueues all 0-indegree starts; DFS loops over all nodes).
- Recursion depth: DFS on n=2000 may hit Python’s default recursion limit; use iterative DFS or
sys.setrecursionlimit(10**6)for safety.
6.6 Lex-smallest topological order
Replace deque with heapq (min-heap):
heap = [v for v in range(n) if indegree[v] == 0]
heapq.heapify(heap)
while heap:
v = heapq.heappop(heap)
result.append(v)
for u in adj[v]:
indegree[u] -= 1
if indegree[u] == 0:
heapq.heappush(heap, u)
O((V + E) log V). Standard follow-up.
6.7 Cycle reporting (not just detection)
Sometimes interviewers ask “return the cycle.” Modify DFS: when you hit a GRAY node, walk parent pointers backwards from current node to the GRAY node to recover the cycle. Useful in dependency-resolution diagnostics.
6.8 Production use cases
- Build systems (Make, Bazel, Ninja): topological order of compilation targets.
- Task schedulers (Airflow, Luigi, Dagster): DAG of operators.
- Spreadsheet engines (Excel, Google Sheets): cell evaluation order.
- Package managers (npm, pip, apt): install order respecting deps.
- DB query optimizers: rule application order.
Every one of these crashes / loops on a cycle, so detection is essential.
7. Anti-Pattern Analysis
- Wrong edge direction —
adj[a].append(b)instead ofadj[b].append(a). Produces a valid reverse-topo, fails the LC test. - Indegree decremented to negative — bug from duplicate edges or wrong direction.
- DFS without GRAY state — only WHITE/BLACK; cannot detect cycles, mistakes back-edges for cross-edges.
- Recursive DFS without raised recursion limit — silent crash for n > 1000.
- Forget to check
len(result) == nfor Kahn — partial result returned on a cycle, LC fails. - Use a set instead of indegree counter — works but adds log/hash overhead, makes follow-up “lex-smallest” awkward.
8. Skills & Takeaways
- Kahn’s BFS (indegree) and DFS post-order — both in pocket.
- Three-color DFS for cycle detection.
- Adjacency list + indegree counter pattern.
- Iterative DFS via explicit stack for large graphs.
- Lex-smallest variant via
heapq.
9. When to Move On
- Solve LC 207 (Course Schedule, cycle detection only) in 5 min.
- Solve LC 210 with BOTH algorithms in 15 min total.
- Solve LC 269 (Alien Dictionary) — topo sort over inferred char ordering.
- Solve LC 1857 (Largest Color Value in DAG) — topo + DP.
- Explain three-color DFS in 60 seconds.
10. Company Context
- Amazon: L5/L6; productized as “service deployment order across regions.”
- Meta: E5; LC 269 follow-up common.
- Google: L5; lex-smallest variant a frequent twist.
- Microsoft: Compiler / build-system teams.
- LinkedIn: L5; skill-tree / course-recommendation analogue.
Scorecard for strong: “Implemented both Kahn and DFS, articulated three-color cycle detection, mentioned lex-smallest extension via heap, related to build systems.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Edge direction | Confused | Correct after prompt | Correct + articulates | + draws DAG to verify |
| Algorithm choice | One only | Both names | Both implementable | + chooses based on follow-up potential |
| Cycle detection | Forgets | Indegree check | + DFS GRAY state | + reports cycle, not just detects |
| Extensions | None | LC 207 | LC 269 alien dict | + Tarjan SCC + Kahn-with-heap lex-smallest |
12. Level Delta
- Mid (L4): Kahn’s BFS, correct edge direction, returns
[]on cycle. - Senior (L5): + DFS post-order + three-color cycle detection + lex-smallest with
heapq+ LC 269 alien dictionary. - Staff (L6): + iterative DFS to avoid recursion limit + Tarjan SCC awareness + reports actual cycle + production parallels (build systems).
- Principal (L7+): + parallel topological sort (Kahn’s BFS is naturally levelizable across cores) + incremental topo-sort (Pearce-Kelly 2007 algorithm for online edge insertions) + transitive closure / DAG reachability via DFS+memoization.
13. Follow-up Q&A
Q1: Return only “can it be done” (cycle detection).
A1: LC 207. Same algorithm; return len(result) == n (Kahn) or not cycle (DFS).
Q2: Lex-smallest topological order.
A2: Kahn’s with heapq instead of deque. O((V+E) log V).
Q3: All valid topological orders (not just one). A3: Backtrack on the set of “ready” (indegree-0) nodes; for each choice, decrement and recurse. Exponential in pathological cases (a DAG with no constraints has n! orders).
Q4: Detect AND report the cycle.
A4: In DFS, maintain parent pointers. On encountering GRAY neighbor u from v, walk parent chain from v back to u to recover the cycle.
Q5: What if prerequisites has 10^9 edges?
A5: Adjacency list won’t fit. Stream edges; if you can bound indegree-0 sets externally or process by batches; otherwise the problem isn’t tractable in-memory. Production: graph DBs (Neo4j) for huge dependency graphs.
14. Solution Walkthrough
See solution.py:
find_order_kahn— BFS with indegree counter anddeque.find_order_dfs— three-color DFS, post-order push, reverse at end.find_order_brute— try all permutations (stress n ≤ 6).
15. Beyond the Problem
Principal code review: “Topological sort is one of the dozen algorithms you’ll touch every week of an engineering career. Every build system, every CI pipeline, every workflow engine, every database query plan — they all reduce to this. The interview asks for an order; production asks for parallel scheduling (which subset can run concurrently?), incremental rebuilds (only redo what changed downstream?), cycle diagnostics (when a circular dep is added in code review, surface WHICH cycle?). Kahn’s algorithm naturally answers “what can run now” — it levelizes. That’s why Bazel’s job scheduler is essentially Kahn-with-a-thread-pool. Learn the algorithm; then learn its three or four most common extensions; you’ll use them forever.“
Hints — p62 Course Schedule II
Hint 1. This is topological sort. Edge convention: [a, b] means “b before a” → directed edge b → a. Get the direction right or you’ll output the reverse order.
Hint 2. Kahn’s BFS: build adjacency + indegree array. Push all indegree-0 nodes onto a queue. Pop, append to result, decrement neighbors’ indegrees; if any drops to 0, push. Cycle iff final result has fewer than n nodes.
Hint 3. DFS alternative: three colors WHITE/GRAY/BLACK. On entering, mark GRAY; recurse on neighbors; if any is GRAY → back-edge → cycle. On finish, mark BLACK and append to a post-order list. Reverse it at the end.
Hint 4. Both are O(V + E). Watch for: self-loops [a, a] (cycle of length 1), disconnected components (loop over all start nodes).
Hint 5. Use iterative DFS (explicit stack) for n > 1000 to avoid Python recursion limit, or call sys.setrecursionlimit. Follow-up “lex-smallest order” → swap Kahn’s deque for a heapq.
If stuck: see solution.py.
p63 — Network Delay Time
Source: LeetCode 743 · Medium · Topics: Graph, Dijkstra, Shortest Path Companies (2024–2025): Amazon (high), Google (high), Microsoft (high), Meta (medium-high), Uber (medium-high) Loop position: Dijkstra canonical. Single-source shortest path on a non-negative-weight directed graph. Heap version O((V+E) log V).
1. Quick Context
Network: n nodes 1..n, edges times[i] = [u, v, w] directed u → v with travel time w. From source k, return the time for the signal to reach ALL nodes (= max shortest-path distance). Return -1 if any node unreachable.
Standard Dijkstra:
dist = [INF] * (n+1); dist[k] = 0
heap = [(0, k)]
while heap:
d, u = heappop(heap)
if d > dist[u]: continue # stale entry
for v, w in adj[u]:
if d + w < dist[v]:
dist[v] = d + w
heappush(heap, (d+w, v))
answer = max(dist[1..n]) if all reachable else -1
O((V + E) log V) with binary heap; O(E + V log V) with Fibonacci heap (rarely used in practice).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/network-delay-time/
STOP. 25-min timer.
3. Prerequisite Concepts
- Adjacency list of weighted edges.
heapqas a min-heap (Python only has min-heap natively).- Concept of “stale heap entry” — same node can be pushed multiple times; skip when popped distance > current dist.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: All-targets shortest path from k; answer is the max distance.
2. Clarify: Edge weights non-negative? (Yes — Dijkstra requires this.) Directed. 1-indexed nodes.
3. Examples:
times=[[2,1,1],[2,3,1],[3,4,1]], n=4, k=2 → 2.- Single isolated node not reachable → -1.
5. Brute force: Bellman-Ford O(V·E). Or BFS-with-priority (= Dijkstra).
6. Target: O((V+E) log V).
7. Pattern: Dijkstra.
8. Develop: see solution.py.
9. Test: disconnected; one node; self-loops; multiple edges same pair; LC examples.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Dijkstra — the cut property
Cut property: when we pop (d, u) from the heap with d == dist[u], this d is the true shortest-path distance to u.
Proof sketch: All popped distances are non-decreasing (heap property). Suppose dist[u] were larger than the true shortest distance — then on the true shortest path from k to u, the predecessor p of u was popped earlier with the true dist[p], and we would have relaxed dist[u] := dist[p] + w(p, u) to the correct value, contradiction. This argument requires non-negative weights — a negative edge (p, u, w<0) could decrease dist[u] after p is popped, breaking the invariant.
6.2 Why Dijkstra fails on negative edges
Counterexample: edges A→B(2), A→C(3), C→B(-2). From A: Dijkstra pops B(2) first, finalizes dist[B]=2. Later finds C→B = 3-2 = 1. Wrong.
For graphs with negative edges (but no negative cycles), use Bellman-Ford O(V·E) or, if you need many sources, Johnson’s algorithm (Bellman-Ford reweighting + Dijkstra per source).
6.3 Stale entries — the “if d > dist[u]: continue” line
heapq doesn’t support decrease-key. When a shorter path is found, we push (new_d, u) AGAIN; the old entry remains in the heap. When the old (larger) entry is popped, skip via the staleness check. Without it, you’d re-relax outgoing edges from u redundantly — correctness OK but performance degrades.
Alternative: use IndexedHeap / heapdict for true decrease-key, common in production graph libs (e.g., NetworkX).
6.4 SSSP variants you should know
| Algorithm | Time | When |
|---|---|---|
| BFS | O(V + E) | unweighted (all weights = 1) |
| 0/1 BFS (deque) | O(V + E) | weights ∈ {0, 1} |
| Dijkstra (binary heap) | O((V+E) log V) | non-negative |
| Dijkstra (Fibonacci) | O(E + V log V) | huge graphs, asymptotic |
| Bellman-Ford | O(V·E) | allows negative edges |
| Floyd-Warshall | O(V³) | all-pairs, small V |
| Johnson | O(V·E + V² log V) | all-pairs with negative edges |
| SPFA | avg fast, worst O(V·E) | negative edges, often faster than BF in practice |
| A* | depends on heuristic | informed search w/ heuristic |
6.5 A* — Dijkstra with a heuristic
A* is Dijkstra where the priority key is dist[u] + h(u) instead of dist[u]. With an admissible heuristic (h(u) ≤ true_dist_to_target), A* finds the optimal path while exploring fewer nodes. Used in path-finding (games, routing).
When h ≡ 0, A* degenerates to Dijkstra. When h is perfect, A* is greedy best-first.
6.6 Bidirectional Dijkstra
For shortest path between two specific nodes (not all targets), run Dijkstra from both source and target simultaneously; stop when the search frontiers meet. Cuts the effective explored area roughly in half. Standard in production routing (Google Maps, OpenStreetMap).
6.7 0/1 BFS — the deque trick
If all edge weights are 0 or 1, use a deque: push weight-0 edges to the front, weight-1 to the back. Achieves O(V + E) without a heap. Useful for grid problems where some moves are “free.”
6.8 Lower bound
SSSP on directed graphs with non-negative weights has no known O(E) algorithm in the general case (matter of open research). Thorup’s RAM-model O(E) algorithm exists for undirected graphs (1999). For typical problem sizes, binary-heap Dijkstra is the practical choice.
7. Anti-Pattern Analysis
- Use Dijkstra on a graph with negative edges — Silently wrong; use Bellman-Ford.
- Forget the staleness check — Correctness OK, performance degrades to O(E²/V·log V) worst case.
- Build a max-heap by negating distances when you mean min-heap — Python
heapqis already min-heap; double-negation is a common cause of subtle bugs. - Re-relax already-finalized nodes — happens without staleness check; bloats runtime.
- Forget unreachable nodes — return -1 (or
INF) whenmax(dist) == INF; off-by-one if you sum instead. - 1-indexed vs 0-indexed off-by-one — LC 743 is 1-indexed; allocate
dist[n+1]or shift. - Multi-edge between same pair — keep all in adjacency list; Dijkstra naturally takes the min.
8. Skills & Takeaways
- Dijkstra with binary heap + staleness check.
- Cut property as proof tool.
- SSSP algorithm-family awareness (when to use which).
- A* and bidirectional Dijkstra for production routing.
- 0/1 BFS for weight ∈ {0, 1} graphs.
9. When to Move On
- Implement Dijkstra cold in 12 min.
- Solve LC 743, 787 (Cheapest Flights with K stops — see p64), 1631 (Path With Minimum Effort).
- Articulate cut property in 60 seconds.
- Explain why Dijkstra fails on negative edges with a concrete example.
- Know which algorithm to choose given weight constraints.
10. Company Context
- Amazon: L6 onsite; productized as “warehouse routing.”
- Google: L5/L6; Maps team adjacent; A* and bidirectional Dijkstra extensions.
- Microsoft: L65+; Azure routing.
- Uber: L5; ride-matching ETA = SSSP on road network.
- Meta: E5; social-graph distance variants.
Scorecard for strong: “Standard Dijkstra with staleness check; articulated cut property; mentioned A, bidirectional, and Bellman-Ford for negative edges.”*
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Algorithm choice | “shortest path” | “Dijkstra” | Justifies non-neg | + names alternatives by constraint |
| Heap usage | Wrong direction | Correct | + staleness check | + true decrease-key via IndexedHeap |
| Proof | None | Memorized | Cut property sketched | + connects to greedy algorithm invariants |
| Variants | None | Recognizes A* | Implements A* | + bidirectional + Johnson + Thorup awareness |
12. Level Delta
- Mid (L4): Correct Dijkstra, staleness check, returns max dist or -1.
- Senior (L5): + A* with admissible heuristic + 0/1 BFS for weights {0,1} + LC 787 K-stops variant.
- Staff (L6): + bidirectional Dijkstra for s-t queries + Johnson’s algorithm + production routing engines (CRP, contraction hierarchies).
- Principal (L7+): + Thorup’s O(E) undirected SSSP + multi-modal routing (transit + walking + driving) + ALT/landmark preprocessing + scale-time tradeoffs in road networks.
13. Follow-up Q&A
Q1: What if some edge weights are negative (but no negative cycles)? A1: Bellman-Ford in O(V·E). Or run V iterations of edge relaxation; Bellman-Ford also detects negative cycles (a (V+1)-th relaxation that still updates means negative cycle exists).
Q2: All-pairs shortest path on a small graph. A2: Floyd-Warshall O(V³). For large sparse graphs, run Dijkstra from each node: O(V · (V+E) log V).
Q3: Find the shortest path (the sequence of nodes), not just its length.
A3: Maintain parent[v] = the predecessor when dist[v] was last relaxed. Walk back from target.
Q4: K shortest paths (LC 882, Eppstein). A4: Modify Dijkstra to keep the K best distances per node (or use Eppstein’s O(E + V log V + K)).
Q5: Negative cycle on a path — what’s “shortest”? A5: Undefined (you could loop the negative cycle infinitely many times). Bellman-Ford signals this.
14. Solution Walkthrough
See solution.py:
network_delay_dijkstra— heap-based Dijkstra with staleness check.network_delay_bellman_ford— V-iteration relaxation (oracle for verification; works with negative weights too).network_delay_brute— BFS-style exhaustive DP relaxation until stable.
15. Beyond the Problem
Principal code review: “Dijkstra is the algorithm. Every routing service in the world — Google Maps, Uber dispatch, Cisco’s OSPF protocol, Bitcoin’s Lightning Network path-finding — sits on top of some variant of it. The toy version you write in an interview runs in microseconds on a 100-node graph; the production version on the world road graph (~50M nodes, ~150M edges) needs Contraction Hierarchies, Customizable Route Planning, ALT landmarks, transit indices — all built atop Dijkstra. The cut property you should be able to prove on a whiteboard is the same property that makes those production optimizations correct. Memorize the code; understand the proof; respect the depth.”
Hints — p63 Network Delay Time
Hint 1. This is single-source shortest path. The answer is the time for the slowest node to be reached — i.e., max(dist[v]) for all v. Return -1 if any node is unreachable.
Hint 2. All weights non-negative → Dijkstra is the right algorithm. (If any weight were negative, you’d need Bellman-Ford.)
Hint 3. Use heapq (min-heap). Push (dist, node). Pop, skip if popped dist > known dist[node] (stale entry). Else relax outgoing edges.
Hint 4. Python’s heapq doesn’t support decrease-key. You push a new entry every time you find a shorter path; older entries become “stale” and you skip them via if d > dist[u]: continue.
Hint 5. Time O((V+E) log V). After the loop, return -1 if any dist[v] == INF; else return max(dist[1:]). Watch the 1-indexed-vs-0-indexed nodes — LC 743 is 1-indexed.
If stuck: see solution.py.
p64 — Cheapest Flights Within K Stops
Source: LeetCode 787 · Medium · Topics: Graph, Bellman-Ford, BFS-with-state, DP Companies (2024–2025): Amazon (high), Meta (high), Microsoft (medium-high), Google (medium) Loop position: Why Dijkstra fails when you constrain path length. The constraint “≤ K stops” breaks Dijkstra’s correctness — you must use Bellman-Ford (K+1 relaxations) or BFS-with-(node, stops) state. Tests algorithm-selection judgment, not just implementation.
1. Quick Context
n cities, flights[i] = [from, to, price]. Find cheapest price from src to dst using ≤ K stops. Return -1 if impossible.
“K stops” = at most K+1 edges (a direct flight is 0 stops = 1 edge).
Bellman-Ford bounded by K+1 relaxations:
prev = [INF] * n; prev[src] = 0
for _ in range(K + 1):
curr = prev.copy()
for u, v, w in flights:
if prev[u] + w < curr[v]:
curr[v] = prev[u] + w
prev = curr
return prev[dst] if prev[dst] != INF else -1
O(K · E) time, O(V) space.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/cheapest-flights-within-k-stops/
STOP. 30-min timer. The Dijkstra-doesn’t-work insight is the hardest part.
3. Prerequisite Concepts
- Dijkstra (p63) and its correctness conditions.
- Bellman-Ford as iterative edge relaxation.
- BFS-with-state (extended graph:
(node, stops_used)pairs). - Path-length constraints and how they break greedy approaches.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Cheapest path with at most K+1 edges. Standard cheapest path EXCEPT for the length cap.
2. Clarify: Edges have weights ≥ 0? (Yes.) “K stops” = K intermediate cities = K+1 edges. Verify with interviewer.
3. Examples:
n=3, flights=[[0,1,100],[1,2,100],[0,2,500]], src=0, dst=2, K=1 → 200(0→1→2 uses 1 stop).- Same with
K=0 → 500(direct only).
5. Brute force: DFS with depth limit K+1; exponential in depth.
6. Target: O(K · E).
7. Pattern: Bellman-Ford bounded relaxation, OR BFS-with-state.
8. Develop: see solution.py.
9. Test: K=0 (direct only); K=n (effectively unbounded); unreachable; multi-edge same pair.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why Dijkstra FAILS here
Dijkstra greedily finalizes the cheapest unfinished node. With a stops-constraint, the “cheapest reachable so far” may have used too many stops to be extensible, while a more expensive path with fewer stops remains better for downstream extensions.
Counterexample:
0 → 1 (1)
0 → 2 (5)
1 → 2 (1)
2 → 3 (1)
K = 1, src=0, dst=3
From 0, Dijkstra explores: dist[1]=1, dist[2]=2 (via 0→1→2), dist[3]=3 (via 0→1→2→3). But 0→1→2→3 uses 2 stops — exceeds K=1. The correct answer is 0→2→3 = 6 (1 stop).
Dijkstra needs augmentation: track (node, stops_used) as the state, OR switch to Bellman-Ford.
6.2 Bellman-Ford with K+1 iterations — why it’s correct
Bellman-Ford’s invariant after iteration i: dist[v] = cheapest cost reaching v using ≤ i edges. Run exactly K+1 iterations; dist[dst] = cheapest with ≤ K+1 edges = ≤ K stops. Done.
Critical: use prev and curr arrays separately. If you relax in-place with a single array, you can chain multiple relaxations in one iteration — each “iteration” then represents using ≤ many edges, not exactly one more. Wrong answer.
for _ in range(K + 1):
curr = prev.copy()
for u, v, w in flights:
curr[v] = min(curr[v], prev[u] + w)
prev = curr
The prev.copy() is THE detail. Skipping it is the most common interview bug.
6.3 BFS-with-state alternative
Extend the state space: each state is (node, stops_used). Run BFS / modified Dijkstra over states. Transitions from (u, k): for each outgoing edge (u, v, w), go to (v, k+1) with added cost w, allowed if k+1 ≤ K+1.
Heap-based Dijkstra on this extended graph works correctly because the augmented state has no constraint that breaks the cut property.
Time: O((V·K + E) log (V·K)) — usually slower than Bellman-Ford for this problem unless K is small relative to V.
6.4 SPFA / Modified Bellman-Ford
You can speed up Bellman-Ford with SPFA (Shortest Path Faster Algorithm) — a queue-based relaxation that only revisits updated nodes. Same K-iteration bound. In practice often faster than plain BF.
6.5 Why the constraint changes everything
Adding a “≤ K edges” constraint converts the SSSP from a cut-property problem to a layered one. The DP is over layers (= edge counts), not over distances. This is a general lesson: path-constraint shortest path = layered DP, NOT cut-based greedy. Other instances: “exactly K edges” (uses powers of adjacency matrices), “at most K colors used,” “bounded resource consumption.”
6.6 Equivalent formulations
- DP over edges:
dp[k][v]= min cost to reachvusing ≤kedges. Bellman-Ford is this DP rolled to 1D. - Min-plus matrix power:
A^k[i][j](in min-plus semiring) = min cost using exactlykedges. Sum overk ≤ K+1. Matrix exponentiation gives O(V³ log K) — useful when K is huge.
6.7 Subtleties
- K=0: direct flights only. Common edge case bug.
- Self-loops or multi-edges: must keep them all in the adjacency / edge list; Bellman-Ford handles correctly.
- Cycle in graph: doesn’t matter; the K-iteration cap prevents infinite traversal.
7. Anti-Pattern Analysis
- Use Dijkstra naively — silently wrong on stops-constrained paths.
- Bellman-Ford in-place (no
prev.copy()) — chains relaxations within one iteration; counts wrong number of stops. - Off-by-one on K: “K stops” = “K+1 edges.” Confuse and you compute the wrong path-length cap.
- Heap-based Dijkstra over
(node, cost)with visited-set — visited-set prevents revisiting nodes via cheaper-stops paths; wrong. - Track stops without state — single
dist[v]array can’t disambiguate paths with same cost different stops. - DFS without memoization — exponential blowup; on LC times out.
8. Skills & Takeaways
- Recognize when a constraint breaks Dijkstra’s correctness.
- Bellman-Ford as bounded-edge-count DP.
- BFS-with-state for augmented graphs.
- The “prev vs curr” array discipline in iterative DP.
- General pattern: constraint on path = extra DP dimension.
9. When to Move On
- Solve LC 787 cold in 18 min with both Bellman-Ford and BFS-with-state.
- Articulate why Dijkstra fails in 60 seconds with the explicit counterexample.
- Solve LC 743 (Dijkstra, p63) and LC 787 back-to-back, switching algorithms appropriately.
- Solve LC 1928 (Min Cost to Reach Destination In Time) — another layered DP.
10. Company Context
- Amazon: L6 onsite; productized as “warehouse-to-warehouse with hop limit.”
- Meta: E5; expects clean Bellman-Ford with
prev.copy(). - Microsoft: L65+; Azure Front Door routing with hop limit.
- Google: L5; less common but appears as the airline-pricing variant.
- Uber: L5; multi-modal hop-bounded routing.
Scorecard for strong: “Recognized Dijkstra fails; chose Bellman-Ford; correctly used prev/curr arrays; articulated K=K+1-edges; mentioned BFS-with-state alternative.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Algorithm choice | Dijkstra (wrong) | “BFS” vague | Bellman-Ford justified | + BFS-with-state alternative + tradeoff |
| Prev/curr discipline | Bug | After prompt | Cold | + explains why in-place fails |
| K semantics | Confused | Correct after example | K=K+1 edges cold | + connects to general layered-DP pattern |
| Counterexample | None | Sketches | Constructs | + generalizes to broader class |
12. Level Delta
- Mid (L4): Bellman-Ford with K+1 iterations, prev/curr arrays.
- Senior (L5): + BFS-with-state via
heapqon(cost, node, stops)+ counterexample for Dijkstra. - Staff (L6): + min-plus matrix exponentiation for huge K + SPFA optimization + general “constraint = extra DP dimension” pattern.
- Principal (L7+): + LP relaxation framing + resource-constrained shortest path NP-hardness for arbitrary resources + production airline-pricing systems (multi-fare-class with bag/seat constraints).
13. Follow-up Q&A
Q1: K is huge (say, 10^9). A1: Bellman-Ford is O(K·E) — too slow. Use min-plus matrix exponentiation: O(V³ log K). Or observe: if K ≥ V-1, you’re effectively running unbounded shortest path (use Dijkstra).
Q2: Allow negative weights. A2: Bellman-Ford still works (it natively handles negatives). Dijkstra would already fail. Detect negative cycles by running one extra iteration; any further improvement indicates a negative cycle.
Q3: Reconstruct the actual flight sequence.
A3: Maintain parent[k][v] during Bellman-Ford (the predecessor on the iteration when v’s distance improved). Walk back.
Q4: Multiple sources, single destination. A4: Add a “super-source” with 0-cost edges to all real sources; run Bellman-Ford from super-source.
Q5: Why does prev.copy() matter?
A5: Without it, in-place updates can cascade: relaxing dist[v] then immediately using new dist[v] to relax dist[w] chains TWO edges in one iteration, blowing the K-edge cap.
14. Solution Walkthrough
See solution.py:
find_cheapest_price_bf— Bellman-Ford with prev/curr, K+1 iterations.find_cheapest_price_bfs— Dijkstra over(cost, node, stops_used)states.find_cheapest_price_brute— DFS with depth-limit K+1, exhaustive.
15. Beyond the Problem
Principal code review: “This is the problem that teaches you when to STOP reaching for Dijkstra. Every time someone proposes a path-finding problem with an extra constraint — ‘at most K hops,’ ‘within budget B,’ ‘using at most M road segments’ — the same lesson applies: the constraint creates a new DP dimension, and greedy algorithms that rely on cut properties break. The fix is one of three patterns: Bellman-Ford with bounded iterations, augmented state in BFS/Dijkstra, or LP relaxation when the constraints are linear. In airline-pricing systems at scale, fares, bag rules, fare classes, layover lengths, and bilateral agreements all add dimensions; the production solver is a fully integer-programming engine, not a graph algorithm at all. Knowing where the line is — when Dijkstra suffices vs when you need bigger machinery — is what separates a coder from a path-planning engineer.”
Hints — p64 Cheapest Flights Within K Stops
Hint 1. “K stops” = K intermediate cities = K+1 edges. A direct flight is 0 stops.
Hint 2. Dijkstra is the WRONG choice here. With a hop-limit, the cheapest cumulative cost to a node may use too many edges to extend further, while a more expensive earlier-stage path with fewer edges could lead to a cheaper full solution. Construct a small counterexample to convince yourself.
Hint 3. Bellman-Ford bounded to K+1 iterations works. Invariant: after iteration i, dist[v] = cheapest cost to reach v using ≤ i edges. Run K+1 times.
Hint 4. Critical: use TWO arrays (prev and curr). At each iteration, curr = prev.copy(), then relax edges into curr using prev[u]. Updating in-place lets a single iteration chain multiple edges — wrong number of stops.
Hint 5. Alternative: BFS/Dijkstra over an extended state (node, stops_used). Push (cost, u, k); relax to (v, k+1) if allowed. Slightly more code but no prev.copy() discipline needed.
If stuck: see solution.py.
p65 — Number of Connected Components in an Undirected Graph
Source: LeetCode 323 · Medium · Topics: Union-Find (DSU), DFS/BFS Companies (2024–2025): Amazon (high), Google (high), Meta (medium-high), Microsoft (medium-high), LinkedIn (medium) Loop position: Union-Find / Disjoint Set Union (DSU) canonical. Tests path compression + union by rank — the two optimizations that make DSU effectively O(α(n)) per op (inverse Ackermann, ~constant).
1. Quick Context
n nodes 0..n-1, undirected edges. Return number of connected components.
DSU approach: start with n singleton components. For each edge (u, v), union u and v. Final count = number of distinct roots.
parent = list(range(n))
rank = [0] * n
def find(x):
while parent[x] != x:
parent[x] = parent[parent[x]] # path compression (halving)
x = parent[x]
return x
def union(x, y):
rx, ry = find(x), find(y)
if rx == ry: return False
if rank[rx] < rank[ry]: rx, ry = ry, rx
parent[ry] = rx
if rank[rx] == rank[ry]: rank[rx] += 1
return True
count = n
for u, v in edges:
if union(u, v): count -= 1
return count
Time: O((V + E) · α(V)) ≈ O(V + E) practically.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/number-of-connected-components-in-an-undirected-graph/ (LC Premium) Equivalent free problem: LeetCode 547 “Number of Provinces” (matrix form).
STOP. 20-min timer.
3. Prerequisite Concepts
- DSU data structure:
parentarray +find+union. - Two key optimizations: path compression (in find) and union by rank/size (in union).
- Why α(n) ≤ 4 for all n ≤ 2^65536 (Tarjan).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Count connected components in undirected graph.
2. Clarify: Self-loops? Duplicate edges? Both fine; DSU handles via find returning same root.
3. Examples:
n=5, edges=[[0,1],[1,2],[3,4]] → 2.n=5, edges=[[0,1],[1,2],[2,3],[3,4]] → 1.
5. Brute force: DFS / BFS from each unvisited node; count components. O(V + E).
6. Target: DSU with α(V) per op.
7. Pattern: Union-Find.
8. Develop: see solution.py.
9. Test: empty edges; single node; all in one component; all isolated; duplicate edges; self-loops.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 DSU operations
find(x): return the root (representative) of x’s component. Walk parent chain.
union(x, y): connect the components containing x and y. Find both roots; if different, hang one under the other.
Both ops with optimizations are O(α(n)) amortized.
6.2 Path compression — three variants
Full compression (recursive):
def find(x):
if parent[x] != x:
parent[x] = find(parent[x])
return parent[x]
Cleanest; risks recursion limit.
Path halving (iterative):
def find(x):
while parent[x] != x:
parent[x] = parent[parent[x]]
x = parent[x]
return x
No recursion; nearly as good as full compression.
Path splitting: like halving but updates parent[x] to grandparent then moves to OLD parent (not grandparent). Theoretical curiosity.
Both halving and full compression give α(n) bound.
6.3 Union by rank vs union by size
Union by rank: rank ≈ depth upper bound. Hang shorter tree under taller. Rank increases only on ties.
Union by size: track component size; hang smaller under larger.
Both achieve O(log n) tree height without compression, O(α(n)) with compression. Equivalent in practice; rank uses less memory if components are small; size is more intuitive and gives the component size as a side benefit.
6.4 Why α(n) — Tarjan’s theorem
Tarjan (1975, 1984) proved: with both path compression AND union by rank/size, m operations on n elements take O((m + n)·α(n)) time, where α is the inverse Ackermann function. α(n) ≤ 4 for all n ≤ 2^2^2^…^2 (65536 twos). Effectively constant.
Without either optimization, naive DSU is O(n) per op worst case.
6.5 Variants worth knowing
- Weighted DSU / Potentials: store offset
weight[x]= “x’s value minus root’s.” Used for LC 399 (Evaluate Division), arithmetic constraints, parity checks. - DSU on intervals (offline): track “next unmarked index ≥ i” — classic LC 1488 type problem.
- DSU with rollback (persistent): for offline dynamic connectivity; segment-tree-divide-and-conquer over edge lifetimes.
- Small-to-large merging: generic “merge two sets” pattern; O(n log² n) without DSU.
6.6 Connection to Kruskal’s MST
Kruskal sorts edges by weight, scans through, and unions endpoints — using DSU. Path-compression + union-by-rank is essential for Kruskal’s O(E log E) bound.
6.7 DFS / BFS alternative
For static graphs (all edges known up front), DFS/BFS from each unvisited node gives O(V+E) component count — simpler than DSU, same asymptotics. Choose DSU when:
- Edges arrive in a STREAM (incremental connectivity).
- You need to interleave “is u connected to v?” queries with edge additions.
- The problem is naturally union-based (Kruskal, image segmentation, ETL).
6.8 Decremental / fully dynamic connectivity
DSU handles INCREMENTAL (edge additions). For DECREMENTAL (deletions) or FULLY DYNAMIC, advanced structures: Holm-Lichtenberg-Thorup O(log² n) per op for fully dynamic, link-cut trees for trees only. Out of LC scope but worth knowing exists.
7. Anti-Pattern Analysis
- Path compression WITHOUT union by rank — still works (O(log n) amortized) but loses the α(n) bound.
- Union by rank WITHOUT path compression — same: O(log n), not α(n).
- Naive
find(no compression) on a deep tree — degenerates to O(n) per query in worst case. - Initialize
parent = [0] * n— bug, parents should belist(range(n))(each its own root). - Forget to check
if rx == ryin union — does redundant assignment; correctness OK, but in problems where you increment a counter on successful merge, you’d over-count. - Mix
find(x)withparent[x]later — after compression,parent[x]is the root, but only becausefindwas called. Don’t assumeparent[x]is the root without callingfind.
8. Skills & Takeaways
- DSU with path compression + union by rank/size — the standard combo.
- O(α(n)) ≈ constant per op intuition.
- When to choose DSU vs DFS/BFS for component-counting.
- Pattern recognition: any “merge groups” problem screams DSU.
- Weighted DSU for relational constraints.
9. When to Move On
- Implement DSU with both optimizations cold in 10 min.
- Solve LC 323 (premium), LC 547 (Provinces), LC 684 (Redundant Connection), LC 200 (Islands via DSU), LC 1319 (Make Network Connected) in 10–15 min each.
- Solve LC 399 (Evaluate Division) using weighted DSU.
- Solve Kruskal’s MST.
- Recite Tarjan’s α(n) result and its intuition.
10. Company Context
- Amazon: L5/L6; productized as “cluster detection in service-mesh graphs.”
- Google: L5/L6; image-segmentation pipelines; LC 399 a favorite.
- Meta: E5; account-merging across signals.
- Microsoft: L65+; build-system dependency clustering.
- LinkedIn: L5; people-you-may-know cluster discovery.
Scorecard for strong: “Implemented DSU with both optimizations, articulated α(n) bound, mentioned weighted-DSU and Kruskal applications, can solve LC 399.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| DSU correctness | Buggy | Correct after debug | Cold | + iterative find for huge inputs |
| Optimizations | Neither | One of two | Both, explained | + path-halving variant + lower bounds |
| α(n) understanding | None | Memorized name | Sketches intuition | + Tarjan’s proof structure named |
| Applications | LC 323 only | + LC 547 | + Kruskal + LC 399 | + decremental connectivity + small-to-large + image seg pipelines |
12. Level Delta
- Mid (L4): DSU correctness, path compression OR union by rank, returns count.
- Senior (L5): + both optimizations + LC 399 weighted DSU + Kruskal MST.
- Staff (L6): + iterative DFS path-halving for deep trees + small-to-large merging + decremental connectivity awareness + ETL pipelines using DSU for entity resolution.
- Principal (L7+): + Holm-Lichtenberg-Thorup for fully dynamic connectivity + link-cut trees + DSU with rollback for offline problems + image-segmentation production pipelines (graph cuts, watershed).
13. Follow-up Q&A
Q1: Count the SIZE of each component, not just the count.
A1: Track size[x] initialized to 1; on union, size[new_root] += size[other_root]. Final sizes are at roots; call find to look up.
Q2: LC 399 Evaluate Division — given division equations, answer division queries.
A2: Weighted DSU. weight[x] = value of x / value of parent[x]. On union and find, update weights multiplicatively.
Q3: Edges arrive dynamically; report component count after each addition. A3: Maintain a counter, decrement on each successful union. O(α(n)) per edge.
Q4: What if you must also handle edge DELETIONS? A4: Fully dynamic connectivity. Naive DSU doesn’t support deletion. Use Holm-Lichtenberg-Thorup or offline segment-tree-divide-and-conquer over edge lifetimes.
Q5: How does DSU appear in Kruskal’s MST? A5: Sort edges by weight. Scan in order; for each edge, if its endpoints are in different DSU components, add to MST and union them. Skip otherwise. O(E log E) total.
14. Solution Walkthrough
See solution.py:
count_components_dsu— DSU with path halving + union by rank.count_components_dfs— DFS over adjacency list (oracle).count_components_brute— quadratic transitive-closure flood (stress only).
15. Beyond the Problem
Principal code review: “Union-Find is one of the rare data structures that’s both theoretically beautiful (α(n) ≈ constant via Tarjan’s analysis) and constantly practical. Image segmentation in production (think medical imaging, satellite analysis), Kruskal’s MST in network design, entity resolution in data warehouses (‘these three records are the same customer’), percolation simulations in physics, build-system dependency clustering — all DSU. Weighted DSU adds the ability to track relational constraints (parity, ratios, offsets), turning it into a constraint propagator. Persistent DSU (with rollback) enables offline algorithms on dynamic graphs. When you write
parent[ry] = rx, you’re invoking 50 years of algorithm-design history and one of the most elegant amortized analyses in computer science. Memorize the seven lines; learn the dozen applications; respect the depth.”
Hints — p65 Number of Connected Components
Hint 1. Two approaches: DFS/BFS over adjacency list, OR Union-Find (DSU). Both O(V+E)-ish, but DSU is the canonical answer here and tests an important data structure.
Hint 2. DSU basics: parent[i] = i initially. find(x) walks parents to root. union(x, y) joins two roots. Component count = number of distinct roots = n − (successful unions).
Hint 3. Optimization 1 — path compression (or halving): inside find, redirect parents toward the root. Iterative halving avoids recursion-limit risk: parent[x] = parent[parent[x]]; x = parent[x].
Hint 4. Optimization 2 — union by rank: track approximate tree depth (rank). When unioning, hang the shorter tree under the taller. Rank increases only on ties.
Hint 5. Together: O(α(n)) per operation amortized (inverse Ackermann, ~constant ≤ 4 for all practical n). Decrement a components counter on every SUCCESSFUL union (when roots differ). Final count = n − merges.
If stuck: see solution.py.
Month 3 — Graphs & DP Deep
Weeks 13–16. Named-algorithm tier (MST, Tarjan, Hierholzer) → DP deep (interval DP, bitmask DP, digit DP) → string DP and harder bridge problems.
Weeks
- Week 13 — Advanced Graphs: MST (Kruskal + Prim), bipartite, bridges (Tarjan), Alien Dictionary, Reconstruct Itinerary (Hierholzer).
- Week 14 — Advanced DP: interval DP, bitmask DP, harder string DPs. (TBD)
- Week 15 — Game DP + digit DP + counting. (TBD)
- Week 16 — Practical engineering. (TBD)
Why this month
Foundational graphs (Weeks 5–7, 12) gave you BFS, DFS, Dijkstra, Bellman-Ford, Union-Find. Foundational DP (Week 12) gave you 1D and 2D string DP. This month layers on the named algorithms and the harder DP shapes that show up at L5+ interviews: Tarjan, Hierholzer, interval DP, bitmask DP. Each has a precise correctness argument; vagueness is penalized.
Week 13 — Advanced Graphs
Month 3 — Graphs & DP Deep · Week 1 of 4
5 problems, p66–p70. Topics: Minimum Spanning Tree (Kruskal + Prim), bipartite graphs (2-coloring), bridges & articulation points (Tarjan), constrained topological sort over inferred edges (Alien Dictionary), and Eulerian paths (Hierholzer). Each lifts you out of “BFS/DFS” comfort zone into algorithms with subtle invariants and named papers.
Why this week
Weeks 11–12 covered foundational graphs (BFS, DFS, Dijkstra, Bellman-Ford, Union-Find). This week is the named-algorithm tier: MST, Tarjan, Hierholzer. Each has a one-paragraph correctness proof you must be able to state. Senior+ interviews ask for these by reputation — even when not strictly required.
Schedule (Mon–Fri)
| Day | Problem | Focus | Difficulty |
|---|---|---|---|
| Mon | p66 — Min Cost to Connect All Points | MST: Kruskal (sort + DSU) vs Prim (heap) | Medium |
| Tue | p67 — Is Graph Bipartite? | 2-coloring via BFS / DFS, odd-cycle witness | Medium |
| Wed | p68 — Critical Connections in a Network | Tarjan’s bridge algorithm: disc + low | Hard |
| Thu | p69 — Alien Dictionary | Char graph from adjacent word diffs + topo sort + cycle | Hard |
| Fri | p70 — Reconstruct Itinerary | Hierholzer’s algorithm (Eulerian path) | Hard |
Readiness gate (move to Week 14 when)
- Implement Kruskal and Prim cold; explain when each is preferred.
- State Tarjan’s bridge invariant:
low[v] = min(disc[v], low[child], disc[neighbor through back-edge]). - Construct edge cases for Alien Dictionary: prefix-violation (
["abc","ab"]→ “”), cycle. - Explain Hierholzer: why DFS with post-order append yields the Eulerian path.
- Solve all 5 cold in their target windows (25–40 min each).
p66 — Min Cost to Connect All Points
Source: LeetCode 1584 · Medium · Topics: MST, Kruskal, Prim, Union-Find Companies (2024–2025): Amazon (high), Google (high), Microsoft (medium-high), LinkedIn (medium) Loop position: MST canonical. Tests the two classical MST algorithms (Kruskal + Prim) and your judgment on which to pick given density. Often phrased as “connect houses with minimum cable” or “minimum-cost network.”
1. Quick Context
n points in 2D. Cost to connect points i, j = Manhattan distance |xi−xj| + |yi−yj|. Return min total cost to connect all points (so the graph is connected — equivalently, build a spanning tree of min weight).
This is the MST problem on a complete graph with V² edges.
Prim (better here — graph is dense, V² edges):
in_mst = [False] * n
min_cost = [INF] * n
min_cost[0] = 0
total = 0
for _ in range(n):
u = argmin over not-in-mst of min_cost
in_mst[u] = True
total += min_cost[u]
for v in range(n):
if not in_mst[v]:
min_cost[v] = min(min_cost[v], dist(u, v))
return total
O(V²) — perfect for V² edges. Heap-based Prim is O(E log V), Kruskal is O(E log E). With V² edges all are similar; array-Prim wins by constant.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/min-cost-to-connect-all-points/
STOP. 30-min timer.
3. Prerequisite Concepts
- Spanning tree definition: connected, acyclic, V-1 edges, spans all V nodes.
- MST: spanning tree of minimum total edge weight (unique if all weights distinct).
- Cut property: lightest edge crossing any cut belongs to some MST.
- Cycle property: heaviest edge in any cycle is NOT in any MST.
- Union-Find (p65) for Kruskal; heap or array for Prim.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: MST on the complete graph induced by Manhattan distances.
2. Clarify: Distance is Manhattan (problem-stated). All points distinct? (Often yes; even if not, MST is well-defined.) Allowed duplicates? Same answer.
3. Examples: [[0,0],[2,2],[3,10],[5,2],[7,0]] → 20. Construct by hand: edges (0,0)→(2,2)=4, (2,2)→(5,2)=3, (5,2)→(7,0)=4, (5,2)→(3,10)=9. Total 20.
5. Brute force: Enumerate all spanning trees (n^(n-2) by Cayley’s formula). Infeasible.
6. Target: O(V²) via Prim with array (since edge count = V²).
7. Pattern: MST. Choose Prim (array) for dense graphs, Kruskal for sparse.
8. Develop: see solution.py.
9. Test: n=1 → 0; n=2 → distance between them; collinear points; coincident points.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why Prim wins for this problem
Edge count = V·(V-1)/2 ≈ V². Kruskal is O(E log E) = O(V² log V). Prim-with-heap is O((V + E) log V) = O(V² log V). Prim-with-array is O(V²) — strictly better when E ≈ V². On LC’s max constraints (V ≤ 1000), V² = 10⁶ vs V² log V ≈ 2·10⁷. The constant matters.
For SPARSE graphs (E = O(V)), Kruskal or heap-Prim win.
6.2 Prim’s correctness — the cut property
Cut property: For any cut (partition of vertices into S and V\S), the cheapest edge crossing the cut belongs to some MST.
Prim maintains S = set of nodes already in MST. At each step, it adds the cheapest edge from S to V\S. By the cut property, this edge is safe. After V−1 additions, S = V; we have an MST.
6.3 Kruskal’s correctness — also cut property, but globally
Algorithm: Sort edges by weight. For each in order, if its endpoints are in different DSU components, add to MST and union. Skip otherwise.
Why it works: When you consider edge e = (u, v), the partition (component of u) | rest defines a cut. Since e is the next-smallest edge and you haven’t yet connected this cut, e is the cheapest edge crossing it — safe by cut property.
6.4 Array-Prim implementation detail
min_cost[v] = “weight of cheapest edge from {already-in-MST} to v.” Initialize min_cost[0] = 0, all others INF. In each iteration:
- Pick
u= unfinished node with smallestmin_cost[u]. - Add
min_cost[u]to total. - Mark
ufinished. - For each unfinished
v, updatemin_cost[v] = min(min_cost[v], dist(u, v)).
Repeats V times; inner loops O(V). Total O(V²).
6.5 Heap-Prim subtlety
Heap-Prim pushes (weight, v) for each newly-improvable neighbor; doesn’t decrease-key (Python heapq lacks it). May push duplicates; on pop, skip if v is finished. Time: O(E log V) amortized. Cleaner code but slower constants here.
6.6 Manhattan-distance MST in O(V log V)
For Manhattan-distance MST in 2D, there’s a clever O(V log V) algorithm using sweep + KD-tree. Each point has at most 8 “candidate” neighbors in the MST (one per octant). Overkill for LC; mention only at Staff+. The key insight: in L1, the MST is sparse despite the complete underlying graph.
6.7 Variants and friends
- LC 1135 Connecting Cities with Minimum Cost: explicit edges given → Kruskal.
- LC 1631 Path with Minimum Effort: minimax path = MST or binary search.
- LC 1168 Optimize Water Distribution: add a super-source for “well” costs, then MST.
- Steiner Tree: MST when only a SUBSET of nodes must be connected. NP-hard.
6.8 Borůvka’s algorithm (third MST algorithm)
Each iteration: every component picks its cheapest outgoing edge; add all unique ones. Halves component count per round → O(E log V). Parallelizable. Original 1926 algorithm. Worth knowing the name at Staff+.
7. Anti-Pattern Analysis
- Build explicit edge list of V² edges in Python list of tuples for Kruskal — uses ~10⁸ memory for V=10⁴; OOM. Prefer array-Prim.
- Heap-Prim without staleness check — popping a stale
(w, v)and re-processing v double-counts. - Forget
dist(u, u) = 0in array-Prim — handled automatically if you initializemin_cost[0] = 0and add it; common bug is to start total atdist(0, ?)instead of 0. - Use Euclidean distance (square root) — problem says Manhattan; passing tests on a different metric is silently wrong.
- Recompute distance on every loop iteration — fine for clarity at V≤1000; precomputing matrix uses V² extra memory but trades for speed.
- Mix up MST with shortest path — they’re different problems! MST minimizes total tree weight; SSSP minimizes per-vertex distance from a root.
8. Skills & Takeaways
- Two algorithms (Kruskal, Prim), one MST.
- Density determines which is faster: dense → array-Prim, sparse → Kruskal.
- Cut property as the core correctness lemma for both.
- Build complete graphs implicitly (compute distances on the fly) rather than materializing edge lists when V² is large.
9. When to Move On
- Implement array-Prim and Kruskal+DSU cold in 12 min each.
- State the cut property and its role in both proofs in 60 seconds.
- Solve LC 1584, LC 1135, LC 1168, LC 1631 in 25 min each.
- Pick the right algorithm given density without prompting.
10. Company Context
- Amazon: L5/L6; “warehouse network design” / “AWS region interconnect cost.”
- Google: L5/L6; data-center cabling, fiber-laying problems.
- Microsoft: L65+; Azure network topology.
- LinkedIn: L5; cluster formation in connectivity graphs.
Scorecard for strong: “Recognized MST, picked Prim for density reasons, cited cut property, implemented cleanly, mentioned Kruskal as alternative + Steiner tree if subset constraint added.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Recognize MST | “It’s connecting things” | Names MST | + picks algorithm by density | + Borůvka, Steiner, Manhattan MST O(V log V) |
| Correctness | Hand-wavy | Mentions greedy | Cites cut property | + sketches proof + cycle property |
| Implementation | Buggy | One works | Both clean | + array-Prim O(V²) optimization |
| Variants | None | LC 1135 | + LC 1168 super-source trick | + Steiner tree NP-hardness + LP relaxation |
12. Level Delta
- Mid (L4): One MST algorithm correct (Prim or Kruskal); recognizes the problem.
- Senior (L5): + both algorithms + density justifies choice + LC 1168 super-source pattern.
- Staff (L6): + Borůvka’s algorithm + Steiner tree NP-hardness + Manhattan MST O(V log V) sketch + LC 1631 minimax connection.
- Principal (L7+): + LP relaxation framing of MST + production network-design pipelines + multi-criteria MST (cost + latency) + distributed MST (GHS algorithm 1983).
13. Follow-up Q&A
Q1: What if some edges are mandatory (must be in the tree)? A1: Contract them first (union endpoints in DSU; cost += sum). Then run MST on the remainder.
Q2: What if you can only connect a SUBSET of “important” nodes (others are optional intermediates)? A2: That’s the Steiner Tree problem — NP-hard. Heuristics: minimum spanning tree on the important nodes (often a 2-approx).
Q3: What if edges are added DYNAMICALLY over time and you want the MST at each timestep? A3: Link-cut trees give O(log n) per edge insert/delete (Sleator-Tarjan).
Q4: Tie-breaking when multiple MSTs exist? A4: Any spanning tree of minimum weight is valid. To make it unique, perturb weights with small epsilons or break ties lex.
Q5: How does this generalize to k-edge-connected spanning subgraphs? A5: For k=2 (bridge-connected), polynomial via matroid intersection; for general k, harder. Standard MST is the k=1 case.
14. Solution Walkthrough
See solution.py:
min_cost_connect_prim_array— O(V²) Prim withmin_costarray (canonical).min_cost_connect_kruskal— sort all edges, union-find.min_cost_connect_brute— try every spanning tree (Cayley enumeration via Prüfer). For stress; works only n ≤ 6.
15. Beyond the Problem
Principal code review: “MST is the prototypical greedy-is-optimal proof in algorithms textbooks. The cut property tells you that local cheapest choices, made carefully, compose into a global optimum — but the carefulness matters: you must choose edges crossing some cut, not arbitrary edges. Kruskal exploits this via global edge-sorting; Prim via expanding-front. Borůvka via parallel cheapest-edge-per-component. Three different orchestrations of the same greedy idea. In production network design at AWS/GCP scale, MST is the seed; real systems layer on redundancy (k-edge-connected), latency constraints, and multi-criteria optimization, eventually becoming integer programs. But the MST intuition — ‘spanning tree’ as the cheapest ‘just-barely-connected’ state — survives. Knowing when MST is enough vs when you need more machinery is the senior+ judgment call.”
Hints — p66 Min Cost to Connect All Points
Hint 1. “Connect all points with minimum total cost” = Minimum Spanning Tree (MST) on the complete graph whose edge weights are pairwise Manhattan distances.
Hint 2. Two classical MST algorithms:
- Kruskal: sort all edges, scan in order, add if endpoints are in different components (Union-Find).
- Prim: grow MST from a starting node; at each step add the cheapest edge leaving the current tree.
Both correct via the cut property (cheapest edge crossing any cut belongs to some MST).
Hint 3. Edge count = V·(V-1)/2 ≈ V². That’s DENSE. Array-based Prim is O(V²); heap-Prim and Kruskal are O(V² log V). Array-Prim wins.
Hint 4. Array-Prim implementation: keep min_cost[v] = cheapest edge from {in-MST} to v. Each iteration: pick the unfinished node u with smallest min_cost[u]; add its cost to total; update min_cost[v] for all unfinished v using dist(u, v).
Hint 5. Initialize min_cost[0] = 0 (so the first iteration picks node 0 with cost 0). After V iterations all nodes are in the MST and total is the answer.
If stuck: see solution.py.
p67 — Is Graph Bipartite?
Source: LeetCode 785 · Medium · Topics: Graph, BFS, DFS, 2-coloring Companies (2024–2025): Meta (high), Google (medium-high), Amazon (medium), Microsoft (medium) Loop position: Bipartite check canonical. Equivalent to “graph has no odd cycle.” Tests 2-coloring via BFS/DFS — a clean, named, provable algorithm that’s the gateway to matching theory.
1. Quick Context
Undirected graph as adjacency list graph[i]. Return True if you can partition the nodes into two sets A and B such that every edge goes between A and B.
Theorem (König 1936): A graph is bipartite ⟺ it contains no odd-length cycle.
Algorithm: 2-color via BFS. Color start node 0; alternate colors across edges. Conflict → not bipartite.
color = [-1] * n
for start in range(n):
if color[start] != -1: continue
color[start] = 0
q = deque([start])
while q:
u = q.popleft()
for v in graph[u]:
if color[v] == -1:
color[v] = 1 - color[u]
q.append(v)
elif color[v] == color[u]:
return False
return True
O(V + E). Iterate over all start nodes to handle disconnected components.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/is-graph-bipartite/
STOP. 20-min timer.
3. Prerequisite Concepts
- BFS / DFS traversal of an undirected graph.
- Disconnected components (must restart traversal from each unvisited node).
- The 2-coloring invariant: each edge has endpoints of different colors.
- (Background) König’s theorem: bipartite ⟺ no odd cycle.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Partition nodes into two sets so that every edge crosses the partition. Equivalent: 2-color the graph.
2. Clarify: Self-loops? (A self-loop is a length-1 cycle = odd cycle → not bipartite. LC’s graph has none typically.) Multi-edges? Don’t change bipartiteness.
3. Examples: graph = [[1,3],[0,2],[1,3],[0,2]] (4-cycle) → True. graph = [[1,2,3],[0,2],[0,1,3],[0,2]] (has triangle 0-1-2) → False.
5. Brute force: Try all 2^n colorings; verify each. Exponential.
6. Target: O(V + E).
7. Pattern: BFS/DFS + 2-coloring.
8. Develop: see solution.py.
9. Test: disconnected components (each must be independently bipartite); single node (True); single edge (True); triangle (False); odd cycle of length 5 (False); even cycle (True).
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 König’s theorem — the why
Bipartite ⟺ no odd cycle.
(⇒) If graph is bipartite with parts A, B, any cycle alternates A-B-A-B-… and must have even length to return to start.
(⇐) Suppose no odd cycle. BFS from any vertex; color by BFS-depth parity. Claim: this is a valid 2-coloring. If an edge (u, v) has same-color endpoints, then depth(u) ≡ depth(v) (mod 2), but |depth(u) − depth(v)| ≤ 1 (BFS tree edge) so depth(u) = depth(v). The path u→LCA→v in the BFS tree plus edge (u,v) is an odd cycle — contradiction.
This is why BFS-by-depth-parity works as the coloring rule.
6.2 DFS variant
DFS works just as well: color each node opposite to its parent. Conflict on a back-edge with same color → not bipartite.
def dfs(u, c):
color[u] = c
for v in graph[u]:
if color[v] == -1:
if not dfs(v, 1 - c): return False
elif color[v] == c:
return False
return True
Recursive risks stack overflow on long chains (n up to 10⁴ on LC is borderline). Iterative DFS with explicit stack is safer.
6.3 Disconnected components
Critical: iterate over ALL starting nodes (for start in range(n)), not just one. A disconnected graph is bipartite ⟺ EVERY component is bipartite.
Common bug: starting BFS only from node 0 — silently passes connected tests, fails on disconnected ones.
6.4 Connection to matching theory
Bipartite graphs admit polynomial-time maximum matching (König’s theorem: max matching = min vertex cover, both polynomial via augmenting paths / Hopcroft-Karp O(E√V)). Non-bipartite max matching is harder (Blossom algorithm, Edmonds 1965, O(V³)). The “is bipartite” check is the gatekeeper for which matching algorithm to deploy.
6.5 Union-Find variant
You can also detect bipartiteness via DSU with a parity-weighted version:
- For each edge (u, v): union(u, v) with weight 1 (different colors).
- If u and v are already in the same component, check that the parity-distance from u to root XOR distance from v to root = 1.
More complex than BFS for this problem but generalizes to constraint-satisfaction graphs (e.g., “u and v differ by k mod m”).
6.6 Why “is bipartite” is a fundamental check
Many problems implicitly assume bipartiteness:
- Stable matching (Gale-Shapley) requires bipartite.
- Maximum flow problems on bipartite networks reduce to matching.
- Graph coloring with 2 colors = bipartite check.
Knowing the check costs O(V + E) tells you when the cheap algorithm suffices.
6.7 LC 886 Possible Bipartition
Same problem, dressed up: “Given dislike pairs, can you split into two groups so no two who dislike each other are together?” Build graph from pairs; run bipartite check.
LC 785 (this one) gives you the graph; LC 886 makes you build it. Solution is identical.
7. Anti-Pattern Analysis
- Start BFS only from node 0 — fails on disconnected components.
- Color with
0/1and use truthy checkif color[v]— node 0 colored 0 evaluates falsy; bug. Use-1as uncolored sentinel. - Recursive DFS without depth-limit awareness on huge chains — stack overflow.
- Try
setinstead of[-1]*n— slower; lookups become hashable-key overhead. - Skip the second check (
elif color[v] == color[u]) — silent True on non-bipartite graphs. - Bipartite via 2^n brute force — would technically work but trivially blows up; not a serious option.
8. Skills & Takeaways
- 2-coloring as a BFS/DFS invariant.
- König’s theorem and the odd-cycle characterization.
- Disconnected-component handling: iterate over all starts.
- Recognize bipartite check as a gatekeeper for matching / coloring problems.
9. When to Move On
- Solve LC 785 cold in 12 min with BFS.
- Solve LC 886 cold in 15 min (with graph construction).
- State König’s theorem and sketch the proof in 60 seconds.
- Convert your BFS to iterative DFS.
10. Company Context
- Meta: E5; appears as a sub-problem in social-graph community detection / dislike-grouping.
- Google: L5; ad-placement bipartite matching prep.
- Amazon: L5; order-fulfillment bipartite assignment.
Scorecard for strong: “BFS 2-color over all components, used -1 sentinel, cited König’s theorem, mentioned matching applications.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Algorithm | Vague | BFS, single component | + all components, clean | + DFS variant + DSU with parity |
| Proof | None | “alternate colors” | König’s theorem stated | + sketches proof + connects to matching |
| Edge cases | Misses disconnected | Catches with prompt | Handles cleanly | + self-loop + multi-edge analyzed |
| Applications | None | “graph coloring” | + matching theory | + LP relaxation + Hopcroft-Karp + Blossom |
12. Level Delta
- Mid (L4): BFS 2-color, handles disconnected components, returns correct answer.
- Senior (L5): + DFS variant + König’s theorem + LC 886 reduction.
- Staff (L6): + DSU with parity for general constraint graphs + connection to bipartite matching (Hopcroft-Karp O(E√V)) + Blossom for non-bipartite.
- Principal (L7+): + matroid generalization + LP relaxation + production matching systems (ride-hailing, ad-auction).
13. Follow-up Q&A
Q1: Now find the two parts explicitly.
A1: Return color array. Nodes with color 0 = part A, color 1 = part B.
Q2: What if the graph might be directed? A2: Bipartiteness is an undirected concept. For directed, ignore directions (treat as undirected) and run the same check.
Q3: What if you must split into THREE groups (3-coloring)? A3: 3-coloring is NP-complete (vs polynomial for 2). Use backtracking + pruning, or heuristics.
Q4: Now ALSO find the maximum matching in the bipartite graph. A4: Hopcroft-Karp O(E√V); or Hungarian algorithm for weighted bipartite matching.
Q5: What if edges are weighted with “this edge prefers same-side” weights? A5: That’s the MAX-CUT problem (find a cut maximizing crossing-edge weight). NP-hard in general; 0.878-approx via Goemans-Williamson SDP. Vastly harder than 2-coloring.
14. Solution Walkthrough
See solution.py:
is_bipartite_bfs— BFS 2-coloring (canonical).is_bipartite_dfs— iterative DFS 2-coloring.is_bipartite_brute— try all 2^n colorings (n ≤ 8 for stress only).
15. Beyond the Problem
Principal code review: “Bipartite check is one of those algorithms that looks trivial but anchors an entire theory. König’s theorem connects bipartiteness, matching, vertex cover, and LP duality — four notions that turn out to be the same on bipartite graphs and meaningfully different elsewhere. The 30-line BFS you wrote tells you whether you can deploy Hopcroft-Karp’s O(E√V) matching or have to escalate to Edmonds’ O(V³) Blossom. In production systems — ride-hailing matching, ad-auction allocation, kidney-exchange chains — that O(V³) vs O(E√V) choice is the difference between real-time and batch. Memorize the BFS; learn the theorem; respect the consequences.”
Hints — p67 Is Graph Bipartite?
Hint 1. Bipartite means you can split nodes into two groups A and B such that every edge crosses A↔B. Equivalently, you can 2-color the graph with no monochromatic edge.
Hint 2. König’s theorem: a graph is bipartite ⟺ it contains no odd-length cycle. You don’t need to find odd cycles explicitly — the coloring algorithm detects them automatically.
Hint 3. BFS from any starting node. Color the start node 0. Each neighbor gets the opposite color. Maintain color[v] ∈ {-1, 0, 1} (-1 = uncolored).
Hint 4. While exploring, if you encounter an already-colored neighbor whose color equals the current node’s color → odd cycle exists → return False.
Hint 5. Critical: iterate over ALL nodes as potential starting points (for start in range(n): if color[start] == -1: ...). The graph may be disconnected; every component must be independently bipartite.
If stuck: see solution.py.
p68 — Critical Connections in a Network
Source: LeetCode 1192 · Hard · Topics: Graph, Tarjan, Bridges, DFS Companies (2024–2025): Amazon (high), Google (medium-high), Meta (medium), Microsoft (medium) Loop position: Tarjan’s bridge-finding algorithm. The named algorithm interviewers reach for at L5+. Tests
disc/lowinvariants — the single most-tested DFS-with-timestamps pattern after standard DFS itself.
1. Quick Context
Undirected graph (n servers, connections list). A bridge (a.k.a. critical edge / cut edge) is an edge whose removal disconnects the graph. Return all bridges.
Tarjan’s algorithm (1972): single DFS with two timestamps per node:
disc[u]= DFS discovery order (when u was first visited).low[u]= lowestdiscreachable from u’s subtree via tree edges OR ONE back-edge.
Bridge condition: edge (u, v) where v is u’s DFS child is a bridge ⟺ low[v] > disc[u]. (v’s subtree can’t reach above u via any back-edge → cutting (u,v) disconnects.)
O(V + E). Single DFS pass.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/critical-connections-in-a-network/
STOP. 40-min timer. This is HARD.
3. Prerequisite Concepts
- DFS tree (tree edges, back edges, forward edges, cross edges).
- For UNDIRECTED graphs: only tree edges and back edges (no forward or cross).
- DFS timestamps (discovery time).
- Iterative DFS — recursion limit issue on LC (n up to 10⁵).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Find all edges whose removal increases the number of connected components by 1.
2. Clarify: Multi-edges? (LC 1192 says no.) Self-loops? (Never bridges.) Connected initially? (Usually yes; algorithm handles disconnected anyway.)
3. Examples: n=4, connections=[[0,1],[1,2],[2,0],[1,3]] → [[1,3]]. The triangle 0-1-2 has no bridges; the spoke 1-3 is critical.
5. Brute force: Remove each edge, check connectivity with DFS, restore. O(E · (V+E)).
6. Target: O(V + E) via Tarjan.
7. Pattern: Tarjan bridge-finding (DFS with disc and low).
8. Develop: see solution.py.
9. Test: triangle (no bridges); single edge (it’s a bridge); two triangles joined by one edge (that edge is the bridge); tree (every edge is a bridge).
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why low[v] > disc[u] characterizes bridges
In a DFS tree of an undirected graph, the only non-tree edges are back edges (to an ancestor). Forward and cross edges don’t exist (because we’d have visited the destination already by the time we go back up; but in undirected, “back” and “forward” collapse).
For tree edge (u, v) (v is u’s child): removing it disconnects v’s subtree from the rest IFF no back edge from v’s subtree goes to u or an ancestor of u. That’s exactly low[v] > disc[u].
If low[v] ≤ disc[u], there’s an alternate path — bridge is NOT critical.
6.2 Computing low[u] precisely
disc[u] = timer; low[u] = timer; timer += 1
for each neighbor v of u:
if v is parent: continue # don't go back along the SAME edge we came in
if disc[v] != -1: # already visited (back edge)
low[u] = min(low[u], disc[v])
else: # tree edge
dfs(v, parent=u)
low[u] = min(low[u], low[v])
if low[v] > disc[u]:
bridges.append((u, v))
Subtle: When taking back edge to neighbor v, use disc[v], NOT low[v]. (Using low[v] would be wrong — we want the lowest disc reachable, and from a back edge we directly reach v.)
6.3 The “parent” exclusion subtlety (multi-edges)
The “don’t go back to parent” rule works only if there are NO multi-edges. With multi-edges, the second edge to parent is a genuine back edge.
Robust fix: track which EDGE INDEX you came in on (not just the parent node). Skip exactly that one edge, not all edges to the parent node.
6.4 Iterative DFS — required for LC 1192
LC constraints (n up to 10⁵) blow Python’s default recursion limit of 1000. Must convert to iterative DFS with an explicit stack.
Iterative Tarjan is tricky: each stack frame remembers its iterator over neighbors AND a “child index” to know where to resume after a child returns. See solution.py.
6.5 Bridges vs articulation points (cut vertices)
- Bridge: edge whose removal disconnects.
- Articulation point: vertex whose removal disconnects.
Different concepts! Same algorithm framework with different conditions:
- Bridge:
low[v] > disc[u](strictly greater). - Articulation point (non-root):
low[v] ≥ disc[u](greater or equal). - Articulation root: root is articulation if it has ≥ 2 DFS-children.
6.6 Block-cut tree
The 2-edge-connected components (sub-graphs with no bridges) form a forest when contracted; the bridges are the inter-component edges. This block-cut tree is the canonical structure for “how is this graph held together.”
6.7 Online / dynamic bridge maintenance
Static bridges in O(V+E). For DYNAMIC (edges added/removed), Holm-Lichtenberg-Thorup gives O(log² n) per operation. Advanced; mention only at Staff+.
6.8 Generalization: k-edge-connectivity
Bridge = “1-edge-connectivity weakness.” For 2-edge-connectivity (no 2 edges whose removal disconnects), the algorithm extends but becomes more involved. Gomory-Hu trees compute all-pairs k-edge-connectivity.
7. Anti-Pattern Analysis
- Recursive DFS for LC 1192 — Python recursion limit (default 1000); n up to 10⁵; stack overflow on linear chains.
- Use
low[v]instead ofdisc[v]on back edges — silently miscomputes low; wrong answer. - Exclude parent by NODE not by EDGE — fails on multi-edges.
- Compare
low[v] >= disc[u]— that’s the ARTICULATION POINT condition; for bridges use strict>. - Build adjacency without edge indices — multi-edge subtlety can’t be handled.
- Brute O(E·(V+E)) — works for small inputs, TLE on LC 1192’s constraints.
8. Skills & Takeaways
- Tarjan’s
disc/lowframework — the workhorse of advanced DFS algorithms. - The DFS tree structure on undirected graphs (only tree + back edges).
- Bridge ⟺
low[v] > disc[u]; articulation point uses≥(slight but important). - Iterative DFS with neighbor iterators for large inputs.
9. When to Move On
- Implement iterative Tarjan bridge-finder cold in 30 min.
- Explain why
low[v] > disc[u]characterizes bridges in 90 seconds. - Solve articulation points (LC has it as a related problem) with the modified condition.
- Recognize the algorithm by name when an interviewer says “find all critical edges.”
10. Company Context
- Amazon: L6 onsite; service-mesh “find single-point-of-failure connections.”
- Google: L6; SRE networking, fault-tolerant topology design.
- Meta: E6; data-center inter-rack links analysis.
- Microsoft: L65+; Azure VNet topology.
Scorecard for strong: “Named Tarjan, used disc/low, wrote iterative DFS for scale, used edge indices to handle multi-edges, distinguished bridges from articulation points.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Algorithm | Brute O(E·(V+E)) | Recursive Tarjan (TLE) | Iterative Tarjan correct | + edge indices + articulation points |
| Invariant | “low value” hand-wave | Computes correctly | Articulates low[v] > disc[u] precisely | + sketches proof + DFS-tree structure |
| Scale | Doesn’t think | Hits recursion limit, fixes | Anticipates, writes iterative | + block-cut tree + dynamic bridges |
| Generalization | None | “what about vertices?” | Articulation points + condition difference | + 2-edge-connectivity + Gomory-Hu |
12. Level Delta
- Mid (L4): Tarjan recursive (works for small input); names algorithm.
- Senior (L5): + iterative DFS for scale + correctly handles parent exclusion + states bridge condition precisely.
- Staff (L6): + articulation points via
≥+ edge-index parent exclusion for multi-edges + block-cut tree + 2-edge-connected components. - Principal (L7+): + dynamic bridges (Holm-Lichtenberg-Thorup) + Gomory-Hu for k-edge-connectivity + production SRE applications + LP relaxation framing of connectivity.
13. Follow-up Q&A
Q1: Find articulation points (cut vertices) instead.
A1: Same DFS framework, change condition to low[v] >= disc[u] for non-root; root is articulation if it has ≥ 2 DFS-children.
Q2: Find ALL 2-edge-connected components. A2: Same DFS but skip bridges during a second DFS to identify components. Or use the block-cut tree.
Q3: Multi-edges? A3: Track parent EDGE INDEX, not parent NODE. Two edges to same node are both legitimate (one tree, one back).
Q4: Edges added/removed dynamically; report bridges after each. A4: Holm-Lichtenberg-Thorup O(log² n) per update; complex implementation.
Q5: Why does iterative DFS need such elaborate state?
A5: Recursive DFS implicitly carries for v in adj[u] iteration state via the call stack frame. Iterative DFS must explicitly save (current node, iterator into its neighbor list, child being processed) — otherwise you can’t resume properly after a child returns.
14. Solution Walkthrough
See solution.py:
critical_connections_tarjan— iterative Tarjan; canonical.critical_connections_tarjan_recursive— recursive variant; only safe for small inputs.critical_connections_brute— remove each edge, check connectivity (oracle).
15. Beyond the Problem
Principal code review: “Tarjan’s bridge algorithm (1972) and its sibling articulation-point algorithm are the cleanest example of how a single DFS pass, properly instrumented, computes a global structural property of a graph. The
disc/lowpair is one of those data-structure inventions that feels obvious in retrospect: every node maintains both ‘when did I get here’ and ‘what’s the highest I could go back to.’ From those two scalars you derive bridge-ness, articulation-ness, biconnected components, the block-cut tree, even strongly-connected components (for directed graphs, with a sign flip). In production SRE — fault-tolerance analysis of microservice topologies, redundancy planning, blast-radius computations — these algorithms run on graphs with millions of edges and they finish in linear time. The clarity of ‘low[v] > disc[u] means cut this and the subtree falls off’ is what makes Tarjan one of the field’s most-cited papers. Memorize the seven lines; respect the proof.”
Hints — p68 Critical Connections in a Network
Hint 1. A bridge (critical edge) is an edge whose removal disconnects the graph. Brute force: for each edge, remove it and run DFS to check connectivity. O(E·(V+E)) — too slow for LC’s constraints (n up to 10⁵).
Hint 2. Tarjan’s bridge algorithm finds all bridges in a single DFS pass, O(V+E). Assign each node two timestamps:
disc[u]= when u was first discovered (DFS order).low[u]= lowestdiscreachable from u’s subtree via tree edges + AT MOST ONE back edge.
Hint 3. Bridge condition: a tree edge (u, v) where v is u’s DFS child is a bridge ⟺ low[v] > disc[u]. (Meaning: v’s subtree can’t reach above u via any back edge → cutting (u,v) disconnects.)
Hint 4. Computing low[u]:
- Initialize
low[u] = disc[u]. - For each neighbor v: if v is the PARENT (the edge you came in on), skip.
- If v is already visited (back edge):
low[u] = min(low[u], disc[v]). Usedisc[v], notlow[v]— this is the most common bug. - If v is unvisited (tree edge): recurse, then
low[u] = min(low[u], low[v]). Check bridge condition.
Hint 5. LC 1192’s n=10⁵ blows Python’s default recursion limit (1000). You must convert to iterative DFS with an explicit stack carrying (node, parent_edge_id, iterator over neighbors). After popping a child, integrate its low into the parent and check the bridge condition.
If stuck: see solution.py.
p69 — Alien Dictionary
Source: LeetCode 269 · Hard (LC Premium) · Topics: Graph, Topo Sort, Strings Companies (2024–2025): Google (very high), Meta (high), Amazon (high), Microsoft (medium-high) Loop position: The hardest topological sort variant on LC. You don’t get the graph — you must INFER edges from adjacent word comparisons, handle the prefix-violation edge case correctly, and detect cycles. Three independent failure modes; missing any = wrong answer.
1. Quick Context
A list of words from an alien language, sorted lexicographically in some unknown alphabet order. Derive that alphabet order. Return empty string if impossible.
Strategy:
- Build a directed graph over CHARACTERS where edge
a → bmeans “a comes before b.” - Add all distinct characters as nodes (even those with no edges).
- For each adjacent pair
words[i], words[i+1]: find first differing character; that gives ONE edge. - Prefix-violation check: if
words[i]is longer thanwords[i+1]ANDwords[i+1]is a prefix ofwords[i](e.g.,["abc", "ab"]), it’s impossible → return “”. - Topo sort (Kahn or DFS); cycle → return “”.
O(V + E + N·L) where N = number of words, L = max word length.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/alien-dictionary/ (LC Premium)
STOP. 40-min timer. Three independent edge cases must all be handled.
3. Prerequisite Concepts
- Topological sort (Kahn’s BFS — see p62).
- Cycle detection in directed graph.
- Lexicographic order: comparison stops at first differing char.
- The “prefix” subtlety: in dictionary order, shorter prefix comes before longer extension.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Reverse-engineer an alphabet from sorted-word evidence.
2. Clarify: All chars lowercase? (Typically yes, but sometimes any chars.) Are all chars unique across words? No — they may repeat. Tie order ambiguous? (Multiple valid orderings; any one is acceptable.)
3. Examples:
["wrt","wrf","er","ett","rftt"]→"wertf".["abc","ab"]→""(prefix violation).["z","x","z"]→""(cycle: z→x, x→z).
5. Brute force: Try all permutations of distinct chars, check each against the word list. 26! too large but conceptually correct.
6. Target: O(V + E + N·L). V = distinct chars (≤26), E ≤ N-1.
7. Pattern: Topo sort over inferred graph.
8. Develop: see solution.py.
9. Test: prefix violation; cycle; single word (any permutation of its chars valid); duplicate adjacent words; chars-with-no-constraints (must still appear in result).
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Where do edges come from?
Between two adjacent words w1, w2, walk char-by-char:
- If
w1[i] != w2[i]: add edgew1[i] → w2[i]. STOP walking — subsequent characters give no info. - If
w1[i] == w2[i]: continue. - If we exhaust
w2(sow2is a prefix ofw1) ANDlen(w1) > len(w2): invalid input — a prefix must come BEFORE its extension in dictionary order, so["abc","ab"]is impossible.
The “first differing character” rule is critical: characters AFTER the first difference give NO ordering information. (abxxx vs acyyy only tells us b < c, not x vs y.)
6.2 Why include ALL characters as nodes
The result must include every character that appears in any word. A character with no incoming OR outgoing constraints (e.g., only appears in one word with no neighbors) must still be in the output.
Common bug: build the graph only from inferred edges, miss isolated nodes. The final string has the wrong character count.
6.3 Three failure modes
- Prefix violation: shorter word that’s a prefix appears AFTER its extension.
- Cycle: e.g., a→b and b→a inferred. Detected via Kahn (final order length < V) or DFS (back edge).
- No solution despite valid input: shouldn’t happen if input is well-formed; impossible only via 1 or 2.
Each is independent. Tests target all three.
6.4 Kahn vs DFS for topo sort
Kahn (BFS): Build indegrees; queue all nodes with indeg=0; pop, append to result, decrement neighbors. Cycle iff len(result) < V.
DFS with three-color: WHITE=0, GRAY=1, BLACK=2. Cycle iff we hit a GRAY (back edge during DFS).
Both O(V+E). Kahn is slightly cleaner here; gives an explicit order. DFS gives the order in reverse-post-order.
6.5 Multiple valid orderings
When the partial order is incomplete (e.g., only a→b is known, c is unconstrained), multiple total orders are valid. LC accepts any. Kahn with a deque (FIFO) gives one consistent order; using a min-heap gives the lex-smallest.
6.6 Connection to scheduling
Alien Dictionary is just LC 207 Course Schedule with the graph hidden in word-comparison data. Once edges are inferred, it’s a textbook topo sort. The “hardness” is the graph-construction phase + edge cases, not the sorting itself.
6.7 Generalization: order from preferences
Any “I have evidence of pairwise orderings; derive a global order” problem is alien-dictionary-shaped. Examples: tournament rankings from match outcomes, package dependency resolution from build logs, sentence reconstruction from word-adjacency frequencies. Topo sort + cycle detection is the universal hammer.
7. Anti-Pattern Analysis
- Compare entire words, not first-difference char — wrong edges (extra info from later chars).
- Forget the prefix-violation check — silently returns valid order on impossible input.
- Don’t add all chars as nodes — output missing chars; wrong length.
- Use indegree dict without initializing isolated chars to 0 — those chars never get into the Kahn queue.
- Cycle detection by “result not equal to input chars” — only checks length; misses cycles where some chars happen to have indeg 0.
- Compare adjacent words once and stop on first edge but then process EXTRA edges from later chars — silently adds wrong edges.
8. Skills & Takeaways
- Topo sort on inferred graph.
- “First differing character” rule for lex order.
- Prefix-violation invalidity.
- Cycle detection via Kahn’s len-check.
- Universal pattern: pairwise ordering evidence → topo sort.
9. When to Move On
- Solve LC 269 cold in 30 min handling all 3 edge cases.
- State the prefix-violation check in 30 seconds.
- Distinguish “no edges between two chars yet” from “cycle.”
- Solve LC 207, LC 210 (Course Schedule I/II), LC 444 (Sequence Reconstruction) cold.
10. Company Context
- Google: L5/L6; the most-asked Google graph problem alongside LC 207.
- Meta: E5; tests both topo sort AND attention to edge cases.
- Amazon: L6; comes with prefix-violation as the “did you really test it” gotcha.
- Microsoft: L65; similar.
Scorecard for strong: “Built graph correctly from first-differing char, handled prefix violation, added all chars as nodes, used Kahn with cycle detection, mentioned multiple valid orderings.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Graph construction | Compares full words | First diff char | + stops processing after first diff | + handles all 3 edge cases unprompted |
| Prefix violation | Misses | After prompt | Cold | + articulates why (prefix must precede extension) |
| Isolated nodes | Misses | After fail | Adds preemptively | + explains node set ≠ edge endpoint set |
| Cycle detection | None | Kahn len check | + DFS three-color alt | + LC 444 unique-order extension |
12. Level Delta
- Mid (L4): Handles standard case; misses one of the three edge cases.
- Senior (L5): All three edge cases right; cleanly built graph; Kahn or DFS; cites pattern.
- Staff (L6): + LC 444 (Sequence Reconstruction — uniqueness check) + lex-smallest topo order via heap + multiple valid orderings discussion.
- Principal (L7+): + general “pairwise evidence → partial order → topo sort” framing + connection to learning-to-rank + production dependency-resolution systems (Cargo, Nix, NPM).
13. Follow-up Q&A
Q1: Multiple valid orderings — return the lex-smallest. A1: Replace deque with min-heap in Kahn’s algorithm. O((V+E) log V).
Q2: Is the order UNIQUE? (LC 444 Sequence Reconstruction) A2: Yes iff at every Kahn step the queue has exactly one element. If ever ≥ 2 simultaneously, order is ambiguous.
Q3: What if input has only one word? A3: No edges can be inferred. Return that word’s chars in any order (typically their original order in the word, deduplicated).
Q4: What if we’re allowed to be wrong on a few inputs (statistical)? A4: Use feedback-arc-set heuristics; treat as approximate ranking aggregation.
Q5: What if there’s a typo in the input — can you detect? A5: A “typo” might create a spurious cycle or prefix violation. You can identify the minimum-edge edit to make it consistent; that’s an NP-hard variant (minimum feedback arc set).
14. Solution Walkthrough
See solution.py:
alien_order_kahn— Kahn’s BFS topo sort (canonical).alien_order_dfs— iterative DFS three-color variant.alien_order_brute— enumerate permutations of distinct chars; verify each (only for stress with ≤6 chars).
15. Beyond the Problem
Principal code review: “Alien Dictionary is the prototypical ‘graph is hidden in the data’ problem. Half the work is reading the input correctly: extracting evidence from pairwise word comparisons under the strict first-differing-character rule, detecting the prefix-violation that immediately invalidates the input. The other half — topo sort with cycle detection — is the LC 207 you’ve already solved. This pattern recurs across systems: package managers reconstruct build order from declared dependencies; tournament systems infer rankings from match outcomes; learning-to-rank systems aggregate pairwise human preferences. The universal lesson is that ‘partial order + cycle detection + total ordering’ is one tool. When evidence is consistent, topo sort gives you the answer; when it’s not, you need feedback-arc-set heuristics. Knowing which case you’re in is the senior judgment.”
Hints — p69 Alien Dictionary
Hint 1. You’re given lex-sorted words in an unknown alphabet; derive an alphabet order. Pattern: topological sort over a graph of characters. The trick: you must INFER the edges from the word list.
Hint 2. For each adjacent pair words[i], words[i+1], find the first differing character: that gives ONE edge (w1[j] → w2[j]). STOP comparing after that — later characters give NO ordering information.
Hint 3. Prefix-violation edge case: if words[i] is longer than words[i+1] AND words[i+1] is a prefix of words[i] (e.g., ["abc","ab"]), the input is impossible — return "".
Hint 4. Build the graph; collect all distinct characters (not just edge endpoints — isolated chars must still appear in the output). Run Kahn’s BFS topo sort: start with all chars of indeg 0, pop and append, decrement neighbors.
Hint 5. Cycle detection: if the final ordering’s length is less than the number of distinct characters, there’s a cycle — return "". The three failure modes (prefix violation, cycle, missing chars) are independent; test each.
If stuck: see solution.py.
p70 — Reconstruct Itinerary
Source: LeetCode 332 · Hard · Topics: Graph, Eulerian Path, Hierholzer, DFS Companies (2024–2025): Google (high), Amazon (medium-high), Meta (medium), Uber (medium) Loop position: The Eulerian path problem in disguise. Tests Hierholzer’s algorithm — one of the named graph algorithms that distinguishes Senior from Mid. Plus the lex tie-break subtlety that catches everyone.
1. Quick Context
Given a list of flight tickets (departure, arrival), construct the trip that uses every ticket exactly once, starting from “JFK”. When multiple valid itineraries exist, return the lexicographically smallest.
Recognize: This is finding an Eulerian path (a walk that uses every edge exactly once) in a directed multigraph, with a lexicographic tie-break.
Algorithm: Hierholzer’s, with destinations processed in sorted order (min-heap or pre-sorted list). The clever twist: append nodes to the route in post-order during DFS, then reverse at the end.
O((V+E) log E) due to the heap; could be O(V+E) with bucket sort.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/reconstruct-itinerary/
STOP. 35-min timer. The “reverse post-order” insight is non-obvious.
3. Prerequisite Concepts
- DFS over a multigraph (multiple edges between same pair allowed).
- Eulerian path conditions (we don’t need to verify — problem guarantees one exists).
- Min-heap for lexicographic ordering of destinations.
- Why post-order + reverse = correct Eulerian path output.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Find an Eulerian path starting at JFK, lex-smallest among valid ones.
2. Clarify: Always solvable? (LC promises yes.) Multiple tickets between same pair? (Yes — directed multigraph.) Output one itinerary.
3. Examples:
[["MUC","LHR"],["JFK","MUC"],["SFO","SJC"],["LHR","SFO"]]→["JFK","MUC","LHR","SFO","SJC"].[["JFK","SFO"],["JFK","ATL"],["SFO","ATL"],["ATL","JFK"],["ATL","SFO"]]→["JFK","ATL","JFK","SFO","ATL","SFO"](lex-smallest among multiple valid).
5. Brute force: Try all permutations of tickets; for each, check it forms a valid path starting JFK. O(N!). Only for small inputs.
6. Target: O((V+E) log E) via Hierholzer.
7. Pattern: Hierholzer’s Eulerian path with post-order DFS + lex tie-break via heap.
8. Develop: see solution.py.
9. Test: single ticket; loop (returns to JFK); branching where greedy choice leads to dead-end (Hierholzer recovers via post-order).
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why greedy lex-smallest DFS DOESN’T work directly
Naive: at each step go to the lex-smallest unvisited destination. Counterexample: at JFK with destinations {ATL, KUL} where ATL leads to a dead end but KUL has more edges. Greedy goes ATL first → can’t finish → wrong.
The correct algorithm:
- Recursive DFS, but APPEND TO ROUTE AT THE END (post-order), not the beginning (pre-order).
- This guarantees that any “dead-end” subgraph gets added LAST in the right order; the longer paths get explored fully first.
6.2 Why post-order + reverse works (the proof intuition)
When DFS hits a dead end (no more outgoing edges from current node), that node MUST be the LAST node in the itinerary (or last in its subtour). So append at the end of route. As DFS unwinds, nodes are added in reverse order. Reverse the entire route at the end.
Formally: this is Hierholzer’s algorithm. It works on any graph with an Eulerian path; the lex requirement adds the heap.
6.3 Heap of destinations for lex tie-break
For each node, store outgoing destinations in a min-heap. Each DFS step pops the smallest destination, removes that edge (decrements edge count), recurses.
Without the heap, you’d sort destinations once, but as edges are consumed you’d need to re-find the next smallest — heap is cleaner.
6.4 Iterative Hierholzer
The recursive version is elegant but the iterative version is the standard production form:
stack = [start]
route = []
while stack:
u = stack[-1]
if heap[u]:
v = heappop(heap[u])
stack.append(v)
else:
route.append(stack.pop())
return reversed(route)
Same post-order + reverse, but using explicit stack. Beautiful in its compactness.
6.5 Eulerian path conditions (when you’d verify)
A directed graph has an Eulerian PATH iff:
- At most one vertex has
out_degree - in_degree = 1(the start). - At most one vertex has
in_degree - out_degree = 1(the end). - All other vertices have equal in/out degree.
- The graph is connected (considering only vertices with edges).
LC 332 promises this; in interview you may be asked to verify.
6.6 Eulerian CYCLE vs PATH
Cycle: starts and ends at same vertex; all vertices have in_deg = out_deg. Path: starts and ends at different vertices; exactly two vertices have unequal in/out.
Hierholzer’s works for both; the LC problem could be either depending on tickets.
6.7 Chinese Postman Problem (the general case)
If you’re allowed to traverse edges more than once (and want min total), it’s the Chinese Postman Problem — solvable in poly time for undirected, harder for directed (called “windy postman”). Out of scope for LC but worth knowing the name.
6.8 Lex-smallest via destination ordering — sufficient or not?
Claim: with destinations processed in lex order at each step, Hierholzer’s post-order + reverse gives the lex-smallest Eulerian path. The proof uses an exchange argument — out of scope but worth knowing the result holds.
7. Anti-Pattern Analysis
- Greedy lex-smallest pre-order DFS — fails on the standard counterexample; gives wrong itinerary.
- Use the heap but forget to RE-INSERT when backtracking (rare bug; correct algorithm doesn’t backtrack at the edge level since post-order handles it).
- Sort destinations once, walk in order — heap is the clean tool because edges are popped non-trivially.
- Build adjacency as a Counter without sort — lose lex order.
- Recursive DFS with deep itineraries — Python recursion limit. LC 332 inputs are small; iterative is still safer.
- Append nodes in pre-order, reverse — wrong; pre-order doesn’t have the right property.
8. Skills & Takeaways
- Eulerian path recognition: “use every edge exactly once” → Hierholzer.
- Post-order DFS + reverse — a counterintuitive but correct pattern.
- Heap for lexicographic processing under edge removal.
- Iterative Hierholzer with explicit stack.
9. When to Move On
- Solve LC 332 cold in 30 min using iterative Hierholzer.
- Explain why pre-order greedy fails (the dead-end-first counterexample) in 2 minutes.
- Recognize Eulerian path/cycle in disguise (e.g., domino chains, DNA assembly fragments).
10. Company Context
- Google: L5/L6; classic “named algorithm” probe.
- Amazon: L6; logistics/routing flavor.
- Uber: L5; routing/dispatch context.
- Meta: E5; less common, harder DFS variant.
Scorecard for strong: “Recognized Eulerian path; named Hierholzer; explained why post-order; wrote iterative version; cited the dead-end counterexample for naive greedy.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Recognition | DFS attempt | “Eulerian-ish” | Names Hierholzer | + lex proof + LC variants |
| Greedy bug | Doesn’t see | After failing test | Anticipates | + cites counterexample upfront |
| Iterative form | None | Recursive only | Iterative | + Eulerian path conditions verified |
| Generalization | None | None | Notes Euler cycle vs path | + Chinese Postman + DNA assembly + de Bruijn graphs |
12. Level Delta
- Mid (L4): Recursive Hierholzer; works on small inputs; misses iterative.
- Senior (L5): Iterative Hierholzer + lex-smallest via heap + cites greedy counterexample + post-order intuition.
- Staff (L6): + Eulerian path/cycle conditions + verifies feasibility + de Bruijn graph application (DNA assembly).
- Principal (L7+): + Chinese Postman generalization + production routing (vehicle routing problem framing) + the Bose-Hierholzer history.
13. Follow-up Q&A
Q1: What if no Eulerian path exists? A1: Check in/out degree conditions and connectivity first; return [] or error.
Q2: What if input has duplicates (multiple identical tickets)? A2: Multigraph; treat as separate edges. Heap with repeats handles it naturally.
Q3: Find the lex-LARGEST itinerary instead. A3: Use max-heap (negate keys) instead of min-heap.
Q4: How is this related to de Bruijn sequences? A4: A de Bruijn sequence is an Eulerian cycle on a de Bruijn graph; same algorithm. DNA fragment assembly uses this.
Q5: Why does post-order + reverse work but pre-order doesn’t? A5: Post-order ensures dead-end subtours get added last (correct position in itinerary). Pre-order commits to a path before knowing if it leads to a dead end early.
14. Solution Walkthrough
See solution.py:
find_itinerary_hierholzer— recursive Hierholzer with min-heap; canonical.find_itinerary_iterative— iterative Hierholzer with explicit stack.find_itinerary_brute— permutation enumeration (only for stress with ≤6 tickets).
15. Beyond the Problem
Principal code review: “Hierholzer’s algorithm (1873) is one of those discoveries that crystalizes the post-order DFS pattern into a named technique. The intuition — ‘when you dead-end, you must be at the last node’ — is profound and counterintuitive; pre-order DFS feels natural but fails. The same algorithm powers DNA fragment assembly (de Bruijn graphs in computational biology), printer-driver-style flood fill of plotter commands, and a variety of routing problems. The Chinese Postman generalization (visit every edge with minimum total weight) extends to vehicle routing and street-sweeper scheduling. When you see ‘traverse every edge exactly once,’ the answer is always Hierholzer; the only question is which lex tie-break the problem wants. Memorize the iterative form — it’s seven lines and you’ll be asked it again.”
Hints — p70 Reconstruct Itinerary
Hint 1. Recognize the problem: use every ticket (edge) exactly once = Eulerian path in a directed multigraph. Named algorithm: Hierholzer’s.
Hint 2. Naive greedy (“always pick lex-smallest next destination”) FAILS. Counterexample: JFK can go to ATL (dead end) or KUL (long subtour); greedy picks ATL, gets stuck. Need a smarter approach.
Hint 3. Hierholzer’s trick: DFS but append nodes in POST-ORDER (after exploring all outgoing edges), then reverse the route at the end. Dead-end subtours naturally end up in the correct (last) position.
Hint 4. For lex-smallest: store each node’s destinations in a min-heap; always pop the smallest available destination. Each pop removes that ticket from consideration.
Hint 5. Iterative form (production-clean):
stack = ["JFK"]; route = []
while stack:
if heap[stack[-1]]:
stack.append(heappop(heap[stack[-1]]))
else:
route.append(stack.pop())
return route[::-1]
If stuck: see solution.py.
Week 14 — Advanced DP
Month 03, Week 2. The hardest DP patterns interviewers throw at Senior+ candidates.
Theme
Five DP variants that share a common challenge: the recurrence is non-obvious and the order of subproblem decomposition matters. Specifically:
- Interval DP (p71 Burst Balloons): split by last action, not first.
- State-machine DP over two pointers (p72 Regex Match): the
*operator’s “match 0 or many” forces branching. - 2D grid DP from string interleaving (p73): proving non-greedy via counterexample.
- Subsequence counting DP (p74 Distinct Subsequences): include/exclude on every match.
- Partition DP with palindrome preprocessing (p75): O(n²) palindrome table + O(n²) cut DP.
Daily Schedule
| Day | Problem | LC | Pattern |
|---|---|---|---|
| Mon | p71 Burst Balloons | 312 | Interval DP, split by LAST balloon |
| Tue | p72 Regular Expression Matching | 10 | 2D DP, * matches 0 or many |
| Wed | p73 Interleaving String | 97 | 2D DP, why greedy fails |
| Thu | p74 Distinct Subsequences | 115 | 2D DP, include/exclude on match |
| Fri | p75 Palindrome Partitioning II | 132 | 2-phase DP, min cuts |
Readiness Gate
You can leave this week when you can:
- Explain why interval DP for Burst Balloons splits on the LAST balloon (not first).
- Write the regex DP recurrence cold, articulating the
*case in plain English. - Give a concrete counterexample for why greedy interleaving check fails.
- State the include/exclude split for Distinct Subsequences.
- Compute the O(n²) palindrome table separately from the O(n²) cut DP.
Mental Model: “Subproblem Decomposition Choice”
For DP problems beyond the textbook, the key insight is what to enumerate at each step. Burst Balloons fails if you split on “first balloon to burst”; it works splitting on “last balloon remaining in interval (i,j).” This decomposition-direction sensitivity is the hallmark of Hard DP.
p71 — Burst Balloons
Source: LeetCode 312 · Hard · Topics: DP, Interval DP, Divide and Conquer Companies (2024–2025): Google (very high), Amazon (high), Meta (medium), Microsoft (medium) Loop position: The canonical “split on the LAST action” interval DP. If you split on the first balloon to burst, the subproblems depend on each other and DP fails. Splitting on the LAST balloon to burst in a range makes the two halves independent. This counterintuitive flip is the entire trick.
1. Quick Context
n balloons, each with a value. Bursting balloon i gives you nums[left] * nums[i] * nums[right] coins (where left/right are the currently adjacent unburst balloons). After bursting i, its left and right become adjacent. Out-of-bounds neighbors count as value 1. Maximize total coins.
Strategy: Pad nums with 1 on each end. Define dp[i][j] = max coins from bursting all balloons strictly between i and j. Recurrence: enumerate k (the LAST balloon to burst in (i,j)); when k bursts, its neighbors are exactly i and j (because everything else between them is already gone). Sum nums[i] * nums[k] * nums[j] + dp[i][k] + dp[k][j].
O(n³) time, O(n²) space.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/burst-balloons/
STOP. 45-min timer. This is HARD; almost no one gets it without the “last balloon” insight.
3. Prerequisite Concepts
- Interval DP (filling
dp[i][j]by increasing interval length). - Divide and conquer with the split point as the choice variable.
- Why splitting on FIRST action fails for some problems (subproblems coupled).
- Sentinel padding (1s on each end) to unify boundary cases.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Order n balloons for bursting; each burst earns left*self*right based on CURRENT neighbors; maximize.
2. Clarify: Are values positive? (Constraint says 0 ≤ nums[i] ≤ 100.) Range size? (n ≤ 500 → O(n³)=1.25×10⁸ borderline OK.)
3. Examples: [3,1,5,8] → 167. Optimal burst order: 1, 5, 3, 8 yielding 3·1·5 + 3·5·8 + 1·3·8 + 1·8·1 = 15+120+24+8 = 167.
5. Brute force: Try all permutations of burst orders. O(n!).
6. Target: O(n³) interval DP.
7. Pattern: Interval DP, split on LAST burst.
8. Develop: see solution.py.
9. Test: [1], [3,1,5,8] (canonical), [1,5], all zeros.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why splitting on FIRST balloon fails
If dp[i][j] = max coins from bursting balloons in (i,j) and you split on “first balloon k to burst”:
- Earn
nums[i] * nums[k] * nums[j]? No — when k bursts FIRST, its neighbors are whatever balloons are still in (i,j), not i and j directly. The remaining problem on (i,k) and (k,j) is not independent of past choices.
Splitting on LAST balloon to burst:
- When k is the last balloon remaining in (i,j) before bursting, ALL OTHERS in (i,j) are already gone. So k’s neighbors at the time it bursts are exactly i and j. Recurrence becomes clean:
nums[i]*nums[k]*nums[j] + dp[i][k] + dp[k][j].
This insight — “structure your DP around the last action, not the first” — recurs in matrix-chain multiplication, optimal BST construction, polygon triangulation, and any problem where actions affect the cost of subsequent actions.
6.2 The sentinel pad
Padding nums with 1 on each end unifies the boundary case. Without it, the first/last balloon burst would have only one neighbor. With it, every balloon always has two neighbors and the formula is uniform.
6.3 Interval DP filling order
Fill by increasing interval length len = j - i. Inner loops iterate over i (interval start), compute j = i + len, then iterate k from i+1 to j-1.
for length in range(2, n+1): # length of (i,j), exclusive endpoints
for i in range(n - length):
j = i + length
for k in range(i+1, j):
dp[i][j] = max(dp[i][j], dp[i][k] + dp[k][j] + nums[i]*nums[k]*nums[j])
6.4 Memoized recursion variant
@lru_cache on (i, j) recursing on choice of last k. Equivalent O(n³). Often easier to write under interview pressure.
6.5 Connection to Matrix Chain Multiplication
Both problems share the structure: pick a split point k in [i+1, j-1]; combine costs of (i,k) and (k,j) plus an interaction cost. MCM’s interaction is matrix multiplication cost p[i]*p[k]*p[j]; BB’s is nums[i]*nums[k]*nums[j]. Same recurrence shape. If you’ve seen MCM, BB is a small leap.
6.6 Optimal BST construction
Knuth-Yao speedup brings the O(n³) of optimal BST construction down to O(n²) under certain monotonicity conditions. Burst Balloons does NOT satisfy those conditions (no Knuth-Yao speedup), so O(n³) is the best known.
6.7 Polygon triangulation
LC 1039 Minimum Score Triangulation of Polygon is essentially the same problem: pick a “third vertex” k for the triangle formed with edge (i,j); cost v[i]*v[k]*v[j] plus recursive subdivision. Identical to BB.
7. Anti-Pattern Analysis
- Split on FIRST balloon — subproblems are not independent; recurrence wrong.
- Forget sentinel padding — boundary cases multiply by 1 explicitly; ugly and bug-prone.
- Fill the DP table in wrong order (e.g., by row) —
dp[i][j]depends on smaller intervals; must fill by length. - Recurse without memoization — exponential blowup.
- Use
dp[i][j]to mean coins from bursting i through j INCLUSIVE — recurrence becomes messier than the exclusive-endpoint convention. - Try greedy (burst smallest first, etc.) — provably wrong; classic DP problem precisely because greedy fails.
8. Skills & Takeaways
- Interval DP filling by increasing length.
- “Split on LAST action” insight for problems with coupled subproblems.
- Sentinel padding to unify boundaries.
- Connection to MCM, polygon triangulation, optimal BST.
9. When to Move On
- Implement BB iteratively in 30 min with sentinel padding and correct fill order.
- Articulate “last balloon” reasoning in 90 seconds.
- Solve LC 1039 (Polygon Triangulation) by analogy.
- Solve Matrix Chain Multiplication cold.
10. Company Context
- Google: L5/L6; the canonical “hard DP” Google problem.
- Amazon: L6; pricing/auction-flavor variants.
- Meta: E5/E6; less common but high signal.
- Microsoft: L65; OR/optimization context.
Scorecard for strong: “Recognized interval DP, articulated ‘last balloon’ reasoning unprompted, padded sentinels, filled by length, computed O(n³).”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Decomposition | Tries first-balloon, fails | “DP something” hand-wave | Names “split on last” cold | + cites MCM analogy upfront |
| Recurrence | Confused | After hint | Clean exclusive-endpoint formulation | + proves subproblem independence |
| Implementation | Recursive no memo | Memoized | Iterative with length-major fill | + space optimization discussion |
| Generalization | None | None | Notes BB ≡ MCM ≡ polygon triangulation | + Knuth-Yao speedup limits |
12. Level Delta
- Mid (L4): Memoized recursion with the correct recurrence after a hint.
- Senior (L5): Cold, iterative, with sentinel padding + length-major fill + names “split on last” insight.
- Staff (L6): + connects to MCM/polygon triangulation/optimal BST + discusses Knuth-Yao (and why it doesn’t apply here).
- Principal (L7+): + LP relaxation framing for interval-scheduling DPs + production applications (sequence-of-decisions in pricing, recommendation reordering).
13. Follow-up Q&A
Q1: What if values can be negative? A1: Same recurrence; max still works. The “smallest first” greedy intuition definitely fails.
Q2: What if you can burst K balloons at once (group burst)? A2: State expands; possibly need bitmask DP. Out of standard interview scope.
Q3: What if neighbors are non-adjacent (e.g., k-nearest)? A3: Recurrence breaks; subproblems no longer independent. Different algorithm needed.
Q4: Space optimization? A4: Filling diagonal-by-diagonal — can in principle reduce to O(n) but requires careful index gymnastics; rarely worth it.
Q5: Why does memoized recursion work but recursive without memo doesn’t? A5: Same subproblem (i,j) recurs exponentially many times in naive recursion (different burst orderings reach the same (i,j) state). Memoization collapses to O(n²) distinct subproblems.
14. Solution Walkthrough
See solution.py:
max_coins_dp— iterative interval DP, length-major fill; canonical.max_coins_memo— top-down with@lru_cache.max_coins_brute— permutation enumeration (oracle).
15. Beyond the Problem
Principal code review: “Burst Balloons is the cleanest example of why DP-direction matters. The naive ‘first action’ decomposition fails because actions COUPLE — bursting balloon 3 first changes who balloon 5’s neighbors are when it bursts later. The ‘last action’ decomposition succeeds because, by the time k bursts, you’ve already decided everything else in (i,j) and the answer there is
dp[i][k] + dp[k][j]plus the clean interactionnums[i]*nums[k]*nums[j]. This pattern — interval DP with split-on-last — is the engine behind Matrix Chain Multiplication, optimal BST construction, polygon triangulation, and a class of operations-research problems. In production, you’ll see it in compiler optimizer phase ordering, in scheduling sequential resource-allocation decisions, and in dynamic-pricing systems where bid order affects clearing prices. When you see ‘enumerate the last action,’ you’re in interval-DP land; the O(n³) is unavoidable in general but worth every cycle.”
Hints — p71 Burst Balloons
Hint 1. Brute force is permutations of burst orders, O(n!). For n=500 (LC constraint), you need DP. Recognize: interval DP.
Hint 2. Naive interval DP — “let dp[i][j] = max coins from interval [i,j], enumerate the FIRST balloon k to burst” — FAILS. Why? When k bursts first, its neighbors are whatever’s left in [i,j], not the original endpoints. Subproblems aren’t independent.
Hint 3. Flip the decomposition: enumerate the LAST balloon to burst in (i,j). When k is the LAST one remaining in (i,j) before bursting, everything else in (i,j) is already gone, so k’s neighbors at burst time are exactly i and j. Clean.
Hint 4. Pad nums with 1 on each end (sentinel). Define dp[i][j] = max coins from bursting balloons STRICTLY between i and j. Recurrence:
dp[i][j] = max over k in (i,j) of:
nums[i] * nums[k] * nums[j] + dp[i][k] + dp[k][j]
Hint 5. Fill order: by increasing interval length (j - i). This ensures dp[i][k] and dp[k][j] are computed before dp[i][j]. O(n³) time, O(n²) space. Answer: dp[0][n+1] after padding.
If stuck: see solution.py.
p72 — Regular Expression Matching
Source: LeetCode 10 · Hard · Topics: DP, Recursion, String Companies (2024–2025): Google (high), Meta (high), Amazon (medium-high), Microsoft (medium) Loop position: The DP problem where the recurrence has a branching
*case. Tests careful case analysis:*can match ZERO or MORE preceding chars, leading to a “skip the (char,*) pair” vs “consume one char” decision at every step.
1. Quick Context
Implement regex matching for . (matches any single char) and * (matches zero or more of the PRECEDING element). Match the ENTIRE string (not substring).
Strategy: 2D DP. dp[i][j] = True iff s[:i] matches p[:j]. Three cases for the pattern’s last character:
- Plain char or
.:dp[i][j] = dp[i-1][j-1] AND (s[i-1] matches p[j-1]). *(forms pair withp[j-2]): two choices —- Match ZERO of
p[j-2]:dp[i][j-2]. - Match ONE MORE of
p[j-2]:dp[i-1][j]ANDs[i-1] matches p[j-2].
- Match ZERO of
O(m·n).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/regular-expression-matching/
STOP. 40-min timer. The case-analysis is the hardest part.
3. Prerequisite Concepts
- 2D string DP (alignment style).
- Why “match zero” via
*is the non-obvious case. - Empty string can be matched by
a*,a*b*c*, etc. - The
*always pairs with the preceding character — indexj-2.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Boolean: does full string match full pattern with . and * semantics?
2. Clarify: * always follows a real char or . (LC promises valid input). Match is FULL not substring.
3. Examples: s="aa", p="a*" → True. s="ab", p=".*" → True. s="aab", p="c*a*b" → True.
5. Brute force: Recursive char-by-char matching with branching on *. Exponential without memoization.
6. Target: O(m·n) DP.
7. Pattern: 2D DP, alignment-style with a branching * rule.
8. Develop: see solution.py.
9. Test: empty s vs empty p; a* matches empty; .* matches anything; nested stars like a*b*c*.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The dp[i][j] definition matters
Use dp[i][j] = “s[0..i-1] matches p[0..j-1]” (1-indexed end-exclusive). So dp[0][0] = True (empty matches empty).
Initialization row dp[0][j]: empty string matches pattern iff pattern is zero-or-more star pairs. dp[0][j] = dp[0][j-2] whenever p[j-1] == '*'.
6.2 The * case is the entire problem
if p[j-1] == '*':
# Option A: match zero of p[j-2]
dp[i][j] = dp[i][j-2]
# Option B: match one MORE of p[j-2]
if p[j-2] == '.' or p[j-2] == s[i-1]:
dp[i][j] = dp[i][j] or dp[i-1][j]
“Match one more” means: we already used the (char,*) pair to consume zero or more chars up through s[i-2]; now we consume one more (s[i-1] if it matches p[j-2]). This stays on the SAME j because the * is still active.
6.3 Why dp[i-1][j] not dp[i-1][j-2] in Option B
The * is “greedy” in the sense that it remains in scope as we consume more chars. dp[i-1][j] says: “up to s[i-2] matched the SAME pattern p[:j] (including the active *)” — we just extend by one more matching char.
6.4 NFA/DFA framing
a*b*c* is a classic regular language. The DP is essentially simulating an NFA where each state is a position in the pattern, and * transitions are ε-moves. Production regex engines use either Thompson’s NFA construction (linear time) or backtracking (exponential worst case — see the StackOverflow July 2016 outage). This DP is conceptually the NFA simulation, written iteratively.
6.5 Backtracking RegEx engines and ReDoS
Most production regex engines (Perl, Python’s re, JavaScript) use backtracking, which can hit exponential worst cases on patterns like (a+)+$ against "aaaa...!". RE2 (Google) uses NFA construction to guarantee linear time. Knowing this distinction matters at Staff+.
6.6 LC 44 Wildcard Matching
LC 44 uses * differently — it matches ANY sequence of any characters (no preceding char binding). Different DP recurrence: dp[i][j] = dp[i-1][j] OR dp[i][j-1] for *. Easier than LC 10.
6.7 Memoized recursion variant
Often the cleanest for interview pressure: recurse on (i, j) = “does s[i:] match p[j:]?” with memoization. Two cases on next pattern token (regular vs c*). Many candidates find this easier than iterative.
7. Anti-Pattern Analysis
- Treat
*as “matches any single char” — that’s LC 44 wildcard. LC 10’s*modifies the PRECEDING char. - Forget the empty-prefix row
dp[0][j]— wrong answer for inputs likes="", p="a*b*". - Use
dp[i-1][j-1]instead ofdp[i-1][j]for “match one more” — silently wrong ons="aaa", p="a*". - Use 0-indexed
dp[i][j]= match s[:i+1] vs p[:j+1]` — boundary handling becomes painful. - Greedy “match longest first” — fails on
s="aa", p="a*a"and others. - Catastrophic backtracking in a recursive solution — TLE on adversarial inputs without memoization.
8. Skills & Takeaways
- 2D string DP with branching recurrence.
*modifies preceding char; pair withp[j-2].- Three cases at each step (plain,
.,*). - Initialization row for empty-string matching.
- Connection to NFA/DFA theory.
9. When to Move On
- Write iterative or memoized version cold in 35 min.
- State the recurrence in English: “If pattern ends in
*: either skip the (char,*) pair or consume one more matching char.” - Solve LC 44 (Wildcard Matching) by analogy.
- Articulate when backtracking regex engines blow up.
10. Company Context
- Google: L5/L6; “implement a small regex” is a classic.
- Meta: E5/E6; tests case analysis carefully.
- Amazon: L6; rare but discriminating.
- Microsoft: L65; engine-implementation flavor.
Scorecard for strong: “Defined dp[i][j] cleanly with empty-prefix row, articulated *’s two cases, used dp[i-1][j] for ‘match one more’ correctly, mentioned NFA framing.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Case analysis | Confused | After hints | Cold 3-case split | + cites NFA |
* recurrence | Off-by-one | Correct after debugging | Cold | + explains intuition |
| Initialization | Misses | After failing test | Cold | + works backward from empty match |
| Generalization | None | LC 44 by name | LC 44 solved by analogy | + RE2 vs backtracking ReDoS discussion |
12. Level Delta
- Mid (L4): Memoized recursion after hints; passes most tests.
- Senior (L5): Iterative DP cold; correct empty-prefix row; correct
*recurrence; cites LC 44. - Staff (L6): + NFA/Thompson construction reference + ReDoS pitfall + RE2 mention.
- Principal (L7+): + full regex-to-NFA conversion + DFA construction tradeoffs + production grep engine architecture.
13. Follow-up Q&A
Q1: Add + (one or more) and ? (zero or one).
A1: + reduces to a copy: a+ ≡ aa*. ? is “match 0 or 1”: dp[i][j] = dp[i][j-2] or (matching dp[i-1][j-2]).
Q2: What’s the time complexity of brute recursion?
A2: Exponential. Each * doubles the search tree. Memoization reduces to O(m·n).
Q3: Implement using NFA construction. A3: Build Thompson NFA, then simulate with set-of-states. O((m+n)·s) where s is input length.
Q4: What about anchors (^, $)?
A4: LC version assumes full match (implicit anchors). Adding partial-match support requires extending the recurrence.
Q5: Production case where backtracking regex bit you?
A5: ReDoS attacks — (a+)+$ against "aaaaaa!". Use RE2 or rewrite the regex to avoid catastrophic backtracking.
14. Solution Walkthrough
See solution.py:
is_match_dp— iterative 2D DP, canonical.is_match_memo— top-down with@lru_cache.is_match_brute— naive recursion (oracle for small inputs).
15. Beyond the Problem
Principal code review: “Regex matching is the gateway to NFA/DFA theory. The DP you write here is essentially a tabular NFA simulation: each
dp[i][j]cell is ‘can the NFA, after consuming s[:i], be in pattern-state j?’ The branching*recurrence corresponds to ε-transitions in the NFA. Production regex engines split into two camps: backtracking (Perl, Pythonre, PCRE) — concise to implement, exponential worst case (ReDoS), and NFA simulation (Go’s regexp/RE2, Rust’s regex) — slightly more complex code, linear time guaranteed. The 2016 StackOverflow outage and several CVE-rated ReDoS attacks demonstrate that this isn’t an academic distinction. When you write this DP in an interview, mention RE2; it shows you understand that the elegant n×m table is the same algorithm Google’s grep uses for trillions of lines of logs daily.”
Hints — p72 Regular Expression Matching
Hint 1. Define dp[i][j] = True iff s[:i] matches p[:j]. The empty-prefix base case dp[0][0] = True. Empty string can match patterns like a*b*c* — initialize dp[0][j] carefully.
Hint 2. Three cases for the last pattern char p[j-1]:
- Regular char or
.:dp[i][j] = dp[i-1][j-1] AND matches(s[i-1], p[j-1]). *: forms a pair withp[j-2]. Two sub-cases — see hints 3 and 4.
Hint 3. * case A — match ZERO of preceding: dp[i][j] = dp[i][j-2] (skip the (char, *) pair entirely).
Hint 4. * case B — match ONE MORE of preceding: if s[i-1] matches p[j-2], then dp[i][j] = dp[i][j] OR dp[i-1][j]. Note: dp[i-1][j] (not dp[i-1][j-2]) — the * stays active to consume more chars.
Hint 5. Common bugs: forgetting the empty-prefix row; using j-1 instead of j-2 when pairing with *; treating * like LC 44 wildcard (which matches any sequence — different problem). Test s="", p="a*b*" → True.
If stuck: see solution.py.
p73 — Interleaving String
Source: LeetCode 97 · Hard (often listed Medium-Hard) · Topics: DP, String Companies (2024–2025): Amazon (high), Microsoft (medium-high), Google (medium), Meta (medium) Loop position: The DP problem that exposes “why greedy fails.” Surface intuition says “always take from whichever string matches” — that’s wrong. Tests whether candidate can construct a counterexample and articulate why a 2D DP table is required.
1. Quick Context
Given strings s1, s2, s3. Return True iff s3 is an interleaving of s1 and s2 (preserving order within each).
Strategy: 2D DP. dp[i][j] = True iff s3[:i+j] is an interleaving of s1[:i] and s2[:j]. Recurrence:
dp[i][j] = (dp[i-1][j] AND s1[i-1]==s3[i+j-1]) OR (dp[i][j-1] AND s2[j-1]==s3[i+j-1]).
O(m·n) time and space; can reduce to O(min(m,n)) space.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/interleaving-string/
STOP. 30-min timer. Must reject the greedy approach explicitly.
3. Prerequisite Concepts
- 2D DP over two strings.
- Why greedy (“take from whichever matches the current s3 char”) fails.
- Length precondition:
len(s3) == len(s1) + len(s2), else immediate False.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Can s3 be formed by interleaving s1 and s2 while preserving each string’s order?
2. Clarify: Order PRESERVED within each source. Each char in s3 must come from s1 OR s2, not both.
3. Examples: s1="aabcc", s2="dbbca", s3="aadbbcbcac" → True. s1="aabcc", s2="dbbca", s3="aadbbbaccc" → False.
5. Brute force: Recursion on (i, j) = pointers into s1 and s2 (s3 pointer = i+j). Exponential without memoization.
6. Target: O(m·n) DP.
7. Pattern: 2D string DP.
8. Develop: see solution.py.
9. Test: empty s3 with empty s1 and s2; greedy-counterexample case (s1="a", s2="aa", s3="aaa" style); length mismatch.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why greedy fails — concrete counterexample
s1 = "aabc", s2 = "abad", s3 = "aabadabc". Greedy at s3[2]=‘b’: could come from s1 (s1[2]=‘b’) or s2 (s2[1]=‘b’). Wrong choice leads to dead-end.
This is the classic “local choice doesn’t determine global outcome” pattern. When both pointers can match, both branches must be explored. Memoization makes this efficient.
6.2 The dp[i][j] definition
dp[i][j] = True iff s3[:i+j] is an interleaving of s1[:i] and s2[:j]. So dp[0][0] = True. Total answer is dp[m][n].
The recurrence’s two terms capture “last char came from s1” vs “last char came from s2”:
- From s1:
dp[i-1][j]ANDs1[i-1] == s3[i+j-1]. - From s2:
dp[i][j-1]ANDs2[j-1] == s3[i+j-1].
6.3 First row and column initialization
dp[0][j] = dp[0][j-1] AND s2[j-1]==s3[j-1] — purely from s2.
dp[i][0] = dp[i-1][0] AND s1[i-1]==s3[i-1] — purely from s1.
6.4 Space optimization
Each row depends only on the previous row + current row’s left neighbor. Reduce to a 1D array of length min(m,n)+1. Standard rolling-array trick.
6.5 BFS framing
You can frame this as graph search: nodes are (i, j) states, edges are “advance s1 pointer” or “advance s2 pointer” when matching. Goal: reach (m, n). BFS or DFS over this implicit graph; same O(m·n).
6.6 Why this isn’t a trie / suffix-tree problem
The “preserving order within each” constraint means we can’t just ask “is s3’s multiset = s1’s + s2’s” — order matters. Trie/suffix structures don’t help directly; the DP is canonical.
6.7 LC 1216 Valid Palindrome III, LC 583 Delete Operation — same family
These are all “2D alignment DP” problems. Recognizing the family lets you write the recurrence in 30 seconds once the state is identified.
7. Anti-Pattern Analysis
- Greedy (“take whichever string matches first”) — fails on the standard counterexample.
- Forget length precondition — without
len(s3) == len(s1)+len(s2), recurrence may “succeed” on impossible inputs (rare but real edge case at boundaries). - 3D state
(i, j, k)— wasteful; k = i+j is derived, not free. - Recursion without memoization — exponential.
- Use
dp[i][j]but iterate in wrong order — must ensure row-major or col-major order with proper neighbors. - Compare full prefixes — quadratic per comparison; turns O(mn) into O(mn·max(m,n)).
8. Skills & Takeaways
- 2D string alignment DP.
- “Why greedy fails” recognition.
- Constructing counterexamples for surface-correct heuristics.
- Space optimization via rolling array.
9. When to Move On
- Solve LC 97 cold in 25 min with correct O(mn) DP.
- Articulate the greedy counterexample without prompting.
- Reduce to O(min(m,n)) space.
- Recognize the family (LC 583, 1143, 72, 115, 1216).
10. Company Context
- Amazon: L5/L6; “DP and reject greedy” probe.
- Microsoft: L65; less common but appears.
- Google: L5; rare; testing thinking style more than algorithm.
- Meta: E5; tests careful state definition.
Scorecard for strong: “Rejected greedy with concrete counterexample, defined dp[i][j] cleanly, recurrence in 30 seconds, optimized to 1D rolling.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Approach | Greedy | DP after greedy fails | DP cold, articulates greedy counterexample | + space optimization unprompted |
| State | 3D (i,j,k) | 2D (i,j) | 2D, explains k=i+j is derived | + rolling array + BFS framing |
| Recurrence | Off-by-one | Correct after debugging | Cold | + cites LC 583/1143 family |
| Generalization | None | None | “Other alignment problems” | + LP relaxation / sequence-alignment in bioinformatics (Needleman-Wunsch) |
12. Level Delta
- Mid (L4): DP after attempting greedy; correct after one iteration.
- Senior (L5): Cold DP + counterexample for greedy + clean state + O(mn) confirmed.
- Staff (L6): + 1D rolling + cites LC 583, 1143, 72, 115 as same family + bio sequence-alignment framing.
- Principal (L7+): + Needleman-Wunsch / Smith-Waterman applications in bioinformatics + parallelism via anti-diagonal traversal + GPU implementations.
13. Follow-up Q&A
Q1: Return the interleaving (not just yes/no).
A1: Backtrack from dp[m][n] along the True path; reconstruct char by char.
Q2: What if a char appears at both s1[i-1] and s2[j-1] (both match s3)?
A2: Both branches valid; explore both. DP handles this naturally (OR in recurrence).
Q3: What if interleaving allows repetition (each char of s1/s2 can be used multiple times)? A3: Different problem — closer to LCS variants or unbounded-knapsack-of-chars.
Q4: How would you parallelize? A4: DP table can be filled anti-diagonal by anti-diagonal — each anti-diagonal independent, parallelize across threads/GPU lanes.
Q5: Bioinformatics connection? A5: Needleman-Wunsch (global sequence alignment) and Smith-Waterman (local alignment) are 2D DP with the same shape and similar recurrences. Production tools (BLAST) use heuristic versions.
14. Solution Walkthrough
See solution.py:
is_interleave_dp— iterative 2D DP.is_interleave_1d— space-optimized rolling array.is_interleave_memo— top-down with@lru_cache.is_interleave_brute— naive recursion (oracle).
15. Beyond the Problem
Principal code review: “Interleaving String is a 2D-alignment DP whose value lies in teaching that greedy choices CAN be globally suboptimal even when each local choice ‘looks right.’ The structure —
dp[i][j]indexed by progress in each input — is exactly the structure of Needleman-Wunsch global sequence alignment in bioinformatics, of edit distance, of LCS. These O(m·n) DPs power genome alignment tools that run on petabytes of sequencing data. Modern implementations exploit anti-diagonal parallelism (each anti-diagonal is independent), giving GPU speedups of 10–100×. When you writedp[i][j] = dp[i-1][j] OR dp[i][j-1]with the appropriate guards, you’re writing the kernel of an entire field of computational biology. Tell the interviewer this; they’ll remember.”
Hints — p73 Interleaving String
Hint 1. Try greedy first (“take from whichever string matches s3’s next char”). Construct a counterexample where both s1 and s2 could provide the matching char but only one choice leads to success. Convince yourself greedy is wrong → you need DP.
Hint 2. Define dp[i][j] = True iff s3[:i+j] is an interleaving of s1[:i] and s2[:j]. The k-pointer for s3 is derived (k = i+j), not a free dimension.
Hint 3. Recurrence: dp[i][j] = (dp[i-1][j] AND s1[i-1]==s3[i+j-1]) OR (dp[i][j-1] AND s2[j-1]==s3[i+j-1]). Two terms = “last char came from s1” vs “last char came from s2”.
Hint 4. Precondition: len(s3) == len(s1) + len(s2) else immediate False. Initialize dp[0][0] = True. First row/column are purely-from-s2 / purely-from-s1 prefix matches.
Hint 5. Space optimization: rows depend only on previous row + left neighbor → 1D array of size min(m,n)+1. Swap s1/s2 so the inner loop is over the shorter dimension.
If stuck: see solution.py.
p74 — Distinct Subsequences
Source: LeetCode 115 · Hard · Topics: DP, String Companies (2024–2025): Meta (high), Amazon (medium-high), Microsoft (medium), Google (medium) Loop position: The COUNTING DP — not a boolean, not an optimum, but how many ways. Tests whether candidate can derive an additive recurrence cleanly and reason about why the “include vs exclude” split avoids double-counting.
1. Quick Context
Given strings s and t. Return the number of distinct subsequences of s that equal t.
Strategy: 2D DP. dp[i][j] = number of distinct subsequences of s[:i] equal to t[:j]. Recurrence:
- If
s[i-1] != t[j-1]:dp[i][j] = dp[i-1][j](must skip s[i-1]). - If
s[i-1] == t[j-1]:dp[i][j] = dp[i-1][j-1] + dp[i-1][j](either use s[i-1] to match t[j-1], or skip it).
O(m·n) time and space; reducible to O(n).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/distinct-subsequences/
STOP. 35-min timer. Counting DPs trip people who only practice boolean/min/max DPs.
3. Prerequisite Concepts
- Subsequence ≠ substring (no contiguity required).
- Include/exclude framework for counting.
- Distinct = each subsequence counted once even if it appears multiple times.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Count subsequences of s (preserving order, can skip) that equal t exactly.
2. Clarify: “Distinct” means count once per matching index-set, not per string value. Result can be very large (LC guarantees fits in 32-bit signed int).
3. Examples: s="rabbbit", t="rabbit" → 3 (drop one of the three b’s). s="babgbag", t="bag" → 5.
5. Brute force: Enumerate all subsequences of s with length |t|, compare to t. C(m, |t|) combinations → exponential.
6. Target: O(m·n) DP.
7. Pattern: 2D string DP, counting variant.
8. Develop: see solution.py.
9. Test: t longer than s → 0; t empty → 1 (empty subsequence); s == t.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why dp[i-1][j-1] + dp[i-1][j] when chars match
Two disjoint cases when s[i-1] == t[j-1]:
- Use s[i-1] to match t[j-1]: count = ways to form t[:j-1] from s[:i-1] =
dp[i-1][j-1]. - Skip s[i-1]: count = ways to form t[:j] from s[:i-1] =
dp[i-1][j].
The cases are disjoint by construction: the first FIXES the use of s[i-1]; the second EXCLUDES it. No double-counting.
6.2 Base cases
dp[i][0] = 1for all i — empty t has exactly one subsequence (the empty one).dp[0][j] = 0for j ≥ 1 — empty s has no way to form non-empty t.
6.3 The “distinct” trap
“Distinct” doesn’t mean “distinct as strings” — it means each subsequence (as an index set into s) is counted once. s="aa", t="a" → 2 (use either ‘a’). The DP naturally counts index sets, so this just works.
6.4 Space optimization
dp[i][j] depends on dp[i-1][j] and dp[i-1][j-1] — previous row only. Reduce to 1D, iterating j from RIGHT to LEFT (so dp[j-1] still refers to the previous row when read).
6.5 Connection to LC 72 Edit Distance and LC 1143 LCS
Same alignment-DP family. LC 115 differs because it’s COUNTING (additive) instead of BOOLEAN (OR) or OPTIMIZING (min/max). The recurrence shape is identical — just the combiner changes.
6.6 Polynomial coefficient framing
When t = "a" and s has k ’a’s, the answer is k. When t = "aa" and s has k ’a’s, it’s C(k, 2). This counts combinations subject to ordering — a generalization of binomial coefficients.
6.7 Numerical overflow
LC promises 32-bit int, but in general counting DPs can overflow. Mention 64-bit accumulator or modular arithmetic at Staff+.
7. Anti-Pattern Analysis
- Treat “distinct” as set-of-strings — overcomplicates; the DP naturally counts index sets.
- Enumerate combinations of s indices — exponential, won’t pass for m=1000.
- Forget
dp[i][0] = 1— wrong answer (zeros propagate). - Use OR instead of
+— that’s a boolean DP, not counting. - 1D rollback in wrong direction — must iterate j right-to-left to preserve previous-row values.
- Off-by-one in
s[i-1] == t[j-1]check — easy under pressure.
8. Skills & Takeaways
- Counting DP recurrence design.
- Include/exclude framework.
- Distinction between counting / boolean / optimizing DP.
- 1D space optimization with right-to-left iteration.
9. When to Move On
- Solve cold in 30 min with correct O(mn) DP.
- Reduce to O(n) space with right-to-left j-loop.
- Articulate “include vs exclude — disjoint, no double-count.”
- Solve LC 583, 72, 1143 by recognizing the family.
10. Company Context
- Meta: E5/E6; appeared multiple times in onsite rotation.
- Amazon: L6; counting DP discriminator.
- Microsoft: L65; less common, but appears.
- Google: L5; rare; tests case-analysis maturity.
Scorecard for strong: “Defined counting DP with disjoint include/exclude split, dp[i][0]=1 base case, O(mn) recurrence cold, optimized to O(n) with right-to-left scan.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Recurrence | Confused | After hints | Cold | + cites disjointness |
| Base cases | Wrong | After failing test | Cold | + explains why dp[i][0]=1 |
| Space opt | None | 2D only | O(n) with right-to-left | + cites why direction matters |
| DP family | None | None | Cites LC 72, 1143 | + counting vs boolean vs optimizing taxonomy |
12. Level Delta
- Mid (L4): 2D DP after one or two hints; correct on most tests.
- Senior (L5): Cold 2D DP + disjoint include/exclude articulation + O(n) optimization.
- Staff (L6): + DP taxonomy (counting/boolean/optimizing) + overflow discussion + cites combinatorial framing.
- Principal (L7+): + connection to formal grammar parsing (Earley counts) + production code-path counting in static analysis.
13. Follow-up Q&A
Q1: Return the actual subsequences, not just the count.
A1: Backtrack from dp[m][n]; collect index sets where both branches contributed. Output exponential; only feasible for small inputs.
Q2: What if t is much longer than s? A2: Return 0 trivially. Length precheck saves work.
Q3: With repeated patterns in t and large s, can you do faster than O(mn)? A3: For specific structures (e.g., t = single char) collapse to O(m) counting. For general t, O(mn) is tight in worst case.
Q4: Overflow risk?
A4: Real risk for large inputs. Use 64-bit (Python int is unbounded; in Java/C++ use long long) or modular arithmetic if the problem specifies modulo.
Q5: Generalization with weights (count weighted by occurrence)?
A5: Replace + with weighted sum. The recurrence shape is identical; the semiring changes.
14. Solution Walkthrough
See solution.py:
num_distinct_dp— iterative 2D DP, canonical.num_distinct_1d— O(n) space, right-to-left j-loop.num_distinct_memo— top-down with@lru_cache.num_distinct_brute— enumerate viaitertools.combinations(oracle).
15. Beyond the Problem
Principal code review: “Distinct Subsequences is the ‘counting twin’ of the alignment-DP family (LCS, Edit Distance, Interleaving String). Same O(m·n) table shape, same neighbor dependencies — only the combiner changes from
max/min/orto+. Recognizing this is the difference between solving one problem and solving an entire class. The deeper lesson: DPs live in a semiring — replace(+, ×)with(max, +)for shortest paths, with(or, and)for reachability, with(min, +)for tropical algebras used in scheduling. Production parsing libraries (Earley, CYK) use this exact pattern to count parse trees in O(n³). Static analyzers count program paths the same way. When you writedp[i][j] = dp[i-1][j] + dp[i-1][j-1], mention that this is the COUNTING instance of the alignment-DP semiring — interviewers at the Staff+ bar will lean in.”
Hints — p74 Distinct Subsequences
Hint 1. This is a counting DP — not boolean, not min/max. Define dp[i][j] = number of distinct subsequences of s[:i] equal to t[:j]. Combiner will be +, not OR / max / min.
Hint 2. Recurrence — case on whether s[i-1] == t[j-1]:
- If they differ: must skip
s[i-1]→dp[i][j] = dp[i-1][j]. - If they match: two disjoint choices — use
s[i-1]to matcht[j-1](dp[i-1][j-1]) OR skip it (dp[i-1][j]). Sum:dp[i][j] = dp[i-1][j-1] + dp[i-1][j].
Hint 3. Base cases matter: dp[i][0] = 1 for all i (empty t has exactly one matching subsequence — the empty one). dp[0][j] = 0 for j ≥ 1.
Hint 4. “Distinct” doesn’t mean distinct-as-strings — it means each subsequence (an index set into s) is counted once. The DP naturally counts index sets; s="aa", t="a" → 2.
Hint 5. Space optimization: reduce to 1D array of length n+1. Iterate j from RIGHT TO LEFT so dp[j-1] still refers to the previous row.
If stuck: see solution.py.
p75 — Palindrome Partitioning II
Source: LeetCode 132 · Hard · Topics: DP, String Companies (2024–2025): Amazon (medium-high), Microsoft (medium-high), Google (medium), Meta (medium) Loop position: The two-phase DP — first precompute a property (palindrome table), then run a second DP on top. Tests recognition that O(n²) is achievable by separating “palindromicity” from “min cuts” rather than re-checking palindromes during the cut DP.
1. Quick Context
Given a string s, partition it so every substring is a palindrome. Return the MINIMUM number of cuts needed.
Strategy: Two phases.
- Phase 1 (O(n²)):
is_pal[i][j]= iss[i..j]a palindrome. Fill by increasing length. - Phase 2 (O(n²)):
cuts[i]= min cuts fors[:i+1]. For eachi, try everyj ≤ iwheres[j..i]is a palindrome →cuts[i] = min(cuts[i], cuts[j-1] + 1). Ifs[:i+1]itself is a palindrome,cuts[i] = 0.
Total O(n²) time, O(n²) space (reducible).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/palindrome-partitioning-ii/
STOP. 35-min timer.
3. Prerequisite Concepts
- Palindromic substring DP (LC 5 / 647):
is_pal[i][j]=s[i]==s[j] AND is_pal[i+1][j-1]. - Phase separation: don’t conflate the palindrome-check with the cut-count DP.
- “Cuts” vs “partitions”: cuts = partitions − 1.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Min cuts so every piece is a palindrome.
2. Clarify: Min CUTS, not min PIECES (off by one).
3. Examples: s="aab" → 1 (cut between “aa” and “b”). s="a" → 0. s="ab" → 1.
5. Brute force: Enumerate all 2^(n-1) cut positions; check each piece is palindrome. Exponential.
6. Target: O(n²).
7. Pattern: Two-phase DP — precompute property, then DP on top.
8. Develop: see solution.py.
9. Test: single char; all same char; already palindrome; pathological “abcd” → 3.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The phase separation is the whole insight
Naively combining “is palindrome” and “min cuts” inside one loop costs O(n³) (palindrome check is O(n) per call). Phase-separating gives two clean O(n²) passes.
6.2 Phase 1: palindrome table, fill by length
for length in 1..n:
for i in 0..n-length:
j = i + length - 1
if length == 1: is_pal[i][j] = True
elif length == 2: is_pal[i][j] = (s[i] == s[j])
else: is_pal[i][j] = (s[i] == s[j]) and is_pal[i+1][j-1]
Must fill by increasing length so is_pal[i+1][j-1] is already computed.
6.3 Phase 2: cuts DP
cuts[i] = min cuts for s[:i+1]
for i in 0..n-1:
if is_pal[0][i]: cuts[i] = 0
else:
cuts[i] = min over j in 1..i where is_pal[j][i] of (cuts[j-1] + 1)
6.4 Expand-around-center alternative — O(n²) without table
Instead of a is_pal table, expand-around-center from each position (odd and even centers). When we discover s[l..r] is a palindrome, immediately update cuts[r] = min(cuts[r], cuts[l-1] + 1). Same O(n²) but only O(n) memory.
6.5 Manacher’s algorithm — O(n) palindrome enumeration
Manacher’s finds all palindromic substrings in O(n) by exploiting the symmetry of previously-found palindromes. Combined with the cuts DP, total time is O(n²) because the cuts DP itself is O(n²) — Manacher’s doesn’t help the overall bound here. But it’s worth mentioning for palindrome-heavy problems.
6.6 Connection to LC 5, LC 647, LC 131
- LC 5 (longest palindromic substring) — uses the same
is_paltable or expand-around-center. - LC 647 (count palindromic substrings) — counts True entries in the table.
- LC 131 (palindrome partitioning, enumerate ALL) — backtracking on top of the table.
LC 132 (this one) is “min cuts” — DP on top of the table.
6.7 Why we can’t be greedy
Greedy “take longest palindrome from current position” fails: s="aabbaa" greedy might take “aabbaa” whole (1 piece, 0 cuts ✓) but greedy “aa” first leaves “bbaa” needing more cuts.
7. Anti-Pattern Analysis
- Conflate palindrome-check with cuts DP — O(n³), TLE for n=2000.
- Greedy “take longest palindrome” — fails (counterexample above).
- Count partitions instead of cuts — off by one.
- Fill
is_palin wrong order — read uninitializedis_pal[i+1][j-1]. - Recompute palindrome checks inside cuts loop — same O(n³) trap as #1.
- Backtracking — exponential, doesn’t apply for “min” (could memoize but harder than the DP).
8. Skills & Takeaways
- Two-phase DP recognition.
- Palindrome table fill order (by length, not by i/j).
- Expand-around-center alternative.
- Manacher’s algorithm awareness.
9. When to Move On
- Solve cold in 30 min with O(n²) DP.
- Articulate why naive is O(n³) and phase-separation fixes it.
- Implement expand-around-center variant with O(n) extra memory.
- Mention Manacher’s for the palindrome-finding subroutine.
10. Company Context
- Amazon: L5/L6; “DP with preprocessing” probe.
- Microsoft: L65; appears in cluster with LC 5, 647, 131.
- Google: L5; tests phase-separation thinking.
- Meta: E5; rare but discriminating.
Scorecard for strong: “Recognized two-phase O(n²), filled palindrome table by length correctly, articulated why naive is O(n³), mentioned expand-around-center / Manacher’s as alternatives.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Approach | Greedy or exponential | DP after hints | Two-phase O(n²) cold | + expand-around-center / Manacher’s |
| Table fill | Wrong order | After failing | By-length cold | + explains dependency |
| Optimization | None | Two-phase 2D | O(n) memory via expand | + Manacher’s for O(n) palindrome enumeration |
| Cuts vs partitions | Wrong | After failing test | Correct cold | + cites LC 131 (enum) and LC 1278 (DP cuts II) family |
12. Level Delta
- Mid (L4): Two-phase 2D DP after one hint; correct on most tests.
- Senior (L5): Cold two-phase O(n²) + correct fill order + articulates O(n³) trap.
- Staff (L6): + expand-around-center for O(n) memory + Manacher’s mention + LC 131/647/5 family.
- Principal (L7+): + suffix automaton / palindromic tree (Eertree) for advanced palindrome problems + production string-processing library design.
13. Follow-up Q&A
Q1: Return the actual partition, not just the cut count.
A1: Track parent pointer in cuts DP; backtrack from cuts[n-1] to reconstruct cut positions.
Q2: What if we want max cuts (max partitions)? A2: Trivially n cuts (every char is a length-1 palindrome). Boring.
Q3: K palindromic partitions (LC 1278)?
A3: Add a dimension: dp[i][k] = min changes to partition s[:i] into k palindromes (note: that variant allows char changes; different objective).
Q4: O(n log n) or better? A4: Not known. O(n²) appears tight without exploiting input structure.
Q5: Manacher’s — when does it actually help? A5: When you need to enumerate ALL palindromic substrings in O(n) total. For LC 132 it doesn’t change the overall bound (the cuts DP is O(n²)), but it does cut the palindrome-precomputation phase from O(n²) to O(n).
14. Solution Walkthrough
See solution.py:
min_cut_dp— two-phase O(n²) withis_paltable.min_cut_expand— expand-around-center, O(n) extra memory.min_cut_brute— enumerate all 2^(n-1) partitions (oracle).
15. Beyond the Problem
Principal code review: “Palindrome Partitioning II is the cleanest illustration of DP staging: precompute a property in one pass, then run a second DP on top. This pattern recurs everywhere — precompute prefix sums then range queries, precompute suffix array then string queries, precompute reachability then shortest paths. The naive O(n³) trap (recomputing palindromes inside the cuts loop) is what separates engineers who think ‘one DP per problem’ from those who think ‘pipeline of DPs.’ At the Principal bar, articulate the generalization: when an inner DP has a frequently-queried property as part of its recurrence, hoist the property into a precomputed table — Strassen’s matrix-mult, FFT-accelerated convolutions, suffix-tree-based string algorithms, and even modern compilers’ dataflow analyses all use this staging principle.”
Hints — p75 Palindrome Partitioning II
Hint 1. Two-phase DP. Don’t try to check palindromes inside the cuts loop — that’s O(n³). First precompute an is_pal[i][j] table, then run a separate cuts DP that consults it in O(1).
Hint 2. Phase 1 — fill is_pal[i][j] by increasing length. For length 1: True. Length 2: s[i]==s[j]. Length ≥ 3: s[i]==s[j] AND is_pal[i+1][j-1]. Fill order matters so the inner reference is already computed.
Hint 3. Phase 2 — cuts[i] = min cuts for s[:i+1]. If is_pal[0][i] then cuts[i] = 0. Else: cuts[i] = min(cuts[j-1] + 1) over every j such that is_pal[j][i] (every palindrome suffix ending at i).
Hint 4. Cuts vs partitions — off by one. K palindromic pieces need K-1 cuts. The DP returns cuts, not pieces.
Hint 5. Space optimization — expand-around-center alternative: instead of building the is_pal table, expand from each center (odd and even); when you discover s[l..r] is a palindrome, update cuts[r+1] = min(cuts[r+1], cuts[l] + 1). Same O(n²) time, only O(n) memory.
If stuck: see solution.py.
Week 15 — Bitmask & Game DP
Theme: Move beyond standard 2D string DPs into the harder DP archetypes — reverse-direction DP, multi-agent simultaneous DP, adversarial game DP, and bitmask DP over subsets. These are the patterns that distinguish Senior from Staff at the FAANG bar.
Daily schedule
| Day | Problem | LC # | Difficulty | Key archetype |
|---|---|---|---|---|
| Mon | p76 — Dungeon Game | 174 | Hard | Reverse DP — fill from goal back to start |
| Tue | p77 — Strange Printer | 664 | Hard | Interval DP — split on shared endpoints |
| Wed | p78 — Cherry Pickup | 741 | Hard | Multi-agent DP — two simultaneous walks |
| Thu | p79 — Stone Game II | 1140 | Medium-Hard | Game DP — minimax over suffix + parameter M |
| Fri | p80 — Number of Ways to Wear Different Hats | 1434 | Hard | Bitmask DP — flip subject (iterate hats, not people) |
Readiness gate (Friday)
- Articulate why Dungeon Game must be filled from bottom-right to top-left.
- State the Strange Printer “merge shared endpoints” optimization.
- Explain why Cherry Pickup uses one DP with two agents walking simultaneously (not two separate path DPs).
- For Stone Game II, write the minimax recurrence and explain why the suffix-sum + max-of-min structure works.
- For Hats problem, articulate why iterating hats × people-masks (40 × 2¹⁰) is feasible while iterating people × hat-masks (10 × 2⁴⁰) is not.
Why these together
Each problem here breaks a habit formed by easier DPs:
- Dungeon Game — habit of “fill from origin” is wrong; reverse fill is mandatory.
- Strange Printer — natural recurrence is O(n⁴); the “skip k if s[k]==s[i]” insight cuts it to O(n³).
- Cherry Pickup — habit of “find path A, then path B” is wrong; both paths must be considered simultaneously to avoid double-counting cherries.
- Stone Game II — habit of “iterate left-to-right” yields nothing; must iterate suffix-first.
- Hats — habit of “bitmask the obvious set” is wrong; pick the smaller set (people, not hats) as the state.
These five together form the DP archetype recognition pack — by Friday, the candidate should be able to classify a novel DP by archetype within 60 seconds of reading.
p76 — Dungeon Game
Source: LeetCode 174 · Hard · Topics: DP, Matrix Companies (2024–2025): Microsoft (high), Amazon (medium-high), Google (medium), Meta (medium) Loop position: The reverse-DP problem. Tests whether candidate recognizes that the natural forward DP (start → goal) is wrong here, because the answer depends on FUTURE costs that aren’t known at the time you’d write a forward state. Must fill from bottom-right to top-left.
1. Quick Context
Knight starts top-left, must reach bottom-right (princess). Each cell adds/subtracts health. Knight dies if health ≤ 0 at any point. Find minimum starting health so knight survives the entire path (knight chooses optimal path).
Strategy: Reverse DP. Let dp[i][j] = min health knight needs WHEN ENTERING cell (i,j) to survive from (i,j) to goal. Then dp[i][j] = max(1, min(dp[i+1][j], dp[i][j+1]) - dungeon[i][j]). Answer: dp[0][0].
O(m·n) time, O(m·n) space (reducible to O(n)).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/dungeon-game/
STOP. 40-min timer. The first wrong approach (forward DP) will burn 20 min before the candidate realizes it doesn’t work.
3. Prerequisite Concepts
- Grid DP fundamentals (Unique Paths, Min Path Sum).
- Why some DPs require reverse fill direction.
- The “min health on entry” formulation, not “max health accumulated.”
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Min starting health so knight survives all cells on some path from (0,0) to (m-1,n-1).
2. Clarify: Knight may only move RIGHT or DOWN. Health must be ≥ 1 at all times including after entering the princess cell.
3. Examples: [[-2,-3,3],[-5,-10,1],[10,30,-5]] → 7. Path: right→right→down→down with starting health 7 keeps you alive.
5. Brute force: DFS over all paths trying each starting health (or each path’s min health requirement). Exponential.
6. Target: O(m·n).
7. Pattern: Reverse DP on grid.
8. Develop: see solution.py.
9. Test: all positive cells (need 1); all negative; 1×1; path with the “valley then peak” trap (health drops below 1 even though path total is positive).
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why forward DP fails
Try dp[i][j] = “min starting health to reach (i,j) alive.” Recurrence would need min(dp[i-1][j], dp[i][j-1]). Problem: the OPTIMAL path to (i,j) might require less starting health, but force more health LATER. You can’t decide locally without knowing the future. This is the classic “future affects past” pitfall.
Concrete counterexample: at cell (i,j), path A arrives with 1 health and path B arrives with 100 health; later cells require 50 health buffer. Path A appears “optimal” locally (less starting health) but path B is actually feasible. Forward DP picks A and produces wrong answer.
6.2 Why reverse DP works
dp[i][j] = “min health needed at the MOMENT of entering (i,j) to survive from here to goal.” Now the future is encapsulated in dp[i+1][j] and dp[i][j+1] — we know exactly what minimum health each successor demands. We pick the cheaper successor and back-derive:
dp[i][j] = max(1, min(dp[i+1][j], dp[i][j+1]) - dungeon[i][j])
The max(1, ...) enforces the “health must stay ≥ 1” constraint: even if entering this cell gives huge bonus, you still need 1 HP to be alive entering.
6.3 Base case
dp[m-1][n-1] = max(1, 1 - dungeon[m-1][n-1]). The princess cell still must be entered with ≥ 1 HP. If dungeon[m-1][n-1] = -10, you need 11 HP entering.
Pad boundary with +inf (a sentinel for “no successor here”) to unify the recurrence — dp[m][j] = dp[i][n] = +inf for j < n / i < m respectively.
6.4 The max(1, ...) is the entire subtlety
Without it, a cell with dungeon[i][j] = +100 might give dp[i][j] = -50 (i.e., “you can enter with negative health, gain 100, and have positive health to survive the rest”). But you’d already be DEAD before entering. The max(1, ...) says: you need at least 1 HP entering, regardless of any gain you’d get.
6.5 Space optimization
Each row depends only on the row below + the right neighbor in the current row. Reduce to a 1D array of length n+1, iterating bottom-up and right-to-left.
6.6 Why this is “DP archetype 4: state defined relative to goal, not origin”
Most DPs define dp[i] as “best outcome ending at i” (origin-relative). Reverse DPs define dp[i] as “best outcome STARTING from i” (goal-relative). When the cost at i depends on what happens AFTER i (not before), you must reverse. Other examples: jump games where you need future reachability, suffix-array DPs, “min cost from here to goal” with constraints.
6.7 Connection to Bellman optimality
This IS Bellman’s equation in disguise: V*(s) = max_a R(s,a) + γV*(s'). The “max(1, …)” is a state constraint (health ≥ 1) imposed on the value function. The reverse fill is value iteration computed in topological order (the grid has no cycles, so one sweep suffices).
7. Anti-Pattern Analysis
- Forward DP from (0,0) — burns 20 min before failing on the “future affects past” trap.
- Track max-health-along-path instead of min-health-needed — wrong formulation; doesn’t account for required buffer at later cells.
- Forget
max(1, ...)— silently wrong on cells with large positive bonus. - Binary search on starting health + path feasibility check — works but O(mn · log(range)); worse than direct DP. Mention only as alternative.
- Try Dijkstra — graph isn’t well-defined here; the “cost” depends on path history.
- Iterate top-left to bottom-right with reverse formula — wrong order; you’d read uninitialized cells.
8. Skills & Takeaways
- Reverse DP pattern recognition.
- “Future affects past” diagnosis.
- Goal-relative state definition.
max(1, ...)constraint enforcement in DP values.
9. When to Move On
- Solve cold in 30 min with O(mn) reverse DP.
- Articulate why forward DP fails with a concrete counterexample.
- Optimize to O(n) space.
- Recognize the reverse-DP archetype in other problems (e.g., Frog Jump variants, Min Falling Path with constraints).
10. Company Context
- Microsoft: L65/L66; “reverse DP” probe. Common in Bing-team interviews.
- Amazon: L6; tests pattern recognition + Bellman-equation framing.
- Google: L5; appears in optimization rotations.
- Meta: E5; rare but discriminating.
Scorecard for strong: “Diagnosed forward DP as wrong with a concrete counterexample, defined dp[i][j] as ‘min health entering this cell,’ wrote O(mn) reverse DP cold, enforced max(1, …) correctly, optimized to O(n) space, cited connection to Bellman.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Diagnosis | Forward DP, no recognition | Forward DP fails → reverse after hint | Recognizes reverse cold | + diagnoses via “future affects past” lens |
| Recurrence | Confused | Correct after debugging | Cold with max(1, ...) | + explains why constraint is enforced as max |
| Optimization | None | 2D only | O(n) with right-to-left scan | + Bellman framing + value iteration analogy |
| Generalization | None | None | Other reverse-DP examples | + RL value-iteration parallel |
12. Level Delta
- Mid (L4): Tries forward DP, fails, switches to reverse after one hint. Eventually correct.
- Senior (L5): Cold reverse DP + correct
max(1, ...)+ concrete forward-DP counterexample + O(n) space. - Staff (L6): + Bellman optimality framing + value iteration analogy + generalizes “future-affects-past” diagnostic.
- Principal (L7+): + RL value iteration parallel + connection to dynamic-programming-on-MDPs literature + production game-AI applications.
13. Follow-up Q&A
Q1: Knight can move in 4 directions (incl. up/left). A1: Now there are cycles in the state graph. Must use value iteration (repeated sweeps until convergence) or Dijkstra-style with proper cost function. O(mn log mn) with Dijkstra on transformed graph.
Q2: What if some cells are blocked (impassable)?
A2: Set dp[i][j] = +inf for blocked cells; recurrence handles propagation naturally.
Q3: Return the actual optimal path. A3: After filling dp, trace from (0,0) choosing the direction (down or right) with smaller dp value; tie-break arbitrarily.
Q4: Knight has bounded health (≤ H). Feasibility?
A4: Check dp[0][0] ≤ H. If we want to also minimize NUMBER OF HEALTH POTIONS used, problem becomes multi-objective.
Q5: Connect to RL/MDP framework. A5: Each cell is a state, moves are actions, dungeon[i][j] is the reward, terminal at (m-1, n-1). The reverse DP is value iteration on this deterministic MDP. Adding stochastic transitions makes it a real MDP requiring iterative value iteration (not one sweep).
14. Solution Walkthrough
See solution.py:
calculate_minimum_hp_dp— iterative O(mn) reverse DP.calculate_minimum_hp_1d— O(n) space.calculate_minimum_hp_memo— top-down with@lru_cache.calculate_minimum_hp_brute— enumerate all paths, compute min health each, take min over paths (oracle).
15. Beyond the Problem
Principal code review: “Dungeon Game is the cleanest illustration of value iteration in a deterministic MDP — every grid DP textbook should open with it. The ‘future affects past’ diagnostic generalizes: any time a local choice’s quality depends on what happens after it, you must define the state goal-relative, not origin-relative. This is exactly why Bellman’s equation
V*(s) = max_a R(s,a) + γV*(s')is written with V on both sides — the future is encoded in V*(s’), not in a path history. Production game AI (chess engines, Go bots) all run variants of this: define value goal-relative, iterate from terminal states backward. Themax(1, ...)here is a state constraint, the analog of which appears in constrained MDPs and safe reinforcement learning. When you write this in 20 lines, you’re writing the kernel of an entire field.”
Hints — p76 Dungeon Game
Hint 1. Try forward DP from (0,0) first (“min HP to reach this cell alive”). You’ll discover the recurrence depends on FUTURE cells. This is the “future affects past” diagnostic — switch to reverse DP.
Hint 2. Define dp[i][j] = minimum HP the knight needs at the moment of entering cell (i,j) in order to survive from here to the princess. Goal: dp[0][0]. Fill from bottom-right to top-left.
Hint 3. Recurrence: dp[i][j] = max(1, min(dp[i+1][j], dp[i][j+1]) - dungeon[i][j]). Pick the cheaper successor; back-derive HP needed before applying this cell’s effect.
Hint 4. The max(1, ...) is essential. Even if dungeon[i][j] = +100 would let you enter with negative HP and survive, you can’t be alive with HP ≤ 0. Minimum entering HP is always ≥ 1.
Hint 5. Base case: dp[m-1][n-1] = max(1, 1 - dungeon[m-1][n-1]). Pad bottom/right boundary with +inf sentinels so the min(dp[i+1][j], dp[i][j+1]) doesn’t need special-casing. Space-optimize to O(n) with right-to-left scan.
If stuck: see solution.py.
p77 — Strange Printer
Source: LeetCode 664 · Hard · Topics: DP, Interval DP, String Companies (2024–2025): Google (high), Amazon (medium-high), Meta (medium), Microsoft (medium) Loop position: The interval DP with the “skip matching endpoint” optimization. Natural recurrence is O(n⁴); the insight that
s[i] == s[k]lets you absorb one print cuts it to O(n³). Tests whether the candidate can see the matching-endpoint structure.
1. Quick Context
A printer prints sequences of the same character only. To print “aab” requires: print “aaa”, overprint position 2 with “b” → 2 turns. Given a string s, find minimum number of turns to print it.
Strategy: Interval DP. dp[i][j] = min turns to print s[i..j]. Recurrence:
- Base:
dp[i][i] = 1. - General:
dp[i][j] = dp[i+1][j] + 1, then try improvements: for eachkini+1..jwiths[k] == s[i]:dp[i][j] = min(dp[i][j], dp[i+1][k-1] + dp[k][j]).
O(n³).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/strange-printer/
STOP. 45-min timer. The matching-endpoint insight is hard to find cold.
3. Prerequisite Concepts
- Interval DP (MCM, Burst Balloons).
- Why splitting on intervals beats splitting on prefixes for non-overlapping subproblems.
- Recognizing the “matching endpoints absorb a print” pattern.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Min printer turns where each turn paints a contiguous block of the SAME char (which may be later overwritten in a sub-region).
2. Clarify: Each turn paints any contiguous range; later turns can overwrite. The order matters in the sense that we count turns, not positions.
3. Examples: “aaabbb” → 2 (print ‘a’×3 then ‘b’×3). “aba” → 2 (print ‘aaa’, then ‘b’ at middle). “aabbaa” → 2 (print ‘aaaaaa’, then ‘bb’ in middle).
5. Brute force: Try all sequences of (range, char) operations. Exponential.
6. Target: O(n³) interval DP.
7. Pattern: Interval DP with shared-endpoint merging.
8. Develop: see solution.py.
9. Test: single char; all same; alternating; reduce-by-merge example “aabbaa” should be 2 not 3.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The naive recurrence (without the insight)
dp[i][j] = min over k in [i, j) of dp[i][k] + dp[k+1][j]. O(n³) splits × O(n²) intervals = O(n⁴). Often TLE.
6.2 The matching-endpoint insight
Observation: if s[i] == s[j], then dp[i][j] = dp[i][j-1]. Why? Print s[i..j-1] optimally; the same print stroke that painted s[i] can be extended to cover position j (since s[j] = s[i]). No extra turn.
Generalized: dp[i][j] = dp[i+1][j-1] + (1 if s[i] != s[j] else 0) — not quite, but close. The clean form: start with dp[i][j] = dp[i+1][j] + 1 (print s[i] alone, then the rest); then for each k in i+1..j with s[k] == s[i], try dp[i][j] = min(dp[i][j], dp[i+1][k-1] + dp[k][j]) — the stroke painting s[i] extends to also cover s[k], so we don’t count an extra turn at k.
6.3 Why “fix the first char, scan for matches”
When painting s[i], the printer paints a contiguous range. Anything inside that range that’s a DIFFERENT char gets overprinted later. So we only need to think about which OTHER positions of the SAME char s[i] share the same paint stroke as s[i]. For each such k, the segment between (i+1..k-1) is printed independently AFTER, and the segment (k..j) is printed independently of i. The +0 (not +1) for the matching k is the savings.
6.4 String reduction trick
Collapse consecutive duplicates first: “aaabbb” → “ab”. Doesn’t change the answer (one stroke handles consecutive duplicates). Speeds up by collapsing degenerate input.
6.5 Fill order
Length-major: outer loop = interval length 1..n; inner = i with j = i + length - 1. Ensures dp[i+1][j], dp[i+1][k-1], dp[k][j] are all already computed.
6.6 Connection to other interval-DP problems
- LC 312 Burst Balloons: split on LAST burst.
- LC 1547 Min Cost to Cut a Stick: split on WHICH cut to make first.
- LC 132 Palindrome Partitioning II: cuts DP on top of palindrome precomputation.
- LC 1000 Min Cost to Merge Stones: K-merge interval DP.
Each is an O(n³) interval DP with a problem-specific split rule.
6.7 Generalization: this is “polygon triangulation”
Treat the string as vertices on a polygon. Each “stroke” is a triangle (or generalization). The cost is the number of strokes. This is exactly the polygon-triangulation DP family — Knuth-Yao quadrangle inequality applies in some variants and can drop O(n³) to O(n²).
7. Anti-Pattern Analysis
- Greedy “print biggest contiguous same-char run first” — fails on “aba” (greedy paints “a” then “b” then “a” = 3 turns; optimal is 2).
- Naive O(n⁴) interval split without the matching-endpoint insight — TLE for n=100.
- Forget the
+1baselinedp[i][j] = dp[i+1][j] + 1— recurrence becomes a strict improvement loop with no base. - Don’t collapse consecutive duplicates — wastes time; doesn’t break correctness but burns the time budget.
- Wrong fill order — reading uninitialized cells.
- Confusing this with LC 546 (Remove Boxes) — different recurrence (3D state with “k same-color blocks to the right”).
8. Skills & Takeaways
- Interval DP archetype recognition.
- “Matching endpoint absorbs one operation” insight.
- Length-major fill order.
- String preprocessing (collapse duplicates) as O(n) speedup.
9. When to Move On
- Solve cold in 35 min with O(n³) interval DP.
- Articulate the matching-endpoint savings cleanly.
- State the family (Burst Balloons, MCM, Cut a Stick, Merge Stones).
- Mention polygon-triangulation / Knuth-Yao framing at Staff+.
10. Company Context
- Google: L5/L6; interval DP probe.
- Amazon: L6; tests pattern recognition.
- Meta: E5; appears in onsite rotation.
- Microsoft: L65; less common.
Scorecard for strong: “Recognized interval DP, found matching-endpoint savings unprompted, O(n³) cold, correct fill order, mentioned the interval-DP family and polygon-triangulation generalization.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Approach | Greedy | Naive O(n⁴) | O(n³) with matching-endpoint cold | + cites family + polygon triangulation |
| Insight | None | After hint | Cold | + proves savings rigorously |
| Fill order | Wrong | After failing | Length-major cold | + mentions Knuth-Yao bound |
| Generalization | None | None | Cites family | + Eertree / suffix-automaton tangent at Principal |
12. Level Delta
- Mid (L4): Naive interval DP after hints; O(n⁴) accepted on small inputs.
- Senior (L5): O(n³) with matching-endpoint savings cold + correct fill order + collapses duplicates.
- Staff (L6): + cites Burst Balloons / MCM / Cut a Stick family + polygon-triangulation framing.
- Principal (L7+): + Knuth-Yao quadrangle inequality + O(n²) variants in specific structures + production string-processing library architecture.
13. Follow-up Q&A
Q1: What if the printer can paint NON-contiguous ranges? A1: Answer is just the number of distinct characters (one stroke per char). Trivial.
Q2: Variant: printer can only paint ranges of length ≤ K. A2: Add constraint to recurrence; range of valid k limited. Still O(n³).
Q3: Reconstruct the actual sequence of strokes.
A3: Track parent pointer at each DP cell; backtrack from dp[0][n-1].
Q4: What’s the connection to Burst Balloons? A4: Both are interval DP. Burst Balloons splits on “last burst inside interval.” Strange Printer splits on “which other matching char shares the stroke.” Both O(n³).
Q5: Knuth-Yao optimization? A5: When the recurrence satisfies the quadrangle inequality, the optimal split point is monotonic in (i,j), allowing amortized O(n²) total. Doesn’t apply directly here without modification.
14. Solution Walkthrough
See solution.py:
strange_printer_dp— iterative O(n³) with matching-endpoint optimization.strange_printer_memo— top-down with@lru_cache.strange_printer_brute— try all stroke sequences (oracle for small).
15. Beyond the Problem
Principal code review: “Strange Printer is the canonical ‘interval DP with absorption’ problem. The matching-endpoint insight (
s[i] == s[k]lets one stroke serve both) is the same shape as ‘polygon triangulation skip’ — when two vertices share a property, the triangle linking them costs nothing extra. This pattern recurs in compiler optimizations (common subexpression elimination — same expression in multiple places shares one computation), in graph drawing (shared crossings), and in database query planning (shared subtrees in plan trees). At the Staff+ bar, articulate this absorption as the general principle: ‘when the cost function exhibits sub-additivity over shared structure, hoist the shared structure into the recurrence.’ The interval-DP family — Burst Balloons, MCM, Cut a Stick, Strange Printer, Merge Stones — is one of the most-tested archetypes precisely because it forces this kind of structural thinking.”
Hints — p77 Strange Printer
Hint 1. Greedy (“paint biggest run first”) fails on “aba”. You need DP. What’s the state? Try interval DP: dp[i][j] = min turns to print s[i..j].
Hint 2. Base: dp[i][i] = 1. For the general case, start with the naive: dp[i][j] = dp[i+1][j] + 1 (one stroke for s[i] alone, then print the rest).
Hint 3. Key insight: if s[k] == s[i] for some k in (i, j], the stroke painting position i can EXTEND to cover position k in the same turn. So dp[i][j] = min(dp[i][j], dp[i+1][k-1] + dp[k][j]) — the +1 for position k is absorbed.
Hint 4. Fill order: length-major. Outer = interval length 1..n; inner = i with j = i + length - 1. Ensures dp[i+1][j], dp[i+1][k-1], dp[k][j] are all already computed.
Hint 5. Preprocess: collapse consecutive duplicates (“aaabbb” → “ab”). Doesn’t change the answer (one stroke handles a run of duplicates) and shrinks the DP table.
If stuck: see solution.py.
p78 — Cherry Pickup
Source: LeetCode 741 · Hard · Topics: DP, Multi-Agent DP, Grid Companies (2024–2025): Google (high), Amazon (medium), Meta (medium-high) Loop position: The multi-agent DP reframing. Naive “go then return” greedy fails. The insight that “round trip” = “two simultaneous one-way trips” is the entire problem.
1. Quick Context
n×n grid. 0 = empty, 1 = cherry, -1 = thorn. Walk from (0,0) → (n-1,n-1) moving only right/down picking cherries (set to 0). Then walk back (n-1,n-1) → (0,0) moving only left/up. Maximize cherries collected.
Reframing: Equivalent to TWO simultaneous walks from (0,0) → (n-1,n-1), each moving right/down. State (r1, c1, r2); derive c2 = r1 + c1 - r2 (both walks have made the same number of steps). If both at same cell, count cherry once.
O(n³) time / space.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/cherry-pickup/
STOP. 45-min timer. The “two simultaneous paths” reframing is the unlock.
3. Prerequisite Concepts
- Grid DP (unique paths).
- Why greedy “max path forward, max path back” fails (shown in 6.2).
- State compression by invariant (c2 derived from r1+c1-r2).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Max cherries on a round trip; thorns block; eaten cherries don’t regrow.
2. Clarify: What if no valid path? Return 0. What if cells reused on round trip — eat twice? No (cherry consumed on first visit).
3. Examples: [[0,1,-1],[1,0,-1],[1,1,1]] → 5.
5. Brute: Enumerate all (forward, backward) path pairs. Exponential.
6. Target: O(n³).
7. Pattern: Multi-agent DP / dimension reduction by step-count invariant.
8. Develop: see solution.py.
9. Test: all zeros; thorn blocks all forward paths (return 0); single cell.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why the naive “forward greedy then backward greedy” fails
Take forward as “max cherry path to goal”, consume those cherries, then “max cherry path back”. This is suboptimal because the forward path’s greedy choice may starve the backward path of a much-better lane. Counterexample: a grid where two cherry-lanes exist; greedy forward takes one lane fully, leaving the backward to grab only the partial overlap.
6.2 The reframing: round trip = 2 simultaneous one-way trips
A “round trip A→B→A” where forward is right/down and backward is left/up is equivalent to two agents simultaneously walking A→B, both moving right/down. Why? Time-reverse the backward path → it becomes a right/down forward path. Cherries collected on the round trip = cherries collected by the union of two forward paths (with shared-cell deduplication).
6.3 State derivation: c2 = r1 + c1 - r2
Both agents move one step at a time (right or down). After step t, agent 1 is at (r1, c1) with r1+c1 = t; same for agent 2: r2+c2 = t. So c2 = r1 + c1 - r2. This drops the state from O(n⁴) to O(n³).
6.4 Shared-cell dedup
If (r1, c1) == (r2, c2) (same cell, same step), count the cherry once not twice. In code: gain = grid[r1][c1]; if (r1, c1) != (r2, c2): gain += grid[r2][c2].
6.5 Transitions: 4 combinations
Each agent moves down or right → 4 predecessor combos: (D,D), (D,R), (R,D), (R,R). Take max over predecessors that are valid (in-bounds and not thorns).
6.6 -inf for thorn cells / unreachable
If any cell on the path is -1, that state is invalid. Set dp[...] = -inf. Final answer: max(0, dp[n-1][n-1][n-1]).
6.7 Why this is “Cherry Pickup II” pattern foreshadowed
LC 1463 (Cherry Pickup II) has TWO robots starting at top corners, both walking down to bottom. Same multi-agent DP, slightly different state (r is shared since both move down 1 per step; state = (r, c1, c2)).
7. Anti-Pattern Analysis
- Greedy forward-then-backward — counterexample in 6.1.
- Treat as two independent shortest paths — ignores cherry consumption.
- State
(r1, c1, r2, c2)(O(n⁴)) — works but wastes memory; misses the invariant. - Forget the shared-cell dedup — over-counts cherries when paths cross.
- Forget
max(0, ...)— return -inf when no valid path exists; must clamp to 0. - Iterate t (step count) but forget r1+c1=t constraint — visits invalid states.
8. Skills & Takeaways
- Multi-agent DP reframing.
- Dimension reduction by invariant.
- Round-trip = two one-way trips trick.
9. When to Move On
- Solve cold in 40 min with O(n³).
- Explain the “two simultaneous walks” reframing on the whiteboard.
- Handle thorn invalidation cleanly.
- Mention Cherry Pickup II as the K-agent generalization.
10. Company Context
- Google: L5/L6; classic.
- Amazon: L6; harder onsite.
- Meta: E5/E6; rare but signals.
Strong: “Reframed as two simultaneous walks unprompted, derived c2 = r1+c1-r2 invariant, O(n³), correct shared-cell dedup, clean -inf propagation.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Reframing | None | After hint | Cold | + K-agent generalization |
| State | O(n⁴) | O(n³) after hint | O(n³) cold | + cites Cherry Pickup II |
| Dedup | Wrong | After test fail | Cold | + proves correctness rigorously |
| Generalization | None | None | LC 1463 | + multi-agent value iteration framing |
12. Level Delta
- Mid (L4): Brute or wrong-greedy; gets O(n⁴) DP after hints.
- Senior (L5): Reframes cold, derives invariant, O(n³) with dedup.
- Staff (L6): + cites Cherry Pickup II + frames as multi-agent value iteration.
- Principal (L7+): + Bellman optimality on product MDP + K-agent extensions + connections to multi-commodity flow.
13. Follow-up Q&A
Q1: K agents instead of 2?
A1: State (r1, c1, r2, c2, ..., rK, cK); can reduce one dim by step-count invariant; remaining O(n^(2K-1)). Exponential in K.
Q2: What if grid has weighted cherries (different values)?
A2: Same DP; gain uses the weight not 1.
Q3: 4-directional movement instead of right/down only? A3: Loses DAG structure; becomes shortest-path with state including consumed cells → exponential. NP-hard in general.
Q4: Reconstruct the actual two paths? A4: Parent pointers per state; backtrack from (n-1,n-1,n-1).
Q5: Difference from Cherry Pickup II (LC 1463)?
A5: Two robots both START at top (col 0 and col n-1) and both walk DOWN. State (r, c1, c2); both at same r per step.
14. Solution Walkthrough
See solution.py:
cherry_pickup_dp— top-down memoized 3D(r1, c1, r2).cherry_pickup_iter— bottom-up 3D iterative by step count.cherry_pickup_brute— enumerate all forward + backward path pairs (oracle for tiny grids).
15. Beyond the Problem
Principal code review: “Cherry Pickup is the prototype for ‘multi-agent grid DP with shared state’. The reframing — round trip becomes two simultaneous one-way trips — is the kind of move that distinguishes a senior IC from a staff engineer: instead of attacking the problem head-on, restructure the state space until the obstacles vanish. The same move appears in: multi-commodity flow (sending two flows is like sending one in a tensor-product graph), distributed systems consensus (n-replica problems reframed as 2-replica with witnesses), and ML (multi-task learning’s shared encoder = two tasks walking the same network). The general principle: ‘when two processes share structure, model them as one process on a product space.’ At Staff+, articulate this as Bellman optimality on a product MDP, then point out that the dimension reduction (c2 derived from r1+c1-r2) is just exploiting the conserved quantity (step count) — the same trick as exploiting conserved quantities in physics to reduce dimensionality of differential equations.”
Hints — p78 Cherry Pickup
Hint 1. Greedy “best forward path, then best backward path” looks reasonable but fails. Why? The forward greedy may starve the backward path of a better lane. Need DP.
Hint 2. Key reframing: a round trip (right/down then left/up) is equivalent to TWO simultaneous walks from (0,0) to (n-1,n-1), both moving right/down. Time-reverse the backward walk to see this.
Hint 3. State: (r1, c1, r2, c2) is O(n⁴). Observe both walks take the same number of steps: r1 + c1 = r2 + c2 = t. So c2 = r1 + c1 - r2. State becomes 3D: O(n³).
Hint 4. Gain at each state: if (r1,c1) == (r2,c2), count cherry ONCE. Otherwise sum both cells. Thorns (-1) make the state invalid (-inf).
Hint 5. Recurrence: for each of 4 predecessor combos (each agent moved down or right), take the max. Final answer: max(0, dp[n-1][n-1][n-1]) (clamp -inf to 0 if no valid path).
If stuck: see solution.py.
p79 — Stone Game II
Source: LeetCode 1140 · Medium (functionally Hard) · Topics: Game DP, Minimax, Suffix Sums Companies (2024–2025): Google (high), Amazon (medium), Meta (medium) Loop position: The minimax game DP with a dynamic M parameter. State
(i, M); suffix sums collapse “rest of game” into O(1). Tests dynamic state design.
1. Quick Context
Piles[i] stones. Two players alternate. On each turn, current player takes the first X piles where 1 ≤ X ≤ 2M. Then M = max(M, X). Initial M = 1. Both play optimally to maximize their own stones. Alice goes first. Return Alice’s max stones.
Game DP: dp[i][M] = max stones the current player can collect from piles i..n-1 with parameter M. Suffix sum suf[i] = sum(piles[i:]). Recurrence:
dp[i][M] = max over x in 1..min(2M, n-i) of (suf[i] - dp[i+x][max(M, x)])
(Current player takes the first x piles → suf[i] - suf[i+x]; opponent then plays optimally from state (i+x, max(M,x)) collecting dp[i+x][max(M,x)]; so current player gets total = my immediate take + (remaining suffix minus opponent’s optimal). Cleaner form: my total = suf[i] - opponent’s optimal = suf[i] - dp[i+x][max(M,x)].)
Answer: dp[0][1].
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/stone-game-ii/
STOP. 45-min timer. The “my-take = suffix - opponent’s-optimal” identity is the trick.
3. Prerequisite Concepts
- Minimax on games.
- Suffix sums for O(1) “rest of game total”.
- State design with a dynamically-updated parameter (M).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Alternate-turn game where each player takes 1..2M front piles; M grows. Maximize own total.
2. Clarify: Both optimal? Yes. M can exceed n? min(2M, n-i) caps it.
3. Examples: piles=[2,7,9,4,4] → Alice 10. piles=[1,2,3,4,5,100] → Alice 104.
5. Brute: Recursive minimax without memo. Exponential.
6. Target: O(n²) (states: n × n M values; transitions O(n)).
7. Pattern: Zero-sum game DP with suffix-sum identity.
8. Develop: see solution.py.
9. Test: monotonic increasing (last-pile-grab wins); small inputs.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why dp[i][M] is “current player’s max” not “Alice’s max”
Symmetry: roles flip each turn but objective is “maximize my own”. Track from current-player POV; alternate naturally via recursion.
6.2 The suffix-sum identity
If from state (i, M) the current player takes x piles, they get suf[i] - suf[i+x]. Opponent then plays state (i+x, max(M,x)) and collects dp[i+x][max(M,x)] for themselves. Total in suffix from i = suf[i] = my immediate take + opponent’s take + my future take. Rearranging: my total from (i,M) = (suf[i] - suf[i+x]) + (suf[i+x] - dp[i+x][max(M,x)]) = suf[i] - dp[i+x][max(M,x)]. The middle two cancel.
6.3 Why M can be capped at n
M starts at 1 and grows by at most n per turn. Practically M ≤ n suffices since 2M ≥ 2n already lets the player take everything. Cap M at n in the table.
6.4 Base case
i == n: no piles left; dp = 0. Also if 2M >= n - i: current player can take everything → dp[i][M] = suf[i].
6.5 Memoization order
Top-down with @lru_cache is cleanest. Bottom-up: iterate i from n down to 0; M from 1 to n.
6.6 Family: zero-sum game DPs
- LC 877 Stone Game (take from either end; constant difference DP).
- LC 1140 Stone Game II (this one; dynamic M).
- LC 1406 Stone Game III (take 1, 2, or 3 from front; minimax).
- LC 1510 Stone Game IV (take square number; win/lose only, not score).
- LC 1690 Stone Game VII (alternating with score on remaining sum).
All share the “my total = total - opponent’s optimal” identity.
7. Anti-Pattern Analysis
- Track Alice’s total directly — confuses player roles; harder to recurse.
- Forget to update M = max(M, x) — game rule violated.
- No suffix sum — recompute “rest of pile total” each call; quadratic factor explodes.
- Forget the
2M ≥ n - ibase case — extra work; correctness preserved but slower. - Off-by-one on the
1..2Mrange — must include x = 2M. - Try to track both players’ totals as state — explodes the state space; the suffix-sum identity eliminates the need.
8. Skills & Takeaways
- Game DP from current-player POV.
- Suffix sums as zero-cost “rest of game” total.
- Dynamic state parameter (M grows).
9. When to Move On
- Solve cold in 35 min, articulate the suffix-sum identity unprompted.
- Cap M correctly, handle the “take everything” base case.
- Mention the Stone Game family.
10. Company Context
- Google: L5/L6; classic game DP.
- Amazon: L6; alternate-turn DP probe.
- Meta: E5; less common.
Strong: “Game DP from current-player POV with suffix-sum identity; cold; capped M, handled base case, cited family.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| POV | Tries Alice-specific | Current-player after hint | Current-player cold | + cites identity proof |
| Identity | Misses | After hint | Cold | + derives suf cancellation |
| M cap | Misses | After test | Cold | + proves M ≤ n suffices |
| Family | None | None | Stone Game I/III | + minimax-on-DAG framing at Principal |
12. Level Delta
- Mid (L4): Minimax brute; gets memo after struggle.
- Senior (L5): Game DP from current-player POV with suffix-sum identity, cold; correct M cap and base case.
- Staff (L6): + cites Stone Game family + minimax-on-DAG framing.
- Principal (L7+): + alpha-beta pruning + Nash equilibrium for non-zero-sum + connection to MDPs and policy iteration.
13. Follow-up Q&A
Q1: What if it’s Bob’s turn first?
A1: Return total - dp[0][1] since dp gives current-player optimal.
Q2: 3 players?
A2: Cyclic; dp[i][M][turn]; current player’s max but each tracks own. State doubles.
Q3: Non-zero-sum (e.g., bonuses)? A3: Suffix-sum identity breaks; need to track both totals as state (or as differences) — much larger.
Q4: Bob plays adversarially against Alice (not for Bob’s own max)?
A4: Bob’s objective = minimize Alice. Still minimax but Bob’s dp[i][M] = min Alice’s future gain. Identity still works.
Q5: Reconstruct Alice’s actual moves? A5: At each state, record argmax x; replay from (0,1).
14. Solution Walkthrough
See solution.py:
stone_game_ii_dp— bottom-up O(n²) iterative.stone_game_ii_memo— top-down@lru_cache.stone_game_ii_brute— recursive minimax without memo (exponential oracle).
15. Beyond the Problem
Principal code review: “Stone Game II is the prototype for ‘minimax DP with dynamic parameter’. The suffix-sum identity (
my_total = total - opponent_optimal) is the algebraic move that collapses a 2-quantity state into a 1-quantity state. The same move underlies: zero-sum 2-player games in game theory (‘Nash equilibrium reduces to 1-player optimization on a transformed payoff’), backpressure routing (‘queue differential collapses 2 queue states into 1 gradient’), and even gradient descent (‘only the relative loss matters; absolute level is free’). At Staff+, articulate this as: ‘when the game is zero-sum, the state collapses by one dimension via conservation.’ Then point out the next level: non-zero-sum requires Nash equilibrium computation (NP-hard in general; LP for 2-player zero-sum) — and that’s why this problem is solvable in polynomial time only because of the zero-sum structure.”
Hints — p79 Stone Game II
Hint 1. Game DP from CURRENT-PLAYER POV (not Alice-specific). State (i, M) = piles remaining + parameter. dp[i][M] = max stones current player gets from this state.
Hint 2. Suffix sum identity: if current player takes x piles starting at i, total remaining sum is suf[i]. Opponent then plays optimally and gets dp[i+x][max(M,x)] for themselves. So current player gets suf[i] - dp[i+x][max(M,x)]. The middle terms cancel.
Hint 3. Recurrence: dp[i][M] = max over x in 1..min(2M, n-i) of (suf[i] - dp[i+x][max(M,x)]). Don’t forget M = max(M, x) per game rule.
Hint 4. Base cases: i == n → 0 (no piles). 2M >= n - i → take everything, dp[i][M] = suf[i]. M can be capped at n (since 2M ≥ n always suffices).
Hint 5. Fill order: iterate i from n down to 0; M from 1 to n. Or use top-down memoization. Answer: dp[0][1].
If stuck: see solution.py.
p80 — Number of Ways to Wear Different Hats
Source: LeetCode 1434 · Hard · Topics: Bitmask DP, Subject Flip Companies (2024–2025): Google (high), Amazon (high), Meta (medium-high), Microsoft (medium) Loop position: The bitmask-DP “flip the subject” trick. Naive bitmask on hats is 2^40 — infeasible. Flipping to bitmask on people (≤10 → 2^10 = 1024 states) makes it trivial. Tests whether the candidate sees the dimensional flip.
1. Quick Context
n people (≤ 10). 40 hats. hats[i] = list of hat ids person i likes. Each person wears exactly one hat; each hat worn by at most one person. Count ways to assign hats so everyone wears a hat they like. Modulo 10^9 + 7.
KEY: people ≤ 10 → bitmask on people (2^10 = 1024). Iterate hats (40) outer, people-masks inner, people who like current hat in innermost.
dp[h][mask] = number of ways using hats 1..h to satisfy exactly the people in mask. Transitions:
- Skip hat h:
dp[h][mask] += dp[h-1][mask]. - Assign hat h to some person p who likes it AND p in mask:
dp[h][mask] += dp[h-1][mask ^ (1<<p)].
Build liked_by[hat] = list of people who like that hat. O(40 · 1024 · 10) ≈ 410K ops.
Answer: dp[40][(1<<n) - 1] mod 1e9+7.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/number-of-ways-to-wear-different-hats-to-each-other/
STOP. 45-min timer. The “flip subject” insight is the entire problem.
3. Prerequisite Concepts
- Bitmask DP (LC 691, 698, 943).
- “Subject flip” pattern: when one dimension is small (people = 10), iterate on the other (hats = 40) outer.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Count perfect assignments people→hats respecting preferences, distinct hats, mod 1e9+7.
2. Clarify: n ≤ 10? Yes (critical constraint). Hats numbered 1..40. Output mod 1e9+7.
3. Examples: hats=[[3,4],[4,5],[5]] → 1 (only assignment 3,4,5). hats=[[3,5,1],[3,5]] → 4.
5. Brute: Recursively assign person 0 a liked hat, mark used, recurse. Exponential in n (≤ 10! = 3.6M; tractable for n ≤ 10 but slower).
6. Target: O(40 · 2^n · n).
7. Pattern: Bitmask DP with subject flip.
8. Develop: see solution.py.
9. Test: single person; everyone likes same hat (→ multiple but each can wear one); n=10 random.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why naive “bitmask on hats” fails
Naive bitmask dp[hat_mask] = number of ways to assign hats marked in hat_mask to people. States: 2^40 = 1 trillion. Infeasible.
6.2 The flip: iterate over hats, bitmask on people
People = 10 → 2^10 = 1024 states. Iterate hats 1..40 outer (linear), people-mask inner (1024), people-who-like-this-hat innermost (≤10).
dp[h][mask] = number of ways using hats 1..h to satisfy exactly the people in mask.
6.3 Why iterate hats outer (not people outer)
If we iterate people outer (“for each person, assign a hat”), we’d need a hat-bitmask state to track usage → 2^40. Iterating hats outer lets us SKIP unused hats freely (each hat is “use or skip”); only need to track people satisfied so far.
6.4 Precompute liked_by[hat]
For each hat 1..40, list of people who like it. Built in O(sum of hats[i] sizes) one-pass.
6.5 Transitions (in code)
for h in 1..40:
for mask in 0..(1<<n)-1:
dp[h][mask] = dp[h-1][mask] # skip hat h
for p in liked_by[h]:
if mask & (1<<p):
dp[h][mask] = (dp[h][mask] + dp[h-1][mask ^ (1<<p)]) % MOD
6.6 1D rolling
dp[mask] only depends on dp[mask] (skip) and dp[mask ^ (1<<p)] (a subset of mask with one fewer bit). Iterate masks in any order if we use a fresh array per hat — or, since mask ^ (1<<p) has STRICTLY fewer bits than mask, iterate masks from high to low population… actually the safest is two-array.
6.7 General principle: “small-dimension subject flip”
When two dimensions exist (A small, B large), make B the outer loop variable and A the state. The state space is 2^|A|; the loop is |B|. Total O(|B| · 2^|A| · |A|).
Examples:
- LC 943 Shortest Superstring: n ≤ 12 words; bitmask on words.
- LC 691 Stickers to Spell Word: target ≤ 15 chars; bitmask on target chars.
- LC 1601 Maximum Number of Achievable Transfer Requests: n ≤ 16 buildings; enumerate request-subsets.
- LC 1986 Min Sessions: tasks ≤ 14; bitmask on tasks.
7. Anti-Pattern Analysis
- Bitmask on hats (2^40) — infeasible; classic mistake.
- Iterate people outer with bitmask on hats — same trap, different framing.
- Forget to mod — overflow on n=10 with many liked hats; answer wrong.
- Off-by-one on hat index — hats are 1..40, often easier to 0-index internally.
- Iterate the “people who like hat h” outer instead of inner — works but no speedup.
- Try meet-in-the-middle on hats — 2^20 × 2^20 still too much; the flip is cleaner.
8. Skills & Takeaways
- Subject-flip insight on bitmask DP.
- Per-element subset enumeration in O(2^k · k).
- Modular arithmetic discipline.
9. When to Move On
- Solve cold in 35 min with the flip articulated unprompted.
- Cite at least 2 other “small-dimension flip” problems (943, 691).
- Mod correctly; no overflow.
10. Company Context
- Google: L5/L6; bitmask DP is a frequent probe.
- Amazon: L6; assignment problems.
- Meta: E5/E6; sometimes wrapped as scheduling.
Strong: “Identified the flip cold (n ≤ 10 → bitmask people, iterate hats outer), O(40 · 1024 · 10), cited family (Stickers, Superstring), correct mod.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Flip | None (2^40) | After hint | Cold | + cites general principle |
| Subject choice | Wrong | After hint | Right cold | + proves bound |
| Family | None | None | 1+ cites | Multiple cites + Hungarian/assignment LP framing |
| Mod | Forgets | After review | Correct | + frames as ring arithmetic |
12. Level Delta
- Mid (L4): Brute or wrong-bitmask; gets flipped DP after explicit hint.
- Senior (L5): Flip cold; cites at least one similar problem; correct mod and complexity.
- Staff (L6): + general “small-dimension flip” principle + cites 3+ similar problems + Hungarian algorithm comparison.
- Principal (L7+): + assignment problem as LP + permanent of matrix = #perfect matchings = #P-hard in general (but tractable here because one side ≤ 10) + connection to inclusion-exclusion.
13. Follow-up Q&A
Q1: What if n = 30? A1: 2^30 = 1G states; infeasible. Switch to matching-based algorithm (bipartite #perfect-matchings via permanent — #P-hard in general; approximate with Ryser’s formula at 2^n · n).
Q2: What if hats = 1,000,000? A2: Only hats that are liked by SOMEONE matter; filter to active hats first. Active ≤ 10 · max_liked_per_person.
Q3: Maximize assignments instead of count? A3: Bipartite max matching (Hopcroft-Karp); polynomial.
Q4: Each person can wear multiple hats? A4: Reduces to “for each hat, independently pick a person who likes it or skip” → product across hats.
Q5: Min-cost assignment (weighted hats)? A5: Bipartite min-cost matching (Hungarian, O(n³) on small side).
14. Solution Walkthrough
See solution.py:
number_ways_dp— bottom-up O(40 · 2^n · n).number_ways_memo— top-down@lru_cacheon (hat, mask).number_ways_brute— recursive assignment with used-hats set (oracle).
15. Beyond the Problem
Principal code review: “Number of Ways to Wear Different Hats is the canonical ‘subject-flip bitmask DP’. The insight — when two dimensions exist and one is small, flip which one becomes the state — is one of the most-leveraged tricks in competitive programming. The same principle drives: GPU kernels (iterate over the small dim outer, batch the large dim inner), database joins (build hash table on the smaller side, probe with the larger), and randomized algorithms (state-space pruning: small state ⇒ memoize, large state ⇒ Monte Carlo). At the Staff+ bar, frame this as: ‘the optimal subject of memoization is the dimension whose value space is exponential-but-small; the other dimension becomes a linear loop.’ Then connect to the deeper result: counting bipartite perfect matchings is #P-hard via matrix permanent, but tractable when one side is small — making this problem a tractable instance of an intractable general problem, which is exactly the kind of structural observation that distinguishes principal-level engineering judgment.”
Hints — p80 Number of Ways to Wear Different Hats
Hint 1. Naive bitmask on hats = 2^40 states. Infeasible. What’s special about the input? n ≤ 10 people. Flip the subject: bitmask on PEOPLE (2^10 = 1024) instead.
Hint 2. Iterate hats OUTER (1..40), people-mask INNER (0..2^n-1). Why outer hats? Each hat is “use or skip” — no need to track which hats are used; only track which people are satisfied.
Hint 3. Precompute liked_by[hat] = list of people who like that hat. Built in one pass over hats.
Hint 4. dp[h][mask] = ways using hats 1..h to satisfy exactly the people in mask. Transition: skip hat h (+= dp[h-1][mask]); or for each person p in liked_by[h] AND in mask: += dp[h-1][mask ^ (1<<p)].
Hint 5. Mod 1e9+7 every addition. Answer: dp[40][(1<<n) - 1]. Total ops: 40 × 1024 × 10 ≈ 410K. Fast.
If stuck: see solution.py.
Month 4 — Greedy & Systems
Weeks 16+. Greedy with exchange-argument proofs, binary-search-on-answer, and the bridge to practical engineering problems.
Weeks
- Week 16 — Greedy Mastery & Exchange Arguments: Queue Reconstruction (2D sort key), Gas Station (failure-skip lemma), Task Scheduler (heap vs closed form), Burst Balloons (interval scheduling), Split Array Largest Sum (binary search on answer).
Why this month
Greedy is the most dangerous interview category — wrong greedy choices pass the obvious tests and fail silently on the cases that matter. This month focuses on proving greedy correct via exchange arguments and recognizing when binary-search-on-answer is the right tool. The Staff/Principal signal here is justification, not just code.
Week 16 — Greedy Mastery & Exchange Arguments
Theme: Greedy is the most dangerous interview category. The wrong greedy choice produces code that passes 90% of cases and fails silently on the 10% that determine the hire decision. This week trains the exchange argument — the only honest tool for proving a greedy is correct — and contrasts greedy against binary-search-on-answer where they meet in the middle.
Daily schedule
| Day | Problem | LC # | Difficulty | Key archetype |
|---|---|---|---|---|
| Mon | p81 — Queue Reconstruction by Height | 406 | Medium | Sort + insert at index — tallest-first invariant |
| Tue | p82 — Gas Station | 134 | Medium | Prefix-sum greedy — failure-point skip lemma |
| Wed | p83 — Task Scheduler | 621 | Medium | Frequency greedy + slot formula — max-heap simulation vs closed-form |
| Thu | p84 — Min Arrows to Burst Balloons | 452 | Medium | Interval greedy — sort by end, count disjoint |
| Fri | p85 — Split Array Largest Sum | 410 | Hard | Binary-search-on-answer + greedy feasibility — DP↔greedy parity |
Readiness gate (Friday)
- State the exchange argument for sorting tall-first in Queue Reconstruction.
- Prove the “skip past the failure point” lemma for Gas Station (why you never need to restart in the middle).
- Derive the Task Scheduler closed-form
max(n, (max_freq-1)*(n+1) + count_of_max)from the max-heap simulation. - Sort intervals by end (NOT start) for arrow-bursting and explain why start-sort fails.
- For Split Array Largest Sum, articulate the bijection between the DP solution and the binary-search-on-answer solution — why both give the same answer and when each is preferable.
Why these together
These five form the greedy correctness toolkit:
- Queue Reconstruction — exchange argument over a 2D sort key.
- Gas Station — prefix-sum invariant proves no restart is ever beneficial below the failure point.
- Task Scheduler — same problem solved two ways (heap simulation, closed-form), illustrating that recognizing the algebraic structure beats brute simulation.
- Burst Balloons — interval scheduling, the canonical “sort by end” pattern that also underlies activity selection.
- Split Array Largest Sum — bridges greedy (feasibility check) and binary search (parameter search) and contrasts with the O(n²·k) DP. The Staff-level signal is choosing the right tool given constraints.
By Friday, the candidate should be able to prove a greedy correct (not merely test it) and to detect when a problem framed as greedy is actually binary-search-on-answer wearing greedy clothes.
p81 — Queue Reconstruction by Height
Source: LeetCode 406 · Medium · Topics: Greedy, Sorting, Exchange Argument Companies (2024–2025): Google (high), Amazon (medium), Meta (medium), Stripe (medium) Loop position: The canonical 2D-sort-key greedy. Tests whether you can derive (not just recall) the “tallest-first, then insert at index” trick.
1. Quick Context
Input: list of (h, k) pairs — person of height h has k taller-or-equal people in front of them in some valid arrangement. Reconstruct the queue.
Insight: Sort by (-h, k) — tallest first, ties broken by ascending k. Iterate and insert(k, person) into the result list. Because already-placed people are all ≥ this person’s height, the count of “taller-or-equal in front” equals the insertion index k.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/queue-reconstruction-by-height/
STOP. 30-min timer. Try brute permutation first to internalize the constraint, then look for an invariant.
3. Prerequisite Concepts
- Stable sort with composite key.
- List
insert(index, value)semantics (O(n)). - Exchange argument for proving greedy correctness.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Given (h, k) pairs, output the queue arrangement satisfying every person’s k constraint.
2. Clarify: Heights unique? Often yes; if duplicates, ties matter — same-height people count toward each other’s k.
3. Examples: [[7,0],[4,4],[7,1],[5,0],[6,1],[5,2]] → [[5,0],[7,0],[5,2],[6,1],[4,4],[7,1]].
5. Brute: Permute all n! arrangements; check k for each.
6. Target: O(n²) (sort O(n log n) + n inserts at O(n) each).
7. Pattern: Sort by primary descending, secondary ascending; insert at index.
8. Develop: see solution.py.
9. Test: all same height; strictly decreasing; one person; duplicates.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The exchange argument
Claim: place tallest first. Proof: shorter people don’t affect a taller person’s k-count, so inserting shorter people later cannot disturb already-placed taller people’s positions. Conversely, placing a shorter person first would require knowing where future taller people land — circular.
6.2 Why ascending k breaks ties
Among same-height people, the one with smaller k must come first (their k counts include the other same-height people). Sorting (-h, k) ensures [5,0] is placed before [5,2], and insert(0, [5,0]) then insert(2, [5,2]) produces [..., [5,0], _, [5,2], ...] correctly.
6.3 Why insert(k, person) is correct
When inserting person (h, k), every already-placed person has height ≥ h. So whatever index we insert at, that index = count of people in front who are taller-or-equal = k. Insert at k.
6.4 Complexity refinement
O(n²) with Python list. With a balanced BST or order-statistic tree (Fenwick of free slots), O(n log n). Rarely asked at the interview bar; mention as a follow-up.
6.5 Family: 2D-sort-key greedy
- LC 406 Queue Reconstruction (this).
- LC 354 Russian Doll Envelopes (sort by w asc, h desc → LIS on h).
- LC 452 Min Arrows Burst Balloons (sort by end asc).
- LC 1235 Job Scheduling (sort by end, DP + binary search).
The pattern: choose primary key for the “anchor” invariant, secondary key to disambiguate ties.
6.6 When greedy intuition is wrong
If heights could be unbounded and k could exceed n, the problem is ill-posed. The greedy assumes a valid queue exists; doesn’t verify.
7. Anti-Pattern Analysis
- Sort by height ascending — already-placed shorter people don’t satisfy the “taller-or-equal” predicate, so index ≠ k.
- Sort ties by descending k — places
[5,2]before[5,0], breaking the same-height ordering invariant. - Append instead of insert — produces wrong positions.
- Forget that k counts same-height people too — easy to convince yourself k is “strictly taller”.
- Try DP — exponential states; greedy is asymptotically better and conceptually cleaner.
- Brute permutation in production code — n! grows; only acceptable as oracle.
8. Skills & Takeaways
- 2D sort key design.
- Exchange argument for greedy correctness.
- Why insertion-by-rank works when prefix is “all ≥”.
9. When to Move On
- Solve cold in 20 min, articulate the exchange argument unprompted.
- Justify each sort key direction.
- Cite Russian Doll Envelopes as a sibling.
10. Company Context
- Google: L4/L5; greedy correctness signal.
- Amazon: L5; sort-then-insert template.
- Meta: E4/E5; common 2D-sort warmup.
- Stripe: mid; loves greedy proof questions.
Strong: “Sort tallest-first ties broken by ascending k, insert at index k; cold; exchange argument articulated; cited 2D-sort family.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Sort key | Guesses | After hint | Cold | + derives from exchange |
| Tie-break | Misses | Backwards | Correct | + proves both directions wrong |
| Insert vs append | Append | After test | Insert | + cites O(n log n) refinement |
| Family | None | None | Russian Dolls | + Job Scheduling sibling |
12. Level Delta
- Mid (L4): Greedy after hint; struggles with tie-break direction.
- Senior (L5): Sort + insert cold; correct tie-break; explains exchange argument.
- Staff (L6): + Fenwick-of-free-slots O(n log n) refinement + 2D-sort-key family.
- Principal (L7+): + connects to topological-sort variants + when exchange arguments fail (matroid intersection NP-hard in general).
13. Follow-up Q&A
Q1: Heights with duplicates — does tie-breaking matter? A1: Yes. Ascending k among same heights is required.
Q2: What if k can equal the number of people not less than h (instead of taller-or-equal)? A2: Strict version. Sort ties differently; insert offset by count of same-height already inserted. Solvable; cleaner with explicit predicate.
Q3: Streaming version — people arrive one by one? A3: Order-statistics tree of empty slots; O(n log n).
Q4: How to verify an arrangement is valid?
A4: For each person, count j < i with height[j] >= height[i]; compare to k. O(n²).
Q5: What if no valid arrangement exists? A5: Our greedy still outputs something; verify with Q4 to detect.
14. Solution Walkthrough
See solution.py:
reconstruct_queue— sort + insert; O(n²).reconstruct_queue_fenwick— Fenwick of free slots; O(n log² n).reconstruct_queue_brute— try all permutations; oracle.
15. Beyond the Problem
Principal code review: “Queue Reconstruction is the smallest exchange-argument problem with a 2D sort key. The lesson scales: in distributed systems, you constantly face ‘order-by-key-A-then-key-B’ problems — Spanner timestamps (commit-time then transaction-id), Kafka partition-then-offset, paxos round-then-proposer. The exchange argument generalizes: when you can prove that swapping two adjacent items in violation of the order strictly worsens the objective, the sorted order is optimal. The deeper move at Staff+: recognize that greedy works iff the underlying problem is a matroid (Edmonds’ theorem). Queue Reconstruction is a degenerate matroid (the free-slot poset); Russian Dolls is the partial-order matroid; job scheduling is a transversal matroid. When the problem isn’t a matroid, greedy still might work, but you owe a proof — usually via exchange or via reduction to a known matroid.”
Hints — p81 Queue Reconstruction by Height
-
The k field counts taller-or-equal in front, not strictly taller. Same-height people count.
-
If you place the tallest person first, no one you place later can ever invalidate their k-count (shorter people don’t add to “taller-or-equal”).
-
Sort by
(-h, k)— tallest first, then ascending k among ties. Process in this order. -
When inserting person
(h, k): every already-placed person has height ≥ h. So the insertion index = count of taller-or-equal in front = k. Uselist.insert(k, [h, k]). -
Ascending k for ties matters: same-height persons count toward each other’s k, so place the one with smaller k first.
If stuck: see solution.py.
p82 — Gas Station
Source: LeetCode 134 · Medium · Topics: Greedy, Prefix Sums Companies (2024–2025): Amazon (high), Google (medium), Meta (medium), Uber (medium) Loop position: The failure-point skip lemma. Tests whether you can prove a one-pass greedy correct, not just simulate.
1. Quick Context
n gas stations in a circle. gas[i] available at station i; cost[i] to travel from i to i+1. Start with empty tank. Return the starting index that lets you complete the circle, or -1 if impossible.
Key facts:
- If
sum(gas) < sum(cost): impossible. Return -1. - Otherwise a unique-modulo-symmetry start exists.
One-pass greedy: Track total (overall surplus) and tank (current segment surplus). If tank drops below 0 at station i, no start in [start, i] works; reset start = i+1, tank = 0.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/gas-station/
STOP. 25-min timer. The lemma “if you fail at i starting from s, no j in [s, i] succeeds” is the whole problem.
3. Prerequisite Concepts
- Prefix sums on a circular array.
- Pigeonhole / invariant-style proofs.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Find a starting index so the running tank stays ≥ 0 around the circle.
2. Clarify: Negative tank disallowed between stations; -1 if no valid start.
3. Examples: gas=[1,2,3,4,5], cost=[3,4,5,1,2] → start=3.
5. Brute: Try each start; simulate full circle. O(n²).
6. Target: O(n) one pass.
7. Pattern: Failure-point skip + global feasibility check.
8. Develop: see solution.py.
9. Test: sum(gas) < sum(cost); single station; all costs zero.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The skip lemma
Claim: If starting from s, the tank first goes negative at station i (i.e., after refueling at i and trying to depart for i+1), then no start j ∈ [s, i] is valid.
Proof: Let T(a, b) = net tank from a (start tank 0) reaching b (after fueling at b but before departing). By assumption T(s, j) ≥ 0 for all j ∈ [s, i-1] and T(s, i) < 0. For any j ∈ (s, i]: T(j, i) = T(s, i) - T(s, j-1) ≤ T(s, i) < 0 (since T(s, j-1) ≥ 0). So j also fails by station i. ∎
Therefore: when failure happens at i, skip past i: reset start to i+1.
6.2 Why one pass suffices
Combined with the lemma, each station is visited at most once as part of an attempt. If total = sum(gas - cost) ≥ 0, the surviving candidate start must be valid (proof: suppose not; then it fails somewhere, contradiction since total surplus is non-negative and skips already eliminated all earlier candidates).
6.3 Why total ≥ 0 guarantees existence
sum(gas - cost) ≥ 0 ⟹ exists a circular arrangement with non-negative running prefix from some start (a standard “rotate to minimum” argument).
6.4 Equivalent: “rotate-to-minimum-prefix” formulation
Compute diff[i] = gas[i] - cost[i]. Best start = index immediately after the global minimum prefix sum. Equivalent answer; less elegant code than the skip greedy but easier to prove.
6.5 Family: circular prefix-sum problems
- LC 134 Gas Station (this).
- LC 918 Max Subarray Sum Circular (Kadane + suffix or total-min variant).
- LC 525 Contiguous Array (prefix sum + hashmap).
- LC 974 Subarray Sums Divisible by K.
The common move: convert circular to linear via doubled array or via algebraic identity.
6.6 What changes if tank capacity is bounded?
If tank cap C exists, the skip lemma still holds for “≤ 0” failures but you can no longer carry excess from rich-gas regions. Becomes a windowed problem; often solvable with monotonic deque on prefix sums.
7. Anti-Pattern Analysis
- Reset start to i (not i+1) — i itself already failed; off-by-one.
- Don’t compute global total — when no valid start exists, your greedy still produces a “candidate” that doesn’t work.
- Two-loop O(n²) — works but signals you missed the lemma.
- Reset tank but not start — wrong; the candidate must move past the failure point.
- Track minimum prefix without proving it’s optimal — works but unconvincing for a senior interview.
- Allow negative tank temporarily, hoping later gas refills — rule violation; must remain ≥ 0 at every step.
8. Skills & Takeaways
- Failure-point skip greedy.
- Global feasibility check via total surplus.
- Circular ↔ linear conversion.
9. When to Move On
- Solve cold in 20 min, prove the skip lemma unprompted.
- Articulate why
total >= 0is necessary AND sufficient. - Cite Max Subarray Circular as a sibling.
10. Company Context
- Amazon: L5/L6; very common screen.
- Google: L5; greedy correctness probe.
- Meta: E5; expects the lemma proof.
- Uber: mid; logistics-flavor problem.
Strong: “One-pass skip greedy with global feasibility check; proves the skip lemma; cold; cites circular-prefix family.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Skip lemma | None | After hint | Proves cold | + cites prefix-min equivalence |
| Total check | Misses | After test | Cold | + necessity & sufficiency proof |
| Off-by-one | Resets to i | After test | i+1 cold | + invariant comment |
| Family | None | None | Circular Kadane | + bounded-tank deque variant |
12. Level Delta
- Mid (L4): Two-loop O(n²); one-pass after hint.
- Senior (L5): One-pass with skip + total check; proves lemma when asked.
- Staff (L6): + prefix-min equivalence + bounded-tank deque extension.
- Principal (L7+): + control-theory framing (Lyapunov function for “tank” stability) + LP duality (cut interpretation of the minimum prefix).
13. Follow-up Q&A
Q1: Multiple valid starts — return all? A1: All form a contiguous arc (proof from the lemma). Linear scan to find the arc bounds.
Q2: Tank has cap C?
A2: At each step tank = min(C, tank + gas[i]) - cost[i]. Skip lemma weakens; sliding-window/monotonic-deque on prefixes works.
Q3: Variable cost depending on direction (forward vs reverse)? A3: Try both directions; same O(n) per direction.
Q4: Multiple vehicles starting simultaneously, each consuming from station gas? A4: Multi-agent scheduling; usually NP-hard in general; LP relaxation often used.
Q5: Prove total ≥ 0 ⟹ a valid start exists. A5: Let S = index of minimum running prefix sum. Starting at S+1 makes all prefixes non-negative (by rotation). Combined with total ≥ 0, the circular loop closes non-negatively.
14. Solution Walkthrough
See solution.py:
can_complete_circuit— one-pass O(n) skip greedy.can_complete_circuit_prefix_min— O(n) via min-prefix identity.can_complete_circuit_brute— try each start; O(n²) oracle.
15. Beyond the Problem
Principal code review: “Gas Station is the cleanest example of a monotone failure-skip invariant: once a prefix fails, an entire region is eliminated. This pattern reappears in: TCP slow-start / congestion control (once you hit packet loss, an entire window is invalidated and we restart at a known good point), JIT deoptimization (once a speculative assumption fails at PC=i, every speculation rooted in the chain is invalidated and we rebase at the next safe IR boundary), and Paxos prepare-reject (once a higher proposal is observed, all in-flight lower proposals are doomed — the proposer skips ahead). At Staff+, the meta-lesson is: invariants that monotonically eliminate future work are the most powerful tools in algorithm and systems design. They turn O(n²) into O(n), turn quadratic congestion control into linear, and turn distributed consensus from impossible into terminating. The exam question is: when you face an O(n²) brute, is there an invariant — usually expressible as ‘if X fails at i, no j ∈ [start, i] succeeds’ — that lets you skip?”
Hints — p82 Gas Station
-
First check: if
sum(gas) < sum(cost), return -1 immediately. Necessary condition. -
Try simulating from each start (O(n²)). Then look for an invariant that lets you skip ahead.
-
Skip lemma: if tank goes negative at station i starting from s, then no j in [s, i] is valid. Why?
T(j, i) = T(s, i) - T(s, j-1) < 0sinceT(s, j-1) ≥ 0. -
Therefore on failure at i, reset
start = i + 1andtank = 0. Continue the same pass. -
After one pass, if
total = sum(gas - cost) ≥ 0, returnstart; else -1.
If stuck: see solution.py.
p83 — Task Scheduler
Source: LeetCode 621 · Medium · Topics: Greedy, Heap, Counting, Math Companies (2024–2025): Amazon (high), Meta (high), Google (medium) Loop position: Two equivalent solutions; choose your weapon. Heap simulation vs closed-form derivation. Tests whether you can recognize when algebra trumps simulation.
1. Quick Context
Tasks A, B, … need 1 unit time. Same-letter tasks must be separated by ≥ n idle units. Return minimum total time (including idles) to finish all.
Closed form: Let f_max = max frequency, c = count of tasks with that max frequency.
answer = max(len(tasks), (f_max - 1) * (n + 1) + c)
Heap simulation: Always run the highest-remaining-count task that isn’t on cooldown.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/task-scheduler/
STOP. 30-min timer. Solve with heap first, then derive the closed form.
3. Prerequisite Concepts
- Max-heap (Python: negate counts).
- Counter from
collections. - Pigeonhole reasoning over frame slots.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Schedule single-unit tasks with same-letter cooldown of n; minimize total time.
2. Clarify: Cooldown counts idle slots; case sensitive; n can be 0.
3. Examples: tasks=AAABBB, n=2 → A B _ A B _ A B = 8. tasks=A, n=100 → 1.
5. Brute: Backtracking over orderings; exponential.
6. Target: O(N) closed-form or O(N log K) heap (K = distinct task types ≤ 26).
7. Pattern: Frame the most-frequent task as anchor; fit others into gaps.
8. Develop: see solution.py.
9. Test: n=0 (no cooldown → just len); all same letter; tie at max freq.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The frame model
Place A’s first: A _ _ A _ _ A (f_max - 1 frames of size n+1, then a trailing A). Total slot count from this skeleton = (f_max - 1) * (n + 1) + 1. Now fit other tasks into the _ gaps. If they all fit, total = (f_max - 1) * (n + 1) + 1. If multiple tasks tie at f_max, append them to the end: +c where c = ties.
If the skeleton + fills uses fewer slots than there are tasks (when n is small and many distinct tasks), the answer is just len(tasks).
6.2 The closed-form derivation
answer = max(len(tasks), (f_max - 1) * (n + 1) + c)
(f_max - 1) * (n + 1)= slots in the first f_max - 1 frames.+ c= the last frame contains all max-frequency tasks (no idle needed for them, but they take c slots).max(len(tasks), ...)handles the case where many task types fill all idle slots.
6.3 Heap simulation (equivalent answer)
Each cycle of n+1 time units: pop the n+1 highest-count tasks, run them; push back with count - 1 if still positive. If fewer than n+1 tasks available, fill remainder with idles (but only if heap is non-empty after pops, i.e., still tasks remaining). Time per cycle = n+1 (or until heap empty).
6.4 Why both give the same answer
The heap simulation is the execution of the frame model; the closed form is its algebraic summary. The heap is more intuitive; the closed form is O(K log K) (sort once) and trivially extends to “which letter goes in which slot”.
6.5 Family: gap/cooldown scheduling
- LC 621 Task Scheduler (this).
- LC 1834 Single-Threaded CPU (heap of (process_time, idx)).
- LC 1882 Process Tasks Using Servers (two heaps).
- LC 358 Rearrange String k Distance Apart (same problem, return arrangement not length).
6.6 Variant — return the actual schedule
Heap simulation gives the schedule for free. Closed form needs a reconstruction pass: place max-freq tasks first at distances n+1, then round-robin remaining tasks into the gaps.
7. Anti-Pattern Analysis
- Brute backtracking — N! permutations; never finishes.
- Sort tasks by count and lay them out greedily without the frame model — incorrect when ties at f_max.
- Forget
max(len(tasks), ...)— closed form under-counts when many distinct tasks fill all idles. - Use
c = 1always — wrong if multiple tasks tie at f_max. - Off-by-one on
(f_max - 1) * (n + 1)— usingf_max * n + 1or similar. - Heap pops one at a time and decrements n separately — fragile; cleaner to pop a batch of n+1 per cycle.
8. Skills & Takeaways
- Closed-form derivation from a “frame + fill” model.
- Max-heap with cooldown via a temporary buffer.
- Choosing algebra over simulation when structure permits.
9. When to Move On
- Derive the closed form unprompted in 5 min after the heap solution.
- Articulate the
+cterm. - Cite LC 358 (Rearrange String k Distance Apart) as the “return the schedule” variant.
10. Company Context
- Amazon: L5/L6; appears in scheduling-flavored rounds.
- Meta: E5; loves the closed-form derivation.
- Google: L5; checks understanding of
c.
Strong: “Two solutions: O(N log K) heap simulation and O(N + K) closed-form max(N, (f_max-1)(n+1) + c); derived from frame model; cited rearrange-string sibling.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Heap sim | Buggy cooldown | Works | Clean | + extracts schedule |
| Closed form | None | After hint | Derives | + proves equivalence to heap |
+c term | Misses | After test | Cold | + handles edge n=0 cleanly |
| Family | None | None | Rearrange String | + multi-resource scheduling LP |
12. Level Delta
- Mid (L4): Heap simulation buggy; needs guidance on cooldown.
- Senior (L5): Both heap and closed-form, with derivation; correct
candmax(N, ...). - Staff (L6): + extracts actual schedule; + cites LP relaxation of multi-resource version.
- Principal (L7+): + reduces to graph coloring (interval graph) for the multi-machine generalization; + cites EDF (Earliest Deadline First) and rate-monotonic scheduling from RTOS literature.
13. Follow-up Q&A
Q1: Multiple machines (m parallel workers)?
A1: Closed form: max(ceil(N/m), ceil(f_max * (n+1) / m)) — but exact answer is NP-hard for arbitrary task graphs; LP relaxation works.
Q2: Tasks have priorities (some letters cheaper)?
A2: Heap by (priority, count); closed form breaks.
Q3: Return the actual sequence (not length)? A3: Heap pops naturally produce it; or place anchors then round-robin fill.
Q4: Cooldown varies per pair (A-B has cooldown 2, A-C has 5)? A4: General graph coloring; NP-hard.
Q5: Streaming — tasks arrive online? A5: Online scheduling; competitive ratio analysis (e.g., 2-competitive for SRPT).
14. Solution Walkthrough
See solution.py:
least_interval_closed— O(N + K) closed form.least_interval_heap— O(N log K) max-heap simulation.least_interval_brute— backtracking oracle.
15. Beyond the Problem
Principal code review: “Task Scheduler illustrates the gap between simulation and algebraic summary. The heap version is correct, runs in O(N log K), and resembles what a junior would write. The closed form is O(N + K), reveals the actual structure (one anchor frame, ties at top, slack absorbed by other tasks), and generalizes immediately: change n to a per-pair cooldown and the closed form breaks but the model still informs your design. The Staff+ lesson: when you write a simulation, always ask ‘is there a closed form?’ — sometimes the answer is no (NP-hard), sometimes it’s a clever counting argument (this problem), sometimes it’s a known result (Birkhoff polytope decomposition for doubly stochastic matrices). Engineers who reach for simulation reflexively without first looking for structure ship O(n) code where O(1) would do — and worse, they ship code without understanding why it’s correct.”
Hints — p83 Task Scheduler
-
The most-frequent task drives the schedule. If it appears
f_maxtimes with cooldown n, you need at least(f_max - 1) * (n + 1) + 1slots to space them. -
If
ctasks tie atf_max, append them to the end:+cextra slots (one per tie). -
But if many distinct task types fill every idle, the answer is just
len(tasks). -
Closed form:
max(len(tasks), (f_max - 1) * (n + 1) + c). -
Heap simulation alternative: every cycle of n+1 time, pop the n+1 highest-count tasks, decrement, push back; idle the remainder if heap still non-empty.
If stuck: see solution.py.
p84 — Minimum Number of Arrows to Burst Balloons
Source: LeetCode 452 · Medium · Topics: Greedy, Sorting, Intervals Companies (2024–2025): Amazon (high), Google (medium), Meta (medium) Loop position: Canonical interval-scheduling greedy — sort by END. Tests whether you can articulate why end-sort beats start-sort.
1. Quick Context
n balloons specified as [x_start, x_end]. An arrow shot at coordinate x bursts every balloon containing x. Return the min number of arrows.
Equivalent: Min arrows = max number of pairwise-disjoint intervals (after greedy partitioning into non-overlapping groups; each group bursts with one arrow).
Greedy: Sort by x_end. Track current arrow position (initially the first interval’s end). For each subsequent interval: if start > arrow, need a new arrow; update arrow = end.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/minimum-number-of-arrows-to-burst-balloons/
STOP. 25-min timer. Don’t peek; derive why sort-by-end.
3. Prerequisite Concepts
- Activity selection / interval scheduling.
- Greedy correctness via exchange argument.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Cover all intervals with the minimum count of vertical-line “stabs”.
2. Clarify: Endpoints included? Yes (closed intervals). Empty input → 0.
3. Examples: [[10,16],[2,8],[1,6],[7,12]] → 2 (arrows at x=6 and x=12).
5. Brute: Try all subsets of “arrow positions” = endpoints; check covering. Exponential.
6. Target: O(n log n) sort + O(n) sweep.
7. Pattern: Sort by end; commit minimum-end first; greedy stab at that end.
8. Develop: see solution.py.
9. Test: all overlapping (1 arrow); all disjoint (n arrows); single balloon.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why sort by END, not start
Exchange argument. Suppose an optimal solution uses arrow at position a. Consider the interval with the smallest end that a covers. Moving a to that end (right edge) can only extend coverage rightward — every interval covered by a had start ≤ a ≤ end_min, so moving to end_min keeps those covered (since end_min ≤ end_i) AND may add more. So the greedy “stab at the smallest end” is optimal.
If we sort by start: [[1, 100], [2, 3], [4, 5]] — start-sort puts [1,100] first; greedy might stab at 100, missing [2,3] and [4,5]. Forced to think more carefully. End-sort makes the greedy choice obvious.
6.2 The arrow-position invariant
After sorting by end, walk left-to-right. The current arrow is at the end of the most recent “uncommitted” interval. Each new interval either:
- Starts at or before the arrow: covered by current arrow; skip.
- Starts after the arrow: needs a new arrow; advance arrow to this interval’s end.
6.3 Equivalence to “max disjoint intervals”
Min arrows = number of disjoint intervals you can carve out. Same greedy: sort by end, take intervals whose start > previous_end.
6.4 Edge case: touching endpoints
[[1, 2], [2, 3]]: x=2 bursts both. Greedy: sort by end → process [1,2] first, arrow=2; [2,3] has start=2 ≤ arrow=2 → no new arrow. Count=1. Correct.
6.5 Family: interval-scheduling greedy
- LC 452 Burst Balloons (this; min covers).
- LC 435 Non-overlapping Intervals (min removals to make disjoint).
- LC 56 Merge Intervals (sort by start; merge consecutive).
- LC 253 Meeting Rooms II (heap of end times; min rooms).
Pattern: sort by end for “max disjoint” / “min stab”; sort by start for “merge” / “scheduling overlap counts”.
6.6 Overflow gotcha
Endpoint values can be near INT_MIN/INT_MAX. In Python no overflow; in C++/Java, use long for comparisons or start > arrow (not arrow + 1 < start).
7. Anti-Pattern Analysis
- Sort by start, then greedy — fails on
[[1,100],[2,3],[4,5]]. - Use
arrow + 1 < start— overflow in fixed-width languages. - Try interval-merging then count — works but conceptually fuzzy.
- Track all overlapping intervals at each x — quadratic.
- Heap of intervals by start, popping when arrow position changes — overcomplicated; sweep suffices.
- Forget to handle empty input — return 0.
8. Skills & Takeaways
- Sort-by-end + sweep template.
- Exchange argument for “stab at the smallest end”.
- Equivalence between min-cover and max-disjoint.
9. When to Move On
- Solve cold in 15 min, articulate the exchange argument.
- Justify end-sort vs start-sort with a counterexample.
- Cite Non-overlapping Intervals (LC 435) as a sibling.
10. Company Context
- Amazon: L4/L5; very common phone screen.
- Google: L5; expects exchange-argument justification.
- Meta: E4; calendar-scheduling flavor.
Strong: “Sort by end, sweep; cold; exchange-argument justified; cited Non-overlapping Intervals and Meeting Rooms II as siblings.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Sort key | Start | After test | End cold | + counterexample for start |
| Exchange | None | None | After hint | + cold proof |
| Touching ends | Off-by-one | After test | Cold | + cites closed-interval semantics |
| Family | None | None | LC 435 | + Meeting Rooms II + interval graph |
12. Level Delta
- Mid (L4): Sorts by start, fails; corrects after test; counts correctly.
- Senior (L5): Sorts by end cold; correct on touching ends; cites family.
- Staff (L6): + exchange-argument proof + max-disjoint equivalence + Meeting Rooms II generalization.
- Principal (L7+): + recognizes this as interval graph clique cover (NP-hard in general, polynomial for intervals via this exact greedy); + cites Helly’s theorem on the line (mutual pairwise overlap ⟹ common point).
13. Follow-up Q&A
Q1: Return the actual arrow positions, not just count.
A1: Track them during the sweep — push current end to a list when we commit a new arrow.
Q2: What if intervals are open (exclusive endpoints)?
A2: Change start > arrow to start >= arrow. Touching intervals no longer share an arrow.
Q3: 2D — disks instead of intervals, vertical arrows?
A3: Project to x-intervals (disk i covers [cx-r, cx+r] for any vertical arrow at the projected y); same greedy in x. For general 2D point cover, NP-hard.
Q4: Weighted balloons — each requires an arrow with cost = weight? A4: Min-cost set cover; NP-hard in general. For intervals, LP relaxation is integral; solvable by DP.
Q5: Streaming — balloons arrive online? A5: Lower bound: 2-approximation by stabbing immediately at the right end of the first unstabbed interval seen.
14. Solution Walkthrough
See solution.py:
find_min_arrow_shots— sort by end + sweep; O(n log n).find_min_arrow_shots_brute— try all subsets of endpoints as candidate arrow sets; oracle.
15. Beyond the Problem
Principal code review: “Burst Balloons is the canonical interval-clique-cover problem solved by an exchange argument. The interesting structural fact: on the real line, interval graphs are perfect — chromatic number equals clique number, max clique can be found in O(n log n) by sweep, and clique cover (this problem) equals max independent set on the complement (max disjoint intervals). This is not true in general graphs (clique cover is NP-hard). The Staff+ insight: when a problem reduces to a graph optimization, ask ‘is the graph in a structured class — interval, chordal, planar, bipartite?’ If yes, problems that are NP-hard in general become polynomial. The exchange argument here is the constructive witness of intervals’ perfection. The follow-up: the same problem on a circle (circular-arc graphs) is already harder — chromatic number is NP-hard in general, though the clique-cover version remains polynomial. Topology matters.”
Hints — p84 Min Arrows to Burst Balloons
-
Each arrow bursts a contiguous “cluster” of overlapping balloons. Goal: partition into min clusters.
-
Sort by END (not start). Counterexample to start-sort:
[[1,100],[2,3],[4,5]]— start-sort puts the wide one first and misleads the greedy. -
Walk left to right. Maintain
arrow= end of the most recently committed interval. For each new[s, e]: ifs > arrow, commit a new arrow ate. -
Exchange argument: stabbing at the smallest end can only cover more later intervals (since later starts are ≥ this start in some optimal ordering).
-
Touching endpoints (closed intervals):
[[1,2],[2,3]]share x=2 → 1 arrow. Use strict>not>=.
If stuck: see solution.py.
p85 — Split Array Largest Sum
Source: LeetCode 410 · Hard · Topics: Binary Search on Answer, DP, Greedy Companies (2024–2025): Google (high), Amazon (high), Meta (medium), Stripe (medium) Loop position: The DP↔binary-search bijection. Two complete solutions; choosing correctly given constraints is the Staff signal.
1. Quick Context
Given non-negative nums and integer k, split nums into k contiguous subarrays so that the maximum subarray sum is minimized. Return that minimum.
Two solutions:
- DP
dp[i][j]= min largest-sum of splittingnums[:i]intojparts. O(n² · k). - Binary search on answer: search
S ∈ [max(nums), sum(nums)]; feasibility check = “can we partition into ≤ k parts each with sum ≤ S?” Greedy left-to-right. O(n log(sum)).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/split-array-largest-sum/
STOP. 40-min timer. Try DP first to understand the problem; then derive binary search on answer.
3. Prerequisite Concepts
- Binary search on a monotone predicate (parametric search).
- 1D DP with index split.
- Monotonicity of “feasible to fit in ≤ k parts” as a function of cap S.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Partition into k contiguous parts; minimize the maximum part-sum.
2. Clarify: Contiguous (yes); non-negative (yes); k ≤ n.
3. Examples: nums=[7,2,5,10,8], k=2 → 18 (split [7,2,5,10] | [8] would be max 24; [7,2,5] | [10,8] is max 18).
5. Brute: Try all C(n-1, k-1) split positions; track min of max. Combinatorial.
6. Target: O(n log(sum(nums))) binary search.
7. Pattern: Binary search on answer with greedy feasibility.
8. Develop: see solution.py.
9. Test: k=1 (= sum); k=n (= max); all zeros; single huge element.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why binary search works (monotonicity)
Predicate feasible(S) = “can we partition into ≤ k parts with each sum ≤ S”. This is monotonic in S: if S works, any S’ ≥ S works. The minimum S where feasible(S) = True is the answer.
6.2 Search bounds
- Lower bound
lo = max(nums): any part must contain at least one element, so part-sum ≥ max(nums). - Upper bound
hi = sum(nums): a single part trivially fits.
6.3 The greedy feasibility check
Greedy from left: start a new part; accumulate while sum ≤ S; when adding next element would exceed S, close part and start new. Count parts. Feasible iff count ≤ k.
Why greedy is optimal for feasibility: if you close a part earlier than needed, the suffix has more to fit → never helps. (Standard exchange argument for “first-fit on a line”.)
6.4 DP formulation
dp[i][j] = min over split point t < i of: max(dp[t][j-1], sum(nums[t:i]))
base: dp[i][1] = prefix_sum[i]
O(n² · k) with prefix sums. Reconstructing actual splits: track argmin t.
6.5 DP vs binary search: when to pick which
- Binary search when nums are large (sum dominates k), n moderate, only the answer matters. O(n log(sum)).
- DP when you need the actual partition or k is tiny. O(n²·k).
- Practical: at the FAANG bar, binary search is the expected answer; DP is the warmup.
6.6 Family: binary-search-on-answer
- LC 410 Split Array Largest Sum (this).
- LC 1011 Capacity to Ship Packages in D Days (identical structure).
- LC 875 Koko Eating Bananas (per-pile feasibility).
- LC 1482 Min Days to Make m Bouquets (per-day feasibility).
- LC 1283 Smallest Divisor (per-element ceil-division feasibility).
All share: search a numeric parameter; greedy feasibility check.
7. Anti-Pattern Analysis
- Search bounds wrong —
lo = 0letsfeasible(0)be tested unnecessarily;lo = max(nums)is tight. - Off-by-one in binary search — use
lo < hiwithhi = midon success,lo = mid + 1on failure; final answer =lo. - Greedy that splits “after exceeding” — must split before exceeding (close current part, start new with this element).
- DP without prefix sums — recomputes sums; O(n³·k).
- Confuse “≤ k parts” with “= k parts” — feasibility uses ≤ k (fewer parts = larger max, won’t help); but problem requires exactly k, which is fine since any ≤k-feasible can be “split more finely” → still ≤ S.
- Try the DP first in a 30-min interview — runs out of time.
8. Skills & Takeaways
- Binary search on numeric answer with monotone predicate.
- Greedy first-fit feasibility check.
- Recognizing when to prefer binary search over DP.
9. When to Move On
- Solve both ways cold in 30 min.
- Justify the monotonic predicate.
- Cite Capacity to Ship Packages and Koko as siblings.
10. Company Context
- Google: L5/L6; binary-search-on-answer is a fixture.
- Amazon: L5/L6; load-balancing flavor.
- Meta: E5; expects you to identify the pattern fast.
- Stripe: mid+; rate-limiting / capacity problems.
Strong: “Two solutions: O(n log(sum)) binary search on answer with greedy feasibility, and O(n²·k) DP; chose binary search for the constraints; cited the binary-search-on-answer family.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Monotone predicate | None | After hint | Cold | + proves monotonicity formally |
| Bounds | [0, sum] | [max, sum] after test | [max, sum] cold | + tight reasoning |
| Greedy feasibility | Off-by-one | After test | Cold | + proves exchange-argument optimality |
| Family | None | None | LC 1011 | + LC 875/1482/1283 + DP↔BSA tradeoff |
12. Level Delta
- Mid (L4): DP after struggle; binary search after hint.
- Senior (L5): Both solutions cold; correct bounds and greedy.
- Staff (L6): + articulates DP↔BSA tradeoff + reconstructs split.
- Principal (L7+): + parametric-search framework (Megiddo) + connects to scheduling / load balancing LP relaxations + recognizes this as a discrete analog of the continuous max-min partition problem (solvable in O(n) via prefix-sum balancing).
13. Follow-up Q&A
Q1: Return the actual partition? A1: After finding S via binary search, run greedy once more and record cut points.
Q2: Negative numbers allowed? A2: Monotonicity of greedy breaks (adding a negative could let you fit more). Use DP.
Q3: Maximize the minimum part-sum instead? A3: Symmetric binary search; greedy fits “≥ S” parts, count ≥ k means feasible.
Q4: Parts don’t have to be contiguous? A4: NP-hard (multiway number partitioning); LP relaxation or approximations.
Q5: 2D — grid into k rectangles? A5: Becomes NP-hard for general k; the 1D contiguity is what makes it tractable.
14. Solution Walkthrough
See solution.py:
split_array_bsa— O(n log(sum)) binary search on answer with greedy feasibility.split_array_dp— O(n²·k) DP.split_array_brute— try all combinations of split points; oracle.
15. Beyond the Problem
Principal code review: “Split Array Largest Sum is the prototype for the binary-search-on-answer pattern, which dominates real-world capacity-planning problems: ship-loading deadlines, autoscaler thresholds, rate-limit configuration, batch-size tuning. The Staff+ recognition is: whenever you need to find the smallest (or largest) numeric value X such that some greedy predicate P(X) holds, and P is monotonic in X, prefer binary search. The cost is
log(range)predicate evaluations vs DP’s polynomial in input size — usually a huge win. The deeper move: parametric search (Megiddo) lets you do this even when the bounds aren’t obvious, by running the greedy symbolically on an unknown X and resolving comparisons via a smaller subproblem. Most engineers never reach for this even when it’s perfect — they default to DP because that’s what was drilled. The exam: when you see ‘minimize the maximum of…’ or ‘maximize the minimum of…’, binary-search-on-answer should be your first move.”
Hints — p85 Split Array Largest Sum
-
Reframe: instead of searching over partitions, search over the answer itself — the max part-sum S.
-
Predicate
feasible(S)= “can we split into ≤ k contiguous parts, each with sum ≤ S?” This is monotone in S: feasible at S ⟹ feasible at S+1. -
Binary-search S in
[max(nums), sum(nums)]. Lower boundmax(nums)because any single part containing the max needs ≥ max. -
Greedy feasibility check (O(n)): accumulate left-to-right; when adding the next element would exceed S, start a new part. Count parts; feasible iff ≤ k.
-
DP alternative:
dp[i][j] = min over t of max(dp[t][j-1], prefix[i]-prefix[t]). O(n²·k). Use prefix sums.
If stuck: see solution.py.
Week 17 — Advanced Greedy & Scheduling
Theme: Heap-driven greedy. Every problem this week reduces to “at each decision point, the heap tells you the right move”. The Staff signal is recognizing the two-heap or sort-then-heap pattern and proving each greedy choice optimal via exchange.
Daily schedule
| Day | Problem | LC # | Difficulty | Key archetype |
|---|---|---|---|---|
| Mon | p86 — Meeting Rooms II | 253 | Medium | Min-heap of end times (or sweep line on events) |
| Tue | p87 — Reorganize String | 767 | Medium | Max-heap of frequencies with cooldown buffer |
| Wed | p88 — IPO | 502 | Hard | Two heaps: min on capital + max on profit — affordability gate |
| Thu | p89 — Course Schedule III | 630 | Hard | Sort by deadline + max-heap swap — counterintuitive exchange |
| Fri | p90 — Single-Threaded CPU | 1834 | Medium-Hard | Sort by enqueue + min-heap of (proc_time, idx) event loop |
Readiness gate (Friday)
- For Meeting Rooms II, derive both the min-heap solution AND the sweep-line solution; explain which is preferable when.
- For Reorganize String, articulate the feasibility condition
max_freq ≤ (n+1)/2and prove it tight. - For IPO, justify why lazy admission (only add affordable projects to max-heap as capital grows) beats sorting.
- For Course Schedule III, state and prove the swap-out lemma: if a course can’t fit, swap with the heap’s longest-duration accepted course iff it’s longer.
- For Single-Threaded CPU, articulate the two-phase event loop: sort by enqueue, advance time when idle, pop heap by (proc_time, idx).
Why these together
These five problems share a structural backbone: scan items in some natural order; at each step, the heap maintains the “best so far” view. The greedy correctness in each comes from a different exchange argument:
- Meeting Rooms II — exchange between rooms is free; only count matters.
- Reorganize String — letters at the top of the heap are the constraint; cooldown buffer prevents same-letter adjacency.
- IPO — affordability is monotone in capital; lazy admission preserves optimality.
- Course Schedule III — counterintuitive: replace a longer accepted course with a shorter incoming one to free up future slack.
- Single-Threaded CPU — the heap simulates the OS scheduler; ordering by (proc_time, idx) is the tie-break rule.
By Friday, the candidate should be able to classify a scheduling problem by “what does the heap hold?” within 60 seconds.
p86 — Meeting Rooms II
Source: LeetCode 253 · Medium · Topics: Heap, Sweep Line, Greedy Companies (2024–2025): Amazon (very high), Google (high), Meta (high), Microsoft (medium), Bloomberg (medium) Loop position: The canonical “two ways to solve it” problem. Min-heap of end times OR sweep line on events. Knowing both — and when to use each — separates Mid from Senior.
1. Quick Context
Given meeting intervals [start, end), return the minimum number of rooms required so no two simultaneous meetings share a room.
Two solutions:
- Min-heap of end times: Sort by start. For each meeting, if the earliest-ending room is free (heap top ≤ current start), reuse it; else allocate a new room. Push current end. Answer = heap size.
- Sweep line: Sort starts and ends separately. Walk two pointers; on
start ≤ end, rooms++; else rooms–. Max value = answer. (Or sort all events; +1 for start, -1 for end.)
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/meeting-rooms-ii/
STOP. 25-min timer. Try both approaches; compare.
3. Prerequisite Concepts
- Min-heap (
heapq) for “earliest end” lookup. - Sweep-line event aggregation.
- Half-open intervals
[start, end).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Min number of rooms to host all meetings simultaneously.
2. Clarify: Open intervals? [start, end) means end == next_start shares no overlap → can reuse room.
3. Examples: [[0,30],[5,10],[15,20]] → 2.
5. Brute: For each meeting, scan all others to find overlap; build conflict graph; chromatic number. NP-hard in general but polynomial for interval graphs.
6. Target: O(n log n).
7. Pattern: Min-heap of end times OR sweep line.
8. Develop: see solution.py.
9. Test: all overlapping (n rooms); none overlapping (1); touching (end == start reuses); empty input.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Min-heap solution — why it’s correct
Sort by start. When meeting i begins, the min-heap holds end times of currently-in-use rooms. If heap_min ≤ start_i, that room is free — reuse it (pop, push new end). Else allocate new room (push new end).
Exchange argument: any policy uses at least max overlap rooms. The heap-greedy never uses more, because it only opens a new room when every existing room is still busy.
6.2 Sweep-line solution — the events view
Aggregate +1 at every start, -1 at every end. Sort. Sweep and track running sum; max sum = answer.
Tie-break: when start time == end time, process end first (because [start, end) means the meeting ended before the new one starts; they share no overlap). Sort key: (time, +1 for start | -1 for end) — end comes first since -1 < +1.
6.3 Heap vs sweep: which is preferable?
- Heap if you need the actual room assignment per meeting.
- Sweep if you only need the count (slightly faster, simpler).
- Sweep wins on streaming pre-sorted events.
- Heap wins when meetings have features (priorities) and you must pick which to evict.
6.4 The (time, kind) tie-break trap
In sweep line, on tied times, end events must come before start events for half-open intervals [start, end). The natural sort (time, +1) puts ends first because -1 < +1 — but only if you encode ends as -1 and starts as +1. The opposite encoding requires explicit care.
6.5 Equivalent: max overlap = chromatic number of interval graph
This is the famous result that interval graphs are perfect — chromatic number equals max clique = max overlapping intervals at any instant. The greedy here is the constructive proof.
6.6 Family: heap-of-times / sweep-line
- LC 253 Meeting Rooms II (this).
- LC 1094 Car Pooling (sweep + cumulative passengers).
- LC 731/732 Calendar II/III (segment tree or sweep line).
- LC 1851 Minimum Interval to Include Each Query (sort + heap).
- LC 218 The Skyline Problem (sweep + heap with lazy deletion).
7. Anti-Pattern Analysis
- Use closed intervals —
[0,30]and[30,60]then “overlap”; produces 2 rooms instead of 1. - Sweep line tie-break wrong — sort starts and ends with
(time, +1/-1)but flip the sign; produces n+1 instead of n. - Heap not sorted by start — pop arbitrary end times; never matches the time progression.
- O(n²) overlap counting — works but signals you missed the pattern.
- Build a Gantt-chart 2D bitmap — works for small inputs only; not interview-worthy.
- Forget that touching intervals reuse the room — off-by-one on the comparison (
<vs≤).
8. Skills & Takeaways
- Min-heap of “what’s currently in use”.
- Sweep line for event aggregation.
- Tie-break rules for half-open intervals.
9. When to Move On
- Solve both ways cold in 15 min.
- Articulate which is preferable when.
- Cite LC 1094 Car Pooling as the +1/-1 sibling.
10. Company Context
- Amazon: L4/L5; one of the most common phone screens of the decade.
- Google: L5; expects both solutions.
- Meta: E4/E5; standard.
- Microsoft: L4; calendar-flavored.
Strong: “Two solutions: min-heap of end times O(n log n) and sweep-line O(n log n); chose heap when room assignment needed, sweep otherwise; cited Car Pooling and Skyline as siblings.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Heap soln | After hint | Cold | Cold + clean | + per-room assignment |
| Sweep soln | None | After hint | Cold | + cites interval graph perfection |
| Tie-break | Wrong | After test | Cold | + [start,end) justification |
| Family | None | None | LC 1094 | + Skyline lazy deletion |
12. Level Delta
- Mid (L4): Heap solution with guidance; sweep after hint.
- Senior (L5): Both cold; correct tie-break; cites family.
- Staff (L6): + cites interval-graph perfection; + Skyline generalization.
- Principal (L7+): + recognizes this as graph coloring on interval graphs (polynomial) vs general chromatic-number (NP-hard) + cites real-world scheduling LP: integrality gap is 1 for interval graphs.
13. Follow-up Q&A
Q1: Return the actual room assignment.
A1: Heap holds (end, room_id). On reuse, the popped room_id is assigned to the new meeting; else allocate next room_id.
Q2: Rooms have different capacities; meetings have attendee counts. A2: Becomes bin-packing flavored; NP-hard for arbitrary capacities. Approximations: first-fit decreasing.
Q3: Meetings have priorities; if not enough rooms, drop lowest priority. A3: Max-heap of accepted meetings by priority; on conflict, replace if new priority > min in heap. Similar to Course Schedule III.
Q4: Streaming meetings (online). A4: Maintain heap of end times; per arrival O(log n).
Q5: What’s the minimum number of rooms if meetings can be rescheduled within [start, deadline]?
A5: Becomes a scheduling problem with windows; LP / max-flow.
14. Solution Walkthrough
See solution.py:
min_meeting_rooms_heap— min-heap of end times.min_meeting_rooms_sweep— sweep line on events.min_meeting_rooms_brute— count max overlap at every start time.
15. Beyond the Problem
Principal code review: “Meeting Rooms II is the graph-coloring problem in disguise — and the only reason it’s polynomial is that the underlying conflict graph is an interval graph, which is perfect. The two solutions are dual: the heap explicitly builds the coloring (each room is a color); the sweep computes the max clique (max simultaneous overlap), and perfection of interval graphs guarantees these are equal. At Staff+, recognize this is a special case of a deep theorem: graph coloring is NP-hard in general, polynomial for perfect graphs, and interval graphs are the canonical perfect graph. The lesson scales: when you face a ‘min resources for tasks with conflicts’ problem in production (CI scheduling, GPU allocation, DB connection pooling), if you can show the conflict graph is interval (or even chordal), you get a polynomial-time optimal scheduler for free. If not, you’re in NP-hard territory and your best bet is LP relaxation or rounding heuristics.”
Hints — p86 Meeting Rooms II
-
Half-open intervals
[start, end): meeting ending at 5 and meeting starting at 5 share no overlap and can reuse the same room. -
Heap approach: sort by start time. Min-heap holds end times of in-use rooms. For each meeting, if
heap_min <= start, reuse (replace top); else push (new room). -
Sweep approach: create +1 events at starts, -1 at ends. Sort. Cumulative max = answer. At ties, end (-1) comes before start (+1) — natural sort works since -1 < +1.
-
Brute (oracle): at every meeting’s start, count how many other meetings are active. Max over starts.
-
Answer = max overlap at any instant = chromatic number of the interval conflict graph (a perfect graph, so polynomial).
If stuck: see solution.py.
p87 — Reorganize String
Source: LeetCode 767 · Medium · Topics: Heap, Greedy, Counting Companies (2024–2025): Amazon (high), Meta (medium), Google (medium), TikTok (medium) Loop position: Pigeonhole feasibility + max-heap with cooldown buffer. Tests whether you derive the tight feasibility condition before coding.
1. Quick Context
Rearrange string s so no two adjacent chars are equal. Return any valid rearrangement, or "" if impossible.
Feasibility: Possible iff max_freq ≤ (n + 1) // 2. Proof: if some letter appears > (n+1)/2 times, by pigeonhole two of them must be adjacent (only ≤ (n-1)/2 + 1 = (n+1)/2 slots non-adjacent).
Construction (max-heap + 1-step cooldown):
- Push (-count, char) for each letter.
- Pop top; append char; decrement count; buffer it (don’t push back yet).
- On next iteration, pop top (different letter, guaranteed because buffered letter was the most frequent), append; push the buffered one back if count > 0; buffer current.
Alternative: place into even indices first (0, 2, 4, …) then odd indices, starting with the most frequent letter.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/reorganize-string/
STOP. 30-min timer. Derive the feasibility condition before coding.
3. Prerequisite Concepts
- Max-heap via negation in Python’s
heapq. collections.Counter.- Pigeonhole reasoning.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Rearrange so no two adjacent chars are equal.
2. Clarify: Lowercase only? Usually yes. Empty string?
3. Examples: "aab" → "aba". "aaab" → "".
5. Brute: Permute n! and check.
6. Target: O(n log k) where k = distinct letters ≤ 26.
7. Pattern: Max-heap with cooldown buffer.
8. Develop: see solution.py.
9. Test: all distinct; majority letter; exact feasibility threshold; n=1.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The pigeonhole feasibility bound — tight
Letters must occupy alternating slots. n positions can hold at most ⌈n/2⌉ = (n+1)//2 copies of any one letter (positions 0, 2, 4, …). If max_freq > (n+1)//2, infeasible.
6.2 Why the cooldown buffer works
After placing letter L, never pick L again until we’ve placed at least one other letter. The heap holds all available letters; the buffered letter is unavailable for this turn. After the next placement, push buffer back.
Correctness: as long as feasibility holds, at every turn there exists ≥ 2 distinct letters with count > 0 or exactly one letter left with count = 1. In the first case the heap has ≥ 2 entries and we pick the freshest; in the second case the buffer is empty.
6.3 Even-index construction — simpler proof
Sort letters by frequency descending. Fill positions 0, 2, 4, ... with copies of the most frequent letter; continue with next-most-frequent. After filling all even positions, fill 1, 3, 5, ....
Why it works: the most frequent letter goes to even positions only — no two of it are adjacent. Other letters’ counts are smaller, so they can’t form adjacency either (by pigeonhole on smaller frequency).
6.4 Comparison: heap vs even-index
- Heap: more flexible (handles weighted variants, k-distance generalization LC 358).
- Even-index: simpler, O(n + k log k).
- Production: even-index for this exact problem; heap for the k-distance generalization.
6.5 Generalization: LC 358 Rearrange String k Distance Apart
Same problem but require ≥ k distance between same letters. Cooldown buffer becomes a queue of size k (FIFO); push back into heap after k turns.
6.6 Family: cooldown / spacing problems
- LC 767 Reorganize String (this; k=2).
- LC 358 Rearrange String k Distance Apart (general k).
- LC 621 Task Scheduler (cooldown by time, not adjacency).
- LC 1054 Distant Barcodes (same as 767).
7. Anti-Pattern Analysis
- Use min-heap by frequency — places sparsest letters first; fails on
aab. - Forget the feasibility check — heap algorithm “succeeds” but produces an invalid string.
- Push buffered letter back too early — adjacent placements of same letter.
- Greedy without proof — works but unconvincing.
- Sort string and zip with itself — only works in very narrow cases.
- Use Counter.most_common() once — counts change as you place; must reuse heap.
8. Skills & Takeaways
- Pigeonhole-derived feasibility.
- Max-heap with explicit cooldown buffer.
- Even-index construction as an alternative.
9. When to Move On
- Solve cold in 20 min using either approach.
- Derive
max_freq ≤ (n+1)//2. - Cite k-distance generalization.
10. Company Context
- Amazon: L5; common screen.
- Meta: E5; heap pattern signal.
- Google: L5; pigeonhole derivation expected.
- TikTok: mid; standard heap-greedy.
Strong: “Heap with cooldown buffer OR even-index construction; pigeonhole feasibility max_freq ≤ (n+1)//2 derived; cited LC 358 k-distance generalization.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Feasibility | None | After test | Cold | + tight pigeonhole proof |
| Heap+cooldown | Buggy | Works | Clean | + k-distance generalization |
| Even-index | None | None | After hint | + correctness proof |
| Family | None | None | LC 358 | + LC 621 connection + LP framing |
12. Level Delta
- Mid (L4): Heap greedy buggy on cooldown; corrects with hint.
- Senior (L5): Both heap and even-index; feasibility derivation.
- Staff (L6): + generalizes to k-distance (LC 358).
- Principal (L7+): + framing as chromatic number of distance-1 conflict path graph + LP relaxation for k-distance + connection to broadcast scheduling.
13. Follow-up Q&A
Q1: Return lexicographically smallest valid string? A1: Heap by (count desc, char asc) breaks pure correctness; the standard greedy doesn’t give lex-smallest. Use DP over (position, last_char, remaining_counts) or backtracking.
Q2: k-distance apart (LC 358)? A2: Cooldown queue size k. Each placement, pop oldest from cooldown (push back to heap if count > 0), then pop heap top.
Q3: Streaming letters arrive online; output as they arrive? A3: Maintain heap; emit whenever the heap top differs from the last emitted char. May need to buffer.
Q4: Multiple constraints (no adjacent + balanced subsequence)? A4: DP over state; typically NP-hard for arbitrary regex-like constraints.
Q5: What if counts include weights (a different letter is “twice as bad”)? A5: Weighted greedy; max-heap by weight × count.
14. Solution Walkthrough
See solution.py:
reorganize_string_heap— max-heap with cooldown buffer.reorganize_string_even_index— fill even then odd indices.reorganize_string_brute— permutation oracle.
15. Beyond the Problem
Principal code review: “Reorganize String is the smallest version of a broadcast / interleaving scheduling problem that appears in WiFi MAC layer design (avoid same-station back-to-back transmissions), CPU branch prediction (avoid same-target speculation chains), and audio/video frame interleaving (avoid temporal clustering of same-codec frames). The Staff+ recognition: the pigeonhole bound
max_freq ≤ ⌈n/2⌉generalizes to any periodic schedule with period k: feasibility becomesmax_freq ≤ ⌈n/k⌉. This is the load-bound in scheduling theory — a hard combinatorial constraint that no algorithm can violate. The interesting follow-up is when some letters can be relaxed (low-priority interleaved with high-priority) and the bound becomes a weighted load with LP duality. The lesson: whenever you face a ‘no two X adjacent’ problem, your first move is to compute the pigeonhole bound and check feasibility. If infeasible, the problem is unsolvable; if feasible, a constructive algorithm exists — heap, even-index, or for harder variants, LP rounding.”
Hints — p87 Reorganize String
-
Feasibility check first:
max_freq <= (n + 1) // 2(pigeonhole). If violated, return"". -
Heap approach: max-heap of (-count, char). Place top; buffer it for one turn so it can’t be picked again immediately. Next turn, pop a different letter, then push the buffered one back.
-
Even-index approach: sort letters by frequency descending. Fill positions 0, 2, 4, … with copies of the most frequent letter; continue with next-most-frequent; wrap to odd positions when even positions exhausted.
-
Both achieve O(n log k) or O(n + k log k). Even-index is simpler; heap generalizes to k-distance (LC 358).
-
Validation: every adjacent pair must differ; total counts must match input.
If stuck: see solution.py.
p88 — IPO
Source: LeetCode 502 · Hard · Topics: Heap, Greedy Companies (2024–2025): Goldman Sachs (high), Bloomberg (high), Amazon (medium), Google (medium) Loop position: Two heaps coordinated by a gating condition. Min-heap-by-capital gates affordability; max-heap-by-profit picks the best affordable. Canonical pattern for “filter by one key, optimize by another”.
1. Quick Context
Given initial capital w, do at most k projects. Each project i requires capital[i] to start and yields profits[i] (added to your capital). Maximize final capital.
Solution: Two-heap pattern.
- Min-heap on capital: all unaffordable projects (with current
w), keyed by required capital. - Max-heap on profit: all currently-affordable projects, keyed by profit.
Loop k times: drain min-heap into max-heap while top capital ≤ w; if max-heap empty break (nothing affordable); pop max profit and add to w.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/ipo/
STOP. 30-min timer.
3. Prerequisite Concepts
- Two-heap pattern (min for one key, max for another).
- Greedy with “promotion” between data structures as state changes.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: k rounds; each round, pick max-profit affordable project; add profit to capital.
2. Clarify: Same project picked twice? No, one-time. Can w stay zero? Yes.
3. Examples: k=2, w=0, profits=[1,2,3], capital=[0,1,1] → w=0+1=1; now both [2,3] affordable → +3 → 4.
5. Brute: Try all combinations of size ≤ k respecting capital order; O(C(n,k)·k).
6. Target: O((n + k) log n).
7. Pattern: Two heaps.
8. Develop: see solution.py.
9. Test: k > # projects; w=0 and all need capital; one project blocks others.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why max-profit-affordable is greedy-optimal
Exchange argument: suppose OPT picks project P at round i but not max-profit-affordable Q at that round. Q yields ≥ P profit, and Q is affordable at round i. Swap P ← Q: Q’s higher profit means future affordability is at least as good (more capital). So Q dominates P. Repeat ⇒ greedy = OPT.
6.2 The two-heap coordination
- Min-heap-by-capital = “waiting room” for projects whose capital requirement hasn’t been met.
- Max-heap-by-profit = “available pool” of projects ready to execute.
- Promotion: each round, drain min-heap → max-heap while top capital ≤ w.
Each project moves at most once between heaps ⇒ amortized O(log n) per project.
6.3 Why a single sort doesn’t work
You can sort by capital ascending and use a max-heap on profit alone (no min-heap needed). Walk a pointer through sorted projects; push profits while capital[ptr] ≤ w. This is equivalent and slightly simpler.
projects = sorted(zip(capital, profits))
ptr = 0
max_heap = []
for _ in range(k):
while ptr < n and projects[ptr][0] <= w:
heapq.heappush(max_heap, -projects[ptr][1])
ptr += 1
if not max_heap:
break
w += -heapq.heappop(max_heap)
Both are O((n + k) log n); the pointer version saves the min-heap overhead.
6.4 If k = ∞ — what changes?
Pick every project with profit > 0 (in some feasible order). The two-heap pattern still works — just keep looping until both empty. No new optimization needed.
6.5 Family: two-heap / gated greedy
- LC 502 IPO (this).
- LC 295 Find Median from Data Stream (max + min for two halves).
- LC 480 Sliding Window Median.
- LC 1942 Number of Smallest Unoccupied Chair (free chairs + busy ends).
- LC 1834 Single-Threaded CPU (waiting + ready).
7. Anti-Pattern Analysis
- Pick min-capital affordable — wastes turns on low-profit projects.
- Pick max-profit overall (ignoring capital) — picks unaffordable; no progress.
- Re-sort projects each round — O(k n log n) instead of O((n+k) log n).
- Forget early termination when nothing is affordable — wastes loop iterations (harmless but signals weak code).
- Use min-heap-by-profit — same wrong direction as anti-pattern 1.
8. Skills & Takeaways
- Two-heap pattern (or sort + single heap).
- Greedy exchange argument for round-by-round optimization.
- Affordability gating.
9. When to Move On
- Solve cold in 20 min with two-heap or sort-then-heap.
- Articulate exchange argument.
- Cite LC 1834 as a sibling.
10. Company Context
- Goldman Sachs: finance-themed; loves this problem.
- Bloomberg: very common.
- Amazon: L5/L6.
- Google: L5; expects exchange argument.
Strong: “Two-heap or sort + max-heap; gate by affordability; greedy exchange argument for max-profit-affordable optimality; cited LC 1834 CPU as sibling.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Two-heap | Buggy | After hint | Cold | + sort + single heap equivalence |
| Exchange | None | None | Verbal | + formal write-up |
| Family | None | None | LC 1834 | + LC 1942 chairs |
| Edge | k > n | After test | Cold | + nothing affordable proof |
12. Level Delta
- Mid (L4): Two-heap with guidance; misses early termination.
- Senior (L5): Cold; exchange argument verbal.
- Staff (L6): + sort + single heap equivalence + family.
- Principal (L7+): + framing as online matroid-intersection / activity selection on weighted matroid + LP duality for fractional relaxation.
13. Follow-up Q&A
Q1: Each project takes time; you have a time budget instead of k rounds. A1: Becomes knapsack-flavored; NP-hard. Approximations: dynamic programming on (time, capital) state if discretized.
Q2: Projects expire (deadline). A2: Like Course Schedule III: sort by deadline, swap-out lemma.
Q3: Maximize expected profit when profits are random. A3: Greedy on E[profit] if independent; otherwise stochastic DP.
Q4: k = ∞. A4: Pick every project with profit > 0; loop until both heaps drain or no affordable.
Q5: Negative profits allowed. A5: Skip them — never optimal to take a negative profit if not forced.
14. Solution Walkthrough
See solution.py:
find_maximized_capital_two_heap— explicit min + max heap.find_maximized_capital_sort— sort by capital + single max heap (simpler, same asymptotic).find_maximized_capital_brute— all subsets ≤ k.
15. Beyond the Problem
Principal code review: “IPO is the smallest online matroid intersection problem you can write down: at each step you pick the max-weight element subject to a feasibility constraint that changes with your state (here, capital). Two-heap is the standard implementation, but the deeper observation is that affordability is monotone in capital — once you can afford a project, you can always afford it later (capital never decreases) — so the ‘promotion’ direction is one-way. This monotonicity is what makes the greedy optimal: it’s the analog of matroid-greedy-optimality but lifted to an online setting where the matroid evolves. The Staff+ recognition: when you see ‘pick k things from a pool gated by current state, where state grows monotonically’, you almost always have a two-heap (or sort + heap) solution with a clean exchange-argument proof. Real-world analogs: ad-bidding budget allocation, GPU scheduling with VRAM constraints, CI test selection under cumulative-time budget. The lesson scales: monotone state + max-value-feasible greedy = optimal, and the implementation is always a coordinated heap pair.”
Hints — p88 IPO
-
Greedy: at every round, pick the max-profit affordable project. Exchange-argument proves optimality.
-
Two heaps: min-heap-by-capital = waiting room of unaffordable projects; max-heap-by-profit = pool of affordable projects.
-
Each round: drain min-heap into max-heap while top capital ≤ current w. Pop max-profit; add to w. Break if max-heap empty (nothing affordable).
-
Alternative (simpler): sort projects by capital ascending. Walk a pointer through sorted list; push profit to max-heap whenever capital[ptr] ≤ w.
-
Both run in O((n + k) log n). Each project is promoted at most once.
If stuck: see solution.py.
p89 — Course Schedule III
Source: LeetCode 630 · Hard · Topics: Heap, Greedy, Exchange Argument Companies (2024–2025): Google (high), Amazon (medium), Meta (medium) Loop position: The swap-out lemma. Sort by deadline; max-heap of accepted durations; if a new course fits, accept; else if a longer one is already accepted, swap. Proves dramatic mastery of greedy exchange.
1. Quick Context
Given courses [duration, lastDay], you can take at most one course at a time. A course must finish by its lastDay. Maximize the number of courses you can take.
Solution (deadline-sorted swap-out greedy):
- Sort courses by
lastDayascending. - Max-heap of accepted durations; track
cur_time. - For each
(d, dl):- If
cur_time + d <= dl: accept (push d, cur_time += d). - Else if heap and
heap_max > d: swap — replace heap_max with d,cur_time -= (heap_max - d). - Else skip.
- If
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/course-schedule-iii/
STOP. 35-min timer.
3. Prerequisite Concepts
- Greedy with rollback / swap (not just monotone accept).
- Deadline-sorted scheduling (Moore-Hodgson, EDF).
- Max-heap for “longest committed”.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Max number of courses; each must finish by its deadline; serial execution.
2. Clarify: Can start at time 0? Yes. All distinct deadlines? No.
3. Examples: [[100,200],[200,1300],[1000,1250],[2000,3200]] → 3.
5. Brute: Sort by deadline; try every subset; check feasibility. Exponential.
6. Target: O(n log n).
7. Pattern: Moore-Hodgson swap-out.
8. Develop: see solution.py.
9. Test: all fit; none fit; chain that requires swap.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why sort by deadline?
Exchange argument: suppose OPT has any two courses i, j with dl_i < dl_j but i scheduled after j. Swap them: still feasible (j fits in i’s old slot iff i fit in j’s old slot, both ≤ dl_j). After enough swaps, OPT respects deadline order. So WLOG, schedule in deadline-sorted order.
6.2 The swap-out lemma
Claim: when considering course c with cur_time + d_c > dl_c, if heap_max > d_c, swapping gives same count but smaller cur_time, which never hurts future courses.
Proof:
- Same count: heap size unchanged.
- Smaller cur_time:
cur_time' = cur_time - (heap_max - d_c) < cur_time. - Feasibility of past commitments: all past courses had deadlines ≤ dl_c (sorted); their feasibility was based on cur_time at the time they were scheduled, not now. Removing the max and adding d_c only reduces cur_time ⇒ all past feasibility still holds.
6.3 Why max-heap (longest), not min-heap?
We swap out the longest to free maximum time. Swapping out the shortest gains nothing.
6.4 Moore-Hodgson algorithm — classical reference
This is exactly Moore’s algorithm (1968) for “minimize number of late jobs on a single machine, all weights = 1”. The optimal complement: max number of on-time jobs = total - min late.
6.5 What if weights matter (weighted version)?
Becomes weighted Moore-Hodgson — NP-hard in general. DP on (time, course index) for small inputs.
6.6 Family: deadline-sorted swap-out
- LC 630 Course Schedule III (this).
- LC 1834 Single-Threaded CPU (different: minimizing total wait, not max count).
- LC 502 IPO (different: gating by capital, not deadline).
- LC 1353 Maximum Number of Events That Can Be Attended (similar deadline pattern).
7. Anti-Pattern Analysis
- Sort by duration — fails; deadline order is essential.
- Min-heap of durations — swap-out lemma breaks; no gain.
- No swap; just skip if doesn’t fit — misses swaps that would yield same count + lower cur_time.
- Reset cur_time per swap-out incorrectly — must subtract
(heap_max - d_new), not heap_max alone. - Sort by
lastDaydescending — backward; misses Moore-Hodgson structure. - DP on (time, count) — exponential; works on tiny inputs but signals you missed greedy.
8. Skills & Takeaways
- Moore-Hodgson swap-out greedy.
- Deadline-ordered scheduling with rollback.
- Max-heap for “what to evict”.
9. When to Move On
- Solve cold in 25 min.
- State and prove swap-out lemma.
- Cite Moore-Hodgson by name.
10. Company Context
- Google: L5/L6; loves swap-out exchange arguments.
- Amazon: L6.
- Meta: E5/E6.
Strong: “Moore-Hodgson swap-out: sort by deadline, max-heap, accept-or-swap. Exchange argument: swap-out never hurts count and lowers cur_time. Cited weighted version is NP-hard.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Sort key | Wrong | After hint | Deadline | + exchange proof |
| Swap lemma | None | None | Verbal | + formal claim |
| Family | None | None | LC 1353 | + Moore-Hodgson reference |
| NP-hard variant | None | None | None | + weighted version |
12. Level Delta
- Mid (L4): Tries duration-sort; fails; corrects with hint.
- Senior (L5): Cold; verbal swap-out reasoning.
- Staff (L6): + cites Moore-Hodgson; + weighted NP-hard.
- Principal (L7+): + LP relaxation of weighted Moore-Hodgson + connection to earliest deadline first (EDF) scheduling theorem in real-time systems.
13. Follow-up Q&A
Q1: Courses can be preempted (resumed later).
A1: Preemptive EDF; feasibility check by Liu-Layland utilization bound Σ d_i/dl_i ≤ 1. Different algorithm.
Q2: Multiple machines. A2: Parallel-machine scheduling; NP-hard for m ≥ 2 even with all weights = 1 (m-machine Moore-Hodgson).
Q3: Weighted (each course has a value). A3: NP-hard; DP for small instances; LP relaxation for approximations.
Q4: Online (courses arrive over time). A4: Same swap-out works online — but you can’t “uncommit” time you’ve already spent on a course you’re partway through (depends on preemption model).
Q5: Trace the algorithm on [[5,5],[4,6],[2,6]].
A5: dl-sorted; take 5→cur=5; 4 doesn’t fit (cur+4=9>6) but heap_max=5>4 → swap, cur=4; 2 fits (4+2=6) → take, cur=6. Result: 2 courses ({4-duration, 2-duration}).
14. Solution Walkthrough
See solution.py:
schedule_course_greedy— Moore-Hodgson swap-out.schedule_course_brute— try all subsets in deadline order.
15. Beyond the Problem
Principal code review: “Course Schedule III is Moore-Hodgson (1968) in disguise — the canonical ‘minimize late jobs on a single machine, unit weights’ problem. The swap-out lemma is what makes it polynomial; the moment weights enter, the problem becomes NP-hard and you fall back to LP-relaxation rounding or DP for small instances. The Staff+ insight: the swap-out is a localized 1-step rollback that preserves count while improving the state (cur_time). This pattern — ‘reject by swapping out the worst already-accepted’ — recurs in: page-replacement (LRU is a special case of competitive swap-out), caching, online matching with deletion. The lesson scales: whenever your greedy doesn’t have a clean monotone-accept structure, look for a swap-out: ‘is there an already-committed element that, if evicted, would still maintain optimality?’ If yes, the algorithm is O(n log n) with a heap; if no, you’re in NP-hard territory. Real-world analogs: meeting-room reservation with priorities, virtual-machine consolidation in datacenters, scheduling preemptible batch jobs on a fixed CPU budget.”
Hints — p89 Course Schedule III
-
Sort courses by deadline ascending. Exchange argument: any feasible schedule can be reordered to respect deadline order without loss.
-
Walk in deadline order; maintain
cur_time(total time committed) and a max-heap of accepted durations. -
For each
(d, dl): ifcur_time + d ≤ dl, accept and push to heap. Else if heap’s max > d, swap: replace the longest accepted with this one, adjustingcur_time -= (max - d). -
The swap-out is the key insight: same count, but smaller
cur_time⇒ more room for future courses. Never hurts. -
This is classical Moore-Hodgson (1968): minimize late jobs on a single machine with unit weights. Weighted version is NP-hard.
If stuck: see solution.py.
p90 — Single-Threaded CPU
Source: LeetCode 1834 · Medium · Topics: Heap, Simulation, Sorting Companies (2024–2025): Amazon (high), Google (medium), Microsoft (medium), Bloomberg (medium) Loop position: Discrete-event simulation with a ready-queue heap. Combines sort-by-enqueue + min-heap of (proc_time, idx) tie-break + time fast-forward when idle.
1. Quick Context
Given tasks[i] = [enqueueTime, processingTime], simulate a CPU:
- Idle CPU picks the shortest-processing-time task that has arrived (tie-break: smallest index).
- Return the order tasks are processed.
Algorithm:
- Sort tasks by
enqueueTimekeeping original index. - Walk a pointer + maintain
cur_time+ min-heap of (proc_time, idx) for ready tasks. - Each iteration:
- If heap empty, fast-forward
cur_timeto next task’s enqueue. - Push all tasks with
enqueueTime ≤ cur_time. - Pop heap; append idx to order;
cur_time += proc_time.
- If heap empty, fast-forward
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/single-threaded-cpu/
STOP. 30-min timer.
3. Prerequisite Concepts
- Discrete-event simulation.
- Two-pointer over sorted events.
- Heap tie-break by lexicographic tuple.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Simulate SJF (shortest job first) on a single CPU, breaking ties by original index.
2. Clarify: Insertion order in heap = arrival order? Yes via (proc_time, idx) tuple.
3. Examples: tasks = [[1,2],[2,4],[3,2],[4,1]] → time 1 push idx0; pop (2,0); time 3 push 1,2; pop (2,2); time 5 push 3; pop (4,1); push (1,3) earlier; … result: [0,2,3,1].
5. Brute: Same simulation with naive O(n²) ready-set scan.
6. Target: O(n log n).
7. Pattern: Sort + heap + fast-forward.
8. Develop: see solution.py.
9. Test: all enqueue at 0; single task; large gap between enqueues.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why sort by enqueue?
Tasks arrive over time. Pointer walks through enqueue-sorted list; everything before the pointer with enqueue ≤ cur_time is in the heap. Sort makes this O(n) over all iterations.
6.2 The fast-forward step
When the CPU is idle and heap is empty, time jumps to the next task’s enqueue time. Don’t simulate every clock tick — costly and unnecessary.
Specifically: cur_time = max(cur_time, tasks[ptr].enqueue).
6.3 Tie-break correctness
Python’s tuple comparison compares element-wise. (proc_time, original_idx) ensures:
- Shorter proc_time wins.
- On tie, smaller original_idx wins.
6.4 Why min-heap, not other structures?
- Sorted set / BST works but heavier.
- Min-heap is the natural fit for “pop the smallest-priority element repeatedly”.
- Average O(log n) per push/pop.
6.5 SJF vs FIFO vs round-robin
This is Shortest Job First (SJF) with non-preemptive policy. SJF minimizes average wait time for known job sizes. Real OS schedulers use approximations (multilevel feedback queues) since job size is unknown.
6.6 Family: discrete-event simulation with priority queue
- LC 1834 Single-Threaded CPU (this).
- LC 1882 Process Tasks Using Servers (two-heap: free servers + busy servers).
- LC 1942 Number of Smallest Unoccupied Chair (similar pattern).
- LC 502 IPO (two-heap with gating).
- LC 295 Find Median from Data Stream (two-heap for medians).
7. Anti-Pattern Analysis
- Use just one queue (FIFO) — produces wrong order; not SJF.
- Sort by processingTime initially — destroys enqueue ordering; lookups become quadratic.
- Iterate per clock tick — O(maxTime) blow-up.
- Forget tie-break by idx — heap pops in proc_time order but unstable on ties.
- Push all tasks into heap upfront — picks tasks before they “arrive”.
- Use
tasks.pop(0)from sorted list — O(n) per pop = O(n²) total.
8. Skills & Takeaways
- Discrete-event simulation pattern.
- Heap with tuple tie-break.
- Fast-forward when idle.
9. When to Move On
- Solve cold in 20 min.
- Articulate the fast-forward and tuple tie-break.
- Cite LC 1882 servers as the next step.
10. Company Context
- Amazon: L4/L5; very common phone screen.
- Google: L5.
- Microsoft: L5; OS-flavored.
- Bloomberg: common.
Strong: “Sort by enqueue + min-heap of (proc_time, idx); fast-forward when idle; cited LC 1882 servers as the two-heap generalization.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Heap pattern | After hint | Cold | Clean | + fast-forward proof |
| Tie-break | Misses | After test | Cold | + tuple comparison reasoning |
| Family | None | None | LC 1882 | + LC 295 median + LC 502 IPO |
| Edge | Forgets idle | Time jump | Cold | + bounded |
12. Level Delta
- Mid (L4): Heap + sort, may miss fast-forward.
- Senior (L5): Cold; tuple tie-break correct; fast-forward.
- Staff (L6): + cites SJF/EDF connection; + multi-server generalization.
- Principal (L7+): + framing as non-preemptive SJF optimality for total wait time + LP analog + connection to OS schedulers (CFS, MLFQ).
13. Follow-up Q&A
Q1: Multi-CPU? A1: LC 1882: two heaps (free servers min-heap by weight+idx; busy servers min-heap by free_time). Standard generalization.
Q2: Preemptive (interruptible task)? A2: Preemptive SJF (SRTF — Shortest Remaining Time First) — heap of remaining-time; on each new arrival, compare against currently-running.
Q3: Tasks have priorities, not just sizes.
A3: Replace (proc_time, idx) with (priority, proc_time, idx).
Q4: Tasks can be batched (parallel up to k). A4: Becomes flow-shop / open-shop; NP-hard for k ≥ 2 with arbitrary processing times.
Q5: What’s the avg wait time under this algorithm? A5: SJF is provably optimal for minimizing average wait time when sizes are known a priori.
14. Solution Walkthrough
See solution.py:
get_order— sort + heap + fast-forward.get_order_brute— O(n²) simulation oracle.
15. Beyond the Problem
Principal code review: “Single-Threaded CPU is SJF (Shortest Job First) non-preemptive in textbook form. The Staff+ insight is that SJF is provably optimal for minimizing average waiting time when job sizes are known — a foundational result in scheduling theory (Smith’s rule, 1956). Real OS schedulers can’t use SJF directly because job sizes are unknown; instead they approximate via Multi-Level Feedback Queues (MLFQ) and Completely Fair Scheduler (CFS, Linux) which use aging to approximate SJF without prior knowledge. The lesson scales: discrete-event simulation with a priority queue is the workhorse for any ‘simulate this system’ interview question, and the structure is always: (1) sort events by time, (2) maintain a heap of currently-ready entities, (3) pop in priority order, (4) fast-forward time when idle. This pattern appears in ad-auction simulators, packet routing simulators, financial-trading match engines (Bloomberg loves this), and game-engine event loops. Once you internalize the four steps, every simulation problem collapses to filling in the priority key and the state update.”
Hints — p90 Single-Threaded CPU
-
This is Shortest Job First (SJF) non-preemptive with tie-break by original index.
-
Sort tasks by
enqueueTime(keeping original index). Walk a pointer through the sorted list. -
Maintain a min-heap of (proc_time, original_idx) for ready tasks. Tuple comparison handles SJF + tie-break automatically.
-
Fast-forward: when CPU is idle (heap empty), jump
cur_timeto next task’s enqueue time. Don’t simulate every clock tick. -
Each iteration: push all arrived tasks (enqueue ≤ cur_time); pop the smallest (proc_time, idx); append idx to order; advance cur_time.
If stuck: see solution.py.
Week 18 — Advanced Data Structures
Theme: The four advanced data structures that separate Senior from Staff: segment tree / Fenwick (BIT), KMP, rolling hash, monotonic deque. Each one is a tool that turns an O(n²) brute force into O(n log n) or O(n).
Schedule
- Mon — p91 Range Sum Query Mutable (LC 307): point-update + range-sum. Two implementations: Fenwick (BIT) for sum; segment tree (iterative, 2n nodes) for generality.
- Tue — p92 Count Smaller Numbers After Self (LC 315): BIT over coordinate-compressed values, walk right-to-left querying
sum(0..v-1). Also merge-sort with index tracking. - Wed — p93 Implement strStr / KMP (LC 28): failure function (longest proper prefix that is also a suffix) + match loop that never backtracks
i. Z-algorithm sketch as bonus. - Thu — p94 Repeated DNA Sequences (LC 187): polynomial rolling hash (Rabin-Karp), constant-time slide via
h = (h*BASE - first*POW + new) % MOD. Hash collision discussion. - Fri — p95 Sliding Window Maximum (LC 239): monotonic decreasing deque holding indices; pop from back while smaller, pop from front when out of window.
Readiness Gate
- Implement Fenwick tree from scratch in 10 min (point update + prefix sum).
- Implement iterative segment tree from scratch in 15 min.
- Write KMP failure function and articulate the loop invariant.
- Write polynomial rolling hash with modular sliding update.
- Write monotonic deque for sliding window max.
If any of these takes > 1 hint, re-do the problem.
Cross-References
- FRAMEWORK.md — Approach 1–9.
- phase-03-advanced-data-structures/ — lab support for segment trees, KMP, hashing.
- phase-09-language-runtime/python/ — Python
bisect,collections.deque.
What “Done” Looks Like
- All 5 problems solved cold within ~25 min, no hints.
- Can implement each DS without referring to notes.
- Can articulate when to choose each one (BIT vs segment tree, KMP vs rolling hash).
- Can cite at least one production analog per problem.
p91 — Range Sum Query Mutable
Source: LeetCode 307 · Medium · Topics: Fenwick Tree (BIT), Segment Tree, Design Companies (2024–2025): Google (high), Meta (medium), Amazon (medium), Bloomberg (high) Loop position: The canonical “build a data structure” warm-up. Tests whether you’ve memorized Fenwick AND can derive segment-tree alternative on the spot.
1. Quick Context
Design a data structure supporting:
update(index, val)— setarr[index] = val.sumRange(left, right)— returnsum(arr[left..right])inclusive.
Two solutions:
- Fenwick Tree (BIT): 1-indexed array
tree[]wheretree[i]coversi & (-i)elements ending at i. Point update + prefix sum, both O(log n). - Iterative Segment Tree: array of size 2n. Leaves at
tree[n..2n-1], internal nodes attree[1..n-1]. Update: walk up from leaf. Query: walk up from both endpoints accumulating disjoint chunks.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/range-sum-query-mutable/
STOP. 30-min timer. Implement Fenwick from memory — no looking up i & -i.
3. Prerequisite Concepts
- Two’s complement (
i & -iisolates the lowest set bit). - Recursive segment tree → iterative segment tree compression.
- Prefix sums as a building block.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Mutable array with O(log n) updates and O(log n) range sum.
2. Clarify: How often update vs query? If updates rare: prefix-sum + O(n) update. If queries rare: O(1) update, O(n) query. Balanced: BIT/segment tree.
3. Examples: [1,3,5], update(1, 2) → [1,2,5], sumRange(0,2)=8.
5. Brute: Store array; O(n) per query.
6. Target: O(log n) per op.
7. Pattern: BIT or segment tree.
8. Develop: see solution.py.
9. Test: all updates then all queries; interleaved; out-of-order updates.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Fenwick tree — the i & -i trick
tree[i] stores the sum of arr[i - lowbit(i) + 1 .. i] where lowbit(i) = i & -i. Update at index i propagates upward via i += lowbit(i). Prefix query at i walks down via i -= lowbit(i).
Why it works: every prefix [1..n] can be decomposed into O(log n) disjoint intervals of length lowbit. The structure is a binary-indexed forest where each node’s range is determined by its trailing zeros.
6.2 Fenwick vs segment tree
| Aspect | Fenwick | Iterative Segment Tree |
|---|---|---|
| Code lines | ~10 | ~20 |
| Space | n+1 | 2n |
| Sum/prefix-sum | ✅ | ✅ |
| Range max/min | ❌ (or hack) | ✅ |
| Range update + point query | ✅ (diff trick) | ✅ |
| Range update + range query | ❌ (BIT^2 hack) | ✅ (lazy) |
Rule: sums only → BIT. Anything else (max, min, gcd, lazy) → segment tree.
6.3 Iterative segment tree — the 2n layout
Leaves at indices [n, 2n). Internal node i has children 2i and 2i+1. Update: i = leaf; while i > 1: tree[i/2] = combine(tree[i], tree[i^1]); i /= 2. Query: walk from l += n and r += n + 1 upward, accumulating off-spine values.
6.4 Range update + point query via difference array on BIT
Store diff[i] = arr[i] - arr[i-1]. Then arr[i] = prefix_sum(diff, i). Range update [l, r] += v becomes two point updates: diff[l] += v, diff[r+1] -= v. O(log n).
6.5 Range update + range query (BIT^2)
Maintain two BITs: B1 and B2. Range update [l,r] += v: B1.update(l, v); B1.update(r+1, -v); B2.update(l, v*(l-1)); B2.update(r+1, -v*r). Range query: prefix_sum(i) = B1.prefix(i)*i - B2.prefix(i). Surprisingly clean.
6.6 Family: range query / point update DS
- LC 307 Range Sum Query Mutable (this).
- LC 308 Range Sum Query 2D Mutable (2D BIT or 2D segment tree).
- LC 315 Count Smaller After Self (BIT over value space).
- LC 327 Count of Range Sum (BIT or merge-sort).
- LC 218 Skyline Problem (segment tree on heights).
- LC 715 Range Module (segment tree with lazy or sorted intervals).
7. Anti-Pattern Analysis
- Square root decomposition — O(√n) per op; works but signals you didn’t know BIT/segment tree.
- Recompute prefix sum on update — O(n) update.
- Recursive segment tree — works but verbose; iterative is the interview standard.
- 0-indexed BIT — possible but error-prone; use 1-indexed.
- Forget that BIT stores deltas, not the array — confused initial-build code.
- Build BIT in O(n log n) — possible to build in O(n) using
tree[i + lowbit(i)] += tree[i].
8. Skills & Takeaways
- Two implementations of the same interface.
i & -ilowbit trick.- Iterative segment tree layout.
9. When to Move On
- Implement Fenwick in 10 min cold.
- Implement iterative segment tree in 15 min cold.
- Decide which to use based on operation type.
10. Company Context
- Google: L5/L6; very common.
- Meta: E5; expects BIT.
- Amazon: L6.
- Bloomberg: loves BIT for financial range queries.
Strong: “Fenwick (BIT) for point-update + range-sum, O(log n) each. Segment tree if I needed range max or lazy propagation. Cited BIT^2 for range-update + range-query.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| BIT impl | After hint | Cold | Clean + 1-indexed | + O(n) build |
| Lowbit | Doesn’t know | Knows trick | Explains | + derives from two’s complement |
| ST alternative | None | Mentions | Sketches | + iterative + lazy variant |
| Family | None | None | LC 315 | + BIT^2 range-range |
12. Level Delta
- Mid (L4): BIT after a hint.
- Senior (L5): Cold BIT; can sketch segment tree.
- Staff (L6): + BIT^2 for range/range; + LC 308 2D.
- Principal (L7+): + framing as prefix-sum query over a tree-shaped commutative monoid + recognizes BIT as a Morton-encoded prefix decomposition + can implement persistent / immutable segment tree for time-travel queries.
13. Follow-up Q&A
Q1: Range update + range query? A1: BIT^2 (two BITs) or segment tree with lazy propagation. Same O(log n).
Q2: Range max instead of sum? A2: BIT supports max in narrow case (only “max of prefix”) — segment tree is the right call.
Q3: 2D version?
A3: 2D BIT — update(r,c) walks r+=lowbit(r) and c+=lowbit(c) nested. O(log² n) per op.
Q4: Online (streaming inserts at arbitrary positions)? A4: Balanced BST or order-statistic tree (skiplist, sorted-container). BIT requires fixed index space.
Q5: Persistent (query any historical version)? A5: Persistent segment tree; O(log n) per op with O(log n) extra space.
14. Solution Walkthrough
See solution.py:
NumArrayBIT— Fenwick implementation.NumArraySegTree— iterative segment tree.NumArrayBrute— oracle (O(n) updates/queries).
15. Beyond the Problem
Principal code review: “Range Sum Query Mutable is the smallest ‘mutable aggregate over an index space’ problem. The Staff+ insight is recognizing this as a query over a commutative monoid (here, integers under addition) on a sequential index space. Once you frame it that way, the data structure choice falls out: BIT for monoids with inverses (sum, XOR), segment tree for monoids without inverses (max, min, gcd, set union). The lesson scales: when you face any ‘mutable array with aggregate queries’ problem — financial OHLC bars, real-time analytics, ad-auction histograms, time-series DB — the answer is a segment tree or BIT, and the only question is whether your aggregation has an inverse. Real systems: ClickHouse / DuckDB use column-oriented segment trees for analytical queries; Redis ZADD uses a skiplist (similar O(log n) tradeoff); ad-tech systems use 2D BITs for geographic + temporal aggregations. Knowing this pattern lets you design these systems from scratch.”
Hints — p91 Range Sum Query Mutable
-
O(n) per query is too slow for many queries; O(n) per update is too slow if frequent. Need O(log n) for both.
-
Fenwick Tree (BIT): 1-indexed array where
tree[i]stores the sum of a range ending atiof lengthlowbit(i) = i & -i. Update propagatesi += lowbit(i); prefix sum walksi -= lowbit(i). -
Iterative Segment Tree: 2n-node array. Leaves at
[n, 2n). Internal nodeiaggregatestree[2i]andtree[2i+1]. Range query: walk both endpoints upward, accumulate off-spine nodes. -
Range sum
[l, r]via BIT =prefix(r+1) - prefix(l)using 1-indexed inclusive prefix. -
Choice: BIT for sum/XOR (has inverse); segment tree for max/min/gcd (no inverse) or for lazy propagation (range updates).
If stuck: see solution.py.
p92 — Count of Smaller Numbers After Self
Source: LeetCode 315 · Hard · Topics: Fenwick Tree, Merge Sort, Coordinate Compression Companies (2024–2025): Google (high), Meta (medium), Amazon (medium), Bytedance (high) Loop position: Inversion counting in disguise. Tests the leap from “what does the problem ask” to “this is an inversion problem; use BIT or merge sort”.
1. Quick Context
For each index i, count how many j > i have nums[j] < nums[i]. Return the counts array.
Reframe: for each element, count smaller values to its right = count of inversions involving this element on the right.
Two solutions:
- BIT over coordinate-compressed values, right-to-left: walk i from n-1 down to 0; query
prefix(rank(nums[i]) - 1)to count smaller already-seen-to-the-right; thenupdate(rank(nums[i]), +1). - Merge sort with index tracking: during merge, when taking from left, the count of right-elements already merged contributes to that left-index’s inversion count.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/count-of-smaller-numbers-after-self/
STOP. 35-min timer.
3. Prerequisite Concepts
- Fenwick tree (LC 307).
- Coordinate compression (
sorted(set(nums))+ dict for rank). - Merge sort with index permutation.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Inversion count per element with j > i.
2. Clarify: Values can be negative; large range → compress.
3. Examples: [5,2,6,1] → [2,1,1,0].
5. Brute: O(n²) double loop.
6. Target: O(n log n).
7. Pattern: BIT or merge sort.
8. Develop: see solution.py.
9. Test: sorted asc (all 0); sorted desc (n-1, n-2, …); duplicates.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 BIT-right-to-left intuition
Walk i from right to left. At each step, the BIT holds the multiset of values to the right of i. Query “how many smaller than nums[i] have I already seen?” → bit.prefix(rank(nums[i]) - 1).
Coordinate compression maps n possibly-huge values into ranks [1..k] where k ≤ n.
6.2 Merge-sort intuition
Standard inversion-count adapted: during the merge step, maintain original indices. When you pop from the LEFT half, the count of elements already popped from the RIGHT half is the # of elements that are smaller AND to the right of this left element.
sort sublist by value while tracking original index
during merge, when left[i] is popped, ans[original_idx[left[i]]] += right_popped_so_far
6.3 Why both run in O(n log n)
- BIT: each query/update O(log n); n elements.
- Merge sort: O(n log n) standard; per-level merge is O(n) and there are log n levels.
6.4 BIT requires coordinate compression for arbitrary integer ranges
Without compression, BIT size = max_value − min_value + 1, blowing up memory. Compression maps to dense ranks.
6.5 Variants
- LC 327 Count of Range Sum: similar reframe; sort prefix sums and BIT-count.
- LC 493 Reverse Pairs: count pairs (i, j) with i < j and nums[i] > 2·nums[j]; merge-sort variant.
- LC 1505 Min Possible Integer After K Swaps: BIT to track which digits remain.
6.6 Family: inversion / order-statistics
- LC 315 Count Smaller After Self (this).
- LC 327 Count of Range Sum.
- LC 493 Reverse Pairs.
- LC 1395 Count Number of Teams.
- LC 2179 Count Good Triplets in an Array.
7. Anti-Pattern Analysis
- Naive O(n²) — works on small inputs; fails on n > 10⁴.
- Sort upfront, then index — destroys positional info needed.
- BIT without coordinate compression — TLE/MLE on large values.
- Off-by-one in BIT prefix — query
prefix(rank-1)notprefix(rank)(else includes equal values). - Merge sort modifying values directly — must track original indices in an auxiliary array.
- Use a sorted list with
bisect.insort— O(n²) worst case because insert is O(n).
8. Skills & Takeaways
- Inversion-counting reframe.
- BIT over coordinate-compressed values.
- Merge sort with index tracking.
9. When to Move On
- Solve cold in 25 min.
- Articulate why BIT solves inversion problems naturally.
- Cite LC 493 Reverse Pairs as a generalization.
10. Company Context
- Google: L6; expects BIT + compression.
- Meta: E5/E6.
- ByteDance: very common; loves merge-sort variant.
- Amazon: L6.
Strong: “Reframed as inversion count. BIT over coordinate-compressed values, right-to-left scan. Also achievable via merge sort with index tracking; O(n log n) either way.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Reframe | Misses | After hint | Cold | + cites inversion theory |
| BIT impl | None | After hint | Clean | + coordinate compression |
| Merge sort | None | None | Sketch | + clean index-tracking |
| Family | None | None | LC 327 | + LC 493 + LP framing |
12. Level Delta
- Mid (L4): BIT after hint; works but messy compression.
- Senior (L5): Cold BIT or merge sort.
- Staff (L6): + both solutions + LC 493.
- Principal (L7+): + framing as rank-aggregation problem / Kendall-tau distance + connection to sorting-network lower bounds (Ω(n log n) comparisons).
13. Follow-up Q&A
Q1: Count larger numbers after self instead.
A1: Symmetric. prefix(max_rank) - prefix(rank) on the same BIT, or process left-to-right.
Q2: Reverse pairs (i < j, nums[i] > 2·nums[j]) — LC 493.
A2: Merge sort variant; during merge, for each left[i], two-pointer scan right half counting r[j] < left[i]/2.
Q3: Online (streaming inserts) — count smaller after each insert. A3: Order-statistic balanced BST (skiplist or Treap) supporting insert + rank query in O(log n).
Q4: 2D version (count points dominated below-right). A4: Offline sweep + 2D BIT or persistent segment tree.
Q5: Total inversion count (sum of counts). A5: Just sum the result, or equivalent classical merge-sort inversion count in O(n log n).
14. Solution Walkthrough
See solution.py:
count_smaller_bit— Fenwick + coordinate compression.count_smaller_mergesort— merge sort with index tracking.count_smaller_brute— O(n²) oracle.
15. Beyond the Problem
Principal code review: “Count Smaller After Self is inversion counting dressed as an array problem. The Staff+ leap is recognizing this from the problem statement in 30 seconds — and once you have, the solution is mechanical: either BIT over coordinate-compressed values (right-to-left) or merge sort with index tracking. The deeper lesson: inversion count is the Kendall-tau distance between the input and its sorted form, and it’s a fundamental measure in rank aggregation (used in: recommender systems for ranking similarity, search-result re-ranking metrics, statistical tests like Mann-Whitney U). The principal-level recognition is that ANY problem reducible to ‘how many (i,j) pairs satisfy this monotone predicate’ can be solved in O(n log n) by BIT or merge sort. This pattern appears in: financial reconciliation (out-of-order trades), distributed system event reordering analysis, A/B test ranking metrics. Once you internalize the inversion-counting lens, dozens of seemingly-different problems collapse to the same algorithm.”
Hints — p92 Count Smaller After Self
-
Reframe: for each i, count of
nums[j] < nums[i]withj > i= inversion contribution. -
BIT approach: walk i from right to left. BIT holds the multiset of values seen so far (i.e., to the right of i).
ans[i] = BIT.prefix(rank(nums[i]) - 1). ThenBIT.update(rank(nums[i]), +1). -
Coordinate compression is necessary for arbitrary-range values: map each distinct value to its rank in sorted order.
-
Merge sort alternative: during merge, track original indices. Each time you pop from left, add
right_popped_so_fartoans[original_idx]. -
Off-by-one: query
prefix(rank - 1)to count strictly smaller; usingprefix(rank)would include equal values.
If stuck: see solution.py.
p93 — Implement strStr() / KMP
Source: LeetCode 28 · Easy (deceptively) · Topics: String, KMP, Z-algorithm Companies (2024–2025): Google (high), Amazon (medium), Meta (medium) Loop position: The one “Easy” problem that separates levels. Naive O(nm) gets you the job at Mid. KMP O(n+m) signals Senior. Articulating the failure function invariant is Staff.
1. Quick Context
Return the index of the first occurrence of needle in haystack, or -1.
Naive: O(n·m). Slide window of length m; compare char-by-char.
KMP: O(n+m). Precompute failure function lps[i] = length of the longest proper prefix of needle[0..i] that is also a suffix. On mismatch, skip via j = lps[j-1] instead of restarting.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/find-the-index-of-the-first-occurrence-in-a-string/
STOP. 30-min timer. Write KMP from memory — including the failure function loop.
3. Prerequisite Concepts
- Strings as char arrays.
- Proper prefix vs suffix.
- Amortized analysis (KMP runs in O(n+m) despite nested loop).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Substring search.
2. Clarify: Empty needle? Return 0. Needle longer than haystack? Return -1.
3. Examples: haystack="sadbutsad", needle="sad" → 0. needle="abc" → -1.
5. Brute: O(n·m).
6. Target: O(n+m).
7. Pattern: KMP failure function.
8. Develop: see solution.py.
9. Test: match at start; match at end; no match; needle = haystack; periodic needle (aaab in aaaaab).
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The failure function — what it stores
lps[i] = max k < i+1 such that needle[0..k-1] == needle[i-k+1..i].
It’s the longest prefix of needle ending at position i that’s also a prefix of needle. On mismatch, we know the last lps[j-1] chars matched a prefix → resume comparing from position lps[j-1] instead of 0.
6.2 Building lps in O(m) — the trick
lps[0] = 0
length = 0
for i in 1..m-1:
while length > 0 and needle[i] != needle[length]:
length = lps[length - 1]
if needle[i] == needle[length]:
length += 1
lps[i] = length
Amortized O(m): length can increase by at most 1 per iteration (total m increases). Each decrement strictly reduces length. So total work O(m).
6.3 KMP match loop
i = 0 # haystack
j = 0 # needle
while i < n:
if haystack[i] == needle[j]:
i += 1; j += 1
if j == m: return i - m # full match
else:
if j > 0: j = lps[j - 1] # fall back
else: i += 1 # nothing to fall back on
Critical property: i never decreases. That’s what gives O(n+m).
6.4 Why amortized linear
Total i increments: O(n). Total j increments: ≤ i ⇒ O(n). Total j decrements: ≤ total j-increments ⇒ O(n). So overall O(n+m).
6.5 Alternatives
- Z-algorithm: computes
z[i]= length of longest substring starting at i that matches a prefix. O(n+m) by concatenatingneedle + sentinel + haystack. - Rabin-Karp: polynomial rolling hash. O(n+m) expected, O(nm) worst.
- Boyer-Moore: sublinear average via bad-character + good-suffix heuristics.
6.6 Family: substring / pattern matching
- LC 28 strStr (this).
- LC 459 Repeated Substring Pattern (KMP failure function reveals period).
- LC 686 Repeated String Match.
- LC 214 Shortest Palindrome (KMP on
s + '#' + reverse(s)). - LC 1392 Longest Happy Prefix (lps[m-1] directly).
7. Anti-Pattern Analysis
- Use Python’s
str.find— works but signals you avoided the question. - O(n·m) naive — passes LC but interviewer flags as Junior signal.
- Off-by-one in lps building — common;
lps[length - 1]notlps[length]on mismatch. - Forget the
if j > 0branch — infinite loop when j=0 mismatches. - Decrement
ion mismatch — defeats the point of KMP. - Confuse
jwith i when explaining loop invariants — interviewers test this.
8. Skills & Takeaways
- Failure function construction.
- Match loop never decrementing
i. - Amortized linear-time argument.
9. When to Move On
- Implement KMP cold in 20 min including lps build.
- Explain amortized O(n+m).
- Cite Z-algorithm as alternative.
10. Company Context
- Google: L5; expects KMP cold + amortized argument.
- Amazon: L4/L5.
- Meta: E5.
Strong: “Naive is O(nm); KMP via failure function in O(n+m). lps[i] = longest proper prefix of needle[0..i] that’s also a suffix. On mismatch, fall back to lps[j-1] without decrementing i; amortized linear.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Naive | Works | Cold | Cold | Cold |
| KMP | None | After hint | Cold | + amortized proof |
| Failure fn | None | None | Builds | + explains semantics |
| Family | None | None | LC 459 | + Z-algorithm |
12. Level Delta
- Mid (L4): Naive O(nm); mentions KMP.
- Senior (L5): Implements KMP cold.
- Staff (L6): + amortized proof + LC 459 period detection.
- Principal (L7+): + framing as DFA over needle alphabet (Aho-Corasick generalization for multi-pattern) + suffix automaton connection.
13. Follow-up Q&A
Q1: Find ALL occurrences, not just first.
A1: On full match, instead of return, append i - m to results and set j = lps[m - 1] to continue.
Q2: Multi-pattern search (k patterns). A2: Aho-Corasick — KMP generalized to a trie with failure links. O(n + Σ|p_i| + #matches).
Q3: Rotated string matching.
A3: KMP on s + s for needle = original; or KMP on doubled haystack.
Q4: Approximate matching with k errors. A4: Bit-parallel (Shift-Or with k). Or DP.
Q5: What does lps[m-1] tell you about the string?
A5: Length of the longest proper prefix that’s also a suffix → if m - lps[m-1] divides m, the string is periodic with that period (LC 459).
14. Solution Walkthrough
See solution.py:
strstr_kmp— KMP match._build_lps— failure function.strstr_brute— naive O(n·m).
15. Beyond the Problem
Principal code review: “KMP (1977) is one of the deepest ‘easy’ algorithms in computer science. The Staff+ recognition is that the failure function is a DFA — specifically, a partial deterministic automaton that recognizes any string ending in
needle. Aho-Corasick (1975) generalizes this to multi-pattern matching by building a trie of all needles and adding ‘failure links’ (the multi-needle version of lps). The lesson scales: any ‘detect this pattern in a stream’ problem at production scale runs Aho-Corasick or KMP — virus signature scanning (ClamAV), intrusion detection (Snort), DNS-blocklist matching, content filtering, regex engines (RE2 uses NFA but falls back to KMP-like for literal substrings). The deeper lesson is the amortized linear-time technique: KMP’s correctness rests on the fact thatinever decreases, which is the same argument used in two-pointer problems, monotonic-stack problems, and the Boyer-Moore voting algorithm. Once you see this pattern — ‘each character is examined O(1) amortized’ — you recognize it everywhere.”
Hints — p93 Implement strStr / KMP
-
Naive O(n·m) is the baseline. KMP gets O(n+m) by never restarting
iin the haystack. -
Precompute lps (longest proper prefix-suffix):
lps[i]= length of longest proper prefix ofneedle[0..i]that’s also a suffix. -
Building lps: maintain
length= previous longest. On match,length += 1, lps[i] = length, i++. On mismatch, fall back vialength = lps[length-1](without advancing i), unlesslength == 0(thenlps[i] = 0, i++). -
Match loop: walk
ithrough haystack andjthrough needle. On mismatch withj > 0, setj = lps[j-1]; elsei++. Onj == m, found match ati - m. -
Amortized linear:
istrictly increases;jincreases at most n times, decreases at most n times. Total O(n+m).
If stuck: see solution.py.
p94 — Repeated DNA Sequences
Source: LeetCode 187 · Medium · Topics: Hash Set, Rolling Hash, Bit Manipulation Companies (2024–2025): Google (medium), Amazon (medium), Bytedance (high) Loop position: The Rabin-Karp warm-up. Tests whether you can implement a rolling polynomial hash without copy-pasting it.
1. Quick Context
Find all 10-letter substrings of a DNA string (alphabet {A,C,G,T}) that appear more than once.
Three solutions:
- Set of substrings: O(n·k) time & memory; trivial.
- Bit-encoded fixed-width hash: since k=10 and alphabet is 4 chars, encode each substring in 20 bits (10 × 2 bits). Maintain rolling 20-bit integer. O(n) time, O(n) memory.
- Polynomial rolling hash (Rabin-Karp): general technique for arbitrary alphabet and k. O(n) expected with collision handling.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/repeated-dna-sequences/
STOP. 25-min timer. Implement bit-encoded version then rolling-hash version.
3. Prerequisite Concepts
- Hash sets.
- Sliding window.
- Modular arithmetic (for rolling hash).
- Bit manipulation.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Find all duplicate 10-length windows.
3. Examples: "AAAAACCCCCAAAAACCCCCCAAAAAGGGTTT" → ["AAAAACCCCC", "CCCCCAAAAA"].
5. Brute: Set of all substrings, return duplicates. O(n·k).
6. Target: O(n) or O(n + #matches).
7. Pattern: Rolling hash.
8. Develop: see solution.py.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Bit-encoded fixed-width hash (the DNA-specific trick)
Map A→00, C→01, G→10, T→11. Each 10-mer fits in 20 bits. Sliding window:
mask = (1 << 20) - 1
h = (h << 2 | code[c]) & mask
After first 10 chars, h uniquely identifies the current 10-mer. Use a dict {hash → count} or set + seen.
6.2 Polynomial rolling hash (Rabin-Karp, general)
For arbitrary alphabet:
h = (h * BASE - s[i - k] * BASE^k + s[i]) % MOD
where BASE > alphabet size and MOD is a large prime (e.g., (1<<61) - 1).
Collision handling: hash matches → still verify substrings character-by-character. Use two hashes with different MODs for ultra-low collision probability (1/MOD²).
6.3 Why this is the canonical “rolling hash” lesson
Rolling hash underpins:
- Rabin-Karp (substring search).
- Karp-Rabin minimum-rotation.
- Longest common substring via hashing + binary search on length.
- Polynomial-hash-based suffix array construction.
- Zobrist hashing (game-tree DP).
6.4 Mersenne primes for fast modulo
(1<<61) - 1 is Mersenne. x mod ((1<<61)-1) = (x & ((1<<61)-1)) + (x >> 61) then conditional subtract. Avoids slow %. Useful in performance-critical C++/Rust.
6.5 Hash families and adversarial inputs
Constant BASE and MOD can be exploited by adversarial inputs to force collisions. In competitive programming use randomized BASE chosen at runtime; in security-sensitive code use SipHash or a keyed cryptographic hash.
6.6 Family: rolling hash / substring repeats
- LC 187 Repeated DNA (this).
- LC 1044 Longest Duplicate Substring (rolling hash + binary search on length).
- LC 28 strStr / KMP (Rabin-Karp alternative).
- LC 1316 Distinct Echo Substrings.
- LC 718 Maximum Length of Repeated Subarray (rolling hash variant).
7. Anti-Pattern Analysis
- Use Python’s set of substrings without rolling hash — works on LC but signals you didn’t engage the problem.
- Build hash from scratch every window — O(n·k); same as brute.
- Forget collision verification — Rabin-Karp without verify is incorrect.
- Use BASE = 4 with simple mod — works for k=10 here but breaks for general alphabets.
- Use
hash(str)builtin — Python’s hash is randomized per run; can’t be used as portable rolling hash.
8. Skills & Takeaways
- Bit-encoded fixed-alphabet rolling.
- Polynomial rolling hash with two-hash collision handling.
- Recognizing when rolling hash applies.
9. When to Move On
- Implement both bit-encoded and polynomial versions cold.
- Cite LC 1044 (binary search + rolling hash).
- Explain Mersenne-prime mod trick.
10. Company Context
- Google: L5.
- ByteDance: loves rolling hash.
- Amazon: L5/L6.
Strong: “Bit-encoded since alphabet has 4 chars; 20-bit window suffices. For general case, polynomial rolling hash with double-hash collision verification.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Set approach | Cold | Cold | Cold | Cold |
| Bit-encoded | None | After hint | Cold | + explains uniqueness |
| Polynomial | None | None | Sketch | + collision handling |
| Family | None | None | LC 1044 | + Mersenne + adversarial |
12. Level Delta
- Mid (L4): Set of substrings.
- Senior (L5): Bit-encoded rolling.
- Staff (L6): + polynomial rolling for general alphabet + double hash.
- Principal (L7+): + Mersenne-prime mod + randomized BASE for adversarial safety + connection to fingerprinting / Bloom-filter false-positive analysis.
13. Follow-up Q&A
Q1: Generalize to arbitrary k and alphabet. A1: Polynomial rolling hash; collision verify.
Q2: Longest repeated substring (LC 1044). A2: Binary search on length L; for each L, rolling hash all length-L windows; if any repeats, L is achievable.
Q3: Probability of collision with single hash MOD ~ 10^9. A3: ~ n²/MOD by birthday. For n = 10^5, ~10 collisions; need verify or double hash.
Q4: Stream of DNA — find new repeats online. A4: Maintain dict {hash → first_index}; on each new window check membership.
Q5: Memory-bound k = 10^6, n = 10^9? A5: Bloom filter or Count-Min sketch for probabilistic dedup.
14. Solution Walkthrough
See solution.py:
find_repeated_set— naive set-based.find_repeated_bit— bit-encoded rolling.find_repeated_polynomial— polynomial rolling hash + verify.
15. Beyond the Problem
Principal code review: “Repeated DNA Sequences is the gateway to rolling hash, one of the most leveraged techniques in real systems. The Staff+ recognition: this is the same algorithm as rsync’s rolling checksum (Adler-32), content-defined chunking in deduplication systems (Restic, borg, Plan9’s Venti), Git’s pack-file delta compression, and the Karp-Rabin substring matcher used inside grep-like tools when patterns are long. The bit-encoded approach here is the special case where alphabet is small enough to encode in fixed bits — same trick used by bioinformatics tools (BLAST, minimap2) for k-mer indexing of genomes. Once you internalize ‘rolling hash for substring problems’, a whole class of problems becomes O(n) instead of O(n²) — and once you understand collision verification, you understand why production systems use double-hashing or cryptographic-grade rolling functions (SipHash for HashDoS resistance). The deeper lesson: hashing is a probabilistic data structure, and engineering with it requires reasoning about collision probability and adversarial inputs.”
Hints — p94 Repeated DNA Sequences
-
Trivial: set of every length-10 substring → repeats. Works but O(n·k) memory.
-
Bit-encoded rolling hash: alphabet is
{A,C,G,T}(4 letters); encode each base in 2 bits. A 10-mer fits in 20 bits. Slide:h = ((h << 2) | code[c]) & ((1 << 20) - 1). -
After feeding the first 10 chars,
huniquely identifies the current window. Maintainseenset of hashes; on collision, add the substring to result. -
General Rabin-Karp: polynomial rolling hash
h = (h * BASE - s[i-k] * BASE^k + s[i]) mod MOD. Use a large MOD (e.g.,(1<<61) - 1) and verify substring on hash match. -
Double-hashing (two independent (BASE, MOD) pairs) reduces collision probability to ~1/MOD² without needing to verify substrings.
If stuck: see solution.py.
p95 — Sliding Window Maximum
Source: LeetCode 239 · Hard · Topics: Deque, Monotonic Queue, Sliding Window Companies (2024–2025): Amazon (high), Google (medium), Meta (medium), Bytedance (high) Loop position: The canonical monotonic-deque problem. Required to articulate why a monotonic deque achieves O(n) where a heap achieves O(n log n).
1. Quick Context
Given array nums and window size k, return the max of every size-k window.
Solutions:
- Brute: O(n·k). Recompute max per window.
- Heap with lazy deletion: O(n log k). Push (-val, idx); pop top while stale.
- Monotonic decreasing deque: O(n). Deque stores indices, values strictly decreasing from front to back.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/sliding-window-maximum/
STOP. 30-min timer.
3. Prerequisite Concepts
collections.dequepush/pop from both ends.- Monotonic stack (LC 84, 496).
- Amortized analysis.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Max of every length-k window.
3. Examples: [1,3,-1,-3,5,3,6,7], k=3 → [3,3,5,5,6,7].
5. Brute: O(n·k).
6. Target: O(n).
7. Pattern: Monotonic deque.
8. Develop: see solution.py.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Monotonic deque invariant
Deque stores indices i_1 < i_2 < ... < i_m such that nums[i_1] > nums[i_2] > ... > nums[i_m] (strictly decreasing). Front is always the max of the current window.
Per element i:
- Pop from back while
nums[back] <= nums[i](back can never be the max if a later equal-or-greater value exists). - Pop from front while
front <= i - k(out of window). - Push i.
- If
i >= k - 1, recordnums[deque[0]].
6.2 Why O(n) amortized
Each index is pushed once and popped at most once → 2n deque operations total.
6.3 Why heap is O(n log k), not O(n)
Heap doesn’t support efficient deletion of arbitrary elements. Lazy deletion is the workaround: keep popping stale top entries. Each push/pop is O(log k); each element pushed once and lazy-popped at most once.
6.4 Variants
- Sliding minimum: flip comparison; monotonic increasing deque.
- Both min and max: maintain two deques.
- Sliding median: harder; requires balanced BST or two heaps; O(n log k).
- Sliding k-th order statistic: order-statistics tree.
6.5 Family: monotonic stack/queue
- LC 239 Sliding Window Max (this).
- LC 84 Largest Rectangle in Histogram.
- LC 496 Next Greater Element.
- LC 862 Shortest Subarray with Sum ≥ K (monotonic deque on prefix sums).
- LC 1438 Longest Subarray with Absolute Diff ≤ Limit (two monotonic deques).
7. Anti-Pattern Analysis
- Brute O(n·k) — Junior signal; fine on tiny n but flagged.
- Heap without lazy deletion — incorrect (stale entries linger).
- Store values not indices in deque — can’t tell when something falls out of window.
- Use
<not<=when popping back — for duplicates either works for max but<=keeps deque smaller. - Use
>instead of>=when checking out-of-window — off-by-one; condition isfront <= i - k. - Forget the
i >= k - 1guard before recording output.
8. Skills & Takeaways
- Monotonic deque pattern.
- Amortized linear via push/pop accounting.
- Heap with lazy deletion as alternative.
9. When to Move On
- Implement monotonic deque cold in 15 min.
- Explain amortized O(n).
- Cite LC 862, LC 1438 as deeper monotonic-deque problems.
10. Company Context
- Amazon: L5/L6; very common.
- Google: L5.
- ByteDance: loves monotonic deque.
Strong: “Monotonic decreasing deque of indices; front is the window max. Pop back while ≤ new, pop front when out-of-window. Each index touched twice → O(n).”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Brute | Cold | Cold | Cold | Cold |
| Heap | None | After hint | Cold | + lazy deletion |
| Deque | None | None | Cold | + amortized analysis |
| Family | None | None | LC 84 | + LC 862 deque-on-prefix-sum |
12. Level Delta
- Mid (L4): Heap O(n log k).
- Senior (L5): Monotonic deque cold.
- Staff (L6): + amortized argument + cites LC 862 prefix-sum + monotonic deque.
- Principal (L7+): + framing as incremental order-statistic maintenance + connection to streaming-percentile sketches (P², GK) for harder variants.
13. Follow-up Q&A
Q1: Sliding minimum.
A1: Symmetric; pop back while >= new.
Q2: Sliding median.
A2: Two-heap balanced approach with lazy deletion, OR SortedList (skiplist), OR an order-statistics tree. O(n log k).
Q3: Window of dynamic size. A3: Two-pointer + monotonic deque still works.
Q4: Streaming (online) max with arbitrary window age cutoffs. A4: Sliding-window aggregation via monotonic deque; for time-based windows, deque of (timestamp, value).
Q5: Max of any subarray of size ≥ k. A5: Monotonic deque again, but record max once entered window.
14. Solution Walkthrough
See solution.py:
max_sliding_window_deque— monotonic deque O(n).max_sliding_window_heap— heap + lazy deletion.max_sliding_window_brute— O(n·k) oracle.
15. Beyond the Problem
Principal code review: “Sliding Window Maximum is the canonical monotonic-deque problem, and the Staff+ recognition is that the pattern — ‘maintain a monotone-ordered structure of indices/values, evicting from one end as the window slides’ — appears everywhere in production. It’s the algorithm behind real-time bid-ask queries in trading systems, windowed top-K computations in streaming analytics (Flink, Spark Structured Streaming use specialized operators built on this), rate-limiter sliding-window-max-burst implementations, and monitoring systems (Prometheus’s
max_over_timeuses an equivalent structure). The deeper lesson is amortized analysis via the credit method: every element pays O(1) to enter the deque, and that credit covers its eventual eviction — same accounting used in two-pointer problems, KMP, and union-find with path compression. Once you internalize ‘each element does O(1) amortized work across the entire algorithm’, you stop being scared of nested-loop-looking code that’s actually linear. The principal-level extension: when the operation is not just max but any associative function (sum, gcd, matrix-multiply), use the ‘two-stack queue’ trick to get amortized O(1) per query; this is the building block for batched streaming aggregations in production data pipelines.”
Hints — p95 Sliding Window Maximum
-
Brute is O(n·k); target O(n).
-
Monotonic decreasing deque of indices. Deque values (in nums) strictly decreasing from front to back. Front is always the current window’s max.
-
For each i: (a) pop from BACK while
nums[back] <= nums[i](they can never be max again); (b) push i; (c) pop from FRONT iffront <= i - k(out of window); (d) ifi >= k - 1, recordnums[deque[0]]. -
Why O(n)? Each index pushed once and popped at most once → 2n total deque operations.
-
Alternative: max-heap with lazy deletion. Push
(-v, i). Before reading top, pop while top’s index is out of window. O(n log n).
If stuck: see solution.py.
Week 19 — Practical Engineering: Design & Parsing
Month 04 · Week 19 · Theme: Production-style design problems
This week shifts from algorithmic puzzles to engineering judgment: parsing, OOP design, and APIs that real interviewers use as the “second-round system question that’s still coding”.
Why This Week Matters
The bar for Senior+ engineers is no longer “can you solve LC Hard”. It’s “can you design a class hierarchy under time pressure that handles edge cases, scales, and has clean APIs”. This week trains that muscle on five canonical design problems.
Daily Schedule
| Day | Problem | Pattern |
|---|---|---|
| Mon | p96 — Basic Calculator II | LC 227 · stack-based parsing |
| Tue | p97 — Text Justification | LC 68 · greedy line packing + simulation |
| Wed | p98 — Design Circular Queue | LC 622 · fixed-size ring buffer |
| Thu | p99 — Design Log Storage System | LC 635 · range query on timestamps |
| Fri | p100 — Design In-Memory File System | LC 588 · OOP + trie + path parsing |
Readiness Gate
By Friday you should be able to:
- Write a recursive-descent or stack-based expression parser cold.
- Implement Text Justification with all spacing rules right the first time.
- Implement a circular buffer with
front/rear/sizeinvariants. - Design a hierarchical filesystem with
ls/mkdir/addContentToFile/readContentFromFilein 30 min. - Explain time/space tradeoffs for each design and a follow-up that breaks it (concurrency, persistence, scale).
Connection to Interview Loop
These problems appear in the “design but still coding” round at FAANG — typically 45 min, expects working code AND class-design discussion. Bombing the design discussion at Senior+ is the #1 reason candidates get downleveled.
p96 — Basic Calculator II
Source: LeetCode 227 · Medium · Topics: Stack, String Parsing, Operator Precedence Companies (2024–2025): Google (high), Meta (high), Amazon (medium), Microsoft (medium) Loop position: The canonical parsing problem. Tests whether you can hand-roll a tokenizer + precedence-aware evaluator without leaning on
eval.
1. Quick Context
Evaluate a string expression with + - * / (integer division truncating toward zero) and non-negative integers. No parentheses.
Two solutions:
- Stack of operands, current-op deferred: read each number; based on the previous operator, push, subtract, multiply-with-top, or divide-with-top. Final answer = sum of stack.
- Two-pass (tokenize → reduce): tokenize, then collapse
* /first pass, sum second pass.
Follow-up (LC 224): add parentheses → recursive descent or stack-of-stacks.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/basic-calculator-ii/
STOP. 30-min timer. No eval. Handle multi-digit and leading/trailing spaces.
3. Prerequisite Concepts
- Tokenization basics.
- Operator precedence (
*/before+-). - Integer division semantics in Python (
//rounds toward-inf; for truncation toward zero useint(a/b)or sign-careful logic).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Evaluate infix expression with precedence; integer division truncates toward zero.
2. Clarify: Negative results from 7-3*2? Yes, 1. Division truncation: -3 / 2 → -1, not -2.
3. Examples: "3+2*2" → 7. " 3/2 " → 1. " 3+5 / 2 " → 5. "14-3/2" → 13.
5. Brute: Python eval (forbidden) or shunting-yard.
6. Target: O(n) time, O(n) space (stack of operands).
7. Pattern: Single-pass deferred-operator stack.
8. Develop: see solution.py.
9. Test: all four ops; multi-digit; leading/trailing spaces; embedded spaces; single number; chained * /; truncation of negative quotients.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The “deferred operator” pattern
Track prev_op (initially +) and cur_num (the number being built). When you hit an operator or end-of-string, apply prev_op to cur_num against the stack:
+:stack.append(cur_num)-:stack.append(-cur_num)*:stack.append(stack.pop() * cur_num)/:stack.append(int(stack.pop() / cur_num))
Then prev_op = current operator, cur_num = 0.
Final answer = sum(stack).
Why it works: +/- are stored as-is (commute with addition). *// reduce immediately with the previous operand (higher precedence). At end, the stack contains only additively-combinable terms.
6.2 Integer division — truncation toward zero
Python’s // rounds toward -inf: -3 // 2 == -2. LC requires truncation: -3 / 2 → -1. Use int(a / b) (float division then truncate) or -(-a // b) when a < 0.
In real C/Java/Rust, integer / already truncates toward zero. This Python-specific gotcha is a common interview trap.
6.3 With parentheses (LC 224)
Two clean approaches:
- Recursive descent: parse_expr / parse_term / parse_factor, where
parse_factorrecurses on(. - Stack with sign accumulator: maintain running
resultandsign; on(, push (result, sign); on), pop and combine.
6.4 General expression parsing — shunting yard
Dijkstra’s shunting-yard algorithm converts infix to postfix (RPN) using two stacks (output + operator). Evaluate postfix with a single operand stack. The deferred-operator trick is a specialized inlining for the 4-op case.
6.5 Family: parsing / expression eval
- LC 227 Basic Calculator II (this).
- LC 224 Basic Calculator (with parens, +/-).
- LC 772 Basic Calculator III (+ parens + */).
- LC 770 Basic Calculator IV (symbolic).
- LC 394 Decode String (stack-based parsing).
- LC 726 Number of Atoms (recursive descent on chemistry).
7. Anti-Pattern Analysis
eval(s)— disallowed; also unsafe in production.- Python
//for negative quotients — wrong; must truncate toward zero. - Process operator immediately when seen — forces a state machine that fails on
3 + 5 * 2 / 4. - Forget to flush
cur_numat end-of-string — last number missed. - Skip whitespace ad-hoc — clean is
if c.isspace(): continueorif c == ' 'early-continue. - Build
cur_numwithint(c)thencur_num * 10 + int(c)is right; using string concatcur_num + cthenint(cur_num)works but allocates.
8. Skills & Takeaways
- Deferred-operator stack pattern.
- Truncation-toward-zero division in Python.
- Tokenization while scanning.
9. When to Move On
- Solve cold in 20 min.
- Extend to parens (LC 224) cleanly.
- Articulate shunting-yard generalization.
10. Company Context
- Google: L5; expects clean parser + correctly handles negative truncation.
- Meta: E5; favorite for “design but coding” round; often extended to parens mid-interview.
- Amazon: L6.
- Microsoft: L62/L63.
Strong: “Single-pass scan with a prev_op deferred-operator stack: +/- push signed, *// reduce against top. Final = sum(stack). O(n) time, O(n) space. Python: use int(a/b) to truncate toward zero on division.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Tokenize | Buggy | Cold | Clean | + streaming |
| Precedence | Misses | After hint | Deferred-op | + shunting-yard |
| Truncation | Bug | Fixed after | Cold | + cross-language semantics |
| Parens ext | None | Buggy | Sketch | + recursive descent cold |
12. Level Delta
- Mid (L4): Works on happy path; fails on
-3/2truncation. - Senior (L5): Cold solution + handles edge cases.
- Staff (L6): + extends to parens + cites shunting-yard + production safety note (never
evaluser input). - Principal (L7+): + recursive-descent grammar in BNF + tokenizer/parser separation + recognizes as the foundation of all expression DSLs (formula bars in Excel, query languages, configuration languages).
13. Follow-up Q&A
Q1: Add parentheses.
A1: Stack-of-stacks: on (, push current state (result, sign); on ), pop and combine. Or recursive descent (cleaner).
Q2: Add ^ (exponent, right-associative).
A2: Shunting-yard with associativity flags; or precedence-climbing parser.
Q3: Symbolic variables (e.g., 2*x + 3)?
A3: Build expression tree; defer evaluation; return AST.
Q4: Float arithmetic instead of integer.
A4: Replace int(a/b) with a/b; tokenizer must accept decimal point.
Q5: Why is eval() dangerous in production?
A5: Arbitrary code execution. Even sandboxed, leaks side effects, memory, CPU. Use a real parser.
14. Solution Walkthrough
See solution.py:
calculate— single-pass deferred-operator stack.calculate_shunting_yard— infix → postfix → eval (general).calculate_brute— uses Python AST (oracle).
15. Beyond the Problem
Principal code review: “Basic Calculator II is the smallest non-trivial parser problem. The Staff+ recognition is that you’re building, in miniature, the same machinery that powers every DSL in production: SQL parsers (PostgreSQL, DuckDB), spreadsheet formula engines (Excel, Google Sheets), templating engines (Jinja, Handlebars), configuration languages (HCL, Nginx config), regex compilers, and even shader languages. The lesson scales in three directions: (1) lexer/parser separation (tokenize first, parse second — keeps grammar clean), (2) precedence handling (deferred-op for fixed-precedence ops, precedence-climbing or Pratt parser for extensible grammars, shunting-yard for explicit infix→postfix), (3) evaluation strategy (eager eval vs build-AST-then-walk; the AST approach unlocks optimization passes, partial evaluation, JIT compilation). Production-grade parsers (ANTLR, tree-sitter, LLVM frontend) are ‘just’ large versions of this exercise. The principal-level skill is recognizing when to hand-roll (small grammar, performance-critical, no dependency wanted) vs use a parser generator (large grammar, frequent grammar changes, want LSP/syntax-highlighting support).”
Hints — p96 Basic Calculator II
-
Single-pass scan. Build
cur_numdigit by digit; defer applying the previous operator until you see the next non-digit char (or end). -
Deferred-operator stack: track
prev_op(initially+). When finalizingcur_num: ifprev_opis+/-, push signed; if*//, reduce against stack top in place. -
Final answer =
sum(stack)because*and/already collapsed; only additively-combinable terms remain. -
Truncation gotcha: Python’s
//rounds toward-inf. LC needs truncation toward zero. Useint(a / b). -
With parens (LC 224 / 772): on
(, push current(result, sign); on), pop and combine. Or write a recursive-descent parser.
If stuck: see solution.py.
p97 — Text Justification
Source: LeetCode 68 · Hard · Topics: String, Greedy, Simulation Companies (2024–2025): Google (high), Airbnb (high), Amazon (medium), LinkedIn (medium) Loop position: The “implementation Hard” — algorithmically trivial (greedy line-packing) but brutal on detail. Tests precision under pressure.
1. Quick Context
Given a list of words and a target width maxWidth, format each line:
- Pack as many words per line as fit (with single spaces between).
- Distribute extra spaces evenly among gaps; if uneven, left-leaning slots get one extra.
- Last line is left-justified (single spaces between words, trailing spaces to fill width).
- A line with one word is left-justified.
Output: list of strings, each exactly maxWidth characters.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/text-justification/
STOP. 35-min timer.
3. Prerequisite Concepts
- Greedy line packing.
- Integer division + remainder for distribution.
- String join.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Format paragraphs with full justification, last line left-justified.
2. Clarify: Word longer than maxWidth? LC says guaranteed not. Empty words? No.
3. Examples:
words = ["This","is","an","example","of","text","justification."], maxWidth = 16
→ ["This is an", "example of text", "justification. "]
5. Brute: Brute force isn’t really meaningful here — the problem IS the simulation. Implement straightforwardly; verify with a property-based oracle that re-parses output.
6. Target: O(total chars) time.
7. Pattern: Two phases: (a) greedy pack to find each line’s word range; (b) format each line with spacing rules.
8. Develop: see solution.py.
9. Test: single short line; line with one long word; last line as middle-line edge; exact-fit (sum of words + (k-1) spaces == maxWidth); maxWidth = 1; word.length == maxWidth.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Line-packing greedy
Maintain line_len = sum of word lengths in the current line + (count - 1) for mandatory single spaces. When adding word w would exceed maxWidth (line_len + 1 + len(w) > maxWidth), close the current line.
This greedy is optimal for the LC problem definition. (Note: Knuth-Plass justification — used in TeX — is a global DP optimizing badness; not what LC asks.)
6.2 Even-with-left-leaning space distribution
For a line of k words with total_word_chars total characters and gaps = k - 1 gaps:
- Total spaces to distribute:
maxWidth - total_word_chars. - Base spaces per gap:
total_spaces // gaps. - Leftover:
total_spaces % gaps. Add one extra to the firstleftovergaps.
Edge: k == 1 → no gaps; the whole maxWidth - total_word_chars becomes trailing space.
6.3 Last line — special-cased
Last line is always left-justified: words joined by single spaces, then padded with trailing spaces to width. Same rule applies if k == 1 on any line (whether last or not — but for the last line, the rule is uniform).
6.4 Why this is “easy-Hard”
Algorithm is trivial. The Hard rating comes from edge case density: 5 special cases in 30 lines of code, each silently breakable. Interview signal is precision and discipline, not insight.
6.5 Family: text layout / line breaking
- LC 68 Text Justification (this).
- LC 1531 String Compression II.
- LC 418 Sentence Screen Fitting.
- LC 65 Valid Number (more parsing, similar precision bar).
- Knuth-Plass (TeX) line breaking — DP minimizing sum of squared “badness”.
7. Anti-Pattern Analysis
- Forget last-line is left-justified — fails on the trailing-words test.
- Forget single-word line is left-justified — common bug.
- Distribute extra space evenly using
//without%— last gap gets all the slack, looks wrong. - Use
' ' * (maxWidth - line_len)between every word — collapses spaces incorrectly. - Use
' '.join()then pad — works for last line only; not for full justification. - Track line as a string and append — works but harder to reason about; track word indices instead.
- Off-by-one when checking fit — must account for the implicit space between current last word and the new word.
8. Skills & Takeaways
- Two-phase design: split into “decide line boundaries” + “format one line”.
- Distribute remainder via
divmod. - Implementation Hard requires checklists, not insight.
9. When to Move On
- Implement in 30 min cold with no off-by-ones.
- Cite Knuth-Plass as the global-optimization variant.
10. Company Context
- Google: L4/L5; classic phone screen at this rating.
- Airbnb: loves this as a precision check.
- Amazon: L5.
Strong: “Greedy line-packing (decide where each line ends), then format: distribute maxWidth - total_chars spaces across k - 1 gaps with divmod, left-leaning extras. Last line and single-word lines are left-justified.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Pack | After hint | Cold | Cold | Cold |
| Even distribute | Buggy | Fix after | Cold | + divmod-clean |
| Last line | Forgets | Fixes | Cold | Cold |
| Single-word | Forgets | Forgets | Catches | Catches |
| Family | None | None | None | + Knuth-Plass |
12. Level Delta
- Mid (L4): Works with 1-2 bug fixes in interview.
- Senior (L5): Works first try.
- Staff (L6): + clean two-phase decomposition + cites Knuth-Plass.
- Principal (L7+): + recognizes as line-breaking subproblem in document layout engines (TeX, browsers’ text shapers — HarfBuzz + ICU); + production tradeoff: greedy is fast and predictable; Knuth-Plass produces better typography but is O(n²); browsers use greedy for speed.
13. Follow-up Q&A
Q1: Minimize the total “badness” across all lines (Knuth-Plass).
A1: DP over word indices; dp[i] = min total badness ending line at word i; transition over previous line-start j; O(n²). Used by TeX.
Q2: Right-justify instead of full-justify. A2: Leading spaces instead of trailing; same packing.
Q3: Hyphenation across lines. A3: Pre-compute hyphenation points (TeX dictionary); add hyphen as a virtual word boundary with associated badness.
Q4: Streaming — receive words one at a time, emit lines. A4: Maintain current line buffer; emit when next word doesn’t fit.
Q5: Variable-width fonts (rendering). A5: Width function maps char → pixels; same algorithm but with float widths and a budget.
14. Solution Walkthrough
See solution.py:
full_justify— greedy pack + format._format_line— single-line formatter with all special cases.- Verifier function that re-parses output to confirm round-trip invariants.
15. Beyond the Problem
Principal code review: “Text Justification is document layout in miniature. The greedy approach used here is what browsers, IDEs, and console text renderers use because it’s O(n) and predictable. The Knuth-Plass approach (TeX, 1981) treats line breaking as global optimization: define a ‘badness’ metric per line (deviation from ideal stretch), then minimize the sum across all lines via DP. The result is dramatically better typography — TeX-set documents are visually superior to browser-rendered ones precisely because of this. The Staff+ insight is recognizing the tradeoff: greedy is O(n) and locally optimal; DP is O(n²) but globally optimal. Production systems make this choice deliberately: web browsers (Chrome, Safari) use greedy for performance; PDF generators (LaTeX, ConTeXt) use Knuth-Plass for quality; the in-between (InDesign) lets users choose. The deeper lesson: ‘how good is good enough’ is an engineering judgment, and being able to articulate the spectrum of options between O(n) greedy and O(n²) global-DP is what separates Staff+ from Senior. The same tradeoff appears in compiler instruction scheduling, query planner cost-based vs heuristic optimization, and ML hyperparameter search.”
Hints — p97 Text Justification
-
Two phases: (a) greedy line-packing — decide how many words fit per line; (b) format each line according to rules.
-
Line fit condition:
line_len + 1 + len(next_word) <= maxWidth, whereline_len = sum(word lengths) + (count - 1) for mandatory spaces. -
Even-with-left-leaning distribution:
base, extra = divmod(maxWidth - total_word_chars, k - 1). Firstextragaps getbase + 1; rest getbase. -
Special cases: (a) last line → left-justify (single spaces + trailing pad). (b) single-word line → left-justify (no gaps to fill).
-
Verify by re-parsing output: each line must have length
maxWidthand the words in order must equal the input.
If stuck: see solution.py.
p98 — Design Circular Queue
Source: LeetCode 622 · Medium · Topics: Design, Array, Ring Buffer Companies (2024–2025): Amazon (high), Microsoft (high), Bloomberg (high) Loop position: Foundation of every fixed-size queue in systems software — kernel pipes, audio buffers, lock-free MPMC queues, network packet rings.
1. Quick Context
Implement a fixed-capacity queue with O(1) enQueue, deQueue, Front, Rear, isEmpty, isFull. The “circular” part: when the underlying array’s logical end is reached, wrap to index 0.
State (canonical): arr: List[int], front: int (read index), size: int (count). Rear index = (front + size - 1) % k. Next-write = (front + size) % k.
Alternative: front + rear pointers and a size-disambiguator (or sacrifice one slot to distinguish full from empty).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/design-circular-queue/
STOP. 20-min timer.
3. Prerequisite Concepts
- Modular arithmetic on array indices.
- “Distinguish full from empty” problem (two equal pointers).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Fixed-size FIFO with O(1) ops on a circular array.
2. Clarify: What does enQueue return when full? False. What does Front() return when empty? -1.
3. Examples: k=3; enQ(1),enQ(2),enQ(3) → full; Front()=1, Rear()=3; deQ(); enQ(4) → wraps; Front()=2, Rear()=4.
5. Brute: Use collections.deque(maxlen=k). Works but defeats the point.
6. Target: O(1) every op.
7. Pattern: front + size representation avoids the “full vs empty” ambiguity cleanly.
8. Develop: see solution.py.
9. Test: wrap-around; fill-drain-fill; deQ on empty → False; enQ on full → False.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The “full vs empty” problem
If you store only front and rear pointers (no size), then front == rear is ambiguous: empty OR full. Three resolutions:
- Keep a
sizecounter (this solution). - Sacrifice one slot: treat “full” as
(rear + 1) % k == front. Wastes one slot. - Use an extra “is_full” flag bit.
The size counter is cleanest in software; the “sacrifice one slot” trick is used in hardware (single-producer single-consumer rings where atomic update of two values is impossible).
6.2 Why O(1) matters
In systems software — audio buffers, kernel pipes, packet rings — the queue is on the hot path. An O(n) op (like list.pop(0) in Python, which shifts the entire array) blows the latency budget. Ring buffers give you O(1) FIFO with predictable cache behavior: writes go to a single hot cache line.
6.3 Lock-free MPMC (multi-producer multi-consumer)
Real production queues (Disruptor, boost::lockfree::queue, Linux’s kfifo) are lock-free ring buffers with atomic compare-and-swap on head/tail and memory barriers between writes. The data structure is the same; the concurrency control is the hard part.
6.4 Double-ended (deque) variant
LC 641 Design Circular Deque adds insertFront and deleteLast. Same ring buffer with both ends mutable.
6.5 Family: ring/queue design
- LC 622 Design Circular Queue (this).
- LC 641 Design Circular Deque.
- LC 232 Implement Queue using Stacks (amortized O(1)).
- LC 225 Implement Stack using Queues.
7. Anti-Pattern Analysis
list.pop(0)— O(n); defeats the point.- No wrap (use modulo) — IndexError after k operations.
- Track only
frontandrear, nosize— full-vs-empty ambiguity. enQueuedoesn’t return bool — LC requires bool return.Front()on empty returnsarr[front]— wrong; must return -1.- Use
deque(maxlen=k)— works but trivializes the problem; interviewer wants the manual ring.
8. Skills & Takeaways
- Ring buffer with
front + sizerepresentation. - Full-vs-empty disambiguation.
- O(1) FIFO with cache locality.
9. When to Move On
- Implement cold in 15 min.
- Articulate full-vs-empty resolutions.
- Extend to deque (LC 641).
10. Company Context
- Amazon: L5; “design with code” round staple.
- Microsoft: L62.
- Bloomberg: every Bloomberg systems-y phone screen.
Strong: “Ring buffer with front + size. Next-write index = (front + size) % k. Rear = (front + size - 1) % k. All ops O(1).”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| O(1) ops | After hint | Cold | Cold | Cold |
| Wrap | Forgets | Fixes | Cold | Cold |
| Full vs empty | Confused | Fixes | Cold | + cites variants |
| Concurrency | None | None | Mentions | + Disruptor, kfifo |
12. Level Delta
- Mid (L4): Works after 1 bug fix (off-by-one on rear index).
- Senior (L5): Clean cold + handles all edge cases.
- Staff (L6): + extends to deque + cites the three full-vs-empty resolutions + mentions lock-free variant.
- Principal (L7+): + names production systems (LMAX Disruptor, Linux kfifo, boost lock-free) + discusses memory ordering / barriers / false sharing (padding to avoid head and tail living on the same cache line).
13. Follow-up Q&A
Q1: Thread-safe single-producer single-consumer. A1: Use atomic loads/stores on head and tail with appropriate memory ordering (acquire/release). No locks needed; this is the SPSC ring buffer.
Q2: Multi-producer multi-consumer. A2: CAS on head and tail with seqlock per slot, OR full Disruptor-style with sequence numbers. Significantly harder.
Q3: Make it growable. A3: Allocate new array of 2k; copy elements in logical order (front..front+size); reset front=0. Amortized O(1) like vector.
Q4: Cache-line padding.
A4: Place front and rear on different cache lines (typically 64-byte padding between them) to avoid false sharing in concurrent code.
Q5: Why is deque(maxlen=k) insufficient as an answer?
A5: The interviewer is testing your ability to design the data structure, not your knowledge of stdlib.
14. Solution Walkthrough
See solution.py:
MyCircularQueue—front + sizering buffer.MyCircularQueueDeque—deque(maxlen=k)brute (oracle).- Stress test driving random op sequences against both.
15. Beyond the Problem
Principal code review: “The circular queue is the atomic unit of high-performance messaging. Every system you depend on uses one: Linux kernel
kfifofor inter-process pipes, ALSA for audio device buffers, NVMe submission/completion queues for SSDs, network drivers’ packet rings (e.g., DPDK, io_uring), and userspace concurrent queues (LMAX Disruptor at 100M+ ops/sec, used in financial exchanges). The leetcode problem is the single-threaded skeleton; production complexity is in three orthogonal axes: (1) concurrency (lock-free via atomic head/tail with memory barriers; or full Disruptor with per-slot sequence numbers), (2) cache behavior (pad head and tail to separate cache lines to eliminate false sharing — a single line of pad code can 10× throughput), (3) backpressure semantics (block? drop oldest? drop newest? return error?). The Staff+ recognition is that the data structure itself is a lever: choosing ‘enqueue overwrites oldest on full’ makes it a bounded log (Kafka tail); ‘enqueue blocks’ makes it a flow-control primitive (Go channels, ALSA); ‘enqueue fails’ makes it a back-pressure signal (web server connection queues). Same 30 lines of code; entirely different system semantics.”
Hints — p98 Design Circular Queue
-
State =
arr: List[int]of size k +front: int+size: int. Avoid storingreardirectly — derive it. -
Next write index =
(front + size) % k. Rear index =(front + size - 1) % k. -
Full vs empty disambiguation: with a
sizecounter,size == 0means empty andsize == kmeans full. No pointer-comparison ambiguity. -
Front()/Rear()on empty must return -1 (per LC spec); don’t index intoarr. -
Real-world: this is a ring buffer. Production extensions: lock-free via atomic head/tail (SPSC: easy; MPMC: hard — see LMAX Disruptor); cache-line padding to avoid false sharing.
If stuck: see solution.py.
p99 — Design Log Storage System
Source: LeetCode 635 · Medium · Topics: Design, String, Range Query Companies (2024–2025): Amazon (medium), Bloomberg (medium), Snowflake (low) Loop position: Range-query design problem. Tests whether you recognize that a fixed-format timestamp can be range-queried via string prefix comparison.
1. Quick Context
Two operations:
put(id: int, timestamp: str)— store a log with timestamp formatted"Year:Month:Day:Hour:Minute:Second"(zero-padded:"2017:01:01:23:59:59").retrieve(start: str, end: str, granularity: str) -> List[int]— return IDs whose timestamp falls in[start, end]at the given granularity (one of"Year", "Month", "Day", "Hour", "Minute", "Second").
Key insight: because the format is fixed-width and zero-padded, lexicographic string comparison agrees with chronological comparison. So filtering reduces to comparing fixed-length prefixes.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/design-log-storage-system/
STOP. 20-min timer.
3. Prerequisite Concepts
- Fixed-width zero-padded format → lex order = numeric order.
- Prefix truncation by granularity.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Store (id, timestamp) pairs; retrieve IDs whose timestamps fall in [start, end] at variable granularity.
2. Clarify: Duplicate IDs? Per LC, ids are unique per put. Result order? Any order. Are start/end always inclusive? Yes, inclusive on both ends.
3. Examples:
retrieve("2016:01:01:01:01:01", "2017:01:01:23:00:00", "Year")matches anything with year 2016 or 2017.- Granularity “Hour” → prefix length 13.
5. Brute: Linear scan with substring comparison. This IS the canonical solution; the optimization is the prefix length lookup table.
6. Target: put O(1), retrieve O(n) (acceptable for LC).
7. Pattern: Prefix-length table maps granularity → number of chars to compare.
8. Develop: see solution.py.
9. Test: all 6 granularities; boundary timestamps (exact match start, exact match end); empty store.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why prefix comparison works
The format "YYYY:MM:DD:HH:MM:SS" is:
- Fixed-width — every component is the same number of chars.
- Zero-padded —
"01" < "10"lexicographically AND numerically. - Most-significant-first — Year before Month before Day, etc.
Result: lexicographic comparison agrees with chronological. Truncating to the granularity’s prefix gives correct range queries.
6.2 Granularity → prefix length
| Granularity | Example prefix | Length |
|---|---|---|
| Year | "2017" | 4 |
| Month | "2017:01" | 7 |
| Day | "2017:01:01" | 10 |
| Hour | "2017:01:01:23" | 13 |
| Minute | "2017:01:01:23:59" | 16 |
| Second | "2017:01:01:23:59:59" | 19 |
6.3 Optimization — sorted + bisect
If put is mostly upfront and retrieve is frequent, maintain a sorted list of (timestamp_prefix, id) keyed by truncated timestamp; use bisect_left / bisect_right to find the range in O(log n).
For LC’s constraints (n ≤ 500), the linear scan is fine.
6.4 Production reality — time-series databases
This is a toy version of a time-series database. Real systems (InfluxDB, Prometheus, TimescaleDB) use:
- Inverted index on timestamp for range queries.
- Hierarchical chunking by time bucket (e.g., one chunk per hour).
- Column-oriented storage for compression.
- Write-ahead log + LSM for ingestion.
The Staff-level recognition is: “we’re approximating Prometheus’ tsdb in 30 lines.”
6.5 Family: range query / time series
- LC 635 Log Storage System (this).
- LC 729 My Calendar I.
- LC 731 My Calendar II.
- LC 732 My Calendar III.
- LC 1146 Snapshot Array.
7. Anti-Pattern Analysis
- Parse timestamp into int components — works but unnecessarily complex; string prefix comparison is simpler and equally correct.
- Compare full timestamp at “Year” granularity — fails:
start="2017:..."would not match"2017:05:..."because the full strings differ. <instead of<=— LC range is inclusive.- Forget granularity case sensitivity — LC sends
"Year"exactly (capital Y); use a dict notlower().
8. Skills & Takeaways
- Fixed-width zero-padded format → lex order matches numeric order.
- Prefix truncation is range-query primitive when the format is hierarchical.
9. When to Move On
- Solve cold in 12 min.
- Recognize “lex order = numeric order” as the unlocking insight.
- Articulate sorted + bisect optimization.
10. Company Context
- Amazon: L5; “design simple” warmup.
- Bloomberg: systems round.
Strong: “Fixed-width zero-padded format means lex comparison agrees with chronological. Store as list; on retrieve, truncate start/end to the granularity’s prefix length and string-compare. O(n) per retrieve; sorted + bisect → O(log n + k) if needed.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Insight | None | Hint | Cold | Cold |
| Prefix length | Hardcodes | Table | Clean dict | Clean dict |
| Bisect opt | None | None | Mentions | + production tsdb |
12. Level Delta
- Mid (L4): Works after the “string compare” hint.
- Senior (L5): Cold solution + clean prefix-length dict.
- Staff (L6): + sorted + bisect optimization + cites production time-series DB design.
- Principal (L7+): + chunking strategies, column-oriented storage, retention policies, downsampling.
13. Follow-up Q&A
Q1: Optimize retrieve to sub-linear.
A1: Maintain sorted list of (timestamp, id); on retrieve, bisect_left(start_prefix) and bisect_right(end_prefix + '~') (the ~ is the high sentinel above ‘:’ and digits in ASCII… actually use something like a generous high char or compute the prefix’s successor).
Q2: Support non-ISO formats.
A2: Either canonicalize on put, or parse to integer epoch on put and store both.
Q3: Billions of logs. A3: Real time-series DB: chunked storage per time bucket, secondary index on tags, compression (delta + Gorilla for floats), retention policies, downsampling.
Q4: Concurrent put and retrieve.
A4: RWLock on the store; or COW snapshot for readers.
14. Solution Walkthrough
See solution.py:
LogSystem— linear scan with prefix dict.LogSystemSorted— bisect-based variant.- Stress test driving random puts and retrieves against both.
15. Beyond the Problem
Principal code review: “Log Storage System is a time-series database in 30 lines. The pedagogical magic is the zero-padded fixed-width format that lets lexicographic comparison stand in for chronological — this is the same trick used in Cassandra’s clustering columns, DynamoDB sort keys, Bigtable row keys, and HBase rowkeys: encode time as a sortable string prefix, and range queries become byte-level range scans. The Staff+ recognition is that this is the foundation of every column-family / wide-column NoSQL store: choose your row key carefully (often time-prefixed) and your reads become sequential disk scans, your inserts become sequential writes, your hot-partition problem is solved by salting the prefix. The same insight powers Prometheus tsdb (chunks indexed by time bucket), InfluxDB (TSI index on tags + timestamp), and TimescaleDB (hypertables auto-partitioned by time). When designing data layouts, the question is always: what’s the natural sort order, and can I encode it as a byte-comparable prefix? If yes, you get O(log n) range queries for free from any sorted store (B-tree, LSM, skiplist). This problem is the smallest non-trivial instance of that principle.”
Hints — p99 Design Log Storage System
-
The timestamp format
"YYYY:MM:DD:HH:MM:SS"is fixed-width and zero-padded — so lexicographic string comparison agrees with chronological comparison. -
Granularity → prefix length: Year=4, Month=7, Day=10, Hour=13, Minute=16, Second=19. Store in a dict.
-
On
retrieve(start, end, gran), truncatestartandendto the prefix length, then for each stored timestamp checkstart_pref <= ts[:n] <= end_pref. -
Inclusive on both ends — use
<=not<. -
Optimization: store a sorted list and use
bisectfor O(log n + k) retrieves. For LC’s n ≤ 500, linear scan is fine.
If stuck: see solution.py.
p100 — Design In-Memory File System
Source: LeetCode 588 · Hard · Topics: Design, OOP, Trie, Path Parsing Companies (2024–2025): Google (high), Amazon (high), Meta (medium), Microsoft (high) Loop position: The canonical “design + OOP” interview — operating-system-flavored, tests both your ability to model hierarchical data and to write clean code at scale.
1. Quick Context
Implement an in-memory filesystem with four operations:
ls(path)→ if path is a directory, return its sorted child names; if a file, return[filename].mkdir(path)→ create the directory; intermediate directories must be auto-created.addContentToFile(path, content)→ create file if absent; append content.readContentFromFile(path)→ return file contents.
Paths are Unix-style: absolute, separated by /, no .. or ..
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/design-in-memory-file-system/
STOP. 35-min timer.
3. Prerequisite Concepts
- Trie / n-ary tree where each node is a directory or file.
- Path splitting and traversal.
- OOP: minimal abstraction, no over-engineering.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Hierarchical FS with ls/mkdir/append/read.
2. Clarify: Are file and directory names disjoint per parent? Yes. Is path “/” valid for ls? Yes — returns root’s children. Append semantics: concat to existing content.
3. Examples:
ls("/")→[]mkdir("/a/b/c")→ okaddContentToFile("/a/b/c/d", "hello")ls("/")→["a"]ls("/a/b/c")→["d"]readContentFromFile("/a/b/c/d")→"hello"
5. Brute: Use a flat dict[str, str] for files and dict[str, list[str]] for directories. Works but messy.
6. Target: All ops O(path-length + result-size).
7. Pattern: Trie of Node objects. Each node carries children: dict[name, Node] and content: str (empty string = directory).
8. Develop: see solution.py.
9. Test: root ls; mkdir of nested path; ls of file; append accumulates; mkdir intermediate.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Node design — file or directory in one type
class Node:
children: dict[str, Node]
content: str # "" if directory, else file content
The empty-content-means-directory convention works because empty files aren’t a real LC test case. Cleaner alternative: separate Dir and File classes, or a single class with an is_file: bool flag. All three are acceptable; pick one and stay consistent.
6.2 Path parsing
parts = path.split("/")
# Drop the leading empty string from "/a/b/c" → ["", "a", "b", "c"]
# Keep only nonempty parts.
parts = [p for p in parts if p]
ls("/") produces parts == [], which means “operate on root”.
6.3 Traverse-or-create helper
def _walk(self, parts, create=False) -> Node:
node = self.root
for p in parts:
if p not in node.children:
if not create:
return None # or raise
node.children[p] = Node()
node = node.children[p]
return node
This pattern unifies all four ops:
ls: walk without create; inspect node.mkdir: walk WITH create.addContentToFile: walk WITH create, append.readContentFromFile: walk without create, return content.
6.4 Why this is “design Hard”
The algorithm is trivial. The Hard rating is API surface + correctness: 4 methods, several edge cases each, easy to write 100 lines of buggy code. Senior+ signal is showing you can write 30 clean lines with one _walk helper.
6.5 Family: hierarchical / trie design
- LC 588 In-Memory File System (this).
- LC 211 Add and Search Word (regex-trie).
- LC 1166 Design File System (path → value mapping).
- LC 642 Design Search Autocomplete System.
- Real: Linux VFS, ZFS, FUSE userspace filesystems.
7. Anti-Pattern Analysis
- Separate flat dicts for files and directories — must keep them in sync; error-prone.
isinstancechecks everywhere — symptom of poor data model; unify with one node type.- No
_walkhelper — open-coded traversal in each method — code duplication; bugs creep in. addContentToFileoverwrites instead of appends — read the spec carefully.lson file returns directory listing — must return[basename]when path is a file.- Forget
path.split("/")returns leading empty string — off-by-one on root.
8. Skills & Takeaways
- Trie-of-nodes for hierarchical data.
- Single
_walk(create)helper unifies all ops. - “File or directory in one node” pattern (or its alternatives).
9. When to Move On
- Implement in 25 min cold.
- Explain why one node type is cleaner than two.
- Articulate where a real FS goes beyond (inodes, permissions, journaling).
10. Company Context
- Google: L5; “design + code” is the bar.
- Amazon: L6 onsite.
- Microsoft: Windows team uses this exact problem.
Strong: “Trie of nodes; each node has children dict and content string (empty = dir). A single _walk(parts, create) helper drives all four ops. Path: split on / and drop empties. ls returns sorted children, or [basename] if the node is a file.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Data model | Flat dicts | Mixed | Trie cold | Trie + helper |
| Code quality | Duplicated | Some dup | Clean helper | Single helper + tests |
| ls on file | Misses | Fixes | Cold | Cold |
| Real FS | None | None | Mentions | + inodes, journaling |
12. Level Delta
- Mid (L4): Works with some code duplication.
- Senior (L5): Clean trie +
_walkhelper. - Staff (L6): + extends to symlinks/permissions sketch + cites real FS (Linux VFS, FUSE).
- Principal (L7+): + discusses inode vs dentry separation (one inode can have multiple paths via hardlinks), journaling for crash recovery (ext4, XFS), CoW snapshots (ZFS, Btrfs), and FUSE for userspace FSes.
13. Follow-up Q&A
Q1: Add rm and mv.
A1: rm: walk to parent, delete from children. mv: walk to source parent, remove; walk to dest parent (create=True), insert. Atomic only if no concurrent writers.
Q2: Symbolic links.
A2: New node type Symlink(target_path). On traversal, dereference (with cycle detection, max depth like Linux’s 40).
Q3: Permissions.
A3: Each node gets mode, owner, group. Each op checks against current process identity.
Q4: Make it crash-safe (journaling). A4: Log every metadata change (creates, deletes, renames) to a write-ahead log; replay on startup. ext4 / NTFS use this.
Q5: Multi-threaded. A5: Per-node RWLock; or copy-on-write directory snapshots; or path-prefix lock hierarchy with strict ordering to avoid deadlock.
Q6: Hardlinks. A6: Split node into inode (data + refcount) and dentry (name pointing to inode). Multiple dentries can point to the same inode.
14. Solution Walkthrough
See solution.py:
Node— unified file/directory node.FileSystem— four ops via single_walkhelper.FileSystemBrute— flat-dicts oracle.- Stress test driving random sequences against both.
15. Beyond the Problem
Principal code review: “Design In-Memory File System is operating-systems-101 in 50 lines. The LeetCode version is toy; production filesystems add three orthogonal axes of complexity: (1) Persistence + crash safety: every metadata mutation goes through a journal (ext4’s jbd2, XFS’s metadata log, NTFS’s
$LogFile) so a power loss leaves the FS recoverable. (2) Concurrency: VFS uses per-inode locks, dcache RCU readers, andi_mutexfor atomic ops; getting this right is what makes a production FS hard. (3) Storage layer: extents/blocks/B-trees (ext4 uses HTree dir index, XFS uses B+ trees, ZFS uses Merkle trees for end-to-end checksumming). The Staff+ recognition is inode/dentry separation — the data is decoupled from the name. This is what enables hardlinks (multiple names, one data), bind mounts (the same data appears at multiple paths), and elegant unlink-while-open semantics (you canrman open file in Linux; the inode persists until the last fd closes). The LeetCode ‘children + content’ model collapses this; a senior interviewer might push you toward the separation as a follow-up. The broader lesson: filesystems are the canonical example of a hierarchical naming service, and the same patterns recur in URI routing, DNS, LDAP, package namespaces (Python imports, npm scopes), and config trees (etcd, Consul KV). The trie-with-helper is the universal data structure for hierarchical naming.”
Hints — p100 Design In-Memory File System
-
Model: a trie of nodes. Each
Nodehaschildren: dict[str, Node]andcontent: str(empty = directory). -
Path parsing:
[p for p in path.split("/") if p]. Root"/"yields[]. -
Single
_walk(parts, create)helper drives all four ops:ls,mkdir,addContentToFile,readContentFromFile. Code duplication is the smell. -
lson a file returns[filename], not directory listing. Checknode.content != "". -
addContentToFileappends, doesn’t overwrite. And it must auto-create intermediate directories.
If stuck: see solution.py.
Month 5 — Hards + Systems
Weeks 20+. Hard-tier coding plus the O(1) data structures that show up in every senior-and-above design round.
Weeks
- Week 20 — Cache & High-Throughput Design: LFU Cache (DLL of frequency buckets), Hit Counter (sliding-window time bucket), Twitter (k-way feed merge), Autocomplete (trie + top-k heap), All O(1) (bucket DLL of equal-count keys).
Why this month
Every Senior+ design loop hands you a problem that looks like a system question but is graded on whether your hot path is O(1). This month drills the canonical doubly-linked-list-plus-hashmap, trie-plus-heap, and bucketed-time-window primitives — and the discipline to refuse any sorted() or heapify() on a per-request path.
Week 20 — Cache & High-Throughput Design
Month 05 · Week 20 · Theme: O(1) data structures behind every production system
This week is the design round that’s really an algorithms round. Every problem here is “build a textbook DS from scratch with O(1) guarantees” — exactly the trap question that separates a Mid-level fumble from a Staff-level reflex.
Why This Week Matters
LRU was Week 4 (p16). Every other cache, rate limiter, ranker, and suggestion engine you’ll ever build derives from the same three primitives this week drills:
- Doubly-linked list + hashmap → eviction in O(1).
- Sorted-by-frequency bucket of nodes → LFU in O(1) (not O(log n)).
- Heap-merge of k sorted streams → personalized timelines in O(k log k).
- Trie + per-node top-k heap → autocomplete in O(prefix + suggestions).
- Pre-bucketed time windows → rate limiting in O(1) amortized.
You will fail the design round if you reach for dict + sorted() on the eviction path. These five problems show you what a real production cache, feed, suggester, or rate limiter looks like — and what the interviewer wants to hear when they ask “now scale it 1000×”.
Daily Schedule
| Day | Problem | Pattern |
|---|---|---|
| Mon | p101 — LFU Cache | LC 460 · doubly-linked-list-of-lists, O(1) |
| Tue | p102 — Design Hit Counter | LC 362 · sliding-window time bucket |
| Wed | p103 — Design Twitter | LC 355 · k-way heap merge of user feeds |
| Thu | p104 — Design Autocomplete System | LC 642 · trie + per-prefix top-k heap |
| Fri | p105 — All O(1) Data Structure | LC 432 · bucket DLL of equal-count keys |
Readiness Gate
By Friday you should be able to:
- Implement LFU in 25 min with both
getandputstrictly O(1) — and explain why themin_freqpointer never needs to scan. - Implement a hit counter with both naive
dequeand bucketedcircular arrayvariants and explain when each wins (single-thread vs. wide-time-range queries). - Implement Twitter’s
getNewsFeedas a k-way heap merge in O(k log n) and articulate the alternative fan-out-on-write tradeoff. - Implement Autocomplete with trie + top-k heap maintained per node, including the input-stream state machine (
'#'resets) — most candidates forget the state. - Implement All O(1) inc/dec/getMax/getMin without a
SortedDict, using bucket DLLs — the canonical “I’ve actually built one of these” signal.
Connection to Interview Loop
These are the system-design-but-really-coding problems at Meta (LFU, Twitter), Google (Autocomplete), Microsoft (Hit Counter), and Bloomberg (All O(1)). Failure mode at Senior+ is always the same: candidate reaches for sorted() or heapify() in the hot path. The interviewer says nothing, marks the candidate down one level, and the loop is over.
The transferable lesson: every O(log n) call on a per-request path is a downlevel signal. If you internalize that this week, you’re playing at Staff caliber.
p101 — LFU Cache
Source: LeetCode 460 · Hard · Topics: Design, Hash Table, Doubly Linked List Companies (2024–2025): Meta (very high), Google (high), Amazon (high), Stripe (medium) Loop position: The canonical “LRU was the warm-up” follow-up. If LRU was Week 4 (p16), this is the Senior+ retort: “now evict by frequency, still O(1).”
1. Quick Context
LFU = Least Frequently Used. When the cache is full, evict the key with the lowest access count. Tie-break by LRU within that count. All ops must be O(1).
State (canonical O(1) design):
key_to_node: dict[key, Node]whereNodeholds(key, val, freq).freq_to_dll: dict[freq, DoublyLinkedList]— each list orders its nodes by recency (head = most recent).min_freq: int— pointer to the current minimum frequency.
The trick that makes this O(1): min_freq never needs to scan. It only increments (on _bump) or resets to 1 (on new put).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/lfu-cache/
STOP. 25-min timer. If you reach for heapq or SortedDict, you’re already wrong.
3. Prerequisite Concepts
- LRU Cache (p16) — the
OrderedDict/ dict-of-DLL pattern. - Doubly-linked list with O(1)
remove(node)given a node pointer. - The “bucket by attribute, LRU within bucket” pattern (also used in p105 All O(1)).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Cap-bounded cache; on full, evict min-frequency key (LRU on tie); get and put are O(1).
2. Clarify: What if capacity=0? All puts are no-ops, get always -1. What does put of existing key do? Updates val AND counts as one access.
3. Examples: cap=2; put(1,1), put(2,2), get(1)→1 [freqs: 1→2, 2→1], put(3,3) → evicts 2 (min freq=1), get(2)→-1.
5. Brute: dict[key, (val, freq, last_tick)]; on evict, scan for min (freq, last_tick). O(n) per op.
6. Target: O(1) per op.
7. Pattern: Two-level index — by frequency, then by recency. min_freq increments naturally.
8. Develop: see solution.py.
9. Test: cap=0; cap=1; ties on freq; min_freq jump when the only freq-1 bucket empties.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why min_freq is O(1) without scanning
min_freq only changes in three places:
- New insert (
putof unseen key) →min_freq = 1always (the new key has freq 1). _bumpempties themin_freqbucket →min_freq += 1(the node bumped from min_freq became min_freq+1; nothing else has frequency strictly less).- Capacity-zero / never-set → undefined; never read.
There is no path where min_freq needs to search for the next non-empty bucket. This is the single insight that makes O(1) work.
6.2 Why we need a doubly-linked list, not just a list of keys
We need O(1) remove(node) given only the node. Singly-linked list requires a parent scan (O(n)). Python’s OrderedDict provides this internally as a dict + DLL hybrid — you can use OrderedDict per frequency bucket and call popitem(last=False) for O(1) LRU eviction.
6.3 LRU vs LFU — production tradeoffs
| Property | LRU | LFU |
|---|---|---|
| One-hit wonders | Pollute the cache | Evicted quickly (good) |
| Suddenly-cold “hot” keys | Evict (responsive) | Sticky (bad — old hot keys hog cache) |
| Implementation | Trivial (OrderedDict) | The above 60 lines |
| Production fix for LFU stickiness | — | TinyLFU / LFU with aging / W-TinyLFU (Caffeine, Linux page cache) |
6.4 W-TinyLFU (Caffeine, the JVM cache library)
Production caches don’t use plain LFU because of the “stickiness” problem. W-TinyLFU uses a small Window LRU in front of an admission filter (Count-Min Sketch estimating frequency) plus the main LFU. New entries enter the window LRU; on eviction from window, they compete against the LFU victim — admission is granted only if the new entry’s estimated frequency exceeds the victim’s. Result: handles bursts AND long-tail well. This is what Java’s Caffeine library does (the de facto JVM cache).
6.5 Family: cache eviction
- LC 146 LRU Cache (p16).
- LC 460 LFU Cache (this).
- LC 588 Design In-Memory File System (p100) — different concern.
- LC 1756 Design Most Recently Used Queue — LRU variant.
7. Anti-Pattern Analysis
- Use a
heapqof(freq, tick, key)— heap ops are O(log n), not O(1). And you can’t update a key’s freq without lazy-deletion gymnastics. Downlevel signal at Senior+. - Use
SortedDict/bisect— same problem; O(log n) per op. - Scan for
min_freq— O(n) on every eviction. - Forget that
putof existing key counts as an access — yourget-then-putsequence breaks LRU tie-break. - Forget LRU tie-break within a frequency — interviewer’s hidden test will hit this.
min_freqnot updated when the min bucket empties — silent wrong eviction.- Don’t handle
capacity == 0— crash on firstput.
8. Skills & Takeaways
- Two-level “bucket by attribute, LRU within bucket” pattern (reused in p105).
min_freqinvariant: monotonic-ish pointer that never scans.- Why O(log n) is a downlevel signal on cache hot paths.
9. When to Move On
- Implement cold in 25 min, all ops O(1), no
heapq/SortedDict. - Articulate why
min_freqis O(1) without scanning. - Name W-TinyLFU as the production fix for LFU stickiness.
10. Company Context
- Meta: L5/E5; appears in the “design but coding” round. Recruiters publish it on the prep list.
- Google: L4/L5; “Show me you can build a real cache. LRU is too easy.”
- Amazon: SDE2/SDE3; OE round.
- Stripe: Reliability/Infra teams; “we cache payment-method tokens — design eviction.”
Strong: “Bucket by frequency, doubly-linked list per bucket, min_freq pointer that only ever increments or resets to 1. Both get and put strictly O(1).”
Weak: “I’ll use a min-heap of (freq, time, key).” — interviewer marks “doesn’t know O(1) is achievable.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| O(1) achieved | After heavy hints | After hints | Cold | Cold |
| Reaches for heap | Yes | Sometimes | No | No (and says why not) |
| LRU tie-break | Forgets | Mentions | Implements | Implements + tests for it |
min_freq invariant | Doesn’t see | Guesses | Articulates | Proves the invariant |
| W-TinyLFU / Caffeine | — | — | Knows of | Cites and contrasts |
12. Level Delta
- Mid (L4): Gets to working LFU using
heapqwith lazy deletion. Works; not O(1). Hire-with-reservations. - Senior (L5): Cold O(1) implementation with DLL-per-freq and correct
min_freq. Articulates LRU tie-break. Strong hire. - Staff (L6): + explains why O(log n) is a downlevel signal on hot paths; + cites W-TinyLFU as the production fix; + benchmarks both with the interviewer.
- Principal (L7+): + discusses admission policy (TinyLFU’s Count-Min Sketch) vs eviction policy; + production load patterns where each wins (one-hit-wonder traffic → favor frequency; bursty hot → favor recency; web cache → hybrid). + mentions that Linux’s page cache is essentially 2-list LRU with promotion (active vs inactive LRU lists), a different attack on the same stickiness problem.
13. Follow-up Q&A
Q1: Make it thread-safe.
A1: Coarse: single mutex around get/put. Better: read-write lock with reader-mostly assumption is wrong here (every get mutates). The realistic answer is shard the cache by hash(key) % N so contention is reduced, with one mutex per shard. Caffeine’s lock-free implementation uses async eviction + ring buffers per thread for get-recording.
Q2: Aging — old “hot” keys stay sticky. Fix? A2: Periodically halve all frequencies (decay). Or use W-TinyLFU with Count-Min Sketch that has built-in reset. Or 2Q (a queue-based LRU variant — Linux page cache).
Q3: Persist across restarts.
A3: Write the key-value AND frequency to a log on put. On restart, rebuild. Caches usually don’t bother — recompute is the point of a cache.
Q4: Distributed — eviction across N nodes. A4: Consistent hashing maps keys to nodes (each node runs local LFU). Frequencies aren’t shared across nodes (too chatty). Total fleet capacity = N × per-node capacity.
Q5: Why not Python OrderedDict per frequency bucket?
A5: You absolutely should — it’s a built-in dict+DLL. popitem(last=False) is the O(1) LRU evict. This solution shows the manual DLL for clarity but the OrderedDict version is what you’d write in 10 minutes in an interview.
14. Solution Walkthrough
See solution.py:
LFUCache— canonical DLL-per-frequency withmin_freqpointer.LFUCacheOrderedDict—OrderedDict-per-frequency variant (concise, idiomatic Python).LFUCacheBrute— O(n) scan oracle for stress testing.- Stress test: random
get/putsequences, compare outputs against the brute.
15. Beyond the Problem
Principal code review: “Plain LFU has a well-known failure mode in production: stickiness. A page that was hot for a week and then went cold sits in the cache forever, blocking the actually-active long tail. Your
min_freqpointer never makes progress because the cold-but-high-frequency keys keepmin_freqat 1 (where the new arrivals live). Every production cache you depend on — Caffeine (JVM), Linux page cache (2-list active/inactive LRU with promotion), Redis LFU mode (uses logarithmic counter with decay), memcached LRU+slab — has solved this exact problem in three families of ways: (1) admission filter (TinyLFU, Count-Min Sketch — decide whether a new entry deserves to evict an existing one), (2) counter decay (Redis halves counters periodically; Linux ages by clock), (3) two-queue / segmented (Linux’s active/inactive lists; 2Q algorithm). The Staff+ signal in this round is that you implement the textbook LFU cleanly AND name at least one of those production patches without prompting. The interviewer is testing whether you’ve ever read the source of Caffeine or the Linux MM subsystem — the answer that gets you the level is yes, and here’s the tradeoff.”
p101 — LFU Cache — Hints
-
The trap: if you reach for
heapqorSortedDict, you’ve already conceded O(log n). Bothgetandputmust be strictly O(1). Re-read the problem; then ask: what data structure gives O(1)remove(node)given a node pointer? Answer: doubly-linked list. -
Two levels of indexing. You need to find a key fast (level 1:
dict[key, node]). When evicting, you need to find the min-frequency key fast (level 2:dict[freq, list_of_nodes]). The level-2 list must be ordered by recency so the tail is the LRU within that frequency. -
The
min_freqpointer never scans. It can only change in two ways: (a) a brand-newputsets it to1; (b) the bucket atmin_freqbecomes empty and the just-bumped node now lives atmin_freq + 1→ somin_freq += 1. Convince yourself there is no other case. -
putof an existing key is an access too. It updates the value AND counts as agetfor tie-breaking. If you forget, your eviction picks the wrong key on the next full insert. -
Edge case the autograder loves:
capacity == 0. Everyputis a no-op; everygetreturns -1. Don’t blow up on the firstput.
If stuck: see solution.py.
p102 — Design Hit Counter
Source: LeetCode 362 · Medium · Topics: Design, Queue, Binary Search Companies (2024–2025): Microsoft (high), Amazon (high), Google (medium), Dropbox (medium), Datadog (medium) Loop position: The rate-limiting / metrics primitive. Every API gateway, observability tool, and DDoS shield has this exact data structure at its core.
1. Quick Context
hit(ts) records a hit at timestamp ts (monotonically nondecreasing seconds). getHits(ts) returns the number of hits in the past 300 seconds (i.e., in window [ts-299, ts]).
Two designs, very different scaling:
- Deque of timestamps — append on hit; popleft while head <
ts - 299. Best when QPS is low; O(n) worst-casehitif a burst tails old. - Circular array of (ts, count) per bucket — 300 buckets indexed
ts % 300.hitincrements one bucket in O(1).getHitssums 300 buckets in O(300) = O(1).
The bucket design is what production rate limiters use (token bucket variant; Stripe’s, Cloudflare’s, AWS API Gateway’s).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/design-hit-counter/
STOP. 20-min timer. Sketch both designs before reading further.
3. Prerequisite Concepts
- Sliding window over time.
collections.dequeO(1) append/popleft.- Circular buffer indexed by
(value % capacity).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Streaming hits with monotonic timestamps; count hits in trailing 300s window.
2. Clarify: Are timestamps guaranteed monotonic? Yes (per LC). Out-of-order? No. Granularity? 1 second. What if the same second is hit 1M times? → the deque blows up; the bucket design handles it in O(1).
3. Examples: hit(1), hit(2), hit(3), getHits(4)=3, getHits(300)=3, getHits(301)=3, getHits(302)=2 (hit at ts=1 dropped).
5. Brute: list[int] of all hits; on getHits, iterate and count. O(n) per query, unbounded memory.
6. Target: O(1) amortized hit; O(1) getHits (bucket design).
7. Pattern: Sliding-window via bucketed time.
8. Develop: see solution.py.
9. Test: burst at same second; bucket-recycling at window boundary; long quiet gap (stale buckets must be ignored, not summed).
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The deque design’s hidden failure mode
hit(ts) is O(1) amortized. But if 1M hits happen at second 100 and then no hits until second 401, the first getHits(401) must popleft all 1M entries in one call → that one query is O(1M). For a latency-sensitive metrics endpoint (P99 matters), this is unacceptable. The bucket design bounds the work per getHits to 300 ops regardless of traffic shape.
6.2 The bucket design’s elegance
Index ts % 300 into a 300-slot array. Each slot stores (stored_timestamp, count). On hit(ts):
- If
bucket[ts%300].timestamp == ts, increment count. - Else, the bucket is stale → overwrite with
(ts, 1).
On getHits(ts):
- Sum
bucket[i].countfor all i wherebucket[i].timestamp > ts - 300.
The “stale” check is what makes it correct across the wrap: a bucket from 300+ seconds ago must not contribute.
6.3 Production: from this to a rate limiter
A rate limiter is “hit counter that returns whether to admit”. Same bucket design. Cloudflare’s rate limiter, AWS API Gateway throttling, GitHub’s REST API limits, Stripe’s idempotency-key dedup window — all run this exact data structure (with finer buckets, e.g., 10 buckets of 1s for a 10s window, to balance accuracy vs memory).
6.4 Sub-second / nanosecond hits — out of scope, but…
LC restricts ts to integer seconds. Real systems use leaky bucket or token bucket with floating-point fill rates. Same algorithm shape; the “bucket” is now a counter that decays continuously rather than discretely.
6.5 Family: time-window aggregates
- LC 362 Design Hit Counter (this).
- LC 933 Number of Recent Calls — deque variant.
- LC 1845 Seat Reservation Manager — different concern.
- LC 1396 Design Underground System — different (paired check-in/out).
7. Anti-Pattern Analysis
- Unbounded list of timestamps — memory leak under load.
getHitsusesbisect_lefton a sorted list — O(log n) is fine for queries buthitbecomes O(n) if you ever need to reclaim memory.- Forget the bucket staleness check — a hit from second 5 still counted at second 405 because
bucket[5 % 300]=bucket[5]still has count from the old hit. - Use
time.time()instead of the passedts— fails the test harness. - Cleanup runs during
hitbut is unbounded — moves the latency fromgetHitstohitbut doesn’t solve it; bucket design bounds both.
8. Skills & Takeaways
- Bucketed sliding window — the rate-limiter primitive.
- Why “amortized O(1)” can still be unacceptable under bursty load.
- Recognizing when memory must be bounded by
window_size, nottotal_events.
9. When to Move On
- Implement both deque and bucket variants cold in 15 min.
- Articulate the burst-then-quiet failure of the deque design.
- Extend to a configurable window (5min, 1hr).
10. Company Context
- Microsoft: Azure SDK team; phone screen staple.
- Amazon: SDE2; “design rate limiter” round.
- Google: L4; “extend to per-user rate limit” follow-up.
- Dropbox / Datadog: observability/SRE roles.
Strong: “Bucket array of 300 slots, indexed ts % 300, each storing (stored_ts, count) with staleness check. Both hit and getHits strictly O(window_size).”
Weak: “Deque of timestamps.” → “OK but what about a burst then a long gap?” → if you stall, you’re a mid.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Bounded memory | After hint | Cold | Cold | Cold |
| Bucket design | — | After hint | Cold | Cold |
| Burst failure mode | — | After hint | Articulates | + cites rate-limiter products |
| Sub-second extension | — | — | Mentions | + leaky/token bucket math |
12. Level Delta
- Mid (L4): Deque solution. Works. Hire.
- Senior (L5): Bucket design cold + articulates the burst-then-quiet failure of deque. Strong hire.
- Staff (L6): + names Cloudflare / AWS API Gateway / Stripe as users of this exact pattern; + designs per-user rate-limit extension (sharded buckets per
(user_id, time_bucket)). - Principal (L7+): + discusses distributed rate limiting: Redis with TTL’d buckets, eventual consistency tradeoffs, Lua scripts for atomicity, and “burstable” tokens (token bucket as a more accurate model than fixed-window). + Mentions request smoothing via leaky bucket and where in the stack this lives (edge vs gateway vs service mesh).
13. Follow-up Q&A
Q1: 1-hour window instead of 5 minutes? A1: 3600 buckets. Same algorithm; 12× memory. Or: coarser buckets (60-second granularity → 60 buckets, ±60s error).
Q2: Per-user counter.
A2: dict[user_id, BucketArray]. Memory grows with active-user count; evict idle users (LRU on the dict — see p101).
Q3: Distributed.
A3: Redis with INCR + EXPIRE per (user, time_bucket) key. Read = sum of N keys. For atomicity under concurrent ops, use Lua scripts. Cloudflare and AWS WAF do exactly this.
Q4: Out-of-order timestamps.
A4: Bucket design still works if ts falls within [now - window, now]. Older = drop or count toward original bucket if still active.
Q5: Why not a count-min sketch? A5: CMS approximates per-key counts, useful for “hits per user” with bounded memory. Overkill for a single counter. CMS shows up if the question becomes “top-K users by request rate”.
14. Solution Walkthrough
See solution.py:
HitCounter— deque-based (idiomatic for LC; passes all tests).HitCounterBucket— circular array of 300 buckets; production-grade.HitCounterBrute— list of all hits; O(n) per query oracle.- Stress test: random hit + getHits sequences with monotonic timestamps.
15. Beyond the Problem
Principal code review: “What you’ve built is the fixed-window counter — the simplest of three rate-limiter families. Production systems pick differently per concern: (1) fixed-window (this; simple, burst-on-window-boundary problem — a user can fire
2x limitin one second by hitting the end of one window and the start of the next), (2) sliding-window log (the deque variant; accurate, memory-unbounded), (3) sliding-window counter (interpolate between two fixed windows — Cloudflare’s published design — smooth and bounded), (4) leaky bucket (constant outflow; smooths bursts at the cost of latency), (5) token bucket (constant refill; allows burstiness up to bucket size — AWS API Gateway). The Staff-grade answer in this round names the burst-on-boundary issue with fixed-window and reaches for the sliding-window counter as the production compromise. The Principal-grade answer ties it to the edge vs gateway vs service placement: edge (Cloudflare) does coarse fixed-window for DDoS; gateway (Kong, AWS API GW) does token bucket for fairness; service mesh (Envoy) does adaptive concurrency for back-pressure. Same algorithm family; entirely different role per layer.”
p102 — Design Hit Counter — Hints
-
Memory must be bounded by the window, not by total hits. A naive
listgrows forever. A deque is bounded but only aftergetHits(or a cleanup step onhit) actually pops the old entries. -
Two designs cover the spectrum. (a)
dequeof timestamps —hitappend,getHitspopleft-while-stale thenlen. (b) Circular array of 300 buckets indexedts % 300, each(stored_ts, count)—hitis O(1),getHitsis O(300). -
Burst then quiet — the deque’s weak spot. 1M hits at ts=100, then
getHits(401)must popleft all 1M in one call. P99 latency suffers. The bucket design caps work per call regardless of traffic shape. -
Bucket staleness check is mandatory.
bucket[ts % 300]may hold a hit from 300+ seconds ago. On ahit, ifbucket[i].timestamp != ts, overwrite (don’t increment). OngetHits, skip buckets wherestored_ts <= ts - 300. -
Test the boundary.
getHits(300)should still seehit(1).getHits(301)should not. Off-by-one on<vs<=is the dominant bug.
If stuck: see solution.py.
p103 — Design Twitter
Source: LeetCode 355 · Medium · Topics: Design, Hash Table, Heap, Linked List Companies (2024–2025): Meta (high), Twitter/X (very high — namesake), Snap (medium), LinkedIn (high), Bloomberg (medium) Loop position: The K-way merge feed problem. Every social feed, news aggregator, and activity timeline reduces to “merge K sorted streams, take top N”.
1. Quick Context
Build a tiny social network with:
postTweet(userId, tweetId)getNewsFeed(userId)→ 10 most recent tweets from user + everyone they follow.follow(followerId, followeeId)unfollow(followerId, followeeId)
State:
tweets: dict[userId, list[(time, tweetId)]]— append-only, in post order.follows: dict[userId, set[userId]].time: int— monotonic counter.
getNewsFeed = k-way merge of at most K = (1 + #followed) sorted streams, take top 10. Heap-based: O(K log K + 10 log K) = O(K log K).
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/design-twitter/
STOP. 25-min timer.
3. Prerequisite Concepts
- Heap merge of K sorted streams (LC 23 Merge K Sorted Lists, p9-heap-top-k pattern).
- Reverse iteration of per-user tweet lists (newest first).
- Following = adjacency set (self-implicitly followed).
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Mini-Twitter; news feed = top-10 most recent across user + followees.
2. Clarify: Self-follow allowed? No (LC). But user always sees own tweets. unfollow someone you don’t follow? No-op. Re-follow? Idempotent.
3. Examples: user1 posts t5; user2 posts t6; user1 follows user2; user1.feed = [t6, t5].
5. Brute: On getNewsFeed, gather all relevant tweets, sort desc by time, take 10. O(N total tweets log N).
6. Target: O(K log K + 10 log K) where K = followee count + 1.
7. Pattern: Heap-merge K sorted streams.
8. Develop: see solution.py.
9. Test: user follows self (must be no-op or implicit); follow then unfollow then re-follow; getNewsFeed on user with 0 tweets and 0 followees; user with only 3 tweets across feed.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why heap-merge, not sort-everything
If a user follows 1M accounts and each has 10K tweets, sort-everything is 10B-element sort. Heap-merge of 1M streams pulling top 10 is (1M heap init) + (10 × log(1M)) = ~30M ops. The interviewer’s “scale” probe is satisfied only by this.
In practice, “heap init” is also dominated by only ever peeking the newest tweet per followee (lazy: push only the newest per stream initially, pop-then-push-next as you consume).
6.2 Fan-out-on-write vs fan-out-on-read
This solution is fan-out-on-read: feed is computed on getNewsFeed. Production tradeoff:
| Model | Post cost | Read cost | Wins when |
|---|---|---|---|
| Fan-out-on-read | O(1) | O(K log K) | Few reads, many follows, viral users |
| Fan-out-on-write | O(F) F=#followers | O(1) | Many reads, average users |
| Hybrid (Twitter prod) | O(F) for normals, O(1) for celebs | O(K_celebs log K_celebs) + 1 | Mix |
Twitter famously uses hybrid: for users with > ~1M followers (celebs), fan-out-on-write would mean writing 1M timelines per tweet — too expensive. Their tweets are pulled at read time. Normal users fan-out-on-write.
6.3 The “self-tweets included” trap
getNewsFeed(uid) returns tweets from uid AND followees. Half of LC submissions forget the user’s own tweets. The clean fix: when building the candidate set, always include uid itself.
6.4 Time ordering — monotonic counter, not wall clock
Wall-clock time.time() can collide for tweets posted in the same microsecond. The monotonic counter (a single self.time int incremented on each post) is the canonical answer. Production: hybrid (wall_clock_ms, sequence) tuple — Twitter’s Snowflake IDs embed this exactly: 41 bits ms-timestamp + 10 bits machine + 12 bits sequence.
6.5 Family: feed merging
- LC 355 Design Twitter (this).
- LC 23 Merge K Sorted Lists — pure heap merge primitive.
- LC 218 Skyline — heap-merge variant.
- LC 692 Top K Frequent Words — heap top-K.
7. Anti-Pattern Analysis
- Sort all relevant tweets — O(N total log N); fails the scale probe.
- Forget user’s own tweets — most common LC submission bug.
- Use wall-clock
time.time()— collisions; nondeterministic across runs. followsas alist— O(F) forunfollow; should beset.unfollowof self — must be no-op (not an error).- Push entire tweet lists onto heap upfront — wasted work if most are dropped; lazy “push next on pop” is the production pattern.
8. Skills & Takeaways
- K-way heap merge of pre-sorted streams.
- Monotonic counter for deterministic ordering.
- Fan-out-on-read vs fan-out-on-write tradeoff vocabulary.
9. When to Move On
- Implement cold in 25 min, heap-based.
- Articulate the celebrity / fan-out-on-write tradeoff.
- Extend to
getRecentMentions(userId)follow-up.
10. Company Context
- Meta / X / Snap / LinkedIn: Feed team interview staple.
- Bloomberg: Real-time feed / news ticker.
- Reddit: Front-page ranking variant.
Strong: “Per-user tweet lists in post order. getNewsFeed is a 10-element heap merge of K = (followees + 1) streams, lazy push-on-pop. Monotonic time counter.”
Weak: “Concatenate all tweets and sort.” → 1M-follow follow-up kills you.
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| Heap merge | After hint | Cold | Cold | Cold |
| Includes self-tweets | Forgets | Fixes | Cold | Cold |
| Fan-out vocabulary | — | — | Mentions | Articulates tradeoff |
| Celebrity hybrid | — | — | — | Cites Twitter prod |
12. Level Delta
- Mid (L4): Sort-everything works. Survives. Doesn’t pass scale probe.
- Senior (L5): Heap merge cold + correct self-tweets + monotonic counter.
- Staff (L6): + lazy “push-next-on-pop” + articulates fan-out-on-read vs write + describes celebrity-user hybrid.
- Principal (L7+): + discusses timeline materialization (Twitter’s “home timeline” stored in Redis per user, capped at ~800 entries); + read-after-write consistency (your own posts must appear instantly even if Redis hasn’t caught up — query merges materialized + recent self-tweets); + multi-DC issues (your follower fan-out crosses regions); + Snowflake IDs as the canonical monotonic time + machine-shard scheme.
13. Follow-up Q&A
Q1: Scale to 1M followees. A1: Lazy heap-merge already handles it: K=1M but we only pop 10. Initial heap-init is O(K), still dominant. To improve further: persist the timeline (fan-out-on-write).
Q2: Hot user (celebrity) with 100M followers. A2: Don’t fan-out-on-write for celebs. At read, merge celebrity tweets in at request time (hybrid model).
Q3: Delete a tweet.
A3: Add deleted: set[tweet_id]; filter in getNewsFeed. Or remove from per-user list (O(N_user)).
Q4: Trending topics. A4: Count-min sketch + heavy-hitters algorithm (Misra-Gries) over a sliding time window. Different problem; same “streaming top-K” family.
Q5: Read-after-write for own posts.
A5: On getNewsFeed(uid), merge user’s own recent tweets at request time even if the materialized timeline is stale. Production: this is exactly what Twitter does for self-views.
14. Solution Walkthrough
See solution.py:
Twitter— heap-merge feed usingheapqwith lazy push-on-pop.TwitterBrute— gather-all + sort oracle.- Stress test: random sequences of post/follow/unfollow/feed.
15. Beyond the Problem
Principal code review: “The textbook answer is fan-out-on-read with heap merge — that’s what your code does, and it’s the right starting point. But Twitter’s actual production architecture (well documented in their engineering blog) is a hybrid based on followee fan-out distribution: for the median user (< ~100 followers), they fan-out-on-write into per-follower Redis timelines (capped at ~800 entries each, ~600 GB RAM total for the active user set). For celebrities (> ~1M followers), they do fan-out-on-read at request time — writing 100M timelines per tweet from @ladygaga would crash the cluster. The hybrid ‘merge celebrity in at read’ adds latency to celebrity-follower reads but bounds write amplification. The Staff/Principal signal in this round is naming this pull vs push asymmetry and identifying the viral fan-in event (e.g., a tweet by a new account that suddenly goes viral) as the hardest case — you can’t pre-compute timelines for an unknown future celebrity. Bonus signal: knowing that the timeline is itself capped (you don’t get tweets from 2014 in your feed), which means the merge is bounded by
min(800, K × per_user_recent), not unbounded history. Snowflake IDs (their open-sourced monotonic ID scheme: 41 bits ms-time + 10 bits machine + 12 bits sequence) replace yourself.timecounter and solve cross-shard ordering without coordination.”
p103 — Design Twitter — Hints
-
State first.
tweets: dict[user, list[(time, tweet_id)]]appended in post order.follows: dict[user, set[user]]. A single monotonicself.timecounter incremented on eachpostTweet. That’s the whole skeleton. -
Don’t forget the user’s own tweets.
getNewsFeed(uid)merges[uid] + sorted(follows[uid]). The most common LC submission bug is omittinguiditself. -
Top-10 across K streams = heap merge, not sort-everything. Push
(neg_time, user, idx)for the most-recent tweet of each user (start at last index per list) into a max-heap (negate for Python’s min-heap). Pop 10; on each pop, push the next-older tweet from that same user’s list. -
setfor follows, notlist.unfollowandfollowmust be O(1). AlistmakesunfollowO(F) and re-followinsert duplicates. -
unfollow(uid, uid)is a no-op (per LC). Don’t crash. Andunfollowof someone you don’t follow is also a no-op. Useset.discard, notset.remove.
If stuck: see solution.py.
p104 — Design Autocomplete System
Source: LeetCode 642 · Hard · Topics: Design, Trie, Heap, Stream Companies (2024–2025): Google (very high — namesake), Amazon (high), Apple (medium), Yelp (high), Airbnb (medium) Loop position: The trie-plus-heap hard. Every search bar with suggestions runs this exact algorithm. Maps to the question that gets you the L5+ at Google.
1. Quick Context
Stream of characters arrives via input(c). Build up the current query until '#' arrives (signals end of sentence — commit it to history, reset state). After every non-'#' char, return the top 3 sentences that have the current input as a prefix, ranked by (frequency desc, sentence asc lexicographically).
State:
- Trie of sentences, each terminal node weighted by frequency count.
- Current input buffer (
self.buf) + pointer to the current trie node (self.cur). - On
'#': increment the buffer’s count in the trie, reset buf + cur to root. - Otherwise: append char, descend (or mark “off-trie” if no such child).
Two engineering choices for top-3:
- Maintain top-3 at every node on insert —
O(L)per query (just read the node),O(L × log K)per insert where L = sentence length, K = top-K size. - DFS subtree on each query —
O(N_subtree log N_subtree)per query,O(L)per insert.
For LC, both pass. For production scale, (1) wins reads, (2) wins writes. Real systems use per-node top-K heap maintained on insert.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/design-search-autocomplete-system/
STOP. 30-min timer. This is the longest of the week; budget accordingly.
3. Prerequisite Concepts
- Trie (insert + descend by prefix).
- Top-K with a fixed-size min-heap.
- Stream-state machine (input is a stream; reset on
'#').
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Stream char-by-char; after each, return top-3 sentences with current prefix.
2. Clarify: Casing? Case-sensitive. Spaces? Allowed as ordinary chars. Empty query? (after '#') Returns []. Off-trie? Empty list and stay off-trie until '#'.
3. Examples: History [(“i love you”, 5), (“island”, 3), (“ironman”, 2), (“i love leetcode”, 2)]. Stream i, , a, #. After i → [“i love you”, “island”, “i love leetcode”]. After → [“i love you”, “i love leetcode”]. After a → [] (no sentence “i a…”). After # → [] and commit “i a” with freq +1.
5. Brute: Linear scan of all (sentence, freq) on each char.
6. Target: O(L) per input (read-optimized).
7. Pattern: Trie + per-node maintained top-3 list.
8. Develop: see solution.py.
9. Test: off-trie behavior; commit increments count, not duplicates; top-3 tie-break by sentence lex.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 The state-machine you’ll forget
input(c) is stateful. After c arrives:
- If
c == '#': commitself.bufto trie (increment count), thenself.buf = "",self.cur = root. - Else: append
ctoself.buf. Ifself.cur is not None, descend (self.cur = self.cur.children.get(c)). If we go off-trie,self.cur = None— and stay off-trie for the rest of this query.
Half of LC submissions implement input as if it were a one-shot query (return top3(query)) and fail the state machine. The clue: input returns suggestions AFTER each char, which only makes sense if you remember the prefix so far.
6.2 Per-node top-K vs DFS-on-query
| Choice | Read cost | Write cost | When to use |
|---|---|---|---|
| Per-node top-K maintained on insert | O(L) | O(L · K log K) | Production search bars (read-heavy) |
| DFS subtree on each query | O(subtree · log subtree) | O(L) | Write-heavy / mostly-cold dictionary |
| Per-node sorted list (no heap) | O(L) | O(L · K) | Tiny K (3 here); simplest code |
Google’s real autocomplete uses (1) per-node top-K AND tier-loads from disk (top suggestions in RAM, long tail on SSD); the prefix node points to a precomputed list.
6.3 Tie-break: (frequency DESC, sentence ASC)
The comparator is not natural. A (freq, sentence) tuple sorted descending sorts sentences DESCENDING too — wrong. You need: sort by (-freq, sentence). In a min-heap of size 3 (keeping top 3), the heap key is (freq, neg_sentence) so the worst element pops correctly. Or use a custom class with __lt__. This is the bug most candidates hit and don’t notice until the LC harness tells them.
6.4 Stream-friendliness vs query-friendliness
input returning suggestions on every char is a UX choice (Google does it). But it implies the state machine. Compare to a hypothetical suggest(prefix) — stateless, no buf, no cur — which would be trivial. The interview question rewards understanding why the API is shaped this way.
6.5 Family: prefix data structures
- LC 642 Autocomplete (this).
- LC 208 Implement Trie.
- LC 211 Add and Search Word — Data Structure (wildcard).
- LC 720 Longest Word in Dictionary.
- LC 745 Prefix and Suffix Search.
7. Anti-Pattern Analysis
- Stateless implementation —
input(c)treats each char as a one-shot query, ignoringself.buf/self.cur. Off-by-everything. - Wrong tie-break —
sorted(candidates, reverse=True)sorts sentences descending; needskey=(-freq, sentence). - Reset state on every call — defeats the streaming abstraction.
- Off-trie recovery — descending into a missing child should not crash. Mark
self.cur = Noneand return [] for the rest of this query (until'#'). - Re-descend the trie from root on every
input(c)— O(L) per char, total O(L²) per sentence. Use theself.curpointer. - Forget
'#'commit semantics — must increment count of existing sentence or insert with count 1; resets state.
8. Skills & Takeaways
- Trie with per-node augmentation (frequency, top-K).
- Custom tie-break:
(-frequency, sentence). - Stream state-machine pattern.
9. When to Move On
- Implement cold in 30 min with correct state machine + tie-break.
- Articulate the per-node top-K vs DFS-on-query tradeoff.
- Extend to typo tolerance (Levenshtein-1 prefix match).
10. Company Context
- Google: Search team and Cloud Search; “you wrote autocomplete; now we want you to be able to grade an autocomplete implementation in code review.”
- Amazon: Product search (
amazon.comsuggestions); A9 team. - Apple: Spotlight, Maps search.
- Yelp / Airbnb: Place / location autocomplete.
Strong: “Trie with per-node frequency. State machine: input(c) descends one node; ‘#’ commits + resets. Top-3 via per-node maintained list on insert OR DFS-then-sort on query. Tie-break by (-freq, sentence). Off-trie stays off-trie until #.”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| State machine | Misses | Bugs | Cold | Cold |
| Tie-break | Wrong | Fixes | Cold | Cold |
| Off-trie handling | Crashes | After hint | Cold | Cold |
| Per-node top-K | — | — | Mentions | Implements + benchmarks |
| Typo tolerance | — | — | Mentions | Edit-distance + BK-tree |
12. Level Delta
- Mid (L4): State machine works; uses DFS-then-sort. Passes LC. Doesn’t pass scale probe.
- Senior (L5): State machine clean; per-node maintained top-3. Articulates tie-break. Survives scale probe.
- Staff (L6): + names BK-tree / Levenshtein automaton for typo tolerance + describes how Google tiers suggestions (RAM hot prefix → SSD cold). + handles personalization (per-user history weighted in).
- Principal (L7+): + discusses query log mining as the source of suggestion frequencies (Google’s suggestions come from anonymized query logs, not a curated dictionary) + privacy (differential privacy on aggregation) + multilingual (per-locale trie, language detection) + freshness (last-24h trending injected via separate index).
13. Follow-up Q&A
Q1: Top-K instead of fixed 3. A1: Parameterize. Per-node list of size K — bounded. Heap operations are still O(K log K) per insert.
Q2: Typo tolerance — “leetcoke” → “leetcode”. A2: Levenshtein automaton (descend trie + DP state of edit distance, prune branches with distance > threshold). Or BK-tree (precomputed metric tree on edit distance).
Q3: Per-user personalization. A3: Per-user trie blended with global trie by weight. Or per-user “boost” map applied on top-K read.
Q4: 1B sentences — won’t fit in RAM. A4: Tier trie: hot prefixes (~top 100k) in RAM, cold suffixes on SSD. Or shard trie by first 2 chars across machines (consistent hashing on prefix). LSM-tree storage of (prefix → top-K) is what production search uses.
Q5: Live updates (new trending query in last 5 min). A5: Separate hot index updated every minute, merged into top-K at read time. (Like Twitter trending; same shape as the count-min-sketch + heavy-hitters pattern.)
14. Solution Walkthrough
See solution.py:
AutocompleteSystem— trie with per-node frequency, state machine, DFS-on-query top-3.AutocompleteSystemBrute— linear scan oracle.- Stress test: random sentence histories + random char streams interspersed with
'#'.
15. Beyond the Problem
Principal code review: “What you’ve built is the single-machine, single-language, single-user autocomplete. Google’s production autocomplete is the same trie shape but distributed across three orthogonal scaling axes: (1) corpus — query logs reach hundreds of TB per locale; the trie shards by first 2-3 chars over thousands of machines, with consistent hashing for live re-balancing; (2) freshness — a new trending query needs to surface in minutes, not at the next index build; there is a hot in-memory delta index merged into top-K at read time (same architecture as Elasticsearch’s
near_real_timesegments or LSM trees); (3) personalization + privacy — your search history weights suggestions but must satisfy differential privacy on aggregation. The Staff/Principal signal is naming all three axes AND articulating that the query-log mining that produces the suggestions is itself a non-trivial pipeline (de-duplication, spell normalization, language detection, profanity filtering, geo-binning). Bonus signal: knowing that Bayesian inference on click-through-rate dominates raw frequency for suggestion ranking — a high-frequency-but-low-CTR suggestion is demoted. The simple(freq, lex)tie-break in this problem is the first step of a 30-feature ranking model.”
p104 — Design Autocomplete System — Hints
-
It’s a state machine, not a one-shot query.
input(c)builds up a buffer over many calls. You needself.buf: strandself.cur: Optional[Node]as instance state. Reset both on'#'. -
Trie node holds frequency at the terminal (or as
countat every internal sentence-end). Insert a sentence by descending one char at a time, then+= freqon the terminal node’s count field. -
Tie-break is
(-frequency, sentence), not(-frequency, -sentence). Frequencies sort descending, sentences sort ASCENDING on ties. Easiest: collect all candidates from the subtree, sort withkey=lambda x: (-x[1], x[0]), take first 3. -
Off-trie behavior. If
self.curisNone(we’ve already gone off-trie this query), or if the next char has no child, setself.cur = Noneand return[]. Stay off-trie until'#'arrives — don’t try to re-find a matching prefix mid-stream. -
'#'commits the current buffer. Increment count if the sentence already exists; insert with count 1 if new. Then reset state. The buffer can contain spaces (treat them as ordinary characters).
If stuck: see solution.py.
p105 — All Oone Data Structure
Source: LeetCode 432 · Hard · Topics: Design, Hash Table, Doubly Linked List Companies (2024–2025): Bloomberg (very high), Meta (medium), Amazon (medium), Two Sigma (medium), Citadel (medium) Loop position: The week’s capstone DLL-of-buckets puzzle. Generalizes the LFU pattern (p101): bucket by count, DLL of buckets, O(1) extremes.
1. Quick Context
Build a key-value store supporting:
inc(key)— count of key +1 (insert with count 1 if absent).dec(key)— count of key -1; remove key when count reaches 0. (Guaranteed key exists.)getMaxKey()/getMinKey()— any key with the max/min count; “” if empty.
All four operations must be O(1).
State:
key_to_count: dict[key, int].- A doubly-linked list of buckets, each bucket =
(count, set_of_keys_with_that_count). The buckets are kept sorted ascending bycount(low at head, high at tail). count_to_bucket: dict[int, BucketNode]for O(1) bucket lookup.
getMinKey = any key in head.next bucket; getMaxKey = any key in tail.prev bucket. Both O(1).
inc(key): move key from its current bucket to the (count+1) bucket — creating that bucket adjacent if needed.
dec(key): move to (count-1) — or drop if count was 1.
2. LeetCode Link + Attempt Gate
🔗 https://leetcode.com/problems/all-oone-data-structure/
STOP. 30-min timer. This is harder than LFU. Sketch the bucket-DLL on paper first.
3. Prerequisite Concepts
- LFU Cache (p101) — same “bucket by integer attribute, DLL of buckets” shape.
- Sentinel-head/tail doubly linked list.
- O(1)
add/discardonsetfor bucket membership.
4. How to Approach (FRAMEWORK 1–9)
1. Restate: Multiset with counts. O(1) inc, dec, min, max.
2. Clarify: dec is only called on existing keys (per LC). getMin/getMax on empty → “”. Multiple keys at the same count? Return any one.
3. Examples: inc(“a”), inc(“b”), inc(“b”). max=b (2), min=a (1). inc(“a”). Now a=b=2; min and max both return “a” or “b”.
5. Brute: dict[key, count] + min(values) / max(values) per query. O(n) per query.
6. Target: O(1) for all four.
7. Pattern: Bucket DLL ordered by count.
8. Develop: see solution.py.
9. Test: sequence that exercises bucket-creation between existing ones; dec to zero (key removal AND bucket removal if last); inc adjacent.
5. Progressive Hints
See hints.md.
6. Deeper Insight
6.1 Why the buckets must be a DLL (not a sorted list)
We need O(1) insert-after / insert-before / remove given a pointer to a node. A sorted array would be O(log n) for insert position. DLL is the answer; same reason as LRU and LFU.
6.2 Bucket creation is the subtle part
inc("foo") where foo’s current count is c needs to move foo to the c+1 bucket. Cases:
c+1bucket exists adjacent → just move foo.c+1bucket doesn’t exist → create it adjacent to thecbucket, then move.foowas the only key in its oldcbucket → after moving, the old bucket is empty and must be deleted.
dec is symmetric with c-1, plus the “if c was 1, just delete the key” branch.
6.3 Why this is harder than LFU
LFU only needs the min (for eviction). This requires both min and max simultaneously O(1). The DLL gives both as endpoints — but the bookkeeping for adjacency on bucket creation is denser. A common bug: creating a new bucket on the wrong side of the existing one.
6.4 Family: “rank by integer attribute” structures
- LC 460 LFU (p101) — bucket by frequency.
- LC 432 All O(1) (this) — bucket by count.
- LC 716 Max Stack — different pattern (LDLL + max-tracking).
- LC 895 Maximum Frequency Stack — bucket by frequency + stack-per-bucket.
6.5 Production use case: top-K trending words
Real systems use this exact bucket-DLL when they need “top-K most-incremented keys over a window with O(1) updates”. Examples: real-time word counts in stream processors (Heron, Flink), live-vote tallies, slow-query trackers. Combined with a sliding-window decay, it’s the core of the Misra-Gries heavy-hitters algorithm’s data structure.
7. Anti-Pattern Analysis
min(counts)/max(counts)on a dict — O(n) per query.- Use
heapqwith lazy deletion — O(log n) and gets ugly withdec. - Single bucket ordered by
countwith binary search to find position — O(log n). - Forget to delete empty buckets — bucket DLL grows unboundedly; min/max becomes wrong (they sit at an empty bucket).
- Create the new bucket on the wrong side —
incshould create(c+1)AFTERc(toward tail);decshould create(c-1)BEFOREc(toward head). getMinKey/getMaxKeyon empty doesn’t return""— LC test failure.
8. Skills & Takeaways
- Bucket-DLL pattern (generalizes LFU’s “bucket by frequency” to “bucket by any integer attribute, O(1) extremes”).
- Adjacent bucket creation/deletion bookkeeping.
- Why O(log n) is a downlevel signal on this specific problem.
9. When to Move On
- Implement cold in 30 min, all 4 ops O(1), all buckets correctly created/destroyed.
- Articulate why a heap fails.
- Connect to LFU (p101) and heavy-hitters streaming.
10. Company Context
- Bloomberg: Phone + onsite staple. They love the bucket-DLL.
- Two Sigma / Citadel: “Real-time vote tally / order book level count” — same data structure.
- Meta / Amazon: Less common but appears in cache / rate-limit follow-ups.
Strong: “Doubly-linked list of buckets sorted by count. Each bucket holds a set of keys at that count. count_to_bucket map for O(1) lookup. Inc moves a key to the (count+1) bucket (creating it adjacent if needed); dec mirrors. Empty buckets are deleted. All four ops strictly O(1).”
11. Interviewer’s Lens
| Signal | Junior | Mid | Senior | Staff+ |
|---|---|---|---|---|
| O(1) achieved | — | After hints | Cold | Cold |
| Bucket-DLL idea | — | After hints | Cold | Cold |
| Empty bucket cleanup | Forgets | Fixes | Cold | Cold |
| LFU connection | — | — | Mentions | Articulates |
| Misra-Gries link | — | — | — | Cites |
12. Level Delta
- Mid (L4): Heap with lazy deletion. Works (slow). Senior+ marks it as a downlevel.
- Senior (L5): Bucket-DLL cold, all four ops O(1), correct empty-bucket cleanup.
- Staff (L6): + connects to LFU (p101 — same pattern); + names the generalization “bucket by integer attribute, DLL of buckets” and applies it elsewhere; + handles thread-safety strategy.
- Principal (L7+): + names Misra-Gries and Space-Saving heavy-hitters algorithms as the streaming generalization; + discusses memory cost vs accuracy (exact O(distinct keys) vs sketch O(k)); + production placement (used in Flink for top-K trending, in Kafka Streams for windowed aggregations); + how an order book in a financial exchange uses the same bucket-DLL keyed by price level.
13. Follow-up Q&A
Q1: Make it thread-safe.
A1: Coarse lock around all four ops (they’re all short). Fine-grained: per-bucket lock + RW lock on the DLL spine — fragile. Realistic: shard by hash(key) % N and merge results for global min/max (which becomes O(N) for the merge).
Q2: Concurrent inc on the same key.
A2: With a single lock, sequenced. Lock-free is very hard here (the bucket moves are multi-step pointer updates).
Q3: getTopK() for K > 1.
A3: Walk the tail buckets, collecting keys, until K. Worst case O(n) but typically O(K). For large K, return iterator over buckets.
Q4: count(key) query.
A4: key_to_count[key] — O(1) trivially. We already have it.
Q5: Persist across restarts.
A5: Snapshot key_to_count only; rebuild buckets lazily on first access. The bucket structure is derived state.
14. Solution Walkthrough
See solution.py:
AllOne— bucket-DLL withcount_to_bucketmap.AllOneBrute— dict + scan oracle.- Stress test: random inc/dec/getMin/getMax sequences with the dec-precondition respected.
15. Beyond the Problem
Principal code review: “What you’ve built is the exact data structure behind the order book at every financial exchange (price levels = bucket counts; orders = keys; best bid =
getMaxKey, best ask =getMinKey; on every order insert/cancel, O(1) updates AND O(1) BBO read). It is also the data structure behind real-time top-K trending (Twitter trends, YouTube trending, Reddit Hot) when memory permits exact counting — for huge key spaces, Space-Saving (Metwally et al., 2005) and Misra-Gries (1982) are approximations that trade accuracy for O(K) memory regardless of distinct-key count. The Staff/Principal signal is recognizing that this 80-line data structure is the kernel of multiple production systems AND knowing when to swap exact (this) for approximate (Space-Saving) — the threshold is whenO(distinct_keys)memory becomes intolerable. The bucket-DLL family — LRU (p16), LFU (p101), All-O(1) (this) — is one of the three most important patterns in systems coding (the others being the segment tree family and the heap-merge family); mastering all three is a precondition for the L6+ design loop.”
p105 — All O(1) Data Structure — Hints
-
The bucket-DLL is the only path to O(1) extremes. Group keys by count: each bucket =
(count, set_of_keys). Keep buckets in a doubly-linked list sorted ascending by count.getMinKey= any key in head bucket.getMaxKey= any key in tail bucket. Both O(1). -
count_to_bucket: dict[int, BucketNode]is mandatory. Without it,inc(key)would need to scan the DLL to find the bucket at the new count. -
inc(key)is a 3-case move-or-create:- key absent → insert into bucket(1); create bucket(1) at the head of the DLL if missing.
- key at count
c→ move from bucket(c) to bucket(c+1); create bucket(c+1) immediately AFTER bucket(c) if missing. - If the old bucket becomes empty after the move, unlink it from the DLL and delete it from the map.
-
dec(key)mirrorsincwith two extra cases:- key was at count 1 → delete the key entirely.
- else → move from bucket(c) to bucket(c-1); create bucket(c-1) immediately BEFORE bucket(c) if missing.
-
getMinKey()/getMaxKey()on empty must return"". Use a sentinel head and tail;head.next is tailmeans empty.
If stuck: see solution.py.
Phase 0 — Interview Execution Baseline
Target level: Beginner → Easy Expected duration: 1 week (12-week track) / 1–2 weeks (6-month) / 2 weeks (12-month) Weekly cadence: 7 labs + 25 Easy problems applying the framework rigorously
Why This Phase Exists
Most candidates fail interviews not because they lack algorithms knowledge, but because of execution failures: they jump to coding before understanding the problem, panic when stuck, miss obvious edge cases, or communicate so poorly that the interviewer can’t tell whether they actually solved it.
Phase 0 fixes execution. It does not teach algorithms. It teaches you how to use what you already know.
Concepts To Master
1. How Coding Interviews Are Evaluated
Every modern coding interview at Big Tech uses a multi-dimensional rubric. The dimensions are roughly:
- Problem understanding — did you grasp what was asked?
- Approach quality — did you find a reasonable solution?
- Optimality — did you reach the optimal complexity?
- Implementation correctness — does your code actually work?
- Code quality — would your code pass code review?
- Testing — did you verify your solution?
- Communication — could the interviewer follow your thinking?
- Tradeoff awareness — do you understand what you chose and why?
Each is scored ~independently. A “hire” decision typically requires strong scores on most dimensions, not perfect on one. Code that works but is uncommunicated is often a no-hire.
2. How To Communicate While Solving
See ../COMMUNICATION.md in full. The summary: narrate your intent, not your typing. Pause at decision points. Ask before assuming.
3. How To Ask Clarifying Questions
Good questions:
- “Can the input be empty?”
- “Are there duplicates allowed?”
- “What’s the expected size of N — is 10^5 reasonable?”
- “Does the order of output matter?”
- “What should happen on invalid input?”
- “If multiple valid answers exist, can I return any one?”
Bad questions:
- Re-asking what’s already in the problem statement (signals poor reading)
- “What’s the optimal complexity?” (this is your job to derive)
- 15 questions in a row (drip them as relevant)
4. How To Derive Constraints
Constraints dictate the algorithm. Memorize this table:
| N | Acceptable Complexity | Likely Approach |
|---|---|---|
| ≤ 10 | O(N!) or O(2^N · N) | Backtracking, full enumeration |
| ≤ 20 | O(2^N · N) | Bitmask DP, meet-in-the-middle |
| ≤ 100 | O(N^4) | Multi-loop brute, Floyd-Warshall |
| ≤ 500 | O(N^3) | Interval DP, matrix chain |
| ≤ 5,000 | O(N^2) | 2D DP, edit distance |
| ≤ 100,000 | O(N log N) | Sort + scan, heap, segment tree |
| ≤ 1,000,000 | O(N) | Linear scan, hashmap, two pointers |
| ≤ 10^8 | O(log N) | Binary search, math closed form |
| ≤ 10^18 | O(log N) | Binary exponentiation, math |
Rule of thumb: in modern judges, 10^8 simple operations per second is safe; 10^9 is risky.
5. How To Create Examples
The given examples are minimum-coverage. You should construct:
- Trivial: size 0, size 1.
- Boundary: all duplicates, all negatives, all sorted, all reversed.
- Adversarial: max-constraint values, edge of integer range.
- Multi-answer: if multiple valid outputs exist, pick a specific one.
Working through your own examples often reveals the pattern faster than reading the problem 5 more times.
6. How To Identify Edge Cases
Universal checklist (run through this every problem):
- Empty input
- Null input (where applicable)
- Single element
- Two elements
- Duplicates
- Negative numbers
- Maximum constraint values (overflow risk)
- Sorted input
- Reversed input
- Disconnected graph
- Cycle in graph
- Multiple valid answers
- All identical values
- Off-by-one at boundaries
7. How To Start With Brute Force
The brute force is mandatory. Even when you “see” the optimal:
- It anchors correctness — you have a working oracle.
- It’s a starting point to optimize from.
- It’s a fallback if optimization fails.
- It demonstrates you understand the problem at all.
State the brute force in pseudocode within the first 3 minutes.
8. How To Optimize
Optimization checklist:
- Pattern recognition — does this match a known pattern? See Phase 2.
- Repeated work — what does the brute force recompute? That’s your DP / memo target.
- Sortedness — would sorting help? Two pointers, binary search, sweep line.
- Monotonicity — is the answer monotonic in some parameter? Binary search on answer.
- State compression — can the state space be made smaller? Bitmask, prefix sum.
- Math — closed form, modular arithmetic, combinatorics.
- Data structure — would a heap / BST / segment tree change the complexity?
9. How To Prove Correctness
- Greedy: exchange argument — show that any optimal solution can be modified to use the greedy choice without losing optimality.
- DP: state definition, transition, base cases, evaluation order.
- Two-pointer / sliding window: loop invariant.
- Graph algorithms: cite the algorithm’s correctness theorem and verify preconditions hold.
- Math: induction or direct calculation.
10. How To Explain Complexity
State time and space. Mention:
- Whether the bound is tight.
- Amortized vs worst case (e.g., dynamic array push is O(1) amortized, O(N) worst case).
- Assumptions (hash table O(1) average requires non-adversarial input).
- Constants when they matter (e.g., bitset gives 64× speedup over bool array).
11. How To Write Clean Code Under Time Pressure
See ../CODE_QUALITY.md. Quick rules:
- Meaningful names.
- Helper functions for distinct units.
- Validate at the boundary, not in hot loops.
- No globals.
- Standard library used idiomatically.
12. How To Test Manually
- Walk through given example by hand. Record intermediate state.
- Walk through one trivial case (empty / size 1).
- Walk through one adversarial case.
- Find at least one bug before submitting.
13. How To Recover When Stuck
Use the stuck protocol. Restate, brute force, examine constraints, try smaller examples, look for repeated work, look for monotonicity, look for graph modeling, ask for a hint.
14. How To Use Hints Without Failing The Interview
A hint is not a failure. Frozen silence is. Sample phrasing:
“I’ve explored two pointers and sliding window, but I’m having trouble seeing how to handle duplicates without O(N^2). Could you nudge me toward the right family of approach?”
Receive the hint, restate it, commit out loud, resume.
Why These Concepts Matter In Interviews
Algorithms are necessary but not sufficient. Of all rejected candidates with strong algorithm knowledge, the most common failure modes are:
- “Could not communicate clearly” (60%+)
- “Did not test their code” (40%+)
- “Could not articulate complexity” (30%+)
- “Got stuck and couldn’t recover” (30%+)
- “Misunderstood the problem” (20%+)
(Percentages are approximate, based on common interviewer feedback patterns.)
Phase 0 fixes all of these.
Required Problem Categories
Phase 0 problems are not algorithmically hard. The challenge is execution. Use only Easy problems:
- Two Sum
- Reverse String
- Valid Palindrome
- Maximum Subarray (Kadane’s)
- Best Time To Buy And Sell Stock
- Single Number
- Merge Two Sorted Lists
- Linked List Cycle
- Binary Tree Inorder Traversal
- Symmetric Tree
- Maximum Depth Of Binary Tree
- Climbing Stairs
- Move Zeros
- Contains Duplicate
- Intersection Of Two Arrays
- Reverse Linked List
- Valid Parentheses
- Implement Stack Using Queues
- First Bad Version
- Squares Of A Sorted Array
Solve each one applying the full framework. The point is not the answer; it’s the execution discipline.
Recommended Resources
- This curriculum’s FRAMEWORK.md, COMMUNICATION.md, CODE_QUALITY.md
- LeetCode Easy tier (filter by Easy)
- “Cracking the Coding Interview” (Gayle Laakmann McDowell) — chapters 1, 2, 6, 7 (the soft-skills chapters; skip the rest until later)
Hands-On Labs
Complete in order. Each lab uses the strict lab format from the curriculum spec.
- Lab 01 — Problem Statement Reading
- Lab 02 — Constraints Extraction
- Lab 03 — Brute Force First
- Lab 04 — Optimization Pathway
- Lab 05 — Manual Testing
- Lab 06 — Communication
- Lab 07 — Stuck Recovery
Common Mistakes In Phase 0
- Skipping Phase 0 thinking “I already know this stuff” — execution skills are different from knowledge
- Solving Easy problems silently — the whole point is communication practice
- Skipping the brute force because the optimal is obvious
- Skipping the testing step because the code “looks right”
- Not timing yourself — you don’t know your real solving speed until you measure
- Treating clarifying questions as a checklist — they should feel natural and motivated, not robotic
Mastery Checklist
- Solve any LeetCode Easy in 12 minutes including brute force, optimization, testing, complexity
- Restate every problem in your own words without re-reading the prompt
- Ask 3+ relevant clarifying questions on every problem
- Always state the brute force first (even if 10 seconds long)
- Walk through at least one example by hand before submitting
- Explain complexity correctly on every problem
- Find at least one bug pre-submission on at least 30% of problems (this is good!)
- Never go silent for >60 seconds when working a problem
- Recover from being stuck using the stuck protocol within 3 minutes
Exit Criteria
Move to Phase 1 only when:
- Mastery checklist 100% checked
- Completed all 7 labs
- Solved 25 Easy problems applying the full framework
- Recorded yourself solving 3 problems and reviewed the playback for communication quality
- Run a self-mock with one Easy problem you’ve never seen — pass with full framework execution
If any item fails, repeat Phase 0 with another 25 problems. Do not move forward with a weak baseline.
Lab 01 — Problem Statement Reading
Goal
Train the discipline of reading a problem statement deliberately — extracting structure, constraints, examples, and ambiguity — before any solving begins. Most candidate failures begin in the first 60 seconds when the candidate reads superficially and locks in a wrong mental model.
Background Concepts
- Active reading vs passive reading
- Structural parsing of a problem (input shape, output shape, constraints, examples, follow-ups)
- Identifying ambiguity vs underspecification
- Restating in your own words as a comprehension test
Interview Context
In a real interview, the prompt is often delivered verbally with a brief written version. You have ~3 minutes to load the entire problem into working memory before you start engaging. If you misunderstand something here, every subsequent step is wasted. Strong candidates always restate the problem out loud and ask 2–4 clarifying questions before touching anything.
Problem Statement
Given the problem statement below, in 5 minutes:
- Read it twice.
- Restate it in your own words.
- List all constraints (explicit + implicit).
- List 3 ambiguities you’d ask the interviewer about.
- Construct 3 examples beyond the one given.
The problem (use as a fixed text for this lab):
“Given a list of meeting time intervals, determine if a single person could attend all meetings.”
Example:
[[0,30],[5,10],[15,20]]→false.
Constraints
The problem deliberately omits constraint specification. That’s the point.
Clarifying Questions (you should generate)
Examples of good questions to surface from the prompt above:
- Are intervals inclusive on both ends, or
[start, end)? - Can intervals be zero-duration (
[5,5])? - Is the input pre-sorted?
- Are negative times possible?
- What are the realistic bounds on N and on the time values?
- Can two meetings sharing an endpoint be attended (e.g. one ends at 10, next starts at 10)?
- Are there any null or invalid intervals to validate?
Examples (you should generate)
Beyond the given:
[]→true(empty)[[1,5]]→true(single)[[1,5],[5,10]]→ endpoint-sharing case (depends on inclusivity)[[1,5],[2,3]]→ fully nested overlap[[10,20],[1,5]]→ unsorted, non-overlapping[[1,1],[1,1]]→ zero-duration duplicates
Initial Brute Force
Compare every pair: O(N^2). For each pair, check if intervals overlap (max(a.start, b.start) < min(a.end, b.end) for half-open).
Brute Force Complexity
Time O(N^2), space O(1).
Optimization Path
Sort by start time, then walk through and check intervals[i].start >= intervals[i-1].end. Two thoughts emerge during sorting: ties on start, and the inclusivity semantics — both surface back to the clarifying questions. Optimal after the clarification.
Final Expected Approach
Sort + linear scan. O(N log N) time, O(1) extra space (or O(N) if sorting requires a copy).
Data Structures Used
- The interval array (input)
- Sort comparator on start time
Correctness Argument
After sorting by start, two meetings overlap ↔ some adjacent pair overlaps. Proof: if intervals i < j overlap, then for all k with i ≤ k < j, since intervals[k].start ≤ intervals[j].start < intervals[i].end ≤ intervals[k].end (after sort), k and k+1 overlap somewhere too. By induction, adjacent pairs cover all overlap detection.
Complexity
- Time: O(N log N) (sort dominates)
- Space: O(1) auxiliary if in-place sort, else O(N)
Implementation Requirements
For this lab, do NOT implement. Instead produce a written deliverable (described below).
Tests
Not applicable for this lab — written exercise only.
Follow-up Questions
- “What if instead of yes/no, we wanted the number of conflicts?”
- “What if we had to schedule N people across M rooms?” (becomes Meeting Rooms II)
- “What if the meetings stream in one at a time and we want online detection?”
Product Extension
This is “calendar conflict detection” — a real product feature. Ask yourself: what would Google Calendar do for 10,000 events? (Hint: indexed by day → small per-day scan, or interval tree for general queries.)
Language/Runtime Follow-ups
- In Python, sort is
O(N log N)Timsort, stable. Beware ofkey=lambda x: x[0]— closure overhead vsoperator.itemgetter(0). - In Java, sort uses dual-pivot quicksort for primitives, Timsort for objects. Comparator allocation can dominate small inputs.
- In Go,
sort.Sliceis reflection-based and slow; prefersort.Sliceonly when convenient or usesort.Sortwith a typed slice. - In C++,
std::sortis introsort; comparator must be strict-weak-order or behavior is UB.
Common Bugs
- Using
<=instead of<(or vice versa) due to misreading inclusivity - Mutating input when interviewer expects pure function
- Off-by-one on boundary cases (
[1,5]and[5,9]) - Not handling empty input
Debugging Strategy
If a test fails:
- Print pairs where the algorithm decided “overlap” or “no overlap”
- Walk through the smallest failing case by hand
- Check inclusivity assumption — single source of most bugs here
Deliverable For This Lab
In your notebook (or a markdown file beside this lab), write:
- Restatement. A 1–2 sentence paraphrase of the problem in your own words.
- Clarifying questions list. 6+ questions, prioritized.
- Implicit constraints list. What did the prompt fail to specify? (Inclusivity, sortedness, N bounds, etc.)
- Examples list. 8+ examples covering: trivial, boundary, adversarial, multi-answer.
- Brute force pseudocode. O(N^2) approach.
- Optimization sketch. Just one paragraph.
- Self-critique. Where in your reading did you make assumptions that the prompt didn’t justify?
Mastery Criteria
You complete this lab to mastery when:
- You restated the problem without re-reading the prompt
- You produced 6+ clarifying questions, none of which were answered by the prompt
- You found 3+ implicit constraints
- You produced 8+ examples spanning all category types
- You can articulate which of your assumptions were wrong (everyone makes some — the skill is noticing)
Repeat this lab with 3 different problem statements (pick any from LeetCode Easy) before declaring mastery.
Lab 02 — Constraints Extraction
Goal
Train the discipline of converting written constraints into algorithmic targets. Given 1 ≤ N ≤ 10^5, you should immediately think “O(N log N) is the budget, O(N^2) will TLE”.
Background Concepts
- The constraint-to-complexity mapping (FRAMEWORK.md)
- Operations-per-second budget on competitive judges (~10^8 simple ops/sec safe)
- Implicit constraints (memory, integer overflow, recursion depth)
Interview Context
In live interviews, the interviewer often omits explicit constraints, expecting you to ask. Constraints discipline the algorithm choice. Candidates who skip this step often write an O(N^2) algorithm that the interviewer was hoping they’d avoid.
Problem Statement
For each of the 10 problem prompts below, derive:
- The complexity budget
- At least one algorithm family that fits
- At least one approach that does not fit and why
Prompts:
1 ≤ N ≤ 20— count subsets satisfying property X1 ≤ N ≤ 200— shortest path in weighted graph with up to N^2 edges1 ≤ N ≤ 10^4, queries≤ 10^5— range sum1 ≤ N ≤ 10^5— find duplicate1 ≤ N ≤ 5 × 10^5— kth smallest in array1 ≤ N ≤ 10^6, integers up to 10^9 — count of pairs with sum ≤ KN ≤ 10^9, queries Q ≤ 10^5 — count of integers in[1,N]divisible by KT test cases, T ≤ 10^4, each withN ≤ 10^3— pairwise XOR maximum1 ≤ a, b ≤ 10^18— computea^b mod pStream of up to 10^7 elements— top-K running
Constraints
The point of this lab is constraints. Treat each prompt as separate.
Clarifying Questions
For each prompt, list:
- What’s the operation budget?
- Is the time limit explicit (e.g., 1 sec, 2 sec)?
- Is there a memory limit (e.g., 256 MB)?
- Are values within int32 / int64 range?
Examples
Worked example for prompt #4 (N ≤ 10^5, find duplicate):
- Budget: ~10^7–10^8 ops → O(N), O(N log N), O(N · log_max) all fit
- Fits: hashset O(N), sort+scan O(N log N), Floyd’s cycle if input ∈
[1,N] - Doesn’t fit: O(N^2) brute force (10^10 ops)
Initial Brute Force
Not applicable — meta-lab.
Brute Force Complexity
N/A.
Optimization Path
The optimization path here is constraint-to-algorithm mapping. Memorize the table from FRAMEWORK.md.
Final Expected Approach
For each prompt, write your final answer as:
Prompt #K:
Budget: O(...)
Fitting algorithm family: ...
Disqualified approach: ... because budget / memory / overflow / etc.
Data Structures Used
The point is to map N range → DS choice:
N ≤ 20: bitmask, recursion (no DS needed)N ≤ 200: 2D arrays (Floyd, etc.)N ≤ 10^4: O(N^2) DP arrays, simple sortN ≤ 10^5: hashmap, heap, sorted set, segment tree, FenwickN ≤ 10^6: array + linear scan, two pointers, hash, no log factorsN ≤ 10^9: math, binary search on answer, sieve segment
Correctness Argument
The argument here is budget: justify why your chosen algorithm fits within ~10^8 ops/sec * time-limit. Be explicit: N · log N = 10^5 · 17 ≈ 1.7 × 10^6 — comfortably fits 1-second limit.
Complexity
For each prompt, you produce both:
- The budget
- The justification per the table
Implementation Requirements
Written deliverable. No code.
Tests
Not applicable for this lab.
Follow-up Questions
For prompt #6 (10^6 elements, pairs with sum ≤ K):
- “What if values can be negative?” → may need different sort + two-pointer logic
- “What if we want to enumerate the pairs, not just count?” → output limit changes everything
- “What if the array streams in?” → online algorithm needed
For prompt #9 (a, b ≤ 10^18):
- “What if
pis not prime?” → can’t use Fermat’s little theorem inverse - “What if we need
a^bexactly (no mod)?” → impossible for these sizes
Product Extension
In production, “constraint” often means “expected QPS × max payload size × peak time”. A request handler that’s O(payload_size^2) is fine for size 10 but catastrophic for size 10^4. Same intuition as competitive judges, just different vocabulary.
Language/Runtime Follow-ups
- Python: constant factor ~30–100× slower than C++. Halve your effective budget.
N=10^5with O(N^2) is risky in Python. - Java: ~2–4× slower than C++. Beware autoboxing in hot loops.
- Go: ~2× slower than C++. Map operations have higher constants than
unordered_map. - C++: baseline. Use
ios_base::sync_with_stdio(false)for fast I/O. - JavaScript: ~3–10× slower than C++. Avoid object lookups in hot loops; prefer typed arrays.
Common Bugs
- Forgetting Q queries multiplies your budget (10^5 queries × 10^5 per-query = 10^10!)
- Forgetting T test cases (e.g., T = 10^4 with O(N^2) per test, N = 10^3 → 10^10)
- Underestimating constants in Python/JS
- Forgetting recursion depth limits (Python default 1000)
- Forgetting integer overflow at int32 boundary (~2 × 10^9)
Debugging Strategy
When code TLEs:
- Recompute total operations: outer loops × inner work × test cases × queries
- Check the constant factor (string concat in a loop is a classic disaster)
- Profile to find the actual hotspot (often I/O, not algorithm)
Deliverable
For each of the 10 prompts above, write the structured answer block. Compare yours to the table at FRAMEWORK.md.
Mastery Criteria
- Correctly mapped 10/10 prompts to budget within 30 seconds each
- Identified at least one disqualifying approach for each (per query / total ops)
- Recognized the multiplier effect of T test cases / Q queries
- Identified at least 2 prompts where Python/JS would need extra care
Lab 03 — Brute Force First
Goal
Internalize the discipline of always producing a brute force before optimizing — even when the optimal is obvious. The brute force is your correctness oracle, your fallback, and your communication anchor.
Background Concepts
- Brute force as a baseline: it must be correct, even if slow
- Brute force as oracle: cross-check optimal output with brute on random small inputs
- Brute force as recovery: when stuck, you have something to deliver
Interview Context
Many candidates hear a problem, immediately recognize the optimal pattern, and start coding. The interviewer can’t tell whether they understand the problem or just memorized the answer. State the brute force first, even if it takes 30 seconds. Then, “I see this can be optimized to O(N log N) using … — would you like me to go straight to that, or step through a middle approach?”
Problem Statement
For each of the 5 problems below, in 10 minutes total:
- State the brute force in pseudocode
- State its complexity
- State the optimization path (one or two sentences)
Problems:
- Two Sum: find indices
i, jsuch thata[i] + a[j] == target - Maximum Subarray: maximum sum contiguous subarray
- Longest Substring Without Repeating Characters
- Trapping Rain Water: given heights, total water trapped
- Median Of Two Sorted Arrays
Constraints
For each problem, assume N ≤ 10^5 for sizing.
Clarifying Questions
For each problem, ask: “Are we returning the value or the actual subarray/indices?” — this affects implementation but not the brute-force complexity.
Examples
For Two Sum, brute is for i: for j > i: if a[i] + a[j] == target: return [i,j]. Test on [2,7,11,15], target=9 → [0,1].
Initial Brute Force
Per problem:
- Two Sum: O(N^2) double loop
- Maximum Subarray: O(N^3) — three nested loops (i, j, sum) — or O(N^2) with running sum
- Longest Substring No Repeat: O(N^2) or O(N^3) — for each pair (i, j), check if substring has duplicates (set construction is O(N) per pair, so O(N^3))
- Trapping Rain Water: O(N^2) — for each index, find max-left and max-right by scanning
- Median Of Two Sorted Arrays: O(N+M) — merge then take median; even simpler O((N+M) log(N+M)) — concatenate + sort
Brute Force Complexity
As listed above. Be explicit about which O(N^k) you mean.
Optimization Path
- Two Sum: Hashmap of seen values → O(N).
- Maximum Subarray: Kadane’s running sum, reset to 0 if negative → O(N).
- Longest Substring No Repeat: Sliding window with hashset → O(N).
- Trapping Rain Water: Two-pointer with running max-left, max-right → O(N); or precompute prefix/suffix max → O(N) space.
- Median Of Two Sorted Arrays: Binary search on partition → O(log min(N,M)).
Final Expected Approach
In the deliverable, write:
Problem K:
Brute force pseudocode: ...
Brute complexity: O(...)
Optimization sketch: <pattern> → O(...)
Why the optimization works (1 sentence)
Data Structures Used
For each problem, identify the diff between brute force DS and optimized DS — that diff is usually where the optimization lives.
Correctness Argument
For brute force, correctness is by exhaustive enumeration — usually trivial. For the optimized version, the correctness lives in why exhaustive enumeration is unnecessary (e.g., “if a[i] is the answer’s left half, then target - a[i] must be in the seen set”).
Complexity
Stated per problem above.
Implementation Requirements
For this lab, write the brute force only for problems 1, 2, 3 in your preferred language. Run on the given examples. Do not write the optimal version. The exercise is to make the brute force a habit.
Tests
For each implemented brute force:
- Smoke: 1 given example
- Edge: empty / single
- Random: 10 random small inputs (N ≤ 20), check no crash, plausible output
Follow-up Questions
- For Two Sum: what if there can be multiple valid pairs? Return all? First found?
- For Trapping Rain Water: what about 2D version (Trapping Rain Water II)? — different algorithm (heap-based BFS).
- For Median: what if K-th smallest instead of median? — same binary-search-on-partition idea.
Product Extension
In production code review, the brute force is often the only defensible code if you can’t justify the optimal. Reviewers prefer “obviously correct slow code” over “supposedly fast code with a subtle bug” — until the optimization is properly tested.
Language/Runtime Follow-ups
- Python: brute force should still complete for N ≤ 10^3 in <1s. Use it as oracle.
- Java: beware of
Integerboxing inHashMap<Integer, Integer>for Two Sum — measurable slowdown. - C++: brute force in C++ for
N ≤ 5 · 10^3finishes in well under 1s, useful for stress testing.
Common Bugs
- Off-by-one:
for j = i+1vsfor j = i(Two Sum: must j > i unless duplicates allowed) - Maximum Subarray: starting
max_so_far = 0instead of-infinityfails on all-negative arrays - Sliding Window No-Repeat: not removing characters from the set when shrinking the window
- Trapping Rain Water: using
<vs<=when comparing left and right pointers
Debugging Strategy
When the optimal disagrees with the brute on a small input: trust the brute. The brute is by construction correct. Your optimized version has the bug.
This is the stress testing pattern:
- Write brute (slow, correct)
- Write optimal (fast, suspicious)
- Generate random small inputs
- Compare outputs; on mismatch, print the input and inspect
Deliverable
In a notebook:
- For each of the 5 problems, the structured block (brute pseudocode, complexity, optimization sketch).
- For problems 1, 2, 3: real code for the brute force, with smoke + edge + random tests.
- A 3-sentence reflection: which problem was hardest to resist writing the optimal first?
Mastery Criteria
- Wrote a brute force for all 5 in <2 minutes each (verbally or on paper)
- Stated the brute force complexity correctly
- Stated the optimization path in one sentence
- Resisted the urge to write the optimal first
- Used the brute force as oracle in at least one stress test
Lab 04 — Optimization Pathway
Goal
Train the explicit transition from brute force to optimal: identify what the brute force wastes (repeated work, missed monotonicity, overlooked sortedness), then close that gap with a specific technique.
Background Concepts
The optimization checklist (canonical):
- Repeated work → memoize / DP
- Sortedness → two pointers, binary search, sweep line
- Monotonicity → binary search on answer
- Local + global structure → sliding window, prefix sums
- Pattern match to known DS → heap, monotonic stack/queue, segment tree, trie, union find
- State compression → bitmask, hash of state
- Math closed form → combinatorics, modular arithmetic, geometry
- Graph modeling → BFS/DFS/Dijkstra/topo even on non-graph problems
- Randomization / hashing → reservoir sampling, rolling hash
- Approximation / amortization → when exact O(N) is hard, amortize
Interview Context
The interviewer wants to see your thinking process during optimization, not just the answer. Narrate: “The brute is O(N^2) because we recompute the prefix sum on every iteration. If we precompute prefix sums once, each query is O(1) and total is O(N).”
Problem Statement
For each of the 7 brute force descriptions below, identify which optimization checklist item applies, and produce a one-paragraph optimized approach.
Problems:
- Brute: for each i, sum a[i..j] for all j; O(N^2). Goal: answer multiple range-sum queries.
- Brute: for each pair (i, j), check if
a[i] + a[j] = target; O(N^2). Goal: find pair sum. - Brute: generate all subsets, count those with sum K; O(2^N). N up to 30. Goal: count.
- Brute: for each query (l, r), find min in
a[l..r]; O(N) per query, N queries → O(N^2). Goal: range min queries on static array. - Brute: simulate game state recursively; many overlapping subproblems; O(2^depth). Goal: optimal game value.
- Brute: for each i, find next j > i with
a[j] > a[i]; O(N^2). Goal: next-greater-element array. - Brute: binary search would work if we knew the answer was in
[L, R]. The answer is monotonic in some parameter. O(N · K) brute. Goal: find min K satisfying property.
Constraints
Assume N ≤ 10^5 for #1, #2, #4, #6, #7. N ≤ 30 for #3, allowing 2^N/2 meet-in-the-middle. N is the state-space size for #5.
Clarifying Questions
- “Are queries online or offline?” — offline allows different algorithms (Mo’s, offline BIT)
- “Is the array static or updated?” — static allows sparse table, dynamic needs Fenwick / segment tree
Examples
#1: prefix sums → query in O(1). #7: monotonic predicate → binary search the answer in O(log range).
Initial Brute Force
As stated above per problem.
Brute Force Complexity
As stated.
Optimization Path
For each problem, write the checklist item that applies and the optimized approach:
- Sortedness / static structure: prefix sums → O(N) preprocess, O(1) query
- Sortedness / hashing: hashmap of
target - a[i]→ O(N) - State compression: subset-sum DP O(N · K), or meet-in-the-middle O(2^(N/2)) → fits N=30
- Pattern → known DS: sparse table O(N log N) preprocess, O(1) query (static); segment tree if dynamic
- Repeated work: memoization → O(unique states)
- Pattern → monotonic stack: O(N)
- Monotonicity: binary search on answer → O(log range × verify)
Final Expected Approach
For your deliverable: each of the 7 problems gets:
- Checklist item identified
- Optimized algorithm
- Final complexity
- One-line reasoning
Data Structures Used
- Prefix sum array
- Hashmap
- DP table / memo dict
- Sparse table (immutable RMQ) / segment tree
- Monotonic stack
- (Binary search needs no DS)
Correctness Argument
For each optimization, the correctness argument is “the brute force result is preserved because X”:
- Prefix sums:
sum(l..r) = prefix[r+1] - prefix[l]— algebraic identity - Hashmap pair sum: every pair
(i, j)withi < jis examined exactly once when we processjand look uptarget - a[j] - Subset-sum DP: state
(i, sum)covers all subsets ofa[0..i]with given sum - Sparse table: range min over a range of length L is
min(table[k][l], table[k][r - 2^k + 1])wherek = log2(L)— overlap is fine because min is idempotent - Memoization: same input → same output → cached
- Monotonic stack: pop while top is not greater than current; top after popping is the next-greater for popped elements
- Binary search on answer: predicate is monotonic by problem assumption
Complexity
Stated per problem.
Implementation Requirements
Implement #1, #2, #6 in your preferred language. Verify against brute force on random small inputs.
Tests
- Smoke: given example
- Edge: empty, single
- Random: 50 random tests against brute force; on mismatch, dump input
- Large: stress test at constraint maximum, time it
Follow-up Questions
- For #1: what if the array is updated? → Fenwick tree
- For #2: what if k-sum (k > 2)? → recurse to (k-1)-sum with target adjustment, or sort + multi-pointer
- For #4: what about range max? Range gcd? Range sum? → which are idempotent (sparse table) vs which need segment tree
- For #6: what about previous greater? Next smaller? → mirror the stack
- For #7: what about minimum fractional answer? → binary search on real numbers with precision
Product Extension
The optimization checklist is a real code-review tool. When reviewing a colleague’s PR with O(N^2) in a hot path, run through this checklist mentally. 80% of N^2 → N transitions are one of: prefix sum, hashmap, sort + two-pointer, monotonic stack.
Language/Runtime Follow-ups
- Python: prefix-sum approach gets a 5–10× speedup if you use NumPy
np.cumsuminstead of Python list. - Java: monotonic stack with
Deque<Integer>(autoboxing) is slower than a plainint[]with manual top index. - C++:
std::lower_bound/upper_boundgive log-N binary search on sorted vectors with no manual implementation. - JavaScript:
Mapis generally faster than plain object for hashmap when keys are non-string.
Common Bugs
- Prefix sum: off-by-one in
prefix[r+1] - prefix[l] - Two-pointer: forgetting to advance one pointer
- DP: wrong base case or wrong evaluation order
- Sparse table: log table off-by-one
- Monotonic stack: comparing
>vs>=for the next-greater-or-equal variant - Binary search on answer: monotonicity direction reversed
Debugging Strategy
For each optimization, do stress testing: write brute, write optimal, run on 100 random small inputs, compare outputs. This catches off-by-one bugs that survive the given tests.
Deliverable
- The 7-problem structured analysis
- Real implementation + tests for #1, #2, #6
- A reflection paragraph: for which problem was the optimization checklist most useful? Was there a problem where you’d’ve gone the wrong direction without it?
Mastery Criteria
- Correctly identified the optimization checklist item for all 7
- Implemented #1, #2, #6 with passing tests including stress
- Can explain why each optimization preserves correctness, not just that it’s faster
- Found at least 1 bug via stress testing (good — that’s the point)
Lab 05 — Manual Testing
Goal
Train the discipline of manually walking through your code before claiming it’s done — and finding bugs before the interviewer does.
Background Concepts
- Trace tables: writing variable state row-by-row through a loop
- Edge cases as a reflex
- The “rubber duck” verbalization while tracing
- Pre-submission checklist
Interview Context
Strong candidates always test their code by walking through at least one example by hand, vocalizing variable state. The interviewer learns whether you can verify your own work — a critical engineering signal.
Weak candidates write the code and say “I think this works” without verification, then submit and the interviewer finds the bug. Even when the candidate would have caught the bug if they’d traced.
Problem Statement
You are given 4 small functions below, each with at least one bug. For each:
- Construct a trace table for one input
- Identify the bug
- State a fix
- Construct a test case that exposes the bug
Function 1 — is_palindrome(s: str) -> bool:
def is_palindrome(s):
i, j = 0, len(s)
while i < j:
if s[i] != s[j]:
return False
i += 1
j -= 1
return True
Function 2 — binary_search(a: list[int], target: int) -> int:
def binary_search(a, target):
lo, hi = 0, len(a)
while lo < hi:
mid = (lo + hi) // 2
if a[mid] == target:
return mid
elif a[mid] < target:
lo = mid
else:
hi = mid
return -1
Function 3 — reverse_linked_list(head):
def reverse_linked_list(head):
prev = None
curr = head
while curr.next is not None:
nxt = curr.next
curr.next = prev
prev = curr
curr = nxt
return prev
Function 4 — max_subarray(a: list[int]) -> int:
def max_subarray(a):
best = 0
cur = 0
for x in a:
cur += x
if cur < 0:
cur = 0
if cur > best:
best = cur
return best
Constraints
Standard.
Clarifying Questions
- For palindrome: case-sensitive? Treat as is.
- For binary search: input is sorted ascending, no duplicates.
- For reverse linked list: input may be empty.
- For max subarray: input may be all negative.
Examples
For each function, work a non-trivial input by trace table.
Initial Brute Force
N/A — debugging lab.
Brute Force Complexity
N/A.
Optimization Path
N/A.
Final Expected Approach
For each function, identify:
- The bug
- The minimal fix
- The triggering input
Reference answers (DON’T peek until you’ve tried):
-
is_palindrome:
j = len(s)should belen(s) - 1(or change loop toi < jand uses[i] == s[j-1]style — but as written, immediate IndexError on non-empty input). Trigger: any non-empty string. -
binary_search:
lo = midshould belo = mid + 1, otherwise infinite loop. Trigger: target larger thana[lo]but smaller thana[hi-1], e.g.,a=[1,2,3], target=3. Actually wait — let’s check:lo=0, hi=3, mid=1, a[1]=2 < 3, lo=1(correct in the buggy version; mid would still advance). Trya=[1,3], target=3: lo=0, hi=2, mid=1, a[1]=3 → return 1. OK that works. Trya=[1,2], target=2: lo=0, hi=2, mid=1, a[1]=2 → return 1. Trya=[1], target=2: lo=0, hi=1, mid=0, a[0]=1<2, lo=0 → infinite loop. Trigger: target greater than max element. -
reverse_linked_list: loop condition
while curr.next is not Noneskips reversing the last node. Should bewhile curr is not None. Also crashes on empty input (head = None). -
max_subarray: initializing
best = 0fails on all-negative input — returns 0 instead of the largest (least negative) element. Should bebest = -infinityorbest = a[0].
Data Structures Used
- Trace table: a small ASCII grid of variable values per loop iteration
Correctness Argument
A function is correct iff for every valid input it produces the right output. Manual tracing on edge cases is a cheap, reliable way to falsify “I think it works”.
Complexity
N/A.
Implementation Requirements
For each function:
- Build a trace table on paper
- Identify the bug
- Write the fix
- Write 4 unit tests covering the bug and adjacent cases
Tests
For function 1:
assert is_palindrome("") == True
assert is_palindrome("a") == True
assert is_palindrome("aa") == True
assert is_palindrome("ab") == False
assert is_palindrome("racecar") == True
For function 2:
assert binary_search([], 5) == -1
assert binary_search([5], 5) == 0
assert binary_search([1,2,3], 0) == -1
assert binary_search([1,2,3], 4) == -1 # the trigger
assert binary_search([1,3,5,7], 5) == 2
(Build similar for #3, #4.)
Follow-up Questions
- “How would you find this bug in production?” — logs, test failure, customer report
- “How would you prevent this class of bug?” — property-based testing, fuzz testing
- “What’s your debugging strategy when you can’t reproduce locally?” — see phase-10-testing-debugging/
Product Extension
These four bugs are real bug archetypes:
- Off-by-one (#1, #2)
- Loop termination (#2, #3)
- Initialization (#4)
In code review, scan specifically for: array bounds, loop conditions, accumulator initial values, null inputs.
Language/Runtime Follow-ups
- Python: index
s[len(s)]raisesIndexErrorimmediately — defensive crash - C/C++: would silently read garbage and possibly continue. Always be more defensive
- Java:
ArrayIndexOutOfBoundsExceptionlike Python - Go: panic with clear message
Common Bugs
The four bugs themselves are the common bugs:
- Off-by-one in array bounds
- Off-by-one in binary search update
- Loop terminates one iteration early
- Wrong initial accumulator value
Debugging Strategy
- Trace table for the smallest failing input
- Print state every iteration (the printf approach)
- Verbalize the loop invariant — does the code uphold it?
- Step through with a debugger (last resort, not first — debugger usage is slower than tracing for small problems)
Deliverable
- Trace tables for all 4 functions (one per function)
- Bug identification + fix + test suite for each
- A “trace table template” you’ll reuse going forward
Mastery Criteria
- Found all 4 bugs via tracing (without peeking at answers)
- Wrote correct fixes
- Wrote tests that exposed each bug
- Can construct a trace table for an unseen function in <2 minutes
- Adopted “always trace one example before submitting” as a permanent habit
Lab 06 — Communication
Goal
Train explicit, structured communication during a coding interview: narrate intent, signpost transitions, ask before assuming, summarize at decision points.
Background Concepts
- The 6-phase communication structure (see COMMUNICATION.md)
- “Signposting” — short phrases that tell the listener which phase you’re in
- “Thinking out loud” without rambling
- Pause-points and decision-points
Interview Context
In a real interview, silence is your enemy. After ~30 seconds of silence, the interviewer becomes uncertain about your progress. After 90 seconds, they may interrupt — disrupting your thinking. Skilled candidates fill the time with structured narration that costs them little extra cognitive load.
Problem Statement
Record yourself solving 3 problems out loud, applying explicit communication scaffolding. Then transcribe and grade.
Recommended problems (Easy to keep cognitive load low):
- Two Sum
- Valid Parentheses
- Linked List Cycle
For each, follow this script structure:
Phase 1 — Restatement (60–90 sec)
- “Let me restate to make sure I have it: …”
- Ask 2–3 clarifying questions explicitly
Phase 2 — Examples (60 sec)
- “Let me work through an example by hand: …”
- Construct one trivial and one adversarial example
Phase 3 — Brute Force (60–90 sec)
- “The simplest approach would be …”
- “That’s O(…) — clearly not optimal because …”
Phase 4 — Optimization Discussion (90 sec)
- “I notice
; that suggests …” - Explicitly mention the checklist item you matched
Phase 5 — Coding (5–10 min)
- Narrate intent at each block: “Now I’ll initialize the hashmap, then iterate…”
- Pause and verify after each block
Phase 6 — Testing & Closing (2–3 min)
- “Let me trace through the given example…”
- “Edge cases I should check: …”
- “Final complexity: …”
Constraints
Time-box yourself: 12 minutes total per problem. If you’re going over, cut narration first, not coding.
Clarifying Questions
For Two Sum:
- “Can the input be empty?”
- “Can there be duplicates?”
- “Is exactly one valid pair guaranteed?”
- “Can I assume integers, or could they be floats?”
For Valid Parentheses:
- “Are only
()[]{}allowed, or other characters?” - “Does an empty string count as valid?”
For Linked List Cycle:
- “Should I detect the cycle’s start, or just whether it exists?”
- “Can I modify the list?”
Examples
For each problem, narrate one example by hand before coding.
Initial Brute Force
State and narrate.
Brute Force Complexity
State out loud.
Optimization Path
Narrate the transition: “The brute is O(N^2) because we compare every pair. If I use a hashmap, I can check in O(1) whether target - a[i] is already seen, giving O(N).”
Final Expected Approach
State the final approach as a single sentence before coding.
Data Structures Used
State the data structure explicitly: “I’ll use a hashmap from value → index.”
Correctness Argument
State the loop invariant: “After processing index i, the map contains every (value, index) pair from a[0..i].”
Complexity
State at end: time + space + amortized vs worst case.
Implementation Requirements
You must produce:
- Three audio recordings (or written transcripts) of yourself solving the 3 problems with the script structure
- Working code for all three (the bar is correct + clean, not optimal at first try)
Tests
For each problem, state aloud:
- “Smoke test: given example”
- “Edge: empty input”
- “Edge: single element / smallest non-trivial”
- “Adversarial: all duplicates / all negatives / max constraint”
Then trace through at least one example.
Follow-up Questions
After your solution, narrate:
- “If we wanted all pairs instead of one, we’d …”
- “If the input were a stream, we’d …”
- “If memory were tight, we could …”
Product Extension
These same communication patterns scale to design interviews, on-call discussions, and code reviews. The skill is permanent.
Language/Runtime Follow-ups
When narrating, mention language-specific notes:
- “In Python I’m using
dictwhich is O(1) average; with adversarial input collisions it degrades to O(N).” - “I’m using
enumeratefor index + value to keep the code clean.”
Common Bugs (in communication)
- Going silent for >60 seconds. Even saying “I’m trying to figure out the right invariant for the inner loop” is enough.
- Narrating typing instead of intent. “Now I’m typing
for i in range(n)…” — bad. “Now I’ll iterate forward to maintain the prefix…” — good. - Asking too many questions at once. Drip them; ask after motivation appears.
- Restating the prompt verbatim. Paraphrase, demonstrate comprehension.
- Refusing hints when stuck. Frozen silence is far worse than asking for a nudge.
- Forgetting to summarize at the end. “So the final approach is X with O(N) time and O(N) space; tested on the given example and edge cases.”
Debugging Strategy
After recording, listen back and grade yourself on:
- Did I signpost each phase?
- Did I ever go silent for >60 seconds?
- Did I narrate intent or just typing?
- Did I summarize at decision points?
- Did I sound confident or apologetic?
- Was my code-talk congruent with my code? (i.e., did I narrate one thing while coding another?)
Deliverable
- 3 recordings or transcripts (12 minutes each)
- Self-grading rubric per recording (the 6 questions above + score 0–10 per phase)
- A list of 3 phrases / patterns you’ll reuse next time, and 3 you’ll avoid
Mastery Criteria
- Recorded all 3 problems
- Self-graded honestly (most candidates score 4–6/10 on first attempt — that’s expected)
- Identified at least 3 specific things to improve
- On a re-recording of one problem, scored at least 2 points higher
- No silent gaps > 30 seconds in your final recording
- Narration tracked code 90%+ of the time (no divergence)
Lab 07 — Stuck Recovery
Goal
Train the explicit “stuck protocol” so that when you don’t see the optimal in an interview, you don’t freeze — you systematically work through a checklist that has a high chance of unblocking you within 3 minutes.
Background Concepts
- The 11-step stuck protocol (see FRAMEWORK.md)
- The difference between productive struggle and unproductive freeze
- How to ask for a hint without losing the interview
Interview Context
Every candidate gets stuck. The signal in your favor is how you respond. A candidate who gets stuck and applies a visible recovery protocol is a strong signal. A candidate who gets stuck and goes silent is a weak signal regardless of whether they recover.
Problem Statement
Three problems below are deliberately chosen to be above what you can solve cold without a known pattern. For each:
- Apply the 11-step stuck protocol explicitly, narrating each step
- Time-cap each step at 60 seconds
- If you’ve used 8+ steps without progress, ask for a hint (in this lab, “ask for a hint” means: read the hint section at the bottom)
- Continue until solved
- Document which step generated the breakthrough
Problems (medium-difficulty, but you don’t have the pattern yet):
- Container With Most Water: given heights, find two indices
i, jsuch thatmin(h[i], h[j]) * (j - i)is maximum. - Longest Palindromic Substring: find the longest palindromic substring of a string.
- Search In Rotated Sorted Array: given a sorted array rotated at an unknown pivot, find target in O(log N).
(If you already know all 3 cold, replace with 3 unfamiliar mediums.)
Constraints
Standard. Assume N ≤ 10^5.
Clarifying Questions
For each problem, list 3 you’d ask the interviewer.
Examples
Construct 3 examples for each.
Initial Brute Force
For each problem, write the O(N^2) brute force first.
Brute Force Complexity
State.
Optimization Path — Apply The Stuck Protocol
The 11 steps in order:
- Restate the problem. Out loud, in your own words.
- Write the brute force. It is correct, even if slow.
- Examine the constraints. What does N suggest?
- Construct smaller examples. Solve N=2, N=3 by hand.
- Look for repeated work. What does the brute force recompute?
- Look for sortedness or monotonicity. Could sorting help? Is something monotonic?
- Look for symmetry / pattern from a known DS family. Two-pointer? Sliding window? Heap?
- Try graph modeling. Some non-graph problems become BFS/DFS when you reframe.
- Try a math approach. Closed form? Modular arithmetic? Combinatorial identity?
- State the simplest approach you have. Even if not optimal — get it down.
- Ask for a nudge. Phrase: “I’ve explored A, B, and C. I think the answer might involve D-family of approaches but I’m not seeing it cleanly. Could you nudge me?”
Final Expected Approach
For each problem, the recovery protocol leads to:
-
Container With Most Water: Two pointers from both ends, moving the smaller one inward. Step 7 (two-pointer pattern) gets you there. Loop invariant: the optimal answer doesn’t include any pair
(L, R)we’ve already discarded. -
Longest Palindromic Substring: Either expand-around-center (step 4 — by hand on N=3,4,5 you spot the center idea) or DP (step 5 — repeated work on overlapping substrings). Manacher’s is optimal but step 4/5 is enough.
-
Search In Rotated Sorted Array: Modified binary search. Step 6 — “is something monotonic?” — at least one half is sorted. Use that to decide which half to recurse into.
Data Structures Used
- Two pointers (#1)
- Center expansion / DP table (#2)
- Modified binary search (#3)
Correctness Argument
For each, the post-recovery solution has a clear correctness argument:
- The “moving the smaller one” strategy is correct because moving the larger pointer cannot increase
min(h[i], h[j])(still bounded by the smaller) and reduces width. - Center-expansion enumerates every (center, length) pair exactly once.
- The half that is sorted contains target iff
targetis in[lo, mid](or[mid, hi]); otherwise recurse on the other half.
Complexity
- O(N) time, O(1) space
- O(N^2) time (center expansion); Manacher’s is O(N)
- O(log N) time, O(1) space
Implementation Requirements
For each problem, after applying the protocol, implement it and run tests. The deliverable is the protocol log + working code.
Tests
- Smoke: given example
- Edge: N=0, N=1, N=2
- Adversarial: all-equal heights (#1), all-same-char string (#2), no rotation (#3 with sorted input)
- Random: 50 small random inputs vs brute
Follow-up Questions
- “Why didn’t you see the optimal immediately?” — be honest: “I hadn’t internalized the two-pointer pattern yet” or “I tried sweeping linearly and missed the symmetry.”
- “What pattern would you check next time?” — internalize the answer.
- “When would you fail to recognize this in an interview?” — when nervous, or when the problem is wrapped in unfamiliar context.
Product Extension
In production, “stuck on a bug” follows the same shape: restate symptom, find a minimal reproduction, narrow the search, hypothesize cause, verify by experiment, ask a teammate. The protocol generalizes.
Language/Runtime Follow-ups
- Python: for #2, naive O(N^2) center expansion may TLE on
N = 10^5due to constant factors; consider Manacher’s for full credit - Java:
String.substringin older versions copies; in newer versions it shares — beware version differences - C++: raw character arrays beat
std::stringfor hot loops
Common Bugs
- Stuck protocol going too fast — you skip step 4 (small examples) and miss the breakthrough that lives there
- Stuck protocol going too slow — you spend 5 minutes on step 5 instead of moving on
- Not asking for a hint when 11 steps are exhausted (rare, but happens — pride costs the interview)
- Asking for a hint too early (before step 6) — looks weak
Debugging Strategy
If your protocol runs took >15 minutes per problem, you spent too long per step. Cap individual steps at 60 seconds; if no progress, move on.
If you finished without a hint, congratulations — but verify that you didn’t already know the pattern, in which case use a different problem.
Deliverable
- Per-problem stuck protocol log: for each of the 3 problems, a written record of which step you tried, how long, what you tried, did it advance you?
- The breakthrough step annotated.
- Working implementation + tests for each problem.
- Reflection: which 2 steps are weakest for you? (e.g., many candidates skip step 4 — small examples — because it feels too elementary, but it’s often the highest-yield step.)
Mastery Criteria
- Solved all 3 problems with explicit protocol use
- Spent ≤ 15 min per problem (including coding + tests)
- Identified the breakthrough step accurately
- Used at most 1 hint across the 3 problems
- Identified your two weakest protocol steps
- Re-attempted with the weak steps as starting points and noticed faster progress
Hints (peek only after attempting)
Container With Most Water hint: “Think about what happens at the two extreme ends — and which pointer moving inward could possibly improve the answer.”
Longest Palindrome hint: “Every palindrome has a center. How many possible centers are there?”
Search In Rotated Sorted Array hint: “Even though the full array is rotated, one half across any midpoint must be sorted. Can you tell which half?”
Phase 1 — Programming & Data Structure Foundations
Target level: Easy → low-Medium Expected duration: 2 weeks (12-week track) / 4 weeks (6-month track) / 4 weeks (12-month track) Weekly cadence: ~5 labs/week + 30–50 problems applying every data structure under the framework
Why This Phase Exists
Phase 0 fixed your execution — how to read, communicate, derive constraints, brute force, optimize, test, and recover. Phase 1 fixes your vocabulary: the data structures and language-runtime concepts that 95% of every coding interview rests on.
If you cannot, on demand, state the amortized cost of a dynamic-array push, the worst-case behavior of a hash map under adversarial keys, why s += c in a Python loop is O(N²), what happens to your for-loop when you mutate the collection it iterates, and the difference between deep and shallow copy in your language — you do not have a foundation. You have a list of words.
This phase makes the foundation real. Every concept comes with: internal representation, complexity table, memory behavior, language-specific gotchas (Python, Java, Go, C++, JS/TS), interview traps, common bugs, and testing strategy.
What You Will Be Able To Do After This Phase
- Pick the correct data structure for any Easy/Medium problem in under 60 seconds.
- State the worst-case, average-case, and amortized complexity of every operation on every fundamental DS.
- Predict your code’s memory and cache behavior, not just its asymptotic time.
- Write idiomatic code in your interview language without falling into language-specific traps.
- Recognize when a problem is really a hash map / heap / monotonic stack problem in disguise.
- Implement, from scratch, every data structure listed below — without notes — in under 20 minutes each.
Concepts To Master
You must master every item below before moving to Phase 2. Pattern problems in Phase 2 assume fluency with these primitives.
Data Structures
- Arrays — operations, complexity, memory layout, cache behavior, dynamic resizing amortization, gotchas per language
- Strings — immutability, encoding pitfalls, concat-in-loop blowup, substring complexity
- Hash Maps — hashing, collisions, load factor, adversarial inputs, ordered vs unordered, custom hash for tuples
- Hash Sets — operations, set algebra, when to use vs map
- Linked Lists — singly/doubly, sentinels, tail pointer, common manipulation patterns
- Stacks — array-backed vs linked, monotonic stack preview
- Queues — deque, ring buffer, priority queue preview
- Heaps — binary heap, sift up/down, complexity, top-K pattern
- Sorted arrays / sorted sets — binary search, bisect, sorted-set complexity per language
- Trees — binary, BST, traversals iterative + recursive, balanced vs unbalanced
- Tries — operations, space cost, alphabet size considerations
- Graphs — adjacency list, matrix, edge list, when each
- Disjoint Set Union — path compression + union by rank
- Bitsets / Bit Manipulation — set/clear/popcount, common idioms
- Counters / Multisets —
Counter,HashMap+counterpattern, multiset alternatives
Runtime Concepts
- Stack vs heap memory
- Scope and lifetime
- Value vs reference semantics
- Mutable vs immutable
- Hash collisions and adversarial keys
- Iterator invalidation
- Garbage collection basics (refcount vs tracing)
- Memory leaks (especially in GC’d languages)
- Deep vs shallow copy
- Recursion depth and stack overflow
Why These Concepts Matter In Interviews
Most “I knew the algorithm but couldn’t get it to pass” failures aren’t algorithmic. They are foundation failures:
- “My hash map is slow” → adversarial collision pattern in the input.
- “My recursion crashed” → no idea Python’s default recursion limit is 1000.
- “I edited the list while iterating and got weird output” → iterator invalidation.
- “My BFS queue is slow” → using
list.pop(0)instead ofdeque.popleft. - “Java said
ConcurrentModificationException” → the JDK’s fail-fast iterator policy. - “Go map iteration ordered differently every run” → intentional non-determinism for hash-flooding defense.
- “C++ vector reference invalidated after push_back” → reallocation moved the storage.
- “JS object stringified my int keys to strings” → all object keys are strings; use
Map.
Every one of these is on the rubric somewhere as “implementation correctness” or “language fluency.” Phase 1 closes them.
Inline Data Structure Reference
The remainder of this README is a data-structure and runtime reference manual. Read it linearly the first time. Skim it as a reference any time you forget a complexity, language API, or gotcha.
1. Arrays
Internal Representation
A contiguous block of memory holding fixed-size elements indexed by offset. Static arrays have fixed size; dynamic arrays (Python list, Java ArrayList, Go slice, C++ vector, JS Array) wrap a static array and grow it geometrically.
Operations and Complexity
| Operation | Static | Dynamic (avg) | Dynamic (worst) |
|---|---|---|---|
Index read/write a[i] | O(1) | O(1) | O(1) |
Append push_back | n/a | O(1) amortized | O(N) (resize) |
| Prepend / insert middle | n/a | O(N) | O(N) |
| Pop end | n/a | O(1) | O(1) |
| Pop front | n/a | O(N) | O(N) |
| Search (unsorted) | O(N) | O(N) | O(N) |
| Search (sorted, binary search) | O(log N) | O(log N) | O(log N) |
| Length | O(1) | O(1) | O(1) |
Memory Layout & Cache Behavior
Contiguous memory means the CPU prefetcher can stream elements predictively, giving arrays the best cache locality of any data structure. A linear scan over an int[] is typically 5–20× faster than the same scan over a linked list of the same length, even though both are O(N). When constants matter (HFT, hot loops), this difference dominates.
Dynamic Resizing Amortization
Geometric growth (typically 2× or 1.5×) gives O(1) amortized append: doubling means total work to grow from 1 to N is 1 + 2 + 4 + … + N = 2N - 1, amortized to O(1) per push. Linear growth (+1) gives O(N) amortized — never use it.
Language-Specific Gotchas
- Python:
listoverallocates with growth factor ~1.125;arr.pop(0)is O(N); usecollections.dequefor queue. Lists are heterogeneous (each slot is aPyObject*), defeating cache locality. Usearray.arrayor NumPy for primitive-typed storage. - Java:
ArrayListisObject[]— boxing for primitives. Useint[]for hot paths.ArrayList.remove(0)is O(N).Arrays.asList(arr)returns a fixed-size view, not a realArrayList. - Go: Slices are a 3-tuple
(ptr, len, cap).appendmay or may not reallocate — appending to one slice can silently mutate another sharing storage. Always check capacity.s = s[:0]reuses storage;s = nilreleases it. - C++:
std::vector<T>reallocates onpush_backpast capacity, invalidating all iterators and references. Reserve up front when you know the size.vector<bool>is a packed bitset, notbool[]— itsoperator[]returns a proxy. - JS/TS: Arrays are dense or sparse; sparse arrays (
a[1000000] = 1) have terrible perf.arr.shift()is O(N). Holes (new Array(10)) skip duringforEachbut not duringfor.
Common Interview Traps
- “Insert in the middle” — be sure they don’t actually want a different DS.
- “In-place” — explicitly disallows the cheap copy-to-new-array solution.
- Off-by-one at boundaries:
[l, r)vs[l, r].
2. Strings
Internal Representation
An array of code units (bytes for char[], UTF-16 code units in Java/JS, variable-width in Python 3 with PEP 393 latin-1/UCS-2/UCS-4). In most languages, strings are immutable — every “modification” allocates a new string.
Operations and Complexity
| Operation | Complexity |
|---|---|
Index s[i] | O(1) for fixed-width encoding, O(i) for UTF-8 by codepoint |
| Length | O(1) (cached) |
Concat s + t | O(|s| + |t|) — new allocation |
Substring s[l:r] | O(r − l) (copy) in most langs, O(1) (view) in Go and Java pre-7u6 |
| Equality | O(min len) but fast with hash compare first |
| Find substring (naive) | O(NM) |
| Find substring (KMP / Z) | O(N + M) |
Immutability
Java/Python/JS strings are immutable. So this loop:
s = ""
for c in chars:
s += c # O(N) per iteration → O(N²) total
This bug shows up in real interviews and gets candidates dinged for not knowing language internals. Use "".join(chars) (Python), StringBuilder (Java), [].join('') (JS), strings.Builder (Go).
Encoding Pitfalls
- “String length” in characters vs bytes vs grapheme clusters can all differ.
"é"may be 1 codepoint or 2 (e + combining accent) → 2 codepoints, 1 grapheme. - Python
len("😀") == 1; Java"😀".length() == 2(UTF-16 surrogate pair). - JS
"😀".length === 2for the same reason; iterate withfor…ofto get codepoints. - Always clarify: “Are inputs ASCII?” — if no, ask whether the unit of “character” is byte, codepoint, or grapheme.
Substring Complexity
Substring extraction copies the underlying chars in most modern languages. Go strings are immutable byte slices and substring is O(1) view (but converting to/from []byte is O(N)).
Language-Specific Gotchas
- Python: strings are immutable; use lists of chars and
"".join()for building.s[::-1]is idiomatic reverse. - Java:
String s += c→ quadratic. UseStringBuilder.String.intern()exists but rarely needed in interviews. - Go:
stringis immutable bytes;[]byte(s)andstring(b)each allocate. Range over a string yields(i, rune), not(i, byte). - C++:
std::stringis mutable; SSO (small string optimization) keeps short strings on the stack.s.c_str()is null-terminated. - JS/TS: strings are UTF-16 code units; emoji and non-BMP chars are 2 units long.
3. Hash Maps
Internal Representation
A bucket array, indexed by hash(key) % capacity. Collisions resolved by either separate chaining (linked list / tree per bucket — Java since 8 promotes long chains to red-black trees) or open addressing (Python, Ruby — probe sequence in the same array).
Operations and Complexity
| Operation | Average | Worst |
|---|---|---|
| Insert | O(1) | O(N) (all collide) |
| Lookup | O(1) | O(N) |
| Delete | O(1) | O(N) |
| Iterate | O(N + capacity) | O(N + capacity) |
Load Factor
The capacity / size ratio. When load factor exceeds a threshold (~0.75 typical), the table doubles and rehashes — this is amortized O(1) per insert but O(N) for the resize itself.
Adversarial Inputs
A static, public hash function lets an attacker craft N keys that all hash to the same bucket → O(N²) insertion. Real-world history: this brought down Java web servers in 2003 and PHP in 2011. Modern languages (Python PYTHONHASHSEED, Go map randomization, Java tree fallback) defend against this.
In an interview, if the problem says “the input may be adversarial,” do not rely on hash maps for worst-case bounds — use sorting + binary search or a balanced BST.
Ordered vs Unordered
- Insertion-ordered: Python 3.7+
dict, JavaLinkedHashMap, JSMap. - Sorted (by key): Java
TreeMap, C++std::map, Pythonsortedcontainers.SortedDict. - Hash, no order guarantee: C++
std::unordered_map, JavaHashMap(unordered as of 8+), Gomap(intentionally randomized).
If the problem requires ordered iteration, do not use a plain hash map.
Custom Hash for Tuples / Composite Keys
- Python: tuples of hashable items are hashable for free.
- Java: must implement both
equalsandhashCodefor any custom key class. Forgetting one is a top-10 interview bug. - Go: map keys must be “comparable types” — structs of comparables work, slices/maps don’t.
- C++: must specialize
std::hash<T>or pass a custom hasher tounordered_map. - JS: object keys are coerced to strings; use
Mapfor object-keyed maps.
Common Interview Traps
- Mutating a key after insertion (its hash changes; the map can’t find it).
- Iterating while mutating →
ConcurrentModificationException(Java) /RuntimeError(Python). - Assuming O(1) without acknowledging worst case.
4. Hash Sets
Operations and Complexity
Same as hash map (a hash set is conceptually a hash map with null values).
| Operation | Average | Worst |
|---|---|---|
| add / contains / remove | O(1) | O(N) |
| union | O(|A| + |B|) | |
| intersection | O(min(|A|, |B|)) | |
| difference | O(|A|) |
Set Algebra
- Union:
A ∪ B— PythonA | B, JavaA.addAll(B), C++ manual. - Intersection:
A ∩ B— iterate the smaller, lookup in the larger. - Difference:
A \ B— iterate A, skip if in B. - Symmetric difference:
A △ B—(A ∪ B) \ (A ∩ B).
Set vs Map: When To Choose
Use a set when you only need presence; use a map when you need associated value (count, index, parent, etc.). Many “use a set” problems become trivially easier with a map (e.g., Two Sum needs a value → index map, not a set).
Language-Specific Gotchas
- Python:
setandfrozenset; tuples are hashable, lists are not. - Java:
HashSet,LinkedHashSet,TreeSet. - Go: no built-in set — use
map[T]struct{}(zero-byte value). - C++:
std::unordered_set,std::set(sorted). - JS/TS:
Setpreserves insertion order; objects are reference-equal.
5. Linked Lists
Singly vs Doubly
- Singly: each node has
next. Reverse, cycle detection, two-pointer dance. - Doubly: each node has
nextandprev. Required for O(1) erase given an iterator (LRU cache pattern).
Sentinels (Dummy Nodes)
A dummy head node simplifies edge cases: “what if the list is empty?” “what if we delete the first node?” become non-special. Always use a dummy when you have to return a head pointer that may change.
dummy = ListNode(0)
dummy.next = head
prev = dummy
# … operations on prev.next …
return dummy.next
Tail Pointer
Maintaining a tail pointer makes append O(1) (otherwise O(N)). Always-mark whether your manipulation invalidates the tail.
Common Manipulation Patterns
- Reverse iteratively: three pointers
prev,curr,next. - Reverse recursively:
reverse(head.next)thenhead.next.next = head; head.next = None. - Find middle: slow/fast pointers; fast moves 2× slow.
- Detect cycle: Floyd’s tortoise and hare.
- Merge two sorted lists: dummy head + zip pattern.
- Remove Nth from end: lead pointer ahead by N.
Operations and Complexity
| Operation | Singly | Doubly |
|---|---|---|
| Index | O(N) | O(N) |
| Insert at known node | O(1) | O(1) |
| Delete at known node | O(N)* | O(1) |
| Search | O(N) | O(N) |
*Singly: O(N) because you need the previous node; O(1) if you have the previous pointer.
Language-Specific Gotchas
- Python: define
class ListNode: __slots__ = ('val', 'next')for memory; default uses a__dict__. - Java:
LinkedList<T>exists but is rarely the right choice;ArrayDequebeats it on most operations. - Go: standard library has
container/list(doubly linked, generic-erased before Go 1.18). - C++:
std::list(doubly),std::forward_list(singly). - JS: typically write
class Node { constructor(v) { this.val = v; this.next = null; } }.
6. Stacks
Implementations
Array-backed (dynamic array, push/pop end) or linked (push/pop head). Array-backed is faster in practice due to cache locality.
Operations and Complexity
| Operation | Complexity |
|---|---|
| push | O(1) amortized |
| pop | O(1) |
| peek (top) | O(1) |
Monotonic Stack Preview
A monotonic stack maintains elements in increasing or decreasing order. Used for next-greater / next-smaller / largest rectangle in histogram. Preview only — full pattern in Phase 2.
for x in arr:
while stack and stack[-1] < x:
# process element being popped: x is its next-greater
stack.pop()
stack.append(x)
Language-Specific Gotchas
- Python: use
listdirectly withappend/pop. Don’t usequeue.LifoQueue(locks). - Java: prefer
ArrayDequeover the legacyStack(synchronized, slow). - Go: slice with
s = append(s, x)ands[len(s)-1]/s = s[:len(s)-1]. - C++:
std::stack<T>adapter onstd::deque; or just usevector. - JS: array
push/pop.
7. Queues
Variants
- Plain queue (FIFO): enqueue rear, dequeue front.
- Deque (double-ended): push/pop both ends in O(1).
- Ring buffer / circular buffer: fixed-capacity deque on a static array.
- Priority queue: see Heaps.
Operations and Complexity
| Operation | Linked | Array deque | Ring buffer |
|---|---|---|---|
| enqueue rear | O(1) | O(1) amortized | O(1) (if not full) |
| dequeue front | O(1) | O(1) | O(1) |
| peek front | O(1) | O(1) | O(1) |
Language-Specific Gotchas
- Python:
collections.dequefor queue; never uselist.pop(0)(O(N)). - Java:
ArrayDeque<T>for both stack and queue.LinkedListworks but slower. Avoidjava.util.Queue<T> q = new LinkedList<>();for hot paths. - Go: no built-in deque;
container/listexists. Most CP code uses a slice as a queue withq[head:](lazy popfront). - C++:
std::deque(random-access amortized O(1)) orstd::queueadapter. - JS: array
push/shiftworks butshiftis O(N); use a custom ring buffer for large queues.
8. Heaps
Binary Heap
A complete binary tree where each parent ≤ children (min-heap) or ≥ (max-heap). Stored in an array: parent of i is (i-1)/2, children are 2i+1 and 2i+2.
Operations and Complexity
| Operation | Complexity |
|---|---|
| push (sift up) | O(log N) |
| pop top (sift down) | O(log N) |
| peek top | O(1) |
| heapify (build from N elements) | O(N) |
| decrease-key | O(log N) (if you know the index) |
| arbitrary delete | O(log N) (if you know the index), else O(N) |
Top-K Pattern (Preview)
- “Top K largest” → min-heap of size K. Push every element; if size > K, pop. Final heap = top K.
- “Stream median” → max-heap (lower half) + min-heap (upper half), balanced.
Language-Specific Gotchas
- Python:
heapqis a min-heap of any orderable; for max-heap, push-x. Tuples break ties lexicographically — careful with non-orderable secondary keys. - Java:
PriorityQueue<T>is a min-heap by default; passComparator.reverseOrder()for max.peek()may return null on empty. - Go:
container/heaprequires you to implement theheap.Interface. Significant boilerplate. - C++:
std::priority_queue<T>is a max-heap by default. Usestd::priority_queue<T, vector<T>, greater<T>>for min-heap. Or usemake_heap+push_heap+pop_heapon a vector. - JS: no built-in; write your own or use a library.
9. Sorted Arrays / Sorted Sets
Binary Search
On a sorted array, find a target or its insertion point in O(log N). Three canonical variants:
- Lower bound: smallest index where
a[i] >= target. - Upper bound: smallest index where
a[i] > target. - Exact match: lower bound + check
a[i] == target.
Operations on Sorted Set / Multiset
| Operation | Complexity |
|---|---|
| Insert | O(log N) |
| Erase | O(log N) |
| Find / lower_bound / upper_bound | O(log N) |
| Iterate in order | O(N) |
| Min / max | O(1) (or O(log N)) |
| Kth element | O(log N) (with order statistics tree) or O(K) iteration |
Language-Specific Gotchas
- Python:
bisect.bisect_left/bisect_rightfor sorted lists.sortedcontainers.SortedListfor an ordered multiset (O(log N) insert). - Java:
TreeSet<T>/TreeMap<K,V>;floor,ceiling,higher,lowerare essential APIs to know. - Go: no standard sorted set — must implement or use a third-party library.
- C++:
std::set/std::multiset(red-black tree);lower_bound/upper_boundmember functions. - JS: no standard sorted set; use a sorted array with binary search or write a treap.
10. Trees
Binary Tree Definitions
- Binary tree: each node has ≤ 2 children.
- BST: left subtree < node < right subtree (in-order traversal yields sorted sequence).
- Balanced (AVL, RB): height O(log N) guaranteed.
- Unbalanced: worst case O(N) (degenerate to linked list).
Traversals (Recursive)
def inorder(n):
if not n: return
inorder(n.left)
visit(n)
inorder(n.right)
Pre-order: visit, left, right. Post-order: left, right, visit. Level-order: BFS with a queue.
Traversals (Iterative)
- In-order with stack: push left chain, pop and visit, then go right.
- Pre-order with stack: push root, pop, visit, push right then left.
- Post-order with stack: trickier — use a marker or a 2-stack trick.
- Morris traversal: O(1) extra space using threaded pointers; advanced.
Operations and Complexity (Balanced)
| Operation | Balanced | Unbalanced |
|---|---|---|
| Insert / delete / search | O(log N) | O(N) |
| Min / max | O(log N) | O(N) |
| In-order traversal | O(N) | O(N) |
Language-Specific Gotchas
- Python: sys.setrecursionlimit; CPython has no tail-call elimination.
- Java:
TreeMapis red-black; recursion depth limited by JVM stack (~1000s). - Go: no standard balanced BST.
- C++:
std::map/std::setare red-black; iterators traverse in order. - JS: no standard balanced BST; recursion depth is engine-dependent.
11. Tries
Internal Representation
Each node has up to alphabet-size children (e.g., 26 for lowercase, 256 for byte, larger for unicode). End-of-word flag per node.
Operations and Complexity
| Operation | Complexity |
|---|---|
| Insert word of length L | O(L) |
| Search word of length L | O(L) |
| Prefix search | O(L) |
| Space | O(total chars × alphabet size) |
Alphabet Size Considerations
Fixed array per node: O(σ) memory per node, fast O(1) child lookup. HashMap per node: O(actual children) memory, slightly slower lookup. For 26 letters use array; for full unicode use hash map.
Language-Specific Gotchas
- Python: dict of dicts; can use
defaultdict(dict). - Java:
Map<Character, TrieNode>orTrieNode[26]. - Go: struct with
[26]*TrieNode. - C++: struct with
TrieNode* children[26]. Manual memory management orunique_ptr. - JS: plain objects or
Map.
12. Graphs
Representations
| Representation | Space | Edge query | Iterate neighbors |
|---|---|---|---|
| Adjacency list | O(V + E) | O(deg(v)) | O(deg(v)) |
| Adjacency matrix | O(V²) | O(1) | O(V) |
| Edge list | O(E) | O(E) | O(E) |
When To Use Each
- Adjacency list: sparse graphs (E ≪ V²) — almost always the right answer for interviews.
- Adjacency matrix: dense graphs (E ≈ V²), Floyd-Warshall, when V is small (≤ 500).
- Edge list: Kruskal’s MST, when you need to sort edges, when graph is given as edges and you don’t need neighbor queries.
Common Forms in Interviews
List<List<Integer>>adjacency.Map<String, List<String>>for non-integer node IDs.- Implicit graph (grid: neighbors are
(±1, 0)and(0, ±1)).
Language-Specific Gotchas
- Python:
defaultdict(list)is ideal foradj[u].append(v). - Java:
List<List<Integer>>with explicitArrayListinitialization in a loop; primitive int adjacency lists need third-party (Eclipse Collections, fastutil). - Go:
[][]intslice-of-slices. - C++:
vector<vector<int>>; for performance, usevector<int>with offsets (CSR format). - JS: array of arrays or
Map<string, string[]>.
13. Disjoint Set Union (Union-Find)
Operations
find(x): which component isxin?union(x, y): merge components ofxandy.
Optimizations
- Path compression: during
find, set parent of every visited node to the root. - Union by rank/size: attach the shorter tree under the taller.
- Together: O(α(N)) per op (inverse Ackermann — practically constant, ≤ 4 for any realistic N).
Naive vs Optimized
| Variant | find | union |
|---|---|---|
| Naive | O(N) | O(N) |
| Path compression only | O(log N) amortized | O(log N) amortized |
| Path compression + union by rank | O(α(N)) amortized | O(α(N)) amortized |
Reference Implementation (Python)
parent = list(range(N))
rank = [0] * N
def find(x):
while parent[x] != x:
parent[x] = parent[parent[x]] # path compression
x = parent[x]
return x
def union(x, y):
rx, ry = find(x), find(y)
if rx == ry: return False
if rank[rx] < rank[ry]: rx, ry = ry, rx
parent[ry] = rx
if rank[rx] == rank[ry]: rank[rx] += 1
return True
Language-Specific Gotchas
- Python: the recursive form blows past the recursion limit for N > 1000; always iterative.
- Java: prefer
int[] parentoverMap<Integer, Integer>for primitive perf. - Go: straightforward with
[]int. - C++:
vector<int>parent and rank. - JS: typed array
Int32Arrayfor speed.
14. Bitsets / Bit Manipulation
Common Idioms
| Operation | Idiom |
|---|---|
Set bit i | x | (1 << i) |
Clear bit i | x & ~(1 << i) |
Toggle bit i | x ^ (1 << i) |
Test bit i | (x >> i) & 1 |
| Lowest set bit | x & -x |
| Pop lowest set bit | x & (x - 1) |
| Popcount | language builtin (__builtin_popcount, Integer.bitCount, bin(x).count('1')) |
| Iterate subsets of mask | s = mask; while s > 0: …; s = (s - 1) & mask |
Bitsets
A packed array of bits — 64× the density and 64× the throughput of bool[] for many ops. Use when N up to ~10⁵ and you need fast set operations.
Language-Specific Gotchas
- Python: ints are arbitrary precision; no overflow but no SIMD either.
bin(x).count('1')works butint.bit_count()(3.10+) is faster. - Java:
intis 32-bit,longis 64-bit. Negative numbers:>>is arithmetic,>>>is logical. - Go: untyped constants vs typed; explicit cast required (
int(x),uint32(x)). - C++:
std::bitset<N>for compile-time-known N;vector<bool>is a specialization (proxy reference). - JS: bitwise ops coerce to 32-bit signed int — beware truncation. Use
BigIntfor 64-bit ops.
15. Counters / Multisets
Counter Pattern
A hash map from key to count. Used for: frequency analysis, anagram detection, sliding window distinct-element count.
Operations
| Operation | Complexity |
|---|---|
| Increment count[k] | O(1) |
| Decrement / remove if zero | O(1) |
| Total count | O(distinct keys) |
| Sorted by count | O(N log N) |
Multiset
A counter is essentially a multiset (allows duplicates, remembers count). For an ordered multiset (need min/max/kth in order), use a TreeMap+count or sortedcontainers.SortedList.
Language-Specific Gotchas
- Python:
collections.Counter—Counter(s),most_common(k), arithmetic operators (c1 - c2drops zeros). - Java:
HashMap<K, Integer>withgetOrDefault(k, 0) + 1andmerge(k, 1, Integer::sum). There is no built-inCounter. - Go:
map[K]int. Manual increment. - C++:
std::unordered_map<K, int>;++m[k]works because default-constructed int is 0. - JS:
Map(preserves insertion order) or plain object (string keys only).
Inline Runtime Concepts Reference
These concepts cut across all data structures. They are interview rubric line items in their own right.
1. Stack vs Heap Memory
The call stack holds function frames: locals, args, return addresses. Fast, fixed-size (typically 1–8 MB in interview environments). Allocations are pointer-bump and bound to the frame’s lifetime.
The heap is dynamically managed memory — new, malloc, Python objects, JVM objects. Slower allocation, can be GC’d or manually freed. Survives beyond the function that created it.
Why It Matters In Interviews
- “Why does my recursion overflow at depth 10⁵?” → call stack ~1 MB / ~64 bytes per frame ≈ 16K frames before crash.
- “Why is my linked list slower than my array?” → heap-allocated nodes scattered in memory, no cache locality.
Per-Language Gotchas
- Python: all “values” except small ints are heap objects. Locals are name bindings, not stack-allocated values.
- Java: primitives in locals are stack-allocated; objects always heap. Escape analysis can sometimes elide a heap alloc.
- Go: escape analysis decides stack vs heap;
&xin a returned closure forces heap allocation. - C++: explicit (
int x;is stack,new intis heap). Stack allocation is much faster. - JS: all objects are heap; primitives may be stack-internal but the language hides it.
2. Scope and Lifetime
Scope = where a name is visible. Lifetime = how long the value lives.
These can differ! In a closure, a variable’s scope ends with the function but its lifetime extends as long as the closure references it.
Per-Language Gotchas
- Python: late binding in closures —
[lambda: i for i in range(3)]all return 2, not 0/1/2. - Java: local variables captured by lambdas must be effectively final.
- Go: loop variable capture changed in Go 1.22 — pre-1.22,
for i := range … { go func() { use(i) }() }captures the samei. - C++: dangling reference if you return
&local. Lifetime extension viaconst&is a niche rule. - JS:
varis function-scoped,let/constare block-scoped. Hoisting trap.
3. Value vs Reference Semantics
Does assigning or passing a variable copy the value or share a reference?
| Language | Primitives | Objects/Arrays |
|---|---|---|
| Python | by value (immutable) | by reference |
| Java | by value | reference passed by value |
| Go | by value (struct, array) | slice/map/chan = ref-ish |
| C++ | by value (default) | explicit & or * for ref |
| JS | by value | by reference |
Trap
“I passed my array to the function, modified it inside, and the caller saw the change!” — yes, because the array (Python list, Java int[], JS array) is passed by reference.
“I passed my int to the function, modified it inside, but the caller didn’t see the change!” — yes, because the int is by value.
4. Mutable vs Immutable
Immutable values cannot be modified after creation; “modification” returns a new value.
| Language | Immutable types |
|---|---|
| Python | str, int, tuple, frozenset, bytes |
| Java | String, all primitive wrappers, LocalDate, etc. |
| Go | string |
| C++ | const-qualified |
| JS | primitives (string, number, boolean, undefined, null, bigint, symbol) |
Trap
- Using a mutable object as a hash map key, then mutating it → key lost.
- “Why is
s += cslow?” — string is immutable, every iteration copies the whole string.
5. Hash Collisions and Adversarial Keys
A hash map’s O(1) average requires a “good” hash function and “uniform” inputs. An adversary who knows the hash function can craft keys all colliding to one bucket → O(N²) blowup.
Defenses
- Random seed (Python’s
PYTHONHASHSEED, Go map random seed) — attacker can’t predict the function. - Tree fallback — Java
HashMapsince 8 converts long collision chains into red-black trees, capping worst case at O(log N). - Cryptographic hash — overkill, but immune.
Interview Note
If the problem explicitly says “adversarial” or “competitive” inputs, do not rely on hash maps. Use a sorted structure (TreeMap, sortedcontainers, std::map).
6. Iterator Invalidation
Modifying a collection while iterating can break the iterator.
Behaviors
- Python: mutating a dict during iteration raises
RuntimeError. - Java:
ConcurrentModificationException(fail-fast iterator). - Go: map iteration order is randomized; modifying the map mid-iteration is technically allowed but the new keys may or may not be visited.
- C++:
vector::push_backinvalidates all iterators if it reallocates.unordered_map::insertinvalidates all on rehash. - JS:
MapandSetiteration sees later insertions; deletions are honored.
Safe Pattern
Collect mutations into a list during iteration; apply after the loop.
7. Garbage Collection Basics
Two main strategies:
- Reference counting (refcount): each object has a count of references to it; when 0, freed. Fast but cannot collect cycles. CPython uses refcount + cycle collector.
- Tracing GC (mark-and-sweep, generational): periodically traces from roots; unreachable objects freed. Java, Go, JS, C# use variants.
Why It Matters
- Refcount:
del xis immediate; deterministic destruction. - Tracing: deallocation happens “later” — pauses (“stop-the-world”) historically, mostly amortized in modern collectors.
Per-Language Gotchas
- Python: cycles between objects with
__del__finalizers may never collect (pre-3.4). - Java: “GC pause” is a real concern in latency-sensitive interviews; mention G1, ZGC.
- Go: GC is concurrent and non-generational; predictable sub-millisecond pauses.
- C++: no GC — must
deletewhat younew. RAII (smart pointers) automates this. - JS: V8 uses generational GC.
8. Memory Leaks (in GC’d Languages)
A “leak” in a GC’d language means: the object is unreachable from the developer’s intent, but reachable from the GC’s perspective — so it’s never freed.
Common Sources
- Listeners not removed: event handler holds a reference to the listener forever.
- Caches without eviction: map grows monotonically.
- Closure capture: closure holds reference to large enclosing object.
- Static fields: lives forever.
- ThreadLocals not cleared in pooled threads: classic JVM leak.
Per-Language Gotchas
- Python: circular refs with finalizers; module-level state.
- Java:
ThreadLocal+ thread pool; classloader leaks (PermGen/Metaspace). - Go: goroutine leak (blocking on a channel that never receives).
- C++: real memory leak via missing
delete. - JS: detached DOM nodes, timers not cleared.
9. Deep vs Shallow Copy
- Shallow copy: new container, but elements are shared references.
- Deep copy: recursively copies elements too.
When It Matters
Backtracking: if you store snapshots of a mutable list, you need a deep (or at least one-level) copy, otherwise all snapshots reflect the latest state.
result = []
path = []
def backtrack(...):
if done: result.append(path[:]) # MUST copy; otherwise all entries are the same list
Per-Language Gotchas
- Python:
list(x)/x[:]shallow;copy.deepcopy(x)deep. - Java:
clone()is shallow by default; deep requires explicit traversal. - Go:
copy(dst, src)shallow; for deep, write your own. - C++: copy constructor: shallow by default for raw pointers; smart pointers and STL containers do “value” copies.
- JS: spread
[...arr]and{...obj}are shallow; deep viastructuredClone(x)(modern) or JSON round-trip (lossy).
10. Recursion Depth and Stack Overflow
Each recursive call adds a frame to the call stack. The stack has a fixed size; exceeding it crashes (or in Python, raises RecursionError).
Default Limits
- Python:
sys.getrecursionlimit()defaults to 1000. Raise withsys.setrecursionlimit(10**6)plusthreading.stack_size(...). - Java: ~10K frames typical; tune with
-Xss. - Go: stacks start small (8KB) and grow up to 1 GB.
- C++: thread stack ~1 MB default; tune at thread creation.
- JS: ~10K frames typical, engine-dependent.
Mitigation
- Convert to iteration with an explicit stack (DFS, in-order traversal).
- Tail-call optimization is not present in most popular languages (no Python, no Java, no JS, no Go). C++ and Scheme can do it.
- For trees, balance matters: a degenerate (chain-shaped) tree blows recursion at depth N, not log N.
Recommended Problem Categories
After mastering the concepts, drill these problem categories. Each maps to a lab below.
| Category | Sample Problems |
|---|---|
| Array manipulation | Rotate Array, Move Zeros, Plus One, Spiral Matrix |
| String mechanics | Reverse Words, Implement strStr, Length of Last Word |
| Hash map / set | Two Sum, Group Anagrams, Contains Duplicate II, Intersection of Two Arrays |
| Linked list | Reverse Linked List, Merge Two Sorted Lists, Palindrome Linked List, Linked List Cycle |
| Stack / queue | Valid Parentheses, Min Stack, Implement Queue using Stacks, Daily Temperatures |
| Heap | Kth Largest in Stream, Last Stone Weight, K Closest Points to Origin |
| Binary search | First Bad Version, Search Insert Position, Find Peak Element |
| Recursion | Generate Parentheses, Letter Combinations of a Phone Number, Subsets |
| Tree traversal | Inorder Traversal (iterative + recursive), Symmetric Tree, Same Tree, Binary Tree Paths |
Aim for ~30 problems across these categories minimum (more for the 6/12-month tracks).
Mastery Checklist
- Implement, from scratch, in 20 minutes each, with passing tests: dynamic array, linked list (singly + doubly), stack, queue (deque), binary heap, hash map (open addressing), trie, union-find.
- State worst-case, average-case, and amortized complexity for every operation in the table above.
-
Explain why
s += cin a loop isO(N²)in three of {Python, Java, JS, Go}. - Write iterative in-order, pre-order, and post-order tree traversals (no recursion).
- Solve any LeetCode Easy in 15 minutes using the framework.
- Solve any LeetCode Easy-Medium hash-map problem in under 20 minutes.
- Recognize when a problem needs a heap vs sorted set vs counter without prompting.
- Articulate the iterator invalidation rules for your interview language.
- Articulate refcount vs tracing GC for your interview language.
- Predict whether a recursive solution will overflow on N = 10⁵ in your interview language.
Exit Criteria
Move to Phase 2 only when:
- Mastery checklist 100% complete.
- All 9 labs completed with passing tests.
- Solved ≥ 30 problems across the recommended categories using the framework, each reviewed via REVIEW_TEMPLATE.md.
- Implemented every primary data structure from scratch — no notes, no internet, just the language’s basic features.
- Self-mocked one Easy and one low-Medium problem with full framework execution; passed using READINESS_CHECKLIST.md section criteria for “data-structure fluency.”
- For every failure during these problems, ran the diagnosis in FAILURE_ANALYSIS.md and queued it via SPACED_REPETITION.md.
If any item fails, do not move on. Phase 2 patterns assume Phase 1 fluency. Without it, you will plateau.
Hands-On Labs
Complete in order. Each lab uses the strict 22-section format used throughout the curriculum.
- Lab 01 — Array Fundamentals
- Lab 02 — String Mechanics
- Lab 03 — Hashmap Mastery
- Lab 04 — Linked List Pointers
- Lab 05 — Stack & Queue Applications
- Lab 06 — Heap Priority
- Lab 07 — Binary Search Fundamentals
- Lab 08 — Recursion & Stack
- Lab 09 — Tree Traversal Fundamentals
Common Mistakes In Phase 1
- Memorizing complexities without understanding them — when an interviewer probes “why is dict insert O(1) on average?”, you must derive it.
- Skipping “from scratch” implementations — using
import heapqis fine for problems, but you must be able to write the heap yourself. - Treating Python list as a queue —
pop(0)is O(N); usedeque. - Forgetting to check overflow in C++/Java —
int + intoverflows silently; cast tolong. - Recursion in Python without setting the limit — N=10⁴ trees blow the default stack.
- Confusing
==withis/equals/===— identity vs equality differs by language. - Using a mutable object as a hash key.
- Not knowing your language’s iterator invalidation rules.
If any of these still trip you up, you are not done with Phase 1.
Lab 01 — Array Fundamentals: Rotate Array In Place
Goal
Master in-place array rotation. The deliverable shows you understand pointer arithmetic, the O(N) reversal trick, dynamic-array memory layout, and edge cases that catch ~70% of candidates on this exact problem.
Background Concepts
Arrays as contiguous memory; index arithmetic mod N; in-place vs auxiliary-space transformations; the “three reversals” identity: rotate(arr, k) == reverse(reverse(arr[:k]), reverse(arr[k:])). Review the Arrays section of the Phase 1 README and the value-vs-reference rules in section 3 of the runtime concepts.
Interview Context
This is the canonical “looks easy, traps everyone” Easy/low-Medium problem. Real interviews from Microsoft, Amazon, Meta, Apple, Google. The interviewer is watching for: do you do the auxiliary-array brute force first? Do you spot the O(N) in-place trick? Do you handle k > N? Do you survive k == 0 and N == 1?
Problem Statement
Given an integer array nums and a non-negative integer k, rotate the array to the right by k steps in place. After the rotation, element originally at index i ends up at index (i + k) % N.
Constraints
1 ≤ N ≤ 10^5-2^31 ≤ nums[i] ≤ 2^31 - 10 ≤ k ≤ 10^9- Must run in O(N) time and O(1) extra space.
Clarifying Questions
- Can
kbe greater thanN? (Yes — must reduce mod N.) - Can
kbe 0? (Yes — should be a no-op, no array mutation.) - Can the array be empty / size 1? (Per the constraints,
N ≥ 1. Confirm.) - Right rotation, not left? (Confirm direction; getting it backward is a top-3 bug here.)
- Must it be in place, or is auxiliary memory allowed? (In place — that’s the spirit of the problem.)
Examples
| Input | k | Output | Notes |
|---|---|---|---|
[1,2,3,4,5,6,7] | 3 | [5,6,7,1,2,3,4] | Standard case |
[1,2] | 3 | [2,1] | k > N: effective k = 1 |
[1,2,3] | 0 | [1,2,3] | No-op |
[1] | 100 | [1] | Trivial size |
[1,2,3] | 3 | [1,2,3] | k == N: no-op |
Initial Brute Force
Allocate out[N]; for each i, set out[(i + k) % N] = nums[i]; copy out back into nums.
def rotate_brute(nums, k):
n = len(nums)
k %= n
out = [0] * n
for i in range(n):
out[(i + k) % n] = nums[i]
nums[:] = out
Brute Force Complexity
Time: O(N). Space: O(N) auxiliary. Fails the in-place constraint despite optimal time.
Optimization Path
We need O(1) extra space. Two well-known approaches:
- Cyclic replacement: start at index 0, jump to
(0 + k) % N, place the displaced element, continue. Visits each index exactly once. Tricky whengcd(N, k) > 1(multiple disjoint cycles). Correctness needs a counter for elements moved. - Three reversals: reverse the whole array, reverse the first
k, reverse the lastN-k. This works because rotation bykis reversal of (reversal of left, reversal of right). Easier to write correctly.
Pick the three-reversal approach for the interview unless the interviewer explicitly asks for cyclic replacement.
Final Expected Approach
Three-reversal in place.
def reverse(nums, l, r):
while l < r:
nums[l], nums[r] = nums[r], nums[l]
l += 1
r -= 1
def rotate(nums, k):
n = len(nums)
k %= n # normalize
reverse(nums, 0, n - 1)
reverse(nums, 0, k - 1)
reverse(nums, k, n - 1)
Data Structures Used
A single mutable array. No auxiliary structures.
Correctness Argument
Loop invariant for reverse(l, r): at each iteration, elements at positions less than l and greater than r are already correctly placed (i.e., are mirror swaps of each other). Termination at l ≥ r leaves the closed range fully reversed. The three-reversal identity is verifiable with a 2-step example: [A B C D E] k=2 → reverse all → [E D C B A] → reverse first 2 → [D E C B A] → reverse last 3 → [D E A B C]. Equivalent to original rotated right by 2.
Complexity
Time: O(N) (three passes, each touching at most N elements). Space: O(1) (in-place swaps; no allocation).
Implementation Requirements
- Helper
reverse(nums, l, r)with explicit bounds (closed interval). - Always do
k %= nfirst, before any loop or reversal. - Handle
k == 0andk == n: both reduce to no-op viak %= n. - No allocation outside the input array (verify by reading your code).
- Clean variable names (
l,r,n,kare interview-acceptable).
Tests
- Smoke: the canonical
[1,2,3,4,5,6,7]with k=3. - Unit: k=0 (no-op), k=N (no-op), k=1 (single right shift).
- Edge:
N=1,N=2 with k=1, largek=10^9modN. - Large: N=10^5, k=N//2; assert in-place (capture
id(nums)in Python). - Random: generate random arrays and
ks; check against brute force as oracle. - Invalid: negative
k(per constraints not allowed; if interviewer extends, decide left rotation semantics).
Follow-up Questions
- “Can you do it without the modulo?” (Yes, but ugly: branch on
k <= n.) - “What if the array is given as a linked list?” (Different problem — find length, find pivot, splice.)
- “What if
kcan be negative (left rotation)?” (Convert viak = ((k % n) + n) % n.) - “Solve using a single reverse loop without a helper.” (Inline the swaps three times.)
- “Implement with cyclic replacement instead.” (Demonstrate the gcd cycle counter trick.)
Product Extension
A circular buffer for a metrics dashboard storing the last N seconds of samples. Rotation isn’t done on append — instead, a head index advances mod N. The “three reversals” trick is what you do when the buffer must be flattened to a linear export. Discuss tradeoffs: head-index buffer is O(1) per append but harder to debug; rotation on read is O(N) but storage is always linear.
Language/Runtime Follow-ups
- Python:
nums[:] = nums[-k:] + nums[:-k]is one-line and Pythonic but allocates O(N). Acceptable to mention but interviewer may rule it “not in place.” Pure swap version uses no allocation. - Java:
int[](primitive) avoids boxing. Don’t reach forCollections.rotate; understand it. - Go: slice indexing, careful with
n := len(nums)capture;nums = nums[:]aliasing makes no copy. - C++:
std::reverse(nums.begin() + l, nums.begin() + r + 1); use the standard. - JS: in-place using
[a, b] = [b, a]swap ortemp = a.arr.reverse()is in-place.
Common Bugs
- Forgetting
k %= n— when k > n the reversals overlap incorrectly. - Off-by-one in
reverse(l, r)— usingrvsr - 1as the bound; using<vs≤. - Reversing wrong segments — confusing first-k with last-k. Right rotation:
[reverse first k of reversed array]then[reverse last n-k]. - Allocating in disguise —
nums = nums[-k:] + nums[:-k]rebinds the local name and does not mutate the caller’s array (in Python). Usenums[:] = …. - Left vs right confusion — re-read the problem statement once before submitting.
Debugging Strategy
- Print the array after each of the three reversals; compare to a hand-traced
[1,2,3,4,5,6,7]k=3 walk-through. - If output is wrong by a constant shift, suspect an off-by-one in segment bounds.
- If output looks reflected (
[3,2,1, 7,6,5,4]instead of[5,6,7,1,2,3,4]), one of the three reversals fired in the wrong region.
Mastery Criteria
- Wrote the three-reversal solution in under 4 minutes, no bugs, in-place verified.
- Traced through
k > N,k == 0,N == 1without prompting. - Stated the loop invariant for
reversealoud. - Named the cyclic-replacement alternative and acknowledged its
gcdcomplication. - Identified and avoided the Python
nums = nums[-k:] + nums[:-k]allocation trap.
Lab 02 — String Mechanics: Reverse Words In A String
Goal
Master string immutability, builder patterns, encoding gotchas, and the cost of naive concatenation. The deliverable: reverse the order of words in a sentence efficiently in your interview language, demonstrating you understand why the “obvious” solution can be O(N²) in some languages and O(N) in others.
Background Concepts
String immutability and the resulting cost of s += c loops; StringBuilder / strings.Builder / "".join() patterns; substring complexity; whitespace tokenization; Unicode pitfalls. Review the Strings section of the Phase 1 README and item 4 in the runtime concepts (mutable vs immutable).
Interview Context
A staple of Microsoft, Amazon, and Bloomberg phone screens. The trap is candidates who write result = ""; for w in reversed(words): result += w + " " and don’t realize they just shipped O(N²) code in Java or Python. Strong candidates state the immutability fact aloud and choose a builder pattern.
Problem Statement
Given a string s representing a sentence, return a new string with the order of words reversed. A “word” is a maximal run of non-space characters. Multiple spaces between words and leading/trailing spaces must be collapsed to single spaces; the output has no leading/trailing space.
Constraints
1 ≤ |s| ≤ 10^4scontains printable ASCII characters and spaces.scontains at least one word.
Clarifying Questions
- Is the input ASCII or arbitrary Unicode? (Affects iteration model; ASCII is the default unless stated.)
- Should multiple internal spaces be preserved or collapsed? (Standard problem says collapse; confirm.)
- Trim leading/trailing whitespace? (Yes — output has none.)
- Punctuation: is
"hello,"one word? (Per problem: a “word” is non-space-separated;"hello,"is one word.) - Can I allocate a new string, or must I work in place? (For Python/Java/JS — strings are immutable, so a new string is unavoidable. For C++
std::string, in-place is feasible.)
Examples
| Input | Output |
|---|---|
"the sky is blue" | "blue is sky the" |
" hello world " | "world hello" |
"a good example" | "example good a" |
"single" | "single" |
" " | invalid per constraints |
Initial Brute Force
Split on whitespace, reverse the list, join with single spaces.
def reverse_words(s):
return " ".join(reversed(s.split()))
s.split() (no arg) collapses runs of whitespace and trims, which is exactly the spec. This is one line in Python — but the interviewer wants you to explain what it does.
Brute Force Complexity
Time: O(N) — split is one linear pass, reversed is O(k) where k is word count, join is one linear pass. Space: O(N) for the list of words and the output. This is already optimal asymptotically.
Optimization Path
For interviews where the one-liner is “too easy,” the interviewer escalates: “Do it without split/join; use only character-level operations.” Or: “Reverse in place in a char[] with O(1) extra memory.”
The classic in-place trick on a mutable buffer: reverse the entire buffer, then reverse each word, then collapse internal whitespace. This is the same three-reversal identity from Lab 01, applied to characters.
Final Expected Approach
State the one-liner first. Then offer the manual two-pointer approach for languages with mutable strings or as an “I-understand-the-internals” demonstration.
def reverse_words(s):
# tokenize without builtins
words = []
i = 0
n = len(s)
while i < n:
while i < n and s[i] == ' ':
i += 1
j = i
while j < n and s[j] != ' ':
j += 1
if j > i:
words.append(s[i:j])
i = j
# reverse and join via builder
out = []
for w in reversed(words):
out.append(w)
return ' '.join(out)
In Java, replace the final join with StringBuilder. In Go, with strings.Builder.
Data Structures Used
A list of word strings (or substrings); a builder for the output. No advanced structures.
Correctness Argument
Tokenization invariant: at each outer iteration, i points at the start of unscanned input; the inner loops skip whitespace and capture a word. Each character is examined O(1) times, so tokenization is O(N). Reversed iteration over words produces them in opposite order; joining with ' ' produces single-space separation; no leading/trailing space because we never push empty words and we don’t terminate with a separator (Python join handles this).
Complexity
Time: O(N) — single tokenization pass plus single output assembly pass. Space: O(N) — output and word list.
Implementation Requirements
- Use an explicit builder (
StringBuilder,strings.Builder,[].join,''.join). - Never use
+=to build the output in a loop in Java/Python/JS. - Don’t rely on regex unless the interviewer is explicitly fine with it (
s.split(/\s+/)works but is overkill). - Verify trimming works on
" word ". - Verify multiple internal spaces collapse on
"a b".
Tests
- Smoke:
"the sky is blue"→"blue is sky the". - Unit: leading/trailing spaces; multiple internal spaces; single-word input.
- Edge: all-spaces input (per constraints invalid; handle gracefully if extended); single character; punctuation as part of word.
- Large: N = 10^4 input with 10^3 words; assert no quadratic behavior (time it).
- Random: randomly generate space-and-letter strings; cross-check against
" ".join(reversed(s.split())). - Invalid: non-ASCII in extended versions (define behavior per language’s iteration model).
Follow-up Questions
- “Reverse character order within each word as well.” (Reverse each word in place after splitting.)
- “Reverse in O(1) extra space on a mutable
char[].” (The three-reversal trick.) - “Handle Unicode where ‘word’ is grapheme-cluster bounded.” (Need an ICU library or equivalent.)
- “Preserve original whitespace runs.” (Don’t collapse; keep separator tokens.)
- “What if the string is huge and streamed?” (Process word-by-word from a buffered reader.)
Product Extension
A search-engine query normalizer. Inputs from users have inconsistent whitespace, varying word order. Reverse word order is a feature for “did-you-mean” inversion testing. In production: keep the original for display, normalize for indexing, and accept that the cost of String immutability in Java means hot paths use StringBuilder or even byte arrays directly.
Language/Runtime Follow-ups
- Python:
s.split()(no args) is the magical normalizer."".join(...)is a single allocation; never use+=onstrin a loop. - Java:
String.split("\\s+")returns an array;String.trim()separately. Output viaStringBuilder.String.join(" ", parts)is the modern one-liner. - Go:
strings.Fields(s)splits on any whitespace and trims;strings.Join(parts, " ")rebuilds. Both are O(N). - C++:
std::stringstreamfor tokenizing; build viastd::string +=, which has small-string optimization but still amortized O(N). - JS/TS:
s.trim().split(/\s+/).reverse().join(' '). Beware: the empty-string case"".split(/\s+/)returns[""], not[]. - Unicode subtlety: in Java/JS,
lengthcounts UTF-16 code units;'😀'is length 2. Doesn’t matter here unless emoji-as-word.
Common Bugs
- Off-by-one on whitespace — leaving a trailing space after
join. s += cin a loop — O(N²) in Python, Java, JS. Catastrophic on large input.- Splitting by
' 'instead of by whitespace —"a b".split(' ')returns["a", "", "b"]in Java/JS; you get empty tokens. - Trim missed — leading whitespace becomes a leading empty token.
- Mutating the input — in some languages strings are immutable so this is a type error; in C++/Go-byte-slice it’s a semantic bug.
Debugging Strategy
- Print the tokenized list. If it has empty strings, your splitter doesn’t collapse.
- If output has trailing space, your join builds it manually rather than via the standard library.
- Time on a 10^4 input. If it’s > 1 second, you have a quadratic concat hidden somewhere.
Mastery Criteria
- Wrote the one-liner in 30 seconds and explained why each piece is needed.
- Wrote the manual tokenizer in under 5 minutes.
- Stated aloud “in this language, strings are immutable, so I will use a builder.”
- Identified the difference between
split(' ')andsplit('\\s+')/split()(no-arg). - Acknowledged the in-place
char[]three-reversal alternative without writing it (or wrote it on follow-up). - Tested with
" a b "and confirmed clean output.
Lab 03 — Hashmap Mastery: Group Anagrams
Goal
Master hash-map design with composite keys, adversarial input awareness, and the equality/hashcode contract. The deliverable groups N strings into anagram buckets in O(N · L) time and articulates exactly why your hash key works and what an adversary could do to break it.
Background Concepts
Hash function design; key equality contract; adversarial inputs and load factor; ordered-vs-unordered map choice; counter pattern. Review the Hash Maps and Hash Sets sections of the Phase 1 README, plus runtime concept 5 (hash collisions).
Interview Context
Group Anagrams is interview-evergreen: appears at Meta, Google, Amazon, Microsoft. The interview signal is whether you reach for the right key. Naive candidates compare every pair (O(N² · L)). Decent candidates sort each string into a key (O(N · L log L)). Strong candidates use a counter tuple key (O(N · L)). Elite candidates discuss adversarial hash flooding and language-specific custom-hash mechanics.
Problem Statement
Given an array strs of N lowercase-ASCII strings, group the strings that are anagrams of each other. Return the groups as a list of lists. Within and across groups, any order is acceptable.
Constraints
1 ≤ N ≤ 10^40 ≤ |s_i| ≤ 100s_iconsists of lowercase English letters.- Total characters:
Σ |s_i| ≤ 10^6.
Clarifying Questions
- Lowercase only? (Per constraints — confirm.)
- Do empty strings group together? (Yes — the empty string is an anagram of itself.)
- Is output order significant within or across groups? (Standard problem: no.)
- Are duplicates in the input allowed? (Yes —
["aa","aa"]is one group of size 2.) - Memory constraints? (Should fit comfortably; mention you’ll discuss tradeoffs.)
Examples
| Input | Output (any order) |
|---|---|
["eat","tea","tan","ate","nat","bat"] | [["eat","tea","ate"], ["tan","nat"], ["bat"]] |
[""] | [[""]] |
["a"] | [["a"]] |
["abc","cab","bca","xyz"] | [["abc","cab","bca"], ["xyz"]] |
Initial Brute Force
For each pair (i, j), check if strs[j] is an anagram of strs[i] (e.g., by sorting both). Use a seen[] array. O(N² · L log L).
def group_anagrams_brute(strs):
seen = [False] * len(strs)
groups = []
for i, s in enumerate(strs):
if seen[i]: continue
g = [s]
seen[i] = True
ks = sorted(s)
for j in range(i + 1, len(strs)):
if not seen[j] and sorted(strs[j]) == ks:
g.append(strs[j])
seen[j] = True
groups.append(g)
return groups
Brute Force Complexity
Time: O(N² · L log L) due to repeated sorting. Space: O(N) for the seen array plus O(L) per sort. Fails for N = 10^4 (10⁸ operations × log).
Optimization Path
The key insight: anagrams have the same multiset of characters. We need a hashable key derived from this multiset. Two canonical forms:
- Sorted string as key:
sorted(s) → "aet"for"eat","tea","ate". Cost per key: O(L log L). Total: O(N · L log L). - Count tuple as key: a length-26 tuple of counts. Cost per key: O(L). Total: O(N · L). Optimal for large L.
Pick the count tuple unless L is tiny.
Final Expected Approach
Bucket strings by count tuple in a hash map.
from collections import defaultdict
def group_anagrams(strs):
buckets = defaultdict(list)
for s in strs:
counts = [0] * 26
for c in s:
counts[ord(c) - ord('a')] += 1
buckets[tuple(counts)].append(s)
return list(buckets.values())
Data Structures Used
- A hash map keyed by a 26-tuple of int counts; values are lists of strings.
- A constant-size 26-int array for tallying (per word).
Correctness Argument
Two strings are anagrams iff they have identical character multisets iff their count vectors are equal. Equal count vectors hash to the same bucket and compare equal under tuple equality, so anagrams land in the same bucket. Different vectors compare unequal under tuple equality, so non-anagrams don’t share a bucket (modulo accidental hash collisions, which the equality check resolves correctly — that’s the equality/hashcode contract at work).
Complexity
Time: O(Σ |s_i|) = O(N · L) for tallying plus O(N · 26) for tuple hashing = O(N · L). Space: O(N · L) for the buckets and keys.
Implementation Requirements
- Use a hashable key — tuples in Python,
String(built from the count array) in Java, struct or stringified key in Go,std::array<int, 26>in C++. - Don’t use the sorted string for very large L (suboptimal but acceptable for interview presentation).
- Use
defaultdict(list)or equivalent (computeIfAbsentin Java) to avoid manual “if not in map” branching. - Return values as a list-of-lists, not the dict itself.
Tests
- Smoke: the canonical 6-string example above.
- Unit: singletons, all-anagrams (
["abc","bca","cab"]), no-anagrams (["a","b","c"]). - Edge: empty strings, single-char strings, duplicates (
["aa","aa"]). - Large: N = 10⁴, L = 100, mix of group sizes; assert sub-second.
- Random: generate random words; verify bucketing matches a reference (e.g., the sorted-string variant).
- Invalid: uppercase or non-ASCII (per constraints disallowed; if extended, normalize first).
Follow-up Questions
- “What if strings can be Unicode?” → switch to a
Counter/HashMap<Char, Int>as the key (more expensive hashing). Or, use the sorted string with a Unicode-aware sort. - “What if the input is streamed?” → emit groups lazily as you find duplicates, but you can’t finalize a group until input ends.
- “What if memory is tight (you can’t store N count arrays)?” → use the sorted-string key (only 1 allocation per word, free after bucketing) or a rolling hash with secondary check.
- “Adversarial input — can the interviewer construct N strings whose count tuples all hash to the same bucket?” → yes for predictable-hash languages; mitigation is randomized hash seeds or a TreeMap fallback.
- “Implement without a hash map.” → sort all strings by their sort-key, then group consecutive equal keys. O(N · L log L) due to sorting strings.
Product Extension
A duplicate-document detector. Each document is hashed by a content fingerprint (e.g., sorted shingles); documents with the same fingerprint are grouped. The same data-structure pattern (hash by canonical form) underlies large-scale dedup at file-storage and email systems. Discuss false positives (two different docs with the same fingerprint), the role of secondary equality check, and the tradeoff between fingerprint cost and accuracy.
Language/Runtime Follow-ups
- Python:
tuple(counts)is hashable;Counteris hashable only viafrozenset(c.items()).defaultdict(list)is the idiom. - Java: must build a
String(or use a hash of the int[] array combined withequalson the array — which means a custom class with properhashCode/equals). The classic interview shortcut is to convert the count array to a string like"1#0#0#…#1". Beware boxing inHashMap<int[], List<String>>—int[]does not overridehashCode/equals, defaults to object identity. This is a top-3 Java bug on this problem. - Go: map keys must be comparable.
[26]intis comparable;[]intis not. Use the array. - C++:
std::array<int, 26>is hashable withboost::hashor a customstd::hashspecialization. Or stringify. - JS/TS:
Mapkeys can be any value but use reference equality for arrays/objects. Use a string key likecounts.join(',')or aMap<string, string[]>. - Adversarial keys: Java’s
String.hashCodeis well-known and allows hash flooding. Java HashMap mitigates with tree-to-bucket conversion past 8 collisions.
Common Bugs
- Java
int[]as map key — uses object identity, not value equality. Every entry creates a new bucket. Fix: stringify or useArrays.hashCode+ custom wrapper. - Mutating the count array between map ops — if you reuse one buffer and mutate, your inserted keys all alias the same buffer. Allocate fresh per word.
- Off-by-one in
ord(c) - ord('a')— non-lowercase input goes negative or out of range. - Empty-string handling — count array is all zeros; should still bucket correctly. Verify.
- Returning
dict.values()directly in Python — works but the type isdict_values, notlist. Wrap withlist(...).
Debugging Strategy
- Print the keys of the resulting map. If you see N keys for N strings, your key derivation is wrong (likely identity-based).
- For Java: assert
myKey.hashCode() == myOtherKey.hashCode()for a hand-crafted anagram pair. - Time on N=10⁴: should run in well under a second.
Mastery Criteria
- Selected the count-tuple approach within 60 seconds, explaining why over the sorted-string approach.
- Stated the equality/hashcode contract and how it affects key choice in Java.
- Identified the
int[]reference-equality trap (or its language equivalent) before coding. - Articulated the adversarial-input concern and the language’s defense.
- Wrote a clean implementation in under 8 minutes.
- Tested with the empty-string and all-duplicates edge cases.
Lab 04 — Linked List Pointers: Reverse Linked List
Goal
Master pointer manipulation under aliasing, the classic three-pointer iterative reverse, the recursive variant with stack-frame analysis, and the dummy-node technique. The deliverable: reverse a singly linked list iteratively and recursively, articulating exactly which references move when, and identifying the recursion-depth risk on long lists.
Background Concepts
Pointers as references; aliasing; null sentinels; recursion stack frames; tail-call elimination (or absence thereof). Review the Linked Lists section of the Phase 1 README, plus runtime concepts 3 (value vs reference) and 10 (recursion depth).
Interview Context
Reverse Linked List is the warm-up question at every FAANG. The signal isn’t whether you can do it — most candidates can — it’s whether you can do it cleanly, in two ways, on the whiteboard, while talking through pointer movement. It’s also the “are you ready for harder linked-list problems” gate.
Problem Statement
Given the head of a singly linked list 1 → 2 → 3 → 4 → 5 → null, return the head of the reversed list 5 → 4 → 3 → 2 → 1 → null. The original list nodes are reused (no new node allocation).
Constraints
0 ≤ N ≤ 5000(LeetCode classic) — but realistic interview lists may have N up to 10⁵; recursion depth matters.-5000 ≤ node.val ≤ 5000(irrelevant for traversal logic).
Clarifying Questions
- Is the list singly or doubly linked? (Singly — affects whether we need to update
prevpointers.) - Is
headevernull? (Yes — returnnull. Top edge case.) - Single-node list? (Return the same node; its
nextis alreadynull.) - Should the reversal be in place (reuse nodes) or allocate new nodes? (In place is the standard; allocating new nodes is a different problem.)
- Should I also support reversing a segment
[m, n]? (That’s a follow-up — see “Reverse Linked List II”.)
Examples
| Input | Output |
|---|---|
1 → 2 → 3 → 4 → 5 | 5 → 4 → 3 → 2 → 1 |
1 → 2 | 2 → 1 |
1 | 1 |
null | null |
Initial Brute Force
Walk the list, push values onto a stack, walk again and reassign values from the stack.
def reverse_brute(head):
vals = []
cur = head
while cur:
vals.append(cur.val)
cur = cur.next
cur = head
while cur:
cur.val = vals.pop()
cur = cur.next
return head
Brute Force Complexity
Time: O(N). Space: O(N) auxiliary. Two passes. Doesn’t reverse pointers — only mutates values, which violates the spirit (and breaks if val is immutable, e.g., final field).
Optimization Path
We want O(1) extra space by manipulating pointers directly. Two canonical approaches:
- Iterative three-pointer:
prev,cur,next. Walk forward, flipcur.nexttoprev, advance. - Recursive: reverse the tail, then attach the head behind it. Beautiful but O(N) stack.
Iterative is preferred for production (no stack-overflow risk). Recursive is preferred for explaining the idea. Strong candidates write both.
Final Expected Approach
Iterative, three pointers.
def reverse_list(head):
prev = None
cur = head
while cur:
nxt = cur.next # save the rest of the list
cur.next = prev # flip
prev = cur # advance prev
cur = nxt # advance cur
return prev # prev is the new head
Recursive form:
def reverse_list_rec(head):
if head is None or head.next is None:
return head
new_head = reverse_list_rec(head.next)
head.next.next = head
head.next = None
return new_head
Data Structures Used
The input list itself; three local pointers. No new allocation.
Correctness Argument
Iterative invariant: before each iteration, the sub-list ending at prev is fully reversed and cur points at the head of the not-yet-reversed remainder. The body of the loop preserves the invariant: we save cur.next, flip cur.next to point at the reversed prefix, then advance prev and cur by one. When cur is None, the entire input has been processed and prev is the head of the reversed list.
Recursive correctness: by induction on length. Base: list of length 0 or 1 is its own reverse. Inductive step: assume reverse_list(head.next) correctly returns the head of the reversed tail. The original head is now at the end of the reversed tail; head.next is the last node of the reversed-tail (the original second node). Set head.next.next = head to append head; set head.next = None to terminate.
Complexity
Iterative: O(N) time, O(1) space. Recursive: O(N) time, O(N) space due to the call stack.
Implementation Requirements
- Three named pointers:
prev,cur,nxt(ornext— but watch out for shadowing built-ins in some languages). - Initialize
prev = null. Top bug: forgetting this means the head’snextbecomes self-referential or stale. - Save
cur.nextbefore overwriting it. Forgetting to save loses the rest of the list. - Return
prev, notcur(which isnullat termination). - For recursion: handle the base case at
head is Nonefirst.
Tests
- Smoke:
1 → 2 → 3 → 4 → 5. - Unit: length 1, length 2, length 3.
- Edge:
nullhead; list with all-equal values; list with cycle (should not be passed in — but if defensive, detect with Floyd’s). - Large: N = 10⁵; if recursive, expect StackOverflow in Java/Python without
sys.setrecursionlimit. - Random: build random lists, reverse, reverse again, assert equality with the original.
- Invalid: ensure the original head’s
nextisnullafter reversal (it’s now the tail).
Follow-up Questions
- “Reverse a sublist
[m, n].” → “Reverse Linked List II” — needs a dummy node and careful pointer wiring. - “Reverse in groups of K.” → “Reverse Nodes in K-Group” — apply the iterative reverse on each chunk.
- “Reverse a doubly linked list.” → swap
prev/nextper node. - “Detect and handle a cycle before reversing.” → Floyd’s tortoise and hare.
- “Iterative without saving
next(write it as a swap).” → trickier; usually a teaching exercise. - “Why is iterative preferred in production?” → no stack-overflow risk on long lists.
Product Extension
A document undo/redo stack implemented as a linked list. To replay actions in reverse temporal order, you reverse the list. In-place reversal is preferred because the list nodes carry references to large objects (action payloads) and reallocation would be expensive. The null-handling and dummy-node patterns transfer directly to LRU-cache implementations and free-list management.
Language/Runtime Follow-ups
- Python: no tail-call elimination;
sys.setrecursionlimit(N+100)for deep lists. Default recursion limit is 1000. - Java: typical stack ~500K frames; expect
StackOverflowErrorfor recursive on N=10⁵+. - Go: stack starts small (8 KB) and grows automatically. Recursion is safe for moderate N. Pointers are explicit (
*ListNode). - C++: stack usually 1–8 MB; recursive risk depends. Use
-fsanitize=addressto catch use-after-free if you mis-rewire. - JS/TS: V8 doesn’t reliably tail-call optimize. Iterative is the only safe choice for large N.
- Pointer aliasing: mutating
cur.nextwhile another reference (e.g.,head) still points to the same node is exactly the operation we want — but only because we intentionally preserve the oldnextinnxtfirst.
Common Bugs
- Losing the rest of the list — overwriting
cur.nextbefore saving it. Symptoms: list has 2 elements after “reverse.” Fix: always save first. - Forgetting to set the original head’s
nexttonull— in recursive form, omittinghead.next = Nonemakes the original head point at itself or its successor, creating a cycle. - Returning
headinstead ofprev— returns the now-tail of the reversed list. Always returnprev. - Initializing
prevtoheadinstead ofnull— first iteration creates a self-loop. - Using
nextas a variable name in Python — shadows the built-in iterator function. Harmless here but tags you as junior.
Debugging Strategy
- Hand-trace on a 3-node list. Draw arrows. After each iteration, write down where
prev,cur,nxtpoint. - After running, walk the result and assert it terminates at
nullwithin N steps (cycle check). - If output is shortened (only 1 element), you lost the rest — debug the save step.
- If output reverses but the last element points back to the previous, you forgot the
head.next = None(recursive only).
Mastery Criteria
- Wrote the iterative version cleanly in under 90 seconds.
- Wrote the recursive version on demand and explained the inductive correctness argument.
- Identified the
nullhead and length-1 edge cases without prompting. - Stated why iterative is safer for production.
- Drew the three-pointer dance on a whiteboard (or in comments) for one full iteration.
- Acknowledged that recursive depth = N and called out the stack risk.
Lab 05 — Stack & Queue Applications: Valid Parentheses + Min Stack
Goal
Master the stack as a structural matching tool, the dual-stack technique for augmented operations, and the queue/deque distinction. The deliverable: validate balanced bracket strings in linear time, then extend to a Min Stack supporting O(1) push, pop, top, getMin.
Background Concepts
LIFO discipline; stack invariants; dual-stack trick for tracking auxiliary state; deque vs queue. Review the Stacks and Queues sections of the Phase 1 README, plus runtime concept 1 (stack vs heap memory).
Interview Context
Valid Parentheses is the warm-up; Min Stack is the follow-up. Together they probe whether you grasp the stack as a general tool (not just a recursion bookkeeping device). Asked at Amazon, Google, Bloomberg, Microsoft. The signal: do you generalize from “match ()” to “match (){}[]”? Do you reach for the dual-stack trick on Min Stack instead of O(N) getMin?
Problem Statement
Part A (Valid Parentheses): Given a string s of bracket characters from (){}[], return true iff every opener is matched with the correct closer in the correct order.
Part B (Min Stack): Design a stack that supports push(x), pop(), top(), and getMin() all in O(1).
Constraints
- A:
1 ≤ |s| ≤ 10^4;scontains only the six bracket characters. - B:
pop,top,getMinare not called on an empty stack; up to3 · 10^4operations.
Clarifying Questions
- A: Are non-bracket characters possible? (Per constraints, no — but if extended, ignore them.)
- A: Is the empty string valid? (Conventionally yes — vacuous truth.)
- B: Are integer values bounded? (Affects whether
intsuffices.) - B: Is
getMinof an empty stack defined? (Per constraints, never called on empty.) - B: Should
topandpopbe separate, or ispopreturning the value acceptable? (LeetCode classic: separate. Match the spec.)
Examples
A:
| Input | Output |
|---|---|
"()" | true |
"()[]{}" | true |
"(]" | false |
"([)]" | false (interleaved, not nested) |
"{[]}" | true |
"" | true |
B: Sequence push(-2), push(0), push(-3), getMin() → -3, pop(), top() → 0, getMin() → -2.
Initial Brute Force
A: Repeatedly scan for adjacent matching pairs (), [], {} and remove them. If string empties, valid; else invalid.
def valid_brute(s):
while True:
new = s.replace("()", "").replace("[]", "").replace("{}", "")
if new == s: break
s = new
return s == ""
B: Push x to a normal stack. getMin walks the entire stack each call.
Brute Force Complexity
A: O(N²) worst case (each pass removes constant pairs). B: getMin is O(N) per call, total O(N²) for N operations.
Optimization Path
A: Single pass with a stack: push openers, on closer pop and verify match.
B: Maintain a parallel min-stack so each push records the current minimum. On pop, also pop from min-stack. getMin returns the top of min-stack.
Final Expected Approach
A — Valid Parentheses:
def is_valid(s):
pairs = {')': '(', ']': '[', '}': '{'}
stack = []
for c in s:
if c in '([{':
stack.append(c)
else:
if not stack or stack.pop() != pairs[c]:
return False
return not stack
B — Min Stack:
class MinStack:
def __init__(self):
self.s = [] # main stack
self.m = [] # min stack: m[i] = min of s[0..i]
def push(self, x):
self.s.append(x)
self.m.append(x if not self.m else min(x, self.m[-1]))
def pop(self):
self.s.pop()
self.m.pop()
def top(self):
return self.s[-1]
def getMin(self):
return self.m[-1]
Data Structures Used
- A: A single stack of opener characters.
- B: Two parallel stacks, both supporting O(1) push/pop.
Correctness Argument
A: Loop invariant — at each iteration, the stack contains the unclosed openers of the prefix of s consumed so far, in order. A closer is valid iff it matches the most-recent opener (LIFO). At end, an empty stack means all openers were closed in order.
B: Invariant — m[i] is min(s[0..i]). When we push x, the new minimum is min(x, current_min) — pure local computation. When we pop, both stacks shrink; the new top of m is correct because it was correct before this push. Hence getMin is m[-1], O(1).
Complexity
A: O(N) time, O(N) space (worst case, all openers). B: O(1) time per operation, O(N) total space.
Implementation Requirements
- Use a single map
closer → openerto avoid six-wayif/else. - For Min Stack, use two stacks (or one stack of pairs); never recompute min by scanning.
- Don’t pre-validate
s(e.g., for invalid characters) unless the problem demands. - Handle the empty stack case before popping in
is_valid.
Tests
- A smoke:
"()[]{}"valid;"(]"invalid. - A unit: unbalanced opener-only
"(("; closer-first")"; nested correctly"{[]}"; interleaved"([)]". - A edge: empty string; single character.
- A large: 10⁴ openers followed by 10⁴ closers; should still run in milliseconds.
- B smoke: the canonical sequence above.
- B edge: push the same value twice, pop, ensure min is still correct (this is the “duplicate-min trap” — naive single-stack solutions fail here).
- Random: generate random op sequences; cross-check against a “min via scan” reference.
Follow-up Questions
- A: “What if
smay contain other characters (letters, digits)?” → ignore them or treat as “skip.” - A: “Return the index of the first invalid bracket.” → modify the loop to return
iinstead ofFalse. - A: “Generate all valid bracket strings of length 2N.” → that’s “Generate Parentheses” (Lab 08).
- B: “Use only one stack.” → store
(value, current_min)as pairs. - B: “Use only constant extra space (no parallel stack).” → encoding trick: store
2x - currentMinwhenx < currentMin, then decode on pop. Watch for overflow. - B: “Add
getMax.” → add a third parallel stack.
Product Extension
A real-time expression evaluator for a spreadsheet engine. As users type formulas, you validate parenthesization on every keystroke (must be O(N) for snappy UX) and maintain a “min/max running aggregate” for selected cells (Min Stack pattern). The two-stack technique generalizes to maintaining any associative aggregate over a stack-shaped sliding context.
Language/Runtime Follow-ups
- Python:
listis a fast stack viaappend/pop(). Don’t uselist.pop(0)(O(N)). - Java: prefer
ArrayDequeoverStack(the latter is synchronized, slower, and inherits fromVector).Deque<Integer>withpush/pop/peek. - Go: slices as stacks:
s = append(s, x)andx, s = s[len(s)-1], s[:len(s)-1]. - C++:
std::stack(LIFO adapter) or juststd::vector.std::stack’spopreturns void; usetopthenpop. - JS/TS:
Array.push/Array.popare O(1) amortized. The same array is fine. - Memory: stacks here grow on the heap (the data structure), even though the conceptual abstraction is named “stack.” Don’t confuse with the call stack.
Common Bugs
- A — popping an empty stack — Python raises
IndexError. Checkif not stackfirst. - A — accepting
"(("— forgetting the finalif stackcheck. The string ends with openers still on the stack. - A — wrong pair table —
{')': '(', ']': '[', '}': '{'}. Off-by-one easy to typo. - B — naive
getMin— scanning the stack is O(N), violating the contract. - B — duplicate-min handling — if you maintain “the min” as a single field and pop the value equal to it without secondary tracking, the min is wrong after pop. Two-stack design avoids this.
- B — pop on empty stack — per constraints not called, but if defensive, raise.
Debugging Strategy
- A: print the stack and the current char at each step; trace
"([)]". - B: print both stacks after each op; verify
m[i] == min(s[0..i]). - For the duplicate-min trap, manually trace
push(-1), push(-1), pop(), getMin() → must still be -1.
Mastery Criteria
- Wrote
is_validcleanly in under 4 minutes; under 5 lines of logic. - Recognized the closer-table pattern over a six-way conditional.
- Designed Min Stack with two stacks; explained why one stack with a single min field fails on duplicates.
- Sketched the “encoded delta” optimization without needing it.
- Handled the empty-stack defensive checks.
- Selected
ArrayDequeoverStackin Java without prompting.
Lab 06 — Heap Priority: Kth Largest In A Stream
Goal
Master the binary heap as the canonical streaming top-K device, the min-heap-of-size-K trick, and the cost model for push/pop. The deliverable: an online data structure that, after O(K) initialization, returns the Kth largest element in O(log K) per add.
Background Concepts
Binary heap as an array; sift-up / sift-down; min-heap vs max-heap; heapify is O(N); push/pop are O(log N). Review the Heaps section of the Phase 1 README and runtime concept 1 (stack vs heap memory — note the distinction between the call stack and the heap data structure).
Interview Context
Asked at Amazon, Apple, Bloomberg, and any role touching streaming systems. The signal: do you reach for a heap immediately when “online K-th largest” is mentioned? Do you choose a min-heap of size K (not a sorted list, not a max-heap)? Do you state the O(log K) per add?
Problem Statement
Design a KthLargest(k, nums) class. The constructor receives the integer k and an initial array nums. The method add(val) inserts val into the stream and returns the Kth largest element among all elements seen so far.
Constraints
1 ≤ k ≤ 10^40 ≤ |nums| ≤ 10^4-10^4 ≤ val, nums[i] ≤ 10^4- At most
10^4calls toadd. - Guaranteed: at the time of any
addreturn, there are at leastkelements seen.
Clarifying Questions
- Is
kfixed for the lifetime of the object? (Yes — set once.) - Are duplicates allowed? (Yes —
add(5)twice keeps both.) - What if fewer than
kelements have been seen? (Per constraints, won’t happen at return time. Confirm.) - Is “Kth largest” 1-indexed? (Yes —
K=1is the maximum.) - Streaming: do we ever remove elements? (No — additions only.)
Examples
KthLargest(3, [4, 5, 8, 2])
add(3) → 4 // sorted desc: 8, 5, 4, 3, 2 → 3rd is 4
add(5) → 5 // 8, 5, 5, 4, 3, 2 → 3rd is 5
add(10) → 5 // 10, 8, 5, 5, 4, 3, 2 → 3rd is 5
add(9) → 8 // 10, 9, 8, 5, 5, 4, 3, 2 → 3rd is 8
add(4) → 8 // 10, 9, 8, 5, 5, 4, 4, 3, 2 → 3rd is 8
Initial Brute Force
Maintain a sorted list. On add, insert in sorted order (O(N)) and read index N-k.
class KthLargestBrute:
def __init__(self, k, nums):
self.k = k
self.arr = sorted(nums)
def add(self, val):
# binary search insertion
import bisect
bisect.insort(self.arr, val)
return self.arr[-self.k]
Brute Force Complexity
bisect.insort is O(log N) for the search but O(N) for the actual insertion (array shift). Total O(N) per add. For 10⁴ adds and 10⁴ initial size: 10⁸ ops. Borderline.
Optimization Path
We don’t need to track all elements. Only the top K. A min-heap of size K keeps the K largest seen, with the smallest of them at the top — that’s the Kth largest.
- On
add: push, then if size > K, pop (the smallest, which is no longer in the top K). - Return
heap[0].
Final Expected Approach
import heapq
class KthLargest:
def __init__(self, k, nums):
self.k = k
self.heap = []
for x in nums:
self.add(x)
def add(self, val):
heapq.heappush(self.heap, val)
if len(self.heap) > self.k:
heapq.heappop(self.heap)
return self.heap[0]
For the constructor, you can do better: take the first k elements, heapify them (O(k)), then for each remaining element, push if it beats the top, else skip. But the simple version above is acceptable and amortizes the same.
Data Structures Used
A binary min-heap. Underlying storage: a dynamic array. Capacity: K.
Correctness Argument
Invariant: self.heap contains the K largest values seen so far (when ≥ K have been seen), and self.heap[0] is the minimum of those — i.e., the Kth largest overall.
After heappush(val): heap may have K+1 elements; the smallest is at the top. Popping removes it. The remaining K elements are still the K largest (we only removed the smallest of K+1, which by definition is excluded from the top K of K+1). Hence self.heap[0] is the Kth largest of K+1 = the Kth largest overall.
Complexity
- Constructor: O(N log K) using the per-element approach; O(N) using bottom-up heapify on first K then sift the rest.
add: O(log K).- Space: O(K).
Implementation Requirements
- Use the language’s built-in min-heap; don’t roll your own unless asked.
- Bound the heap size to K explicitly; if you don’t, you’ve built a sorted set, not the optimization.
- For max-heap-only languages (e.g., Java’s
PriorityQueueis min-heap by default — fine here), use the natural orientation. If you need a max-heap, negate or pass a comparator. - Don’t allocate fresh on every add.
Tests
- Smoke: the canonical example above.
- Unit: K=1 (always returns max); K==N (returns min after each add).
- Edge: empty
numsand a stream that brings size up to K; duplicate values; negative values. - Large: 10⁴ adds of random ints with K=100; assert per-call O(log K) by timing.
- Random: maintain a brute-force sorted-list reference; assert equality of returned value on each call.
- Invalid:
addbefore reaching K elements (per constraints not happening; if defensive, raise or buffer).
Follow-up Questions
- “What if the stream supports
remove(val)?” → switch to a balanced BST or two heaps with lazy deletion. - “Maintain the K smallest.” → max-heap of size K (mirror).
- “K-th most frequent element in a stream.” → counter + heap with re-inserts on count change.
- “Top K trending hashtags over a sliding 1-hour window.” → heap + circular buffer + lazy deletion of stale entries.
- “Implement the min-heap from scratch.” → array-backed, sift-up on push, sift-down on pop, parent at
(i-1)//2, children at2i+1,2i+2. - “Why O(N) heapify rather than N pushes?” → bottom-up sift-down sums to O(N); pushes sum to O(N log N).
Product Extension
A leaderboard service that streams game scores and surfaces the top 100. Memory budget per shard is tight; the min-heap-of-size-K pattern is the standard approach. Combine with sharding (each shard maintains its own top-100; the aggregator maintains a heap of heap-tops). The same pattern powers “top-N alerts,” “p99 latency tracking,” and “trending content” feeds.
Language/Runtime Follow-ups
- Python:
heapqis min-heap only. For max-heap behavior, push-xand negate on pop.heapq.heapify(list)is O(N) in place. - Java:
PriorityQueuedefaults to min-heap.PriorityQueue<Integer> pq = new PriorityQueue<>();. Reversed:new PriorityQueue<>(Comparator.reverseOrder()).pq.poll()andpq.peek()are O(log N) and O(1). - Go: must implement the
heap.Interface(Len,Less,Swap,Push,Pop). Verbose; stand-alone helpers in thecontainer/heappackage. - C++:
std::priority_queue<int>is a max-heap by default. Usestd::priority_queue<int, std::vector<int>, std::greater<int>>for a min-heap. - JS/TS: no built-in heap. Must implement or pull a library. This is a not-uncommon interview surprise.
- Memory model: the data-structure heap lives in the process heap (not the call stack). Sizes up to 10⁴ are trivial.
Common Bugs
- Maintaining a max-heap — works for finding max, but you’d need to extract K elements per call. Wrong tool.
- Forgetting to bound size to K — heap grows to N; per-add cost becomes O(log N) instead of O(log K) (small impact for small N, but conceptually wrong and uses more memory).
- Returning
heap[-1]— Python’sheap[-1]is not the largest; onlyheap[0]is the min. Other indices are unordered. - Off-by-one on K — K=1 should track the maximum; if you accidentally maintain K-1 elements, you’re answering the wrong query.
- Java
PriorityQueuereversed Comparator typo — using(a, b) -> b - aoverflows for large negative ints. UseInteger.compare(b, a).
Debugging Strategy
- After each
add, print the heap. Should be ≤ K elements with the K-th-largest at index 0. - Cross-check against
sorted(all_seen)[-K]. - For perf: time 10⁴ adds; should be milliseconds.
Mastery Criteria
- Selected the min-heap-of-size-K pattern within 30 seconds of hearing “Kth largest streaming.”
- Stated the loop invariant aloud.
- Wrote the implementation in under 5 minutes.
- Identified the K=1 and K=N degenerate cases.
- Knew the language idiom:
heapqPython,PriorityQueueJava,priority_queue<…, greater<…>>C++. - Mentioned the
(a, b) -> b - aoverflow trap in Java. - Sketched the
O(N)bottom-up heapify alternative for the constructor.
Lab 07 — Binary Search Fundamentals
Goal
Master the half-open invariant [lo, hi), the overflow-safe midpoint, the lower-bound / upper-bound generalizations, and the discipline that makes binary search bug-free. The deliverable: implement Search Insert Position cleanly and explain why your loop terminates.
Background Concepts
Sorted arrays; monotone predicates; loop invariants; integer overflow on (lo + hi) / 2. Review the Sorted Arrays / Sorted Sets section of the Phase 1 README.
Interview Context
Binary search is asked at every FAANG and is the #1 source of “I solved it but had off-by-one bugs” complaints. The signal isn’t whether you can find an exact match — it’s whether you can correctly answer “first index where predicate flips from false to true” in one of three loop variants without bugs. Lower-bound and upper-bound are the general tools.
Problem Statement
Given a sorted array nums of distinct integers and a target target, return the index where target is found, or the index where it would be inserted to keep nums sorted.
This is exactly lower_bound (first index i such that nums[i] >= target).
Constraints
1 ≤ |nums| ≤ 10^4-10^4 ≤ nums[i], target ≤ 10^4numsis sorted ascending and contains no duplicates.
Clarifying Questions
- Are duplicates possible? (Per constraints, no — but the lower-bound formulation handles them: returns leftmost.)
- Can
numsbe empty? (Per constraints no, but the implementation handles it vialo = hi = 0.) - Should we return
len(nums)if target exceeds all elements? (Yes — it inserts at the end.) - Is the result expected to be the first match or any match? (For Search Insert, lower-bound semantics: leftmost.)
- Recursive or iterative? (Iterative is preferred — no stack growth.)
Examples
nums | target | Output |
|---|---|---|
[1, 3, 5, 6] | 5 | 2 |
[1, 3, 5, 6] | 2 | 1 |
[1, 3, 5, 6] | 7 | 4 |
[1, 3, 5, 6] | 0 | 0 |
[1] | 1 | 0 |
[] | 5 | 0 |
Initial Brute Force
Linear scan: walk the array, return the first index i where nums[i] >= target. If none, return len(nums).
def insert_pos_brute(nums, target):
for i, x in enumerate(nums):
if x >= target:
return i
return len(nums)
Brute Force Complexity
O(N) time, O(1) space. For 10⁴ elements with 10⁴ queries, 10⁸ ops — borderline. Misses the entire point of “sorted.”
Optimization Path
Sorted + monotone predicate = binary search. The predicate is nums[i] >= target, monotone false → true as i increases. We want the first true index.
Three loop styles compete:
- Closed
[lo, hi]:while lo <= hi, terminate atlo > hi. - Half-open
[lo, hi):while lo < hi, terminate atlo == hi. Recommended. - Inclusive find-or-not-found:
while lo < hi, post-loop checklovalidity.
Half-open is the cleanest because the answer-pointer lo always satisfies the invariant “all indices < lo have predicate false; all indices >= hi have predicate true.” When lo == hi, that’s the boundary.
Final Expected Approach
def search_insert(nums, target):
lo, hi = 0, len(nums) # half-open [lo, hi)
while lo < hi:
mid = lo + (hi - lo) // 2 # overflow-safe
if nums[mid] < target:
lo = mid + 1 # predicate false → exclude mid
else:
hi = mid # predicate true → keep mid as candidate
return lo # lo == hi; first true index
Data Structures Used
The input array. Three integer indices: lo, hi, mid.
Correctness Argument
Invariant: at every iteration, the answer (the smallest index i such that nums[i] >= target, or len(nums) if none) lies in [lo, hi] (closed interval over the half-open search range). Equivalently: nums[lo-1] < target (or lo == 0) and nums[hi] >= target (or hi == len(nums)).
Body: if nums[mid] < target, predicate at mid is false, so the answer is in [mid+1, hi]. We set lo = mid+1. Otherwise predicate at mid is true; the answer is in [lo, mid]. We set hi = mid. Both branches strictly shrink the range.
Termination: each iteration shrinks hi - lo by at least 1 (since mid is in [lo, hi-1]). Loop exits when lo == hi. Invariant gives us: lo is the answer.
Complexity
O(log N) time. O(1) space.
Implementation Requirements
- Use
lo + (hi - lo) // 2, never(lo + hi) // 2— integer overflow in Java/C++/Go for large ints. - Half-open
[lo, hi)withhi = len(nums)initial. - Loop condition
lo < hi. - Update
lo = mid + 1(exclude mid);hi = mid(include mid as candidate). - Return
lo. - Don’t write three nested if/else — there are only two branches.
Tests
- Smoke: the table above.
- Unit: target equals an existing element; target less than all; target greater than all; single-element array (target equal, less, greater).
- Edge: empty array → return 0.
- Large: N = 10⁵, sorted; binary search with random targets. Time should be sub-millisecond.
- Random: generate sorted random arrays; cross-check against linear scan.
- Invalid: array not sorted (undefined behavior; if defensive, assert).
Follow-up Questions
- “Find the last index where the predicate is true (upper-bound).” → flip the predicate; or use
bisect.bisect_right. - “Search in a rotated sorted array.” → modify the comparison: identify which half is sorted.
- “Search for a peak element.” → ternary-search-like: compare
midwithmid+1. - “First bad version (the predicate is the only oracle).” → same exact loop with
is_bad(mid)as the predicate. - “Search a 2D matrix.” → flatten conceptually if rows are sorted continuations; else two passes.
- “Why does
lo + (hi - lo) // 2matter in Python?” → it doesn’t (Python ints are unbounded), but it’s the universal idiom.
Product Extension
A timestamp-indexed log store. Find the first log line at or after a given timestamp: that’s lower_bound. The same primitive powers range queries (lower_bound(start) to upper_bound(end)) and is the basis for B-tree leaf-node lookups. Library functions like bisect, lower_bound, Arrays.binarySearch already implement this; a senior engineer reaches for them, not for a hand-rolled loop.
Language/Runtime Follow-ups
- Python:
bisect.bisect_left(nums, target)is the library answer. Returns exactly the lower-bound index. - Java:
Arrays.binarySearch(arr, target)returns either the match index or-(insertion_point) - 1. Decode withresult < 0 ? -result - 1 : result. Note the bit-shift idiom(lo + hi) >>> 1for unsigned-right-shift to avoid overflow. - Go:
sort.SearchInts(arr, target)returns the lower-bound directly. - C++:
std::lower_bound(v.begin(), v.end(), target) - v.begin(). Returns an iterator; subtractbegin()for the index. - JS/TS: no library. Must implement.
- Overflow: Java/C++ ints are 32-bit by default.
(lo + hi)can overflow when both ~2³⁰. Use the safe form.
Common Bugs
(lo + hi) // 2overflow in Java/C++/Go (32-bit ints). Uselo + (hi - lo) // 2or>>> 1.- Wrong update direction —
lo = mid(instead ofmid + 1) on the false branch. Causes infinite loop whenlo + 1 == hi. - Closed-interval
while lo <= hiwith half-open updates — mixing the two styles. Pick one and stick to it. - Returning
midinstead oflo—midis wherever the loop happens to stop, not the answer. - Off-by-one on the initial
hi—hi = len(nums) - 1for closed;hi = len(nums)for half-open. - Forgetting the empty-array case — half-open form handles it naturally (
lo = hi = 0); closed form needs an explicit check.
Debugging Strategy
- Print
lo,hi,mid, andnums[mid]each iteration. The range should strictly shrink. - If you hit an infinite loop, you almost certainly have
lo = mid(notmid + 1) on the false branch. - For random testing, compare against
bisect.bisect_leftas the reference.
Mastery Criteria
- Wrote the half-open form from memory in under 2 minutes, no off-by-ones.
- Stated the invariant aloud: “all indices < lo are false; all indices ≥ hi are true.”
- Identified the overflow trap and used the safe midpoint.
- Recognized that Search Insert Position is
lower_bound. - Knew the library function in Python, Java, Go, C++.
- Solved a follow-up (rotated sorted array OR upper-bound) in under 10 minutes by reusing the same skeleton.
Lab 08 — Recursion & Stack: Generate Parentheses
Goal
Master backtracking with partial-state validity, the recursion tree as a mental model, the bound on recursion depth, and the Catalan-number cost analysis. The deliverable: enumerate all well-formed parenthesizations of N pairs and explain why the count is C_n (the n-th Catalan number).
Background Concepts
Recursion as a tree of choices; partial-state pruning vs full-state validation; recursion depth = call-stack frames; iterative backtracking as an explicit-stack alternative. Review runtime concept 10 (recursion depth) in the Phase 1 README and the Stacks section.
Interview Context
Generate Parentheses is asked at Google, Microsoft, Meta. The signal: do you generate only valid prefixes (prune early) instead of generating all 2^(2n) strings and filtering? Do you know the Catalan-number complexity? Can you also produce an iterative version using an explicit stack?
Problem Statement
Given an integer n, return all combinations of well-formed parentheses using exactly n pairs of ( and ).
Constraints
1 ≤ n ≤ 8— the count grows asC_n = (2n)! / ((n+1)! n!).C_8 = 1430.- Output order is not specified; any valid enumeration is acceptable.
Clarifying Questions
- Should output be sorted? (Usually no — but lexicographic falls out naturally if we always try
(before).) - Is duplication possible? (No — each generated string is unique by construction.)
- Should we return a list or stream the results? (List is canonical; streaming/yield is a follow-up.)
- Empty case
n = 0? (Per constraintsn ≥ 1. If allowed: return[""].) - Are the parentheses always
(and)? (Yes for the canonical problem; brackets and braces is a generalization.)
Examples
n | Output |
|---|---|
| 1 | ["()"] |
| 2 | ["(())", "()()"] |
| 3 | ["((()))", "(()())", "(())()", "()(())", "()()()"] |
| 4 | 14 strings |
Initial Brute Force
Generate all 2^(2n) strings of length 2n over {(, )}. Filter by validity (use the stack from Lab 05). Return the survivors.
def gen_brute(n):
out = []
def rec(s):
if len(s) == 2*n:
if is_valid(s): # Lab 05 routine
out.append(s)
return
rec(s + "(")
rec(s + ")")
rec("")
return out
Brute Force Complexity
2^(2n) strings; each takes O(n) to validate. Total O(n · 4^n). For n = 8: ~10⁵ operations — fast, but for n = 16 it would be billions.
Optimization Path
Prune as we build. Track (open, close) counts; the rules are:
- We may add
(ifopen < n. - We may add
)ifclose < open(otherwise we’d close before opening).
Every leaf of this pruned tree is a valid string; no validation needed. The number of leaves is exactly C_n.
Final Expected Approach
def generate_parentheses(n):
out = []
def backtrack(s, opens, closes):
if len(s) == 2 * n:
out.append(s)
return
if opens < n:
backtrack(s + "(", opens + 1, closes)
if closes < opens:
backtrack(s + ")", opens, closes + 1)
backtrack("", 0, 0)
return out
Data Structures Used
- The recursion call stack (depth =
2n). - An accumulator string built up by concatenation (or, for efficiency, a list of chars joined at the leaf).
- An output list of strings.
Correctness Argument
Soundness: every leaf has length 2n, opens == n, closes == n (else we wouldn’t reach length 2n under the pruning rules). At every prefix, closes ≤ opens (we only added ) when closes < opens). Therefore every leaf is balanced.
Completeness: any valid parenthesization satisfies the same two rules at every prefix (it’s the characterization of valid prefixes). Therefore the recursion explores it. By induction on length: every valid prefix s of length < 2n extended by ( (if extensible) or ) (if extensible) appears in the tree.
Uniqueness: at each node we make distinct choices (( vs )), so two leaves cannot have the same string.
Complexity
- Number of leaves:
C_n = (2n)! / ((n+1)! n!) ≈ 4^n / (n^(3/2) · √π). - Cost per leaf:
O(n)to copy the final string. - Total time:
O(n · C_n)=O(4^n / √n). - Space: output is
O(n · C_n). Recursion stack isO(n).
Implementation Requirements
- Two counters:
opens,closes. Don’t track the full prefix’s validity — the counters are sufficient. - Termination at
len(s) == 2 * n, not when both counters hitn(equivalent, but the length check is clearer). - Pass
simmutably (string concat) for clarity, or mutate a list andappend/popfor performance — but forn ≤ 8the difference is negligible. - Don’t generate then filter. The whole point is to not visit invalid branches.
Tests
- Smoke:
n = 3→ 5 strings. - Unit:
n = 1→["()"];n = 2→ 2 strings. - Edge:
n = 0(if allowed) →[""]. - Property: count of returned strings equals
C_n(compute reference Catalan number). - Property: every string in the output is valid (run Lab 05’s
is_valid). - Property: all strings are distinct (length =
len(set(out))). - Large:
n = 8returns 1430 strings in milliseconds.
Follow-up Questions
- “Generate iteratively using an explicit stack.” → push partial states
(s, opens, closes); pop and expand. - “Return only the count, not the strings.” → that’s just
C_n; closed form:comb(2n, n) // (n + 1). - “Brackets, braces, and parens (multi-type).” → much harder; can’t be solved by simple counters because the closer must match the most-recent opener.
- “Stream results lazily (generator/yield).” → in Python,
yieldfrom each leaf; saves memory. - “Memoize.” → the canonical formulation has no overlapping subproblems on
(opens, closes, s)becausesis unique at every state. If you parametrize by just counts, you lose the actual string. - “Why is the count
C_n?” → bijection with Dyck paths, balanced trees ofn+1leaves, etc.
Product Extension
A SQL/expression-grammar generator for fuzz testing. Generating syntactically valid parenthesized expressions is a backtracking-with-pruning problem; arbitrary depth-bounded grammars use the same technique. The code-generation engine inside any compiler’s “synthesize a small valid program” tool uses this exact pattern.
Language/Runtime Follow-ups
- Python: strings are immutable, so each
s + "("allocates. For largern, build with alistand"".join(...)at the leaf. - Java: use
StringBuilderanddelete/setLengthat backtrack — the canonical mutable-builder pattern. Pass the builder by reference; remember to undo each append on return. - Go: strings are immutable; use
[]byteorstrings.Builder. Beware: astrings.Builderdoes not support truncation; use a byte slice with a manual length pointer. - C++: use
std::stringmutated in place withpush_back/pop_back. Pass by reference. - JS/TS: strings are immutable; concat is fine for small
n. For larger, use an array. - Recursion depth:
2n. Forn ≤ 8, depth ≤ 16 — trivial. Evenn = 1000(academic) is safe in most languages. - Tail-call optimization: absent in Python and JS; this code isn’t tail-recursive anyway because there are two recursive branches.
Common Bugs
- Adding
)without checkingcloses < opens— generates)(...prefixes that can never become valid; produces duplicates and invalid strings. - Adding
(without checkingopens < n— overshoots; never closes; never reaches the leaf condition. - Wrong termination —
if opens == n and closes == ninstead ofif len(s) == 2nis fine but harder to reason about. - Backtracking with mutation but not undoing — append
(, recurse, forget to pop before recursing again. Adds spurious chars. - Catalan miscount — saying complexity is
O(2^(2n))instead ofO(4^n / √n)is a forgivable but suboptimal answer.
Debugging Strategy
- Print the recursion tree: indent by
len(s)and show(s, opens, closes). - Run for
n = 2and verify the output is exactly["(())", "()()"]. - Count outputs and compare to
comb(2n, n) // (n + 1).
Mastery Criteria
- Identified backtracking-with-pruning as the right tool within 60 seconds.
- Wrote the two pruning rules without help.
- Stated complexity as
O(n · C_n)≈O(4^n / √n). - Acknowledged the Catalan-number connection.
- Wrote the iterative-with-explicit-stack version on demand.
- Selected the appropriate language idiom (StringBuilder, []byte, etc.) and remembered to undo mutations on backtrack.
Lab 09 — Tree Traversal Fundamentals
Goal
Master the three depth-first traversals (preorder, inorder, postorder) in both recursive and iterative forms, plus level-order (BFS). Understand the explicit-stack simulation of recursion, the postorder trick, and Morris traversal as the O(1)-space follow-up. The deliverable: implement iterative inorder cleanly and explain the stack invariant.
Background Concepts
Binary trees; DFS vs BFS; recursion as implicit stack; explicit stack as iterative replacement; visited flags. Review the Trees section and the Stacks section of the Phase 1 README.
Interview Context
Tree traversal is a Day-1 interview topic. Recursive forms are trivial; the interesting signal is iterative inorder (the canonical “implement recursion with a stack” question). Postorder iteratively is harder still — and Morris traversal (O(1) space) shows up in senior interviews.
Problem Statement
Given the root of a binary tree, return the inorder traversal as a list. Implement iteratively (no recursion).
class TreeNode:
def __init__(self, val=0, left=None, right=None):
self.val = val
self.left = left
self.right = right
Constraints
- Number of nodes in
[0, 10^4]. -100 ≤ Node.val ≤ 100.- The tree is not necessarily balanced.
Clarifying Questions
- What’s the node definition? (As above — confirm with interviewer.)
- Empty tree allowed? (Yes — return
[].) - Duplicate values? (Allowed; doesn’t affect traversal.)
- Are we limited on stack space? (For
10^4nodes in a degenerate (linked-list) tree, recursive blows Python’s default 1000-deep stack. Iterative is required.) - Must we use O(1) extra space? (If yes — Morris traversal. Otherwise the explicit stack is fine.)
Examples
| Tree | Inorder |
|---|---|
[1, null, 2, 3] (LeetCode array form) | [1, 3, 2] |
[] | [] |
[1] | [1] |
BST [2, 1, 3] | [1, 2, 3] (sorted!) |
Initial Brute Force
Recursive — visit left, root, right.
def inorder_recursive(root):
out = []
def rec(node):
if not node: return
rec(node.left)
out.append(node.val)
rec(node.right)
rec(root)
return out
Brute Force Complexity
O(N) time, O(H) space (recursion stack, where H = tree height). H = N for a degenerate tree, log N for balanced.
Optimization Path
The iteration cost is the same — O(N) — but recursion uses the system stack which has a fixed limit (~1000 frames in Python by default). For N = 10⁴ in a worst-case skewed tree, recursion crashes. We replace with an explicit stack.
The pattern: “go left as far as possible, pushing each node along the way; when we can’t go further left, pop, visit, then move to right child and repeat.”
Final Expected Approach
def inorder_iterative(root):
out, stack, node = [], [], root
while node or stack:
while node: # walk left, pushing
stack.append(node)
node = node.left
node = stack.pop() # leftmost unvisited
out.append(node.val) # visit
node = node.right # explore right subtree
return out
For preorder: visit before descending left.
def preorder_iterative(root):
if not root: return []
out, stack = [], [root]
while stack:
node = stack.pop()
out.append(node.val)
if node.right: stack.append(node.right) # right first → left popped first
if node.left: stack.append(node.left)
return out
For postorder: hardest. Two clean approaches:
- Modified preorder + reverse: do preorder but visit right before left; reverse the result.
- Visited-flag trick: push
(node, visited)tuples; on first pop, push back as visited and push children; on second pop, emit.
def postorder_iterative(root):
if not root: return []
out, stack = [], [root]
while stack:
node = stack.pop()
out.append(node.val)
if node.left: stack.append(node.left)
if node.right: stack.append(node.right)
return out[::-1]
Data Structures Used
- An explicit stack (list/deque/
Stack/std::stack). - A pointer/cursor
node. - An output list.
- For Morris: only the output list and tree pointers themselves.
Correctness Argument
Inorder invariant: at the start of each outer-loop iteration, the stack contains the chain of ancestors (along left-edges) of the next-to-visit node, all of whose left subtrees are pending. The inner while node walk pushes new ancestors. After popping, we’ve finished its left subtree (it was either null or fully consumed in earlier iterations); we visit it; then move to its right subtree, where the same invariant resumes.
Termination: every node is pushed exactly once and popped exactly once (visited once). N pushes + N pops = O(N) work. Loop ends when both node is null and stack is empty — meaning we’ve returned from the rightmost-rightmost subtree.
Complexity
O(N) time, O(H) auxiliary space (the stack holds at most one chain of ancestors).
For a skewed tree, H = N. For balanced, H = log N.
Implementation Requirements
- Initialize
node = root,stack = []. - Outer loop:
while node or stack. - Inner loop walks left and pushes.
- After inner loop: pop, append val, set
node = popped.right. - Don’t push
None. Don’t visit on the way down. - For preorder, push right before left (LIFO order).
- For postorder, the “modified preorder + reverse” form is the cleanest one-pass solution.
Tests
- Smoke: the example table.
- Unit: single node; empty tree; left-skewed (degenerate); right-skewed; balanced BST.
- Property: inorder of a BST is sorted ascending.
- Property: preorder + inorder uniquely determine a binary tree (round-trip test if you implement the reconstruction).
- Edge: root with only-left child; root with only-right child.
- Large: N = 10⁴ skewed tree — must not stack-overflow.
Follow-up Questions
- “Implement Morris traversal (O(1) space).” → temporarily rewire the tree using “threaded” pointers from inorder predecessors back to successors; restore on the way through.
- “Level-order traversal.” → BFS with a queue.
- “Zigzag level-order.” → BFS with alternating direction; reverse every other level.
- “Reconstruct a tree from its preorder + inorder.”
- “Boundary traversal of a tree.” → combination of left boundary, leaves left-to-right, right boundary reversed.
- “Verify a BST.” → inorder traversal must be strictly ascending; or carry
(min, max)bounds recursively.
Product Extension
A directory tree being indexed: BFS gives breadth-first crawl (sibling priority); preorder DFS visits parent before children (renderers); postorder visits children before parent (size accumulation, deletion). A code formatter walks the AST in postorder so children’s formatted text is available when the parent emits its own. A serializer uses preorder. The choice of traversal is a design decision tied to dependency direction.
Language/Runtime Follow-ups
- Python:
listworks as stack viaappend/pop.collections.dequeis faster for very deep stacks. Default recursion limit is 1000 —sys.setrecursionlimit(10**5)if recursing on huge trees. - Java:
Deque<TreeNode> stack = new ArrayDeque<>(). Avoidjava.util.Stack(legacy, synchronized). For BFS, useDeque<TreeNode> q = new ArrayDeque<>(). - Go: slices as stacks:
stack = append(stack, n), pop withstack = stack[:len(stack)-1]. No generic stack in stdlib pre-1.18. - C++:
std::stack<TreeNode*>andstd::queue<TreeNode*>. - JS/TS: array
push/pop. For BFS,Array.shift()is O(N); use a real deque or an index pointer to avoid quadratic blowup on large trees. - Recursion depth: Python ~1000 default; Java ~10⁴ on default
-Xss; Go grows stacks dynamically. For 10⁴ skewed trees, iterative is mandatory in Python.
Common Bugs
- Pushing
Noneonto the stack — bloats the stack and requires defensive pops. - Visiting on the way down for inorder — that’s preorder.
- In preorder iterative, pushing left before right — gives reverse-of-preorder.
- For postorder, forgetting to reverse in the modified-preorder approach.
- BFS using
list.pop(0)in Python — O(N) shift on every level; quadratic on deep trees. Usecollections.dequeandpopleft(). - Inner loop not consuming left children — only pushing root; you never reach the leftmost node.
- Mutation in Morris traversal — forgetting to restore the threaded pointer; leaves the tree corrupted.
Debugging Strategy
- Print stack contents and
nodeat the top of each outer-loop iteration. - For inorder, the first visit should be the leftmost node.
- For a known BST, the output of inorder must be sorted; if not, the loop is wrong.
- For large skewed inputs, iterative must finish without
RecursionError.
Mastery Criteria
- Wrote iterative inorder from memory in under 3 minutes.
- Stated the stack invariant (chain of ancestors along left edges).
- Wrote preorder iteratively and explained the right-before-left push order.
- Articulated two postorder strategies (reverse-preorder vs visited-flag).
- Knew that BFS uses a queue; can explain why
list.pop(0)is wrong in Python. - Could sketch Morris traversal at a high level: rewire to predecessors, restore on the way through.
- Recognized that inorder of a BST is sorted (and used it to verify a BST).
Phase 2 — Standard Coding Interview Patterns
Target level: Medium → Medium-Hard Expected duration: 4 weeks (12-week track) / 4 weeks (6-month track) / 4 weeks (12-month track) Weekly cadence: ~7 patterns introduced per week + 50–80 problems applying them under the framework
Why This Phase Is The Keystone
Phase 0 fixed your execution. Phase 1 fixed your vocabulary. Phase 2 fixes the only thing standing between you and a 95% Medium solve rate: pattern recognition.
Here is the empirical claim, and it is the entire reason this phase exists:
Any unseen LeetCode Medium becomes a 5-minute problem if you immediately recognize the pattern. The recognition takes ~30 seconds. The remaining 4–5 minutes are mechanical: instantiate the template, adapt to the problem’s specifics, write clean code, test.
Candidates who fail Mediums almost never fail because the pattern was hard. They fail because they did not recognize the pattern, so they tried to derive the algorithm from first principles in 25 minutes — a task the original algorithm researcher needed weeks for. Pattern recognition is not memorization; it is the compiled, searchable index of the entire algorithmic literature, indexed by problem-statement signal.
This is the difference between a candidate who looks at “find longest substring with at most K distinct characters” and thinks “sliding window with a frequency map, variable-size, shrink while violation, O(N)” in 20 seconds — and one who thinks “hmm, maybe two pointers? or sort? or…” and starts coding the wrong thing.
The 28 patterns below cover >90% of the problems asked at Big Tech, infrastructure companies, quant firms, and systems-engineering interviews. They are not all the patterns in existence — Phases 3–7 add advanced data structures, hard graphs, DP families, and competitive-programming techniques. But these 28 are the ones that, once internalized, transform Medium-level problems from “puzzles to solve” into “templates to instantiate”.
What You Will Be Able To Do After This Phase
- For any Medium problem, recognize the dominant pattern in <2 minutes of reading the problem statement.
- For each of the 28 patterns, write the canonical template from memory in <5 minutes.
- Distinguish between superficially-similar patterns (e.g., binary search on index vs binary search on answer) by their signal, not their syntax.
- Combine two patterns when one alone is insufficient (e.g., monotonic deque inside a sliding window; trie inside a backtracking DFS).
- Diagnose, when a pattern almost fits but not quite, exactly which generalization is needed (e.g., “this is sliding window but the window is variable and we need the max — we need a monotonic deque, not just a counter”).
- Communicate the pattern out loud at the moment of recognition: “This is X because of signal Y; the template is Z; expected complexity is W; the canonical pitfall is P.”
How To Read This Phase
This README is a reference manual for all 28 patterns, plus a recognition cheat sheet, plus a mastery checklist. Each pattern entry has a fixed structure:
- Signal recognition — the words/structure in the problem statement that should fire this pattern within 2 minutes of reading
- Canonical template — pseudocode you should be able to write from memory
- Complexity — time and space, with the constants that matter
- Common variants — the family tree (e.g., sliding window has fixed-size, variable-size, count-based variants)
- Classic problems — 4–8 LeetCode problems where this pattern is the intended solution
- Common bugs — the specific failure modes seen on this pattern in interviews
Read it linearly the first time. Refer back to specific patterns as you work the labs. After all labs, re-read the cheat-sheet table at the bottom — it should now read as obvious.
A Word On The 28 Patterns As A System
The patterns are not 28 unrelated tricks. They form a small number of meta-strategies:
- Linear scans with state (1, 2, 3, 4, 5, 9, 10, 11) — one pass, maintain a structure
- Reduce-to-sorted (6, 7, 11) — sort first, then exploit order
- Decision-on-monotonic-axis (8) — binary search where the axis is the answer itself
- Local-update primitives on linear/tree/graph topology (12, 13, 14, 15, 16, 17, 18) — propagate information along edges/pointers
- Enumerate with pruning (19) — exhaustive search with backtracking
- Memoize over a state space (20, 21, 22, 23, 24, 25) — cache answers to a DAG of subproblems
- Specialized structures for prefix/order queries (26, 27, 28) — trie, heap, K-way merge
Recognizing the meta-strategy first, then drilling down to the specific pattern, is often faster than trying to match all 28 patterns linearly.
Inline Pattern Reference
1. Two Pointers (opposite ends + same direction)
Signal Recognition (<2 min)
- The input is sorted (or can be sorted cheaply) and the problem asks for a pair/triplet with a property.
- The problem says “in-place” and you are scanning an array.
- The answer is symmetric: it depends on values from both ends shrinking inward.
- “Find pair such that
a + b = target” with sorted input. - “Remove duplicates in place” / “Move zeros”.
Canonical Template (Opposite Ends)
l, r = 0, len(a) - 1
while l < r:
if condition(a[l], a[r]):
# record / move both
l += 1; r -= 1
elif a[l] + a[r] < target:
l += 1
else:
r -= 1
Canonical Template (Same Direction / Read-Write Pointers)
write = 0
for read in range(len(a)):
if keep(a[read]):
a[write] = a[read]
write += 1
return write # new length
Complexity
Time O(N) (each pointer moves monotonically — total moves bounded by N). Space O(1).
Common Variants
- Opposite ends: Two Sum on sorted, 3Sum, container with most water, valid palindrome.
- Same direction (slow/fast): remove duplicates, move zeros, partitioning around a pivot.
- Two arrays merge: merge two sorted arrays / lists.
- Cycle detection (Floyd): linked-list two-pointer where fast moves 2× slow.
Classic Problems
- LeetCode 1 — Two Sum (variant: sorted input becomes two pointers)
- LeetCode 15 — 3Sum
- LeetCode 11 — Container With Most Water
- LeetCode 26 — Remove Duplicates from Sorted Array
- LeetCode 75 — Sort Colors (Dutch national flag)
- LeetCode 125 — Valid Palindrome
- LeetCode 167 — Two Sum II Sorted
- LeetCode 283 — Move Zeroes
Common Bugs
- Forgetting to advance both pointers when a match is recorded → infinite loop.
- Off-by-one in
while l < rvsl <= r(depends on whether single element is meaningful). - Skipping duplicates: forgetting the inner
while l < r and a[l] == a[l+1]: l += 1after a recorded match (3Sum).
2. Sliding Window (fixed size + variable size)
Signal Recognition (<2 min)
- “Longest / shortest / count of subarrays / substrings with property X.”
- “Maximum sum of K consecutive elements.”
- “Subarray containing all of …” / “smallest substring that contains all chars of T.”
- The brute force is O(N²) over all subarrays. The property is monotone as the window grows or shrinks.
Canonical Template (Variable Size, Shrink-While-Violation)
l = 0
state = init()
best = 0
for r in range(len(a)):
state = add(state, a[r])
while violates(state):
state = remove(state, a[l])
l += 1
best = max(best, r - l + 1)
Canonical Template (Fixed Size K)
state = init()
for i in range(K): state = add(state, a[i])
best = report(state)
for r in range(K, len(a)):
state = add(state, a[r])
state = remove(state, a[r - K])
best = update(best, report(state))
Complexity
Time O(N) — each element enters and leaves the window at most once. Space O(window state size).
Common Variants
- Fixed-size: maximum sum, average, min/max via deque.
- Variable-size with constraint to shrink under: at most K distinct, sum ≤ S, no repeats.
- Variable-size with constraint to grow until satisfied: smallest window containing all of T (then shrink while still satisfying).
- Count of “good” windows = count of “good” right endpoints, often
count += r - l + 1after each step.
Classic Problems
- LeetCode 3 — Longest Substring Without Repeating Characters
- LeetCode 76 — Minimum Window Substring
- LeetCode 209 — Minimum Size Subarray Sum
- LeetCode 340 — Longest Substring with At Most K Distinct Characters
- LeetCode 424 — Longest Repeating Character Replacement
- LeetCode 567 — Permutation in String
- LeetCode 992 — Subarrays with K Different Integers (the “exactly K = atMost(K) − atMost(K-1)” trick)
- LeetCode 1004 — Max Consecutive Ones III
Common Bugs
- Updating the answer inside the shrink loop instead of after — leads to recording invalid windows.
- Forgetting that
while(notif) is required when shrinking — a single character can violate by>1. - Counting “exactly K” as
atMost(K)instead ofatMost(K) − atMost(K-1). - For “no repeats”, forgetting that the freq map needs decrement on shrink, not just delete.
3. Prefix Sums (1D + 2D)
Signal Recognition (<2 min)
- “Sum/count over a range
[l, r]” with many queries or asked once with N up to 10^5. - “Subarray with sum equal to K” (prefix sum + hashmap of seen prefix sums).
- “Number of subarrays with sum divisible by K” (prefix sums mod K).
- 2D: “matrix region sum” / “rectangle of ones”.
Canonical Template (1D)
prefix = [0] * (n + 1)
for i in range(n):
prefix[i + 1] = prefix[i] + a[i]
# range sum a[l..r]: prefix[r + 1] - prefix[l]
Canonical Template (2D)
P = [[0] * (m + 1) for _ in range(n + 1)]
for i in range(n):
for j in range(m):
P[i+1][j+1] = a[i][j] + P[i][j+1] + P[i+1][j] - P[i][j]
# region (r1,c1)..(r2,c2):
# P[r2+1][c2+1] - P[r1][c2+1] - P[r2+1][c1] + P[r1][c1]
Complexity
Build O(N) (1D) or O(NM) (2D). Each query O(1). Space O(N) or O(NM).
Common Variants
- Subarray-sum-equals-K with hashmap
{prefix_sum: count}. - XOR prefix for “subarray XOR equals K” — same trick, different operator (any group operator works).
- Mod K prefix for “subarray sum divisible by K” — bucket prefixes by their value mod K.
- Count of negative numbers in a sorted matrix via row prefix.
- 2D rectangle sum, 2D max-sum submatrix.
Classic Problems
- LeetCode 303 — Range Sum Query Immutable
- LeetCode 304 — Range Sum Query 2D Immutable
- LeetCode 560 — Subarray Sum Equals K
- LeetCode 525 — Contiguous Array
- LeetCode 974 — Subarray Sums Divisible by K
- LeetCode 1248 — Count Number of Nice Subarrays
- LeetCode 1314 — Matrix Block Sum
Common Bugs
- Off-by-one:
prefixis size N+1, indexed 0..N. Range[l,r]isprefix[r+1] - prefix[l]. Get this wrong once and it’s wrong forever. - 2D inclusion-exclusion sign flip.
- Initializing the hashmap:
{0: 1}is needed for “subarrays starting at index 0” in subarray-sum-equals-K. - Integer overflow: prefix sums at N=10^5 with values up to 10^9 exceed 32-bit. Use 64-bit.
4. Difference Arrays (range update O(1))
Signal Recognition (<2 min)
- “Apply many range updates
(l, r, +v)then query the final array.” - “How many flights on day X” given
(start, end, count)triples. - “Maximum overlap of intervals.”
- The brute would be O(N · Q); a difference array makes it O(N + Q).
Canonical Template
diff = [0] * (n + 1)
for (l, r, v) in updates:
diff[l] += v
diff[r + 1] -= v
a = [0] * n
cur = 0
for i in range(n):
cur += diff[i]
a[i] = cur
Complexity
O(N + Q) total. Space O(N).
Common Variants
- Booking-system style: count of overlapping intervals at each point.
- 2D difference (imos method): stamp rectangles, prefix-sum twice.
- Sweep line equivalence: events at
landr+1are exactly the events of a sweep; difference array is the “discretized sweep”. - Range add + point query with later updates: Fenwick/BIT becomes more flexible (Phase 3).
Classic Problems
- LeetCode 1109 — Corporate Flight Bookings
- LeetCode 1854 — Maximum Population Year
- LeetCode 2381 — Shifting Letters II
- LeetCode 370 — Range Addition
- LeetCode 731 — My Calendar II
- LeetCode 2536 — Increment Submatrices by One (2D diff)
Common Bugs
- Forgetting the
r + 1cancellation → all suffixes get incremented. - Using
[0] * ninstead of[0] * (n + 1)— causes index OOB ondiff[r + 1]. - For 2D: forgetting the inclusion-exclusion of all four corners.
5. Hashing Patterns (frequency / complement / grouping)
Signal Recognition (<2 min)
- “Find
target - x” for some target → complement in a hashmap. - “Most/least frequent X” → frequency map, often paired with heap/sort.
- “Group by canonical form” (anagrams, isomorphic strings) → grouping map keyed by canonical form.
- “Has any element appeared twice within K positions?” → sliding window of size K with a hashset.
Canonical Templates
# complement
seen = {}
for i, x in enumerate(a):
if (target - x) in seen:
return [seen[target - x], i]
seen[x] = i
# frequency
from collections import Counter
freq = Counter(a)
top = freq.most_common(K)
# grouping by canonical form
from collections import defaultdict
groups = defaultdict(list)
for s in strs:
groups[canonical(s)].append(s)
Complexity
O(N) average (hash). Space O(N) worst case. Adversarial inputs may degrade to O(N²) — see Phase 1 §3.
Common Variants
- Two-Sum (complement).
- Group anagrams (grouping by char-count tuple).
- Longest consecutive sequence (set-membership test for
x-1to find sequence starts). - Subarray sum = K (prefix-sum + complement — see pattern 3).
- Bullet-proof word ladder (wildcards as keys).
Classic Problems
- LeetCode 1 — Two Sum
- LeetCode 49 — Group Anagrams
- LeetCode 128 — Longest Consecutive Sequence
- LeetCode 217 — Contains Duplicate
- LeetCode 219 — Contains Duplicate II
- LeetCode 347 — Top K Frequent Elements
- LeetCode 451 — Sort Characters by Frequency
Common Bugs
- Java
int[]as a key — uses object identity, not value equality. (See Phase 1 lab 03.) - Inserting into
seenbefore the lookup, when the problem needs distinct indices. - Using ordered map when unordered suffices (e.g., Java
TreeMapinstead ofHashMap) → log-N factor. - Reusing a mutable buffer as a key — all keys alias to the latest buffer.
6. Sorting + Greedy (sort to enable greedy)
Signal Recognition (<2 min)
- “Maximum number of non-overlapping …” → sort by end, take earliest end.
- “Minimum number of meeting rooms / arrows / platforms” → sort by start; sweep.
- “Schedule jobs to maximize profit / minimize lateness.”
- “Pair items optimally” → sort one or both, pair by index.
- The brute force is “try all pairings” (factorial); sortedness collapses it to linear.
Canonical Template
a.sort(key=lambda x: x[1]) # sort by end
chosen = []
last_end = -inf
for (s, e) in a:
if s >= last_end:
chosen.append((s, e))
last_end = e
return len(chosen)
Complexity
Time O(N log N) for the sort, O(N) for the sweep. Space O(1) beyond sort buffer.
Common Variants
- Activity selection — sort by end, take earliest end.
- Minimum platforms / arrows — sort by start (or by end for arrows).
- Pairing: sort and pair by index (e.g., “minimum pair-sum to fit a target”).
- Two arrays joined — sort both, two-pointer merge.
- Custom comparator — sort by a derived value (profit/time, deadline-then-profit, etc.) requires proving the exchange argument.
Classic Problems
- LeetCode 56 — Merge Intervals
- LeetCode 252 — Meeting Rooms (and 253 — Meeting Rooms II)
- LeetCode 435 — Non-overlapping Intervals
- LeetCode 452 — Minimum Number of Arrows to Burst Balloons
- LeetCode 502 — IPO (sort by capital, pq by profit)
- LeetCode 630 — Course Schedule III
- LeetCode 881 — Boats to Save People
Common Bugs
- Sorting by the wrong key (start vs end). Activity selection by start is wrong.
- Forgetting to prove the exchange argument before committing to greedy. (See Phase 6.)
- For “non-overlap” problems: confusing
s >= last_end(touching allowed) vss > last_end(strict). - For comparator: subtraction overflow in Java/JS when sorting
intdifferences.
7. Binary Search On Index (sorted array)
Signal Recognition (<2 min)
- The input is sorted (or has a sorted property like a rotated sorted array).
- The task is “find X” / “find first / last X” / “find insertion point”.
- N is large (10^5+), and the brute O(N) is acceptable but O(log N) is wanted (or there are many queries).
Canonical Template (lower_bound)
def lower_bound(a, target):
lo, hi = 0, len(a)
while lo < hi:
mid = (lo + hi) // 2
if a[mid] < target:
lo = mid + 1
else:
hi = mid
return lo # first index with a[i] >= target
Complexity
Time O(log N) per query. Space O(1).
Common Variants
lower_bound,upper_bound, exact-match.- Rotated sorted array — pick the half that is sorted, decide which half contains the target.
- Search in 2D matrix — flatten coordinates, binary search the 1D index, or descend from top-right.
- Find peak — local-property binary search (no global sort required).
Classic Problems
- LeetCode 33 — Search in Rotated Sorted Array
- LeetCode 34 — Find First and Last Position
- LeetCode 35 — Search Insert Position
- LeetCode 74 — Search a 2D Matrix
- LeetCode 153 — Find Minimum in Rotated Sorted Array
- LeetCode 162 — Find Peak Element
- LeetCode 240 — Search a 2D Matrix II (descend from top-right; not binary search per se)
Common Bugs
(lo + hi) // 2overflow in C++/Java — uselo + (hi - lo) // 2.- Wrong loop condition (
<vs<=) interacting with wrong update (midvsmid + 1vsmid - 1) — pick a single canonical form (we use half-open[lo, hi)here) and stick with it. - Off-by-one when reconstructing the actual index after finding the bound.
- For rotated arrays, forgetting that duplicates break the binary search invariant.
8. Binary Search On Answer (parametric / monotonic predicate)
Signal Recognition (<2 min)
- The problem asks for the minimum X such that property P(X) holds (or maximum X such that ¬P).
Pis monotonic in X (if P(X) holds, P(X+1) also holds — or vice versa).- Direct construction is hard, but verifying a candidate answer in O(N) or O(N log N) is easy.
- Constraints: answer’s range is enormous (10^9, 10^18), but verification per candidate is cheap.
- Keywords: “smallest capacity / speed / time”, “largest minimum”, “split into K parts minimize max sum”.
Canonical Template
def feasible(x): ... # returns True if x is a valid answer or larger
lo, hi = LOW, HIGH
while lo < hi:
mid = (lo + hi) // 2
if feasible(mid):
hi = mid
else:
lo = mid + 1
return lo # smallest feasible value
Complexity
Time O(log(range) · cost_of_feasible). Space O(1) beyond feasible.
Common Variants
- Min-max / max-min (split array into K parts to minimize the maximum part sum).
- Capacity / rate (capacity to ship within D days; Koko eating bananas).
- Time (earliest day to finish; latest day before failure).
- K-th smallest in matrix / multiplication table (binary search the value, count “≤ value” entries).
- Floating-point binary search — replace
lo < hiwithhi - lo > epsand pick the right output.
Classic Problems
- LeetCode 410 — Split Array Largest Sum
- LeetCode 875 — Koko Eating Bananas
- LeetCode 1011 — Capacity To Ship Packages Within D Days
- LeetCode 1283 — Find Smallest Divisor Given a Threshold
- LeetCode 1482 — Minimum Number of Days to Make m Bouquets
- LeetCode 668 — Kth Smallest Number in Multiplication Table
- LeetCode 1539 — Kth Missing Positive Number
Common Bugs
- Wrong direction of monotonicity — verify by hand on small cases before committing.
- Wrong search bounds (lo too high → miss the answer; hi too low → infinite loop).
feasiblehas a subtle off-by-one — write and testfeasibleindependently before plugging it into the binary search.- Returning
lo - 1orhi + 1accidentally — the half-open[lo, hi)template returnslo, period.
9. Monotonic Stack (next-greater / histogram / span)
Signal Recognition (<2 min)
- “Next/previous greater/smaller element” on each index.
- “Largest rectangle in histogram” / “max area of submatrix of 1’s” (uses histogram per row).
- “Daily temperatures” / “stock span” / “trapping rainwater” (an O(N) variant).
- The brute force is “for each i, scan right (or left) until …” — O(N²); the monotonic stack collapses it to O(N).
Canonical Template (Next Greater)
n = len(a)
result = [-1] * n
stack = [] # indices, values strictly decreasing
for i in range(n):
while stack and a[stack[-1]] < a[i]:
result[stack.pop()] = a[i]
stack.append(i)
Complexity
Time O(N) — each index pushed and popped at most once. Space O(N) for the stack.
Common Variants
- Next/previous, greater/smaller (4 combinations) — a sign flip and a comparator change.
- Histogram problems: maintain stack of indices with strictly increasing heights; on pop, the popped index sees the current as its right boundary and the new top as its left.
- Sum of subarray minimums — for each element, count subarrays where it is the min.
- Trapping rainwater — stack of decreasing heights; each pop produces a “trapped” volume.
- Sliding window max — uses a monotonic deque (pattern 10), not stack.
Classic Problems
- LeetCode 84 — Largest Rectangle in Histogram
- LeetCode 85 — Maximal Rectangle (histogram per row)
- LeetCode 42 — Trapping Rain Water (stack variant)
- LeetCode 496 — Next Greater Element I
- LeetCode 503 — Next Greater Element II (circular)
- LeetCode 739 — Daily Temperatures
- LeetCode 901 — Online Stock Span
- LeetCode 907 — Sum of Subarray Minimums
Common Bugs
- Comparator:
<vs<=matters when there are duplicates and the problem wants “strictly greater” vs “greater-or-equal”. Pick the variant that gives unique boundary assignment. - Forgetting to drain the stack at the end (for problems where unprocessed elements have no next-greater).
- Histogram: forgetting the sentinel
0appended at the end — without it the last bar may not be evaluated. - Storing values vs indices — almost always store indices, derive values when needed.
10. Monotonic Queue (sliding window max/min in O(N))
Signal Recognition (<2 min)
- “Maximum / minimum of every window of size K” (or variable size) in O(N).
- DP transitions of the form
dp[i] = max(dp[j] + ...)forjin some window — the deque maintains the candidatejs. - Constrained Subsequence Sum, Jump Game VI.
Canonical Template (Sliding Window Max)
from collections import deque
dq = deque() # holds indices, a[dq] strictly decreasing
result = []
for i, x in enumerate(a):
while dq and a[dq[-1]] <= x:
dq.pop()
dq.append(i)
if dq[0] <= i - K:
dq.popleft()
if i >= K - 1:
result.append(a[dq[0]])
Complexity
Time O(N). Space O(K) for the deque.
Common Variants
- Sliding-window min (flip comparator).
- DP optimization: when
dp[i] = f(max{dp[j] : j ∈ window}), the deque maintains the max efficiently. - Shortest subarray with sum at least K (LC 862) — combine prefix sums with a monotonic deque on prefix-sum values.
Classic Problems
- LeetCode 239 — Sliding Window Maximum
- LeetCode 862 — Shortest Subarray with Sum at Least K
- LeetCode 918 — Maximum Sum Circular Subarray
- LeetCode 1425 — Constrained Subsequence Sum
- LeetCode 1696 — Jump Game VI
Common Bugs
- Storing values, not indices — lose the ability to evict by window position.
<=vs<for the back-eviction (with duplicates, the wrong choice can leave stale entries that survive past their window).- Forgetting to evict the front when its index is out of window.
- Reporting before the window is full (
i >= K - 1).
11. Intervals (sort by start, merge / sweep)
Signal Recognition (<2 min)
- “Merge overlapping intervals”, “insert interval”, “remove minimum to make non-overlapping”.
- “Meeting rooms” / “minimum platforms” / “maximum concurrent events”.
- “Employee free time” / “interval intersection”.
Canonical Template (Merge)
intervals.sort(key=lambda x: x[0])
merged = []
for s, e in intervals:
if merged and merged[-1][1] >= s:
merged[-1][1] = max(merged[-1][1], e)
else:
merged.append([s, e])
Canonical Template (Sweep Line)
events = []
for s, e in intervals:
events.append((s, +1))
events.append((e, -1)) # or (e + 1, -1) for closed intervals on integers
events.sort()
cur = peak = 0
for _, delta in events:
cur += delta
peak = max(peak, cur)
Complexity
Sort O(N log N), sweep O(N). Space O(N) for events.
Common Variants
- Merge (sort by start, fold).
- Sweep (events at endpoints, count concurrent).
- Heap-of-end-times (for “minimum platforms / rooms”).
- Interval trees / balanced BSTs (Phase 3) for online updates.
- Tie-breaking: end events before start events (or vice versa) depending on whether endpoint contact counts as overlap.
Classic Problems
- LeetCode 56 — Merge Intervals
- LeetCode 57 — Insert Interval
- LeetCode 252 — Meeting Rooms
- LeetCode 253 — Meeting Rooms II
- LeetCode 435 — Non-overlapping Intervals
- LeetCode 759 — Employee Free Time
- LeetCode 986 — Interval List Intersections
- LeetCode 1851 — Minimum Interval to Include Each Query
Common Bugs
- Sorting by end when start was needed (or vice versa).
- Tie-breaking events at the same time wrong — touching intervals counted as overlap (or not) depending on the problem.
- Mutating the input list while iterating (Java
ConcurrentModificationException).
12. Linked List Manipulation (reverse / detect cycle / merge)
Signal Recognition (<2 min)
- The data structure given is
ListNode. - Tasks: reverse, reverse in groups, detect cycle, find middle, merge sorted, partition, deep copy.
- Often combined with a dummy head for return-pointer simplification.
Canonical Templates
# reverse
prev, curr = None, head
while curr:
nxt = curr.next
curr.next = prev
prev, curr = curr, nxt
return prev
# detect cycle (Floyd)
slow = fast = head
while fast and fast.next:
slow = slow.next
fast = fast.next.next
if slow is fast: return True
return False
Complexity
All operations O(N) time, O(1) space (recursion variants are O(N) stack).
Common Variants
- Reverse, reverse in K-group, reverse between m..n.
- Floyd’s cycle detection + finding cycle start (mathematical trick: reset slow to head, advance both at speed 1).
- Find middle with slow/fast (handle even/odd length).
- Merge two sorted with dummy.
- Deep copy with random pointer — interleave clones, then split.
- LRU cache (Phase 3) is doubly-linked list + hashmap.
Classic Problems
- LeetCode 21 — Merge Two Sorted Lists
- LeetCode 25 — Reverse Nodes in k-Group
- LeetCode 138 — Copy List with Random Pointer
- LeetCode 141 — Linked List Cycle
- LeetCode 142 — Linked List Cycle II
- LeetCode 206 — Reverse Linked List
- LeetCode 234 — Palindrome Linked List
- LeetCode 876 — Middle of the Linked List
Common Bugs
- Not using a dummy head when the head can change → special-case branches everywhere.
- Reverse: losing the next pointer because of assignment order.
- Cycle detection: incorrect math for finding the cycle start.
- Returning the wrong end (
curris null after the loop;previs the new head).
13. Tree DFS (preorder / inorder / postorder)
Signal Recognition (<2 min)
- The input is a tree (binary, n-ary, or just a graph that happens to be a tree).
- The answer is computed by combining results from subtrees (postorder) or by augmenting a top-down state (preorder).
- BST in-order traversal yields sorted values.
- “Validate”, “diameter”, “lowest common ancestor”, “serialize / deserialize”, “path sum”.
Canonical Template (Postorder)
def dfs(node):
if not node: return base
L = dfs(node.left)
R = dfs(node.right)
return combine(node.val, L, R)
Complexity
Time O(N) — each node visited once. Space O(H) recursion (H = height; up to N for skewed trees).
Common Variants
- Inorder for BSTs (yields sorted; use for “kth smallest” / “validate BST”).
- Preorder when state flows top-down (e.g., max value on path).
- Postorder when answer combines subtree results (e.g., diameter, LCA).
- Iterative with explicit stack — required when recursion depth could overflow (N=10^5 in Python ≈ stack limit).
- Morris traversal for O(1) extra space (Phase 3).
Classic Problems
- LeetCode 94 — Binary Tree Inorder Traversal
- LeetCode 98 — Validate Binary Search Tree
- LeetCode 104 — Maximum Depth of Binary Tree
- LeetCode 124 — Binary Tree Maximum Path Sum
- LeetCode 230 — Kth Smallest Element in a BST
- LeetCode 236 — Lowest Common Ancestor of a Binary Tree
- LeetCode 297 — Serialize and Deserialize Binary Tree
- LeetCode 543 — Diameter of Binary Tree
Common Bugs
- “Validate BST” by checking only
left.val < node.val < right.val(local check) — must pass(min, max)bounds top-down. - Confusing “max path through node” with “max path starting at node” — the diameter trick.
- Stack overflow for deep trees in Python (default limit 1000) —
sys.setrecursionlimitor go iterative. - Mutating shared
pathlist without backtracking → wrong “all paths” output.
14. Tree BFS (level order / right side view)
Signal Recognition (<2 min)
- “Level order / level by level / per-depth” output.
- “Right (or left) side view” — last node at each level.
- “Minimum depth” — first leaf encountered in BFS.
- “Connect next pointers per level” (LC 116/117).
- “Vertical / diagonal traversal” — same machinery with different keying.
Canonical Template
from collections import deque
q = deque([root])
levels = []
while q:
size = len(q)
cur = []
for _ in range(size):
node = q.popleft()
cur.append(node.val)
if node.left: q.append(node.left)
if node.right: q.append(node.right)
levels.append(cur)
Complexity
Time O(N), space O(W) (W = max width; up to N/2 for balanced).
Common Variants
- Plain level order, with or without per-level grouping.
- Zigzag (alternate appending direction).
- Right-/left-side view (last/first per level).
- Minimum depth (first leaf — cuts BFS short on the goal).
- Bottom-up (collect all levels then reverse).
Classic Problems
- LeetCode 102 — Binary Tree Level Order Traversal
- LeetCode 103 — Binary Tree Zigzag Level Order Traversal
- LeetCode 107 — Binary Tree Level Order Traversal II
- LeetCode 111 — Minimum Depth of Binary Tree
- LeetCode 116 — Populating Next Right Pointers in Each Node
- LeetCode 199 — Binary Tree Right Side View
- LeetCode 314 — Binary Tree Vertical Order Traversal
Common Bugs
- Using
list.pop(0)(Python) → O(N²). Usedeque. - Forgetting to capture
size = len(q)before the inner loop — q grows during the loop and you’d over-iterate. - Returning the level structure backwards (or forwards) accidentally.
- Null root not handled.
15. Graph DFS (cycle / connected components / topo via DFS)
Signal Recognition (<2 min)
- The structure is a graph (general, not necessarily tree).
- Tasks: count connected components, detect cycle, topologically order, find bridges/articulation points (Phase 4).
- Recursion is fine (or you simulate it with an explicit stack).
Canonical Template (Connected Components)
visited = [False] * n
def dfs(u):
visited[u] = True
for v in adj[u]:
if not visited[v]: dfs(v)
components = 0
for u in range(n):
if not visited[u]:
dfs(u); components += 1
Canonical Template (Cycle Detection in Directed Graph)
WHITE, GRAY, BLACK = 0, 1, 2
color = [WHITE] * n
def has_cycle(u):
color[u] = GRAY
for v in adj[u]:
if color[v] == GRAY: return True
if color[v] == WHITE and has_cycle(v): return True
color[u] = BLACK
return False
Complexity
Time O(V + E). Space O(V) for the recursion + visited arrays.
Common Variants
- Connected components (undirected).
- Strongly connected components (Tarjan / Kosaraju — Phase 4).
- Cycle detection: undirected uses a parent-check; directed uses tri-color (WHITE/GRAY/BLACK).
- Topological sort via DFS — postorder of a DAG, reversed.
- Number of islands / flood fill on grid graphs.
Classic Problems
- LeetCode 200 — Number of Islands
- LeetCode 207 — Course Schedule (cycle detection in directed graph)
- LeetCode 261 — Graph Valid Tree
- LeetCode 323 — Number of Connected Components
- LeetCode 695 — Max Area of Island
- LeetCode 1192 — Critical Connections (Tarjan bridges)
Common Bugs
- Forgetting to mark visited before recursing → infinite recursion.
- For undirected cycle detection, treating “going back to parent” as a cycle.
- Stack overflow for deep recursion in Python on N=10^5 — convert to iterative.
- For grid problems, going out of bounds because the bounds check is after the recursion call.
16. Graph BFS (shortest unweighted / multi-source / 0-1)
Signal Recognition (<2 min)
- Shortest path on an unweighted graph (or weights ∈ {0, 1}).
- “Minimum number of steps / moves / transformations.”
- Multi-source: “shortest distance from any of these sources” (rotting oranges, walls and gates).
- 0-1 BFS: use a deque, push 0-weight to front, 1-weight to back.
Canonical Template
from collections import deque
dist = [-1] * n
q = deque([src]); dist[src] = 0
while q:
u = q.popleft()
for v in adj[u]:
if dist[v] == -1:
dist[v] = dist[u] + 1
q.append(v)
Complexity
Time O(V + E). Space O(V).
Common Variants
- Standard BFS for unweighted shortest path.
- Multi-source BFS — push all sources with distance 0, then run.
- 0-1 BFS with deque for graphs with weights ∈ {0, 1}.
- Bidirectional BFS for shortest path between fixed source and target — both halves explore O(b^(d/2)) instead of O(b^d).
- Word-ladder pattern — neighbors via wildcard buckets, not adjacency list.
Classic Problems
- LeetCode 127 — Word Ladder
- LeetCode 286 — Walls and Gates (multi-source)
- LeetCode 542 — 01 Matrix (multi-source)
- LeetCode 752 — Open the Lock
- LeetCode 994 — Rotting Oranges (multi-source)
- LeetCode 1162 — As Far from Land as Possible
- LeetCode 2290 — Minimum Obstacle Removal to Reach Corner (0-1 BFS)
Common Bugs
- Marking visited at dequeue time (lets duplicates pile up) instead of at enqueue time.
- Using BFS for weighted graphs (distinct positive weights) — wrong; use Dijkstra (Phase 4).
- Forgetting that “minimum depth of binary tree” is BFS, not DFS — DFS visits all leaves; BFS halts on the first.
17. Topological Sort (Kahn’s / DFS-based)
Signal Recognition (<2 min)
- “Order tasks given dependencies” / “course prerequisites” / “build order” / “alien dictionary”.
- Detecting whether a DAG has a cycle (failure = there’s a cycle).
- DP on DAG (some tasks need a topological order to evaluate).
Canonical Template (Kahn’s BFS)
indeg = [0] * n
for u in range(n):
for v in adj[u]:
indeg[v] += 1
q = deque([u for u in range(n) if indeg[u] == 0])
order = []
while q:
u = q.popleft()
order.append(u)
for v in adj[u]:
indeg[v] -= 1
if indeg[v] == 0: q.append(v)
return order if len(order) == n else [] # else: cycle
Canonical Template (DFS Postorder)
order = []
WHITE, GRAY, BLACK = 0, 1, 2
color = [WHITE] * n
def dfs(u):
color[u] = GRAY
for v in adj[u]:
if color[v] == GRAY: raise CycleError
if color[v] == WHITE: dfs(v)
color[u] = BLACK
order.append(u)
for u in range(n):
if color[u] == WHITE: dfs(u)
order.reverse()
Complexity
Time O(V + E). Space O(V + E).
Common Variants
- Kahn’s — gives a valid order; cycle detection by
len(order) != n. - DFS postorder reversed — alternate algorithm; same result.
- Lexicographically smallest topological order — use a min-heap instead of FIFO queue (Kahn’s).
- All possible topological orderings — backtracking over orderings (LC 1632).
- DP on DAG following topological order.
Classic Problems
- LeetCode 207 — Course Schedule
- LeetCode 210 — Course Schedule II
- LeetCode 269 — Alien Dictionary
- LeetCode 310 — Minimum Height Trees (peeling leaves — relative)
- LeetCode 329 — Longest Increasing Path in a Matrix (DP on DAG)
- LeetCode 444 — Sequence Reconstruction
- LeetCode 1136 — Parallel Courses
- LeetCode 2115 — Find All Possible Recipes
Common Bugs
- Building the graph with reversed edges (prerequisites vs unlocks).
- Not detecting cycles (returning a partial order silently).
- For lexicographic smallest, using a regular queue instead of a heap.
- Indegrees off-by-one when the same edge is duplicated in input.
18. Union-Find (connectivity / Kruskal preview)
Signal Recognition (<2 min)
- “Are X and Y connected after these operations?” (online connectivity).
- “Number of connected components” with dynamic union operations (DFS once-and-done is sufficient for static).
- Kruskal’s MST (Phase 4) — sort edges, union components.
- Equation problems (LC 399 — Evaluate Division — weighted union-find).
Canonical Template
class DSU:
def __init__(self, n):
self.parent = list(range(n))
self.rank = [0] * n
def find(self, x):
while self.parent[x] != x:
self.parent[x] = self.parent[self.parent[x]] # path compression
x = self.parent[x]
return x
def union(self, a, b):
ra, rb = self.find(a), self.find(b)
if ra == rb: return False
if self.rank[ra] < self.rank[rb]: ra, rb = rb, ra
self.parent[rb] = ra
if self.rank[ra] == self.rank[rb]: self.rank[ra] += 1
return True
Complexity
Per operation: amortized O(α(N)) ≈ O(1) with both path compression and union by rank/size. Without rank: O(log N) amortized. Without compression: O(log N) amortized. Naive: O(N) worst case.
Common Variants
- Vanilla connectivity.
- With size — track component sizes for “find largest component”.
- Weighted — each edge has a multiplier (LC 399 — Evaluate Division).
- With rollback (Phase 3) — for offline / divide-and-conquer queries.
- Kruskal MST — sort edges by weight, union the endpoints if they’re in different components.
Classic Problems
- LeetCode 200 — Number of Islands (DSU alternative)
- LeetCode 261 — Graph Valid Tree
- LeetCode 305 — Number of Islands II (online)
- LeetCode 399 — Evaluate Division
- LeetCode 547 — Number of Provinces
- LeetCode 684 — Redundant Connection
- LeetCode 721 — Accounts Merge
- LeetCode 1319 — Number of Operations to Make Network Connected
Common Bugs
- Forgetting path compression — TLE on adversarial chain inputs.
findrecursion that deepens the stack (use iterative or two-pass).- Updating
rankonly when unequal — but updating it always makes the rank wrong by+1. - Comparing
parent[x] == xvsfind(x) == x— they differ before compression converges.
19. Backtracking (subsets / permutations / combinations / N-queens)
Signal Recognition (<2 min)
- “Find all subsets / permutations / combinations / arrangements satisfying X.”
- “Place K items respecting constraints” (N-queens, Sudoku).
- The brute force is exponential, and you can’t shave it polynomially — but you can prune aggressively.
Canonical Template
def backtrack(state, choices):
if is_solution(state):
record(state); return
for choice in choices:
if not valid(state, choice): continue
apply(state, choice)
backtrack(state, next_choices(choices, choice))
undo(state, choice)
Complexity
- Subsets: O(N · 2^N).
- Permutations: O(N · N!).
- N-queens: O(N!) worst case, dramatically pruned in practice. Space O(depth) for recursion + O(state size).
Common Variants
- Subsets (include/exclude each element).
- Permutations (choose unused; track used set or swap-in-place).
- Combinations (start index to avoid reordering duplicates).
- Partition into subsets (assign each element to a bucket; prune by sorting + skipping equal-prefix bucket).
- Constraint satisfaction (N-queens, Sudoku) — prune with row/column/box bitmasks.
- String backtracking (palindrome partitioning, restore IP addresses, generate parentheses).
Classic Problems
- LeetCode 17 — Letter Combinations of a Phone Number
- LeetCode 22 — Generate Parentheses
- LeetCode 39 — Combination Sum
- LeetCode 46 — Permutations
- LeetCode 51 — N-Queens
- LeetCode 78 — Subsets
- LeetCode 79 — Word Search
- LeetCode 90 — Subsets II (with duplicates)
- LeetCode 131 — Palindrome Partitioning
- LeetCode 212 — Word Search II (with trie)
Common Bugs
- Forgetting to undo the choice before returning → state corruption.
- Recording
stateby reference, not by copy → all results alias the final state. - Duplicate handling: forgetting
if i > start and a[i] == a[i-1]: continue(for sorted input with duplicates). - For grid backtracking, forgetting to mark visited or not unmarking on return.
20. Basic DP (memoization vs tabulation)
Signal Recognition (<2 min)
- “Number of ways to …” / “Min/max … over choices.”
- Recursive structure with overlapping subproblems: the same sub-question is asked multiple times.
- Optimal substructure: the optimal answer combines optimal answers to subproblems.
- The brute is exponential; with memo the state space is polynomial.
Canonical Template (Top-Down Memoization)
from functools import lru_cache
@lru_cache(maxsize=None)
def solve(state):
if is_base(state): return base_value(state)
return combine(solve(next_state(state, c)) for c in choices(state))
Canonical Template (Bottom-Up Tabulation)
dp = init_table()
for state in topological_order_of_states():
dp[state] = combine(dp[prev] for prev in predecessors(state))
return dp[final_state]
Complexity
Time = (# states) × (work per state). Space = (# states), often optimizable to a slice.
Common Variants (covered separately below)
- 1D DP (pattern 21).
- 2D DP (pattern 22).
- Knapsack (pattern 23).
- Subsequence DP (pattern 24).
- String DP (pattern 25).
- Bitmask / interval / digit / probability / tree (Phase 5).
Classic Problems
- LeetCode 70 — Climbing Stairs
- LeetCode 198 — House Robber
- LeetCode 322 — Coin Change
- LeetCode 416 — Partition Equal Subset Sum
Common Bugs
- Wrong state definition — too coarse to reconstruct, too fine to fit in memory.
- Wrong base case (off-by-one in the empty / single-element base).
- Wrong evaluation order in tabulation — predecessors computed after dependents.
- Memo key collisions when two different state tuples accidentally hash equal.
21. 1D DP (climbing stairs / house robber / decode ways)
Signal Recognition (<2 min)
- The state is a single index: “the answer at position i depends on positions ≤ i”.
- Transitions look at the last 1–3 positions.
- The answer is at
dp[n]ordp[n-1].
Canonical Template
dp = [0] * (n + 1)
dp[0] = base
for i in range(1, n + 1):
dp[i] = combine(dp[i - 1], dp[i - 2], a[i - 1])
return dp[n]
Complexity
Time O(N), space O(N) — usually compressible to O(1).
Common Variants
- Climbing stairs (Fibonacci-shaped).
- House robber / robber II (linear / circular).
- Decode ways (transitions depend on a 2-digit window).
- Maximum subarray (Kadane’s).
- Min cost climbing stairs.
Classic Problems
- LeetCode 53 — Maximum Subarray (Kadane’s)
- LeetCode 70 — Climbing Stairs
- LeetCode 91 — Decode Ways
- LeetCode 121 — Best Time to Buy and Sell Stock
- LeetCode 198 — House Robber
- LeetCode 213 — House Robber II
- LeetCode 264 — Ugly Number II
- LeetCode 746 — Min Cost Climbing Stairs
Common Bugs
- Off-by-one in the
dpsize (nvsn+1). - Wrong base for empty input.
- Decode ways: forgetting that
0is not a valid single-digit decoding. - Compressing to O(1) but reading
dp[i-2]afterdp[i-1]is overwritten.
22. 2D DP (grid / unique paths / LCS preview)
Signal Recognition (<2 min)
- The state is a pair
(i, j)— typically(row, col)or(index_in_A, index_in_B). - Transitions look at neighboring cells:
dp[i-1][j],dp[i][j-1],dp[i-1][j-1]. - Common shapes: grid path counting/min sum, LCS, edit distance, palindrome substring.
Canonical Template
dp = [[0] * (m + 1) for _ in range(n + 1)]
for i in range(1, n + 1):
for j in range(1, m + 1):
dp[i][j] = combine(dp[i-1][j], dp[i][j-1], dp[i-1][j-1], a[i-1], b[j-1])
return dp[n][m]
Complexity
Time O(NM), space O(NM) — often compressible to O(min(N, M)) by keeping two rows.
Common Variants
- Grid DP: unique paths, min path sum, max path sum.
- Two-string DP: LCS, edit distance, regex matching, distinct subsequences (covered in 24/25).
- Matrix DP: maximal square, dungeon game.
- Backwards-traversal (start from
(n, m)) when transitions need future state.
Classic Problems
- LeetCode 62 — Unique Paths
- LeetCode 64 — Minimum Path Sum
- LeetCode 72 — Edit Distance
- LeetCode 174 — Dungeon Game
- LeetCode 221 — Maximal Square
- LeetCode 1143 — Longest Common Subsequence
Common Bugs
- Initializing the first row/column wrong (not the additive identity for the operator).
- Allocating
[[0] * m] * nin Python — all rows alias the same list (top-3 Python DP bug). - 1D-compression bug: reading the new value when the old one was needed (or vice versa).
- For grids with obstacles: forgetting the obstacle ⇒ 0-paths-here invariant.
23. Knapsack (0/1 + unbounded)
Signal Recognition (<2 min)
- “Pick a subset of items to maximize value subject to a capacity constraint” (0/1 knapsack — each item once).
- “Pick items with repetition allowed” (unbounded knapsack — coin change min coins, integer break).
- “Number of ways to make sum K from given items” (counting variant).
Canonical Template (0/1 Knapsack — Compressed 1D)
dp = [0] * (W + 1)
for v, w in items:
for c in range(W, w - 1, -1): # reverse to avoid re-using item
dp[c] = max(dp[c], dp[c - w] + v)
Canonical Template (Unbounded Knapsack)
dp = [0] * (W + 1)
for c in range(1, W + 1): # forward, so each item can be reused
for v, w in items:
if c >= w: dp[c] = max(dp[c], dp[c - w] + v)
Complexity
Time O(N · W). Space O(W) (compressed) or O(N · W) (uncompressed).
Common Variants
- 0/1 vs unbounded vs bounded (limited copies of each item).
- Min coins, count-of-ways, can-we-make-this-sum.
- Subset sum (knapsack with
value = weight). - Partition equal subset sum (subset sum to total/2).
Classic Problems
- LeetCode 322 — Coin Change (unbounded, min)
- LeetCode 416 — Partition Equal Subset Sum (0/1, decision)
- LeetCode 474 — Ones and Zeroes (0/1 with two capacities)
- LeetCode 494 — Target Sum (count-of-ways)
- LeetCode 518 — Coin Change II (unbounded, count)
- LeetCode 879 — Profitable Schemes
- LeetCode 1049 — Last Stone Weight II
Common Bugs
- 0/1 with forward inner loop → double-counts items.
- Unbounded with reverse inner loop → behaves like 0/1.
- For “count of ways” with order-insensitive: outer is items, inner is capacity (LC 518). Order-sensitive: opposite (LC 377).
- Forgetting that
dp[0] = 1for count-of-ways,dp[0] = 0for max-value.
24. Subsequence DP (LIS / LCS / edit distance)
Signal Recognition (<2 min)
- “Longest increasing / common / non-decreasing subsequence.”
- “Edit distance / minimum operations to transform A to B.”
- “Distinct subsequences / supersequences.”
- “Longest palindromic subsequence” (it’s LCS of
sands[::-1]).
Canonical Template (LIS, O(N log N))
import bisect
tails = []
for x in a:
i = bisect.bisect_left(tails, x) # bisect_right for non-decreasing
if i == len(tails): tails.append(x)
else: tails[i] = x
return len(tails)
Canonical Template (LCS / Edit Distance — 2D DP)
dp = [[0] * (m + 1) for _ in range(n + 1)]
for i in range(1, n + 1):
for j in range(1, m + 1):
if a[i-1] == b[j-1]:
dp[i][j] = dp[i-1][j-1] + 1 # LCS
else:
dp[i][j] = max(dp[i-1][j], dp[i][j-1])
Complexity
LIS: O(N log N) (patience-sort) or O(N²) (DP). LCS / edit distance: O(NM).
Common Variants
- LIS, longest non-decreasing, count of LIS, reconstruction.
- LCS, shortest common supersequence (
N + M − LCS). - Edit distance (Levenshtein), with weighted operations.
- Distinct subsequences (count occurrences of T as subsequence of S).
- Longest palindromic subsequence (= LCS of
sand reverseds).
Classic Problems
- LeetCode 72 — Edit Distance
- LeetCode 115 — Distinct Subsequences
- LeetCode 300 — Longest Increasing Subsequence
- LeetCode 354 — Russian Doll Envelopes (LIS in 2D)
- LeetCode 516 — Longest Palindromic Subsequence
- LeetCode 583 — Delete Operation for Two Strings
- LeetCode 673 — Number of LIS
- LeetCode 1143 — Longest Common Subsequence
Common Bugs
- LIS with
bisect_leftvsbisect_rightcontrols strict vs non-strict — pick the wrong one and ties are mishandled. - Edit distance: forgetting that the base row/col is
0..n(i deletions / insertions to reach empty). - Reconstruction: walking the
dptable backward, easy to off-by-one.
25. String DP (palindrome / partitioning)
Signal Recognition (<2 min)
- “Longest palindromic substring / subsequence.”
- “Minimum cuts to partition into palindromes.”
- “Word break / segment string into dictionary words.”
- “Regex / wildcard matching.”
Canonical Template (Palindrome Substring DP)
n = len(s)
dp = [[False] * n for _ in range(n)]
for i in range(n): dp[i][i] = True
for length in range(2, n + 1):
for i in range(n - length + 1):
j = i + length - 1
if s[i] == s[j] and (length == 2 or dp[i+1][j-1]):
dp[i][j] = True
Complexity
Most string-DP problems: O(N²) time, O(N²) space (often compressible to O(N)). Manacher (Phase 3) gives O(N) for longest-palindrome.
Common Variants
- Longest palindromic substring (DP, expand-around-center, or Manacher).
- Longest palindromic subsequence (LCS-based).
- Minimum cuts (palindrome partitioning II).
- Word break (boolean DP) and word break II (recover all decompositions).
- Regex / wildcard matching (
?,*,.).
Classic Problems
- LeetCode 5 — Longest Palindromic Substring
- LeetCode 10 — Regular Expression Matching
- LeetCode 44 — Wildcard Matching
- LeetCode 132 — Palindrome Partitioning II
- LeetCode 139 — Word Break
- LeetCode 140 — Word Break II
- LeetCode 516 — Longest Palindromic Subsequence
- LeetCode 647 — Palindromic Substrings
Common Bugs
- Iteration order: filling
dp[i][j]requiresdp[i+1][j-1]already filled — iterate by length, not byithenj. - Word break: building all decompositions naively is O(2^N) — memoize, but be aware total output can still be exponential.
- Wildcard
*matching empty vs many — both transitions needed. - Off-by-one when
j = i + length - 1.
26. Trie (prefix queries / autocomplete preview)
Signal Recognition (<2 min)
- “Many strings, prefix queries” — does any word start with X? Count words starting with X?
- Autocomplete / spell check.
- Word search II (LC 212) — combine trie with backtracking on a grid.
- Maximum XOR pair (LC 421) — bit-level trie.
- Replace words / dictionary lookup.
Canonical Template
class TrieNode:
__slots__ = ('children', 'end')
def __init__(self):
self.children = {}
self.end = False
class Trie:
def __init__(self): self.root = TrieNode()
def insert(self, word):
node = self.root
for c in word:
node = node.children.setdefault(c, TrieNode())
node.end = True
def search(self, word):
node = self._walk(word)
return node is not None and node.end
def starts_with(self, prefix):
return self._walk(prefix) is not None
def _walk(self, s):
node = self.root
for c in s:
if c not in node.children: return None
node = node.children[c]
return node
Complexity
Insert / search / prefix: O(L) per operation. Space O(N · L) worst case (no shared prefixes).
Common Variants
- Character trie (by char).
- Bit trie (by bit) — for XOR / Hamming-distance problems.
- Compressed (radix) trie — Phase 3.
- Trie + DFS for “all words on a board” (LC 212) — early-prune by failing nodes.
- Suffix trie / suffix automaton — Phase 3 / Phase 12.
Classic Problems
- LeetCode 208 — Implement Trie (Prefix Tree)
- LeetCode 211 — Design Add and Search Words Data Structure
- LeetCode 212 — Word Search II
- LeetCode 336 — Palindrome Pairs
- LeetCode 421 — Maximum XOR of Two Numbers in an Array (bit trie)
- LeetCode 648 — Replace Words
- LeetCode 677 — Map Sum Pairs
- LeetCode 1268 — Search Suggestions System
Common Bugs
- Forgetting the
endflag (or whatever marks a complete word) —search("app")returns true when only"apple"was inserted. - Using a 26-element array vs a hashmap — array is faster but only for fixed alphabets.
- Iterating
node.childrenmistakenly using insertion order assumptions. - For LC 212, not pruning nodes after they’ve been used (still works correctly but wastes time).
27. Heap Top-K (k-largest / k-frequent / k-closest)
Signal Recognition (<2 min)
- “Find the K largest / smallest / most frequent / closest.”
- Online/streaming K-th element.
- Median maintenance (two heaps).
- “Merge K sorted streams” (pattern 28 — see below).
Canonical Template (Top-K with Min-Heap of Size K)
import heapq
heap = []
for x in stream:
if len(heap) < K:
heapq.heappush(heap, x)
elif x > heap[0]:
heapq.heapreplace(heap, x)
return heap # K largest, unsorted
Complexity
Time O(N log K). Space O(K). Compare to full sort O(N log N) — beats it when K << N.
Common Variants
- Top K largest / smallest.
- Top K frequent — bucket sort gives O(N) when frequencies fit (LC 347).
- K closest points to origin — heap of K by distance.
- Median from data stream — two heaps (max-heap of low half, min-heap of high half).
- K-th smallest in matrix / K-th smallest in BST — heap or controlled traversal.
Classic Problems
- LeetCode 215 — Kth Largest Element in an Array
- LeetCode 295 — Find Median from Data Stream
- LeetCode 347 — Top K Frequent Elements
- LeetCode 451 — Sort Characters by Frequency
- LeetCode 692 — Top K Frequent Words
- LeetCode 703 — Kth Largest Element in a Stream
- LeetCode 973 — K Closest Points to Origin
- LeetCode 1046 — Last Stone Weight
Common Bugs
- Using a max-heap for top-K-largest — wrong; use a min-heap of size K (we evict the smallest).
- Java
PriorityQueueis min-heap by default; useComparator.reverseOrder()for max. - Python
heapqis min-heap only; negate values for max-heap. - For “top K frequent words” with tie-breaking (alphabetical) — comparator gets tricky in Java/Python.
28. K-Way Merge (merge K lists / smallest range covering K lists)
Signal Recognition (<2 min)
- K sorted lists/streams; merge them into one sorted output.
- “Find the smallest range that contains at least one element from each of K lists.”
- “Find the K-th smallest in K sorted lists / matrix.”
- External-merge / external-sort flavor.
Canonical Template (Merge K Sorted Lists)
import heapq
heap = []
for i, lst in enumerate(lists):
if lst: heapq.heappush(heap, (lst[0].val, i, lst[0]))
dummy = ListNode(0); tail = dummy
while heap:
val, i, node = heapq.heappop(heap)
tail.next = node; tail = tail.next
if node.next:
heapq.heappush(heap, (node.next.val, i, node.next))
return dummy.next
Complexity
Time O(N log K) where N is the total number of elements. Space O(K) for the heap.
Common Variants
- Merge K sorted lists / arrays / streams.
- Smallest range covering at least one from each list — heap holds one element per list, track current max; pop the min and advance the popped list.
- K-th smallest in sorted matrix — heap of (value, row, col); pop, push next-in-row (or use binary search on answer instead).
- Find smallest pair sums (LC 373) — heap from two sorted lists.
- Skyline problem (LC 218) — sweep over events with a heap of active heights.
Classic Problems
- LeetCode 23 — Merge K Sorted Lists
- LeetCode 218 — The Skyline Problem
- LeetCode 264 — Ugly Number II
- LeetCode 313 — Super Ugly Number
- LeetCode 373 — Find K Pairs with Smallest Sums
- LeetCode 378 — Kth Smallest Element in a Sorted Matrix
- LeetCode 632 — Smallest Range Covering Elements from K Lists
- LeetCode 1675 — Minimize Deviation in Array
Common Bugs
- Heap items must include a tiebreaker — comparing
ListNodedirectly raises TypeError in Python. - Forgetting to push the next element in the same list after popping.
- For “smallest range”: confusing max so far (cheap to maintain) with re-scanning the heap (O(K)).
- Off-by-one when one list is exhausted before others.
Pattern Recognition Cheat Sheet (Signal → Pattern)
This is the table you should be able to traverse, top-to-bottom, in <60 seconds for any new Medium.
| Signal in problem statement | Likely pattern(s) | First template to try |
|---|---|---|
| Sorted input + pair/triplet sum | Two pointers (1) | opposite-ends two-pointer |
| In-place removal / partition | Two pointers (1) | read/write pointer |
| Subarray with property over contiguous elements | Sliding window (2) or prefix sum (3) | shrink-while-violation |
| Max/min of every window K | Monotonic queue (10) | deque indices |
| Subarray sum equals K / divisible K | Prefix sum + hash (3, 5) | prefix + complement map |
| Many range updates then final state | Difference array (4) | diff + prefix |
| “Find pair summing to target” | Hash complement (5) | seen[target − x] |
| “Group by canonical form” | Hashing — grouping (5) | dict[canonical] → list |
| “Maximum non-overlapping …” | Sort + greedy (6, 11) | sort by end, sweep |
| “Number of meeting rooms” | Intervals — sweep (11) | events, +1/−1 |
| Sorted, find element / first ≥ X | Binary search on index (7) | lower_bound |
| “Min capacity / time / speed s.t. P” | Binary search on answer (8) | binary search + feasible() |
| “Next greater / span / histogram” | Monotonic stack (9) | strictly-decreasing stack |
| Linked-list reverse / cycle / merge | Linked-list patterns (12) | dummy + 3-pointer |
| Tree value combined from subtrees | Tree DFS postorder (13) | recursive combine |
| Tree level-by-level | Tree BFS (14) | queue, capture size |
| Graph “connected components” | Graph DFS (15) | visited + DFS |
| Shortest path on unweighted graph | Graph BFS (16) | distances + queue |
| Shortest path with weights ∈ {0,1} | 0-1 BFS (16) | deque, push-front 0 |
| “Order tasks given deps” | Topological sort (17) | Kahn’s BFS |
| “Connectivity with online unions” | Union-find (18) | DSU with path compression |
| “Kruskal MST / spanning tree” | Union-find (18) | sort edges + DSU |
| “All subsets / permutations” | Backtracking (19) | recurse + undo |
| “Constraint satisfaction (N-queens)” | Backtracking (19) | bitmask state |
| “Min/max ways with overlapping subproblems” | DP (20) | memoize state |
| “Single-index recurrence” | 1D DP (21) | dp[i] from dp[i-1..i-3] |
| “Two-index recurrence” | 2D DP (22) | dp[i][j] from neighbors |
| “Pick subset under capacity” | 0/1 knapsack (23) | reverse inner loop |
| “Pick with repetition” | Unbounded knapsack (23) | forward inner loop |
| “LIS / LCS / edit distance” | Subsequence DP (24) | 2D dp or patience sort |
| “Longest palindromic *” / “min cuts” | String DP (25) | palindrome dp + outer loop |
| “Many strings, prefix queries” | Trie (26) | trie + insert/search |
| “K largest/smallest/closest” | Heap top-K (27) | min-heap of size K |
| “Merge K sorted …” | K-way merge (28) | heap of one-per-list |
When two patterns plausibly fit, try both signals on a small example. Often one fits cleanly and the other forces awkward special cases.
Mastery Checklist For This Phase
You are ready to leave Phase 2 when, for every one of the 28 patterns:
- You can recognize the signal in <2 minutes on a fresh Medium.
- You can write the canonical template from memory in <5 minutes, without lookup.
- You can articulate the time/space complexity precisely, including amortized vs worst case.
- You can name 4+ classic LeetCode problems where this pattern is the intended solution.
- You can list at least 2 common bugs that the pattern is famous for.
- You have solved at least 5 problems applying this pattern, with at least 2 reviewed via REVIEW_TEMPLATE.md.
And, more generally:
- You can produce the cheat-sheet table above from scratch (or close to it) on a whiteboard.
- You can name, given a signal, the most likely pattern plus the second-most-likely (because tricky problems disguise themselves).
- You can combine two patterns when one alone is insufficient (e.g., monotonic deque inside sliding window for LC 862; trie inside backtracking for LC 212).
- You have run mock interviews on Mediums and your time-to-recognize is reliably under 2 minutes.
Required Problem Volume
The patterns are not learned from this README. They are learned from solving lots of problems per pattern and reviewing each via REVIEW_TEMPLATE.md, then revisiting via SPACED_REPETITION.md.
Recommended minimums for Phase 2 completion (per pattern):
- 5–8 Medium problems that explicitly use the pattern as their intended solution.
- 2 mock-interview Mediums where the pattern is not hinted (you must recognize it).
- 1 problem at the boundary where two patterns plausibly apply — pick one, justify, solve.
Total over the phase: ~150–200 Mediums. This is the cost. The benefit is that, after Phase 2, almost every Medium becomes a 5-minute problem.
Exit Criteria
You may move to Phase 3 — Advanced Data Structures when:
- You score ≥ 75% on a 10-problem random-Medium mock (35 min each, no hints, no lookup), with the pattern recognition and template write-up completed in the first 5 minutes of each problem.
- You can pass the READINESS_CHECKLIST.md entries for “pattern recognition” without lookup.
- You have completed all 9 labs in labs/ with the format-required deliverables.
- You have at least 40 problems in your spaced-repetition queue from this phase, with first reviews passed.
If your unaided solve rate on random Mediums is below 75%, do not advance. Spend another 1–2 weeks at this level, focusing on the patterns where you missed. The patterns calcify here. Calcify wrong patterns and Phase 3+ becomes a fight against your own intuition.
How This Phase Connects To The Rest
- Phase 0 / Phase 1 are prerequisites. You cannot recognize patterns if you cannot read the problem and you do not know the data structures.
- Phase 3 (Advanced DS) generalizes patterns 9, 10, 18, 26, 27, 28 with segment trees, BITs, balanced BSTs, suffix arrays.
- Phase 4 (Graphs) generalizes patterns 15, 16, 17, 18 with Dijkstra, Bellman-Ford, SCC, max flow.
- Phase 5 (DP) generalizes patterns 20–25 with bitmask, interval, tree, digit, probability DP.
- Phase 6 (Greedy) formalizes the proof techniques behind pattern 6 (sort + greedy).
- Phase 9 (Language/Runtime) drills the language-specific footguns called out in each pattern’s “Common Bugs”.
You will return to this README dozens of times over the rest of the curriculum — each return faster than the last, until eventually the patterns are no longer something you look up but something you simply see.
Lab 01 — Two Pointers: 3Sum (Deep Duplicate Handling)
Goal
Master the opposite-ends two-pointer pattern on a sorted array, with the specific discipline required for correct duplicate suppression. Deliverable solves 3Sum in O(N²) time, O(1) extra space, returning all unique triplets — and you can articulate, line by line, why each duplicate-skip is needed and what bug it prevents.
Background Concepts
Sorting as a precondition for two-pointer; loop invariants on (l, r); duplicate suppression at three loci (the outer i, the inner l, the inner r); the relationship between 3Sum and 2Sum-on-sorted; why hash-set deduplication is the wrong tool here. Review pattern Two Pointers and pattern Hashing.
Interview Context
3Sum is one of the single most asked problems in Big Tech onsite loops — it appears at Meta, Amazon, Bloomberg, and Apple. The signal is whether you can articulate the duplicate-handling logic before you code, not retroactively. Naive candidates produce O(N²) triplets and dedup via a hash set (set(tuple(sorted(triplet)))), which works but signals weak invariant reasoning. Strong candidates handle duplicates inside the two-pointer loop with three skip-clauses and explain each one’s purpose. Elite candidates also address the trade-off vs the hash-based 3Sum (when the input is unsorted and you can’t sort).
Problem Statement
Given an integer array nums, return all unique triplets [nums[i], nums[j], nums[k]] such that i, j, k are distinct indices and nums[i] + nums[j] + nums[k] == 0. The result must not contain duplicate triplets.
Constraints
3 ≤ N ≤ 3000-10^5 ≤ nums[i] ≤ 10^5- Output is a list of triplets in any order; each triplet’s internal order doesn’t matter.
- Triplets are deduplicated by value, not by index:
[-1, 0, 1]from indices(0, 2, 4)and from(1, 3, 5)count once.
Clarifying Questions
- Should
[-1, 0, 1]and[0, 1, -1]count as different triplets? (No — value-set duplicates.) - Are duplicates in the input array allowed? (Yes — many test cases hinge on this.)
- Can the input be empty / size < 3? (Per constraints, N ≥ 3, but mention you’d guard.)
- Can values exceed 32-bit range when summed? (Per constraints, max |sum| ≤ 3·10^5, safe in 32-bit, but you should mention overflow as a habit.)
- Is there a strict no-extra-space requirement, or is the output allocation OK?
Examples
| Input | Output |
|---|---|
[-1,0,1,2,-1,-4] | [[-1,-1,2],[-1,0,1]] |
[0,0,0,0] | [[0,0,0]] |
[1,2,-2,-1] | [] |
[] | [] |
[0] | [] |
Initial Brute Force
Three nested loops; check nums[i] + nums[j] + nums[k] == 0; dedup with a hash set of sorted triplets.
def three_sum_brute(nums):
out = set()
n = len(nums)
for i in range(n):
for j in range(i + 1, n):
for k in range(j + 1, n):
if nums[i] + nums[j] + nums[k] == 0:
out.add(tuple(sorted((nums[i], nums[j], nums[k]))))
return [list(t) for t in out]
Brute Force Complexity
Time O(N³). Space O(N²) worst case for out (number of unique triplets). At N=3000, ~2.7×10^10 operations — far too slow (>30 seconds in any language).
Optimization Path
The sub-problem after fixing nums[i] is 2Sum on the remaining sorted array with target -nums[i]. 2Sum-on-sorted is O(N) via opposite-ends two-pointer. Total: O(N) outer × O(N) inner = O(N²).
Why sort? Sortedness gives 2Sum-on-sorted an O(N) two-pointer algorithm; without sort, 2Sum is O(N) via hash but combining hash-2Sum with outer triplet enumeration makes duplicate-handling much trickier.
Final Expected Approach
def three_sum(nums):
nums.sort()
n = len(nums)
out = []
for i in range(n - 2):
if nums[i] > 0: break # early exit
if i > 0 and nums[i] == nums[i - 1]: continue # skip i-duplicate
l, r = i + 1, n - 1
while l < r:
s = nums[i] + nums[l] + nums[r]
if s < 0:
l += 1
elif s > 0:
r -= 1
else:
out.append([nums[i], nums[l], nums[r]])
l += 1; r -= 1
while l < r and nums[l] == nums[l - 1]: l += 1 # skip l-duplicate
while l < r and nums[r] == nums[r + 1]: r -= 1 # skip r-duplicate
return out
Data Structures Used
- The input array, sorted in place.
- An output list of triplets.
- Three integer indices (
i,l,r).
No hash structures; no auxiliary lists beyond output.
Correctness Argument
After sorting, fix i. The remaining array nums[i+1..n-1] is sorted, and we run the canonical opposite-ends 2Sum: when the sum is too small we advance l, when too large we retreat r, when equal we record and advance both. The standard 2Sum-on-sorted invariant proves that every value pair (nums[l], nums[r]) with l < r and nums[l] + nums[r] == target is found exactly once, in sorted order.
For duplicates: the three skip-clauses ensure that each distinct triplet by value is recorded exactly once.
if i > 0 and nums[i] == nums[i-1]: continue— the previous outer iteration already enumerated all triplets with first element equal tonums[i]. Without this,[-1,-1,0,1,2]would record[-1,-1,2]and[-1,0,1]twice (once for each-1as the outer pivot).while l < r and nums[l] == nums[l-1]: l += 1(after recording) — if multiplenums[l]values exist within(i, r), they all pair with the samenums[r]to give the same triplet. Skip them.while l < r and nums[r] == nums[r+1]: r -= 1(symmetric for the right pointer) — same reasoning.
The early exit if nums[i] > 0: break is correct because once the smallest element of the triplet is positive, no triplet sums to zero (the array is sorted).
Complexity
Time O(N²): O(N log N) sort + O(N²) total work in the nested two-pointer (each l and r move monotonically over a window of size O(N), and the outer i runs N times). Space O(1) extra (excluding output and the sort’s O(log N) recursion stack).
Implementation Requirements
- Sort in place (do not allocate a sorted copy).
- Skip duplicates inside the loop, not via a hash set on output.
- Use the early-exit
if nums[i] > 0: break(small but real speedup for typical inputs). - Move both pointers on a match, then run the skip loops — not before, or you’ll never advance off the matched element.
- Return a
List<List<Integer>>/list[list[int]]— not a set, not tuples.
Tests
- Smoke:
[-1,0,1,2,-1,-4]→[[-1,-1,2],[-1,0,1]]. - Unit: all-zeros (
[0,0,0,0]→[[0,0,0]]); no triplets ([1,2,3]→[]); minimum size ([0,0,0]→[[0,0,0]]). - Edge: size 0 / 1 / 2 →
[]; all-negative; all-positive (sum > 0 from index 0 → break immediately, return[]). - Large: N = 3000, values in
[-10^5, 10^5]; assert sub-second; verify count against the brute force on a 100-element prefix. - Random: generate 50 random inputs of size ≤ 200; compare against the brute force solution as oracle.
- Adversarial:
[0]*3000(all zeros — exactly one triplet[0,0,0]); long run of duplicates of a single value. - Invalid: non-integer / null input — out of scope per constraints, but mention you’d validate at the boundary.
Follow-up Questions
- “What about kSum?” → recurse:
kSum(nums, target, k)calls(k-1)Sum(remaining, target - nums[i]), base case is 2Sum-sorted. Time O(N^(k-1)). - “What if the array is unsorted and you cannot sort it?” → 2Sum-with-hash inside an outer enumeration; duplicate handling becomes per-output-set deduplication, more memory.
- “What if values repeat extremely (e.g., 99% zeros)?” → the duplicate skips handle this in O(N²) worst case, but in practice each outer iteration is O(1) for the duplicate values; you’d see a near-O(N) effective runtime on that adversarial input.
- “Can you do better than O(N²)?” → not under standard reductions: 3Sum has a known conditional lower bound of Ω(N²) (3SUM-hardness conjecture). Subquadratic 3Sum implies subquadratic many problems in computational geometry.
- “What about returning closest triplet to a target?” → 3SumClosest variant; same skeleton, track the best
|s - target|seen.
Product Extension
Detecting fraud rings of size 3 in a transaction graph where the signed sum of three transactions must net to zero (cancel out), under the constraint that the transactions hashed-distinctly. The same skeleton — fix one transaction, two-pointer the rest by signed amount — works, with the wrinkle that “duplicate” must be defined carefully (transactions are distinct by ID even if amounts are equal, so the duplicate skipping is replaced by a per-amount enumeration that emits all distinct ID combinations summing to zero).
Language/Runtime Follow-ups
- Python:
nums.sort()is in place. Usenums.append([nums[i], nums[l], nums[r]])notnums.append((..))if a list-of-lists is required (LC accepts either, but the contract is list). - Java: sort with
Arrays.sort(nums)— note this is dual-pivot quicksort for primitives, average O(N log N). ForInteger[]it’s TimSort. UseList<List<Integer>>and addArrays.asList(a, b, c)per triplet —Arrays.asListreturns a fixed-size list, which is fine because we never mutate it. - Go:
sort.Ints(nums)sorts in place; the rest is pointer arithmetic over indices. - C++:
std::sort(nums.begin(), nums.end()). Usestd::vector<std::vector<int>>for output. Beware: integer additionnums[i] + nums[l] + nums[r]can overflow 32-bit if value bounds were larger; with the given constraints it’s safe, but as a habit, uselong long. - JS/TS:
Array.prototype.sort()defaults to lexicographic comparison — this is a top-3 JS interview bug. Usenums.sort((a, b) => a - b). Also,a - bcan overflow 32-bit if values exceed ±2^30; for very large values useMath.sign(a - b). - Adversarial: sorting is the dominant constant; if the input is already sorted (best case for many sorts) this is faster than the brute by another factor.
Common Bugs
- Forgetting the
nums[i] == nums[i-1]skip on outer — produces duplicate output triplets like[-1,-1,2]repeated. - Forgetting to advance pointers on match —
l += 1; r -= 1must come before the inner duplicate-skip loops; otherwisenums[l] == nums[l-1]is comparingnums[l]to itself and the skip loop runs forever (well, runs untill == r, but produces no progress on the matched value). - Using
<vs<=inwhile l < r—l == rwould mean the same index appearing as both pointers, which is invalid (i, j, k must be distinct indices). - JS lexicographic sort —
[-1, -1, 2, -4, 0, 1]after default sort is[-1, -1, -4, 0, 1, 2](string-sorted). Always pass a numeric comparator. - Missing 32-bit overflow in C++/Java/Go when constraints allow large values. With LC-3Sum’s constraints it doesn’t bite, but the habit costs nothing and saves you on related problems (4Sum, kSum-with-target).
- Hash-set deduplication on output — works, but signals you didn’t internalize the invariant. Time still O(N²) but space O(N²) instead of O(1).
- Sorting twice by accident (once explicitly, once implicit in a downstream API) — innocuous but signals carelessness.
Debugging Strategy
- Run on
[0,0,0,0]first — should return[[0,0,0]]. If you get[[0,0,0],[0,0,0]], your outer-skip is broken. If you get[], your inner loop never matches (likelyl < rtypo). - Run on
[-2,0,0,2,2]. Expected:[[-2,0,2]]. If you get[[-2,0,2],[-2,0,2]], your inner-rskip is broken. - Diff against the brute force on 50 random inputs of size 50. The answers should match as sets (order of triplets and inner order doesn’t matter; sort each triplet and the list of triplets to compare).
- Time a 3000-element random input. Should run in well under a second in Python; under 50ms in C++ / Java / Go.
Mastery Criteria
- Recognized the signal “sorted, find triplet summing to target” as 3Sum within 30 seconds of reading.
- Articulated the three skip-clauses before writing them.
- Wrote a correct implementation in under 8 minutes, no lookup.
- Passed the all-zeros, single-duplicate, and large-N tests on the first attempt.
- Stated the conditional Ω(N²) lower bound when asked “can you do better?”.
- Identified the JS lexicographic-sort trap (or its language equivalent) without prompting.
- Generalized verbally to kSum and to 3SumClosest.
Lab 02 — Sliding Window: Longest Substring With At Most K Distinct Characters
Goal
Master the variable-size sliding window with a frequency map and a “shrink while violation” invariant. Deliverable solves the problem in O(N) time, O(K) extra space, with the loop invariant articulated explicitly: at the end of every iteration, s[l..r] contains at most K distinct characters.
Background Concepts
The shrink-while-violation skeleton; using a count of distinct items vs a full hashmap traversal; the difference between “at most K” and “exactly K” (the atMost(K) - atMost(K-1) trick); when fixed-size and variable-size windows apply. Review pattern Sliding Window.
Interview Context
This problem (LeetCode 340) is a Google / Meta favorite, and a near-direct ancestor of LeetCode 76 (Minimum Window Substring), 159 (At Most Two Distinct), 992 (Subarrays With K Different Integers), and 424 (Longest Repeating Character Replacement). The interview signal is whether you maintain the invariant cleanly: a single while violation: loop that shrinks until valid, then unconditional answer-update. Weak candidates write nested if-statements with off-by-one errors. Strong candidates write the canonical 5-line shrink-and-update.
Problem Statement
Given a string s and an integer K, return the length of the longest substring of s that contains at most K distinct characters.
Constraints
1 ≤ |s| ≤ 5 × 10^40 ≤ K ≤ |s|sconsists of arbitrary characters (Unicode in some variants; ASCII in the standard variant).
Clarifying Questions
- Is the alphabet ASCII or Unicode? (If Unicode, hashmap; if ASCII-lowercase, an
int[26].) - What if
K == 0? (Empty substring is valid; answer is 0.) - What if
K >= number of distinct chars in s? (Answer islen(s).) - Is the answer the length or the substring itself? (LC asks length; mention you can record
(start, length)for the substring.) - Can
sbe empty? (Per constraints,|s| ≥ 1, but the function should return 0 on empty input.)
Examples
| Input | Output | Note |
|---|---|---|
s="eceba", K=2 | 3 | “ece” |
s="aa", K=1 | 2 | “aa” |
s="abcabc", K=2 | 4 | “bcbc” or “cabc” — wait, “cabc” has 3 distinct; correct: “bcbc” or “abca” — both length 4 |
s="a", K=0 | 0 | empty |
s="abcdef", K=10 | 6 | full string |
Initial Brute Force
Enumerate all substrings, check distinct-count, track max.
def longest_brute(s, K):
best = 0
for i in range(len(s)):
seen = set()
for j in range(i, len(s)):
seen.add(s[j])
if len(seen) <= K:
best = max(best, j - i + 1)
else:
break
return best
The inner loop can break on first violation, so this is effectively O(N · K) average and O(N²) worst case (when K is large enough that no break happens).
Brute Force Complexity
Time O(N² · |alphabet|) at worst (set ops). At N=5×10^4, that’s ~2.5×10^9 — too slow in any language.
Optimization Path
Observation: as r advances, the set of distinct characters in s[l..r] is monotone non-decreasing; as l advances (keeping r fixed), it is monotone non-increasing. So we can use a two-pointer / sliding window: advance r, and while the window has more than K distinct chars, advance l. Each character enters and leaves the window at most once, total O(N).
Use a frequency map freq[c] keyed by character; distinct is the count of characters with freq[c] > 0. Increment distinct when a key first reaches 1; decrement when a key drops to 0.
Final Expected Approach
def longest_at_most_k_distinct(s, K):
if K == 0: return 0
freq = {}
l = 0
best = 0
for r, c in enumerate(s):
freq[c] = freq.get(c, 0) + 1
while len(freq) > K:
freq[s[l]] -= 1
if freq[s[l]] == 0:
del freq[s[l]]
l += 1
best = max(best, r - l + 1)
return best
We use len(freq) directly as the distinct count, since we delete keys that hit zero.
Data Structures Used
- A hashmap
freq: char → intof size at most K+1 during the violation, ≤ K otherwise. - Two integer pointers
l,r. - A running maximum
best.
Correctness Argument
Invariant: at the start of each iteration of the outer loop and after the inner shrink, freq contains exactly the characters of s[l..r] with their counts, and len(freq) ≤ K.
Base: before iteration 0, l = 0, freq = {}, len(freq) = 0 ≤ K. ✓
Step: we add s[r] to freq. If this brings len(freq) > K, we shrink: decrement freq[s[l]], delete on zero, advance l. The shrink loop terminates because l ≤ r always (proved: each shrink-step removes one character, and at most r - l + 1 characters can be in the window, so after at most r - l + 1 shrinks we have len(freq) ≤ 1 ≤ K).
Optimality (max): for each r, the smallest l such that the window has ≤ K distinct is recorded; this gives the longest valid window ending at r. Taking the max over all r gives the global longest.
Why while, not if: in this problem each new character can add at most one to distinct, so if would also work. But the canonical sliding-window template uses while because in cousin problems (LC 76) a single r-advance can violate by more than 1 (when adding required chars). Always default to while; pay the (zero) cost of generality.
Complexity
Time O(N): each character is added once, removed at most once. Each hash op is O(1) average. Space O(min(N, K+1)) for the frequency map.
Implementation Requirements
- Use
len(freq)(or maintain adistinctcounter) — do not iterate the map to count. - Delete keys that hit zero, or your
len(freq)will be wrong. - Update
bestafter the shrink, not inside it. (Inside the shrink, the window is invalid.) - For ASCII-lowercase, an
int[26]plus a separatedistinctcounter is faster than a hashmap (no hashing constant). - Guard
K == 0(orK < 0) at the top.
Tests
- Smoke:
("eceba", 2) → 3,("aa", 1) → 2. - Unit:
K = 0→ 0;K ≥ |distinct(s)|→|s|; single-character string. - Edge:
s = ""→ 0;K = 1, s = "abcdef"→ 1;K = |s|→|s|. - Large:
|s| = 5 × 10^4, alphabet 26,K = 3; should run in well under 100ms. - Random: generate 100 random strings of length ≤ 200, alphabet of varying size, varying K; cross-check against the brute force.
- Adversarial: all-same-character (
"aaaa...", K=1 → N); strictly increasing alphabet ("abcdef...", K → answer is K). - Unicode follow-up: make sure your Java/JS code iterates by codepoint, not by char, if the spec extends to Unicode.
Follow-up Questions
- “Now solve exactly K distinct.” →
atMost(K) - atMost(K-1)for the count variant; for the longest-with-exactly-K, run the same sliding window but only recordbestwhenlen(freq) == K(not≤ K). - “Now solve LC 76 (Minimum Window Substring).” → same skeleton, but the violation is “window does not yet contain all required chars”; we grow until satisfied, then shrink while still satisfied, recording the minimum each time we’re satisfied.
- “What if the string is streamed and only
radvances?” → two-pointer doesn’t apply directly; you’d need an order-preserving structure. (Out of scope here.) - “What if K can change with each query?” → preprocess differently; this problem is not amenable to a single offline sliding window for many K values.
- “What if the input is very large, alphabet huge, but
sonly contains a few distinct chars?” → no change; the hashmap is bounded bymin(K+1, |distinct(s)|).
Product Extension
A real-time content-moderation system tracks the longest run of messages in a chat where at most K distinct emojis are used (an indicator of spam-bot activity, which tends to use a small bag of emojis on repeat). The sliding window updates per message in O(1) amortized. The same skeleton applies to “longest session window with at most K distinct user-agents” for fraud detection, and “longest range of cells with at most K distinct values” for spreadsheet anomaly detection.
Language/Runtime Follow-ups
- Python:
dictoverhead is real; for ASCII-lowercase,[0]*26plus adistinctint is ~3× faster. Uses = list(s)only if you need indexing speed (string indexing is already O(1)). - Java:
HashMap<Character, Integer>boxes keys and values. For ASCII, useint[128]and trackdistinctmanually.chars()method exists buts.charAt(i)is fine. - Go:
map[byte]intfor ASCII;map[rune]intfor Unicode.range son a string yields(byte_index, rune)— this is a top-3 Go string trap:for i, c := range sdoes not give youias character index for multi-byte chars. - C++:
std::unordered_map<char, int>works; for ASCII,std::array<int, 128>is faster. - JS/TS:
Map<string, number>works. Iteratingfor (const c of s)yields codepoints, not UTF-16 code units — important if Unicode is in scope. Otherwise,s[i]works for ASCII. - Unicode caveat: “distinct character” might mean codepoint, grapheme cluster, or UTF-16 code unit — clarify with the interviewer.
Common Bugs
- Updating
bestinside the shrink loop — records invalid windows, returns wrong answers when the only valid window length is small. - Forgetting to delete zero-count keys —
len(freq)becomes stale, breaks the violation check. Equivalent bug: maintaining a separatedistinctcounter and forgetting to decrement when a count hits zero. - Using
ifinstead ofwhile— works for this problem but breaks for LC 76 and friends. Build the habit ofwhile. - Off-by-one in
r - l + 1—ris inclusive,lis inclusive; window length isr - l + 1. - Java boxing in
HashMap<Character, Integer>— autoboxing tax is ~3× over the primitiveint[]approach for ASCII. - Go string range bug — iterating with
for i := 0; i < len(s); i++and indexing ass[i]gives bytes, not runes; for UTF-8 data this misclassifies multi-byte chars as multiple distinct ones. - JS UTF-16 surrogate pair bug —
s.lengthfor"😀abc"is 5 (surrogate + 3 ASCII), ands[0],s[1]are the surrogate halves, not the emoji.
Debugging Strategy
- Trace
("eceba", 2)by hand. Window evolution:e | ec | ece | (shrink to ce, then) cebᴬ— whenbenters, window hasc, e, b(3 distinct), shrink: removec(was at position 1), window ise bthen add … etc. If your trace doesn’t match, your shrink logic is wrong. - Diff against the brute force on 50 random inputs.
- Print
(l, r, len(freq), best)per iteration;len(freq)should never exceed K after the shrink completes.
Mastery Criteria
- Recognized “longest substring with property” as sliding window in <30 seconds.
- Wrote the canonical shrink-while-violation template with the answer-update outside the shrink, in <5 minutes.
- Articulated the loop invariant and the termination of the shrink loop.
- Generalized to “exactly K distinct” via the
atMost(K) - atMost(K-1)trick. - Identified the language-specific Unicode / boxing trap.
- Solved LC 76, LC 424, LC 992 within a week, observing the same skeleton.
Lab 03 — Prefix Sums: Subarray Sum Equals K
Goal
Master the prefix-sum + hashmap-of-complements pattern. Deliverable solves LeetCode 560 in O(N) time, O(N) space, and you can articulate why a hashmap of prefix sums (not of values) is the right abstraction, why {0: 1} is the required base case, and why this generalizes to subarray-XOR-equals-K, subarray-sum-divisible-by-K, and friends.
Background Concepts
Prefix sum identity: sum(a[l..r]) = prefix[r+1] - prefix[l]. Reformulating “find subarrays with sum K” as “find pairs of prefix sums differing by K”. Hashmap as a complement-finder. Generalization to any group operation (XOR, mod, addition over any abelian group). Review pattern Prefix Sums and Hashing.
Interview Context
This is a rite-of-passage Medium: appears at Meta, Google, Amazon, Stripe. Naive candidates write O(N²) double loops over all subarrays. Decent candidates write a prefix-sum array then double loop over endpoints — still O(N²). Strong candidates collapse to O(N) with a hashmap-of-prefix-counts. Elite candidates immediately generalize to LC 974 (subarray sums divisible by K) and LC 525 (contiguous array — recast as prefix-balance equals zero) without prompting.
Problem Statement
Given an integer array nums and an integer K, return the total number of contiguous subarrays whose sum equals K.
Constraints
1 ≤ N ≤ 2 × 10^4-1000 ≤ nums[i] ≤ 1000-10^7 ≤ K ≤ 10^7- The array can contain negative numbers (this matters — sliding window does not apply).
Clarifying Questions
- Are values negative or non-negative? (Per constraints — both. This is the crucial clarification: with non-negatives, sliding window works in O(N); with negatives, you must use prefix sums.)
- Are zeros allowed? (Yes, and they create multiple subarrays of the same sum; the count must reflect this.)
- Empty subarrays — count them? (No; subarrays have ≥ 1 element. But the prefix-sum technique uses an “empty prefix” of value 0, hence
{0: 1}initialization.) - Is
Kalways reachable? (No assumption.) - Can
K = 0? (Yes — counts subarrays summing to 0, including those that are entirely zero.)
Examples
| Input | Output | Note |
|---|---|---|
nums=[1,1,1], K=2 | 2 | [1,1] at indices (0,1) and (1,2) |
nums=[1,2,3], K=3 | 2 | [3] and [1,2] |
nums=[1,-1,0], K=0 | 3 | [1,-1], [0], [1,-1,0] |
nums=[0,0,0], K=0 | 6 | every contiguous subarray |
nums=[100], K=100 | 1 | trivial |
Initial Brute Force
Two nested loops over (l, r), sum nums[l..r], count.
def subarray_sum_brute(nums, K):
count = 0
for l in range(len(nums)):
s = 0
for r in range(l, len(nums)):
s += nums[r]
if s == K: count += 1
return count
Brute Force Complexity
Time O(N²). Space O(1). At N=2×10⁴, ~4×10⁸ ops — borderline; passes in C++ but TLEs in Python.
Optimization Path
The key reformulation: a subarray nums[l..r] sums to K iff prefix[r+1] - prefix[l] == K, i.e., prefix[l] == prefix[r+1] - K.
So as we walk r from 0 to N-1, computing the running prefix sum, we ask: “How many earlier prefix sums equal prefix - K?” This is a hashmap lookup. Each step is O(1). Total O(N).
The base case is subtle and important: before processing any element, we have prefix sum 0, “seen once”. This accounts for subarrays starting at index 0 (where the missing earlier prefix is the empty prefix of value 0).
Final Expected Approach
def subarray_sum(nums, K):
count = 0
prefix = 0
seen = {0: 1} # empty prefix
for x in nums:
prefix += x
count += seen.get(prefix - K, 0)
seen[prefix] = seen.get(prefix, 0) + 1
return count
Crucial ordering: lookup before insert. Otherwise the case K == 0 over-counts: every position would match itself.
Data Structures Used
- A hashmap
seen: int → intmapping each prefix-sum value to the number of times it has occurred. - A running
prefixinteger. - A running
countinteger.
Correctness Argument
Let p_i = sum(nums[0..i-1]) (so p_0 = 0). A subarray nums[l..r] sums to K iff p_{r+1} - p_l = K iff p_l = p_{r+1} - K.
For each r, the number of valid l ∈ [0, r] is |{l : p_l == p_{r+1} - K}|. As we iterate, seen after processing index r contains exactly {p_0, p_1, ..., p_{r+1}} with multiplicities. Looking up seen[prefix - K] before inserting prefix gives |{l ∈ [0, r] : p_l == p_{r+1} - K}| — the count of valid l for the current r.
Summing over all r gives the total count of valid subarrays. The {0: 1} initialization handles the case l == 0 (where p_0 == 0 is consulted).
Complexity
Time O(N) average (hashmap ops). Space O(N) for the hashmap (worst case: all distinct prefix sums). Worst-case time degrades to O(N²) under hash collisions on adversarial input — see Phase 1 §3.
Implementation Requirements
- Initialize
seen = {0: 1}before the loop. Forgetting this causes off-by-one on subarrays starting at index 0. - Lookup before insert. This isn’t just style — for
K == 0, swapping the order miscounts trivially. - Use 64-bit accumulator if values × N could overflow 32-bit (here,
2×10^4 × 1000 = 2×10^7, safe; but build the habit). - Don’t precompute the prefix-sum array if you don’t need to — a single running int suffices.
Tests
- Smoke:
([1,1,1], 2) → 2. - Unit:
K = 0cases;Klarger than total sum (returns 0); single element matching K (returns 1); single element not matching K (returns 0). - Edge:
nums = [0]*N, K = 0→N*(N+1)/2. This validates that zeros are correctly counted. - Adversarial: alternating positives and negatives summing to K many times. Construct
[1,-1,1,-1,...,1,-1]with K=0; expected count is N/2 + N/2*(N/2-1)/2 + … — easier to validate against the brute force. - Large: N = 2×10⁴, random values; assert ms-level. Compare to brute force on a 1000-element prefix.
- Random: 100 random inputs of size ≤ 200; cross-check against brute.
- Negative K: include K negative; must work because prefix sums and
prefix - Kuse the same arithmetic.
Follow-up Questions
- “Subarray sum divisible by K?” → key by
prefix % K(taking care to normalize to non-negative for languages where%can return a negative value:((prefix % K) + K) % K). Count pairs of equal residues. - “Subarray with XOR equal to K?” → exact same skeleton, replace
+with^. The group operation just needs an identity and an inverse. - “Longest subarray with sum K?” → store the first occurrence of each prefix-sum value; when you see
prefix - K, the length isr+1 - first[prefix - K]. - “Contiguous array (LC 525) — equal 0s and 1s?” → re-encode 0s as -1; the problem becomes “longest subarray with sum 0”.
- “If memory is tight (cannot store hashmap)?” → if values are non-negative, sliding window in O(1) extra; otherwise no known better.
- “What if the array is updated?” → Fenwick tree for prefix sums; each update O(log N), each query O(log N) (Phase 3).
Product Extension
Anomaly detection in cumulative metric streams. Imagine network ingress where a “subarray sum equals K” query asks: “how many time windows had exactly K bytes of traffic?” — useful for detecting periodicity or replay patterns. The same prefix-hashmap pattern, run online, detects these in O(1) per event with O(window-size) memory. Extends to log-aggregation systems (CloudWatch, Datadog) where dashboards scan millions of events and need O(N) algorithms.
Language/Runtime Follow-ups
- Python:
defaultdict(int)is cleaner thandict.get(...). Performance is similar. - Java:
HashMap<Long, Integer>for safety. Boxing! EachgetOrDefault(prefix, 0) + 1followed byput(prefix, ...)does up to 4 boxes; pre-boxing ormerge()with Integer::sum reduces it. - Go:
map[int]int; the zero value of int is 0 soseen[prefix - K]returns 0 if absent — no need forokcheck on lookups (but you do need to insertseen[0] = 1initially). - C++:
std::unordered_map<long long, int>. Beware:unordered_map<int, int>::operator[]inserts on read of a missing key, growing the map by one for every miss. Usefind()for lookup-only. - JS/TS: prefer
Map<number, number>;Objectkeys are coerced to strings. For very large prefix sums (> 2^53), useBigIntor strings as keys. - Adversarial hashing: crafted inputs producing many distinct prefix sums that collide can degrade to O(N²) in languages with deterministic hash. Java’s
HashMapupgrades long chains to red-black trees; Python and Go randomize; C++ does not by default.
Common Bugs
- Missing
{0: 1}initialization — off-by-one for any subarray starting at index 0. Concretely:([3], 3)returns 0 instead of 1. - Insert before lookup — for
K == 0, every position matches itself once, returning N spurious counts. - Using
dict[key]instead ofdict.get(key, 0)in Python — KeyError on first miss. - Java autoboxing penalty — silent ~3× slowdown; use primitive maps (e.g., Eclipse Collections
IntIntMap) orHashMap.merge. - C++
unordered_map::operator[]insertion-on-read — bloats the map and ruins iteration order; usefind()for read-only. - Negative-modulo bug in the LC 974 follow-up —
-3 % 5is-3in C++/Java/Go but2in Python. Normalize:((x % K) + K) % K. - Recomputing
prefix - Ktwice — minor, but(prefix - K)should be a single local for readability.
Debugging Strategy
- Trace
([1,1,1], 2)by hand. Expectedseenevolution:{0:1} → {0:1,1:1} → {0:1,1:1,2:1} → {0:1,1:1,2:1,3:1}. At each step, lookupprefix - K: at step 2,prefix=2, lookup0: count += 1. At step 3,prefix=3, lookup1: count += 1. Total 2. - For
K = 0issues: trace([0,0,0], 0).seenevolves{0:1} → (lookup 0: count+=1) → {0:2} → (lookup 0: count+=2) → {0:3} → (lookup 0: count+=3). Total 6 =3*(3+1)/2. ✓ - Diff against brute force on 100 random small inputs.
Mastery Criteria
- Recognized “subarray with sum X” + negatives as prefix-sum + hashmap in <30 seconds.
- Articulated the
{0: 1}base case before coding. - Lookup-before-insert ordering correct on first try.
- Generalized to LC 974 (mod) and LC 525 (re-encoding) without lookup.
- Identified C++
operator[]insertion-on-read trap. - Solved on the first try with no off-by-one bugs in 5 random small tests.
Lab 04 — Binary Search On Answer: Capacity To Ship Packages Within D Days
Goal
Master the parametric/binary-search-on-answer pattern: identify the monotonic predicate, prove its monotonicity, write a correct feasible(x) independently, then drive a half-open binary search over the answer space. Deliverable solves LC 1011 in O(N · log(sum)) time, O(1) space, and you can articulate the bounds, the predicate’s monotonicity proof, and the canonical [lo, hi) template’s termination.
Background Concepts
Decision problem vs optimization problem; parametric search; monotone predicates; the half-open [lo, hi) invariant feasible(hi-1) = true, feasible(lo-1) = false; greedy verification of feasibility. Review pattern Binary Search On Answer and the constraint→complexity table.
Interview Context
Binary search on answer is the single highest-value pattern for distinguishing strong from elite candidates in 30-minute Mediums. Most candidates can do binary search on a sorted index. Few can recognize that “min capacity such that we can ship in D days” is the same binary search, just over a different domain. Once you internalize this, an entire family of problems collapses: Koko bananas, split-array-largest-sum, smallest divisor, minimum days for bouquets, magnetic force balls, kth-smallest-in-multiplication-table. The pattern is identical; only feasible changes.
Problem Statement
A conveyor belt has packages with weights weights[i]. Each day you load packages in order and ship them on a boat with weight capacity C. Once loaded, the boat ships and starts again the next day. Return the minimum capacity C such that all packages ship within D days.
Constraints
1 ≤ D ≤ |weights| ≤ 5 × 10^41 ≤ weights[i] ≤ 500
Clarifying Questions
- Must packages be loaded in given order? (Yes — this is the crux. If we could reorder, it becomes a bin-packing problem, which is NP-hard.)
- Can a package’s weight exceed the daily capacity? (No — capacity must be at least
max(weights), else that package never ships.) - Can the boat be partially filled and still ship that day? (Yes — when adding the next package would exceed
C, you ship the current load and start fresh.) - Is
Dstrict (must use all D days) or upper-bound (≤ D days)? (Upper-bound: any number of days ≤ D is acceptable.) - Are weights integers, and if so, is the answer always an integer? (Yes and yes.)
Examples
| Input | Output |
|---|---|
weights=[1,2,3,4,5,6,7,8,9,10], D=5 | 15 |
weights=[3,2,2,4,1,4], D=3 | 6 |
weights=[1,2,3,1,1], D=4 | 3 |
weights=[10,50,100,100,50,100,100,100], D=5 | 200 |
For the first: C=15 ships as [1,2,3,4,5] (15) | [6,7] (13) | [8] (8) | [9] (9) | [10] (10) — 5 days. C=14 would split day 1 into [1,2,3,4] | [5,6,7] | … and need 6+ days.
Initial Brute Force
Try every capacity from max(weights) to sum(weights); for each, simulate and check if D days suffice; return the first that works.
def ship_capacity_brute(weights, D):
for C in range(max(weights), sum(weights) + 1):
if feasible(weights, D, C): return C
def feasible(weights, D, C):
days, load = 1, 0
for w in weights:
if load + w > C:
days += 1; load = 0
load += w
return days <= D
Brute Force Complexity
Time O(N · range), where range = sum - max ≈ N · 500. So O(N²·500) ≈ 1.25×10^9. TLEs.
Optimization Path
Key observation: feasibility is monotonic in capacity. If C works (ships in ≤ D days), any C' > C also works (the greedy simulation can only do better — it never has to start a new day earlier). Equivalently, if C doesn’t work, no C' < C works.
So the set {C : feasible(C) == true} is an upward-closed half-line [C*, ∞). We binary search for C*.
Bounds:
- Low:
max(weights)— any capacity below this can’t even ship the heaviest package alone. - High:
sum(weights)— capacity ≥ total ships everything in 1 day, trivially ≤ D days.
Final Expected Approach
def ship_capacity(weights, D):
def feasible(C):
days, load = 1, 0
for w in weights:
if load + w > C:
days += 1; load = w
else:
load += w
if days > D: return False
return True
lo, hi = max(weights), sum(weights) + 1 # half-open [lo, hi)
while lo < hi:
mid = (lo + hi) // 2
if feasible(mid):
hi = mid
else:
lo = mid + 1
return lo
Note hi = sum + 1 (half-open) and the canonical lo < hi loop condition.
Data Structures Used
None beyond the input array and three integers (lo, hi, mid) plus two more inside feasible.
Correctness Argument
Monotonicity: Let feasible(C) be true. For C' > C, the greedy simulation with capacity C' packs at least as much per day as it would with C (because the “if load + w > C” branch fires at most as often). So days-needed-with-C' ≤ days-needed-with-C ≤ D. Hence feasible(C') is true. The set of feasible C is upward-closed.
Greedy feasibility: the greedy “ship today as much as fits” is optimal for min days given fixed C, by an exchange argument: if there’s a schedule that ships in fewer days, we can shift weight from a later day to an earlier day without exceeding C, monotonically reducing days. So feasible(C) correctly reports whether some schedule fits in D days.
Binary search invariant (half-open [lo, hi)): feasible(lo - 1) = false (or lo - 1 < max(weights)) and feasible(hi) either true or hi = sum + 1 (definitely feasible). The loop preserves this. On termination lo == hi, and feasible(lo) = true, feasible(lo - 1) = false. So lo is the minimum feasible C.
Termination: each iteration strictly shrinks hi - lo by at least 1 (in the mid + 1 branch) or halves it (in the hi = mid branch). Bounded by O(log(hi - lo)).
Complexity
Time O(N · log(sum - max)). With N = 5×10^4 and sum bounded by 2.5×10^7, that’s ~5×10^4 × 25 ≈ 1.25×10^6 ops. Easily sub-100ms in any language. Space O(1) additional.
Implementation Requirements
- Write
feasiblefirst, test it independently on the examples, then wrap it in binary search. Many bugs are infeasible, not the search. - Use
hi = sum + 1half-open, orhi = sumand<=; pick one convention and stick to it. The half-open[lo, hi)template returnsloexactly. - Early-return from
feasibleoncedays > D— saves time on smallC. - Don’t forget that on the “doesn’t fit” branch, the new day starts with the current package as its load, not zero (or else the next package may also overflow).
Tests
- Smoke: the four examples above.
- Unit:
D = N(each package its own day) → answer ismax(weights).D = 1→ answer issum(weights). - Edge: all-equal weights,
[5,5,5,5], D=2→ answer 10. Single weight,[42], D=1→ 42. - Independence test for
feasible: for the first example, verifyfeasible(15)=true, feasible(14)=false, feasible(55)=true, feasible(10)=false. - Large: N = 5×10⁴, weights random in
[1, 500], D = 1000; assert sub-100ms. Cross-check against brute on a 100-element prefix. - Adversarial: sorted ascending, sorted descending, all-max (weights all 500). The greedy is still optimal regardless of ordering (well, the answer depends on ordering since reorder is forbidden).
Follow-up Questions
- “What if package order is flexible?” → bin packing, NP-hard. Approximation: First-Fit-Decreasing achieves 11/9 OPT.
- “What if D is very large (D ≥ N)?” → answer is
max(weights); you can early-return. - “What if you must use exactly D days?” → still binary-searchable (monotonicity holds for “≤ D days”; for “exactly D”, parametrize differently — but in practice you almost always want ≤ D).
- “What about Koko Eating Bananas (LC 875)?” → identical pattern;
feasible(speed) = sum(ceil(p / speed) for p in piles) ≤ H. - “Split array largest sum (LC 410)?” → identical pattern;
feasible(maxSum) = (greedy partition into pieces of sum ≤ maxSum, count ≤ K). - “Floating-point answer?” → loop until
hi - lo < eps, returnlo. Watch for non-termination if eps is below float precision.
Product Extension
Capacity planning for a build pipeline: given a stream of CI jobs with known durations and a deadline D, find the minimum machine-count or the minimum machine-capacity that meets the deadline. Same pattern: monotonic in capacity, binary-search on answer with greedy feasibility. Generalizes to load-balancer auto-scaling: minimum number of pods s.t. p99 latency stays below SLA, given a workload trace. (The feasibility check becomes a simulator instead of a one-line greedy, but the structure is identical.)
Language/Runtime Follow-ups
- Python:
(lo + hi) // 2is fine — Python ints are arbitrary precision. No overflow. - Java:
int mid = lo + (hi - lo) / 2;to avoid 32-bit overflow whenlo + hiexceedsInteger.MAX_VALUE. (For this problem’s constraints,lo + hiwon’t overflow, but build the habit.) - Go: same overflow caveat as Java; use
lo + (hi - lo) / 2. - C++: same; use
lo + (hi - lo) / 2.intmay also be too small forsum(weights)if constraints expand — uselong long. - JS/TS: numbers are 64-bit floats, integer-precise up to 2^53. No overflow risk for this problem. But beware:
Math.floor((lo + hi) / 2)is slower than(lo + hi) >>> 1(zero-fill right shift), and the latter is what idiomatic JS binary-search uses. - Edge: for floating-point binary search (not this problem), terminate by iteration count (e.g., 100 rounds) rather than
hi - lo < epsto dodge non-termination near float precision.
Common Bugs
- Wrong bounds.
lo = 1instead oflo = max(weights)— the search may return a value that doesn’t fit the heaviest package (well, the binary search finds the smallest feasible value; if you start lo too low andfeasibleis correct, you still get the right answer — but you waste log(max) iterations and the symmetry of the bounds is broken).hi = suminstead ofsum + 1withlo < himis-handles the case where the answer issum. - Inverted predicate direction. Searching for “max C such that infeasible” instead of “min C such that feasible” — flips the bounds and breaks the invariant.
feasiblebug: forgetting to start the new day with the current package. Settingload = 0instead ofload = wafter exceeding capacity corrupts the count for runs of large weights.feasiblebug: startingdays = 0instead ofdays = 1. The first day exists before any package ships.- Off-by-one in the binary search template. Mixing
<with<=andmidwithmid - 1andmid + 1is the most common interview bug. Memorize one template (we use the half-open one) and never deviate. - C++/Java/Go integer overflow on
(lo + hi) / 2for very large constraints. - Calling
feasiblewith the wrong arg type (e.g., float when int expected) in dynamically-typed languages — silent rounding.
Debugging Strategy
- Test
feasibleindependently on the given examples for several values of C. The interactive tracefeasible(15)=true, feasible(14)=false, …is your ground truth. - Add an iteration counter to the binary search; cap it at 100. If the cap fires, your bounds or update direction is wrong.
- Print
(lo, mid, hi, feasible(mid))per iteration; thelo, hiinterval should shrink monotonically to a singleton. - For overflow suspicions, replace ints with
long/long long/bigintand rerun.
Mastery Criteria
- Identified “minimum X such that property P” + monotone P as binary-search-on-answer in <60 seconds.
- Stated the monotonicity argument in plain English before coding.
- Wrote
feasiblefirst, tested it independently, then wrapped it in binary search. - Used a single canonical binary-search template (half-open) without confusing
<vs<=. - Generalized verbally to LC 875 (Koko), LC 410 (Split Array Largest Sum), LC 1283 (Smallest Divisor) without prompting.
- Identified the language-specific overflow / template trap.
- Solved a similar new problem from this family in <10 minutes within a week of completing this lab.
Lab 05 — Monotonic Stack: Largest Rectangle In Histogram
Goal
Master the monotonic-stack pattern on its hardest canonical problem. Deliverable solves LC 84 in O(N) time, O(N) space, and you can articulate why each bar is pushed and popped at most once, why a sentinel 0 at the end is required (or how to handle the leftover stack), and how this generalizes to maximal-rectangle-of-1s in a 2D grid.
Background Concepts
Monotonic stack invariant; index-vs-value storage in stacks; sentinel technique for clean termination; amortized analysis (each index pushed and popped once); the rectangle’s “left boundary = new top after pop” / “right boundary = current index” trick. Review pattern Monotonic Stack.
Interview Context
LC 84 is one of the hardest commonly-asked Mediums (often labeled Hard). It appears at Google, Meta, and quant firms. The interview signal is whether you can derive the algorithm from a smaller cousin (LC 496 — Next Greater Element). Naive candidates write O(N²) “for each bar, expand left and right”. Decent candidates derive a left-bounds array and right-bounds array via two monotonic stack passes. Strong candidates do it in one pass with the popped-bar’s-rectangle trick. Elite candidates immediately observe that LC 85 (Maximal Rectangle) is just LC 84 applied per row of a derived heights array.
Problem Statement
Given an array heights representing histogram bar heights of equal width 1, return the area of the largest rectangle that fits within the histogram.
Constraints
1 ≤ N ≤ 10^50 ≤ heights[i] ≤ 10^4
Clarifying Questions
- Are heights non-negative? (Per constraints, yes.)
- Can heights be zero? (Yes — and zero-height bars effectively reset the candidates, since no rectangle can include them.)
- Are bars unit-width? (Yes — width 1 each, so the rectangle’s width is just the number of consecutive bars that all have at least the rectangle’s height.)
- Multiple equal-area rectangles — return any area, or specify? (Just the area; LC asks for the max.)
- Is the answer always achievable in 32-bit? (
max_height × N = 10^4 × 10^5 = 10^9, fits 32-bit signed but barely. Use 64-bit for safety in C++/Java.)
Examples
| Input | Output |
|---|---|
[2,1,5,6,2,3] | 10 (rectangle of height 5 covering indices 2..3 — width 2 — wait, height 5 × width 2 = 10? actually height 5 spans indices 2..3 (heights 5,6), so width 2 rectangle of height 5 — area 10. Or height 6 over index 3 alone, area 6. Best is 10.) |
[2,4] | 4 |
[2,1,2] | 3 (rectangle of height 1 spanning all 3 bars) |
[6,7,5,2,4,5,9,3] | 16 |
[0] | 0 |
Initial Brute Force
For each bar i, expand left and right while heights stay ≥ heights[i]; rectangle area is heights[i] × (right - left + 1).
def largest_rect_brute(heights):
n = len(heights)
best = 0
for i in range(n):
l = r = i
while l > 0 and heights[l - 1] >= heights[i]: l -= 1
while r < n - 1 and heights[r + 1] >= heights[i]: r += 1
best = max(best, heights[i] * (r - l + 1))
return best
Brute Force Complexity
Time O(N²) worst case (all-equal heights). Space O(1). At N=10⁵, ~10^10 ops — TLEs everywhere.
Optimization Path
Observation: for each bar i, we want the largest rectangle of at least height heights[i] that contains i. Width = (next-smaller-to-the-right) − (previous-smaller-to-the-left) − 1. If we know “previous smaller index” pl[i] and “next smaller index” pr[i] for every bar, area is heights[i] * (pr[i] - pl[i] - 1), computed in O(N) total.
Both pl and pr are computable by a single monotonic-stack pass each. Even better: a single pass with a stack-of-indices in strictly increasing height order. When we encounter a smaller bar, we pop the stack; for each popped index j, the current index i is its pr[j] and the new top of the stack (after the pop) is its pl[j]. Compute j’s rectangle area on the spot.
Final Expected Approach
def largest_rectangle_area(heights):
stack = [] # indices, heights[stack] strictly increasing
best = 0
n = len(heights)
for i in range(n + 1):
cur = 0 if i == n else heights[i]
while stack and heights[stack[-1]] > cur:
top = stack.pop()
left = stack[-1] if stack else -1
width = i - left - 1
best = max(best, heights[top] * width)
stack.append(i)
return best
The trick: iterate to n + 1 with a sentinel cur = 0. This forces every remaining bar to be popped (since 0 is strictly less than any positive height), so we don’t need a separate post-loop drain.
Data Structures Used
- A stack of indices into
heights, holding indices whose heights are strictly increasing from stack-bottom to stack-top. - A running
bestinteger.
Correctness Argument
Stack invariant: at every point, heights[stack[0]] < heights[stack[1]] < ... < heights[stack[-1]].
Maintenance: before pushing i, we pop all indices j with heights[j] >= heights[i] (using > ensures strict; for this problem, > is correct and >= would over-pop). After popping, all remaining stack entries have height < heights[i], so pushing i preserves the invariant.
Per-popped-bar rectangle is correct: when we pop top = stack.pop(), by the invariant the new top’s height < heights[top], so pl[top] is stack[-1] (or -1 if stack empty). The current index i is the first index since top with heights[i] < heights[top] (because all indices between top+1 and i-1 had heights ≥ heights[top] and were either still on the stack or popped earlier — but if popped earlier, they were popped by a strictly-smaller bar, contradiction). So pr[top] = i. Width is i - pl[top] - 1.
Sentinel correctness: at i = n we use cur = 0, smaller than any positive height. This pops every remaining index, computing each one’s rectangle with pr = n.
Amortized O(N): each index pushed once, popped at most once. Inner while loop’s total iterations across the outer loop sum to ≤ N.
Complexity
Time O(N) amortized. Space O(N) for the stack worst case (strictly increasing input).
Implementation Requirements
- Store indices, not heights, in the stack — you need indices to compute width.
- Use the sentinel trick (
i in range(n + 1)withcur = 0ati = n) for clean code, OR drain the stack after the main loop withpr = n. Pick one; the sentinel is preferred. - Use strict
>in the pop condition. For “largest rectangle”,>and>=give the same answer (rectangles with equal heights are accounted for by their leftmost bar), but>=over-pops and confuses the bookkeeping in cousin problems. - Use 64-bit arithmetic in Java/C++ for the area:
10^4 × 10^5 = 10^9, fits 32-bit, butInteger.MAX_VALUE = 2.1×10^9and habits matter.
Tests
- Smoke:
[2,1,5,6,2,3] → 10. - Unit:
[1] → 1,[1,1,1,1] → 4,[1,2,3,4,5] → 9(rectangle of height 3 over indices 2..4),[5,4,3,2,1] → 9. - Edge:
[0] → 0,[0,0,0] → 0,[N copies of 1] → N,[10000] → 10000. - Adversarial: strictly increasing — every bar stays on the stack until the sentinel; tests the drain. Strictly decreasing — every bar popped immediately; tests the per-bar bookkeeping.
- Large: N = 10⁵, random heights; assert <100ms. Cross-check against brute on a 1000-prefix.
- All same:
[7,7,7,7,7,7,7,7] → 56. - Random: 100 random inputs of size ≤ 200 against brute.
Follow-up Questions
- “Maximal rectangle of 1s in a 2D matrix (LC 85)?” → for each row
r, buildheights[c] = (heights[c] + 1 if mat[r][c] == 1 else 0). Then run LC 84 on each row’s heights. Total O(R·C). - “Largest rectangle of equal value?” → modify the predicate; same skeleton.
- “Number of submatrices with all 1s (LC 1504)?” → variant where for each row + column we count.
- “What if heights can be updated?” → segment tree with merge function; Phase 3 territory.
- “What if N is so large the stack doesn’t fit?” → the stack is bounded by N; if N doesn’t fit in memory, you have a bigger problem. (Answer: stream-based algorithms with reduced memory exist for some restricted versions.)
- “Why does
>=give the same answer here?” → bars of equal height to the popped bar fail to extend its rectangle leftward (they’d be popped first or become the new left boundary), so the answer is unchanged. Subtle, worth a sentence in the interview.
Product Extension
In ad-hoc analytics, the “largest rectangle” maps to “the longest time window during which all of K monitored systems exceeded a threshold” — useful for SLA breach detection. Each system’s per-time-bin status forms a histogram; the largest rectangle is the worst sustained breach. The same single-pass stack algorithm processes a stream of metric snapshots in O(1) amortized per snapshot.
Language/Runtime Follow-ups
- Python: native
listas a stack —append/popare O(1) amortized. Skipcollections.dequehere; the stack-only access pattern doesn’t benefit from a deque. - Java: prefer
ArrayDeque<Integer>overStack(the latter is a synchronized legacy class with overhead). Or useint[]with a manual top index — fastest for hot loops. - Go: slice-as-stack —
stack = append(stack, i),top := stack[len(stack)-1]; stack = stack[:len(stack)-1]. Beware: capacity may grow geometrically and not shrink — fine here since N is bounded. - C++:
std::vector<int>is fastest.std::stack<int>adds an unnecessary wrapper. Reserve capacity withstack.reserve(n). - JS/TS: native
Array.prototype.push/pop— O(1) amortized. Not as fast as a typedInt32Arrayfor hot loops. - Hot-loop: in Java,
int[]+topint outperformsArrayDeque<Integer>by ~3× due to no boxing.
Common Bugs
- Off-by-one width:
i - left - 1vsi - left. The popped bar’s rectangle starts atleft + 1and ends ati - 1(both inclusive), width =i - left - 1. - Forgetting the sentinel / drain. Without the sentinel
0, indices remaining on the stack are never processed. Their rectangles extend ton - 1withpr = n. - Using
>=instead of>(or vice versa). For LC 84, both happen to produce the right answer, but cousin problems break — pick the variant that gives a unique boundary. - Storing heights instead of indices. You then can’t compute width.
- Integer overflow in C++/Java for max-bar × max-N. Use 64-bit.
- Recursive simulation instead of iterative — Python’s default recursion limit is 1000, breaks at N > 1000.
- Java boxing in
Stack<Integer>orArrayDeque<Integer>— silent slowdown. Useint[]with manual top.
Debugging Strategy
- Trace
[2,1,5,6,2,3]. Stack evolution: push 0; at i=1 (height 1), pop 0 (height 2, left=-1, width=1, area=2); push 1; push 2; push 3; at i=4 (height 2), pop 3 (height 6, left=2, width=1, area=6); pop 2 (height 5, left=1, width=2, area=10); push 4; push 5; sentinel pops everything. Best = 10. ✓ - Trace
[1,1,1,1]. With>, the stack just accumulates[0,1,2,3]; sentinel pops 3 (area 1), pops 2 (area 2), pops 1 (area 3), pops 0 (area 4). Best = 4. ✓ - Cross-check 50 random inputs of size 50 against brute force.
Mastery Criteria
- Recognized “largest rectangle in histogram” as a monotonic-stack problem within 60 seconds.
- Stated the strict-monotonic-stack invariant before coding.
- Used the sentinel-
0trick on first attempt (or correctly drained post-loop). - Wrote
i - left - 1correctly, no off-by-one. - Generalized to LC 85 (max rectangle of 1s) without prompting.
- Articulated the amortized O(N) bound (each index pushed and popped once).
- Solved a cousin problem (LC 42 trapping rain water with stack, or LC 901 stock span) in <10 minutes within a week.
Lab 06 — Intervals: Meeting Rooms II (Heap Of Ends + Sweep-Line Alternate)
Goal
Master the two canonical interval algorithms — min-heap of end times and event-based sweep line — applied to the same problem (LC 253). Deliverable solves it both ways, in O(N log N) time, O(N) space, and you can articulate the tie-breaking rule, why one approach is more intuitive while the other generalizes more cleanly to “max concurrent X” problems.
Background Concepts
Sorting by start time as the canonical interval-prep; min-heap of end times tracking active intervals; sweep line as event stream (time, ±1) with stable tie-breaking; the “end before start” tie-break for closed-on-start, open-on-end intervals. Review pattern Intervals and Heap.
Interview Context
Interval problems appear at Meta, Google, and Amazon — and Meeting Rooms II in particular is a top-15 most-asked Medium. The interview signal is whether you can recognize that “min number of rooms” = “max concurrent meetings”, and then compute concurrency either via heap-of-ends or sweep. Strong candidates do one approach correctly; elite candidates do both, articulate the trade-off, and handle the open/closed interval tie-break correctly without prompting.
Problem Statement
Given an array of meeting time intervals intervals[i] = [start_i, end_i], return the minimum number of meeting rooms required.
Constraints
1 ≤ N ≤ 10^40 ≤ start_i < end_i ≤ 10^6- Each interval is half-open
[start, end): a meeting ending at timetand one starting at timetcan share a room.
Clarifying Questions
- Are intervals half-open or closed? (Crucial:
[1,3]and[3,5]— same room or not? Per LC 253, half-open: same room. This dictates the tie-break.) - Can two meetings start at the same time? (Yes — they need separate rooms.)
- Are intervals sorted? (No assumption.)
- Is
start < endstrict? (Per constraints, yes; no zero-duration meetings.) - Are room IDs significant, or just the count? (Just the count.)
Examples
| Input | Output |
|---|---|
[[0,30],[5,10],[15,20]] | 2 |
[[7,10],[2,4]] | 1 |
[[1,5],[5,10],[10,15]] | 1 (chained, half-open) |
[[1,5],[2,5],[5,10]] | 2 (overlap at [2,5]; the [5,10] reuses) |
[[1,2]] | 1 |
Initial Brute Force
For each pair (i, j), count overlaps; max over all time points.
def min_rooms_brute(intervals):
times = sorted(set(t for s, e in intervals for t in (s, e)))
best = 0
for t in times:
count = sum(1 for s, e in intervals if s <= t < e) # half-open
best = max(best, count)
return best
Brute Force Complexity
Time O(N²). Space O(N). At N=10⁴, ~10⁸ ops — borderline; passes in C++ but TLEs in Python.
Optimization Path A — Heap of End Times
Sort intervals by start. Maintain a min-heap of end times for currently active meetings. For each new interval (start, end):
- If the heap’s smallest end ≤ start, that room frees up — pop it.
- Push the new end.
The number of rooms needed = max heap size ever, which equals the final heap size if we don’t pop more than we push (and we don’t, by the invariant). Actually, simpler: rooms = heap size at end, since we only pop when reusing, never net.
import heapq
def min_rooms_heap(intervals):
intervals.sort(key=lambda x: x[0])
heap = []
for s, e in intervals:
if heap and heap[0] <= s:
heapq.heappop(heap) # reuse a room
heapq.heappush(heap, e)
return len(heap)
Optimization Path B — Sweep Line
Convert intervals to events (start, +1), (end, -1). Sort: by time, with end events before start events on ties (because [1,5] and [5,10] share a room). Sweep, tracking max concurrent.
def min_rooms_sweep(intervals):
events = []
for s, e in intervals:
events.append((s, +1))
events.append((e, -1))
events.sort() # (time, +1)>(time, -1): -1 sorts first since -1 < 1
cur = best = 0
for _, delta in events:
cur += delta
best = max(best, cur)
return best
The tie-break is automatic: (t, -1) < (t, +1) because -1 < 1 lexicographically. End events fire before start events at the same t, so a freed room is reused.
Final Expected Approach
Either A or B is acceptable; mention both in the interview.
Data Structures Used
- Heap approach: the input array (sorted) + a min-heap of end times.
- Sweep approach: an events array of size 2N + a single integer counter.
Correctness Argument (Heap)
Invariant: after processing intervals 0..i-1 in sorted-by-start order, heap contains the end times of all rooms that are still in use at time intervals[i-1].start. Equivalently, heap is the multiset of end times of meetings that haven’t ended yet by the time we’d schedule the next one.
When processing (s, e):
- Any heap top
≤ scorresponds to a room whose meeting has ended; it’s reusable. Pop one. - Push
efor the new meeting.
The final len(heap) is the cumulative maximum number of concurrent meetings, since we only pop when a room frees up (so the heap size strictly decreases only when an old room is reused for a new meeting; otherwise, it grows). Equivalently, max-active-at-any-point = max-rooms-ever-needed.
(Note: we only pop one room even if multiple have ended, but that’s fine — each subsequent meeting will pop its own.)
Correctness Argument (Sweep)
A sweep at time t maintains cur = number of intervals whose start ≤ t < end, half-open. The max of cur over all t is the max concurrency, which is the min rooms needed. The tie-break “end before start at the same time” implements the half-open convention: an end at t decrements cur before a start at t increments it, so cur correctly reflects “intervals active at exactly t” with the half-open semantics.
Complexity
Both: time O(N log N) (sort dominates; heap and sweep each O(N log N) and O(N) respectively after the sort). Space O(N).
Implementation Requirements
- Sort by start for the heap approach. By start ascending; tie-break doesn’t matter for the heap (because we always pop if
heap[0] ≤ s, which is correct for either tie-break). - For the sweep approach, the natural tuple sort
(time, delta)withdelta ∈ {-1, +1}already gives the right tie-break; don’t reverse the comparator. - Use a min-heap (Python
heapq, JavaPriorityQueuedefault, C++priority_queue<int, vector<int>, greater<int>>). - Don’t sort start-times and end-times separately into two arrays — that’s a third valid approach (the “two pointers” approach, equivalent to sweep) but make sure you understand it’s distinct from the heap approach.
Tests
- Smoke: the five examples above.
- Unit: all-disjoint intervals → 1; all-identical intervals → N; chain
[1,2],[2,3],[3,4]→ 1. - Edge: N = 1 → 1; intervals all start at 0 → N (all overlap).
- Tie-break:
[[5,10],[10,15]]→ 1 (half-open). If your test returns 2, your sort comparator is wrong. - Adversarial: tournament-bracket —
[[1,4],[2,5],[7,9],[1,5]](mix); validate against brute on small inputs. - Large: N = 10⁴, random intervals; both heap and sweep should run sub-50ms.
Follow-up Questions
- “Return the actual schedule (which interval goes in which room)?” → augment heap entries to
(end, room_id); reuse the poppedroom_idfor the new interval. - “Real-time scheduling: intervals arrive in order, decide room on the fly?” → still works; no offline assumption needed for the heap version (sweep needs all events upfront).
- “Closed intervals
[s, e]instead of half-open?” → flip the tie-break: start before end at the same time, i.e., sort(t, +1)before(t, -1). Or in heap version, changeheap[0] <= stoheap[0] < s. - “Maximum number of overlapping intervals (LC 1851)?” → identical pattern.
- “Insert / delete intervals dynamically?” → balanced BST keyed on start; O(log N) per op (Phase 3).
- “Generalize to weighted intervals?” → DP on intervals (interval scheduling maximization, LC 1235), Phase 3.
Product Extension
Resource allocation in CI/CD: minimum number of build agents required to handle a queue of (start, duration) jobs without delay, given they’re all queued ahead of time. Or: minimum servers needed to handle a known load profile (each request has a known service window). The same heap-of-ends pattern, run on a stream, computes peak concurrency in real time.
Language/Runtime Follow-ups
- Python:
heapqis min-only; for max-heap, negate.heapq.heappop(h)andheapq.heappush(h, x)are O(log N). For sweep,events.sort()on a list of tuples works directly thanks to lexicographic tuple ordering. - Java:
PriorityQueue<Integer>defaults to min-heap. Boxing tax on primitives — for hot loops, useIntPriorityQueuefrom a primitive collections lib. Sort intervals viaArrays.sort(intervals, (a, b) -> a[0] - b[0])— beware overflow (useInteger.compare(a[0], b[0])). - Go:
container/heaprequires implementing theheap.Interface(5 methods). Verbose but flexible. - C++:
std::priority_queue<int, std::vector<int>, std::greater<int>>for min-heap (default is max). Sort withstd::sort. - JS/TS: no built-in heap. Implement one (~30 lines) or use a library. For sweep,
events.sort((a, b) => a[0] - b[0] || a[1] - b[1]). Beware:Array.sortis not stable in all engines for older versions of V8 (it’s stable since ES2019 — so it’s fine on modern Node, but mention it). - Edge: Java’s
PriorityQueue.peek()is O(1);poll()is O(log N). C++’stop()is O(1);pop()is O(log N) but doesn’t return the value — calltop()first.
Common Bugs
- Wrong tie-break in sweep. For half-open intervals, end events must fire before start events at the same time. The natural
(time, ±1)tuple sort gives this for free (since -1 < +1); reversing the comparator breaks it. - Heap comparator on max-heap default (Java’s
PriorityQueuehas a min-heap default; C++’spriority_queuehas a max-heap default — easy to forget which is which). - Sorting by end instead of start in the heap approach — gives wrong room counts.
- Popping heap on
heap[0] < sinstead of<=— for half-open[s, e),<=is correct (a meeting ending atshas ended, the room is free for one starting ats). - Forgetting to push the new end after popping — heap loses an entry, undercount.
- Comparator overflow in Java:
a[0] - b[0]overflows when values are huge; useInteger.compare. - Sweeping events without separating same-time events — fragile; always tie-break explicitly even if the data “happens” to not have ties.
Debugging Strategy
- Trace
[[0,30],[5,10],[15,20]]with the heap approach: sort doesn’t change order. Process[0,30]: heap =[30]. Process[5,10]: 30 > 5, no pop; push 10; heap =[10, 30]. Process[15,20]: 10 ≤ 15, pop; push 20; heap =[20, 30]. Final size = 2. ✓ - For half-open tie-break: trace
[[5,10],[10,15]]. Heap:[10]; second interval,10 <= 10→ pop, push 15; heap =[15]. Final size 1. ✓ If you used<, you’d get 2. - For sweep: events
[(5,+1),(10,-1),(10,+1),(15,-1)]sorted:[(5,+1),(10,-1),(10,+1),(15,-1)]. Sweep:curbecomes 1, 0, 1, 0; max 1. ✓ - Validate against brute on 30 random inputs of size ≤ 50.
Mastery Criteria
- Recognized “min rooms” / “max concurrent intervals” within 30 seconds.
- Wrote both heap and sweep solutions within 10 minutes total.
- Articulated the half-open tie-break before coding.
- Handled the language-specific heap default (min vs max) without bugs.
- Identified the connection to LC 1851 (max overlapping intervals) and LC 1094 (Car Pooling).
- Solved LC 56 (Merge Intervals) and LC 57 (Insert Interval) within a week, observing the same sort-by-start skeleton.
Lab 07 — Topological Sort: Course Schedule II (Kahn’s Vs DFS)
Goal
Master both topological sort algorithms — Kahn’s BFS and DFS-postorder — applied to LC 210. Deliverable produces a valid course order in O(V + E) time, O(V + E) space, with cycle detection wired in. You can articulate when each algorithm is preferable, the standard cycle-detection check, and how this generalizes to dependency resolution and build systems.
Background Concepts
DAG topological order; indegree array as a Kahn’s prerequisite; DFS three-color marking (white/gray/black) for cycle detection; postorder reverse for DFS-topo; the equivalence of “topo order exists” and “graph is a DAG”; existence of cycle = len(order) != V for Kahn’s. Review pattern Topological Sort and Graph Foundations.
Interview Context
Topological sort appears at Meta (build dependencies), Amazon (course scheduling, package ordering), Google (Spanner schema migrations). The interview signal is whether you naturally pick Kahn’s for explicit ordering (where the BFS structure makes the algorithm self-documenting) and DFS for cycle detection or recursive structure (where the call stack mirrors the recursion). Weak candidates only know one. Strong candidates code Kahn’s and explain the DFS variant. Elite candidates discuss stable ordering (preserving input order on ties via priority queue) and comment on parallelizability.
Problem Statement
You must take numCourses numbered 0..n-1. Some courses have prerequisites: prerequisites[i] = [a, b] means you must take b before a. Return any valid ordering of courses to finish all of them, or an empty array if impossible (cycle).
Constraints
1 ≤ numCourses ≤ 20000 ≤ |prerequisites| ≤ 5000- All
[a, b]pairs unique;a != b.
Clarifying Questions
- Are duplicate prerequisite pairs possible? (Per constraints, no — but worth confirming, as it affects indegree counting.)
- Are self-loops
[a, a]possible? (Per constraints,a != b— no self-loops.) - If multiple valid orderings exist, return which one? (Any. But mention you can return the lex-smallest with a min-heap-based Kahn’s.)
- Output empty array on cycle, or null/exception? (LC says empty.)
- Are course IDs always
0..n-1contiguous? (Yes per LC; otherwise you’d need to map.)
Examples
| numCourses | prerequisites | Output |
|---|---|---|
| 2 | [[1,0]] | [0,1] |
| 4 | [[1,0],[2,0],[3,1],[3,2]] | [0,1,2,3] or [0,2,1,3] |
| 2 | [[1,0],[0,1]] | [] (cycle) |
| 1 | [] | [0] |
| 3 | [[0,1],[1,2],[2,0]] | [] (cycle) |
Initial Brute Force
Repeatedly find a course with no remaining prerequisites; remove it and its outgoing edges; repeat. If at some point no such course exists but courses remain, there’s a cycle.
def find_order_brute(n, prereqs):
deps = [set() for _ in range(n)]
for a, b in prereqs:
deps[a].add(b)
order = []
for _ in range(n):
for i in range(n):
if deps[i] is not None and not deps[i]:
order.append(i)
deps[i] = None
for j in range(n):
if deps[j] is not None: deps[j].discard(i)
break
else:
return []
return order
Brute Force Complexity
Time O(V³) (V scans × V lookups × V edge updates per round). At V=2000, ~8×10⁹ — TLEs.
Optimization Path
Kahn’s algorithm replaces “find a no-prereq course” + “remove it” with O(1) amortized operations:
- Compute
indegree[v]= number of incoming edges. - Initialize a queue with all
vhavingindegree[v] == 0. - Repeat: pop
ufrom queue, append to order, decrement indegree of each successorv; ifindegree[v]becomes 0, pushv. - If
len(order) == V: that’s the topological order. Else: cycle.
Each edge processed once. Each vertex enqueued/dequeued once. Total O(V + E).
DFS variant: run DFS from each unvisited node; on finish (postorder), prepend the node to the order. Detect cycles via the gray-vertex check.
Final Expected Approach (Kahn’s)
from collections import deque
def find_order(n, prereqs):
adj = [[] for _ in range(n)]
indeg = [0] * n
for a, b in prereqs:
adj[b].append(a) # edge b -> a (b is prereq of a)
indeg[a] += 1
q = deque(v for v in range(n) if indeg[v] == 0)
order = []
while q:
u = q.popleft()
order.append(u)
for v in adj[u]:
indeg[v] -= 1
if indeg[v] == 0:
q.append(v)
return order if len(order) == n else []
Final Expected Approach (DFS)
def find_order_dfs(n, prereqs):
adj = [[] for _ in range(n)]
for a, b in prereqs:
adj[b].append(a)
color = [0] * n # 0=white, 1=gray (on stack), 2=black (done)
order = []
def dfs(u):
if color[u] == 1: return False # back edge -> cycle
if color[u] == 2: return True
color[u] = 1
for v in adj[u]:
if not dfs(v): return False
color[u] = 2
order.append(u)
return True
for u in range(n):
if color[u] == 0 and not dfs(u):
return []
return order[::-1]
Data Structures Used
- Kahn’s: adjacency list, indegree array, queue, output list.
- DFS: adjacency list, color array, output list, implicit recursion stack.
Correctness Argument (Kahn’s)
Invariant: at any point, order contains a valid prefix of some topological order, and the queue contains exactly the unprocessed vertices with no remaining unsatisfied prerequisites (i.e., remaining indegree zero in the subgraph of unprocessed vertices).
Maintenance: when we pop u, all its prereqs are already in order (since indeg[u] == 0 in the residual graph means all its prereqs have been processed). Adding u extends a valid topo-prefix. For each successor v, decrementing indeg[v] reflects that u is now “satisfied”; if v’s residual indegree hits 0, all its prereqs are satisfied, so it’s eligible.
Cycle detection: if len(order) < n, some vertices were never enqueued, meaning their residual indegree never reached 0 — they’re inside a strongly connected component with a cycle (or downstream of one).
Correctness Argument (DFS)
The classical theorem: a directed graph is a DAG iff DFS encounters no back edges. We mark vertices gray when entering DFS, black on finish. Encountering a gray vertex via an outgoing edge is a back edge → cycle. Postorder reversed gives a valid topological order: when u finishes, all reachable-from-u vertices have already finished and are earlier in order; reversing puts them after u.
Complexity
Both: O(V + E) time, O(V + E) space (adjacency + queue/recursion-stack).
Implementation Requirements
- Build the adjacency list with edge direction
prereq → course(so we can decrementindeg[course]when aprereqis processed). The reverse direction also works but flips the topo-order interpretation; pick one convention and stay consistent. - Use
deque(Python) /ArrayDeque(Java) /container/list(Go) for the BFS queue, not a Pythonlist.pop(0)which is O(N). - DFS recursion: watch Python’s default recursion limit (1000) for large N. Either iterative DFS or
sys.setrecursionlimitfor V > ~900. - Cycle check after the loop (Kahn’s) or during (DFS gray check).
Tests
- Smoke:
(2, [[1,0]])→[0,1]. - Unit: no prereqs (
(3, [])→ any permutation of[0,1,2]); single chain ((4, [[1,0],[2,1],[3,2]])→[0,1,2,3]). - Cycle:
(2, [[1,0],[0,1]])→[]; longer cycle(3, [[0,1],[1,2],[2,0]])→[]. - DAG with multiple roots:
(4, [[2,0],[2,1],[3,2]])→ either[0,1,2,3]or[1,0,2,3]. Validate by checking the output is a permutation and all prereqs respected. - Edge: N=1, no prereqs →
[0]. - Large: N=2000, E=5000, random DAG; assert sub-100ms.
- Validator helper (write this!):
def is_valid_topo(order, n, prereqs): if len(order) != n or set(order) != set(range(n)): return False pos = {v: i for i, v in enumerate(order)} return all(pos[b] < pos[a] for a, b in prereqs)
Follow-up Questions
- “Return the lex-smallest valid order.” → replace the queue with a min-heap. Time O((V + E) log V).
- “Detect all vertices in cycles, not just whether any exist.” → SCC decomposition (Tarjan’s / Kosaraju’s), Phase 3.
- “Topological sort under updates (edges added/removed)?” → online topological order maintenance, hard problem; offline batched updates with reordering.
- “Parallel topological sort?” → at each round, all indegree-0 vertices can run in parallel; this is the natural parallelization for build systems (Bazel, Buck).
- “Schedule with time costs per task: minimize total wall time?” → critical-path method; longest path in DAG, computable in O(V + E) after topo-sort.
- “If V is huge (10^9) and the graph is implicit?” → streaming variant; need indegree of each vertex computed via input stream.
Product Extension
This pattern is dependency resolution. Build systems (Bazel, Make, Maven, npm/yarn lockfile resolution), database migration runners, terraform depends_on, container orchestration (Kubernetes init-containers), spreadsheet recalc, even React’s effect-dependency ordering. Course Schedule II is the toy version of “given declared dependencies, output a valid execution order” — and the DFS variant is what most build systems use, because they want to detect cycles early with a clear error path showing the offending cycle.
Language/Runtime Follow-ups
- Python:
collections.dequefor BFS queue;sys.setrecursionlimit(10**5)if doing DFS on large input. List-as-queue withpop(0)is O(N) — never use it. - Java:
ArrayDeque<Integer>is the canonical queue. Boxing tax forQueue<Integer>— for hot loops, useint[]ring buffer. - Go: no built-in queue; use a slice and
q = q[1:](cheap ifcap(q)doesn’t change) orcontainer/list(heavier). Slice-as-queue grows O(N) memory per shrink because Go doesn’t truncate-and-shift; if memory matters, periodically copy the live portion. - C++:
std::queue<int>for Kahn’s;std::vector<int>+ recursive DFS (iterative if V > ~10⁵ to avoid stack overflow with default 8MB stack). - JS/TS:
Array.prototype.shift()is O(N) — use index-based queue (let head = 0; q[head++]) for O(1) amortized. - Stack overflow: any DFS-topo on V > recursion-limit needs iterative implementation. The iterative version uses an explicit stack of
(vertex, iterator)pairs.
Common Bugs
- Edge direction confusion.
[a, b]means “b before a”, so the edge isb → a. Reversing it inverts the topological order and breaks the indegree computation. - Forgetting cycle detection. Returning
ordereven whenlen(order) < nproduces a partial order that misses some courses. - Using
list.pop(0)in Python (orQueue.poll()on aLinkedList<Integer>— actually fine, butArrayList.remove(0)is O(N)). - Python recursion limit in DFS on V > 1000 — silent
RecursionError. Set the limit explicitly. - DFS gray check missing. Without distinguishing gray (on-stack) from black (finished), you can’t detect back edges; you’d accept cyclic graphs with the wrong order.
- Java boxing penalty in
Queue<Integer>— ~2× slowdown vsint[]ring buffer. - Adjacency list as
Map<Integer, List<Integer>>when courses are 0..n-1 — wastes time on hashing; useList<List<Integer>>indexed by ID.
Debugging Strategy
- Trace
(4, [[1,0],[2,0],[3,1],[3,2]])Kahn’s: indeg =[0,1,1,2]. Queue:[0]. Pop 0, decrement indeg of 1 and 2: indeg =[0,0,0,2]. Push 1, 2. Pop 1, decrement indeg of 3:[0,0,0,1]. Pop 2, decrement:[0,0,0,0]. Push 3. Pop 3. Order =[0,1,2,3]. ✓ - Run the validator helper on every output during development.
- For cycle issues: build a small cycle by hand, ensure your code returns
[], not a partial order.
Mastery Criteria
- Recognized “valid order respecting prereqs” as topological sort within 30 seconds.
- Wrote Kahn’s correctly within 8 minutes, with cycle detection.
- Wrote DFS variant within 8 more minutes, with three-color cycle detection.
- Articulated both correctness arguments without prompting.
- Identified the language-specific recursion-limit / boxing trap.
- Generalized to LC 207 (Course Schedule, just yes/no), LC 269 (Alien Dictionary, infer edges), LC 1136 (Parallel Courses, layer-by-layer Kahn’s) within a week.
Lab 08 — Backtracking: Word Search II With Trie Pruning
Goal
Master the backtracking-with-pruning pattern in its highest-yield form: a grid DFS guided by a trie, with in-place visit-marking and post-recurse restoration. Deliverable solves LC 212 in O(M·N·4·3^(L-1) + W·L) time, where M·N is grid size, L is max word length, W is number of words. You can articulate why a trie cuts the brute O(W·M·N·4·3^(L-1)) by a factor of W, and why dead-branch pruning of the trie is the speed-of-light optimization.
Background Concepts
DFS on a 2D grid; backtracking with explicit make/undo of state; trie as a multi-pattern matcher; pruning via “if no children, abandon”; visit-marking via in-place mutation (board[r][c] = '#') vs an explicit visited set. Review pattern Backtracking and Trie.
Interview Context
Word Search II is an Amazon / Apple / Bloomberg favorite, and a top-3 hardest commonly-asked Mediums (often listed Hard). The interview signal is recognizing that running LC 79 (Word Search single-word) once per word is catastrophically slow for many words, and that a trie collapses the W independent searches into a single grid traversal. Strong candidates code the trie + DFS in 25 minutes. Elite candidates also implement trie pruning (deleting fully-found subtrees) to avoid revisiting.
Problem Statement
Given an M x N grid of characters and a list of words, return all words from the list that exist in the grid. A word can be constructed from letters of sequentially adjacent cells (horizontal/vertical), each cell used at most once per word.
Constraints
1 ≤ M, N ≤ 121 ≤ |words| ≤ 3 × 10^41 ≤ |word| ≤ 10board[i][j]andwords[i][j]are lowercase English letters.- All
wordsare distinct.
Clarifying Questions
- Each word individually uses each cell ≤ once, but can different words reuse the same cells? (Yes — independent searches per word.)
- Diagonal adjacency? (No — only 4-connected.)
- Are words guaranteed distinct? (Per constraints, yes.)
- Are duplicates in the result allowed? (No — return distinct words found.)
- Lower-case only? (Per constraints, yes — alphabet of size 26 simplifies trie nodes to fixed arrays.)
Examples
board = [["o","a","a","n"],
["e","t","a","e"],
["i","h","k","r"],
["i","f","l","v"]]
words = ["oath","pea","eat","rain"]
output = ["eat","oath"]
board = [["a","b"],["c","d"]]
words = ["abcb"]
output = [] (cells not adjacent or reused)
Initial Brute Force
For each word, run LC 79 (single-word search) on the grid.
def find_words_brute(board, words):
return [w for w in words if exists_in_grid(board, w)]
def exists_in_grid(board, word):
M, N = len(board), len(board[0])
def dfs(r, c, i):
if i == len(word): return True
if not (0 <= r < M and 0 <= c < N) or board[r][c] != word[i]: return False
ch, board[r][c] = board[r][c], '#'
ok = dfs(r+1,c,i+1) or dfs(r-1,c,i+1) or dfs(r,c+1,i+1) or dfs(r,c-1,i+1)
board[r][c] = ch
return ok
return any(dfs(r, c, 0) for r in range(M) for c in range(N))
Brute Force Complexity
Per word: O(M·N · 4·3^(L-1)) — for each starting cell, DFS explores at most 4 branches initially then 3 (one cell visited). Across W words: O(W · M · N · 4 · 3^(L-1)). With W = 3×10⁴, L = 10, M·N = 144: ~3×10⁴ × 144 × 4 × 3⁹ = enormous (~10^11). TLEs.
Optimization Path
Insight: all W single-word searches share grid traversal. If we have a trie of all words, a single DFS over the grid can simultaneously match all words, advancing through trie nodes as we step. At each grid cell (r, c), instead of asking “does this cell match word[i]”, we ask “does the current trie node have a child for board[r][c]”. If yes, descend. If the current trie node has the word field set, that word has been found — record and clear (to dedupe).
Pruning: after backtracking, if the trie node we just descended into has no children left (all its words have been found and we cleared them), prune it from the parent. This avoids revisiting empty subtrees on later starting cells.
Final Expected Approach
def find_words(board, words):
# 1) Build trie
trie = {}
for w in words:
node = trie
for c in w:
node = node.setdefault(c, {})
node['$'] = w # marker: word ends here
M, N = len(board), len(board[0])
found = []
def dfs(r, c, parent):
ch = board[r][c]
node = parent.get(ch)
if not node: return
if '$' in node:
found.append(node.pop('$')) # dedup: clear marker
board[r][c] = '#'
for dr, dc in ((1,0),(-1,0),(0,1),(0,-1)):
nr, nc = r+dr, c+dc
if 0 <= nr < M and 0 <= nc < N and board[nr][nc] != '#':
dfs(nr, nc, node)
board[r][c] = ch
if not node: # prune dead branch
parent.pop(ch)
for r in range(M):
for c in range(N):
dfs(r, c, trie)
return found
Data Structures Used
- Trie: nested dict (Python). In Java/C++, an explicit
TrieNodeclass withTrieNode[26]children array. - Grid: mutated in place (
'#'marker for visited). - Output list: distinct words found.
Correctness Argument
Trie invariant: the trie initially encodes all words; each leaf-marker '$' carries the word. The DFS descends one trie level per grid step; arriving at a node with '$' means we’ve matched the complete word from the start cell.
Backtracking correctness: in-place marking with '#' and explicit restoration in the post-recurse line guarantee that on entry to any DFS call, the grid reflects only ancestor cells as visited. The restore is symmetric to the mark; no leaks.
Dedup via pop('$'): clearing the marker on first find ensures each word is reported exactly once even if multiple grid paths spell it.
Pruning correctness: pruning a child after recursion only removes a subtree that has no remaining words to find (no '$' markers anywhere below). Future searches that would have entered this subtree gain nothing from doing so, so pruning is safe and accelerates the algorithm.
Complexity
Time O(M·N · 4·3^(L-1)) after trie pruning, in the best case (when found words deplete the trie quickly). Worst case (all words distinct, no early termination): O(M·N · 4·3^(L-1)) for the grid traversal + O(W·L) for trie build. Space: O(W·L) for the trie + O(L) for recursion.
The W factor is gone because all words share the traversal.
Implementation Requirements
- One DFS per starting cell, not W DFSes per starting cell. The trie unifies them.
- Restore the cell (
board[r][c] = ch) on every code path. The cleanest pattern: mark before the recursive calls, restore after — never restore inside conditionals. - Prune by removing the child entry when its subtree empties. This is a 5%-50% speedup depending on input.
- Dedup by
pop('$')on found, not byif word not in found: found.append(word)(the latter is O(W) per check). - For Python, use a nested dict; explicit
TrieNodeclasses are slower due to attribute lookup.
Tests
- Smoke: the LC example above.
- Unit: single-cell grid, single-letter word; word identical to one row.
- No matches: words with letters not in the grid →
[]. - All matches: every word findable.
- Diagonal trap: word that exists only along a diagonal — should NOT be found.
- Reuse trap:
board=[["a","a"]], words=["aaa"]→[](cell reuse forbidden). - Stress: M=12, N=12, W=3×10⁴, L=10, random; assert <1s in optimized languages, <5s in Python.
- Adversarial: words with long common prefix (e.g., 1000 variants of
"prefix....") — exercises the trie’s prefix-sharing benefit.
Follow-up Questions
- “What if words can use diagonal adjacency too?” → 8 directions; everything else identical.
- “What if the same cell can be reused?” → no marking needed; but then word length is unbounded by grid size, exponential blowup risk; need a cycle guard (e.g., max-length cap = some threshold).
- “What if you want all paths spelling each word, not just whether it exists?” → don’t dedup; collect on every match.
- “Memory blow-up for huge dictionaries?” → use a compressed trie (radix tree) or DAWG.
- “Distributed: shard grid across machines?” → grids small enough to not matter; for huge grids, partition with overlap of size L-1.
- “Online: words added/removed dynamically?” → trie supports insert/delete in O(L); the search needs no change.
Product Extension
This pattern underlies multi-pattern string matching in DLP (data-loss prevention) — scanning documents for any of a list of forbidden phrases — and in IDE autocomplete-on-context (which words from the dictionary can be formed by adjacent identifiers in scope?). It’s also used in spell-checkers that operate on keyboard layouts (find dictionary words spellable by adjacent keys). The Aho-Corasick algorithm is the streaming generalization (multi-pattern matching in O(text + total-pattern-length + matches)).
Language/Runtime Follow-ups
- Python: nested
dictis the fastest trie representation in Python; class-based is slower due to attribute lookup.dict.setdefault(c, {})is the canonical insert;dict.pop(key, None)is safe pop with default. - Java: explicit
TrieNode { TrieNode[] next = new TrieNode[26]; String word; }is fastest.HashMap<Character, TrieNode>is slower (autoboxing + hash). - Go: struct with
[26]*TrieNodearray. Map-based is slower. - C++: struct with
TrieNode* next[26] = {nullptr};andstring* word. Avoidstd::map<char, TrieNode>. - JS/TS: plain object as a hashmap is fine, but for hot loops, a
Mapor fixed-array index works. - Recursion depth: Python L = 10 fits the default 1000-frame stack. For deeper word lengths, set
sys.setrecursionlimit. - Mutation safety: in-place grid mutation with
'#'is fast but has thread-safety implications; if the function must be re-entrant, use an explicitvisitedset per call.
Common Bugs
- Forgetting to restore the cell. Causes false negatives later: cells stay marked
'#'permanently, blocking unrelated words. - Restoring inside
if/returnpaths. Always restore at the end, after all branches have explored. Easiest: structure asmark; for each direction: recurse; restore;. - Visited check on the neighbor, not on the current cell. You should refuse to step into a
'#'cell, but the current cell mark happens after entering it. - Adding the same word multiple times. Without
pop('$'), a word findable by 5 paths gets reported 5 times. - Walking the trie root-back-to-root for each starting cell in non-trie code — re-enumerating words you’ve already found.
- Boxing in Java’s
Map<Character, TrieNode>— silent ~3× slowdown vsTrieNode[26]. - Using
word in found(O(W)) instead ofpop('$')(O(1)) for dedup. - Pruning incorrectly: popping the trie node’s
'$'marker but leaving its emptydictin the parent — then visited subtrees are revisited as empty traversals. Always checkif not node: parent.pop(ch)after recursion.
Debugging Strategy
- Build a tiny trie by hand for
["oath", "oat"]— verify the structure with a print. - Trace one DFS from cell
(0,0)for the smoke example. The cellomatches root’s'o'child; descend; mark; try neighbors; etc. Verifyoathis found exactly once. - Test the prune with a single word: after finding
"oath", the trie should be empty. Subsequent starts at any cell return immediately. - Cross-check against the brute-force LC 79 per word for the smoke and stress tests on M=N=4, W=10.
Mastery Criteria
- Recognized “many words in a grid” as trie + DFS within 60 seconds.
- Built the trie in ≤ 8 lines with
setdefault. - Wrote the DFS with
mark; recurse; restoresymmetry on first attempt. - Implemented dedup via
pop('$')and pruning viaif not node: parent.pop(ch). - Articulated the W factor savings vs running LC 79 W times.
- Identified the language-specific trie-representation trade-off.
- Solved LC 79 (single-word) in <8 minutes within a week.
- Solved LC 1268 (Search Suggestions System — autocomplete) within two weeks.
Lab 09 — Heap For Top-K: Top K Frequent Elements (Heap Vs Bucket Sort)
Goal
Master the two canonical “top K” algorithms — min-heap of size K for streaming/general cases and bucket sort by frequency for bounded-frequency cases. Deliverable solves LC 347 with both, articulates the time-space trade-off, and recognizes which language-specific gotcha (Python heapq is min-only; Java PriorityQueue boxes; C++ defaults to max-heap) applies.
Background Concepts
Min-heap of size K as the canonical “running top K” structure: pop when size exceeds K, ensuring O(N log K). Bucket sort by frequency (not value) when frequencies are bounded by N. The duality “top K frequent” / “K largest” / “K smallest” via heap-direction inversion. QuickSelect as the O(N) average alternative. Review pattern Heap Top K and Heap Foundations.
Interview Context
Top-K problems are interview gold — they appear at every Big Tech, often as the warmup or the second problem. The interview signal is whether you can match the right structure to the input shape: heap when N is huge or streaming, bucket sort when frequencies are bounded (which they always are in this problem since the max frequency is N). Strong candidates write the heap solution. Elite candidates write the heap solution, then also mention the O(N) bucket-sort alternative and articulate when each is preferred.
Problem Statement
Given an integer array nums and an integer K, return the K most frequent elements. The answer can be returned in any order.
Constraints
1 ≤ |nums| ≤ 10^5-10^4 ≤ nums[i] ≤ 10^4Kis in the range[1, |distinct values in nums|].- The answer is unique (no ties at the K-th position that would create ambiguity).
Clarifying Questions
- Tie-breaking — what if two values share the K-th frequency rank? (Per constraints, the answer is unique. But if it weren’t, ask: arbitrary, or some specified rule like smallest value first?)
- Output order — sorted by frequency, by value, or any? (LC: any.)
- Are floats / strings possible, or strictly ints? (Per constraints, ints — but the algorithm extends trivially to any hashable type.)
- Streaming or offline? (Offline, but extending to streaming — windowed top-K — is a follow-up.)
- Memory-constrained? (No special constraint, but the algorithm should be O(N) space because the frequency map is unavoidable.)
Examples
| Input | Output |
|---|---|
nums=[1,1,1,2,2,3], K=2 | [1,2] |
nums=[1], K=1 | [1] |
nums=[4,1,-1,2,-1,2,3], K=2 | [-1,2] |
Initial Brute Force
Count frequencies; sort by frequency; take top K.
from collections import Counter
def top_k_brute(nums, K):
cnt = Counter(nums)
return [x for x, _ in sorted(cnt.items(), key=lambda kv: -kv[1])[:K]]
Brute Force Complexity
Time O(N + D log D) where D is the number of distinct values. Space O(D). At N=10⁵, D≤2×10⁴+1, this is fast — but it sorts more than needed (full O(D log D) when we only want top K).
Optimization Path A — Min-Heap of Size K
Build a frequency map. Walk the (value, freq) pairs maintaining a min-heap of size K keyed by freq. For each pair, push; if heap size > K, pop. The K survivors are the top K.
Why min-heap? We want to drop the smallest frequency when the heap overflows; min-heap.peek() gives the smallest in O(1).
Time: O(N) frequency count + O(D log K) heap. For D >> K, this is much faster than O(D log D).
Optimization Path B — Bucket Sort by Frequency
Frequencies are integers in [1, N]. Create buckets bucket[f] = list of values with freq == f. Walk f from N down to 1, collecting values into the result until we have K.
Time: O(N). Space: O(N). The cleanest O(N) solution.
Final Expected Approach (Heap)
import heapq
from collections import Counter
def top_k_frequent_heap(nums, K):
cnt = Counter(nums)
heap = [] # (freq, value), min-heap
for v, f in cnt.items():
heapq.heappush(heap, (f, v))
if len(heap) > K:
heapq.heappop(heap)
return [v for _, v in heap]
Final Expected Approach (Bucket Sort)
from collections import Counter
def top_k_frequent_bucket(nums, K):
cnt = Counter(nums)
buckets = [[] for _ in range(len(nums) + 1)]
for v, f in cnt.items():
buckets[f].append(v)
out = []
for f in range(len(nums), 0, -1):
for v in buckets[f]:
out.append(v)
if len(out) == K: return out
return out
Data Structures Used
- Heap approach:
Counter(hashmap) + min-heap of size K. - Bucket approach:
Counter+ a list-of-lists buckets indexed by frequency.
Correctness Argument (Heap)
Invariant: after processing the first i distinct values, the heap contains the top-min(K, i) most frequent among them. Adding the next value either grows the heap (if size < K) or replaces the min if the new freq exceeds it (push then pop-if-over-K does both correctly: push always grows; pop removes the smallest, which is the new value if it’s the smallest, leaving the heap unchanged).
Final step: after processing all D values, heap = top K. ✓
Correctness Argument (Bucket)
Frequencies range in [1, N], so buckets indexed [0..N] capture all. Walking f from high to low and collecting until K values found gives exactly the K most-frequent (with arbitrary tie-breaking within a bucket, acceptable per constraints).
Complexity
| Approach | Time | Space |
|---|---|---|
| Brute (full sort) | O(N + D log D) | O(D) |
| Heap | O(N + D log K) | O(N + K) |
| Bucket | O(N) | O(N) |
| QuickSelect | O(N) average, O(N²) worst | O(D) |
For D ≈ N and K ≪ D, heap and bucket are both linear-class; bucket’s hidden constant is smaller. For streaming, heap is the natural choice.
Implementation Requirements
- Use a min-heap, even though we want the largest K — popping the min when overflowing keeps the largest K in the heap.
- For Python’s
heapq, push tuples(freq, value)— the comparison is lexicographic, so freq dominates. - Java’s
PriorityQueueis a min-heap by default. No comparator inversion needed for “top-K largest” via the min-heap-of-size-K trick. Pushint[]{freq, value}withComparator.comparingInt(a -> a[0]). - C++’s
priority_queueis a max-heap by default. For min-heap:priority_queue<pair<int,int>, vector<pair<int,int>>, greater<>>. - For bucket sort, size buckets
N + 1(frequencies1..N, plus index 0 unused).
Tests
- Smoke:
([1,1,1,2,2,3], 2) → [1,2](in any order). - Unit:
K = 1→ most frequent only;K = D→ all distinct values. - All distinct values:
[1,2,3,4,5], K=3→ 3 of these (any 3, since frequencies tie at 1 — but per problem statement, the answer is unique, so this case wouldn’t be a test… unlessK = 5, returning all). - All same:
[7]*100, K=1 → [7]. - Negative values:
[-1,-1,1,1,2], K=2→[-1,1](tie at freq 2 — problem assumes unique answer; for this test, K = 1 →[-1]or[1]). - Large: N = 10⁵, K = 10, random values; both approaches sub-50ms.
- Adversarial: all-distinct → bucket sort touches all of
bucket[1]; heap pushes D items.
Follow-up Questions
- “Streaming top-K — elements arrive one at a time, query top K at any moment.” → maintain a min-heap of size K plus a hashmap; per element, look up old freq, decrement-and-rebuild. Or use Misra-Gries / Space-Saving sketches for approximate.
- “Top K from multiple sorted streams (LC 23 generalized)?” → merge-K-sorted via min-heap.
- “K closest points to origin (LC 973)?” → max-heap of size K keyed by distance, push and pop-if-over-K.
- “Kth largest element (LC 215)?” → min-heap of size K → root is K-th largest. Or QuickSelect for O(N) average.
- “Sliding window top-K?” → monotonic deque (LC 239 for K=1) or balanced BST (Phase 3).
- “Memory-constrained: top K of a billion items?” → heap-of-K only — O(K) memory, O(N log K) time.
Product Extension
Recommendation systems, trending topics, top-N queries on dashboards, “most frequent error code in last 5 minutes” log monitors — all are top-K problems. Real-world systems use approximate algorithms (Count-Min Sketch + heap) for cardinality at internet scale, but the exact algorithm is what an interview is testing. The Misra-Gries summary (heavy hitters) generalizes the heap to a streaming, memory-bounded, approximate version.
Language/Runtime Follow-ups
- Python:
heapqis min-only. To use it as max-heap, push-x(or(-freq, value)and decode at the end).Counter(nums).most_common(K)is a one-liner shortcut — mention it in the interview as the “Pythonic” answer, but be ready to write the algorithm by hand. - Java:
PriorityQueue<int[]>withComparator.comparingInt(a -> a[0])is the canonical pattern. Boxing tax if you usePriorityQueue<Integer>. Sorting viaArrays.sort(arr, comparator)works onInteger[]but notint[]. - Go:
container/heaprequires implementing 5 methods. Verbose. For Top-K problems, a simple sortsort.Slice(items, less)and slicing top K is often clearer. - C++:
priority_queue<int>is a max-heap by default. For min-heap:priority_queue<int, vector<int>, greater<int>>. Thetop()is O(1),pop()is O(log N) and returns void — you must calltop()first. - JS/TS: no built-in heap. Implement (~30 LOC) or use a library. For interview,
Array.sortwith a comparator and slicing is often acceptable for offline cases; mention you’d need a heap for streaming. - Heap tuple comparison: Python compares tuples lexicographically; if freq ties, comparison falls back to value, which can fail for non-comparable types. Wrap in a class with
__lt__defined onfreqonly, or pre-encode as(freq, hash(value)).
Common Bugs
- Using a max-heap of size D instead of a min-heap of size K — wastes time and space; O(D log D) instead of O(D log K).
- Java boxing in
PriorityQueue<Integer>— silent ~2-3× slowdown. - Python
heapqconfusion: forgetting it’s min-only; writingheappush(heap, -x)for max-heap and forgetting to negate when reading. Use a tuple with negated key cleanly:heappush(heap, (-freq, value)). - C++ default direction wrong:
priority_queue<int>is max-heap; using it for “min-heap of size K” gives wrong results. - Heap size check after push:
if len(heap) > K: popis correct. If you swap toif len(heap) >= K: popyou’ll never have K items. - Bucket-sort allocation cost:
[[] for _ in range(N + 1)]is O(N), but in C++ you canvector<vector<int>> buckets(N + 1);— same idea, just be aware of the per-bucket overhead. - Unstable comparison on ties in heap tuples — for
(freq, value), tied freqs compare values. If values are non-comparable (custom objects), this errors. Pre-encode or wrap.
Debugging Strategy
- Trace
([1,1,1,2,2,3], 2). Counter:{1:3, 2:2, 3:1}. Heap evolution: push (3,1) → heap[(3,1)], size 1 ≤ 2 ✓; push (2,2) →[(2,2),(3,1)], size 2 ≤ 2 ✓; push (1,3) →[(1,3),(3,1),(2,2)], size 3 > 2 → pop (1,3); heap =[(2,2),(3,1)]. Result: values[2, 1]. ✓ - Bucket trace: counter same; buckets
[[], [3], [2], [1], [], [], [], ...]. Walk f=6,5,4,3 → take 1; f=2 → take 2; size=2, return[1, 2]. ✓ - Cross-check the two approaches on 50 random inputs (compare as sets).
Mastery Criteria
- Recognized “Top K” pattern within 30 seconds.
- Wrote the min-heap-of-size-K solution within 6 minutes.
- Mentioned bucket sort as the O(N) alternative when frequencies are bounded.
- Identified the language-specific heap-direction trap (Python min-only, Java min-default, C++ max-default).
- Solved LC 215 (Kth Largest), LC 973 (K Closest), LC 23 (Merge K Sorted Lists) within a week — same pattern, different keys.
- Discussed QuickSelect as an O(N)-average alternative when asked.
Phase 3 — Advanced Data Structures
Target level: Medium → Hard Expected duration: 3 weeks (12-week track) / 3 weeks (6-month track) / 4 weeks (12-month track) Weekly cadence: ~8 advanced structures per week + 30–60 problems applying them under the framework
Why Advanced Data Structures Unlock Hards
Phase 2 gave you 28 patterns that solve the vast majority of Mediums. The patterns work because each one carries an O(N) or O(N log N) algorithm in its template — you recognize the signal, instantiate the template, and the runtime falls out for free.
Hards are different. The signal still fires — you still recognize “this is sliding window with a tricky max”, “this is DP with a state transition”, “this is shortest path with a constraint” — but the vanilla template’s complexity is one factor too high. A sliding-window max over a stream of N updates becomes O(N²) with a sorted list. A DP with N=20 and “set of visited” in the state explodes to O(2^N · N²) without bitmask compression. A range-sum problem with both updates and queries blows past prefix sums. A string match against a pattern of length M in a text of length N is O(N·M) with naive comparison; that’s 10^10 ops at N=10^5, M=10^5.
The advanced data structures in this phase are the augmented engines that bring Hard problems back into reach. Each one is a 1–2 log-factor improvement over a naive structure. They are not “tricks”. They are well-defined, well-proven structures with known invariants, well-understood failure modes, and known operating ranges. The skill is not to invent them — it is to recognize when the vanilla template is one log factor short, identify which augmented structure plugs the hole, and instantiate it correctly under interview pressure.
There are roughly three families:
- Range query / range update structures — segment tree, Fenwick tree, sparse table, sqrt decomposition. These turn O(N) per range query into O(log N) or O(√N), and (with augmentation) handle range updates the same way. They show up whenever the problem has a sequence and you need both updates and aggregates over arbitrary subranges in the same workload.
- String-matching / hashing / suffix structures — KMP, Z, Manacher, rolling hash, suffix array, suffix automaton, Aho-Corasick, tries. These bring per-character work down from O(M) (full pattern recompare) to O(1) amortized, enabling O(N+M) or O(N log N) algorithms over strings. They show up whenever the problem mentions “substring”, “match”, “occurrence”, “palindrome”, or “common”.
- State-compression and amortization — bitmask DP, meet-in-the-middle, DSU with α(N), bit manipulation idioms, Bloom/skip/LRU-LFU. These exploit problem-specific structural facts (small N, splittable input, near-constant amortized work, probabilistic acceptance) to clear constraints that naive DP/search cannot.
You will not memorize 24 implementations cold. You will understand the invariants well enough to derive each implementation in 5–15 minutes under pressure, and instantly recognize which one is needed from the problem signal.
After this phase you can solve unmistakably-Hard LeetCode problems on first attempt: range queries with updates, palindromic counts in linear time, multi-pattern matching, exact-cover by bitmask DP, subset-sum at N=40 by meet-in-the-middle, equation-solving by weighted DSU, dynamic LRU caches. You also become visibly stronger in mock interviews because you no longer flinch at “what if the input is updated?”, “what if N is 40?”, or “what if there are 10^5 patterns to match?”.
What You Will Be Able To Do After This Phase
- For any range-query Hard, identify within 60 seconds whether vanilla prefix sums suffice, or whether Fenwick / segment tree / sparse table / sqrt decomposition is required, and why.
- Implement a segment tree (point update, range query) from memory in <12 minutes.
- Add lazy propagation when the problem demands range updates, and articulate the lazy-tag-push invariant.
- Recognize a string-match Hard and pick the right tool: KMP (single pattern), Z (border + offsets), Manacher (palindromes), rolling hash (probabilistic equality, multi-substring), Aho-Corasick (multi-pattern), suffix array/automaton (overview-level for “longest common substring”, “distinct substrings”).
- Build a trie augmented with counts / deletion / prefix-sum cache for word-search and autocomplete-class problems.
- Recognize bitmask DP from
N ≤ 20constraint, formulate the state, and implement the transition without bugs. - Recognize meet-in-the-middle from
N ≤ 40(split into 20+20) and code the two-half merge. - Implement DSU with path compression + union by rank, and prove the α(N) amortized bound.
- Use bit manipulation idioms (popcount, lowbit, isolate trailing one, parity tricks) without thinking.
How To Read This Phase
Read the inline reference below in two passes. Pass 1: linear, end to end, to assemble a mental map of which structure plugs which hole. Pass 2: as you work through the labs, refer back to the structure entries to clarify invariants and pitfalls. Each entry has a fixed shape:
- When to use — the problem signal that should fire this structure within 2 minutes of reading.
- Complexity — build, query, update, space.
- Implementation pitfalls — the bugs that consume the most interview minutes.
- Classic problems — 3–6 representative problems where the structure is the intended solution.
Where labs cover the structure hands-on, the entry references the lab. Where the structure is overview-only (rare in interviews but expected of strong candidates), the entry says so explicitly.
Inline Advanced Data Structure Reference
1. Segment Tree (point update + range query)
When To Use
- Sequence of N elements, with both point updates and range aggregates (sum / min / max / gcd / xor) on the same workload.
- The aggregate is associative — i.e., it can be combined from two disjoint segments.
- Q queries + Q updates, with N · Q too large for naive O(N) per query and prefix sums (which are O(1) per query but O(N) per update) too rigid for the update mix.
Complexity
Build O(N). Point update O(log N). Range query O(log N). Space O(4N) — the tree array is conventionally sized 4N to fit any nearly-balanced binary tree on N leaves.
Key Implementation Pitfalls
- Off-by-one on the recursive boundaries —
query(node, nl, nr, ql, qr): total miss isqr < nl or ql > nr(closed intervals); total cover isql <= nl and nr <= qr. Mixing open and closed intervals is the #1 cause of broken segment trees. - Sizing the tree array —
4Nis safe;2 * next_power_of_two(N)is tight. If you pick2Nyou’ll segfault for non-power-of-two N. - Combining two subtree results — for sum, just add; for min/max, take the extreme; for gcd, recurse. Define a single
combine(a, b)so you can swap aggregates without rewriting the body. - Recursive vs iterative — iterative segment tree with
nrounded to power of 2, leaves at[n, 2n), is shorter and faster. Pick one style and stick to it.
Classic Problems
- LeetCode 307 — Range Sum Query Mutable
- LeetCode 308 — Range Sum Query 2D Mutable (with row segment trees)
- LeetCode 1157 — Online Majority Element In Subarray (segment tree of candidates + frequency check)
- LeetCode 715 — Range Module (segment tree of intervals, or coordinate-compressed)
Hands-on: see Lab 01.
2. Segment Tree With Lazy Propagation (range update + range query)
When To Use
- Same as #1 but updates affect ranges, not single points: “add v to all elements in [l, r]”, “set all elements in [l, r] to v”, “flip all elements in [l, r]”.
- The update operation has a clean composition rule: applying update u₁ then u₂ to a node is equivalent to a single combined update.
Complexity
All ops O(log N). Space O(4N) for the values + O(4N) for the lazy tags.
Key Implementation Pitfalls
- Pushdown order — before recursing into children, push the parent’s lazy tag down. After recursing, recompute the parent’s value from the children. Forgetting either side breaks the structure silently — queries return stale values.
- Composing lazy tags — for “set” and “add” mixed, “set” must override the pending “add”. Define a clear
apply(child_lazy, parent_lazy)rule and write it down before coding. - Tag identity / no-op — every lazy slot needs an “identity” value (e.g., 0 for add, sentinel for set) that means “no pending op”. Don’t conflate “identity” with a legal value.
- Range-set with empty intersection — apply only on full-cover nodes; recurse only on partial overlap.
Classic Problems
- LeetCode 732 — My Calendar III (count of overlapping intervals — coord-compressed lazy seg tree)
- LeetCode 2569 — Handling Sum Queries After Update (range flip + range sum, lazy + bitmask)
- Codeforces “EDU: Segment Tree” sections
Hands-on: see Lab 02.
3. Fenwick Tree / Binary Indexed Tree (prefix sums with updates)
When To Use
- Workload is point updates + prefix queries (or range queries via two prefix queries).
- The aggregate is invertible (sum, xor) — Fenwick can’t express min/max naturally because they’re not invertible.
- You want a smaller, faster, simpler structure than a segment tree, with smaller constants and easier code.
Complexity
Build O(N) (or O(N log N) trivially). Point update O(log N). Prefix query O(log N). Range query (for sum/xor) = query(r) - query(l - 1). Space O(N).
Key Implementation Pitfalls
- 1-indexed — Fenwick trees are conventionally 1-indexed. Calling
update(0, v)infinite-loops because0 & -0 == 0. Useupdate(i + 1, v)if your data is 0-indexed. i += i & -i(update) vsi -= i & -i(query) — the directions are not symmetric. Memorize: update goes up, query goes down.- Range-update + point-query is a different Fenwick variant — subtract on
r+1, add onl. Not the same code path. - Range-update + range-query needs two Fenwick trees (the BIT² trick). Out of interview scope unless the problem explicitly demands.
Classic Problems
- LeetCode 307 — Range Sum Query Mutable (canonical)
- LeetCode 315 — Count of Smaller Numbers After Self (canonical Fenwick on coord-compressed values)
- LeetCode 327 — Count of Range Sum
- LeetCode 493 — Reverse Pairs
Hands-on: see Lab 03.
4. 2D Fenwick Tree (matrix prefix sums with updates)
When To Use
- Matrix problems with both point updates and rectangle-sum queries.
- Static prefix-sum + occasional updates — but updates make recomputing the prefix-sum O(NM), so 2D Fenwick.
- Coordinate-compressed if the matrix is sparse (most cells empty).
Complexity
Update O(log N · log M). Rectangle query O(log N · log M). Space O(N · M).
Key Implementation Pitfalls
- Nested
i & -i— outer loop walks the row index, inner loop walks the column index. Independent. - Inclusion-exclusion for rectangle sum:
Q(r2, c2) - Q(r1-1, c2) - Q(r2, c1-1) + Q(r1-1, c1-1). Forgetting the+corner is the canonical bug. - Sparse matrices — if N · M = 10^10 you cannot allocate the array. Use a
dictofdictor coordinate compression along each axis.
Classic Problems
- LeetCode 308 — Range Sum Query 2D Mutable (canonical)
- LeetCode 327 — Count of Range Sum (often reduced to 2D via coords)
- LeetCode 1505 — Min Number of Swaps to Make Strings K-Equal (variants use 2D BIT)
5. Sparse Table (immutable RMQ)
When To Use
- The array is static (no updates) and you need idempotent range queries (min, max, gcd, “is there a 1 in this range”).
- O(1) query is required (segment tree’s O(log N) is too slow).
- Builds in O(N log N) — fine for one-time use.
Complexity
Build O(N log N). Query O(1) for idempotent ops, O(log N) for non-idempotent ops (sum). Space O(N log N).
Key Implementation Pitfalls
- Idempotent op required for O(1) — the trick is
query(l, r) = combine(table[k][l], table[k][r - 2^k + 1])wherek = floor(log2(r - l + 1)). The two halves overlap; this only works if combining the same element twice equals once. Sum is not idempotent (counts twice). Min, max, gcd, “or”, “and” are. floor(log2(len))precomputation — building alog_floor[]table of size N is required for true O(1). Computinglog2per query is too slow.- Memory — at N = 10^6, N log N ≈ 2 × 10^7 ints = 80 MB. Plan for it.
Classic Problems
- LeetCode 1851 — Minimum Interval to Include Each Query (one approach uses sparse table + offline sort)
- Codeforces RMQ classics
Hands-on: see Lab 04.
6. Sqrt Decomposition (block-based queries)
When To Use
- Workload mixes point/range updates and range queries, but the operation is hard to fit into segment tree (e.g., “sum of distinct values in a range”, “k-th smallest in a range with offline queries”).
- You want a much simpler implementation than a segment tree, accepting an O(√N) factor.
- Mo’s algorithm (offline range queries, total cost O((N+Q) · √N)) is a sqrt-decomp specialization.
Complexity
Build O(N). Query/update O(√N) per op. Space O(N).
Key Implementation Pitfalls
- Block size choice — block size
B = ⌈√N⌉minimizes total costQ · (N/B + B). Slightly larger B (e.g., 1.5√N) sometimes wins for cache reasons. - Edge of block — left partial block (l to end of block) and right partial block (start of block to r) handled as scalars; full middle blocks summed via block totals.
- Mo’s algorithm — sort queries by
(block of l, r)(with even/odd block hack to halve constants), then move pointers. Don’t confuse “block of l” sorting with sorting by l.
Classic Problems
- “Range Distinct Count” (offline) — Mo’s classic.
- LeetCode 850 — Rectangle Area II (sweep + sqrt-block coord compression)
- Codeforces “EDU: Sqrt Decomposition”
7. Persistent Segment Tree (overview)
When To Use
- You need to query prior versions of an array — “sum over [l, r] as of version v”, “k-th smallest in [l, r] (offline, treat as 2D)”.
- Classic application: “find k-th smallest in subarray” via merging persistent trees indexed by position.
- Rare in standard interviews; expected for Grandmaster / CP-style.
Complexity
Update O(log N) creating a new version (path-copy). Query on any version O(log N). Space O(N + Q log N) over Q versions.
Key Implementation Pitfalls
- Reference each version’s root — store an array
roots[v]; never mutate an old node. - Memory blow-up — N + Q log N at N = 10^5, Q = 10^5 ≈ 1.7M nodes × ~24 bytes = 40 MB. Pre-allocate node pool.
- Garbage collection — in GC’d languages, hold the root references to keep nodes alive; in C++, use a node pool + manual indices.
Classic Problems
- “K-th smallest in range” (offline-via-persistent-seg-tree).
- “Count distinct in range” (with persistent seg tree of last-occurrence positions).
This is overview-only in this phase; you should know it exists, what it solves, and the rough cost. Implementing it is a Phase 7 exercise.
8. Treap / Implicit Treap (overview)
When To Use
- Balanced BST with expected O(log N) operations via randomized priorities — simpler than red-black or AVL.
- Implicit treap keys by position in an array, supporting O(log N) array splice / split / merge / range reverse / range sum — classic for “rope” data structures.
- Order-statistics tree (find k-th, count less than) when sorted-container library doesn’t expose it.
Complexity
Insert / delete / split / merge / range-op all O(log N) expected.
Key Implementation Pitfalls
- Heap property on random priorities — re-bubble after insertion / deletion. Without correct rotations, you lose the log bound.
- Lazy reverse / lazy add on implicit treap mirrors lazy segment tree — push tag before recursing.
- Expected vs worst case — adversarial input can’t degrade because priorities are random; this is the entire point.
Classic Problems
- “Array splice” with O(log N) per op — implicit treap canonical.
- Order-statistics queries on an indexed multiset.
Overview only.
9. Splay Tree (overview, when used)
When To Use
- Self-adjusting BST — recently accessed nodes move toward the root.
- Useful when the access pattern has temporal locality (LRU-like): O(log N) amortized but O(1) for hot keys.
- Library-link: the data structure underlying many compiler symbol tables.
Complexity
All ops O(log N) amortized; individual ops up to O(N) worst case. The amortization argument uses a potential function.
Key Implementation Pitfalls
- Splay step — zig (one rotation), zig-zig (same-side double), zig-zag (opposite-side double). The choice depends on grandparent direction; getting it wrong destroys amortization.
- Splay on every access — including failed search. Forgetting this means the amortization breaks.
Classic Problems
- Rare in competitive interviews; common in systems-engineering follow-ups about LRU implementations.
Overview only.
10. KMP (Knuth–Morris–Pratt failure function)
When To Use
- Single pattern P matched against a text T.
- You need either all occurrences of P in T, or just first, in O(N + M).
- Or you need the longest border (longest proper prefix = suffix) of a string — that’s the failure function itself.
Complexity
Failure-function build O(M). Match O(N + M). Space O(M).
Key Implementation Pitfalls
- Failure function recursion —
fail[i]is the length of the longest proper prefix ofP[0..i]that’s also a suffix. The recurrence walksj = fail[i - 1]backward viaj = fail[j - 1]until matched. - 0 vs 1 indexed — pick one and stick with it. Most resources are 0-indexed; many CP templates are 1-indexed.
- Forgetting to reset
jto 0 between independent matches — the matcher state is per-text, not global.
Classic Problems
- LeetCode 28 — Find the Index of the First Occurrence in a String (canonical strstr)
- LeetCode 459 — Repeated Substring Pattern (one-shot via failure function)
- LeetCode 214 — Shortest Palindrome (failure function on
s + '#' + reverse(s)) - LeetCode 1392 — Longest Happy Prefix
Hands-on: see Lab 05.
11. Z Algorithm
When To Use
- Compute, for each position i in a string S, the longest prefix of S that starts at i (
z[i]). - Substring matching — concatenate
P + '#' + Tand look forz[i] = Min the T half. - Pattern problems where “longest prefix matching at offset” is the natural query.
Complexity
Build O(N) using a sliding [l, r] window of the rightmost reaching match. Match O(N + M).
Key Implementation Pitfalls
- Maintaining the
[l, r]Z-box — wheni ≤ r, copy fromz[i - l]capped atr - i + 1, then extend; otherwise start fresh from i. Off-by-one on the cap is the canonical bug. - Sentinel character — must not appear in either P or T. Use
\0or a fresh symbol; in Python, a tuple of(0, ord(c))for the sentinel and(1, ord(c))for real chars works. - Z and KMP overlap — they solve the same problems with different invariants. Picking one and being fluent is better than knowing both shallowly.
Classic Problems
- Same as KMP — LC 28, 214, 459, 1392.
- “Number of occurrences of P in T overlapping” — count
z[i] >= Min the T region.
12. Manacher’s Algorithm (longest palindrome in O(N))
When To Use
- “Longest palindromic substring” or “count of palindromic substrings”.
- Naive expand-around-center is O(N²); Manacher’s is O(N).
- The trick is mirroring around the rightmost-reaching palindrome center.
Complexity
O(N). Space O(N).
Key Implementation Pitfalls
- Even-length palindromes — Manacher’s classic trick is to insert
#between every pair of chars (and at ends):"abba"→"#a#b#b#a#". Now every palindrome (odd or even original) is odd-length in the transformed string. P[i]vs original-string radius —P[i]after the transform is the radius in the transformed string. The original palindrome has lengthP[i].- Maintaining the rightmost-reaching center C and right-boundary R —
P[i] = min(R - i, P[2C - i])ifi < R, else expand from scratch. Off-by-one onR - i(vsR - i + 1) is the canonical bug.
Classic Problems
- LeetCode 5 — Longest Palindromic Substring (canonical)
- LeetCode 647 — Palindromic Substrings (count)
- LeetCode 1960 — Maximum Product of Two Palindromic Substrings
13. Rolling Hash (Rabin–Karp + double hashing)
When To Use
- Compare a sliding-window substring against a pattern in O(1) per shift (after O(M) preprocessing).
- Find duplicate substrings of length L in O(N) instead of O(N²) — the canonical “longest duplicate substring” via binary search on L + hashing.
- Compare two substrings of S equal in O(1) — precompute prefix hashes.
Complexity
Preprocess O(N). Per-comparison O(1) hash plus O(M) verify (in adversarial settings; often skipped). For “longest duplicate substring”: O(N log N).
Key Implementation Pitfalls
- Hash collisions — single hash with mod ~10^9 has ~50% collision probability over 10^5 strings (birthday paradox). Always double-hash in interview answers, or single-hash + explicit verify on match.
- Modular arithmetic — base ~30 (alphabet) to ~10^9 prime; mod a large prime ~10^9. In Python, use
pow(base, k, mod)for the negative-power inverse trick. In Java/C++, uselongto avoid overflow onbase * value. - Anti-hash adversarial inputs — Codeforces has problems specifically constructed to break common base/mod choices. Use random base from
[26, mod-1]per run.
Classic Problems
- LeetCode 187 — Repeated DNA Sequences (canonical small-window hashing)
- LeetCode 1044 — Longest Duplicate Substring (binary search on length + rolling hash)
- LeetCode 28 — Find the Index of the First Occurrence (Rabin–Karp variant)
- LeetCode 1392 — Longest Happy Prefix
Hands-on: see Lab 06.
14. Suffix Array (overview, applications)
When To Use
- All suffixes of S sorted lexicographically — enables binary search for any pattern in O(M log N).
- “Longest common substring of two strings” via combined suffix array + LCP.
- “Number of distinct substrings of S” =
N(N+1)/2 − Σ LCP[i].
Complexity
Build O(N log² N) (radix-sort + double-the-rank trick) or O(N log N) (DC3 / SA-IS, harder). LCP via Kasai’s algorithm O(N). Per-pattern search O(M log N).
Key Implementation Pitfalls
- Doubling-the-rank — sort suffixes by their first 1, 2, 4, …, N characters using the previous round’s ranks. Each round is a radix sort.
- LCP array — Kasai’s algorithm: walk in original-index order, decrement an
hcounter that tracks current LCP. Subtle but linear. - Sentinel — append a unique smaller-than-all-others character to avoid prefix-of-another-suffix issues.
Classic Problems
- “Longest common substring of two strings” (SA + LCP + range minimum).
- “Number of distinct substrings”.
- “K-th lexicographically smallest substring”.
Overview only — labs don’t drill suffix arrays directly; rolling hash + Z/KMP cover most interview cases.
15. Suffix Automaton (overview, applications)
When To Use
- Smallest DFA accepting all substrings of S, built in O(N) — strictly stronger than a suffix tree for many queries.
- “Number of distinct substrings”, “longest common substring”, “count occurrences of a pattern” — all O(M) per query after O(N) build.
Complexity
Build O(N · σ) where σ is alphabet size. Space O(N · σ).
Key Implementation Pitfalls
link(suffix-link) pointers — the equivalence-class tree of states. Subtle to derive; templates exist.cntaugmentation — counting occurrences requires DP on the suffix-link tree.- Online construction — add char at a time, maintain the latest state.
Classic Problems
- “Distinct substrings count”.
- “Longest common substring of K strings”.
- Codeforces / SPOJ string problems.
Overview only. Rare in interviews; expected at Grandmaster.
16. Trie Variants (compressed, with counts, with deletion)
When To Use
- Prefix queries: “does any inserted word start with prefix P?”, “all words with prefix P”.
- Multi-word search (LC 212 — Word Search II): compile dictionary into a trie, DFS the grid against the trie for O((R · C) · 4^L) instead of per-word DFS.
- Autocomplete with frequency ranking — trie augmented with word counts and best-K-at-each-node.
Complexity
Insert O(L). Search prefix O(L). Space O(Σ L · σ) for plain array-based trie; O(Σ L · branching) for hash-based.
Key Implementation Pitfalls
- Array-of-26 vs hash-of-char — array-of-26 is faster (no hashing); hash is more memory-efficient on sparse tries. Pick one based on problem constraints.
- End-of-word marker — use a separate
is_endboolean, not a sentinel char that could collide with input. - Compressed (radix) trie — chains of single-child nodes are merged into a single edge labeled with the substring. Saves memory at the cost of more complex insertion (split an edge mid-substring).
- Deletion — typically just clear
is_endand prune empty subtrees. Don’t free shared nodes. - Counts at each node — increment on insert, decrement on delete; useful for “count words with prefix P” in O(L).
Classic Problems
- LeetCode 208 — Implement Trie (canonical)
- LeetCode 211 — Design Add and Search Words (with
.wildcard — DFS) - LeetCode 212 — Word Search II (canonical trie-on-grid)
- LeetCode 421 — Maximum XOR of Two Numbers in an Array (binary trie)
- LeetCode 642 — Design Search Autocomplete System
Hands-on: see Lab 07.
17. Aho–Corasick (multi-pattern matching)
When To Use
- Match a set of K patterns simultaneously against a text T — “find all dictionary words that occur in T”.
- Naive: O(N · ΣM) — too slow for K = 10^4 patterns.
- Aho–Corasick: O(N + ΣM + #matches) — linear in everything.
Complexity
Build O(ΣM · σ). Match O(N + #matches). Space O(ΣM · σ).
Key Implementation Pitfalls
- Failure links — analog of KMP’s failure function on a trie. Built via BFS over the trie.
- Output (dict-suffix) links — for each node, follow failure links to collect every matching pattern that ends at this position. Without dict-suffix links, you’d miss patterns that are suffixes of other patterns.
- Node pool size — total nodes ≤ ΣM + 1. Pre-allocate.
Classic Problems
- LeetCode 1032 — Stream of Characters (canonical reverse-trie + Aho-Corasick)
- “Find all dictionary words occurring in document”.
18. Bloom Filter (probabilistic membership)
When To Use
- “Is X in the set?” with a tolerated false-positive rate, zero false negatives.
- Memory tight (you can’t store the full set), or you want a fast pre-filter before a slow exact check (e.g., disk lookup).
- Streaming dedup with a fixed false-positive budget.
Complexity
Insert / query O(K) where K is the number of hash functions. Space O(M) bits where M is chosen for the target false-positive rate (1 - e^(-KN/M))^K.
Key Implementation Pitfalls
- K (#hash functions) and M (#bits) — given target FPR
pand capacityn, optimum isM = -n ln(p) / ln(2)²,K = (M/n) ln 2. - No deletion — standard Bloom can’t delete (can’t tell which other element shares the bit). Counting Bloom does, with K counters.
- False positives compound with set size — a 1% Bloom filter at capacity is 1% per query, not 1% over a workload. Rebuild when growing.
Classic Problems
- System-design follow-ups (“how do you check whether a URL has been crawled in the last 30 days?”).
- LeetCode-adjacent: not common as a graded problem, but expected in design interviews.
19. Skip List (overview)
When To Use
- Randomized alternative to balanced BST — O(log N) expected, simpler to implement than AVL/RB.
- Used in practice in Redis sorted sets, LevelDB MemTable.
- Order-statistics, range queries on a sorted set, with concurrent-modification flexibility.
Complexity
Insert / delete / search O(log N) expected. Space O(N) expected (geometric level distribution).
Key Implementation Pitfalls
- Level distribution — each new node’s level is geometric with p = 1/2 (or 1/4 in production). Cap at log N.
- Update array on insert/delete — track the predecessor at each level; splice carefully.
- Concurrent skip list — much simpler than concurrent BST; standard library impls in Java (
ConcurrentSkipListMap).
Classic Problems
- LeetCode 1206 — Design Skiplist (canonical implementation problem)
- System-design discussions of Redis ZSET / LevelDB.
Overview only; the implementation problem (LC 1206) is good practice but rare.
20. LRU / LFU Implementation Deep Dive
When To Use
- Cache eviction problems with O(1) get/put requirement.
- LRU: hash map + doubly-linked list. Touched node moves to head; evict tail.
- LFU: hash map of (key → node) + hash map of (freq → doubly-linked list). On hit, move node to next-freq list; on evict, drop tail of min-freq list.
Complexity
LRU: O(1) get and put. Space O(capacity).
LFU: O(1) get and put. Space O(capacity). Maintaining min_freq is the subtle bookkeeping bit.
Key Implementation Pitfalls
- LRU: doubly-linked list with sentinel head/tail — eliminates null checks. Always add at head, evict from tail.
- LRU: hash map points to nodes, not keys — so you can splice the node in O(1) without searching.
- LFU:
min_freqinvariant — increment when freq-list at min_freq becomes empty only if the touched node was the cause. - LFU: list-per-frequency — implement as a doubly-linked list of nodes; ordering within a freq is LRU.
Classic Problems
- LeetCode 146 — LRU Cache (canonical)
- LeetCode 460 — LFU Cache
- LeetCode 432 — All O(1) Data Structure (frequency buckets)
Both are bread-and-butter for systems-engineering interviews.
21. Disjoint Set Union (DSU) with Path Compression and Union by Rank — Proof of α(N) Amortization
When To Use
- Online connectivity (#1 trigger).
- Kruskal’s MST.
- Equation problems (weighted DSU — LC 399 Evaluate Division).
- Offline divide-and-conquer queries with rollback (advanced).
Complexity
Each op amortized inverse-Ackermann α(N) — for all practical N (up to 2^65536), α(N) ≤ 4. Effectively constant.
Proof Sketch (Tarjan)
- Without compression or rank: worst-case chain → O(N) per op.
- Path compression alone: each find shortens the path. Amortized O(log N) per op.
- Union by rank (or size) alone: depth bounded by O(log N). Per-op O(log N) worst case.
- Both together: per-op amortized O(α(N)). Tarjan’s potential function counts “blocks” of nodes by rank and shows the total cost over M ops is O(M · α(N)). The proof uses Ackermann’s hierarchy
A(k, n)and α(N) is its inverse. - For an interview: state “with both heuristics, amortized O(α(N)), where α grows so slowly it’s ≤ 4 for any N you’ll see in practice; you treat it as O(1)”. Cite Tarjan 1975.
Key Implementation Pitfalls
- Recursive
findblows the stack at N = 10^5 in Python. Use iterative two-pass: walk up to root, then walk again compressing. - Path-halving variant (
parent[x] = parent[parent[x]]per step) — simpler, asymptotically equivalent, often faster than full compression in practice. - Union by rank vs union by size — both work. Rank is the height upper bound of the tree (compression doesn’t decrease rank); size is the count of nodes in the tree. Pick one.
- Forgetting to update rank when ranks are equal — break the tie and increment the survivor’s rank.
Classic Problems
- See pattern 18 in Phase 2 README.
- LeetCode 200 — Number of Islands (DSU alternative)
- LeetCode 305 — Number of Islands II (online DSU canonical)
- LeetCode 547 — Number of Provinces
- LeetCode 684 — Redundant Connection
- LeetCode 721 — Accounts Merge
- LeetCode 952 — Largest Component Size by Common Factor
- LeetCode 399 — Evaluate Division (weighted DSU)
Phase 2 covered DSU mechanically. Phase 3’s contribution is the proof of α(N) and the weighted / rollback variants.
22. Bit Manipulation Idioms (popcount, lowbit, isolate trailing one, parity)
When To Use
- Bitmask DP (pattern 23) requires fluency in these primitives.
- Subset enumeration, parity tricks, fast set operations.
- Hot-loop optimization where each int represents a tiny set (≤ 64 elements).
Idioms
- Popcount:
__builtin_popcount(x)(C/C++/Java viaInteger.bitCount),bin(x).count('1')(Python — slow),x.bit_count()(Python 3.10+, fast). - Lowbit / lowest set bit:
x & -xgives the value of the lowest set bit.x & (x - 1)clears it. - Isolate trailing ones:
x & ~(x + 1). Set trailing zero:x | (x + 1). - Iterate subsets of mask:
s = mask; while s: ... ; s = (s - 1) & maskenumerates each subset ofmaskexactly once. - Iterate set bits:
while x: lb = x & -x; ... ; x ^= lb. Each step does O(1) work, total O(popcount). - Parity:
bin(x).count('1') & 1or — faster — XOR-fold:x ^= x >> 16; x ^= x >> 8; x ^= x >> 4; x ^= x >> 2; x ^= x >> 1; return x & 1. - Power of two test:
x > 0 and (x & (x - 1)) == 0. - Swap without temp:
a ^= b; b ^= a; a ^= b— academic; never use in production.
Classic Problems
- LeetCode 191 — Number of 1 Bits (popcount).
- LeetCode 338 — Counting Bits (DP using
dp[x] = dp[x >> 1] + (x & 1)). - LeetCode 461 — Hamming Distance (popcount of XOR).
- LeetCode 78 — Subsets via bitmask iteration.
Mastery here is a prerequisite for Pattern 23.
23. Bitmask DP Foundation
When To Use
- N ≤ ~20 and the problem asks for an optimum over subsets or assignments of N items.
- Examples: traveling salesman (TSP), assignment problem, “shortest path visiting all nodes”, “minimum cost to cover all groups”.
- The state is a bitmask of “which items are used / visited / completed”.
Canonical Forms
- Permutation DP:
dp[mask][i] = min over j in mask\{i}: dp[mask \ {i}][j] + cost(j, i). Result:min over i: dp[full_mask][i](or back-to-start for TSP cycle). - Subset cover DP:
dp[mask] = min over partition of mask into subset s and rest: cost(s) + dp[mask \ s]. - Assignment DP:
dp[mask] = min cost to assign people 0..popcount(mask)-1 to the jobs in mask.
Complexity
Permutation DP: O(2^N · N²) time, O(2^N · N) space. N=20 → 4 × 10^8 — borderline. Subset DP: O(3^N) time (enumerating subset of subset). N=15 → 14M — comfortable.
Key Implementation Pitfalls
- Subset-of-subset enumeration uses
s = mask; while s: ... ; s = (s - 1) & mask. Themaskinvariant is critical. - Initial mask — for permutation DP, initialize
dp[1 << i][i]for the first city; iterate masks in increasing order so dependencies are resolved. - Reconstruction — to recover the order, store predecessor
(mask, i) → (prev_mask, prev_i)and walk back. - N too large — N > 20 is not bitmask DP territory. Reach for meet-in-the-middle (#24) or heuristics.
Classic Problems
- LeetCode 847 — Shortest Path Visiting All Nodes (canonical bitmask + BFS)
- LeetCode 1125 — Smallest Sufficient Team (subset cover DP)
- LeetCode 943 — Find the Shortest Superstring (TSP-like, DP over permutations)
- LeetCode 698 — Partition to K Equal Sum Subsets (subset assignment)
- LeetCode 1494 — Parallel Courses II
- LeetCode 1879 — Minimum XOR Sum of Two Arrays (assignment DP)
Hands-on: see Lab 08.
24. Meet-in-the-Middle (split, sort, two-pointer)
When To Use
- N ≤ 40 (or up to 50), the problem asks for “subset with property X”, and
2^Nis too large but2^(N/2)is fine. - Examples: “subset sum closest to T at N=40”, “count subsets with XOR equal to K”, “split items into two groups minimizing difference”.
Canonical Template
left, right = a[:n // 2], a[n // 2:]
sums_left = sorted(sum(combo) for combo in subsets(left)) # 2^(N/2)
sums_right = sorted(sum(combo) for combo in subsets(right)) # 2^(N/2)
# for each L in sums_left, binary-search the closest R such that L + R ≈ T.
Complexity
Time O(2^(N/2) · N/2) for enumeration + O(2^(N/2) · log(2^(N/2))) = O(N · 2^(N/2)) for sort, then O(2^(N/2) · log) for the merge. At N=40 → ~10^6 ops. Space O(2^(N/2)) for the two subset-sum lists.
Key Implementation Pitfalls
- Enumerate subsets correctly —
for mask in range(1 << k): sum = sum of bits set in maskvia popcount-iteration. Or recursive include/exclude. - Two-pointer or binary search — once both halves are sorted, sweep with two pointers (one from each end) to minimize / count target.
- Memory — at N=40, half-mask space is 2^20 = 1M entries × 8 bytes = 8 MB. Comfortable, but watch out at N=44.
- Counting (not just existence) — careful binary-search for
lo, hibounds; usebisect_leftandbisect_right.
Classic Problems
- LeetCode 1755 — Closest Subsequence Sum (canonical N=40)
- LeetCode 956 — Tallest Billboard (subset DP alternative; meet-in-the-middle viable)
- LeetCode 805 — Split Array With Same Average (meet-in-the-middle)
- “Subset sum at N=40” — competitive-programming staple.
Hands-on: see Lab 09.
Recognition Cheat Sheet
| Problem Signal | Structure |
|---|---|
| Range query + point update, sum/min/max | Segment tree (#1) or Fenwick (#3, if invertible) |
| Range query + range update | Lazy segment tree (#2) |
| Static range min/max with O(1) queries | Sparse table (#5) |
| Range distinct count, hard-to-segment-tree aggregate | Sqrt decomposition / Mo’s (#6) |
| Single pattern in text | KMP (#10) or Z (#11) |
| Longest palindrome / count palindromes | Manacher (#12) |
| Many-substring equality / longest duplicate | Rolling hash (#13) |
| Multi-pattern dictionary in text | Aho–Corasick (#17) |
| Prefix queries, autocomplete, word-on-grid | Trie variants (#16) |
| Probabilistic membership | Bloom filter (#18) |
| Cache with O(1) get/put | LRU / LFU (#20) |
| Connectivity / equation graphs | DSU (#21) |
| N ≤ 20, subset / assignment optimum | Bitmask DP (#23) |
| N ≤ 40, subset existence / closest sum | Meet-in-the-middle (#24) |
| Bit-level state mechanics | Bit idioms (#22) |
Mastery Checklist
You have completed Phase 3 when you can, on demand and from memory:
- Implement a segment tree (point update, range sum/min/max) in <12 minutes, with no off-by-ones, on the first attempt.
- Add lazy propagation for range-add + range-sum in <20 minutes, articulating the push-down invariant.
- Implement a Fenwick tree (1-indexed, prefix-sum + point update) in <8 minutes.
- State why Fenwick can’t do range-min naturally and which segment-tree augmentation handles it.
-
Build a sparse table for static RMQ in <10 minutes, including the
log_floor[]precompute. - Choose between segment tree, Fenwick, sparse table, and sqrt decomposition based on the workload (read-only vs mixed; aggregate type) in <30 seconds.
- Compute KMP’s failure function on a string of length 20 by hand, no errors.
- Implement KMP match in <12 minutes.
- Implement Manacher’s longest palindrome in <20 minutes (this one is hard; that’s expected).
- Implement double-hashing rolling hash in <15 minutes; explain why single hash is insufficient.
- Implement a trie (insert, search, startsWith) in <8 minutes.
- Implement Aho–Corasick at the conceptual level (failure + dict-suffix links) and state its complexity.
- State the Bloom filter formula: target FPR p, capacity n → M = -n ln(p) / ln(2)², K = (M/n) ln 2.
- Implement LRU cache (146) in <10 minutes; LFU (460) in <25 minutes.
- Implement DSU with path compression + union by rank in <8 minutes; state the α(N) bound and cite Tarjan.
-
Use
x & -x,x & (x - 1), subset-of-mask enumeration without thinking. - Recognize bitmask-DP from N ≤ 20 and write the transition in <10 minutes for an unfamiliar problem.
- Recognize meet-in-the-middle from N = 40 and write both halves + merge in <20 minutes.
If any of these takes >2× the budget, drill it again — that structure is your weakest link. Hards rarely fail because all your structures are weak; they fail because one of them is, and that’s the one this Hard happened to need.
Exit Criteria
You may proceed to Phase 4 — Graph Mastery only when:
- All 9 labs are complete, with the deliverable code written, tested, and reviewed via the REVIEW_TEMPLATE.
- Mastery checklist is fully ticked.
- 30+ Hard problems solved across the structures above (10 segment-tree-class, 5 string-algo, 5 trie/AC, 5 DSU/bitmask, 5 free choice).
- Mock interview at Phase 3 level: you receive a Hard segment-tree problem, a Hard string problem, and a Medium-Hard bitmask DP problem in a 90-minute window. Solve at least 2 of the 3 cleanly.
- No structure is “the one I always get wrong” — drill it until it isn’t.
If any of these fails, do not proceed. Phase 4 builds on the assumption that DSU, segment trees, and bitmask are reflexes. If they are not, Phase 4’s harder graph problems will compound the gap.
Labs
| # | Lab | Structure | Canonical Problem |
|---|---|---|---|
| 01 | Segment tree (range query) | Point update + range sum/min/max | LC 307 |
| 02 | Segment tree with lazy propagation | Range update + range query | Range-add + range-sum |
| 03 | Fenwick tree (BIT) | Coord-compressed Fenwick | LC 315 |
| 04 | Sparse table for RMQ | Static O(1) RMQ | Range-min array |
| 05 | KMP string matching | Failure function + match | LC 28 / 459 |
| 06 | Rolling hash | Double hashing | LC 187 / 1044 |
| 07 | Trie applications | Trie with is_end + DFS-on-trie | LC 208 / 212 |
| 08 | Bitmask DP | Permutation DP over subsets | LC 847 |
| 09 | Meet-in-the-middle | Split-sort-merge | LC 1755 |
| 10 | Z-function & Manacher | Linear-time prefix-match + palindrome radii | LC 5 / 214 |
| 11 | LCA via binary lifting | Up-table ancestor jumps + path aggregates | LC 1483 / 236 |
| 12 | Sqrt decomposition & Mo’s algorithm | Block decomposition + offline query ordering | SPOJ DQUERY |
| 13 | Treap & ordered set | Split/merge BST with random priorities | LC 315 |
Common Failures At This Phase
These are the failure modes that consume the most candidate time at Phase-3 level. Tag them when they occur using FAILURE_ANALYSIS.md.
- Segment tree off-by-ones — closed-vs-open intervals mixed mid-recursion. Fix: always closed
[l, r], never mix with[l, r). - Fenwick tree 0-index trap —
update(0, …)infinite-loops. Fix: shift to 1-index at the boundary. - KMP failure function off-by-one —
j = fail[j - 1]vsj = fail[j]. Fix: derive on a 5-char example. - Rolling hash single-mod collisions — pass random unit tests, fail adversarial. Fix: double-hash always.
- Bitmask DP transition direction —
dp[mask]fromdp[mask & ~bit](forward) vsdp[mask | bit] from dp[mask](backward). Both work; mixing them mid-implementation breaks. Fix: pick one before coding. - DSU recursive find stack overflow at N=10^5 in Python. Fix: iterative two-pass.
- Lazy segment tree forgetting to push before recursing into a child. Fix: write
push_down(node)as the first line of any non-leaf recursion.
Cross-References
- FRAMEWORK.md — apply on every Hard.
- CODE_QUALITY.md — Hards do not get graded leniency; clean code still required.
- COMMUNICATION.md — out loud at the recognition step, the structure name and complexity must be explicit. “I’ll use a segment tree with lazy propagation; build O(N), query and update O(log N), space O(4N).”
- SPACED_REPETITION.md — segment tree and KMP should be on a 7-day cycle for the first month after this phase. Bitmask and meet-in-the-middle on 14-day.
- Phase 4 — Graphs — DSU shows up immediately; review #21 the day before starting Phase 4.
- Phase 5 — DP — bitmask DP is the bridge. Without #23 fluency, Phase 5’s “DP on graphs / DAG / interval” labs will hurt.
- Phase 7 — Competitive — persistent seg tree, suffix array/automaton, splay/treap deepen here.
Lab 01 — Segment Tree (Point Update + Range Query)
Goal
Implement a segment tree from scratch that supports point updates and range-sum / range-min / range-max queries on an array of N integers. Build in O(N), query and update in O(log N) each. Internalize the recursion structure so you can re-derive any aggregate variant on the fly. After this lab you should be able to write a working segment tree from blank slate in under 12 minutes with zero off-by-ones.
Background Concepts
A segment tree represents an array as a near-balanced binary tree where each internal node stores the aggregate (sum / min / max / gcd / …) of a contiguous range. Leaves correspond to individual array elements; internal nodes correspond to the union of their children’s ranges. The tree has depth O(log N), so any range [l, r] decomposes into at most 2 · log₂(N) disjoint subtree-ranges. That is the entire complexity argument: each query and update walks O(log N) nodes.
The tree is conventionally stored in a flat array of size 4N (worst-case nearly-balanced binary tree on N leaves) with the root at index 1, left child at 2 · i, right child at 2 · i + 1. This avoids pointer overhead and is cache-friendly.
The aggregate must be associative so that subtree results can be combined. Sum, min, max, gcd, xor, “and”, “or”, and matrix multiplication all qualify. Median, mode, and “k-th smallest” do not combine cleanly and need different structures.
Interview Context
Range queries with updates appear in 3–5% of FAANG-tier Hard pools, but they appear more often on the bar-raiser round. Recognizing that prefix sums (O(1) query, O(N) update) won’t survive the workload — that you need O(log N) for both — is the reflex this lab builds. Companies that screen with segment trees: Meta (frequent), Google (occasional), Amazon (rare), Stripe / HFT shops (very frequent — order book, sliding aggregates). Bombing this is a no-hire signal at L5+.
Problem Statement
Implement a class NumArray initialized with an integer array nums. Support:
update(i, val): setnums[i] = val.sumRange(left, right): return the sum ofnums[left..right]inclusive.
Both must be O(log N). After implementing the sum variant, refactor so swapping combine = + for combine = min or combine = max requires changing one line.
Constraints
- 1 ≤ N ≤ 3 × 10⁴
- −100 ≤ nums[i] ≤ 100
- 0 ≤ i, left, right < N
- Up to 3 × 10⁴ calls to
updateandsumRangecombined.
Clarifying Questions
- Are queries inclusive on both ends? (Yes —
[left, right].) - Is
numsmutable in place, or owned byNumArray? (Owned; copy on construction.) - Are the values guaranteed to fit in int32? (Yes; sum across N elements at value ±100 fits comfortably.)
- Is
updatean assignment or delta? (Assignment — set, not add.)
Examples
NumArray([1, 3, 5])
sumRange(0, 2) → 9
update(1, 2) // array becomes [1, 2, 5]
sumRange(0, 2) → 8
update(0, 10) // array becomes [10, 2, 5]
sumRange(1, 2) → 7
Initial Brute Force
Store nums as a plain list. update(i, v): nums[i] = v (O(1)). sumRange(l, r): sum(nums[l:r+1]) (O(N)). Updates are fast; queries are linear.
Brute Force Complexity
Update O(1). Query O(N). Total over Q queries + U updates: O(N · Q + U). At N = Q = 3 × 10⁴: 9 × 10⁸ ops — TLE.
Optimization Path
Two natural alternatives.
Prefix sums. prefix[i] = nums[0] + ... + nums[i-1]. Query is prefix[r+1] - prefix[l] in O(1). But update(i, v) requires recomputing prefix[i+1..N] in O(N). Wrong tradeoff for this workload.
Sqrt decomposition. Block size √N; per-block sums. Update O(1), query O(√N). Total O(N · √N) = O(N^1.5) — at N = 3 × 10⁴, ~5 × 10⁶ ops. Passes but is suboptimal and crusty.
Segment tree. Build O(N), update O(log N), query O(log N). Total O((N + Q) log N) = ~5 × 10⁵ ops. Clean fit.
Final Expected Approach
Recursive segment tree on a flat array of size 4N.
- Build
build(node, nl, nr): ifnl == nr, leaf =arr[nl]; else recurse left and right, settree[node] = tree[2node] + tree[2node + 1]. - Update
update(node, nl, nr, idx, val): recurse into the child whose range containsidx; on return, recompute parent. - Query
query(node, nl, nr, ql, qr): total miss → identity (0 for sum, +∞ for min); total cover → returntree[node]; partial → recurse both children and combine.
Public API wraps with node = 1, nl = 0, nr = N - 1.
Data Structures Used
- A single integer array
tree[]of size4N(sum aggregates). - Optional integer
nstoring the original length.
Correctness Argument
Build establishes the invariant tree[node] = combine over [nl, nr] by induction on subtree size. Update: along the recursion, only nodes whose range contains idx are touched; each is recomputed from its (now-correct) children, so the invariant is preserved. Query decomposes [ql, qr] into a disjoint union of subtree ranges; the result is the combine of those. The decomposition has size ≤ 2 log N because along any root-to-leaf path the recursion either stops (full cover or miss) or splits at most twice (once for the left boundary, once for the right). The total work is O(log N).
Complexity
| Operation | Time | Space |
|---|---|---|
| Build | O(N) | O(4N) |
| Update | O(log N) | O(log N) recursion |
| Query | O(log N) | O(log N) recursion |
Implementation Requirements
class NumArray:
def __init__(self, nums):
self.n = len(nums)
self.tree = [0] * (4 * self.n)
self._build(1, 0, self.n - 1, nums)
def _build(self, node, nl, nr, a):
if nl == nr:
self.tree[node] = a[nl]; return
mid = (nl + nr) // 2
self._build(2*node, nl, mid, a)
self._build(2*node + 1, mid + 1, nr, a)
self.tree[node] = self.tree[2*node] + self.tree[2*node + 1]
def update(self, i, val):
self._update(1, 0, self.n - 1, i, val)
def _update(self, node, nl, nr, idx, val):
if nl == nr:
self.tree[node] = val; return
mid = (nl + nr) // 2
if idx <= mid: self._update(2*node, nl, mid, idx, val)
else: self._update(2*node + 1, mid + 1, nr, idx, val)
self.tree[node] = self.tree[2*node] + self.tree[2*node + 1]
def sumRange(self, l, r):
return self._query(1, 0, self.n - 1, l, r)
def _query(self, node, nl, nr, ql, qr):
if qr < nl or ql > nr: return 0 # miss
if ql <= nl and nr <= qr: return self.tree[node] # cover
mid = (nl + nr) // 2
return self._query(2*node, nl, mid, ql, qr) + self._query(2*node + 1, mid + 1, nr, ql, qr)
Refactor to support min by changing the identity (+∞), the leaf assignment (still a[nl]), and the combine (min).
Tests
- N=1:
update(0, 5); sumRange(0, 0) == 5. - All zeros: every range query returns 0.
- All same value:
sumRange(l, r) == val * (r - l + 1). - After
update(i, v):sumRange(i, i) == v; sum across full range matches direct sum of array. - Random fuzz: 1000 ops alternating updates and queries against a brute-force list.
- Min variant: build
[3, 1, 4, 1, 5, 9, 2, 6],query(2, 5) == 1; afterupdate(3, 10),query(2, 5) == 4.
Follow-up Questions
- “Now I want range updates.” → Lab 02 (lazy propagation).
- “Now I want O(1) queries on a static array.” → sparse table (Lab 04).
- “Now the array is 2D.” → segment tree of segment trees, O(log² N) per op.
- “Make it iterative.” → power-of-two-padded leaves at indices
[N, 2N); update walksi // 2upward, query walksl, rtoward the middle. - “How would you support
count of values ≥ K in [l, r]?” → merge sort tree (segment tree with sorted lists) or wavelet tree.
Product Extension
Real-time analytics dashboards: a stream of N metrics with both edits and arbitrary-range aggregates (e.g., “total revenue from days 17–24” while a correction is being applied to day 19). The naive list is fine until the workload has both fast updates and fast arbitrary-range queries on the same data — then a segment tree over the time axis, keyed by index, is what powers the underlying store.
Language/Runtime Follow-ups
- Python: recursion at N=3×10⁴ is fine but the per-op constant is high. For larger N convert to iterative or use
sys.setrecursionlimit. Considerarray.array('i', ...)over plainlistfor cache locality. - Java: use
int[] tree = new int[4 * n]; the4nallocation is critical because computingnext_power_of_two(n) * 2is a fencepost-error magnet. Method dispatch has a real cost — inline the recursion if hot. - Go: no recursion-limit issue; keep the slice. Use
int(sized to platform) unless the problem demandsint64. - C++: the canonical implementation.
vector<long long> tree(4 * n). Inline the body; mark methodsinline. For competitive problems use the iterative version (template by Adrian Panaete or Codeforces “EDU”). - JS/TS: typed arrays —
new Int32Array(4 * n)— outperform plain arrays. Recursion depth at N=3×10⁴ is fine in V8.
Common Bugs
- Mixing closed-interval
[ql, qr]with half-open[ql, qr)between query and recursion. Always pick closed and stay consistent. - Sizing
treeas2Ninstead of4N: works for power-of-two N, segfaults otherwise. - Forgetting to recompute
tree[node]after recursing inupdate. The leaf updates correctly but the parent stays stale. - Identity wrong for the aggregate: 0 for sum, but +∞ (
float('inf')/Long.MAX_VALUE) for min and −∞ for max. Returning 0 from a min query missing-range gives wrong answers silently. - Building from
nums[mid + 1]vsnums[mid]— pick one slicing convention. - Iterative version: forgetting to round N up to a power of two before placing leaves.
Debugging Strategy
When tests fail, drop into a tiny instance (N=4, indices 0..3) and print(tree) after each op. Verify by hand: tree[1] should equal sum over [0,3], tree[2] over [0,1], tree[3] over [2,3]. If those don’t hold post-build, your build recursion is broken — fix that before touching update or query. Add assert for the cover/miss/partial branches printing (node, nl, nr, ql, qr) to spot which sub-call returns the wrong total.
Mastery Criteria
- Recognized the segment-tree signal in <60 seconds from a “range query + point update” problem statement.
- Wrote build/update/query on a blank screen in <12 minutes with no off-by-ones, first try.
- Refactored sum → min → max in <2 minutes by changing identity + combine only.
- Stated complexity (build O(N), update/query O(log N), space O(4N)) without prompting.
- Solved LC 307 in <15 minutes from cold start.
- Solved one cousin problem (LC 308 or LC 1157) in <30 minutes from cold start.
Lab 02 — Segment Tree With Lazy Propagation
Goal
Extend Lab 01’s segment tree to support range updates in O(log N) using lazy propagation. Implement range-add + range-sum and articulate the push-down invariant so cleanly that you can re-derive lazy on a different aggregate (range-set + range-sum, range-flip + range-count) under interview pressure.
Background Concepts
Lazy propagation defers work. When an update covers a whole subtree, instead of recursing into all O(2^depth) descendants, you stamp a single lazy tag on that subtree’s root and update its aggregate in O(1). The descendants stay stale until something forces a deeper visit; at that point you push_down the tag — apply it to the children’s aggregates and merge into their lazy slots — and clear the parent’s tag.
This works whenever:
- The update operation has an O(1) batch form: applying “add v to all of [nl, nr]” to
tree[node]istree[node] += v * (nr - nl + 1). - The lazy tags compose: a pending “add 3” followed by a new “add 5” composes to “add 8”. Without composition, you cannot stack tags; you must push first.
- There is a identity lazy value (e.g., 0 for add) meaning “nothing pending”.
For mixed update types (“add” and “set” both), composition needs an explicit rule: a new “set” wipes any pending “add”; a new “add” composes with a pending “set” by changing the set value.
Interview Context
Asked at: companies with high-frequency-trading or analytics flavor (Stripe, Two Sigma, Jane Street), and Meta in bar-raiser slots. Most interview problems that need this dress up as “support add v to a range and report sum of a range” or as a count-of-overlapping-intervals problem like LC 732. Failing to know this structure caps you at Mediums; recognizing it and implementing it correctly is a green-light at L5+.
Problem Statement
Implement a class RangeArray over n integers (initially zero) supporting:
add(l, r, v): addvto every index in[l, r].sumRange(l, r): return the sum ofarr[l..r].
Both O(log n).
Constraints
- 1 ≤ n ≤ 10⁵
- 1 ≤ Q ≤ 10⁵ ops total.
- −10⁴ ≤ v ≤ 10⁴.
- Sums fit in 64-bit (max |sum| ≈ 10⁵ · 10⁵ · 10⁴ = 10¹⁴).
Clarifying Questions
- Endpoints inclusive? (Yes.)
- Is
addcumulative or assignment? (Cumulative — additive.) - Should
arrbe mutable in place at the leaves? (Conceptually yes; in practice the segment tree owns it.) - 0-indexed or 1-indexed externally? (0-indexed.)
Examples
RangeArray(5)
add(0, 2, 3) // [3, 3, 3, 0, 0]
sumRange(0, 4) → 9
add(1, 3, 2) // [3, 5, 5, 2, 0]
sumRange(2, 4) → 7
sumRange(0, 0) → 3
Initial Brute Force
Plain list. add(l, r, v) → for i in range(l, r+1): arr[i] += v (O(N)). sumRange → sum(arr[l:r+1]) (O(N)). Combined per-op O(N).
Brute Force Complexity
O(N) per op. Total O(N · Q) = 10¹⁰ at the limits. TLE by 4 orders of magnitude.
Optimization Path
Difference array? diff[l] += v; diff[r+1] -= v is O(1) per add, but you can only query the final prefix sum after all updates — not interleaved with sum queries. Doesn’t survive mixed workload.
Two Fenwick trees (BIT-RU + BIT-PQ)? Yes, this works for range-add + range-sum specifically — the BIT² trick. Slightly faster constants than segment tree, but only handles invertible aggregates. Segment tree generalizes to range-set, range-min-after-add, range-affine, etc.
Lazy segment tree is the canonical answer.
Final Expected Approach
Augment Lab 01’s tree with a parallel lazy[] array of size 4N, all initialized to 0 (the identity for add).
push_down(node, nl, nr): iflazy[node] != 0, apply it to both children’s aggregates (tree[child] += lazy[node] * child_len) and compose it intolazy[child] += lazy[node]. Then clearlazy[node] = 0. Called at the start of any non-leaf update or query that recurses into children.update(node, nl, nr, ql, qr, v):- If
qr < nl or ql > nr: return (miss). - If
ql <= nl and nr <= qr: stamptree[node] += v * (nr - nl + 1); lazy[node] += v; return. - Else:
push_down; recurse both children;tree[node] = tree[left] + tree[right].
- If
query(node, nl, nr, ql, qr): identical structure, withpush_downbefore recursing.
Data Structures Used
tree[]— sum aggregates, size4N,int64.lazy[]— pending add tags, size4N,int64.
Correctness Argument
Invariant: for every node, tree[node] equals the correct aggregate over its range as if all pending lazy tags up to and including this node have been applied. Specifically, tree[node] is correct; tree[child] may be stale by exactly lazy[node] * child_len.
push_down repairs the children: it adds the missing contribution to their aggregates and composes the tag into theirs (so their own descendants will, later, be repaired similarly). It then clears lazy[node]. After push_down, tree[node] is unchanged and the children are now correct, so descendants of children may be stale only by the children’s own pending tags.
update either (a) misses, doing nothing, (b) totally covers, applying the O(1) batch update directly to tree[node] and stamping the tag, or (c) partially overlaps, which requires push_down before recursing so the children are correct, then recomputes tree[node] from now-current children.
query symmetric.
Complexity
| Operation | Time | Space |
|---|---|---|
| Build | O(N) | O(4N) tree + O(4N) lazy |
| Range update | O(log N) | O(log N) recursion |
| Range query | O(log N) | O(log N) recursion |
Implementation Requirements
class RangeArray:
def __init__(self, n):
self.n = n
self.tree = [0] * (4 * n)
self.lazy = [0] * (4 * n)
def _push_down(self, node, nl, nr):
if self.lazy[node]:
mid = (nl + nr) // 2
left, right = 2*node, 2*node + 1
self.tree[left] += self.lazy[node] * (mid - nl + 1)
self.lazy[left] += self.lazy[node]
self.tree[right] += self.lazy[node] * (nr - mid)
self.lazy[right] += self.lazy[node]
self.lazy[node] = 0
def add(self, l, r, v):
self._add(1, 0, self.n - 1, l, r, v)
def _add(self, node, nl, nr, ql, qr, v):
if qr < nl or ql > nr: return
if ql <= nl and nr <= qr:
self.tree[node] += v * (nr - nl + 1)
self.lazy[node] += v
return
self._push_down(node, nl, nr)
mid = (nl + nr) // 2
self._add(2*node, nl, mid, ql, qr, v)
self._add(2*node + 1, mid + 1, nr, ql, qr, v)
self.tree[node] = self.tree[2*node] + self.tree[2*node + 1]
def sumRange(self, l, r):
return self._sum(1, 0, self.n - 1, l, r)
def _sum(self, node, nl, nr, ql, qr):
if qr < nl or ql > nr: return 0
if ql <= nl and nr <= qr: return self.tree[node]
self._push_down(node, nl, nr)
mid = (nl + nr) // 2
return self._sum(2*node, nl, mid, ql, qr) + self._sum(2*node + 1, mid + 1, nr, ql, qr)
Tests
- N=1, single index:
add(0, 0, 5);sumRange(0, 0) == 5. - All-zero: any sumRange before any add returns 0.
- Disjoint adds:
add(0, 2, 1),add(3, 5, 2);sumRange(0, 5) == 3 + 6 = 9. - Overlapping adds:
add(0, 4, 1),add(2, 4, 1);sumRange(2, 4) == 2 + 2 + 2 = 6. - Stress: 10⁴ random adds + queries against a brute-force list.
- Stack of pending tags:
add(0, n-1, 1)100 times;sumRange(i, i) == 100for all i.
Follow-up Questions
- “Now
addbecomesset(assignment, not delta).” → identity = sentinelNone; on push_down, iflazy_parent != None, replace child’streeand replace child’slazy. - “Both
addandsetoperations.” → two lazy slots. Composition: a new “set” wipes pending “add”; a new “add” applied while a “set” is pending modifies the set value. - “Range-flip on a binary array, with range-count-of-ones.” →
tree[node] = count of 1s; flip →tree[node] = (nr-nl+1) - tree[node]; lazy is a boolean toggle. - “Range-affine: replace
a[i]withb · a[i] + c.” → lazy holds(b, c); composition:(b₂, c₂) ∘ (b₁, c₁) = (b₂ b₁, b₂ c₁ + c₂).
Product Extension
A live spreadsheet with array formulas — =ARRAYFORMULA(A1:A1000 + 5) — is exactly range-add. With range-set you get fill-down. With range-affine you get scaling formulas. The backend has to support thousands of these per second per spreadsheet; lazy segment trees are one viable engine.
Language/Runtime Follow-ups
- Python: 4 × 10⁵ allocations are slow; warm the lists once, never resize. Recursion at N=10⁵ depth ≈ 17, fine.
- Java:
long[] tree, lazyto avoid sum overflow at the limits. Synchronization-free for single-thread;LongAdderis unrelated. - Go: same template;
[]int64slices. - C++: the canonical use case.
vector<long long>. Compile with-O2; benchmark on N=10⁶. - JS/TS:
BigInt64Arrayis heavy; if values fit in Number’s 53-bit safe range, useFloat64Arraydespite being float (the IEEE 754 representation is exact for ±2⁵³ integers).
Common Bugs
- Forgetting
push_downbefore recursing on partial cover. The leaf updates correctly but its sibling subtree returns stale aggregates on later queries. Manifests as queries that depend on update order. - Push down on full-cover branch — wasted work but not wrong; only push on partial overlap.
- Identity confusion: 0 is identity for add but a legal value for set. Use
Noneor a sentinel for set-style lazy. - Composition direction: when stamping a new tag onto a parent that already has a tag, write the rule down before coding. For add it’s commutative; for set it isn’t.
intoverflow in Java/C++ — sums of up to 10⁵ · 10⁴ values ≈ 10⁹, doubles to 10¹⁴ with adds. Use 64-bit.- Calling
push_downon a leaf — guardif nl != nr.
Debugging Strategy
Add an assert_consistent() helper that walks the tree and verifies, for every internal node, tree[node] == tree[left] + tree[right] + lazy[node] * (nr - nl + 1). Wait — that’s not quite the invariant, since lazy[node] has not been pushed yet but tree[node] already includes it. The correct invariant is tree[node] == tree[left] + tree[right] + (lazy[node] * (nr - nl + 1)) only if you treat children’s tree as “before this lazy stamp”. An easier debug helper: after each op, force a full push_down from root to leaves and rebuild aggregates; compare against the brute-force array. If they diverge, you have a push-down-order bug.
Mastery Criteria
- Stated the push-down invariant in one sentence.
- Wrote range-add + range-sum lazy seg tree from scratch in <20 minutes, first try.
- Adapted the same template to range-set + range-sum in <10 additional minutes.
- Solved LC 732 (My Calendar III) using a coord-compressed lazy seg tree.
- Stated when not to use lazy (single-point updates → no benefit; non-composing operations → impossible).
-
Pinpointed the canonical bug (missing
push_down) within 5 minutes of seeing a failing test.
Lab 03 — Fenwick Tree (Binary Indexed Tree)
Goal
Implement a Fenwick tree (BIT) and use it to solve LeetCode 315 — Count of Smaller Numbers After Self. Internalize the bit-tricks (i & -i) and the 1-indexed convention so well that you can write a Fenwick tree in under 8 minutes from a blank page.
Background Concepts
A Fenwick tree is a clever encoding of prefix sums in O(N) space supporting prefix_sum(i) and point_update(i, delta) each in O(log N). The key insight: index i in 1-indexed form is associated with a “responsibility range” of size i & -i (the lowest set bit of i). Index 12 = 1100₂ has responsibility for the 4 values at positions 9..12. Index 8 = 1000₂ for 1..8. Walking up the tree (i += i & -i) accumulates non-overlapping responsibility ranges that span exactly [1, i] for query, and exactly the buckets that contain i for update.
The structure is invertible-only: it stores prefix sums and you derive range_sum(l, r) = prefix(r) - prefix(l - 1). This is fine for sum, xor, count, and “frequency-prefix” aggregates. It does not generalize to min/max because subtraction doesn’t undo a min.
For LC 315, the trick is coordinate compression + Fenwick of frequencies. Process the array right-to-left; for each nums[i], query “how many values strictly less than nums[i] have I seen so far?” by computing prefix(rank(nums[i]) - 1) on the frequency Fenwick; then increment update(rank(nums[i]), 1).
Interview Context
Fenwick trees are asked roughly as often as segment trees but the audience skews more competitive-programming. Stripe, Jane Street, Two Sigma, Bloomberg quant — all reach for them. The signal is “count inversions / count-of-X-after-Y / range-sum-with-updates and the aggregate is invertible”. Faster constants than segment tree, ~5x fewer lines of code; if both work, prefer Fenwick.
Problem Statement
Given an integer array nums, return an array counts where counts[i] is the number of elements to the right of nums[i] that are strictly smaller than nums[i].
Constraints
- 1 ≤ N ≤ 10⁵
- −10⁴ ≤ nums[i] ≤ 10⁴
Clarifying Questions
- Strictly smaller, or ≤? (Strictly smaller.)
- Return order: same as input order? (Yes —
counts[i]aligns withnums[i].) - Are duplicates allowed? (Yes — they don’t count toward “smaller”.)
Examples
nums = [5, 2, 6, 1] → counts = [2, 1, 1, 0]
5: indices 1,3 (vals 2,1) are smaller → 2
2: index 3 (val 1) is smaller → 1
6: index 3 (val 1) is smaller → 1
1: nothing to the right is smaller → 0
Initial Brute Force
For each i, scan j > i and count nums[j] < nums[i]. O(N²).
Brute Force Complexity
O(N²) time. At N=10⁵: 10¹⁰ ops. TLE by 4 orders of magnitude.
Optimization Path
Merge sort with inversion counting. During the merge step, when copying from the right half, every remaining element on the left half is strictly larger and hasn’t yet been placed — for each, increment its inversion count. O(N log N) time and space. Works, idiomatic.
Fenwick of frequencies after coordinate compression. Equally O(N log N), simpler to extend (e.g., to “count of values in [a, b] after self”).
For this lab, Fenwick is the assigned approach because it generalizes farther.
Final Expected Approach
- Coordinate compression: build
sorted_unique = sorted(set(nums)); map each valuevtorank = bisect_left(sorted_unique, v) + 1(1-indexed for Fenwick). - Right-to-left sweep: for each
ifromn-1down to0:r = rank[nums[i]]counts[i] = bit.prefix(r - 1)— count of strictly smaller previously-seen.bit.update(r, 1).
- Return
counts.
Data Structures Used
BIT(size)— Fenwick tree of size = number of distinct values.rankmap — value → 1-indexed compressed rank.counts[]— output array.
Correctness Argument
After processing index i, the BIT contains exactly the multiset of ranks for nums[i+1..n-1] (the elements to the right of i, since we go right-to-left). bit.prefix(r - 1) returns the count of those whose rank is < r — i.e., strictly smaller than nums[i]. Coordinate compression preserves order, so “rank smaller” iff “value smaller”. The update bit.update(r, 1) then registers nums[i] for the next iteration. By induction the invariant “BIT == multiset of ranks of strictly-right-of-current” is preserved.
Complexity
| Operation | Time | Space |
|---|---|---|
| Coordinate compression | O(N log N) | O(N) |
| Right-to-left sweep | O(N log N) | O(N) Fenwick |
| Total | O(N log N) | O(N) |
Implementation Requirements
class BIT:
def __init__(self, n):
self.n = n
self.tree = [0] * (n + 1) # 1-indexed
def update(self, i, delta):
while i <= self.n:
self.tree[i] += delta
i += i & -i
def prefix(self, i):
s = 0
while i > 0:
s += self.tree[i]
i -= i & -i
return s
from bisect import bisect_left
def countSmaller(nums):
sorted_unique = sorted(set(nums))
rank = {v: i + 1 for i, v in enumerate(sorted_unique)}
bit = BIT(len(sorted_unique))
counts = [0] * len(nums)
for i in range(len(nums) - 1, -1, -1):
r = rank[nums[i]]
counts[i] = bit.prefix(r - 1)
bit.update(r, 1)
return counts
Tests
[5, 2, 6, 1] → [2, 1, 1, 0].[1, 2, 3, 4] → [0, 0, 0, 0](already sorted).[4, 3, 2, 1] → [3, 2, 1, 0](reverse sorted — every pair is an inversion).- All same:
[5, 5, 5] → [0, 0, 0]. - Negatives:
[-1, -1, 0, -2] → [1, 1, 1, 0]. - Single element:
[7] → [0]. - Stress: 10⁴ random arrays of size 1000 against the O(N²) brute force.
Follow-up Questions
- “Now count strictly larger after self.” → mirror: prefix from
rank+1ton=bit.prefix(n) - bit.prefix(rank). - “Count of values in
[a, b]after self.” →bit.prefix(rank[b]) - bit.prefix(rank[a] - 1). - “Reverse pairs (LC 493)”:
nums[i] > 2 * nums[j]fori < j. Adapt the rank/query: compute “count of v’s in BIT withv < nums[i] / 2” — careful with integer division. - “Sum of values smaller after self instead of count.” → BIT stores value sums, not counts;
update(rank, nums[i])instead ofupdate(rank, 1). - “Now updates are interleaved with queries on the original problem.” → Fenwick tree of frequencies still works because both ops are O(log N).
Product Extension
A leaderboard service that streams game scores and reports “your rank percentile” as scores arrive. Fenwick of frequencies indexed by score bucket; as a new score arrives, query prefix to know how many scored less, divide by total. Works at millions-of-events-per-second with log-bucket cost per event.
Language/Runtime Follow-ups
- Python: integer ops are arbitrary-precision but slow; the BIT loop is hot. PyPy or C-extension if N=10⁶.
array.array('q')overlistonly marginally helps. - Java:
int[]for the tree at this N.Math.floorModnot needed (values are positive ranks). Watch forlongif you store sums. - Go: idiomatic —
tree []int. No surprises. - C++: canonical CP template.
vector<int> bit(n + 1, 0). Thei & -ilowbit relies on two’s complement, which all modern compilers guarantee for signedint. - JS/TS:
Int32Array(n + 1)outperforms regular arrays. The bitwise&and unary-on numbers cast through 32-bit signed int, which works for N ≤ 2³¹.
Common Bugs
- 0-indexing the BIT. Calling
update(0, …)isi & -i = 0, the loop never advances, or it loops forever (depends on language). Always 1-index. - Update walks down, query walks up — got the directions reversed. Mnemonic: update goes up the responsibility tree (so future prefix walks see it); query walks down (collecting predecessor ranges).
- Forgetting to add 1 when going from
bisect_leftrank to BIT index. - Compressing
numsbut using the original value when querying. treearray sizennotn + 1for 1-indexed.- Using Fenwick for min/max — it doesn’t work because subtraction is not the inverse of min.
Debugging Strategy
For a length-8 input, print tree[1..8] after each update. Recall: tree[i] stores the sum over [i - lowbit(i) + 1, i]. So tree[8] should equal the sum over [1, 8], tree[12] over [9, 12], etc. Verify by hand for a 3-update sequence.
If the inversion-count is off by 1 at every position, you almost certainly forgot the +1 in rank shifting (1-indexed vs 0-indexed). If it’s off by a lot, your update is going down instead of up, or your prefix is going up instead of down.
Mastery Criteria
-
Wrote
updateandprefixwithi & -icorrectly on first try. - Used 1-indexed throughout without bugs.
- Solved LC 315 in <15 minutes from blank slate.
- Solved one cousin (LC 327, LC 493) in <30 minutes.
- Articulated why Fenwick can’t do range-min.
- Stated when Fenwick beats segment tree (smaller code, smaller constants — pick Fenwick when the aggregate is invertible).
-
Estimated memory at N=10⁶ (~4 MB for
int) without prompting.
Lab 04 — Sparse Table for Range Minimum Queries
Goal
Implement a sparse table supporting O(1) range-min queries on a static array, after O(N log N) preprocessing. Internalize the “two overlapping intervals of the largest power-of-two length” trick. After this lab you should be able to write a sparse table from blank in under 10 minutes and instantly choose between sparse table and segment tree based on whether updates are required.
Background Concepts
A sparse table is a preprocessing structure for range queries on immutable arrays where the aggregate is idempotent — combining the same element twice gives the same result. Min, max, gcd, bitwise OR, bitwise AND, and “is-there-a-1-in-this-range” are idempotent. Sum is not (counts twice).
Construction. st[k][i] = min of arr[i .. i + 2^k - 1]. Build by:
st[0][i] = arr[i]for alli.st[k][i] = min(st[k-1][i], st[k-1][i + 2^(k-1)])— the range of length 2^k splits cleanly into two halves of length 2^(k-1).
Query. Given [l, r], let k = floor(log2(r - l + 1)). Then min(st[k][l], st[k][r - 2^k + 1]). The two intervals each have length 2^k, they cover [l, l + 2^k - 1] and [r - 2^k + 1, r], and their union is exactly [l, r] because l + 2^k - 1 ≥ r - 2^k + 1 whenever 2^k ≥ (r - l + 1) / 2, which holds by choice of k. They overlap, but for an idempotent op that’s harmless.
For O(1) per query you also need a precomputed log_floor[len] table — calling math.log2 each query has too much overhead and floating-point trouble.
Interview Context
Sparse tables show up in problems with a read-only array and many range-min/max queries. The signal: “static array, Q queries with Q ≫ N”. Common cousin: range-LCA via Euler tour + sparse table over depth array — Phase 4 territory but rooted here. Asked at: Google occasionally, CP-flavored shops always. Rejecting an O(log N) segment tree in favor of a sparse table when O(1) queries matter (e.g., 10⁷ queries on a 10⁵ array) is a senior-level signal.
Problem Statement
Given a static integer array arr of length N, build a structure that answers query(l, r) = min of arr[l..r] in O(1) per query.
Constraints
- 1 ≤ N ≤ 10⁵
- 1 ≤ Q ≤ 10⁷ queries
- 0 ≤ l ≤ r < N
- −10⁹ ≤ arr[i] ≤ 10⁹
Clarifying Questions
- Is the array static? (Yes — that’s the entire premise.)
- Inclusive endpoints? (Yes.)
- Min, or min and max? (Just min for this lab; max is identical with
min→max.)
Examples
arr = [3, 1, 4, 1, 5, 9, 2, 6]
query(0, 7) → 1
query(2, 5) → 1
query(4, 7) → 2
query(3, 3) → 1
Initial Brute Force
min(arr[l:r+1]) per query — O(N) per call. Total O(N · Q).
Brute Force Complexity
At N=10⁵, Q=10⁷: 10¹² ops. TLE by 6 orders of magnitude.
Optimization Path
Segment tree gives O(log N) per query, O(N) per build. Total O(Q log N) = ~2 × 10⁸ at the limits — borderline TLE in Python, fine in C++.
Sparse table gives O(1) per query, O(N log N) per build. Total O(N log N + Q) = ~2 × 10⁶ + 10⁷ = 1.2 × 10⁷ ops. Comfortable everywhere.
The deciding factor: updates. Sparse table is read-only. If the array mutates between queries, sparse table is wrong; segment tree is required. The interviewer asking “what if I want updates?” is a real follow-up — answer: “Switch to a segment tree; sparse table doesn’t support point updates without an O(N log N) full rebuild.”
Final Expected Approach
- Precompute
log_floor[1..N]vialog_floor[i] = log_floor[i // 2] + 1, base caselog_floor[1] = 0. - Allocate
stas a 2D array of size(K + 1) × NwhereK = log_floor[N]. st[0][i] = arr[i]for all i.- For
k = 1 .. K: fori = 0 .. N - 2^k:st[k][i] = min(st[k-1][i], st[k-1][i + 2^(k-1)]). query(l, r):k = log_floor[r - l + 1]; returnmin(st[k][l], st[k][r - 2^k + 1]).
Data Structures Used
- 2D array
st[K+1][N], whereK = floor(log2(N)). - 1D array
log_floor[N+1].
Correctness Argument
By induction on k: st[0][i] = arr[i] (length-1 range, trivially correct). Given st[k-1][·] correct: st[k][i] = min(st[k-1][i], st[k-1][i + 2^(k-1)]) covers [i, i + 2^(k-1) - 1] ∪ [i + 2^(k-1), i + 2^k - 1] = [i, i + 2^k - 1]. Min commutes over union.
For query, k = floor(log2(r - l + 1)) ⇒ 2^k ≤ len ≤ 2^(k+1) - 1 ⇒ 2^k ≥ len/2 ⇒ l + 2^k > r - 2^k, so the two intervals [l, l + 2^k - 1] and [r - 2^k + 1, r] overlap (or meet exactly), and their union is [l, r]. Min over the union equals min of the two.
Complexity
| Operation | Time | Space |
|---|---|---|
| Build | O(N log N) | O(N log N) |
| Query | O(1) | — |
At N=10⁵: K ≈ 17, total table cells ≈ 1.7 × 10⁶. At 8 bytes each, ~14 MB.
Implementation Requirements
class SparseTableMin:
def __init__(self, arr):
n = len(arr)
self.log = [0] * (n + 1)
for i in range(2, n + 1):
self.log[i] = self.log[i // 2] + 1
K = self.log[n]
self.st = [list(arr)] + [[0] * n for _ in range(K)]
for k in range(1, K + 1):
half = 1 << (k - 1)
for i in range(n - (1 << k) + 1):
self.st[k][i] = min(self.st[k-1][i], self.st[k-1][i + half])
def query(self, l, r):
k = self.log[r - l + 1]
return min(self.st[k][l], self.st[k][r - (1 << k) + 1])
Tests
- N=1:
query(0, 0) == arr[0]. - All same:
query(l, r) == arr[0]for all valid (l, r). - Sorted ascending:
query(l, r) == arr[l]. - Sorted descending:
query(l, r) == arr[r]. - Random: 10⁴ queries on a length-1000 random array vs brute force.
- Edge:
query(0, n-1)should equalmin(arr). - Power-of-two length and non-power-of-two length both must pass.
Follow-up Questions
- “Now also support range-max.” → second sparse table or pack
(min, max)into each cell. - “Now updates are required.” → switch to segment tree; sparse table cannot support O(log N) updates without rebuild.
- “Now I want range-sum.” → sum is not idempotent. You can still answer in O(log N) by combining
K = log(len)non-overlapping doubling intervals, but at that point segment tree is simpler. - “Range LCA.” → reduce to range-min on Euler tour depth array + sparse table over depths. Lab in Phase 4.
- “Reduce memory at the cost of complexity.” → Fischer–Heun (RMQ ±1) is O(N) preprocessing + O(1) query but conceptually heavy.
Product Extension
Static analytics dashboards (pre-aggregated, refreshed nightly) over time-series metrics: “min latency in this 5-minute window over the last 24 hours, sliding”. Pre-aggregate the time-series as a sparse table at end-of-day; serve queries at the dashboard at single-microsecond latencies. The “static” condition matches because the data is read-only between rebuilds.
Language/Runtime Follow-ups
- Python: list-of-lists is cache-unfriendly; flatten to one big list with manual indexing for ~3x speedup. PyPy if benchmarking.
- Java:
int[][] st = new int[K+1][n]. JIT will hoist invariants. For N=10⁶ allocate carefully. - Go:
[][]intis fine;make([]int, n)inside a loop is idiomatic. - C++:
vector<vector<int>> st(K + 1, vector<int>(n)). With-O2this is the canonical fast implementation. - JS/TS:
Int32Arrayper row beatsArrayfor numeric ops. JS doesn’t have integer log2 —Math.log2is float and slow; use(31 - Math.clz32(x))for 32-bit ints.
Common Bugs
- Computing
log2per query → floating-point rounding errors, e.g.log2(8) → 2.9999..., floored to 2. Always use the precomputedlog_floor[]. - Building
st[k][i]fori + 2^k - 1 ≥ N— out-of-bounds. Loop must end atn - 2^k. - Sizing
stwith too few rows: K = log_floor[N], but allocating K rows misses the K-th. Allocate K+1. - Using sparse table for sum and then puzzling over wrong answers — sum is not idempotent.
- Forgetting that the array must be static. If a query is interleaved with mutation, the structure silently returns stale answers.
- Off-by-one in query:
r - (1 << k) + 1vsr - (1 << k). The interval is[r - 2^k + 1, r]of length2^k— verify by hand on a tiny case.
Debugging Strategy
Print st[k] for small N=8 and verify by hand: st[0] is the array, st[1][i] = min(arr[i], arr[i+1]), st[2][i] = min(arr[i..i+3]), st[3][0] = min(arr[0..7]). If those don’t hold, your build loop is wrong. If queries fail but build is correct, suspect log_floor and/or query indexing — trace the formula on query(2, 5) where len=4, k=2, indices=2 and 5-4+1=2, both pointing at the same precomputed cell.
Mastery Criteria
- Stated the idempotence requirement and gave 3 ops that satisfy it and 1 that doesn’t.
- Wrote sparse table from scratch in <10 minutes.
-
Wrote
log_floortable without usingmath.log2. - Chose sparse table over segment tree for a read-only workload by stating the constants (1 vs log N per query).
- Identified the failure mode “what if updates are needed” and named segment tree as the replacement.
- Solved one classic RMQ problem and one problem reducible to RMQ.
Lab 05 — KMP String Matching
Goal
Implement Knuth–Morris–Pratt (KMP): build the failure function (longest proper prefix-suffix) of a pattern in O(M), then match the pattern against a text in O(N + M). Apply it to LeetCode 28 (strStr) and LeetCode 459 (Repeated Substring Pattern). After this lab, you should be able to derive fail[] on a 10-character string by hand and write the matcher in <12 minutes.
Background Concepts
The naive substring search compares the pattern against every position in the text: O(N · M) worst case. KMP exploits the fact that when a mismatch occurs at pattern position j, the prefix P[0..j-1] did match. So we already know the last j characters of the text. From that we compute “what is the longest proper prefix of P that is also a suffix of P[0..j-1]?” — call that length fail[j-1]. Then we resume matching at pattern position fail[j-1] without backtracking the text pointer.
The failure function (also called “longest proper prefix-suffix” or LPS):
fail[i]= length of the longest proper prefix ofP[0..i]that is also a suffix ofP[0..i].- “Proper” = strictly shorter than
i + 1. fail[0] = 0always.
Build in O(M) using a two-pointer recurrence: j = fail[i-1]; if P[j] == P[i], fail[i] = j + 1; else fall back j = fail[j-1] and retry, until j = 0.
The matcher: walk text pointer i forward; pattern pointer j advances on match, falls back to fail[j-1] on mismatch (without resetting i).
Interview Context
KMP is the bedrock single-pattern string algorithm. Asked at every FAANG, every quant shop, every search-infra team. The give-away signal: “find pattern in text” with N, M up to 10⁵ — naive is 10¹⁰ ops. Most candidates know strStr exists; few can derive fail[] correctly under pressure. Doing it cleanly is a strong signal at L4+.
Problem Statement (LC 28)
Given two strings haystack and needle, return the index of the first occurrence of needle in haystack, or -1 if needle is not part of haystack.
Constraints
- 1 ≤ haystack.length, needle.length ≤ 10⁴ (LC 28); generalize to 10⁵.
- All printable ASCII.
Clarifying Questions
- First occurrence (leftmost), or any? (Leftmost.)
- Return
0for emptyneedle? (Yes — convention.) - Case-sensitive? (Yes by default.)
Examples
strStr("sadbutsad", "sad") → 0
strStr("leetcode", "leeto") → -1
strStr("hello", "ll") → 2
strStr("aabaaabaaac", "aabaaac") → 4
Initial Brute Force
Two nested loops: for each starting index i ∈ [0, N - M], compare text[i..i+M-1] against pattern; return i on full match.
Brute Force Complexity
O((N - M + 1) · M) ≈ O(N · M). Worst case at text = "aaa..a", pattern = "aaa..b": each starting position fails on the last character. At N = M = 10⁵: 10¹⁰ ops. TLE.
Optimization Path
KMP O(N + M). Z-algorithm O(N + M) — equally good, different invariants. Rabin–Karp O(N) expected with hashing — needs verify on collision. Suffix automaton O(N + M) for the text-side, O(M) match — overkill for one pattern.
KMP is the canonical answer because it (a) is exact (no probabilistic concerns), (b) generalizes to “longest border” / “shortest period” follow-ups, (c) is preferred by interviewers as a known-quantity algorithm.
Final Expected Approach
- Build
fail[]of length M for the pattern. - Walk a single pointer
iover the text andjover the pattern.- If
text[i] == pat[j]: advance both; ifj == M, report match ati - M. - If mismatch and
j > 0:j = fail[j - 1](don’t advancei). - If mismatch and
j == 0:i += 1.
- If
Data Structures Used
fail[]— int array of length M.- Two pointers,
iandj.
Correctness Argument
Failure function invariant: at the end of the build loop’s iteration on i, fail[i] equals the length of the longest proper prefix of P[0..i] matching its suffix.
Proof sketch: assume fail[0..i-1] is correct. Set j = fail[i-1] — the longest proper border of P[0..i-1]. Try to extend: if P[j] == P[i], the border extends by one to j + 1, and there is no longer border (any longer would give a longer border at i-1). If not, fall back to the next-shorter border via j = fail[j-1]; repeat.
Match invariant: when the matcher is at text position i and pattern position j, the last j characters of text up to i-1 equal P[0..j-1]. On mismatch, falling back to fail[j-1] finds the longest proper prefix of P that is a suffix of P[0..j-1], which is also a suffix of the text so far — preserving the invariant without rescanning text.
Linearity: in the matcher, the variable i + (i - j) strictly increases on every iteration. Since i ≤ N and j ≥ 0, the loop runs ≤ 2N times. Same trick for the build: i + (i - j) ≤ 2M.
Complexity
| Operation | Time | Space |
|---|---|---|
| Failure function build | O(M) | O(M) |
| Match | O(N + M) | O(M) |
Implementation Requirements
def build_failure(pat):
m = len(pat)
fail = [0] * m
j = 0
for i in range(1, m):
while j > 0 and pat[j] != pat[i]:
j = fail[j - 1]
if pat[j] == pat[i]:
j += 1
fail[i] = j
return fail
def strStr(text, pat):
if not pat: return 0
n, m = len(text), len(pat)
if m > n: return -1
fail = build_failure(pat)
j = 0
for i in range(n):
while j > 0 and text[i] != pat[j]:
j = fail[j - 1]
if text[i] == pat[j]:
j += 1
if j == m:
return i - m + 1
return -1
Tests
- Empty pattern → 0.
- Pattern not in text → −1.
- Pattern at the start:
strStr("abcd", "ab") == 0. - Pattern at the end:
strStr("abcd", "cd") == 2. - Pattern equals text:
strStr("hello", "hello") == 0. - Repeated chars:
strStr("aaaa", "aa") == 0. - Worst-case backtrack:
strStr("aaaaab", "aaab") == 2. - Verify
failfor"aabaaab"→[0, 1, 0, 1, 2, 2, 3]. - Stress: random texts/patterns, compared against
text.find(pat).
Follow-up Questions
- “Find all occurrences.” → on match, instead of returning, record
i - m + 1and continue withj = fail[j - 1]. - “Repeated substring pattern (LC 459).” →
sis composed ofk ≥ 2repetitions of a substring iffn % (n - fail[n-1]) == 0andfail[n-1] > 0. - “Shortest palindrome (LC 214).” → run KMP on
s + '#' + reverse(s); the answer prefix length isfail[-1]. - “Multi-pattern matching.” → Aho–Corasick (Phase 3 #17) generalizes KMP to a trie of patterns.
- “Strict period” (longest period of S) =
n - fail[n-1]; longest border =fail[n-1]. They are dual. - “Z algorithm — implement it instead.” → different invariant, same asymptotics; pick whichever is fluent.
Product Extension
Search-engine snippet generation: for each query term, find the first match in each candidate document. Multi-pattern at scale uses Aho–Corasick; single-pattern intra-doc still uses KMP because of its predictable cache behavior. Anti-virus signature scanning of binaries is the same problem, multi-pattern, with patterns numbered in the millions — Aho–Corasick territory but KMP per-pattern is the building block.
Language/Runtime Follow-ups
- Python: built-in
str.findis C-implemented Two-Way / Crochemore — usually faster than Python-level KMP. KMP wins when you want all occurrences or the failure function for other purposes. - Java:
String.indexOfis a naive scan. KMP wins for adversarial inputs. Usechar[]overString.charAtfor the inner loop. - Go:
strings.Indexuses Rabin–Karp with a fallback. KMP useful when you want explicitfail[]. - C++:
string::findis naive in libstdc++. KMP from scratch is the canonical CP move.std::vector<int>forfail. - JS/TS:
String.prototype.indexOfis engine-dependent; V8 uses Boyer–Moore–Horspool. KMP needed when you implement custom matchers.
Common Bugs
- Setting
fail[0] = 1instead of 0. The “proper” prefix excludes the full string. - In the build, forgetting
while j > 0 and pat[j] != pat[i]: j = fail[j-1]is awhile, not anif. Treating it asifgives wrong answers on patterns like"aabaaab". - Resetting
j = 0between independent matches but forgetting to reseti— leftover from a “match all” loop. - Using
pat[i] != pat[j]vspat[i] != pat[j-1]— pick a consistent indexing for the LPS (length, not index) and don’t mix. - Off-by-one when reporting the match index:
i - m + 1(0-indexed start) vsi - m. - For LC 459, forgetting the
fail[n-1] > 0guard — without it, a non-repeating string passes the divisibility check trivially.
Debugging Strategy
Compute fail[] for a 7-char pattern by hand and compare. For "aabaaab": fail = [0, 1, 0, 1, 2, 2, 3]. Walk the recurrence: at i=4, j=fail[3]=1, pat[1]==‘a’==pat[4]=‘a’ → fail[4]=2. At i=5, j=fail[4]=2, pat[2]==‘b’!=pat[5]=‘a’ → j=fail[1]=1, pat[1]==‘a’==pat[5]=‘a’ → fail[5]=2. If your code disagrees, instrument with prints.
For the matcher, trace (i, j) per iteration on text="aabaaabaaac", pat="aabaaac". After an early mismatch at (i=6, j=6) (text=‘b’ vs pat=‘c’), j=fail[5]=2, so we resume at text 6 vs pat 2.
Mastery Criteria
-
Computed
fail[]for an unfamiliar 8-character pattern by hand in <2 minutes. -
Wrote
build_failureandstrStrin <12 minutes total, no off-by-ones. - Solved LC 28, LC 459, and LC 1392 from cold start.
- Stated longest-border vs shortest-period duality.
- Identified KMP as the single-pattern engine; identified Aho–Corasick as the multi-pattern generalization.
-
Stated the linear-time argument (potential function
i + (i - j)).
Lab 06 — Rolling Hash (Rabin–Karp + Double Hashing)
Goal
Implement a polynomial rolling hash with two independent (base, mod) pairs. Use it to (a) find repeated substrings of a fixed length (LC 187 — Repeated DNA Sequences) and (b) find the longest duplicate substring via binary search on length (LC 1044 — Longest Duplicate Substring). Internalize the modular arithmetic well enough to avoid collisions on adversarial inputs.
Background Concepts
A polynomial rolling hash treats a string as a base-b number mod p:
H(S) = (S[0] · b^(L-1) + S[1] · b^(L-2) + ... + S[L-1]) mod p
The “rolling” part: when you slide the window from S[i..i+L-1] to S[i+1..i+L], you update in O(1):
H_new = ((H_old - S[i] · b^(L-1)) · b + S[i+L]) mod p
For substring equality from prefix hashes, precompute pref[i] = (S[0] · b^(i-1) + ... + S[i-1]) mod p. Then H(S[l..r]) = (pref[r+1] - pref[l] · b^(r - l + 1)) mod p.
Single hash collision risk. A single mod-p hash has birthday-paradox collision probability ~k²/(2p) for k strings. At p ~ 10⁹ and k = 10⁵, that’s ~5% chance of collision. Adversarial inputs constructed to collide on a fixed (b, p) make single hashing unsafe.
Double hashing uses two independent (b₁, p₁) and (b₂, p₂); two strings collide on both with probability ~k²/(p₁ · p₂) ≈ 10⁻⁹ for ~10⁵ strings. Effectively safe for interviews. Anti-hash-resistant code uses random bases per run.
Interview Context
Rolling hash shows up whenever the brute-force algorithm involves “compare every substring of length L to every other” — a quadratic-in-N algorithm. The reduction is “convert string equality to integer equality, get O(1) per comparison”. Asked at: Google (frequent — duplicate detection, plagiarism), search-infra teams, biotech companies (DNA sequence problems), Stripe.
The signal: “many substring equality checks”, “longest repeated”, “find duplicates of length L”.
Problem Statement A (LC 187)
Find all 10-letter sequences that occur more than once in a DNA string s over {A, C, G, T}. Return them in any order.
Problem Statement B (LC 1044)
Given a string s, find the longest duplicated substring (any substring of length ≥ 1 that appears at least twice — overlaps allowed). Return any longest. If no duplicates, return "".
Constraints
- LC 187: 1 ≤ |s| ≤ 10⁵; alphabet
ACGT. - LC 1044: 2 ≤ |s| ≤ 3 × 10⁴; lowercase ASCII.
Clarifying Questions
- Overlaps allowed? (Yes — both problems.)
- Single longest, or all? (LC 1044: any one longest.)
- Case sensitivity / encoding? (As given by problem.)
- May we use suffix array / suffix automaton? (Yes, but the lab assignment is rolling hash.)
Examples
LC 187: s = "AAAAACCCCCAAAAACCCCCCAAAAAGGGTTT" → ["AAAAACCCCC", "CCCCCAAAAA"].
LC 1044: s = "banana" → "ana".
Initial Brute Force
LC 187: hashtable of every length-10 substring. O(N · L) time, O(N · L) memory — already passes.
LC 1044: for each L from N-1 down to 1, scan all length-L substrings and check duplicates. O(N² · L) at worst — 3 × 10¹² at the limits. TLE.
Brute Force Complexity
LC 187: O(N · L) acceptable. LC 1044: O(N²) at minimum, O(N³) naively. TLE.
Optimization Path
LC 187 with rolling hash: O(N) amortized per scan, O(N) total. Useful when L is large; for L = 10 the brute force is fine, but rolling hash demonstrates the technique.
LC 1044 with rolling hash + binary search on L: outer loop binary-searches L in [1, N-1]; inner loop hashes every length-L substring and looks up duplicates in a dict. O(N log N) total, with double-hashing or hash+verify.
Final Expected Approach
def has_duplicate_of_length(s, L):
# Returns the starting index of a duplicate of length L, or -1.
h = 0; pow_L = 1
seen = {}
for i in range(L):
h = (h * BASE + ord(s[i])) % MOD
if i: pow_L = (pow_L * BASE) % MOD
seen[h] = 0
for i in range(1, len(s) - L + 1):
h = ((h - ord(s[i-1]) * pow_L) * BASE + ord(s[i + L - 1])) % MOD
if h in seen:
# verify (or use double-hash) to avoid false-positive
if s[seen[h]:seen[h]+L] == s[i:i+L]:
return i
else:
seen[h] = i
return -1
# Binary search L in [1, N-1].
For LC 187, just collect a multiset of length-10 hashes; output any whose count > 1 (and verify).
Data Structures Used
- Two integer mods (~10⁹ primes), two random bases.
- Dict from hash (or hash-pair) → starting index.
- Precomputed power array
pow_b[i].
Correctness Argument
If two strings have the same content, their polynomial hash is equal — exact, no probability. The other direction (same hash → same content) is not guaranteed; this is why we verify on collision (or use double-hashing for ~10⁻¹⁸ collision probability).
For LC 1044, monotonicity: if a duplicate of length L exists, every L’ ≤ L has a duplicate (a prefix of one of the duplicate occurrences). So the answer set {L : duplicate exists} is a prefix [1, L*], and binary search finds L* in O(log N) calls of has_duplicate_of_length.
Complexity
| Operation | Time | Space |
|---|---|---|
| Single-length scan | O(N) amortized | O(N) |
| LC 187 | O(N · L) | O(N · L) |
| LC 1044 (binary search) | O(N log N) | O(N) |
Verifying on collision adds an O(L) hit per match; with double-hashing it’s negligible.
Implementation Requirements
import random
class RollingHash:
def __init__(self, s):
self.n = len(s)
# Use two (base, mod) pairs.
self.MOD1, self.MOD2 = (1 << 31) - 1, (1 << 61) - 1
self.B1 = random.randint(27, self.MOD1 - 1)
self.B2 = random.randint(27, self.MOD2 - 1)
self.h1 = [0] * (self.n + 1)
self.h2 = [0] * (self.n + 1)
self.p1 = [1] * (self.n + 1)
self.p2 = [1] * (self.n + 1)
for i, c in enumerate(s):
v = ord(c)
self.h1[i+1] = (self.h1[i] * self.B1 + v) % self.MOD1
self.h2[i+1] = (self.h2[i] * self.B2 + v) % self.MOD2
self.p1[i+1] = (self.p1[i] * self.B1) % self.MOD1
self.p2[i+1] = (self.p2[i] * self.B2) % self.MOD2
def hash_pair(self, l, r): # [l, r)
a = (self.h1[r] - self.h1[l] * self.p1[r - l]) % self.MOD1
b = (self.h2[r] - self.h2[l] * self.p2[r - l]) % self.MOD2
return (a, b)
def longestDupSubstring(s):
n = len(s)
rh = RollingHash(s)
def find(L):
seen = {}
for i in range(n - L + 1):
h = rh.hash_pair(i, i + L)
if h in seen: return i
seen[h] = i
return -1
lo, hi, best_start, best_len = 1, n - 1, 0, 0
while lo <= hi:
mid = (lo + hi) // 2
i = find(mid)
if i != -1:
best_start, best_len = i, mid
lo = mid + 1
else:
hi = mid - 1
return s[best_start:best_start + best_len]
Tests
- LC 187 sample →
["AAAAACCCCC", "CCCCCAAAAA"]. - LC 1044
"banana" → "ana". - LC 1044
"abcd"→"". - LC 1044
"aaaaaa"→"aaaaa"(overlapping duplicates). - LC 1044
"abab"→"ab". - Single-hash adversary: paste a Codeforces anti-hash test for
(b=31, p=10^9+7); verify your double-hash survives. - Stress: 1000 random strings of length 100, compare LC 1044 result to a brute-force suffix-array approach.
Follow-up Questions
- “What if I demand zero false positives?” → either verify on every match (O(L) extra) or use suffix array / suffix automaton (deterministic O(N log N) or O(N)).
- “Multiple texts share patterns.” → hash all, group by hash-pair, verify within group.
- “Stream the text.” → keep a rolling hash; emit matches online; constant memory beyond the dictionary.
- “Distinguish substrings cyclically equal.” → hash min-rotation (Booth’s algorithm) or all rotations.
- “Avoid Python big-int slowdown.” → use
(1 << 61) - 1(Mersenne prime) and bitwise reduction; or numpy.
Product Extension
Plagiarism detection over a corpus: shingle each document into k-grams, hash each shingle, store (doc_id, hash) tuples; near-duplicate documents share many shingles. MinHash + LSH (locality-sensitive hashing) is the production technology, built on the same rolling-hash foundation. Snapchat / Imgur-style “hash this image” reuses the same logic with perceptual hashes.
Language/Runtime Follow-ups
- Python: integers are bignums; prefer mods that fit in 63 bits to keep ops fast. Avoid
pow(b, k, m)in hot loops — precompute pow tables. - Java: use
longfor mod ~10⁹ to keepb · h + cfrom overflowingint. For mod 2⁶¹-1 you needMath.floorModand 128-bit (Math.multiplyHighin JDK 9+). - Go:
uint64with mod 2⁶¹-1 lets you(x * y) >> 61reduce. Idiomatic CP technique. - C++:
__int128for mod 2⁶¹-1 multiplications. Otherwiseunsigned long long. - JS/TS:
Numberonly safe for integers ≤ 2⁵³. UseBigInt(slow) or pick mod ≤ 2²⁶ with two-pair hashing to fit safely.
Common Bugs
- Single-hash collision — passes random tests, fails LC 1044 hidden cases. Double-hash always.
- Negative mod in languages with truncated division (Java, C, JS):
(a - b) % MODcan be negative; addMODand re-mod. - Power table off-by-one:
p[i] = b^i.hash[l..r)usesp[r - l], notp[r - l - 1]. - Reusing the same
(b, p)across runs on adversarial inputs — randomize per run. - Forgetting verify on single-hash → wrong answer.
- Boundary in rolling update: drop the leftmost char first (multiply by
pow_L), then add the rightmost.
Debugging Strategy
Construct two strings of the same length with known equality and known inequality; assert your hash_pair(0, L) gives equal pairs iff the strings are equal. For LC 1044, when the binary search returns wrong-length, instrument find(L) to dump (i, hash, prior_index) on each match candidate. If a hash collision passes verify, your hash is wrong; if verify catches it, single-hash worked but barely — switch to double-hash.
Mastery Criteria
- Implemented double-hash rolling hash from scratch in <15 minutes.
- Stated the collision probability for single vs double hash.
- Solved LC 187 in <10 minutes; LC 1044 in <30.
- Recognized binary-search-on-length as the standard “longest duplicate” reduction.
- Used random base per run.
- Stated the alternative (suffix array) and when to prefer it.
Lab 07 — Trie Applications (Implement Trie + Word Search II)
Goal
Build a trie supporting insert, search, startsWith (LC 208), then use a trie to solve LC 212 — Word Search II — by pruning a DFS over a 2D grid against a trie of dictionary words. After this lab, the trie should be a reflex for any “many strings, prefix-shared, batch-query” problem.
Background Concepts
A trie (prefix tree) is a tree where each edge is labeled with one character and each path from the root spells a string. A node may carry an is_end flag meaning “a word terminates here”. Two strings sharing a prefix share the prefix path in the trie, giving O(L) per insert / search regardless of how many words exist — a fundamental advantage over hashtable + per-word check when prefixes overlap.
Children can be stored as:
- Array of 26 (or 256) — fastest dispatch, fixed alphabet, slightly memory-heavy. Best for grid-DFS hot loops.
- Hash map char → node — flexible alphabet, slower constant. Good for arbitrary Unicode or large alphabets.
- Compressed (radix tree / Patricia) — collapse single-child chains; smaller memory, harder to implement.
For LC 212, the killer move is: instead of running KMP / search per word against the grid (O(W · cells)), build a trie of all words and run one DFS that explores the grid while walking the trie in parallel, terminating any branch whose current grid letter has no trie-child. This converts the cost from “many independent searches” into “one search with multi-end”.
Interview Context
Tries are asked at every FAANG with high frequency: Meta, Google, Amazon all have multiple variants in their pool. The pattern is “many strings share prefixes; query, autocomplete, or grid-search”. Recognizing the trie-prune-DFS reduction for LC 212 is a strong-hire signal — it’s a 5-line code change with a 100x speedup.
Cousins: autocomplete (LC 642), longest word in dictionary (LC 720), word break (LC 139 — sometimes trie-friendly), maximum xor of two numbers (LC 421 — bit-trie).
Problem Statement A (LC 208)
Implement Trie with insert(word), search(word) (exact match), startsWith(prefix) (any word with this prefix).
Problem Statement B (LC 212)
Given an m × n grid of letters and a list words, return all words that can be formed by a sequence of adjacent (4-directionally connected) cells in the grid, where each cell is used at most once per word.
Constraints
- LC 208: ≤ 3 × 10⁴ ops; lowercase ASCII; word length ≤ 2000.
- LC 212: 1 ≤ m, n ≤ 12; 1 ≤ |words| ≤ 3 × 10⁴; each word length ≤ 10; lowercase ASCII.
Clarifying Questions
- Are duplicate words possible in the dictionary? (LC 212: assume no, dedupe defensively.)
- Can the same cell be reused across different words? (Yes — only “once per word”.)
- Is alphabet exactly lowercase ASCII? (Yes — use array of 26.)
Examples
LC 212:
board = [['o','a','a','n'],
['e','t','a','e'],
['i','h','k','r'],
['i','f','l','v']]
words = ["oath", "pea", "eat", "rain"]
→ ["oath", "eat"]
Initial Brute Force
For each word, run a DFS from every starting cell that matches the word’s first letter; backtrack on dead ends.
Per-word DFS: O(m · n · 4^L). Total: O(W · m · n · 4^L). With W=3×10⁴, m·n=144, L=10: ~10¹⁰ ops. TLE.
Brute Force Complexity
O(W · m · n · 4^L). At the limits: 1.6 × 10¹⁰. TLE by 4 orders of magnitude.
Optimization Path
Trie-pruned DFS. Build a trie of all words once: O(total chars) ≈ 3 × 10⁵. Then run a single DFS from each cell, walking the trie in parallel; whenever a trie-child for the current letter is missing, prune. Whenever an is_end is hit, record that word.
Total: O(m · n · 4^L) for the DFS structure, but the effective branching is small because most paths are pruned within 2-3 chars. Practical speedup: ~100×.
Optimization on top: when a word is recorded, mark its trie-end node and prune empty trie branches as you backtrack — keeps the trie shrinking.
Final Expected Approach
- Build trie: each node has
children[26],word: Optional[str](set on insert at the terminal). - For each cell (i, j), DFS:
- Read
c = board[i][j]. - If
node.children[c - 'a']is None, return. - Descend:
node = node.children[...]. - If
node.word: append to results, setnode.word = Noneto dedupe. - Mark
board[i][j] = '#'to prevent reuse. - Recurse 4 directions.
- Restore
board[i][j] = c.
- Read
- Return results.
Data Structures Used
TrieNode { children: List[Optional[TrieNode]], word: Optional[str] }.- 2D grid (mutable for the visited-mark trick).
- Result list.
Correctness Argument
The trie-DFS enumerates exactly the set of (path-in-grid, path-in-trie) pairs where each step matches the current grid letter to a trie child. A word is reported iff the DFS reaches a trie node with word != None along a non-self-intersecting grid path — by construction this is exactly the set of words present in the grid. Setting node.word = None after recording deduplicates without affecting other paths (the children remain reachable for other words sharing this prefix).
board[i][j] = '#' ensures non-reuse: the only way to visit a cell already on the path is if the trie has ‘#’ as a child of the current node, which it doesn’t (alphabet is lowercase).
Complexity
| Operation | Time | Space |
|---|---|---|
| Trie build | O(total characters in words) | O(total characters) |
| DFS (worst) | O(m · n · 4 · 3^(L-1)) per starting cell | O(L) recursion |
| Total | O(m · n · 4 · 3^(L-1)) | O(W · L) trie + O(L) recursion |
The 3^(L-1) (not 4^L) is because after the first step, you can’t immediately go back, so each node has ≤ 3 forward neighbors.
Implementation Requirements
class TrieNode:
__slots__ = ('children', 'word')
def __init__(self):
self.children = [None] * 26
self.word = None
def findWords(board, words):
root = TrieNode()
for w in words:
node = root
for c in w:
i = ord(c) - ord('a')
if node.children[i] is None:
node.children[i] = TrieNode()
node = node.children[i]
node.word = w
m, n = len(board), len(board[0])
res = []
def dfs(i, j, parent):
c = board[i][j]
if c == '#': return
idx = ord(c) - ord('a')
node = parent.children[idx]
if node is None: return
if node.word is not None:
res.append(node.word)
node.word = None
board[i][j] = '#'
for di, dj in ((-1,0),(1,0),(0,-1),(0,1)):
ni, nj = i + di, j + dj
if 0 <= ni < m and 0 <= nj < n:
dfs(ni, nj, node)
board[i][j] = c
# Optional pruning: if node has no children and no word, parent.children[idx] = None.
for i in range(m):
for j in range(n):
dfs(i, j, root)
return res
Tests
- LC 212 sample →
["oath", "eat"]. - Empty grid / empty word list →
[]. - Word equals a single cell:
board=[['a']], words=["a"]→["a"]. - Word longer than grid: returns nothing.
- Duplicate paths to same word: appears once thanks to
node.word = None. - Words sharing prefixes:
words=["oath", "oat"]— both should be found if both are present. - Stress: random grids of size 12×12, random word list of 100 words length 5; verify against per-word DFS brute force.
Follow-up Questions
- “How would you support deletion?” → reference-count children or do recursive cleanup; tricky due to shared prefixes.
- “Autocomplete with frequency.” → store
countper word at the terminal; on prefix lookup, walk the subtree and pick top-k by count (or maintain a per-node max-heap). - “What if the alphabet is Unicode?” → switch from
children[26]to adict. Per-node memory grows but dispatch is O(1) hash. - “Compress the trie.” → radix / Patricia trie collapses single-child chains; helpful when you have million-word dictionaries (e.g., DNS).
- “Bit trie for max-XOR (LC 421).” → trie of binary representations, depth 30 for 32-bit ints; greedy descent picks the opposite bit when possible.
Product Extension
Search-as-you-type / autocomplete: as a user types each character, walk the trie down one node and emit the top-k completions stored at that node. Production search engines (ES, Solr) build inverted indices, but for “small dictionary, fast prefix lookup” use cases (CLI command completion, query suggestion within an admin tool), an in-memory trie is the right call. DNS resolution uses radix tries internally.
Language/Runtime Follow-ups
- Python:
__slots__on TrieNode trims memory by ~40%. For LC 212 uselistof children rather thandict— dict’s per-key overhead dominates at this size. - Java:
TrieNode { TrieNode[] children = new TrieNode[26]; String word; }. JIT inlines the array dispatch. - Go:
type TrieNode struct { children [26]*TrieNode; word string }. Value-typed array of pointers is cache-friendlier than a slice. - C++:
struct TrieNode { array<TrieNode*, 26> children{}; string word; };. Use adeque<TrieNode>arena to avoidnewper node. - JS/TS: plain object
{children: Array(26), word: null}works; for large triesMapwith char keys uses less memory than 26-array per node.
Common Bugs
- Forgetting to mark
board[i][j] = '#'before recursing — same cell reused, wrong matches reported. - Forgetting to restore
board[i][j] = cafter recursion — corrupts the board for subsequent DFS calls. - Recording the word multiple times because the same prefix is reached via different paths. Fix:
node.word = Noneafter recording. - Storing the word only at the terminal — works if you carry the path string in DFS, wasteful otherwise. Correct: store the word at the terminal trie node.
- Initializing
children = [None](length 1) instead of[None] * 26— silent error at first non-‘a’ insert. - Walking the trie before reading the grid letter — off-by-one; you should read the grid letter, look up
parent.children[c], then descend.
Debugging Strategy
Trace the DFS on the LC 212 sample by hand: starting at (0, 0)=‘o’, root has child ‘o’ → ok; recurse to (1, 0)=‘e’ or (0, 1)=‘a’; descend to “oa” → child ‘a’; etc. If your output is missing words, add print(node.word) on the entry to dfs and verify the word terminations are correctly placed in the trie. If your output has spurious words, the visited-mark or the is_end placement is wrong.
Mastery Criteria
-
Wrote
Trieclass withinsert/search/startsWithin <8 minutes. - Solved LC 212 in <30 minutes from cold start.
- Stated the speedup over per-word DFS and the complexity of the combined DFS.
-
Used
node.wordstorage (notis_end + carry-string) for clean dedup. - Picked array-of-26 over hashmap for performance and justified it.
- Solved LC 421 (bit-trie max XOR) using the same structure.
Lab 08 — Bitmask Dynamic Programming
Goal
Solve LC 847 — Shortest Path Visiting All Nodes — using bitmask DP over (visited-set, current-node) state. Internalize the recipe: when N ≤ 20-ish and the state involves “subset of which items have been used / visited / assigned”, a bitmask is the state and the transition is a bit OR.
Background Concepts
A bitmask is an integer interpreted as a set: bit k is 1 iff element k is in the set. Set operations:
- Union:
a | b. Intersection:a & b. Difference:a & ~b. Symmetric difference:a ^ b. - Add element k:
a | (1 << k). Remove:a & ~(1 << k). Test:(a >> k) & 1ora & (1 << k). - Iterate all subsets of
mask:s = mask; while s > 0: ...; s = (s - 1) & mask. Iterates 2^popcount(mask) subsets. - Iterate set bits:
while mask: k = (mask & -mask).bit_length() - 1; mask &= mask - 1.
Bitmask DP stores dp[mask][...] indexed by the subset. Useful when N ≤ ~20 (so 2^N ≤ 10⁶) and the state must remember “exactly which subset has been processed”. It generalizes:
- TSP-like:
dp[mask][i] = min cost path that visited exactlymask, ending at i. - Subset-cover:
dp[mask] = min cost to covermask`` summed over groups. - Assignment problem:
dp[mask] = min cost to assign first popcount(mask) people to the jobs inmask``.
For LC 847, we want shortest walk (edges may repeat) visiting all nodes. The state (mask, i) captures “I’ve visited the set mask of nodes (at least once) and I’m currently at node i”. Transitions: from (mask, i), move to any neighbor j, new state (mask | (1 << j), j), cost +1. We want the smallest distance to any state (full_mask, *). BFS suffices since edge cost is 1.
Interview Context
Bitmask DP is a 1-2% problem family but a strong-hire signal when recognized fast. The trigger: N ≤ 20 with a constraint involving subsets. Asked at: Google occasionally, Stripe / Two Sigma, Meta in bar-raiser slot. Common trap is recognizing 2^N · N is feasible at N=15 (~5 × 10⁵) but not at N=25 (~8 × 10⁸).
Problem Statement (LC 847)
Given an undirected, connected graph of N nodes labeled 0..N-1 as adjacency lists, return the length of the shortest path that visits every node. You may start and stop at any node, may revisit nodes and edges.
Constraints
- 1 ≤ N ≤ 12
- 0 ≤ graph[i].length < N
- Graph is connected.
Clarifying Questions
- Length = number of edges (not nodes)? (Yes — number of edges traversed.)
- Are self-loops possible? (No.)
- May the path start and end at different nodes? (Yes.)
- Is the graph guaranteed connected? (Yes — answer always finite.)
Examples
graph = [[1, 2, 3], [0], [0], [0]]
(star with center 0, leaves 1, 2, 3)
shortest path visiting all = 4 (e.g., 1 → 0 → 2 → 0 → 3)
graph = [[1], [0, 2, 4], [1, 3, 4], [2], [1, 2]]
shortest = 4
Initial Brute Force
DFS / backtracking from every starting node, exploring all walks up to some bounded length. Without memoization, walks can be exponential in length even for small graphs. A timeout and a hand-tuned bound make this brittle.
Brute Force Complexity
Unbounded (or exponential with bound). Practically TLE for any non-trivial test.
Optimization Path
The state space is (mask, current_node) with mask ∈ [0, 2^N) and current_node ∈ [0, N). Total states: N · 2^N. For N=12: 12 · 4096 = 49152. Each state has ≤ N-1 outgoing transitions; total edges: N² · 2^N ≈ 6 × 10⁵. Trivially feasible.
Since edge weights are 1, BFS over the state graph from all starting states {(1 << i, i) : i ∈ [0, N)} (all “I’ve visited just myself” states) gives the shortest distance to each state. Stop when we dequeue a state with mask = (1 << N) - 1.
Final Expected Approach
full = (1 << N) - 1.- Initialize a queue with all
(mask=1<<i, node=i)fori ∈ [0, N). seen[(mask, node)]initialized for those starts.- BFS: pop
(mask, u); for each neighborv,new_mask = mask | (1 << v); if(new_mask, v)unseen, enqueue withdist+1. - First time a state with
mask == fullis dequeued, return its distance.
Data Structures Used
dequefor BFS frontier.- 2D
seenof shape[2^N][N](boolean) or a set. - Distance tracked alongside state in the queue (
dist+1per step).
Correctness Argument
The state graph is a directed graph on N · 2^N states; an edge (mask, u) → (mask | (1 << v), v) exists iff v is a graph neighbor of u. A walk in the original graph that visits all N nodes corresponds to a path in the state graph from some (1<<i, i) to some (full, j). Edge costs are 1 (one edge traversed per state-graph edge). Therefore shortest walk = shortest path in state graph from the start set to any final state, computed by multi-source BFS.
BFS visits each state once and terminates at the first finalized state. Correct because all edge weights equal.
Complexity
| Quantity | Value |
|---|---|
| States | N · 2^N |
| Transitions | up to N² · 2^N |
| Time | O(N² · 2^N) |
| Space | O(N · 2^N) |
At N=12: ~6 × 10⁵ ops. Fast.
Implementation Requirements
from collections import deque
def shortestPathLength(graph):
n = len(graph)
if n == 1: return 0
full = (1 << n) - 1
# State: (mask, node). Distance tracked by BFS layer.
seen = [[False] * n for _ in range(1 << n)]
q = deque()
for i in range(n):
seen[1 << i][i] = True
q.append((1 << i, i, 0))
while q:
mask, u, d = q.popleft()
if mask == full:
return d
for v in graph[u]:
new_mask = mask | (1 << v)
if not seen[new_mask][v]:
seen[new_mask][v] = True
q.append((new_mask, v, d + 1))
return -1 # unreachable for connected graphs
Tests
- N=1: return 0.
- N=2 with one edge: return 1.
- Star example: 4.
- Linear chain
0-1-2-3-4: shortest visiting all = 4. - Complete graph K_5: shortest = 4 (any Hamiltonian path).
- Disconnected (constraint says connected, but defensive): return -1 / handle.
- Stress: random connected graphs N=8..12 vs Held-Karp O(N² · 2^N) DP for cross-check.
Follow-up Questions
- “What if edges have weights?” → Dijkstra instead of BFS; same state graph.
- “What if I must start and end at node 0 (TSP)?” → state
(mask, i)with costdp[mask][i]= min cost, recurrencedp[mask | (1 << j)][j] = min(dp[mask][i] + w(i, j)). Answer:min(dp[full][i] + w(i, 0)). - “What if N=20?” → 20 · 2^20 = 2 × 10⁷ states, still ok. At N=25 we hit 8 × 10⁸ — likely TLE. The constraint cap on bitmask DP is N ~ 22.
- “Subset-cover variant (LC 1125 — Smallest Sufficient Team).” →
dp[mask]= min team to cover skill-maskmask; transition: for each person p with skill-maskpm,dp[mask | pm] = min(dp[mask] + 1). - “Assignment problem in bitmask DP.” →
dp[mask] = min cost to assign popcount(mask) people to the jobs in mask; transition over which job personpopcount(mask)takes.
Product Extension
Vehicle routing / drone delivery with ≤ 20 stops: bitmask DP precomputes optimal tours offline. Interview-scheduling problems (LC 1066): assign N workers to N tasks minimizing cost; the assignment-DP variant of bitmask DP runs in O(N · 2^N), beating the O(N · N!) brute force at N=15 by 9 orders of magnitude.
Language/Runtime Follow-ups
- Python: BFS with deque; the inner loop can be slow at N=12. PyPy if benchmarking. For N>14, switch to numpy or pre-flatten the seen array to
bytearray. - Java:
boolean[][] seen = new boolean[1 << n][n]. The dequeArrayDeque<int[]>boxes each state; for performance pack(mask, node, dist)into along. - Go: idiomatic
[][]bool. Use slice queue with head/tail indices to avoid alloc churn. - C++:
vector<vector<bool>> seen(1 << n, vector<bool>(n)). Pack state intoint(mask * n + node) and usevector<bool>of sizen * (1 << n)for cache. - JS/TS:
Uint8Array(n * (1 << n))for seen; bitwise ops are 32-bit signed — fine for n ≤ 30.
Common Bugs
- Forgetting that the path may revisit nodes — implementing as Hamiltonian path (
maskexactly indicates visited once) is wrong for LC 847; use the right transitionmask | (1 << v)(idempotent on already-visited nodes). - Initializing only one start state instead of all N — gives wrong answers because the optimal path may not start at node 0.
- Returning the first full-mask state encountered without distance: BFS guarantees minimal distance only because of FIFO ordering — correct here, but easy to swap for DFS by accident.
- Using
mask & (1 << v)as a boolean test in C/Java — works, but in JS/Python be explicit:(mask >> v) & 1. - Allocating
seen = [[False] * n] * (1 << n)(shared row reference) — Python beginner trap. - Off-by-one on
full = (1 << n) - 1vs(1 << n).
Debugging Strategy
For N=4 star, hand-simulate: start (1, 0=center) at d=0, expand to neighbors {1,2,3} → states (11, 1), (101, 2), (1001, 3) at d=1. From (11, 1), can go back to 0 → (11, 0) at d=2. From (11, 0) expand to 2 or 3 → (111, 2) or (1011, 3) at d=3. From (111, 2) go to 0 → (111, 0) at d=4. From (111, 0) to 3 → (1111, 3) at d=5. But the expected answer is 4! The min path is starting from a leaf: start (2, 1) → (3, 0) → (7, 2) → (15, 3)? Wait, going 1 → 0 → 2 → 0 → 3 is 4 edges. Let me recount: starts (2, 1) at d=0, → (3, 0) at d=1, → (7, 2) at d=2, → (7, 0) at d=3, → (15, 3) at d=4. Yes, 4. The issue with my earlier trace was starting from center.
If your code returns 5, you forgot to seed BFS with all start states.
Mastery Criteria
- Recognized “bitmask DP” within 60 seconds when N ≤ 20 and state involves subsets.
-
Wrote
shortestPathLengthfrom scratch in <20 minutes. - Stated state space size and confirmed feasibility for the given N.
- Solved one cousin (LC 1125, LC 943, LC 691) from cold start.
- Used proper bit operations (no string-based mask handling).
- Articulated when bitmask DP fails (N > 22 → 2^N too large).
Lab 09 — Meet in the Middle
Goal
Solve LC 1755 — Closest Subsequence Sum — via meet-in-the-middle: split the array into two halves, enumerate 2^(N/2) subset sums in each half, sort one half, and use binary search / two-pointer to find the pair-sum closest to the goal. Internalize the technique as the standard recipe whenever N is in the awkward zone 30 ≤ N ≤ 40 — too large for 2^N enumeration, too small for any polynomial DP.
Background Concepts
Meet in the middle (MITM) trades exponential time for exponential space, halving the exponent: instead of enumerating all 2^N subsets in one go (infeasible at N=40), enumerate 2^(N/2) subsets of each half (feasible at N=20: ~10⁶ each), then combine.
The combination step depends on the problem:
- Closest sum to goal (this lab): sort one half’s sums; for each sum
Lof the left half, binary-search the right half’s sums forgoal − L. - Count of subset-pairs with sum ≤ K: sort both halves; two-pointer.
- Find any subset summing to S: hashmap of one half’s sums; for each sum
L, check ifS − Lis in the map. - k-th smallest subset sum: more elaborate — heap-merge two sorted lists.
The asymptotics: O(2^(N/2) · N/2) to enumerate, O(2^(N/2) · log) for the combine. At N=40: 2^20 ≈ 10⁶, total ~2 × 10⁷. Feasible.
Interview Context
MITM is a niche but high-impact technique. Asked at Google (occasional), CP-flavored shops (frequent), and any problem set with N in [30, 45]. The signal: “N is around 30-40, brute force is 2^N, no polynomial DP visible because the state involves arbitrary subset sums”. Recognizing it converts a hopeless problem into a 30-minute solve. Not recognizing it caps you at “I would brute force but it TLEs” — a soft no-hire.
Problem Statement (LC 1755)
Given an integer array nums and integer goal, return the minimum absolute difference |sum(sub) − goal| over all non-empty subsequences (subsets) of nums. (LC 1755 allows the empty subsequence too — sum 0 — so empty is fine.)
Constraints
- 1 ≤ |nums| ≤ 40
- −10⁷ ≤ nums[i] ≤ 10⁷
- −10⁹ ≤ goal ≤ 10⁹
Clarifying Questions
- Subsequence = subset (unordered)? (Yes — LC’s “subsequence” here is order-independent because we only care about sum.)
- Empty subsequence allowed (sum 0)? (Yes per LC 1755.)
- Sums fit in 64-bit? (Max |sum| = 40 · 10⁷ = 4 × 10⁸ — fits in 32-bit Java
int. Use 64-bit defensively.)
Examples
nums = [5, -7, 3, 5], goal = 6 → 0 (5 + 3 - 5 + ... = 6 exactly via {5, 3, -7+5} = subset {5, -7, 3, 5}=6)
nums = [7, -9, 15, -2], goal = -5 → 1 (e.g., -9 + 7 = -2; |-2 - (-5)| = 3; better: -9 + 15 - 2 = 4; |-9 - (-5)|=4; -2 alone gives |-2-(-5)|=3; closest is -9 + 7 - 2 = -4, diff 1)
nums = [1, 2, 3], goal = -7 → 7 (closest is empty sum 0)
Initial Brute Force
Enumerate all 2^N subsets, compute each sum, track minimum |sum - goal|.
Brute Force Complexity
O(2^N · N). At N=40: 4 × 10¹³. TLE by 7 orders of magnitude.
Optimization Path
DP by sum? Sums range over [-4 × 10⁸, 4 × 10⁸] — too wide for a value-indexed DP. So polynomial DP is out.
O(2^(N/2)) enumeration: 2^20 ≈ 10⁶ per half. Feasible in time, requires the combine step.
For closest-sum-to-goal, sort one half’s sums; for each L in the other half, binary-search goal - L; check the two candidates around the insertion point. Total: O(2^(N/2) · N/2 + 2^(N/2) · log(2^(N/2))) = O(2^(N/2) · N).
Final Expected Approach
- Split
numsinto halvesA(first N/2) andB(last N - N/2). - Enumerate all subset sums of
A: listsumsAof size2^|A|. - Enumerate all subset sums of
B: listsumsBof size2^|B|. - Sort
sumsB. best = min(|s - goal|) for s in sumsA(handles the case where the right side contributes 0 — but since we include 0 as a subset sum of B, this is captured by step 6).- For each
ainsumsA: binary-searchsumsBforgoal - a; checksumsB[idx]andsumsB[idx-1](the two closest); updatebest. - Return
best.
Data Structures Used
- Two flat lists of subset sums.
bisect(Python) /Arrays.binarySearch(Java) /sort.Search(Go) /lower_bound(C++).
Correctness Argument
Every subset of nums decomposes uniquely as (left ∪ right) where left ⊆ A and right ⊆ B. So sum(subset) = a + b for some a ∈ sumsA, b ∈ sumsB. We want min_{a, b} |a + b - goal| = min_a min_b |b - (goal - a)|. For a fixed a, the inner min over b is solved by binary search in sorted sumsB: the closest element is at the insertion point or one position to its left. Iterating over all a ∈ sumsA covers all subsets.
Including 0 in both sumsA and sumsB covers the empty-side cases.
Complexity
| Quantity | Value |
|---|---|
| Enumerate sums | O(N · 2^(N/2)) |
| Sort | O(2^(N/2) · log(2^(N/2))) = O(N · 2^(N/2)) |
| Binary search loop | O(2^(N/2) · log(2^(N/2))) = O(N · 2^(N/2)) |
| Total time | O(N · 2^(N/2)) |
| Space | O(2^(N/2)) |
At N=40: ~4 × 10⁷ ops. Fits in 1 sec C++, ~3 sec Python.
Implementation Requirements
from bisect import bisect_left
def minAbsDifference(nums, goal):
def all_sums(arr):
sums = [0]
for x in arr:
sums = sums + [s + x for s in sums]
return sums
n = len(nums)
A = nums[: n // 2]
B = nums[n // 2 :]
sumsA = all_sums(A)
sumsB = sorted(all_sums(B))
best = abs(goal) # corresponds to the empty subset
for a in sumsA:
target = goal - a
idx = bisect_left(sumsB, target)
if idx < len(sumsB):
best = min(best, abs(a + sumsB[idx] - goal))
if idx > 0:
best = min(best, abs(a + sumsB[idx - 1] - goal))
if best == 0:
return 0
return best
Tests
- N=1,
[5], goal=5 → 0. - N=1,
[5], goal=0 → 0 (empty subset). - LC 1755 sample:
[5, -7, 3, 5], goal=6 → 0. - LC 1755 sample 2:
[7, -9, 15, -2], goal=-5 → 1. - LC 1755 sample 3:
[1, 2, 3], goal=-7 → 7. - All zeros: any goal →
|goal|. - Single huge value:
[10^7] * 40, goal=0 → 0 (empty). - Adversary: random N=40 with random values; cross-check against brute force at N=20.
Follow-up Questions
- “Now I need to count subsets with sum exactly
S.” → enumerate sums of both halves; for each sumain A, count occurrences ofS - ain B (bucket by value or use a Counter). - “Now I need subsets with sum in
[L, R].” → sort B; for eacha, binary-search the count of B-elements in[L - a, R - a]using twobisectcalls. - “What if the array has 50 elements?” → 2^25 = 3 × 10⁷ — borderline. Memory at 8 bytes per sum is 256 MB. Need to drop to bitset or stream.
- “Subset-product instead of sum?” → enumerate products; the combine is identical.
- “k-th smallest subset sum across all subsets.” → k-way merge using a min-heap from sorted subset-sum lists per half.
Product Extension
Cryptographic key knapsack (Merkle–Hellman) and certain integer programming problems with ~40 binary variables: MITM is the textbook attack / solver. Portfolio optimization with a small basket of asset switches; molecular conformation enumeration. Whenever you have a “binary vector with cost”, N ≈ 40, and no obvious polynomial structure: MITM is the move.
Language/Runtime Follow-ups
- Python: list-comprehension enumeration as shown is clean. For tighter constants use
numpyto compute all subset sums via repeatedconcatenate(s, s + x). - Java:
long[] sumsA, sumsB.Arrays.sortandArrays.binarySearchare O(log) per call. Watch heap pressure at 2 × 10⁶ longs ≈ 16 MB. - Go:
sort.Slice(sumsB, ...);sort.SearchIntsfor binary search. - C++:
vector<long long>of size 2^20 = 8 MB each.std::sort,std::lower_bound. Idiomatic. - JS/TS:
Numberis safe to ±2⁵³; sums of ±4 × 10⁸ are tiny. Use plainArray+Array.prototype.sort((a, b) => a - b).
Common Bugs
- Splitting halves as
[:n/2]and[n/2:]but accidentally usingn // 2 + 1somewhere → mismatched sizes; you’ll miss subsets. - Forgetting to include 0 (empty subset) in either half — fix by initializing
sums = [0]. - Sorting only one half but binary-searching as if both are sorted, or vice versa.
- Initial
best = float('inf')is fine, but initialbest = abs(goal)is more honest about the empty-subset case. - After binary search, only checking
sumsB[idx]and missingsumsB[idx-1](the next-smaller) — the closest element can be on either side. - Using a
setinstead of sorted list — kills the binary search.
Debugging Strategy
For small N=4, enumerate all 16 subset sums by hand and verify the MITM result. Print sumsA, sumsB (sorted) and walk one binary search by hand. If the result is consistently too large by some |x|, you forgot to include 0; if too small, you’re double-counting (e.g., overlapping halves).
Mastery Criteria
- Recognized N=30..45 + arbitrary subset-sum constraint as the MITM trigger within 90 seconds.
- Wrote MITM closest-sum from scratch in <25 minutes.
- Stated time complexity O(N · 2^(N/2)) and space O(2^(N/2)) without prompting.
- Solved LC 1755 in <30 minutes from cold start.
- Solved one cousin (LC 956 — Tallest Billboard, with a twist; or “find subset with sum closest to half”) from cold start.
- Articulated MITM’s failure point (N ≥ 50 → memory and time both blow up).
Lab 10 — Z-Function & Manacher’s Algorithm
Goal
Implement two linear-time string scans. The Z-array: z[i] = length of the longest common prefix of s and s[i:] — “how far does the string match itself starting at i”. Manacher’s algorithm: the longest palindrome centered at every position, in O(n) exact. Apply Manacher to LC 5 (Longest Palindromic Substring) and Z to pattern matching and LC 214. Target: either scan from cold start in <12 minutes, both amortization arguments stated from memory.
Background Concepts
Z-box maintenance. The Z algorithm maintains the rightmost matched window [l, r): an interval where s[l..r-1] is already known to equal s[0..r-l-1]. When computing z[i] for i < r, position i mirrors position i − l in the prefix, so z[i] ≥ min(z[i−l], r−i) for free; only the part past r needs explicit comparison. Since r only ever advances, total explicit comparisons are O(n).
Relation to the KMP failure function (lab-05). fail[i] is “longest border looking backward at i”; z[i] is “prefix match looking forward from i”. They carry the same information and are interconvertible in O(n) in both directions. Z is often easier to reason about under pressure because z[i] is a direct statement about one suffix, not a chain of borders.
Manacher’s transform. Palindromes come in odd and even lengths — two center types. Transform s → t = '#' + '#'.join(s) + '#' (length 2n+1). Every palindrome in t is odd-length; rad[i] (radius in t) equals the length in s of the palindrome centered at transformed position i, and its start in s is (i − rad[i]) / 2. The mirror trick is the palindromic twin of the Z-box: inside the rightmost palindrome (c, r), position i inherits min(rad[2c−i], r−i) from its mirror 2c−i, then extends explicitly.
Interview Context
LC 5 is among the most frequently asked problems anywhere. The expected answer at most companies is expand-around-center O(n²); Manacher is the differentiator at L5+/quant/CP-flavored interviews — name it, state O(n), derive it if pushed. Z shows up wherever you need “prefix match at every position”: exact matching without hash-collision hand-waving, periodicity, string compression, LC 214. Interviewers who ask for Z usually want the amortization argument, not just the code.
Problem Statement (LC 5)
Given a string s, return the longest palindromic substring of s. Companion problem: given pattern and text, return all occurrence indices (generalized LC 28) — solve it with Z, no hashing.
Constraints
- 1 ≤ s.length ≤ 1000 (LC 5); the O(n) solutions generalize to 10⁶.
- s consists of digits and English letters (algorithm is alphabet-agnostic).
- For
search: pattern and text up to 10⁵ each; separator character must not occur in either.
Clarifying Questions
- If several longest palindromes exist, which one? (Any — LC 5 accepts all.)
- Is a single character a palindrome? (Yes — answer length ≥ 1 for nonempty s.)
- Substring or subsequence? (Substring, contiguous. Subsequence is LC 516 — a DP, different lab.)
- Case-sensitive? (Yes by default.)
- For search: overlapping occurrences reported? (Yes, all of them.)
Examples
longest_palindrome("babad") → "bab" (or "aba")
longest_palindrome("cbbd") → "bb" (even case — the transform earns its keep)
longest_palindrome("aaaa") → "aaaa"
search("aa", "aaaa") → [0, 1, 2]
z_function("aabxaayaab") → [10, 1, 0, 0, 2, 1, 0, 3, 1, 0]
manacher("abba") → [0, 1, 0, 1, 4, 1, 0, 1, 0] (radii over "#a#b#b#a#")
Initial Brute Force
Palindromes: for each of the 2n−1 centers, expand outward while characters match; keep the best. Z: for each i, compare s[i:] against s character by character.
Brute Force Complexity
Both O(n²) worst case ("aaa…a" makes every expansion run to the boundary), O(1) extra space. At n = 1000 (LC 5) expand-around-center passes comfortably; at n = 10⁵–10⁶ it does not. Naive matching is O(n·m).
Optimization Path
Palindromes: expand-around-center O(n²) → binary search on answer length + rolling hash O(n log n), probabilistic (lab-06) → Manacher O(n), exact. Matching: naive O(n·m) → KMP (lab-05) or Z, both O(n+m) — same information, different indexing; Z’s concatenation trick (pattern + sep + text) turns matching into a single array scan with no matcher state machine to get wrong.
Final Expected Approach
def z_function(s):
"""z[i] = length of the longest common prefix of s and s[i:]. z[0] = n."""
n = len(s)
z = [0] * n
if n:
z[0] = n
l, r = 0, 0 # z-box: rightmost window with s[l:r] == s[0:r-l]
for i in range(1, n):
if i < r:
z[i] = min(r - i, z[i - l]) # mirror value, clamped to the box
while i + z[i] < n and s[z[i]] == s[i + z[i]]:
z[i] += 1 # explicit extension past the box
if i + z[i] > r:
l, r = i, i + z[i] # box only ever advances
return z
def search(pattern, text):
"""All start indices of pattern in text. O(n + m)."""
m = len(pattern)
if m == 0:
return list(range(len(text) + 1))
z = z_function(pattern + '\x00' + text)
return [i - m - 1 for i in range(m + 1, len(z)) if z[i] >= m]
def manacher(s):
"""Radii over t = '#' + '#'.join(s) + '#'. rad[i] equals the length (in s)
of the longest palindrome centered at transformed position i."""
t = '#' + '#'.join(s) + '#'
n = len(t)
rad = [0] * n
c, r = 0, 0 # center / right edge of rightmost palindrome
for i in range(n):
if i < r:
rad[i] = min(r - i, rad[2 * c - i]) # mirror trick
while (i - rad[i] - 1 >= 0 and i + rad[i] + 1 < n
and t[i - rad[i] - 1] == t[i + rad[i] + 1]):
rad[i] += 1
if i + rad[i] > r:
c, r = i, i + rad[i]
return rad
def longest_palindrome(s):
"""LC 5. O(n)."""
if not s:
return ""
rad = manacher(s)
i = max(range(len(rad)), key=lambda k: rad[k])
start = (i - rad[i]) // 2 # transformed index → s index
return s[start:start + rad[i]]
Data Structures Used
z/rad— int arrays over the (transformed) string.- Two maintained windows: z-box
[l, r), rightmost palindrome(c, r). Nothing else — that is the point.
Correctness Argument
Z invariant: [l, r) is a window with s[l..r-1] == s[0..r-l-1] and maximal r seen so far. For i < r, the mirror gives z[i] ≥ min(z[i−l], r−i). If z[i−l] < r−i, the value is exact: a longer match at i would extend the mirror’s match inside the known-equal window — contradiction. If z[i−l] ≥ r−i, everything up to r matches but beyond r is unknown — extend explicitly. Either way z[i] is correct by induction.
Manacher: identical structure with palindromic mirroring — inside palindrome (c, r), the palindrome at i reflects the one at 2c−i; strictly-inside radii are exact, boundary-touching radii are extended explicitly. The '#' sentinels make every palindrome in t odd-length, so one center type covers both parities of s; the parity bookkeeping is free because a radius that “ends on a letter” is impossible in t.
Linearity (both): every successful explicit comparison advances r, and r never decreases → ≤ n successes total; each position adds at most one failed comparison → O(n).
Complexity
| Operation | Time | Space |
|---|---|---|
| z_function | O(n) | O(n) |
| search | O(n + m) | O(n + m) |
| manacher / longest_palindrome | O(n) | O(n) |
Implementation Requirements
- State the
z[0]convention explicitly (here:z[0] = n; some templates leave 0 — either is fine, mixing them is not). searchmust use a separator that cannot occur in pattern or text ('\x00'for printable input).manacherreturns radii over the transformed string; the caller maps back withstart = (i − rad[i]) // 2.- No recursion, single pass each, no per-iteration slicing.
Tests
- Fixed:
longest_palindromeon"babad","cbbd","a","","aaaa". search("sad", "sadbutsad") == [0, 6]; overlapping:search("aa", "aaaa") == [0, 1, 2]; miss →[].- Hand-checked arrays:
z_function("aabxaayaab") == [10,1,0,0,2,1,0,3,1,0],z_function("aaaaa") == [5,4,3,2,1],manacher("abba") == [0,1,0,1,4,1,0,1,0]. - Stress: random strings over {a,b} and {a,b,c}, z vs char-by-char brute force, palindrome length vs O(n³) brute force,
searchvs slicing scan. (All of the above verified against brute force before this lab shipped.)
Follow-up Questions
- Convert z[] to fail[] and back. → Both O(n). Z→fail: for each i with z[i] > 0 set
fail[i+z[i]−1] = max(·, z[i]), then sweep right-to-left withfail[i] = max(fail[i], fail[i+1]−1). Fail→Z is symmetric. Knowing this means you never need both memorized under pressure. - Why is the while-loop amortized O(n)? → Each successful comparison pushes
rforward andris monotone non-decreasing, bounded by n; failures are ≤ 1 per position. Potential function: r itself. - Shortest Palindrome (LC 214). → Longest palindromic prefix of s. Via Z: build
t = s + '#' + reverse(s); the prefix of length k is palindromic iffz[len(t)−k] == k; take the max k, prependreverse(s[k:]). Via KMP:fail[-1]of the same concatenation. Same trick, two dialects. - Count distinct palindromic substrings. → Manacher counts palindromes with multiplicity (Σ⌈rad[i]/2⌉ over t solves LC 647). Distinct needs the palindromic tree (Eertree): O(n) nodes because a string has ≤ n+1 distinct palindromic substrings — one new node per appended character. Name-drop it; building one live is beyond interview scope.
- String periodicity. → p is a period of s iff
z[p] == n − p(the shifted suffix matches the prefix exactly). Smallest period: scan p = 1, 2, … Via the failure function the smallest period isn − fail[n−1]directly; s is an exact repetition iff that divides n (LC 459). Both characterizations are equivalent by border–period duality. - Z vs rolling hash for matching? → Z is exact and branch-predictable; hashing (lab-06) handles multiple patterns of the same length and 2-D cases but needs collision care. Prefer Z when one pattern, exact answer.
Product Extension
Genomics: restriction-enzyme sites and hairpin structures are (reverse-complement) palindromes — Manacher over a complement-mapped alphabet finds all maximal ones per chromosome in one linear pass, 10⁸+ characters. Log-pipeline dedup: Z-arrays of template + '\x00' + line classify millions of log lines against extracted templates without regex backtracking blowups. LZ77-style compressors compute Z-like longest-previous-match arrays as their core primitive.
Language/Runtime Follow-ups
- Python: never slice inside the loop —
s[z[i]] == s[i + z[i]]is O(1),s[i:].startswith(...)is O(n) per call and silently reintroduces O(n²). For 10⁶ chars, convert tobytesand index ints. - C++: the canonical CP versions;
std::string+vector<int>. Some templates use sentinels'^'/'$'around t to delete the bounds checks — measurable at 10⁷. - Java:
s.charAtper comparison is fine post-JIT; convert tochar[]for hot loops. - Go: index
[]bytedirectly; avoidstrings.Builderin the transform for large n — preallocate. - JS/TS:
charCodeAtbeats string equality on V8; typedInt32Arrayforz/rad.
Common Bugs
- Unclamped mirror copy: taking
z[i] = z[i−l]withoutmin(·, r−i)— beyond the box nothing is known; produces overcounts on inputs like"abababab". - Updating the box unconditionally: writing
l, r = i, i+z[i]even wheni+z[i] ≤ rshrinks the box. Answers stay correct (it is still a valid match window) but linearity dies — the classic “my Z is O(n²) on aaaa…” bug. - Skipping the transform: running odd-only expansion on raw
ssilently misses every even palindrome ("cbbd" → "c"instead of"bb"). - Wrong map-back: the start in s is
(i − rad[i]) // 2, not(i − rad[i] − 1) // 2— off-by-one here returns a string one char wide of the palindrome. - Separator inside the alphabet: joining with
'#'insearchwhen the text may contain'#'lets matches straddle the boundary. Use'\x00'or a length guard. - z[0] convention drift: template A sets
z[0] = n, template B leaves 0; pasting the search loop from B under the array from A (or vice versa) is fine only because search skips index 0 — until someone iterates from 0.
Debugging Strategy
Hand-compute z for a 10-char string and diff ("aabxaayaab" → [10,1,0,0,2,1,0,3,1,0]). Instrument the loop to print (i, l, r, z[i]): r must be non-decreasing across the entire run — if it ever drops, bug 2 is present. For Manacher, print t with rad aligned beneath it and eyeball the peak: for "abba", the 4 must sit on the middle '#' at index 4. If the peak sits on a letter for an even palindrome, the transform is wrong.
Mastery Criteria
-
Wrote
z_functionfrom cold start in <8 minutes, correct on"aaaaa"and"abababab"first run. -
Wrote
manacher+longest_palindromein <12 minutes, even palindromes included. - Solved LC 5, LC 214, LC 647 from cold start.
-
Computed
zfor an unfamiliar 10-char string by hand in <2 minutes. - Stated the r-monotonicity amortization argument without prompting.
-
Stated the period test both ways (z-based scan;
n − fail[n−1]via lab-05) and when each applies.
Lab 11 — LCA via Binary Lifting
Goal
Answer lowest-common-ancestor queries on a rooted tree in O(log n) after O(n log n) preprocessing: build the jump table up[k][v] = 2^k-th ancestor, then implement lca(u, v), kth_ancestor(v, k) (LC 1483 verbatim), and dist(u, v). Extend the table to path aggregates (max edge on path). Target: full class from cold start in <15 minutes.
Background Concepts
Jump table. up[0][v] = parent of v; up[k][v] = up[k−1][up[k−1][v]] — the 2^k-th ancestor is the 2^(k−1)-th ancestor of the 2^(k−1)-th ancestor. LOG = ⌈log₂ n⌉ rows, filled row by row after one BFS/DFS computes depth and parents. Any k-step walk decomposes into the binary expansion of k: ≤ LOG jumps.
LCA query. Lift the deeper node by depth[u] − depth[v] to equalize depths. If they now coincide, done. Otherwise lift both simultaneously, largest jumps first, taking a jump only when it does not land them on the same node — this keeps both strictly below the LCA; after all bits, both are children of the LCA.
Alternatives. Euler tour + sparse-table RMQ: LCA(u,v) = shallowest node in the Euler tour between the first occurrences of u and v — O(n log n) build, O(1) query via lab-04’s sparse table. Farach-Colton–Bender exploits that adjacent Euler depths differ by ±1 to reach O(n)/O(1). Offline: Tarjan’s algorithm answers all queries in one DFS with a DSU, O(n·α(n) + q). Binary lifting is the interview default: simplest to write, and it is the only one that also gives kth_ancestor and path aggregates for free.
Interview Context
LC 236 (LCA of a Binary Tree) is the toy: one query, O(n) recursion, everyone’s seen it. The real interview escalation is “now answer q = 2×10⁵ queries” — per-query O(n) walks become 4×10¹⁰ operations on an adversarial chain. Binary lifting is the expected answer at L5+; LC 1483 (Kth Ancestor of a Tree Node) is the canonical LeetCode instance and is exactly the jump table with no LCA logic at all. Path-aggregate variants (max edge between u and v) appear in system-flavored rounds via MST verification.
Problem Statement (LC 1483)
Given a tree of n nodes, node 0 as root, and parent[i] for each node, implement TreeAncestor(n, parent) with getKthAncestor(node, k) returning the k-th ancestor or −1. Generalized target: on the same tree, answer up to 2×10⁵ mixed queries lca(u, v) and dist(u, v).
Constraints
- 1 ≤ n ≤ 2×10⁵ nodes; up to 2×10⁵ queries (LC 1483: 5×10⁴ each — same regime).
- 0 ≤ k < n; tree given as parent array or adjacency list; static (no structural updates).
- Recursion is a liability: a chain of 2×10⁵ overflows Python’s default stack.
Clarifying Questions
- Is the tree rooted? (Yes, at node 0; if unrooted, root it arbitrarily — LCA answers depend on the root.)
- One query or many? (Up to 2×10⁵ — preprocessing amortizes; for a single query, walk parents.)
- Weighted edges? (Unweighted here; for weighted
dist, keep prefix sums root→v and use the same formula.) - Static tree? (Yes. Insertions/deletions between queries need link-cut trees — out of scope.)
- Are u and v guaranteed valid, possibly equal? (Yes;
lca(u, u) = umust hold.)
Examples
parent = [-1, 0, 0, 1, 1, 2, 2] (perfect binary tree on 7 nodes)
kth_ancestor(3, 1) → 1 kth_ancestor(5, 2) → 0 kth_ancestor(6, 3) → -1
lca(3, 4) → 1 lca(3, 5) → 0 lca(1, 1) → 1
dist(3, 5) → 4 dist(0, 6) → 2
Initial Brute Force
Store only parent and depth. Per query: walk the deeper node up until depths match, then step both up in lockstep until they meet. kth_ancestor: k single steps.
Brute Force Complexity
O(depth) per query, O(n) worst case on a chain → O(n·q) = 4×10¹⁰ at n = q = 2×10⁵. Fine for one query (this is the right answer for LC 236); dead for many.
Optimization Path
Naive walk O(n)/query → binary lifting O(n log n) build, O(log n) query → Euler tour + sparse table O(n log n)/O(1) (lab-04) → Farach-Colton–Bender O(n)/O(1) → Tarjan offline O(n·α(n) + q). In interviews, lifting wins on code-length-to-power ratio; mention the O(1)-query reductions to show you know the landscape.
Final Expected Approach
from collections import deque
class LCA:
"""Binary lifting over a rooted tree. O(n log n) build, O(log n) query."""
def __init__(self, n, adj, root=0):
self.LOG = max(1, (n - 1).bit_length())
self.depth = [0] * n
up = [[-1] * n for _ in range(self.LOG)]
seen = [False] * n
seen[root] = True
dq = deque([root]) # iterative BFS: no recursion limit
while dq:
v = dq.popleft()
for w in adj[v]:
if not seen[w]:
seen[w] = True
up[0][w] = v
self.depth[w] = self.depth[v] + 1
dq.append(w)
for k in range(1, self.LOG): # up[k][v] = 2^k-th ancestor
upk, prev = up[k], up[k - 1]
for v in range(n):
mid = prev[v]
upk[v] = -1 if mid == -1 else prev[mid] # guard: -1 wraps in Python!
self.up = up
def kth_ancestor(self, v, k):
"""Binary expansion of k. Returns -1 if the walk leaves the tree."""
if k > self.depth[v]:
return -1
b = 0
while k:
if k & 1:
v = self.up[b][v]
k >>= 1
b += 1
return v
def lca(self, u, v):
if self.depth[u] < self.depth[v]:
u, v = v, u
u = self.kth_ancestor(u, self.depth[u] - self.depth[v]) # equalize depths
if u == v:
return u
for b in range(self.LOG - 1, -1, -1): # descend: largest jumps first
if self.up[b][u] != self.up[b][v]:
u, v = self.up[b][u], self.up[b][v]
return self.up[0][u]
def dist(self, u, v):
return self.depth[u] + self.depth[v] - 2 * self.depth[self.lca(u, v)]
Data Structures Used
up— LOG × n int table (the sparse “jump pointers”; ~n log n ints, the dominant memory cost).depth— int array; BFS queue during build.- For path aggregates: a parallel
mx[k][v]table of the max edge weight on each 2^k jump.
Correctness Argument
Table: by induction on k — up[k][v] composes two correct 2^(k−1) jumps, and 2^(k−1) + 2^(k−1) = 2^k. The −1 guard preserves “walked off the root” through compositions.
kth_ancestor: the depth check rejects k > depth(v); otherwise k ≤ depth(v) < 2^LOG, so every set bit of k has a table row, and the jumps compose to exactly k steps regardless of bit order (ancestor composition is associative).
lca: after equalization, u and v are at equal depth. Invariant of the descending loop: u ≠ v and both lie strictly below their LCA. A jump of 2^b is taken only when up[b][u] ≠ up[b][v] — two distinct nodes at equal depth cannot both be ancestors of anything shared, so the jump keeps both strictly below the LCA. Descending bit order is what makes the greedy exact: after considering bit b, the remaining distance to the LCA’s children is < 2^b (else the b-jump would have been taken). After bit 0, the distance is 0: both nodes are children of the LCA, and up[0][u] is the answer.
Complexity
| Operation | Time | Space |
|---|---|---|
| Build (BFS + table) | O(n log n) | O(n log n) |
| kth_ancestor / lca / dist | O(log n) | — |
| q queries total | O((n + q) log n) | O(n log n) |
Implementation Requirements
- Iterative BFS/DFS only — a 2×10⁵ chain kills recursive DFS in Python and default-stack C++.
LOG = max(1, (n−1).bit_length())— handles n = 1 and exact powers of two.- Explicit −1 guard when composing the table (Python’s
list[-1]is the last element, not an error). lca(u, u) == uandlca(root, v) == rootmust fall out without special-casing.- For LC 1483 submit only
__init__(from parent array) +getKthAncestor.
Tests
- Fixed 7-node perfect binary tree — every pair hand-checkable (see Examples).
- Chain of 2000:
lca(n−1, n/2) == n/2,kth_ancestor(n−1, n−1) == 0,kth_ancestor(n−1, n) == −1,dist(0, n−1) == n−1. - Star:
lcaof any two leaves is the center. - n = 1:
lca(0,0) == 0,kth_ancestor(0, 1) == −1. - Stress: 300 random trees (≤ 64 nodes), every query checked against parent-walking brute force and BFS distance. (All verified before this lab shipped.)
- Perf: chain of 2×10⁵ + 2×10⁵ random queries — must finish in seconds, not minutes.
Follow-up Questions
- Reduce LCA to RMQ and back. → Euler tour (2n−1 entries) turns LCA(u,v) into “index of minimum depth between first[u] and first[v]” — a lab-04 sparse table gives O(1) queries. Adjacent tour depths differ by ±1, which Farach-Colton–Bender exploits (√-sized blocks, precomputed in-block patterns) for true O(n)/O(1). The reduction also runs backwards: any RMQ instance becomes LCA on a Cartesian tree.
- Max edge on the path u–v (MST verification / bottleneck routing)? → Augment:
mx[k][v]= max edge on v’s 2^k jump, composed withmaxexactly asupcomposes.path_max(u,v)= max over the jumps both lifts take. An MST is optimal iff every non-tree edge (u,v,w) has w ≥ path_max(u,v). Works for any idempotent/associative aggregate (min, max, gcd, bitwise-or) — not sums under updates. - Why does lifting beat naive parent-walking? → Adversarial shape: a chain with queries pairing the two ends — naive is Θ(n) per query, Θ(nq) total; lifting caps every query at ⌈log₂ n⌉ ≈ 18 jumps. The table costs 18× the parent array in memory — the classic time/space trade.
- Jump pointers on functional graphs? → LC 1483 never uses “tree-ness”: any graph where each node has exactly one out-edge (successor function) admits the same table. k-th successor, cycle entry detection, “where is chip x after 10⁹ steps” — all the same doubling. This is also how sparse tables on permutations work.
- When is lifting not enough? → Path updates (“add w to every edge on u–v, then query”). The table is a baked snapshot; rebuilding is O(n log n) per update. Heavy-light decomposition (phase-12 lab-03) maps paths to O(log n) segment-tree ranges and handles updates in O(log² n). Rule of thumb: static path queries → lifting; dynamic → HLD.
- Memory too tight at n = 10⁶? → Euler tour + sparse table stores the same O(n log n) but over 2n entries of the tour; the true fix is FCB O(n) or an ancestor-labeling scheme; or answer offline with Tarjan + DSU in O(n·α(n)).
Product Extension
Org-chart services answer “nearest common manager” and “reporting distance” for pairs of employees — exactly lca/dist with the CEO as root, precomputed nightly. File systems and DOM trees use LCA for “closest common container” (permission inheritance, event delegation). Network engineering: on a spanning tree of the topology, bottleneck bandwidth between two hosts is the path-min aggregate — the mx-table variant with min. Phylogenetics: LCA = most recent common ancestor, dist = evolutionary distance; biology databases precompute exactly this table.
Language/Runtime Follow-ups
- Python: lay the table out as LOG lists of n ints (as above), not n lists of LOG — row-major matches the fill loop and halves cache misses; PyPy or
sys.setrecursionlimitnever fixes what iteration fixes for free. - C++:
vector<array<int,18>>or a flatint up[LOG][N]; watch stack size if you insist on recursive DFS. - Java:
int[][] upfine; bewareArrayDequeautoboxing — useint[]as an explicit stack. - Go: slices of slices are fine; BFS with an index pointer into a preallocated slice beats container/list.
- JS/TS:
Int32Arrayrows; recursion on 2×10⁵ chain overflows V8’s stack — iterate.
Common Bugs
- −1 wraparound:
prev[-1]in Python silently reads the last node, corrupting the table with plausible-looking ancestors. Guardmid == -1explicitly; in C++/Java the same bug is an index crash (kinder). - Ascending bits in the LCA loop: the greedy “jump iff ancestors differ” is only exact largest-first; ascending order can strand the nodes short of the LCA. (kth_ancestor, by contrast, is order-independent.)
- Missing depth equalization: calling the bit loop on nodes of unequal depth returns a node that is an ancestor of neither.
- Returning before the
u == vcheck: when one node is the ancestor of the other, equalization already lands on the LCA; falling through toup[0][u]returns its parent — off by one level. - LOG off-by-one:
n.bit_length()vs(n−1).bit_length()— for n an exact power of two the former wastes a row, the latter is tight; too small a LOG silently caps jumps and fails only on deep chains. - Recursive DFS at n = 2×10⁵: RecursionError in Python, stack overflow elsewhere; always build iteratively.
Debugging Strategy
Build the table for an 8-node chain 0←1←…←7 and print it: row k must read “shift by 2^k, −1 past the root” (up[1][5] == 3, up[2][5] == 1, up[2][3] == -1). If any row disagrees, the composition loop or the −1 guard is wrong. Then trace lca(7, 4) by hand: equalize (7→4… depths 7 vs 4, lift 7 by 3 → node 4), equal → return 4. For a branching failure, print (b, u, v) per loop iteration — u and v must stay at equal depth and never coincide inside the loop; if they ever become equal mid-loop, bug 2 or 4.
Mastery Criteria
- Wrote the full class (build + lca + kth_ancestor + dist) from cold start in <15 minutes.
- Solved LC 1483 and LC 236 from cold start; stated why LC 236 alone does not need lifting.
- Augmented the table with path-max on paper in <5 minutes and stated the MST-verification use.
- Stated the Euler-tour RMQ reduction and the O(n)/O(1) name (Farach-Colton–Bender).
- Explained descending-bit necessity in the LCA loop without prompting.
- Named the boundary where lifting yields to HLD (path updates).
Lab 12 — Sqrt Decomposition & Mo’s Algorithm
Goal
Answer offline range queries in O((n + q)·√n) when the aggregate does not fit a segment tree — the canonical case: “number of distinct values in a[l..r]”. Warm-up: block decomposition for point-update/range-sum. Main event: Mo’s algorithm — sort queries into block order, slide a window with O(1) add/remove. Target: Mo’s from cold start in <20 minutes, movement bound derived on a whiteboard.
Background Concepts
Block decomposition (warm-up). Split the array into blocks of size B ≈ √n; keep one aggregate per block. Point update touches one block: O(1). Range query = two partial blocks scanned element-wise + whole blocks read from the per-block table: O(B + n/B), minimized at B = √n → O(√n). It is a segment tree with exactly two levels — worse asymptotics, trivial code, and it tolerates any per-block precomputation (sorted copies for “count < x in range”, etc.).
Mo’s algorithm. Some aggregates are easy to maintain incrementally but impossible to merge: distinct-count has no combine(left_half, right_half). Mo’s exploits incrementality: maintain a current window [cl, cr] with add(i)/remove(i) in O(1), and answer queries offline in an order that keeps total pointer travel small. Sort queries by (block of l, then r — with r’s direction alternating per block, “snake order”). Movement bound: within one l-block, r is monotone → r travels ≤ n per block, n/B blocks → O(n²/B); l wiggles ≤ B per query plus B per block change → O(qB + n). At B = √n both terms are O((n+q)√n); at B = n/√q the total tightens to O(n√q).
Interview Context
This is a competitive-programming staple that leaks into interviews at HFT shops and infra teams as “design a batch analytics query engine”. The signal being tested: recognizing that distinct-count is not decomposable (a segment tree node cannot summarize it), and that “offline” is a license to reorder work. Saying “Mo’s algorithm, O((n+q)√n), needs all queries upfront” plus the sort trick is the entire expected answer; the code is secondary. Contrast question interviewers love: LC 1157 (Online Majority Element) looks adjacent but is online — different tool entirely (segment tree over Boyer-Moore candidates + binary-searched occurrence lists).
Problem Statement
Given an array a of n integers and q queries (l, r), report the number of distinct values in a[l..r] for each. Canonical judge: SPOJ DQUERY (n ≤ 3×10⁴, q ≤ 2×10⁵). There is no exact LeetCode equivalent — LeetCode’s range-query problems are online or decomposable; say so honestly rather than forcing a bad analogue.
Constraints
- n ≤ 3×10⁴, q ≤ 2×10⁵ (DQUERY); the technique scales to n, q ≈ 10⁵–10⁶ in compiled languages.
- Values up to 10⁶ — coordinate-compress so
cntis an array. - Queries known upfront (offline); no updates interleaved.
- Ranges inclusive; DQUERY is 1-indexed (convert at the boundary).
Clarifying Questions
- Online or offline? (Offline — all queries available before answering. Mo’s is unusable online.)
- Updates between queries? (No. With updates → Mo with time dimension, O(n^{5/3}) — see follow-ups.)
- Value range? (Up to 10⁶ — compress to [0, n) so the count table is a flat array.)
- Inclusive ranges? 0- or 1-indexed? (Inclusive; this lab is 0-indexed, DQUERY input is 1-indexed.)
- Answer order? (Answers must be emitted in the original query order — Mo reorders internally only.)
Examples
a = [1, 1, 2, 1, 3]
query(0, 4) → 3 (values {1, 2, 3})
query(1, 3) → 2 (values {1, 2}) ← DQUERY sample, 0-indexed
Warm-up: a = [3, 1, 4, 1, 5], sum(1, 3) → 6, update(2, 0), sum(1, 3) → 2
Initial Brute Force
Per query, scan the range and build a set; answer is its size.
Brute Force Complexity
O(n) per query → O(n·q) = 6×10⁹ element visits at DQUERY limits, each with a hash insert. TLE by two orders of magnitude. Space O(n) per query for the set.
Optimization Path
Brute force → try a segment tree and fail: distinct-count does not merge (knowing each half has 3 distinct values says nothing about the union) → Mo’s O((n+q)√n) with O(1) add/remove — the general tool whenever the window aggregate is incremental → for distinct-count specifically, an offline BIT sorted by r (“keep a 1 only at each value’s last occurrence ≤ r”) achieves O((n+q) log n) and beats Mo — know it exists; Mo remains the answer for non-BIT-able aggregates (mode, count of pairs, XOR-frequency queries) → online requirement → persistent segment tree (see follow-ups).
Final Expected Approach
from math import isqrt
class SqrtBlocks:
"""Warm-up: point update, range sum. O(1) update, O(√n) query."""
def __init__(self, a):
self.a = list(a)
self.b = isqrt(len(a)) + 1
self.block = [0] * ((len(a) + self.b - 1) // self.b)
for i, x in enumerate(a):
self.block[i // self.b] += x
def update(self, i, x): # a[i] = x
self.block[i // self.b] += x - self.a[i]
self.a[i] = x
def query(self, l, r): # sum of a[l..r], inclusive
s = 0
while l <= r and l % self.b: # left partial block
s += self.a[l]; l += 1
while l + self.b - 1 <= r: # whole blocks
s += self.block[l // self.b]; l += self.b
while l <= r: # right partial block
s += self.a[l]; l += 1
return s
def mo_distinct(a, queries):
"""queries: (l, r) inclusive, 0-indexed. Offline. O((n + q)·√n)."""
n, q = len(a), len(queries)
if q == 0:
return []
B = max(1, int(n / q ** 0.5)) # block size ≈ n / √q
def key(i):
l, r = queries[i]
blk = l // B
return (blk, r if blk % 2 == 0 else -r) # snake order: r alternates
order = sorted(range(q), key=key)
comp = {v: i for i, v in enumerate(sorted(set(a)))}
va = [comp[x] for x in a] # coordinate-compress values
cnt = [0] * len(comp)
distinct = 0
ans = [0] * q
cl, cr = 0, -1 # current window [cl, cr], empty
for qi in order:
l, r = queries[qi]
while cl > l: # expand before shrink
cl -= 1
cnt[va[cl]] += 1
if cnt[va[cl]] == 1: distinct += 1
while cr < r:
cr += 1
cnt[va[cr]] += 1
if cnt[va[cr]] == 1: distinct += 1
while cl < l:
cnt[va[cl]] -= 1
if cnt[va[cl]] == 0: distinct -= 1
cl += 1
while cr > r:
cnt[va[cr]] -= 1
if cnt[va[cr]] == 0: distinct -= 1
cr -= 1
ans[qi] = distinct
return ans
Data Structures Used
block— per-block aggregate array (warm-up), size ⌈n/B⌉.cnt— multiplicity table over compressed values;distinct— one running int. The whole “state” of Mo’s is these two.order— the permutation of query indices; answers written back by original index.
Correctness Argument
Blocks: block[j] = Σ of its elements is maintained exactly by each update (delta applied to one block); a query partitions [l, r] into ≤ 2 partial blocks (scanned directly) and whole blocks (read from block) — a disjoint exact cover, so the sum is exact.
Mo’s: invariant — after the four while-loops, cnt[x] is the multiplicity of x in a[cl..cr] and distinct = |{x : cnt[x] > 0}|, with (cl, cr) = (l, r). Each add/remove changes one multiplicity by 1 and adjusts distinct exactly at the 0↔1 boundary, so the invariant is preserved per step; the loops terminate with the window equal to the query. Correctness is independent of the query order — the sort affects only running time. Expanding (cl↓, cr↑) before shrinking guarantees the window never passes through an inverted state, so no count goes negative.
Movement bound: stated in Background; the key observation is that r resets at most once per l-block (not per query), and snake order removes even those resets.
Complexity
| Operation | Time | Space |
|---|---|---|
| SqrtBlocks update / query | O(1) / O(√n) | O(n) |
| Mo’s sort | O(q log q) | O(q) |
| Mo’s pointer movement | O((n + q)·√n) total (O(n√q) with B = n/√q) | O(n) |
Implementation Requirements
cntmust be a flat array over compressed values — a dict is correct but ~5× slower in Python and often the difference between AC and TLE.- Block size clamped:
max(1, int(n / √q))(q ≫ n² would otherwise give B = 0 → division by zero in the sort key). - Answers stored by original query index; never emit in sorted order.
- Pointer discipline: exactly the four loops, expand-before-shrink, window inclusive
[cl, cr]initialized to (0, −1). - Warm-up struct: no lazy anything — that is the point of the exercise; it is 20 lines.
Tests
- DQUERY sample: a = [1,1,2,1,3], queries (0,4) → 3, (1,3) → 2.
- Single element, empty query list, all-equal array (every answer 1), all-distinct array (answer = r−l+1).
- Degenerate blocks: q ≫ n² (B clamps to 1); q = 1 (B = n).
- SqrtBlocks: interleaved random updates/queries vs
sum(a[l:r+1]), including negative values. - Stress: 200 random arrays (n ≤ 80, values ≤ 12), all Mo answers vs
len(set(a[l:r+1])). (All of the above verified against brute force before this lab shipped.) - Perf: n = 3×10⁴, q = 2×10⁵ random queries — count total pointer steps, confirm ≈ (n+q)√n, not n·q.
Follow-up Questions
- Why does snake order (alternating r direction) help? → With plain (block, r ascending) sorting, r crashes from ~n back to ~1 at every block boundary — n/B resets ≈ √q · n extra steps. Alternating r’s direction per block turns the reset into a continuation; r sweeps up-then-down like a snake. Same asymptotics, roughly 2× fewer moves in practice — often the AC/TLE line.
- Queries interleaved with point updates? → Mo with a time dimension: each query is (l, r, t) where t = number of updates before it; sort by (l-block, r-block, t) with block size n^{2/3}; three pointers (cl, cr, time), where moving time applies/rolls back one update. Total O(n^{5/3}) for n ≈ q. Painful but mechanical.
- When does a segment tree or BIT win? → Whenever queries are online, updates are frequent, or the aggregate is decomposable (sum, min, max, gcd — anything with a merge). Mo’s is strictly a last resort for incremental-but-not-mergeable aggregates on offline queries. For distinct-count itself, the offline sort-by-r BIT trick (a 1 at each value’s last occurrence) is O((n+q) log n) — faster than Mo; Mo earns its keep on mode, “count pairs with equal values”, and friends.
- Hilbert curve ordering? → Map each query (l, r) to its position d(l, r) on a Hilbert curve over the n×n grid and sort by d alone. Locality in d implies locality in both coordinates simultaneously — provably O(n√q) movement without tuning B, and measurably faster than block sort at large q. Standard drop-in in CP templates.
- Same query, but online? → Persistent segment tree: version r of the tree holds a 1 at the last occurrence ≤ r of each value; distinct(l, r) = range-sum of [l, r] in version r. O(log n) per query, O(n log n) space, fully online — persistent structures are covered in phase-12. This is the standard escape hatch when the interviewer removes the “offline” concession.
- Why B = n/√q instead of √n? → Balancing the two movement terms n²/B (r-travel) and qB (l-travel) gives B = n/√q, total O(n√q). At q ≈ n they coincide at √n; at q ≫ n the tuned block is smaller and wins. Know the derivation — it is a one-line calculus argument interviewers enjoy.
Product Extension
Batch analytics is offline by nature: “distinct users per session window” over yesterday’s event log, thousands of ad-hoc time ranges — exactly Mo’s shape, and the reorder-for-locality idea is the same one behind sorting scan ranges in columnar engines. Exact distinct-counts feed A/B dashboards where HyperLogLog’s ±2% is unacceptable (billing, fraud thresholds). The sqrt-block warm-up is the mental model for time-series downsampling tiers (raw + per-block rollups) in monitoring systems.
Language/Runtime Follow-ups
- Python: inline the add/remove logic into the loops (as above) — closures with
nonlocalcost ~2× in the hot path; use lists, not defaultdict; at DQUERY limits CPython is marginal, PyPy comfortable. - C++: Mo’s home turf —
sortwith a comparator,int cnt[MAX], easily 10⁷+ pointer moves/sec. Useinline void add(int i). - Java: sort an
Integer[]of indices or a primitive index array with a manual sort; avoidHashMapfor cnt at all costs. - Go:
sort.Sliceon the index slice; flat[]intfor cnt; closures are cheap enough here. - JS/TS:
Int32Arrayfor cnt and va; numeric sort comparator(x, y) => keyX - keyYcarefully — default sort is lexicographic.
Common Bugs
- Shrink before expand: reordering the four while-loops can drive the window through an inverted state (cl > cr+1), sending counts negative and
distinctpermanently wrong. Expand (cl↓, cr↑) first, always. - Answers in sorted order: writing
ans.append(distinct)instead ofans[qi] = distinct— output silently permuted; passes single-query tests, fails everything real. - Block size zero:
int(n / √q)truncates to 0 when q > n² — every l lands in “block 0/0” → ZeroDivisionError or a degenerate sort. Clamp withmax(1, ·). - Dict instead of compressed array: correct output, 5–10× slowdown in Python — the classic “my algorithm is right but TLEs” report.
- Inclusive/exclusive drift: window is inclusive
[cl, cr]starting at (0, −1); mixing in half-open query ranges off-by-ones every answer by ±1 on boundary elements. - Sorting by (l, r) without blocks: looks plausible, degrades to O(n·q) pointer travel on adversarial queries — the entire algorithm is the sort order.
Debugging Strategy
First verify the window invariant in isolation: run Mo’s with queries in given order (no sort) on a 10-element array and diff every answer against len(set(a[l:r+1])) — if that fails, the bug is in add/remove or loop order, not the sort. Then re-enable the sort and diff again — a newly appearing mismatch means bug 2 (answer indexing). For performance, count pointer steps with a counter and print it: at n = 3×10⁴, q = 2×10⁵ expect ~10⁷ (≈ (n+q)√n), not ~10⁹; a 100× excess means the sort key is wrong (bug 6) or B is degenerate (bug 3). For SqrtBlocks, test l and r inside the same block — the whole-block loop must not fire.
Mastery Criteria
-
Wrote
mo_distinctfrom cold start in <20 minutes, correct on the DQUERY sample first run. - Derived the O((n+q)√n) movement bound on a whiteboard, including the per-block r argument.
- Stated why distinct-count cannot live in a segment tree (no merge), unprompted.
- Wrote the sqrt-block struct in <8 minutes.
- Given a fresh range-query problem, chose correctly between BIT/segment tree, Mo’s, and persistence in <1 minute (online? updates? mergeable?).
- Named the tuned block size n/√q and the snake/Hilbert orderings as constant-factor upgrades.
Lab 13 — Treap & Ordered-Set Operations
Goal
Implement a treap — the one balanced BST you can realistically write in an interview. Build it from exactly two primitives, split and merge, then compose insert, delete, k-th smallest, rank, and count-less-than on top. Understand the implicit treap (position-keyed) for array cut/paste/reverse. Apply it to LC 315. After this lab: split + merge from memory in <10 minutes, the full ordered set in <25.
Background Concepts
A treap stores a key (BST-ordered) and a priority (max-heap-ordered) per node. For distinct priorities the shape is unique: the max-priority node is the root, recurse on both sides. That shape is exactly what you’d get by inserting keys into a plain BST in decreasing-priority order — so random priorities simulate random insertion order, giving expected depth O(log n) regardless of key order.
Everything reduces to two primitives:
split(t, key)→(left, right)whereleftholds keys < key,rightholds keys ≥ key.merge(a, b)→ one treap, precondition: every key ina≤ every key inb.
Insert = split + merge in a new node. Erase = unlink + merge children. Order statistics (k-th, rank) come from an augmented size field maintained by a pull() after every child mutation.
Implicit treap: drop explicit keys; a node’s “key” is its position, computed implicitly as size(left subtree) along the search path. Split by size instead of key. Now you can cut out any array range in O(log n), paste it elsewhere, or reverse it with a lazy flip flag — a rope.
Why this matters: C++ has std::set plus pbds order-statistics trees; Java has TreeMap (no rank); Python has no ordered set at all — sortedcontainers is third-party and banned on some judges. In Python, treap knowledge is the difference between solving rank-query problems and not.
Interview Context
“Implement a balanced BST” is rare, but ordered-set-with-rank problems are constant: LC 315, LC 327, LC 493, sliding-window medians, online percentile queries. AVL and red-black trees are unwritable in 30 minutes (rotation case analysis); a treap is ~70 lines with two primitives, and interviewers accept it. Saying “I’ll use a treap because split/merge composes cleanly” is a strong L5 signal; deriving the E[depth] bound on request seals it.
Problem Statement (LC 315)
Count of Smaller Numbers After Self. Given an integer array nums, return an array counts where counts[i] is the number of elements to the right of nums[i] that are strictly smaller than nums[i].
Constraints
- 1 ≤ nums.length ≤ 10⁵
- −10⁴ ≤ nums[i] ≤ 10⁴ (LC); treat keys as unbounded — the treap doesn’t care.
Clarifying Questions
- Strictly smaller, or ≤? (Strictly smaller — equal elements don’t count.)
- Duplicates in the input? (Yes; the structure must handle equal keys.)
- Is the value range bounded? (On LC yes — which enables a BIT over compressed values, lab-03; a treap needs no compression.)
- Online or offline queries? (Offline here, but the treap solution is fully online — relevant follow-up.)
Examples
countSmaller([5, 2, 6, 1]) → [2, 1, 1, 0]
countSmaller([-1]) → [0]
countSmaller([-1, -1]) → [0, 0]
countSmaller([2, 0, 1]) → [2, 0, 0]
Initial Brute Force
For each i, scan j > i and count nums[j] < nums[i]. Two nested loops.
Brute Force Complexity
O(n²) time, O(1) extra space. At n = 10⁵: 10¹⁰ comparisons. TLE.
Optimization Path
Three O(n log n) routes: (a) merge-sort with cross-count — offline only; (b) BIT over compressed values (lab-03) — needs bounded/compressible keys, offline compression pass; (c) balanced BST with subtree sizes — online, unbounded keys, no preprocessing. Sweep right→left; for each element, query rank(x) = count of inserted keys < x, then insert x. The treap is route (c) made writable: both the query and the insert are O(log n) expected.
Final Expected Approach
_state = 0x9E3779B97F4A7C15 # fixed-seed LCG: deterministic runs
def _pri():
global _state
_state = (_state * 6364136223846793005 + 1442695040888963407) & (2**64 - 1)
return _state
class Node:
__slots__ = ("key", "pri", "size", "l", "r")
def __init__(self, key):
self.key, self.pri, self.size = key, _pri(), 1
self.l = self.r = None
def sz(t):
return t.size if t else 0
def pull(t):
t.size = 1 + sz(t.l) + sz(t.r)
def split(t, key):
"""(left, right): left holds keys < key, right holds keys ≥ key."""
if t is None:
return None, None
if t.key < key:
a, b = split(t.r, key)
t.r = a
pull(t)
return t, b
a, b = split(t.l, key)
t.l = b
pull(t)
return a, t
def merge(a, b):
"""Every key in a ≤ every key in b (caller's contract)."""
if a is None: return b
if b is None: return a
if a.pri > b.pri:
a.r = merge(a.r, b)
pull(a)
return a
b.l = merge(a, b.l)
pull(b)
return b
def insert(t, key): # duplicates allowed
a, b = split(t, key)
return merge(merge(a, Node(key)), b)
def erase(t, key): # remove one occurrence
if t is None:
return None
if t.key == key:
return merge(t.l, t.r)
if key < t.key:
t.l = erase(t.l, key)
else:
t.r = erase(t.r, key)
pull(t)
return t
def kth(t, k): # 0-indexed k-th smallest
while t:
ls = sz(t.l)
if k < ls:
t = t.l
elif k == ls:
return t.key
else:
k -= ls + 1
t = t.r
raise IndexError("k out of range")
def rank(t, key): # = count of keys < key
c = 0
while t:
if t.key < key:
c += sz(t.l) + 1
t = t.r
else:
t = t.l
return c
def countSmaller(nums): # LC 315
root, out = None, []
for x in reversed(nums):
out.append(rank(root, x))
root = insert(root, x)
out.reverse()
return out
Recursion in split/merge/erase runs to tree depth — O(log n) expected (measured: depth 39 after 10⁵ sorted inserts), safely under Python’s default limit of 1000. kth/rank are iterative anyway.
Data Structures Used
- Node with
key, pri, size, l, rand__slots__(halves memory, speeds attribute access). - Fixed-seed 64-bit LCG for priorities — deterministic, reproducible failures.
- No parent pointers: split/merge never need them.
Correctness Argument
Shape/depth: the treap’s shape equals the BST obtained by inserting keys in decreasing-priority order; random priorities ⇒ uniformly random insertion order. Node x is an ancestor of y iff x has the max priority among the keys between them — probability 2/(d+1) for keys d apart. Summing harmonics gives E[depth] = O(log n), the same recursion-tree analysis as randomized quicksort.
split: induction on tree size. The root routes by BST order into one output; the recursive call splits the relevant subtree; reattaching the returned piece preserves BST order (all its keys are on the correct side of the root) and heap order (parent priorities unchanged). pull on unwind fixes sizes.
merge: precondition max(a) ≤ min(b). The higher-priority root becomes the merged root, preserving heap order; BST order holds since everything in b exceeds everything in a.
rank: walking down, every time we go right past a node with key < target, its left subtree and itself are all < target; nothing else < target is skipped. Sizes are exact because every mutation is followed by pull.
Complexity
| Operation | Time (expected) | Space |
|---|---|---|
| split / merge / insert / erase | O(log n) | O(n) total |
| kth / rank | O(log n) | O(1) |
| LC 315 total | O(n log n) | O(n) |
Implementation Requirements
pull()after every child reassignment in split/merge/erase — no exceptions.- Split convention
< key | ≥ keymust matchrank(strict<) so duplicates aren’t counted as smaller. eraseremoves exactly one occurrence;kthis 0-indexed — document both.__slots__on the node class; pure-Python treaps are constant-heavy without it.
Tests
- The four fixed LC 315 cases above.
- 300 random arrays (length ≤ 60, values in [−20, 20], duplicates) vs the O(n²) brute force — exact match.
- 200 random op sequences (insert/erase/kth/rank, 200 ops each) vs a
bisect-maintained sorted list oracle, asserting sizes after every op. - Adversarial order: 10⁵ sorted inserts; assert depth < 3 log₂ n (measured 39 vs 16.6 log₂).
Follow-up Questions
- Why do random priorities give E[depth] = O(log n)? → Shape = BST built in decreasing-priority (i.e., random) insertion order. x is y’s ancestor iff x’s priority is max over the key-interval between them: P = 2/(d+1). E[depth] = Σ 2/(d+1) ≈ 2 ln n. Identical to randomized quicksort’s recursion tree — the root is a random pivot for its key range.
- Range reverse on an array in O(log n)? → Implicit treap: split by size into [0,l), [l,r], (r,…]; set a lazy
flipflag on the middle; push down by swapping children and propagating flags. This rope structure is what text editors use for million-line buffers. - Treap vs AVL/red-black in an interview? → Same O(log n) asymptotics (treap expected, others worst-case). AVL/RB need rotation case analysis nobody writes correctly in 30 minutes; a treap is two primitives plus composition. Say this trade-off out loud.
- Make it persistent? → Path-copying: split/merge only touch O(log n) nodes per op; copy them instead of mutating. Old roots remain valid versions — a persistent ordered set/rope for cheap, same trick as persistent segment trees.
- Skip list — the other writable balanced structure? → Probabilistic level towers, expected O(log n), no rotations at all. Redis chose skip lists for sorted sets: simpler code, cache-friendly forward scans for range queries. Treap wins when you need split/merge as first-class ops.
- When is the BIT (lab-03) better for LC 315? → Bounded or compressible values, offline: BIT is ~5 lines, tiny constants. The treap wins online, with unbounded keys, or when you also need k-th/delete.
Product Extension
A live leaderboard service: rank(score) answers “what percentile am I?”, kth(k) answers “who’s #k?”, both on a stream of score updates (erase old, insert new). This is exactly Redis ZRANK/ZRANGE — Redis backs it with a skip list; a treap is the interview-writable equivalent. Implicit-treap ropes back collaborative editor buffers: cut/paste of a 10⁶-char range in O(log n).
Language/Runtime Follow-ups
- Python: nothing built-in.
sortedcontainers.SortedList(block-list, third-party) is faster in practice; a treap is the from-scratch answer.__slots__mandatory; consider array-based nodes (parallel lists) to dodge object overhead. - C++:
std::setlacks order statistics;__gnu_pbds::treewithtree_order_statistics_node_updategivesfind_by_order/order_of_key. Treap still needed for split/merge/implicit tricks. - Java:
TreeMaphas no rank; augmenting requires a hand-rolled tree anyway — treap it. - Go: no ordered set in the stdlib; treaps are the common CP choice.
- JS/TS: nothing; for small n a sorted array with binary-search insert (O(n) shift) often passes — know when the constant beats the asymptotics.
Common Bugs
- Missing pull(): a child reassignment without a size recompute rots
sizesilently;kth/rankreturn garbage much later. Every mutation path must end inpull. - Split boundary mismatch:
splitsending≤ keyleft whilerankcounts< keydouble-counts duplicates — LC 315 fails only on inputs with equal elements. - Heap direction inconsistency: comparing
a.pri > b.priin one merge branch but effectively min-heap in insert logic — the tree still “works” but degenerates to O(n) depth. - Merging unordered trees:
merge(a, b)silently produces a non-BST if the key precondition is violated (e.g., swapped return values from split). Assert the contract in debug builds. - Weak priorities: using
id(node)or a constant seed per-node — correlated priorities recreate the sorted-insert worst case. One global LCG orrandom.random(). - kth off-by-one: mixing 0- and 1-indexed k between
kthand callers; thek == lsbranch is the giveaway to re-check.
Debugging Strategy
Write a debug-only O(n) validator: in-order traversal asserts keys non-decreasing, every parent priority ≥ children, and size == 1 + sz(l) + sz(r) at every node. Run it after every operation in the randomized test; the first failing op localizes the bug to one function. Then shrink: rerun the failing seed with shorter sequences until minimal, and hand-trace split/merge on the 4–5 node result. For LC 315 disagreements, print the in-order key list next to the oracle’s sorted list — a duplicate-boundary bug shows up as an off-by-one exactly at repeated values.
Mastery Criteria
-
Wrote
split+mergefrom memory, correct first run, in <10 minutes. - Full ordered set (insert/erase/kth/rank) passing the randomized oracle in <25 minutes.
- Solved LC 315 three ways from cold start: merge-sort count, BIT, treap.
- Explained E[depth] = O(log n) via the quicksort correspondence in <2 minutes.
- Sketched implicit-treap range-reverse (size-split + lazy flip) on a whiteboard.
- Stated the treap-vs-AVL/RB and treap-vs-skip-list trade-offs unprompted.
Phase 4 — Graph Mastery
Target level: Medium → Hard Expected duration: 3 weeks (12-week track) / 3 weeks (6-month track) / 4 weeks (12-month track) Weekly cadence: ~7 algorithms per week + 40–70 problems applying them under the framework
Why Graphs Are The Most-Tested Algorithm Family In Senior Interviews
Phase 2 taught you 28 patterns that solve most Mediums. Phase 3 taught you the augmented data structures that make Hards tractable. Phase 4 teaches the single algorithmic family that shows up more often than any other in senior, staff, and infrastructure interviews: graphs.
Here is the empirical claim, and it is the entire reason this phase exists as its own three-to-four-week unit:
Roughly one in three of all Medium-Hard and Hard interview problems at top-tier companies is a graph problem in disguise. Of those, at least half are not labeled “graph” — they are labeled “string”, “scheduler”, “permission system”, “currency conversion”, “build pipeline”, or “bus route”. The first job is recognizing that the problem is a graph problem. The second is picking the right algorithm. The third is implementing it without bugs.
Why graphs dominate senior interviews specifically:
- Graphs are the universal modeling language. Almost every relational or topological structure in a real production system — service dependencies, ACL inheritance, package builds, request routing, social networks, fraud rings, knowledge bases, scheduling DAGs, currency markets — is a graph. A senior engineer is expected to see the graph in a problem that doesn’t mention one.
- Graphs combine almost every Phase 1–3 building block. A real graph problem will fold in a hash map (adjacency list), a queue (BFS), a stack (DFS), a heap (Dijkstra), a DSU (Kruskal / cycle detection), a topological sort (dependency resolution), and sometimes a segment tree (Euler tour + RMQ for LCA). The interviewer gets to test ten primitives in a single 35-minute round.
- Graphs have a clean correctness story. Each algorithm here is a named result with a known proof, known preconditions, and a known complexity. There is no “I think this works because…” — there is “this is BFS, BFS gives shortest paths in unweighted graphs, the precondition is unit-weight edges, the proof is on layer numbers.” Senior interviewers want to hear that proof come out unprompted.
- The graph algorithm space is large but finite. Roughly 20 algorithms cover everything you’ll see at L4–L6. Past that, max-flow, min-cost-max-flow, and matching variants cover staff-and-above. There is a definite ceiling — but it’s higher than candidates expect, and it’s where senior interviewers live.
Candidates who stall on graph rounds almost always fail at recognition or modeling, not at the algorithm itself. They fail because:
- They didn’t recognize “alien dictionary” as a topological sort over inferred constraints.
- They didn’t see “minimum cost to connect all points” as Kruskal/Prim on the complete metric graph.
- They reached for Dijkstra on a graph with negative weights and produced wrong answers.
- They tried BFS on a 0-1 weighted graph instead of 0-1 BFS or Dijkstra.
- They forgot to coordinate-compress an implicit graph and exploded the state space.
This phase is structured to make those failures impossible. You will internalize the signal for each of 21 algorithms, the modeling reflex for implicit graphs, and the algorithm-selection decision tree that maps a problem statement to a single correct technique within 90 seconds.
After this phase, you can solve unmistakably-Hard graph problems on first attempt: alien dictionary in 20 minutes, network delay time in 10, cheapest flights in K stops in 25, accounts merge in 20, bus routes in 30, min cost to connect all points in 15. You also become visibly stronger in mock interviews because you immediately reach for adjacency lists, write from collections import deque before you write any logic, and articulate which algorithm you’re running and why.
What You Will Be Able To Do After This Phase
- Recognize that a problem is a graph problem in <2 minutes of reading, even when neither “graph”, “node”, nor “edge” appears in the statement.
- Choose between BFS / DFS / Dijkstra / Bellman-Ford / Floyd-Warshall / 0-1 BFS / topological sort / DSU / MST in <60 seconds based on the problem’s edge weights, query type, and size.
- Implement a clean adjacency-list representation in <2 minutes for any graph variant (directed, undirected, weighted, multi-edge, self-loop, implicit grid).
- Implement BFS, DFS (recursive + iterative), Dijkstra (eager and lazy), and topological sort (Kahn + DFS) from memory in <8 minutes each.
- Detect cycles in directed and undirected graphs by both DFS-coloring and DSU.
- Run Kruskal and Prim end-to-end with a DSU you write by hand.
- Identify when a Hard problem reduces to bipartite matching or max-flow at the modeling level (you do not need to memorize Dinic).
- Articulate the correctness theorem for every algorithm you use (“Dijkstra is correct because the heap always extracts the next-closest unsettled node, and that node’s tentative distance is its true shortest distance under non-negative weights”).
- Recognize negative-cycle problems and reach for Bellman-Ford / SPFA correctly.
- Construct the implicit graph for grid problems, word ladders, state-space search, and bus-route problems without ever materializing all edges.
How To Read This Phase
Read this README in two passes. Pass 1: linear, end to end, building a mental map of which algorithm plugs which signal. Pass 2: as you work the labs, refer back to specific algorithm entries to clarify invariants and pitfalls.
Each algorithm entry has a fixed shape:
- When To Use — the problem signal that should fire this algorithm in <2 minutes.
- Complexity — time and space, with the assumptions that matter.
- Correctness Sketch — one paragraph that you should be able to recite under interviewer pressure.
- Common Pitfalls — the bugs that consume the most interview minutes.
- Classic Problems — 3–6 representative LeetCode problems where the algorithm is the intended solution.
The phase ends with a Graph-Modeling Cheat Sheet (how to recognize a graph problem in disguise), an Implicit-Graph Catalog (grid / word-ladder / state-space), a Mastery Checklist, and Exit Criteria.
Inline Graph Algorithm Reference
1. Graph Representation
When To Use
Every graph problem starts here. The choice between adjacency list, adjacency matrix, edge list, and implicit graph is the first decision you make, and it shapes every subsequent algorithm’s complexity.
- Adjacency list — the default.
adj[u]is a list of (neighbor, weight) pairs. Use adictoflist(Python),Map<Integer, List<int[]>>(Java),[][]intormap[int][]edge(Go),vector<vector<pair<int,int>>>(C++). - Adjacency matrix —
M[u][v]is the edge weight (or 0 / ∞ for absence). Use only when (a)V ≤ 500so the O(V²) memory fits, (b) you do many(u, v)edge-existence queries, or (c) you’re running Floyd-Warshall. - Edge list — a flat list of
(u, v, w)triples. Use only when the algorithm is edge-centric: Kruskal, Bellman-Ford. - Implicit graph — never materialize the edges. The neighbors of a state are computed on demand. Used for grids, word ladders, sliding puzzles, state-space search.
Complexity
| Representation | Space | Edge query | Iterate neighbors |
|---|---|---|---|
| Adjacency list | O(V + E) | O(deg(u)) | O(deg(u)) |
| Adjacency matrix | O(V²) | O(1) | O(V) |
| Edge list | O(E) | O(E) | O(E) |
| Implicit | O(state) | O(neighbor-fn) | O(neighbor-fn) |
Correctness Sketch
The representation is a faithful encoding of the graph; the algorithm’s correctness is independent of representation as long as iteration over neighbors is exhaustive and edge weights are preserved. Use the representation that minimizes the algorithm’s dominant cost.
Common Pitfalls
- Undirected edges added once instead of twice.
adj[u].append(v)withoutadj[v].append(u)silently breaks every traversal that relies on bidirectionality. - Multi-edges silently lost when using a
setinstead of alistfor neighbors. If the problem permits multi-edges, use lists; if not, decide explicitly. - Self-loops can appear in problems that don’t seem to allow them (e.g., topological sorts of “course depends on itself”). Handle defensively.
- Indexing on string keys — convert string node IDs to ints once, up front. Hash lookups inside hot loops cost real time.
Classic Problems
- LeetCode 261 — Graph Valid Tree (tests representation + cycle detection).
- LeetCode 332 — Reconstruct Itinerary (multi-edges matter; use a heap-of-destinations).
- LeetCode 547 — Number of Provinces (matrix vs list tradeoff).
2. BFS — Breadth-First Search (Unweighted Shortest Path)
When To Use
- Find shortest path in number of edges in an unweighted (or unit-weight) graph.
- Layer-by-layer traversal: “all nodes at distance ≤ k”, “minimum number of moves”, “level-order traversal”.
- The problem says “minimum / shortest / fewest” and the edge weights are all equal.
- Implicit graph variants: shortest word ladder, shortest path in a maze, fewest knight moves on a chessboard.
Complexity
Time O(V + E). Space O(V) for the queue and visited set.
Correctness Sketch
BFS visits nodes in non-decreasing order of distance from the source. When a node is first dequeued, its distance is exactly the shortest path length, because any earlier-enqueued node has distance ≤ the current node’s distance, and the current node was enqueued by a neighbor at distance d - 1 — so any other path to it must go through some node at distance ≥ d - 1, giving total distance ≥ d.
Common Pitfalls
- Marking visited on dequeue, not on enqueue. If you mark on dequeue, the same node can be enqueued by every neighbor before it’s processed once — exploding the queue to O(E) size and degrading performance.
- Tracking distance via
len(queue)confusion. Use either a(node, dist)tuple or process the queue in level batches viafor _ in range(len(queue)). - Not separating the visited check from the enqueue.
if v not in visited: visited.add(v); queue.append(v)is the canonical idiom. - Forgetting to handle the source itself. The source’s distance is 0; it should be marked visited at start.
Classic Problems
- LeetCode 102 — Binary Tree Level Order Traversal.
- LeetCode 127 — Word Ladder (canonical BFS on implicit graph). See Lab 01.
- LeetCode 200 — Number of Islands (BFS variant on grid).
- LeetCode 433 — Minimum Genetic Mutation.
- LeetCode 1091 — Shortest Path in Binary Matrix.
3. DFS — Depth-First Search (Recursive + Iterative; Pre/Post Numbering)
When To Use
- Connected-component enumeration, cycle detection, topological sort, tree traversal, articulation-point detection.
- Backtracking-style problems where you exhaustively explore a state space.
- When path matters more than distance — DFS finds some path, not necessarily the shortest.
- When the graph has small branching but deep paths.
Complexity
Time O(V + E). Space O(V) for the recursion stack (or explicit stack).
Correctness Sketch
DFS explores each edge exactly twice (once in each direction for undirected, once for directed). Pre-order numbering captures discovery time; post-order captures finish time. The discovery/finish interval structure underpins SCC, articulation-point, and bridge algorithms (Tarjan’s lowlink uses pre-order numbers as ranks).
Common Pitfalls
- Stack overflow at V = 10^5 in Python. Default recursion limit is 1000. Either
sys.setrecursionlimit(2 * 10**5)or convert to an explicit stack. - Iterative DFS state. When converting to a stack, you need to track where in the neighbor iteration you are — a tuple of
(node, iterator)works; a tuple of(node, neighbor_index)is faster. - Pre vs post processing confusion. “Print on entry” is pre-order; “print on completion” is post-order; topological sort uses reverse post-order.
- Visited semantics differ from BFS. For cycle detection in directed graphs, you need three states: white (unvisited), gray (on the current DFS path), black (fully explored). A single boolean visited is insufficient.
Classic Problems
- LeetCode 200 — Number of Islands. See Lab 02.
- LeetCode 695 — Max Area of Island.
- LeetCode 207 — Course Schedule (cycle detection via DFS coloring).
- LeetCode 332 — Reconstruct Itinerary (Hierholzer’s = post-order DFS).
- LeetCode 332 — Surrounded Regions.
4. Multi-Source BFS
When To Use
- “From any of these K starting points, what’s the shortest distance to every other node?” — common in grids.
- Equivalent to adding a virtual super-source connected to all K starts with weight 0 and running single-source BFS. But you don’t materialize the super-source: you just enqueue all K starts at distance 0 simultaneously.
- Examples: “rotting oranges” (every rotten orange is a source), “walls and gates” (every gate is a source), “01 matrix distance from nearest zero” (every zero is a source).
Complexity
Time O(V + E). Same as single-source BFS — the K starts add O(K) but are absorbed into the V + E term.
Correctness Sketch
The super-source argument: imagine a node S₀ connected to every start with weight 0. Single-source BFS from S₀ visits each real node in non-decreasing distance order, and the distance is 1 + min(dist(start_i)). By initializing the queue with all starts at distance 0 instead of materializing S₀, we get the same layer-by-layer behavior with the same correctness proof.
Common Pitfalls
- Initializing one start at a time in a loop and running single-source BFS K times. That’s O(K · (V + E)), not O(V + E).
- Forgetting to mark all starts visited up front. If you only mark the first as visited, the others are treated as unvisited targets and get re-enqueued at distance > 0.
- Mixing source types in problems where some sources and some targets are both special (e.g., “rotting oranges” has rotten=source, fresh=target, empty=skip). Always classify cells in a single pass before BFS.
Classic Problems
- LeetCode 994 — Rotting Oranges. See Lab 03.
- LeetCode 286 — Walls and Gates.
- LeetCode 542 — 01 Matrix.
- LeetCode 1162 — As Far From Land As Possible.
- LeetCode 815 — Bus Routes (multi-source on the bus-line graph). See Lab 09.
5. 0-1 BFS
When To Use
- Edge weights are in
{0, 1}(or any two values, with 0-weight as the “free” edge). - The graph mixes “free” transitions (0-weight) and “step” transitions (1-weight). Examples: grid with portals, terrain with roads (free) and trails (cost 1).
- Dijkstra would also work but has a
log Voverhead. 0-1 BFS is O(V + E) — strictly faster.
Complexity
Time O(V + E). Space O(V).
Correctness Sketch
Use a deque. When relaxing an edge of weight 0, push the neighbor to the front; when relaxing weight 1, push to the back. The deque thus holds nodes in non-decreasing order of tentative distance, with at most two distinct distance values present at any moment. The first time a node is popped, its distance is final — same correctness argument as Dijkstra, with the deque playing the role of a 2-bucket priority queue.
Common Pitfalls
- Pushing weight-1 edges to the front is the canonical bug — it breaks the monotone-distance invariant.
- Re-processing nodes because you didn’t check
if d > dist[u]: continueafter popping. This is a Dijkstra-style guard. - Generalizing to weights
{0, k}for k > 1 doesn’t work directly; you need either a multi-bucket BFS or actual Dijkstra.
Classic Problems
- LeetCode 1368 — Minimum Cost to Make at Least One Valid Path in a Grid (canonical 0-1 BFS).
- LeetCode 2290 — Minimum Obstacle Removal to Reach Corner.
- “Shortest path with at most K edges of cost 1, others free” — folklore.
6. Dijkstra (Lazy + Eager Variants; Non-Negative Weights)
When To Use
- Single-source shortest path in a graph with non-negative edge weights.
- The default for any “shortest / cheapest / minimum cost” path problem with weighted edges. If weights can be negative, use Bellman-Ford instead.
- Variants: “shortest path with at most K edges” (relax with a
(dist, edges_used)state), “second shortest path” (two distance arrays), “shortest path on multi-criteria” (state expansion).
Complexity
- Lazy (binary heap): O((V + E) log V). The heap holds up to E entries because we don’t decrease-key — we re-insert and skip stale entries on pop.
- Eager (binary heap with decrease-key): O((V + E) log V) — same asymptotic, smaller constant, but decrease-key requires an indexed heap.
- Fibonacci heap: O(E + V log V) — theoretical, never used in interviews.
- Space O(V) for the dist array + O(E) for the heap (lazy).
Correctness Sketch
Maintain a tentative distance dist[v] for every node, initialized to ∞ except the source (0). Repeatedly extract the unsettled node u with smallest dist[u] (the heap gives this in O(log V)). At the moment of extraction, dist[u] is final: any other path to u must go through some other unsettled node w with dist[w] ≥ dist[u], and since edges are non-negative, the total path length to u via w is ≥ dist[u]. Relax all outgoing edges from u and push updated neighbors. Termination: every node is extracted at most once.
Common Pitfalls
- Using Dijkstra on negative-weight edges. It produces wrong answers — the relaxation invariant fails. Use Bellman-Ford.
- Lazy variant: forgetting the staleness check.
if d > dist[u]: continueafter popping. Without it, you re-process nodes and the asymptotic blows up to O(E²). - Pushing
(dist[u], u)instead of(new_dist, neighbor)when relaxing — the heap orders on the first tuple element, so putdistfirst. - Initializing
dist[source] = 0but not pushing the source to the heap. The first pop must be the source. - Forgetting to handle disconnected components.
dist[v] = ∞is the answer for unreachable v; check before printing.
Classic Problems
- LeetCode 743 — Network Delay Time. See Lab 04.
- LeetCode 1631 — Path With Minimum Effort.
- LeetCode 778 — Swim in Rising Water.
- LeetCode 787 — Cheapest Flights Within K Stops (modified Dijkstra with edge-budget).
- LeetCode 1514 — Path with Maximum Probability (Dijkstra on max-multiplicative).
7. Bellman-Ford (Negative Weights, Negative-Cycle Detection)
When To Use
- Shortest path with negative edge weights but no negative cycle reachable from source.
- Negative-cycle detection itself: if a V-th iteration relaxes any edge, the graph has a negative cycle on the source’s reachable component.
- “Shortest path with at most K edges” — Bellman-Ford’s iteration index is the edge budget. This is the canonical reframing of LeetCode 787.
- All-pairs negative shortest paths via Johnson’s algorithm (Bellman-Ford + Dijkstra), but this is overview-only.
Complexity
Time O(V · E). Space O(V).
Correctness Sketch
After i rounds of relaxing all E edges, dist[v] equals the shortest path from source to v using at most i edges. By induction: in round 1, only the source’s neighbors are relaxed (1-edge paths). In round i, any shortest i-edge path’s last edge (u, v) was relaxed because dist[u] was already correct for i - 1 edges. Since shortest paths in a graph with no negative cycle have ≤ V - 1 edges, V - 1 rounds suffice. A V-th round that still relaxes an edge proves a negative cycle.
Common Pitfalls
- Iterating until no edge is relaxed (early termination) is a common variant — but you still need V - 1 rounds in the worst case for correctness, and the V-th round for cycle detection.
- Using a dict for distances instead of an array indexed by int — slow on hot iteration loops.
- Confusing “shortest path with at most K edges” with “K hops” — read the problem carefully. K stops in LC 787 is K + 1 edges.
- Negative-cycle reachability. A negative cycle exists in the graph but doesn’t affect the source if it’s unreachable. Run from the source, not globally.
Classic Problems
- LeetCode 787 — Cheapest Flights Within K Stops (canonical). See Lab 05.
- “Detect arbitrage in currency markets” (negative cycle in
-log(rate)graph). - “Minimum steps to make k operations” with negative-cost shortcuts.
8. SPFA (Shortest Path Faster Algorithm)
When To Use
- Bellman-Ford with a queue-based optimization that avoids re-relaxing edges whose source
uhasn’t been updated since the last visit. - In practice, ~2–10× faster than vanilla Bellman-Ford on sparse graphs with random structure.
- Caveat: worst-case is still O(V · E). On adversarial inputs (e.g., gridded negative cycles), SPFA can be slower than Bellman-Ford. Codeforces problems are sometimes designed to break SPFA. Use it for negative-weight graphs in interviews only when you’ve stated the caveat.
Complexity
Average O(k · E) for small k (often k ≤ 2). Worst-case O(V · E). Space O(V) for the queue + an in_queue flag.
Correctness Sketch
A node u is enqueued whenever its dist[u] improves. On dequeue, relax all outgoing edges. The in_queue flag prevents duplicate enqueues. Termination follows from the fact that each dist[u] decreases monotonically and is bounded below — for a graph with no negative cycle, the total number of decreases is ≤ V · E.
Common Pitfalls
- Forgetting the
in_queueflag. Without it, the queue can grow to O(E) size and SPFA degrades. - Negative-cycle detection in SPFA requires tracking the number of times each node is relaxed; if a node is relaxed ≥ V times, there’s a negative cycle.
- Adversarial inputs. State the caveat to the interviewer; don’t claim asymptotic improvement.
Classic Problems
- Same as Bellman-Ford. Choose Bellman-Ford for the K-edge-budget framing (LC 787); SPFA only when raw single-source negative shortest path is the goal and average performance matters.
9. Floyd-Warshall (All-Pairs Shortest Paths)
When To Use
- All-pairs shortest paths on a small graph: V ≤ ~500 (V³ = 10^8, ~1 second).
- Negative weights are fine (no negative cycle assumed).
- Transitive closure: replace
min/+withOR/ANDto compute reachability in O(V³). - Density independence: the algorithm is V³ regardless of E. So on dense graphs (E ~ V²) it’s the only practical all-pairs algorithm; on sparse graphs (E ~ V) Dijkstra-from-each-node is V · E · log V = better when V is large.
Complexity
Time O(V³). Space O(V²) for the distance matrix.
Correctness Sketch
Define dp[k][i][j] = shortest path from i to j using only intermediate vertices in {1, ..., k}. The recurrence is dp[k][i][j] = min(dp[k-1][i][j], dp[k-1][i][k] + dp[k-1][k][j]) — either don’t use k, or use it as a midpoint. The 2D in-place version dist[i][j] = min(dist[i][j], dist[i][k] + dist[k][j]) works because in iteration k, the row dist[i][k] and column dist[k][j] are updated only with paths that don’t use k as an intermediate (since k-as-intermediate requires using k twice, which is a cycle and dominated by no-use).
Common Pitfalls
- Loop order is
k, theni, thenj— and noti, j, k. The latter computes garbage. - Negative-cycle detection is
dist[i][i] < 0after the algorithm completes. - Initializing diagonals —
dist[i][i] = 0; missing edges are∞(use a large finite number like10^9to avoid overflow when summing). - Path reconstruction requires a parent matrix — easy to add but double the memory.
Classic Problems
- LeetCode 1334 — Find the City With the Smallest Number of Neighbors at a Threshold Distance (canonical V ≤ 100 Floyd-Warshall).
- LeetCode 743 — Network Delay Time (single-source, but Floyd-Warshall on V ≤ 100 still passes).
- “Transitive closure of a DAG” via
OR/ANDFloyd-Warshall.
10. Topological Sort (Kahn + DFS-Based; Cycle-Detection Equivalence)
When To Use
- DAG ordering where edge
u → vmeans “u must come before v”. - “Course schedule”, “task dependencies”, “build order”, “alien dictionary” (after extracting constraint edges).
- Used as a precondition check: if topological sort fails (cycle present), the problem has no valid ordering.
Complexity
Time O(V + E). Space O(V).
Correctness Sketch — Kahn’s
Repeatedly remove a node with in-degree 0, append it to the order, and decrement its neighbors’ in-degrees. If the order has length V at the end, the graph is a DAG and the order is valid. If not, the remaining nodes form a cycle. Correctness: a node with in-degree 0 has no predecessors, so it can safely come first. After removal, the remaining graph is still a DAG (removing vertices can’t create cycles).
Correctness Sketch — DFS-Based
Run DFS from each unvisited node; on finishing a node (post-order), prepend it to the result. This works because of the white-path lemma: if u → v and DFS visits u before v, then v is in u’s subtree, so v finishes before u, so v is appended before u and prepended after u — meaning u comes before v in the final order.
Common Pitfalls
- Edge direction confusion. “X depends on Y” usually means edge
Y → X(Y must come first). Read the problem carefully. - Detecting cycles via Kahn’s — count nodes processed; if < V, cycle exists.
- Multiple valid orders — both Kahn’s and DFS-based produce some valid order, not a unique one. If the problem demands lexicographically smallest, use Kahn’s with a min-heap instead of a queue.
Classic Problems
- LeetCode 207 — Course Schedule.
- LeetCode 210 — Course Schedule II.
- LeetCode 269 — Alien Dictionary. See Lab 06.
- LeetCode 1136 — Parallel Courses.
- LeetCode 2115 — Find All Possible Recipes from Given Supplies.
11. Cycle Detection In Directed Graphs (DFS Color States)
When To Use
- “Does this directed graph have a cycle?” — in dependency / scheduling / DAG-validation problems.
- More fine-grained than Kahn’s: tells you which edge closes a cycle.
Complexity
Time O(V + E). Space O(V) for the color array.
Correctness Sketch
Maintain three colors: white (unvisited), gray (on the current DFS path), black (fully explored). On entering a node, mark it gray; on finishing, mark it black. If during DFS we encounter a gray neighbor, that’s a back-edge and proves a cycle. White → recurse. Black → already processed; not part of current path; safe to ignore. Correctness: a back-edge is exactly an edge from a descendant to an ancestor in the DFS tree, which closes a cycle. Forward and cross edges don’t.
Common Pitfalls
- Using a single boolean visited. A visited node could be a back-edge target (cycle) or a forward/cross-edge target (no cycle). Two-state visited can’t distinguish. Three colors are required.
- Resetting gray to white on finish. Wrong — you’d keep re-marking black nodes as gray on subsequent DFS calls and produce phantom cycles. Mark black on finish and stay black.
- Forgetting to check both directions in undirected graphs. This algorithm is for directed graphs; for undirected, see #12.
Classic Problems
- LeetCode 207 — Course Schedule (DFS-color variant).
- LeetCode 802 — Find Eventual Safe States (reverse-direction + cycle-detection).
- LeetCode 1059 — All Paths from Source Lead to Destination.
12. Cycle Detection In Undirected Graphs (DFS / Union-Find)
When To Use
- “Is this undirected graph a tree (acyclic + connected)?”, “redundant edge”, “valid tree”.
- Two equivalent approaches: DFS with parent tracking, or DSU.
Complexity
DFS: O(V + E). DSU: O(E · α(V)).
Correctness Sketch — DFS
In an undirected graph, every edge is bidirectional in the adjacency list. When DFS visits v from u, the edge (v, u) shows up in v’s adjacency. Skip the parent: for w in adj[v]: if w != parent: dfs(w, v). If a non-parent neighbor is already visited, that’s a cycle-closing edge.
Correctness Sketch — DSU
Process edges in any order. For each edge (u, v): if find(u) == find(v), the edge would close a cycle (both endpoints already in same component); else union(u, v). After processing all edges, the graph is acyclic iff no cycle was reported.
Common Pitfalls
- DFS: forgetting to skip the parent. Without
if w != parent, every edge looks like a back-edge. - DFS: parallel edges — if the graph has multi-edges between
uandv, the second edge looks like a back-edge through a non-parent neighbor. Track edge IDs, not just parent node. - DSU: ignoring connectedness. “Valid tree” requires both acyclic and connected (V - 1 edges + DSU returning a single component).
Classic Problems
- LeetCode 261 — Graph Valid Tree.
- LeetCode 684 — Redundant Connection (DSU canonical).
- LeetCode 685 — Redundant Connection II (directed variant — harder).
13. Strongly Connected Components (Kosaraju + Tarjan)
When To Use
- Decompose a directed graph into maximal sets where every pair
(u, v)has paths in both directions. - Reduces a directed graph to a DAG of SCCs (the condensation).
- Used in 2-SAT, transitive-closure compression, and “find all nodes that can reach all others”.
Complexity
Both Kosaraju and Tarjan: O(V + E). Space O(V).
Correctness Sketch — Kosaraju
- DFS on the original graph, pushing nodes to a stack in post-order.
- Build the reverse graph.
- Pop nodes from the stack; for each unvisited node, DFS on the reverse graph — that DFS visits exactly one SCC.
The post-order ordering ensures the first node popped is in a “source” SCC of the condensation; reverse-DFS from it can only reach nodes in its own SCC because no other SCC’s nodes have a forward path back to it.
Correctness Sketch — Tarjan
Single DFS maintaining a stack of “currently active” nodes plus disc[u] (discovery time) and low[u] (lowest discovery time reachable via the subtree + at most one back-edge). When DFS finishes a node u and low[u] == disc[u], pop the stack down to and including u — those popped nodes form an SCC. Tarjan does it in one pass; Kosaraju in two passes but with simpler bookkeeping.
Common Pitfalls
- Kosaraju: forgetting to reverse all edges in the second graph. Use a separate adjacency list.
- Tarjan: confusing
lowwithdisc—low[u] = min(low[u], low[v])for tree-edge children;low[u] = min(low[u], disc[v])for back-edge neighbors that are still on the stack. Both updates are needed. - Tarjan: handling cross-edges to other SCCs. A neighbor that’s been finished is already in a different SCC; do not update
low[u]from it.
Classic Problems
- LeetCode 1192 — Critical Connections (Tarjan for bridges; SCC-adjacent).
- “2-SAT solvability” via SCCs in the implication graph.
- “Number of source SCCs in the condensation” — Codeforces classic.
14. Bridges And Articulation Points (Tarjan’s Lowlink)
When To Use
- A bridge is an edge whose removal disconnects the graph.
- An articulation point is a vertex whose removal disconnects the graph.
- Used in “critical connections” problems and network-resilience analysis.
Complexity
O(V + E). Single DFS, one pass.
Correctness Sketch
Compute disc[u] and low[u] as in Tarjan’s SCC. For an edge (u, v) where v is a tree child: low[v] > disc[u] ⇒ (u, v) is a bridge (no back-edge from v’s subtree reaches anything at-or-above u, so removing (u, v) disconnects). For articulation points: u is an articulation point if (a) u is the DFS root and has ≥ 2 tree children, or (b) u is not the root and has a tree child v with low[v] ≥ disc[u].
Common Pitfalls
- Using ≥ vs > — bridges use
low[v] > disc[u]; articulation points uselow[v] ≥ disc[u]. The off-by-one between them is critical. - DFS root special case for articulation points — must count tree children; with one tree child, removing the root doesn’t disconnect.
- Multi-edges — a multi-edge between
uandvis not a bridge (the parallel edge keeps the graph connected). Treat parallel edges by edge-ID, not endpoint pair.
Classic Problems
- LeetCode 1192 — Critical Connections in a Network (canonical bridges).
- “Find all articulation points in a network” — Codeforces classic.
15. Minimum Spanning Tree — Kruskal (With DSU)
When To Use
- Connect all V nodes with the minimum total edge weight, in a graph with V - 1 chosen edges.
- Edge-centric algorithm: sort edges, add greedily, skip those that close a cycle (DSU detects).
- Best on sparse graphs (E ~ V) where sorting E edges dominates.
Complexity
O(E log E) for sort + O(E · α(V)) for DSU = O(E log E). Space O(V).
Correctness Sketch (Cut Property)
The minimum-weight edge crossing any cut of the graph belongs to some MST. Kruskal repeatedly takes the next-cheapest edge; if it doesn’t close a cycle (DSU find(u) != find(v)), it’s the cheapest edge crossing the cut between its DSU components. By the cut property, it’s in some MST. Adding it preserves the invariant that the chosen edges are a subset of some MST. After V - 1 edges, the chosen set is a spanning tree.
Common Pitfalls
- Forgetting to sort edges. Kruskal without sorting is just a random spanning tree.
- DSU bugs. Phase 3’s α(N) DSU is required. Recursive
findblows the stack at V = 10^5; use iterative two-pass or path-halving. - Disconnected graph. If after processing all edges DSU has > 1 component, no spanning tree exists. Return failure or compute a minimum spanning forest.
- Tie-breaking on equal weights — any tie-breaking rule works; they all produce some MST.
Classic Problems
- LeetCode 1584 — Min Cost to Connect All Points. See Lab 08.
- LeetCode 1135 — Connecting Cities With Minimum Cost.
- LeetCode 1489 — Find Critical and Pseudo-Critical Edges in MST.
16. Minimum Spanning Tree — Prim (With Priority Queue)
When To Use
- Same problem as Kruskal — minimum total edge weight to connect all V.
- Vertex-centric algorithm: grow the tree one vertex at a time, always adding the minimum-weight edge from the tree to a non-tree vertex.
- Best on dense graphs where the heap pays off relative to sorting all E edges.
Complexity
With binary heap: O((V + E) log V). With Fibonacci heap: O(E + V log V) (theoretical). Space O(V + E).
Correctness Sketch (Cut Property, Variant)
At every step, the partial tree T defines a cut (T vs not-T). The minimum-weight edge crossing this cut is added next. By the cut property, it belongs to some MST. The invariant — chosen edges ⊆ some MST — is preserved. After V - 1 additions, the tree spans all of V.
Common Pitfalls
- Lazy vs eager Prim. Lazy: push every (weight, neighbor) to the heap, skip duplicates on pop. Eager: maintain a “best known weight to enter T” per non-tree vertex and use decrease-key. Lazy is simpler and asymptotically equivalent.
- Disconnected graph — same as Kruskal; the heap empties before V - 1 edges are added.
- Picking starting vertex — any vertex works for connected graphs.
Classic Problems
- LeetCode 1584 — Min Cost to Connect All Points (also solvable via Prim).
- “Maximum spanning tree” via negated weights.
17. Bipartite Check (BFS/DFS 2-Coloring)
When To Use
- “Can we partition the V into two groups such that every edge crosses groups?”
- Equivalent to: graph has no odd cycle.
- Used as a precondition for bipartite matching, and in problems like “is this set of dislike-pairs separable into two camps?”
Complexity
O(V + E). Space O(V).
Correctness Sketch
BFS/DFS, coloring each visited node alternately (color 0 or 1) from its parent. If we ever try to color a visited node with a color different from its existing color, the graph has an odd cycle and is not bipartite. Correctness: BFS layers alternate colors; an edge within a layer (or skipping ≥ 2 layers) violates bipartiteness; specifically, any odd cycle forces a same-layer edge.
Common Pitfalls
- Disconnected graph. Run BFS from every unvisited node; the bipartiteness of each component is independent.
- Initializing colors as
-1(uncolored),0,1. Aboolean visitedis insufficient; you need the actual color. - Counting color-0 vs color-1 sizes when the problem asks for “minimum group size” — but the partitioning is unique only up to swapping the two colors per connected component.
Classic Problems
- LeetCode 785 — Is Graph Bipartite?
- LeetCode 886 — Possible Bipartition.
- “2-coloring as a sanity check before bipartite matching.”
18. Bipartite Matching (Hungarian / Hopcroft-Karp Overview)
When To Use
- Maximum cardinality matching in a bipartite graph: pair up as many left-side nodes with right-side nodes as possible, using each at most once.
- Job assignment, “find as many distinct words to slots as possible”, “schedule maximum tasks to workers”.
- Hungarian algorithm: O(V · E) via repeated augmenting-path BFS — ~O(V²·√V) on bipartite.
- Hopcroft-Karp: O(E · √V) — strictly better; the algorithm of choice for large bipartite matching.
Complexity
Hungarian: O(V · E). Hopcroft-Karp: O(E · √V). Space O(V + E).
Correctness Sketch (König’s Theorem and Augmenting Paths)
Berge’s theorem: a matching M is maximum iff there is no augmenting path (a path alternating unmatched / matched edges, starting and ending at unmatched vertices). The Hungarian algorithm repeatedly finds augmenting paths via BFS/DFS and augments. Hopcroft-Karp accelerates by finding all shortest augmenting paths in a single BFS phase, then augmenting all of them in one DFS phase.
Common Pitfalls
- Confusing maximum matching with maximum-weight matching. The latter is min-cost-max-flow; harder algorithm.
- Implementing matching from scratch in interviews is rare. State the algorithm by name, reduce to it, and note the complexity. Senior interviewers accept “this is bipartite matching, O(E√V) via Hopcroft-Karp” without code.
- Modeling. The hard part is recognizing the bipartite structure. “Are there two disjoint sets where edges only cross sets?”
Classic Problems
- LeetCode 1947 — Maximum Compatibility Score Sum (small N: bitmask DP. Large N: bipartite matching + weights).
- “Maximum number of distinct words to slots” — folklore.
- LeetCode 1349 — Maximum Students Taking Exam (bipartite + bitmask).
Overview-level only. Implementation drills are Phase 7 / 12.
19. Max Flow (Ford-Fulkerson / Edmonds-Karp / Dinic — Overview + When To Use)
When To Use
- “Maximum amount of flow from source S to sink T in a capacitated network.”
- Reductions: bipartite matching → max flow. Edge-disjoint paths → max flow. Min cut → max flow.
- Algorithms:
- Ford-Fulkerson (DFS-based augmenting): O(E · max-flow) — pseudo-polynomial; can loop forever on irrational capacities.
- Edmonds-Karp (BFS-based augmenting): O(V · E²). Polynomial.
- Dinic’s: O(V² · E) — uses BFS levels + DFS-blocking-flow. Practical for V, E up to 10^4–10^5.
Complexity
See above. Space O(V + E) for the residual graph.
Correctness Sketch (Max-Flow Min-Cut Theorem)
The maximum flow equals the minimum cut capacity. An augmenting path in the residual graph proves the current flow is not maximum; absence of any augmenting path proves it is. Dinic’s enhancement: BFS to compute layered graph, DFS to push blocking flow, repeat until no augmenting path. Each phase strictly increases the BFS distance from S to T, bounded by V phases.
Common Pitfalls
- Implementation in 35-minute interviews is rare. State the algorithm by name, model the problem, and let the interviewer guide depth.
- Residual graph forgetting reverse edges. Every forward edge
u → vof capacitycadds a reverse edgev → uof capacity 0; pushing flowfon forward subtracts from forward residual and adds to reverse residual. Without reverse edges, augmenting paths can’t “undo” prior bad choices and the max-flow can be wrong. - Modeling errors. “Each node has capacity” requires node-splitting (split
vintov_inandv_outwith edge capacity = node capacity).
Classic Problems
- “Maximum bipartite matching via max-flow.”
- “Edge-disjoint paths from S to T.”
- LeetCode-style: very rare. Common in Google L5+ system rounds and competitive programming.
Overview-level. Implementation in Phase 12.
20. Min-Cut / Max-Flow Duality (Problem Modeling)
When To Use
- “Minimum cost to disconnect S from T” → min-cut problem.
- “Minimum number of edges to remove to disconnect” → min-cut on unit-capacity edges.
- “Image segmentation as binary labeling” → min-cut on a constructed graph.
- “Project selection” (some projects depend on others; pick a subset to maximize profit) → min-cut.
Complexity
Same as max-flow (compute the cut from the residual graph after max-flow terminates).
Correctness Sketch (Max-Flow Min-Cut Theorem)
In any flow network, max-flow value = min-cut capacity. After running max-flow, the min cut consists of the edges from {nodes reachable from S in residual graph} to {nodes not reachable}. Their original capacities sum to the max-flow value.
Common Pitfalls
- Recognizing the model. This is the hardest part. “Project selection” doesn’t look like a flow problem; recognizing the bipartite encoding is the senior-level skill.
- Edge orientation. Min-cut on undirected graphs: each undirected edge becomes two directed edges, each with capacity
c.
Classic Problems
- “Project Selection Problem” — folklore.
- “Image Segmentation via Min-Cut” — vision systems.
- “Minimum number of edges to disconnect” — Menger’s theorem.
21. Eulerian Path / Circuit (Hierholzer’s Algorithm)
When To Use
- “Visit every edge exactly once” — Eulerian path/circuit.
- An undirected graph has an Eulerian circuit iff every vertex has even degree (and the graph is connected on edges). It has an Eulerian path iff exactly 0 or 2 vertices have odd degree.
- A directed graph has an Eulerian circuit iff every vertex has in-degree = out-degree. It has an Eulerian path iff exactly one vertex has out-degree − in-degree = 1 (start) and one has in-degree − out-degree = 1 (end).
- Hierholzer’s algorithm finds the path/circuit in O(E).
Complexity
O(V + E).
Correctness Sketch (Hierholzer’s)
DFS from the start vertex, consuming edges as you traverse them (remove from adjacency). When stuck at a vertex with no outgoing edges, prepend it to the path. Backtrack and continue from earlier vertices that still have unused outgoing edges. The final reverse of the recorded sequence is an Eulerian path. Correctness: each edge is consumed exactly once; the post-order finishing structure naturally constructs the path in reverse.
Common Pitfalls
- Multi-edges and self-loops are common in Eulerian problems. Use a multiset or a list of edges with a “consumed” flag.
- Disconnected components on edges (vs vertices). Isolated vertices with degree 0 are fine; they don’t break Eulerian-ness.
- Lexicographically smallest Eulerian path (LC 332) — sort each adjacency list and use a multiset / heap; pop the smallest unused edge first.
Classic Problems
- LeetCode 332 — Reconstruct Itinerary (canonical Hierholzer’s).
- LeetCode 753 — Cracking the Safe (Eulerian path on de Bruijn graph).
Graph-Modeling Cheat Sheet — How To Recognize A Graph Problem In Disguise
The hardest skill in this phase is modeling: recognizing that a problem is a graph problem when nothing in the statement says “graph”. Here is a battery of signals.
| Signal in problem statement | Graph interpretation | Likely algorithm |
|---|---|---|
| “Depends on” / “must come before” / “prerequisite” | DAG, edge pre → post | Topological sort |
| “Connected” / “linked” / “merged” / “in same group” | Undirected, components | DFS / BFS / DSU |
| “Shortest” / “fewest steps” / “minimum moves” with unit cost | Unweighted graph | BFS |
| “Cheapest” / “minimum cost” with non-negative weights | Weighted graph | Dijkstra |
| “Cheapest with negative discounts” | Weighted graph with neg edges | Bellman-Ford |
| “Minimum cost to connect all” | Spanning tree | Kruskal / Prim |
| “Cycle” / “loop” / “redundant” | Graph + cycle test | DFS coloring / DSU |
| “Two groups” / “partition” / “no two together” | Bipartite | 2-coloring |
| “Pair up X with Y” | Bipartite matching | Hungarian / Hopcroft-Karp |
| “Maximum throughput” / “bottleneck” / “max disjoint paths” | Flow network | Max-flow |
| “Minimum to disconnect” / “critical edges” | Min-cut / bridges | Max-flow / Tarjan |
| “Visit all edges once” | Eulerian | Hierholzer’s |
| “Visit all vertices once with min cost” | Hamiltonian / TSP | Bitmask DP (Phase 3) |
| “Currency conversion” / “exchange rate” | Weighted directed; cycles = arbitrage | Bellman-Ford |
| “ACL inheritance” / “permission propagation” | Reachability | DFS / BFS / transitive closure |
| “Build pipeline” / “task DAG” | Topological + critical path | Topo sort + DP |
| “Friend of friend” / “social network” | Undirected | BFS / DSU |
| “Word transformation” / “step-by-step transform” | Implicit graph on states | BFS |
| “Sliding puzzle” / “8-puzzle” / “Rubik’s cube” | Implicit state graph | BFS / IDA* |
| “Routes between cities” / “flight network” | Directed weighted | Dijkstra |
| “Spread / infection / contamination over time” | Multi-source unweighted | Multi-source BFS |
Common Implicit Graphs
These are the four canonical implicit-graph patterns. You should be able to spot all four within 30 seconds of reading the problem.
1. Grid Graphs
Each cell (r, c) is a node; edges go to the 4 (or 8) neighbors that satisfy bounds and the cell-type constraint. Never materialize all V·M edges — compute neighbors on demand.
DIRS = [(-1, 0), (1, 0), (0, -1), (0, 1)]
def neighbors(r, c):
for dr, dc in DIRS:
nr, nc = r + dr, c + dc
if 0 <= nr < R and 0 <= nc < C and grid[nr][nc] != '#':
yield (nr, nc)
Variants: 8-connected, weighted edges (cost = cell value), constraint-aware (can only enter from certain directions), gravity-based.
2. Word-Ladder Graphs
Each word is a node; an edge connects two words that differ in exactly one character. With N words of length L over alphabet σ, materializing all edges is O(N²) worst case. The trick: for each word, generate L · σ “wildcarded” patterns and use a dict-of-list to find neighbors in O(L · σ) per word.
buckets = defaultdict(list)
for w in words:
for i in range(len(w)):
buckets[w[:i] + '*' + w[i+1:]].append(w)
def neighbors(w):
for i in range(len(w)):
yield from buckets[w[:i] + '*' + w[i+1:]]
3. State-Space Search
Each “state” of the system is a node; transitions are edges. The state encodes the full configuration: e.g., a tuple of (position, keys_collected) for “shortest path with key-collection”.
def neighbors(state):
pos, keys = state
for next_pos in adjacent_cells(pos):
if next_pos has key K:
yield (next_pos, keys | (1 << K))
else:
yield (next_pos, keys)
4. Bipartite “Token / Container” Graphs
Two sets of nodes — e.g., users and groups, buses and stops, courses and prerequisites. An edge connects a token to a container it belongs to. Multi-source BFS over this graph gives “minimum tokens needed to traverse from container A to container B” — the canonical “bus routes” framing in LeetCode 815.
See Lab 09 for the bus-routes modeling exercise.
Mastery Checklist
Before exiting this phase, verify all of these:
- You can build an adjacency list from an edge list (directed and undirected) in <2 minutes, in your primary language.
- You can write BFS from memory in <5 minutes, including correct visited-on-enqueue semantics.
- You can write DFS recursively and iteratively (with explicit stack) from memory in <8 minutes total.
- You can write Dijkstra from memory in <8 minutes, lazy variant, including the staleness-skip line.
- You can write Kahn’s topological sort from memory in <6 minutes.
- You can write DSU with path compression and union by rank from memory in <5 minutes.
- You can write Kruskal’s MST from memory in <10 minutes (DSU + sort).
- You can recognize “this is a graph problem” within 2 minutes of reading any of the 30 classic graph problems on this list.
- You can correctly choose between BFS / Dijkstra / Bellman-Ford / 0-1 BFS based on edge weights.
- You can model the bus-routes problem (LC 815) as a graph in <5 minutes, articulating the bipartite structure.
- You can model the alien-dictionary problem (LC 269) as a topological sort in <5 minutes, articulating the constraint-extraction step.
- You can articulate the cut property and why it makes Kruskal correct, in <30 seconds.
- You can articulate why Dijkstra fails on negative weights, in <30 seconds.
- You can articulate the white-path lemma and its connection to topological sort via reverse post-order.
Exit Criteria
You may move to Phase 5 (Dynamic Programming) when all of the following are true:
- You have completed all nine labs in this phase, with the lab’s mastery criteria checked off for each.
- You have solved at least 40 unaided graph problems from LeetCode (mix of Medium, Medium-Hard, Hard) and reviewed each via REVIEW_TEMPLATE.md.
- Your unaided success rate on Medium-Hard graph problems is ≥ 70%.
- In a mock interview (phase-11-mock-interviews/), you correctly identify the algorithm family within 2 minutes for at least 8 of 10 graph problems.
- You can write Dijkstra, BFS, DFS, Kahn’s topological sort, and DSU + Kruskal — five algorithms — from a blank slate in under 45 minutes total.
If any of these fails, do another 15–25 graph problems before moving on. Skipping this gate calcifies bad habits that compound in Phase 5 (where DP-on-graphs and DAG-DP build directly on this material).
Labs
Hands-on practice. Each lab follows the strict 22-section format from Phase 0.
- Lab 01 — BFS Shortest Path (Word Ladder)
- Lab 02 — DFS Connected Components (Number of Islands)
- Lab 03 — Multi-Source BFS (Rotting Oranges / Walls and Gates)
- Lab 04 — Dijkstra (Network Delay Time / Cheapest Flights)
- Lab 05 — Bellman-Ford (Cheapest Flights Within K Stops)
- Lab 06 — Topological Sort (Alien Dictionary)
- Lab 07 — Union-Find Applications (Accounts Merge / Provinces)
- Lab 08 — MST Kruskal (Min Cost to Connect All Points)
- Lab 09 — Graph Modeling (Bus Routes)
- Lab 10 — Strongly Connected Components & 2-SAT
- Lab 11 — Bridges & Articulation Points (Critical Connections)
- Lab 12 — Eulerian Path (Reconstruct Itinerary)
← Phase 3: Advanced Data Structures · Phase 5: Dynamic Programming → · Back to Top
Lab 01 — BFS Shortest Path (Word Ladder)
Goal
Implement an unweighted shortest-path search on an implicit graph where the nodes are dictionary words and the edges connect any two words that differ in exactly one character. After this lab you should be able to recognize a word-ladder / state-transformation problem in <60 seconds, build the wildcard-bucket adjacency in <3 minutes, and write the BFS body from a blank screen in <5 minutes with correct visited-on-enqueue semantics.
Background Concepts
BFS on an unweighted graph visits nodes in non-decreasing order of distance from the source: the source first (distance 0), then its neighbors (distance 1), then their neighbors (distance 2), and so on. The first time a node is dequeued, its distance is final. This phase teaches the wildcard-bucket trick that makes word-ladder graphs tractable: rather than checking all O(N²) word-pairs for adjacency, build a dict mapping each L-character “wildcarded” pattern (e.g., h*t, *ot, ho*) to the list of words matching that pattern. Two words are adjacent iff they share at least one bucket.
The buckets are O(N · L) entries total; constructing them is O(N · L); finding all neighbors of a word is O(L · σ) where σ is the average bucket size. The total BFS cost is O(N · L²) instead of O(N² · L).
Interview Context
Word Ladder (LC 127) is a top-50 interview problem at Meta and Amazon — both companies have asked it within the past year. Variants appear at Google (“minimum genetic mutation” — LC 433) and Bloomberg. It tests three things at once: (1) recognizing the implicit graph, (2) building it efficiently, (3) running clean BFS. Candidates who try for each pair: if differs by one: connect time out at N = 5000. Bombing this problem on a phone screen is a serious negative signal at L4+.
Problem Statement
Given a beginWord, an endWord, and a list wordList, return the length of the shortest transformation sequence from beginWord to endWord such that:
- Only one letter changes per step.
- Every intermediate word must be in
wordList. beginWorddoes not need to be inwordList.
Return 0 if no such sequence exists.
Constraints
- 1 ≤
beginWord.length≤ 10 - All words have the same length, L.
- 1 ≤
wordList.length≤ 5000 - All words consist of lowercase English letters.
beginWord != endWord.- All words in
wordListare unique.
Clarifying Questions
- Is
beginWordrequired to differ fromendWord? (Yes — guaranteed.) - Does the answer count
beginWordandendWord? (Yes — sequence length includes both endpoints.) - If
endWordis not inwordList, is the answer 0? (Yes — by the rules.) - Are case-sensitive comparisons required? (No — all lowercase.)
- Can
beginWorditself appear inwordList? (Yes; treat normally.)
Examples
beginWord = "hit", endWord = "cog", wordList = ["hot","dot","dog","lot","log","cog"]
→ 5 (hit → hot → dot → dog → cog)
beginWord = "hit", endWord = "cog", wordList = ["hot","dot","dog","lot","log"]
→ 0 (cog not in wordList)
Initial Brute Force
Materialize the graph: for each pair of words, check if they differ by one letter (O(L) check per pair). Build adjacency in O(N² · L). Then run BFS in O(V + E) = O(N²) edges worst case.
Brute Force Complexity
Time O(N² · L) for graph construction, O(N²) for BFS. Total O(N² · L). Space O(N²). At N = 5000, L = 10: 2.5 × 10^8 ops — borderline TLE in Python; passes in C++ tightly.
Optimization Path
The bottleneck is graph construction. Wildcard buckets eliminate it: for each word, generate L wildcards and append to a bucket dict. Construction is O(N · L²) — at N = 5000, L = 10, that’s 5 × 10^5 ops, two orders of magnitude better. Neighbor enumeration is also faster: only words sharing a bucket are candidates, which prunes dramatically vs scanning all N words.
Bidirectional BFS is a further optimization that ~halves the work in practice (search from both ends, meet in the middle), but adds complexity and is overkill at N = 5000.
Final Expected Approach
- If
endWordnot inwordList, return 0. - Build
buckets: a dict mapping each wildcard pattern to the list of words matching it. - BFS from
beginWordwith distance 0; on dequeue, generate L wildcards; for each, look up the bucket and enqueue any unvisited word with distance + 1. - On reaching
endWord, return distance + 1 (because the answer counts both endpoints). - If queue exhausts, return 0.
Data Structures Used
dict[str, list[str]]— wildcard buckets.set[str]— visited words.collections.deque— BFS queue, holding(word, distance)tuples.
Correctness Argument
BFS on an unweighted graph: when endWord is first dequeued, its distance is the minimum number of edges from beginWord. Each edge represents a one-letter change between dictionary words. Therefore the distance equals the minimum number of one-letter changes — and the sequence length is distance + 1. Visited-on-enqueue ensures each word enters the queue at most once, so total work is O(V · L · σ) where σ is the average bucket size.
Complexity
| Operation | Time | Space |
|---|---|---|
| Build buckets | O(N · L²) | O(N · L) |
| BFS | O(N · L²) | O(N) |
| Total | O(N · L²) | O(N · L) |
Implementation Requirements
from collections import defaultdict, deque
def ladderLength(beginWord, endWord, wordList):
word_set = set(wordList)
if endWord not in word_set:
return 0
L = len(beginWord)
buckets = defaultdict(list)
for w in word_set:
for i in range(L):
buckets[w[:i] + '*' + w[i+1:]].append(w)
visited = {beginWord}
queue = deque([(beginWord, 1)])
while queue:
word, d = queue.popleft()
if word == endWord:
return d
for i in range(L):
pat = word[:i] + '*' + word[i+1:]
for nb in buckets[pat]:
if nb not in visited:
visited.add(nb)
queue.append((nb, d + 1))
buckets[pat] = [] # optional: clear bucket to avoid reprocessing
return 0
Tests
- Standard:
hit → cogwith full path → 5. - Missing endWord: return 0.
beginWord == endWord: technically violates constraints, but should return 1 if asked.- Single-step:
hit → hotwith wordList=[“hot”] → 2. - No path: disconnected words → 0.
- Long L: words of length 10, N = 5000 (load test).
- All same-length: invariant must hold; assert in code.
Follow-up Questions
- “Return all shortest paths, not just length.” → BFS to identify the layer of
endWord, then DFS backward through parent pointers stored at each layer. - “What if word lengths differ across the list?” → Edges are now insert/delete/substitute; problem reduces to edit-distance graph. Out of scope here.
- “What if N = 10^6?” → Bidirectional BFS halves the layer count; trie-based neighbor finding can replace bucket dicts.
- “Stream of new words being added live.” → Maintain buckets incrementally; BFS becomes a per-query operation.
- “What if changing a letter has a cost?” → Now weighted; switch to Dijkstra.
Product Extension
Spell-correctors, fuzzy-matching APIs, and DNA-mutation analyzers all use similar implicit-graph BFS. Google’s “did you mean” suggestion historically used Levenshtein-distance graphs over its query log; word-ladder BFS is the toy version of that.
Language/Runtime Follow-ups
- Python:
defaultdict(list)andcollections.dequeare essential. String slicing is O(L); thew[:i] + '*' + w[i+1:]pattern allocates a new string per call (3 × 10^5 per call * 5000 words = 1.5 × 10^9 — measured ~1.5s in Python). Acceptable; for faster, precompute patterns once per word. - Java: use
HashMap<String, List<String>>andArrayDeque<String>.StringBuilderfor pattern construction is faster than string concat. Useintdistance via a parallel map or wrap in a custom record. - Go:
map[string][]stringand a slice-based queue (q = q[1:]is O(1) amortized for slice queues, but channels are easier). Strings are immutable so building patterns is O(L) regardless. - C++:
unordered_map<string, vector<string>>andqueue<pair<string,int>>. Preallocate to avoid rehashing. Usestring_viewif possible to avoid copies; or(word_index, distance)to avoid string keys entirely. - JS/TS:
Mapand an array used as a queue (shift()is O(N) — instead, use a deque library or two-stack approach). Strings are immutable; pattern construction allocates.
Common Bugs
- Forgetting to check
endWord in word_setup front — wastes work if missing. - Visited check on dequeue, not on enqueue — exponential blowup of queue size.
- Returning the BFS distance instead of distance + 1 (or vice versa) — off by one.
- Including
beginWordinword_setand then visiting it on a wildcard match — easy if you don’t initializevisited = {beginWord}first. - Generating wildcards with the wrong character (
'_'vs'*') and getting collision-free buckets that are also empty. - Forgetting that
wordListmay contain duplicates if you stored as list — use a set.
Debugging Strategy
Print the buckets for a tiny example (["hot", "dot", "dog"], L=3) and verify each wildcard pattern maps to the expected words. Print the BFS queue state after each layer. If the BFS terminates too early, trace which word was dequeued at the failure point and which neighbors weren’t generated. If too slow, profile with cProfile (Python) — likely you’re not visited-marking on enqueue.
Mastery Criteria
- Recognized the implicit-graph signal (one-character-difference adjacency) within 60 seconds.
- Wrote the bucket construction from blank screen in <3 minutes, no off-by-ones.
- Wrote correct BFS with visited-on-enqueue in <5 minutes from cold start.
- Stated O(N · L²) complexity unprompted.
- Solved LC 127 in <15 minutes from cold start.
- Solved LC 433 (Minimum Genetic Mutation, a near-clone) in <10 minutes.
- Articulated the wildcard-bucket vs all-pairs tradeoff in <30 seconds when asked.
Lab 02 — DFS Connected Components (Number of Islands)
Goal
Implement DFS on a 2D grid to enumerate connected components, both recursively and iteratively. After this lab you should be able to write numIslands from a blank slate in <8 minutes, convert recursive DFS to iterative DFS in <3 minutes, and extend the template to any grid-component problem (max area, perimeter, surrounded regions) by changing 5 lines or fewer.
Background Concepts
A grid graph treats each cell (r, c) as a node and edges as adjacencies between 4-connected (or 8-connected) neighboring cells of compatible type. A connected component is a maximal set of cells reachable from each other. Counting components reduces to: scan all cells; whenever an unvisited “land” cell is found, increment the count and DFS-mark its entire component as visited.
DFS is naturally recursive: enter a cell, mark visited, recurse to each valid neighbor. The recursion depth equals the longest path in the component; for an R × C grid the worst case is R · C — at 300 × 300 that’s 9 × 10^4, exceeding Python’s default 1000 recursion limit. The iterative version uses an explicit stack and avoids this entirely.
Interview Context
Number of Islands (LC 200) is the most-asked grid-DFS problem in interview history — it has appeared at virtually every FAANG company at least once, and at Amazon, Meta, and Google in the last year. It’s a stock phone-screen problem at L3-L4 and a warm-up at L5+. Bombing it is a no-hire signal at any senior level. The interviewer expects you to know it cold, and the value-add comes from how you handle the follow-ups: max area, surrounded regions, online updates (LC 305), grid as adjacency matrix (LC 547).
Problem Statement
Given an m × n 2D binary grid where '1' represents land and '0' represents water, return the number of islands. An island is a maximal group of land cells connected 4-directionally (horizontally or vertically).
Constraints
- 1 ≤ m, n ≤ 300
grid[i][j]is'0'or'1'.- The grid is surrounded implicitly by water on all sides.
Clarifying Questions
- Is diagonal adjacency 4-connected or 8-connected? (4-connected.)
- Are the cell values strings
'0'/'1'or ints? (Strings, per LeetCode.) - Can the input be modified in place? (Usually yes — saves O(R·C) visited memory.)
- Are very tall thin grids possible? (Yes — 1 × 90000 is allowed by the constraint.)
- Is the count required to fit in int32? (Yes; max islands ≈ 4.5 × 10^4.)
Examples
grid = [["1","1","0"],
["1","0","0"],
["0","0","1"]]
→ 2
grid = [["1","1","1"],
["1","1","1"],
["0","0","0"]]
→ 1
Initial Brute Force
For each cell, if it’s land and unvisited, increment count and recursively flood-fill. There is no significantly worse “naive” — DFS is the natural approach.
Brute Force Complexity
O(R · C) — every cell is visited exactly once. Space O(R · C) for the recursion stack worst case.
Optimization Path
There’s no asymptotic improvement; only constant-factor and stack-depth improvements:
- In-place marking (mutate
'1'→'0') — saves O(R · C) memory. - Iterative DFS via stack — avoids Python recursion-limit blow-up at R · C ≥ 10^4.
- DSU as alternative — slightly slower in practice (α factor) but composable for online problems (LC 305).
- BFS variant — same asymptotic, different constant; sometimes preferred in Python because deque-pop has lower per-call cost than function calls.
Final Expected Approach
Iterative DFS with in-place marking:
- Iterate over every cell.
- If
grid[r][c] == '1': increment count, push(r, c)to stack. - While stack non-empty: pop
(r, c); if already water, skip; mark'0'; push 4 neighbors that are land.
Recursive DFS is acceptable for small grids; mention recursion-limit and in-place marking on follow-up.
Data Structures Used
- The input grid itself as the visited bookmark (in-place mutation).
- An explicit stack (list, in Python) for iterative DFS.
Correctness Argument
Component-counting via DFS is correct because: (1) DFS from an unvisited land cell visits exactly the cells in its component (closure under adjacency); (2) marking visited prevents re-counting; (3) the outer loop ensures every cell is examined; (4) we increment count only on the first cell of each component. Termination follows from finite grid size.
Complexity
| Operation | Time | Space |
|---|---|---|
| Whole algorithm | O(R · C) | O(R · C) recursion or O(min(R,C)) for BFS |
Implementation Requirements
def numIslands(grid):
if not grid:
return 0
R, C = len(grid), len(grid[0])
DIRS = [(-1, 0), (1, 0), (0, -1), (0, 1)]
count = 0
for r0 in range(R):
for c0 in range(C):
if grid[r0][c0] != '1':
continue
count += 1
stack = [(r0, c0)]
while stack:
r, c = stack.pop()
if grid[r][c] != '1':
continue
grid[r][c] = '0'
for dr, dc in DIRS:
nr, nc = r + dr, c + dc
if 0 <= nr < R and 0 <= nc < C and grid[nr][nc] == '1':
stack.append((nr, nc))
return count
Tests
- All water:
[["0"]]→ 0. - All land:
[["1"]*5 for _ in range(5)]→ 1. - Diagonal 1s only: 4-connected → many islands.
- Single column:
[["1"],["0"],["1"],["0"],["1"]]→ 3. - Single row: same logic.
- Snake pattern: alternating
1rows /0rows → R/2 components. - Large: 300 × 300 random — must complete in <1s.
Follow-up Questions
- “Max area of an island.” (LC 695) → DFS returns area; track max.
- “Number of distinct islands” (LC 694) → record canonical-form path of each DFS; dedupe by hash.
- “Surrounded regions” (LC 130) → DFS from border, mark; flip rest.
- “Add land online and report island count after each addition” (LC 305) → DSU.
- “Are two grids the same modulo rotation?” — open-ended modeling; involves shape signatures.
Product Extension
Connected-component analysis underpins image segmentation (region labeling in computer vision), ACL group expansion in identity systems, and graph-clustering for fraud-ring detection. The grid is just a constrained adjacency; the structure generalizes to any sparse adjacency.
Language/Runtime Follow-ups
- Python: recursion limit at 1000 means recursive DFS fails at large grids. Use iterative or
sys.setrecursionlimit(10**6). List-as-stack is fast;tuplekeys in any auxiliary structures are fine. - Java: write
int[][] gridorchar[][]. UseDeque<int[]>(ArrayDeque) for stack;int[]{r, c}instead ofPairfor performance. Don’t useStack(legacy synchronized class). - Go:
[][]byteor[][]int32. Slice-as-stack:s = s[:len(s)-1]. Two-int struct (type cell struct{ r, c int }) avoids slice allocation per push. - C++:
vector<vector<char>>orvector<string>.stack<pair<int,int>>. Useemplace_backto avoid copies. - JS/TS: arrays as stacks (
push/popare O(1)). For 2D grids prefergrid[r][c]over flat-array indexing for clarity; the speed diff is negligible at N = 9 × 10^4.
Common Bugs
- Bounds check missing on neighbor (negative or out-of-grid index).
- Marking visited after pushing instead of on push — same node enters stack many times via different neighbors.
- Recursive DFS without
sys.setrecursionlimitblowing the stack on 300 × 300 dense islands. - Mutating the input when the caller didn’t allow it — clarify in interviews.
- Treating
'1'as integer1(or vice versa) — equality check fails silently. - Off-by-one on
RandC(using< Rvs<= R). - Forgetting one of the four directions (typo in DIRS).
Debugging Strategy
For a 3 × 3 grid, print the grid after each DFS call to verify in-place marking. Add a print((r, c)) on each pop and verify the 4 neighbors are correctly considered. If count is too high, you’re probably re-counting a visited component (visited check missing). If too low, your DFS is exiting early — check the bounds and the equality on '1' vs 1.
Mastery Criteria
- Recognized “count connected groups in a grid” as DFS/BFS in <30 seconds.
- Wrote both recursive and iterative DFS from cold start in <8 minutes total.
- Handled the recursion-limit trap correctly when asked about 10^4 × 10^4 grids.
- Stated O(R · C) complexity unprompted.
- Solved LC 200 in <10 minutes from cold start.
- Solved LC 695 (Max Area) in <12 minutes by extending the template.
- Solved LC 130 (Surrounded Regions) in <20 minutes by inverting the search.
- Articulated when DSU is preferable to DFS (online updates, no spatial constraint).
Lab 03 — Multi-Source BFS (Rotting Oranges)
Goal
Implement multi-source BFS on a grid to compute the minimum time for a process to spread from multiple simultaneous starting points. After this lab you should be able to recognize the multi-source BFS signal (any “infection / spread / nearest-source” problem) in <60 seconds, initialize the queue correctly with all sources at distance 0, and write the layer-by-layer time-tracking logic without off-by-ones.
Background Concepts
Multi-source BFS is single-source BFS with a virtual super-source connected to all real sources by zero-weight edges. We don’t materialize the super-source; we just enqueue all real sources at distance 0 simultaneously. The BFS then proceeds layer by layer, and each cell’s distance is min over all sources of (path length to that source). Critically, this is O(V + E) — same as single-source BFS — not O(K · (V + E)) for K sources.
The “rotting oranges” problem asks: given a grid where some cells contain rotten fruit (sources) and some contain fresh fruit (targets), how many time steps until all fresh fruit rots? Each minute, every rotten orange infects its 4-connected fresh neighbors. The answer is the maximum BFS distance among fresh oranges, or -1 if any fresh orange is unreachable.
Interview Context
Rotting Oranges (LC 994) appears at Amazon, Meta, and Microsoft phone screens regularly. The trap is candidates running single-source BFS K times — one per rotten cell — which is O(K · R · C) and blows up at K = R · C / 2. Multi-source BFS is the senior signal here; recognizing it within the first 90 seconds and stating it explicitly differentiates a strong L4 from a struggling one.
Problem Statement
Given an m × n grid where each cell is:
0: empty,1: fresh orange,2: rotten orange,
each minute every rotten orange rots its 4-connected fresh neighbors. Return the minimum minutes until no fresh orange remains, or -1 if some fresh orange can never rot.
Constraints
- 1 ≤ m, n ≤ 10
grid[i][j]∈ {0, 1, 2}- (Note: small grid in LC 994; the algorithm scales to 10^4 × 10^4 trivially.)
Clarifying Questions
- If the grid has no fresh oranges initially, what’s the answer? (0 — already done.)
- If a rotten orange has no fresh neighbors and there are no fresh oranges anywhere, return 0. If there are unreachable fresh oranges, return -1.
- Are ties between sources broken consistently? (Doesn’t matter — we want minimum distance, which is unambiguous.)
- Can a rotten orange “re-rot” a previously-rotted cell? (No — once rotten, stays rotten.)
Examples
grid = [[2,1,1],
[1,1,0],
[0,1,1]]
→ 4
grid = [[2,1,1],
[0,1,1],
[1,0,1]]
→ -1 (bottom-left is unreachable)
grid = [[0,2]]
→ 0
Initial Brute Force
Simulate minute by minute: at each step, find every rotten orange, infect its fresh neighbors, count rotted cells. Repeat until no change. Each step is O(R · C); total steps ≤ R + C; total O((R + C) · R · C) — at 10 × 10 trivially fast, but doesn’t scale.
Brute Force Complexity
O(R · C · max-distance). At 10 × 10, ~10^4 ops. Passes LC bounds easily but is “embarrassing” — interviewer wants the BFS framing.
Optimization Path
Multi-source BFS gives the optimal O(R · C):
- Scan grid: count fresh oranges; enqueue every rotten orange at distance 0.
- BFS layer by layer; on rotting a fresh cell, decrement the fresh count and enqueue at next distance.
- After BFS, if fresh count > 0, return -1; else return the maximum distance reached.
The simulation is mathematically equivalent but presents better in interviews — it’s the canonical multi-source signal.
Final Expected Approach
fresh_count = count of '1' in grid
queue = deque of (r, c, 0) for every '2' in grid
time = 0
while queue:
r, c, t = queue.popleft()
time = max(time, t)
for each 4-neighbor (nr, nc):
if in bounds and grid[nr][nc] == 1:
grid[nr][nc] = 2
fresh_count -= 1
queue.append((nr, nc, t + 1))
return -1 if fresh_count > 0 else time
Data Structures Used
collections.dequeof(row, col, time)tuples.- The input grid as the visited marker (mutate fresh → rotten on rot).
- An integer
fresh_count.
Correctness Argument
The super-source argument: imagine a virtual node S₀ connected to every initial rotten cell by a zero-weight edge. BFS from S₀ visits cells in non-decreasing distance order; each cell’s distance is 1 + min over rotten cells of (path length). By initializing with all rotten cells at distance 0 instead, we get the same distances. The “minutes until no fresh remains” equals the maximum BFS distance among initially-fresh cells; this is exactly the time the last cell rots. Unreachable fresh cells are detected by the fresh_count > 0 check post-BFS.
Complexity
| Operation | Time | Space |
|---|---|---|
| Initial scan | O(R · C) | O(R · C) for queue worst case |
| BFS | O(R · C) | (already counted) |
| Total | O(R · C) | O(R · C) |
Implementation Requirements
from collections import deque
def orangesRotting(grid):
R, C = len(grid), len(grid[0])
queue = deque()
fresh = 0
for r in range(R):
for c in range(C):
if grid[r][c] == 2:
queue.append((r, c, 0))
elif grid[r][c] == 1:
fresh += 1
if fresh == 0:
return 0
DIRS = [(-1, 0), (1, 0), (0, -1), (0, 1)]
time = 0
while queue:
r, c, t = queue.popleft()
time = t
for dr, dc in DIRS:
nr, nc = r + dr, c + dc
if 0 <= nr < R and 0 <= nc < C and grid[nr][nc] == 1:
grid[nr][nc] = 2
fresh -= 1
queue.append((nr, nc, t + 1))
return -1 if fresh > 0 else time
Tests
- All rotten: 0.
- All fresh, no rotten: -1.
- Mixed with one isolated fresh: -1.
- Single rotten in corner, all fresh: distance = R + C - 2.
- Empty grid (all zeros): 0.
- 1×1 grid with
[[0]]: 0;[[1]]: -1;[[2]]: 0. - Stress: 10×10 random; verify against simulation brute force.
Follow-up Questions
- “What if rotting takes a different amount of time per cell?” → Edge weights vary; use Dijkstra.
- “What if there are obstacle cells?” → Add
grid[nr][nc] == 0skip; same algorithm. - “What if oranges can only rot orthogonally to one direction?” → Replace DIRS with the allowed subset.
- “Walls and Gates (LC 286): from each empty room, find distance to nearest gate.” → Same multi-source BFS pattern; gates are the sources.
- “01 Matrix (LC 542): for each cell, distance to nearest 0.” → Sources are the 0s; targets are the 1s.
Product Extension
Spreadable processes — disease propagation in epidemiology, fire spread in simulations, viral content propagation in social graphs, “blast radius” of a deployment failure across a service mesh — all map to multi-source BFS. The minute-by-minute simulation in production tracking systems is exactly this algorithm.
Language/Runtime Follow-ups
- Python:
collections.dequeis the canonical queue. Tuples for(r, c, t). Avoidlist.pop(0)— that’s O(N). - Java:
ArrayDeque<int[]>withint[]{r, c, t}. UsepollFirst/offerLast. - Go: a slice as queue (
q[1:]is O(1) amortized, or use container/list). A[3]intstruct is fine. - C++:
queue<tuple<int,int,int>>.emplacefor efficiency.auto [r, c, t] = q.front()in C++17. - JS/TS: array
shift()is O(N) — use a deque or pointer-based queue.queueMicrotaskis irrelevant here.
Common Bugs
- Counting time as
time + 1after the BFS terminates — the last layer’s t is already correct. - Forgetting to enqueue all rotten cells before starting — single-source BFS, missing K-1 sources.
- Initializing time to -1 vs 0 to handle the “no fresh” case — corner case.
- Decrementing
fresh_counttoo late, double-decrementing on revisits. - BFS correctly terminating but reporting
tfrom the wrong layer (e.g., the last enqueued, not last popped). - Mutating
1to2outside the bounds check. - Mistaking the answer “minutes” for “max steps” when the grid has no fresh oranges (answer is 0, not “the BFS terminates immediately”; you must short-circuit).
Debugging Strategy
Trace a 3 × 3 grid by hand: print the queue contents and grid after each pop. Verify time increments by exactly 1 between layers. If fresh > 0 at the end, identify the unreachable cell — confirm it has no path to any source by visual inspection. If time is off by 1, you’re probably tracking time = t + 1 on enqueue instead of t on pop.
Mastery Criteria
- Recognized “spread from multiple sources” as multi-source BFS in <60 seconds.
- Initialized the queue with all sources at distance 0 unprompted.
- Wrote correct BFS with time tracking from cold start in <8 minutes.
- Stated O(R · C) complexity unprompted; explained why running K single-source BFSs is wrong.
- Solved LC 994 in <12 minutes from cold start.
- Solved LC 286 (Walls and Gates) in <12 minutes by extending the template.
- Solved LC 542 (01 Matrix) in <12 minutes.
- Articulated the super-source equivalence in <30 seconds.
Lab 04 — Dijkstra (Network Delay Time)
Goal
Implement Dijkstra’s algorithm (lazy variant, binary heap) for single-source shortest path on a non-negative-weighted directed graph. After this lab you should be able to write Dijkstra from a blank screen in <8 minutes, including the staleness-skip line; recognize the non-negative-weight signal in <30 seconds; and adapt the template to “shortest path with constraints” (e.g., max K edges) by extending the state.
Background Concepts
Dijkstra’s algorithm computes the shortest path from a source s to every other node in a graph with non-negative edge weights. The core invariant: when a node u is extracted from the priority queue (heap), its tentative distance dist[u] is final. The proof relies on non-negativity: any other path to u must go through some node w not yet extracted, with dist[w] ≥ dist[u], and the path’s total length is dist[w] + (non-negative tail) ≥ dist[u].
Two variants:
- Lazy: push every relaxation
(new_dist, neighbor)to the heap; on pop, skip ifnew_dist > dist[neighbor](stale entry). Heap holds up to E entries. Simpler. - Eager: maintain a decrease-key indexed heap; each node appears at most once. Faster constants, more code.
In interviews, lazy is the default. State that explicitly.
Interview Context
Dijkstra appears in the top 5 graph algorithms tested at FAANG. Network Delay Time (LC 743) is the canonical version, asked at Google, Amazon, and Bloomberg. Cheapest Flights (LC 787) is the constrained variant. The senior signal is: state “this is Dijkstra, weights are non-negative, lazy variant with binary heap, O((V+E) log V)” within the first two minutes, before writing code. Candidates who don’t articulate this and just dive in lose points even if the code works.
Problem Statement
You are given a network of n nodes labeled 1..n. A list times of edges where times[i] = (u, v, w) means it takes w time for a signal to travel from u to v. A signal is sent from node k. Return the minimum time for all nodes to receive the signal, or -1 if some node never receives it.
Constraints
- 1 ≤ n ≤ 100, 1 ≤ |times| ≤ 6000
- 1 ≤ u, v, k ≤ n; u ≠ v
- 0 ≤ w ≤ 100
- Edges are directed; possibly multi-edges.
Clarifying Questions
- Are weights non-negative? (Yes — Dijkstra applies.)
- Are nodes 1-indexed? (Yes — adjust array sizes accordingly.)
- Are duplicate edges possible? (Yes — they’re allowed; minimum-weight edge between (u, v) is what matters effectively.)
- Is the graph guaranteed connected? (No — return -1 if unreachable.)
- What’s the answer if the graph has only the source? (0 — already received.)
Examples
times = [[2,1,1],[2,3,1],[3,4,1]], n=4, k=2
→ 2 (signal reaches 1, 3 at time 1; 4 at time 2)
times = [[1,2,1]], n=2, k=1
→ 1
times = [[1,2,1]], n=2, k=2
→ -1
Initial Brute Force
Bellman-Ford: V - 1 rounds of relaxing all E edges. Time O(V · E). At V=100, E=6000: 6 × 10^5 ops — passes easily but is the wrong answer when the interviewer asks complexity.
Alternative brute force: BFS treating equal-weight edges. Wrong answers on weighted graphs unless all weights = 1.
Brute Force Complexity
Bellman-Ford: O(V · E) = 6 × 10^5. BFS-as-Dijkstra: incorrect for varying weights.
Optimization Path
Dijkstra with binary heap: O((V + E) log V) = ~5 × 10^4 with these constraints. The right answer.
For very dense graphs (E ~ V²), simple Dijkstra without a heap (just scan for the min unsettled node) is O(V²) and can be faster. Floyd-Warshall is O(V³) all-pairs — overkill for single-source, but valid here at V=100.
Final Expected Approach
- Build adjacency list from edge list.
adj[u]is a list of(weight, neighbor)pairs. dist[i] = ∞for all i exceptdist[k] = 0.- Push
(0, k)to a min-heap. - While heap non-empty: pop
(d, u); ifd > dist[u], skip (stale); else relax all edgesu → v: ifd + w < dist[v], update and push. - After the loop,
max(dist[1..n])is the answer; if any is ∞, return -1.
Data Structures Used
- Adjacency list:
dict[int, list[(int, int)]]orlist[list[(int, int)]]indexed 1..n. - Distance array:
list[int]of size n+1, init toinf. - Priority queue: Python’s
heapq(min-heap).
Correctness Argument
Loop invariant: when (d, u) is popped from the heap with d == dist[u], that distance is final. Proof: any other path to u goes through some node w not yet popped (else dist[u] would have been updated to a smaller value). Since w is unsettled, dist[w] ≥ dist[u] (by heap order). The path’s total length is dist[w] + tail ≥ dist[u] (non-negativity of tail). So dist[u] is optimal.
Termination: each node is finalized at most once (the staleness-skip ensures repeated pops don’t re-relax). At most V finalizations + E heap pushes: O((V + E) log V).
Complexity
| Operation | Time | Space |
|---|---|---|
| Build adjacency | O(V + E) | O(V + E) |
| Dijkstra | O((V + E) log V) | O(V + E) heap |
| Total | O((V + E) log V) | O(V + E) |
Implementation Requirements
import heapq
from collections import defaultdict
def networkDelayTime(times, n, k):
adj = defaultdict(list)
for u, v, w in times:
adj[u].append((w, v))
INF = float('inf')
dist = [INF] * (n + 1)
dist[k] = 0
heap = [(0, k)]
while heap:
d, u = heapq.heappop(heap)
if d > dist[u]:
continue
for w, v in adj[u]:
nd = d + w
if nd < dist[v]:
dist[v] = nd
heapq.heappush(heap, (nd, v))
ans = max(dist[1:n+1])
return -1 if ans == INF else ans
Tests
- Standard: 4-node star → 2.
- Disconnected: source can’t reach a node → -1.
- Single node, source = only node → 0.
- Duplicate edges: pick min weight on relaxation.
- Self-loop (problem disallows but defend): doesn’t affect answer; skip.
- Stress: V=100, E=6000 random, compare against Bellman-Ford reference.
- Adversarial: dense graph forcing many heap pushes.
Follow-up Questions
- “Now find shortest path with at most K edges.” (LC 787) → Bellman-Ford OR Dijkstra with state
(node, edges_used). See Lab 05. - “Now weights can be negative.” → Dijkstra is wrong; use Bellman-Ford.
- “Find path with maximum probability” (LC 1514) → Dijkstra with max-heap and product (or
-logweights and standard Dijkstra). - “All-pairs shortest path.” → Run Dijkstra from every node, O(V · (V+E) log V), or Floyd-Warshall O(V³).
- “Graph evolves online.” → Recompute on each query, or use dynamic shortest-path structures (advanced).
Product Extension
Network monitoring tools use Dijkstra-like algorithms for path-cost estimation. Routing protocols like OSPF use Dijkstra (link-state routing) to compute shortest paths in IP networks. CDN edge selection, traffic engineering, and request routing in microservices meshes all rely on shortest-path computations parameterized by latency, cost, or QoS.
Language/Runtime Follow-ups
- Python:
heapqis a min-heap; for max-heap, negate weights or use tuples(-w, ...). Tuple comparison is element-by-element —(d, u)compares bydfirst. - Java:
PriorityQueue<int[]>with comparator on the weight index, orPriorityQueue<long[]>if weights overflow. Don’t usePairfrom JavaFX (deprecated). - Go:
container/heaprequires implementing theheap.Interface. Tedious; many candidates inline a slice-based heap. For small N, even O(V²) Dijkstra without heap is fine. - C++:
priority_queue<pair<int,int>, vector<pair<int,int>>, greater<>>for min-heap. Or negate weights with default max-heap. - JS/TS: no built-in heap. Use a library (
heap-js) or hand-roll. For small N, an O(V²) scan is acceptable.
Common Bugs
- Forgetting the staleness check
if d > dist[u]: continue— the heap pops the same node multiple times after stale updates; without the skip, you re-relax incorrectly and blow up complexity. - Pushing
(u, dist[u])instead of(dist[u], u)— heap orders on first element, so distance must come first. - 1-indexed vs 0-indexed off-by-one. The problem is 1-indexed; size arrays at
n + 1. - Initial
distnot set to ∞ — using 0 makes the source’s neighbors look “already optimal”. - Pushing the source to the heap but forgetting to set
dist[source] = 0. - Returning
min(dist)(which catches the unreachable -∞) instead ofmax(dist)(which is the actual question). - Negative weights — Dijkstra silently produces wrong answers; you won’t see this fail unless you stress-test.
Debugging Strategy
Print the heap contents and dist array after each pop. Verify the popped distance equals dist[u] for non-stale entries. For wrong answers, trace a specific node v where dist[v] is wrong: identify the predecessor u that should have relaxed it; check that (d_u + w_uv) is computed correctly. For complexity blowup, check that the staleness skip is present and triggers.
Mastery Criteria
- Recognized “shortest path with non-negative weights” as Dijkstra in <30 seconds.
- Wrote Dijkstra from blank screen with the staleness check in <8 minutes.
- Stated O((V + E) log V) complexity unprompted.
- Articulated why Dijkstra fails on negative weights in <30 seconds.
- Solved LC 743 in <15 minutes from cold start.
- Solved LC 1631 (Path With Minimum Effort) in <20 minutes by adapting weights = max along path.
- Solved LC 778 (Swim in Rising Water) in <20 minutes.
- Stated lazy vs eager difference in <30 seconds.
Lab 05 — Bellman-Ford (Cheapest Flights Within K Stops)
Goal
Implement Bellman-Ford for shortest path on a graph that may contain negative weights, and exploit its iteration structure to solve the canonical “shortest path with at most K edges” problem. After this lab you should be able to recognize the K-edge-budget signal in <60 seconds, write Bellman-Ford from a blank slate in <10 minutes, and articulate the negative-cycle detection extension cleanly.
Background Concepts
Bellman-Ford runs V - 1 rounds; in each round, relax all E edges. After round i, dist[v] equals the shortest path from source to v using at most i edges. This invariant is the key insight for “K-edge-budget” problems: run the algorithm for K rounds (or K + 1, depending on edge-vs-stop semantics) and read off the answer. A V-th round that still relaxes any edge proves a negative cycle reachable from the source.
The complexity is O(V · E). It tolerates negative weights (unlike Dijkstra) but is slower on graphs without them. The “shortest path with at most K edges” framing is the most common interview reason to use Bellman-Ford.
Interview Context
Cheapest Flights Within K Stops (LC 787) is asked at Amazon, Bloomberg, and Adobe. The trap is candidates reaching for Dijkstra and being unable to bound edge count. The strong answer: “this is Bellman-Ford with K + 1 iterations, where K stops means K + 1 edges, complexity O(K · E).” That single sentence wins the round. Negative-cycle detection (e.g., currency arbitrage) is rarer at L4-L5 but standard at staff and on competitive programming exercises.
Problem Statement
You are given n cities and a list flights[i] = (from, to, price) of directed flights. Return the cheapest price from src to dst using at most K stops (i.e., K intermediate cities, K + 1 edges). Return -1 if no such route exists.
Constraints
- 1 ≤ n ≤ 100; 0 ≤ |flights| ≤ n · (n − 1) / 2
- 1 ≤ price ≤ 10^4
- 0 ≤ src, dst, K < n; src ≠ dst
Clarifying Questions
- “K stops” — does this mean K intermediate cities (so K + 1 edges) or K edges total? (Per LC 787 statement: K stops = K intermediate = K + 1 edges.)
- Are prices positive? (Yes; no negative-cycle concerns here.)
- Are duplicate flights allowed? (Possible; pick min on relaxation.)
- Should we count
srcanddstas “stops”? (No — those are endpoints.) - Is K = 0 allowed (direct flight only)? (Yes —
0 ≤ K.)
Examples
n=3, flights=[[0,1,100],[1,2,100],[0,2,500]], src=0, dst=2, K=1
→ 200 (0 → 1 → 2 uses 1 stop)
n=3, flights=[[0,1,100],[1,2,100],[0,2,500]], src=0, dst=2, K=0
→ 500 (only direct allowed)
Initial Brute Force
DFS from src exploring all paths up to K + 1 edges; track minimum cost. Time exponential — O(V^(K+1)) — at K = 99, V = 100: infeasible.
Brute Force Complexity
O(V^(K+1)) — TLE for K > ~10.
Optimization Path
Bellman-Ford with K + 1 iterations: O((K + 1) · E). At K = 99, E ~ 5000: 5 × 10^5 ops — fast. The key technique: keep two distance arrays, prev and curr. In each iteration, compute curr[v] = min(prev[u] + w_uv) over all edges. Using prev (last round’s snapshot) prevents using more than one edge per round.
Modified Dijkstra with state (node, edges_used) also works: push (cost, node, edges_remaining) to a heap, expand only if edges_remaining > 0. Slightly slower than Bellman-Ford for this problem because the heap doesn’t prune effectively; both are correct.
Final Expected Approach
prev[i] = ∞ for all i; prev[src] = 0
for round in 1 .. K + 1:
curr = prev.copy()
for (u, v, w) in flights:
if prev[u] + w < curr[v]:
curr[v] = prev[u] + w
prev = curr
return prev[dst] if prev[dst] != ∞ else -1
The curr = prev.copy() ensures each round uses only edges from the previous round’s distances — bounding the path to at most one new edge per round.
Data Structures Used
- Two arrays of size n:
prevandcurr. - The flight list as the edge list (no need to build adjacency).
Correctness Argument
Invariant: after round i, prev[v] = shortest path from src to v using at most i edges. Proof by induction. Base (i = 0): only src has prev = 0, all others ∞. Inductive step: any shortest i-edge path is either an (i-1)-edge path (already in prev[v]) or extends some (i-1)-edge path to u with edge (u, v); the inner loop catches the latter via prev[u] + w_uv → curr[v]. After K + 1 rounds, prev[dst] is the shortest at-most-K+1-edge path.
Why two arrays: without the copy, in-place updates could chain multiple edges in one round, breaking the at-most-i-edges invariant.
Complexity
| Operation | Time | Space |
|---|---|---|
| Whole algorithm | O((K + 1) · E) | O(V) |
Implementation Requirements
def findCheapestPrice(n, flights, src, dst, K):
INF = float('inf')
prev = [INF] * n
prev[src] = 0
for _ in range(K + 1):
curr = prev[:]
for u, v, w in flights:
if prev[u] + w < curr[v]:
curr[v] = prev[u] + w
prev = curr
return prev[dst] if prev[dst] != INF else -1
Tests
- Standard: 0 → 1 → 2 with K=1 → 200; K=0 → 500.
- No path: disconnected → -1.
- Direct flight only: K=0, no direct → -1.
- K very large (≥ V - 1): equivalent to unrestricted Bellman-Ford.
- Tie: two K-stop paths with same cost — return the cost.
- Stress: V=100 dense, K=99 random — verify against modified Dijkstra.
Follow-up Questions
- “Negative weights? Negative cycles?” → Run V rounds (not K); if round V relaxes any edge, negative cycle reachable from src exists.
- “All-pairs shortest path with negative weights, no cycles.” → Johnson’s algorithm: Bellman-Ford to reweight, then Dijkstra from each source.
- “Currency arbitrage detection.” → Build graph with
weight = -log(rate); negative cycle = profitable arbitrage. - “K is up to 10^9.” → Matrix exponentiation on the (min, +) semiring; O(V³ log K).
- “Online updates: new flights added live.” → Difficult. Restart Bellman-Ford on each query; or maintain incrementally with limited optimizations.
Product Extension
Travel-search engines (Kayak, Google Flights, Hipmunk) treat flight networks as graphs and apply variants of K-stop shortest path. The “fewest connections” filter is exactly K-stop. The “cheapest with up to 2 stops” is K=2 Bellman-Ford. Currency-arbitrage bots run Bellman-Ford continuously on FX-rate graphs to detect profit cycles in milliseconds.
Language/Runtime Follow-ups
- Python:
prev[:]is O(V) and acceptable.array.array('d', ...)for floats can speed up cache locality. - Java:
int[] prev = new int[n]; Arrays.fill(prev, Integer.MAX_VALUE);Watch overflow when summing — uselongif weights × edges can overflowint. - Go:
prev := make([]int, n)with manual init to a large constant;copy(curr, prev)for the snapshot. Builtinmath.MaxInt32is fine. - C++:
vector<int> prev(n, INT_MAX);Uselong longif weights are large.std::copyfor the snapshot. - JS/TS:
Array.from({length: n}, () => Infinity)andprev.slice()for copy.
Common Bugs
- In-place updates without the prev/curr split — chains multiple edges per round, gives wrong answers.
- Running K rounds instead of K + 1 — off by one (K stops = K + 1 edges).
- Treating “K stops” as K edges — read carefully; LC 787 means K + 1 edges.
- Forgetting to copy
prevtocurrat the start of each round — uses stale curr from previous iteration. - Returning
prev[dst]without the unreachable check; INF leaks into output. - Integer overflow on
prev[u] + wwhenprev[u]is set toINT_MAX— guard with the unreachable check first. - Confusing “K = 0 means direct only” with “K = 0 means src only”.
Debugging Strategy
For a small case, print prev at the end of each round. Verify that round 1 has dist[v] = w(src → v) for direct neighbors only; round 2 has 2-edge paths, etc. If the answer is wrong, compare with a brute-force enumeration of paths up to K + 1 edges. For negative-cycle problems, verify that round V actually relaxes an edge by tracking a “changed” flag.
Mastery Criteria
- Recognized “shortest path with at most K edges” as Bellman-Ford in <60 seconds.
- Articulated the iteration-as-edge-budget invariant unprompted.
- Wrote Bellman-Ford from blank screen with prev/curr split in <10 minutes.
- Stated O((K + 1) · E) complexity unprompted.
- Solved LC 787 in <20 minutes from cold start.
- Articulated negative-cycle detection in <30 seconds when asked.
- Articulated when Dijkstra is preferable (no edge budget, non-negative weights) in <30 seconds.
Lab 06 — Topological Sort (Alien Dictionary)
Goal
Build a topological sort over an inferred constraint graph — a problem whose graph is not given but must be extracted from the input. After this lab you should be able to recognize “ordering with constraints” problems in <60 seconds, write Kahn’s algorithm from a blank screen in <8 minutes, and identify and handle the three degenerate cases of alien-dictionary parsing (prefix violation, no constraint differs, cyclic dependency).
Background Concepts
A topological sort orders the vertices of a DAG such that every directed edge u → v has u before v. Kahn’s algorithm repeatedly removes a node of in-degree 0, decrementing its neighbors’ in-degrees. If the final order has length V, the graph is a DAG and the order is valid. Otherwise, a cycle exists.
For alien dictionary, the input is a list of words known to be sorted lexicographically in some unknown alien alphabet. Each adjacent pair of words gives at most one ordering constraint between two characters: the first position where they differ tells you a < b for those characters. Build the constraint graph, run topological sort, output an order. Handle three degenerate inputs:
- No constraint differs between adjacent words but the second is a prefix of the first (e.g.,
["abc", "ab"]) — invalid; return"". - No constraint differs but it’s because the words are equal up to a common length and the second is the longer one (
["ab", "abc"]) — fine; no constraint added. - Cycle in the constraint graph — invalid; return
"".
Interview Context
Alien Dictionary (LC 269) is one of the most-asked Hard graph problems at Meta, Google, and Airbnb. Premium-only on LeetCode but widely leaked. It tests: (1) recognizing topological sort, (2) constraint extraction from non-graph input, (3) handling all three edge cases. Candidates who solve only the happy path lose major points. The senior signal: enumerate the three failure modes upfront, before writing code.
Problem Statement
There is a new alien language using English letters. The order of letters is unknown. Given a list of words sorted lexicographically by the alien language’s rules, return any valid letter ordering. If no valid ordering exists, return "". If multiple are valid, return any.
Constraints
- 1 ≤ |words| ≤ 100
- 1 ≤ |words[i]| ≤ 100
- All words consist of lowercase English letters.
Clarifying Questions
- Is the answer unique? (No — any valid topological order.)
- Does the answer include letters not appearing in any word? (No — only letters that appear.)
- Are duplicate words possible? (Possible; treat normally — they yield no constraint.)
- What does “lexicographically sorted” mean for words of different length? (Standard prefix rule; if A is a prefix of B, then A < B; if B is a prefix of A, the input is invalid.)
- What if
wordsis a single word? (Output any permutation of its unique letters.)
Examples
words = ["wrt","wrf","er","ett","rftt"]
→ "wertf" (one valid ordering)
words = ["z","x"]
→ "zx"
words = ["z","x","z"]
→ "" (z and x must precede each other — cycle)
words = ["abc","ab"]
→ "" (prefix violation — "abc" can't come before "ab")
Initial Brute Force
Enumerate all permutations of the alphabet of size ≤ 26; for each, verify that the input words are in that lexicographic order. O(26!) — infeasible.
Brute Force Complexity
O(26! · ΣL) — astronomical.
Optimization Path
The answer requires O(V + E) where V = number of distinct letters (≤ 26) and E = number of pairwise constraints (≤ |words| - 1). Kahn’s algorithm runs in O(V + E). Constraint extraction is O(Σ |words[i]|).
The total is O(V + E + Σ L) — bounded by Σ L since V ≤ 26. Trivially fast.
Final Expected Approach
- Initialize
in_degree[c] = 0for every letter that appears anywhere. - Build adjacency: for each adjacent pair
(w1, w2)inwords:- Walk both words in parallel; at the first index
iwhere they differ, add edgew1[i] → w2[i](if not already present); break. - If no differing index found and
len(w1) > len(w2): prefix violation — return"".
- Walk both words in parallel; at the first index
- Run Kahn’s: queue all letters with
in_degree[c] == 0; pop, append to order, decrement neighbors. - If the order length equals the number of distinct letters, return the order; else cycle → return
"".
Data Structures Used
defaultdict(set)for adjacency (set prevents duplicate edges).dict[char, int]forin_degree.collections.dequefor the Kahn queue.list[char]for the result.
Correctness Argument
Each adjacent pair contributes at most one constraint — the first differing character. This is sound: if the words are correctly sorted, the relative order of w1[i] and w2[i] (at the first differing index) must be w1[i] < w2[i] in the alien alphabet. No constraint can be inferred from differences after the first; those are consistent with but not implied by the sortedness.
Topological sort over these constraints produces an ordering where every constraint a < b is satisfied (a precedes b in the output). If a cycle exists, the constraints are unsatisfiable and the input is impossible. The prefix-violation case is the only constraint-extraction-time invalid input.
Complexity
| Operation | Time | Space |
|---|---|---|
| Constraint extraction | O(Σ L) | O(unique edges) ≤ O(26²) |
| Kahn’s | O(V + E) | O(V + E) |
| Total | O(Σ L) | O(unique letters + edges) |
Implementation Requirements
from collections import defaultdict, deque
def alienOrder(words):
adj = defaultdict(set)
in_deg = {c: 0 for w in words for c in w}
for i in range(len(words) - 1):
w1, w2 = words[i], words[i+1]
found = False
for j in range(min(len(w1), len(w2))):
if w1[j] != w2[j]:
if w2[j] not in adj[w1[j]]:
adj[w1[j]].add(w2[j])
in_deg[w2[j]] += 1
found = True
break
if not found and len(w1) > len(w2):
return ""
queue = deque([c for c, d in in_deg.items() if d == 0])
order = []
while queue:
c = queue.popleft()
order.append(c)
for nb in adj[c]:
in_deg[nb] -= 1
if in_deg[nb] == 0:
queue.append(nb)
return "".join(order) if len(order) == len(in_deg) else ""
Tests
- Standard:
["wrt","wrf","er","ett","rftt"]→ some topo ofw<e, r<t, t<f. - Single word:
["abc"]→ some permutation of{a,b,c}. - Prefix violation:
["abc","ab"]→"". - Tie/equal words:
["a","a"]→"a". - Cycle:
["z","x","z"]→"". - All same length, all letters used:
["aa","ab","cb"]. - Single letter:
["z"]→"z". - Long words, tiny alphabet: stress for in-degree correctness.
Follow-up Questions
- “Find the lexicographically smallest valid order (in standard a-z order).” → Kahn’s with a min-heap instead of a queue.
- “Find all valid orders.” → Backtracking over Kahn’s choices; exponential.
- “Verify a given ordering.” → For each adjacent word pair, scan for the first differing char and check ordering. O(Σ L).
- “Online: words arrive one by one.” → Maintain the adjacency incrementally; rerun Kahn’s lazily on query.
- “What if the input has typos (wrongly-ordered pairs)?” → Return any consistent ordering, or report the conflict edge.
Product Extension
Build systems (Bazel, Make, Gradle) compute build orders via topological sort over the dependency DAG; cycle detection is a critical correctness property. Database query planners use topo sort over join-graph dependencies. Distributed task schedulers (Airflow, Argo) execute DAGs of jobs in topological order.
Language/Runtime Follow-ups
- Python:
defaultdict(set)andcollections.dequeare essential.dict.items()iteration is fine. - Java:
Map<Character, Set<Character>>andMap<Character, Integer>for in-degree;ArrayDeque<Character>for queue. Useint[26]for in-degree if alphabet is fixed. - Go:
map[byte]map[byte]boolfor adjacency;map[byte]intfor in-degree; slice as queue. Or[26]intfor in-degree as alphabet is fixed. - C++:
unordered_map<char, unordered_set<char>>;array<int, 26>for in-degree;queue<char>. - JS/TS:
Map<string, Set<string>>andMap<string, number>; array-as-queue with care (shiftis O(N)).
Common Bugs
- Adding duplicate edges to in-degree — use a set for adjacency, check membership before incrementing.
- Missing the prefix violation check —
["abc","ab"]returns"abc"if you don’t handle this. - Building in-degree only for letters that have outgoing edges, missing letters that only appear as targets.
- Initializing
in_degreeonly for the first word’s letters — letters appearing only in later words get missed. - Comparing
len(order) == 26instead of== len(in_deg)(only used letters count). - Using a list instead of a set for adjacency, then double-incrementing in-degree.
- Returning the order in the wrong direction (Kahn’s gives the right direction; DFS post-order needs to be reversed).
Debugging Strategy
Print the adjacency and in-degree maps after constraint extraction. Verify each constraint is justified by tracing back to the input pair. For cycles, print the in-degree map at the point Kahn’s stalls — the remaining-positive in-degrees identify nodes in the cycle. For prefix-violation false negatives, print the pair (w1, w2) at each iteration to confirm the check fires.
Mastery Criteria
- Recognized “ordering with constraints” as topological sort in <60 seconds.
- Wrote constraint extraction from word pairs from cold start in <5 minutes.
- Wrote Kahn’s algorithm from blank screen in <6 minutes.
- Enumerated the three degenerate cases (cycle, prefix violation, equal-prefix-shorter-second) before coding.
- Solved LC 269 in <25 minutes from cold start.
- Solved LC 207 (Course Schedule) in <8 minutes by extracting the constraint structure from cold.
- Articulated the white-path lemma / DFS-post-order alternative in <60 seconds.
Lab 07 — Union-Find Applications (Accounts Merge)
Goal
Implement a disjoint-set union (DSU) with path compression and union by rank, then apply it to a real merge problem where the “elements” are emails and the “groups” are accounts. After this lab you should be able to write DSU from a blank screen in <6 minutes, recognize the merge-by-shared-attribute signal in <60 seconds, and articulate when DSU beats DFS for connectivity (online updates, no spatial structure, simple connectivity-only queries).
Background Concepts
A disjoint-set union (DSU, aka union-find) maintains a partition of N elements under two operations:
find(x): return the representative (“root”) of x’s set.union(x, y): merge the sets containing x and y.
With path compression (find rewrites every visited node to point directly at the root) and union by rank/size (always attach the smaller tree under the larger), both operations run in O(α(N)) amortized, where α is the inverse Ackermann function — effectively constant for any practical N.
DSU is the natural choice when you receive a stream of “merge x and y” operations and need to answer “are x and y in the same group” — and you don’t care about paths between them, only connectivity.
Interview Context
Accounts Merge (LC 721) is a top-tier Hard at Amazon and Google. Number of Provinces (LC 547) is the easier sibling at Meta. The trap: candidates default to BFS/DFS on the implicit graph (emails as nodes, “shared email between two accounts” as edges), which works but is messier code than DSU. The senior signal is recognizing the partition structure and reaching for DSU within 90 seconds.
Problem Statement
Given a list of accounts, where accounts[i] = [name, email1, email2, ...], two accounts belong to the same person if they share any common email (names alone are not enough — multiple people can share a name). Merge accounts: return a list where each element is [name, ...sorted unique emails], accounts in any order.
Constraints
- 1 ≤ |accounts| ≤ 1000
- 2 ≤ |accounts[i]| ≤ 10 (one name + 1..9 emails)
- 1 ≤ |email| ≤ 30
- Emails are lowercase, contain
@.
Clarifying Questions
- Are emails case-sensitive? (Per LC: lowercase already.)
- Two accounts with the same name but no shared email — are they merged? (No — names don’t merge.)
- Should the output emails be sorted within each account? (Yes — alphabetically.)
- Order of accounts in output? (Any order is accepted.)
- Total emails: ≤ 1000 × 9 = 9000 — DSU on emails is fine.
Examples
accounts = [
["John","[email protected]","[email protected]"],
["John","[email protected]","[email protected]"],
["Mary","[email protected]"],
["John","[email protected]"]
]
→ [["John","[email protected]","[email protected]","[email protected]"],
["Mary","[email protected]"],
["John","[email protected]"]]
Initial Brute Force
Build implicit graph: each email is a node; for each account, connect all its emails to the first email of that account; run DFS/BFS to enumerate connected components; emit each component with the corresponding name. Works, but DSU is cleaner.
Brute Force Complexity
O(Σ |emails| · α) with DSU; O(Σ |emails|) with DFS — both linear in total email count. The DFS version requires building an adjacency list explicitly, which DSU skips.
Optimization Path
DSU directly:
- Treat each unique email as a DSU element.
- For each account, union all its emails to the first email.
- After all unions, group emails by
find(email)root. - For each group, attach the name (looked up via any email in the group → its account → the account’s name).
- Sort emails within each group; output.
This is the cleanest expression. No explicit graph construction needed.
Final Expected Approach
parent = {}
def find(x): if parent[x] != x: parent[x] = find(parent[x]); return parent[x]
def union(x, y): parent[find(x)] = find(y)
email_to_name = {}
for account in accounts:
name = account[0]
for email in account[1:]:
if email not in parent: parent[email] = email
email_to_name[email] = name
union(account[1], email)
groups = defaultdict(list)
for email in parent: groups[find(email)].append(email)
return [[email_to_name[group[0]]] + sorted(group) for group in groups.values()]
Data Structures Used
dict[str, str]forparent(DSU).dict[str, str]foremail_to_name.defaultdict(list)for grouping by root.
Correctness Argument
DSU correctness: Initially every element is its own set. Each union merges two sets. find returns a canonical representative. After path compression, find(x) == find(y) iff they were ever transitively unioned. Path compression and union by rank/size preserve this invariant and amortize each op to α(N).
Reduction correctness: Two emails belong to the same person iff there is a chain of accounts where consecutive accounts share an email. The unions on each account’s emails form precisely these chains; the resulting partition matches the equivalence-class definition.
Output correctness: Each component’s name is unambiguous because (a) every account contributing to the component has the same name as the others in that component (otherwise they’d be different people, and the input is well-formed by problem statement), and (b) any email in the group recovers the name via email_to_name.
Complexity
| Operation | Time | Space |
|---|---|---|
| Building DSU | O(Σ E · α) where E = total emails | O(Σ E) |
| Grouping + sort | O(Σ E log E) for sorting within each group | O(Σ E) |
| Total | O(Σ E log E) | O(Σ E) |
Implementation Requirements
from collections import defaultdict
class DSU:
def __init__(self):
self.parent = {}
self.rank = {}
def find(self, x):
if self.parent[x] != x:
self.parent[x] = self.find(self.parent[x])
return self.parent[x]
def union(self, x, y):
rx, ry = self.find(x), self.find(y)
if rx == ry: return
if self.rank[rx] < self.rank[ry]: rx, ry = ry, rx
self.parent[ry] = rx
if self.rank[rx] == self.rank[ry]: self.rank[rx] += 1
def add(self, x):
if x not in self.parent:
self.parent[x] = x
self.rank[x] = 0
def accountsMerge(accounts):
dsu = DSU()
email_to_name = {}
for account in accounts:
name = account[0]
first = account[1]
for email in account[1:]:
dsu.add(email)
email_to_name[email] = name
dsu.union(first, email)
groups = defaultdict(list)
for email in dsu.parent:
groups[dsu.find(email)].append(email)
return [[email_to_name[g[0]]] + sorted(g) for g in groups.values()]
Tests
- Standard: 3-account merge → 1 merged + 2 separate.
- All accounts disjoint → each emerges separately.
- All accounts share one email → all merge into one.
- Single account → unchanged.
- Same name, different emails → separate accounts.
- Empty emails (problem disallows but defend): account[1:] is empty → no unions, no groups; account is dropped if no emails. Verify behavior.
Follow-up Questions
- “Number of Provinces (LC 547): given an N × N adjacency matrix, count groups.” → DSU over N nodes; union if
M[i][j] == 1; count distinct roots. - “Online: accounts arrive in a stream.” → DSU handles this natively; just keep adding and unioning.
- “What if names matter (same name + shared email merges, different name doesn’t)?” → Keep DSU but check name-compatibility before union; conflict means error or skip.
- “What if you need to remove an account?” → DSU doesn’t support remove. Use Link-Cut Trees or rebuild from scratch.
- “What if path compression isn’t allowed (read-only
find)?” → Use union by rank only; O(log N) per op instead of α.
Product Extension
Identity-resolution at LinkedIn, Salesforce, and ad networks merges user records by shared email/phone using DSU. Image segmentation libraries (OpenCV’s connectedComponents) use DSU under the hood. Distributed-system membership-protocols use DSU-like merges to track partition healing. Kruskal’s MST (Lab 08) uses DSU as its core data structure.
Language/Runtime Follow-ups
- Python: recursion in
findmay exceed limit at N > 10^4; use iterative two-pass (find root, then compress). - Java:
int[] parentfor integer keys is significantly faster thanHashMap<Integer, Integer>. Use iterativefind. - Go:
parent := make(map[string]string)for string keys, or[]intfor integer indices. - C++:
vector<int> parent(N); iota(parent.begin(), parent.end(), 0);is the clean pattern. Iterativefind. - JS/TS:
Map<string, string>is fine; use iterativefindto avoid call-stack issues at large N.
Common Bugs
- Recursion limit in Python’s
find— at N = 10^4 with worst-case chains, blows the stack. Use iterative or increase limit. - Forgetting path compression —
findbecomes O(N), not α. Functionality correct but TLE. - Union without rank — same TLE risk on adversarial inputs.
- Comparing
parent[x] == xvsfind(x) == xfor “is root” — onlyparent[x] == xis correct;find(x) == xis always true afterfindrewrites. - Forgetting to add an email to
parentbefore union (unioncallsfindwhich dereferencesparent[email]) — KeyError. - Mapping
email_to_nameper-account but overwriting — last write wins; usually fine here, but be deliberate. - Not deduplicating emails within an account (LC inputs may not, but the algorithm is robust either way).
Debugging Strategy
Print parent and rank after each union. For wrong groupings, trace which email failed to union with which other and which account broke the chain. For TLE, verify both path compression and union by rank are present; profile to confirm find dominates.
Mastery Criteria
- Recognized “merge by shared attribute” as DSU in <60 seconds.
- Wrote DSU with path compression and union by rank from blank screen in <6 minutes.
- Stated O(α) amortized complexity unprompted.
- Articulated when DFS is a valid alternative (offline, no online updates) and when DSU is mandatory (online stream of merges).
- Solved LC 547 (Number of Provinces) in <8 minutes.
- Solved LC 721 (Accounts Merge) in <20 minutes from cold start.
- Solved LC 305 (Number of Islands II) in <20 minutes — the canonical online-DSU problem.
- Articulated path compression’s effect on amortization in <60 seconds.
Lab 08 — MST via Kruskal (Min Cost to Connect All Points)
Goal
Build a minimum spanning tree (MST) on a complete graph derived from N points using Kruskal’s algorithm. After this lab you should be able to recognize the MST signal in <60 seconds, write Kruskal from a blank screen (sort + DSU + early-exit) in <8 minutes, and reason about when Kruskal beats Prim (sparse graphs, edge list already given) and vice versa (dense graphs, adjacency matrix).
Background Concepts
A spanning tree of a connected graph G is a subgraph that includes all V vertices and exactly V - 1 edges with no cycles. The MST is the spanning tree with minimum total edge weight. Two canonical algorithms:
- Kruskal’s: Sort all edges by weight ascending. Iterate; for each edge, union the endpoints if they’re in different components (use DSU); add to MST. Stop after V - 1 edges. Time O(E log E).
- Prim’s: Start from any vertex; maintain a min-heap of crossing edges; repeatedly extract the lightest edge to a new vertex. Time O(E log V) with a binary heap; O(E + V log V) with a Fibonacci heap.
For “connect all points” with edge weights = pairwise Manhattan distance, the graph is complete: E = V·(V-1)/2 ≈ V². At V = 1000, E ≈ 5 × 10^5 edges. Either algorithm works; Kruskal with DSU is the cleanest expression because we already have the edge list.
Interview Context
Min Cost to Connect All Points (LC 1584) is a Medium asked at Amazon, Bloomberg, and Salesforce. It’s a clean MST signal: “minimum total cost to make everything connected.” The senior signal is naming the problem ("this is MST on a complete graph") within 60 seconds, then choosing Kruskal vs Prim consciously based on density. Strong candidates also state the Cut Property as the correctness foundation.
Problem Statement
Given an array points where points[i] = [xi, yi] represents a point in 2D, the cost of connecting two points is the Manhattan distance between them: |xi - xj| + |yi - yj|. Return the minimum cost to connect all points, where any two points are connected if there is a path between them.
Constraints
- 1 ≤ |points| ≤ 1000
- −10^6 ≤ xi, yi ≤ 10^6
- All points are distinct.
Clarifying Questions
- Manhattan, Euclidean, or other metric? (Manhattan, per problem.)
- Are diagonal connections counted? (Implicitly yes — we connect any two points directly.)
- Are points distinct? (Yes, per constraint.)
- Single point — cost? (0, no edges needed.)
- Should the answer fit in 32-bit? (Max cost ≈ 999 · 4 × 10^6 ≈ 4 × 10^9 — use 64-bit just in case, though Python int is unbounded.)
Examples
points = [[0,0],[2,2],[3,10],[5,2],[7,0]]
→ 20
points = [[3,12],[-2,5],[-4,1]]
→ 18
points = [[0,0]]
→ 0
Initial Brute Force
Enumerate all spanning trees and pick the one with minimum total weight. There are exponentially many. Infeasible.
A second “brute force” is Prim’s via array-scan (no heap): O(V²). At V = 1000: 10^6 ops — passes easily and is simpler than the heap version. This is actually a competitive option for this problem.
Brute Force Complexity
Spanning-tree enumeration: O(V^(V-2)) by Cayley’s formula. Infeasible. Array-scan Prim: O(V²). At V = 1000: 10^6 ops, well within budget.
Optimization Path
For dense graphs (E ~ V²), array-scan Prim is O(V²) — wins over Kruskal’s O(E log E) = O(V² log V). For sparse graphs, Kruskal or heap-Prim wins.
For LC 1584 specifically (V = 1000, dense), all three pass:
- Kruskal: O(V² log V²) = O(V² log V) = ~10^7 ops; passes in 1-2s.
- Heap-Prim: O(V² log V); same.
- Array-scan Prim: O(V²); fastest.
In interviews, Kruskal is the “safer” choice because the code is mechanical: edges → sort → DSU → loop. Show you can choose array-scan Prim when asked about dense graphs.
Final Expected Approach (Kruskal)
- Generate all V·(V-1)/2 edges with weight = Manhattan distance.
- Sort by weight ascending.
- Initialize DSU with V components.
- Iterate edges; if endpoints differ, union and add weight to total; count edges added.
- Stop when edges added == V - 1 (early exit).
Data Structures Used
- List of edges as
(weight, u, v)tuples. - DSU as in Lab 07 (path compression + union by rank).
- Integer accumulator for total cost.
Correctness Argument
Cut Property: For any cut (partition of vertices into two non-empty sets), the minimum-weight edge crossing the cut belongs to some MST. Kruskal greedily picks the lightest edge that doesn’t create a cycle (i.e., the lightest edge crossing some cut between two components); by the Cut Property, this edge is safe — there is an MST containing it. Repeating this V - 1 times produces an MST.
No-cycle invariant: DSU’s find ensures we add an edge only when its endpoints are in different components. Since adding an edge between same-component endpoints creates a cycle, this is exactly the cycle-prevention check.
Termination: Each union reduces component count by 1; after V - 1 unions, the graph is connected. We stop early.
Complexity
| Operation | Time | Space |
|---|---|---|
| Edge generation | O(V²) | O(V²) |
| Sort | O(V² log V) | (in-place possible) |
| DSU loop | O(V² · α) | O(V) |
| Total | O(V² log V) | O(V²) |
Implementation Requirements
class DSU:
def __init__(self, n):
self.parent = list(range(n))
self.rank = [0] * n
def find(self, x):
while self.parent[x] != x:
self.parent[x] = self.parent[self.parent[x]] # path compression by halving
x = self.parent[x]
return x
def union(self, x, y):
rx, ry = self.find(x), self.find(y)
if rx == ry: return False
if self.rank[rx] < self.rank[ry]: rx, ry = ry, rx
self.parent[ry] = rx
if self.rank[rx] == self.rank[ry]: self.rank[rx] += 1
return True
def minCostConnectPoints(points):
n = len(points)
edges = []
for i in range(n):
xi, yi = points[i]
for j in range(i + 1, n):
xj, yj = points[j]
edges.append((abs(xi - xj) + abs(yi - yj), i, j))
edges.sort()
dsu = DSU(n)
total, count = 0, 0
for w, u, v in edges:
if dsu.union(u, v):
total += w
count += 1
if count == n - 1: break
return total
Tests
- Standard: 5 points → 20.
- Single point: 0.
- Two points: their Manhattan distance.
- Collinear points (all on x-axis): MST weight = max(x) - min(x).
- Stress: V = 1000 random — verify Kruskal and Prim agree.
- Adversarial: points on a grid — many edges of equal weight (test tie-breaking is stable).
Follow-up Questions
- “Connecting Cities With Minimum Cost (LC 1135)” → Same MST template; if multiple MSTs valid, return any total cost; -1 if disconnected.
- “Optimize Water Distribution in a Village (LC 1168)” → Add a virtual node 0 connected to each house with the well-cost; run MST on V + 1 nodes.
- “Critical Connections (bridges)” → Different problem (Tarjan’s bridge algorithm), see Phase README.
- “Maximum Spanning Tree.” → Sort descending; same algorithm.
- “Online edge insertion: maintain MST.” → Link-Cut Trees; advanced.
Product Extension
Network design (laying fiber, planning power grids), clustering algorithms (single-linkage clustering = MST followed by cut-largest-edges), image segmentation, and approximation algorithms for TSP all use MST as a primitive. AWS / Azure data-center backbone planning uses MST variants weighted by latency × cost.
Language/Runtime Follow-ups
- Python:
edges.sort()on tuples sorts lexicographically —(weight, u, v)works.heapqis overkill since we need all edges sorted upfront, not on-demand. - Java:
Arrays.sort(int[][])with a comparator on the weight column. Useint[]triples for cache locality. - Go:
sort.Slice(edges, func(i, j int) bool { return edges[i].w < edges[j].w }). - C++:
vector<tuple<int,int,int>>with default<ordering;sort(edges.begin(), edges.end()). - JS/TS:
edges.sort((a, b) => a[0] - b[0]). Avoida[0] < b[0]returning a boolean (subtle bug).
Common Bugs
- Forgetting the early exit when edges added = V - 1 — works but processes more edges than needed.
- Generating duplicate edges (i.e., (i, j) and (j, i)) — wastes time but doesn’t break correctness.
- Off-by-one in V - 1 — counting edges incorrectly leads to incomplete tree.
- Comparator returns boolean in JS — use subtraction.
- Integer overflow on edge weights — Manhattan distance bounded by 4 × 10^6; sum bounded by ~4 × 10^9, fits in 64-bit. In Python no issue; in Java use
long. - DSU’s
unionreturns nothing vs returns success boolean — pick a consistent API. - Returning the count of edges instead of total weight — silent.
Debugging Strategy
For small inputs, print the sorted edge list and the DSU state after each union. Verify the total edges added is exactly V - 1. If the result is too large, check that you’re not adding edges within the same component (the cycle check might be skipped). Compare against a Prim’s reference for stress tests.
Mastery Criteria
- Recognized “minimum cost to connect” as MST in <60 seconds.
- Wrote Kruskal from blank screen in <8 minutes.
- Chose Kruskal vs Prim consciously based on density when asked.
- Stated the Cut Property as the correctness foundation in <30 seconds.
- Solved LC 1584 in <20 minutes from cold start.
- Solved LC 1135 (Connecting Cities) in <12 minutes by extending the template.
- Solved LC 1168 (Water Distribution) in <15 minutes with the virtual-node trick.
- Stated the V² Prim option for dense graphs in <30 seconds.
Lab 09 — Graph Modeling (Bus Routes)
Goal
Practice the modeling skill that separates competent graph candidates from L5+ candidates: given a problem with no obvious graph, invent the right node and edge definition. After this lab you should be able to enumerate at least two valid graph models for a given problem, choose the one with the smallest state space, and justify the choice in <90 seconds.
Background Concepts
The hardest interview graph questions don’t say “graph.” Examples:
- Bus Routes (LC 815): “fewest buses to take” — model: nodes = bus routes (not stops!); edges = “two routes share a stop”; multi-source BFS from all routes containing the source stop.
- Word Ladder (LC 127): nodes = words; edges = one-character difference. (See Lab 01.)
- Open the Lock (LC 752): nodes = 4-digit states; edges = single-digit ±1; BFS.
- Sliding Puzzle (LC 773): nodes = board states (encode as string); edges = legal swap; BFS.
The modeling decision space is:
- What is a node? — Often the natural object (stop, word, board state) is wrong; a higher-order or quotient object yields a smaller state space.
- What is an edge? — Direct adjacency (one move), or “shared resource” (two routes share a stop).
- Is it weighted? — If yes → Dijkstra. If unweighted → BFS.
- Sources/targets? — Single, multiple, or “any-of-set” (multi-source BFS).
- Implicit vs explicit? — Build the adjacency upfront or compute on the fly.
The Bus Routes trap: candidates model nodes = stops, edges = “stops on the same route.” Then BFS distance is steps within a route, not number of routes taken. The whole question collapses to nonsense. Modeling nodes = routes makes the BFS distance number of routes, which is the answer ± 1.
Interview Context
Bus Routes (LC 815) is asked at Google, Meta, and Amazon at L5+. The question itself is mid-level once modeled correctly; the difficulty is the modeling. Interviewers probe modeling explicitly: “tell me how you’d represent this as a graph” — this is the test. If you flounder for 5 minutes, you’re done. The senior signal: state both modelings (stops vs routes), explain why the route-model gives the right BFS distance, then code.
Problem Statement
You are given an array routes where routes[i] is a bus route that the ith bus repeats forever. For example, routes[0] = [1, 5, 7] means the 0th bus travels in the sequence 1 → 5 → 7 → 1 → 5 → 7 → … forever. You start at source and want to reach target. You can travel between stops by buses only. Return the fewest number of buses required, or -1 if impossible.
Constraints
- 1 ≤ |routes| ≤ 500
- 1 ≤ |routes[i]| ≤ 10^5
- Σ |routes[i]| ≤ 10^5
- 0 ≤ routes[i][j] < 10^6
- 0 ≤ source, target < 10^6
Clarifying Questions
- Do
source == targetcases return 0? (Yes — no bus needed.) - Is
sourceguaranteed to be on some route? (Not necessarily — return -1 if not.) - Are routes circular as stated? (Yes — but that doesn’t matter for “buses taken”; what matters is which stops a route covers.)
- Are stop numbers unique within a route? (Per LC, yes — but treat defensively.)
- Can two routes share a stop? (Yes — that’s exactly how transfers happen.)
Examples
routes = [[1,2,7],[3,6,7]], source = 1, target = 6
→ 2 (take bus 0 from 1 to 7, then bus 1 from 7 to 6)
routes = [[7,12],[4,5,15],[6],[15,19],[9,12,13]], source = 15, target = 12
→ -1
Initial Brute Force
BFS over stops as nodes and edges between any two stops sharing a route. Build adjacency: for each route, add all (stop, stop’) pairs as edges. At Σ |routes| = 10^5, with a 10^5-stop route, that’s 5 × 10^9 edges — TLE / OOM.
Brute Force Complexity
O((Σ L)²) edges in the worst case. Infeasible at Σ L = 10^5.
Optimization Path
Switch the modeling: nodes = routes, edges = “two routes share a stop.” Build a stop → list of routes containing it index in O(Σ L). For each pair of routes sharing a stop, that’s an edge — but we never enumerate all such pairs explicitly. Instead, in BFS, when we visit route r, we expand to every other route sharing any of r’s stops, looked up via the index.
To avoid revisiting, mark routes (not stops) as visited. Also mark stops as visited (after expanding all routes through that stop) to avoid the O(L²) blowup of re-expanding a popular stop.
Final BFS distance = number of routes used. Answer is the BFS layer at which we find any route containing target.
Final Expected Approach
- If
source == target, return 0. - Build
stop_to_routes: dict of stop → set of route indices. - BFS over routes. Initialize queue with all routes containing
source, distance = 1. - For each popped
(route, d), scan all stops inroutes[route]. If any istarget, return d. - Mark each stop visited (skip if already). For each unvisited stop, enqueue all unvisited routes containing it, distance d + 1.
- If queue empties without finding
target, return -1.
Data Structures Used
defaultdict(set)forstop_to_routes.collections.dequefor BFS queue.setfor visited routes and visited stops.
Correctness Argument
Distance interpretation: Initializing the queue with routes containing source at distance 1 means: distance d = “number of routes taken so far, including the current one.” When a route at distance d covers target, the answer is d.
No double-counting: Marking stops visited prevents the O(L²) blowup; marking routes visited prevents re-expansion. Both are needed: a stop is visited once we’ve added all its routes; a route is visited once we’ve expanded its stops.
BFS optimality: Standard BFS argument on the route-graph — first time a route is popped, its distance is minimum. Therefore the first time a route containing target is popped, the answer is its distance.
Complexity
| Operation | Time | Space |
|---|---|---|
| Index build | O(Σ L) | O(Σ L) |
| BFS | O(Σ L + R²) where R = routes count | O(Σ L) |
| Total | O(Σ L + R²) | O(Σ L) |
(R² because in the worst case every pair of routes shares some stop and we add an edge.)
Implementation Requirements
from collections import defaultdict, deque
def numBusesToDestination(routes, source, target):
if source == target:
return 0
stop_to_routes = defaultdict(set)
for i, r in enumerate(routes):
for s in r:
stop_to_routes[s].add(i)
if source not in stop_to_routes:
return -1
visited_routes = set()
visited_stops = {source}
queue = deque()
for r in stop_to_routes[source]:
queue.append((r, 1))
visited_routes.add(r)
while queue:
route, d = queue.popleft()
for stop in routes[route]:
if stop == target:
return d
if stop in visited_stops:
continue
visited_stops.add(stop)
for nr in stop_to_routes[stop]:
if nr not in visited_routes:
visited_routes.add(nr)
queue.append((nr, d + 1))
return -1
Tests
- Standard:
[[1,2,7],[3,6,7]], src=1, tgt=6 → 2. - src == tgt: → 0.
- src not on any route: → -1.
- Single route covering both: → 1.
- Disconnected routes: → -1.
- Long route (10^5 stops, 1 route): src and tgt on it → 1; tgt not on it → -1.
- Many routes sharing one hub: BFS expansion through hub.
Follow-up Questions
- “What if buses have different costs?” → Dijkstra over routes with edge weight = cost.
- “What’s the actual sequence of buses taken?” → Maintain parent pointers in BFS; reconstruct.
- “What if you can walk between adjacent stops?” → Add walking edges; might or might not change the model.
- “Multi-source: any of K starting stops to any of M targets.” → Multi-source BFS on the route-graph.
- “Online: routes added/removed live.” → Recompute on each query, or use dynamic-graph techniques.
Product Extension
Real transit-routing systems (Google Maps, Citymapper, Apple Maps) model transit as a multi-modal graph: nodes are stops, walking edges connect nearby stops, transit edges represent “ride a route between two of its stops.” The route-as-node model used here is a simplification useful for “fewest transfers” queries. Production systems combine with time-dependent shortest-path (CSA, RAPTOR) for actual journey planning.
Language/Runtime Follow-ups
- Python:
defaultdict(set)anddeque. The inner loop scansroutes[route]once per route popped — bound this by visiting stops only once. - Java:
HashMap<Integer, Set<Integer>>for the index;ArrayDeque<int[]>for(route, distance). - Go:
map[int]map[int]boolormap[int][]int. Slice-as-queue. - C++:
unordered_map<int, vector<int>>.queue<pair<int,int>>. Reserve capacity if Σ L is known. - JS/TS:
Map<number, Set<number>>and array-as-queue (use a deque polyfill if N large).
Common Bugs
- Modeling stops as nodes — leads to O((Σ L)²) edge count and TLE.
- Returning d instead of d - 1 (or vice versa) — make sure the distance semantics match: d = number of buses taken.
- Not marking the source stop as visited — re-expanding routes through it.
- Not marking routes as visited — exponential queue blowup.
- Returning -1 when source == target instead of 0.
- Forgetting that
source not in stop_to_routesis a -1 case. - Stack overflow in BFS (no — BFS is iterative; this is a DFS-only problem).
Debugging Strategy
For a 3-route example, print the stop_to_routes index. Walk through BFS by hand: queue contents, visited sets after each pop. Verify d increases exactly once per layer of routes. If TLE, profile to confirm visited-stop marking is preventing repeated route expansion. If wrong answer, check whether you’re returning d or d - 1.
Mastery Criteria
- Recognized “fewest buses” as a graph problem in <30 seconds.
- Enumerated both stop-as-node and route-as-node models in <90 seconds.
- Articulated why route-as-node yields the correct distance unprompted.
- Wrote BFS with both visited-routes and visited-stops sets in <12 minutes.
- Stated O(Σ L + R²) complexity unprompted.
- Solved LC 815 in <30 minutes from cold start.
- Solved LC 752 (Open the Lock) in <20 minutes by encoding state as a string.
- Solved LC 773 (Sliding Puzzle) in <30 minutes by encoding the 2D board as a string state.
- When given a new “no obvious graph” problem, can produce a correct model in <3 minutes.
Lab 10 — Strongly Connected Components & 2-SAT
Goal
Implement Tarjan’s one-pass SCC algorithm iteratively (explicit stack — mandatory in Python at n = 10⁵), understand the condensation DAG, and apply SCC to its killer application: 2-SAT in O(n + m) with assignment extraction. After this lab: iterative Tarjan in <20 minutes, the 2-SAT reduction from memory in <10.
Background Concepts
An SCC is a maximal set of mutually reachable vertices. Tarjan finds all of them in one DFS using two numbers per vertex: disc[v] (discovery time) and low[v] = min disc reachable from v’s subtree using any number of tree edges plus at most one edge into a still-open SCC.
Stack invariant: vertices are pushed on a side stack at discovery and popped only when their SCC’s root closes. A vertex stays on the stack exactly while its SCC is still open — i.e., it can still reach an earlier unfinished vertex. When DFS finishes v with low[v] == disc[v], v is the first-discovered vertex (root) of its SCC, and its SCC is precisely v plus everything above it on the stack. Pop them together.
Because an SCC closes only after every SCC it reaches has closed, Tarjan emits SCCs in reverse topological order — component ids double as a reverse topo sort for free.
Kosaraju is the two-pass alternative: DFS once recording finish order, then DFS the reversed graph in decreasing finish order; each second-pass tree is one SCC. Easier to re-derive under pressure, but needs the reverse graph and two passes.
Condensation: contract each SCC to one node → a DAG. Every DAG technique now applies: topo-order DP, longest path, reachability.
2-SAT: variables x₀..x_{n−1}, clauses (a ∨ b). Each clause is two implications: ¬a→b and ¬b→a (twin edges — always both). Build this implication graph on 2n literal-vertices. The formula is satisfiable iff no variable shares an SCC with its negation. A valid assignment: set each literal true iff its SCC comes later in topological order than its negation’s.
Interview Context
SCC is a staple at trading firms and infra teams (deadlock detection, dependency cycles). 2-SAT appears disguised as “pairwise constraints: pick one of two options per item, some pairs conflict — feasible?” Recognizing the implication-graph reduction is a senior-level signal. LC has no direct 2-SAT problem (stated honestly below); Codeforces and interviews at HFT shops do. Course Schedule (LC 207/210) is the condensation warm-up; don’t confuse LC 1192 (Critical Connections) — that’s bridges, the undirected twin, next lab.
Problem Statement
2-SAT (Codeforces-style; LeetCode has no canonical 2-SAT problem — this is the honest framing). Given n_vars boolean variables and m clauses, each (l₁ ∨ l₂) where a literal is a variable or its negation, decide satisfiability and return one satisfying assignment or report none. Literal encoding: 2v = x_v, 2v+1 = ¬x_v; negation is lit ^ 1.
Constraints
- 1 ≤ n_vars, m ≤ 10⁵ → implication graph has 2·10⁵ vertices, 2m edges.
- O(n + m) required; recursion depth may hit 2·10⁵ → iterative DFS.
Clarifying Questions
- Exactly two literals per clause? (Yes — three makes it 3-SAT, NP-complete.)
- May a clause repeat a literal, e.g. (x ∨ x)? (Yes — it forces x true; the twin edges handle it.)
- One assignment, all, or a count? (One; counting solutions is #P-complete.)
- Any satisfying assignment, or lexicographically smallest? (Any — lex-smallest needs a different, greedy technique.)
Examples
# (x0 ∨ x1) ∧ (¬x0 ∨ x1) ∧ (¬x1 ∨ x2)
solve_2sat(3, [(0, 2), (1, 2), (3, 4)]) → [False, True, True] (one valid answer)
# x0 ∧ ¬x0, encoded as (x0 ∨ x0) ∧ (¬x0 ∨ ¬x0)
solve_2sat(1, [(0, 0), (1, 1)]) → None
Initial Brute Force
Enumerate all 2ⁿ assignments; check every clause for each.
Brute Force Complexity
O(2ⁿ · m). Unusable past n ≈ 25; at n = 10⁵, astronomically dead.
Optimization Path
Key observation: (a ∨ b) ≡ (¬a → b) ∧ (¬b → a). Chains of implications are paths; “x forces ¬x and ¬x forces x” is exactly “x and ¬x mutually reachable” — one SCC. So satisfiability reduces to SCC computation: Tarjan (one pass) or Kosaraju (two passes), both O(n + m). No search, no DPLL — 2-SAT is polynomial precisely because clauses are binary implications.
Final Expected Approach
def tarjan_scc(n, adj):
"""Iterative Tarjan. Returns (count, comp); comp ids are assigned in
reverse topological order: edge u→v across SCCs ⇒ comp[u] > comp[v]."""
UNSEEN = -1
disc = [UNSEEN] * n
low = [0] * n
on_stk = [False] * n
comp = [UNSEEN] * n
stk, work = [], []
timer = count = 0
for s in range(n):
if disc[s] != UNSEEN:
continue
work.append((s, 0))
while work:
v, i = work[-1]
if i == 0: # first visit
disc[v] = low[v] = timer
timer += 1
stk.append(v)
on_stk[v] = True
if i < len(adj[v]):
work[-1] = (v, i + 1)
w = adj[v][i]
if disc[w] == UNSEEN:
work.append((w, 0)) # tree edge: recurse
elif on_stk[w]:
low[v] = min(low[v], disc[w]) # back/cross into open SCC
else: # v is finished
work.pop()
if work:
low[work[-1][0]] = min(low[work[-1][0]], low[v])
if low[v] == disc[v]: # v is its SCC's root
while True:
w = stk.pop()
on_stk[w] = False
comp[w] = count
if w == v:
break
count += 1
return count, comp
def solve_2sat(n_vars, clauses):
"""clauses: list of (l1, l2) meaning l1 ∨ l2.
Literal encoding: 2v = x_v, 2v+1 = ¬x_v; negate with lit ^ 1.
Returns a satisfying list of bools, or None."""
adj = [[] for _ in range(2 * n_vars)]
for a, b in clauses:
adj[a ^ 1].append(b) # ¬a → b
adj[b ^ 1].append(a) # ¬b → a
_, comp = tarjan_scc(2 * n_vars, adj)
assign = []
for v in range(n_vars):
if comp[2 * v] == comp[2 * v + 1]:
return None # x and ¬x entangled
# smaller comp id = later in topo order = "chosen" literal
assign.append(comp[2 * v] < comp[2 * v + 1])
return assign
Data Structures Used
- Adjacency lists over 2·n_vars literal-vertices.
disc,low,compint arrays;on_stkboolean array.- Two explicit stacks:
workholds(vertex, next-edge-index)frames (the recursion replacement);stkis Tarjan’s SCC stack.
Correctness Argument
Root detection: low[v] == disc[v] after v’s subtree finishes means no vertex in the subtree reaches an earlier open vertex. So v’s SCC is entirely within the stack segment above v (everything pushed after v that couldn’t escape below it) — pop it as one component. Induction over pop order makes each popped set maximal.
Reverse topo ids: if SCC A has an edge into SCC B (A→B, A ≠ B), DFS finishes B’s root strictly before A’s — so comp[B] < comp[A]. Verified as an explicit edge-property check in the tests.
2-SAT, necessity: x and ¬x in one SCC gives x→¬x and ¬x→x — either value of x forces its own negation. UNSAT.
2-SAT, sufficiency of the assignment rule: the implication graph is skew-symmetric — every edge l→l′ has a mirror ¬l′→¬l. Rule: literal l is chosen iff comp[l] < comp[¬l]. Suppose a chosen l implies a rejected l′: then comp[l] ≥ comp[l′] (edge, reverse-topo ids) and, by the mirror edge ¬l′→¬l, comp[¬l′] ≥ comp[¬l]. Chain: comp[¬l] ≤ comp[¬l′] < comp[l′] ≤ comp[l] < comp[¬l] — contradiction. So the chosen set is closed under implication and hits exactly one of each pair: every clause’s implications are satisfied.
Complexity
| Step | Time | Space |
|---|---|---|
| Build implication graph | O(n + m) | O(n + m) |
| Tarjan SCC | O(n + m) | O(n) |
| Extract assignment | O(n) | O(n) |
Implementation Requirements
- Iterative DFS is non-negotiable: 2·10⁵ recursion depth kills Python even with
sys.setrecursionlimit(the C stack overflows before the Python limit triggers). - The
i == 0guard must gate the first-visit block — frames are re-entered after each child. on_stkcheck before usingdisc[w]: edges into closed SCCs must not updatelow.- Always add both twin edges per clause;
lit ^ 1for negation keeps this one line each.
Tests
- The two fixed examples above (satisfiable with forced x₁, x₂; and x ∧ ¬x → None).
- SCC: 300 random digraphs (n ≤ 12) vs a reachability-based brute force — same partition, same count, plus the reverse-topo edge property
comp[u] > comp[v]for every cross-SCC edge. - 2-SAT: 400 random instances (n ≤ 8, up to 3n clauses) vs full 2ⁿ enumeration — satisfiability agrees (307 SAT / 93 UNSAT in the verified run), and every returned assignment is checked clause-by-clause.
- Scale: a 10⁵-vertex directed cycle → exactly 1 SCC, no recursion error.
Follow-up Questions
- Tarjan vs Kosaraju — when would you pick each? → Tarjan: one pass, no reversed graph, ids come out in reverse topo order for free. Kosaraju: two simple DFSes, trivial to re-derive when nervous, but needs G-reversed (extra O(n+m) memory) and touches every edge twice. Same asymptotics; in an interview, name both and write the one you’re fluent in.
- Why does reverse topological order give a valid assignment? → The skew-symmetry chain above: a chosen literal implying a rejected one forces
comp[¬l] < comp[¬l]. The intuition: picking the topologically-later SCC means “nothing you imply can escape into a rejected literal,” because implications only move to smaller comp ids. - What DP runs on the condensation? → Any DAG DP: longest path where node weight = SCC size (“longest chain of mutually-reachable groups”), reachability counting, min path cover. Process SCCs in comp-id order (already reverse topo) — no separate topo sort needed.
- Edges arrive online — maintain SCCs incrementally? → Genuinely hard; known incremental algorithms are amortized and complex. Practical answer: offline batching — collect edges, recompute, or divide-and-conquer over the edge timeline. Saying “I’d batch offline” is the right interview move.
- Why does 3-SAT break this? → (a ∨ b ∨ c) has no binary-implication form: falsifying one literal only yields a disjunction, not an implication edge. That’s the 2→3 cliff — 2-SAT is in P (even NL-complete), 3-SAT is NP-complete. Interviewers probe whether you know the reduction fails structurally, not just “3-SAT is hard.”
- Where does 2-SAT show up in disguise? → “Each item has two placements, some pairs of placements conflict” — radio frequencies, seating, flag placement on a map, choosing one of two orientations per component.
Product Extension
A feature-flag dependency engine: rules like “flag A requires flag B” (¬A ∨ B) and “A conflicts with B” (¬A ∨ ¬B) are 2-SAT; the solver either emits a valid flag configuration or pinpoints the entangled variable pair (comp[x] == comp[¬x]) as the human-readable conflict. Same machinery: package managers detecting dependency cycles via SCC, and deadlock detectors condensing wait-for graphs.
Language/Runtime Follow-ups
- Python: iterative only —
sys.setrecursionlimit(3*10⁵)still segfaults the C stack on deep chains. Use flat int lists, not dicts; localizeadjlookups in the hot loop. - C++: recursive Tarjan is fine to ~10⁶ depth with an enlarged stack (or compilers’ default 8 MB handles ~10⁵); most CP code is recursive.
- Java: default thread stack (~512 KB) dies around 10⁴ frames — the classic fix is running DFS in a
new Thread(null, run, "dfs", 1 << 26). - Go: goroutine stacks grow dynamically — recursive DFS is safe. Rare and worth knowing.
- JS/TS: engine call-stack caps near 10⁴ frames; iterative required, same frame-stack pattern.
Common Bugs
- Missing on_stk guard: updating
lowfrom a vertex in an already-closed SCC (cross edge) merges components that are merely reachable, not mutually reachable. The single most common Tarjan bug. - Re-running the first-visit block: in the iterative form, a frame is visited once per child; without the
i == 0gate,discis overwritten and the vertex is pushed ontostkrepeatedly. - Forgetting the parent low-merge at pop:
low[parent] = min(low[parent], low[child])must happen when the child frame pops — dropping it makes every vertex its own root. - Recursive DFS at scale: RecursionError (or a hard segfault) at 10⁵ nodes in Python; see Language notes.
- Wrong 2-SAT edge direction: adding a→b for clause (a ∨ b) instead of ¬a→b, or adding only one twin — breaks skew-symmetry, silently wrong on ~half of random instances.
- Flipped assignment comparison:
comp[x] < comp[¬x]is correct for Tarjan’s reverse-topo ids; with Kosaraju-ordered ids (forward topo) the inequality flips. Porting code between the two without flipping is a classic silent bug.
Debugging Strategy
Self-check the output first — it’s O(m) and catches everything: for SCC, assert comp[u] ≥ comp[v] for every edge u→v (equality only within a component); for 2-SAT, evaluate every clause under the returned assignment. On failure, shrink with the random tester (n ≤ 5) until minimal, then hand-trace: print (v, disc[v], low[v], stk) at each pop and check against a pencil run of the 5-node graph. If low values look right but components are wrong, the bug is the on_stk guard; if components pop too early, it’s the parent low-merge.
Mastery Criteria
- Wrote iterative Tarjan from memory, passing the random SCC oracle, in <20 minutes.
- Wrote the 2-SAT reduction (twin edges + SCC check + assignment) in <10 minutes on top of it.
- Stated the stack invariant and root condition in <2 minutes without notes.
- Reproduced the skew-symmetry proof of the assignment rule on a whiteboard.
- Solved LC 207/210 by explicit condensation framing from cold start.
- Explained the 2-SAT vs 3-SAT cliff (why implications break at width 3) in <1 minute.
Lab 11 — Bridges & Articulation Points
Goal
Find every bridge (edge whose removal disconnects the graph) and every articulation point (vertex ditto) in a single iterative low-link DFS. This is the undirected twin of Tarjan’s SCC low-link and the core of the connectivity-robustness toolkit: 2-edge-connected components, block-cut trees, single-point-of-failure analysis. Target problem: LC 1192, a real and frequently-asked FAANG Hard. After this lab: LC 1192 iterative in <25 minutes.
Background Concepts
DFS on an undirected graph produces only tree edges and back edges — no cross edges. When you examine edge (u, w) and w is already visited, w is an ancestor or descendant of u; two side-by-side subtrees can never touch, or the edge would have been a tree edge.
Per vertex: disc[v] (discovery time) and low[v] = min disc reachable from v’s subtree via tree edges plus back edges. Then:
- Bridge (tree edge u→v, u parent):
low[v] > disc[u]. Strict — v’s subtree has no back edge reaching u or above, so the tree edge is the subtree’s only exit. - Articulation, non-root u: some child v has
low[v] ≥ disc[u]. Non-strict — a back edge reaching exactly u doesn’t help once u itself is removed. - Articulation, root: ≥ 2 DFS-tree children. The low-link test is vacuous at the root (
low[child] ≥ disc[root]always); what matters is whether the root’s children are connected only through it.
Parent skip must be by edge id, not parent vertex. With parallel edges (u, v), (u, v), the second copy is a genuine back edge — the pair forms a 2-cycle and neither is a bridge. Vertex-based skip ignores both copies and misreports a bridge.
These conditions induce two structures: remove all bridges → 2-edge-connected components (contract them: the bridge tree); articulation points + biconnected components → the block-cut tree. Both turn connectivity queries into tree queries.
Relation to SCC (lab-10): same disc/low machinery; there the guard is “on the open-SCC stack,” here it’s “not the parent edge.” Learn them as one pattern with two guards.
Interview Context
LC 1192 (Critical Connections in a Network) is a canonical FAANG Hard — heavily asked at Amazon in “network reliability” clothing. Most candidates who recognize “bridges” still fail on (a) recursion depth at n = 10⁵ in Python and (b) the ≥ vs > distinction between the two conditions. Nailing the iterative version with a clean edge-id skip is a strong senior signal.
Problem Statement (LC 1192)
There are n servers numbered 0..n−1 connected by undirected connections. A critical connection is a connection that, if removed, makes some pair of servers unable to reach each other. Return all critical connections in any order.
Constraints
- 2 ≤ n ≤ 10⁵; n − 1 ≤ connections.length ≤ 10⁵.
- No repeated connections on LC; the implementation below tolerates parallel edges and self-loops anyway — that generality is exactly what the edge-id skip buys.
Clarifying Questions
- Is the graph connected? (On LC, yes; the code handles forests via the outer loop regardless.)
- Parallel edges or self-loops possible? (Not on LC; a robust solution treats a parallel pair as a cycle — never a bridge — and ignores self-loops.)
- Output order? (Any; edges in either endpoint order.)
- Do you also want the vertices whose removal disconnects the network? (That’s the articulation-point variant — included.)
Examples
criticalConnections(4, [[0,1],[1,2],[2,0],[1,3]]) → [[1,3]]
criticalConnections(2, [[0,1]]) → [[0,1]]
bridges_and_cuts(2, [(0,1), (0,1)]) → ([], []) # parallel pair: no bridge
bridges_and_cuts(3, [(0,1), (1,2)]) → ([(0,1),(1,2)], [1]) # path: middle is a cut
Initial Brute Force
For each edge: delete it, run BFS/DFS, check whether the component count grew. Same per vertex for articulation points.
Brute Force Complexity
O(E · (V + E)) for bridges, O(V · (V + E)) for cut vertices. At V = E = 10⁵: ~10¹⁰ operations. TLE — but it is the perfect test oracle for small graphs.
Optimization Path
Every bridge is a tree edge of any DFS tree (a back edge lies on a cycle by construction). So one DFS suffices if each subtree can report “the highest ancestor I can reach without my parent edge” — that is low. Computing low bottom-up during the same DFS gives all bridges and articulation points in O(V + E), one pass, no reverse graph. This is Tarjan’s 1974 low-link technique; the SCC version (lab-10) is the directed sibling.
Final Expected Approach
def bridges_and_cuts(n, edges):
"""One iterative low-link DFS. Returns (bridges, cut_vertices).
Skips the parent EDGE by id, not the parent vertex — parallel edges
(u,v),(u,v) must count as a cycle, and vertex-skip would miss that."""
adj = [[] for _ in range(n)]
for eid, (u, v) in enumerate(edges):
adj[u].append((v, eid))
adj[v].append((u, eid))
disc = [-1] * n
low = [0] * n
children = [0] * n
is_cut = [False] * n
bridges = []
timer = 0
for s in range(n):
if disc[s] != -1:
continue
stack = [(s, -1, 0)] # (vertex, parent edge id, next child)
while stack:
v, pe, i = stack[-1]
if i == 0: # first visit
disc[v] = low[v] = timer
timer += 1
if i < len(adj[v]):
stack[-1] = (v, pe, i + 1)
w, eid = adj[v][i]
if eid == pe: # the one tree edge we came in on
continue
if disc[w] != -1: # back edge
low[v] = min(low[v], disc[w])
else:
stack.append((w, eid, 0)) # tree edge
else: # v finished: report to parent
stack.pop()
if stack:
pv = stack[-1][0]
low[pv] = min(low[pv], low[v])
children[pv] += 1
if low[v] > disc[pv]: # bridge condition
bridges.append(edges[pe])
if stack[-1][1] != -1 and low[v] >= disc[pv]:
is_cut[pv] = True # non-root articulation
if children[s] >= 2: # root articulation
is_cut[s] = True
return bridges, [v for v in range(n) if is_cut[v]]
def criticalConnections(n, connections): # LC 1192
return bridges_and_cuts(n, connections)[0]
Data Structures Used
- Adjacency lists of
(neighbor, edge_id)pairs — the id is what makes parallel edges and the parent skip correct. disc,low,childrenint arrays;is_cutboolean array.- One explicit frame stack of
(vertex, parent_edge_id, next_child_index)— the recursion replacement.
Correctness Argument
No cross edges: if (u, w) is examined with w visited and w is neither ancestor nor descendant of u, then w’s subtree finished before u was discovered — but DFS would have traversed (u, w) from w’s side then, making it a tree edge. Contradiction. So every non-tree edge’s disc[w] is an ancestor’s (or a finished descendant’s, whose larger disc is harmless in the min).
Bridge ⇔ low[v] > disc[u]: (⇐) every path out of subtree(v) uses either the tree edge (u,v) or a back edge from the subtree; low[v] > disc[u] says no back edge reaches u or above, so removing (u,v) isolates the subtree. (⇒) contrapositive: a back edge to disc ≤ disc[u] gives a second edge-disjoint route, so (u,v) lies on a cycle — not a bridge.
Articulation: removing non-root u disconnects child subtree v from the rest iff no back edge from subtree(v) climbs strictly above u — reaching u exactly is useless once u is gone, hence ≥. The root has no “rest above”; it separates its children iff it has ≥ 2 of them in the tree (each child subtree explored fully before the next starts, so two children ⇒ no path between them avoiding the root).
Once-per-edge reporting: a bridge is emitted only when its child endpoint’s frame pops — exactly once per tree edge.
Complexity
| Step | Time | Space |
|---|---|---|
| Build adjacency | O(V + E) | O(V + E) |
| Low-link DFS | O(V + E) | O(V) stacks/arrays |
| Total | O(V + E) | O(V + E) |
Implementation Requirements
- Iterative DFS — LC 1192’s n = 10⁵ path graph is a 10⁵-deep recursion; Python dies even with a raised limit.
- Parent skip by edge id (
eid == pe), never by vertex. - The
i == 0gate: frames re-enter once per child; discovery bookkeeping must run once. low[parent] = min(low[parent], low[child])and the bridge/cut checks happen at child-pop time, not push time.- Outer loop over all vertices: handles disconnected inputs and forests.
Tests
- LC example →
[[1,3]]; two-vertex single edge → that edge; triangle → no bridges, no cuts. - Parallel pair
(0,1)×2→ no bridges, no cuts. Self-loop(0,0)→ tolerated, nothing reported. - Path 0–1–2 → both edges bridges, vertex 1 the only cut.
- 400 random graphs (n ≤ 10, up to 2n edges, duplicates allowed) vs the deletion brute force: bridge iff removing the edge increases component count; cut iff removing the vertex does. Exact match on both outputs.
- Scale: 10⁵-vertex path — all n−1 edges bridges, n−2 cuts, no recursion error.
Follow-up Questions
- Why must the parent edge id be skipped rather than the parent vertex? → With parallel edges (u,v),(u,v), the two copies form a cycle: neither is a bridge. Vertex-skip discards both copies from v’s scan, so subtree(v) shows no back edge and (u,v) is misreported as a bridge. Edge-id skip discards only the one tree edge; the twin correctly updates
low[v] = disc[u]. Any multigraph input (and many real datasets) needs this. - What is a block-cut tree and what does it answer? → Nodes = biconnected components (blocks) + cut vertices; a cut vertex connects to each block containing it. It’s a tree. “Does every a→b path pass through v?” ⇔ v is a cut vertex lying on the block-tree path between a’s and b’s blocks. Turns separation queries into LCA/path queries.
- “Add the fewest edges to make the network 2-edge-connected”? → Contract 2-edge-connected components → the bridge tree (bridges become tree edges). Answer: ⌈leaves/2⌉ — pair up leaves across the tree. Classic follow-up to LC 1192.
- Edges are being deleted online — maintain bridges dynamically? → Genuinely hard. Fully dynamic connectivity is Holm–de Lichtenberg–Thorup O(log² n) amortized; deletions-only usually go offline: divide-and-conquer over the query timeline with a rollback DSU. Name these; don’t attempt them in 45 minutes.
- Is there a directed analogue? → Yes: strong bridges and strong articulation points (removal breaks strong connectivity), computable via dominator trees of G and G-reversed (Italiano et al.). Rarely asked — knowing they exist and that “run undirected low-link” is wrong on digraphs is the whole point.
Product Extension
Datacenter fault analysis: model switches/routers as vertices, links as edges; bridges are the links whose failure partitions the network, articulation points the routers. Run after every topology change; feed the bridge tree into capacity planning (“which single link additions remove all bridges?” = the ⌈leaves/2⌉ computation). The same audit applies to microservice call graphs and power grids — “critical connection” is a literal SRE term here.
Language/Runtime Follow-ups
- Python: iterative mandatory (C-stack segfault beats
sys.setrecursionlimiton 10⁵-deep paths). Tuples-in-list frames are fastest; avoid per-frame objects. - C++: recursive is idiomatic and survives ~10⁵–10⁶ depth on an 8 MB stack;
vector<pair<int,int>>adjacency. - Java: default thread stack dies near 10⁴ frames — the
new Thread(null, task, "dfs", 1 << 26)trick or an explicit stack. - Go: goroutine stacks grow on demand; recursive DFS is safe.
- JS/TS: ~10⁴-frame engine cap; iterative required. LC’s JS judge on 1192 punishes recursive solutions with stack overflow.
Common Bugs
- Parent-vertex skip: the headline bug — correct on simple graphs, silently wrong on multigraphs; a parallel pair gets reported as a bridge. Skip by edge id.
- > vs ≥ swapped: bridges need strict
low[v] > disc[u]; articulation needslow[v] ≥ disc[u]. Swapping makes cycle edges bridges or misses cut vertices whose subtree loops back exactly to them. - Root handled by the low-link test:
low[child] ≥ disc[root]is always true — every root with one child gets flagged. The root needs the ≥ 2-tree-children rule. - Counting children per examined edge: incrementing
childrenwhen scanning edges (rather than when a child frame pops) counts back edges as children — the root of a triangle gets flagged. - Missing low propagation at pop: in iterative form, forgetting
low[parent] = min(low[parent], low[child])makes every tree edge look like a bridge. - Bridge appended from both directions: emitting on the scan of each endpoint duplicates every bridge; emit only at child-pop.
Debugging Strategy
Property-check the output before reading the code: for each reported bridge, delete it and assert the component count grows; assert no unreported edge does the same (O(E²) but fine at test scale) — this instantly classifies false positives vs false negatives. False positives on graphs with parallel edges → the skip is vertex-based. False positives everywhere → missing low propagation or swapped ≥/>. Then hand-trace the 4-node LC example printing (v, disc[v], low[v]) at each pop: expected low = [0,0,0,3], and only edge (1,3) satisfies low[3] > disc[1].
Mastery Criteria
- Solved LC 1192 with an iterative DFS, clean first submission, in <25 minutes.
- Stated all three conditions (bridge, non-root cut, root cut) with correct >/≥ in <1 minute.
- Explained why the parent edge, not vertex, is skipped — with the parallel-edge counterexample — in <1 minute.
- Derived the bridge condition from the no-cross-edges lemma on a whiteboard.
- Answered the bridge-tree follow-up (⌈leaves/2⌉ edge additions) without prompting.
- Mapped this lab onto lab-10: same low-link, different guard (parent edge vs SCC stack).
Lab 12 — Eulerian Path & Hierholzer’s Algorithm
Goal
Decide the existence of an Eulerian path (uses every edge exactly once) or Eulerian circuit (path that returns to start) in directed and undirected graphs, and construct one in O(E) via Hierholzer’s algorithm. Apply to LeetCode 332 (Reconstruct Itinerary). After this lab you should state the degree conditions cold for both graph flavors and write the iterative Hierholzer in <15 minutes.
Background Concepts
An Eulerian circuit traverses every edge once and ends where it started; an Eulerian path traverses every edge once but may start and end at distinct vertices. Existence is a pure degree-and-connectivity question — no search needed to decide, only to construct.
Undirected. Let a graph be connected on the vertices that touch at least one edge (isolated vertices are irrelevant). Then:
- Eulerian circuit ⟺ every vertex has even degree.
- Eulerian path ⟺ exactly 0 or 2 vertices have odd degree. (2 odd ⇒ they are the endpoints; 0 odd ⇒ it is a circuit.)
Directed. Let the graph be connected when edge directions are ignored (on edge-touched vertices). Then:
- Eulerian circuit ⟺
in(v) == out(v)for every vertex. - Eulerian path ⟺ exactly one vertex has
out − in = +1(the start), exactly one hasin − out = +1(the end), and all others satisfyin == out.
Hierholzer builds the trail in O(E). The clean formulation is the post-order append: walk greedily from the start, consuming edges as you go; when a vertex has no unused outgoing edge, append it to the output; reverse the output at the end. Equivalently: DFS that emits a vertex only once its adjacency list is exhausted. This is a correct, single-pass rendering of the “walk a cycle, splice in sub-cycles” idea — the recursion stack is the splice.
Why naive greedy-DFS fails. A plain DFS that commits to the first path it finds and returns it can strand edges: from JFK it might fly JFK→KUL (dead end) and never come back for JFK→NRT→JFK. The post-order trick avoids this because a vertex is only emitted after all its edges are drained, so stranded cycles get spliced automatically.
Euler vs Hamilton. Eulerian (every edge once) is decidable and constructible in O(E) by a degree check. Hamiltonian (every vertex once) is NP-complete — see Phase 13’s TSP lab. One is a local parity condition; the other is a global search. Do not confuse the two under interview pressure.
Interview Context
Eulerian-path problems are disguised: “reconstruct the itinerary,” “arrange the pairs into a chain,” “shortest string containing all codes.” The tell is use every edge/pair exactly once, order matters. Interviewers like it because the naive candidate reaches for backtracking (exponential) while the strong candidate recognizes the Euler structure and writes O(E) Hierholzer. Getting the lexicographic tie-break right on LC 332 while keeping linearity is the L5 signal.
Problem Statement (LC 332)
You are given a list of airline tickets where tickets[i] = [from_i, to_i]. Reconstruct the itinerary in order and return it. All tickets belong to a man who departs from "JFK", so the itinerary must begin with "JFK". If multiple valid itineraries exist, return the one that is lexicographically smallest when read as a single concatenated list of airports. You may assume all tickets form at least one valid itinerary; you must use all the tickets exactly once.
Constraints
- 1 ≤ tickets.length ≤ 300 (LC 332); generalize the technique to E up to 10⁵.
- Airport codes are 3 uppercase letters.
- A valid Eulerian path is guaranteed to exist.
Clarifying Questions
- Must the path start at a fixed vertex? (Yes —
"JFK"for LC 332; in the generic version the start is forced by the degree imbalance.) - Are multi-edges (duplicate tickets) allowed? (Yes — treat each ticket as its own edge.)
- Is a valid Eulerian path guaranteed, or must I detect infeasibility? (LC 332 guarantees one; the generic
find_euler_pathmust returnNonewhen none exists.) - Circuit or open path? (Either — the degree check tells you which; the algorithm is the same.)
- Directed or undirected? (LC 332 is directed; undirected needs edge used-flags by id.)
Examples
tickets = [[MUC,LHR],[JFK,MUC],[SFO,SJC],[LHR,SFO]]
→ [JFK, MUC, LHR, SFO, SJC]
tickets = [[JFK,SFO],[JFK,ATL],[SFO,ATL],[ATL,JFK],[ATL,SFO]]
→ [JFK, ATL, JFK, SFO, ATL, SFO] (not JFK,SFO,... — lexicographically smaller)
tickets = [[JFK,KUL],[JFK,NRT],[NRT,JFK]]
→ [JFK, NRT, JFK, KUL] (greedy JFK→KUL would strand the NRT cycle)
Initial Brute Force
Backtracking: from the start vertex, try each unused outgoing edge in sorted order, recurse, and accept the first complete trail that consumes all E edges. Because ties are explored in lexicographic order, the first success is the lexicographically smallest itinerary.
Brute Force Complexity
Exponential in the worst case: the recursion branches over unused edges at each step, O(E!) in adversarial multigraphs, with backtracking on dead ends. Fine for LC 332’s E ≤ 300 only because valid solutions are dense; it collapses at E = 10⁵.
Optimization Path
The problem is exactly “find an Eulerian path.” Hierholzer solves construction in O(E). For the lexicographic requirement, store each vertex’s outgoing edges in a min-heap (or pre-sort and consume with a pointer); popping the smallest destination first makes the post-order-reversed output lexicographically minimal. The heap variant is O(E log E); pre-sorting once is also O(E log E) but with a cheaper inner loop. Both beat backtracking decisively.
Final Expected Approach
- Build adjacency lists; heapify each so the smallest next hop pops first.
- Iterative Hierholzer with an explicit stack: push the start; while the top vertex has an unused edge, pop its smallest destination and push it; when it has none, pop it into
route. - Reverse
route.
import heapq
from collections import defaultdict
def find_itinerary(tickets):
graph = defaultdict(list)
for src, dst in tickets:
graph[src].append(dst)
for src in graph:
heapq.heapify(graph[src]) # min-heap per vertex: lex tie-break
route, stack = [], ["JFK"]
while stack:
while graph[stack[-1]]: # drain smallest edges first
stack.append(heapq.heappop(graph[stack[-1]]))
route.append(stack.pop()) # post-order: emit when exhausted
return route[::-1] # reverse to get forward trail
def find_euler_path(n, edges):
"""Generic directed Eulerian path over vertices 0..n-1, or None."""
adj = [[] for _ in range(n)]
indeg, outdeg = [0] * n, [0] * n
for u, v in edges:
adj[u].append(v); outdeg[u] += 1; indeg[v] += 1
start = -1
for v in range(n): # locate the unique +1 out-imbalance
d = outdeg[v] - indeg[v]
if d == 1:
if start != -1: return None # two candidate starts
start = v
elif d not in (0, -1):
return None # imbalance beyond ±1
ends = sum(1 for v in range(n) if indeg[v] - outdeg[v] == 1)
if (start != -1) != (ends == 1):
return None # start without a matching end
if start == -1: # circuit: begin anywhere with edges
start = next((v for v in range(n) if outdeg[v]), 0)
ptr = [0] * n # next unused edge index per vertex
route, stack = [], [start]
while stack:
v = stack[-1]
if ptr[v] < len(adj[v]):
stack.append(adj[v][ptr[v]]); ptr[v] += 1
else:
route.append(stack.pop())
if len(route) != len(edges) + 1: # unreached edges ⇒ disconnected
return None
return route[::-1]
Data Structures Used
- Adjacency lists (
defaultdict(list)), min-heaps for the lexicographic variant. indeg[],outdeg[]int arrays for the directed degree test.- An explicit stack for iterative Hierholzer, plus a per-vertex
ptr[](or the heap itself) tracking the next unused edge. route[]built in reverse.
Correctness Argument
Edge-disjoint cycle decomposition. If every vertex satisfies the balance condition, the edge set decomposes into edge-disjoint cycles (start any walk from a balanced vertex; it can only get stuck by returning to the start, forming a cycle; remove it and recurse). Hierholzer stitches these cycles: the outer walk is one cycle; whenever it revisits a vertex with leftover edges, the stack dives into another cycle and splices it in at exactly that point.
Why post-order append is correct. A vertex is appended only when its edge list is exhausted, i.e. after every cycle through it has been fully walked. Reversing yields a sequence where each vertex precedes the continuation of the trail that leaves it — a valid Eulerian ordering. The reversal is what turns “deepest-finished-first” into “first-traversed-first.”
Completeness check. If the graph is disconnected on edge-touched vertices, some edges are never reached, so len(route) != E + 1; returning None is correct. The degree scan rejects imbalances that make any Eulerian trail impossible before the walk begins.
Complexity
| Operation | Time | Space |
|---|---|---|
| Degree/existence check | O(V + E) | O(V) |
| Hierholzer (pointer) | O(E) | O(E) |
| Hierholzer (heap, LC 332) | O(E log E) | O(E) |
Implementation Requirements
- Build
routein reverse and reverse once at the end — neverinsert(0, …)(that is O(E²)). - Use an iterative stack, not recursion: E up to 10⁵ blows the Python recursion limit.
- For the heap variant,
heapq.heapifyeach list once; do not re-sort per pop. - For the generic version, return
Noneon any of: imbalance beyond ±1, start-without-end mismatch, or unreached edges.
Tests
- LC 332 example 1 →
[JFK,MUC,LHR,SFO,SJC]. - LC 332 example 2 →
[JFK,ATL,JFK,SFO,ATL,SFO](lex tie-break exercised). - Greedy-killer
[[JFK,KUL],[JFK,NRT],[NRT,JFK]]→[JFK,NRT,JFK,KUL]. find_euler_path(3, [(0,1),(1,2),(2,0)])→ a valid circuit.find_euler_path(3, [(0,1),(0,2)])→None(two out-imbalances).find_euler_path(4, [(0,1),(1,0),(2,3),(3,2)])→None(disconnected).- Stress: random directed multigraphs, compare
find_euler_pathexistence against a backtracking oracle and validate every returned trail.
Follow-up Questions
- Why does the post-order append produce a valid trail rather than garbage? The edge set decomposes into edge-disjoint cycles; a vertex is emitted only after all its cycles are drained. The recursion/stack order is a DFS over that cycle structure, and reversing the finish order yields a topological-like ordering of the splice points — exactly one consistent Eulerian traversal.
- How do De Bruijn sequences relate (LC 753, Cracking the Safe)? Build a graph whose vertices are length-(n−1) strings over a k-symbol alphabet and whose edges are length-n strings (edge
a·s → s·b). Every vertex has in-degree = out-degree = k, so an Eulerian circuit exists; walking it emits every length-n string exactly once, giving a shortest cyclic De Bruijn sequence. Cracking the Safe is precisely this Euler circuit. - How does the undirected variant differ in bookkeeping? Each undirected edge is one resource shared by two endpoints, so you cannot just pop from an adjacency list — you tag each edge with an id and keep a
used[id]flag (or remove it from both endpoints). Existence needs 0 or 2 odd-degree vertices; if 2, start at an odd one. - Where does this appear in genome assembly? Real short-read assemblers build a de Bruijn graph over k-mers and find an Eulerian path through it to reconstruct the sequence — “k-universal string” and “DNA fragment assembly” are the same Eulerian-path problem. Euler beats the Hamiltonian-path formulation (overlap graph) precisely because Euler is polynomial and Hamilton is NP-complete.
- What is Valid Arrangement of Pairs (LC 2097)? The pure directed-Eulerian-path problem with no lexicographic constraint: arrange all pairs so consecutive pairs chain. Run
find_euler_pathstarting from the vertex without − in = +1(or any vertex if balanced). - What if some vertices have odd degree and I still want to cover every edge? That is the Chinese Postman (Route Inspection) problem: pair up odd-degree vertices via minimum-weight matching, duplicate the shortest paths between matched pairs to make all degrees even, then take an Eulerian circuit of the augmented graph.
Product Extension
Network monitoring and testing frameworks that must traverse every link exactly once (link-coverage test suites, packet-tracing) are Eulerian-circuit problems on the topology graph. Automated test-path generation over a state-transition model — cover every transition (edge) with the fewest walks — reduces to Eulerian path plus, when the model has odd-degree states, a Chinese-Postman augmentation. Genome assemblers (SPAdes, Velvet) are the highest-profile production use: the de Bruijn Eulerian path is the algorithmic core.
Language/Runtime Follow-ups
- Python: use an explicit stack, never recursion —
sys.setrecursionlimitis a smell here.heapqon the adjacency lists gives the lex order;collections.defaultdict(list)avoids key checks. - Java:
PriorityQueue<String>per source for LC 332; use anArrayDequeas the Hierholzer stack andCollections.reverseat the end, oraddFirstinto aLinkedList. - C++:
multiset<string>or a sortedvectorwith an iterator per vertex;std::reversethe route. Watch that erasing from amultisetinvalidates only the erased iterator. - Go: sort each adjacency slice once and consume via an index pointer; append to a slice and reverse in place.
- JS/TS: heaps aren’t built in — pre-sort adjacency arrays and
shift()(or index-pointer to avoid O(n) shifts), thenreverse().
Common Bugs
- Naive greedy without splicing: committing to the first complete-looking path strands cycles (the
JFK→KULtrap). The post-order emit is not optional — it is the algorithm. route.insert(0, v)instead of append+reverse: turns O(E) into O(E²); TLE at E = 10⁵.- Recursion depth overflow: recursive Hierholzer hits Python’s ~1000-frame limit on long trails. Use the explicit stack.
- Wrong start vertex in the generic version: an open path must start at the
out − in = +1vertex, not an arbitrary one; starting elsewhere strands the imbalanced tail. - Missing the connectivity check: the degree conditions are necessary but not sufficient — two balanced components pass the parity test yet have no single Eulerian trail. Verify
len(route) == E + 1. - Undirected edge counted once: forgetting to mark an undirected edge used from both endpoints re-traverses it; use edge ids with a
used[]flag.
Debugging Strategy
Print the degree table first: for the directed case list (v, in, out, out−in) and confirm at most one +1 and one −1. If existence is wrong, the bug is in the scan, not the walk. For the walk, trace the stack and route on the greedy-killer [[JFK,KUL],[JFK,NRT],[NRT,JFK]]: the stack should dive JFK→KUL, pop KUL (dead end) into route, then unwind and take the NRT cycle — final route before reversal is [KUL, JFK, NRT, JFK]. If your route length ≠ E + 1, you have a disconnected graph or a consumed-edge accounting bug.
Mastery Criteria
- Stated both directed and undirected existence conditions (degree + connectivity) from memory in <2 minutes.
- Wrote iterative Hierholzer for LC 332 with correct lex tie-break in <15 minutes.
- Explained why post-order append works via edge-disjoint cycle decomposition.
- Solved LC 332, LC 2097, and LC 753 from a cold start.
- Distinguished Eulerian (O(E)) from Hamiltonian (NP-complete) instantly.
- Name-dropped De Bruijn / genome assembly and the Chinese Postman extension.
Phase 5 — Dynamic Programming (Basic → Extreme)
Target level: Medium → Very Hard Expected duration: 4 weeks (12-week track) / 5 weeks (6-month track) / 6 weeks (12-month track) Weekly cadence: ~6 DP topics per week + 30–60 problems applying them under the framework
Why Dynamic Programming Is The Single Hardest Pattern Family In Coding Interviews
Phase 4 taught you that one-in-three Medium-Hard interview problems is a graph problem in disguise. The other big share is dynamic programming. DP shows up in roughly one in four Medium-Hard rounds at top-tier companies, and the share rises further in staff/principal and quant interviews where exact-counting and optimization questions dominate. More importantly, DP is the topic where the gap between candidates who have a framework and those who don’t is widest. A candidate without a DP framework freezes on dp[i][j] = ?; a candidate with one writes the recurrence in 90 seconds and spends the remaining time on edge cases.
The empirical claim that drives this entire phase:
The hard part of DP is not the code. The hard part is deriving the state. Almost every wrong DP solution is wrong because the state is wrong — too small to capture all the information needed, or so large that the table doesn’t fit in memory or time. Once the state is right, the transition writes itself, the base cases follow from the state, and the code is mechanical.
DP is also the topic where candidates most often memorize problems instead of internalizing the technique, and it is the topic where memorization fails most spectacularly. There are perhaps 60 named DP problems on LeetCode that everyone has seen; an interviewer who wants to filter out the memorizers asks the 61st. The solution, then, is not to drill 60 problems — it is to drill the derivation process until you can derive any DP from the recursive formulation.
This phase is built around one teaching device that we will use on every single problem from start to finish: the brute → memo → tabulated → space-optimized progression. Every problem you solve in this phase will be solved four times in succession:
- Brute force — usually exponential recursion that explores every choice.
- Memoized — the same recursion plus a cache, top-down DP, O(states × transition) time.
- Tabulated — bottom-up loop in topological order on the state DAG, O(states × transition) time, no recursion stack.
- Space-optimized — a rolling-array transformation that keeps only the previous one or two layers and reduces space from O(N · M) to O(M) or O(1).
By the end of this phase, you will execute this progression unconsciously. When an interviewer asks “can you reduce the space?”, you will already have written the tabulated version with the rolling array in mind. When the interviewer asks “what’s the recurrence?”, you will already have derived it from the brute-force recursion in the first 90 seconds of the problem. The four-stage progression is the single most valuable interview-time discipline taught in this entire curriculum, because it converts an open-ended “design a DP” question into a deterministic four-step recipe.
After this phase, you can solve canonical Hard DP problems on first attempt: edit distance in 25 minutes with full progression, longest increasing subsequence at both O(N²) and O(N log N), partition equal subset sum, coin change (count and minimum), burst balloons (interval DP), house robber III (tree DP), and shortest path visiting all nodes (bitmask DP). You will also become visibly stronger in mock interviews because you will reach for dp[i][j] notation on the whiteboard within 90 seconds and articulate the state definition out loud before writing any code.
What You Will Be Able To Do After This Phase
- Recognize that a problem is a DP problem within 2 minutes, even when the words “DP” or “memoize” never appear in the statement.
- Derive the state from a recursive brute force in <3 minutes by identifying the parameters that change across recursive calls.
- Write the transition as a closed-form max/min/sum over a small set of choices, with clear correctness justification.
- Identify base cases as the recursive function’s return statements at the smallest input.
- Write the tabulated version by inverting the recursion into a loop with the right evaluation order on the state DAG.
- Apply the rolling-array trick to reduce O(N · M) space to O(M) or O(1) when the recurrence depends only on the previous row(s).
- Distinguish 0/1 knapsack from unbounded knapsack by a single change in the inner-loop direction.
- Recognize when a “subset” or “partition” problem reduces to subset sum with target
total / 2. - Implement LIS at both O(N²) (canonical DP) and O(N log N) (patience sorting + binary search) and explain the equivalence.
- Implement edit distance with all four variants (brute, memo, tabulated, O(M)-space) in <25 minutes.
- Implement tree DP via post-order recursion, returning multi-tuple state (e.g., “best with root included” and “best with root excluded”).
- Implement interval DP with the canonical
for length, for left, right = left + length - 1loop structure. - Implement bitmask DP with state
(mask, last)for TSP-style problems and(mask)alone for set-cover-style problems. - Articulate the correctness argument for every DP you write: state definition, transition justification, evaluation order, base case.
- Spot the standard DP bugs unprompted: wrong base case, wrong evaluation order, off-by-one in indices, missed edge case at empty input.
How To Read This Phase
Read this README in two passes. Pass 1: linear, end-to-end, building a mental map of which DP variant solves which problem signal. Do this in one sitting. Pass 2: as you work the labs, refer back to specific topic entries to clarify state-design choices and pitfalls.
Each topic entry has a fixed shape:
- When To Use — the problem signal that should fire this DP variant in <2 minutes.
- State Design — what the state is, why these are the right parameters, why no fewer suffice.
- Transition — the recurrence in closed form.
- Complexity — time and space, and what space optimization is possible.
- Common Pitfalls — the bugs that consume the most interview minutes for this DP variant specifically.
- Classic Problems — 3–5 representative LeetCode problems where this DP is the intended solution.
The phase ends with a DP-Recognition Cheat Sheet (problem signals → DP variant), a Common-Bug Catalog, a Mastery Checklist, and Exit Criteria.
The DP Framework
Before any topic, internalize this framework. Use it on every DP problem.
1. State Definition
The state is the smallest set of parameters that uniquely determines the answer to a subproblem. Write it down explicitly:
dp[i] =the answer to the subproblem ending at / using up to / for prefix-of-length / etc., parameteri.
Sentences that begin “let dp[i] be” are the most valuable two seconds of the entire problem. If you can’t finish the sentence, you don’t have a state — you have a vague hope.
The state must be sufficient (encodes everything the future needs to know) and necessary (every parameter actually changes the answer). A common bug is a state that’s sufficient but not necessary — e.g., tracking both index and remaining-budget when budget is determined by index. Another is necessary but not sufficient — e.g., tracking only the index when the choice depends on what was picked earlier.
A useful test: two equal states must produce equal answers. If two different histories arrive at the same state but have different optimal continuations, the state is missing a dimension.
2. Transition Function
The transition expresses dp[state] as a function of dp[smaller_state] for one or more smaller states. It is the recurrence. For optimization problems, it is min or max over a small set of choices; for counting problems, it is a sum.
The transition has three parts:
- Choices — the discrete set of moves at this state (include item or skip; pick this character or that one; rob this house or skip).
- Cost / value — the contribution of each choice (the item’s weight, the operation’s cost, the gain from picking).
- Aggregation —
min/max/sum/ORover the choices.
Write the transition as:
dp[state] = aggregate over choice c in C(state): contribution(c) + dp[state - effect(c)]
Always keep C(state) finite and small — typically O(1), O(K), or O(N). Transitions that aren’t O(small) usually indicate a missing state dimension.
3. Base Cases
The base cases are the values of dp at the smallest (recursion-stopping) states. They are not optional; a missing or wrong base case is the single most common DP bug.
Identify base cases by writing the recursion first and looking at the return-statements:
def f(i):
if i == 0: return 0 # ← THIS is the base case
return f(i-1) + something
For 2D DP, the base cases are typically the entire first row and first column — set them explicitly before the main loop. For 3D DP and beyond, they’re a hyper-plane of dimension one less than the state.
A subtle base case bug: two different recursive paths reach the same base case but expect different return values. Usually this means the state is wrong (missing a dimension) and the base case has to “remember” which path it came from — impossible.
4. Evaluation Order
DP states form a DAG: state A points to state B iff dp[B] appears in the recurrence for dp[A]. To compute dp[A] we must have already computed dp[B]. The evaluation order is a topological order of this DAG.
For 1D DP indexed by i, the order is usually i = 0, 1, 2, …, N (increasing) or i = N, N-1, …, 0 (decreasing) — depending on whether your transition looks “back” or “forward”. Both work; pick one consistently.
For 2D DP indexed by (i, j), the order is usually row-major (for i: for j:) or column-major. The right one is the one that fills dp[i-1][j] and dp[i][j-1] before dp[i][j].
For interval DP indexed by (left, right), the order is by interval length ascending: for length in 1..N: for left, right = left + length - 1. This guarantees all sub-intervals are filled before the enclosing interval.
For tree DP, the order is post-order DFS: children are filled before the parent.
For bitmask DP, the order is by popcount ascending or by mask value ascending (since a sub-mask of m is < m).
5. Space Optimization (The Rolling-Array Technique)
If dp[i][j] depends only on dp[i-1][*] and dp[i][j-1], then once row i is computed we don’t need any earlier row. Keep only the current and previous row — O(N · M) becomes O(M).
If dp[i][j] depends only on dp[i-1][j] and dp[i-1][j-1] (no same-row dependency), you can collapse to a single row using right-to-left iteration in j — O(M) space.
If dp[i][j] depends on more rows (e.g., dp[i-2]), keep that many rows.
The rolling-array transformation is mechanical once the tabulated version is written. The interviewer often asks for it: “can you reduce the space?” Practice the transformation on every lab so it becomes reflex.
6. The Brute → Memo → Tabulated → Space-Optimized Progression
Apply this on every problem in this phase. It is the mandatory teaching device of this phase.
- Brute force: exponential recursion that tries every choice. Often O(2^N) or O(N!). Don’t skip writing this — it is the direct source of your state and transition.
- Memoized: same recursion + a cache (
@lru_cachein Python, aHashMapin Java, an array in C++). Time becomes O(states × transition); space includes the recursion stack. - Tabulated: replace recursion with a loop in topological order on the state DAG. Same time complexity, but no recursion overhead, and the loop structure makes the dependency pattern explicit.
- Space-optimized: roll the table down to O(M) or O(1) by exploiting the recurrence’s locality.
Each stage is a strictly smaller change from the previous: brute → memo is “add a cache”, memo → tabulated is “invert the call graph”, tabulated → space-optimized is “drop dimensions you don’t reuse”. This is deterministic engineering, not invention.
Inline DP Topic Reference
1. Memoization Vs Tabulation Tradeoffs
When To Use
Both compute the same thing — they’re two evaluation orders on the same state DAG. Choose deliberately.
- Memoization (top-down): when the reachable set of states is much smaller than the full state space (sparse DP). Examples: regex matching where many
(i, j)pairs are never visited. Also when the transition is easier to express recursively than as a forward loop. - Tabulation (bottom-up): when the state space is dense (most states are visited), when you need to reduce space via rolling arrays (which require explicit loop structure), or when recursion depth would exceed the stack limit (e.g., N=10^5 in Python with default
setrecursionlimit=1000).
Common Pitfalls
- Memoization in Python with default
setrecursionlimitoverflows at N ≈ 1000. Eithersys.setrecursionlimit(10**6)or convert to tabulation. - Memoization with mutable arguments (e.g.,
lru_cacheon a function taking a list) — Pythonlru_cacherequires hashable arguments; pass tuples or indices, not lists. - Tabulation with the wrong loop order silently produces garbage — see Section 5.
Classic Problems
- LC 322 — Coin Change. Memoized recursion is natural; tabulated is faster in tight loops. See Lab 04.
- LC 10 — Regular Expression Matching. Memoization is cleaner here. See String DP below.
2. State Design Principles
When To Use
Every DP problem starts here. Follow this discipline:
- Identify what changes across recursive calls. Those parameters are candidate state dimensions.
- Drop any parameter that is determined by the others.
- Keep any parameter whose value affects the optimal continuation.
- Verify: two states with identical parameters must have identical optimal values.
State Design Patterns
- Prefix DP:
dp[i]= answer for firstielements. Used in LIS, house robber, decode ways. - Two-pointer / interval DP:
dp[i][j]= answer for elements in[i, j]. Used in matrix chain, burst balloons, palindromic subsequence. - Knapsack-style:
dp[i][w]= best using firstiitems with budgetw. Used in 0/1 knapsack, partition, coin change. - Two-string DP:
dp[i][j]= answer for prefix-i of A and prefix-j of B. Used in LCS, edit distance, regex matching. - Tree DP:
dp[v]= answer for subtree rooted atv. Often a tuple of values (e.g., “rob” / “skip”). - Bitmask DP:
dp[mask]ordp[mask][last]= answer over the subset specified bymask. - Game DP:
dp[state]= best score the current player can guarantee.
Common Pitfalls
- Adding a parameter that doesn’t affect the answer — wastes time and space. E.g., tracking
step_countwhen the recurrence already encodes it via the index. - Missing a parameter that does — produces wrong answers because two materially different histories collapse to the same
dpcell. - Encoding choices in the state instead of the transition. The state is “where are we now”; the transition decides “what to do next”. Keep them separate.
3. Transition Function Design
When To Use
Once the state is defined, the transition is constrained: it must express dp[state] in terms of strictly smaller states.
Design Steps
- List all choices available at this state (include / skip / pick which / move where).
- For each choice, identify the contribution and the resulting smaller state.
- Aggregate:
minfor shortest/cheapest,maxfor longest/most-valuable,+for counting.
Common Pitfalls
- Forgetting a choice — usually “don’t take the item” or “skip this position”. Often the trivial choice that the recurrence still depends on.
- Double-counting — particularly in counting problems where two distinct paths to the same state are aggregated naively. Often signals a missing dimension.
- Off-by-one in the resulting smaller state —
dp[i-1][j-1]vsdp[i][j-1]is the difference between “use this character” and “use the prefix ending here”.
4. Base Case Identification
When To Use
After defining state and transition, the recursion bottoms out at some smallest state. The base case is what dp returns there.
Identifying Base Cases
- For prefix DP
dp[i]: the base case isdp[0]— the empty prefix. Its value is the natural identity (0 for sums, 1 for counts of “the empty product”, -∞ or +∞ for unreachable). - For interval DP
dp[i][j]: the base case isdp[i][i](length-1 interval) — its value depends on the problem. - For two-string DP
dp[i][j]: the base cases aredp[0][j] = janddp[i][0] = ifor edit distance, ordp[0][j] = dp[i][0] = 0for LCS.
Common Pitfalls
- Wrong identity for counting problems — the empty prefix has count
1(one way to make nothing), not0. - Wrong identity for
min/max— initialize to+∞/-∞, not0. Initializing to0silently makes “do nothing” look optimal. - Forgetting to set base cases on the boundary of a 2D table — leaves them as the language’s default (
0in Java arrays;nullin JS; uninitialized garbage in C).
5. Evaluation Order (Topological Order On The State DAG)
When To Use
Every DP. The evaluation order must be consistent with the dependency structure.
Determining The Order
Treat states as nodes; draw an edge from A to B iff the recurrence for A reads dp[B]. The order to evaluate is reverse topological (children before parents).
For most DPs the order is obvious: increasing i, increasing j, increasing interval length, post-order over the tree, increasing popcount of mask. When in doubt, fix a small example, write out the dependency arrows, and read off the order.
Common Pitfalls
- 2D DP iterated in the wrong order silently computes garbage. The classic bug: iterating
for j: for i:when the recurrence readsdp[i-1][j]anddp[i][j-1]. The latter is fine only if the inner loop fills the column top-to-bottom and you compute it in the right order. - Interval DP iterated in
(left, right)order instead of(length, left)— fails because you computedp[0][N-1]beforedp[1][N-1]. - Bitmask DP iterated by some-arbitrary-order instead of mask value ascending — fails if any sub-mask is read after the enclosing mask is written.
6. Space Optimization (Rolling Array Technique)
When To Use
Whenever dp[i][...] depends on only the previous one or two i values, you can keep just those rows.
Mechanical Transformation
- Replace
dp[i][j]withdp_curr[j]anddp[i-1][j]withdp_prev[j]. - After each
i, swap or copy. - If
dp[i][j]doesn’t depend ondp[i][k]fork < j(no same-row dependency), collapse further to a single 1Ddp[j]. Iteratejcarefully: if the recurrence readsdp[i-1][j-1], iteratejfrom right-to-left so you read the old value before overwriting.
Common Pitfalls
- Iterating left-to-right when right-to-left is needed — overwrites the value you’ll need next. This is the canonical 0/1 knapsack vs unbounded knapsack distinction:
- 0/1 knapsack: iterate weight right-to-left to use the previous-row’s
dp[w-wi]. - Unbounded knapsack: iterate weight left-to-right to use the current row’s
dp[w-wi](because items can be reused).
- 0/1 knapsack: iterate weight right-to-left to use the previous-row’s
- Forgetting to reset the rolling array between outer iterations — old values bleed through.
- Optimizing space prematurely, before the tabulated version is correct. Always verify tabulated against memoized on small inputs first.
7. 1D DP
When To Use
The state is a single integer index — dp[i]. Examples: climbing stairs, house robber, decode ways, max subarray (Kadane).
State Design
dp[i] = answer for the prefix ending at index i, OR for the first i elements. Pick one convention and stick with it (the lab uses “answer for first i elements” consistently).
Transition
dp[i] = f(dp[i-1], dp[i-2], ..., dp[i-k]) for some small k. Examples:
- House robber:
dp[i] = max(dp[i-1], dp[i-2] + house[i-1]). - Climbing stairs:
dp[i] = dp[i-1] + dp[i-2](Fibonacci). - Decode ways:
dp[i] = (dp[i-1] if s[i-1] is valid 1-digit) + (dp[i-2] if s[i-2:i] is valid 2-digit).
Complexity
Time O(N). Space O(N) tabulated, O(1) space-optimized (since dependence on at most O(1) previous values).
Common Pitfalls
- Off-by-one between
dp[i]andhouse[i]— confusion between “first i houses” (useshouse[i-1]as the latest) and “ending at index i” (useshouse[i]). Pick one and never mix. - Forgetting the empty case —
dp[0]for “first 0 elements” must be the identity.
Classic Problems
- LC 70 — Climbing Stairs. See Lab 01.
- LC 198 — House Robber.
- LC 91 — Decode Ways.
- LC 53 — Maximum Subarray (Kadane).
- LC 746 — Min Cost Climbing Stairs.
8. 2D DP
When To Use
The state is a pair of integers — dp[i][j]. Examples: unique paths on a grid, minimum path sum, longest common subsequence, edit distance.
State Design
For grid problems, dp[i][j] = answer for getting to cell (i, j). For two-string problems, dp[i][j] = answer for prefix-i of one string and prefix-j of the other.
Transition
For grid: dp[i][j] = dp[i-1][j] + dp[i][j-1] (count of paths) or min(dp[i-1][j], dp[i][j-1]) + grid[i][j] (min path sum).
Complexity
Time O(N · M). Space O(N · M) tabulated, O(M) with rolling rows, O(M) with right-to-left collapse to 1D when there’s no same-row dependency.
Common Pitfalls
- Initializing first row and first column wrong for grid path problems — these are not always 0 or 1; they may carry obstacles or grid values.
- Adding
grid[i][j]to all transitions including the boundary — the boundary needs special handling.
Classic Problems
- LC 62 — Unique Paths.
- LC 63 — Unique Paths II (with obstacles). See Lab 02.
- LC 64 — Minimum Path Sum.
- LC 120 — Triangle.
9. 0/1 Knapsack
When To Use
A set of N items each with weight w_i and value v_i; capacity W; maximize value subject to total weight ≤ W. Each item used at most once. Recognized by discrete choices over a budget.
State Design
dp[i][w] = max value using first i items with capacity w.
Transition
dp[i][w] = max(dp[i-1][w], dp[i-1][w - w_i] + v_i) if w >= w_i, else dp[i-1][w]. The two cases: skip item i, or take it.
Complexity
Time O(N · W). Space O(W) with right-to-left collapse: iterate w from W down to w_i.
Common Pitfalls
- Iterating
wleft-to-right in the 1D-collapsed version — turns 0/1 knapsack into unbounded knapsack, allowing the same item to be picked multiple times. - Treating
Was a free variable when it’s actually constrained by problem size — atW = 10^9the table doesn’t fit; switch to meet-in-the-middle or branch-and-bound (out of scope here).
Classic Problems
- LC 416 — Partition Equal Subset Sum (0/1 knapsack reformulation: target = total / 2). See Lab 03.
- LC 494 — Target Sum.
- LC 474 — Ones and Zeroes (2D knapsack).
10. Unbounded Knapsack
When To Use
Same as 0/1 knapsack but each item can be used any number of times. Recognized by “unlimited supply” / “any number of coins” / “items can be reused”.
State Design
dp[w] = best value with capacity w, considering all items as candidates at every step.
Transition
dp[w] = max(dp[w], dp[w - w_i] + v_i) for every item i such that w >= w_i.
Complexity
Time O(N · W). Space O(W). Iterate w left-to-right.
Common Pitfalls
- Iterating wrong direction — same as 0/1 knapsack but inverted. Left-to-right makes items reusable; right-to-left makes them one-use.
- Confusing “min number of items” with “max value” — in coin change (min coins), initialize to
+∞, transition isdp[w] = min(dp[w], dp[w - c] + 1). - Counting orderings vs combinations: for “number of ways to make change as combinations”, the outer loop is over coins and inner over sums; for “number of ordered sequences”, swap them. The two produce different counts.
Classic Problems
- LC 322 — Coin Change (min coins). See Lab 04.
- LC 518 — Coin Change II (count combinations).
- LC 279 — Perfect Squares.
- LC 139 — Word Break (unbounded with “items” = dictionary words).
11. Subset Sum / Partition Equal Subset Sum
When To Use
“Can we pick a subset summing to T?” Recognized in: partition problems, target-sum problems, equal-sum-subsets.
Reformulation
Subset sum is 0/1 knapsack with v_i = w_i and target W = T. Use dp[w] = bool (reachable or not) instead of “max value”, and aggregate with OR instead of max.
Complexity
Time O(N · T). Space O(T) (bool array, can use a bitset for O(T / 64) space and time).
Common Pitfalls
- Forgetting that target
Tmay be huge — for “partition equal subset sum”,T = total / 2; iftotalis odd, return false immediately. - Using
maxinstead ofORfor boolean aggregation.
Classic Problems
- LC 416 — Partition Equal Subset Sum. See Lab 03.
- LC 698 — Partition to K Equal Sum Subsets (harder; bitmask DP).
12. LIS — Longest Increasing Subsequence
When To Use
“Longest subsequence with property P” where P is monotonic (increasing, non-decreasing, or some order relation).
State Design (O(N²) DP)
dp[i] = length of LIS ending at index i and using arr[i] as the last element.
Transition
dp[i] = 1 + max(dp[j] for j < i if arr[j] < arr[i]). Answer is max(dp[1..N]).
Complexity
O(N²) time, O(N) space.
Patience Sort / O(N log N) Variant
Maintain tails[k] = smallest possible tail of any increasing subsequence of length k+1. For each arr[i], find the leftmost tails[k] >= arr[i] via binary search and replace it with arr[i] (or append if arr[i] > all). The length of tails at the end is the LIS length.
This is patience sorting — laying cards onto piles where each pile is strictly decreasing top-to-bottom, and the number of piles is the LIS length.
Complexity
O(N log N) time, O(N) space.
Common Pitfalls
- Confusing “LIS length” with “LIS itself” —
tailsis not the LIS; reconstructing the actual sequence requires storing predecessors during scan. - Strict vs non-strict — for non-decreasing, use
bisect_rightinstead ofbisect_left.
Classic Problems
- LC 300 — Longest Increasing Subsequence. See Lab 05.
- LC 354 — Russian Doll Envelopes (sort + LIS).
- LC 673 — Number of Longest Increasing Subsequences.
13. LCS / Edit Distance Family
When To Use
Two strings, asking for similarity, alignment, or transformation cost. Includes longest common subsequence, edit distance (Levenshtein), longest common substring (different state!), and shortest common supersequence.
State Design
dp[i][j] = answer for prefix-i of A and prefix-j of B.
Transitions
- LCS:
dp[i][j] = dp[i-1][j-1] + 1ifA[i-1] == B[j-1], elsemax(dp[i-1][j], dp[i][j-1]). - Edit distance (Levenshtein): if match,
dp[i][j] = dp[i-1][j-1]; else1 + min(dp[i-1][j-1], dp[i-1][j], dp[i][j-1])for replace / delete / insert. - Longest common substring (different!): if match,
dp[i][j] = dp[i-1][j-1] + 1; elsedp[i][j] = 0. Answer ismax(dp[i][j])over all(i, j). The “else = 0” is what makes it substring vs subsequence.
Complexity
Time O(N · M). Space O(N · M) tabulated, O(M) with two rolling rows, O(M) with one row + a single saved diagonal value.
Common Pitfalls
- Confusing subsequence and substring — they have different recurrences. Subsequence allows skipping; substring requires contiguity.
- Edit distance with non-unit costs (insert/delete/replace each have a custom cost) — works the same with custom weights instead of
+1. - Reconstructing the alignment requires backtracking through
dpchoices; store back-pointers or reconstruct from values.
Classic Problems
- LC 1143 — Longest Common Subsequence.
- LC 72 — Edit Distance. See Lab 06.
- LC 583 — Delete Operation for Two Strings.
- LC 712 — Minimum ASCII Delete Sum.
- LC 718 — Maximum Length of Repeated Subarray (LCS variant; substring).
14. Palindrome DP
When To Use
Anything about palindromic substrings or subsequences: count, longest, partition into palindromes, minimum cuts.
Variant 1: Longest Palindromic Subsequence
dp[i][j] = length of longest palindromic subsequence in s[i..j].
dp[i][j] = dp[i+1][j-1] + 2 if s[i] == s[j]
= max(dp[i+1][j], dp[i][j-1]) otherwise
Answer: dp[0][N-1]. Evaluation order: by interval length ascending.
Variant 2: Longest Palindromic Substring
is_pal[i][j] = boolean. is_pal[i][j] = (s[i] == s[j]) and (j - i < 2 or is_pal[i+1][j-1]). Track max length and start during fill.
(Manacher’s algorithm gives O(N) for this; see Phase 3.)
Variant 3: Palindrome Partitioning Min Cuts
cuts[i] = min cuts to partition s[0..i] into palindromes.
cuts[i] = 0 if s[0..i] is itself a palindrome
= min(cuts[j-1] + 1) for all j ≤ i with s[j..i] palindrome
Precompute is_pal[i][j] first (O(N²)), then run the cut DP (O(N²)). Total O(N²).
Common Pitfalls
- Computing
is_palafter the cut DP — circular dependency. - Wrong evaluation order in
dp[i][j]— must fill smaller intervals first; iterate by length ascending.
Classic Problems
- LC 516 — Longest Palindromic Subsequence. See Lab 07.
- LC 5 — Longest Palindromic Substring.
- LC 132 — Palindrome Partitioning II. See Lab 07.
- LC 647 — Palindromic Substrings.
15. String DP
When To Use
Pattern matching with wildcards or operators: regex, glob/wildcard, interleaving, distinct subsequences. The state is two indices (one per string).
Variant: Regex Matching (LC 10)
dp[i][j] = does p[0..j] match s[0..i]?
if p[j-1] == '*':
dp[i][j] = dp[i][j-2] # match zero of preceding
or (matches(s[i-1], p[j-2]) and dp[i-1][j]) # match one more
elif matches(s[i-1], p[j-1]):
dp[i][j] = dp[i-1][j-1]
else:
dp[i][j] = False
Variant: Wildcard Matching (LC 44)
Similar but * matches any sequence: dp[i][j] = dp[i-1][j] or dp[i][j-1] when p[j-1] == '*'.
Variant: Interleaving Strings (LC 97)
dp[i][j] = can s3[0..i+j] be formed by interleaving s1[0..i] and s2[0..j]? Transition: take from s1 if s1[i-1] == s3[i+j-1]; take from s2 symmetrically; OR them.
Common Pitfalls
- Off-by-one between pattern index and
dpindex — almost universal source of regex DP bugs. *semantics differ between regex and glob; read the problem carefully.
Classic Problems
- LC 10 — Regular Expression Matching.
- LC 44 — Wildcard Matching.
- LC 97 — Interleaving String.
- LC 115 — Distinct Subsequences.
16. Tree DP
When To Use
The structure is a tree (rooted or rootable); the answer at a node depends on its subtree. Examples: house robber III, max path sum, longest path / diameter.
State Design
dp[v] = answer for the subtree rooted at v. Often a tuple: (best_with_v_chosen, best_without_v_chosen). Tuples are essential when the parent’s decision depends on whether the child was used.
Evaluation Order
Post-order DFS — fill children before the parent.
Transition
Aggregate over children. For house robber III: rob[v] = val[v] + sum(skip[c] for c in children); skip[v] = sum(max(rob[c], skip[c]) for c in children).
Complexity
Time O(V). Space O(V) for the recursion stack.
Common Pitfalls
- Stack overflow at deep trees in Python (default limit 1000) —
sys.setrecursionlimit(2 * 10**5)or convert to iterative post-order. - Mishandling N-ary vs binary children — N-ary requires summing over a dynamic list; binary is hard-coded
(left, right). - Forgetting to handle null children — return identity values (0 or -∞).
Classic Problems
- LC 337 — House Robber III. See Lab 08.
- LC 124 — Binary Tree Maximum Path Sum.
- LC 543 — Diameter of Binary Tree (variant).
- LC 968 — Binary Tree Cameras (multi-state tree DP).
17. Interval DP
When To Use
The state is (left, right) — an interval — and the transition picks a “split point” k in [left, right]. Examples: matrix chain multiplication, burst balloons, palindrome partitioning, optimal BST, stone game.
State Design
dp[i][j] = answer for interval [i, j]. Often the meaningful question is “what is the last operation on this interval”, which forces a choice of split point k.
Transition
dp[i][j] = aggregate over k in [i..j]: dp[i][k-1] + dp[k+1][j] + cost(i, j, k).
The cost(i, j, k) typically depends on the boundaries of the interval — not just k — because the interval’s neighbors after the split are still i-1 and j+1.
Evaluation Order
By interval length ascending: for length in 1..N: for left in 0..N-length: right = left + length - 1.
Complexity
Time O(N³) in general (O(N²) intervals × O(N) split points). Space O(N²).
Common Pitfalls
- Iterating
(i, j)in the wrong order — must fill smaller intervals first. Length-ascending is the canonical order. - Choosing the wrong “thing” to split on — e.g., for burst balloons, the right state is “last balloon to burst in
[i, j]” rather than “first balloon”. - Confusing the boundaries — the cost in burst balloons uses
nums[i-1]andnums[j+1]as multipliers because those are the surviving neighbors at the moment the last balloon in[i, j]is burst.
Classic Problems
- LC 312 — Burst Balloons. See Lab 09.
- LC 1547 — Minimum Cost to Cut a Stick (matrix-chain-like).
- LC 87 — Scramble String.
- LC 132 — Palindrome Partitioning II. See Lab 07.
18. Bitmask DP
When To Use
Small-N (typically N ≤ 20) problems where the state must remember which subset of items has been used. Examples: TSP, assignment problem, set cover, “shortest path visiting all nodes”.
State Design
dp[mask] = best value over subsets specified by mask. Or dp[mask][last] = best path ending at node last and visiting exactly the nodes in mask (TSP-style).
Transition
For TSP: dp[mask | (1 << v)][v] = min(dp[mask | (1 << v)][v], dp[mask][u] + dist(u, v)) for all u in mask and v not in mask.
Evaluation Order
By mask value ascending — guarantees dp[submask] is filled before dp[mask] whenever submask ⊂ mask. Equivalently, by popcount(mask) ascending.
Complexity
Time O(2^N · N²) for TSP-style. Space O(2^N · N) — at N=20 this is 20 × 10^6 = 20M cells, fits in memory.
Common Pitfalls
- Iterating bitmasks in the wrong order — by-mask-value ascending is the safe default.
- Off-by-one on
1 << vvs1 << (v-1)depending on 0- or 1-indexed nodes. - Forgetting
maskincludes the source when initializing. - Underestimating memory — at N=22, 2^N × N = 92M cells; at N=24, 400M+. Bitmask DP is strictly small-N.
Classic Problems
- LC 847 — Shortest Path Visiting All Nodes. See Lab 10.
- LC 943 — Find the Shortest Superstring.
- LC 1349 — Maximum Students Taking Exam (bitmask over rows).
- LC 1125 — Smallest Sufficient Team.
19. Digit DP (Overview)
When To Use
“Count numbers in [L, R] with property P” where P is digit-defined (sum of digits, no consecutive equal, contains a digit, etc.). The state is (position, tight, accumulator…).
State Design
dp[pos][tight][...accumulated state] where tight is a flag indicating whether the prefix so far equals the upper bound’s prefix (so the next digit is bounded).
Transition
For each digit d in 0..(9 if not tight else upper_bound[pos]), recurse to pos + 1 with tight' = tight and d == upper_bound[pos], updating the accumulator.
Complexity
Time O(D × 2 × digit_range × accumulator_size), typically tractable for D = 18 (decimal) and small accumulator.
Common Pitfalls
- Off-by-one between
[L, R]and[0, R]— answer iscount(R) - count(L-1). - Leading zeros — track a “started” flag; otherwise “001” and “1” are conflated.
- Memoizing on
tight=Truepaths — they’re path-specific and shouldn’t be memoized; only memoize thetight=Falsebranch.
Classic Problems
- LC 233 — Number of Digit One.
- LC 902 — Numbers At Most N Given Digit Set.
- LC 1012 — Numbers With Repeated Digits.
Overview-only in this phase; depth in Phase 7 (Competitive Programming).
20. DP On DAG
When To Use
The graph is acyclic; you want longest / shortest / count of paths. The DAG itself defines the topological order; the DP runs along it.
State Design
dp[v] = answer for paths ending at v (or starting from v).
Transition
For longest path: dp[v] = max(dp[u] + w(u, v) for u in predecessors(v)). Run in topological order on the DAG.
Complexity
Time O(V + E). Space O(V).
Common Pitfalls
- Running on a graph that has cycles — the recurrence diverges or memoization loops. Confirm DAG-ness with topological sort first.
- Confusing “longest path” (NP-hard in general graphs) with “longest path in a DAG” (polynomial) — always say “in a DAG” out loud.
Classic Problems
- LC 329 — Longest Increasing Path in a Matrix (implicit DAG).
- LC 1857 — Largest Color Value in a Directed Graph.
- “Longest path in a DAG” — folklore.
21. Game DP (Minimax / Nim / Stone Game)
When To Use
Two-player zero-sum perfect-information game; ask whether the first player wins, or by what margin. Examples: stone game, Nim, predict-the-winner.
State Design
dp[state] = the optimal score the current player can guarantee, assuming both play optimally. Often dp[i][j] with i, j being the two ends of a contested range.
Transition
The current player picks the choice that maximizes their own score. The opponent then plays from the resulting state, also optimally — so the value at the resulting state is what the opponent nets, not the current player. Hence:
dp[i][j] = max(stones[i] - dp[i+1][j], stones[j] - dp[i][j-1])
The -dp[...] flips perspective — the opponent’s optimal score becomes a deduction from the current player’s view.
Common Pitfalls
- Forgetting the perspective flip —
+dp[...]instead of-dp[...]. Produces nonsensical “both players cooperate” answers. - Confusing “current player wins” with “first player wins” —
dp[state]is from the perspective of whoever moves at this state, which may not be the original first player after several moves.
Classic Problems
- LC 486 — Predict the Winner.
- LC 877 — Stone Game.
- LC 1140 — Stone Game II.
- LC 464 — Can I Win (game DP + bitmask).
22. Probability And Expected Value DP
When To Use
Random walks, expected number of steps, probability of reaching a state. Examples: knight probability, dice problems, Markov chains in disguise.
State Design
dp[state] = probability of being in state after the random process, OR expected value of some random variable from state.
Transition
For probability: dp[next] = sum(P(s -> next) × dp[s]) over all predecessors. For expected value (with stopping): E[state] = expected_immediate + sum(P(s -> next) × E[next]) for non-terminal states; E[terminal] = 0.
Complexity
Same as the underlying state-space DP.
Common Pitfalls
- Conflating probability DP and expected-value DP — they have different recurrences; pick the right one for the question.
- Numerical stability — many small probabilities multiplied; use
logor rational arithmetic when extreme. - Infinite expected steps — if there’s a non-zero probability of never reaching the terminal, the expected value is infinite; check reachability first.
Classic Problems
- LC 688 — Knight Probability in Chessboard.
- LC 837 — New 21 Game.
- “Expected number of dice rolls to reach sum N” — folklore.
DP-Recognition Cheat Sheet
The hardest skill in this phase is recognizing that a problem is DP. Here is a battery of signals.
| Signal in problem statement | Likely DP variant |
|---|---|
| “Count number of ways” | Counting DP — sum over choices |
| “Maximum / minimum cost” with sequential choices | Optimization DP |
| “Pick subset with property P” / “partition” | Subset / knapsack |
| “Longest / shortest subsequence” | LIS / LCS family |
| “Edit / transform A into B” | Edit distance family |
| “Each item used at most once” | 0/1 knapsack |
| “Each item can be reused” | Unbounded knapsack |
| “Substring / subarray / contiguous” | 1D DP (often Kadane-like) |
| “Subsequence (non-contiguous)” | LCS / LIS family |
| “Palindromic” | Interval DP, expand-around-center, or LCS(s, rev(s)) |
“Match a pattern with * / .” | Regex / wildcard DP |
| “Tree” + “subtree answer aggregates” | Tree DP, post-order |
| “N ≤ 20” + “visit all” / “subset” | Bitmask DP |
| “N ≤ 100” + “split into intervals” / “merge intervals” | Interval DP, length-ascending |
| “Two-player game, both optimal” | Game DP, perspective-flip |
| “Probability” / “expected” + “random walk / dice” | Probability/EV DP |
| “Number of digits ≤ 18, range [L, R]” | Digit DP |
| “Acyclic graph + longest/count paths” | DP on DAG |
| “Climbing / hopping with steps {a, b, c}” | 1D DP, Fibonacci-like |
| “Decide YES/NO with budget K” | Reachability DP, often boolean knapsack |
Common DP Bugs
A taxonomy. Each one shows up in at least 30% of submitted DP solutions.
- Wrong base case.
dp[0]initialized to 0 when it should be 1 for counting, or 0 when it should be+∞for min. Check by running tabulated against memoized on N=0, 1, 2. - Wrong evaluation order. 2D DP iterated in
(j, i)order when the recurrence readsdp[i-1][j]. Interval DP iterated in(left, right)instead of(length, left). Bitmask DP iterated in arbitrary mask order. - Off-by-one between value-array index and DP index. If
dp[i]is “first i elements”, the latest element isarr[i-1], notarr[i]. Ifdp[i]is “ending at index i”, the latest element isarr[i]. Pick one and never mix. - Missing a choice in the transition. The “skip” / “do nothing” choice is the most-often-forgotten. Without it, you over-constrain the answer.
- Wrong direction in 1D-collapsed knapsack. Left-to-right (unbounded) vs right-to-left (0/1). Silently flipping turns one problem into the other.
- Counting orderings instead of combinations. In coin change variants, the loop nesting (coins outer vs sums outer) determines combinations vs permutations.
- Not handling unreachable states.
+∞propagation: if you computedp[w] = dp[w - c] + 1anddp[w-c] = +∞, yourdp[w]becomes a large finite number (in fixed-width integer types) — overflow. UseINF = 10^9 + 7and guard with explicitif dp[w-c] == INF: continue. - Recursion stack overflow in Python at N > 1000 — convert to iterative, or
sys.setrecursionlimit(10**6)and accept memory cost. - Memoizing on mutable arguments.
lru_cacherequires hashable args; lists / dicts must be tuples / frozensets. - Wrong perspective flip in game DP.
+dp[...]instead of-dp[...]. Both players appear to cooperate in your model. - Including or excluding the boundary of the table inconsistently. Off-by-one in iterators, inclusive/exclusive bounds.
- Time / space estimate ignoring constants. “O(N · M) at N = M = 10^4” is 10^8 — TLE in Python, fine in C++. State the constant honestly.
Mastery Checklist
Before exiting this phase, verify all of these:
- You can derive a state from a recursive brute force in <3 minutes for any DP problem.
- You can write the recurrence (transition) in <2 minutes once the state is fixed.
- You execute the brute → memo → tabulated → space-optimized progression on every DP problem in this phase, without skipping stages.
- You can write tabulated 1D DP (house robber, climbing stairs) in <5 minutes from a blank screen.
- You can write tabulated 2D DP (unique paths, edit distance) in <8 minutes from a blank screen.
- You can space-optimize 2D DP to O(M) on demand, including the right-to-left collapse trick for 0/1 knapsack.
- You can implement LIS at O(N²) and at O(N log N) in <15 minutes total.
- You can implement edit distance with full progression in <25 minutes.
- You can implement house robber III (tree DP) with the (rob, skip) tuple pattern in <15 minutes.
- You can implement burst balloons (interval DP) with the length-ascending iteration in <25 minutes.
-
You can implement TSP-style bitmask DP (
dp[mask][last]) in <30 minutes. -
You can articulate why iterating
for j: for i:in 2D DP can produce garbage — i.e., the topological-order argument — in <30 seconds. -
You can articulate why 0/1 knapsack iterates
wright-to-left and unbounded iterates left-to-right — in <30 seconds. - You can articulate the perspective-flip in game DP — in <30 seconds.
Exit Criteria
You may move to Phase 6 (Greedy and Mathematical Thinking) when all of the following are true:
- You have completed all ten labs in this phase, with each lab’s mastery criteria checked off.
- You have solved at least 50 unaided DP problems from LeetCode (mix of Medium, Medium-Hard, Hard) and reviewed each via REVIEW_TEMPLATE.md.
- Your unaided success rate on Medium-Hard DP problems is ≥ 65%.
- In a mock interview (phase-11-mock-interviews/), you correctly identify the DP variant within 2 minutes for at least 7 of 10 DP problems and produce the recurrence within 4 minutes for at least 6 of 10.
- You execute the brute → memo → tabulated → space-optimized progression on every DP problem in mocks, even when the interviewer doesn’t ask for all four stages — this is the single discipline of this phase, and skipping it is a phase-failure.
If any of these fails, do another 20–30 DP problems before moving on. Skipping this gate calcifies bad habits that destroy you in Phase 7 (competitive programming) where DP shows up at every turn.
Labs
Hands-on practice. Each lab follows the strict 22-section format and demonstrates the four-stage progression in detail.
- Lab 01 — 1D DP Foundations (House Robber)
- Lab 02 — 2D DP (Unique Paths with Obstacles)
- Lab 03 — 0/1 Knapsack (Partition Equal Subset Sum)
- Lab 04 — Unbounded Knapsack (Coin Change)
- Lab 05 — LIS (Longest Increasing Subsequence)
- Lab 06 — LCS / Edit Distance
- Lab 07 — Palindrome DP (LPS + Min Cuts)
- Lab 08 — Tree DP (House Robber III)
- Lab 09 — Interval DP (Burst Balloons)
- Lab 10 — Bitmask DP (Shortest Path Visiting All Nodes)
- Lab 11 — Digit DP (Numbers With Repeated Digits)
- Lab 12 — SOS DP (Sum over Subsets)
← Phase 4: Graph Mastery · Phase 6: Greedy → · Back to Top
Lab 01 — 1D DP Foundations (House Robber)
Goal
Implement House Robber (LC 198) four times — brute recursion, memoized, tabulated, and space-optimized — to internalize the brute → memo → tabulated → space-optimized progression that this entire phase is built around. After this lab you should be able to recognize a 1D DP problem in <60 seconds, derive the state and recurrence in <90 seconds, and produce the O(1)-space final solution from a blank screen in under 5 minutes.
Background Concepts
A 1D DP has state dp[i] indexed by a single integer — the prefix length, the position, or the day. The recurrence reads only O(1) previous values, which is what makes the rolling-array (O(1)-space) trick work. House Robber is the canonical example because it has exactly two choices per state (rob this house or skip), each of which determines the next state cleanly. The recursive formulation f(i) = max(f(i-1), f(i-2) + house[i-1]) reads two previous values; the tabulated version is a direct loop; the space-optimized version keeps two scalars.
The four-stage progression is the discipline of this lab. Don’t skip stages. The interviewer at staff level routinely asks “show me the recursive version first” specifically to test whether you can derive the recurrence from a brute force. Candidates who memorized the iterative solution but never wrote the recursion fail this question.
Interview Context
House Robber is a top-30 phone-screen DP problem at Amazon, Google, Microsoft, and Meta. Its variants — House Robber II (circular), House Robber III (tree, see Lab 08) — extend it. Bombing this problem on a phone screen is a near-instant rejection at L4. The reason: it has the simplest possible state (a single integer) and the simplest possible recurrence (two-way choice). If you can’t do this one, you can’t do any DP.
Problem Statement
You are a robber planning to rob houses arranged in a line. Each house has a non-negative integer amount of cash, given by nums[i]. You cannot rob two adjacent houses (the alarm system links them). Return the maximum amount of cash you can rob.
Constraints
- 1 ≤
nums.length≤ 100 - 0 ≤
nums[i]≤ 400
Clarifying Questions
- Are amounts non-negative? (Yes — given.)
- Can
numsbe empty? (No, length ≥ 1 by constraint, but always confirm.) - Are houses arranged in a line or a circle? (Line for LC 198; LC 213 is the circular variant.)
- Can two adjacent houses both be skipped? (Yes — skipping is always allowed.)
- Must we rob at least one house? (No — robbing nothing is allowed if all values are 0; in practice, since amounts are non-negative, the optimum is always ≥ 0.)
Examples
nums = [1, 2, 3, 1] → 4 (rob houses 0 and 2: 1 + 3)
nums = [2, 7, 9, 3, 1] → 12 (rob houses 0, 2, 4: 2 + 9 + 1)
nums = [2, 1, 1, 2] → 4 (rob houses 0 and 3: 2 + 2)
nums = [5] → 5
nums = [0, 0, 0] → 0
Initial Brute Force
At each house, two choices: rob it (and skip the next) or skip it. Recursively try both:
def rob_brute(nums):
def f(i):
if i >= len(nums):
return 0
return max(f(i + 1), nums[i] + f(i + 2))
return f(0)
Brute Force Complexity
Each call branches into 2 recursive calls, so we visit O(2^N) subproblems. At N=100, that’s 2^100 = 1.27 × 10^30 — far beyond any time limit. Space is O(N) for the recursion stack.
Optimization Path
The brute force is exponential because the same f(i) is recomputed exponentially many times. There are only N+1 distinct values of i, so memoization collapses the work to O(N). From there, tabulation removes the recursion overhead. Finally, since the recurrence reads only dp[i-1] and dp[i-2], we keep two scalars instead of the full array — O(1) space.
Each stage strictly improves on the previous: brute → memo (cache; from O(2^N) to O(N) time), memo → tabulated (loop instead of recursion; same complexity, no stack overhead), tabulated → space-optimized (drop the array; from O(N) to O(1) space).
Final Expected Approach
Define dp[i] = maximum cash robbed from the first i houses. Recurrence:
dp[0] = 0 (no houses to rob)
dp[1] = nums[0] (one house — rob it)
dp[i] = max(dp[i-1], # skip house i-1
dp[i-2] + nums[i-1]) # rob house i-1
Answer: dp[N]. Since the recurrence reads only dp[i-1] and dp[i-2], keep two scalars: prev2 and prev1.
Data Structures Used
- A 1D array
dpof sizeN+1(tabulated). - Two scalars
prev2,prev1(space-optimized). - For brute / memo: function call stack and a memoization dict /
lru_cache.
Correctness Argument
By induction on i. Base: dp[0] = 0 (correct — no houses). dp[1] = nums[0] (correct — one house, rob it). Inductive step: at step i, the optimal robbery either does or does not rob house i-1. If it does, the remaining is the optimal over the first i-2 houses (since we can’t rob i-1’s neighbors), giving dp[i-2] + nums[i-1]. If it does not, the remaining is the optimal over the first i-1 houses, giving dp[i-1]. Taking the max covers both cases — this exhausts the choice space, so the recurrence is correct.
Complexity
| Stage | Time | Space |
|---|---|---|
| Brute force | O(2^N) | O(N) (stack) |
| Memoized | O(N) | O(N) (cache + stack) |
| Tabulated | O(N) | O(N) |
| Space-optimized | O(N) | O(1) |
Implementation Requirements
All four stages are required.
# ---- Stage 1: Brute force ----
def rob_brute(nums):
def f(i):
if i >= len(nums):
return 0
return max(f(i + 1), nums[i] + f(i + 2))
return f(0)
# ---- Stage 2: Memoized ----
from functools import lru_cache
def rob_memo(nums):
@lru_cache(None)
def f(i):
if i >= len(nums):
return 0
return max(f(i + 1), nums[i] + f(i + 2))
return f(0)
# ---- Stage 3: Tabulated ----
def rob_tab(nums):
n = len(nums)
if n == 0: return 0
dp = [0] * (n + 1)
dp[1] = nums[0]
for i in range(2, n + 1):
dp[i] = max(dp[i-1], dp[i-2] + nums[i-1])
return dp[n]
# ---- Stage 4: Space-optimized ----
def rob(nums):
prev2, prev1 = 0, 0
for x in nums:
prev2, prev1 = prev1, max(prev1, prev2 + x)
return prev1
Tests
[]→ 0 (defensive, even if constraint disallows).[5]→ 5.[5, 1]→ 5.[1, 2, 3, 1]→ 4.[2, 7, 9, 3, 1]→ 12.[0, 0, 0, 0]→ 0.[400, 400, 400]→ 800 (rob ends).- All four implementations should produce identical results — write a randomized stress comparator on
numsof length 1..15 and checkrob_brute == rob == rob_tab == rob_memo.
Follow-up Questions
- “What if houses are in a circle?” (LC 213 — House Robber II) → Run the line algorithm twice: once excluding house 0, once excluding house N-1; take the max.
- “What if houses are nodes of a binary tree?” (LC 337 — House Robber III) → Tree DP with
(rob, skip)tuple per node. See Lab 08. - “Reconstruct which houses were robbed.” → Track back-pointers in tabulated version, or re-derive by walking
dpbackwards: at eachi, robbed iffdp[i] > dp[i-1]. - “What if the no-rob constraint extends to k-apart instead of adjacent?” →
dp[i] = max(dp[i-1], dp[i-k-1] + nums[i-1]). - “What if amounts can be negative?” → Same recurrence;
dp[i-2] + nums[i-1]may be less thandp[i-1], so the max correctly drops it.
Product Extension
Variations of this problem appear in real systems: scheduling non-conflicting jobs (interval scheduling with profit), selecting non-overlapping ad slots, and assigning tasks to time-slots with cooldown. The 1D DP framework generalizes when “no two adjacent” becomes “no two within window K” or “must wait at least T”.
Language/Runtime Follow-ups
- Python:
lru_cachemakes memoization a one-line addition. At N>1000, default recursion limit overflows — bump withsys.setrecursionlimit(10**6)or use the tabulated version. The space-optimized version is idiomatic and fast. - Java: use
int[] dpfor tabulated;Arrays.fill(dp, -1)+ recursion for memoization. Java’s default stack is ~512KB; recursion overflows around N=10000. - Go: tabulated is idiomatic; Go has no
lru_cacheso memoization needs a manualmap[int]intor[]int. - C++: tabulated with
vector<int>; memoization with avector<int> memo(N, -1)and a recursive helper. - JS/TS: same idiom as Python but no
lru_cache— useMapfor memoization.
Common Bugs
- Returning
dp[N-1]instead ofdp[N](or vice versa) — depends on whetherdp[i]indexes “first i houses” or “ending at i”. Pick one convention and stick to it. - Initializing
dp[0] = nums[0]anddp[1] = max(nums[0], nums[1])— works, but only if you handleN=1separately. The cleaner convention isdp[0]=0, dp[1]=nums[0]. - Off-by-one in
dp[i-2] + nums[i-1]vsdp[i-2] + nums[i-2]— depends on the index convention. Verify on[5]. - Forgetting that
numscan be empty — guard withif not nums: return 0even though constraints say N ≥ 1. - Space-optimized version: swapping
prev2, prev1 = prev1, max(prev1, prev2 + x)in the wrong order. Tuple-assignment in Python evaluates the RHS first, so this is correct; in Java/C++ you need an explicit temp.
Debugging Strategy
When the answer is wrong by a small amount: print the entire dp array for nums = [2, 7, 9, 3, 1] (expected dp = [0, 2, 7, 11, 11, 12]). If it differs, trace the iteration step where dp first deviates and inspect the recurrence at that index. When the answer is wildly wrong (negative, or much smaller): suspect index off-by-one or an if condition that’s flipped. When TLE: confirm you’re not running the brute force.
Mastery Criteria
- Recognized House Robber as a 1D DP problem within 60 seconds.
- Wrote the brute recursive formulation in <2 minutes from cold start.
-
Added
@lru_cacheto produce the memoized version in <30 seconds. - Wrote the tabulated version in <3 minutes from blank screen, passing all five test cases first try.
- Wrote the space-optimized version in <2 minutes after the tabulated.
- Stated O(N) time and O(1) space unprompted.
- Articulated the inductive correctness argument in <30 seconds.
- Solved LC 198 unaided in <8 minutes total (all four stages).
- Solved LC 213 (House Robber II) unaided in <12 minutes by running the line algorithm twice.
Lab 02 — 2D DP (Unique Paths with Obstacles)
Goal
Solve Unique Paths II (LC 63) with the full brute → memo → tabulated → space-optimized progression. Internalize the canonical 2D DP loop structure (for i: for j:) and the rolling-row trick that reduces O(N · M) space to O(M). After this lab you should be able to write any grid-DP problem from a blank screen in <8 minutes and apply the rolling-row collapse on demand.
Background Concepts
A 2D DP has state dp[i][j] indexed by two integers. For grid problems, (i, j) is a cell, and the recurrence aggregates over the (at most) two predecessors (i-1, j) and (i, j-1). Because each row depends only on the previous row, the table can be rolled down to a single 1D array of length M+1 — half the memory, identical answers.
The grid-DP family is the cleanest 2D DP family because the dependency graph is trivially layered (row by row). It is the right place to learn the rolling-row mechanic before applying it to harder 2D DPs (knapsack, edit distance, LCS).
Interview Context
Unique Paths II is a top-50 Medium DP problem at Microsoft, Amazon, and Bloomberg. The non-obstacle variant (LC 62) shows up at every L3 phone screen. The obstacle variant adds a wrinkle: cells with grid[i][j] == 1 are blocked and contribute 0. Candidates who try a closed-form combinatorial answer (C(N+M-2, N-1)) get stuck the moment obstacles appear — the only general approach is DP. Showing all four stages (brute, memo, tabulated, O(M)-space) signals senior fluency.
Problem Statement
Given an m × n grid obstacleGrid where each cell is either 0 (open) or 1 (obstacle), count the number of distinct paths from (0, 0) to (m-1, n-1). Movement is restricted to right or down by one cell. If the start or end is blocked, the answer is 0.
Constraints
- 1 ≤ m, n ≤ 100
obstacleGrid[i][j]is0or1.- The result is guaranteed to fit in a 32-bit signed integer.
Clarifying Questions
- Can the start or end be an obstacle? (Yes — answer is
0if so.) - Are diagonal moves allowed? (No — only right and down.)
- Are paths considered distinct if they share intermediate cells? (Yes — only the sequence of moves matters.)
- Modular arithmetic required? (No — fits in int32.)
- Is
m=1, n=1valid? (Yes; answer is1if open,0if blocked.)
Examples
[[0,0,0],
[0,1,0],
[0,0,0]] → 2
[[0,1],
[0,0]] → 1
[[1]] → 0 (start blocked)
[[0]] → 1
Initial Brute Force
At each open cell, try moving right or down recursively:
def paths_brute(grid):
m, n = len(grid), len(grid[0])
def f(i, j):
if i >= m or j >= n or grid[i][j] == 1:
return 0
if i == m - 1 and j == n - 1:
return 1
return f(i + 1, j) + f(i, j + 1)
return f(0, 0)
Brute Force Complexity
Each call branches into two; depth is m + n - 2. Worst-case calls: 2^(m+n-2). At m=n=100, that’s 2^198 ≈ 4 × 10^59 — TLE. Space is O(m+n) for the recursion stack.
Optimization Path
The brute force recomputes f(i, j) exponentially. There are only m × n distinct (i, j) pairs, so memoization collapses time to O(m · n). Tabulation replaces recursion with a row-major loop. Since dp[i][j] reads dp[i-1][j] and dp[i][j-1], the previous row plus the in-progress row are sufficient — collapse to a single 1D array iterated left-to-right (no same-row dependency conflict because we read dp[j-1] before overwriting it, and dp[j] from the previous row is what’s already there).
Final Expected Approach
Define dp[i][j] = number of paths from (0, 0) to (i, j). Recurrence:
dp[0][0] = 1 if grid[0][0] == 0 else 0
dp[i][j] = 0 if grid[i][j] == 1
= dp[i-1][j] + dp[i][j-1] otherwise (treat out-of-bounds as 0)
Roll to 1D: dp[j] += dp[j-1] for each row, with dp[j] = 0 if blocked.
Data Structures Used
- 2D array
dpof sizem × n(tabulated). - 1D array
dpof sizen(rolled). - For brute / memo: recursion stack +
lru_cache.
Correctness Argument
Every path to (i, j) arrives via the cell above or the cell to the left. The number of paths to (i, j) is the sum of paths to those two predecessors (when neither is out-of-bounds and both are open). This holds because the two predecessor paths are disjoint (the last move differs) and exhaust all paths. Blocked cells contribute 0 directly. Base: dp[0][0] = 1 if open, else 0. Induction over the row-major topological order proves correctness for all cells.
Complexity
| Stage | Time | Space |
|---|---|---|
| Brute force | O(2^(m+n)) | O(m+n) |
| Memoized | O(m · n) | O(m · n) |
| Tabulated | O(m · n) | O(m · n) |
| Space-optimized | O(m · n) | O(n) |
Implementation Requirements
All four stages.
# ---- Stage 1: Brute force ----
def paths_brute(grid):
m, n = len(grid), len(grid[0])
def f(i, j):
if i >= m or j >= n or grid[i][j] == 1:
return 0
if i == m - 1 and j == n - 1:
return 1
return f(i + 1, j) + f(i, j + 1)
return f(0, 0)
# ---- Stage 2: Memoized ----
from functools import lru_cache
def paths_memo(grid):
m, n = len(grid), len(grid[0])
@lru_cache(None)
def f(i, j):
if i >= m or j >= n or grid[i][j] == 1: return 0
if i == m - 1 and j == n - 1: return 1
return f(i + 1, j) + f(i, j + 1)
return f(0, 0)
# ---- Stage 3: Tabulated ----
def paths_tab(grid):
m, n = len(grid), len(grid[0])
if grid[0][0] == 1 or grid[m-1][n-1] == 1: return 0
dp = [[0] * n for _ in range(m)]
dp[0][0] = 1
for i in range(m):
for j in range(n):
if grid[i][j] == 1:
dp[i][j] = 0
continue
if i > 0: dp[i][j] += dp[i-1][j]
if j > 0: dp[i][j] += dp[i][j-1]
return dp[m-1][n-1]
# ---- Stage 4: Space-optimized (1D rolled) ----
def uniquePathsWithObstacles(grid):
m, n = len(grid), len(grid[0])
if grid[0][0] == 1: return 0
dp = [0] * n
dp[0] = 1
for i in range(m):
for j in range(n):
if grid[i][j] == 1:
dp[j] = 0
elif j > 0:
dp[j] += dp[j-1]
return dp[n-1]
Tests
[[0,0,0],[0,1,0],[0,0,0]]→ 2.[[0,1],[0,0]]→ 1.[[1]]→ 0.[[0]]→ 1.[[0,0],[1,1],[0,0]]→ 0 (no path past the blocking row).[[0,0,0,0,0]]→ 1.[[0],[0],[0]]→ 1.- m=n=100 with random 10% obstacles — performance test.
Follow-up Questions
- “What if there are diagonal moves?” →
dp[i][j] += dp[i-1][j-1]as a third predecessor. - “Each cell has a cost; minimize total path cost.” → Min-path-sum (LC 64);
mininstead of+. - “K obstacles can be removed.” → 3D DP
dp[i][j][k]= paths to(i, j)having removedkobstacles. - “Reconstruct one valid path.” → Backtrack through
dpfrom the target; at each cell pick a predecessor with non-zero contribution. - “Grid is enormous (m=n=10^9) but obstacles are sparse.” → Combinatorial answer (
C(m+n-2, n-1)) minus inclusion-exclusion over obstacles. Out of scope here.
Product Extension
Routing on a city grid with road closures, robot path planning with obstacles, dependency-graph traversal with disabled edges. The grid-DP framework generalizes to any DAG where the topological order is row-major.
Language/Runtime Follow-ups
- Python:
[[0]*n for _ in range(m)]allocates correctly;[[0]*n]*mshares row references — a classic bug. Use a comprehension. - Java:
int[][] dp = new int[m][n];zero-initializes by default. - Go: pre-allocate the slice-of-slices explicitly.
- C++:
vector<vector<int>> dp(m, vector<int>(n, 0));. - JS/TS:
Array.from({length: m}, () => new Array(n).fill(0))to avoid the shared-reference trap.
Common Bugs
- Shared row references in Python:
dp = [[0]*n]*mmakes all rows alias the same list. Use a comprehension. - Forgetting to check
grid[0][0] == 1: if the start is blocked, the answer is 0, butdp[0][0] = 1would propagate non-zero counts through the grid. - Using
if i > 0 and j > 0instead of two separateifs — silently misses one of the two predecessors. - Iterating columns outer, rows inner — works for this problem since
dp[i][j]only reads upward and leftward, but breaks the rolled-1D version. - Rolled 1D version: forgetting to set
dp[j] = 0on obstacle — old non-zero value persists from the previous row.
Debugging Strategy
Print the full dp table for the 3×3 obstacle example. Expected:
1 1 1
1 0 1
1 1 2
If yours diverges at row 1, suspect the obstacle handling. If at row 2, suspect the dp[j] = 0 reset. For the rolled-1D version, print dp after each row.
Mastery Criteria
- Recognized this as a 2D grid DP within 60 seconds.
- Wrote the brute recursion in <2 minutes.
- Wrote the tabulated 2D version in <5 minutes from blank screen.
- Performed the rolling-row collapse to 1D in <2 minutes from the tabulated version.
- Stated O(m·n) time and O(n) space for the final solution unprompted.
- Articulated why the rolled-1D version iterates left-to-right (no same-row conflict).
- Solved LC 62 (no obstacles) in <5 minutes.
- Solved LC 63 unaided in <12 minutes total.
-
Solved LC 64 (min path sum) in <8 minutes by changing
+tomin.
Lab 03 — 0/1 Knapsack (Partition Equal Subset Sum)
Goal
Solve Partition Equal Subset Sum (LC 416) by reducing it to 0/1 knapsack. Internalize the right-to-left iteration that makes the 1D-collapsed knapsack correct, and articulate why left-to-right iteration would silently turn it into unbounded knapsack. After this lab you should recognize any subset-sum / partition / target-sum / select-with-budget problem as 0/1 knapsack within 90 seconds.
Background Concepts
0/1 knapsack: N items each with weight w_i and value v_i; pick a subset with total weight ≤ W maximizing total value. The 2D DP has state dp[i][w] = max value using first i items with capacity w. The 1D-collapsed version uses dp[w] and iterates w from W down to w_i — the right-to-left iteration is what prevents an item from being reused within the same outer iteration.
Subset sum is 0/1 knapsack with v_i = w_i and a boolean dp instead of integer-valued. Partition equal subset sum reduces to “is there a subset summing to total / 2?”; if total is odd, return false immediately.
Interview Context
Partition Equal Subset Sum is a top-25 Medium DP problem at Amazon and Microsoft. The 0/1 knapsack pattern shows up in disguise constantly: target sum (LC 494), ones and zeroes (LC 474), last stone weight II (LC 1049), tallest billboard (LC 956). Recognizing the reduction is half the battle. The other half is the right-to-left iteration trick — getting that wrong is one of the most common DP bugs across the entire interview corpus.
Problem Statement
Given a non-empty array nums of positive integers, determine whether it can be partitioned into two subsets with equal sums.
Constraints
- 1 ≤
nums.length≤ 200 - 1 ≤
nums[i]≤ 100
So the maximum total sum is 200 × 100 = 20,000, and the target is at most 10,000. The 2D DP has 200 × 10001 = 2 × 10^6 cells — comfortable.
Clarifying Questions
- Are elements positive? (Yes — given.)
- Must the partition use all elements? (Yes — that’s what “partition” means.)
- Is the empty subset allowed on either side? (Yes if total is 0 — vacuously true. Not the case here since
nums[i] ≥ 1.) - Are duplicates allowed? (Yes — they’re treated as separate items.)
- Return value: bool (true / false).
Examples
[1, 5, 11, 5] → true (1+5+5 == 11)
[1, 2, 3, 5] → false (total=11, odd)
[1, 2, 5] → false (total=8, target=4, no subset sums to 4)
[2, 2, 1, 1] → true (2+1 == 2+1)
[100] → false (total=100, target=50, no subset)
Initial Brute Force
For each element, recurse on “include it” and “skip it”:
def can_partition_brute(nums):
total = sum(nums)
if total % 2: return False
target = total // 2
def f(i, remain):
if remain == 0: return True
if i == len(nums) or remain < 0: return False
return f(i + 1, remain - nums[i]) or f(i + 1, remain)
return f(0, target)
Brute Force Complexity
O(2^N) time, O(N) stack. At N=200, 2^200 — completely infeasible.
Optimization Path
There are only N × (target + 1) distinct (i, remain) pairs, so memoization gives O(N · target) time and space. Tabulation replaces recursion with a 2D loop. Since dp[i][w] only reads dp[i-1][...], roll to 1D dp[w] — but iterate w right-to-left so that each item is considered at most once per outer iteration.
The right-to-left direction is the defining trick of 0/1 knapsack. If we iterate left-to-right, then dp[w - w_i] may have already been updated to include item i from the current outer iteration; we’d then re-include item i, turning the algorithm into unbounded knapsack.
Final Expected Approach
Reduce to subset sum: target = total / 2 (or return false if total is odd).
dp[w] = True if some subset sums to exactly w, considering items processed so far.
dp[0] = True (empty subset sums to 0).
For each num x in nums:
for w in range(target, x - 1, -1):
dp[w] = dp[w] or dp[w - x]
Answer: dp[target]
Data Structures Used
- 2D
dp[N+1][target+1]boolean array (tabulated). - 1D
dp[target+1]boolean array (space-optimized). - For brute / memo: recursion +
lru_cache.
Correctness Argument
Inductive on items processed. dp[w] = True iff some subset of items processed so far sums to w. Base: dp[0] = True (empty subset). Inductive step: when we process item x, the new dp[w] is True iff (a) it was True before (subset not using x sums to w), OR (b) dp[w - x] was True before processing x (subset summing to w - x plus item x). The right-to-left iteration ensures we read the previous dp[w - x], not the in-iteration one. Termination: we want dp[target] after all items are processed.
Complexity
| Stage | Time | Space |
|---|---|---|
| Brute force | O(2^N) | O(N) |
| Memoized | O(N · target) | O(N · target) |
| Tabulated | O(N · target) | O(N · target) |
| Space-optimized | O(N · target) | O(target) |
For LC 416: N≤200, target≤10000, so ~2×10^6 ops — fast.
Implementation Requirements
All four stages.
# ---- Stage 1: Brute force ----
def can_partition_brute(nums):
total = sum(nums)
if total % 2: return False
target = total // 2
def f(i, remain):
if remain == 0: return True
if i == len(nums) or remain < 0: return False
return f(i + 1, remain - nums[i]) or f(i + 1, remain)
return f(0, target)
# ---- Stage 2: Memoized ----
from functools import lru_cache
def can_partition_memo(nums):
total = sum(nums)
if total % 2: return False
target = total // 2
@lru_cache(None)
def f(i, remain):
if remain == 0: return True
if i == len(nums) or remain < 0: return False
return f(i + 1, remain - nums[i]) or f(i + 1, remain)
return f(0, target)
# ---- Stage 3: Tabulated 2D ----
def can_partition_tab(nums):
total = sum(nums)
if total % 2: return False
target = total // 2
n = len(nums)
dp = [[False] * (target + 1) for _ in range(n + 1)]
for i in range(n + 1):
dp[i][0] = True
for i in range(1, n + 1):
for w in range(1, target + 1):
dp[i][w] = dp[i-1][w]
if w >= nums[i-1]:
dp[i][w] = dp[i][w] or dp[i-1][w - nums[i-1]]
return dp[n][target]
# ---- Stage 4: Space-optimized 1D ----
def canPartition(nums):
total = sum(nums)
if total % 2: return False
target = total // 2
dp = [False] * (target + 1)
dp[0] = True
for x in nums:
for w in range(target, x - 1, -1): # RIGHT-TO-LEFT
dp[w] = dp[w] or dp[w - x]
return dp[target]
Tests
[1, 5, 11, 5]→ True.[1, 2, 3, 5]→ False (odd total).[1, 2, 5]→ False (no valid subset).[2, 2, 1, 1]→ True.[100]→ False.[1, 1]→ True.- N=200, all
nums[i]=1→ True (target=100; pick 100 of them). - All four implementations should produce identical bool results — randomized comparator on N≤15.
Follow-up Questions
- “Return one valid subset, not just yes/no.” → Track parent pointers in the 2D DP; reconstruct by walking backwards through
(i, w). - “Partition into K equal subsets.” (LC 698) → 0/1-knapsack-style DP becomes intractable; use bitmask DP or backtracking with pruning.
- “Target sum: how many ways to assign +/- to each number to total exactly
T?” (LC 494) → Reduce to subset sum: count subsets summing to(total + T) / 2. - “Minimum subset sum difference.” (LC 1049) → Find largest
s ≤ total/2reachable; answer istotal - 2s. - “What if
nums[i]can be huge (up to 10^9)?” → Knapsack space blows up. Use Karp-style or reduce by GCD; otherwise NP-hard in general.
Product Extension
Resource allocation (split a budget across two teams equally), load balancing (split a workload across two workers), and “is there a subset with this exact total?” appear in billing systems, accounting reconciliation, and cluster-resource schedulers.
Language/Runtime Follow-ups
- Python:
dp = [False] * (target + 1)is fine; the inner loop’srange(target, x - 1, -1)is the canonical right-to-left form. - Java:
boolean[] dp = new boolean[target + 1];defaults to false. Use aBitSetfor ~64x speedup:dp.or(dp << x)does the entire row-update in O(target / 64). - Go:
make([]bool, target+1)and a manual reversed loop. - C++:
vector<bool>is bit-packed;bitset<10001>is faster but fixed-size. - JS/TS:
new Uint8Array(target + 1)to avoid thefalsedefault-equals-undefined trap.
Common Bugs
- Iterating
wleft-to-right in the 1D version — turns 0/1 into unbounded; spuriousTrueanswers. - Forgetting the odd-total short circuit — wastes time and may TLE on edge cases.
- Using
dp[w] = dp[w-x]instead ofdp[w] or dp[w-x]— wipes out previously-set True values. - Off-by-one in
range(target, x - 1, -1)— should includew == x(sincedp[x] = dp[x] or dp[0] = Truefor anyx ≤ target). - Setting
dp[0] = Trueonly on the first iteration — must be set once before any item is processed.
Debugging Strategy
For [1, 5, 11, 5]: after processing [1], dp = [T, T, F, F, F, F, F, F, F, F, F, F] (indexes 0..11). After [1, 5]: dp[6] = T (1+5). After [1, 5, 11]: dp[11] = T. Print dp after each item; if dp[target] becomes True earlier than expected, you’re allowing item-reuse (left-to-right bug).
Mastery Criteria
- Recognized partition-equal-subset as 0/1 knapsack within 90 seconds.
-
Wrote the reduction to subset sum (
target = total / 2) before any code. - Wrote the brute recursion in <2 minutes.
- Wrote the 2D tabulated version in <5 minutes.
- Performed the 1D collapse with right-to-left iteration in <2 minutes.
- Articulated why left-to-right would be wrong (item reuse → unbounded knapsack) in <30 seconds.
- Stated O(N · target) time and O(target) space unprompted.
- Solved LC 416 unaided in <12 minutes (all four stages).
- Solved LC 494 (Target Sum) in <12 minutes via the reduction.
Lab 04 — Unbounded Knapsack (Coin Change)
Goal
Solve Coin Change (LC 322 — minimum coins) and Coin Change II (LC 518 — count combinations) with the full four-stage progression. Internalize the left-to-right iteration that makes 1D-collapsed unbounded knapsack correct, and the loop-nesting trick that distinguishes counting combinations from counting permutations. After this lab you can solve any “unlimited supply” knapsack in <10 minutes from cold start.
Background Concepts
Unbounded knapsack: items can be reused any number of times. The 1D-collapsed DP iterates w left-to-right, the opposite of 0/1 knapsack. That single direction-change is the entire mechanical difference. The semantic difference: when we read dp[w - c] in the left-to-right pass, it has already been updated this round to include coin c zero or more times — so this round’s update can stack another c on top, achieving “use c multiple times”.
Coin Change has two flavors. LC 322 asks for the minimum number of coins to reach amount; the recurrence is dp[w] = min(dp[w], dp[w - c] + 1), initialized to +∞ with dp[0] = 0. LC 518 asks for the count of combinations summing to amount; the recurrence is dp[w] += dp[w - c], initialized to dp[0] = 1. The combinations-vs-permutations trap: with coins outer, sums inner, you count combinations (each combination of coins is counted once regardless of order); with sums outer, coins inner, you count permutations (different orderings of the same coins count separately).
Interview Context
Coin Change (LC 322) is a top-15 phone-screen DP problem at every major company. Coin Change II is asked roughly half as often but tests the deeper combinations-vs-permutations distinction. Bombing LC 322 at L4+ is a near-instant rejection. Senior interviewers often follow up with LC 518 specifically to test whether you understand why the loop nesting matters — the hand-wavy candidate is filtered by this question.
Problem Statement
LC 322 (minimum coins): Given coins of distinct denominations and an integer amount, return the fewest number of coins needed to make up amount. Return -1 if unreachable. Each coin denomination has unlimited supply.
LC 518 (count combinations): Given coins and amount, return the number of distinct combinations that sum to amount.
Constraints
- 1 ≤
coins.length≤ 12 (LC 322) / 300 (LC 518). - 1 ≤
coins[i]≤ 2^31 − 1. - 0 ≤
amount≤ 10^4 (LC 322) / 5000 (LC 518). - LC 518: answer fits in a signed 32-bit integer.
Clarifying Questions
- Are coins distinct? (Yes — given.)
- Can each coin be used multiple times? (Yes — unlimited supply; this is what makes it unbounded.)
- Is amount=0 valid? (Yes; minimum coins = 0; combinations = 1 — the empty combination.)
- LC 518: is
[1, 2]different from[2, 1]? (No — combinations only, not permutations.) - Coins can exceed
amount? (Yes; just unusable for that amount.)
Examples
LC 322:
coins=[1,2,5], amount=11 → 3 (5+5+1)
coins=[2], amount=3 → -1 (unreachable)
coins=[1], amount=0 → 0
coins=[1,2,5], amount=100 → 20 (twenty 5-coins)
LC 518:
coins=[1,2,5], amount=5 → 4 ([5], [2,2,1], [2,1,1,1], [1×5])
coins=[2], amount=3 → 0
coins=[10], amount=10 → 1
Initial Brute Force (LC 322)
At each step, try every coin:
def coinChange_brute(coins, amount):
def f(remain):
if remain == 0: return 0
if remain < 0: return float('inf')
best = float('inf')
for c in coins:
best = min(best, f(remain - c) + 1)
return best
ans = f(amount)
return ans if ans != float('inf') else -1
Brute Force Complexity
Each call branches into len(coins) recursive calls; depth amount / min(coins). Worst case O(K^(amount)) — exponential. At K=12, amount=10^4, completely infeasible.
Optimization Path
There are only amount + 1 distinct values of remain, so memoization gives O(K · amount) time. Tabulation replaces recursion with a loop. The 1D version uses left-to-right iteration so each coin can be reused.
For LC 518 (counting), the order matters: coins outer (combinations) vs sums outer (permutations). The combinations interpretation is what LC 518 wants.
Final Expected Approach
LC 322 (minimum):
dp[w] = min coins to make w. dp[0] = 0; dp[w > 0] = INF.
For each w in 1..amount:
For each c in coins where c <= w:
dp[w] = min(dp[w], dp[w - c] + 1)
Answer: dp[amount] if dp[amount] != INF else -1.
LC 518 (count combinations):
dp[w] = number of combinations summing to w. dp[0] = 1; rest 0.
For each c in coins: # COINS OUTER
For each w in c..amount: # SUMS INNER, left-to-right
dp[w] += dp[w - c]
Answer: dp[amount].
Data Structures Used
- 1D
dp[amount+1]integer array. - For brute / memo: recursion +
lru_cache.
Correctness Argument
LC 322: dp[w] = min coins to reach w, by induction on w. Base: dp[0] = 0. Inductive step: any optimal solution for w ends with some coin c, leaving w - c to be solved optimally — dp[w - c] + 1. Take the minimum over all coins. Unreachable states stay at INF, propagating correctly under min.
LC 518: by the outer-coins loop, after processing coins c_1, ..., c_k, dp[w] counts combinations of those coins summing to w. Inductive step: when we process c_{k+1} with the inner left-to-right loop, the update dp[w] += dp[w - c_{k+1}] adds combinations that use at least one c_{k+1}. Because the inner loop is left-to-right, dp[w - c_{k+1}] already includes solutions using c_{k+1} zero or more times — so this update accounts for using c_{k+1} exactly 1, 2, 3, … times in turn. Each combination is counted exactly once because every combination has a latest coin index, and only the iteration on that coin index counts it. Outer-coins prevents reordering: [1, 2] and [2, 1] are not separately counted.
Complexity
| Stage | Time | Space |
|---|---|---|
| Brute force | O(K^amount) | O(amount) |
| Memoized | O(K · amount) | O(amount) |
| Tabulated | O(K · amount) | O(amount) |
| Space-optimized | (same as tabulated; already 1D) | O(amount) |
Implementation Requirements
All four stages for LC 322; tabulated only for LC 518.
# ==== LC 322: Coin Change (minimum coins) ====
# ---- Stage 1: Brute force ----
def coinChange_brute(coins, amount):
def f(remain):
if remain == 0: return 0
if remain < 0: return float('inf')
return min((f(remain - c) + 1 for c in coins), default=float('inf'))
ans = f(amount)
return ans if ans != float('inf') else -1
# ---- Stage 2: Memoized ----
from functools import lru_cache
def coinChange_memo(coins, amount):
@lru_cache(None)
def f(remain):
if remain == 0: return 0
if remain < 0: return float('inf')
return min((f(remain - c) + 1 for c in coins), default=float('inf'))
ans = f(amount)
return ans if ans != float('inf') else -1
# ---- Stage 3+4: Tabulated 1D (already optimal space) ----
def coinChange(coins, amount):
INF = amount + 1
dp = [INF] * (amount + 1)
dp[0] = 0
for w in range(1, amount + 1): # SUMS OUTER, COINS INNER for min variant
for c in coins:
if c <= w:
dp[w] = min(dp[w], dp[w - c] + 1)
return dp[amount] if dp[amount] != INF else -1
# ==== LC 518: Coin Change II (count combinations) ====
def change(amount, coins):
dp = [0] * (amount + 1)
dp[0] = 1
for c in coins: # COINS OUTER
for w in range(c, amount + 1): # SUMS INNER, LEFT-TO-RIGHT
dp[w] += dp[w - c]
return dp[amount]
Tests
- LC 322:
coins=[1,2,5], amount=11→ 3. - LC 322:
coins=[2], amount=3→ -1. - LC 322:
coins=[1], amount=0→ 0. - LC 322:
coins=[186, 419, 83, 408], amount=6249→ 20. - LC 518:
coins=[1,2,5], amount=5→ 4. - LC 518:
coins=[2], amount=3→ 0. - LC 518:
coins=[10], amount=10→ 1. - LC 518:
amount=0→ 1 (empty combination). - Compare: For LC 518 with sums-outer (the wrong way),
coins=[1,2], amount=3gives 3 (1+1+1, 1+2, 2+1) instead of 2 (1+1+1, 1+2).
Follow-up Questions
- “Reconstruct one valid combination.” → Track which coin produced each
dp[w]; backtrack fromdp[amount]. - “What if coins are large (up to 10^9)?” → Knapsack table doesn’t fit; switch to BFS over reachable amounts (still O(amount × K)) or to coin-set-specific number theory.
- “Constraint: at most K coins total.” → Add a dimension:
dp[w][k]= min/count using ≤ k coins. - “All combinations summing to exactly amount, not just count.” → Backtracking; output is exponential in worst case.
- “What if some coins have limited supply?” → Bounded knapsack; binary-decompose each coin’s count and reduce to 0/1.
Product Extension
Cash-register optimization (which bills/coins to dispense for change), packet-payload composition (combining MTU-aware fragments), and currency-change problems in financial systems — all reduce to coin change variants.
Language/Runtime Follow-ups
- Python:
INF = amount + 1(since at mostamountcoins of value 1) avoidsfloat('inf')arithmetic. - Java:
int[] dp = new int[amount+1]; Arrays.fill(dp, amount+1); dp[0]=0;. - Go: pre-fill via loop; no
Arrays.fillshortcut. - C++:
vector<int> dp(amount+1, amount+1); dp[0]=0;. - JS/TS:
new Array(amount+1).fill(amount+1)thendp[0]=0.
Common Bugs
- LC 322: iterating coins outer, sums inner — works for the minimum variant; misleading for those who later try LC 518 with the same nesting and get permutations.
- LC 518: iterating sums outer, coins inner — counts permutations (
[1,2]and[2,1]separately) instead of combinations. - Forgetting
dp[0] = 1in LC 518 — every count becomes 0. - Using
float('inf') + 1arithmetic in Python — works (inf + 1 == inf), but slower and obscures intent. PreferINF = amount + 1. - Forgetting the
c <= wguard — out-of-bounds indexdp[w - c]whenw < c. - Off-by-one in
range(1, amount + 1)— must reachamountinclusive.
Debugging Strategy
For LC 322 with coins=[1,2,5], amount=11: after the loop, dp = [0,1,1,2,2,1,2,2,3,3,2,3]. Print dp and check dp[11] = 3. For LC 518 with coins=[1,2,5], amount=5: after processing coin 1, dp = [1,1,1,1,1,1]. After coin 2: dp = [1,1,2,2,3,3]. After coin 5: dp = [1,1,2,2,3,4]. Walking through this manually catches loop-nesting bugs.
Mastery Criteria
- Recognized “unlimited coins” as unbounded knapsack within 60 seconds.
- Wrote LC 322 brute recursion in <2 minutes.
- Wrote LC 322 tabulated in <5 minutes.
- Articulated why unbounded uses left-to-right iteration in <30 seconds.
- Wrote LC 518 with the correct outer-coins loop in <5 minutes.
- Articulated why coins-outer counts combinations and sums-outer counts permutations in <60 seconds.
- Stated O(K · amount) time and O(amount) space.
- Solved LC 322 unaided in <10 minutes (full progression).
- Solved LC 518 unaided in <10 minutes.
Lab 05 — LIS (Longest Increasing Subsequence)
Goal
Solve LC 300 with two distinct algorithms: the canonical O(N²) DP and the patience-sort + binary-search O(N log N) variant. Internalize why both produce the same answer despite very different mechanics. After this lab you can produce both solutions from a blank screen in <15 minutes total and explain the equivalence on a whiteboard.
Background Concepts
The LIS problem is the canonical example of a problem with two equally-valid algorithmic angles. The O(N²) DP defines dp[i] = length of LIS ending at index i; the O(N log N) algorithm maintains an array tails where tails[k] = smallest tail of any increasing subsequence of length k+1. Both produce the same length; the binary-search version is faster but harder to prove correct.
Patience sorting: imagine dealing cards onto piles such that each pile is strictly decreasing top-to-bottom (place each card on the leftmost pile whose top is ≥ the new card; if none exists, start a new pile). The number of piles equals the LIS length, by Dilworth’s theorem. The tails array tracks the top of each pile.
Interview Context
LIS is a top-20 Medium DP problem and shows up at Google, Bloomberg, and Microsoft regularly. The follow-up “can you do better than O(N²)?” is asked specifically to test whether you know patience sorting. Candidates who know only O(N²) are shipped to L4; candidates who can derive O(N log N) from scratch (or articulate it cleanly) are L5+ material. LIS is also the building block for LC 354 (Russian Doll Envelopes) and LC 673 (Number of LIS).
Problem Statement
Given an integer array nums, return the length of the longest strictly increasing subsequence.
Constraints
- 1 ≤
nums.length≤ 2500 (canonical LeetCode constraint) - −10^4 ≤
nums[i]≤ 10^4
Clarifying Questions
- Strictly increasing or non-decreasing? (Strictly —
nums[i] < nums[j].) - Subsequence or subarray? (Subsequence — non-contiguous selections allowed.)
- Return the length or the actual sequence? (Length only, per problem.)
- Are duplicates handled? (Yes; strict means duplicates can’t both be in the LIS.)
- Is the empty subsequence allowed (length 0)? (Yes, but since
nums.length ≥ 1, the answer is ≥ 1.)
Examples
[10, 9, 2, 5, 3, 7, 101, 18] → 4 ([2, 3, 7, 101] or [2, 5, 7, 101])
[0, 1, 0, 3, 2, 3] → 4 ([0, 1, 2, 3])
[7, 7, 7, 7] → 1
[1] → 1
[5, 4, 3, 2, 1] → 1
Initial Brute Force
For each index, recursively decide include or skip, tracking the previous chosen element to enforce strict-increasing:
def lengthOfLIS_brute(nums):
def f(i, prev):
if i == len(nums):
return 0
skip = f(i + 1, prev)
take = 0
if prev == -1 or nums[i] > nums[prev]:
take = 1 + f(i + 1, i)
return max(skip, take)
return f(0, -1)
Brute Force Complexity
O(2^N) — each step has two choices.
Optimization Path
The state is (i, prev) where prev is the last chosen index (or -1 for none). There are O(N²) such states, so memoization gives O(N²) time and space. Tabulation: define dp[i] = length of LIS ending exactly at index i; recurrence reads only smaller j < i, so we don’t even need the prev dimension — dp[i] = 1 + max(dp[j] for j < i if nums[j] < nums[i]). Final answer is max(dp).
For O(N log N): maintain tails such that tails[k] is the smallest tail of any LIS of length k+1. For each nums[i], binary-search for the first element in tails ≥ nums[i]; if found, replace; if not (i.e., nums[i] exceeds all), append.
Final Expected Approach
O(N²) DP: prefix DP indexed by ending position; recurrence iterates over all earlier indices.
O(N log N) patience sort: maintain tails as an increasing array; bisect_left(tails, nums[i]) gives the position to replace; if equal to len(tails), append.
The equivalence: each tails[k] corresponds to “the smallest endpoint of a length-(k+1) IS we’ve seen”. When we process nums[i], replacing tails[k] with nums[i] represents “we’ve found a length-(k+1) IS with smaller tail” — which can only help future extensions. The length of tails at the end is the LIS length.
Data Structures Used
- 1D
dpof size N (O(N²) version). - 1D
tailsarray (O(N log N) version), Python’sbisectmodule.
Correctness Argument
O(N²): by induction. dp[i] = 1 + max(dp[j] : j < i, nums[j] < nums[i]). Base: dp[0] = 1. Inductive step: any LIS ending at i has a previous element at some j < i with nums[j] < nums[i], contributing dp[j] + 1. The max over all valid j is the optimum. Answer is max_i dp[i].
O(N log N) (Patience sort): invariant — tails[k] is the smallest possible tail of any IS of length k+1 over nums[0..i]. When processing nums[i]: binary-search for the leftmost position k with tails[k] >= nums[i]. If k = len(tails), append (we’ve extended the longest IS by one). Otherwise, replace tails[k] with nums[i] (we’ve found a length-(k+1) IS with smaller tail; future extensions are now easier). The invariant is preserved at every step. The length of tails is the LIS length.
Complexity
| Algorithm | Time | Space |
|---|---|---|
| Brute force | O(2^N) | O(N) |
| Memoized | O(N²) | O(N²) |
| Tabulated O(N²) DP | O(N²) | O(N) |
| Patience sort | O(N log N) | O(N) |
Implementation Requirements
All four stages.
# ---- Stage 1: Brute force ----
def lengthOfLIS_brute(nums):
def f(i, prev):
if i == len(nums):
return 0
skip = f(i + 1, prev)
take = 0
if prev == -1 or nums[i] > nums[prev]:
take = 1 + f(i + 1, i)
return max(skip, take)
return f(0, -1)
# ---- Stage 2: Memoized ----
from functools import lru_cache
def lengthOfLIS_memo(nums):
@lru_cache(None)
def f(i, prev):
if i == len(nums): return 0
skip = f(i + 1, prev)
take = 0
if prev == -1 or nums[i] > nums[prev]:
take = 1 + f(i + 1, i)
return max(skip, take)
return f(0, -1)
# ---- Stage 3: Tabulated O(N^2) ----
def lengthOfLIS_tab(nums):
n = len(nums)
if n == 0: return 0
dp = [1] * n
for i in range(1, n):
for j in range(i):
if nums[j] < nums[i]:
dp[i] = max(dp[i], dp[j] + 1)
return max(dp)
# ---- Stage 4: Patience sort O(N log N) ----
from bisect import bisect_left
def lengthOfLIS(nums):
tails = []
for x in nums:
k = bisect_left(tails, x)
if k == len(tails):
tails.append(x)
else:
tails[k] = x
return len(tails)
Tests
[10, 9, 2, 5, 3, 7, 101, 18]→ 4.[0, 1, 0, 3, 2, 3]→ 4.[7, 7, 7, 7]→ 1.[1]→ 1.[1, 2, 3, 4, 5]→ 5 (already sorted).[5, 4, 3, 2, 1]→ 1 (decreasing).- N=2500 random — performance test for both algorithms.
- Cross-check: random N≤15, the four implementations should agree.
Follow-up Questions
- “Return the actual LIS, not just the length.” → Track parent pointers in the O(N²) DP, or in O(N log N) keep alongside
tailsan arraytails_idxof indices intonumsand parent links. - “Number of distinct LIS’s of maximum length.” (LC 673) → Augment
dp[i]withcnt[i]= number of LIS’s ending ati. - “Longest non-decreasing subsequence.” →
bisect_rightinstead ofbisect_left. - “2D version: stack envelopes (LC 354).” → Sort by width ascending and height descending (to break ties); run LIS on heights.
- “Longest bitonic subsequence.” → Compute LIS forward and LIS backward; combine at each split point.
Product Extension
LIS underlies version-history compression, longest-monotonic-trend analysis in time-series (e.g., longest streak of growing daily users), and dependency-resolution heuristics. The O(N log N) algorithm is what production code uses when N is large.
Language/Runtime Follow-ups
- Python:
bisect_leftis in the standard library and uses C-level binary search — extremely fast. - Java:
Arrays.binarySearch(tails, 0, size, x)returns negative for not-found; convert to insertion point with-(ret + 1). - Go:
sort.SearchInts(tails, x)forbisect_leftequivalent. - C++:
lower_bound(tails.begin(), tails.end(), x)forbisect_left;upper_boundforbisect_right. - JS/TS: no built-in binary search — implement manually or use a third-party
lodash.sortedIndex.
Common Bugs
bisect_rightvsbisect_left— strict-increasing usesbisect_left; non-decreasing usesbisect_right. Off-by-one in this choice silently gives the wrong LIS variant.- Treating
tailsas the actual LIS — it isn’t; it’s just the smallest-tails-by-length array. Reconstructing the LIS requires extra bookkeeping. - O(N²) DP: starting
dp[i] = 0instead of1— every element is itself an LIS of length 1. - Returning
dp[N-1]instead ofmax(dp)— the LIS may end anywhere, not necessarily at the last index. - Memoization on
(i, prev)withprev=-1not recognized as initial state — works in Python with@lru_cachesince -1 is hashable, but easy to forget.
Debugging Strategy
For [10, 9, 2, 5, 3, 7, 101, 18]: trace tails after each element: [10] → [9] → [2] → [2,5] → [2,3] → [2,3,7] → [2,3,7,101] → [2,3,7,18]. Length 4 is the LIS length. If your trace diverges, you’ve made a bisect mistake. For the O(N²) DP, print dp after the loop: [1,1,1,2,2,3,4,4].
Mastery Criteria
- Recognized “longest increasing subsequence” as LIS within 30 seconds.
- Wrote the brute recursion in <2 minutes.
- Wrote the O(N²) DP from blank screen in <4 minutes.
- Wrote the O(N log N) patience-sort version in <5 minutes.
-
Articulated the patience-sort invariant (“
tails[k]is the smallest tail of length-(k+1) IS”) in <30 seconds. - Stated O(N log N) time complexity and explained why binary-search is correct here.
- Solved LC 300 unaided in <12 minutes (both algorithms).
- Solved LC 354 (Russian Doll Envelopes) by reduction to LIS in <15 minutes.
-
Articulated
bisect_leftvsbisect_rightfor strict vs non-strict in <30 seconds.
Lab 06 — LCS / Edit Distance
Goal
Solve Edit Distance (LC 72 — Levenshtein) with the full four-stage progression. Internalize the canonical two-string DP dp[i][j] indexed by prefix lengths, and the three-way min over insert / delete / replace. After this lab you can write any LCS-family DP from a blank screen in <12 minutes and apply the rolling-row collapse to O(M) space.
Background Concepts
Edit distance — sometimes called Levenshtein distance — is the minimum number of single-character edits (insert, delete, replace) needed to transform string A into string B. The state dp[i][j] indexes prefix-i of A and prefix-j of B; the recurrence has one branch per edit operation plus a free pass on character match.
LCS (longest common subsequence) and edit distance are the foundational two-string DPs. They share the index convention (dp[i][j] = answer for prefix-i of A, prefix-j of B), the boundary handling (dp[i][0] = i, dp[0][j] = j), and the rolling-row space optimization (O(N · M) → O(M)). Mastering edit distance gives you LCS for free.
Interview Context
Edit Distance is a top-15 Hard-tagged DP problem at Google, Microsoft, and Amazon. It shows up in coding rounds at staff level routinely, often paired with a follow-up “now reconstruct the alignment”. LCS (LC 1143) is the gentler Medium variant and tests the same skill. Candidates who can derive both recurrences from scratch and articulate the four edit operations precisely demonstrate fluency that translates to nearly every two-string DP problem in the corpus (regex match, distinct subsequences, interleaving strings, longest common substring).
Problem Statement
Given two strings word1 and word2, return the minimum number of operations required to convert word1 into word2. Allowed operations: insert a character, delete a character, replace a character (each cost 1).
Constraints
- 0 ≤
word1.length,word2.length≤ 500 word1andword2consist of lowercase English letters.
Clarifying Questions
- Are insert/delete/replace each cost 1? (Yes — Levenshtein.)
- Are there any other operations (transpose, e.g.)? (No — Damerau-Levenshtein adds transpose; not in scope.)
- Are characters lowercase only? (Yes — given.)
- Are empty strings valid inputs? (Yes; answer is
len(word1) + len(word2)’s difference, specificallymax(len(word1), len(word2))when one is empty.) - Return the count or the alignment? (Count — alignment is a follow-up.)
Examples
word1="horse", word2="ros" → 3 (horse→rorse→rose→ros)
word1="intention", word2="execution" → 5
word1="", word2="abc" → 3 (insert 3)
word1="abc", word2="" → 3 (delete 3)
word1="abc", word2="abc" → 0
Initial Brute Force
def edit_brute(w1, w2):
def f(i, j):
if i == 0: return j # insert remaining w2
if j == 0: return i # delete remaining w1
if w1[i-1] == w2[j-1]:
return f(i-1, j-1) # match: no edit
return 1 + min(
f(i-1, j), # delete w1[i-1]
f(i, j-1), # insert w2[j-1]
f(i-1, j-1), # replace
)
return f(len(w1), len(w2))
Brute Force Complexity
Each non-base call branches into 3; recursion depth N + M. Worst case O(3^(N+M)). At N=M=500, completely infeasible.
Optimization Path
There are (N+1)(M+1) distinct (i, j) pairs — memoization gives O(N · M) time. Tabulation replaces recursion with a row-major loop. Since dp[i][j] depends only on dp[i-1][j-1], dp[i-1][j], dp[i][j-1], the previous row plus the in-progress row are enough — collapse to two 1D arrays of size M+1. With careful use of a saved diagonal, you can collapse to a single 1D array.
Final Expected Approach
dp[i][j] = edit distance between word1[:i] and word2[:j].
dp[0][j] = j (insert j chars to get word2[:j] from empty word1[:0])
dp[i][0] = i (delete i chars from word1[:i] to get empty word2[:0])
dp[i][j] = dp[i-1][j-1] if word1[i-1] == word2[j-1]
= 1 + min(dp[i-1][j-1], dp[i-1][j], dp[i][j-1]) otherwise
The three operations correspond to:
dp[i-1][j-1] + 1— replaceword1[i-1]withword2[j-1].dp[i-1][j] + 1— deleteword1[i-1].dp[i][j-1] + 1— insertword2[j-1].
Data Structures Used
- 2D
dp[(N+1) x (M+1)]array (tabulated). - Two 1D
prev,currarrays of sizeM+1(rolled).
Correctness Argument
By induction on (i, j) in row-major order. Base cases: dp[0][j] = j (clearly j inserts), dp[i][0] = i (clearly i deletes). Inductive step: an optimal alignment of word1[:i] with word2[:j] ends with one of: (a) match — word1[i-1] == word2[j-1] aligned; cost is dp[i-1][j-1]. (b) replace — pair word1[i-1] with word2[j-1]; cost dp[i-1][j-1] + 1. (c) delete — word1[i-1] deleted, word1[:i-1] aligned with word2[:j]; cost dp[i-1][j] + 1. (d) insert — word2[j-1] inserted, word1[:i] aligned with word2[:j-1]; cost dp[i][j-1] + 1. The min of these is the optimum. (a) and (b) are mutually exclusive based on character equality.
Complexity
| Stage | Time | Space |
|---|---|---|
| Brute force | O(3^(N+M)) | O(N+M) |
| Memoized | O(N · M) | O(N · M) |
| Tabulated | O(N · M) | O(N · M) |
| Space-optimized | O(N · M) | O(min(N, M)) |
Implementation Requirements
All four stages.
# ---- Stage 1: Brute force ----
def edit_brute(w1, w2):
def f(i, j):
if i == 0: return j
if j == 0: return i
if w1[i-1] == w2[j-1]: return f(i-1, j-1)
return 1 + min(f(i-1, j), f(i, j-1), f(i-1, j-1))
return f(len(w1), len(w2))
# ---- Stage 2: Memoized ----
from functools import lru_cache
def edit_memo(w1, w2):
@lru_cache(None)
def f(i, j):
if i == 0: return j
if j == 0: return i
if w1[i-1] == w2[j-1]: return f(i-1, j-1)
return 1 + min(f(i-1, j), f(i, j-1), f(i-1, j-1))
return f(len(w1), len(w2))
# ---- Stage 3: Tabulated 2D ----
def edit_tab(w1, w2):
n, m = len(w1), len(w2)
dp = [[0] * (m + 1) for _ in range(n + 1)]
for j in range(m + 1): dp[0][j] = j
for i in range(n + 1): dp[i][0] = i
for i in range(1, n + 1):
for j in range(1, m + 1):
if w1[i-1] == w2[j-1]:
dp[i][j] = dp[i-1][j-1]
else:
dp[i][j] = 1 + min(dp[i-1][j], dp[i][j-1], dp[i-1][j-1])
return dp[n][m]
# ---- Stage 4: Space-optimized O(M) ----
def minDistance(w1, w2):
n, m = len(w1), len(w2)
if n < m:
w1, w2, n, m = w2, w1, m, n # ensure m is the smaller dim
prev = list(range(m + 1))
for i in range(1, n + 1):
curr = [i] + [0] * m
for j in range(1, m + 1):
if w1[i-1] == w2[j-1]:
curr[j] = prev[j-1]
else:
curr[j] = 1 + min(prev[j], curr[j-1], prev[j-1])
prev = curr
return prev[m]
Tests
("horse", "ros")→ 3.("intention", "execution")→ 5.("", "abc")→ 3.("abc", "")→ 3.("abc", "abc")→ 0.("a", "b")→ 1.("a"*500, "b"*500)→ 500 — performance test.- All four implementations equivalent on random N≤8 inputs.
Follow-up Questions
- “Reconstruct the alignment (sequence of operations).” → Backtrack from
dp[n][m]: at each(i, j), look at which of the three (or four) predecessors matches the current value; emit the corresponding operation. - “Custom costs for insert / delete / replace.” → Replace
+1with the appropriate cost in each branch; works without other change. - “Levenshtein with transpositions (Damerau-Levenshtein).” → Add a fourth branch
dp[i-2][j-2] + 1ifword1[i-1]==word2[j-2] and word1[i-2]==word2[j-1]. - “Longest common subsequence.” (LC 1143) → Same shape; recurrence:
dp[i][j] = dp[i-1][j-1] + 1if match, elsemax(dp[i-1][j], dp[i][j-1]). - “Minimum ASCII delete sum.” (LC 712) → Variant where deletes cost ASCII value of the deleted char.
Product Extension
Edit distance is the engine behind diff/patch tools, spell correctors, fuzzy search (“did you mean”), DNA-sequence alignment (Needleman-Wunsch is a generalization with custom scoring matrices), and code-review side-by-side comparison. Real systems use Hirschberg’s algorithm to reconstruct the alignment in O(M) space.
Language/Runtime Follow-ups
- Python:
lru_cacheworks on(i, j)since both are ints. The space-optimized version benefits from swapping to ensurem ≤ n. - Java:
int[][] dpwith explicit boundary fill. For O(M) space, twoint[m+1]arrays. - Go: 2D slice;
make([][]int, n+1)then per-rowmake([]int, m+1). - C++:
vector<vector<int>>2D; or twovector<int>(m+1)for O(M). - JS/TS: 2D array via
Array.from({length: n+1}, () => new Array(m+1).fill(0)).
Common Bugs
- Off-by-one between string index and DP index —
word1[i-1]notword1[i]. The convention “prefix-i” means indexiis exclusive in the prefix, so the latest char isword1[i-1]. - Forgetting the boundary
dp[0][j] = j,dp[i][0] = i— defaults to 0 and produces nonsense answers. - Using
maxinstead ofminin the recurrence — wrong direction. - Including
dp[i-1][j-1] + 1in the match branch — match has no edit cost; should bedp[i-1][j-1]exactly. - Space-optimized version: forgetting
curr[0] = i— the new row’s column 0 isi(deletingichars to match empty prefix), not 0. - Mistakenly thinking insert and delete are symmetric in cost across both strings — they aren’t; insert into
word1is the same as delete fromword2. Levenshtein conflates them in our recurrence which is fine because all costs are 1.
Debugging Strategy
For ("horse", "ros"), the full table is:
"" r o s
"" 0 1 2 3
h 1 1 2 3
o 2 2 1 2
r 3 2 2 2
s 4 3 3 2
e 5 4 4 3
Print and check. If the boundary row/column is wrong, the entire table is. For the rolled version, print prev after each row.
Mastery Criteria
- Recognized “min operations to transform” as edit distance within 60 seconds.
- Wrote the brute recursion with all four cases (match / replace / delete / insert) in <3 minutes.
- Wrote the 2D tabulated version in <5 minutes.
- Performed the rolling-row collapse to two 1D arrays in <3 minutes.
- Stated O(N · M) time and O(M) space.
- Articulated which DP cell corresponds to which edit operation.
- Solved LC 72 unaided in <20 minutes (full progression).
- Solved LC 1143 (LCS) in <8 minutes by changing the recurrence.
- Solved LC 583 (Delete Operation for Two Strings) in <8 minutes (LCS + arithmetic).
Lab 07 — Palindrome DP
Goal
Solve Longest Palindromic Subsequence (LC 516) and Palindrome Partitioning II (LC 132 — minimum cuts) with the four-stage progression. Internalize the length-ascending iteration that makes interval DP correct, and the is_pal[i][j] precompute that powers most palindrome problems. After this lab you can solve any palindrome-DP variant in <12 minutes.
Background Concepts
Palindrome DP problems split into two families:
-
Subsequence palindromes (LC 516):
dp[i][j]= length of longest palindromic subsequence ofs[i..j]. Recurrence: ifs[i] == s[j],dp[i][j] = 2 + dp[i+1][j-1]; elsedp[i][j] = max(dp[i+1][j], dp[i][j-1]). -
Substring palindromes (LC 132, LC 5, LC 647): precompute
is_pal[i][j] = (s[i..j] is a palindrome)via interval DP, then layer the partitioning / counting DP on top.
The shared mechanic: iterate by length ascending, since dp[i][j] depends on intervals strictly shorter. This is the defining feature of interval DP (deeper exploration in Lab 09).
Interview Context
Longest Palindromic Subsequence is a top-30 Medium DP problem; Palindrome Partitioning II is a Hard variant asked at Google and Microsoft. The is_pal precompute is the unlock for ~10 distinct LeetCode problems (5, 131, 132, 516, 647, 1278, 1312, 1771, …). Candidates who can derive the length-ascending loop and articulate the substring-vs-subsequence distinction signal solid interval-DP fluency.
Problem Statement
LC 516 (LPS subsequence): Given a string s, return the length of the longest palindromic subsequence.
LC 132 (min cuts): Given a string s, return the minimum number of cuts needed to partition s into palindromic substrings.
Constraints
- 1 ≤
s.length≤ 1000 (LC 516) / 2000 (LC 132). sis lowercase English.
Clarifying Questions
- Subsequence or substring? (Subsequence for LC 516; substring for LC 132 partitioning.)
- Is a single character a palindrome? (Yes — length 1.)
- Is the empty string a palindrome? (Conventionally yes.)
- LC 132: must each part be non-empty? (Yes.)
- LC 132: 0 cuts means the entire string is a palindrome.
Examples
LC 516:
"bbbab" → 4 ("bbbb")
"cbbd" → 2 ("bb")
"a" → 1
LC 132:
"aab" → 1 ("aa" | "b")
"a" → 0
"ab" → 1
"aabb" → 1 ("aa" | "bb")
"abcbm" → 2
"abc" → 2
Initial Brute Force (LC 516)
def lps_brute(s):
def f(i, j):
if i > j: return 0
if i == j: return 1
if s[i] == s[j]: return 2 + f(i+1, j-1)
return max(f(i+1, j), f(i, j-1))
return f(0, len(s) - 1)
Brute Force Complexity
O(2^N) worst case — every mismatch branches. At N=1000, infeasible.
Optimization Path
(i, j) has only O(N²) distinct values, so memoization is O(N²) time and space. Tabulation: iterate length = 1..N, then i = 0..N-length, then derive j = i + length - 1. The length-ascending order ensures all shorter intervals are computed first.
LC 132 strategy: precompute is_pal[i][j] in O(N²). Then cuts[i] = min cuts for s[:i+1]; cuts[i] = -1 if s[:i+1] is itself a palindrome; else cuts[i] = min(cuts[j-1] + 1 : 0 ≤ j ≤ i, s[j..i] palindrome).
Final Expected Approach
LC 516:
dp[i][j] = LPS of s[i..j]
dp[i][i] = 1; dp[i][j] = 0 for i > j
For length 2..N:
For i in 0..N-length:
j = i + length - 1
if s[i] == s[j]:
dp[i][j] = (2 if length == 2 else 2 + dp[i+1][j-1])
else:
dp[i][j] = max(dp[i+1][j], dp[i][j-1])
Answer: dp[0][N-1]
LC 132:
1. Compute is_pal[i][j] (O(N^2)).
2. cuts[i] = min cuts to partition s[0..i].
cuts[i] = 0 if is_pal[0][i].
else cuts[i] = min(cuts[j-1] + 1 : 1 <= j <= i, is_pal[j][i]).
Answer: cuts[N-1].
Data Structures Used
- 2D
dp[N][N](LC 516). - 2D
is_pal[N][N]boolean + 1Dcuts[N](LC 132).
Correctness Argument
LC 516: by induction on length. Base: length 1 → 1. Inductive: an LPS of s[i..j] either uses both endpoints (must be equal, contributing 2 + LPS of s[i+1..j-1]) or skips at least one endpoint (LPS of s[i+1..j] or s[i..j-1]). The max covers all cases.
LC 132: every valid partition has a last cut at some position j, splitting into s[0..j-1] + s[j..i] where s[j..i] is a palindrome. The minimum is over all valid j. This exhausts all partitions.
Complexity
| Problem | Time | Space |
|---|---|---|
| LC 516 brute | O(2^N) | O(N) |
| LC 516 memo | O(N²) | O(N²) |
| LC 516 tab | O(N²) | O(N²) |
| LC 516 space-opt | O(N²) | O(N) |
| LC 132 | O(N²) | O(N²) |
Implementation Requirements
# ==== LC 516: Longest Palindromic Subsequence ====
# ---- Stage 1: Brute force ----
def lps_brute(s):
def f(i, j):
if i > j: return 0
if i == j: return 1
if s[i] == s[j]: return 2 + f(i+1, j-1)
return max(f(i+1, j), f(i, j-1))
return f(0, len(s) - 1)
# ---- Stage 2: Memoized ----
from functools import lru_cache
def lps_memo(s):
@lru_cache(None)
def f(i, j):
if i > j: return 0
if i == j: return 1
if s[i] == s[j]: return 2 + f(i+1, j-1)
return max(f(i+1, j), f(i, j-1))
return f(0, len(s) - 1)
# ---- Stage 3: Tabulated 2D ----
def lps_tab(s):
n = len(s)
dp = [[0] * n for _ in range(n)]
for i in range(n): dp[i][i] = 1
for length in range(2, n + 1):
for i in range(n - length + 1):
j = i + length - 1
if s[i] == s[j]:
dp[i][j] = 2 if length == 2 else 2 + dp[i+1][j-1]
else:
dp[i][j] = max(dp[i+1][j], dp[i][j-1])
return dp[0][n-1]
# ---- Stage 4: Space-optimized 1D ----
def longestPalindromeSubseq(s):
n = len(s)
dp = [0] * n
for i in range(n - 1, -1, -1):
new = [0] * n
new[i] = 1
for j in range(i + 1, n):
if s[i] == s[j]:
new[j] = 2 + (dp[j-1] if j-1 >= i+1 else 0)
else:
new[j] = max(dp[j], new[j-1])
dp = new
return dp[n-1]
# ==== LC 132: Palindrome Partitioning II ====
def minCut(s):
n = len(s)
# Step 1: precompute is_pal in O(N^2)
is_pal = [[False] * n for _ in range(n)]
for i in range(n): is_pal[i][i] = True
for length in range(2, n + 1):
for i in range(n - length + 1):
j = i + length - 1
if s[i] == s[j] and (length == 2 or is_pal[i+1][j-1]):
is_pal[i][j] = True
# Step 2: cuts DP
cuts = [0] * n
for i in range(n):
if is_pal[0][i]:
cuts[i] = 0
continue
cuts[i] = i # worst case: cut after every character
for j in range(1, i + 1):
if is_pal[j][i]:
cuts[i] = min(cuts[i], cuts[j-1] + 1)
return cuts[n-1]
Tests
- LC 516:
"bbbab"→ 4."cbbd"→ 2."a"→ 1."ac"→ 1."aaaa"→ 4. - LC 132:
"aab"→ 1."a"→ 0."ab"→ 1."aabb"→ 1."abcbm"→ 2."abacabaca"→ 0 (already a palindrome? no: check) → actually"abacaba"is, but"abacabaca"is not — answer 1. - Cross-implementation check on random N≤10.
Follow-up Questions
- “Count palindromic substrings.” (LC 647) → Sum
is_pal[i][j]over all(i, j)withi ≤ j. - “Longest palindromic substring.” (LC 5) → Return the length / actual string of the largest
(j - i + 1)withis_pal[i][j]. - “Minimum insertions to make palindrome.” (LC 1312) →
len(s) - LPS(s). - “All palindrome partitions.” (LC 131) → Backtracking; output exponentially many.
- “Palindromes with one allowed mismatch.” → Add a dimension
dp[i][j][k]wherek∈ {0, 1}.
Product Extension
Palindrome detection appears in DNA-sequence analysis (palindromic motifs are biologically meaningful: restriction sites, hairpins), text-search systems, and as a non-trivial benchmark for compiler optimization. The is_pal precompute is also useful in interview problems that don’t strictly need DP (just O(N²) precomputation).
Language/Runtime Follow-ups
- Python: 2D arrays via list comprehensions;
lru_cachefor memoization. - Java:
boolean[][] isPal = new boolean[n][n];defaults to false.int[][] dp = new int[n][n];. - Go: pre-allocate slice-of-slices; can fuse
is_palandcutscomputation in a single function. - C++:
vector<vector<bool>> is_pal(n, vector<bool>(n, false));. - JS/TS: as Python; watch for shared-reference trap.
Common Bugs
- Iterating
i, jin row-major for LPS — fails becausedp[i][j]depends ondp[i+1][j-1], which hasn’t been computed yet. Must iterate by length. - Forgetting
dp[i][i] = 1— base case for single-char palindromes. - Edge case
length == 2—dp[i+1][j-1]isdp[i+1][i], an invalid range. Special-case to 0 or just use2. - LC 132: forgetting the
is_pal[0][i]short-circuit — gives wrong answer for already-palindromic input. - LC 132: cuts initialization — initialize to
i(worst case: cut after every character ofs[0..i]). - Confusing subsequence and substring — LC 516 wants subsequence; many candidates accidentally solve substring (which is LC 5, harder).
Debugging Strategy
For LC 516 "bbbab": trace the table by length. Length 1: diagonal all 1. Length 2: dp[0][1]=2 (bb), dp[1][2]=2, dp[2][3]=1, dp[3][4]=1. Length 3: dp[0][2]=3 (bbb). Length 4: dp[0][3]=3, dp[1][4]=3. Length 5: dp[0][4]=4 (bbbb). For LC 132 "aab": is_pal = [[T,T,F],[F,T,F],[F,F,T]]; cuts = [0, 0, 1].
Mastery Criteria
- Recognized “longest palindromic subsequence” as interval DP within 60 seconds.
- Articulated the length-ascending iteration order in <30 seconds.
- Wrote LC 516 brute recursion in <2 minutes.
- Wrote LC 516 tabulated in <5 minutes.
-
Wrote
is_palprecompute correctly in <4 minutes. -
Wrote LC 132 cuts DP using
is_palin <5 minutes. - Stated O(N²) time and space.
- Solved LC 516 unaided in <12 minutes (full progression).
- Solved LC 132 unaided in <15 minutes.
-
Solved LC 5 (longest palindromic substring) in <8 minutes via
is_pal.
Lab 08 — Tree DP (House Robber III)
Goal
Solve House Robber III (LC 337) with post-order DFS returning (rob, skip) per node. Internalize the post-order DP pattern where each node returns a tuple of “best with this node included” and “best with this node excluded”. After this lab you recognize tree DP within 60 seconds and can write any post-order tuple-DP from blank screen in <8 minutes.
A note on the four-stage progression: tree DP doesn’t have a clean tabulated form (there’s no natural row-major order for a tree), and “space optimization” is implicit (each post-order call returns a constant-size tuple, so the working memory is O(1) per node). Stages we can show: brute recursion, memoized recursion, post-order DFS with tuple return (the canonical form), and an iterative post-order using an explicit stack. The tuple-return version is what you write in interviews.
Background Concepts
Tree DP: state is per node; recurrence aggregates children’s states. The natural evaluation order is post-order — process all descendants before the node itself. Most tree DPs return a tuple (or struct) per node carrying the answers for “this node included” vs “this node excluded” (or whatever the binary split is). The parent combines children’s tuples in O(1) per child, giving O(N) total.
House Robber III is the canonical example. Each node v returns (rob_v, skip_v):
rob_v = val[v] + sum(skip_c for c in children(v))— robv, must skip every child.skip_v = sum(max(rob_c, skip_c) for c in children(v))— skipv, each child is independently best.
Final answer: max(rob_root, skip_root).
Interview Context
LC 337 is a top-30 Medium DP problem at Amazon and Microsoft. The post-order tuple pattern recurs in: LC 124 (Binary Tree Maximum Path Sum), LC 543 (Diameter of Binary Tree), LC 968 (Binary Tree Cameras), LC 1372 (Longest ZigZag Path). Mastering it here generalizes broadly. Senior interviewers specifically test whether you write the tuple-return version (clean, O(N)) versus the memoized-but-redundant version that recursively descends three times per call.
Problem Statement
A binary tree where each node holds an integer amount of money. The thief cannot rob two directly-linked houses (parent–child). Return the maximum amount the thief can rob without alerting the police.
Constraints
- 1 ≤ tree size ≤ 10^4
- 0 ≤
node.val≤ 10^4
Clarifying Questions
- Is the tree binary or general? (Binary, per LC 337.)
- Are values non-negative? (Yes — given.)
- What does the tree representation look like? (Standard
TreeNodewithleft,right.) - Can the tree be empty? (Yes — return 0.)
- Does “linked” mean parent–child only or also siblings? (Parent–child only.)
Examples
3
/ \
2 3
\ \
3 1 → 7 (rob 3 + 3 + 1)
3
/ \
4 5
/ \ \
1 3 1 → 9 (rob 4 + 5)
Initial Brute Force
For each subtree rooted at v: try rob-v (must skip children, recurse on grandchildren) or skip-v (recurse on children).
def rob_brute(root):
def f(v):
if v is None: return 0
# Skip v
skip = f(v.left) + f(v.right)
# Rob v: must skip both children
rob = v.val
if v.left: rob += f(v.left.left) + f(v.left.right)
if v.right: rob += f(v.right.left) + f(v.right.right)
return max(rob, skip)
return f(root)
Brute Force Complexity
Each call recurses on children and on grandchildren — the same node is visited multiple times via different paths. Worst case O(N · 2^depth). Memoization on the node identity collapses to O(N), but cleaner is to return the tuple in a single post-order traversal.
Optimization Path
The brute force descends three times per node (for the rob branch) and two for skip. With memoization on the node, every subtree is computed twice (once as a “child” call, once as a “grandchild” call). Use a dict keyed by id(node) or, much cleaner, return both values in one tuple per node — a single post-order pass with no memoization needed.
Final Expected Approach
Post-order DFS returning (rob, skip):
def f(v):
if v is None: return (0, 0)
lr, ls = f(v.left)
rr, rs = f(v.right)
rob_v = v.val + ls + rs
skip_v = max(lr, ls) + max(rr, rs)
return (rob_v, skip_v)
answer = max(f(root))
Time O(N) (each node visited once). Space O(H) for the call stack (H = tree height; O(N) worst case for skewed trees, O(log N) average).
Data Structures Used
- Binary tree (input).
- Recursion stack.
- For brute / memo: optional
dictkeyed by node identity.
Correctness Argument
By structural induction on the tree. Base: empty tree → (0, 0). Inductive: assume f(v.left) and f(v.right) correctly return (rob, skip) for those subtrees. Then:
rob_v= robvand skip both children. Since the children must be skipped (parent-child constraint), the contribution from each child subtree isskip_child. Plusv.val.skip_v= skipv, each child subtree independently maximized:max(rob_child, skip_child).
The max of the two is the overall optimum, but we return both (not the max) because the parent of v needs skip_v distinct from rob_v. The final answer at the root is max(rob_root, skip_root).
Complexity
| Stage | Time | Space |
|---|---|---|
| Brute force | O(N · 2^H) worst | O(H) |
| Memoized (node-keyed) | O(N) | O(N) |
| Tuple-return post-order | O(N) | O(H) |
| Iterative post-order | O(N) | O(N) (explicit stack) |
Implementation Requirements
class TreeNode:
def __init__(self, val=0, left=None, right=None):
self.val, self.left, self.right = val, left, right
# ---- Stage 1: Brute force (recompute via grandchildren) ----
def rob_brute(root):
def f(v):
if v is None: return 0
skip = f(v.left) + f(v.right)
rob = v.val
if v.left: rob += f(v.left.left) + f(v.left.right)
if v.right: rob += f(v.right.left) + f(v.right.right)
return max(rob, skip)
return f(root)
# ---- Stage 2: Memoized on node identity ----
def rob_memo(root):
memo = {}
def f(v):
if v is None: return 0
if id(v) in memo: return memo[id(v)]
skip = f(v.left) + f(v.right)
rob = v.val
if v.left: rob += f(v.left.left) + f(v.left.right)
if v.right: rob += f(v.right.left) + f(v.right.right)
memo[id(v)] = max(rob, skip)
return memo[id(v)]
return f(root)
# ---- Stage 3: Tuple-return post-order (canonical) ----
def rob(root):
def f(v):
if v is None: return (0, 0)
lr, ls = f(v.left)
rr, rs = f(v.right)
rob_v = v.val + ls + rs
skip_v = max(lr, ls) + max(rr, rs)
return (rob_v, skip_v)
return max(f(root))
# ---- Stage 4: Iterative post-order with explicit stack ----
def rob_iter(root):
if root is None: return 0
stack, order = [root], []
while stack:
v = stack.pop()
order.append(v)
if v.left: stack.append(v.left)
if v.right: stack.append(v.right)
# order is now reverse post-order; iterate reversed for true post-order
state = {} # id(v) -> (rob, skip)
for v in reversed(order):
lr, ls = state.get(id(v.left), (0, 0)) if v.left else (0, 0)
rr, rs = state.get(id(v.right), (0, 0)) if v.right else (0, 0)
state[id(v)] = (v.val + ls + rs, max(lr, ls) + max(rr, rs))
return max(state[id(root)])
Tests
Build trees from level-order input:
[3,2,3,null,3,null,1]→ 7.[3,4,5,1,3,null,1]→ 9.[1]→ 1.[]→ 0 (empty tree).- Linear left-skewed chain
1→2→3→4→5→ 9 (rob 1, 3, 5). - Random N=1000 tree — performance test.
- Cross-check all four implementations on random trees of size N≤8.
Follow-up Questions
- “Reconstruct which nodes were robbed.” → Augment the tuple to also return the set of robbed nodes (or backtrack from the root after the post-order pass).
- “K-ary tree.” → Same idea; sum over all children.
- “Constraint relaxes to: cannot rob two nodes within distance K.” → Augment state to track distance to last robbed node; state grows by factor K.
- “Negative values allowed.” → Same recurrence; max correctly handles.
- “Maximum path sum of an arbitrary path (LC 124).” → Different but related: post-order returns “best one-sided path from this node”. Combine left + right + node at the root for the best path through it.
Product Extension
Tree DP underlies code-review priority computation in commit-trees, expression-tree evaluation in compilers, hierarchical resource allocation (org charts, file systems), and game-tree value computation. The post-order-with-tuple pattern is a workhorse.
Language/Runtime Follow-ups
- Python: tuple unpacking is idiomatic. Default recursion limit (1000) overflows on deep skewed trees; bump with
sys.setrecursionlimit. - Java: define a small inner
int[]of size 2 or aPair<Integer, Integer>(or just useint[]). Recursion depth is O(H); JVM stack default 512KB, OK for N≤10000. - Go: return
(int, int)directly via multiple-return. - C++:
pair<int,int>returned by value. - JS/TS: return
[rob, skip]array.
Common Bugs
- Returning only
max(rob_v, skip_v)instead of the tuple — the parent then can’t distinguish the two cases and the recurrence is wrong. - Computing
rob_vasv.val + skip_v— wrong, becauseskip_valready includesmax(rob_child, skip_child)not justskip_child. - Forgetting
v is Nonebase case — null pointer /AttributeError. - Confusing which children’s value to sum:
rob_vsumsls + rs(skip both children);skip_vsumsmax(lr, ls) + max(rr, rs). - Iterative version: traversing in pre-order and reversing — works for binary trees because reverse pre-order with right-before-left equals post-order. Easy to flip and break.
- Using
@lru_cacheon the functionf(v)directly —TreeNodeis unhashable by default; useid(v)or define__hash__.
Debugging Strategy
For [3,2,3,null,3,null,1]:
- Leaves:
f(3-leaf-left) = (3, 0).f(1-leaf) = (1, 0).f(3-leaf-mid) = (3, 0). - Mid-left node (val 2, child 3):
f(2-mid) = (2 + 0, 3) = (2, 3). - Mid-right (val 3, right child 1):
f(3-right) = (3 + 0, 1) = (3, 1). - Root (val 3):
f(root) = (3 + 3 + 1, max(2,3) + max(3,1)) = (7, 6). Answer: 7. ✓
If your tuple values diverge, print (rob, skip) per node in post-order and locate the first inconsistency.
Mastery Criteria
- Recognized this as tree DP within 60 seconds.
-
Articulated the
(rob, skip)tuple invariant in <30 seconds. - Wrote the tuple-return post-order in <5 minutes from blank screen.
- Stated O(N) time and O(H) space.
- Articulated why the tuple is necessary (parent needs both rob_child and skip_child) in <30 seconds.
- Solved LC 337 unaided in <8 minutes.
- Solved LC 124 (Binary Tree Max Path Sum) in <12 minutes using the same post-order pattern.
- Solved LC 543 (Diameter) in <8 minutes.
- Identified the brute-force redundancy (descend twice via children + grandchildren) without prompting.
Lab 09 — Interval DP (Burst Balloons)
Goal
Solve Burst Balloons (LC 312) with the four-stage progression. Internalize the “think backwards” trick: instead of asking “which balloon to burst first?” (which fragments the array), ask “which balloon is burst last in the interval (i, j)?” — that balloon’s left and right neighbors are guaranteed to be nums[i] and nums[j] (the surviving boundary). After this lab you can identify and solve interval DP problems in <15 minutes.
Background Concepts
Interval DP: state dp[i][j] is the answer over the subarray (or substring) from index i to j. Recurrence iterates over a “split point” k in (i, j) and combines two sub-intervals. Defining feature: iterate by interval length ascending, so all shorter intervals are computed before they’re needed.
Burst Balloons is famous because the naive “first to burst” formulation creates non-contiguous subproblems. The “last to burst” reformulation (think backwards) restores contiguity: in interval (i, j), if k is the last balloon to burst, then by the time we burst it, all of (i, k) and (k, j) have already been burst, and k’s left and right neighbors are exactly nums[i] and nums[j] (the original boundary).
Interview Context
Burst Balloons is a top-Hard interval DP problem at Google and Microsoft. It is the problem that teaches the “think backwards” trick. Candidates who fail to recognize the contiguity issue with the forward formulation (and then naively try memoization on subsets — which is 2^N states) get stuck. Senior interviewers love this problem precisely because the reformulation is non-obvious and tests insight, not memorization.
Other interval DP problems: Matrix Chain Multiplication (LC 1547), Stone Game (LC 877), Strange Printer (LC 664), Remove Boxes (LC 546).
Problem Statement
You are given n balloons indexed 0 to n−1, each with a number nums[i] painted on it. You are asked to burst all the balloons. If you burst balloon i, you get nums[i-1] * nums[i] * nums[i+1] coins (use 1 if neighbor is out of bounds). After bursting, the neighbors become adjacent. Return the maximum coins you can collect.
Constraints
- N == nums.length
- 1 ≤ N ≤ 500
- 0 ≤ nums[i] ≤ 100
Clarifying Questions
- Are values non-negative? (Yes — given.)
- After bursting, do neighbors become adjacent? (Yes — that’s the rule.)
- What if a neighbor is out of bounds (edge balloon)? (Treat as 1.)
- Must we burst all balloons? (Yes — the question asks max coins from bursting all.)
- Are zero values allowed? (Yes — bursting a zero-balloon gives 0 coins.)
Examples
nums = [3, 1, 5, 8] → 167
Burst order: 1, 5, 3, 8.
3 * 1 * 5 = 15; 3 * 5 * 8 = 120; 1 * 3 * 8 = 24; 1 * 8 * 1 = 8; total 167.
nums = [1, 5] → 10 (burst 5 → 1*5*1=5; burst 1 → 1*1*1=1; total 6.
Better: burst 1 first → 1*1*5=5; burst 5 → 1*5*1=5; total 10.)
nums = [9] → 9
nums = [1] → 1
Initial Brute Force
For each balloon, try bursting it first; recurse on the two halves. Note: this naive form has a fundamental contiguity issue — after bursting k first, the left and right halves’ boundary values change depending on which balloons remain, so the subproblems aren’t independent. We can still write it but it requires passing the current array (or the active set) down.
def burst_brute(nums):
arr = [1] + nums + [1]
def f(active):
if not active: return 0
best = 0
for idx in range(len(active)):
k = active[idx]
left = active[idx-1] if idx > 0 else 0
right = active[idx+1] if idx+1 < len(active) else len(arr)-1
gain = arr[left] * arr[k] * arr[right]
best = max(best, gain + f(active[:idx] + active[idx+1:]))
return best
return f(list(range(1, len(arr) - 1)))
Brute Force Complexity
O(N!) — every permutation of bursts. At N=500, completely infeasible.
Optimization Path
The “first to burst” formulation cannot be memoized cleanly on (i, j) because the subproblems depend on what’s outside (i, j). Reframe: ask “in the final interval (i, j), which balloon k is burst last?”. By the time k is burst, all balloons in (i, k) and (k, j) have been burst — independently of each other. k’s neighbors at that moment are nums[i] and nums[j] (the surviving boundary). The recurrence becomes:
dp[i][j] = max over k in (i, j) of:
dp[i][k] + nums[i] * nums[k] * nums[j] + dp[k][j]
Pad nums with 1 at both ends to handle out-of-bounds neighbors uniformly. Iterate by interval length.
Final Expected Approach
1. arr = [1] + nums + [1] (length N+2)
2. dp[i][j] = max coins from bursting all balloons strictly between i and j (boundaries i, j untouched)
3. dp[i][i+1] = 0 (no balloons between i and i+1)
4. For length 2..N+1:
For i in 0..N+1-length:
j = i + length
dp[i][j] = max over k in (i, j) of dp[i][k] + arr[i]*arr[k]*arr[j] + dp[k][j]
5. Answer: dp[0][N+1]
Data Structures Used
- 2D
dp[N+2][N+2]. - Padded array
arrof sizeN+2.
Correctness Argument
Claim: dp[i][j] = max coins from bursting all balloons in (i, j) (exclusive), assuming balloons at indices i and j are still present. Proof by induction on length.
Base: length 1 → (i, i+1) has no balloons strictly between → 0.
Inductive step: any optimal bursting order has a last balloon k in (i, j). When k is burst, all other balloons in (i, k) and (k, j) have already been burst, and k’s neighbors are arr[i] and arr[j] (because k is the last to go). The two subintervals (i, k) and (k, j) are independent — neither affects the other since they’re separated by k itself, which is alive until the end. So:
dp[i][k]= max coins from bursting(i, k)(boundariesi,kalive).dp[k][j]= max coins from bursting(k, j)(boundariesk,jalive).arr[i] * arr[k] * arr[j]= coins from burstingklast with neighborsi,j.
Sum and maximize over k. The induction works because both subintervals are strictly shorter than (i, j).
Complexity
| Stage | Time | Space |
|---|---|---|
| Brute force | O(N!) | O(N) |
| Memoized | O(N³) | O(N²) |
| Tabulated | O(N³) | O(N²) |
| Space-optimized | (no further reduction; subproblems span all of (i, j)) | O(N²) |
At N=500, N³ = 1.25 × 10^8 — close to the edge but passes.
Implementation Requirements
# ---- Stage 1: Brute force ----
def burst_brute(nums):
arr = [1] + nums + [1]
def f(active):
if not active: return 0
best = 0
for idx in range(len(active)):
k = active[idx]
left = active[idx-1] if idx > 0 else 0
right = active[idx+1] if idx+1 < len(active) else len(arr) - 1
gain = arr[left] * arr[k] * arr[right]
best = max(best, gain + f(active[:idx] + active[idx+1:]))
return best
return f(list(range(1, len(arr) - 1)))
# ---- Stage 2: Memoized (think-backwards reformulation) ----
from functools import lru_cache
def burst_memo(nums):
arr = [1] + nums + [1]
@lru_cache(None)
def f(i, j):
if j - i < 2: return 0
return max(
f(i, k) + arr[i] * arr[k] * arr[j] + f(k, j)
for k in range(i + 1, j)
)
return f(0, len(arr) - 1)
# ---- Stage 3: Tabulated 2D ----
def maxCoins(nums):
arr = [1] + nums + [1]
n = len(arr)
dp = [[0] * n for _ in range(n)]
for length in range(2, n):
for i in range(n - length):
j = i + length
best = 0
for k in range(i + 1, j):
cand = dp[i][k] + arr[i] * arr[k] * arr[j] + dp[k][j]
if cand > best: best = cand
dp[i][j] = best
return dp[0][n-1]
# ---- Stage 4: No further space optimization (full 2D needed); presented as a tighter inner loop ----
def maxCoins_tight(nums):
arr = [1] + nums + [1]
n = len(arr)
dp = [[0] * n for _ in range(n)]
for length in range(2, n):
for i in range(n - length):
j = i + length
ai, aj = arr[i], arr[j]
best = 0
for k in range(i + 1, j):
cand = dp[i][k] + ai * arr[k] * aj + dp[k][j]
if cand > best: best = cand
dp[i][j] = best
return dp[0][n-1]
Tests
[3, 1, 5, 8]→ 167.[1, 5]→ 10.[9]→ 9.[1]→ 1.[]→ 0.[1, 1, 1]→ 3.- N=100 random — performance smoke test.
- Cross-check brute vs memo on N≤7.
Follow-up Questions
- “Find the burst order.” → Track the
argmax kin eachdp[i][j]; recursively reconstruct. - “Each balloon has a different gain function (not multiplicative).” → Same DP shape, plug in any commutative-on-boundaries function.
- “Can we burst at most M balloons?” → Add a 3rd dimension
dp[i][j][m]. - “Stones Game family (LC 877).” → Interval DP with two players;
dp[i][j]= max-score-difference. - “Matrix Chain Multiplication.” → Same shape:
dp[i][j] = min over k of dp[i][k] + dp[k+1][j] + cost(i,k,j).
Product Extension
Interval DP underlies optimal binary search tree construction, optimal parenthesization for matrix chains in linear algebra libraries, and pricing problems in algorithmic finance (“when to buy/sell a contract that opens an interval”). The “think backwards / last to act” reframing recurs in algorithmic game theory and contract design.
Language/Runtime Follow-ups
- Python: at N=500, the inner triple-loop is
1.25 × 10^8iterations; Python may TLE. PyPy or rewriting the inner loop as amax(...)generator helps. - Java/Go/C++: no concern at this size.
- Memoization in Python:
lru_cache(None)is fine; works with(i, j)int tuples. - Iterative version: triple-nested loop is most efficient; avoid generator overhead.
Common Bugs
- Trying “first to burst” recurrence and memoizing on
(i, j)— incorrect because the subproblems’ boundaries change as outer balloons are burst. - Forgetting to pad with 1 at both ends — out-of-bounds neighbors then need special-casing in every loop iteration.
- Iterating by
i, jrow-major —dp[i][k]fork > j(which we never compute) is never read, butdp[k][j]fork < jis; the row-major order computesdp[i][k]beforedp[i][j]only sometimes. Iterate by length. - Off-by-one in
range(i + 1, j)—kmust be strictly betweeniandj. Easy to writerange(i, j)orrange(i + 1, j + 1)and break the recurrence. - Initializing
best = -1— wrong because all values are non-negative anddp[i][j] = 0for empty intervals is correct.
Debugging Strategy
For [3, 1, 5, 8] (padded to [1, 3, 1, 5, 8, 1]):
- Length 2 (no balloons strictly between): all
dp = 0. - Length 3: e.g.,
dp[0][2]=arr[0]*arr[1]*arr[2]=1*3*1=3.dp[1][3] = 3*1*5 = 15.dp[2][4] = 1*5*8 = 40.dp[3][5] = 5*8*1 = 40. - Length 4: e.g.,
dp[0][3]= max over k=1,2 ofdp[0][k] + 1*arr[k]*5 + dp[k][3]= max(0 + 1*3*5 + 15,3 + 1*1*5 + 0) = max(30, 8) = 30. - Continue up to
dp[0][5] = 167.
Print dp row by row and verify against the trace.
Mastery Criteria
- Recognized the contiguity issue with “first to burst” within 90 seconds.
- Articulated the “think backwards / last to burst” reformulation in <60 seconds.
- Wrote the corrected recurrence on a whiteboard in <2 minutes.
- Wrote the tabulated O(N³) solution in <8 minutes from blank screen.
-
Padded
numswith sentinels correctly without prompting. - Iterated by interval length without prompting.
- Stated O(N³) time and O(N²) space.
- Solved LC 312 unaided in <20 minutes.
- Solved Matrix Chain Multiplication in <12 minutes via the same template.
Lab 10 — Bitmask DP (Shortest Path Visiting All Nodes)
Goal
Solve Shortest Path Visiting All Nodes (LC 847) with both BFS over (node, mask) states and an iterative DP variant. Internalize bitmask state encoding: when N is small (≤ 20), the subset S ⊆ {0, …, N-1} fits in a single integer mask and a 1D array of size 2^N indexes all subsets. After this lab you can handle any “visit all / cover all / select subset” problem with N ≤ 20 in <15 minutes.
Background Concepts
Bitmask DP encodes a subset as an integer’s bits. For N=12, there are 2^12 = 4096 subsets — small enough that dp[mask][i] (mask × last-visited-node) has 4096 × 12 ≈ 50K states. Each state’s transition is O(N), giving O(N² · 2^N) total — feasible for N ≤ 16, manageable for N ≤ 20 with care.
Common bitmask DP patterns:
- Visit-all / TSP-like:
dp[mask][i]= min cost to visit nodes inmaskending ati. Final: min overiofdp[(1<<N)-1][i](+ return-to-start cost for TSP). - Subset-cover:
dp[mask]= best score selecting items whose indicator ismask. - Assignment problems: assign N people to N tasks with min total cost —
dp[mask]wheremask= set of tasks already assigned, withpopcount(mask)people processed so far.
LC 847 is unusual: it’s a shortest unweighted path problem (BFS), not a min-cost (Dijkstra) problem. So BFS over (node, mask) is the natural approach. We can also frame it as DP, but BFS is cleaner here.
Interview Context
Shortest Path Visiting All Nodes (LC 847) is a top-Hard graph + bitmask problem at Google and Meta. The bitmask-on-graph technique recurs in: LC 943 (Find the Shortest Superstring), LC 1125 (Smallest Sufficient Team), LC 1349 (Maximum Students Taking Exam), LC 526 (Beautiful Arrangement). The trick of recognizing N ≤ 20 → bitmask is itself an interview heuristic: any time N is suspiciously small, consider bitmask.
Problem Statement
You have an undirected, connected graph of n nodes labeled from 0 to n − 1. graph[i] is the list of neighbors of node i. Return the length of the shortest path that visits every node. You may start and stop at any node, may revisit nodes, and may reuse edges.
Constraints
- 1 ≤ n ≤ 12
graph.length== n- 0 ≤
graph[i].length< n - The graph is connected.
Clarifying Questions
- Edges are undirected? (Yes — given.)
- Edges weighted or unweighted? (Unweighted — count edges traversed.)
- Can the same node be visited multiple times? (Yes.)
- Can we start anywhere? (Yes — the answer minimizes over all starting nodes.)
- Must the graph be connected? (Yes — given. Otherwise the answer is infeasible.)
Examples
graph = [[1,2,3],[0],[0],[0]] → 4 (visit order: 1→0→2→0→3 or 2→0→1→0→3 etc.)
graph = [[1],[0,2,4],[1,3,4],[2],[1,2]] → 4
graph = [[1],[0]] → 1 (just edge 0–1)
graph = [[]] → 0 (single node, already visited)
Initial Brute Force
Try every permutation of node visits as a starting path; sum the shortest-path-lengths between consecutive nodes (precomputed via BFS). N! permutations × O(N) work per permutation. At N=12, 12! ≈ 5 × 10^8 — borderline.
from itertools import permutations
from collections import deque
def shortestPathLength_brute(graph):
n = len(graph)
if n == 1: return 0
# Precompute pairwise shortest path lengths via BFS
dist = [[float('inf')] * n for _ in range(n)]
for src in range(n):
dist[src][src] = 0
q = deque([src])
while q:
u = q.popleft()
for v in graph[u]:
if dist[src][v] == float('inf'):
dist[src][v] = dist[src][u] + 1
q.append(v)
best = float('inf')
for perm in permutations(range(n)):
cost = sum(dist[perm[i]][perm[i+1]] for i in range(n - 1))
best = min(best, cost)
return best
Brute Force Complexity
O(N! · N) time. At N=12, infeasible.
Optimization Path
Observation: at any point, the relevant state is (current_node, set_of_visited_nodes). There are N · 2^N such states. Two ways to solve:
-
BFS over states (canonical for unweighted): each state is a node in a meta-graph of
(node, mask); transitions to(neighbor, mask | (1 << neighbor)). BFS gives shortest-path lengths to all states; the answer is the smallest distance to any state withmask = (1 << N) - 1. TimeO(N · 2^N · degree)≈O(N² · 2^N). -
Iterative DP (Held-Karp style for TSP — but TSP minimizes weighted paths; for unweighted with revisits the BFS is more natural).
We present BFS as the primary solution (canonical for LC 847) with the iterative-DP variant as a follow-up.
Final Expected Approach
1. Initialize a queue with all (node, 1 << node) states (one per starting node), distance 0.
2. BFS:
- Pop (u, mask). If mask == ALL = (1<<N)-1, return current distance.
- For each neighbor v of u:
new_mask = mask | (1 << v)
if (v, new_mask) not yet visited:
mark, enqueue, distance + 1.
Data Structures Used
dequefor BFS.- 2D
visited[node][mask]boolean (or asetof(node, mask)tuples). - For brute force: pairwise
disttable (BFS-precomputed) anditertools.permutations.
Correctness Argument
The state graph has nodes (u, mask) and edges (u, mask) → (v, mask | (1 << v)) for every graph neighbor v of u. A path in the original graph that visits all nodes corresponds 1-to-1 to a path in the state graph from some starting state (s, 1 << s) to a “complete” state (t, ALL) for some t. Since the state-graph edges are unweighted, BFS finds the shortest such path. Multi-source BFS over all starting states minimizes over all start nodes simultaneously. Termination: the state graph has N · 2^N nodes; BFS visits each at most once.
Complexity
| Stage | Time | Space |
|---|---|---|
| Brute force (perm + BFS dist) | O(N! · N + N²·E) | O(N²) |
| BFS over (node, mask) | O(N² · 2^N) | O(N · 2^N) |
| DP (popcount-ascending fill) | O(N² · 2^N) | O(N · 2^N) |
At N=12: 12² · 4096 = 590K ops — fast.
Implementation Requirements
from collections import deque
from itertools import permutations
# ---- Stage 1: Brute force ----
def shortestPathLength_brute(graph):
n = len(graph)
if n == 1: return 0
dist = [[float('inf')] * n for _ in range(n)]
for src in range(n):
dist[src][src] = 0
q = deque([src])
while q:
u = q.popleft()
for v in graph[u]:
if dist[src][v] == float('inf'):
dist[src][v] = dist[src][u] + 1
q.append(v)
best = float('inf')
for perm in permutations(range(n)):
cost = sum(dist[perm[i]][perm[i+1]] for i in range(n - 1))
if cost < best: best = cost
return best
# ---- Stage 2: Memoized DFS over (node, mask) — works but BFS is preferred for unweighted ----
from functools import lru_cache
def shortestPathLength_memo(graph):
n = len(graph)
ALL = (1 << n) - 1
@lru_cache(None)
def f(u, mask):
if mask == ALL: return 0
best = float('inf')
for v in graph[u]:
new_mask = mask | (1 << v)
if new_mask != mask: # only progress if v is newly visited
best = min(best, 1 + f(v, new_mask))
# also allow revisiting (no-op for mask but adds 1 to path length) -- but that's wasteful, skip
return best
# Try every starting node
return min(f(s, 1 << s) for s in range(n))
# NOTE: this memo version misses cases where you must transit through already-visited nodes.
# BFS handles that natively because (v, new_mask) where new_mask == mask is allowed if not yet seen.
# ---- Stage 3: BFS over (node, mask) — canonical solution ----
def shortestPathLength(graph):
n = len(graph)
if n == 1: return 0
ALL = (1 << n) - 1
visited = set()
q = deque()
for s in range(n):
state = (s, 1 << s)
visited.add(state)
q.append((s, 1 << s, 0))
while q:
u, mask, dist = q.popleft()
if mask == ALL: return dist
for v in graph[u]:
new_mask = mask | (1 << v)
state = (v, new_mask)
if state not in visited:
visited.add(state)
q.append((v, new_mask, dist + 1))
return -1 # unreachable; should not happen for connected graphs
# ---- Stage 4: DP filling by mask in popcount order (alternative formulation) ----
def shortestPathLength_dp(graph):
n = len(graph)
if n == 1: return 0
ALL = (1 << n) - 1
INF = float('inf')
# dp[mask][u] = min edges to reach state (u, mask) from any starting node
dp = [[INF] * n for _ in range(1 << n)]
q = deque()
for s in range(n):
dp[1 << s][s] = 0
q.append((s, 1 << s))
while q:
u, mask = q.popleft()
for v in graph[u]:
new_mask = mask | (1 << v)
if dp[new_mask][v] > dp[mask][u] + 1:
dp[new_mask][v] = dp[mask][u] + 1
q.append((v, new_mask))
return min(dp[ALL][u] for u in range(n))
Tests
[[1,2,3],[0],[0],[0]]→ 4.[[1],[0,2,4],[1,3,4],[2],[1,2]]→ 4.[[1],[0]]→ 1.[[]]→ 0 (single node).- N=12 with sparse and dense connectivity — performance smoke.
- Cross-check brute vs BFS on N≤6 random connected graphs.
Follow-up Questions
- “What if edges are weighted?” → Replace BFS with Dijkstra (priority queue). Still O((N · 2^N) log(N · 2^N) + edges).
- “Must return to start (TSP closed tour).” (Held-Karp) → Compute
dp[mask][u]= min cost from start touvisitingmask; final answer min overuofdp[ALL][u] + dist[u][start]. - “Can revisit, weighted, must visit all.” → Floyd-Warshall preprocess to get all-pairs shortest paths, then Held-Karp on the dense complete graph induced by those distances.
- “N up to 20 — does this still fit?” → 2^20 = 10^6, N²·2^N ≈ 4×10^8. Borderline; need bit tricks and tight inner loops, often C++/Java only.
- “All Hamiltonian paths (visit each node exactly once).” → Same DP; track exact
popcount(mask) == N. NP-hard but bitmask handles N≤20.
Product Extension
The bitmask DP / Held-Karp algorithm is the gold-standard exact solution for small TSP-like problems. Real applications: drone delivery routing for ≤ 20 stops, layout optimization in chip design, scheduling N jobs on a single machine with sequence-dependent setup times, optimal-question-ordering in adaptive testing.
Language/Runtime Follow-ups
- Python: bit operations (
|,&,<<) on ints are arbitrary-precision;(1 << N) - 1works for any N. Usebin(mask).count('1')for popcount, ormask.bit_count()in Python 3.10+. - Java:
Integer.bitCount(mask). Useintfor N ≤ 31,longfor N ≤ 63. - Go:
bits.OnesCount(uint(mask))frommath/bits. - C++:
__builtin_popcount(mask)(orpopcountin C++20). Compiles to a single CPU instruction. - JS/TS: bitwise ops are 32-bit signed; for N > 30 use
BigInt(slower).
Common Bugs
- Forgetting to multi-source initialize the BFS — a single starting node misses better starts.
- Treating
(v, new_mask)wherenew_mask == maskas visited and skipping — this is correct for BFS if the state was already enqueued, but new code may forget that revisiting is sometimes necessary (transit through known nodes). The state-key(v, mask)handles this automatically. mask | (1 << v)missing the parentheses:mask | 1 << vparses as(mask | 1) << vin C/Java/JS — wrong. Always parenthesize the shift.- Forgetting the
if mask == ALL: return distcheck at dequeue time — checking only at enqueue can miss adist+1opportunity. setof tuples performance: in Python, a 2Dboolarray is faster than asetfor the visited check at high N. Use[[False]*N for _ in range(1<<N)].
Debugging Strategy
For [[1,2,3],[0],[0],[0]] (N=4): BFS expands (0,0001), (1,0010), (2,0100), (3,1000) at distance 0. From (1,0010), we go to (0,0011) at distance 1. From (0,0011), we go to (1,0011), (2,0111), (3,1011) at distance 2. From (2,0111) we go to (0,0111) at distance 3. From (0,0111) we go to (3,1111) at distance 4 — done. Print (u, bin(mask), dist) per dequeue and locate where your trace diverges.
Mastery Criteria
- Recognized “N ≤ 12, visit all” as bitmask DP within 60 seconds.
-
Stated the state space
(node, mask)and its sizeN · 2^Nin <30 seconds. - Wrote the multi-source BFS in <8 minutes from blank screen.
- Articulated why multi-source initialization is correct (start anywhere).
- Stated O(N² · 2^N) time complexity.
- Solved LC 847 unaided in <20 minutes.
- Articulated the difference between BFS-over-states (unweighted) and Held-Karp (weighted/TSP) in <60 seconds.
-
Wrote
mask.bit_count()/__builtin_popcountcorrectly without prompting. -
Solved LC 526 (Beautiful Arrangement) in <12 minutes using
dp[mask]over assignments.
Lab 11 — Digit DP
Goal
Count integers in [0, N] (and [L, R] by subtraction) that satisfy a per-digit property, in O(digits × states × 10). Master the canonical recursion dp(pos, tight, state, leading_zero) and apply it to LeetCode 902, 2719, and the hard classic LeetCode 1012. After this lab you should write a digit-DP template from a cold start in <15 minutes and adapt its state to a new property in <5.
Background Concepts
Digit DP builds the number one decimal position at a time, left to right, counting how many completions satisfy the property. The four canonical parameters:
pos— index into the digits ofN,0= most significant.tight— are we still bounded byN’s prefix? IfTrue, the current digit may not exceedN[pos]; if it equalsN[pos]we stay tight, otherwise we go loose. Loose stays loose forever.state— everything the property needs to remember (a used-digit bitmask, a running remainder mod k, a “previous digit,” a digit-sum, …).leading_zero— are we still in the leading-zero region? A leading zero is not a real digit: it must not mark the mask, must not count as “used digit 0,” and often must not start counting a length.
Why tight collapses the exponential tree. Only one prefix of the search tree is ever tight — the one equal to N’s prefix. Every loose branch is fully free and its subtree count depends only on (pos, state), so it memoizes. The tight branch is a single spine of length = len(N). That is why the whole thing is polynomial: the state space is pos × state × {tight} × {leading}, and tight/leading each multiply by at most 2.
Answering [L, R]. Count is monotone and prefix-summable: answer(L, R) = f(R) − f(L − 1), where f(X) counts valid numbers in [0, X]. Handle L = 0 by defining f(−1) = 0.
Leading-zero pitfalls. For properties about which digits are used, 00007 uses only the digit 7, not 0 — so while leading_zero is True, a chosen 0 must recurse without touching the mask. Get this wrong and you overcount (e.g., treating short numbers as if padded with real zeros).
Interview Context
Digit DP is the sledgehammer for “how many numbers up to N have property P.” Candidates who haven’t seen it try to iterate [0, N] (10⁹+ — TLE) or hand-roll fragile combinatorics that break on the tight boundary. Recognizing the pattern and producing the memoized dp(pos, tight, state, leading) cleanly — especially the leading-zero and complement-counting subtleties — is a strong L5 signal. The tell: N up to 10⁹ or 10¹⁸ and “count numbers such that …”.
Problem Statement (LC 902)
Given an array digits of sorted distinct decimal digit characters (a subset of 1..9) and a positive integer n, count how many positive integers can be written using digits from digits (with repetition, any length) that are less than or equal to n. The main implementation showcase in this lab is the harder sibling, LC 1012 (Numbers With Repeated Digits), which exercises every subtlety — bitmask state, leading zeros, and complement counting.
Constraints
- 1 ≤ digits.length ≤ 9, digits distinct and sorted, each in
1..9(LC 902). - 1 ≤ n ≤ 10⁹ (LC 902); 1 ≤ N ≤ 10⁹ (LC 1012). Technique scales to 10¹⁸ with 64-bit ints.
Clarifying Questions
- Is the range inclusive of
N? (Yes —fcounts[0, N].) - Do we count
0itself? (LC 1012 counts positives[1, N]; the DP naturally produces the count including 0, so subtract or start at 1.) - Are leading zeros part of the number? (No —
007is the number7; leading zeros mark nothing.) - For “repeated digit,” does a single-digit number ever count? (No — one digit cannot repeat.)
- Is the property about the value (mod k, digit-sum) or about the set of digits used? (Determines whether
stateis a remainder/sum or a bitmask.) - For
[L, R], isLpossibly0? (If so,f(L − 1) = f(−1) = 0.)
Examples
LC 1012 N = 20 → 1 (only 11 has a repeated digit)
LC 1012 N = 100 → 10 (11,22,…,99,100)
LC 1012 N = 1000 → 262
LC 902 digits=[1,3,5,7], n=100 → 20 (4 one-digit + 16 two-digit)
LC 902 digits=[1,4,9], n=1000000000 → 29523
Initial Brute Force
Loop x from 0 to N, convert each to a string, test the property (len(set(str(x))) < len(str(x)) for repeated digits), and count. Correct and trivial to write — use it as the oracle.
Brute Force Complexity
O(N × D) where D = number of digits (the per-number string work). At N = 10⁹ that is ~10¹⁰ character operations — TLE by orders of magnitude. The whole point of digit DP is to replace the N factor with len(N) ≈ log₁₀ N.
Optimization Path
Replace the linear scan with the position-by-position count. Direct counting has D positions, each with 2 values of tight, 2 of leading, and at most 2^10 mask states (for the used-digits property) or k remainder states — times 10 transitions. That is O(D × states × 10): for LC 1012, 10 × 2 × 2 × 1024 × 10 ≈ 4×10⁵ — instant.
Complement counting is the key trick for “repeated digit”: counting numbers with a repeat directly is awkward, but counting numbers with all-distinct digits is a clean bitmask DP. Then repeated = N − distinct. Always look for the easier complement.
Final Expected Approach
Count all-distinct-digit numbers in [1, N] with a memoized dp(pos, mask, tight, leading); the answer to LC 1012 is N − that. The [L, R] wrapper subtracts two f calls. LC 902 is the same template with a digit-set filter and no mask.
from functools import lru_cache
def count_distinct(N):
"""Count x in [1, N] with pairwise-distinct decimal digits."""
if N < 1:
return 0
s = list(map(int, str(N)))
n = len(s)
@lru_cache(maxsize=None) # nested def ⇒ cache is per-call
def dp(pos, mask, tight, leading):
if pos == n:
return 0 if leading else 1 # all-zeros path is the integer 0
hi = s[pos] if tight else 9
total = 0
for d in range(hi + 1):
nt = tight and d == hi
if leading and d == 0:
total += dp(pos + 1, mask, nt, True) # 0 marks nothing yet
elif (mask >> d) & 1:
continue # digit d already used
else:
total += dp(pos + 1, mask | (1 << d), nt, False)
return total
ans = dp(0, 0, True, True)
dp.cache_clear() # tight/leading are per-N: never persist
return ans
def num_dup_digits_at_most_n(N): # LC 1012, via complement
return N - count_distinct(N)
def distinct_in_range(L, R): # [L, R] = f(R) − f(L−1)
return count_distinct(R) - count_distinct(L - 1)
def at_most_n_given_digit_set(digits, N): # LC 902, same template, no mask
s = str(N)
n = len(s)
D = sorted(int(c) for c in digits)
@lru_cache(maxsize=None)
def dp(pos, tight):
if pos == n:
return 1
total = 0
hi = int(s[pos])
for d in D:
if tight and d > hi:
break
total += dp(pos + 1, tight and d == hi)
return total
shorter = sum(len(D) ** k for k in range(1, n)) # all 1..n−1 digit numbers
ans = shorter + dp(0, True)
dp.cache_clear()
return ans
Data Structures Used
- The digit list of
N(list(map(int, str(N)))) as the positional bound. functools.lru_cachekeyed on(pos, mask, tight, leading)for memoization.- A bitmask (
1 << d) as the used-digit state for LC 1012; a remainder or digit-sum scalar for other properties.
Correctness Argument
Positional decomposition. Every integer in [0, N] has a unique decimal representation read left to right; the DP enumerates each exactly once by fixing digits position by position. tight guarantees the tight spine never emits a number exceeding N, and once loose, all remaining positions range freely 0..9 — so the union of tight-spine and loose-subtree completions is exactly [0, N].
Leading-zero handling. While leading holds, a chosen 0 recurses without touching mask and keeps leading = True, so 00007 reaches the base case having marked only digit 7 — it is counted as the number 7 with a one-element digit set, never as “using digit 0.” The base case returns 0 for the all-leading path so the empty number isn’t miscounted as a positive.
Complement. distinct + repeated = N over [1, N] (partition by whether any digit repeats), so repeated = N − distinct; the DP computes distinct exactly, hence LC 1012 is exact.
Complexity
| Quantity | Value |
|---|---|
| States | pos (≈10) × mask (≤1024) × tight (2) × leading (2) |
| Transitions | ×10 digits |
| Time | O(digits × states × 10) — ~10⁵ for LC 1012 |
| Space | O(states) for the cache |
Implementation Requirements
- Memoize with a nested
dpand calldp.cache_clear()(or use a freshlru_cacheperN) — see the caching pitfall below. - Handle
leading_zeroseparately fromtight; they are independent booleans. - For
[L, R], always computef(R) − f(L − 1), definingf(x) = 0forx < 0. - Prefer complement counting when the target property is messy and its negation is clean.
Tests
- LC 1012:
N = 20 → 1,N = 100 → 10,N = 1000 → 262. - Exhaustive: for every
Nin1..1999,num_dup_digits_at_most_n(N)equals the brute-force count. [L, R]wrapper vs brute force on random1 ≤ L ≤ R ≤ 3000.- LC 902:
([1,3,5,7], 100) → 20,([1,4,9], 10⁹) → 29523. - Edge:
N = 9(all single-digit ⇒ 0 repeats);N = 10(still 0);N = 11(first repeat).
Follow-up Questions
- Why must
tight(andleading) not persist across differentN? Their meaning depends onN’s specific digits, so a cache entry(pos, mask, tight=True, …)computed for oneNis wrong for another. The nested-def+cache_clear()(or a per-calllru_cache) resets the cache each call. The trulyN-independent entries are the loose ones (tight=False), which is exactly why loose subtrees memoize within a single call. - How do leading zeros interact with a “digits used” property? A leading
0must not set bit 0 of the mask, because007uses no zero. So the transition keepsleading=Trueand leavesmaskuntouched on a chosen0; only the first nonzero digit flipsleadingoff and starts marking. Conflating the two overcounts short numbers as zero-padded. - How do you do digit DP in another base — e.g., count numbers ≤ N with no two consecutive 1-bits (LC 600)? Same recursion over the binary digits of
N:state= “was the previous bit 1,” transition skipsd=1when the previous was 1.posruns over ~30 bits instead of ~10 decimal digits. - How do you handle a sum-type query — sum of digit-sums over
[L, R], not just a count? Carry an aggregate pair(count, total)up the recursion. On appending digitdtocountcompletions contributingtotal, the parent getscountmore numbers andtotal + d × countadded digit-sum. Combine children by summing both components. - How do you count numbers divisible by k in a range? Add
remainder mod ktostate; the transition updates(rem × 10 + d) % k, and the base case counts onlyrem == 0. State size grows by a factor of k. - What is LC 2719 (Count of Integers) and how does it fit? Count integers in
[num1, num2]whose digit-sum lies in[min_sum, max_sum].stateis the running digit-sum (capped atmax_sum), and[L, R]is thef(num2) − f(num1 − 1)wrapper — a textbook sum-in-range digit DP.
Product Extension
Analytics and fraud systems that bucket over huge id/amount ranges use digit-DP-style counting to answer “how many account numbers in this block match a structural rule” without materializing the range. Lexicographic/odometer generation (next-valid-code, license-plate or coupon-code counting under alphabet constraints) is digit DP over a custom alphabet. Query optimizers estimating “how many keys in [lo, hi] satisfy a predicate on their digits” reuse the same positional counting. Anywhere the range is astronomically large but the property is per-digit, digit DP replaces enumeration.
Language/Runtime Follow-ups
- Python:
functools.lru_cacheon a nesteddefis the cleanest memo; callcache_clear()betweenNs. Big ints are free, so 10¹⁸ needs no special handling. - Java: use an explicit
Integer[][][][] memo(orlong), and reset it per call;-1sentinel for “unvisited.” Bewareintoverflow on counts — uselong. - C++:
vectoror a fixedlong long dp[20][1<<10][2][2]with avisitedflag; pass the digit array by reference. Reset between test cases. - Go: 4-D slice memo built per call; Go has no memo decorator, so wire it manually. Use
int64for counts. - JS/TS:
Mapkeyed on a serializedpos,mask,tight,leadingstring, or nested arrays;BigIntonly if counts exceed 2⁵³.
Common Bugs
- Persisting
tightacrossNin a shared cache: a cache reused betweenf(R)andf(L−1)withtightin the key returns stale counts. Clear the cache per call or key it correctly. - Leading zero marks the mask: treating a leading
0as “used digit 0” undercounts distinct-digit numbers (every short number wrongly conflicts on 0). Skip the mask whileleading. - Counting the number 0 as positive: the all-leading path must return
0, not1, orfis off by one and LC 1012’s complement is wrong. hinot clamped bytight: iteratingdto 9 on a tight branch emits numbers > N.hi = s[pos] if tight else 9.nt = tight and d == hicomputed with the wronghi: if you loosenhito 9 first,d == hino longer detects the boundary. Computentagainsts[pos]on tight branches.- Off-by-one in
[L, R]: usingf(R) − f(L)instead off(R) − f(L − 1)dropsLitself.
Debugging Strategy
Run the DP against the brute-force oracle for every N in 1..2000 — a single mismatch pinpoints the boundary or leading-zero bug by its smallest failing N. If it first fails at a power of 10 (100, 1000), suspect the tight-boundary or length-transition logic. If it fails at repdigit-adjacent values (11, 22, 100), suspect the mask/leading interaction. Instrument dp to print (pos, mask, tight, leading) → return on the smallest failing N and compare against a hand trace of the two-digit case.
Mastery Criteria
-
Wrote the
dp(pos, tight, state, leading)template from cold in <15 minutes. - Solved LC 1012 via complement counting with a distinct-digit bitmask.
-
Adapted
stateto a new property (mod k, digit-sum, no-consecutive-ones) in <5 minutes. -
Explained why
tightcollapses the tree to polynomial and why it must not persist acrossN. - Handled leading zeros correctly for a “digits used” property.
-
Built the
[L, R] = f(R) − f(L − 1)wrapper without an off-by-one.
Lab 12 — SOS DP (Sum over Subsets)
Goal
Compute F[mask] = Σ A[sub] over all sub ⊆ mask, for all masks simultaneously, in O(2ⁿ · n) — replacing the O(4ⁿ) all-pairs-of-masks or O(3ⁿ) subset-enumeration approaches. Apply it to LeetCode 982 (Triples with Bitwise AND Equal To Zero). After this lab you should write the forward transform, its Möbius inverse, and the 3ⁿ subset loop from memory, and recognize when each wins.
Background Concepts
The naive “for each mask, sum over its subsets” costs O(3ⁿ) (each of 2ⁿ masks enumerates its subsets; Σ over masks of 2^popcount = 3ⁿ). SOS DP does it in O(2ⁿ · n) by a dimension-by-dimension relaxation — the subset-sum (zeta) transform:
def sos(A, n):
F = A[:]
for i in range(n): # one bit-dimension at a time
bit = 1 << i
for mask in range(1 << n):
if mask & bit:
F[mask] += F[mask ^ bit]
return F
Interpretation as n-dimensional prefix sum. Think of a mask as a point in the hypercube {0,1}ⁿ. Dimension i sweeps the “prefix sum along axis i”: after processing bit i, F[mask] holds the sum over all sub-masks that agree with mask on bits > i (already processed) and are ≤ mask on bits ≤ i. After all n sweeps, F[mask] = sum over every sub ⊆ mask. It is exactly the standard multi-dimensional prefix-sum, one axis per bit, each axis of length 2.
The 3ⁿ subset-enumeration trick is the intermediate power tool:
sub = mask
while True:
# visit sub ⊆ mask
if sub == 0: break
sub = (sub - 1) & mask # next lower submask
Use 3ⁿ when you need only one mask’s subset sum, or when the per-subset work is non-additive. Use SOS’s 2ⁿ·n when you need the sums for all masks. Crossover: for n ≈ 20, 3ⁿ ≈ 3.5×10⁹ vs 2ⁿ·n ≈ 2×10⁷ — SOS wins by two orders of magnitude when all masks are needed.
Inverse (Möbius) transform. Undo the zeta by subtracting in the same sweep structure: F[mask] -= F[mask ^ bit]. Zeta ∘ Möbius = identity. This is the inclusion-exclusion inverse.
Superset sums are the mirror image: process the unset bit direction (F[mask] += F[mask | bit]), or equivalently run subset SOS on complemented indices.
Interview Context
SOS DP is niche but a hard differentiator: it appears in bitmask-heavy counting problems where a candidate’s first instinct is O(4ⁿ) pair enumeration. Recognizing “I need, for every mask, the aggregate over all its submasks” and reaching for the 2ⁿ·n transform is a senior/competitive signal. LeetCode has few pure hits — the honest framing is that SOS is a technique you fold into subset-DP problems (Maximum Students Taking Exam, Count Subtrees) and AND/OR pair-counting. LC 982 is the cleanest canonical showcase.
Problem Statement (LC 982)
Given an integer array nums, return the number of triples of indices (i, j, k) (each independently ranging over all indices, repetition allowed, order counted) such that nums[i] & nums[j] & nums[k] == 0, where & is bitwise AND.
Constraints
- 1 ≤ nums.length ≤ 1000.
- 0 ≤ nums[i] < 2¹⁶ (so masks live in a 16-bit universe,
2¹⁶ = 65536).
Clarifying Questions
- Are
i, j, kordered and independently chosen? (Yes —(0,1,2)and(2,1,0)are distinct;i = jis allowed.) - What’s the bit width? (Values < 2¹⁶, so a 16-bit mask universe —
U = 2¹⁶.) - Do we count value-triples or index-triples? (Index-triples — duplicates in
numseach count.) - Is the answer possibly large? (Up to
1000³ = 10⁹— fits in 64-bit, but note it in languages with 32-bit ints.) - Is
AND == 0a subset or a disjointness condition? (x & y & z == 0means the ANDed bits are a subset of the complement of… — see the reduction below.)
Examples
nums = [2, 1, 3] → 12
nums = [0, 0, 0] → 27 (every triple ANDs to 0)
Initial Brute Force
Triple nested loop over all (i, j, k), test nums[i] & nums[j] & nums[k] == 0, count. O(m³) with m = len(nums) — the oracle.
Brute Force Complexity
O(m³). At m = 1000 that is 10⁹ AND-and-compare operations — borderline-to-TLE in Python, and the pattern generalizes badly. A better structure: precompute a histogram of pairwise ANDs (pairs[v] = number of (i, j) with nums[i] & nums[j] == v) in O(m²), then for each k count pairs whose AND is disjoint from nums[k].
Optimization Path
The reduction: x & z == 0 ⟺ x ⊆ ~z (every set bit of x is unset in z). So nums[i] & nums[j] & nums[k] == 0 ⟺ (nums[i] & nums[j]) ⊆ ~nums[k]. Therefore:
- Build
pairs[v]= #{(i, j) : nums[i] & nums[j] == v} in O(m²). - SOS:
F[mask] = Σ_{v ⊆ mask} pairs[v]= number of pairs whose AND is a submask ofmask, for all masks in O(U · log U). - Answer =
Σ_k F[~nums[k] & (U−1)]— for eachk, count pairs disjoint fromnums[k].
Total O(m² + U log U) instead of O(m³). The SOS step is what turns “for each k, re-scan all pair-values” (another O(U) or O(m²)) into an O(1) table lookup per k.
Final Expected Approach
def sos(A, n):
"""F[mask] = Σ A[sub] over all sub ⊆ mask, all masks, in O(2ⁿ · n)."""
F = A[:]
for i in range(n):
bit = 1 << i
for mask in range(1 << n):
if mask & bit: # relax along bit-dimension i
F[mask] += F[mask ^ bit]
return F
def inverse_sos(F, n):
"""Möbius inverse: recover A from F = sos(A). Subtract in the same order."""
A = F[:]
for i in range(n):
bit = 1 << i
for mask in range(1 << n):
if mask & bit:
A[mask] -= A[mask ^ bit]
return A
def count_and_triples(nums):
"""LC 982: # ordered triples (i,j,k) with nums[i] & nums[j] & nums[k] == 0."""
B = 16
FULL = (1 << B) - 1
pairs = [0] * (1 << B)
for x in nums: # O(m²) pairwise-AND histogram
for y in nums:
pairs[x & y] += 1
F = sos(pairs, B) # F[m] = #pairs with AND ⊆ m
return sum(F[FULL ^ z] for z in nums) # AND & z == 0 ⟺ AND ⊆ ~z
# 3ⁿ single-mask subset sum (the intermediate power tool)
def subset_sum_of(A, mask):
total, sub = 0, mask
while True:
total += A[sub]
if sub == 0:
break
sub = (sub - 1) & mask # next lower submask of mask
return total
Data Structures Used
- A flat array
Fof length2ⁿindexed by bitmask — the transform is entirely in-place-ish over it. - The
pairs[]histogram of size2¹⁶for LC 982. - Bit operations:
1 << i,mask ^ bit,(sub − 1) & mask,~z & FULLfor the complement.
Correctness Argument
Loop invariant of the zeta transform. After processing dimensions 0..i−1, F[mask] equals Σ A[sub] over all sub that equal mask on bits ≥ i and are arbitrary (≤ mask) on bits < i. Base: before any dimension, F[mask] = A[mask] (only sub = mask). Step: when we process bit i for a mask with bit i set, F[mask] += F[mask ^ bit] fuses the “bit-i-set” family with the “bit-i-clear” family, both already summed over bits < i — extending the invariant to include bit i. After all n dimensions, the free set is all bits, i.e. every sub ⊆ mask. Each sub is added exactly once because each dimension merges disjoint families.
Möbius inverts zeta. Subtracting A[mask ^ bit] in the same sweep order peels off precisely the contribution the forward pass added, so inverse_sos(sos(A)) == A — the standard subset-lattice inclusion-exclusion pair.
LC 982 reduction. a & z == 0 ⟺ a ⊆ ~z. F[~z] sums pairs[a] over all a ⊆ ~z, i.e. counts pairs (i, j) whose AND is disjoint from z = nums[k]. Summing over k counts all valid ordered triples.
Complexity
| Step | Time | Space |
|---|---|---|
| Pair histogram (LC 982) | O(m²) | O(2ⁿ) |
| SOS forward / inverse | O(2ⁿ · n) | O(2ⁿ) |
| Answer aggregation | O(m) | O(1) |
| 3ⁿ single-mask subset sum | O(3ⁿ) total over all masks | O(1) |
Implementation Requirements
- Copy the input (
F = A[:]) — the transform mutates in place; don’t clobber the caller’s array unless intended. - Iterate dimensions outer, masks inner; swapping the loop order breaks the prefix-sum semantics.
- Only update
F[mask]whenmask & bit— skipping this doubles-adds. - For LC 982, use the full 16-bit universe even if
max(nums)is small; sizing tomax+1bits is fine but must cover every complement~z.
Tests
sos(A, n)vs the 3ⁿ subset-enumeration sum for every mask, randomA,n ∈ {3,4,5}.inverse_sos(sos(A, n), n) == Afor randomA.- LC 982:
[2,1,3] → 12,[0,0,0] → 27. - LC 982 vs O(m³) brute force on random arrays (values up to 2¹⁶ and small values), dozens of trials.
- Superset-sum variant: verify
F[mask] += F[mask | bit]gives Σ over supersets.
Follow-up Questions
- What are the zeta and Möbius transforms, and how do they connect to inclusion-exclusion? The subset-sum transform is the zeta transform on the subset lattice; its inverse is the Möbius transform, whose kernel is
(−1)^popcount(mask \ sub). Alternating-sign subset sums are inclusion-exclusion — the same machinery as Phase 12’s inclusion-exclusion lab, specialized to the boolean lattice where Möbius is just “subtract each dropped bit.” - How do you convolve over subsets —
C[mask] = Σ_{s∪t=mask, s∩t=∅} A[s]·B[t]? Subset-sum convolution: rank the arrays by popcount inton+1layers, zeta-transform each layer, multiply layer-wise with a popcount-carrying convolution, then Möbius back — O(2ⁿ · n²). It fixes the double-counting that a plain OR-convolution would introduce. - What is OR-convolution vs AND-convolution vs XOR-convolution? OR-convolution = zeta (subset SOS) → pointwise multiply → Möbius. AND-convolution is the same with the superset transform. XOR-convolution needs the Walsh–Hadamard transform instead of zeta (cross-reference Phase 12 lab 07’s follow-up) — a different butterfly, but the same “transform, pointwise-multiply, inverse-transform” pipeline.
- When does 3ⁿ beat 2ⁿ·n? When you need submask sums for only a few masks, or when the per-subset operation isn’t additive (so the incremental zeta relaxation doesn’t apply). For all-masks additive aggregation, 2ⁿ·n dominates.
- How does SOS help other bitmask DPs? Precompute a table of “best value achievable by any submask of
mask” via a max-flavored SOS (replace+=with amax), then answer meet-in-the-middle or set-cover pruning queries in O(1). The transform generalizes to any commutative-monoid aggregate (sum, max, min, count). - How do you get superset sums instead of subset sums? Flip the merge direction:
if not (mask & bit): F[mask] += F[mask | bit], or run subset SOS on the bit-complemented index space. Useful for AND-based counting where you want “supersets of a given mask.”
Product Extension
Feature-flag and permission systems: given a value assigned to each exact capability set, SOS answers “total value across all subsets of these permissions” in one pass — e.g. cohort counts over every attribute combination for analytics roll-ups. Ad/recommendation systems that bucket users by a bitmask of interests use subset/superset sums to answer “how many users have at least these interests” (superset) or “value summed over all interest subsets” (subset) without re-scanning. Any OLAP-style cube aggregation over a small set of boolean dimensions is an n-dimensional prefix sum — literally SOS.
Language/Runtime Follow-ups
- Python: the tight
2ⁿ·nloop is slow in pure Python; for n = 20 prefernumpyvectorization per dimension (F[mask_has_bit] += F[mask_has_bit ^ bit]via boolean indexing) or accept the constant factor. Copy withA[:]. - Java/C++: trivial
for (i) for (mask)withint[]/vector<long long>; watch overflow — LC 982 counts reach 10⁹, uselong/long long. - C++: this is the canonical competitive form;
for(int i=0;i<n;i++) for(int m=0;m<(1<<n);m++) if(m>>i&1) F[m]+=F[m^1<<i];. - Go:
int64slice, same double loop; no generics needed. - JS/TS:
Int32Array/Float64Arrayof length2ⁿ; useNumberup to 2⁵³,BigIntonly if counts overflow.
Common Bugs
- Loop order swapped: iterating masks outer and dimensions inner computes garbage — the zeta relaxation requires each bit-dimension to fully sweep before the next.
- Missing the
mask & bitguard: updating every mask (not just those with bitiset) double-adds and inflates sums. - In-place clobber: transforming the caller’s array without copying corrupts data the caller still needs (e.g. reusing
pairsaftersos). - Wrong universe size: sizing
Ftomax(nums)+1bits but then indexing~z(which can set high bits) reads out of range or misses pairs. Mask complements with& FULLand size for the full bit width. - Integer overflow: LC 982’s answer approaches
m³ = 10⁹; a 32-bit accumulator overflows in Java/C/Go. - Confusing subset and superset direction: using
F[mask ^ bit](subset) when the problem wants supersets (F[mask | bit]) inverts the meaning of the aggregate.
Debugging Strategy
Validate sos against the 3ⁿ subset loop on tiny n first — if they disagree, the transform itself is wrong, independent of the problem. Then check inverse_sos(sos(A)) == A to confirm the Möbius direction. For LC 982, run the O(m³) oracle on random small arrays: a mismatch localizes to either the pairs histogram (O(m²) step) or the complement lookup F[FULL ^ z]. Print F for a 2-bit universe by hand — F[0b11] should equal A[0]+A[1]+A[2]+A[3] — to catch loop-order and guard bugs instantly.
Mastery Criteria
- Wrote the forward SOS transform from memory in <5 minutes.
- Wrote the Möbius inverse and the 3ⁿ subset loop cold.
- Explained SOS as an n-dimensional prefix sum with the loop invariant.
- Solved LC 982 via the AND-to-submask reduction and SOS in <20 minutes.
- Stated the crossover between 3ⁿ and 2ⁿ·n and picked correctly.
- Connected zeta/Möbius to inclusion-exclusion and named the OR/AND/XOR-convolution pipeline (WHT for XOR).
Phase 6 — Greedy, Proofs & Mathematical Thinking
Target level: Medium → Hard Expected duration: 1.5 weeks (12-week track) / 2 weeks (6-month track) / 2.5 weeks (12-month track) Weekly cadence: ~7 greedy concepts plus 6 labs plus 25–40 problems applying them under the framework
Why Greedy Is The Single Most Dangerous Pattern Family In Coding Interviews
Greedy is the topic where the largest number of candidates fail confidently. Unlike dynamic programming, where the failure mode is “I cannot derive the recurrence” — a visible failure that the interviewer can help with — greedy’s failure mode is “I have a plausible algorithm, I have run it on the given example, it works, and I am wrong.” The candidate writes a clean function, the test cases pass, the complexity is excellent, and the algorithm is silently incorrect on the third hidden test. By the time the interviewer reveals the counterexample, the candidate has consumed 25 minutes building a wrong solution and has 10 minutes left to either patch it (impossible without re-deriving) or restart with DP (also impossible).
The empirical claim that drives this entire phase:
The hard part of greedy is not the algorithm. The hard part is the proof of correctness. Almost every wrong greedy solution is wrong because the candidate convinced themselves “this seems to work” without an exchange argument or invariant — and an interviewer who suspects the candidate is guessing will deliberately construct a counterexample. A candidate who can produce an exchange argument out loud, before the interviewer asks, is signalling “I know what I’m doing”; a candidate who can’t is signalling “I memorized this”.
Greedy is also the topic where the gap between a good engineer and a great one is widest in interview signal. Most candidates can write sort + scan + counter. Very few can articulate why the sort criterion is correct. The ability to say, in 60 seconds, “Suppose for contradiction the optimal solution uses a different first choice; I can exchange it with my greedy choice without making the solution worse, therefore my greedy is also optimal” — that one paragraph is the entire difference between an L4 hire and an L5 hire on greedy questions.
This phase is built around one teaching device that we will use on every single problem from start to finish: the proof comes before the code. Every lab in this phase requires you to write the correctness argument — exchange argument, invariant, or monovariant — before you write the implementation. The implementation is mechanical once the proof is solid; the proof is the whole skill.
After this phase, you can recognize when a problem is amenable to greedy in <2 minutes, produce an exchange argument out loud in <90 seconds, write the algorithm in <5 minutes, and — crucially — identify when a problem looks greedy but is not, falling back to DP from Phase 5 without panic.
What You Will Be Able To Do After This Phase
- Recognize greedy candidates in <2 minutes by spotting the greedy choice property signal: “the locally optimal choice cannot hurt the global optimum.”
- Distinguish greedy-applicable problems from DP-required problems on first read, using the Greedy-vs-DP flowchart.
- Produce an exchange argument for any greedy you propose, in the canonical four-step form (assume optimal differs, locate first divergence, exchange, prove no-worse).
- Cite the cut property for MST correctness and explain why both Kruskal and Prim are correct under it.
- Use loop invariants to scaffold proofs of greedy algorithms whose correctness is not obvious from a single exchange.
- Use monovariants (strictly-decreasing or strictly-increasing quantities) to prove termination and correctness of iterative greedy algorithms.
- Apply amortized analysis (potential method, accounting method, aggregate analysis) to bound the cost of greedy data-structure operations.
- Identify counterexamples to plausible-looking greedy heuristics, including the canonical “0/1 knapsack ≠ fractional knapsack” trap.
- Implement and prove correct: interval scheduling, jump game II, task scheduler, gas station, Huffman coding, and the greedy-vs-DP comparison on coin change.
- Articulate the failure modes of greedy unprompted: missing counterexample, “feels right” without proof, confusing local optimum with global optimum.
How To Read This Phase
Read this README in two passes. Pass 1: linear, end-to-end, building the mental discipline that “I will not ship a greedy solution without an exchange argument.” Do this in one sitting. Pass 2: as you work the labs, refer back to specific concept entries when stuck on a proof.
Each concept entry has a fixed shape:
- Precise Definition — what the concept means, mathematically.
- When Applicable — the problem signal that should fire this concept.
- Worked Example — the concept applied to a canonical problem, end-to-end.
- Common Misuse — the concrete failure mode this concept guards against.
The phase ends with a Greedy-vs-DP flowchart, a Common Greedy Bugs catalog, a Mastery Checklist, and Exit Criteria.
Inline Concept Reference
1. Greedy Choice Property
Precise Definition
A problem has the greedy choice property if there exists an ordering of the input such that, after making the locally optimal choice (the “greedy choice”) at each step, the result is a globally optimal solution. Formally: at every step i, there is a choice c_i such that some globally optimal solution to the original problem extends c_1, c_2, …, c_i. Equivalently, after the greedy choice the remaining problem is a smaller instance of the same problem, and combining the greedy choice with any optimal solution to the residual problem yields an optimal solution to the original.
This is the formal cousin of optimal substructure (which DP also requires) plus the additional claim that a single locally optimal choice — not a search over choices — suffices at each step.
When Applicable
The greedy choice property holds when:
- The problem can be solved by a sequence of irreversible decisions.
- At each step, there is a most attractive choice by some natural metric (earliest deadline, smallest weight, largest ratio, latest start time).
- An exchange argument or cut property can be invoked to prove that the most-attractive choice is never the wrong one.
The greedy choice property does not hold when:
- The right choice at step
idepends on choices made at stepi+1,i+2, … (i.e., you must “look ahead” to decide). This is the DP regime. - There are multiple incomparable “locally optimal” candidates and the wrong one creates suboptimal residual problems.
Worked Example: Activity Selection
Given n activities with start and end times, pick the maximum number of non-overlapping activities.
The greedy choice: pick the activity with the earliest end time among those still compatible with the previous picks. This satisfies the greedy choice property because: any optimal solution either contains the earliest-ending activity, or — if it doesn’t — we can swap its first picked activity for the earliest-ending one without overlap and without changing the count, producing a new optimal solution that does. (See Lab 01 — Interval Scheduling for the full exchange argument.)
Common Misuse
The most common error is to assume the greedy choice property without proving it, on the basis that “earliest deadline first feels intuitive”. Counter-examples are everywhere; for example, “earliest start time” also feels intuitive but is wrong (consider one activity from 1 to 100 versus dozens of short activities from 2 to 3, 3 to 4, …). The discipline: every greedy claim must be paired with a proof.
2. The Exchange Argument — The Canonical Greedy Proof Technique
Precise Definition
An exchange argument proves that a greedy solution G is optimal by showing that any other solution O can be transformed into G without increasing cost (or decreasing value) via a sequence of exchanges. Each exchange replaces an element of O with the corresponding element of G and proves the swap is non-worsening. After all exchanges, O has been transformed into G, so G is at least as good as O. Since O was arbitrary, G is at least as good as any solution — i.e., G is optimal.
Step-By-Step Recipe
The recipe is rigid. Memorize it. Use it on every greedy proof.
- Let
Gbe the greedy solution. Define it precisely (e.g., “the activities chosen by earliest-end-time-first”). - Let
Obe an arbitrary optimal solution. AssumeO ≠ G(otherwise we’re done). - Locate the first index where they differ. Let
ibe the smallest index such thatO[i] ≠ G[i]. By construction,G[0..i-1] = O[0..i-1]. - Perform the exchange. Replace
O[i]withG[i], producing a new solutionO'. - Prove
O'is feasible (still satisfies all constraints). - Prove
O'is no worse thanO(same objective value, or no-worse if it’s a max/min). - Repeat.
O'agrees withGon more positions thanOdid. Iterate; after finitely many exchanges,Ohas been transformed intoG. ThereforeGis optimal.
Worked Example: Activity Selection
Greedy G = activities sorted by end time, picked greedily. Suppose O ≠ G is optimal. Let i be the first divergence. By the greedy rule, G[i] has the earliest end time among activities compatible with G[0..i-1] = O[0..i-1]. So G[i].end ≤ O[i].end. Replace O[i] with G[i]: feasibility holds because G[i] ends no later than O[i], so all subsequent activities in O[i+1..], which all start after O[i].end, also start after G[i].end. The objective (count of activities) is unchanged. Repeat until O = G. QED.
Common Misuse
- Skipping step 5 (feasibility). Many “exchanges” produce an infeasible solution, invalidating the proof.
- Skipping step 6 (no-worse-than). The exchange must be non-worsening, not merely “different”.
- Stopping after one exchange. A single exchange shows
G[0] = O'[0]; you must iterate. - Picking the wrong index to exchange. Exchanging at the last difference rather than the first often fails because residual structure differs.
- Treating the exchange as a swap of types rather than concrete elements. Exchange a specific element of
Owith a specific element ofG, not “exchange the early-ending one with the late-ending one”.
3. The Cut Property (MST Correctness)
Precise Definition
For a connected, weighted, undirected graph G = (V, E), the cut property states: for any cut (S, V\S) (a partition of vertices into two non-empty sets), the minimum-weight edge crossing the cut belongs to some minimum spanning tree of G. (If the minimum is unique, it belongs to every MST; if tied, at least one MST contains it.)
When Applicable
The cut property is the correctness theorem for greedy MST algorithms — Kruskal’s, Prim’s, Borůvka’s. It also generalizes to matroid theory: the greedy algorithm is correct on a structure iff the structure is a matroid.
Worked Example: Kruskal’s Algorithm
Kruskal sorts edges ascending by weight and adds each edge that doesn’t create a cycle. Correctness via the cut property: when Kruskal adds edge (u, v), the union-find structure tells us u and v are in different components — call them S and V\S (where S is u’s component and everything else, including v’s component, is V\S). Edge (u, v) crosses this cut. Because Kruskal scans edges in ascending order and (u, v) is the first edge (by weight) that crosses this cut without forming a cycle, it is the minimum-weight edge crossing the cut. By the cut property, (u, v) belongs to some MST. By induction, the set of edges Kruskal has added so far is a subset of some MST. After processing all edges, Kruskal’s set of edges is exactly an MST.
(Phase 4’s MST labs cover this in algorithmic detail; this phase’s job is the proof.)
Common Misuse
- Applying the cut property to directed graphs. It only applies to undirected MST.
- Assuming the MST is unique. Tie-breaking matters; multiple MSTs may exist.
- Forgetting connectedness. On a disconnected graph, you compute a minimum spanning forest, not an MST.
4. Loop Invariants (Proof Scaffolding)
Precise Definition
A loop invariant is a property P(state) that holds before the loop, after every iteration, and after the loop terminates. To prove a loop is correct, show:
- Initialization —
Pholds before the first iteration (i.e., on the initial state). - Maintenance — if
Pholds at the start of iterationk, thenPholds at the end of iterationk. - Termination — when the loop exits,
P(combined with the exit condition) implies the desired postcondition.
Loop invariants are the workhorse of proving greedy algorithms whose correctness isn’t a single one-shot exchange argument — they’re scaffolding for “this thing stays true throughout the run”.
Worked Example: Gas Station (LC 134)
Greedy: scan stations once, maintain tank = 0 and start = 0. At station i, tank += gas[i] - cost[i]. If tank < 0, set start = i + 1 and reset tank = 0. Final answer: start if total gas ≥ total cost, else -1.
Loop invariant: at the end of iteration i, tank equals the net fuel accumulated from start to i, and no station in [start, i] (other than possibly start) is a valid starting point.
(Full proof in Lab 04 — Gas Station.)
Common Misuse
- Inventing an invariant after the fact that conveniently equals “the answer is correct”. The invariant must be precisely statable independent of the conclusion.
- Failing to prove maintenance — usually because the invariant is too weak (doesn’t survive one iteration) or too strong (false at initialization).
- Skipping termination — the invariant might hold every iteration but the loop might not terminate at all, or might terminate in a state that doesn’t imply the postcondition.
5. Monovariants (Termination Arguments)
Precise Definition
A monovariant is a quantity that strictly increases (or strictly decreases) with every iteration of an algorithm and is bounded below (or above) by a known value. The existence of a monovariant proves termination: a strictly decreasing integer-valued quantity bounded below by 0 cannot decrease more than its initial value, so the loop runs at most that many iterations.
In greedy proofs, monovariants are also used to prove progress: each iteration makes irreversible progress toward the goal, so we never need to undo a choice.
Worked Example: Jump Game II (LC 45)
Greedy with two pointers: current_end and farthest. Scan; for each index, update farthest = max(farthest, i + nums[i]). When i == current_end, jump: jumps += 1, current_end = farthest.
Monovariant: farthest is non-decreasing across iterations; in fact farthest ≥ i + 1 for all i (otherwise we’d be stuck at an unreachable position, which is impossible if a solution exists). The monovariant guarantees we never need to backtrack — every position contributes to extending farthest, and current_end only ever moves forward.
(Full proof in Lab 02 — Jump Game II.)
Common Misuse
- Confusing “increasing” with “non-decreasing”. A non-decreasing monovariant doesn’t prove termination — the algorithm might loop indefinitely with the quantity stuck. Use strictly increasing/decreasing.
- Using a real-valued monovariant without an explicit lower bound on the rate of change. Real values can decrease toward an infimum without ever reaching it (Zeno’s paradox in algorithm form).
- Treating monovariant as a correctness proof when it’s only a termination proof. Termination + invariant gives correctness; one of them alone does not.
6. Amortized Analysis
Amortized analysis bounds the cost of a sequence of operations by an average per-operation cost, even when individual operations may be expensive. It is essential for proving the cost of greedy data-structure operations — union-find with path compression, dynamic arrays, splay trees — and shows up in Phase 3 and Phase 7. We cover the three classical methods here.
6a. Aggregate Analysis
Bound the total cost of n operations by some function T(n), then divide: amortized cost per operation is T(n) / n. Simple to apply, hardest to derive a tight T(n) for.
Example. A dynamic array (Python list, Java ArrayList) doubles its capacity when full. n push operations cost: n for the actual writes, plus 1 + 2 + 4 + … + n/2 ≤ n for the resizes, total ≤ 2n. Amortized cost per push: 2n / n = O(1).
6b. The Accounting Method
Charge each operation a fixed amount (the “amortized cost”) which may exceed its actual cost. The excess is stored as “credit” on data-structure elements. Expensive operations pay using accumulated credits, never going into debt. If you can maintain “credits ≥ 0” as an invariant, the amortized cost is a valid upper bound.
Example. Dynamic array push: charge 3 per push. Actual cost is 1 for the write; 2 is stored as credit on the just-written element. When the array doubles, each element being moved already has 2 credits on it — exactly enough to pay for the move and the copy of one new element from the old half. Credits never go negative; amortized cost per push is O(1).
6c. The Potential Method
Define a potential function Φ(D) over data-structure states D, with Φ(D₀) = 0 initially and Φ(D) ≥ 0 always. The amortized cost of an operation is actual_cost + ΔΦ. Total amortized cost over n operations is Σ actual_cost + Φ(D_n) - Φ(D₀) ≥ Σ actual_cost, so it’s a valid upper bound.
Example. Dynamic array: let Φ = 2 · size − capacity. After a doubling, Φ = 0. Each push that doesn’t trigger doubling: actual cost 1, ΔΦ = +2, amortized 3. Doubling push: actual cost size + 1, ΔΦ = 2 − size, amortized 3. Constant O(1) amortized per push.
Common Misuse
- Conflating amortized with average-case. Amortized is a worst-case bound on a sequence of operations; average-case is over a probability distribution on inputs. They are not the same.
- Applying amortized bounds to a single operation. A single resize is
O(n); only the average over a sequence isO(1). - Claiming credits go negative in the accounting method — invalidates the bound. Always verify the invariant.
- Using a potential function that can become negative — invalidates the bound;
Φ ≥ 0is required.
7. When Greedy FAILS (Counterexamples)
The skill of greedy is not just knowing when it works but recognizing when it doesn’t before you commit. Memorize these traps.
7a. 0/1 Knapsack ≠ Fractional Knapsack
Fractional knapsack (you can take any fraction of an item): greedy by value-to-weight ratio is optimal. Sort items by v_i / w_i descending; take items in order, taking a fraction of the last item if needed.
0/1 knapsack (each item is take-or-skip): greedy by ratio is not optimal. Counterexample:
| Item | Weight | Value | Ratio |
|---|---|---|---|
| A | 1 | 1.5 | 1.5 |
| B | 2 | 2 | 1.0 |
| C | 2 | 2 | 1.0 |
Capacity = 3. Greedy by ratio takes A (capacity left 2), then B (capacity left 0), total 3.5. Optimum is B + C = 4.
The trap: ratio-greedy is correct under fractional flexibility, fails under integrality. The 0/1 version requires DP — see Phase 5 Lab 03.
7b. Coin Change With Arbitrary Denominations
For US coins {1, 5, 10, 25}, greedy (largest first) is optimal. For arbitrary denominations like {1, 3, 4} with target 6, greedy gives 4 + 1 + 1 = 3 coins; optimum is 3 + 3 = 2 coins. Greedy fails. Fall back to DP — see Lab 06 — Greedy vs DP for the canonical counterexample analysis.
7c. Scheduling With Weights
Interval scheduling (unweighted): greedy by earliest end time is optimal. Weighted interval scheduling (each interval has a weight, maximize total weight of non-overlapping picks): greedy fails. DP with binary-search predecessor pointer is correct, O(N log N).
7d. “Greedy By Farthest Reach” In Reachability
Some problems look like jump game but are not: e.g., on a weighted graph, “earliest arrival” by greedy farthest-reach is not Dijkstra; it requires the priority-queue refinement.
Common Misuse
- Trusting “looks like jump game” reasoning. Always verify the greedy choice property formally. The signal is “I have an exchange argument” not “the example worked”.
- Failing the cross-test. Try N=2 and N=3 by hand. Try a hostile counterexample. Try the input where all elements are equal, all are distinct, all are decreasing.
- Forgetting the DP fallback. Many problems are “greedy if X, DP otherwise.” Know which side you are on before coding.
Greedy-Vs-DP Decision Flowchart
When a problem looks optimization-flavored, this flowchart determines whether to attempt greedy or jump straight to DP.
START → Is the problem an optimization (max / min) or counting?
│
├── No (search / decision / construction) → Greedy candidate; check exchange argument.
│
└── Yes
│
├── Can I sort the input by a single criterion (deadline, weight, ratio)
│ such that processing in that order has the greedy choice property?
│ │
│ ├── Yes — and I can write a 4-step exchange argument in <90s →
│ │ GREEDY. Code in <5 minutes.
│ │
│ └── No / not sure →
│ Does the optimal answer at step i depend on choices at i+1, i+2, …?
│ │
│ ├── Yes → DP. State = (position, accumulated state needed for future).
│ │ See Phase 5.
│ │
│ └── No / unclear → Try greedy with a small N=3, N=4 stress-test against
│ brute force. If matches → write proof attempt.
│ If diverges → DP.
│
└── Try the canonical counterexamples for this problem class:
- Fractional vs 0/1 (knapsack)
- Sorted vs arbitrary (coin change)
- Unweighted vs weighted (scheduling)
If any counterexample defeats your greedy → DP.
The flowchart’s discipline: never commit to greedy without an exchange argument. The 90-second time-box for the proof attempt is exactly the safety check that prevents the failure mode of confidently submitting wrong greedy code.
Common Greedy Bugs
A taxonomy. Each one shows up in at least 30% of submitted greedy solutions in mock interviews.
- Claiming greedy without proof. “I’ll sort by end time and pick” with no exchange argument. The interviewer asks “why?” and the candidate either restates the algorithm or stalls. Fix: practice the 4-step exchange recipe so it comes out automatically when you propose the algorithm.
- Wrong sort key. Sorting by start time when end time is correct (interval scheduling); sorting by weight when ratio is correct (fractional knapsack). Fix: before coding, do an exchange argument with each candidate sort key on a hand-crafted small example. The wrong key fails the exchange step quickly.
- Ignoring counterexamples. “My algorithm passes the examples, ship it.” Fix: always run the algorithm against brute force on N=2, N=3, N=4 random inputs before committing.
- Assuming local optimum = global optimum without justification. “At each step pick the smallest” works for some problems and fails for others. Fix: the greedy choice property is not free; it must be proven for every problem.
- Confusing fractional and integer regimes. Ratio-greedy on 0/1 knapsack; “take half of this” interpretation in problems where items are atomic. Fix: read the integrality constraint before coding.
- Off-by-one in “earliest end time” when ties exist. Ties must be broken consistently — typically by start time ascending — and the proof must handle ties. Fix: state the tie-breaker explicitly and verify the exchange argument handles it.
- Greedy with backtracking masquerading as greedy. Some “greedy” solutions secretly maintain a stack and pop on conflict (e.g., remove-k-digits LC 402, candy distribution LC 135). The pure greedy claim doesn’t apply; the algorithm is greedy plus an undo step. Fix: if your algorithm has an
if conflict: undo, it’s not pure greedy; the proof must cover the undo logic. - Mixing up the proof with the algorithm. “Earliest deadline first works because earliest deadline first is the best choice” is circular. Fix: the proof must reference an exchange or invariant or cut, not restate the algorithm.
- Not handling the empty / size-1 case in scan-based greedy. Fix: explicit guard at the start.
- Using a heap when sorting suffices (or vice versa). Heap is for online greedy where future inputs aren’t yet visible (e.g., merge K sorted lists, scheduling with deadlines streaming in). Sort is for offline where all inputs are known. Fix: ask “is the input streamed or batched?” up front.
Mastery Checklist
Before exiting this phase, verify all of these:
- You can recognize a greedy candidate within 2 minutes by spotting the greedy choice property signal.
- You can produce a 4-step exchange argument for any greedy claim within 90 seconds, out loud, without writing code first.
- You can cite the cut property and apply it to prove Kruskal’s / Prim’s correctness from scratch.
- You can articulate the difference between an invariant, a monovariant, and an exchange argument — and pick the right tool for a given proof.
- You can perform aggregate, accounting, and potential-method amortized analyses on a dynamic-array example, in <5 minutes total.
- You can name and articulate three canonical greedy counterexamples (0/1 vs fractional knapsack; coin change {1,3,4}; weighted interval scheduling).
- You can implement interval scheduling, jump game II, task scheduler, gas station, and Huffman coding with full proofs in <90 minutes total.
- You can articulate the greedy-vs-DP decision in <30 seconds for any optimization problem.
- You catch yourself before committing to a greedy without an exchange argument — every time.
Exit Criteria
You may move to Phase 7 (Competitive Programming Acceleration) when all of the following are true:
- You have completed all six labs in this phase, with each lab’s mastery criteria checked off.
- You have solved at least 25 unaided greedy problems from LeetCode (mix of Medium and Hard) and reviewed each via REVIEW_TEMPLATE.md. On at least 20 of them, you wrote the exchange argument or invariant in the review before peeking at any solution.
- Your unaided success rate on Medium-Hard greedy problems is ≥ 65%.
- In a mock interview (phase-11-mock-interviews/), you correctly identify greedy-applicability within 2 minutes for at least 7 of 10 greedy-flavored problems and produce an exchange argument within 90 seconds for at least 6 of 10. You correctly reject greedy in favor of DP on at least 2 of the 10, citing a counterexample.
- You have never in this phase shipped a greedy solution without a written proof. This is the single discipline of Phase 6, and skipping it is a phase-failure.
If any of these fails, do another 15–20 greedy problems before moving on. Skipping this gate produces engineers who pattern-match “looks like greedy” and ship wrong code under deadline pressure — exactly the failure mode that gets candidates rejected at staff level.
Labs
Hands-on practice. Each lab follows the strict 22-section format. Every lab’s Correctness Argument section contains an explicit exchange argument or invariant + monovariant proof. This is the whole point of Phase 6.
- Lab 01 — Interval Scheduling (Activity Selection) — canonical exchange argument
- Lab 02 — Jump Game II — greedy reach + monovariant
- Lab 03 — Task Scheduler With Cooldown — greedy + math formula
- Lab 04 — Gas Station — greedy + invariant proof
- Lab 05 — Huffman Coding — greedy via heap + exchange-argument optimality
- Lab 06 — Greedy Vs DP (Coin Change Counterexample) — when greedy fails and DP is required
← Phase 5: Dynamic Programming · Phase 7: Competitive Programming → · Back to Top
Lab 01 — Interval Scheduling (Activity Selection)
Goal
Master the canonical greedy problem: maximum non-overlapping interval selection. Internalize the earliest-end-time-first greedy and produce its exchange argument out loud, without help, in under 90 seconds.
Background
Interval scheduling is the prototype greedy problem in every algorithms textbook because it has the cleanest exchange argument and the largest gap between “intuitive but wrong” choices (earliest start, shortest duration, fewest conflicts) and the one correct one (earliest end). Mastering this lab is mastering the discipline of proof-before-code for the rest of Phase 6. The same exchange-argument template recurs in LC 452 (minimum arrows to burst balloons), LC 1353 (maximum events attended), and dozens of variants.
Interview Context
A staff-level interviewer at FAANG-tier companies will not accept “I’ll sort by end time” without a justification. The signal they’re testing for is: can this candidate distinguish between intuition and proof? The exchange argument, delivered cleanly in 60–90 seconds before code, is the strongest possible signal. Conversely, candidates who code first and then mumble “earliest end is correct because… well… it just is” almost always fail the question even when their code passes the tests.
Problem Statement
Given n activities, each with a start time s_i and an end time e_i, select the maximum-cardinality subset of activities that are pairwise non-overlapping (an activity ending at time t does not overlap with one starting at time t — boundary touching is allowed). Return the size of that subset, or the subset itself if requested.
LeetCode reference: LC 435 — Non-overlapping Intervals (return the count of intervals to remove to make the rest non-overlapping; equivalent to n − maxNonOverlapping).
Constraints
1 ≤ n ≤ 10^5−5 · 10^4 ≤ s_i < e_i ≤ 5 · 10^4- Boundary contact (an interval ending at
tand another starting att) does NOT count as overlap. - Time complexity must be
O(n log n); spaceO(1)extra (sort in place).
Clarifying Questions
- “Are intervals open, closed, or half-open?” — LC convention is half-open:
[s, e)is a common interpretation; ask the interviewer. - “Is an interval where
s == e(zero-duration) allowed?” — usually yes, treat as a single-point interval. - “Should I return the maximum non-overlapping count or the minimum to remove?” — LC 435 asks the latter; the answer is
n − maxNonOverlapping. - “Can the intervals be unsorted? Are they ever pre-sorted?” — assume unsorted unless stated; sort is part of the algorithm.
- “Are there ties on end time?” — yes; tie-break by start time ascending so the algorithm is deterministic and the proof handles ties cleanly.
Examples
[[1,2], [2,3], [3,4], [1,3]]→ max non-overlapping = 3 (pick[1,2],[2,3],[3,4]); remove count = 1.[[1,2], [1,2], [1,2]]→ max non-overlapping = 1; remove count = 2.[[1,2], [2,3]]→ max non-overlapping = 2; remove count = 0 (boundary contact does not overlap).[]→ 0.
Initial Brute Force
Try every subset; for each, check pairwise non-overlap; return the size of the largest valid subset.
from itertools import combinations
def max_non_overlap_brute(intervals):
n = len(intervals)
best = 0
for r in range(n + 1):
for subset in combinations(intervals, r):
ok = all(subset[i][1] <= subset[j][0]
for i in range(len(subset))
for j in range(len(subset))
if subset[i][0] < subset[j][0])
if ok:
best = max(best, len(subset))
return best
Brute Force Complexity
Time O(2^n · n^2) — every subset checked pairwise. Space O(n) for combinations. Infeasible at n = 25. Useful only as a stress-test oracle for the greedy on n ≤ 12.
Optimization Path
- Brute force (above) — establishes correctness baseline.
- DP — sort by end time;
dp[i]= max non-overlapping using intervals0..i. Transition:dp[i] = max(dp[i-1], 1 + dp[predecessor(i)])wherepredecessor(i)is the latestj < iwithe_j ≤ s_i. TimeO(n log n)with binary search. SpaceO(n). This is weighted interval scheduling’s structure when we drop weights. - Greedy — sort by end time, scan once, pick whenever compatible.
O(n log n)time,O(1)extra space. Optimality from the exchange argument below.
The DP step is worth deriving even though greedy beats it, because the moment intervals carry weights, greedy fails and DP is the only correct approach. Keep DP in your back pocket.
Final Expected Approach
Sort intervals by end time ascending (tie-break by start ascending). Maintain last_end = -∞. For each interval [s, e] in sorted order, if s ≥ last_end, accept it (count += 1, last_end = e); else skip. Return count.
For LC 435 the answer is n − count.
Data Structures Used
- A sorted list / array of intervals.
- One scalar
last_end. - One counter
count.
No heap, no DSU, no DP table. The algorithm is sort-and-scan.
Correctness Argument
This is the section the rest of Phase 6 hangs on. We use the canonical 4-step exchange argument.
Setup. Let G = g_1, g_2, …, g_k be the greedy solution: intervals sorted by end time, picked greedily. Let O = o_1, o_2, …, o_m be any optimal solution, also written in sorted-by-end-time order (we can always re-sort an optimal solution; non-overlap is preserved). Assume for contradiction m > k (O is strictly better than G); we will derive a contradiction by showing we can transform O into G step by step without decreasing its size — implying m ≤ k.
Step 1 — Locate the first divergence. Let i be the smallest index where g_i ≠ o_i. By construction, g_1 = o_1, …, g_{i-1} = o_{i-1}.
Step 2 — Compare end times at the divergence. Greedy picks the interval with the earliest end time among those compatible with g_1, …, g_{i-1} — equivalently, those compatible with o_1, …, o_{i-1}. So g_i.end ≤ o_i.end (with equality possible if there is a tie and o_i happens to be the tied alternative).
Step 3 — Exchange. Replace o_i with g_i in O, producing O' = o_1, …, o_{i-1}, g_i, o_{i+1}, …, o_m. We must verify (a) feasibility and (b) size.
- Feasibility.
g_iis compatible witho_{i-1} = g_{i-1}because greedy ensured this.g_iis compatible witho_{i+1}:o_{i+1}.start ≥ o_i.end ≥ g_i.end, sog_iends no later thano_idid, and the rest ofOwas already non-overlapping witho_i’s end. SoO'is feasible. - Size.
|O'| = |O|(we replaced one interval with one interval).
Step 4 — Iterate. O' agrees with G on positions 1..i. Repeat the argument on O' versus G to find the next divergence. After at most min(k, m) exchanges, the resulting solution agrees with G on the first min(k, m) positions.
Conclude. If m > k, after k exchanges the transformed O looks like g_1, …, g_k, o_{k+1}, …, o_m. But greedy stopped at g_k, which means there was no interval compatible with g_1, …, g_k. Therefore o_{k+1} cannot exist — contradiction. So m ≤ k, i.e., G is at least as large as any feasible solution. G is optimal. QED.
Complexity
- Time
O(n log n)for the sort,O(n)for the scan; totalO(n log n). - Space
O(1)extra beyond the input (sort in place);O(log n)for sort recursion in some implementations.
Implementation Requirements
- Sort key:
(end, start)— explicit tie-break. - Boundary semantics:
s_next ≥ e_previs “compatible” (touching allowed). The interview’s clarifying question about open/closed determines this; default to half-open. - Handle empty input (
n = 0→ return 0). - Single pass, no nested loop.
Tests
def test_interval_scheduling():
# canonical
assert max_non_overlap([[1,2],[2,3],[3,4],[1,3]]) == 3
# all overlapping
assert max_non_overlap([[1,2],[1,2],[1,2]]) == 1
# no overlap
assert max_non_overlap([[1,2],[2,3]]) == 2
# empty
assert max_non_overlap([]) == 0
# single
assert max_non_overlap([[5,10]]) == 1
# tie on end time
assert max_non_overlap([[1,3],[2,3],[3,4]]) == 2 # one of {[1,3],[2,3]} + [3,4]
# negative coords
assert max_non_overlap([[-5,0],[0,5],[5,10]]) == 3
# nested
assert max_non_overlap([[1,10],[2,3],[4,5],[6,7]]) == 3
Stress test: generate random n ≤ 12 and compare greedy to brute force on 1000 trials.
Follow-up Questions
- Weighted version (each interval has weight
w_i; maximize total weight of chosen non-overlapping). Greedy fails. Solution: DP with binary search predecessor pointer,O(n log n). - Online / streaming version (intervals arrive one at a time, decide accept/reject immediately, no recall). Different problem class — competitive ratio analysis.
- K-machine extension (
kparallel resources; each interval scheduled on any one of them; maximize total scheduled). Greedy withk“last_end” trackers + min-heap.
Product Extension
Calendar systems (Google Calendar’s “find a meeting time” feature, AWS spot-instance scheduling, ad slot allocation): the unweighted version is the prototype, and weighted variants are exactly what production schedulers solve.
Language / Runtime Follow-ups
- Python:
sorted(intervals, key=lambda x: (x[1], x[0]))— stable sort, tuple comparison handles tie-break for free. - Java:
Arrays.sort(intervals, (a, b) -> a[1] != b[1] ? Integer.compare(a[1], b[1]) : Integer.compare(a[0], b[0])). UseInteger.compareto avoid integer-overflow ona[1] - b[1]. - Go:
sort.Slice(intervals, func(i, j int) bool { if intervals[i][1] != intervals[j][1] { return intervals[i][1] < intervals[j][1] }; return intervals[i][0] < intervals[j][0] }). - C++:
std::sortwith a lambda; preferstd::tie(a[1], a[0]) < std::tie(b[1], b[0])for clarity. - JS/TS:
intervals.sort((a, b) => a[1] - b[1] || a[0] - b[0])— the|| 0falls through to start-comparison on tie.
Common Bugs
- Sorting by start time instead of end time (intuitive, wrong).
- Wrong tie-break (e.g., descending start) breaking determinism in the proof.
- Using
>instead of≥for compatibility, rejecting touching intervals. - Forgetting to update
last_endafter accepting an interval. - Returning the count when the question asked for the removal count (LC 435), or vice versa.
Debugging Strategy
If your greedy disagrees with brute force on some n ≤ 12 input:
- Print both solutions side by side.
- Find the first interval where they differ.
- Manually run the exchange argument step on that point — does the swap preserve feasibility? If not, your sort key or tie-break is wrong.
- If the exchange preserves feasibility but your code didn’t pick
g_i, your scan logic has an off-by-one in the compatibility check.
Mastery Criteria
- You can write the algorithm in <5 minutes from a blank screen, including the tie-break in the sort key.
- You can deliver the 4-step exchange argument out loud in <90 seconds, without notes.
- You can extend to LC 452 (minimum arrows to burst balloons) by recognizing it as the same problem with renaming, in <2 minutes.
- You can articulate why the weighted version requires DP, in <30 seconds.
- You correctly reject the wrong sort keys (earliest start, shortest duration, fewest conflicts) by giving a concrete counterexample for each, in <2 minutes total.
← Phase 6 README · Lab 02 — Jump Game II →
Lab 02 — Jump Game II
Goal
Internalize the greedy reach pattern: a single forward scan with two pointers (current_end, farthest) producing the minimum number of jumps. Prove correctness via a loop invariant + monovariant pair.
Background
Jump Game II (LC 45) is the canonical “greedy with monovariant” problem. It looks like BFS — and indeed, the greedy is BFS in disguise — but the BFS is implemented in O(n) time and O(1) space because the levels are contiguous index ranges. The proof is a two-part argument: an invariant (“at any point, the farthest position reachable in j jumps equals current_end after the j-th jump”) plus a monovariant (farthest non-decreasing, current_end strictly increasing per jump).
Interview Context
This problem (or its variants — LC 1306, LC 1326, LC 1024) appears at every FAANG-tier interview. The wrong solution is “BFS with a queue, mark visited” (O(n²) time, O(n) space). The right solution looks too simple to be correct unless you have the proof — which is why the interviewer asks for it.
Problem Statement
Given a non-negative integer array nums where nums[i] is the maximum jump length from index i, return the minimum number of jumps to reach the last index, starting from index 0. Assume the last index is always reachable.
Constraints
1 ≤ n ≤ 10^40 ≤ nums[i] ≤ 1000- Last index is always reachable (no
-1case).
Clarifying Questions
- “Is
0a valid value at intermediate positions, and does it mean we get stuck?” — yes; but the problem guarantees reachability, so we won’t actually get stuck. - “Can
n == 1? Then the answer is 0 (already at the end).” — yes, edge case to handle. - “Do I return the count of jumps or the path?” — count.
- “Are negative jumps allowed?” — no; non-negative only.
Examples
[2,3,1,1,4]→ 2 (0 → 1 → 4).[2,3,0,1,4]→ 2.[1]→ 0 (already at end).[1,1,1,1]→ 3.[5,1,1,1,1]→ 1 (one jump from 0 to 4).
Initial Brute Force
Recursive DP with memoization: dp[i] = min jumps from index i to n-1.
from functools import lru_cache
def jumps_brute(nums):
n = len(nums)
@lru_cache(maxsize=None)
def f(i):
if i >= n - 1: return 0
if nums[i] == 0: return float('inf')
return 1 + min(f(i + j) for j in range(1, nums[i] + 1) if i + j < n)
return f(0)
Brute Force Complexity
Time O(n · max(nums)) — each cell tried against up to max(nums) next cells. At max(nums) = 1000, that’s 10^7 — borderline. Space O(n) for memo + recursion. The greedy is O(n) time and O(1) space.
Optimization Path
- Recursive DP (above) — clear correctness.
- Iterative DP —
dp[i]filled left to right;dp[i] = 1 + min(dp[i+j]). Same complexity. - BFS — view jumps as edges; level number = answer.
O(n²)worst case if naively implemented (revisiting);O(n)if you track the boundary. The boundary tracking is exactly the greedy. - Greedy with two pointers —
O(n)time,O(1)space. The BFS layers are contiguous ranges[L, R]; we process the range and compute the next range’s right endpoint asmax(i + nums[i])fori ∈ [L, R].
Final Expected Approach
def jump(nums):
n = len(nums)
if n <= 1: return 0
jumps = 0
current_end = 0
farthest = 0
for i in range(n - 1): # don't iterate past n-1
farthest = max(farthest, i + nums[i])
if i == current_end:
jumps += 1
current_end = farthest
if current_end >= n - 1:
break
return jumps
Two pointers: current_end is the right boundary of the BFS layer we’re currently processing; farthest is the right boundary of the next BFS layer being assembled.
Data Structures Used
- Three integers:
jumps,current_end,farthest. No arrays, no queue, no recursion. The simplicity is the point.
Correctness Argument
We prove: at termination, jumps equals the minimum number of jumps to reach index n - 1.
Setup. Define level(j) = the set of indices reachable from 0 in exactly j jumps and not in fewer. By induction on j: level(0) = {0}; level(j+1) = {i + k : i ∈ level(j), 1 ≤ k ≤ nums[i]} − ⋃_{j' ≤ j} level(j'). By a simple induction, level(j+1) is a contiguous range of indices [L_{j+1}, R_{j+1}] immediately to the right of R_j. (Proof: level(j) is contiguous by induction; the union of [i, i + nums[i]] over i in a contiguous range is itself a contiguous range; subtracting earlier levels removes a contiguous prefix.)
Loop invariant. At the top of iteration i:
current_end = R_{jumps}— i.e., the right boundary of the layer reached injumpsjumps so far.farthest = max_{k ≤ i, k ≤ current_end} (k + nums[k])— the farthest reachable from any index processed so far in the current layer.
Initialization. Before the loop: jumps = 0, current_end = 0, farthest = 0. Layer 0 is {0} with R_0 = 0 ✓.
Maintenance. At iteration i:
- We update
farthest = max(farthest, i + nums[i]). Ifi ≤ current_end, this maintains invariant (2). - If
i == current_end, we’ve finished processing the current layer. We “jump”:jumps += 1,current_end = farthest. By invariant (2),farthest = R_{jumps_old + 1}= right boundary of the next layer. So invariant (1) is restored withjumps_new = jumps_old + 1.
Monovariant. farthest is non-decreasing across the loop (each iteration takes a max). At each “jump” event, current_end strictly increases (otherwise the last index isn’t reachable, contradicting the problem’s guarantee). The loop runs n - 1 iterations and the number of “jump” events is bounded above by n - 1, so the algorithm terminates and produces a finite jumps value.
Termination + correctness. The loop terminates after n - 1 iterations (or earlier if current_end ≥ n - 1). At that point, current_end ≥ n - 1 (because the last index is reachable; if it weren’t, we’d have current_end < n - 1 and farthest = current_end — no progress — but the problem guarantees reachability, so farthest > current_end whenever current_end < n - 1). Therefore jumps is exactly the layer index of n - 1 in the BFS — i.e., the minimum jump count. QED.
The two key proof devices: the invariant that current_end tracks layer boundaries, and the monovariant that farthest is non-decreasing. Together they give correctness; the monovariant alone gives termination.
Complexity
- Time
O(n)— single forward scan. - Space
O(1)— three integers.
Implementation Requirements
- Loop bound
i < n - 1, noti < n. Iterating ton - 1would cause an extra spurious jump count whencurrent_end == n - 1. if i == current_end:— the trigger for layer transition. The check happens afterfarthestis updated.if current_end >= n - 1: break— early exit.- Handle
n == 1as a special case (answer 0) before the loop.
Tests
def test_jump():
assert jump([2,3,1,1,4]) == 2
assert jump([2,3,0,1,4]) == 2
assert jump([1]) == 0
assert jump([1,1,1,1]) == 3
assert jump([5,1,1,1,1]) == 1
assert jump([1,2]) == 1
assert jump([0]) == 0 # n=1, no jumps needed
# large jump from start
assert jump([100, 1, 1, 1, 1, 1, 1]) == 1
Stress-test versus the recursive DP for n ≤ 20.
Follow-up Questions
- LC 55 (Jump Game I) — can we reach the end?
farthest ≥ iinvariant suffices. - LC 1306 (Jump Game III) — bidirectional jumps with fixed offsets; not greedy, BFS.
- LC 1340 (Jump Game V) — descent-only, weighted; DP territory.
- Min jumps with cost — DP, not greedy. The cost asymmetry breaks the layer-contiguity argument.
Product Extension
Network packet routing: minimum hops to reach a destination when each node has a maximum forward-reach. Game pathfinding when movement primitives have variable range.
Language / Runtime Follow-ups
- Python: as shown above. Beware the off-by-one on
range(n - 1). - Java:
int n = nums.length; if (n <= 1) return 0; int jumps = 0, end = 0, far = 0; for (int i = 0; i < n - 1; i++) { far = Math.max(far, i + nums[i]); if (i == end) { jumps++; end = far; if (end >= n - 1) break; } }. - Go: identical structure; use
if i+nums[i] > far { far = i + nums[i] }to avoidmath.Maxfloat overhead. - C++: same; use
std::max. - JS/TS: same;
Math.max(...). Watch fornums.lengthre-evaluation cost in tight loops on engines that don’t hoist.
Common Bugs
- Iterating
for i in range(n)and double-counting the last jump. - Updating
current_endbefore the bookkeeping checki == current_end. - Forgetting the
n == 1early return. - Using BFS with a queue when the contiguous-range observation makes it
O(1)space. - Confusing
farthest(assembling next layer) withcurrent_end(current layer’s right edge) — labelling them consistently is essential.
Debugging Strategy
- Print
(i, current_end, farthest, jumps)at each iteration on a small input. The state should evolve predictably:farthestrises,current_endjumps tofarthestexactly whenicatches up. - If your output is one too many: check the loop bound (
n - 1notn). - If your output is one too few: check that you increment
jumpsat the layer boundary, not after the last iteration.
Mastery Criteria
- You can write the algorithm in <5 minutes from blank.
- You can articulate the BFS-layer interpretation in <30 seconds.
- You can state the loop invariant precisely and run through initialization → maintenance → termination in <2 minutes.
- You can name the monovariant and explain why it implies termination, in <30 seconds.
- You can extend to LC 55 (Jump Game I) in <3 minutes by simplifying the same template.
← Lab 01 — Interval Scheduling · Phase 6 README · Lab 03 — Task Scheduler →
Lab 03 — Task Scheduler With Cooldown
Goal
Master the frequency-greedy pattern: schedule a stream of tasks with a per-type cooldown, minimizing total CPU cycles. Derive the closed-form formula (maxFreq - 1) * (n + 1) + countMax, prove its optimality via an exchange argument, and recognize when the formula breaks (when actual task count exceeds the formula).
Background
LC 621 is the canonical “greedy + counting formula” problem. The exchange argument is short but subtle: the most-frequent task type must be scheduled in the densest possible pattern (every n+1 slots), and any optimal schedule that does not schedule the most-frequent type maximally densely can be modified to do so without increasing the total cycle count. This produces the formula. The skill being tested is recognition of the pattern, derivation of the formula from first principles, and the discipline to also show the alternative max-heap simulation that handles the same problem operationally.
Interview Context
Asked at Amazon, Google, Meta. The interviewer is testing whether you can derive a formula or whether you reach for a heap reflexively. Both solutions are accepted; the formula version with proof is the stronger signal. Watch for the follow-up: “What if a new task type can arrive mid-execution?” — that breaks the formula and forces the heap.
Problem Statement
Given an array of tasks (uppercase letters) and an integer n (cooldown), schedule the tasks so that the same task type is separated by at least n other slots (the slots can be idle if necessary). Return the minimum number of CPU cycles to finish all tasks.
LeetCode reference: LC 621 — Task Scheduler.
Constraints
1 ≤ |tasks| ≤ 10^4tasks[i]is uppercase English letter (so at most 26 distinct types).0 ≤ n ≤ 100.
Clarifying Questions
- “Does the schedule have to be returned, or just the cycle count?” — count only.
- “Can multiple tasks of different types execute in the same cycle?” — no, one task per cycle (or one idle).
- “If
n == 0, can same-type tasks run back to back?” — yes; answer is justlen(tasks). - “Are tasks pre-sorted or arrive in any order?” — order doesn’t matter; only the frequency vector matters.
- “Can a single task type appear more times than total cycles?” — no; the formula handles this naturally.
Examples
tasks = ["A","A","A","B","B","B"], n = 2→ 8. Schedule:A B _ A B _ A B.tasks = ["A","A","A","B","B","B"], n = 0→ 6.tasks = ["A","A","A","A","A","A","B","C","D","E","F","G"], n = 2→ 16. Formula:(6-1)*(2+1) + 1 = 16.tasks = ["A","B","C","D","E","A","B","C","D","E"], n = 4→ 10. Formula gives(2-1)*5 + 5 = 10; actual = 10. Tight.tasks = ["A","B","C","D","E","F"], n = 100→ 6. Formula gives(1-1)*101 + 6 = 6sincemaxFreq = 1.
Initial Brute Force
Simulate. At each cycle, pick any task type with cooldown elapsed and remaining count > 0; if multiple, pick whichever (the simulation is correct under any tiebreaker, but optimal requires the highest-frequency one). If none available, idle. Repeat until all tasks done.
Brute Force Complexity
O(T) where T is the answer. Tight bound T ≤ |tasks| * (n + 1), so O(|tasks| * n) worst case. Acceptable but slower than the formula.
Optimization Path
- Brute simulation — easy to write, slow.
- Max-heap simulation — at each cycle, pop highest-count types, decrement, push back to a temporary “cooling” queue with cooldown timestamp. After
n + 1cycles or a complete pass, restore from cooling queue. - Closed-form formula — derive from the structure of an optimal schedule.
O(|tasks|)time,O(1)space.
Final Expected Approach
from collections import Counter
def least_interval(tasks, n):
cnt = Counter(tasks)
max_freq = max(cnt.values())
count_max = sum(1 for v in cnt.values() if v == max_freq)
return max(len(tasks), (max_freq - 1) * (n + 1) + count_max)
The max(len(tasks), …) handles the case where the formula gives less than total tasks — i.e., when there are so many distinct task types that we never need to idle.
Data Structures Used
- A frequency counter (26-entry array or
Counter). - Two integers:
max_freq,count_max.
Correctness Argument
We prove the formula T = max(|tasks|, (max_freq - 1)(n + 1) + count_max) is optimal.
Setup. Let M = max_freq and K = count_max (number of types tied for the maximum frequency).
Lower bound (no schedule can do better than T). Consider any task type with frequency M. The M instances of this type must be separated by at least n other slots, so the schedule spans at least (M - 1)(n + 1) + 1 cycles for one such type. If K types are all tied at frequency M, then in cycles 1, n+2, 2n+3, … we must place an instance of each tied type — actually, we must place all K of them in one of the n+1-slot windows. The last window (after the last instance of the most-frequent type’s predecessor) has only the final instances of each tied type, contributing exactly K cycles after (M - 1)(n + 1). So T ≥ (M - 1)(n + 1) + K.
Also trivially T ≥ |tasks| (every task runs in its own cycle).
So T ≥ max(|tasks|, (M - 1)(n + 1) + K).
Upper bound (the formula’s value is achievable). We construct a schedule of length exactly (M - 1)(n + 1) + K (or |tasks| if larger) and show it is feasible.
-
Case 1:
(M - 1)(n + 1) + K ≥ |tasks|. Lay outM − 1complete frames ofn + 1slots each, followed by a final frame ofKslots. In each frame, slotj(for0 ≤ j < K) is reserved for thej-th most-frequent task type. The remainingn + 1 − Kslots in each frame are filled by other task types in any order; if there aren’t enough non-cooldown candidates, idle. Exchange argument step: suppose an optimal schedule does not place the most-frequent task at slots0, n+1, 2(n+1), …of consecutive frames. Then there is a slotiwhere it could have been placed but wasn’t. Swap it with whatever is there; cooldown is preserved (we are moving an instance to a slot that’sn + 1away from the previous instance, which is within bounds); other tasks are not constrained by this swap. Iterate. The result is the formula schedule, with the same length. -
Case 2:
(M - 1)(n + 1) + K < |tasks|. Then there are more total tasks than the formula’s slot count; we have so much variety that no idle is needed. Schedule any feasible permutation; total cycles =|tasks|. This is achievable because at each cycle we have at least 26 distinct types to choose from (modulo cooldowns), and the cooldown constraint cannot exceednslots, which is dominated by the diversity.
In both cases, the constructed schedule’s length matches the lower bound. Optimal. QED.
The exchange argument is the crucial step: it converts “the formula is one possible schedule’s length” into “no schedule is shorter.” Without the exchange, you have only an existence claim.
Complexity
- Time
O(|tasks|)— single pass to compute frequencies. - Space
O(1)— at most 26 distinct task types, frequency dictionary is constant-bounded.
Implementation Requirements
- Use
Counteror a 26-entry array. - Compute
max_freqandcount_maxin one pass. - Return
max(len(tasks), (max_freq - 1) * (n + 1) + count_max)— do not skip themax.
Tests
def test_least_interval():
assert least_interval(["A","A","A","B","B","B"], 2) == 8
assert least_interval(["A","A","A","B","B","B"], 0) == 6
assert least_interval(["A","A","A","A","A","A","B","C","D","E","F","G"], 2) == 16
assert least_interval(["A","B","C","D","E","A","B","C","D","E"], 4) == 10
assert least_interval(["A","B","C","D","E","F"], 100) == 6
assert least_interval(["A"], 5) == 1
assert least_interval(["A","A"], 0) == 2
assert least_interval(["A","A","A","A"], 3) == 13 # (4-1)*4 + 1 = 13
Follow-up Questions
- Streaming tasks. New tasks arrive mid-execution; formula no longer applies because frequencies change. Use the max-heap simulation.
- Different cooldowns per task type. Heap with per-type cooldown tracker.
- Print the actual schedule. Heap simulation produces a schedule; the formula does not directly give one (you’d reconstruct from the proof).
- What if
ncan be huge (n = 10^9)? Same formula; constant-time arithmetic.
Product Extension
OS task scheduling with cooldown (e.g., a process that touched a hot resource must wait n cycles before re-touching). API rate limiting at the user level. Workout-program scheduling with muscle-group recovery windows.
Language / Runtime Follow-ups
- Python:
Counter(tasks)isO(|tasks|). - Java:
int[26]array indexed byc - 'A'. Faster thanHashMap<Character, Integer>for the constant-alphabet case. - Go:
[26]intworks the same. - C++:
std::array<int, 26>;std::max_elementformax_freq. - JS/TS:
Map<string, number>or 26-entry array; the latter is faster.
Common Bugs
- Forgetting
max(len(tasks), formula)— fails on inputs with many distinct task types. - Using
count_max = 0when there’s only one max-frequency type (should be 1). - Off-by-one in the formula:
(M - 1) * (n + 1)notM * (n + 1). - Heap simulation: forgetting to push the cooled-down task back, or pushing it back at the wrong cycle.
Debugging Strategy
- For each test case, hand-compute
M,K, and the formula. If formula matches expected, your code is wrong somewhere mechanical. - If formula doesn’t match expected, you have a conceptual error: either
Mis wrong (multi-counting) orKis wrong (counting non-tied types) or themax(|tasks|, …)clamp is missing. - Run brute simulation as a stress oracle for
|tasks| ≤ 20.
Mastery Criteria
- You can derive the formula from first principles in <3 minutes.
- You can deliver the exchange argument out loud in <2 minutes.
- You can write the formula-based solution in <3 minutes.
- You can write the heap-based simulation in <10 minutes when asked for the streaming variant.
-
You can articulate why the
max(|tasks|, formula)clamp is necessary and which case it covers, in <60 seconds.
← Lab 02 — Jump Game II · Phase 6 README · Lab 04 — Gas Station →
Lab 04 — Gas Station
Goal
Master the single-pass invariant greedy: O(n) time, O(1) space, with a non-trivial correctness invariant proving why we can skip ahead instead of retrying every starting station.
Background
LC 134 is the canonical “one-pass with reset” greedy. The naive approach is O(n²): for each candidate starting station, simulate the trip. The greedy collapses this to O(n) via the invariant: if the running tank goes negative at station k starting from station s, then no station in [s, k] can be a valid starting point. Once you see this, the algorithm shrinks to a few lines and the proof is the entire test of skill.
Interview Context
Asked at Google, Bloomberg, Amazon. The candidate who codes O(n²) first then asks “can we do better?” is fine. The candidate who jumps to O(n) without the invariant proof is in danger — interviewers test by asking “why is start = k + 1 correct?” and a candidate without the invariant answers “uh, intuition.” That answer fails staff-level interviews.
Problem Statement
There are n gas stations on a circular route. Station i has gas[i] units of gas; traveling from station i to station i + 1 costs cost[i] units. Starting with an empty tank at some station, find the unique starting station that allows you to complete the full circle, or return -1 if impossible.
LeetCode reference: LC 134 — Gas Station.
Constraints
1 ≤ n ≤ 10^50 ≤ gas[i], cost[i] ≤ 10^4- The solution is unique if it exists.
- Time
O(n), spaceO(1).
Clarifying Questions
- “Is the route guaranteed circular?” — yes; from station
n - 1you go to0. - “Can the answer be ambiguous (multiple valid starts)?” — no, the problem guarantees uniqueness when a solution exists.
- “Can
gas[i]orcost[i]be negative?” — no, both non-negative. - “Should I return the index or the boolean feasibility?” — index, or
-1.
Examples
gas = [1,2,3,4,5], cost = [3,4,5,1,2]→ 3 (start at index 3: tank0 + 4 - 1 = 3, 3 + 5 - 2 = 6, 6 + 1 - 3 = 4, 4 + 2 - 4 = 2, 2 + 3 - 5 = 0).gas = [2,3,4], cost = [3,4,3]→ -1 (total gas = 9, total cost = 10, infeasible).gas = [5], cost = [4]→ 0.gas = [3,1,1], cost = [1,2,2]→ 0.
Initial Brute Force
For each candidate start s, simulate the full trip; return the first s that succeeds.
def can_complete_brute(gas, cost):
n = len(gas)
for s in range(n):
tank = 0
for k in range(n):
i = (s + k) % n
tank += gas[i] - cost[i]
if tank < 0:
break
else:
return s
return -1
Brute Force Complexity
Time O(n²), space O(1). At n = 10^5, n² is 10^{10} — too slow.
Optimization Path
- Brute —
O(n²). - Total-feasibility check — if
sum(gas) < sum(cost), no solution exists; return-1immediately. Reduces wasted work but stillO(n²)worst case. - One-pass with reset —
O(n). The invariant below is the key.
Final Expected Approach
def can_complete_circuit(gas, cost):
if sum(gas) < sum(cost):
return -1
start = 0
tank = 0
for i in range(len(gas)):
tank += gas[i] - cost[i]
if tank < 0:
start = i + 1
tank = 0
return start
Data Structures Used
- Two integers:
start,tank. Plus the inputs.
Correctness Argument
We prove two things: (1) if sum(gas) ≥ sum(cost), the algorithm returns a valid starting index; (2) if sum(gas) < sum(cost), no solution exists.
Part 2 is trivial. Across one full lap, the tank changes by exactly sum(gas) - sum(cost). If this is negative, the tank cannot remain non-negative throughout any lap from any start — so no solution.
Part 1 — the key invariant.
Invariant (key claim): suppose we run the algorithm starting from index
start = sand the runningtankfirst goes negative at indexk(so the partial sumtankafter processing indexkis< 0, but it was≥ 0after processing every index in[s, k - 1]). Then no index in[s, k]can be a valid starting point.
Proof of the key claim. Let T(a, b) = sum(gas[a..b]) - sum(cost[a..b]) be the net fuel from a to b. By assumption, T(s, k - 1) ≥ 0 (we made it past k - 1) and T(s, k) < 0 (we failed at k).
Consider any candidate start s' ∈ [s, k]. To complete the lap from s', we need partial sums T(s', i) ≥ 0 for every i between s' and s' + n - 1 (mod n) — in particular, T(s', k) ≥ 0 (assuming s' ≤ k; otherwise we’d be considering s' = k + 1 which is outside the claim’s range).
But T(s', k) = T(s, k) - T(s, s' - 1) ≤ T(s, k) < 0 (since T(s, s' - 1) ≥ 0 by the assumption that we made it past every index in [s, s' - 1]). So starting from s', the tank goes negative at index k, and s' is not a valid start.
Therefore, after a failure at k, we can safely skip all of [s, k] and resume the search from k + 1. QED for the key claim.
Wrapping up. Each index is visited at most once as part of either a successful prefix or the “reset point.” The algorithm runs n iterations. If sum(gas) ≥ sum(cost), the final start is a valid starting point — because the algorithm has effectively eliminated all other candidates, and the problem guarantees a unique solution when one exists. (Formally: from start to the end of the array, no negative event occurred. The wrap-around portion (from index 0 back to start - 1) accumulates at most sum_total - tank_so_far ≤ sum_total = T_total ≥ 0, but we need the running tank non-negative, which follows from the invariant: every prefix from start is non-negative until end, and the wrap-around is the complement, which by total non-negativity stays non-negative.)
The careful formal completion: since T_total ≥ 0, and T(start, n - 1) ≥ 0, we have T(0, start - 1) = T_total - T(start, n - 1) ≤ T_total, but we need positivity of partial sums. The invariant from each reset proved that no earlier candidate works; combined with uniqueness, start is the unique answer.
Complexity
- Time
O(n)— one pass forsum, one pass for the loop. - Space
O(1).
Implementation Requirements
- Pre-check
sum(gas) < sum(cost)is optional (the algorithm itself returns the rightstarteither way only if a solution exists; the pre-check is cheap and avoids returning a bogus value). - Reset
tank = 0(nottank = gas[i+1] - cost[i+1]) when starting fresh. start = i + 1after failure ati.- The variant where you maintain both running and total in a single pass is also acceptable:
def can_complete_circuit_one_pass(gas, cost):
total = tank = start = 0
for i in range(len(gas)):
diff = gas[i] - cost[i]
total += diff
tank += diff
if tank < 0:
start = i + 1
tank = 0
return start if total >= 0 else -1
Tests
def test_gas_station():
assert can_complete_circuit([1,2,3,4,5], [3,4,5,1,2]) == 3
assert can_complete_circuit([2,3,4], [3,4,3]) == -1
assert can_complete_circuit([5], [4]) == 0
assert can_complete_circuit([3,1,1], [1,2,2]) == 0
assert can_complete_circuit([5,1,2,3,4], [4,4,1,5,1]) == 4
# exact match (zero margin)
assert can_complete_circuit([1,2,3], [3,2,1]) in (0, 1, 2) # one of these
# all zeros
assert can_complete_circuit([0,0,0], [0,0,0]) == 0
# cannot start
assert can_complete_circuit([1,1,1], [2,2,2]) == -1
Stress-test versus brute force for n ≤ 50.
Follow-up Questions
- Find the index where you must idle if the route is infeasible. Slightly different problem; same scan structure.
- Multiple cars on the same circuit. Independent problems per car.
- Variable tank capacity. Now state is two-dimensional; greedy may fail; revert to DP / simulation.
- Two-direction route. Run the greedy in both directions; combine.
Product Extension
Battery-powered EV routing with charging stations of variable wattage and costs. Drone delivery routes with refuel points. Spacecraft trajectory planning with gravity-assist maneuvers (highly idealized).
Language / Runtime Follow-ups
- Python: as shown.
- Java: identical structure;
int total = 0, tank = 0, start = 0;. - Go:
total, tank, start := 0, 0, 0. - C++:
int total = 0, tank = 0, start = 0;. Watch overflow ifgas[i]andcost[i]are at the upper end andn = 10^5:10^4 * 10^5 = 10^9, withinint32range, but borderline; uselong longto be safe. - JS/TS:
let total = 0, tank = 0, start = 0;. JS numbers are 64-bit floats, no overflow worry at this scale.
Common Bugs
- Resetting
tank = gas[i] - cost[i]instead oftank = 0after failure (you’d be double-counting the failure point). - Setting
start = iinstead ofstart = i + 1after failure. - Forgetting the
total < 0 → -1check, returning a bogus index. - Iterating in the wrong direction or two passes when one suffices.
Debugging Strategy
- Print
(i, gas[i] - cost[i], tank, start)at each step. The trajectory should show: tank rises and falls, and on each fall below 0 thestartjumps toi + 1. - If your output is off by one (returns
start - 1orstart + 1), check the assignment in the failure branch. - If you return a
startbut the route is actually infeasible, you missed thetotal < 0gate.
Mastery Criteria
- You can write the algorithm in <4 minutes from blank.
-
You can state the key invariant (“if tank goes negative at
kfrom starts, no station in[s, k]can be a valid start”) in <30 seconds. -
You can prove the invariant (using the partial-sum decomposition
T(s', k) = T(s, k) - T(s, s' - 1)) in <2 minutes, out loud. -
You can articulate why
total < 0 → -1is sufficient and necessary, in <30 seconds. - You can produce the brute-force baseline as a stress-test oracle in <3 minutes when asked.
← Lab 03 — Task Scheduler · Phase 6 README · Lab 05 — Huffman Coding →
Lab 05 — Huffman Coding
Goal
Implement Huffman coding from scratch using a min-heap, and prove its optimality via the canonical exchange argument: in some optimal prefix-free code tree, the two least-frequent symbols are siblings at maximum depth.
Background
Huffman coding is the apex example of “greedy via min-heap” and the most-cited example of greedy optimality in CS curricula. The proof has two non-trivial steps: a swap-to-leaf-depth lemma (any internal node at maximum depth can be assumed to have the two least-frequent symbols), and an induction on the merged tree (the greedy is optimal on n - 1 symbols, and combining the two smallest preserves optimality). Mastering this proof teaches a more sophisticated form of exchange argument than the linear-scan greedy of Labs 1–4.
Interview Context
Huffman is asked occasionally at top-tier interviews — Google, Apple, AWS — usually as an open-ended “design a compression algorithm” or as a follow-up to a lab on heap usage. More commonly, the technique (greedy via min-heap, with optimality proof) appears in adjacent problems: LC 1167 — Minimum Cost to Connect Sticks, LC 23 — Merge K Sorted Lists, and the rope-merging problem. Mastery of Huffman = mastery of the entire family.
Problem Statement
Given a frequency map freq: Symbol -> int over n distinct symbols (n ≥ 2), construct a prefix-free binary code such that the expected code length Σ freq[s] * len(code[s]) is minimized. Return either the code map or the encoding tree.
For interview formulation, often phrased as: “Given n ropes of given lengths, you can merge two ropes at a cost equal to the sum of their lengths. Find the minimum total cost to merge all ropes into one.” (Equivalent to Huffman; rope lengths = frequencies.)
LeetCode reference: LC 1167 — Minimum Cost to Connect Sticks (the rope formulation).
Constraints
2 ≤ n ≤ 10^41 ≤ freq[s] ≤ 10^4- Time
O(n log n); spaceO(n). - Tie-break on equal frequencies: any order is acceptable; the optimal cost is invariant.
Clarifying Questions
- “Should I return the codes or just the cost?” — usually cost (rope formulation); for full Huffman, return the tree or the codes.
- “Are frequencies guaranteed positive?” — yes (zero-frequency symbols don’t need codes).
- “Are there always at least 2 symbols?” — assume yes; with 1 symbol, prefix-free coding is trivially “0” (or empty, depending on definition).
- “Is
n = 0a valid input?” — typically no. - “Should the codes be canonical?” — usually no; any optimal-length code is acceptable.
Examples
- Frequencies:
{a: 5, b: 9, c: 12, d: 13, e: 16, f: 45}. Codes (one valid set):f: 0, c: 100, d: 101, a: 1100, b: 1101, e: 111. Total cost:5*4 + 9*4 + 12*3 + 13*3 + 16*3 + 45*1 = 20 + 36 + 36 + 39 + 48 + 45 = 224. - Ropes
[2, 4, 3]: merge 2+3=5 (cost 5), then 5+4=9 (cost 9), total 14. - Ropes
[1, 8, 3, 5]: merge 1+3=4, merge 4+5=9, merge 9+8=17. Total = 4+9+17=30. Or: 1+3=4, 4+5=9, 8+9=17 → same. Min cost 30.
Initial Brute Force
Try every binary-tree topology over the leaves; compute the weighted external path length; return the minimum-cost tree. Catalan-number many trees → infeasible past n = 10.
# Sketched only — exponential
def huffman_brute(freq):
# enumerate all binary trees with leaves = freq, return min weighted path length
...
Brute Force Complexity
O(C_n) where C_n is the Catalan number — C_{10} ≈ 16800, C_{15} ≈ 9.7M. Useful only as stress-test for n ≤ 8.
Optimization Path
- Brute — exhaustive trees.
- DP on intervals — possible if leaves are ordered (matrix-chain style), but Huffman’s leaves are unordered, so this doesn’t apply.
- Greedy with min-heap —
O(n log n)time,O(n)space. Optimality from the exchange argument below.
Final Expected Approach
import heapq
def huffman_cost(freqs):
heap = list(freqs) # frequencies only, for cost-only variant
heapq.heapify(heap)
total = 0
while len(heap) > 1:
a = heapq.heappop(heap)
b = heapq.heappop(heap)
s = a + b
total += s
heapq.heappush(heap, s)
return total
def huffman_codes(freqs):
# freqs: list of (symbol, freq) tuples
heap = [[f, [[s, ""]]] for s, f in freqs]
heapq.heapify(heap)
while len(heap) > 1:
lo = heapq.heappop(heap)
hi = heapq.heappop(heap)
for pair in lo[1]:
pair[1] = '0' + pair[1]
for pair in hi[1]:
pair[1] = '1' + pair[1]
heapq.heappush(heap, [lo[0] + hi[0], lo[1] + hi[1]])
return dict(heap[0][1])
Data Structures Used
- A min-heap of (frequency, optional payload) pairs.
- The implicit binary tree formed by the merge sequence.
Correctness Argument
We prove the greedy is optimal by induction on n (the number of symbols), using two lemmas.
Lemma 1 (Swap-to-deepest). In some optimal prefix code tree T*, the two least-frequent symbols x and y are siblings at the maximum depth of any leaf.
Proof. Take any optimal tree T'. Let a and b be two siblings at the maximum-depth leaf level of T' (such a pair exists in any full binary tree where every internal node has 2 children). If {a, b} = {x, y}, done. Otherwise, suppose WLOG freq[x] ≤ freq[a] and x ≠ a. Swap x with a (place the symbol x at a’s leaf and vice versa). The cost change is:
Δ = freq[x] * depth(a) + freq[a] * depth(x) - freq[x] * depth(x) - freq[a] * depth(a)
= (freq[a] - freq[x]) * (depth(x) - depth(a))
Since freq[a] ≥ freq[x] (because x is among the two least-frequent) and depth(a) ≥ depth(x) (because a is at maximum depth), Δ ≤ 0. Equality holds; the new tree is also optimal. Repeat with y and b. We’ve moved x, y to the maximum-depth pair without increasing cost. QED for Lemma 1.
Lemma 2 (Greedy preserves optimality on the residual). Let x, y be the two least-frequent symbols. Construct freq' by replacing x and y with a single super-symbol z of frequency freq[x] + freq[y]. Then any optimal tree T* for freq' extends to an optimal tree for freq by replacing z’s leaf with an internal node whose children are leaves for x and y.
Proof. Let T_extended be the extension of T* (replace z-leaf with internal node + x, y children). The cost satisfies:
cost(T_extended) = cost(T*) + freq[x] + freq[y]
(The + freq[x] + freq[y] comes from x, y being one level deeper than z was.)
Conversely, any tree T for freq where x, y are siblings (which by Lemma 1 we can assume WLOG) collapses to a tree T_collapsed for freq' by merging x, y into z:
cost(T_collapsed) = cost(T) - freq[x] - freq[y]
So cost(T) = cost(T_collapsed) + freq[x] + freq[y] ≥ cost(T*) + freq[x] + freq[y] = cost(T_extended). Therefore T_extended is at least as good as any tree where x, y are siblings; combined with Lemma 1, T_extended is optimal for freq. QED for Lemma 2.
Inductive proof of greedy optimality. Base case n = 2: only one tree possible, greedy gives it. Inductive step: greedy merges the two least-frequent symbols x, y first, recurses on the residual of size n - 1, and by inductive hypothesis the recursive call produces an optimal tree for freq'. By Lemma 2, the extended tree is optimal for freq. QED.
The two lemmas together are the full exchange-argument proof. Lemma 1 is the swap step; Lemma 2 is the induction step.
Complexity
- Time
O(n log n)—n - 1merge operations, each with twopopand onepush, eachO(log n). - Space
O(n)— heap and tree.
Implementation Requirements
- Use a min-heap (
heapqin Python uses a min-heap by default; in Java,PriorityQueueis min by default). - Tie-breakers: when frequencies are equal, the heap may pick either; correctness is unaffected. For deterministic output, add a secondary index.
- For the cost-only variant, the symbol payload can be omitted.
Tests
def test_huffman_cost():
assert huffman_cost([2, 3, 4]) == 14 # 2+3=5, 5+4=9
assert huffman_cost([1, 8, 3, 5]) == 30
assert huffman_cost([5]) == 0 # n=1 edge: merge cost is 0
# uniform
assert huffman_cost([1, 1, 1, 1]) == 8 # 1+1=2, 1+1=2, 2+2=4. Total=2+2+4=8.
# large skew
assert huffman_cost([1, 1, 1000]) == 1003 # 1+1=2, 2+1000=1002. Total=2+1002=1004?
# actually: heap [1,1,1000] → pop 1, pop 1, push 2. heap [2, 1000]. pop 2, pop 1000, push 1002. cost = 2 + 1002 = 1004.
Wait — let me re-verify the last test. Heap [1, 1, 1000]. Pop 1 + 1 = 2 (cost contribution: 2). Push 2. Heap [2, 1000]. Pop 2 + 1000 = 1002 (cost contribution: 1002). Total = 2 + 1002 = 1004. So the test should be assert huffman_cost([1, 1, 1000]) == 1004. Correct your tests.
Follow-up Questions
- Adaptive Huffman — frequencies are unknown a priori; encoder and decoder maintain a tree that updates as symbols arrive. Used in older compression standards.
- Canonical Huffman — codes are normalized so only code lengths need to be transmitted, not the tree structure. Used in DEFLATE / zlib.
- Length-limited Huffman (max code length
L) — the package-merge algorithm, more complex than vanilla Huffman. - Arithmetic coding — beats Huffman for non-power-of-2 frequencies; not greedy.
Product Extension
Used in: gzip / DEFLATE (with length-limited variant), HTTP/2 HPACK header compression, JPEG entropy coding stage, MP3 audio coding. Whenever a known-frequency distribution must be losslessly compressed with a prefix-free code, Huffman or its variants are the workhorse.
Language / Runtime Follow-ups
- Python:
heapqfor the heap. Watch out:heapqis min-heap; for tie-breaking, use(freq, counter, payload)to avoid comparing payloads. - Java:
PriorityQueue<Node>withComparator.comparingInt(n -> n.freq). - Go: implement
heap.Interface(5 methods) on a slice of nodes; standard library does not provide a generic typed heap pre-1.21. - C++:
std::priority_queue<Node, std::vector<Node>, std::greater<Node>>. Defineoperator<onNodeto compare by frequency. - JS/TS: no built-in heap; either bring a library (
@datastructures-js/priority-queue) or hand-roll a binary heap.
Common Bugs
- Mixing up min-heap and max-heap: with a max-heap, you’d merge the two largest — the answer is wrong by a lot.
- Pushing the merged node back with the wrong frequency (e.g.,
max(a, b)instead ofa + b). - For the codes variant: assigning ‘0’ to high-freq and ‘1’ to low-freq and forgetting that the prefix is built bottom-up (so the last prepended bit is the root’s assignment — make sure the prepend order is right).
- Heap of size 1 at the start (single symbol): the
while len(heap) > 1loop is correct; cost is 0.
Debugging Strategy
- Hand-trace a small example (e.g.,
[1, 1, 1, 1]) and verify each merge step. - Compare cost output against the brute force for
n ≤ 6. - For codes: verify the tree visually — every internal node has exactly two children, every leaf is a symbol, and code lengths are weighted appropriately.
Mastery Criteria
- You can implement Huffman cost-only in <8 minutes from blank.
- You can implement Huffman with full code map in <15 minutes.
- You can deliver Lemma 1 (swap-to-deepest) in <2 minutes, out loud.
- You can deliver Lemma 2 (induction on residual) in <2 minutes, out loud.
- You can recognize LC 1167 / connect-sticks as a Huffman variant in <30 seconds.
- You can articulate when Huffman is not optimal (e.g., when the alphabet allows non-binary codes, or when arithmetic coding is admissible).
← Lab 04 — Gas Station · Phase 6 README · Lab 06 — Greedy Vs DP →
Lab 06 — Greedy Vs DP (Coin Change Counterexample)
Goal
Internalize the failure mode of greedy by walking through the canonical counterexample: coin change with denominations [1, 3, 4] and target 6. Greedy gives 4 + 1 + 1 = 3 coins; DP gives 3 + 3 = 2 coins. Make this the test you run on every “looks like greedy” problem before committing.
Background
Many candidates correctly solve coin change with US denominations [1, 5, 10, 25] greedily, then assume greedy works for any denomination set. It does not. The failure on [1, 3, 4] target 6 is the most-cited counterexample in algorithms textbooks (Cormen, Kleinberg-Tardos, Erickson) precisely because it cleanly demonstrates that “greedy felt right” is not a proof. The lesson generalizes: for greedy to work on a problem, the underlying combinatorial structure typically must be a matroid — a property that is rarely obvious from problem statements and almost never holds for arbitrary inputs.
Interview Context
This lab is the meta lab of Phase 6. Its purpose is not to drill a new algorithm but to drill the discipline of testing greedy hypotheses against counterexamples before coding. Interviewers love to ask coin-change variants specifically because they expose candidates who pattern-match without proof. A candidate who says “I’ll greedy by largest denomination” is asked “what about [1, 3, 4] target 6?” and either (a) recovers gracefully and switches to DP, or (b) doubles down and ships wrong code. (b) ends the interview.
Problem Statement
Given an array of distinct positive coin denominations coins and a non-negative integer amount, return the minimum number of coins needed to sum to amount, or -1 if impossible. You have an unlimited supply of each denomination.
LeetCode reference: LC 322 — Coin Change.
Constraints
1 ≤ |coins| ≤ 121 ≤ coins[i] ≤ 2^31 - 10 ≤ amount ≤ 10^4- Coins are distinct.
1may or may not be in the set; if not, some amounts are unreachable.
Clarifying Questions
- “Are coins guaranteed sorted?” — typically no; sort if needed.
- “Is
1always present?” — no; the inputcoins = [3, 5]andamount = 4is unsolvable, return-1. - “Can
amount = 0?” — yes; answer is0. - “Is the order of coins in the answer significant?” — no, just the count.
- “Should I count the coins or list them?” — count.
Examples
coins = [1, 3, 4], amount = 6→ 2 (3 + 3).coins = [1, 5, 10, 25], amount = 30→ 2 (25 + 5); greedy works.coins = [1, 5, 10, 25], amount = 41→ 4 (25 + 10 + 5 + 1); greedy works.coins = [2], amount = 3→ -1.coins = [1], amount = 0→ 0.coins = [186, 419, 83, 408], amount = 6249→ 20 (random hostile case).
The Greedy Hypothesis (And Why It Fails)
The natural greedy: sort denominations descending, take the largest that fits, recurse on the remainder.
def coin_change_greedy_WRONG(coins, amount):
coins = sorted(coins, reverse=True)
count = 0
for c in coins:
while amount >= c:
amount -= c
count += 1
return count if amount == 0 else -1
Run this on coins = [1, 3, 4], amount = 6:
- Pick 4: amount = 2, count = 1.
- Pick 3? No, 2 < 3. Skip.
- Pick 1: twice. amount = 0, count = 3.
Result: 3 coins. Optimum: 3 + 3 = 2 coins. Greedy is wrong.
Why? The greedy choice property does not hold: taking the largest coin (4) at step 1 forces a residual problem (amount = 2) where the available coins ([1, 3, 4]) cannot reach 2 with fewer than 2 coins (1 + 1). But not taking 4 leaves us with amount = 6 and the optimal residual 3 + 3 = 2 coins. The local optimum (largest fits) is not the global optimum.
The exchange-argument failure at this concrete level:
- Greedy picks coin 4 first. Optimal picks 3 first.
- Try to exchange the optimal’s first 3 with greedy’s 4: residual amount becomes
6 - 4 = 2, which cannot be made with one more coin of denomination ≥ 3. So the swap breaks feasibility / minimality. - The exchange argument fails. Therefore the greedy is not provably optimal — and indeed isn’t.
DP Fallback (The Correct Algorithm)
def coin_change_dp(coins, amount):
INF = float('inf')
dp = [0] + [INF] * amount
for w in range(1, amount + 1):
for c in coins:
if c <= w and dp[w - c] + 1 < dp[w]:
dp[w] = dp[w - c] + 1
return dp[amount] if dp[amount] != INF else -1
This is unbounded knapsack — see Phase 5 Lab 04 — Unbounded Knapsack (Coin Change) for the full derivation.
Complexity: O(amount * |coins|) time, O(amount) space.
When Does Greedy WORK On Coin Change?
Greedy is optimal on a coin system iff it is canonical — a property that depends on the specific denominations. Sufficient conditions:
[1, c, c², c³, …](powers of a fixed base) — always canonical.[1, 5, 10, 25, 50, 100](US currency) — canonical.[1, 2, 5, 10, 20, 50](euro) — canonical.
Necessary and sufficient condition: the set is canonical iff for every amount m in the range [c_{k+1} + 1, c_{k+1} + c_k - 1] (where c_k is the k-th denomination from largest), the greedy answer matches the optimal. Verifying this requires checking O(c_max²) amounts — feasible for small denomination sets.
For interview purposes: never assume canonicity unless the problem explicitly states the denominations are canonical (e.g., “US currency” with the standard set). Default to DP.
A Glance At Matroid Theory (Why Some Greedy Problems Work)
A matroid M = (E, I) is a pair where E is a set of elements and I ⊆ 2^E is a family of “independent sets” satisfying:
- Hereditary: if
A ∈ IandB ⊆ A, thenB ∈ I. - Exchange property: if
A, B ∈ Iand|A| < |B|, then there existsb ∈ B \ Asuch thatA ∪ {b} ∈ I.
Theorem (Edmonds–Rado): the greedy algorithm produces a maximum-weight independent set on M iff M is a matroid.
Examples of matroids: the cycle-free edge sets of a graph (graphic matroid → Kruskal works), linearly-independent subsets of vectors (linear matroid), independent sets in a uniform matroid.
Coin change is not a matroid problem — there is no natural matroid structure under which “fewest coins to reach amount” is a max-weight independent set, which is why greedy doesn’t work for arbitrary denominations. Interval scheduling is effectively a matroid problem (the set of compatible activities forms an “interval matroid”), which is why earliest-end-time greedy works.
You don’t need to memorize matroid theory for interviews. You do need to know the empirical signal: if greedy doesn’t pass the counterexample stress test, fall back to DP without panic.
Decision Recipe (The Whole Point Of This Lab)
For any optimization problem that “looks greedy”:
- Hypothesize a greedy choice (e.g., largest first, smallest first, by ratio).
- Run it on a hand-crafted small input of size 4–6 with adversarial denominations / weights.
- Compare to brute force (recursive enumeration of all choices).
- If greedy ≠ brute force on any input → fall back to DP, no further deliberation.
- If greedy = brute force on all stress tests → try to prove via exchange argument.
- If exchange argument works → ship greedy.
- If exchange argument fails or is unclear → fall back to DP. Better safe than sorry under interview time pressure.
The discipline: greedy is opt-in, requires positive proof. DP is the default for optimization problems unless greedy is clearly justified.
Tests
def test_coin_change_dp():
assert coin_change_dp([1, 3, 4], 6) == 2
assert coin_change_dp([1, 2, 5], 11) == 3
assert coin_change_dp([2], 3) == -1
assert coin_change_dp([1], 0) == 0
assert coin_change_dp([1, 5, 10, 25], 30) == 2
assert coin_change_dp([1, 5, 10, 25], 41) == 4
assert coin_change_dp([186, 419, 83, 408], 6249) == 20
def test_greedy_fails_on_counterexample():
"""Document the failure for posterity."""
assert coin_change_greedy_WRONG([1, 3, 4], 6) == 3 # WRONG; correct is 2
assert coin_change_dp([1, 3, 4], 6) == 2 # Right answer
def test_greedy_works_on_canonical():
assert coin_change_greedy_WRONG([1, 5, 10, 25], 30) == 2
assert coin_change_greedy_WRONG([1, 5, 10, 25], 41) == 4
Correctness Argument (For DP)
DP correctness follows from optimal substructure: dp[w] = 1 + min(dp[w - c] : c ∈ coins, c ≤ w). Each dp[w] is computed from strictly smaller subproblems, so the table fills in O(amount * |coins|) time. The minimum is over all first-coin choices, exhaustively — so we never miss the optimal first choice (in contrast to greedy, which commits to one). See Phase 5 Lab 04 for the full proof.
Common Bugs (In The DP)
- Initializing
dp[0] = INFinstead of0. (dp[0] = 0because zero amount needs zero coins.) - Iterating
for c: for w(orderings DP) whenfor w: for c(combinations DP) is intended for count of orderings — not the issue for min coins, but the analogous bug appears in LC 518 (number of ways to make change). - Returning
dp[amount]without checkingINF— returns a giant number instead of-1.
Common Bugs (In The Greedy, If You Do Try It)
- Assuming canonicity. Always test against DP on hostile cases first.
- Forgetting to return
-1when amount is not zero at the end. - Treating
coins = [1]as always feasible — true, but easy to forget the early return.
Mastery Criteria
-
You can deliver the
[1, 3, 4]target6counterexample by heart, in <30 seconds, without notes. -
You can articulate why greedy on coin change works for
[1, 5, 10, 25]but fails for[1, 3, 4]— the canonicity property — in <60 seconds. - You can write the DP solution in <5 minutes from blank.
- You can name three other classic problems where greedy fails but DP works (0/1 knapsack, weighted interval scheduling, longest path in a general graph).
- When proposing a greedy solution to any problem in mock interviews, you stress-test it against brute force on small adversarial inputs before writing production code.
← Lab 05 — Huffman Coding · Phase 6 README
Phase 7 — Competitive Programming Acceleration
Target level: Hard → Codeforces Div 2 D (rating ~1900–2100) Expected duration: 2 months (12-month Elite track) / 4 weeks selective topics (6-month Serious track) / skipped or read-only (12-week Accelerated track) Weekly cadence: ~10 competitive topics + 6 labs + 2 contests/week + 30–60 problems applying them under the framework
A Direct Note On ROI Before You Spend Two Months Here
This phase has the lowest direct ROI per hour for FAANG SWE2 / L4 prep of any phase in this curriculum. If your goal is a Google L4, Meta E4, Amazon SDE2, or similar — you can skip this phase entirely and lose nothing. Phases 01 → 06 plus Phase 8 (practical engineering) cover essentially every problem you will see in those interviews. Modular inverse will not appear in your loop. Convex hull will not appear in your loop. Mo’s algorithm will not appear in your loop. The opportunity cost of two months on competitive programming is two months you could have spent on system design, behavioral prep, or sleep.
This phase has the highest direct ROI per hour for: HFT/quant interviews (Jane Street, HRT, Citadel, Two Sigma, Optiver, IMC, Jump), compiler/runtime/database internals teams (Google’s compiler infra, Microsoft’s CLR, Oracle’s HotSpot, ClickHouse, Snowflake’s query engine), distributed systems coding rounds at the senior+ level where contest-style problems are deliberately used as filters, ICPC-flavored test rounds at startups founded by ex-CP champions, and any interview where the explicit goal is to filter out everyone except the top ~5% of candidates by raw algorithmic horsepower. In those loops, the topics in this phase are not optional decoration — they are the test. A candidate who cannot derive a modular inverse, write binary exponentiation, or sweep events along a coordinate cannot pass an Optiver onsite no matter how good their system design is.
So: decide your target before you start this phase, and do not feel guilty about skipping it if the ROI calculation says skip. The rest of this README assumes you’ve decided to do the work.
What “Competitive Programming Acceleration” Actually Means
Competitive programming is not just “harder LeetCode”. It is a different sport with a different culture, different problem-solving rhythm, and different correctness bar. The differences that matter for interview prep:
- Constraints are everything. A LeetCode Hard might say
1 ≤ N ≤ 10^5and accept anyO(N log N)solution. A Codeforces problem will say1 ≤ N ≤ 5·10^5, T ≤ 10^4 testcases, sum of N ≤ 5·10^5, 2 second time limit, and yourO(N log^2 N)solution will TLE whileO(N log N)will pass with 200ms to spare. Reading constraints first — before the problem statement — is the single biggest skill jump from LeetCode to CP. - Problems are short. A typical CF Div 2 problem is 3–8 sentences plus 2 example testcases. Information density per word is 5–10× LeetCode. Skim-then-deep-read is wrong; deep-read on first pass is correct.
- Brute force is a starting point, not an ending point. Submitting brute force on a contest problem to “lock in partial credit” is a LeetCode habit. On CF you submit only when you believe you have the intended complexity, because wrong submissions cost 50 points each (penalty time).
- Stress testing is a normal part of the workflow. Top CP grandmasters run brute-vs-candidate stress tests against random inputs during a contest, on every problem they’re not 100% certain about. This is the muscle Lab 06 builds.
- Editorials are a separate skill. After a contest, reading editorials productively (extracting transferable techniques, not just patching your specific solution) is half the learning. Most candidates read an editorial and take away nothing because they read it as a solution rather than as a textbook.
The competitive programming skill set translates to interview signal in three ways: (1) speed — you become physically faster at typing and debugging, which buys time for harder questions; (2) vocabulary — when an interviewer says “this is a sweep line problem” or “use binary search on the answer”, you have a direct reference rather than re-deriving from scratch; (3) pattern coverage — the long tail of “weird trick” problems that interviewers reach for to filter senior candidates is exactly the long tail of CP techniques.
What You Will Be Able To Do After This Phase
- Read a Codeforces Div 4 / Div 3 problem in <2 minutes, decide brute-vs-intended in <1 minute, and submit Div 4 A–F or Div 3 A–E within contest time.
- Reach Div 2 C consistently and attempt Div 2 D in ~50% of contests.
- Read AtCoder Beginner Contest problems A–F and solve A–E reliably; reach F in ~50% of contests.
- Reach AtCoder Regular Contest A–C, with C being the contest-finisher you usually upsolve afterward rather than solve in-contest.
- Compute
nCr mod pforpprime andnup to10^7, with precomputed factorials and modular inverses, in <5 minutes from blank. - Implement binary exponentiation (
a^b mod m) in <2 minutes and recognize when matrix exponentiation reduces a linear recurrence fromO(N)toO(K^3 log N). - Implement the Sieve of Eratosthenes (basic and linear), the smallest-prime-factor sieve, and trial-division factorization, knowing when each is appropriate.
- Implement modular inverse via Fermat’s Little Theorem (when modulus is prime) and via extended Euclidean (when it isn’t), and know which to reach for.
- Implement Andrew’s monotone chain convex hull in <15 minutes and explain why cross product replaces division for orientation.
- Implement a sweep line for the skyline problem and 1D rectangle union; recognize the “sort events, scan, maintain active set” pattern under disguise.
- Implement coordinate compression as a one-line preprocessing step and combine it with Fenwick tree to count inversions in
O(N log N). - Implement Mo’s algorithm with the canonical block-sqrt sorting comparator and explain its
O((N + Q) √N)complexity. - Compute Sprague-Grundy numbers for impartial games and reduce composite games via XOR.
- Write a stress-testing harness — brute, candidate, random generator, comparator — and use it to find a planted bug in <5 minutes.
- Solve interactive CP problems (binary search a hidden value, query a hidden function) using line-buffered I/O and explicit
flushdiscipline. - Configure fast I/O in your language of choice —
cin/coutdesync in C++,bufio.NewReader+bufio.NewWriterin Go,sys.stdin.readline+sys.stdout.writein Python,BufferedReader+PrintWriterin Java — without thinking about it.
How To Read This Phase
Read this README once, linearly, end-to-end. Do not try to memorize it. The 19 inline topic sections are reference material — internalized when you actually use them on contest problems, not by re-reading. The 9 progression sections are playbooks — they tell you which contests to enter and what the success bar is.
After the linear pass, do this in order:
- Set up your CP toolchain — install your language compiler, configure fast I/O templates, get accounts on Codeforces and AtCoder.
- Work Lab 01 through Lab 06 in order. The labs are designed so each one builds a primitive you reuse in the next.
- Start the contest progression — Div 4 first, then Div 3, then Div 2. Do not skip Div 4 thinking it’s “too easy”; the goal there is speed, not difficulty.
- After every contest, spend at least 2× the contest time on upsolving (problems you didn’t solve in-contest, with the editorial open). Upsolving is where the learning happens.
Each topic entry has a fixed shape:
- Definition — what the technique is.
- When Used — the problem signal that fires this technique.
- Complexity — the canonical time/space.
- Classic Problems — 2–4 representative LC / CF / AtCoder problems.
- Pitfalls — the bugs that consume the most contest minutes for this technique.
The phase ends with a Mastery Checklist, Exit Criteria, and links to all six labs.
CP Problem-Solving Methodology — The Five-Step Loop
The single most teachable skill in competitive programming is the read → constraints → brute → submit → stress loop. Apply it to every problem.
- Read fast. First read takes ~60 seconds. Goal: identify the problem class (graph? DP? math? sweep? game?) and the input/output format. Don’t try to solve yet. If you don’t understand on first read, re-read — but do not start sketching code.
- Look at constraints before optimizing. This is the single biggest behavioral difference between CP and LeetCode habits. The constraint
N ≤ 18says bitmask DP.N ≤ 22says meet-in-the-middle.N ≤ 5000saysO(N²).N ≤ 2·10^5saysO(N log N).N ≤ 10^9says you don’t iterateNat all — math, binary search on the answer, or a closed form. The constraint is the algorithm choice. Read it first; do not write a single line of code without it. - Brute-force first, in your head. Even if brute force won’t pass, the brute force gives you (a) a correctness oracle for stress testing, (b) a starting point for optimization, (c) a 100% reliable answer to “do I understand the problem?”. If you can’t write the brute force, you don’t understand the problem yet — re-read the statement.
- Submit early and often, but only when confident. Do not submit a partial / “maybe correct” solution to lock in points; CF/AtCoder penalize wrong submissions. If your code passes the sample inputs, that is necessary but not sufficient. Sample inputs are the easiest possible cases by construction; passing them is the floor, not the ceiling. Stress-test before submitting on any problem you’re <90% confident on.
- Stress test if uncertain. Lab 06 builds this muscle. The pattern: brute (definitely correct, exponential), candidate (your fast solution), random generator (small inputs,
N ≤ 10), comparator that runs both and dies on mismatch. Run it for 1000 random tests in 30 seconds. If it doesn’t fail in 1000 trials, it probably won’t fail on the judge.
The loop applies recursively. If you’re stuck in step 3 (can’t write brute force), drop to “what’s the absolute simplest version of this problem?” — usually a smaller N, a special case, or a related problem. Solve that first. That’s almost always how the intended solution is derived.
Inline Topic Reference
Math
1. Modular Arithmetic
Definition
Arithmetic over the residue ring Z/pZ (typically p = 10^9 + 7 or p = 998244353). Addition, subtraction, multiplication, and exponentiation are all defined modulo p. Division is not defined directly — see modular inverse.
When Used
Whenever the answer is “huge” — count of arrangements, count of paths, sum over all subsets — and the problem says output mod 10^9 + 7. This is the most common modifier on counting problems in CP.
Complexity
Addition / subtraction / multiplication are O(1). Watch for overflow: in C++ (a * b) % p overflows int when p ≈ 10^9; cast to long long first. In Java, % is signed (negative % of negative integer is negative); use ((a % p) + p) % p after subtraction. In Python, integers are arbitrary precision so overflow doesn’t happen, but performance suffers — keep numbers under p aggressively.
Classic Problems
- CF 1342E (Placing Rooks) — counting arrangements mod
10^9 + 7. - AtCoder ABC 174 F — count distinct elements queries (off-topic but illustrates
modergonomics). - LC 920 (Number of Music Playlists) — DP with mod.
Pitfalls
- Forgetting to mod after every multiplication; the value silently overflows and silently corrupts answers.
- Negative numbers after subtraction in C++/Java; always
((x % p) + p) % p. - Using
%ondouble(always wrong; mod is integer-only).
See Lab 01 — Modular Arithmetic.
2. Modular Inverse
Definition
The modular inverse of a mod p is the unique x in [0, p) such that a · x ≡ 1 (mod p), when it exists. Existence requires gcd(a, p) = 1. When p is prime, every a ≠ 0 has an inverse.
Two computation methods:
- Fermat’s Little Theorem (FLT): if
pis prime,a^(p-1) ≡ 1 (mod p), soa^(p-2) ≡ a^(-1) (mod p). Use binary exponentiation inO(log p). - Extended Euclidean Algorithm (extgcd): find
x, ysuch thata·x + p·y = gcd(a, p). Ifgcd = 1, thenx mod pis the inverse.O(log min(a, p)).
When Used
- Division by
amodp(replacen / awithn · inv(a)). - Computing
nCr mod pfrom precomputed factorials:nCr = fact[n] · inv(fact[r]) · inv(fact[n-r]). - Probability problems where the answer is a fraction
p/qmodulo a prime; the answer isp · q^(-1) mod prime.
Complexity
O(log p) per inverse via either method. For batched inverses of n values, there’s a clever O(n) algorithm using the running product trick — useful when precomputing inverse factorials.
Classic Problems
- CF 1462E2 (Close Tuples to Arrays, Hard) —
nCr mod pheavy. - AtCoder ABC 178 F (Contrast) — combinatorics with mod.
- CF 1342E — modular inverse for counting.
Pitfalls
- Using FLT when
pis composite — incorrect, must use extgcd. - Forgetting that
inv(0)is undefined; guard before calling. - Using FLT when the modulus is prime but you accidentally pass
p - 1instead ofp - 2.
3. Binary Exponentiation (Fast Power)
Definition
Compute a^b (or a^b mod m) in O(log b) time by exploiting the binary representation of b. The recurrence: a^b = (a^(b/2))^2 if b is even, a · a^(b-1) if b is odd.
long long power(long long a, long long b, long long m) {
long long res = 1 % m;
a %= m;
while (b > 0) {
if (b & 1) res = res * a % m;
a = a * a % m;
b >>= 1;
}
return res;
}
When Used
Anywhere you’d otherwise loop b times multiplying a. With b up to 10^18, naive looping is impossible; binary exponentiation is mandatory. Also the implementation engine for FLT-based modular inverse and matrix exponentiation.
Complexity
O(log b) multiplications. Each multiplication is O(1) for integers but O(K^3) for K×K matrices (giving O(K^3 log b) for matrix exponentiation).
Classic Problems
- CF 630I, 630J — direct power computation.
- LC 50 (Pow(x, n)) — the canonical binary exponentiation problem.
- AtCoder ABC 178 D — DP with mod, uses fast power for inverses.
Pitfalls
- Negative exponents (LC 50): handle as
1 / power(x, -n)and watch forINT_MIN(negating overflows). - Base case
b = 0returning1, but1 % mifm = 1should be0— start withres = 1 % m.
See Lab 02 — Binary Exponentiation.
4. Matrix Exponentiation
Definition
For a linear recurrence f(n) = c_1 · f(n-1) + c_2 · f(n-2) + ... + c_k · f(n-k), the state vector [f(n), f(n-1), ..., f(n-k+1)] is obtained from the state vector at step n-1 by multiplying by a fixed k×k companion matrix M. Therefore the state at step n is M^n · initial_state, and M^n is computed by binary exponentiation in O(k^3 log n) time.
When Used
Linear recurrences where n is up to 10^18 and k (the recurrence depth) is small (typically k ≤ 60). The textbook example is Fibonacci modulo a prime for n = 10^18. Also: counting walks of length n in a graph (M = adjacency matrix), counting paths in a DFA, certain combinatorial DPs over a small fixed state space.
Complexity
O(k^3 log n) time, O(k^2) space. For k = 2 (Fibonacci), 8 log n multiplications mod p ≈ 500 ops for n = 10^18.
Classic Problems
- Fibonacci mod p, n = 10^18 — the canonical introduction.
- CF 392C (Yet Another Number Sequence) — matrix exponentiation with polynomial coefficients.
- AtCoder DP Contest R (Walk) — counting walks of length
Kin a graph.
Pitfalls
- Index off-by-one in the state vector (forgetting that the last entry is
f(n-k+1), notf(n-k)). - Forgetting to mod every matrix multiplication entry.
- Using nested Python lists instead of NumPy for matrices — Python is too slow for
K ≈ 50andlog n ≈ 60.
See Lab 02 — Matrix Exponentiation for Fibonacci.
5. Sieve of Eratosthenes (and Linear Sieve)
Definition
Build a boolean array is_prime[0..N] in O(N log log N) time by, for each prime p ≤ √N, marking all multiples of p (starting from p²) as composite. The linear sieve variant produces the smallest prime factor (SPF) for every integer up to N in exactly O(N) time using the invariant “every composite is sieved once, by its smallest prime factor”.
When Used
- Counting primes up to
NforN ≤ 10^7(Sieve of Eratosthenes is faster than trial division). - Generating all primes up to
Nfor prime-related problems. - Building a smallest-prime-factor table for fast factorization (see Topic 6).
- Euler’s totient
phi(n)for alln ≤ NinO(N log log N).
Complexity
Sieve of Eratosthenes: O(N log log N), space O(N) (or N/8 with a bitset). Linear sieve: O(N), space O(N) for the SPF table.
Classic Problems
- LC 204 (Count Primes) — sieve introduction.
- CF 17A (Noldbach problem) — primes near pairs of primes.
- Project Euler 10 (sum of primes below 2M) — sieve of size
2·10^6.
Pitfalls
- Iterating to
Ninstead of√Nin the outer loop (correctness OK butO(N²)-flavor slow). - Starting the inner loop at
2pinstead ofp²(correct but slower;p², p²+p, p²+2p, ...is the optimal start). - Using
vector<bool>in C++ is fine;bool[]is also fine.unordered_set<int>is not fine — too slow.
See Lab 03 — Sieve and Factorization.
6. Prime Factorization
Definition
Decompose n into its prime factors. Two main techniques:
- Trial division. For each
p = 2, 3, 5, ..., √n, whilep | n, divide. Finaln > 1is itself prime.O(√n)per number. - Smallest-prime-factor (SPF) sieve. Precompute
spf[i]= smallest prime dividingi, for alli ≤ N. Then factor anyi ≤ NinO(log i)by repeatedly replacingiwithi / spf[i].O(N log log N)preprocessing;O(log i)per query.
When Used
- Trial division when factoring a single large
n(up to10^14is feasible). - SPF sieve when factoring many numbers in a range
[1, N]forN ≤ 10^7. - For
nup to10^18, trial division is too slow; use Pollard’s rho (out of scope for this phase, in Phase 12).
Complexity
Trial division: O(√n). SPF sieve: O(log n) per query after O(N log log N) preprocessing.
Classic Problems
- CF 1325E — factor and sum of exponents.
- LC 263 (Ugly Number) — recursive division by small primes.
- AtCoder ABC 169 D — factor and count exponents.
Pitfalls
- Forgetting that after the loop
if n > 1: append n as final prime. Easy to miss; corrupts every factorization wherenhas a prime factor >√n_initial. - Trial-dividing past
√n; oncep > √n,nis either 1 or itself prime. - Mixing up “number of distinct primes” with “number of prime factors with multiplicity” — these are very different (e.g., 12 = 2²·3 has 2 distinct, 3 with multiplicity).
See Lab 03 — Sieve and Factorization.
7. Combinatorics (nCr mod p)
Definition
Compute binomial coefficients modulo a prime. For repeated queries, precompute fact[i] = i! mod p and inv_fact[i] = (i!)^(-1) mod p for i up to N. Then nCr = fact[n] · inv_fact[r] · inv_fact[n-r] mod p in O(1) per query.
For n very large (up to 10^18) and p small (p ≤ 10^5), use Lucas’s theorem: C(n, r) mod p = ∏ C(n_i, r_i) mod p, where n_i, r_i are the base-p digits of n, r. The inner C(n_i, r_i) are computed directly because n_i, r_i < p.
When Used
- Counting paths in a grid (
C(m+n, m)). - Stars-and-bars: distribute
nidentical items intokbins →C(n+k-1, k-1). - Inclusion-exclusion sums.
- Probability with combinatorial denominators.
- Lucas: when
nis up to10^18(e.g., AtCoder ABC 167 E or grid problems with huge dimensions).
Complexity
Preprocess O(N). Each nCr query O(1). Lucas’s theorem: O(p + log_p(n)) per query (assuming preprocessed factorials up to p).
Classic Problems
- CF 1342E — uses
nCr. - LC 62 (Unique Paths) — direct
C(m+n-2, m-1). - AtCoder ABC 167 E (Colorful Blocks) — inclusion-exclusion with
nCr.
Pitfalls
- Forgetting to precompute
inv_factseparately; computing each query asfact[n] / (fact[r] · fact[n-r])and trying to use integer division modp(this is wrong; you need modular inverse). - Off-by-one in
fact[]array (forgettingfact[0] = 1). - For Lucas, forgetting that any
r_i > n_igivesC(n_i, r_i) = 0, so the whole product is0.
See Lab 01 — Modular Arithmetic.
8. GCD, LCM, Extended Euclidean
Definition
gcd(a, b)is the greatest common divisor ofa, b. Computed by Euclidean:gcd(a, b) = gcd(b, a mod b), base casegcd(a, 0) = a.lcm(a, b) = a · b / gcd(a, b). Compute asa / gcd(a, b) · bto avoid overflow on intermediatea · b.- Extended Euclidean algorithm finds, alongside
gcd(a, b), integersx, ysuch thata·x + b·y = gcd(a, b). This is the engine for modular inverse when the modulus isn’t prime.
When Used
- Reducing fractions.
- Solving linear Diophantine equations
a·x + b·y = c(solution exists iffgcd(a, b) | c). - Modular inverse via extgcd when the modulus is composite.
- Cycle-length problems where the answer involves an LCM.
Complexity
O(log min(a, b)) for both gcd and extgcd.
Classic Problems
- CF 822A — direct LCM use.
- LC 1071 (Greatest Common Divisor of Strings) — repurposed gcd.
- AtCoder ABC 162 D — gcd in a counting problem.
Pitfalls
lcm(a, b) = a * b / gcd(a, b)overflows whena, bare around10^9. Reorder:lcm = a / gcd * b.gcd(0, 0)is conventionally0, but C++__gcd(0, 0)returns0; some libraries return undefined. Guard.- Negative
a, b:gcdshould always be non-negative; some implementations return signs. Useabs().
Geometry
9. Coordinate Geometry Basics (Cross Product, Orientation)
Definition
For two 2D vectors u = (ux, uy) and v = (vx, vy), the cross product is the scalar ux·vy − uy·vx. Its sign tells you the relative orientation of the vectors: positive = counter-clockwise turn, negative = clockwise, zero = collinear. The orientation of three points A, B, C is the sign of the cross product of B − A and C − A; this is the most-used primitive in computational geometry.
When Used
- Determining whether three points form a left turn, right turn, or are collinear (convex hull, polygon orientation).
- Determining whether a point is on, left of, or right of a line.
- Computing twice the signed area of a triangle (the cross product is twice the signed area).
- Computing twice the signed area of a polygon (shoelace formula = sum of cross products).
Complexity
O(1) per cross product / orientation test.
Classic Problems
- LC 587 (Erect the Fence) — convex hull, uses orientation.
- CF 70D (Dynamic Convex Hull) — uses orientation heavily.
- AtCoder ABC 207 D — geometry with cross products.
Pitfalls
- Using floating-point for cross product when integer arithmetic would suffice — introduces rounding errors that cause “almost collinear” misclassifications. Use
long long(or arbitrary precision) when coordinates are integers. - Confusing CCW (counter-clockwise) with CW (clockwise) sign convention.
- Overflow in cross product: with coordinates up to
10^9, the product is up to10^18, which fitslong longbut notint.
10. Convex Hull (Andrew’s Monotone Chain)
Definition
Given a set of 2D points, the convex hull is the smallest convex polygon containing all of them. Andrew’s monotone chain algorithm sorts points by (x, y), then builds the lower hull left-to-right and the upper hull right-to-left, using the cross-product orientation test to pop points that make a right turn (in the lower hull) or left turn (in the upper hull).
sort(points.begin(), points.end());
vector<P> hull;
// lower hull
for (auto &p : points) {
while (hull.size() >= 2 && cross(hull[hull.size()-2], hull.back(), p) <= 0)
hull.pop_back();
hull.push_back(p);
}
// upper hull
int lower_size = hull.size() + 1;
for (int i = points.size() - 2; i >= 0; --i) {
while (hull.size() >= lower_size && cross(hull[hull.size()-2], hull.back(), points[i]) <= 0)
hull.pop_back();
hull.push_back(points[i]);
}
hull.pop_back(); // last point is the start, duplicated
When Used
- Smallest enclosing polygon problems.
- Diameter of a point set (rotating calipers on the hull).
- Pre-step for various 2D optimization problems (convex layers, dynamic hulls).
Complexity
O(N log N) — dominated by the sort. The two hull-building passes are O(N) amortized.
Classic Problems
- LC 587 (Erect the Fence) — direct convex hull.
- CF 1093E — uses convex hull as a subroutine.
Pitfalls
<= 0vs< 0in the orientation test:<= 0removes collinear points from the hull (giving the strict hull);< 0keeps them (giving the inclusive hull). LC 587 wants the inclusive hull (use< 0); most CP problems want the strict hull (use<= 0).- Forgetting to remove the duplicated last point.
- Sorting tuples lexicographically without a tie-break — for points with the same
xbut differenty, the sort order matters;(x, y)lexicographic is the right tie-break.
11. Closest Pair of Points (Divide & Conquer Overview)
Definition
Given N points in 2D, find the pair with the smallest Euclidean distance. The naive O(N²) algorithm is to compare every pair. The classical O(N log N) algorithm sorts by x, recursively solves the left and right halves, finds the minimum distance d of the two halves, then merges by inspecting only points within horizontal distance d of the dividing line — and within those, only y-neighbors within distance d. The merge step is O(N) because each strip point only needs to compare against ~6 nearest y-neighbors.
When Used
- Direct closest-pair problems.
- Any problem where you need a guarantee on minimum spacing (geometric clustering, collision detection).
Complexity
O(N log N) time, O(N) space. The recursion T(N) = 2 T(N/2) + O(N) resolves to O(N log N).
Classic Problems
- Codeforces educational round problems labeled “closest pair”.
- UVa 10245 (The Closest Pair Problem) — direct.
Pitfalls
- For most interview problems,
Nis small enough (≤ 5000) thatO(N²)brute force passes, and writing the divide-and-conquer version is not worth the complexity. - Floating-point distance comparison: compare squared distances (integers, exact) instead of square-rooted distances (floats, lossy).
Sweep & Queries
12. Sweep Line
Definition
A sweep line algorithm imagines a vertical (or horizontal) line sweeping across the plane (or 1D number line) and processing events in the order the sweep encounters them. At each event, you update an “active set” — typically a balanced BST or a multiset — and answer queries based on the current state. The key insight is that between events, the active set is constant, so you only need to process at events.
When Used
- 1D rectangle/interval union (sum of lengths).
- 2D rectangle union area (sweep
y-coordinate; active set =x-intervals). - Segment intersection problems (Bentley-Ottmann).
- Skyline problem (LC 218).
- Closest pair (alternate formulation).
Complexity
Typically O((N + E) log N) where E is the number of events; for rectangle union, E = O(N), giving O(N log N).
Classic Problems
- LC 218 (The Skyline Problem) — canonical.
- LC 850 (Rectangle Area II) — 2D rectangle union.
- AtCoder ABC 188 D — 1D event-sweep counting.
Pitfalls
- Tie-breaking on event time: when multiple events occur at the same
x, process all opens before closes (or vice versa, problem-dependent). Wrong order → off-by-one in the active set. - Using
set<int>for the active set when you need to handle duplicate values; switch tomultiset<int>. - Updating the answer based on the active set after processing all events at the current
x, not in the middle.
See Lab 04 — Sweep Line for Skyline.
13. Coordinate Compression
Definition
Replace large/sparse coordinate values with their ranks in the sorted set of distinct coordinates. If your data has values [10^9, 5, 10^7, 5, 1], compression maps them to ranks [3, 1, 2, 1, 0]. The transformed problem has the same structure but coordinates fit in [0, N), enabling array-indexed data structures (Fenwick tree, segment tree, bucket sort).
sorted_unique = sorted(set(values))
rank = {v: i for i, v in enumerate(sorted_unique)}
compressed = [rank[v] for v in values]
When Used
- Counting inversions with a Fenwick tree (values up to
10^9→ compress to[0, N)). - 2D rectangle union via sweep line + segment tree on
y-coordinates. - DP with a state indexed by a coordinate that’s too large to enumerate.
- Almost any problem with
value ≤ 10^9where you would otherwise need a hashmap of sizeN.
Complexity
O(N log N) for sorting and deduplication; O(N) after that to relabel.
Classic Problems
- LC 315 (Count of Smaller Numbers After Self) — Fenwick + compression.
- CF 51A — geometry with coord compression.
- AtCoder ABC 174 F — distinct elements queries.
Pitfalls
- Forgetting to use
set(deduplicate) before sorting; otherwise duplicate values get different ranks, breaking equality comparisons. - Using compression when not needed (values already in a small range) — adds complexity for no benefit.
See Lab 05 — Coordinate Compression for Inversions.
14. Mo’s Algorithm
Definition
An offline algorithm for answering Q range queries on an array in O((N + Q) √N) total time when (a) you can move the answer from [l, r] to [l-1, r], [l+1, r], [l, r-1], [l, r+1] in O(1) (or O(log N)); and (b) the queries can be reordered. The trick: sort queries by (l / B, r) where B = √N. Then within a block, r only increases, so total r movement is O(N) per block × √N blocks = O(N √N). Across blocks, l movement is O(√N) per query × Q queries = O(Q √N).
When Used
- “Number of distinct values in
[l, r]” queries. - “Sum of
f(count(v))for distinctvin[l, r]” queries. - Mode of a range (with auxiliary frequency-of-frequency structure).
- Many problems labeled “offline range queries with no updates” on Codeforces.
Complexity
O((N + Q) √N). With N = Q = 10^5, that’s about ~3.2·10^7 operations — passes a 2-second limit comfortably.
Classic Problems
- CF 86D (Powerful array) — sum of
cnt² · vover distinctv. - SPOJ DQUERY — distinct values in a range.
- CF 220B — count of values equal to their frequency.
Pitfalls
- The optimal block size is
√N; smaller values ofBcause TLE becauser-movement within a block is too long. - Mo’s algorithm doesn’t handle online queries — queries must be batched and reordered.
- The “add/remove element in O(1)” requirement is strict; an
O(log N)add/remove makes the totalO((N + Q) √N · log N), which usually TLEs.
15. Offline Binary Search / Parallel Binary Search
Definition
When you have Q independent binary-search queries, each of which would naively take O(log V · F) where F is some function evaluation cost, parallel binary search runs all Q queries’ binary searches in lockstep. At each binary-search step, group the queries by their current candidate midpoint, evaluate F once per group, and update each query’s interval. Total cost: O(log V · (F + Q)) instead of O(Q · log V · F).
When Used
- “For each query, find the smallest
tsuch that some propertyP_query(t)holds”, wherePis monotone intand evaluatingPis expensive (e.g., requires processing the firsttoperations of a stream). - Problems where each query is a binary search over time/index/threshold and the function
F(t)changes globally witht.
Complexity
O(log V · (F + Q)). For V = N, F = O(N), Q = N: O(N log N) total.
Classic Problems
- CF 813F (Bipartite Checking) — parallel binary search on offline DSU.
- POI Meteors — the canonical introduction.
Pitfalls
- Forgetting that this technique requires queries to be independent (one query’s answer doesn’t depend on another).
- Conceptually heavier than Mo’s algorithm; for interview prep, knowing the technique exists and recognizing its signal is more important than implementing it from blank.
Game Theory & Misc
16. Sprague-Grundy / Nim
Definition
In an impartial two-player game (both players have the same available moves at every position), each position has a Grundy number g(pos) defined recursively: g(pos) = mex { g(next) : next ∈ moves(pos) }, where mex is the minimum excluded value (smallest non-negative integer not in the set). A position has Grundy number 0 iff it’s losing for the player to move. Nim’s theorem: the Grundy number of a sum of independent games is the XOR of their individual Grundy numbers.
When Used
- Any “two-player game, both move optimally, who wins?” problem with no chance / no hidden information / both players have the same moves.
- Decomposing complex games into sums of simpler subgames.
- Standard Nim (multiple piles, take any number from any pile, last to move wins): the answer is “first player wins iff XOR of pile sizes ≠ 0”.
Complexity
Computing Grundy via memoization: O(states · branching). For games with state space up to 10^6, this is feasible.
Classic Problems
- Standard Nim, multi-pile — XOR of pile sizes.
- CF 95A — Grundy on a stair-step game.
- AtCoder ABC 195 D — game DP related (not pure Grundy but related).
Pitfalls
- The theorem only applies to impartial games. Partisan games (chess, where pieces are colored) don’t satisfy Sprague-Grundy.
- “Last to move loses” (misère convention) does NOT in general have the simple XOR rule — only the “last to move wins” (normal convention) does, except in degenerate cases.
- Large branching factor + large state space → memoization table doesn’t fit. Look for closed-form patterns by computing Grundy for small
nand spotting periodicity.
17. Randomized Algorithms / Stress Testing
Definition
Two related concepts:
- Randomized algorithms: algorithms that use random choices to achieve good expected complexity (randomized quicksort, treap, hash-based string matching). Las Vegas algorithms are always correct, randomized in time; Monte Carlo are randomized in correctness.
- Stress testing (the bigger interview-prep topic): writing a small brute-force solver, your candidate optimal solver, a random input generator, and a comparator that runs both on every random input until they disagree. This is how CP grandmasters find bugs in their own solutions.
When Used
- Stress testing: on every problem you’re not 100% confident about, before submitting.
- Randomized algorithms: when a deterministic guarantee isn’t required (probabilistic data structures: Bloom filter, count-min sketch, treap, randomized convex hull).
Complexity
Stress testing is overhead-only — if both your brute and candidate are fast enough on small inputs, stress testing is essentially free. 1000 random tests on N = 10 finishes in <1 second.
Classic Problems
- This is a meta-skill, not a problem class. See Lab 06 — Stress Testing.
Pitfalls
- Random generator that doesn’t cover edge cases (e.g., always generating distinct elements when the bug is in duplicate handling). Generate adversarially: small
N, small value range, allow duplicates. - Comparator that compares output as strings without normalizing whitespace — false positives.
- Not seeding the RNG; one accidentally-passing run hides the bug.
18. Interactive Problems (CP-Style)
Definition
The problem statement defines an interactive protocol: you ask the judge queries (e.g., “is element i greater than element j?”), the judge answers, and after at most K queries you must report the answer. The judge runs as a subprocess and communicates via stdin/stdout. The technique is usually a binary search, ternary search, or adaptive query strategy bounded to O(log N) queries.
When Used
- “Find a hidden value in
[1, N]in at mostlog₂ Nqueries” — straight binary search. - “Find the minimum of a unimodal function” — ternary search.
- Adversarial / interactive game-tree problems.
Complexity
O(log N) queries for binary/ternary search; the algorithm is otherwise mostly bookkeeping.
Classic Problems
- CF 1207E (XOR Guessing) — adaptive queries.
- CF 1486D (Max Median) — interactive binary search.
- AtCoder ABC 178 D — not interactive but related decision-problem-as-binary-search.
Pitfalls
- Forgetting to flush stdout after every query. This is the single most common interactive bug. In C++:
cout << ... << endl;(orcout.flush();); in Python:print(...); sys.stdout.flush()orprint(..., flush=True). In Go:bufio.NewWriterwith explicitFlush(). If you don’t flush, the judge sees nothing, your program waits for input that never comes, you TLE. - Mixing
cin/coutwithscanf/printf— buffering interleaves badly. - Reading the judge’s response on the wrong line because of an off-by-one in the query loop.
19. Fast I/O
Definition
The default I/O mechanisms in most languages are line-buffered, locale-aware, and format-aware — which makes them slow. For CP, where you might read 10^6 integers in a 1-second time limit, fast I/O is mandatory. The technique varies by language:
- C++:
ios_base::sync_with_stdio(false); cin.tie(nullptr);— disconnectscin/coutfrom Cstdio. Speed-up: ~5×. Even faster:scanf/printfdirectly. - Java:
BufferedReader+StreamTokenizerfor input;PrintWriter(with explicitflush()) for output.Scanneris too slow for CP — never use it. - Python:
sys.stdin.readlineinstead ofinput();sys.stdout.writeinstead ofprintfor hot loops. For massive input:data = sys.stdin.buffer.read().split()and parse from there. PyPy3 is 5–10× faster than CPython for raw computation; use it whenever available. - Go:
bufio.NewReader(os.Stdin)andbufio.NewWriter(os.Stdout); always deferwriter.Flush().fmt.Scanis slow; use a custom token-by-token reader. - JavaScript / TypeScript (Node.js):
process.stdinraw read, parse all input at once. Generally the slowest mainstream CP language; not recommended forN ≥ 5·10^5.
When Used
- Always, on every CP problem with large input. Cost-of-not-using-fast-I/O: 5–10× slowdown, the difference between AC and TLE.
- Less critical on LeetCode-style problems where input is already parsed for you.
Complexity
I/O is O(input_size) either way; fast I/O reduces the constant by ~5–10×.
Classic Problems
- Any problem with
N ≥ 10^6integers as input. The constraint itself is a hint: “you need fast I/O”.
Pitfalls
- Mixing buffered and unbuffered I/O in the same program. In C++, after
sync_with_stdio(false), do not mixcinwithscanforcoutwithprintf. The buffers are independent and output appears out of order. - Forgetting to
Flush()in Go. Your output disappears entirely. - Java
Scanner. Don’t. - Python
printin a loop of10^6iterations. Each call locks stdout, flushes, formats — lethal for CP. Buffer with'\n'.join(...)and one finalsys.stdout.write.
Progression Playbooks (How To Practice Each Contest Track)
The 19 topics above are the vocabulary. The progression playbooks below are the training plans — which contests to enter, what the success bar is, and what to do after.
1. Codeforces Div 4 Progression
Target rating: unrated → 1200. Goal: solve A–F reliably, in <90 min, cleanly. Contest cadence: every Div 4, ~2/month. Why Div 4: the floor of CF; problems are LC-Easy to LC-Medium difficulty but with CF-style constraints. The skill being trained is speed — you should never get stuck on a Div 4 problem; if you do, the bottleneck is reading/typing speed, not algorithm knowledge.
How to practice: enter every Div 4 live. Aim for A–E in <60 min, F in <90 min total. After contest, upsolve any problems you missed, with editorial open. Track your fastest A–E solve times in a spreadsheet — they should drop ~30% over your first 5 Div 4 contests.
Exit criterion: solve A–F in 6/6 problems consistently in <100 min.
2. Codeforces Div 3 Progression
Target rating: 1200 → 1500. Goal: solve A–E reliably, attempt F. Contest cadence: every Div 3, ~2/month. Skill trained: pattern recognition. Div 3 problems mostly use the patterns from this curriculum (sweep, two pointers, basic DP, basic graph), but the framing is less explicit than LeetCode. You’ll see “given an array, do f(...)” with no LC tag telling you “this is binary search on the answer”.
How to practice: enter every Div 3 live. Aim for A–D in <60 min, E in <120 min. Upsolve E and F after contest. Read the editorial carefully — even if you solved it, see if there’s a slicker approach.
Exit criterion: solve A–E in 5/6 problems consistently within contest time.
3. Codeforces Div 2 Progression
Target rating: 1500 → 1900. Goal: solve A–C reliably, attempt D in 50% of contests. Contest cadence: every Div 2, ~3/month. Skill trained: problem-solving creativity. Div 2 D is where “knowing the technique” stops being enough — you must combine techniques. Div 2 D might require a sweep + DP, or a binary search + greedy, or a Fenwick tree + coordinate compression.
How to practice: enter every Div 2 live. Don’t worry about D in your first 10 contests. Aim for A, B, C clean. After contest, upsolve D with the editorial open; the goal is to learn techniques, not to spend 4 hours stuck. After 10 contests, start attempting D in-contest.
Exit criterion: solve A–C in 8/10 contests; D in 4/10 contests.
4. AtCoder Beginner Contest Progression
Target rating: unrated → 1400 (AtCoder). Goal: solve A–F. Contest cadence: every Saturday/Sunday, ~4/month. Why ABC: AtCoder’s problem statements are remarkably clean (often a single math problem stated tersely), and the difficulty curve A–F is smoother than CF Div 2/3. ABC F is roughly equivalent to CF Div 2 D but with cleaner statements.
How to practice: enter every ABC live. Aim for A–E in <60 min, F in <100 min. ABC F is famous for requiring exactly one “aha” insight per problem; if you don’t see it, move on and upsolve afterward — don’t grind in-contest.
Exit criterion: solve A–E in 9/10 contests; F in 5/10 contests.
5. AtCoder Regular Contest Progression
Target rating: 1400 → 1800 (AtCoder). Goal: solve A–C; attempt D. Contest cadence: ~1/month. Why ARC: ARC is harder than ABC and tests deeper CP techniques — segment tree beats, advanced combinatorics, harder geometry. ARC C is approximately CF Div 1 B / Div 2 E.
How to practice: enter every ARC live. Don’t expect to finish A–C in your first 5 attempts. Upsolve C and D after every contest. ARC is the contest where editorial reading delivers the most learning per minute, because the problems are designed around a specific technique that the editorial will name.
Exit criterion: solve A–B in 8/10 contests; C in 3/10 contests.
6. Stress Testing Methodology
Skill trained: finding bugs in your own solutions before the judge does. Cadence: every problem you’re <90% sure about. Tools: brute solver, candidate solver, random generator, comparator script. See Lab 06 for the full implementation.
How to practice: during a contest, when your candidate passes the samples but you have any doubt, write the stress test. It takes ~3 minutes; it saves the 50-point penalty of a wrong submission, plus the 30 minutes of re-solving after a WA. After contest, on every problem you got wrong, write a stress test that finds your bug. This builds the habit until it’s automatic.
Exit criterion: in 3 consecutive contests, no wrong submissions caused by bugs that would have been caught by stress testing.
7. Reading Editorials Productively
Skill trained: extracting transferable techniques rather than specific solutions. Cadence: after every contest, every problem. Why it matters: the difference between a 1500-rated and a 1900-rated coder is mostly that the 1900 has read 5× more editorials and retained them.
How to practice:
- Read the editorial before re-implementing your wrong solution. Do not patch your code; rewrite from blank using the editorial’s approach.
- Identify the technique name the editorial uses. “This is binary search on the answer.” “This is a two-pointer sliding window.” Add it to your personal technique catalog (a markdown file with one line per technique → one problem where you saw it).
- If the editorial is terse (AtCoder editorials are famously curt), look for community write-ups on Codeforces blogs.
- Re-implement from blank, in your own style.
Exit criterion: for every solved problem in your CP journal, you can name the technique and cite one other problem that uses it.
8. Implementation Speed Drills
Skill trained: typing your way out of the problem-statement-to-AC pipeline as fast as possible. Cadence: weekly, 30 min. Why it matters: in a 2-hour Div 2, you have ~25 min per problem on average. If you spend 10 min on syntax errors, you’ve lost 40% of your budget.
How to practice: pick 5 problems you’ve already solved. Type them again, from blank, against a stopwatch. Your second attempt should be 3–5× faster than your first. Do this on the canonical primitives — Sieve, modular inverse, binary exponentiation, Fenwick tree, sweep line — until you can write each from blank in <5 minutes without referring to notes.
Exit criterion: all 6 lab implementations in this phase, written from blank in <15 minutes each.
9. Contest-Time Strategy (Problem Ordering, When To Skip, When To Stress Test)
Skill trained: allocating limited contest time. Cadence: every contest.
The contest-time playbook:
- First 10 minutes: read all problems briefly. Mark each as “trivial / hard / ?”. Solve the trivial ones first to build momentum.
- Next 30 minutes: solve the medium ones. Allocate ~15 min per problem.
- When stuck for 15 min on one problem: skip. Move to the next. Come back later with a fresh perspective. The cost of grinding stuck is paid in opportunity cost on the next problem.
- When passing samples but uncertain: write a stress test. 3 minutes invested, 50-point penalty avoided.
- Last 30 minutes: decide between (a) attempting a hard problem you haven’t started, or (b) re-checking your earlier solutions. (b) is usually higher ROI unless the hard problem is worth a lot.
- Never give up before time expires. Even if every problem is solved or skipped, re-read the hardest unsolved problem one more time — sometimes the third reading triggers an insight the first two didn’t.
Common strategy mistakes:
- Grinding A–B for too long when they should take <15 min total. If A is taking >20 min, you’re misreading; re-read.
- Submitting before stress-testing on a problem you’re <90% sure about. The penalty hurts more than the time investment.
- Skipping the editorial post-contest because “I’ll do it later”. Later never comes.
Mastery Checklist
Tick when each item is true unprompted — i.e., you’d reach for it without consulting notes.
-
Read constraints first on every problem; can articulate why
N ≤ 18says bitmask,N ≤ 5000saysO(N²), etc. - Modular inverse via FLT in <2 minutes from blank; can switch to extgcd when modulus is composite.
- Binary exponentiation for integers in <2 minutes from blank.
- Matrix exponentiation for Fibonacci in <10 minutes from blank, including the matrix multiplication primitive.
- Sieve of Eratosthenes (basic) in <3 minutes; SPF sieve in <5 minutes; trial-division factorization in <2 minutes.
-
nCr mod pwith precomputed factorials in <5 minutes from blank. - gcd/lcm/extgcd in <3 minutes from blank.
- Cross product orientation test on demand; can identify CCW/CW/collinear by sign.
- Convex hull (Andrew’s monotone chain) in <15 minutes from blank.
- Sweep line for skyline in <20 minutes from blank.
- Coordinate compression as a one-line preprocessing.
- Mo’s algorithm template (block sort + add/remove handlers) in <20 minutes from blank.
- Sprague-Grundy on demand for impartial games up to small state space.
- Stress-testing harness (brute, candidate, generator, comparator) in <10 minutes from blank.
-
Interactive-problem template with explicit
flushafter every query. - Fast I/O configured by reflex in your primary language.
- Read an editorial productively: name the technique, find one other problem using it.
- Codeforces Div 3 A–E in 5/6 contests.
- Codeforces Div 2 A–C in 8/10 contests.
- AtCoder ABC A–F in 5/10 contests.
Exit Criteria
You graduate Phase 7 when all of the following hold:
- You have entered ≥10 Codeforces contests live (any division) and ≥10 AtCoder Beginner Contests live.
- You have a Codeforces rating of ≥1500 OR you have solved Codeforces Div 2 D in ≥3 contests, in or out of contest.
- All 6 labs in this phase are completed with all mastery criteria ticked.
- You can explain — out loud, in <60 seconds — what each of the 19 inline topics is, when to reach for it, and what its complexity is.
- You have a personal CP journal with ≥30 entries, each one linking the problem name to the named technique used.
- You have a stress-testing harness saved as a snippet in your editor and have used it to find a bug in your own code at least 5 times.
If any of these is missing — especially the contests and the journal — you have not exited this phase. Add 2 weeks and re-check.
Labs
| # | Lab | What It Builds |
|---|---|---|
| 1 | Modular Arithmetic | nCr mod p with factorial precomputation + modular inverse |
| 2 | Binary Exponentiation | a^b mod p and matrix exponentiation for Fibonacci |
| 3 | Sieve and Factorization | Count primes, sum of primes, SPF table for fast factorization |
| 4 | Sweep Line | The Skyline Problem (LC 218) via canonical sweep |
| 5 | Coordinate Compression | Counting inversions via Fenwick tree + compression |
| 6 | Stress Testing | Brute + candidate + random generator + comparator harness |
| 7 | Combinatorics mod p | Lucas, Catalan, stars-and-bars — beyond lab 1’s nCr |
| 8 | XOR Linear Basis | GF(2) Gaussian elimination: max subset XOR, k-th XOR value |
| 9 | Randomized Algorithms | Quickselect, Fisher-Yates, reservoir + rejection sampling |
← Phase 6 — Greedy · Phase 8 — Practical Engineering →
Lab 01 — Modular Arithmetic: nCr mod p With Precomputed Factorials
Goal
Master modular arithmetic and modular inverse by building a nCr mod p engine that answers any (n, r) query in O(1) after O(N) preprocessing, for n up to 10^7 and p = 10^9 + 7. By the end of this lab you can write the engine from blank in under 5 minutes.
Background
Counting problems modulo a prime are the single most common framing in competitive programming. The output line “print the answer modulo 10^9 + 7” appears on roughly 30% of CF Div 2/3 problems with combinatorial flavor. Behind that line is a fixed engine: precompute fact[i] = i! and inv_fact[i] = (i!)^(-1) modulo p, and C(n, r) = fact[n] · inv_fact[r] · inv_fact[n-r] mod p. Once you have the engine, dozens of problems collapse to “set up the formula, plug in, output”. The same machinery underlies probability problems where the answer is a/b mod p (output a · b^(-1) mod p).
Interview Context
Quant/HFT interviews use modular-counting problems as direct filters: “how many distinct length-k increasing sequences over [1, n], modulo 10^9 + 7?” If you can’t write the formula and the inverse-factorial trick fluently, you’ve failed in 10 minutes. FAANG L4 interviews almost never ask this directly, but a good L5+ candidate signals fluency by reaching for inv_fact[r] without explanation when a counting problem’s answer needs to be reduced. The signal interviewers want is “this candidate has CP background”; that signal is delivered by writing modular inverse without flinching.
Problem Statement
Given Q queries, each (n_i, r_i) with 0 ≤ r_i ≤ n_i ≤ N_max, output C(n_i, r_i) mod p for p = 10^9 + 7. The engine must support Q ≥ 10^6 queries in <2 seconds total.
LeetCode reference: LC 62 (Unique Paths) asks for C(m+n-2, m-1) directly (no mod needed). LC 920 (Number of Music Playlists) is a DP that uses modular arithmetic but not nCr directly. The pure CP framing appears on Codeforces (e.g., CF 1342E, CF 1342B).
Constraints
1 ≤ N_max ≤ 10^7(table size).1 ≤ Q ≤ 10^6(queries).0 ≤ r ≤ n ≤ N_max(well-formed query).p = 10^9 + 7(prime).- Time limit: 2 seconds. Memory limit: 256 MB. The factorial tables fit (
10^7 · 8 bytes ≈ 80 MB).
Clarifying Questions
- “Is
palways prime?” — yes; FLT works iffpprime. If composite, fall back to extgcd-based inverse and watch for non-invertible elements. - “Are
nandralways non-negative?” — yes; ifr < 0orr > n, return0by convention. - “Do queries arrive online or can they be batched?” — for this lab, online (one at a time after preprocessing),
O(1)each. - “Is
N_maxknown in advance?” — yes; we precompute up toN_max. - “Should I support Lucas’s theorem for
nlarger thanN_max?” — out of scope for this lab; see follow-up.
Examples
C(5, 2) mod p=10.C(10, 5) mod p=252.C(0, 0) mod p=1.C(1000000, 500000) mod (10^9+7)= a large nonzero value (engine must handle it).C(5, 7) mod p=0(well-formedness allowsr > nonly as boundary; we return 0).
Initial Brute Force
For each query, compute C(n, r) = n! / (r! · (n-r)!) using arbitrary-precision arithmetic (e.g., Python’s int), then mod p.
from math import factorial
def nCr_brute(n, r, p):
if r < 0 or r > n:
return 0
return (factorial(n) // (factorial(r) * factorial(n - r))) % p
Brute Force Complexity
Time O(n log² n) per query in arbitrary precision (factorial is O(n) multiplications of big ints, each O(log n) digits). For n = 10^7 and Q = 10^6, infeasible by ~6 orders of magnitude. Useful only as a stress oracle on n ≤ 30.
Optimization Path
- Brute force (above) — correctness baseline only.
- Precompute factorials, compute inverse on each query.
fact[i]table built inO(N); each query computesinv(fact[r]) · inv(fact[n-r]) · fact[n], withinv()takingO(log p)via FLT. TotalO(N + Q log p). Better but still2·10^7ops forQ = 10^6— passes but wasteful. - Precompute inverse factorials, query in
O(1). Afterfact[], computeinv_fact[N] = power(fact[N], p-2, p)once, theninv_fact[i] = inv_fact[i+1] · (i+1) mod pgoing backward. TotalO(N + Q). The optimal solution.
The going-backward trick is the key insight: (i!)^(-1) = ((i+1)!)^(-1) · (i+1), because i! · (i+1) = (i+1)!. So one expensive power call plus a backward sweep gives all inverse factorials in O(N).
Final Expected Approach
- Precompute
fact[0..N]withfact[0] = 1,fact[i] = fact[i-1] · i mod p. - Compute
inv_fact[N] = power(fact[N], p - 2, p)via binary exponentiation (FLT). - Compute
inv_fact[i] = inv_fact[i+1] · (i+1) mod pforifromN-1down to0. - Each query:
if r < 0 or r > n: return 0; else return fact[n] · inv_fact[r] · inv_fact[n-r] mod p.
Data Structures Used
- Two flat arrays of
long long(orint64):fact[0..N]andinv_fact[0..N]. - One scalar
p. - A
power(a, b, m)helper (binary exponentiation).
No heaps, no maps, no trees. The whole thing is two arrays and a closed-form formula.
Correctness Argument
FLT proof of inverse. Fermat’s Little Theorem states: for prime p and gcd(a, p) = 1, a^(p-1) ≡ 1 (mod p). Multiplying both sides by a^(-1): a^(p-2) ≡ a^(-1) (mod p). So power(a, p-2, p) is the modular inverse of a whenever a ≢ 0 (mod p). For a = fact[i], since fact[i] = i! < p for i < p and the product of nonzero-mod-p values, gcd(fact[i], p) = 1, so the inverse exists.
Backward inverse-factorial recurrence. We have inv_fact[i+1] = ((i+1)!)^(-1) and want inv_fact[i] = (i!)^(-1). Since (i+1)! = i! · (i+1), taking inverses: ((i+1)!)^(-1) = (i!)^(-1) · (i+1)^(-1), equivalently (i!)^(-1) = ((i+1)!)^(-1) · (i+1). So inv_fact[i] = inv_fact[i+1] · (i+1) mod p. Base case at i = N is given by the explicit FLT call.
nCr formula correctness. C(n, r) = n! / (r! · (n-r)!). In Z/pZ, division by x is multiplication by x^(-1). So C(n, r) ≡ fact[n] · inv_fact[r] · inv_fact[n-r] (mod p). ✓
Complexity
- Preprocess:
O(N + log p)time (onepowercall, two linear sweeps),O(N)space. - Each query:
O(1)time,O(1)space. - Total for
N = 10^7andQ = 10^6: ~10^7 + 10^6mults, well under 2 seconds in C++/Java/Go; in Python use PyPy or numpy-accelerated arithmetic.
Implementation Requirements
- Use
long long(C++) /int64(Go) /int(Python; Javalong). Cast operands before multiplying to avoid overflow:(long long)fact[n] * inv_fact[r] % p. powerhelper handlesb = 0returning1andm = 1returning0.- Guard
r < 0 || r > nreturning0. - Mod once after every multiplication, not at the end.
Tests
def test_nCr():
p = 10**9 + 7
eng = NCrEngine(N=20, p=p)
assert eng.nCr(5, 2) == 10
assert eng.nCr(10, 5) == 252
assert eng.nCr(0, 0) == 1
assert eng.nCr(1, 0) == 1
assert eng.nCr(1, 1) == 1
assert eng.nCr(5, 7) == 0 # r > n
assert eng.nCr(5, -1) == 0 # r < 0
# Stress vs brute on small N:
import random
for _ in range(1000):
n = random.randint(0, 20)
r = random.randint(-2, 22)
assert eng.nCr(n, r) == nCr_brute(n, r, p)
Edge cases: n = 0, r = 0, r = n, r > n, r < 0, n = N_max.
Follow-up Questions
- “What if
pis not prime?” — FLT fails. Use extgcd-based inverse, but be aware that not everyais invertible (only those withgcd(a, p) = 1). - “What if
nis up to10^18butpis small (≤10^5)?” — Lucas’s theorem: writen, rin basep, multiplyC(n_i, r_i) mod pdigit-wise. - “What if you need
C(n, r) mod 4?” — neither FLT nor straightforward Lucas applies; use Kummer’s theorem or direct computation.
Product Extension
Probability/statistics services (e.g., AdTech bid pacing, fraud-risk scoring) compute combinatorial denominators on the fly. The factorial-precomputation engine is the production primitive: build it once at service startup, query it O(1) for the lifetime of the server. Same machinery is in CryptoLib’s prime-modulus arithmetic and in numerical libraries (NumPy’s comb calls down to a similar routine for large arguments).
Language / Runtime Follow-ups
- Python: integers are arbitrary precision, so overflow isn’t a concern, but per-mult cost is ~5× C++. Use
pow(a, b, m)(built-inO(log b)modular exponentiation). Precompute asnumpy.int64arrays only ifN ≤ 10^6; otherwise plain lists. - Java: use
longeverywhere; cast to(long)before multiplying.BigInteger.modPowworks but is 10× slower than a hand-rolled loop. - Go:
int64everywhere;math/bigif you need extra safety. Hand-roll the loop for performance. - C++:
long long(int64_t),(long long)a * b % p. The product of two values <p ≈ 10^9fits inlong long(<2^63). With unsigned overflow concerns, useunsigned long longand% pdefensively. - JS/TS:
Numberisdouble-precision and loses integer precision above2^53. UseBigInt, but it’s ~10× slower; avoid for hot loops larger than10^6ops.
Common Bugs
- Forgetting to mod after every multiplication; silent overflow.
- Using
**(exponentiation) where the language doesn’t have a modular form; computes a 5MB number first, then mods. Usepow(a, b, m)(Python) or your own loop. - Computing
inv_fact[i]directly viapower(fact[i], p-2, p)for everyi. Correct butO(N log p); the backward sweep isO(N). - Off-by-one:
inv_facthasN+1entries (inv_fact[0..N]); allocate accordingly. - Returning
1forC(n, -1)orC(n, n+1)instead of0. Always guard.
Debugging Strategy
If nCr(n, r) disagrees with brute on small cases:
- Print
fact[0..n]and confirmfact[i] = i!fori ≤ 10. - Print
inv_fact[0..n]and confirmfact[i] · inv_fact[i] ≡ 1 (mod p)for eachi. - If step 2 fails: the FLT exponent is wrong. Check
power(fact[N], p-2, p), notp-1. - If step 2 passes but
nCris wrong: the formula isfact[n] · inv_fact[r] · inv_fact[n-r]. Check you’re not usingfact[n-r](withoutinv_).
Mastery Criteria
- Write the engine (factorial table + inverse-factorial backward sweep + query) from blank in <5 minutes.
-
Articulate why
power(fact[N], p-2, p)givesinv_fact[N]in one sentence (FLT). -
Articulate why
inv_fact[i] = inv_fact[i+1] · (i+1)in one sentence (telescope). - Recognize “answer mod prime” + “counting / paths / arrangements” as the trigger for this engine within 60 seconds of reading a problem.
-
Switch to extgcd-based inverse if asked “what if
pis not prime?” - State Lucas’s theorem on demand and explain when it’s needed.
← Phase 7 README · Lab 02 — Binary Exponentiation →
Lab 02 — Binary Exponentiation and Matrix Exponentiation for Fibonacci
Goal
Master O(log b) exponentiation in two settings: integer fast power (a^b mod p) and matrix fast power (Fibonacci F(n) mod p for n up to 10^18). By the end, you can write integer fast power in <2 minutes and matrix Fibonacci in <10 minutes from blank.
Background
Binary exponentiation is the engine behind almost every “compute a^b for huge b” subroutine in CP and cryptography. The same divide-and-conquer pattern — a^b = (a^(b/2))^2 if b even, else a · a^(b-1) — generalizes from integers to any associative operation: matrix multiplication (linear recurrences), polynomial multiplication (signal processing), function composition (rotation by an angle, repeated f(f(f(...)))). Internalize the O(log b) skeleton once and you get all these for free.
Matrix exponentiation is the canonical extension. Fibonacci, defined F(n) = F(n-1) + F(n-2), has the matrix form [F(n+1), F(n)]^T = M · [F(n), F(n-1)]^T where M = [[1, 1], [1, 0]]. Therefore [F(n+1), F(n)]^T = M^n · [F(1), F(0)]^T = M^n · [1, 0]^T. Computing M^n takes O(log n) matrix multiplications, each O(2^3) = O(8) operations. Total: O(log n) for n = 10^18, about 60 multiplications.
Interview Context
Quant interviews use exactly this pair. “Compute 2^(10^18) mod (10^9 + 7)” is a 2-minute warm-up. “Compute F(10^18) mod (10^9 + 7)” is the follow-up that filters out candidates who only know the iterative O(N) Fibonacci. A candidate who reaches for matrix exponentiation reflexively when seeing n ≤ 10^18 and a linear recurrence is signalling 1900+ CF rating, which is a strong positive signal at HFT firms.
Problem Statement
Part 1. Implement power(a, b, m) returning a^b mod m for 0 ≤ a < m, 0 ≤ b ≤ 10^18, 1 ≤ m ≤ 10^9 + 7.
Part 2. Implement fib(n, p) returning F(n) mod p for 0 ≤ n ≤ 10^18, p = 10^9 + 7, where F(0) = 0, F(1) = 1, F(n) = F(n-1) + F(n-2).
LeetCode reference: LC 50 (Pow(x, n)) — Part 1 in real-number form. LC 1137 (N-th Tribonacci) — Part 2 with three-term recurrence (analogous matrix form).
Constraints
- Part 1:
bup to10^18, so naiveO(b)looping is impossible. - Part 2:
nup to10^18, so naiveO(n)Fibonacci is impossible. - Time limit: 2 seconds.
- Memory: O(1) for Part 1, O(1) for Part 2 (matrices are 2×2).
Clarifying Questions
- “Negative
bfor integer power?” — for modulara^b, undefined unlessgcd(a, m) = 1and you wanta^(-1)^|b|. For reala^b(LC 50), return1 / a^|b|and watch forINT_MINoverflow on-b. - “What’s
0^0?” — by convention1. - “Fibonacci indexing — is
F(1) = 1orF(2) = 1?” — confirm; standard CP convention isF(0) = 0, F(1) = 1. - “Matrix exponentiation modulus — same
peverywhere?” — yes. - “Required output format for matrices?” — only the scalar Fibonacci value, but you might be asked to return the full state vector.
Examples
power(2, 10, 1000)=24(2^10 = 1024).power(3, 0, 7)=1.power(5, 1000000000000000000, 10^9 + 7)= some specific value (must compute).fib(0, p)=0,fib(1, p)=1,fib(10, p)=55.fib(10^18, 10^9 + 7)= a specific value the engine must produce.
Initial Brute Force
Part 1: result = 1; for i in 1..b: result = result * a % m. O(b).
Part 2: iterative two-variable Fibonacci. O(n).
def power_brute(a, b, m):
result = 1 % m
for _ in range(b):
result = result * a % m
return result
def fib_brute(n, p):
if n == 0: return 0
a, b = 0, 1
for _ in range(n - 1):
a, b = b, (a + b) % p
return b
Brute Force Complexity
Part 1: O(b) mults, b = 10^18 is 10^17 mults/sec required — impossible. Part 2: same. Useful only as oracles on n ≤ 30.
Optimization Path
Part 1.
- Naive
O(b)(above). - Recursive
O(log b):power(a, b) = power(a, b/2)² if b even, else a · power(a, b-1). - Iterative
O(log b): process bits ofblow-to-high, squareaeach iteration, multiply into result when the current bit ofbis 1. Preferred for stack safety.
Part 2.
- Naive
O(n)(above). - Memoized doubling:
F(2k) = F(k) · (2 · F(k+1) − F(k)),F(2k+1) = F(k)² + F(k+1)².O(log n)recursion. Beautiful but error-prone. - Matrix exponentiation: build
M = [[1,1],[1,0]], computeM^nvia integer-fast-power lifted to matrices, extractM^n[0][1]asF(n).O(log n) · O(8)= ~500 ops forn = 10^18. The general-purpose technique that works for any linear recurrence.
Final Expected Approach
Part 1 (iterative):
long long power(long long a, long long b, long long m) {
long long res = 1 % m;
a %= m;
while (b > 0) {
if (b & 1) res = res * a % m;
a = a * a % m;
b >>= 1;
}
return res;
}
Part 2 (matrix):
typedef vector<vector<long long>> Mat;
const long long P = 1e9 + 7;
Mat matmul(const Mat &A, const Mat &B) {
int n = A.size();
Mat C(n, vector<long long>(n, 0));
for (int i = 0; i < n; ++i)
for (int k = 0; k < n; ++k)
if (A[i][k])
for (int j = 0; j < n; ++j)
C[i][j] = (C[i][j] + A[i][k] * B[k][j]) % P;
return C;
}
Mat matpow(Mat M, long long e) {
int n = M.size();
Mat res(n, vector<long long>(n, 0));
for (int i = 0; i < n; ++i) res[i][i] = 1; // identity
while (e > 0) {
if (e & 1) res = matmul(res, M);
M = matmul(M, M);
e >>= 1;
}
return res;
}
long long fib(long long n) {
if (n == 0) return 0;
Mat M = {{1, 1}, {1, 0}};
Mat R = matpow(M, n);
return R[0][1];
}
Data Structures Used
- Part 1: scalars only.
- Part 2: 2×2 matrices as
vector<vector<long long>>orarray<array<long long, 2>, 2>.
Correctness Argument
Part 1 (binary exponentiation invariant). Let b = sum b_i 2^i be the binary representation of the original exponent. After k iterations, the variable a equals a_initial^(2^k) mod m, and res equals the product of a_initial^(2^i) mod m for all i < k with b_i = 1. After all iterations, res = a_initial^b mod m. The invariant proves correctness; termination is b → 0 after floor(log₂ b) + 1 iterations.
Part 2 (matrix recurrence). Define the column vector v(k) = [F(k+1), F(k)]^T. Then M · v(k) = [[1,1],[1,0]] · [F(k+1), F(k)]^T = [F(k+1) + F(k), F(k+1)]^T = [F(k+2), F(k+1)]^T = v(k+1). By induction, v(n) = M^n · v(0) = M^n · [1, 0]^T, so F(n) = v(n)[1] = (M^n · [1, 0]^T)[1] = M^n[1][0]. Equivalently, M^n[0][1] = F(n) (by symmetry of the Fibonacci matrix). Matrix multiplication is associative, so binary exponentiation lifts directly: same invariant, just with matrices.
Complexity
- Part 1:
O(log b)time,O(1)space. - Part 2:
O(K^3 log n)time forK × Kmatrices (K = 2for Fibonacci →8 log n≈ 480 ops forn = 10^18),O(K^2)space.
Implementation Requirements
- All multiplications mod
pimmediately. - Identity matrix initialization for
matpow. - Handle
n = 0(return 0 directly, notM^0 · [1, 0]^T = [1, 0]^Twhich would giveF(0) = 0correctly anyway, but watch corner cases). - Use
long long(orint64); intermediate products are up to(10^9)² ≈ 10^18, just fittinglong long.
Tests
def test_power():
assert power(2, 10, 1000) == 24
assert power(3, 0, 7) == 1
assert power(0, 0, 7) == 1
assert power(0, 5, 7) == 0
assert power(2, 63, 10**18) == 2**63 % 10**18
def test_fib():
p = 10**9 + 7
expected = [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55]
for i, e in enumerate(expected):
assert fib(i, p) == e
# Stress vs brute up to N = 30
for n in range(31):
assert fib(n, p) == fib_brute(n, p)
# Spot-check large
assert fib(10**18, p) == 209783453 # known value, mod 1e9+7
Follow-up Questions
- “Compute
F(n)for general linear recurrencef(n) = c1 f(n-1) + ... + ck f(n-k)?” — same matrix exponentiation, with ak×kcompanion matrix. - “Compute the number of paths of length
Lbetween two nodes in a graph?” —(adj_matrix)^L, indexed by start/end. SameO(V^3 log L)engine. - “Compute Tribonacci
T(n)inO(log n)?” — 3×3 companion matrix, otherwise identical. - “Avoid
O(K^3)per multiplication for hugeK?” — research topics: Kitamasa’s algorithm reduces toO(K^2 log N); FFT-based polynomial multiplication reduces further. Out of scope here.
Product Extension
Cryptography (RSA encryption: compute m^e mod n with e, n 2048-bit, in milliseconds — same algorithm, with bignum). Computer graphics (rotation matrix R^n for repeated rotations). Markov chain steady-state approximation (P^n for stochastic matrix P, large n). Network reachability (adj^L for paths of length L).
Language / Runtime Follow-ups
- Python: built-in
pow(a, b, m)isO(log b)and ~10× faster than a hand-rolled Python loop. For matrix power: hand-roll the multiplication; numpy is overkill atK = 2(FFI overhead exceeds compute). - Java:
BigInteger.modPowexists but is 10× slower than a hand-rolledlongloop. Use the hand-rolled version unless values exceedlong. - Go:
math/big.Int.Exp(a, b, m)is correct but slow; hand-roll for hot paths. Matrix: use 2D[][]int64arrays, notmath/big. - C++:
__int128for intermediate products ifmexceeds~3·10^9(where(long long)² mod poverflows). For standardm = 10^9 + 7, plainlong longsuffices. - JS/TS:
BigIntis correct but slow; form < 2^32, useNumbercarefully (Math.floorafter* a / m) — easy to get wrong. Matrix: same caveat.
Common Bugs
- Forgetting
res = 1 % mat start (returns1instead of0whenm = 1). - Squaring
abefore checking the bit (results in extra unused multiplication; correctness OK, perf hit). - Matrix multiplication order:
M^n = M · M · ... · M, butmatmulis non-commutative — be deliberate about left/right. - Using
intinstead oflong longfor matrix entries;(10^9)² > 2^31. - Recursive
powerblowing the stack onb = 10^18(recursion depth 60 is fine; just don’t go deeper).
Debugging Strategy
If power(a, b, m) is wrong:
- Test on small cases (
b ≤ 5) where you can verify by hand. - Print binary representation of
band confirm the bits are processed. - If correct on small
bbut wrong on large: overflow. Check thata * a % museslong long, notint.
If fib(n, p) is wrong:
- Verify
M^1 = M,M^2 = [[2,1],[1,1]],M^3 = [[3,2],[2,1]]. Each entry is a Fibonacci number. - Verify
fib(0..10)matches the canonical sequence. - If small
nworks but large doesn’t: same overflow check.
Mastery Criteria
-
Write
power(a, b, m)from blank in <2 minutes. - Write the Fibonacci matrix-exponentiation engine from blank in <10 minutes.
- Articulate the binary-exponentiation invariant in one sentence.
-
Articulate the matrix-Fibonacci recurrence (
v(n+1) = M · v(n)) in 30 seconds. - Generalize to any linear recurrence: given the recurrence, write the companion matrix in <2 minutes.
- Recognize “n up to 10^18 + linear recurrence” as the trigger for matrix exponentiation in <60 seconds.
← Lab 01 — Modular Arithmetic · Phase 7 README · Lab 03 — Sieve and Factorization →
Lab 03 — Sieve of Eratosthenes and Smallest-Prime-Factor Factorization
Goal
Master prime enumeration and integer factorization at competitive scale: count primes up to N = 5 · 10^6 in under 100ms, and factorize Q = 10^5 integers ≤ N in O(log n) each via a precomputed smallest-prime-factor (SPF) table. By the end, you can write the linear sieve from blank in <5 minutes.
Background
Many CP problems reduce to “is x prime?”, “what are the prime factors of x?”, or “how many primes ≤ N?”. For N up to a few million, a sieve answers all three in O(N log log N) (Eratosthenes) or O(N) (Euler/linear sieve), and the SPF byproduct lets you factorize any x ≤ N in O(log x) time. For larger N (up to 10^9), you need Miller-Rabin primality tests and Pollard’s rho for factorization (out of scope here).
The sieve is the single most reused primitive across number-theoretic CP problems. Get this fluent and you save 5–10 minutes per problem.
Interview Context
LC 204 (Count Primes) is the canonical screen at FAANG mid-level. Quant interviews ramp it up: “factorize each of 10^5 numbers up to 10^7”. Both questions test the same primitive, but at different scales — the scale forces the candidate to pick the right data structure (bitset vs vector<bool> vs SPF table). A candidate who reaches for SPF without prompting signals CP fluency.
Problem Statement
Three sub-problems on the same engine:
- Count primes ≤ N. LC 204.
N ≤ 5 · 10^6. - Sum of primes ≤ N. Variant: instead of count, return sum mod
10^9 + 7. - Factorize Q numbers ≤ N. For each
x_i, output its multiset of prime factors.Q ≤ 10^5,x_i ≤ N.
Constraints
N ≤ 5 · 10^6for sub-problems 1 and 2.N ≤ 10^7,Q ≤ 10^5for sub-problem 3.- Time limit: 1 second.
- Memory limit: 256 MB. SPF as
intarray uses4 · 10^7 = 40 MB; fine.
Clarifying Questions
- “Is
0and1prime?” — neither. Standard convention. - “Factorization output: ordered or multiset?” — multiset, e.g.,
12 → [2, 2, 3]. - “Are inputs guaranteed
≤ N?” — yes for this lab. Otherwise switch to trial divisionO(√x)or Pollard’s rho. - “Need primes only or all factors?” — primes only here (canonical). Divisors are a separate problem.
- “Single-threaded?” — yes. Sieves don’t parallelize trivially without contention.
Examples
- Sub-problem 1:
count_primes(10) = 4(2, 3, 5, 7).count_primes(2) = 1.count_primes(1) = 0. - Sub-problem 2:
sum_primes(10) = 17. - Sub-problem 3:
factorize(12) = [2, 2, 3].factorize(1) = [].factorize(7) = [7].factorize(60) = [2, 2, 3, 5].
Initial Brute Force
For sub-problem 1: for each x in [2, N], run trial division up to √x.
def is_prime(x):
if x < 2: return False
for d in range(2, int(x**0.5) + 1):
if x % d == 0: return False
return True
def count_primes_brute(N):
return sum(1 for x in range(2, N) if is_prime(x))
For sub-problem 3: trial division of each x_i.
def factorize_brute(x):
out = []
d = 2
while d * d <= x:
while x % d == 0:
out.append(d); x //= d
d += 1
if x > 1: out.append(x)
return out
Brute Force Complexity
Sub-problem 1 trial division: O(N √N). For N = 5·10^6, that’s ≈ 10^10 ops — impossible in 1 second.
Sub-problem 3 trial division per query: O(√x). For Q = 10^5 and x = 10^7, that’s ~3·10^8 ops — borderline. Sieve-based factorization is O(log x) ≈ 24 ops, ~2.4·10^6 total — comfortably faster.
Optimization Path
- Trial division (above).
- Sieve of Eratosthenes for sub-problems 1, 2: mark composites; remaining are primes.
O(N log log N). - Linear (Euler) sieve + SPF table: each composite is crossed off exactly once via its smallest prime factor; the SPF byproduct enables
O(log x)factorization.O(N)preprocessing.
For interviews, Eratosthenes is fine; for CP the linear sieve is the default once you’ve drilled it.
Final Expected Approach
Sieve of Eratosthenes (sub-problems 1, 2):
vector<bool> sieve(int N) {
vector<bool> is_prime(N + 1, true);
is_prime[0] = is_prime[1] = false;
for (int i = 2; (long long)i * i <= N; ++i)
if (is_prime[i])
for (int j = i * i; j <= N; j += i)
is_prime[j] = false;
return is_prime;
}
Linear sieve with SPF table (sub-problem 3):
vector<int> spf; // smallest prime factor
vector<int> primes;
void linear_sieve(int N) {
spf.assign(N + 1, 0);
primes.clear();
for (int i = 2; i <= N; ++i) {
if (spf[i] == 0) { spf[i] = i; primes.push_back(i); }
for (int p : primes) {
if ((long long)p * i > N || p > spf[i]) break;
spf[p * i] = p;
}
}
}
vector<int> factorize(int x) {
vector<int> out;
while (x > 1) { out.push_back(spf[x]); x /= spf[x]; }
return out;
}
Data Structures Used
vector<bool>(or bitset for memory efficiency) for the sieve mark array.vector<int>for the SPF table (one int per index up toN).vector<int>for the prime list.
A bitset packs 8× denser than vector<bool>; for very large N (5 · 10^7) it matters.
Correctness Argument
Eratosthenes correctness. Inductive claim: when iteration i starts, is_prime[j] is correct for all j < i. If is_prime[i] == true, no j < i has j | i (else it would have unmarked i); so i has no proper divisor < i, so i is prime. We then mark all multiples i², i² + i, i² + 2i, ... as composite. Multiples below i² were already marked by smaller prime divisors. By strong induction, the array is fully correct after the loop terminates.
Linear sieve correctness. Each composite n is marked exactly once, when i = n / spf(n) and the inner loop reaches p = spf(n). The loop guard p > spf[i] ensures p ≤ spf[i], so p · i has smallest prime factor p (because p ≤ spf[i] ≤ all other prime factors of i). So the assignment spf[p * i] = p is correct. Each composite has a unique (i, p) pair, hence is visited exactly once → O(N) total work.
Factorization correctness. While x > 1, spf[x] is x’s smallest prime factor, so emit it and divide. Every iteration strictly decreases x by a factor of at least 2, so loop runs ≤ log₂ x times.
Complexity
- Eratosthenes:
O(N log log N)time,O(N / 8)space with bitset. - Linear sieve + SPF:
O(N)time,O(N)space. - Factorize each query:
O(log x). - Total for sub-problem 3:
O(N + Q log x).
Implementation Requirements
- Use
(long long)i * ito computei²to avoid overflow at largeN. - Sieve inner loop starts at
i², not2i— multiples belowi²are already marked. - For sum-of-primes mod
p, mod the running sum after each addition. - SPF allocation:
N + 1entries (for indexN).
Tests
def test_count_primes():
assert count_primes(10) == 4
assert count_primes(2) == 1
assert count_primes(1) == 0
assert count_primes(100) == 25
# Stress vs brute up to N = 1000
for n in range(2, 1001):
assert count_primes(n) == count_primes_brute(n)
def test_factorize():
linear_sieve(1000)
assert factorize(12) == [2, 2, 3]
assert factorize(7) == [7]
assert factorize(1) == []
assert factorize(60) == [2, 2, 3, 5]
# Stress vs brute
for x in range(1, 1001):
assert sorted(factorize(x)) == sorted(factorize_brute(x))
Edge cases: N = 0, N = 1, N = 2, prime N, prime power N.
Follow-up Questions
- “What if
N = 10^9?” — sieve is impossible (memory + time). Use Miller-Rabin for primality, Pollard’s rho for factorization. - “What if
N = 10^11and you need only the count?” — Meissel-Mertens method,O(N^(2/3)). Out of scope here. - “How would you parallelize the sieve?” — segmented sieve: split
[2, N]into blocks of size√N, sieve each block independently after computing primes ≤√N. Each block fits in cache; threads work on disjoint blocks.
Product Extension
Cryptography service: precompute small primes for trial division as a Miller-Rabin pre-filter. Number-theoretic libraries (GMP, FLINT) cache a small-prime table at startup. RSA key generation does trial division up to ~10^5 before falling back to Miller-Rabin, yielding ~10× speedup on rejecting composites.
Language / Runtime Follow-ups
- Python: plain Python list of bools is slow; use
bytearray(8× faster than list-of-bool). ForN = 5·10^6,bytearraysieve runs in ~0.3 s; PyPy / Cython get to ~50 ms. - Java:
BitSetis more memory-efficient thanboolean[]and ~equally fast. - Go: plain
[]boolis fine; built-inbitsetdoesn’t exist (write your own with[]uint64). - C++:
vector<bool>is bit-packed by default; usebitset<N+1>for stack-allocated speed at compile-time-fixedN. - JS/TS:
Uint8Arrayfor the sieve.Numberinteger math is exact below2^53.
Common Bugs
- Sieve inner loop starting at
2iinstead ofi²— correct but slower. - Marking
is_prime[0]andis_prime[1]as true. - Off-by-one: allocating
Nentries instead ofN + 1. - In linear sieve, missing the
p > spf[i]break condition → quadratic behavior. - Computing
i * iasintand overflowing fori > ~46340. - For factorization output, dividing
xbyspf[x]but forgetting to also pushspf[x].
Debugging Strategy
If sieve is wrong:
- Print primes ≤ 30 from your sieve. Should be
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]. If 1 is included or 9 is missed, check init / inner loop start.
If factorize is wrong:
- Print
spf[2..30]. Should be[2,3,2,5,2,7,2,3,2,11,2,13,2,3,2,17,...](smallest prime factor of each index). - If SPF correct but factorization wrong: the loop should emit
spf[x]and divide; common bug is doing one or the other.
Mastery Criteria
- Write Sieve of Eratosthenes from blank in <3 minutes.
- Write linear sieve with SPF from blank in <5 minutes.
- Articulate why each composite is marked exactly once in linear sieve.
- Use SPF to factorize an integer in <30 seconds.
-
Recognize
N ≤ 10^7+ factor-related question as the sieve trigger within 30 seconds. -
State the alternative when
N > 10^9(Miller-Rabin + Pollard’s rho).
← Lab 02 — Binary Exponentiation · Phase 7 README · Lab 04 — Sweep Line →
Lab 04 — Sweep Line: The Skyline Problem
Goal
Master the sweep-line paradigm by solving the canonical Skyline Problem (LC 218). Process N rectangles by sweeping left-to-right over event points, maintaining a set of active heights, and emitting key points when the maximum height changes. By the end, you can write the full algorithm from blank in <15 minutes.
Background
Sweep line is a meta-technique: convert geometric or interval problems on N objects into a sequence of 2N events (one when an object enters, one when it leaves), sort the events, then process them in order while maintaining a dynamic data structure (multiset, segment tree, BIT) that summarizes the currently active set. The cost shifts from “compare every pair” O(N²) to “sort + log-time updates” O(N log N).
The skyline problem is the canonical sweep-line interview question because it forces every component: event extraction, event sorting (with non-obvious tie-breaking), a dynamic max query, and careful output deduplication. Internalize this and you can derive sweep-line variants for rectangle area unions, interval intersection counting, point-in-rectangle queries, and convex hull (Andrew’s monotone chain).
Interview Context
LC 218 (Hard) — by far the most-asked Hard at FAANG mid-level interviews when the panel wants to test sweep line. The bar is very high: a passing solution must be O(N log N), must handle ties correctly, must deduplicate output, and must not emit phantom key points. A candidate who hand-waves through tie-breaking will be rejected even with otherwise-correct code. Quant interviews use rectangle-union-area, which is the same engine with an integral instead of a max.
Problem Statement
Given N buildings, each described by [left_i, right_i, height_i], return the skyline outline as a list of key points [x, y], where y is the height of the skyline at x and key points appear only when the height changes. The last key point has y = 0 (where the rightmost building ends).
LeetCode reference: LC 218 (The Skyline Problem).
Constraints
1 ≤ N ≤ 10^4(LC), realistically up to10^5for harder variants.0 ≤ left_i < right_i ≤ 2^31 − 1.1 ≤ height_i ≤ 2^31 − 1.- Output is sorted by
x. No two consecutive key points have the samey.
Clarifying Questions
- “Are buildings axis-aligned and non-rotated?” — yes (skyline assumption).
- “Can buildings overlap?” — yes, freely.
- “Is the ground at
y = 0?” — yes; the skyline ends with[x_max, 0]. - “Output format: key points where height changes, or every event point?” — only changes; consecutive duplicates are bugs.
- “Tie-breaking for events at the same
x?” — opens before closes (the building exists at that exactx, contributing its height).
Examples
- Input:
[[2, 9, 10], [3, 7, 15], [5, 12, 12], [15, 20, 10], [19, 24, 8]]. - Output:
[[2, 10], [3, 15], [7, 12], [12, 0], [15, 10], [20, 8], [24, 0]].
Walking through: at x = 2, height jumps from 0 to 10 → emit [2, 10]. At x = 3, height jumps to 15 → emit [3, 15]. At x = 7, building of height 15 ends, max drops to 12 → emit [7, 12]. At x = 12, last “left-cluster” building ends → emit [12, 0]. Then the right cluster starts at 15 ([15, 10]), 19 doesn’t change max (still 10 since 8 < 10), 20 ends 10-building → emit [20, 8], 24 ends 8-building → emit [24, 0].
Initial Brute Force
For each x from 0 to x_max, compute the max height over all buildings covering x. Emit [x, h] whenever h changes.
def skyline_brute(buildings):
x_max = max(b[1] for b in buildings)
out = []
prev_h = 0
for x in range(x_max + 1):
h = max((b[2] for b in buildings if b[0] <= x < b[1]), default=0)
if h != prev_h:
out.append([x, h])
prev_h = h
return out
Brute Force Complexity
O(x_max · N). For coordinates up to 2^31, infeasible. Even on small CP-scale x_max = 10^9, no chance. Useful only as oracle on x_max ≤ 100.
Optimization Path
- Brute (above).
- Event sweep with sorted multiset. Generate
2Nevents:(left, -h, OPEN)and(right, h, CLOSE). Sort. Sweep, maintaining a multiset of active heights. Emit[x, max_active]when max changes.O(N log N). - Event sweep with a max-heap and lazy deletion. Same idea, but the heap doesn’t natively support deletion; instead, store
(height, end_time)pairs and pop stale entries lazily. Sometimes faster constant factor thanmultiset.O(N log N). - Divide and conquer. Split buildings into two halves, solve recursively, merge skylines (similar to merge sort).
O(N log N).
The multiset approach is the cleanest in C++/Java; Python defaults to the heap-with-lazy-deletion style.
Final Expected Approach
Heap with lazy deletion (Python idiom):
import heapq
def get_skyline(buildings):
events = []
for L, R, H in buildings:
events.append((L, -H, R)) # opening: negative height for max-heap via min-heap
events.append((R, 0, 0)) # closing sentinel: process at this x
events.sort()
result = []
heap = [(0, float('inf'))] # (negative height, end_time); ground is height 0 forever
i = 0
n = len(events)
while i < n:
x = events[i][0]
# Process all events at this x: add openings.
while i < n and events[i][0] == x:
L, neg_H, R = events[i]
if neg_H < 0: # an opening
heapq.heappush(heap, (neg_H, R))
i += 1
# Lazy-pop expired buildings.
while heap[0][1] <= x:
heapq.heappop(heap)
cur_max = -heap[0][0]
if not result or result[-1][1] != cur_max:
result.append([x, cur_max])
return result
Data Structures Used
- A list of events, sorted.
- A max-heap (or multiset / sorted set) of active heights with their end times.
- A result list with deduplication on consecutive
y.
Correctness Argument
Sweep correctness. Define H(x) = max heights of buildings covering x. The function H changes value only at event coordinates (the boundaries of buildings). So if we sample H at every event coordinate (in sorted order), we capture every change. The result is the unique sequence of changes in H, which is the skyline.
Tie-breaking at equal x. Multiple events can share an x: an opening, a closing, or both. We process all events at this x together: first add all openings (their buildings exist at this x, contributing their height), then lazy-pop closings (those buildings no longer cover any x ≥ this x). After all events at x are processed, the heap reflects the active set on [x, x+1), and we read cur_max. Emit [x, cur_max] if it differs from the previous emission. This handles “two buildings of different heights both starting at the same x” (emit the taller), “one building closing at the same x another opens” (emit the new max), and “two buildings of equal height with overlapping ranges” (no change at the boundary).
Lazy deletion correctness. We never pop a building from the heap until we’ve passed its end time. The heap’s top might be stale; we pop while heap[0].end ≤ x. Once we stop, the top is the current maximum. Since each building is pushed once and popped at most once, total heap work is O(N log N).
Complexity
O(N log N)time (sort + heap operations).O(N)space (events and heap).- Output size up to
O(N)key points.
Implementation Requirements
- Sort events with the right tie-breaking. The
(x, -h, R)tuple ordering naturally puts openings before closings at the samex(negative height < 0 < zero sentinel). - Process all events at the same
xtogether before emitting. - Deduplicate consecutive key points with equal
y. - Initialize the heap with the ground sentinel
(0, ∞)so it’s never empty.
Tests
def test_skyline():
bs = [[2,9,10],[3,7,15],[5,12,12],[15,20,10],[19,24,8]]
expected = [[2,10],[3,15],[7,12],[12,0],[15,10],[20,8],[24,0]]
assert get_skyline(bs) == expected
# Single building
assert get_skyline([[1, 2, 1]]) == [[1, 1], [2, 0]]
# Two same-x openings, different heights
assert get_skyline([[0, 5, 3], [0, 4, 5]]) == [[0, 5], [4, 3], [5, 0]]
# Stress vs brute on small inputs
import random
for _ in range(200):
N = random.randint(1, 6)
bs = []
for _ in range(N):
L = random.randint(0, 10)
R = random.randint(L + 1, L + 5)
H = random.randint(1, 5)
bs.append([L, R, H])
assert get_skyline(bs) == skyline_brute(bs)
Follow-up Questions
- “Total area covered (rectangle union)?” — same sweep, but accumulate
Δx · max_heightbetween consecutive events.O(N log N). - “Number of overlapping buildings at each
x?” — same events, but track the count of active buildings, not max.O(N log N). - “Online version where buildings stream in?” — segment tree over compressed coordinates, range max query.
O(N log N)total. - “K-th tallest building visible at
x?” — segment tree with “K-th order statistic” support, or a balanced BST.
Product Extension
Logging / event analytics: “concurrent active sessions over time” is the same engine with count instead of max. Cloud autoscaling decision: “what’s the peak demand in this 5-minute window?” Same engine with sum instead of max. Calendar conflict detection: pairs of overlapping events found by sweep + active-set membership. Real-time bidding (RTB): impression eligibility windows with priority-tier counts.
Language / Runtime Follow-ups
- Python:
heapqis min-heap, so use negative heights for max.SortedListfromsortedcontainersisO(log N)insert / delete / max — closest to C++ multiset. - Java:
TreeMap<Integer, Integer>mapping height to count, withlastKey()for max. OrPriorityQueuewith lazy deletion. - Go:
container/heapwith customLess; alternatively asort.IntSliceyou maintain manually. - C++:
multiset<int>with*rbegin()for max,erase(find(...))for single-instance removal. Orpriority_queue+ lazy deletion.
Common Bugs
- Tie-breaking: closing processed before opening at the same
xcauses phantom drops. - Forgetting the ground sentinel
(0, ∞)causes empty-heap crashes when all buildings expire. - Failing to deduplicate consecutive key points: emitting
[5, 10], [7, 10], [9, 10]instead of[5, 10]. - Removing both copies when two buildings have the same height by using
multiset.erase(value)(which erases all). Useerase(find(value)). - Confusing
<=vs<on the lazy-pop condition; off-by-one drops a building one event too early or late. - Sorting events by
xonly and breaking ties arbitrarily — leads to wrong output on dense inputs.
Debugging Strategy
If output has phantom key points or wrong heights:
- Print events after sorting; verify openings appear before closings at the same
x. - Print heap contents after processing each event; verify the top is the true max active height.
- Run on the smallest failing case and compare against brute on
x_max ≤ 30.
If output is missing key points:
- Check the deduplication condition; an off-by-one might filter out real changes.
- Verify all events at the same
xare processed before emitting.
Mastery Criteria
- Write the full skyline algorithm from blank in <15 minutes.
- Articulate the tie-breaking rule and why it’s needed in 30 seconds.
- Adapt the sweep to rectangle-union-area in <5 minutes.
- Recognize “intervals + boundary events + dynamic property” as the sweep-line trigger in <60 seconds.
-
State the
O(N log N)correctness argument in one sentence.
← Lab 03 — Sieve and Factorization · Phase 7 README · Lab 05 — Coordinate Compression →
Lab 05 — Coordinate Compression and Fenwick Tree: Count of Smaller Numbers After Self
Goal
Master coordinate compression as a preprocessing step, then use a Fenwick tree (BIT) over the compressed indices to count, for each element of an array, how many elements to its right are strictly smaller. Solve LC 315 in O(N log N). By the end, you can compress + scan + BIT-update from blank in <10 minutes.
Background
Many problems on integer arrays don’t actually depend on the values, only on the relative ordering. When values can be huge (10^9) but N is small (10^5), allocating an array indexed by value is impossible. Coordinate compression replaces each value v by its rank in sort(unique(values)), mapping the value space down to [0, N). This lets us index a BIT or segment tree by rank instead of value, swapping O(value_range) space for O(N).
The pairing of “compress, then BIT over ranks, then scan one direction” is one of the most reused patterns in CP: count inversions, count smaller-on-right, count pairs with sum in a range, K-th smallest in a sliding window — all instances of the same engine.
Interview Context
LC 315 (Hard) — Count of Smaller Numbers After Self. A classic FAANG senior-level question; also asked at quant firms for “count inversions” or “count pairs (i, j) with i < j and a_i > a_j”. Brute is O(N²) and obvious. The expected O(N log N) solution requires the candidate to either:
- Compress + BIT (this lab), or
- Modified merge sort (counts inversions during the merge step).
Both are valid; BIT is more general and extends to more variants. Senior candidates are expected to know both.
Problem Statement
Given an integer array nums of length N, return an array counts where counts[i] is the number of indices j > i such that nums[j] < nums[i].
LeetCode reference: LC 315 (Count of Smaller Numbers After Self).
Constraints
1 ≤ N ≤ 10^5.−10^4 ≤ nums[i] ≤ 10^4(LC); generalize to−10^9 ≤ nums[i] ≤ 10^9.- Time limit: 1 second.
Clarifying Questions
- “Strictly smaller, or
≤?” — strictly smaller (per LC). For≤, change query upper bound. - “Are duplicates allowed?” — yes. They must not count themselves as “smaller”.
- “Modify input allowed?” — generally yes; the compression step can sort a copy.
- “Return order?” — same order as input (
counts[i]aligned withnums[i]).
Examples
nums = [5, 2, 6, 1] → counts = [2, 1, 1, 0]. For5: indices 1 and 3 have values < 5. For2: only index 3. For6: only index 3. For1: nothing to the right.nums = [-1] → [0].nums = [-1, -1] → [0, 0]. Equal values don’t count.
Initial Brute Force
def count_smaller_brute(nums):
n = len(nums)
out = [0] * n
for i in range(n):
for j in range(i + 1, n):
if nums[j] < nums[i]:
out[i] += 1
return out
Brute Force Complexity
O(N²). For N = 10^5, that’s 10^10 ops — infeasible.
Optimization Path
- Brute (above).
- Modified merge sort. During merge, when an element from the right half is placed before an element from the left half, increment counters for all unplaced left-half elements.
O(N log N). - Coordinate compression + Fenwick tree. Compress values to ranks
[0, N). Scan right-to-left. For each element, query BIT prefix sum on[0, rank(v) − 1](= count of strictly smaller values already seen on the right), then update BIT atrank(v).O(N log N).
The BIT approach is more flexible: it handles “smaller”, “equal”, “in range”, “K-th smallest” with the same engine. The merge sort approach is more efficient in constant factor for pure inversion counting.
Final Expected Approach
Coordinate compression + Fenwick tree, scan right-to-left.
struct BIT {
vector<int> t;
BIT(int n) : t(n + 1, 0) {}
void update(int i, int v) { for (++i; i < (int)t.size(); i += i & -i) t[i] += v; }
int query(int i) { int s = 0; for (++i; i > 0; i -= i & -i) s += t[i]; return s; }
};
vector<int> count_smaller(vector<int>& nums) {
int n = nums.size();
vector<int> sorted_nums(nums.begin(), nums.end());
sort(sorted_nums.begin(), sorted_nums.end());
sorted_nums.erase(unique(sorted_nums.begin(), sorted_nums.end()), sorted_nums.end());
auto rank_of = [&](int v) {
return (int)(lower_bound(sorted_nums.begin(), sorted_nums.end(), v) - sorted_nums.begin());
};
BIT bit(sorted_nums.size());
vector<int> result(n);
for (int i = n - 1; i >= 0; --i) {
int r = rank_of(nums[i]);
result[i] = (r > 0) ? bit.query(r - 1) : 0;
bit.update(r, 1);
}
return result;
}
Data Structures Used
- A sorted-uniqued copy of the input for rank lookup (O(N log N) sort).
- A Fenwick tree of size
unique_countfor prefix-sum updates and queries. - A result array of size
N.
Correctness Argument
Compression preserves order. lower_bound returns the first index whose value is ≥ v; for any v in the original array, this is the unique rank. So rank(a) < rank(b) ↔ a < b, and rank(a) == rank(b) ↔ a == b.
Right-to-left scan invariant. When processing index i, the BIT contains exactly the multiset of ranks for indices [i+1, n−1] (each updated by +1). The query bit.query(rank(nums[i]) − 1) returns the count of those ranks strictly less than rank(nums[i]), which equals the count of nums[j] < nums[i] for j > i. After the query, we insert rank(nums[i]) so it’s visible to the next (lower) i.
Edge case rank == 0. If nums[i] is the minimum, no value is strictly smaller, so result[i] = 0. The code guards r > 0 to avoid querying bit.query(-1).
Complexity
- Sort + unique:
O(N log N). - For each of
Nelements: one rank lookup (O(log N)), one BIT query (O(log N)), one BIT update (O(log N)). - Total:
O(N log N)time,O(N)space.
Implementation Requirements
- Use 1-indexed BIT internally (
++ion entry); 0-indexed externally. - After sort, deduplicate before binary search; otherwise rank would skip values for ties.
- Scan right-to-left; left-to-right would count smaller-on-left, a different problem.
- Handle
n = 0(return empty array) andn = 1(return[0]).
Tests
def test_count_smaller():
assert count_smaller([5, 2, 6, 1]) == [2, 1, 1, 0]
assert count_smaller([-1]) == [0]
assert count_smaller([-1, -1]) == [0, 0]
assert count_smaller([]) == []
assert count_smaller([1, 2, 3, 4]) == [0, 0, 0, 0] # already sorted asc
assert count_smaller([4, 3, 2, 1]) == [3, 2, 1, 0] # sorted desc
assert count_smaller([1, 1, 1]) == [0, 0, 0] # all equal
# Stress vs brute
import random
for _ in range(200):
n = random.randint(0, 50)
nums = [random.randint(-100, 100) for _ in range(n)]
assert count_smaller(nums) == count_smaller_brute(nums)
Follow-up Questions
- “Count strictly greater on right?” — change query to
bit.query(maxRank) − bit.query(rank(v)). - “Count of equal values on right?” —
bit.query(rank(v)) − bit.query(rank(v) − 1). - “Count of values in
[lo, hi]on right?” —bit.query(rank(hi)) − bit.query(rank(lo) − 1). - “Total inversions in array?” — sum the result array.
- “Online streaming version?” — same algorithm with a hash-rank assigned on-the-fly via a balanced BST or order-statistics tree (no compression possible until all values seen).
- “Why not segment tree?” — works equally well; BIT has 4× smaller constant and is shorter to code. Use seg tree if you need range max / range update.
Product Extension
Recommendation systems: “for each user’s recently watched item, how many of their next 10 watches were lower-rated?” — same problem on rating arrays. Quant trading: rank-based features (“how many of the next 100 ticks are below this tick’s price?”) computed in batch via this engine. Search ranking: “for each query, count the number of subsequent queries with shorter session length” — feature engineering pipelines.
Language / Runtime Follow-ups
- Python:
bisect.bisect_leftfor rank lookup; BIT as a plain list.sortedcontainers.SortedListis a 1-line alternative (sl.bisect_left(v); sl.add(v)) but ~5× slower than BIT in pure Python. - Java:
Arrays.binarySearchfor rank lookup,int[]BIT. - Go:
sort.SearchInts,[]intBIT. - C++:
lower_bound+vector<int>BIT. Thepbdsorder-statistics tree (tree<>from__gnu_pbds) givesfind_by_orderandorder_of_keydirectly, but is slower than BIT.
Common Bugs
- Forgetting to deduplicate after sort:
lower_boundstill works, but BIT size becomesNeven if values are mostly equal — wasted space, not incorrect. - Using
upper_boundinstead oflower_boundfor rank: gives wrong answer for duplicates. - Scanning left-to-right instead of right-to-left: solves a different problem.
- 1-indexed vs 0-indexed off-by-one in the BIT.
- Querying
rank − 1without checkingrank > 0:bit.query(-1)may return garbage or crash. - Comparing with
≤instead of<(depends on problem statement).
Debugging Strategy
If output is wrong:
- Print compressed ranks alongside original values; check ordering preserved.
- Print BIT state after each insertion (small input). Verify prefix sums equal “count of values ≤ rank”.
- Run on
[5, 2, 6, 1]and trace right-to-left: ati=3, BIT empty, result=0. Insert rank(1). Ati=2, query rank(6)−1 → count of values with rank ≤ rank(6)−1 already in BIT → expect 1. - Compare against brute on
N ≤ 30random inputs.
Mastery Criteria
-
Write coordinate compression (
sort+unique+lower_bound) from blank in <2 minutes. - Write Fenwick tree (update + query) from memory in <3 minutes.
- Articulate the right-to-left scan invariant in one sentence.
- Adapt the engine to “count in range [lo, hi]” or “total inversions” in <5 minutes.
- Recognize “values up to 10^9, N up to 10^5, count-by-rank query” as the compression+BIT trigger in <60 seconds.
← Lab 04 — Sweep Line · Phase 7 README · Lab 06 — Stress Testing →
Lab 06 — Stress Testing: Finding Bugs You Can’t Spot by Reading
Goal
Build a stress-testing harness in <10 minutes that pits a brute-force oracle against a candidate solution on randomly generated small inputs, automatically catching mismatches. Use it to find two intentionally-injected bugs in a candidate solution where reading the code wouldn’t reveal them. By the end, stress testing is your default tool when a CP solution passes samples but fails hidden tests, and you can build a fresh harness for any problem in <10 minutes.
Background
In CP and high-stakes interviews, you frequently face: “my code passes the samples but WAs on hidden tests.” Reading harder doesn’t help — your mental model of the algorithm is exactly what’s wrong. Stress testing breaks this by replacing your brain with the machine: write a slow-but-obviously-correct brute, write a fast random input generator, and let the computer compare outputs on millions of small cases. The first mismatch is a tiny counterexample you can debug by hand.
Top competitive programmers (red CF) use stress testing constantly. It’s the single most underused tool by interview candidates and the fastest way to debug a “looks right but doesn’t work” solution.
Interview Context
Interview problems rarely give you the freedom to write a stress test under time pressure, but the meta-skill — converting “I’m stuck” into a structured experiment — is exactly what staff-level interviews probe. In quant interviews, “How would you validate this code?” is a standard question; “I’d write a brute oracle and stress-test against random inputs” is a strong answer. In CP, stress testing is required at the Div 2/Div 1 level.
This lab is the meta-lab for the entire phase: build the tooling that will save you in every other lab.
Problem Statement
Given a candidate solution and a brute-force oracle for some problem, build a harness that:
- Generates random inputs of small size (so brute is fast).
- Runs both solutions.
- Compares outputs and prints / dies on the first mismatch.
- Uses a seeded RNG so failures are reproducible.
We’ll use prefix sum range queries as the test problem. Sub-problems:
- Brute: for each query
(l, r), suma[l..r]directly.O(N)per query. - Candidate: precompute
prefix[i] = a[0] + ... + a[i-1], answer each query asprefix[r+1] - prefix[l].O(1)per query.
We will deliberately introduce two bugs in the candidate and use the harness to find each.
Constraints
- For stressing:
N ≤ 50, values in[-10, 10],Q ≤ 50queries. Small enough that brute isO(N · Q) = 2500ops per test, allowing ~10⁵ tests / second. - The candidate should pass when correct, fail clearly when buggy.
Clarifying Questions
- “What’s the comparison rule for outputs?” — exact equality (lists of ints).
- “How small should random inputs be?” — small enough that brute finishes in microseconds per test, big enough to expose edge cases. Rule of thumb: smallest size where the candidate’s structure is non-trivial.
- “Is determinism required?” — yes; seed the RNG so the same failing test re-runs identically.
- “Output format on mismatch?” — input that triggered, both outputs, the seed.
Examples
A passing harness run prints nothing (or a PASSED line). A failing run prints the first counterexample:
MISMATCH at iteration 47 (seed=12345):
input: N=5, a=[3, -2, 1, 4, -1], queries=[(0, 4), (1, 3), (2, 2)]
brute: [5, 3, 1]
candidate:[5, 3, 0]
Initial Brute Force
The brute oracle is the brute force here:
def prefix_sum_brute(a, queries):
return [sum(a[l:r+1]) for l, r in queries]
It’s O(N · Q), slow but unambiguous.
Brute Force Complexity
O(N · Q) per test case. For N = Q = 50, ~2500 ops per test. Running 100,000 stress iterations completes in seconds.
Optimization Path
The harness itself doesn’t optimize. The thing being tested (candidate solution) does. The harness’s job is to detect when the optimization is incorrect.
Final Expected Approach
The candidate (intentionally with bugs to discover):
def prefix_sum_candidate(a, queries):
n = len(a)
prefix = [0] * (n + 1)
for i in range(n):
prefix[i + 1] = prefix[i] + a[i]
# BUG 1: should be prefix[r+1] - prefix[l], not prefix[r] - prefix[l]
return [prefix[r] - prefix[l] for l, r in queries]
The harness:
import random
def stress(brute, candidate, gen, n_iters=10000, seed=0):
rng = random.Random(seed)
for it in range(n_iters):
inp = gen(rng)
b_out = brute(*inp)
c_out = candidate(*inp)
if b_out != c_out:
print(f"MISMATCH at iteration {it} (seed={seed}):")
print(f" input: {inp}")
print(f" brute: {b_out}")
print(f" candidate: {c_out}")
return False
print(f"PASSED {n_iters} iterations.")
return True
def gen_prefix_sum(rng):
n = rng.randint(1, 10)
a = [rng.randint(-5, 5) for _ in range(n)]
q = rng.randint(1, 5)
queries = []
for _ in range(q):
l = rng.randint(0, n - 1)
r = rng.randint(l, n - 1)
queries.append((l, r))
return (a, queries)
stress(prefix_sum_brute, prefix_sum_candidate, gen_prefix_sum, n_iters=1000, seed=42)
The harness will fire and report the first failing input within a few iterations. Fix prefix[r] to prefix[r+1]. Re-run.
Now introduce BUG 2 (subtle): use prefix[i] initialized as 0 for i = 0 but set prefix[i+1] = prefix[i] + a[i+1] (off-by-one in the recurrence). Stress finds it again.
After both fixes, the harness reports PASSED 10000 iterations. and you know the candidate is (probably) correct.
Data Structures Used
- A seeded RNG (
random.Random(seed)in Python;mt19937in C++). - A generator function returning a random valid input.
- A brute oracle.
- A candidate solution.
- The harness loop.
Correctness Argument
Why this works. If the brute oracle is correct (small enough that we can verify by hand), and the candidate disagrees, then the candidate is wrong on that input. We have a counterexample. Conversely, if the candidate matches the brute on n_iters = 10⁵ random small inputs, it’s probably correct — the chance that a non-trivial bug survives all of them is small for most input distributions, but not zero. Add adversarial inputs (all same, all max, all min, edge sizes 0, 1) explicitly to the generator to harden coverage.
Why determinism matters. When the harness fires, you want to re-run with the same seed to reproduce; without a seed, the bug might evaporate next run.
Why small inputs. The smaller the input, the faster brute runs (more iterations), and the easier the counterexample is to debug by hand. CF-grade stress tests use N ≤ 5 for the first pass.
Complexity
Per stress iteration: brute is O(N · Q); candidate is O(N + Q); comparison is O(Q). Harness overhead is negligible. For N = Q = 10 and n_iters = 10⁵, total is ~10⁷ ops — under 1 second in Python.
Implementation Requirements
- Seed the RNG explicitly. Print the seed on failure.
- Generator must produce valid inputs (respects all problem constraints — non-empty arrays, valid index ranges, etc.).
- On mismatch, print the minimal failing input. (Optional refinement: shrink the input by retrying with smaller sizes once you’ve found a failure.)
- Cover edge cases: empty array, single element, all-same values, max-size inputs.
Tests
The harness is itself code; it should be tested.
def test_harness_finds_planted_bug():
def buggy(a, queries):
return [sum(a[l:r]) for l, r in queries] # off-by-one: should be a[l:r+1]
# The harness should fire (return False) on a buggy candidate.
result = stress(prefix_sum_brute, buggy, gen_prefix_sum, n_iters=1000, seed=1)
assert result == False
def test_harness_passes_correct_candidate():
def correct(a, queries):
n = len(a)
prefix = [0] * (n + 1)
for i in range(n):
prefix[i + 1] = prefix[i] + a[i]
return [prefix[r + 1] - prefix[l] for l, r in queries]
result = stress(prefix_sum_brute, correct, gen_prefix_sum, n_iters=1000, seed=1)
assert result == True
Follow-up Questions
- “What if brute is also buggy?” — write brute as straightforwardly as possible (read the problem statement and implement it word-for-word). Skip optimizations. If both brute and candidate agree on a bug, you have no oracle and stress testing fails. Mitigation: cross-check brute against the problem’s sample I/O before stressing.
- “How to shrink a counterexample?” — once a failing input is found, repeatedly remove array elements / queries / values; if it still fails, keep the smaller input. Greedy; good enough.
- “Stress testing for graph problems?” — generator emits random small graphs (
N ≤ 6vertices, random edges). Brute is BFS / DFS over all paths. - “What if the answer isn’t unique?” — write a checker instead of a comparator: validate the candidate’s output as one of many valid answers (e.g., for “any valid topological order”).
- “Multi-threaded stress?” — easy with a process pool; each worker has its own seed offset.
Product Extension
Property-based testing in production: tools like Hypothesis (Python), QuickCheck (Haskell), proptest (Rust) generate random inputs and check invariants — same idea, different framing. Fuzz testing for security: AFL, libFuzzer feed random / mutated bytes to a binary and check for crashes. Differential testing across implementations: compare a new compiler against an old one on random programs (CSmith for C, csmith for SQL via SQLancer, etc.). The harness pattern transfers directly.
Language / Runtime Follow-ups
- Python:
random.Random(seed)—mt19937under the hood.pytest+hypothesisfor property-based testing in production code. - C++:
std::mt19937 rng(seed); std::uniform_int_distribution<int> dist(lo, hi);— never userand()(low-quality, broken seeding on Windows). - Java:
java.util.Random(seed)(LCG, low-quality but reproducible) orSplittableRandom(better statistical quality). - Go:
rand.New(rand.NewSource(seed)). The defaultmath/randglobal is not thread-safe. - JS/TS: seedable RNG requires a library (
seedrandom); built-inMath.random()is not seedable.
Common Bugs
- Forgetting to seed the RNG → non-reproducible failures.
- Brute oracle has the same bug as the candidate → false negative (stress passes a buggy solution).
- Generator produces invalid inputs (e.g., negative array sizes) → both brute and candidate crash → not a useful comparison.
- Generator’s distribution is too narrow → never hits edge cases (all-equal, sorted, reverse-sorted, single element).
- Output comparison uses
==on floats with rounding errors → spurious mismatches; use tolerance. - Harness exits silently on first iteration if generator throws → wrap in try/except and report.
- Letting
Ngrow too large → brute is too slow → fewer iterations → less coverage.
Debugging Strategy
When stress fires:
- Print the failing input. Run just that input through both brute and candidate.
- If brute disagrees with your hand-computed answer, brute is buggy. Fix brute first.
- If candidate disagrees with brute (and brute matches your hand-computed answer): trace candidate’s execution on the failing input by hand or in a debugger. The bug is local to a few lines.
- Once fixed, re-run stress with the same seed; if it passes, increment seed and run again.
When stress passes but the real submission still WA:
- Generator might not cover the failing case. Inspect the failing test’s input distribution (size, value range, special structure) and update the generator.
- Add explicit corner cases: empty input, single element, max-size input, all duplicates, all distinct, sorted asc, sorted desc.
- Push
Nhigher; some bugs surface only at scale.
Mastery Criteria
- Build a stress harness from blank for an array problem in <10 minutes.
- Find a planted off-by-one bug in <30 seconds of harness runtime.
- Articulate why the brute oracle must be obviously-correct in one sentence.
- Generate adversarial corner cases (empty, single, all-equal) deliberately, not only random.
- Use the same harness pattern across array, graph, and string problems.
- When a real submission WAs, default to “stress test” instead of “read the code again”.
← Lab 05 — Coordinate Compression · Phase 7 README
Lab 07 — Combinatorics mod p
Goal
Extend the fact[]/inv_fact[] engine from lab-01 (modular arithmetic) into the full counting toolkit: O(1) binomial coefficients, permutations, multinomials, Catalan numbers, stars and bars, and Lucas’ theorem for n up to 10¹⁸. Every CF round with the line “output the answer modulo 10⁹+7” assumes this is preloaded. Target: write the Comb class from blank in <4 minutes; map a counting problem to the right formula in <2 minutes.
Background Concepts
The engine. Precompute fact[i] = i! mod p forward in O(n). Compute inv_fact[n] = fact[n]^(p−2) mod p with one binary exponentiation (Fermat’s little theorem, prime p — see lab-02), then sweep backward: inv_fact[i−1] = inv_fact[i] · i mod p, because (i−1)!⁻¹ = i!⁻¹ · i. Total O(n + log p). After that:
nCr(n, r) = fact[n] · inv_fact[r] · inv_fact[n−r]— O(1).nPr(n, r) = fact[n] · inv_fact[n−r]— ordered selections.- Multinomial
n! / (k₁! k₂! … k_m!)=fact[n] · Π inv_fact[kᵢ]— arrangements with repeated items.
Stars and bars. Number of ways to write n as an ordered sum of k non-negative integers = C(n+k−1, k−1) (place k−1 bars among n stars). Positive integers: substitute xᵢ = yᵢ+1 → C(n−1, k−1).
Catalan numbers. Cat(n) = C(2n, n)/(n+1) = (2n)! / (n!(n+1)!) = C(2n, n) − C(2n, n+1). They count: valid parentheses sequences of n pairs (the count behind LC 22), distinct BSTs on n nodes (LC 96), Dyck paths (lattice paths that never dip below the diagonal), non-crossing chord pairings, and triangulations of an (n+2)-gon.
Lucas’ theorem. For prime p: C(n, r) ≡ Π C(nᵢ, rᵢ) mod p, where nᵢ, rᵢ are the base-p digits. Any digit with rᵢ > nᵢ zeroes the product. Lets you answer C(10¹⁸, r) mod p for small prime p with a table of size p.
Derangements. D(n) = n! · Σ (−1)ᵏ/k! = (n−1)(D(n−1) + D(n−2)); D(n) is the nearest integer to n!/e. Classic inclusion-exclusion output (phase-12 lab-10).
Composite modulus. Fermat fails. Factor m (lab-03 sieve), compute mod each prime power (Lucas generalizes via Andrew Granville / factorial-with-p-stripped tricks), recombine with CRT — machinery in phase-13 lab-07 (ext-gcd/CRT). Name it in the interview; implement it only in CP.
Interview Context
FAANG rarely asks nCr mod p cold, but LC 62 rewards the formula answer over DP, LC 96 is Catalan recognition, and LC 920 (Hard) is unsolvable in time without modular counting fluency. Quant/HFT interviews use these as direct filters — “how many monotone lattice paths avoid the diagonal?” expects Catalan by reflection, not DP. The strongest signal: reaching for inv_fact without narrating it, the way you’d use a hash map.
Problem Statement (LC 62, LC 96, LC 920)
Canonical framing: count paths / sequences / arrangements, output mod 10⁹+7. Three graded instances:
- LC 62 (Unique Paths): paths from (0,0) to (m−1,n−1) moving only right/down = arrangements of
m−1downs andn−1rights =C(m+n−2, m−1). - LC 96 (Unique BSTs): answer is
Cat(n). - LC 920 (Number of Music Playlists):
ndistinct songs, playlist lengthL, every song played, a song may repeat only afterkothers — DP over distinct-songs-used whose transitions are modular products; closed form usesC(n, j)with inclusion-exclusion.
Constraints
- Table size
N ≤ 10⁶–10⁷ (build once, O(N)). - Queries
Q ≤ 10⁶, O(1) each. p = 10⁹+7(prime) unless stated; Lucas variant:n, r ≤ 10¹⁸,p ≤ 10⁵prime.- LC 62:
1 ≤ m, n ≤ 100. LC 920:0 ≤ k < n ≤ L ≤ 100.
Clarifying Questions
- Is the modulus prime? (Yes, 10⁹+7 — Fermat inverses valid; if composite, ext-gcd inverses + CRT, phase-13 lab-07.)
- How large can
nget relative to my table? (Ifnexceeds table size butpis small, Lucas; otherwise size the table to maxn— and 2n for Catalan.) - Are
r < 0orr > npossible? (Yes at formula boundaries — return 0, don’t crash.) - Ordered or unordered selections? (Determines nPr vs nCr — say it out loud before coding.)
- Do “objects” repeat? (Repetition → multinomial or stars and bars, not plain nCr.)
Examples
C(5, 2) = 10 P(5, 2) = 20 multinomial(4; 2,1,1) = 12
stars&bars: x+y+z = 5, xᵢ ≥ 0 → C(7, 2) = 21
Cat(3) = 5 (LC 96: numTrees(3) = 5; LC 22: 5 valid strings of 3 pairs)
LC 62: m=3, n=7 → C(8, 2) = 28
Lucas: C(10¹⁸, 3) mod 7 = 0 (10¹⁸ ≡ 1 mod 7, so the n−1 factor ≡ 0); C(10¹⁸+5, 3) mod 7 = 6
Initial Brute Force
Pascal’s triangle DP: C[n][r] = C[n−1][r−1] + C[n−1][r] mod p, O(n²) table. Or Python big-int math.factorial(n) // (…) then mod. Both are fine oracles for n ≤ 2000.
Brute Force Complexity
Pascal: O(n²) time and space — 10¹² cells at n = 10⁶. Big-int factorial: O(n) multiplications of numbers with O(n log n) digits ≈ O(n² log n) bit ops per query. Both infeasible at CP scale; keep them as stress oracles.
Optimization Path
- Pascal O(n²) → correctness oracle.
fact[]table + Fermat inverse per query: O(n + Q log p). Passes, wasteful.fact[]+ backwardinv_fact[]sweep: O(n + log p) build, O(1) query. Optimal — onepowtotal.nbeyond any table, small prime p → Lucas, O(p + log_p n) per query.
Final Expected Approach
MOD = 10**9 + 7
class Comb:
"""O(1) nCr/nPr/Catalan after O(n) precomputation, prime modulus."""
def __init__(self, n, mod=MOD):
self.mod = mod
self.fact = fact = [1] * (n + 1)
for i in range(1, n + 1):
fact[i] = fact[i - 1] * i % mod
self.inv_fact = inv = [1] * (n + 1)
inv[n] = pow(fact[n], mod - 2, mod) # the single Fermat inverse
for i in range(n, 0, -1):
inv[i - 1] = inv[i] * i % mod # backward sweep
def ncr(self, n, r):
if r < 0 or r > n:
return 0
return self.fact[n] * self.inv_fact[r] % self.mod * self.inv_fact[n - r] % self.mod
def npr(self, n, r):
if r < 0 or r > n:
return 0
return self.fact[n] * self.inv_fact[n - r] % self.mod
def catalan(self, n):
# Cat(n) = (2n)! / (n! (n+1)!) — table must reach 2n
return self.fact[2 * n] * self.inv_fact[n] % self.mod * self.inv_fact[n + 1] % self.mod
def lucas(n, r, p):
"""C(n, r) mod prime p, p small enough to table; n, r up to 10^18."""
fact = [1] * p
for i in range(1, p):
fact[i] = fact[i - 1] * i % p
res = 1
while n or r:
ni, ri = n % p, r % p
if ri > ni:
return 0
res = res * fact[ni] % p
res = res * pow(fact[ri] * fact[ni - ri] % p, p - 2, p) % p
n //= p
r //= p
return res
def unique_paths_dp(m, n): # LC 62, O(mn) oracle
row = [1] * n
for _ in range(m - 1):
for j in range(1, n):
row[j] = (row[j] + row[j - 1]) % MOD
return row[-1]
def unique_paths_formula(m, n, C): # LC 62, O(1) after tables
return C.ncr(m + n - 2, m - 1)
if __name__ == "__main__":
C = Comb(4000)
for m in range(1, 41):
for n in range(1, 41):
assert unique_paths_dp(m, n) == unique_paths_formula(m, n, C)
assert unique_paths_formula(3, 7, C) == 28
assert [C.catalan(i) for i in range(6)] == [1, 1, 2, 5, 14, 42]
assert lucas(10**18, 3, 7) == 0 and lucas(10**18 + 5, 3, 7) == 6
Data Structures Used
- Two flat int arrays
fact[0..N],inv_fact[0..N]. - Lucas: one array of size
pplus the base-pdigit loop — no big tables.
Correctness Argument
Backward sweep: i! · (i!)⁻¹ ≡ 1 at i = n by Fermat (p prime, fact[n] ≢ 0 since n < p). Inductively, inv_fact[i−1] = inv_fact[i] · i = (i!)⁻¹ · i = ((i−1)!)⁻¹ because (i−1)! · i = i!. So every entry is a true inverse with a single pow.
nCr: n! / (r!(n−r)!) is an integer; modulo a prime, division by r!(n−r)! is multiplication by its inverse, valid because r, n−r < p keeps both factorials nonzero mod p.
Lucas (sketch): in F_p[x], (1+x)^p ≡ 1 + x^p (freshman’s dream — binomials C(p,k) vanish for 0<k<p). Writing n = Σ nᵢ pⁱ gives (1+x)^n ≡ Π (1+x^{pⁱ})^{nᵢ}; extracting the coefficient of x^r digit by digit yields Π C(nᵢ, rᵢ).
LC 62: every path is a length-(m+n−2) string over {D, R} with exactly m−1 D’s; strings ↔ paths is a bijection, hence C(m+n−2, m−1). The DP-vs-formula assertion over 1600 cases is the machine check.
Complexity
| Operation | Time | Space |
|---|---|---|
Build Comb(N) | O(N + log p) | O(N) |
| ncr / npr / catalan | O(1) | — |
| lucas(n, r, p) | O(p + log_p n · log p) | O(p) |
Implementation Requirements
- Exactly one
powcall in the constructor — apowper entry is the classic O(n log p) regression. - Guard
r < 0 or r > n→ 0 in every query path. - Catalan requires the table sized to 2n, and
inv_fact[n+1]— size for it up front. - Lucas must reject composite
p(document it; assert if cheap — Miller–Rabin in phase-13 lab-06). - Keep everything in one
Combclass you can retype from memory.
Tests
ncr/nprvsmath.comb/factorial oracle for alln ≤ 200, spot checks at n ≈ 4000.- Boundaries:
C(0,0)=1,C(n,0)=C(n,n)=1,C(5,7)=0,C(5,−1)=0. - Catalan vs 1, 1, 2, 5, 14, 42, 132, … and vs
comb(2n,n)//(n+1)for n ≤ 200. - Lucas vs
math.comb % pexhaustively forn ≤ 120,p ∈ {2,3,5,7,13}; large-n spot checkC(10¹⁸,3) mod 7 = 0via the polynomialn(n−1)(n−2)/6 mod 7. - Parity law:
C(n,r)odd ⇔r AND (n−r) = 0— checked againstlucas(·,·,2)for n < 64. - LC 62 DP vs formula for all
m, n ≤ 40. (All of the above executed for this lab.)
Follow-up Questions
- Why build
inv_factbackwards instead of n modular inverses? →nFermat calls cost O(n log p) — 30× slower at n = 10⁷. Onepowat the top plus the O(n) identity(i−1)!⁻¹ = i!⁻¹ · igets every inverse. (Bonus:inv[i] = inv_fact[i] · fact[i−1]then gives plain inverses of 1..n in O(n) too.) - When is
C(n, r)odd? → Lucas base 2: odd iff every bit ofris ≤ the corresponding bit ofn, i.e.r AND (n−r) = 0. Kummer’s theorem generalizes: the exponent ofpinC(n, r)equals the number of carries when addingrandn−rin basep. - Derive stars and bars. → arrange
nstars andk−1bars in a row; each arrangement ↔ one ordered solution ofx₁+…+x_k = n,xᵢ ≥ 0; choose bar positions:C(n+k−1, k−1). LC disguises: distribute candies, count non-decreasing digit strings, bounded knapsack counting after inclusion-exclusion on upper bounds. - How do you recognize a Catalan problem? → checklist: balanced open/close structure, a “never dip below zero” prefix condition, non-crossing pairings, or recursive splitting into left/right independent subproblems (
Cat(n) = Σ Cat(i)Cat(n−1−i)). Any one → tryC(2n,n)/(n+1)before writing DP. - Count surjections from n elements onto k labels. → inclusion-exclusion over excluded labels:
Σ (−1)ʲ C(k,j)(k−j)ⁿ. Same alternating pattern gives derangements. Phase-12 lab-10 drills the technique; this lab supplies the O(1)C(k,j). - Modulus 10⁹+6 — now what? → composite (2 × 500000003). Factor, solve mod each prime power, recombine with CRT (phase-13 lab-07); non-invertible factorials require carrying the exponent of each prime separately.
Product Extension
A poker/board-game backend computing exact win equities evaluates thousands of C(n, r) per hand — precompute tables once at service start, never at request time. Same shape in ranking systems: “how many ways can k of n experiments beat control” is a binomial tail; a fintech risk engine summing hypergeometric probabilities is ncr three times per term. The O(n)-build/O(1)-query split is exactly a cache-warming pattern.
Language/Runtime Follow-ups
- Python:
pow(a, -1, mod)(3.8+) is the idiomatic single inverse;math.combis exact big-int — fine as oracle, too slow as engine at 10⁶ queries. - Java: keep everything
long;a · bwith both < 10⁹+7 fits in 63 bits — but three-factor products need an intermediate% mod. - C++:
long longwith% MODafter every multiply, or aMontgomery/mintwrapper;__int128only if you must multiply two 62-bit values. - Go:
uint64works with the same discipline; no operator overloading → writemulmodhelpers. - JS/TS:
Numberbreaks past 2⁵³ — a single product of two ~10⁹ values already overflows. UseBigIntthroughout or split multiplies.
Common Bugs
- Forward inverse loop: computing
inv_fact[i]frominv_fact[i−1]— the identity runs backward; forward needs a division you don’t have. - Table one short:
catalan(n)touchesfact[2n]andinv_fact[n+1]; a table sized tonthrows IndexError (best case) or reads garbage (C++). - Missing range guard:
ncr(5, 7)must return 0; without the guard,inv_fact[−2]silently indexes from the end in Python — wrong answer, no crash. - Negative after subtraction:
C(2n,n) − C(2n,n+1)can be negative pre-mod in C++/Java — addmodbefore%. (Python%is already non-negative.) - Per-query
pow: O(Q log p) instead of O(Q); passes small tests, TLEs at Q = 10⁶ — the regression is invisible until scale. - Lucas with composite p: both Fermat inverses and the theorem itself require prime
p; composite moduli silently produce wrong digits products.
Debugging Strategy
First check the two identities fact[i] · inv_fact[i] ≡ 1 and ncr(n, r) == math.comb(n, r) % p for all n ≤ 50 — this isolates table bugs from formula bugs. For Lucas, print the digit pairs (nᵢ, rᵢ) per iteration: C(10, 4) mod 3 → digits (1,1),(0,1),(1,0) → the (0,1) pair zeroes it; math.comb(10,4)=210=3·70 confirms. For a wrong LC 62/920 answer, shrink to m, n ≤ 5 and diff formula vs Pascal DP row by row — the first divergent cell names the wrong term.
Mastery Criteria
-
Combclass (ncr, npr, catalan) from blank, correct first run, in <4 minutes. -
Explained the backward
inv_factsweep and why onepowsuffices, in <1 minute. - LC 62 via formula and LC 96 via Catalan, each in <5 minutes including the mapping argument.
- LC 920 solved in <35 minutes with modular DP.
-
lucaswritten and hand-verified onC(10, 4) mod 3in <8 minutes. - Stated the composite-modulus escape hatch (CRT, phase-13 lab-07) and the Kummer carry rule without notes.
Lab 08 — XOR Linear Basis
Goal
Build a linear basis over GF(2) for a set of integers and answer the three canonical queries in O(60) each: maximum subset XOR, “can value v be formed as a subset XOR”, and k-th smallest span value. This is Gaussian elimination wearing a bitmask costume — the standard weapon for every CF “XOR of some subset” problem, and the structured alternative to the bitwise trie from phase-03 lab-07.
Background Concepts
Integers are vectors. A 60-bit integer is a vector in GF(2)⁶⁰; XOR is vector addition (each bit position an independent coordinate mod 2). “Some subset XORs to v” = “v is in the span”. All of linear algebra applies: span, independence, rank, basis.
Basis by bit position. Keep basis[b] = a vector whose highest set bit is b (or 0 if none). Insertion: walk x’s bits from high to low; at each set bit b, if basis[b] exists, reduce x ^= basis[b]; else store x there and stop. If x reduces to 0, it was already in the span — dependent. This is online Gaussian elimination, one column at a time.
Span structure. With rank r independent vectors out of n inserted, the span has exactly 2ʳ distinct values, and each is produced by exactly 2^(n−r) of the 2ⁿ subsets (each dependent element doubles every count uniformly). The empty subset contributes 0 — 0 is always in the span.
Row-reduced (canonical) form. After insertion the basis is echelon but not canonical: a row may still contain lower rows’ pivot bits. Eliminating them (each pivot bit appears in exactly one row) makes the rows, sorted ascending, generate span values monotonically: XOR-combining rows selected by the bits of k yields the k-th smallest span value. That’s the k-th query in O(60).
Basis vs trie. The bitwise trie (phase-03 lab-07) answers “max x XOR (one array element)”. The basis answers “max XOR of any subset”. Different questions — see follow-ups.
Interview Context
FAANG asks the trie problems (LC 421, LC 1707); the basis itself is CF Div 1/2 staple, ICPC regular, and a favorite of quant firms because it’s “do you actually know linear algebra” in disguise. Interview signal: the moment you say “these are vectors over GF(2), so I’ll row-reduce”, you’ve separated yourself from everyone pattern-matching on tries. Derive-from-scratch is expected; it’s 25 lines.
Problem Statement (LC 421)
Primary: Maximum XOR subset (classic CF/GFG): given a[1..n], find the maximum XOR over all 2ⁿ subsets. Related LC anchors:
- LC 421 (Maximum XOR of Two Numbers in an Array): max over pairs — trie territory, but the basis’s
max_xor(start)machinery is the same greedy. - LC 1707 (Maximum XOR With an Element From Array): max
x XOR a[i]witha[i] ≤ m— offline sorted trie insertion; the trie-flavored cousin.
Constraints
1 ≤ n ≤ 10⁵;0 ≤ a[i] < 2⁶⁰(fixbits = 60; rank ≤ 60 regardless of n).- Queries (contains / max / k-th): up to 10⁵, each O(bits).
- k-th query:
1 ≤ k ≤ 2^rank.
Clarifying Questions
- Is the empty subset allowed? (Yes — 0 is in the span; matters for “minimum XOR” and k = 1.)
- Max element bit width? (Sets the loop bound; 60 covers 10¹⁸, 30 covers 10⁹.)
- Duplicates in input? (Fine — second copy reduces to 0, rejected as dependent, counted in n−rank.)
- Do I need which subset achieves the max, or just the value? (Value; recovering a witness needs tracking masks alongside — doubles state.)
- Are elements inserted online or all up front? (Basis is naturally online; deletion is NOT supported — rebuild or use offline tricks.)
Examples
a = [3, 10, 5, 25, 2, 8] → rank 5 (8 is dependent), span size 2⁵ = 32
max subset XOR = 31 (witness: 3^5^25), each span value formed 2^(6−5) = 2 ways
a = [1, 2, 3] → rank 2 (3 = 1^2 dependent), span {0, 1, 2, 3}, each formed 2×
contains(3) = True, contains(4) = False
kth on [1, 2, 3]: kth(1) = 0, kth(2) = 1, kth(3) = 2, kth(4) = 3
LC 421 on [3, 10, 5, 25, 2, 8] → best pair 5^25 = 28 (pairs ≠ subsets: 28 < 31)
Initial Brute Force
Enumerate all 2ⁿ subsets, XOR each, take max / collect set for membership and k-th. Ten lines; the stress oracle for n ≤ 20 (pair with phase-07 lab-06 stress testing).
Brute Force Complexity
O(2ⁿ · n) time, O(2ⁿ) space for the value set. n = 40 is already 10¹²; n = 10⁵ is absurd. Meet-in-the-middle (phase-03 lab-09) buys n ≈ 40 for max-subset-XOR — still exponential.
Optimization Path
- Brute force 2ⁿ — oracle only.
- Observe XOR = GF(2) addition → the answer set is a linear span → at most 2⁶⁰ ≠ 2ⁿ structure, rank ≤ 60.
- Online Gaussian elimination: insert each element in O(60) → basis. Max via greedy high-bit descent; membership via reduction; total O(n · 60).
- k-th queries: one O(60²) row reduction, then O(60) per query.
Final Expected Approach
class XorBasis:
"""Linear basis over GF(2) for integers < 2^bits. All ops O(bits)."""
def __init__(self, bits=60):
self.bits = bits
self.basis = [0] * bits # basis[b]: vector with highest bit b, or 0
def insert(self, x):
"""True if x was independent (basis grew), False if x already in span."""
for b in range(self.bits - 1, -1, -1):
if not (x >> b) & 1:
continue
if not self.basis[b]:
self.basis[b] = x
return True
x ^= self.basis[b]
return False
def contains(self, x):
for b in range(self.bits - 1, -1, -1):
if (x >> b) & 1:
if not self.basis[b]:
return False
x ^= self.basis[b]
return True
def max_xor(self, start=0):
res = start
for b in range(self.bits - 1, -1, -1):
if self.basis[b] and (res ^ self.basis[b]) > res:
res ^= self.basis[b]
return res
@property
def rank(self):
return sum(1 for v in self.basis if v)
def reduced(self):
"""Row-reduced echelon rows, ascending: each pivot bit in exactly one row."""
rows = [] # (pivot_bit, vector), pivots ascending
for b in range(self.bits):
if self.basis[b]:
v = self.basis[b]
for pb, pv in rows:
if (v >> pb) & 1:
v ^= pv
rows.append((b, v))
return [v for _, v in rows]
def kth(self, k):
"""k-th smallest span value, 1-indexed (kth(1) == 0). −1 if out of range."""
rows = self.reduced()
if not 1 <= k <= 1 << len(rows):
return -1
k -= 1
res = 0
for i, v in enumerate(rows):
if (k >> i) & 1:
res ^= v
return res
def max_subset_xor(nums):
b = XorBasis(max(nums).bit_length() if nums else 1)
for x in nums:
b.insert(x)
return b.max_xor()
Data Structures Used
- Fixed array
basis[0..bits)— the whole structure is 60 machine words. - Row-reduced copy (list of ≤ 60 ints) built on demand for k-th queries.
Correctness Argument
Insertion invariant: basis[b] is 0 or has highest bit exactly b; the multiset span never changes, because reducing x by basis vectors replaces it with x ⊕ (span element) — elementary row operations preserve span. x reaches 0 iff it was a GF(2) combination of stored vectors, so the return value is exactly independence.
Greedy max: descend bits high→low; taking res ^= basis[b] when it increases res is optimal because a set bit at position b dominates all lower bits combined (2ᵇ > 2ᵇ−1), and lower basis vectors cannot disturb bit b (their highest bit is < b). Exchange argument, same skeleton as the trie greedy in LC 421.
k-th monotonicity: after full reduction, pivot bit pᵢ appears only in row i. For masks k < k' differing highest at bit j: rows above j are chosen identically, rows below cannot touch bit p_j, so value(k’) has bit p_j set where value(k) doesn’t, and all higher bits agree → value(k’) > value(k). Strictly increasing in k, hence kth(k) is the k-th smallest.
Counting: rank r vectors are independent → 2ʳ distinct combinations; each of the n−r dependent inserts doubles the number of subsets hitting every value equally → 2^(n−r) each. (Verified against full subset enumeration in tests.)
Complexity
| Operation | Time | Space |
|---|---|---|
| insert / contains / max_xor | O(bits) | O(bits) total |
| reduced() | O(bits²) | O(bits) |
| kth (after reduce) | O(bits) | — |
| Build over n elements | O(n · bits) | O(bits) |
Implementation Requirements
insertreturns bool — required for rank/counting answers; don’t discard it.- Bit loop bound is a constructor parameter; default 60, never hardcode 32.
max_xor(start)must accept a start value (LC 1707-style “x XOR best” queries).kthmust treat k = 1 as 0 (empty subset) and reject out-of-range k.- No deletion API — document that; rebuilds are O(n · 60).
Tests
- 300 random arrays (n ≤ 10, values < 2⁸) vs full 2ⁿ subset enumeration: max, membership for every probe, k-th for every k in range, rank = independent-insert count, span size 2^rank, every span value formed exactly 2^(n−rank) times.
max_xor(start)vsmax(start ^ v for v in span)with random starts.- Reduced rows sorted ascending and regenerate the same span.
- Degenerates: all zeros (rank 0, span {0}, kth(1)=0), single element, duplicates.
- 60-bit values: insert 50 random < 2⁵⁹, membership of elements and of ad-hoc XOR combos. (All executed for this lab.)
Follow-up Questions
- Basis or trie — how do you choose? → Subset-XOR questions (max subset XOR, “is v reachable”, k-th value) → basis: the answer set is a span, size 2ʳ. Pairwise-XOR questions (LC 421 max pair, LC 1707 bounded max) → trie: you need “best partner among actual elements”, and the span contains values no single element attains.
- Count subsets XORing to exactly v. → 0 if
not contains(v), else 2^(n−rank) — every reachable value is hit by equally many subsets. (Empty subset counts for v = 0; subtract 1 if the problem demands non-empty.) - Merge two bases. → insert every nonzero vector of one into the other: O(60²). Powers the segment-tree-of-bases / divide-and-conquer patterns; note there’s no efficient “un-merge”.
- Range queries: max subset XOR over a[l..r], offline. → prefix bases with time tags: keep for each prefix a basis remembering the latest insertion index per pivot, prefer newer vectors when inserting; query (l, r) uses pivots with tag ≥ l. O((n + q) · 60). CF-standard technique.
- Where’s the “real” linear algebra? → this IS Gaussian elimination over GF(2): insert = eliminate against pivot rows; rank–nullity says #subsets mapping to each value = 2^(dim kernel) = 2^(n−r);
reduced()is Gauss–Jordan back-substitution. Saying it this way in an interview is worth more than the code. - Minimum non-zero subset XOR? → smallest reduced row (= kth(2)). And max is the XOR of all reduced rows — every pivot set.
Product Extension
Erasure coding and RAID parity are literally spans over GF(2): a stripe’s parity blocks are XOR combinations, and “can we reconstruct block v from surviving blocks” is contains(v) on the survivors’ basis. Same math in network coding (butterfly routing) and in deduplicating feature-flag bitmask configs: rank tells you how many independent toggles actually exist across 10⁵ stored configs.
Language/Runtime Follow-ups
- Python: arbitrary-precision ints make
bits > 64free (hash prefixes, 128-bit ids); the O(60) loops are ~µs — no need for numpy. - C++:
uint64_t basis[60]; find the high bit with63 - __builtin_clzll(x)to jump straight to the pivot instead of scanning — insertion becomes O(rank). - Java:
long[]+Long.numberOfLeadingZeros; beware>>>vs>>on the sign bit if you ever use bit 63. - Go:
bits.Len64(x) − 1for the pivot; fixed[60]uint64array keeps it allocation-free. - JS/TS:
NumberXOR truncates to 32 bits (^operates on int32!) — anything beyond 2³¹ needsBigIntthroughout.
Common Bugs
- Low-to-high insertion: reducing from bit 0 upward breaks the invariant “basis[b] has highest bit b” — vectors land in wrong slots and
max_xormisses bits. - Wrong return on dependent insert: returning True when x reduced to 0 corrupts rank, and with it the 2^(n−rank) counting answers.
- k-th on the raw basis: without full row reduction, mask order is not value order — answers are span members but not the k-th. The monotonicity proof needs exclusive pivots.
- Forgetting 0 ∈ span: k-th smallest with k = 1 must be 0 (empty subset); minimum non-zero is kth(2). Off-by-one between the two conventions is endemic.
- Hardcoded 32 bits: values up to 10¹⁸ silently lose their top 28 bits; the basis “works” on samples and fails on real data.
basis[b] > resinstead of(res ^ basis[b]) > res: the greedy must test the XOR result — a basis vector can have bit b set where res also does, making XOR decrease res.
Debugging Strategy
Shrink to 4-bit values and print the basis as binary rows after every insert — the echelon shape (one row per pivot, highest bit on the diagonal) is visually checkable. For [1, 2, 3]: insert 1 → row at bit 0; insert 2 → row at bit 1; insert 3 → 3^2=1, 1^1=0 → rejected, rank 2. If kth misbehaves, print reduced() and confirm no row contains another row’s pivot bit; then table kth(1..2ʳ) against the sorted brute-force span — the first mismatch’s mask tells you which row still carries a stray pivot. Keep the 2ⁿ enumerator from Tests as the permanent oracle (lab-06 stress harness).
Mastery Criteria
-
XorBasiswith insert/contains/max_xor from blank, first-run correct, in <10 minutes. -
Added
reduced()+kthand passed a brute-force diff in <10 more minutes. - Stated the 2^rank / 2^(n−rank) counting facts with proof sketch, unprompted, in <2 minutes.
- Articulated basis-vs-trie decision rule on LC 421 vs max-subset-XOR in <1 minute.
- Solved max-subset-XOR cold (classic CF/GFG) in <15 minutes including the stress test.
- Explained the whole structure as Gaussian elimination + rank–nullity to a linear-algebra-literate interviewer.
Lab 09 — Randomized Algorithms
Goal
Own the randomized toolkit interviews actually test: quickselect with random pivot (expected O(n)), Fisher–Yates shuffle, reservoir sampling for streams, and rejection sampling — plus the probability arguments interviewers push on when they ask “why is that uniform?” and “what’s the expected cost?”. Target: any of the four from blank in <10 minutes, each with its proof sketch on demand.
Background Concepts
Las Vegas vs Monte Carlo. Las Vegas: always correct, runtime is random (quickselect, quicksort). Monte Carlo: runtime fixed, correctness is probabilistic (Miller–Rabin, phase-13 lab-06; hashing checks). Know which you’re offering before the interviewer asks.
Quickselect expectation. A uniformly random pivot lands in the middle half with probability 1/2; conditioned on that, the surviving side is ≤ 3n/4. So the expected work satisfies T(n) ≤ n + T(3n/4)-in-expectation, giving T(n) ≤ n(1 + 3/4 + (3/4)² + …) = 4n = O(n). Worst case remains O(n²), with probability vanishing in n.
Fisher–Yates uniformity. For i from n−1 down to 1, swap a[i] with a[j], j uniform on [0, i] (inclusive — self-swap allowed). Induction: position n−1 receives each element with probability 1/n; conditioned on that, the remaining prefix is a uniform shuffle of n−1 elements. Each permutation: 1/n × 1/(n−1) × … = 1/n!.
Reservoir sampling. Keep the first k items; for item i (0-indexed, i ≥ k) draw j uniform on [0, i] and replace res[j] if j < k — keep probability k/(i+1). Induction: after item i, every item seen so far is in the reservoir with probability k/(i+1). One pass, O(k) memory, stream length unknown in advance.
Randomization as defense. A fixed-base rolling hash (phase-03 lab-06) is broken by published anti-hash tests; drawing the base randomly per run makes the adversary’s collision probability ≈ n²/(2p) — the birthday bound: expect a collision only after ~√p strings. Same principle: languages randomize hash seeds so attackers can’t precompute worst-case buckets.
Interview Context
LC 215 with “do better than O(n log n)” is the single most common randomized prompt; LC 384 is the uniformity-proof filter; LC 382/398 test whether “stream” triggers reservoir; LC 470 tests rejection sampling arithmetic. The differentiator is never the code — it’s whether you can defend uniformity and expected cost under pushback. Rehearse the three induction proofs above; interviewers at Google/Meta/quant shops ask for exactly them.
Problem Statement (LC 215, LC 384, LC 382, LC 398, LC 470)
- LC 215: k-th largest element in an array — expected O(n) quickselect.
- LC 384:
Shuffle an Array— return uniformly random permutations; reset support. - LC 382: random node of a singly linked list, unknown length, O(1) memory — reservoir k=1.
- LC 398: random index among positions where
targetoccurs — reservoir over matches. - LC 470: implement
rand10()givenrand7()— rejection sampling.
Constraints
- LC 215: n ≤ 10⁵, values fit in int; duplicates present.
- LC 384: n ≤ 200 but
shufflecalled up to 5×10⁴ times — per-call O(n). - LC 382/398: single pass, O(1)/O(k) auxiliary memory; stream length unknown.
- LC 470: rand7 is the only entropy source; minimize expected calls.
Clarifying Questions
- K-th largest with duplicates — by value rank or by sorted position? (Sorted position:
sorted(a)[n−k].) - Can I mutate the input array? (Quickselect partitions in place; copy first if not.)
- Does “uniform” mean every permutation equally likely, or every element equally likely per slot? (Every permutation — the stronger condition; slot-uniformity alone is weaker.)
- Is the RNG assumed uniform and independent per call? (Yes; that’s the model all proofs rely on.)
- Adversarial inputs — could someone feed a killer case? (Random pivot/seed makes the adversary blind; without randomization, sorted input kills naive quickselect.)
- Stream too big for memory but k items fit? (Reservoir; if even k doesn’t fit, sample rate must be revisited.)
Examples
quickselect([3,2,1,5,6,4], k=2 largest) → 5
quickselect([3,2,3,1,2,4,5,5,6], k=4 largest) → 4
fisher_yates([0,1,2]) → each of the 6 permutations with probability 1/6
reservoir_sample(stream of 10 items, k=3) → each item kept with probability 3/10
rand10(): (rand7()−1)·7 + rand7() uniform on 1..49; accept ≤ 40, map (v−1) % 10 + 1
Initial Brute Force
- K-th largest: sort, index — O(n log n). Correct, and the oracle for everything below.
- Shuffle: generate a random permutation by sorting on random keys — O(n log n) and subtly wrong with duplicate keys.
- Random stream node: buffer the whole stream, pick uniformly — O(n) memory.
Brute Force Complexity
Sorting: O(n log n) time — fine until the interviewer says “linear expected time”. Buffering a stream: O(n) memory — disqualified by the O(1)-memory constraint, not by speed. The brute forces here fail on constraints, not correctness; keep them as oracles.
Optimization Path
- Sort-based selection O(n log n) → heap of size k, O(n log k) → quickselect, expected O(n).
- Full-buffer stream sampling O(n) memory → reservoir, O(k) memory, one pass.
- Sort-by-random-key shuffle → Fisher–Yates: O(n), provably uniform, in place.
- rand10 via
rand7() % ...hacks (biased) → rejection sampling: exactly uniform, expected 2.45 calls.
Final Expected Approach
import random
def quickselect(nums, k):
"""k-th smallest (0-indexed), expected O(n). Iterative Lomuto, random pivot."""
a = list(nums)
lo, hi = 0, len(a) - 1
while True:
if lo == hi:
return a[lo]
p = random.randint(lo, hi) # random pivot defeats adversaries
a[p], a[hi] = a[hi], a[p]
pivot, i = a[hi], lo
for j in range(lo, hi):
if a[j] < pivot:
a[i], a[j] = a[j], a[i]
i += 1
a[i], a[hi] = a[hi], a[i] # pivot lands in final slot i
if k == i:
return a[i]
if k < i:
hi = i - 1
else:
lo = i + 1
def kth_largest(nums, k): # LC 215
return quickselect(nums, len(nums) - k)
def fisher_yates(arr): # LC 384 core
a = list(arr)
for i in range(len(a) - 1, 0, -1):
j = random.randint(0, i) # [0, i] inclusive — self-swap allowed
a[i], a[j] = a[j], a[i]
return a
def reservoir_sample(iterable, k): # LC 382 / LC 398 generalized
res = []
for i, x in enumerate(iterable):
if i < k:
res.append(x)
else:
j = random.randint(0, i) # keep x with probability k/(i+1)
if j < k:
res[j] = x
return res
def rand7():
return random.randint(1, 7)
def rand10(): # LC 470: rejection sampling
while True:
v = (rand7() - 1) * 7 + rand7() # uniform over 1..49
if v <= 40:
return 1 + (v - 1) % 10 # 40 outcomes → exactly 4 per value
Data Structures Used
- Quickselect: the array itself plus two window pointers — no recursion stack (iterative).
- Reservoir: array of size k.
- rand10: none — just the two-digit base-7 construction.
Correctness Argument
Quickselect invariant: the k-th smallest of the whole array always lies in a[lo..hi], and partitioning is a permutation of that window. After Lomuto, a[i] = pivot with everything strictly smaller to its left, so exactly i elements are < pivot: if k = i we’re done (with duplicates, sorted(a)[i] must equal the pivot since positions 0..i−1 hold the < elements and the right side is ≥ pivot); otherwise recurse into the side containing k — the invariant is maintained.
Fisher–Yates: by downward induction as in Background — P(specific permutation) = Π 1/(i+1) = 1/n!.
Reservoir: induction on i. Base: first k items kept with probability 1. Step: item i survives insertion with probability k/(i+1) directly; a previous item survives with probability [k/i] · (1 − (k/(i+1)) · (1/k)) = k/(i+1). Uniform at every prefix — correct even though the stream length is never known.
rand10: the pair (r₁, r₂) is uniform over 49 outcomes, so v is uniform on 1..49. Conditioning on acceptance (v ≤ 40) keeps uniformity on 1..40, and 40 = 4 × 10 splits into equal preimage classes of size 4 per output. Rejection never biases; truncating with % 10 on 1..49 would (values 1..9 get 5 preimages, 10 gets 4).
Complexity
| Routine | Time | Space |
|---|---|---|
| quickselect | expected O(n), worst O(n²), w.h.p. O(n) | O(n) copy (O(1) if in-place allowed) |
| fisher_yates | O(n) | O(1) extra |
| reservoir_sample | O(n) stream pass, O(1) per item | O(k) |
| rand10 | expected 2 × 49/40 = 2.45 rand7 calls | O(1) |
Implementation Requirements
- Quickselect must be iterative (no stack-depth surprises) and must pick the pivot uniformly from
[lo, hi]— nota[hi]blindly. fisher_yatesdraws j on the inclusive range [0, i]; document it in a comment — this is the bug magnet.reservoir_samplemust accept any iterable (generators included) and never calllen().rand10may only obtain randomness throughrand7().- All routines must behave correctly under duplicates (LC 215 tests this deliberately).
Tests
- Quickselect vs
sorted(nums)[k]on 500 random arrays (n ≤ 60, heavy duplicates from range −20..20), plus both LC 215 examples. - Shuffle: 60 000 shuffles of [0,1,2]; all 6 permutations observed, each count within ±600 of 10 000 (≈7.5σ — flags real bias, ignores noise).
- Reservoir: k=3 over a 10-item stream × 30 000 trials — every element’s inclusion count within ±600 of 9 000; stream shorter than k returns the whole stream.
- rand10: 100 000 draws — support exactly {1..10}, each count within ±700 of 10 000. (All executed for this lab; chi-square-style bucket tolerances chosen ≥ 4σ.)
Follow-up Questions
- Why is “sort by random key” or “swap each i with random j ∈ [0, n−1]” biased? → the n-random-swaps version has nⁿ equally likely execution paths but n! target permutations; n! ∤ nⁿ (e.g. 27 paths onto 6 permutations for n=3), so some permutation must be hit more often. Sort-by-key additionally collides keys. Fisher–Yates has exactly n! paths, one per permutation.
- Can selection be worst-case O(n)? → yes — median-of-medians (groups of 5) guarantees a 30/70 split, T(n) = T(n/5) + T(7n/10) + O(n) = O(n). But constants ≈ 10–20× random pivot; it’s a talking point and the theory answer, not the implementation. C++
nth_elementuses introselect: random pivots with a median-of-medians fallback. - Expected vs amortized vs high probability? → expected: average over the algorithm’s own coin flips, any input (quickselect O(n)). Amortized: worst-case total over an operation sequence, no randomness (vector doubling). W.h.p.: probability ≥ 1 − 1/nᶜ (quickselect exceeds c·n only with polynomially small probability). Interviewers probe whether you conflate them.
- Exact expected cost of rand10? → each round makes 2 rand7 calls and accepts with probability 40/49. Rounds are geometric: E[rounds] = 49/40, so E[calls] = 2 × 49/40 = 2.45. Follow-up: recycle the rejected entropy (v − 40 ∈ 1..9 is a free rand9) to push calls below 2.45.
- Why do languages randomize hash seeds? → hash-flooding DoS: with a public hash, an attacker POSTs 10⁵ keys colliding into one bucket, turning dict inserts O(n²). A per-process random seed (universal hashing: pick h from a family s.t. P(h(x)=h(y)) ≤ 1/m) makes collisions unpredictable. Identical to the anti-hash-test defense for rolling hashes (phase-03 lab-06).
- When is quickselect’s O(n²) tail acceptable? → when inputs aren’t adversarial and the RNG is unseeded by users, the tail probability is ~2⁻ᶜⁿ-ish in practice; for hard latency SLOs, use introselect or a k-size heap (O(n log k), worst-case, streaming-friendly).
Product Extension
Telemetry pipelines reservoir-sample k events per minute per service so downstream storage is O(k) regardless of traffic spikes — the exact LC 382 pattern with k > 1 and sharding. A/B assignment is Fisher–Yates thinking: hash(user_id, salt) mod buckets, salt rotated per experiment (the “random key” done right, deterministically replayable). Load balancers use “power of two random choices” — two random probes, pick the emptier — expected max load drops from Θ(log n/log log n) to Θ(log log n).
Language/Runtime Follow-ups
- Python:
randomis Mersenne Twister — fine for algorithms, never for tokens (usesecrets).random.shuffleIS Fisher–Yates;random.sampledoes reservoir-adjacent selection without materializing when given a range. - C++:
rand() % nis biased and low-entropy; usestd::mt19937_64+uniform_int_distribution.std::shufflerequires a URBG;std::random_shufflewas removed in C++17 for exactly the bias reasons above.nth_element= introselect. - Java:
Collections.shuffleis Fisher–Yates;ThreadLocalRandomover sharedRandomunder concurrency (CAS contention). - Go:
math/randvscrypto/randsplit mirrors Python;rand.Shuffletakes a swap closure — Fisher–Yates underneath. - JS/TS:
Math.randomis unseedable and engine-dependent;sort(() => Math.random() − 0.5)is the canonical broken shuffle (comparison sorts assume consistent comparators — undefined behavior plus bias).
Common Bugs
- Exclusive upper bound in Fisher–Yates:
j ∈ [0, i−1](no self-swap) is Sattolo’s algorithm — it generates only n-cycles, (n−1)! outcomes, never the identity. Tests that only check “looks shuffled” pass; permutation counts fail. - Swap with random j ∈ [0, n−1] every step: the nⁿ-vs-n! bias above. Visually indistinguishable from correct; provably non-uniform for n ≥ 3.
- Deterministic pivot: taking
a[hi]as pivot without the random swap → O(n²) on sorted input; LC 215 includes such tests and Python recursion-based versions also die by stack depth. - k-th largest index arithmetic: k-th largest = index n−k (0-indexed) k-th smallest; off-by-one here returns the (k+1)-th. Assert on
([3,2,1,5,6,4], 2) → 5before submitting. - Reservoir drawing j on [0, i−1]: the i-th item then survives with k/i, not k/(i+1) — early items get overweighted. The inclusive bound carries the proof.
- rand10 truncating instead of rejecting:
rand49 % 10 + 1gives 1..9 five preimages and 10 four — a 25% relative bias no unit test with n=1000 will catch; only bucket-count tests at ~10⁵ trials expose it.
Debugging Strategy
Bias bugs don’t crash — they skew. So test distributions, not runs: bucket-count every randomized routine at ≥ 10⁵ trials and compare against expectation with a ±4σ band (σ = √(Np(1−p))); a correct implementation almost never trips it, and Sattolo/truncation bugs trip it instantly (identity permutation count = 0 is the smoking gun for bug 1). For quickselect, fix random.seed(...) to make failures reproducible, then print (lo, hi, i, k) per iteration — the window must strictly shrink and always contain k. For rand10, histogram the pre-rejection v: it must be flat on 1..49 with everything above 40 discarded.
Mastery Criteria
- Quickselect (iterative, random pivot) from blank, passing duplicates tests, in <10 minutes.
- Fisher–Yates + uniformity proof by induction delivered in <3 minutes without notes.
- Reservoir sampling written and the k/(i+1) induction stated in <8 minutes (LC 382 + LC 398 both cold).
- rand10 with the 2.45-expected-calls derivation in <6 minutes; recycled-entropy variant described.
- Named which of the four is Las Vegas vs Monte Carlo and defended expected-vs-worst-case under pushback.
- Explained hash-seed randomization / hash-flooding in <2 minutes, connecting to phase-03 lab-06.
Phase 8 — Practical Engineering Coding Interviews
Target level: Medium-Hard → Hard (senior+ practical interview track) Expected duration: 4 weeks Weekly cadence: 5–6 labs/week, with each lab requiring a complete working implementation, tests, and rehearsed answers to follow-ups Companies this targets: Big Tech L5+ (Google L5/L6, Meta E5/E6, Amazon SDE-III/Principal, Microsoft Sr/Principal), Stripe, Uber, Airbnb, Cloudflare, Datadog, Snowflake, Databricks, infrastructure-heavy startups
Why This Phase Exists
Phase 2 through Phase 7 trained you to recognize patterns and produce optimal algorithms under a stopwatch. That training is necessary and remains the gating function for the first 30 minutes of most rounds. But there is a second, distinct kind of coding interview that you will face starting at the senior level (and at every level at companies like Stripe, Airbnb, and Uber where the engineering bar is calibrated against production code rather than against contest performance).
That second kind of interview is the practical engineering coding round. You are asked to “build an LRU cache”, “build a rate limiter”, “build a thread pool”, “build a job queue”, “build a small in-memory filesystem”. The problem is not algorithmically extreme — most of these have textbook solutions you could find in a CS curriculum. What the interviewer is testing is whether your code looks like production code:
- Are your data structures encapsulated behind a clean API?
- Are mutations and reads separated cleanly?
- Are concurrency invariants explicit, or did you sprinkle locks “just in case”?
- Do you handle partial failure, shutdown, and resource cleanup?
- Did you write tests that actually exercise the contract — including concurrency tests where relevant?
- Can you answer the inevitable follow-ups about scaling, observability, and operational concerns?
Candidates from a pure LeetCode background routinely fail this round. They produce a one-function LRUCache that passes the LC test cases, then freeze when the interviewer asks “how would you make this thread-safe?” or “how would you observe this in production?” or “what would you do if a put could fail mid-operation?” The interviewer’s note reads: “Strong on the algorithm, weak on engineering. No-hire for senior.”
The bar at senior+ practical interviews is not “did you write code that produces the right answer”. The bar is “did you write code that I would be willing to deploy”. Those are different bars, and this phase trains the second one explicitly.
What Makes Practical Problems Different From LeetCode
| Dimension | LeetCode-style | Practical engineering |
|---|---|---|
| Optimization target | Big-O time, sometimes space | API surface, testability, operational fitness, correctness under concurrency |
| Code length | 20–40 lines | 100–400 lines (a class with several methods + tests) |
| State | Local to a function | Owned by a long-lived object with invariants across calls |
| Concurrency | Almost never tested | Almost always at least raised as a follow-up |
| Failure modes | “Wrong answer on test 47” | Partial failure, restart, poison input, backpressure, shutdown |
| Tests | Provided by the judge | You write them |
| Follow-ups | Variant problems with tweaked constraints | Operational reality questions (“scale to N nodes”, “persist across restarts”) |
| Bar for excellence | Optimal complexity | Production readability + correctness + answers all follow-ups crisply |
A LeetCode answer that nails the algorithm but ships a 60-line wall-of-code with single-letter variables and no separation of concerns will get a no-hire at the senior bar even when the algorithm is correct. Conversely, a practical answer that is a little slower than optimal but is cleanly structured, well-tested, and accompanied by sharp follow-up answers will get a strong hire.
You will not “see” this difference until you’ve practiced enough practical labs to internalize what “clean” looks like at the senior bar. That internalization is the entire point of this phase.
The 13 Standard Follow-Ups
Every problem in this phase will be followed by a subset of these thirteen questions. They are not problem-specific — they are senior-bar questions that recur across the industry. Memorize the question list. Then, for each lab in this phase, rehearse the answer for the 4–6 follow-ups that are most natural for that problem. By the end of Phase 8 you should be able to give a 60-to-90-second answer to any of these for any data structure or service-shaped object you’ve built.
- How would you make it thread-safe? Identify the critical sections, choose between coarse-grained mutex / fine-grained locks / lock-free / CAS / sharded locks, justify the choice, name the failure modes the choice avoids (deadlock, lost update, torn read), and state the contention behavior under load.
- How would you persist state across restarts? Pick between full snapshot, log-structured append (write-ahead log), and snapshot+log; address durability (fsync), atomicity (rename or checksum), and recovery (replay log on boot). State the time-to-recover and the worst-case data loss window.
- How would you scale to N nodes? Decide between sharding (partition the keyspace), replication (read scaling), and routing (consistent hashing + virtual nodes). Address rebalancing, hotspotting, and cross-node operations. Don’t reach for “distribute everything” — most practical objects scale by sharding.
- How would you observe and monitor it? Name the four signals (latency, traffic, errors, saturation — Google’s Golden Signals) and state which metric you’d emit for each. Specify whether you’d export histograms (latency), counters (events), or gauges (queue depth). Describe the dashboard you’d build.
- How would you test it? Three layers minimum: unit tests on each method’s contract; integration / smoke tests on end-to-end flows; concurrency / stress tests where multiple goroutines or threads exercise the object. Mention property-based testing where invariants are clean.
- What metrics would you emit? Per-operation counters (puts, gets, hits, misses); per-operation latency histograms; queue / cache size gauges; failure-class counters (eviction, timeout, retry, poison). Reject the temptation to emit everything — emit what you’d actually look at on a 3 AM page.
- How would you handle backpressure? Decide between blocking the producer, dropping the request, returning an error, or buffering with a bounded queue and rejection policy. State which one you chose and why. The wrong answer here is “we’d have a really big buffer” — that just delays the problem and worsens latency.
- How would you handle partial failure? Identify which operations can fail mid-way (a write that succeeds locally but fails to persist; a network call that times out without confirmation). Choose between idempotent retry, two-phase commit, log-and-recover, or just-fail-fast. Don’t reach for “transactions” reflexively — pick the tool that matches the problem.
- What is the eviction policy and cleanup strategy? For caches: LRU / LFU / TTL / size-bounded. For queues: drop oldest / drop newest / dead-letter. For background state: TTL + scavenger goroutine. State the worst-case eviction storm.
- What is the consistency model? Strong (linearizable), sequential, causal, eventual, monotonic-read. Most in-memory single-process objects are linearizable trivially; the question becomes interesting once replicated. Be precise about what guarantees you offer.
- What configuration knobs would you expose? Capacity, TTL, retry count, backoff base, concurrency limit, shutdown timeout. State sensible defaults. Critically: state the knobs you would not expose, because over-configuration is its own production smell.
- What is the shutdown / draining behavior? On
close()/SIGTERM: stop accepting new work, finish in-flight work up to a deadline, persist or surface anything not finished, release resources. Specify the deadline. Specify what happens when the deadline expires. - How would you handle a poison-pill input? A request that crashes the worker, exhausts memory, or causes an infinite loop. Bound resource usage per request, isolate the worker, route repeat-offending payloads to a dead-letter queue, and emit a metric. Never silently drop them.
For each lab, the Follow-up Questions section selects 4–6 of these and rehearses an answer. Memorizing one bullet per question is not enough — you need to be able to converse about the choice, naming alternatives and tradeoffs.
Implementation Discipline Expected In This Phase
This is the heaviest phase by code volume. Every lab demands a complete working implementation, not pseudocode and not a sketch. The bar is “could a coworker submit this for code review without me being embarrassed?”. Concretely:
- Idiomatic in the chosen language. Python uses snake_case, dataclasses where natural,
withblocks for locks,asynciowhere the lab demands async. Java uses camelCase, prefersjava.util.concurrentprimitives, declares interfaces. Go uses short receiver names, returns errors as last value, prefers channels for fan-out, mutexes for shared state. - Small functions, one concern per function. A method that does both validation and mutation should be split. The exception is hot-path code where inlining matters; if you inline, leave a one-line comment explaining why.
- Names that describe intent, not type.
evict_lru()note(),pending_jobsnotpj,acquire_token_or_block(timeout)nottake(). - Separation of concerns. Storage, eviction policy, concurrency primitives, observability hooks, and configuration are all distinct concerns. Most labs in this phase have natural seams between them — find the seams and respect them. A class that mixes “manages state”, “decides policy”, and “emits metrics” in every method is harder to test than three classes that compose.
- Testable design. Every public method has an obvious test. Constructors take their dependencies (the eviction policy, the clock, the metrics emitter) as parameters so tests can inject fakes. Hardcoded
time.now()calls inside business logic are a code smell — inject a clock. - Explicit error handling. Every external call has a defined behavior on failure. Silent
try/except: passis forbidden unless accompanied by a comment explaining why the exception is benign. - Concurrency invariants documented. If a class is thread-safe, say so in the docstring and name the lock that guards each field. If a class is not thread-safe, say so. The forbidden state is “it might be thread-safe, the author didn’t think about it”.
- No premature abstraction. Two implementations of an interface justify the interface. One implementation does not. Don’t add a
Storageinterface for the in-memory backing map until you actually have a second backing.
The labs do not enforce a single language across the phase. Pick Python, Java, or Go for each lab based on what feels natural. Most candidates default to Python because the standard library is rich and the syntax is dense; Java is a strong choice when concurrency and java.util.concurrent primitives are at the heart of the problem (thread pool, blocking queue, atomic counters); Go is excellent when the problem is naturally concurrent and channel-shaped (job queue, dispatcher, crawler). For each lab, the Language/Runtime Follow-ups section calls out the right idiomatic choice in each of the major languages.
The 23 Labs
| # | Lab | Core Idea (one line) |
|---|---|---|
| 01 | LRU Cache | O(1) get/put via doubly-linked list + hashmap; the canonical practical-coding warmup |
| 02 | LFU Cache | Frequency-bucketed eviction; tie-breaking by recency; harder than LRU |
| 03 | Rate Limiter | Four algorithms compared: token bucket, leaky bucket, sliding window log, sliding window counter |
| 04 | Task Scheduler | Priority-aware task scheduling with retries, backoff, and a dead-letter queue |
| 05 | Thread Pool | Bounded worker pool with work queue, graceful shutdown, and rejection policy |
| 06 | Durable Job Queue | At-least-once delivery semantics with idempotency keys and ack/nack |
| 07 | Autocomplete | Trie + per-prefix top-K with weighted scores and sub-millisecond response |
| 08 | Log Parser | Streaming log line parser with regex extraction and bounded memory |
| 09 | File Deduplication | Three-stage pipeline: size → quick hash → full hash |
| 10 | Consistent Hashing | Hash ring with virtual nodes, minimal key movement on add/remove |
| 11 | Message Dispatcher | Fan-out to N consumers with fairness, priority, and per-consumer backpressure |
| 12 | Pub/Sub | In-memory topic-based publish/subscribe with wildcard subscriptions |
| 13 | Timer Wheel | Hierarchical timer wheel for O(1) amortized timer scheduling |
| 14 | Key-Value Store | In-memory KV with TTL, snapshot+WAL persistence, and crash recovery |
| 15 | Retry With Backoff | Exponential backoff + decorrelated jitter + max-attempts + retryable-error policy |
| 16 | Circuit Breaker | Three-state machine: closed / open / half-open with sliding-window failure counting |
| 17 | Metrics Collector | Counter / gauge / histogram with bounded memory and atomic updates |
| 18 | Web Crawler | Concurrent crawler with depth limit, politeness (per-host throttle), and dedup |
| 19 | In-Memory Filesystem | ls, mkdir, addContentToFile, readContent over a tree of inodes |
| 20 | Snake Game | State machine + collision detection + score; classic OOD round |
| 21 | Tic-Tac-Toe Streaming | O(1) winner check by maintaining row/col/diagonal counters |
| 22 | Text Editor Buffer | Gap buffer / piece table for cursor-local edits; the canonical editor data structure |
| 23 | SQL-Like Engine | Toy parser + executor for SELECT … FROM … WHERE … JOIN … over in-memory tables |
The order is not arbitrary. Labs 1–6 are the canonical warmups (LRU is asked at every senior interview that uses this format). Labs 7–14 stretch into harder data-structure and operational territory. Labs 15–17 are pure operational primitives (retry, circuit breaker, metrics) that show up in service-design rounds. Labs 18–23 are larger, more open-ended OOD-style problems where the interviewer wants to see how you decompose a fuzzy problem into classes.
If you have a 4-week schedule, do six labs per week with a buffer day for the final lab and a mock-interview rehearsal. If you have an 8-week schedule, do three per week and spend the extra time on the follow-ups — that’s where senior interviews are won and lost.
Mastery Checklist
You have completed Phase 8 when you can do the following without prompting:
- Implement LRU cache with thread-safety in <20 minutes from a blank screen, including a unit test suite that exercises eviction order.
- Implement LFU cache with correct tie-breaking in <30 minutes.
- Compare the four rate-limiting algorithms verbally and justify the right pick for a stated load profile in <2 minutes.
- Implement a thread pool with bounded queue, rejection policy, and graceful shutdown in <30 minutes.
- Implement a job queue with at-least-once semantics and explain why exactly-once is impractical in <2 minutes.
- Implement an in-memory KV store with TTL eviction in <25 minutes.
- Implement a circuit breaker with all three states and explain when half-open transitions back to closed in <2 minutes.
- Implement consistent hashing with virtual nodes in <30 minutes and explain the rebalancing cost on add/remove.
- For any of the 23 problems, answer all 13 standard follow-ups crisply (60–90 seconds each) without notes.
- Identify, for any production object you’ve built (in real work or in this phase), the four Golden Signals you’d emit and justify why those four.
- State the consistency model of any data structure you’ve built in one sentence.
- Write a stress test for a concurrent data structure that actually finds bugs (i.e., randomly interleaves operations across threads, asserts invariants after, replays the seed on failure).
- Refactor one of your own LeetCode-style 50-line answers from any earlier phase into a clean, testable, production-shaped class without consulting any reference.
Exit Criteria
You may exit Phase 8 and move on to Phase 9 — Language & Runtime Deep Dive when:
- Lab completion: every one of the 23 labs has been implemented and tested by you, in a language you would actually use at work, with the test suite passing on the first run after a 24-hour gap (no peek-and-debug). The 24-hour gap matters — it tests retention, not short-term memory.
- Follow-up fluency: you can answer the 13 standard follow-ups without prompts for at least 18 of the 23 labs.
- Mock interview: you have done at least 2 mock interviews drawn from this phase’s problem list (Phase 11 — Mock Interview Mastery) with a passing rubric score on both, where “passing” requires hitting both the algorithmic correctness and the production-readiness rubric dimensions.
- Code review readiness: you can take any of your Phase 8 implementations, post it as a hypothetical PR, and write the PR description (motivation, design choices, tradeoffs, test plan) in <10 minutes per implementation.
If any of the four criteria fail, do not move on. Most candidates underestimate (3) — they pass the algorithm dimension but fail the production-readiness dimension because they didn’t rehearse the follow-ups out loud. Read COMMUNICATION.md once more, then re-do the mocks. The mocks are not optional; the practical-engineering bar is calibrated against verbalized reasoning, not solo-coded artifacts.
Cross-References
- FRAMEWORK.md — the universal 16-step framework still applies. Practical problems extend step 16 (production implications), not replace steps 1–15.
- CODE_QUALITY.md — the bar is enforced here more strictly than anywhere else in the curriculum.
- phase-03-advanced-data-structures/ — several labs (LRU, LFU, trie) build on data structures introduced there. If you skipped Phase 3, do at least labs 7 and 8 of that phase before starting here.
- phase-04-graphs/ — the consistent-hashing and dispatcher labs share modeling instincts with graph problems.
- phase-09-language-runtime/ — the next phase. Practical engineering interviews and runtime interviews are deeply intertwined; many follow-up answers in Phase 8 cite runtime facts you’ll formalize in Phase 9.
- phase-11-mock-interviews/ —
mock-08-staff-practical.mdis built around this phase’s problem list.
Lab 01 — LRU Cache
Goal
Implement a thread-unsafe and a thread-safe LRUCache with O(1) get and put, using a doubly-linked list keyed by a hashmap. After this lab you should be able to write a clean, tested LRU cache from a blank screen in under 20 minutes, and answer the 13 standard follow-ups for it crisply.
Background Concepts
A cache is a bounded-capacity associative store that evicts entries when capacity is exceeded. The Least Recently Used (LRU) policy evicts whichever entry was accessed (read or written) the furthest in the past. The reason this is the canonical practical-coding warmup is that both O(1) operations require coordinating two data structures: a hashmap that maps keys to nodes (so get is O(1)), and a doubly-linked list ordered by recency (so eviction and recency-update are O(1)). Either structure alone forces an O(N) operation. That two-structure coordination is the engineering insight the interviewer wants to see.
The doubly-linked list uses sentinel head and tail nodes — this avoids null checks at the ends and reduces the function body from a tangle of if-statements to four pointer assignments per operation.
Interview Context
LRU cache is asked at almost every senior coding interview at Big Tech, Stripe, Uber, and Cloudflare. It is LeetCode 146 verbatim, but the bar at the practical-engineering level is far higher than passing the LeetCode test cases: you must produce production-shaped code (encapsulation, naming, error handling, optional thread-safety) and answer follow-ups about concurrency, persistence, sharding, and observability. Failing to articulate any of those follow-ups crisply is a common no-hire signal even when the algorithm is correct.
Problem Statement
Design a class LRUCache(capacity) supporting:
get(key) -> value or None— return the value forkey, marking the entry as most-recently-used. ReturnNone(or sentinel) if not present.put(key, value)— insert or update. If the cache exceedscapacity, evict the least-recently-used entry.
Both operations must run in O(1) average time.
Constraints
- 1 ≤
capacity≤ 10^5 - Keys are hashable; values are arbitrary
getandputmay be called up to 10^7 times in benchmarks- Thread-safe variant: any number of concurrent callers
Clarifying Questions
- Are keys always hashable, or can they be raw bytes / mutable objects? (Assume hashable.)
- Does
putof an existing key count as a “use” for LRU ordering? (Yes — by convention.) - Is the API allowed to return a sentinel for missing keys, or must it raise? (Both are defensible; pick one and document.)
- Must the cache be thread-safe? (Often “we’ll get to that” — write the single-threaded version first, then add a lock when asked.)
- Eviction callback (notify on evict) needed? (Often a follow-up; design so it can be added without changing the call sites.)
Examples
cache = LRUCache(2)
cache.put(1, "a") # state: [1]
cache.put(2, "b") # state: [2, 1]
cache.get(1) -> "a" # state: [1, 2]
cache.put(3, "c") # evict 2; state: [3, 1]
cache.get(2) -> None
cache.put(1, "z") # update; state: [1, 3]
cache.get(3) -> "c" # state: [3, 1]
Initial Brute Force
A dict plus a list that records the access order. Each get does a linear scan to remove and re-append the key. O(N) per op.
class LRUCacheBrute:
def __init__(self, cap):
self.cap = cap
self.store = {}
self.order = [] # least-recent at front
def get(self, k):
if k not in self.store: return None
self.order.remove(k); self.order.append(k)
return self.store[k]
def put(self, k, v):
if k in self.store:
self.order.remove(k)
elif len(self.store) >= self.cap:
old = self.order.pop(0)
del self.store[old]
self.store[k] = v; self.order.append(k)
Brute Force Complexity
O(N) per get / put because list.remove and list.pop(0) are O(N). At N=10^5 with 10^7 calls, this is 10^12 operations — it will not finish.
Optimization Path
Replace the order-tracking list with a doubly-linked list, and add a hashmap that maps each key to its node. Now removal and append are O(1), so each operation is O(1). The hashmap uses ~3x more memory than the brute, which is the standard space-time tradeoff and is acceptable.
Final Expected Approach
- Node:
(key, value, prev, next). Storing the key in the node is essential — when we evict the LRU node, we need its key to remove it from the hashmap. - Sentinels:
head(most recent) andtail(least recent) sentinel nodes that always exist. Real nodes live between them. _add_after_head(node): insert immediately afterhead._remove(node): splice out by relinking neighbors._touch(node): remove + add after head. This is the recency update.get(k): hashmap lookup → if hit,_touch(node)and return value; elseNone.put(k, v): if exists, update value and_touch; else create node,_add_after_head, hashmap insert, evict if over capacity (the node beforetail).
Data Structures Used
| Structure | Purpose |
|---|---|
dict | key → node lookup, O(1) |
| Doubly-linked list (with sentinels) | recency ordering, O(1) splice |
RLock (thread-safe variant) | guards both structures together |
Correctness Argument
The invariants we maintain after every get or put:
- The hashmap and the linked list contain exactly the same set of keys.
- The list size never exceeds
capacity. - The node immediately after
headwas the most-recently accessed; the node immediately beforetailis the LRU candidate.
Each of get, put, _touch, _remove, _add_after_head preserves all three invariants by inspection of the four pointer assignments per splice. The eviction step in put is the only place that may shrink both structures; it removes exactly one node from both, by reading its key from the node before deleting from the hashmap.
Complexity
get: O(1) average (hashmap lookup + 4 pointer writes)put: O(1) average- Space: O(capacity)
Implementation Requirements
A complete working implementation is required. Below is the canonical Python version with thread-safety and a clean separation between the storage (the linked list + hashmap) and the policy (eviction order). Tests are required.
import threading
from typing import Hashable, Any, Optional
class _Node:
__slots__ = ("key", "val", "prev", "next")
def __init__(self, key=None, val=None):
self.key, self.val = key, val
self.prev = self.next = None
class LRUCache:
"""Thread-safe O(1) LRU cache.
Concurrency: a single RLock guards both the hashmap and the linked list.
The locked region is short (constant pointer work), so contention is low
until very high concurrency. For higher concurrency, see follow-up #1.
"""
def __init__(self, capacity: int):
if capacity <= 0:
raise ValueError("capacity must be positive")
self._cap = capacity
self._map: dict[Hashable, _Node] = {}
self._head, self._tail = _Node(), _Node()
self._head.next = self._tail
self._tail.prev = self._head
self._lock = threading.RLock()
def get(self, key: Hashable) -> Optional[Any]:
with self._lock:
node = self._map.get(key)
if node is None:
return None
self._touch(node)
return node.val
def put(self, key: Hashable, value: Any) -> None:
with self._lock:
node = self._map.get(key)
if node is not None:
node.val = value
self._touch(node)
return
node = _Node(key, value)
self._map[key] = node
self._add_after_head(node)
if len(self._map) > self._cap:
lru = self._tail.prev
self._remove(lru)
del self._map[lru.key]
def __len__(self) -> int:
with self._lock:
return len(self._map)
def _add_after_head(self, node: _Node) -> None:
node.prev = self._head
node.next = self._head.next
self._head.next.prev = node
self._head.next = node
def _remove(self, node: _Node) -> None:
node.prev.next = node.next
node.next.prev = node.prev
def _touch(self, node: _Node) -> None:
self._remove(node)
self._add_after_head(node)
Tests
Required: unit tests for the contract, smoke tests for ordering, concurrency tests for the thread-safe variant.
import unittest, threading, random
class TestLRU(unittest.TestCase):
def test_basic(self):
c = LRUCache(2)
c.put(1, "a"); c.put(2, "b")
self.assertEqual(c.get(1), "a")
c.put(3, "c") # evicts 2
self.assertIsNone(c.get(2))
self.assertEqual(c.get(3), "c")
self.assertEqual(c.get(1), "a")
def test_update_is_a_use(self):
c = LRUCache(2)
c.put(1, "a"); c.put(2, "b"); c.put(1, "z")
c.put(3, "c") # evicts 2 (1 was just updated)
self.assertEqual(c.get(1), "z")
self.assertIsNone(c.get(2))
def test_capacity_one(self):
c = LRUCache(1)
c.put(1, "a"); c.put(2, "b")
self.assertIsNone(c.get(1))
self.assertEqual(c.get(2), "b")
def test_concurrent(self):
c = LRUCache(100)
def worker():
for _ in range(10000):
k = random.randint(0, 200)
if random.random() < 0.5:
c.put(k, k * 2)
else:
c.get(k)
threads = [threading.Thread(target=worker) for _ in range(8)]
for t in threads: t.start()
for t in threads: t.join()
self.assertLessEqual(len(c), 100) # invariant: never over capacity
Follow-up Questions
(1) How would you make it thread-safe? Already shown: a single RLock around the body of every public method. The lock is held only for O(1) work, so contention is bounded. For higher concurrency, shard by hash(key) % N into N independent caches; this is the practical answer for production caches at high QPS. A lock-free LRU is hard and rarely worth it.
(2) How would you persist state across restarts? Snapshot the cache to disk on a configurable interval (write the (key, value, recency-rank) triples in LRU order). On restart, replay the file in order. For stricter durability, write a per-put log entry to a write-ahead log; on restart, replay snapshot + log. Note: most caches choose not to persist — losing the cache on restart is usually fine, and persistence adds complexity for small benefit.
(4) How would you observe and monitor it? Emit hit-rate (hits / (hits + misses)) as a gauge; emit eviction count as a counter; emit cache size as a gauge; emit get/put latency as a histogram. Hit rate is the #1 actionable signal. Set an alert on hit rate dropping below the SLO.
(7) How would you handle backpressure? Caches don’t have classical backpressure since there’s no producer queue, but the analog is memory pressure: if the host is short on memory, the cache should shed load. Either (a) a soft size_in_bytes ceiling that triggers eviction beyond capacity, or (b) integrate with a host-level memory pressure signal (cgroup memory accounting on Linux). Decide explicitly which.
(11) What configuration knobs would you expose? capacity (entries), optionally size_bytes (RAM ceiling), optionally eviction_callback. Knobs not to expose: lock granularity, internal data structure choice, snapshot interval (set sensible default and document).
(12) What is the shutdown / draining behavior? The cache itself is in-memory and stateless from the caller’s perspective; on shutdown, optionally write a snapshot, then release the lock and let GC reclaim. No draining required; no in-flight work to finish.
Product Extension
LRU caches are the workhorse of CDN edge caches, database query result caches (Redis with allkeys-lru), HTTP reverse proxies, and database buffer pools. Real-world variants: two-level cache (L1 in-process + L2 Redis); size-aware LRU (counts bytes, not entries); adaptive LRU/LFU hybrid (ARC, used by ZFS). The data structure you wrote here is the textbook foundation; the variants tune it for specific workloads.
Language/Runtime Follow-ups
- Python:
collections.OrderedDictalready implements LRU semantics —move_to_end()for_touch,popitem(last=False)for evict. In an interview, state that you knowOrderedDictexists, then implement from scratch because the interviewer wants to see the linked list. Production: preferOrderedDictorfunctools.lru_cache(decorator) unless you need eviction callbacks. - Java:
LinkedHashMapwithaccessOrder=trueand an overriddenremoveEldestEntryis the textbook Java LRU. For thread-safety wrap withCollections.synchronizedMap, or use Caffeine (Guava successor) which is O(1) and concurrent. - Go: no stdlib LRU; the
container/listpackage gives a linked list, and amap[K]*list.Elementgives O(1) lookup. Thehashicorp/golang-lrupackage is the de facto standard in production Go. - C++:
std::list<std::pair<K,V>>plusstd::unordered_map<K, list::iterator>. Iterator stability ofstd::listis the reason this works. - JS/TS:
Mappreserves insertion order, somap.delete(k); map.set(k, v)is the recency-update pattern. Eviction ismap.delete(map.keys().next().value). This works becauseMap.prototype.keys()returns keys in insertion order.
Common Bugs
- Forgetting to update the hashmap when evicting (only removing from the linked list). The next
getfor the evicted key returns a dangling node. The fix is to read the node’skeybefore unlinking and use it to delete from the hashmap. - Storing only the value (not the key) in the node, then having no way to find the hashmap entry to delete on eviction.
- Calling
_touchbefore checking whether the key exists — touches aNone. - In the thread-safe variant, taking the lock in
getbut not input, or releasing between the eviction step and the insert step. The whole operation must be atomic. - Using a non-reentrant
Lockand then calling another locked method internally — deadlock. UseRLockif you need to call locked methods from inside a locked method.
Debugging Strategy
If get returns the wrong value: print [(n.key, n.val) for n in walk(self._head)] after each put, compare to the expected access order. If the cache exceeds capacity: assert len(self._map) <= self._cap after every put; the violation tells you which call broke the invariant. For concurrency bugs (rare under the single-lock design), run the concurrent test with pytest --count=1000 until a failure repros, then add print(threading.get_ident(), op, key) traces and minimize.
Mastery Criteria
-
Implemented
LRUCachewith sentinel head/tail in <15 minutes from blank screen. - All four tests pass on first run.
- Articulated invariants (hashmap-list set equality, capacity bound, head/tail recency) without prompting.
- Stated O(1) time and O(capacity) space unprompted.
- Answered follow-ups #1, #4, #7, #11, #12 in <90 seconds each, naming alternatives.
- Refactored a single-threaded version into the thread-safe version in <5 minutes.
Lab 02 — LFU Cache
Goal
Implement a LFUCache with O(1) get and put. The challenge over LRU is the tie-breaking rule: when multiple keys share the minimum frequency, evict the least-recently-used among them. Expect this to take 30 minutes from blank screen on first attempt; aim to bring it down to 20 with practice.
Background Concepts
LFU evicts the least-frequently-used entry. Frequency means “number of get plus put calls referencing that key since insertion”. A naive frequency-counter approach forces an O(N) eviction scan. The O(1) trick is to bucket nodes by frequency: a hashmap from freq → DoublyLinkedList plus a min_freq cursor. On eviction, pop the LRU entry from freq_map[min_freq]. On a hit, move the node from freq_map[f] to freq_map[f+1]; when freq_map[min_freq] empties, advance min_freq.
This is a textbook example of bucket sort applied to a dynamic counter. Each frequency bucket is itself an LRU list, which handles tie-breaking in O(1). Together you get O(1) for both ops at the cost of more code than LRU.
Interview Context
LFU follows LRU as the second-asked cache problem at senior practical interviews. It is LeetCode 460 verbatim. Where LRU is a 20-minute problem, LFU is a 30-to-45-minute problem and tests whether you can design with two coordinated abstractions (the freq map and the per-bucket LRU lists). Most candidates fail by picking a frequency-only design and then hitting the tie-breaking case.
Problem Statement
Design LFUCache(capacity):
get(key) -> value or None— increment frequency on hit.put(key, value)— insert/update; on capacity overflow, evict the LFU entry. Tie-break by LRU within the LFU bucket. Insertion sets frequency to 1.
Both O(1) average.
Constraints
- 1 ≤
capacity≤ 10^5 - 10^7 ops in benchmarks
- Thread-safety: optional follow-up
Clarifying Questions
- Does
putof an existing key increment frequency? (Yes by convention; confirm.) - Tie-breaking: LRU or arbitrary? (LRU is the standard expectation; confirm.)
- What does
getreturn on miss? (Noneor sentinel; confirm.) - Should frequencies decay over time (windowed LFU)? (Often a follow-up; default is “no decay”.)
Examples
c = LFUCache(2)
c.put(1, "a") # freq: {1->1}
c.put(2, "b") # freq: {1->1, 2->1}
c.get(1) -> "a" # freq: {1->2, 2->1}
c.put(3, "c") # evict 2 (freq=1, LRU); state {1, 3}
c.get(2) -> None
c.get(3) -> "c" # freq: {1->2, 3->2}
c.put(4, "d") # tie at freq=2; evict 1 (LRU); state {3, 4}
c.get(1) -> None
Initial Brute Force
dict mapping key → (value, freq, last_access_time). On eviction, scan all entries to find the (min freq, min time) pair. O(N) eviction.
Brute Force Complexity
get is O(1), but put with eviction is O(N). At N=10^5 over 10^7 calls, slow but tolerable for small inputs; will TLE at large N.
Optimization Path
Replace the linear eviction scan with frequency bucketing. Maintain:
key_to_node: dict[K, Node]for O(1) lookupfreq_to_list: dict[int, DoublyLinkedList]mapping frequency to an LRU list of nodesmin_freq: inttracking the current minimum frequency present
On get(k): look up node, remove from freq_to_list[node.freq], increment node.freq, append to freq_to_list[node.freq]. If the old bucket is now empty and it equaled min_freq, increment min_freq.
On put(k, v): if exists, update value and behave like get. Otherwise, evict from freq_to_list[min_freq] if at capacity (pop the front = LRU), then insert with freq=1 and reset min_freq=1.
The reset of min_freq=1 on insertion is the only step that’s not obvious; without it you’d evict the wrong bucket on the very next put.
Final Expected Approach
Three layers: Node, per-frequency DoublyLinkedList (with sentinels), LFUCache orchestrating the two maps. The per-bucket linked list handles LRU tie-breaking automatically — append on insert, pop from front on evict.
Data Structures Used
| Structure | Purpose |
|---|---|
dict[K, Node] | O(1) key lookup |
dict[int, DLList] | O(1) frequency → bucket |
| Doubly-linked list per bucket | LRU within frequency, O(1) splice |
min_freq: int | O(1) eviction target |
Correctness Argument
Invariants maintained after every operation:
key_to_nodeand the union of allfreq_to_list[f]contain the same set of keys.- Every node
nlives infreq_to_list[n.freq]and only there. min_freqis the smallestfsuch thatfreq_to_list[f]is non-empty (or any value when the cache is empty).- Within each bucket, the front is the LRU and the back is the MRU.
Each method preserves these by case analysis. The subtle case is on get: incrementing node.freq may empty the old bucket. We check if was_min_freq_bucket and now_empty: min_freq += 1. This is correct because every other key has frequency ≥ old min_freq + 1, since old min_freq was the global minimum and this was the only node at it (we know that because the bucket is now empty, but the bucket contained this node before the increment — so it was the only node).
Complexity
get: O(1)put: O(1)- Space: O(capacity)
Implementation Requirements
from typing import Hashable, Any, Optional
class _Node:
__slots__ = ("key", "val", "freq", "prev", "next")
def __init__(self, key=None, val=None, freq=1):
self.key, self.val, self.freq = key, val, freq
self.prev = self.next = None
class _DLList:
"""Doubly linked list with sentinels. Front = LRU, back = MRU."""
def __init__(self):
self.head, self.tail = _Node(), _Node()
self.head.next, self.tail.prev = self.tail, self.head
self.size = 0
def append(self, node: _Node) -> None: # add to back (MRU)
prev = self.tail.prev
prev.next = node; node.prev = prev
node.next = self.tail; self.tail.prev = node
self.size += 1
def remove(self, node: _Node) -> None:
node.prev.next = node.next; node.next.prev = node.prev
self.size -= 1
def pop_front(self) -> _Node: # evict LRU
node = self.head.next
self.remove(node)
return node
def is_empty(self) -> bool:
return self.size == 0
class LFUCache:
def __init__(self, capacity: int):
if capacity <= 0:
raise ValueError("capacity must be positive")
self._cap = capacity
self._key_to_node: dict[Hashable, _Node] = {}
self._freq_to_list: dict[int, _DLList] = {}
self._min_freq = 0
def get(self, key: Hashable) -> Optional[Any]:
node = self._key_to_node.get(key)
if node is None:
return None
self._bump(node)
return node.val
def put(self, key: Hashable, value: Any) -> None:
if self._cap == 0:
return
node = self._key_to_node.get(key)
if node is not None:
node.val = value
self._bump(node)
return
if len(self._key_to_node) >= self._cap:
evicted = self._freq_to_list[self._min_freq].pop_front()
del self._key_to_node[evicted.key]
node = _Node(key, value, freq=1)
self._key_to_node[key] = node
self._freq_to_list.setdefault(1, _DLList()).append(node)
self._min_freq = 1
def _bump(self, node: _Node) -> None:
old_list = self._freq_to_list[node.freq]
old_list.remove(node)
if old_list.is_empty() and node.freq == self._min_freq:
self._min_freq += 1
node.freq += 1
self._freq_to_list.setdefault(node.freq, _DLList()).append(node)
def __len__(self) -> int:
return len(self._key_to_node)
Tests
import unittest
class TestLFU(unittest.TestCase):
def test_basic_eviction(self):
c = LFUCache(2)
c.put(1, "a"); c.put(2, "b")
self.assertEqual(c.get(1), "a") # 1.freq=2, 2.freq=1
c.put(3, "c") # evict 2
self.assertIsNone(c.get(2))
self.assertEqual(c.get(3), "c") # 3.freq=2
def test_tie_break_lru(self):
c = LFUCache(2)
c.put(1, "a"); c.put(2, "b") # both freq=1; 1 is LRU
c.put(3, "c") # evict 1 (LRU at freq=1)
self.assertIsNone(c.get(1))
self.assertEqual(c.get(2), "b")
def test_update_increments_freq(self):
c = LFUCache(2)
c.put(1, "a"); c.put(2, "b"); c.put(1, "z") # 1.freq=2
c.put(3, "c") # evict 2 (freq=1)
self.assertEqual(c.get(1), "z")
self.assertIsNone(c.get(2))
def test_capacity_zero(self):
c = LFUCache(0)
c.put(1, "a")
self.assertIsNone(c.get(1))
def test_min_freq_advance(self):
c = LFUCache(2)
c.put(1, "a"); c.put(2, "b")
c.get(1); c.get(1); c.get(2); c.get(2) # both freq=3
c.put(3, "c") # evict 1 (LRU at freq=3)
self.assertIsNone(c.get(1))
self.assertEqual(c.get(2), "b")
Follow-up Questions
(1) Thread-safe? A single RLock around get, put, and _bump is correct and the standard production answer. The locked region is O(1), so contention is bounded. Sharding by hash(key) % N into N independent LFU caches is the higher-throughput choice — but each shard has its own LFU bucketing, which is fine because LFU is per-key.
(4) Observe and monitor? Hit rate (hits / (hits + misses)) as a gauge; eviction count as a counter; frequency distribution as a histogram (10th, 50th, 90th, 99th percentile of frequency at eviction time) — this tells you whether the cache is actually being used “frequency-aware” or whether everything has freq=1. Cache size as a gauge.
(9) Eviction policy / cleanup? This is the eviction policy. The catch: long-tail entries can pile up at high frequency and never get evicted even after they go cold (a value popular yesterday is still freq=1000 today). Solutions: windowed LFU (decay frequencies on a timer), LFU-Aging (halve all frequencies periodically), or TinyLFU (admission filter that uses a count-min sketch). State the limitation and pick a mitigation.
(10) Consistency model? Linearizable in a single process under the lock — every get/put appears to take effect instantly at some moment between invocation and return. Replicated LFU is harder; most distributed caches degrade to eventual consistency on the cache and rely on an authoritative store underneath.
(11) Configuration knobs? capacity, optionally decay_interval (for windowed LFU), optionally eviction_callback. Don’t expose internal data-structure tuning.
Product Extension
LFU is the right policy when access patterns are stationary (popular items stay popular). It outperforms LRU on workloads with strong popularity skew (web caches, recommendation systems, query result caches). It performs worse than LRU on scan-heavy workloads (a large one-time scan pollutes LRU with a single bump but pollutes LFU with cold-but-frequent entries until aging kicks in). TinyLFU (used in Caffeine) combines a count-min admission filter with a small LRU window, getting LFU’s hit rate without the staleness problem.
Language/Runtime Follow-ups
- Python: same approach as shown.
collections.Counterfor frequencies is tempting but doesn’t give the bucketing we need. - Java: build on
LinkedHashMapfor the per-frequency buckets,HashMap<Integer, LinkedHashSet<K>>for the freq map. Caffeine provides production-grade TinyLFU. - Go:
container/listper bucket,map[int]*list.Listfor buckets. Thegroupcachelibrary uses a different policy (LRU); for LFU, write it yourself or usedgraph-io/ristretto(TinyLFU). - C++:
std::list<Node>per bucket;std::unordered_map<int, list>for freq map. - JS/TS:
Map<int, Set<K>>for buckets —Setpreserves insertion order, so LRU-within-bucket is free.
Common Bugs
- Forgetting to advance
min_freqwhen the LRU bucket becomes empty after a_bump. Subsequent eviction picks an empty bucket and crashes. - Resetting
min_freq=1on insert before the eviction step instead of after, evicting the wrong bucket. - Tie-breaking by MRU instead of LRU — appending to the front of the bucket instead of the back. The bucket is itself an LRU list; respect the convention.
- Sharing a single
_DLListacross freq buckets accidentally (e.g., a class-level default). Usesetdefault(freq, _DLList()). - On
putof an existing key, decrementing-then-incrementing frequency instead of bumping — produces correct frequency but breaks the bucketing if you forget to move buckets.
Debugging Strategy
Print the bucketing as {freq: [keys]} after every operation. The bug usually shows as a stale entry in min_freq’s bucket or a missing entry in the new bucket. Add an assert _check_invariants() method that walks the buckets and verifies (a) bucket→key relations, (b) min_freq correctness, (c) hashmap-bucket set equality. Run the test suite with assertions on.
Mastery Criteria
-
Implemented
LFUCachefrom blank screen in <30 minutes (target: <25 after second attempt). - All five tests pass first run.
- Stated tie-breaking-via-per-bucket-LRU explicitly without prompting.
-
Articulated the
min_freqadvancement rule precisely. - Answered follow-ups #1, #4, #9 (LFU-Aging / TinyLFU), #10, #11 in <90 seconds each.
- Compared LFU vs LRU on scan-heavy and popularity-skewed workloads in <60 seconds.
Lab 03 — Rate Limiter
Goal
Implement four rate-limiting algorithms — token bucket, leaky bucket, sliding window log, sliding window counter — and articulate the tradeoffs between them. After this lab you should be able to pick the right algorithm for a stated workload in under 30 seconds and implement any of the four in under 15 minutes.
Background Concepts
A rate limiter caps the number of requests a key (user, IP, API token) may make over a time window. The four standard algorithms differ in how much history they keep and what kind of bursts they allow:
- Token bucket: a bucket of capacity
Brefills at rateRper second; each request consumes one token; if no token, reject. Allows bursts up toB. - Leaky bucket: requests enter a queue of capacity
Qthat drains at rateR; if the queue is full, reject. Smooths bursts (output rate is constant). - Sliding window log: keep a list of timestamps over the last
Wseconds; reject iflen(log) ≥ N. Most accurate, most memory. - Sliding window counter: keep a count for the current and previous fixed window; estimate by linear interpolation. Cheap; mildly inaccurate at boundaries.
The token bucket is by far the most-common production choice (used by AWS, Stripe, GitHub) because it gives sensible burst tolerance with O(1) memory per key. Sliding window log is the choice when you must guarantee strict request-count caps (e.g., quota enforcement against a legal contract). Leaky bucket is used in network shaping. Sliding window counter is the right pick when memory is constrained and approximate is acceptable.
Interview Context
Rate limiter is asked at every senior+ practical round at Stripe, Cloudflare, Uber, and most high-scale API companies. The strong answer compares the four algorithms, picks one, justifies the pick, implements it, and answers the inevitable follow-ups about distributed coordination (multiple servers must share one quota), persistence, and observability. The weak answer implements one variant without acknowledging the others exist.
Problem Statement
Design RateLimiter with allow(key) -> bool. Configurable rate R requests per W seconds. Implement all four algorithms behind a common interface so they can be benchmarked.
Constraints
- Up to 10^6 distinct keys
- Up to 10^5 QPS aggregate
- Sub-millisecond per-call latency
- Configurable rate per key (follow-up)
Clarifying Questions
- Per-key or global limit? (Per-key by convention.)
- Should refused requests be queued or rejected? (Token bucket rejects; leaky bucket queues.)
- Time source: monotonic clock or wall clock? (Always monotonic — wall clock can jump backward.)
- Distributed across N servers, or single-process? (Often a follow-up; default single-process.)
- Burst tolerance — yes or no? (Token bucket allows bursts; sliding window log enforces strict.)
Examples
Rate = 5 req / 1 s.
Token bucket (capacity 5, refill 5/s):
t=0: 5 quick requests → all allow (bucket drains 5→0)
t=0.1: 1 request → reject (bucket=0.5 < 1)
t=1.0: 1 request → allow (bucket refilled to 5; now 4)
Sliding window log (limit 5 over 1 s):
t=0..0.5: 5 requests → all allow
t=0.6: 1 request → reject (6 in last 1 s)
t=1.1: 1 request → allow (the t=0 request slid out)
Initial Brute Force
A dict[key, list[timestamp]] and on every allow, filter out timestamps older than W and check len. This is the sliding window log baseline; it is O(history-size) per call and unbounded memory. Acceptable for low-rate testing; not acceptable for production at 10^5 QPS.
Brute Force Complexity
Per call: O(N) where N is the request count in the window. Memory: O(N · keys). At 10^5 QPS over 1-second windows for 10^6 keys, the memory could approach 10^11 timestamps — orders of magnitude too high.
Optimization Path
For each algorithm, the optimization target is different:
- Token bucket: store
(tokens, last_refill_time)per key. Compute refill on demand:tokens = min(B, tokens + (now - last_refill) * R). O(1) per call, O(1) memory per key. - Leaky bucket: equivalent to token bucket if reject-on-full; if queue requests, store the queue.
- Sliding window log: same as brute force, but trim the prefix lazily on each call. O(amortized 1) per call.
- Sliding window counter: store
(curr_count, prev_count, curr_window_start). Approximate the rolling count withprev * (1 - elapsed/W) + curr. O(1) per call, O(1) memory.
The token bucket is the dominant choice; the other three are presented for comparison.
Final Expected Approach
Define a RateLimiter interface with allow(key) -> bool. Implement four classes. Each takes rate: float (per second) and capacity (or window). Use time.monotonic(). Make all four thread-safe via per-key fine-grained locks (a dict[key, Lock] lazily created — or just a single global lock, which is simpler and adequate for most workloads).
Data Structures Used
| Algorithm | Per-key state |
|---|---|
| Token bucket | (float tokens, float last_refill_t) |
| Leaky bucket (reject) | (float queue_size, float last_drain_t) |
| Sliding log | deque[float] of timestamps |
| Sliding counter | (int curr, int prev, float window_start) |
Correctness Argument
Token bucket: tokens are produced at rate R continuously and capped at B; consumption is one per allowed request. Equivalent to the differential equation dt/dt = R - consumption, integrated by the lazy refill formula. Correct provided we never let tokens go below 0 (we check >= 1 before decrementing) or above B (the min(B, ...)).
Sliding window log: the invariant is “at any time, the deque contains exactly the timestamps in [now - W, now]”. We maintain it by trimming the prefix on every call. Then allow is len(deque) < N.
Sliding window counter: the approximation is estimate = prev_count * (1 - elapsed/W) + curr_count. This is exact when requests are uniformly distributed within each window and an upper bound otherwise (off by at most one window’s burst). Acceptable for most production rate limiters.
Complexity
| Algorithm | Time / call | Space / key |
|---|---|---|
| Token bucket | O(1) | O(1) |
| Leaky bucket | O(1) | O(1) |
| Sliding log | O(1) amortized | O(N) |
| Sliding counter | O(1) | O(1) |
Implementation Requirements
import time, threading
from collections import deque
from typing import Hashable
class TokenBucket:
def __init__(self, rate: float, capacity: float):
self._rate, self._cap = rate, capacity
self._state: dict[Hashable, list[float]] = {} # key -> [tokens, last_t]
self._lock = threading.Lock()
def allow(self, key: Hashable) -> bool:
now = time.monotonic()
with self._lock:
s = self._state.get(key)
if s is None:
s = [self._cap, now]; self._state[key] = s
tokens, last = s
tokens = min(self._cap, tokens + (now - last) * self._rate)
if tokens >= 1:
s[0] = tokens - 1; s[1] = now
return True
s[0] = tokens; s[1] = now
return False
class SlidingWindowLog:
def __init__(self, max_in_window: int, window_s: float):
self._max, self._w = max_in_window, window_s
self._logs: dict[Hashable, deque[float]] = {}
self._lock = threading.Lock()
def allow(self, key: Hashable) -> bool:
now = time.monotonic()
with self._lock:
dq = self._logs.setdefault(key, deque())
cutoff = now - self._w
while dq and dq[0] < cutoff:
dq.popleft()
if len(dq) >= self._max:
return False
dq.append(now)
return True
class SlidingWindowCounter:
def __init__(self, max_in_window: int, window_s: float):
self._max, self._w = max_in_window, window_s
self._state: dict[Hashable, list] = {} # [curr, prev, window_start]
self._lock = threading.Lock()
def allow(self, key: Hashable) -> bool:
now = time.monotonic()
with self._lock:
s = self._state.get(key)
if s is None:
s = [0, 0, now]; self._state[key] = s
curr, prev, ws = s
elapsed = now - ws
if elapsed >= 2 * self._w:
curr = prev = 0; ws = now
elif elapsed >= self._w:
prev, curr = curr, 0; ws += self._w
elapsed -= self._w
estimate = prev * (1 - elapsed / self._w) + curr
if estimate >= self._max:
s[0], s[1], s[2] = curr, prev, ws
return False
s[0], s[1], s[2] = curr + 1, prev, ws
return True
class LeakyBucket:
"""Reject-on-full leaky bucket. Equivalent to token bucket when reject."""
def __init__(self, rate: float, capacity: float):
self._rate, self._cap = rate, capacity
self._state: dict[Hashable, list[float]] = {}
self._lock = threading.Lock()
def allow(self, key: Hashable) -> bool:
now = time.monotonic()
with self._lock:
s = self._state.get(key)
if s is None:
s = [0.0, now]; self._state[key] = s
level, last = s
level = max(0.0, level - (now - last) * self._rate)
if level + 1 > self._cap:
s[0], s[1] = level, now
return False
s[0], s[1] = level + 1, now
return True
Tests
import unittest, time
class TestTokenBucket(unittest.TestCase):
def test_burst_then_steady(self):
rl = TokenBucket(rate=5, capacity=5)
# Burst: 5 quick allows
for _ in range(5):
self.assertTrue(rl.allow("k"))
# 6th: reject (bucket empty)
self.assertFalse(rl.allow("k"))
# Wait 0.4s → 2 tokens accumulated
time.sleep(0.4)
self.assertTrue(rl.allow("k"))
self.assertTrue(rl.allow("k"))
def test_per_key_isolation(self):
rl = TokenBucket(rate=1, capacity=1)
self.assertTrue(rl.allow("a"))
self.assertTrue(rl.allow("b")) # different key, full bucket
self.assertFalse(rl.allow("a"))
class TestSlidingLog(unittest.TestCase):
def test_strict_count(self):
rl = SlidingWindowLog(max_in_window=3, window_s=1.0)
for _ in range(3):
self.assertTrue(rl.allow("k"))
self.assertFalse(rl.allow("k"))
time.sleep(1.05)
self.assertTrue(rl.allow("k")) # log has slid out
Follow-up Questions
(3) How would you scale to N nodes? This is the key follow-up. Options: (a) sticky routing — route all requests for a key to a fixed node by hash(key) % N; each node enforces locally. Simple, but rebalancing on add/remove is painful. (b) Centralized counter in Redis using INCR + EXPIRE per window. Network round-trip per call — only works at moderate QPS. (c) Approximate distributed: each node enforces R/N locally and accepts that bursts up to R are possible. The pragmatic real-world answer; documents your error budget. (d) Token bucket in Redis with Lua script for atomic refill+decrement — Stripe and GitHub do this in production.
(7) How would you handle backpressure? The whole point of a rate limiter is backpressure on upstream traffic. The question becomes: when the limiter rejects, what does the client see? HTTP 429 with Retry-After header is the standard. Enable cooperative backoff so clients don’t retry-storm. Optionally include X-RateLimit-Remaining and X-RateLimit-Reset headers (GitHub convention).
(9) What’s the eviction policy? Per-key state grows unbounded if you never clean up. Two strategies: (a) lazy expiry — when a key has been silent for >2W, drop its state on the next access. (b) Background scavenger — periodically scan and remove stale entries. The lazy approach is preferred; it’s O(0) overhead in the steady state.
(11) What configuration knobs? rate and capacity (or window) per limit class. Optionally per-key overrides (a dict[key, (rate, cap)] for VIP customers). Knobs not to expose: the algorithm choice (pick one and stick).
(4) How would you observe / monitor? Allow rate (counter), reject rate (counter), reject ratio (gauge), per-key reject rate (top-K dashboard for finding hot keys). Bucket-fill / queue-length gauge for diagnosing whether you’re rate-limited because of bursts or steady overload.
Product Extension
Stripe’s API uses token bucket with per-account capacity. AWS API Gateway uses token bucket per stage. GitHub’s API uses sliding window with hourly windows visible to clients. Twitter (X) uses fixed windows for some endpoints, sliding for others. The choice depends on the contract you offer customers (“up to 5 burst requests” → token bucket; “exactly 5000 per hour” → sliding log or sliding counter).
Language/Runtime Follow-ups
- Python: the implementation above. For high QPS, replace the global lock with a
dict[key, Lock]lazily, or shard byhash(key) % N. - Java: Guava’s
RateLimiteris a token bucket with smoothing options. For distributed, Bucket4j is excellent. - Go:
golang.org/x/time/rateis a token bucket (Allow/Wait). For distributed, use Redis with a Lua script. - C++: no stdlib; use
std::chrono::steady_clock::now(). Folly has a token bucket. - JS/TS:
bottleneck(npm) is the canonical client-side. Server-side: Redis-backed for distributed.
Common Bugs
- Using
time.time()(wall clock) instead oftime.monotonic()— clock skew or NTP adjustments cause negative deltas and free tokens. - Token bucket: not capping at
B— bucket grows unbounded over idle periods; first burst is huge. - Sliding log: not trimming on every call, only on insert — memory grows for read-heavy patterns.
- Sliding counter: failing to advance the window pointer when 2W has passed (key idle for long enough that both windows are stale).
- Forgetting per-key isolation — a single shared bucket across all keys.
Debugging Strategy
Log every (key, allow/reject, bucket_state) transition for a single key. Hand-trace against expected behavior. For distributed bugs, capture the Lua script’s input and output and replay against a local Redis. For thundering-herd bugs (many clients see “reset” simultaneously and all retry at once), add jitter on Retry-After (server-side recommends Retry-After: random_in_range(t, 2t)).
Mastery Criteria
- Implemented all four algorithms in <60 minutes total (15 min each).
- All tests pass first run.
- Compared the four algorithms verbally in <90 seconds, naming a workload where each is the right choice.
-
Stated why
time.monotonic()is required without prompting. - Answered follow-ups #3 (distributed), #4, #7, #9, #11 in <90 seconds each.
- Identified that the leaky bucket and reject-on-full token bucket are mathematically equivalent when reject (different when queue).
Lab 04 — Task Scheduler
Goal
Implement a TaskScheduler that accepts tasks with priorities, executes them in priority order via a worker pool, retries on failure with exponential backoff, and routes permanently-failed tasks to a dead-letter queue. After this lab you should be able to design and implement a small in-memory task queue with retry semantics in under 35 minutes.
Background Concepts
A task scheduler is the in-memory cousin of Celery / Sidekiq / RQ. The four moving parts are:
- Priority queue of pending tasks (heap keyed on priority + scheduled-execution-time).
- Worker pool that pops tasks and runs them.
- Retry policy that decides if and when a failed task is re-enqueued (with delayed visibility).
- Dead-letter queue (DLQ) for tasks that have exhausted retries.
The non-trivial design question is how to handle delayed re-enqueue for retries. The clean answer is to use a single priority queue keyed by (priority, ready_at), and have workers wait_until_ready on the head of the queue. This unifies “high-priority now” and “low-priority retry-in-30-seconds” under one data structure.
Interview Context
Task scheduler problems are popular at infrastructure-heavy companies (Uber, Cloudflare, AWS Lambda team, Datadog) because they touch concurrency, priority queues, retry semantics, and DLQ design — the building blocks of every async-job system. The interviewer will probe whether you’ve thought about idempotency, exactly-once vs at-least-once, and observability.
Problem Statement
Design TaskScheduler(n_workers, max_retries):
submit(task: Callable, priority: int, max_attempts: int = 3) -> task_id— enqueue a task with given priority. Lower numeric priority = runs first.start()— start the worker pool.shutdown(timeout)— stop accepting new tasks; finish in-flight up to timeout; return.dead_letters() -> list[FailedTask]— return tasks that exhausted retries.
Behavior: failures (raised exception) → exponential backoff retry up to max_attempts; permanent failure → DLQ.
Constraints
- Up to 10^4 pending tasks
- Up to 100 workers
- Per-task max execution time: 60s (a configurable per-task timeout is a follow-up)
- Tasks may be of arbitrary type but assumed to be deterministic-ish
Clarifying Questions
- Are tasks idempotent? (We’ll assume yes; idempotency is the user’s responsibility for at-least-once correctness.)
- Priority semantics: lower = higher? (Yes by convention, like a min-heap.)
- What does retry mean — the same task is re-run, or a new attempt object? (Same callable, same args, attempt counter incremented.)
- Should retries preserve original priority? (Yes by convention.)
- Cancellation? (Often a follow-up; default no.)
Examples
sched = TaskScheduler(n_workers=2, max_retries=3)
sched.start()
sched.submit(lambda: print("a"), priority=1)
sched.submit(lambda: print("b"), priority=0) # runs first
sched.submit(failing_task, priority=2, max_attempts=2)
# 1st attempt fails; retry after backoff
# 2nd attempt fails; → DLQ
sched.shutdown(timeout=5.0)
sched.dead_letters() # [FailedTask(failing_task, attempts=2, last_error=...)]
Initial Brute Force
A single thread polling a list sorted by priority. Run, retry inline. Single worker, no parallelism. O(N log N) per submit.
Brute Force Complexity
Per submit: O(N log N) on the sort. Per dispatch: O(N) on the linear scan. Acceptable only for tens of tasks.
Optimization Path
Replace the sorted list with a heapq. Replace the single thread with a worker pool of n_workers threads. Add a Condition variable so workers block when the queue is empty. Add a delayed-execution facility: instead of time.sleep(backoff) in the worker, push the task back with ready_at = now + backoff and key the heap on (ready_at, priority).
Final Expected Approach
Single heap (ready_at, priority, attempt, task_id, callable). A condition variable not_empty wakes workers when something becomes available. Workers loop: peek heap → if ready_at > now, wait until ready_at (or until notified). Pop, run, on success: done. On failure: increment attempt, if under max_attempts, push back with ready_at = now + backoff(attempt); else push to DLQ list. shutdown sets a flag, broadcasts the condition, joins all workers with a deadline.
Data Structures Used
| Structure | Purpose |
|---|---|
heapq of tuples | priority + delayed-readiness |
threading.Condition | wait/notify for empty queue and ready-time |
list (DLQ) | failed tasks |
dict[task_id, attempt_count] | retry tracking |
Correctness Argument
Priority ordering: heap orders by (ready_at, priority); workers always pop the smallest. When ready_at <= now, this is the highest-priority ready task. Ties on ready_at go to the smaller priority — correct.
Retry semantics: failure → push back with attempt+1, ready_at = now + 2^attempt * base + jitter. After max_attempts attempts, push to DLQ. The task is never lost: it is either running, in the heap, or in the DLQ — invariant maintained by every transition.
Shutdown: setting _stopping = True and broadcasting wakes every blocked worker. Each worker checks _stopping after the wait and exits if true. The join(timeout) per worker bounds total shutdown time.
Complexity
submit: O(log N)- Worker dispatch: O(log N) per task
- Memory: O(pending + dlq)
Implementation Requirements
import heapq, threading, time, itertools, random, traceback
from dataclasses import dataclass, field
from typing import Callable, Any
@dataclass
class FailedTask:
task_id: int
callable_repr: str
attempts: int
last_error: str
@dataclass(order=True)
class _Heap_Entry:
ready_at: float
priority: int
seq: int
task_id: int = field(compare=False)
fn: Callable = field(compare=False)
attempt: int = field(compare=False)
max_attempts: int = field(compare=False)
class TaskScheduler:
def __init__(self, n_workers: int = 4, base_backoff: float = 0.1):
self._n_workers = n_workers
self._base = base_backoff
self._heap: list[_Heap_Entry] = []
self._dlq: list[FailedTask] = []
self._cond = threading.Condition()
self._stopping = False
self._workers: list[threading.Thread] = []
self._seq = itertools.count()
self._next_id = itertools.count(1)
def submit(self, fn: Callable, priority: int = 5, max_attempts: int = 3) -> int:
if max_attempts <= 0:
raise ValueError("max_attempts must be positive")
tid = next(self._next_id)
e = _Heap_Entry(time.monotonic(), priority, next(self._seq),
tid, fn, attempt=0, max_attempts=max_attempts)
with self._cond:
heapq.heappush(self._heap, e)
self._cond.notify()
return tid
def start(self) -> None:
for i in range(self._n_workers):
t = threading.Thread(target=self._run_worker, name=f"w{i}", daemon=True)
t.start()
self._workers.append(t)
def shutdown(self, timeout: float = 5.0) -> None:
with self._cond:
self._stopping = True
self._cond.notify_all()
deadline = time.monotonic() + timeout
for w in self._workers:
w.join(timeout=max(0.0, deadline - time.monotonic()))
def dead_letters(self) -> list[FailedTask]:
with self._cond:
return list(self._dlq)
def _run_worker(self) -> None:
while True:
with self._cond:
while not self._heap and not self._stopping:
self._cond.wait()
if self._stopping and not self._heap:
return
head = self._heap[0]
wait = head.ready_at - time.monotonic()
if wait > 0:
self._cond.wait(timeout=wait)
continue
e = heapq.heappop(self._heap)
try:
e.fn()
except Exception as ex:
e.attempt += 1
if e.attempt >= e.max_attempts:
with self._cond:
self._dlq.append(FailedTask(
e.task_id, repr(e.fn), e.attempt,
f"{type(ex).__name__}: {ex}"))
else:
backoff = self._base * (2 ** (e.attempt - 1))
backoff *= 0.5 + random.random() # jitter [0.5, 1.5]
e.ready_at = time.monotonic() + backoff
e.seq = next(self._seq)
with self._cond:
heapq.heappush(self._heap, e)
self._cond.notify()
Tests
import unittest, time, threading
class TestScheduler(unittest.TestCase):
def test_priority_order(self):
order = []
sched = TaskScheduler(n_workers=1)
sched.start()
for p in [3, 1, 2]:
sched.submit((lambda x=p: order.append(x)), priority=p)
time.sleep(0.5)
sched.shutdown(timeout=2.0)
self.assertEqual(order, [1, 2, 3])
def test_retry_then_dlq(self):
attempts = []
def always_fail():
attempts.append(1)
raise RuntimeError("boom")
sched = TaskScheduler(n_workers=1, base_backoff=0.01)
sched.start()
sched.submit(always_fail, priority=0, max_attempts=3)
time.sleep(1.0)
sched.shutdown(timeout=2.0)
self.assertEqual(len(attempts), 3)
self.assertEqual(len(sched.dead_letters()), 1)
def test_concurrent_submit(self):
results = []
sched = TaskScheduler(n_workers=4)
sched.start()
def push(i):
for j in range(50):
sched.submit((lambda x=(i, j): results.append(x)), priority=0)
threads = [threading.Thread(target=push, args=(i,)) for i in range(4)]
for t in threads: t.start()
for t in threads: t.join()
time.sleep(0.5)
sched.shutdown(timeout=2.0)
self.assertEqual(len(results), 200)
Follow-up Questions
(2) Persist state across restarts? Tasks live in memory and are lost on restart. To persist, choose: (a) write each submit to a WAL; on boot, replay; on completion, append a “done” marker. (b) Snapshot the heap periodically and write a delta log. The DLQ should be persisted regardless — losing failed tasks is the worst outcome because nobody knows why a job didn’t run.
(8) Partial failure? The interesting case: a worker pops a task and crashes mid-execution. The task is now lost (it’s not in the heap and it didn’t complete). Solution: at-least-once via visibility timeout — the heap pops the task to an “in-flight” map with a TTL; if the worker doesn’t ack before TTL, the task returns to the heap. Idempotency keys make this safe. This is the SQS / Cloud Tasks model.
(9) Eviction / cleanup? The DLQ grows unbounded. Either: cap its size and drop oldest, retain a sliding-window of the last N failures, or persist to durable storage and prune from memory after a TTL. Always emit a per-task DLQ event so downstream alerting can fire.
(11) Configuration knobs? n_workers, base_backoff, default max_attempts. Per-task: priority, max_attempts, optionally timeout. Knobs not to expose: jitter strategy (use decorrelated jitter), heap implementation.
(12) Shutdown / draining? Two modes: graceful (stop accepting; wait for in-flight; return) and forceful (stop accepting; abandon in-flight; return immediately). Always offer both. Default to graceful with a deadline.
(13) Poison pill? A task that always crashes the worker (segfault, unhandled OS exception, infinite loop). Run tasks in subprocess isolation (or with a cooperative timeout). Blacklist by hash of (callable, args) after N consecutive crashes.
Product Extension
This is the heart of Celery, Sidekiq, RQ, AWS SQS + Lambda, GCP Cloud Tasks, and Temporal. Real systems add: visibility timeouts (the in-flight TTL), distributed coordination (multiple workers across hosts), durable storage (RDBMS or Redis with persistence), scheduling (cron-like time-based triggers), and workflow orchestration (Temporal). The core is what you wrote here.
Language/Runtime Follow-ups
- Python: GIL means worker threads don’t parallelize CPU work. For CPU-bound tasks, use a
ProcessPoolExecutorinstead. The implementation above is fine for I/O-bound tasks. - Java:
ScheduledThreadPoolExecutoris the textbook fit — submit with a delay, retries via re-submission.RetryTemplate(Spring Retry) for the policy.DeadLetterPublishingRecoverer(Kafka). - Go: a single channel of tasks plus N goroutines; for delayed retry, use
time.AfterFuncto push back to the channel. Or usegolang.org/x/sync/errgroupfor the worker pool. - C++: a
std::priority_queueplus condition variable. Tasks asstd::function<void()>. - JS/TS: not concurrent (single event loop), but BullMQ (Redis-backed) is the de-facto Node task queue.
Common Bugs
- Workers spinning when the heap head is in the future — wait
ready_at - nowexactly, not poll-loop. - Notifying only one worker on
submit(notify) butnotify_allon shutdown — fine, but check that the heap-shrink case (the popper seeshead.ready_at > nowand sleeps) doesn’t miss a wakeup when a higher-priority task is pushed during the sleep. - Forgetting to update
e.seqon re-push — the heap entry’s identity matters for tie-breaking, but Python’s heapq compares the full tuple in order, so missingsequpdates can cause the same(ready_at, priority, seq)to compare equal and the comparison to fall through to the un-comparableCallable. Always bumpseq. - Catching
Exceptionbut lettingBaseException(e.g.,KeyboardInterrupt) escape — workers die silently. CatchBaseExceptionwith care, or at minimum catchExceptionand log unexpected escapes. - DLQ growing forever — see follow-up #9.
Debugging Strategy
Add a worker trace: every transition (pop, run-start, run-end, retry, dlq) gets a log line with (worker_id, task_id, ts). Replay the log to see the timeline. For “task didn’t run” bugs, walk the heap state at submit time and check that notify was called. For shutdown deadlocks, take a thread dump (Python: faulthandler.dump_traceback_later()) — usually a worker is blocked on wait because notify_all was missed.
Mastery Criteria
- Implemented in <40 minutes; <30 on second attempt.
- All three tests pass.
- Articulated visibility-timeout / at-least-once vs lost-on-crash tradeoff in <90 seconds.
- Answered follow-ups #2, #8, #9, #12, #13 crisply.
- Added at-least-once semantics in <15 minutes when prompted.
- Stated why decorrelated jitter beats fixed jitter in retry backoff.
Lab 05 — Thread Pool
Goal
Implement a bounded ThreadPool with a fixed number of worker threads, a bounded work queue, configurable rejection policy, and graceful shutdown. After this lab you should be able to write a clean ThreadPoolExecutor clone in under 25 minutes and answer the standard concurrency follow-ups.
Background Concepts
A thread pool decouples task submission from task execution by introducing a queue of work items processed by N worker threads. The four design decisions are:
- Pool sizing: fixed-size, dynamic (grow/shrink), or bounded with min/max?
- Queue policy: bounded (block / reject / drop) or unbounded (memory risk)?
- Rejection policy when the queue is full: throw, drop newest, drop oldest, or run-on-caller’s-thread?
- Shutdown semantics: stop accepting and finish queue (
shutdown), or stop accepting and abandon queue (shutdown_now)?
The textbook implementation (and the one Java’s ThreadPoolExecutor uses) is fixed-size pool + bounded blocking queue + caller-runs rejection + graceful shutdown. This is the answer the interviewer wants by default.
Interview Context
Thread pool is a classic concurrency interview question. It tests whether you understand condition variables / blocking queues, can reason about producer-consumer with backpressure, and can structure shutdown so that submit after shutdown is rejected and in-flight tasks complete cleanly. Java candidates are expected to know that ThreadPoolExecutor’s seven-parameter constructor encodes most of these decisions.
Problem Statement
Implement ThreadPool(n_workers, queue_capacity, on_reject):
submit(fn) -> Future— enqueue. If queue full and pool not shut down, applyon_reject.shutdown(wait=True, timeout=None)— stop accepting; ifwait, drain the queue and join workers.shutdown_now() -> list[Callable]— stop accepting; abandon queued tasks and return them.
A Future exposes .result(timeout) to retrieve the task’s return value or raise its exception.
Constraints
- 1 ≤
n_workers≤ 1000 - 0 ≤
queue_capacity≤ 10^4 (0 = SynchronousQueue: hand off directly) - Submission rate up to 10^5 / s
Clarifying Questions
- Is the queue bounded? (Yes by default; “unbounded queue” is a known antipattern that masks production bugs.)
- What happens on full queue? (Reject by default; offer caller-runs as alternative.)
- Should workers be daemons? (Depends on language; in Python yes for graceful interpreter shutdown.)
- Returns a Future? (Yes — async result is the standard contract.)
- Re-entrancy: can a task
submitmore tasks? (Yes — must not deadlock on a full queue from inside a worker.)
Examples
pool = ThreadPool(n_workers=2, queue_capacity=5)
fut = pool.submit(lambda: 1 + 1)
fut.result() -> 2
pool.shutdown(wait=True)
pool.submit(lambda: 1) -> raises RuntimeError (pool shut down)
Initial Brute Force
for fn in tasks: threading.Thread(target=fn).start(). No bound, no reuse, no result tracking. Each task pays full thread-creation cost (~1ms on Linux), and the OS can run out of threads at 10^4+.
Brute Force Complexity
Per task: O(thread creation) ≈ 1 ms in Python. Total: O(N · 1ms). At N=10^5, this is 100 seconds — far too slow. Memory: O(N) thread stacks ≈ 8 MB each.
Optimization Path
Pool the threads. Workers spin on a blocking queue. submit enqueues; the queue blocks when full (or rejects). Per-task overhead drops to microseconds (a queue push and pop). Memory is O(n_workers · stack_size + queue_capacity).
Final Expected Approach
A Queue(maxsize=queue_capacity) (Python’s queue.Queue is thread-safe and supports timeouts). N worker threads loop on q.get(), run the task, set its Future, repeat. A sentinel None posted N times signals shutdown. submit checks the shut-down flag, then either q.put_nowait (raise on full) or q.put (block on full); on full and not blocking, invoke on_reject.
Data Structures Used
| Structure | Purpose |
|---|---|
queue.Queue(maxsize=…) | producer-consumer with bounded blocking |
Future (custom or concurrent.futures.Future) | result + exception delivery |
Sentinel None posted N times | shutdown signal |
_shutdown: bool flag | reject post-shutdown submissions |
Correctness Argument
Liveness: when a task is enqueued and at least one worker is idle, that worker will dequeue and run it. Provided by queue.Queue’s internal Condition (notify on put, wait on get).
Safety / no lost tasks: every put either succeeds (task will be dequeued by some worker) or is explicitly rejected. The shutdown protocol enforces that no put succeeds after _shutdown=True. When shutdown(wait=True) returns, the queue is empty and all workers have exited (proven by the sentinel pattern: each worker sees exactly one None and exits, so all N workers terminate).
Future correctness: the worker’s try/except block sets either set_result(value) or set_exception(ex). Future.result() blocks on a Condition until one of the two is set. Linearizable.
Complexity
submit: O(1) amortized- Worker step: O(1) plus task duration
- Memory: O(n_workers + queue_capacity)
Implementation Requirements
import threading, queue
from typing import Callable, Any, Optional
from concurrent.futures import Future
_SENTINEL = object()
class RejectedExecutionError(RuntimeError):
pass
class ThreadPool:
def __init__(self, n_workers: int, queue_capacity: int = 1024,
on_reject: Optional[Callable] = None):
if n_workers <= 0:
raise ValueError("n_workers must be positive")
self._q: queue.Queue = queue.Queue(maxsize=queue_capacity)
self._workers: list[threading.Thread] = []
self._shutdown = False
self._lock = threading.Lock()
self._on_reject = on_reject or self._default_reject
for i in range(n_workers):
t = threading.Thread(target=self._run, name=f"pool-w{i}", daemon=True)
t.start()
self._workers.append(t)
@staticmethod
def _default_reject(fn, *args, **kwargs):
raise RejectedExecutionError("queue full")
def submit(self, fn: Callable, *args, **kwargs) -> Future:
with self._lock:
if self._shutdown:
raise RejectedExecutionError("pool shut down")
fut: Future = Future()
try:
self._q.put_nowait((fn, args, kwargs, fut))
except queue.Full:
try:
self._on_reject(fn, *args, **kwargs)
except Exception as ex:
fut.set_exception(ex)
else:
fut.set_exception(RejectedExecutionError("rejected"))
return fut
def shutdown(self, wait: bool = True, timeout: Optional[float] = None) -> None:
with self._lock:
if self._shutdown:
return
self._shutdown = True
for _ in self._workers:
self._q.put(_SENTINEL)
if wait:
for w in self._workers:
w.join(timeout=timeout)
def shutdown_now(self) -> list[Callable]:
"""Stop accepting; abandon queued tasks; return abandoned callables."""
with self._lock:
self._shutdown = True
abandoned: list[Callable] = []
try:
while True:
item = self._q.get_nowait()
if item is _SENTINEL: continue
fn, _, _, fut = item
fut.set_exception(RejectedExecutionError("shutdown_now"))
abandoned.append(fn)
except queue.Empty:
pass
for _ in self._workers:
self._q.put(_SENTINEL)
return abandoned
def _run(self) -> None:
while True:
item = self._q.get()
if item is _SENTINEL:
return
fn, args, kwargs, fut = item
if not fut.set_running_or_notify_cancel():
continue
try:
result = fn(*args, **kwargs)
except BaseException as ex:
fut.set_exception(ex)
else:
fut.set_result(result)
# A useful policy: caller-runs (executes inline if queue is full)
def caller_runs(fn, *args, **kwargs):
fn(*args, **kwargs)
Tests
import unittest, time, threading
class TestPool(unittest.TestCase):
def test_basic(self):
pool = ThreadPool(n_workers=2, queue_capacity=10)
futs = [pool.submit(lambda x=i: x * 2) for i in range(10)]
results = [f.result(timeout=2.0) for f in futs]
self.assertEqual(sorted(results), [0, 2, 4, 6, 8, 10, 12, 14, 16, 18])
pool.shutdown()
def test_exception_propagated(self):
pool = ThreadPool(n_workers=1, queue_capacity=10)
f = pool.submit(lambda: 1 / 0)
with self.assertRaises(ZeroDivisionError):
f.result(timeout=2.0)
pool.shutdown()
def test_rejection_when_queue_full(self):
block = threading.Event()
pool = ThreadPool(n_workers=1, queue_capacity=1)
pool.submit(lambda: block.wait()) # occupies the worker
pool.submit(lambda: None) # fills the queue
f = pool.submit(lambda: None) # rejected
with self.assertRaises(RejectedExecutionError):
f.result(timeout=1.0)
block.set()
pool.shutdown()
def test_shutdown_rejects_new(self):
pool = ThreadPool(n_workers=1)
pool.shutdown()
with self.assertRaises(RejectedExecutionError):
pool.submit(lambda: None)
def test_concurrent_submit(self):
pool = ThreadPool(n_workers=8, queue_capacity=200)
results = []
lock = threading.Lock()
def task(x):
with lock: results.append(x)
futs = [pool.submit(task, i) for i in range(200)]
for f in futs: f.result(timeout=2.0)
pool.shutdown()
self.assertEqual(sorted(results), list(range(200)))
Follow-up Questions
(1) Thread-safe? Already designed for concurrency; the Queue handles producer-consumer atomicity. The _shutdown flag is read under a lock to avoid races between submit and shutdown.
(7) Backpressure? The bounded queue is the backpressure. Three policies: (a) put blocks the producer (default queue.put) — pushes backpressure to the caller. (b) reject (raise) — caller decides. (c) caller-runs — caller does the work; throttles naturally. (d) drop oldest — for non-critical telemetry-style work. Pick one explicitly per pool.
(12) Shutdown / draining? shutdown(wait=True) drains the queue (graceful). shutdown_now() abandons the queue and returns abandoned tasks (forceful). The graceful path is the production default; expose shutdown_now for SIGTERM after a deadline.
(8) Partial failure? A worker thread that crashes on an uncaught exception leaves the pool with N-1 workers permanently. Solutions: (a) catch BaseException around the task body (shown), (b) supervise — periodically check live worker count and respawn dead workers. The simplest production design catches and logs, never lets the worker die.
(13) Poison pill? A task that runs forever or consumes all memory blocks one worker permanently. Mitigations: per-task timeout (cooperative with a watchdog thread), memory accounting (rare in Python), or run untrusted tasks in subprocesses. Stating this awareness is the bar.
(11) Configuration knobs? n_workers (often CPU_count or 2 * CPU_count for I/O-bound); queue_capacity (rule of thumb: enough to absorb a ~1 second burst); on_reject policy. Knobs not to expose: queue type, worker thread name (auto-generate).
Product Extension
java.util.concurrent.ThreadPoolExecutor is the textbook reference. Python’s concurrent.futures.ThreadPoolExecutor is the canonical stdlib equivalent, with an unbounded queue by default — your implementation is more correct than the stdlib because you bounded the queue. AWS Lambda’s worker runtime, NGINX worker processes, and most application servers use variants of this pattern.
Language/Runtime Follow-ups
- Python: GIL serializes CPU work; ThreadPool is for I/O-bound tasks. Use
ProcessPoolExecutorfor CPU-bound.concurrent.futures.ThreadPoolExecutorships with Python — but its queue is unbounded by default, which is a footgun. - Java:
new ThreadPoolExecutor(core, max, keepAlive, unit, queue, factory, rejectedHandler). Memorize the seven parameters and the four built-inRejectedExecutionHandlerpolicies (Abort, CallerRuns, Discard, DiscardOldest). - Go: idiomatic Go does not use thread pools — goroutines are cheap. The pattern is a worker-pool of N goroutines reading from a channel of work items. The bounded channel is the bounded queue.
- C++:
std::threadper worker;std::condition_variable+std::queuefor the work queue. Boost.Asio’s thread pool is production-ready. - JS/TS: single event loop; use
worker_threadsfor CPU work. Libraries:piscina(worker pool for Node).
Common Bugs
- Unbounded queue:
Queue()withoutmaxsize— masks production overload as memory growth. - Daemons vs non-daemons: in Python, daemon workers die abruptly on interpreter exit, abandoning in-flight tasks. Non-daemons require explicit
shutdownor the program hangs. Pick deliberately. - Catching
Exceptionbut notBaseExceptionletsKeyboardInterruptkill workers silently. CatchBaseException, restore worker. submitaftershutdownrace: check the flag under the lock and put under the same critical section, or accept that a few enqueues may sneak in between check and put (and handle them in the worker by checking the flag before running).- Forgetting to
set_running_or_notify_cancel()on the Future — cancelled futures still get run.
Debugging Strategy
For deadlocks, take a thread dump: in Python, import faulthandler; faulthandler.dump_traceback_later(5) then trigger the hang. For lost tasks, instrument every put/get/set_result/set_exception with a sequence number and replay. For worker death, log every worker exit with its reason.
Mastery Criteria
-
Implemented
ThreadPoolin <30 minutes; tests pass first run. - Articulated bounded-queue + rejection-policy design without prompting.
- Listed the four standard rejection policies (abort / caller-runs / discard / discard-oldest).
- Answered follow-ups #7, #8, #12, #13 in <90 seconds each.
- Stated when to use ThreadPool vs ProcessPool in Python in <30 seconds.
- Refactored to add per-task timeout in <10 minutes when prompted.
Lab 06 — Durable Job Queue
Goal
Implement a job queue with at-least-once delivery semantics, idempotency keys, and visibility timeouts. After this lab you should be able to articulate why exactly-once delivery is impractical, design at-least-once with idempotency on the consumer, and implement an in-memory queue that simulates SQS-style semantics in under 35 minutes.
Background Concepts
A durable job queue accepts jobs from producers and delivers them to consumers, surviving consumer crashes without losing work. The three classical delivery semantics:
- At-most-once: deliver, forget. Fast but loses jobs on crash. Acceptable for fire-and-forget telemetry.
- At-least-once: deliver, retry until acknowledged. Jobs may be delivered multiple times. Requires consumers to be idempotent.
- Exactly-once: impossible in a distributed system without two-phase commit between queue and consumer. The “exactly-once” branding in real systems (Kafka, Pulsar) means “exactly-once processing semantics given idempotent consumers” — which is at-least-once + idempotency.
The standard primitive that enables at-least-once is the visibility timeout: when a consumer dequeues a job, the queue marks it “in-flight” with a TTL. If the consumer acks before TTL, the job is deleted. If TTL expires (consumer crashed), the job becomes visible again and is redelivered. The consumer must be idempotent because the same job may be processed twice if ack was lost in transit.
Interview Context
This is the central problem in any interview at AWS (SQS), GCP (Pub/Sub), Confluent (Kafka), or any infrastructure team. The interviewer wants to hear: “exactly-once is impractical because of the two-generals problem; at-least-once with idempotency keys is the production answer; visibility timeout is how we implement it.” Then they want to see you build a small version that demonstrates understanding.
Problem Statement
Design JobQueue:
enqueue(payload, idempotency_key=None) -> job_id— push. Ifidempotency_keyis non-Noneand matches a recent job, deduplicate.dequeue(visibility_timeout=30.0) -> Job | None— pop a visible job; mark in-flight with TTL.ack(job_id)— confirm successful processing; permanently delete.nack(job_id, requeue_delay=0)— release back; optionally with a delay.- Background scavenger: jobs whose visibility TTL has expired return to visible state.
Constraints
- Up to 10^5 in-flight jobs
- 10^4 enqueues / second
- Single-process in-memory; persistence is a follow-up
Clarifying Questions
- FIFO or best-effort ordering? (Best-effort is standard SQS; FIFO costs more.)
- Visibility timeout: per-call or queue default? (Per-call, with queue default.)
- Idempotency key TTL — how long do we dedupe? (Typically 5 minutes; configurable.)
- Max retries before DLQ? (Often a follow-up; default unlimited.)
- What happens on
nack? (Requeue, optionally with a delay; this is the natural retry path.)
Examples
q = JobQueue()
job_id = q.enqueue("send-email-123", idempotency_key="email-abc")
q.enqueue("send-email-123", idempotency_key="email-abc") # dedup; same job_id
job = q.dequeue(visibility_timeout=10)
# … process …
q.ack(job.job_id) # done
# Crash scenario:
job = q.dequeue(visibility_timeout=10)
# consumer crashes, never acks
# 11 seconds later:
job_again = q.dequeue() # same payload, same job_id, redelivery_count=2
Initial Brute Force
A list of jobs and a single mutex. dequeue pops the head, sets in-flight; ack removes; nack re-prepends. No visibility timeout. Loses jobs on crash.
Brute Force Complexity
O(1) per op for a deque, O(N) if implemented over a list with re-prepend. Fundamentally wrong for at-least-once because there’s no scavenger.
Optimization Path
Add: (a) visible: deque[Job] — jobs ready to be dequeued; (b) in_flight: dict[job_id, (Job, expires_at)] — taken but not acked; (c) idempotency: dict[key, job_id] with TTL for dedup; (d) a background scavenger that moves expired in-flight jobs back to visible. The scavenger can be lazy (check on each dequeue) instead of a dedicated thread.
Final Expected Approach
Use deque for visible, dict for in-flight with (job, expires_at), dict for idempotency cache. Each dequeue first sweeps in_flight for expired entries (move them to the front of visible). All operations under a single lock — the queue is fast, sub-millisecond critical sections.
Data Structures Used
| Structure | Purpose |
|---|---|
deque[Job] | visible jobs, FIFO best-effort |
dict[job_id, (Job, expires_at)] | in-flight tracking |
dict[idempotency_key, job_id] | dedup cache |
Lock | single-process atomicity |
Correctness Argument
No lost jobs (at-least-once): every job is in exactly one of three states: visible, in_flight, or acked (deleted). Transitions: enqueue → visible; dequeue → in_flight; ack → deleted; nack → visible; scavenger → visible (from in_flight on TTL expiry). No transition discards a job before ack. Therefore, until acked, every job remains in the system and will eventually be redelivered.
Idempotency dedup: if idempotency_key matches a job in either visible or in_flight (or recently acked, within the dedup window), enqueue returns the existing job_id without creating a new job. This makes producer retries safe.
At-least-once, not exactly-once: a consumer that successfully processes the job and crashes before sending ack will see the same job redelivered. The consumer must idempotent-key the work it does (e.g., the email service must dedupe by email-abc).
Complexity
enqueue: O(1)dequeue: O(K) where K is the number of expired in-flight entries swept (amortized O(1))ack/nack: O(1)
Implementation Requirements
import threading, time, itertools
from collections import deque
from dataclasses import dataclass
from typing import Optional, Any
@dataclass
class Job:
job_id: int
payload: Any
delivery_count: int
enqueued_at: float
class JobQueue:
def __init__(self, default_visibility_timeout: float = 30.0,
idempotency_ttl: float = 300.0):
self._visible: deque[Job] = deque()
self._in_flight: dict[int, tuple[Job, float]] = {} # id -> (job, expires_at)
self._idem: dict[str, tuple[int, float]] = {} # key -> (job_id, expires_at)
self._default_vt = default_visibility_timeout
self._idem_ttl = idempotency_ttl
self._lock = threading.Lock()
self._next_id = itertools.count(1)
def enqueue(self, payload: Any, idempotency_key: Optional[str] = None) -> int:
now = time.monotonic()
with self._lock:
self._sweep_idem(now)
if idempotency_key is not None and idempotency_key in self._idem:
existing_id, _ = self._idem[idempotency_key]
return existing_id
job_id = next(self._next_id)
job = Job(job_id, payload, delivery_count=0, enqueued_at=now)
self._visible.append(job)
if idempotency_key is not None:
self._idem[idempotency_key] = (job_id, now + self._idem_ttl)
return job_id
def dequeue(self, visibility_timeout: Optional[float] = None) -> Optional[Job]:
vt = visibility_timeout if visibility_timeout is not None else self._default_vt
now = time.monotonic()
with self._lock:
self._sweep_in_flight(now)
if not self._visible:
return None
job = self._visible.popleft()
job.delivery_count += 1
self._in_flight[job.job_id] = (job, now + vt)
return job
def ack(self, job_id: int) -> bool:
with self._lock:
return self._in_flight.pop(job_id, None) is not None
def nack(self, job_id: int, requeue_delay: float = 0.0) -> bool:
now = time.monotonic()
with self._lock:
entry = self._in_flight.pop(job_id, None)
if entry is None:
return False
job, _ = entry
if requeue_delay > 0:
# For simplicity, treat delay as a delayed visibility:
# park as in-flight with expires_at = now + delay.
self._in_flight[job_id] = (job, now + requeue_delay)
else:
self._visible.appendleft(job) # head, so it's seen first
return True
def stats(self) -> dict:
with self._lock:
return {
"visible": len(self._visible),
"in_flight": len(self._in_flight),
"dedup_keys": len(self._idem),
}
def _sweep_in_flight(self, now: float) -> None:
expired = [(jid, j) for jid, (j, t) in self._in_flight.items() if t <= now]
for jid, j in expired:
del self._in_flight[jid]
self._visible.appendleft(j) # redelivery: front-load
def _sweep_idem(self, now: float) -> None:
# Lazy: only keep recent entries. O(N) but called infrequently.
if len(self._idem) > 4096:
self._idem = {k: v for k, v in self._idem.items() if v[1] > now}
Tests
import unittest, time
class TestJobQueue(unittest.TestCase):
def test_basic_enqueue_ack(self):
q = JobQueue()
jid = q.enqueue("hello")
job = q.dequeue()
self.assertEqual(job.job_id, jid)
self.assertTrue(q.ack(jid))
self.assertIsNone(q.dequeue())
def test_idempotency_dedup(self):
q = JobQueue()
a = q.enqueue("x", idempotency_key="k1")
b = q.enqueue("x", idempotency_key="k1")
self.assertEqual(a, b)
self.assertEqual(q.stats()["visible"], 1)
def test_visibility_timeout_redelivery(self):
q = JobQueue(default_visibility_timeout=0.1)
jid = q.enqueue("x")
job1 = q.dequeue()
time.sleep(0.15)
job2 = q.dequeue()
self.assertEqual(job1.job_id, job2.job_id)
self.assertEqual(job2.delivery_count, 2)
q.ack(job2.job_id)
def test_nack_requeue(self):
q = JobQueue()
jid = q.enqueue("x")
job = q.dequeue()
q.nack(jid)
job2 = q.dequeue()
self.assertEqual(job.job_id, job2.job_id)
self.assertEqual(job2.delivery_count, 2)
Follow-up Questions
(2) Persist state across restarts? Three layers: (a) WAL: every enqueue, ack, nack is appended to a log; on boot, replay. (b) Snapshot: periodic full state dump. (c) Combined: snapshot every N seconds, WAL between snapshots; recovery = latest snapshot + log replay since. SQS uses a replicated multi-AZ store; for an interview, WAL is the right answer.
(8) Partial failure? That’s the entire point of visibility timeout. Consumer crashes mid-processing → TTL expires → job redelivered. The consumer is responsible for idempotency. The queue is responsible for delivering at-least-once.
(9) Eviction / cleanup? Stale in-flight entries (consumer crashed and never acked) are swept on every dequeue. The idempotency cache TTL bounds dedup memory. DLQ (not implemented above) would catch jobs after N redeliveries — a follow-up to add.
(10) Consistency model? Linearizable per-job in a single process; redelivery breaks “exactly-once” but preserves “every job is processed at least once”. Replicated: consensus (Raft) for the metadata, leader-based delivery, replicated log for durability.
(11) Configuration knobs? default_visibility_timeout, idempotency_ttl, max_redeliveries (DLQ trigger), dlq_handler. Knobs not to expose: internal sweep cadence.
(12) Shutdown? On shutdown, refuse new enqueues, sweep in-flight back to visible (so consumers don’t redeliver after restart with stale TTLs), persist state, exit. The graceful invariant: no in-flight at shutdown time.
Product Extension
This is a simplified SQS / Cloud Pub/Sub / Azure Service Bus. Real systems add: replication for durability (across hosts/AZs), partitioning for throughput (multiple shards), DLQ as a separate queue with its own retention, FIFO ordering as an opt-in higher-cost mode, and ordering keys (per-key FIFO with cross-key parallelism — Kafka’s model). Kafka explicitly doesn’t have visibility timeouts; it uses offset-based delivery with consumer-managed checkpoints, which is a different design point.
Language/Runtime Follow-ups
- Python: this implementation. For high-throughput, sharded queues with per-shard locks scale better than the single global lock.
- Java:
ArrayBlockingQueueis too simple (no visibility timeout). The right reference isjava.util.concurrent.DelayQueuefor visibility, plus aConcurrentHashMapfor in-flight tracking. Production: ActiveMQ, RabbitMQ. - Go: a channel-based implementation works for visible queue; in-flight is a
sync.Map; sweeper is a goroutine. NATS JetStream is the production-grade Go choice. - C++: roll-your-own with
std::deque+std::unordered_map+std::mutex. Boost has thread-safe queue templates. - JS/TS: BullMQ (Redis-backed) is the de-facto Node choice; visibility timeout is implemented via Redis sorted sets.
Common Bugs
- Idempotency cache that never expires — memory leak.
- Sweeping in-flight in a separate thread without coordinating the lock — races with
dequeue. Lazy sweep ondequeue(as shown) avoids the extra thread. - Forgetting to increment
delivery_counton redelivery — alerting can’t detect poison-pill jobs (jobs that always crash consumers). nackwith delay implemented by sleeping — blocks the consumer that callednackinstead of just delaying re-visibility.- Treating idempotency dedup as global / forever — if the dedup window is too long, retries after intentional re-submission are silently dropped.
Debugging Strategy
Print stats() periodically to track visible / in-flight counts. A growing in-flight count without acks → consumers are crashing or hanging. Stuck visible count → no consumers are running. For “duplicate processing” complaints, capture the redelivery-count distribution; high tail = consumers crashing or visibility timeout too short.
Mastery Criteria
- Implemented in <40 minutes from blank screen.
- Stated “exactly-once is impractical; at-least-once + idempotency is the answer” without prompting.
- All four tests pass first run.
- Articulated visibility timeout, idempotency keys, and DLQ design in <90 seconds.
- Answered follow-ups #2, #8, #10, #11, #12.
- Compared SQS-style (visibility timeout) vs Kafka-style (offset-based) delivery in <60 seconds.
Lab 07 — Autocomplete
Goal
Implement an autocomplete service that returns the top-K suggestions for any prefix in sub-millisecond time. Use a trie augmented with per-node top-K caches, and support weighted suggestions (popularity-ranked). After this lab you should be able to design and implement the data structure in under 30 minutes and answer follow-ups about scale, freshness, and personalization.
Background Concepts
A trie (prefix tree) maps prefixes to a set of completions in O(prefix length) time. The naive design — at query time, walk to the prefix node and DFS to gather all descendants, then sort by weight — is correct but slow if the prefix matches many descendants. The production trick is to precompute and cache the top-K at each node during insert; query then becomes O(prefix-length + K).
The cache update is the subtle part. When add(word, weight) is called, walk the trie down the word’s path; at each node, merge the new word into the node’s top_k (a sorted list or small heap) and discard anything past K. The data flow is “bottom-up via the path you just walked, but for top-K at every level”.
Interview Context
Autocomplete is a top-10 question at search-heavy companies (Google search, Amazon product search, LinkedIn, Yelp). The bar is: trie + per-node top-K cache + ability to answer follow-ups about distributing the index, refreshing weights, and personalizing.
Problem Statement
Design Autocomplete(K):
add(word, weight)— insert with weight; subsequent adds of the same word update its weight (additive or replace, your choice — pick one).suggest(prefix) -> list[str]— return top-K words by weight starting withprefix. Sub-millisecond average.
Constraints
- Up to 10^6 words
- 10^5 queries / second
- Average word length 10–30
- K ≤ 10
Clarifying Questions
- Weight semantics: replace or additive? (Pick one; “additive” matches “popularity”.)
- Case sensitivity? (Default case-insensitive; lowercase on insert.)
- Tie-breaking on equal weight? (Lexicographic.)
- Real-time updates required, or build-once? (Both supported; weights mutable.)
- K provided per-query or fixed? (Fixed at construction simplifies caching; per-query is a follow-up.)
Examples
ac = Autocomplete(K=3)
ac.add("apple", 5); ac.add("app", 10); ac.add("apply", 3); ac.add("apricot", 1)
ac.suggest("ap") -> ["app", "apple", "apply"]
ac.suggest("app") -> ["app", "apple", "apply"]
ac.suggest("apr") -> ["apricot"]
ac.suggest("z") -> []
Initial Brute Force
Store words in a dict[word, weight]. On suggest, scan all words, filter by startswith(prefix), sort by weight, return top-K. O(N · prefix-length) per query.
Brute Force Complexity
Per suggest: O(N · L) where N = number of words, L = average length. At N=10^6 and 10^5 QPS, this is 10^11 operations / second — far too slow.
Optimization Path
A trie reduces the descend-to-prefix step to O(L). The remaining work — gathering the top-K descendants — is what the per-node top_k cache eliminates. Insert becomes O(L · K log K) (we update the cache at L nodes, each O(K log K)); query becomes O(L + K).
Final Expected Approach
Trie node has children: dict[char, Node] and top_k: list[(weight, word)]. add walks down the path; at each node, runs a small merge to maintain top-K. suggest walks to the prefix node and returns its top_k.
Data Structures Used
| Structure | Purpose |
|---|---|
| Trie nodes | prefix indexing |
Per-node top_k: list[(weight, word)] | precomputed answers |
dict[word, weight] | dedup + weight tracking |
Correctness Argument
Invariant: at any node N, top_k(N) is the top-K (by weight, ties broken lexicographically) of { (weight, word) : word starts with prefix(N) }. After every add(word, weight), we visit exactly the nodes on word’s path. At each visited node, we either (a) update the (word, weight) entry if word is already in top_k, or (b) insert and trim to K. No node off the path’s top_k set could change because word doesn’t extend any non-path prefix.
Edge case: weight updates that decrease a word’s standing — if word was in the top-K and its new weight kicks it out, we need to recompute the node’s top-K from a wider candidate set. The clean approach: store at each node a count_per_word dict and full candidate set restricted to top-K-wide buffer (e.g., top-10K when K=10) — heavy but correct. The simpler approach: on weight decrease, do a DFS to rebuild top_k. Document the choice.
Complexity
add: O(L · K log K)suggest: O(L + K)- Space: O(N · L · K) worst case; in practice much less because most nodes don’t have K distinct descendants
Implementation Requirements
import heapq
from typing import Optional
class _Node:
__slots__ = ("children", "top_k")
def __init__(self):
self.children: dict[str, _Node] = {}
self.top_k: list[tuple[int, str]] = [] # min-heap of (weight, word) ... ish
class Autocomplete:
def __init__(self, k: int = 5):
self._root = _Node()
self._weights: dict[str, int] = {}
self._k = k
def add(self, word: str, weight_delta: int = 1) -> None:
word = word.lower()
new_weight = self._weights.get(word, 0) + weight_delta
self._weights[word] = new_weight
node = self._root
nodes_on_path: list[_Node] = [node]
for ch in word:
node = node.children.setdefault(ch, _Node())
nodes_on_path.append(node)
for n in nodes_on_path:
self._upsert_top_k(n, word, new_weight)
def suggest(self, prefix: str) -> list[str]:
prefix = prefix.lower()
node = self._root
for ch in prefix:
node = node.children.get(ch)
if node is None:
return []
# top_k stored as (weight_neg, word) so sorted asc gives top weights desc
ranked = sorted(node.top_k, key=lambda p: (-p[0], p[1]))
return [w for _, w in ranked[:self._k]]
def _upsert_top_k(self, node: _Node, word: str, weight: int) -> None:
for i, (w, ww) in enumerate(node.top_k):
if ww == word:
node.top_k[i] = (weight, word)
node.top_k.sort(key=lambda p: (-p[0], p[1]))
return
node.top_k.append((weight, word))
node.top_k.sort(key=lambda p: (-p[0], p[1]))
if len(node.top_k) > self._k:
del node.top_k[self._k:]
Tests
import unittest
class TestAutocomplete(unittest.TestCase):
def test_basic(self):
ac = Autocomplete(k=3)
for w, n in [("app", 10), ("apple", 5), ("apply", 3), ("apricot", 1)]:
ac.add(w, n)
self.assertEqual(ac.suggest("ap"), ["app", "apple", "apply"])
self.assertEqual(ac.suggest("app"), ["app", "apple", "apply"])
self.assertEqual(ac.suggest("apr"), ["apricot"])
self.assertEqual(ac.suggest("z"), [])
def test_weight_update(self):
ac = Autocomplete(k=3)
ac.add("a", 1); ac.add("b", 2); ac.add("c", 3)
ac.add("a", 10) # a now weight 11
# Suggest off the empty prefix
self.assertEqual(ac.suggest(""), ["a", "c", "b"])
def test_top_k_truncation(self):
ac = Autocomplete(k=2)
for c, w in [("a", 1), ("b", 2), ("c", 3), ("d", 4)]:
ac.add(c, w)
self.assertEqual(ac.suggest(""), ["d", "c"])
def test_lex_tie(self):
ac = Autocomplete(k=3)
ac.add("banana", 5); ac.add("apple", 5); ac.add("cherry", 5)
self.assertEqual(ac.suggest(""), ["apple", "banana", "cherry"])
Follow-up Questions
(3) Scale to N nodes? Shard by first character (A–Z) → 26 shards, each with its own trie. For more even distribution, shard by hash(prefix[:2]). Each suggestion query routes to one shard. For the empty-prefix query, broadcast and merge — the price you pay for sharding by prefix.
(4) Observe / monitor? Per-prefix-length latency histogram (short prefixes = many candidates, slow); cache hit-rate (proportion of queries hitting precomputed top-K vs needing DFS); query volume per prefix (top hot prefixes). Alert on p99 latency.
(9) Eviction / cleanup? Words may go cold (a celebrity who stopped trending). Strategy: timestamped weight, decay on a schedule (multiply all weights by 0.95 daily), delete when below a threshold. Or use a separate “deletion” path: remove(word) walks the trie, removes from each node’s top_k, and rebuilds top_k from a wider candidate set if the removed word was in it.
(11) Configuration knobs? K, case_sensitivity, weight_decay_factor. Knobs not to expose: trie node layout, sort algorithm.
(8) Partial failure? A query that hits a partially-rebuilt index during a add could see stale top-K. Solutions: (a) atomic per-node update (under a per-node lock), (b) versioned snapshot (queries read a stable version while writes go to a shadow), (c) accept stale results for ~1 second. For autocomplete, eventual consistency is fine.
Product Extension
Google’s autocomplete is far more than a trie: it’s a personalized, context-aware, learning-ranked system. The trie + top-K is the index layer; on top sits a ranker that combines popularity, personalization (your history), context (location, time), and freshness (trending). Production systems also add typo tolerance (Levenshtein-edit-distance fuzzy match within edit-distance ≤ 2) — a much harder problem solved with FSTs or n-gram inverted indexes.
Language/Runtime Follow-ups
- Python: dict-based trie is the simplest; for memory, switch to arrays of 26/128 children once you optimize. The implementation above is fine for 10^6 words; beyond, consider a DAWG (DAG of suffix-shared subtrees).
- Java:
HashMap<Character, Node>per node, or arrays for ASCII. Apache Commons Collections hasTrieMap(PATRICIA trie). - Go:
map[rune]*Nodeper node. Excellent for this workload because of GC’s tolerance for many small allocations. - C++: same pattern. For best performance, use
std::array<Node*, 26>for ASCII. - JS/TS:
Map<string, Node>per node; no concurrency concerns in single-threaded Node.
Common Bugs
- Inserting and forgetting to update top-K at every node on the path (only updating the leaf). Subsequent prefix queries return empty.
- Sorting top-K by weight only, forgetting lex tie-break. Tests with equal weights become flaky.
- On weight decrease, leaving a stale entry in top-K. Solution: full rebuild of top-K on decrease.
- Case mismatch: insert lowercase, query as-is. Lowercase both.
- Memory: storing the full word in every node’s top-K — at 10^6 words and depth 30, this is 30M strings. Store an integer ID and look up the word in a side dict for memory savings.
Debugging Strategy
For “wrong suggestions” bugs: print node.top_k at the prefix node and verify it matches the expected top-K. For “missing word” bugs: walk down the trie from root, printing node.children at each step, confirm the path exists. For weight bugs: dump self._weights[word] and compare to expected. For performance, profile: most hot paths are dict.setdefault and the in-place sort.
Mastery Criteria
- Implemented trie + per-node top-K in <30 minutes.
- All four tests pass first run.
- Stated trie + top-K caching tradeoff (insert is K-times slower, query is L+K instead of L+all-descendants).
- Answered follow-ups #3 (sharding), #4 (observability), #9 (decay), #8 (eventual consistency) crisply.
- Compared trie vs DAWG vs FST for memory.
- Articulated typo-tolerance design (BK-tree / fuzzy n-grams) at a high level.
Lab 08 — Log Parser
Goal
Implement a streaming log parser that reads log lines (potentially gigabytes), extracts structured fields via regex, aggregates per-field counts, and emits structured output — all under bounded memory. After this lab you should be able to write a clean streaming text-processing class with bounded memory in under 25 minutes.
Background Concepts
Log parsing has two patterns: batch (load file, parse all, output) and streaming (read one line at a time, emit incremental output). The bar at senior interviews is the streaming variant because real production logs are too large to load — multi-gigabyte files where batch processing would OOM.
The two streaming primitives are:
- Line-by-line iteration with a generator (
for line in file:in Python). Memory is O(line size), not O(file size). - Bounded aggregation: when counting unique IPs over a 1 TB log, you cannot keep all distinct IPs in a
dict. Bound the aggregation by either (a) sketch (HyperLogLog for distinct counts, count-min for top-K), or (b) “top-K with eviction” using a min-heap of size K.
The regex itself is mundane. The interview signal is the discipline of bounded memory and clean separation between parser, extractor, and aggregator.
Interview Context
Log parsing is a popular question at logging / observability companies (Datadog, Splunk, Honeycomb, Cribl, Elastic) and at any infrastructure company that processes high-volume telemetry. It tests streaming discipline, regex fluency, and bounded-memory awareness. It also exposes weak engineering: a candidate who writes lines = file.readlines() instantly fails the bounded-memory criterion.
Problem Statement
Design LogParser(pattern, top_k=10):
parse_stream(line_iter) -> Iterator[dict]— yield a dict per line with extracted named fields. Skip malformed lines (count them).aggregate(line_iter) -> dict— return per-field top-K aggregates (e.g., top 10 IPs, top 10 paths, top 10 status codes). Bounded memory.
The regex is provided at construction; the parser must use named capture groups.
Constraints
- Input file size: up to 100 GB
- Aggregator memory: ≤ 100 MB
- Target throughput: 50 MB/s on a single core
Clarifying Questions
- Is the log format known? (Yes — caller provides regex with named groups.)
- Malformed lines: skip, error, or quarantine? (Skip + count by default; quarantine optionally.)
- Aggregation: which fields, what kind (count, distinct, top-K)? (Caller specifies.)
- Time-series: are we computing per-time-window aggregates? (Optional; default is whole-stream.)
- Encoding: UTF-8? Binary? (UTF-8 default; binary is a follow-up.)
Examples
pattern = r'(?P<ip>\S+) - - \[(?P<ts>[^\]]+)\] "(?P<method>\S+) (?P<path>\S+) HTTP/[\d.]+" (?P<status>\d+) (?P<bytes>\d+)'
parser = LogParser(pattern, top_k=3)
for record in parser.parse_stream(open("access.log")):
print(record)
# {'ip': '1.2.3.4', 'ts': '20/May/2026:12:00:00', 'method': 'GET', 'path': '/x', 'status': '200', 'bytes': '1234'}
agg = parser.aggregate(open("access.log"))
# {'ip': [('1.2.3.4', 1500), ('5.6.7.8', 1200), ...],
# 'path': [('/api', 5000), ('/login', 3000), ...],
# 'status': [('200', 90000), ('404', 4000), ('500', 200)],
# 'malformed': 12,
# 'total': 100000}
Initial Brute Force
open(file).read() then re.finditer. Loads everything; OOMs on 100 GB.
Brute Force Complexity
Memory: O(file size). At 100 GB, instant OOM on a 32 GB machine.
Optimization Path
Stream line-by-line with for line in file:. For aggregation, replace the unbounded dict[ip, count] with: (a) keep all counts during the stream because the unique key cardinality is what matters (often 100k unique IPs is fine), or (b) for very high cardinality, use HyperLogLog (HLL) for distinct counts and count-min sketch + min-heap for top-K. For most workloads at moderate cardinality, the dict is fine; at extreme cardinality, sketches are required.
Final Expected Approach
Compile the regex once. Stream lines via the iterator. For each line, match and yield the groupdict() if matched, else increment malformed count. Aggregator: per configured field, maintain a Counter (which is a dict); at end, most_common(K). For very high cardinality, switch to count-min + heap.
Data Structures Used
| Structure | Purpose |
|---|---|
Compiled re.Pattern | match each line in O(line length) |
Counter per field | exact top-K within bounded cardinality |
| Min-heap of (count, key) of size K | bounded top-K when cardinality is unbounded |
Counters for total, malformed | observability |
Correctness Argument
Streaming: for line in file: reads at most one line buffer at a time. Memory is O(longest line + aggregator state). For 100 GB files with 1 KB lines, memory stays at ~aggregator-state size.
Aggregation: Counter.most_common(K) returns the exact top-K when all keys are tracked. When using a count-min sketch + bounded heap, the result is approximate with bounded error: actual_count ≤ estimate ≤ actual_count + ε · total with probability ≥ 1 − δ. We pick ε, δ to fit memory.
Complexity
- Per line: O(L · regex-complexity) for parsing + O(F) for F fields aggregated
- Total: O(N · L · regex)
- Memory: O(unique keys per field) for exact aggregation; O(width × depth) for sketch
Implementation Requirements
import re
from collections import Counter
from typing import Iterator, Iterable, Optional
class LogParser:
def __init__(self, pattern: str, top_k: int = 10,
aggregate_fields: Optional[list[str]] = None):
self._re = re.compile(pattern)
self._k = top_k
self._fields = aggregate_fields # None = aggregate all named groups
def parse_stream(self, lines: Iterable[str]) -> Iterator[dict]:
for line in lines:
line = line.rstrip("\n")
m = self._re.match(line)
if m is None:
continue
yield m.groupdict()
def aggregate(self, lines: Iterable[str]) -> dict:
counters: dict[str, Counter] = {}
total = malformed = 0
for line in lines:
line = line.rstrip("\n")
total += 1
m = self._re.match(line)
if m is None:
malformed += 1
continue
d = m.groupdict()
fields = self._fields or list(d.keys())
for f in fields:
v = d.get(f)
if v is None: continue
counters.setdefault(f, Counter())[v] += 1
out = {f: c.most_common(self._k) for f, c in counters.items()}
out["total"] = total
out["malformed"] = malformed
return out
# Bounded-memory variant: top-K only via heap
import heapq
class BoundedTopK:
"""Approximate top-K using count-min sketch + min-heap of size K.
For high-cardinality streams. Replace LogParser._field_counters with this.
"""
def __init__(self, k: int, width: int = 2048, depth: int = 5):
self._k = k
self._w, self._d = width, depth
import random
self._table = [[0] * width for _ in range(depth)]
# Independent hash seeds.
self._seeds = [random.randint(1, 2**31 - 1) for _ in range(depth)]
self._heap: list[tuple[int, str]] = [] # (count, key)
self._in_heap: dict[str, int] = {} # key -> count seen at insert
def add(self, key: str) -> None:
est = self._increment(key)
if key in self._in_heap:
# Best-effort: refresh heap entry. (Lazy: do nothing; entries are stale.)
self._in_heap[key] = est
return
if len(self._heap) < self._k:
heapq.heappush(self._heap, (est, key))
self._in_heap[key] = est
return
if est > self._heap[0][0]:
old_count, old_key = heapq.heappushpop(self._heap, (est, key))
self._in_heap.pop(old_key, None)
self._in_heap[key] = est
def _increment(self, key: str) -> int:
ests = []
for i in range(self._d):
j = (hash((self._seeds[i], key))) % self._w
self._table[i][j] += 1
ests.append(self._table[i][j])
return min(ests)
def top_k(self) -> list[tuple[str, int]]:
return sorted(((k, c) for c, k in self._heap), key=lambda p: -p[1])
Tests
import unittest, io
LOG_PATTERN = (r'(?P<ip>\S+) - - \[(?P<ts>[^\]]+)\] '
r'"(?P<method>\S+) (?P<path>\S+) HTTP/[\d.]+" '
r'(?P<status>\d+) (?P<bytes>\d+)')
SAMPLE = """1.1.1.1 - - [01/Jan/2026:00:00:00 +0000] "GET /a HTTP/1.1" 200 100
2.2.2.2 - - [01/Jan/2026:00:00:01 +0000] "GET /b HTTP/1.1" 200 200
1.1.1.1 - - [01/Jan/2026:00:00:02 +0000] "POST /a HTTP/1.1" 500 0
malformed log line junk junk junk
1.1.1.1 - - [01/Jan/2026:00:00:03 +0000] "GET /a HTTP/1.1" 200 100
"""
class TestParser(unittest.TestCase):
def test_parse_stream(self):
p = LogParser(LOG_PATTERN, top_k=3)
recs = list(p.parse_stream(io.StringIO(SAMPLE)))
self.assertEqual(len(recs), 4)
self.assertEqual(recs[0]["ip"], "1.1.1.1")
self.assertEqual(recs[0]["status"], "200")
def test_aggregate(self):
p = LogParser(LOG_PATTERN, top_k=3, aggregate_fields=["ip", "status"])
agg = p.aggregate(io.StringIO(SAMPLE))
self.assertEqual(agg["total"], 5)
self.assertEqual(agg["malformed"], 1)
self.assertEqual(agg["ip"][0], ("1.1.1.1", 3))
self.assertEqual(dict(agg["status"]), {"200": 3, "500": 1})
def test_streaming_memory(self):
# Generate a synthetic stream and ensure parse_stream is lazy
def gen():
for i in range(10000):
yield f'1.1.1.1 - - [now] "GET /p{i % 100} HTTP/1.1" 200 100'
p = LogParser(LOG_PATTERN)
# consume one record at a time
it = p.parse_stream(gen())
first = next(it)
self.assertEqual(first["path"], "/p0")
Follow-up Questions
(4) Observe / monitor? Throughput (lines/sec), parse error rate (malformed/total), per-field cardinality (gauge), p99 line size (latency surrogate). Alert on parse error rate spiking — usually means upstream changed the format.
(5) Tests? Unit on regex correctness with hand-crafted lines; property-based tests with random-line generators; smoke on a real prod-shaped sample (1 MB); large-input test that asserts memory stays bounded (tracemalloc.get_traced_memory() in Python).
(7) Backpressure? If the consumer of parse_stream is slow, the iterator naturally pauses — Python generators are pull-based. For the producer side (file reads), no backpressure issue. If shipping to a downstream like Kafka, buffer with a bounded queue and drop on full (with a counter).
(11) Configuration knobs? pattern, top_k, aggregate_fields, bounded_memory: bool (toggle exact vs sketch-based). Knobs not to expose: regex compilation cache.
(13) Poison pill? A line that takes O(catastrophic backtracking) on the regex (regex DoS via specific patterns). Mitigation: line length cap (skip lines > N bytes), regex timeout (Python: only available in regex package, not stdlib re), or pre-compile with anchors and avoid .* at the start.
Product Extension
Production systems use one of: logstash / fluentd (regex-based extraction with field rules), CloudWatch Logs Insights (column-based after extraction), Datadog Logs / Splunk (full pipeline with grok patterns and ingest-time enrichment). The data structure that powers most “top-K-over-stream” dashboards is count-min + heap; HLL powers distinct-count widgets; reservoir sampling powers “show me 100 random matching events”.
Language/Runtime Follow-ups
- Python:
reis fast enough for most logs but doesn’t compile to DFA — backtracking is a real risk. Use theregexpackage for timeout support. For raw speed,pyre2(re2 binding) avoids backtracking entirely. - Java:
Pattern.compile(...)once; reuse.Matcheris mutable per match. For very high throughput, RE2/J avoids backtracking. - Go:
regexppackage is RE2-based — guaranteed linear time, no catastrophic backtracking. Idiomatic for log parsing. - C++:
std::regexis slow; prefer Boost.Regex or PCRE2 in production. - JS/TS: V8’s regex is backtracking; same DoS concern as Python’s
re. Node has no built-in regex timeout.
Common Bugs
- Loading the file:
open(f).readlines()orf.read().split("\n")— instant OOM on large files. - Recompiling the regex per line — 100x slowdown.
- Forgetting to strip
\n— the last named group captures\nand breaks comparisons. - Using
.*greedily inside the pattern — catastrophic backtracking on long lines. - Aggregator dict grows unbounded on high-cardinality fields (e.g., user-agent string with version churn). Cap or use sketch.
Debugging Strategy
For parse failures: print the first 5 malformed lines and inspect the regex against them. For wrong field values: print m.groupdict() of one matching line. For OOM: tracemalloc.start(); ...; print(tracemalloc.get_traced_memory()) at intervals — find the structure that grows. For slowness: cProfile and check whether the hot spot is regex match or dict update.
Mastery Criteria
-
Implemented streaming
LogParserwith bounded aggregation in <25 minutes. - All three tests pass first run.
-
Stated
for line in file:lazy iteration without prompting. -
Explained when to switch from
Counterto count-min + heap (when unique-key memory exceeds budget). - Answered follow-ups #4, #7, #11, #13 (regex DoS) crisply.
- Identified backtracking risk in user-supplied regexes.
Lab 09 — File Deduplication
Goal
Find all groups of duplicate files in a directory tree using a three-stage filter: size → quick hash (first/last K bytes) → full content hash. After this lab you should be able to design and implement an efficient file deduper that minimizes I/O on huge directories in under 25 minutes.
Background Concepts
The naive approach — read every file and group by full hash — is correct but wastes I/O on files that are clearly not duplicates (different sizes). The classical optimization is a three-stage cascade:
- Group by size. Two files of different sizes cannot be duplicates. This is a
statcall (cheap, no read). - Group by quick hash. For each size-group with ≥2 files, hash the first and last K bytes (e.g., 4 KB each). Files with different head+tail hashes are not duplicates.
- Group by full hash. For each surviving group with ≥2 files, hash the full content. This is the only stage that does full reads.
This works because in real datasets, most files of the same size differ in their first/last few KB (think .docx files with embedded timestamps, video files with different headers, log files with different first lines). The cascade reduces total I/O by ~10–100×.
Interview Context
This is a popular practical problem at infrastructure / file-storage companies (Dropbox, Box, Google Drive, AWS S3 dedup). It tests I/O awareness, hashing fluency, and ability to design a multi-stage filter pipeline. The interviewer wants to hear “I’d minimize I/O by going size → quick-hash → full-hash” before any code.
Problem Statement
Design find_duplicates(root) -> list[list[Path]]:
- Walk the directory tree under
root. - Return groups (lists) of paths that have identical content. Each group has ≥ 2 paths.
- Use the three-stage cascade.
Constraints
- Up to 10^6 files
- Up to 1 TB total bytes
- Memory: ≤ 1 GB
- Read budget: minimize bytes read (the goal of optimization)
Clarifying Questions
- Symlinks — follow or skip? (Skip by default to avoid loops; configurable.)
- Hidden files? (Include by default; let caller filter.)
- Empty files (size 0) — duplicates of each other? (Yes — include them as a group; or filter; ask.)
- Hash function? (
hashlib.blake2bis fast;sha256is the cryptographic default;md5is fine for non-adversarial dedup. Pick non-cryptographic-fast for performance, cryptographic if the result is durable.) - Concurrency? (Not strictly required; CPU is hashing, I/O is reads — both parallelize well. State as a follow-up.)
Examples
root/
├── a.txt "hello"
├── b.txt "hello"
├── c.txt "world"
├── d.txt "world"
└── e.txt "different"
find_duplicates("root")
-> [[Path("root/a.txt"), Path("root/b.txt")],
[Path("root/c.txt"), Path("root/d.txt")]]
Initial Brute Force
For every pair of files, compare byte-for-byte. O(N² · L) reads. Catastrophic at 10^6 files.
Brute Force Complexity
O(N²) pairwise comparisons, each O(L). At N=10^6 and L=1 MB, this is 10^18 byte comparisons — never completes.
Optimization Path
Three-stage cascade:
- Group by size: O(N)
statcalls. Memory O(N · path). - Quick-hash within each size group: O(K · |group|) per group; only run on size-groups with ≥2 files.
- Full-hash within each quick-hash group: O(L · |group|).
Total reads: most files are excluded at stage 1 (different sizes); of the rest, most are excluded at stage 2 (quick hash differs). Only true (or very near) duplicates get a full read.
Final Expected Approach
Walk the tree once, building dict[size, list[Path]]. For groups with len ≥ 2, build dict[quick_hash, list[Path]]. For surviving groups, build dict[full_hash, list[Path]]. Output groups with len ≥ 2.
Data Structures Used
| Structure | Stage |
|---|---|
dict[int, list[Path]] | size grouping |
dict[bytes, list[Path]] | quick-hash grouping |
dict[bytes, list[Path]] | full-hash grouping |
Correctness Argument
Soundness: every group output has identical full content (verified by full-hash equality, modulo collision probability of 2^-256 for SHA-256, negligible).
Completeness: two files A, B with identical content have:
- equal size (so they survive stage 1)
- equal quick-hash (so they survive stage 2)
- equal full-hash (so they end up in the same output group)
Therefore A and B appear in the same output group. The cascade does not miss any duplicate.
Complexity
- Time: O(N) for size grouping; O(K · N_size_dups) for quick-hash; O(L · N_full_dups) for full-hash. K ≪ L, N_full_dups ≪ N.
- Space: O(N) for path bookkeeping.
Implementation Requirements
import os, hashlib
from pathlib import Path
from collections import defaultdict
from typing import Iterable
QUICK_BYTES = 4096 # 4 KB head + 4 KB tail
def _quick_hash(path: Path) -> bytes:
h = hashlib.blake2b(digest_size=16)
size = path.stat().st_size
with open(path, "rb") as f:
head = f.read(QUICK_BYTES)
h.update(head)
if size > 2 * QUICK_BYTES:
f.seek(-QUICK_BYTES, os.SEEK_END)
tail = f.read(QUICK_BYTES)
h.update(tail)
return h.digest()
def _full_hash(path: Path, chunk: int = 1 << 20) -> bytes:
h = hashlib.blake2b(digest_size=32)
with open(path, "rb") as f:
while True:
buf = f.read(chunk)
if not buf: break
h.update(buf)
return h.digest()
def find_duplicates(root: str | Path,
follow_symlinks: bool = False,
include_empty: bool = False) -> list[list[Path]]:
root = Path(root)
by_size: dict[int, list[Path]] = defaultdict(list)
for dirpath, _, files in os.walk(root, followlinks=follow_symlinks):
for name in files:
p = Path(dirpath) / name
try:
st = p.stat() if follow_symlinks else p.lstat()
if not include_empty and st.st_size == 0: continue
if not p.is_file(): continue
by_size[st.st_size].append(p)
except (OSError, PermissionError):
continue
candidates_after_size: list[list[Path]] = [g for g in by_size.values() if len(g) >= 2]
by_quick: list[list[Path]] = []
for group in candidates_after_size:
sub: dict[bytes, list[Path]] = defaultdict(list)
for p in group:
try: sub[_quick_hash(p)].append(p)
except OSError: continue
for g in sub.values():
if len(g) >= 2: by_quick.append(g)
out: list[list[Path]] = []
for group in by_quick:
sub: dict[bytes, list[Path]] = defaultdict(list)
for p in group:
try: sub[_full_hash(p)].append(p)
except OSError: continue
for g in sub.values():
if len(g) >= 2: out.append(g)
return out
Tests
import unittest, tempfile, os
from pathlib import Path
class TestDedup(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.root = Path(self.tmp.name)
def tearDown(self):
self.tmp.cleanup()
def _w(self, name: str, content: bytes):
p = self.root / name
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(content)
return p
def test_basic(self):
a = self._w("a", b"hello")
b = self._w("sub/b", b"hello")
c = self._w("c", b"world")
d = self._w("sub/d", b"world")
e = self._w("e", b"unique")
groups = find_duplicates(self.root)
flat = sorted([sorted(map(str, g)) for g in groups])
self.assertEqual(len(flat), 2)
self.assertIn(sorted([str(a), str(b)]), flat)
self.assertIn(sorted([str(c), str(d)]), flat)
def test_size_excludes_non_duplicates(self):
self._w("a", b"x" * 100)
self._w("b", b"x" * 200) # different size
self.assertEqual(find_duplicates(self.root), [])
def test_quick_hash_disambiguates(self):
# Same size, different head: stage 2 separates them
self._w("a", b"head1" + b"x" * 8000)
self._w("b", b"head2" + b"x" * 8000)
self.assertEqual(find_duplicates(self.root), [])
def test_large_groups(self):
for i in range(5):
self._w(f"copy{i}", b"same content")
groups = find_duplicates(self.root)
self.assertEqual(len(groups), 1)
self.assertEqual(len(groups[0]), 5)
Follow-up Questions
(3) Scale to N nodes? Distributed dedup over a fleet: size grouping is local on each node; for cross-node dedup, broadcast (size, quick_hash, node, path) tuples to a coordinator, group by (size, quick_hash), then have nodes that share a group exchange full hashes. The full read happens locally; only hashes (32 bytes) move over the network.
(4) Observe / monitor? Files scanned (counter), bytes read at each stage (gauge — quantifies the savings of the cascade), groups found (gauge), errors per stage (counter). The “bytes read at full-hash stage / total bytes” ratio is the key savings metric.
(8) Partial failure? A file deleted/replaced mid-scan: the second-stage hash may differ from the first-stage size or hash. Solutions: (a) treat the file as missing on OSError and skip, (b) snapshot the FS (LVM snapshot, ZFS, Btrfs) before scanning. (a) is the practical answer.
(13) Poison-pill input? A multi-TB file (or a sparse file with a 1 PB apparent size) blows up the full-read stage. Mitigation: cap files by size (skip if > max_file_size), or use Merkle-tree chunked hashing where each chunk is independent and partial reads can be cached/resumed.
(11) Configuration knobs? quick_bytes (default 4 KB), chunk_size for full read, max_file_size, follow_symlinks, include_empty. Knobs not to expose: hash algorithm (pick BLAKE2b for speed unless a contract requires SHA-256).
Product Extension
fdupes, rmlint, jdupes are the canonical Linux tools and use exactly this cascade. Dropbox’s chunked dedup operates at the 4 MB block level (each file is split into chunks, each chunk hashed and deduplicated separately) — letting two files that share a 1 MB prefix store the prefix once. ZFS dedup operates at the block level too. The “full-file dedup” you wrote here is the simplest version; production systems often go to chunk-level dedup for higher savings.
Language/Runtime Follow-ups
- Python:
os.walkis the standard;pathlib.Path.rglob('*')is more idiomatic but slower.hashlib.blake2bis in stdlib and ~3x faster than SHA-256. - Java:
Files.walk(Path)is the equivalent;MessageDigestfor hashing.BLAKE2requires BouncyCastle. - Go:
filepath.Walk(deprecated forfilepath.WalkDir).crypto/sha256is fast;golang.org/x/crypto/blake2bfor BLAKE2. - C++:
std::filesystem::recursive_directory_iterator. OpenSSL’sEVP_DigestUpdatefor hashing. - JS/TS:
fs.promises.readdir(dir, {recursive: true})(Node 20+).crypto.createHash('blake2b512')for hashing.
Common Bugs
- Treating symlinks as files — infinite loops or duplicate phantom matches. Use
lstatand explicitis_file()check. - Forgetting to filter size-1 groups before stage 2 (running quick-hash on isolated files wastes I/O).
- Hashing with
md5and getting bitten by collision in adversarial datasets. Use SHA-256 or BLAKE2b. - Reading the entire file into memory at full-hash stage instead of streaming. OOM on large files.
- Not handling permission errors — first
OSErrorhalts the entire scan instead of skipping the file.
Debugging Strategy
For “missed duplicates”: verify with cmp (Unix) or byte-by-byte. Print the size/quick-hash/full-hash of both files. Most missed-dup bugs are quick-hash logic (e.g., not seeking to end correctly for files smaller than 2 * QUICK_BYTES).
Mastery Criteria
- Articulated the three-stage cascade in <60 seconds before coding.
- Implemented in <25 minutes; all four tests pass.
- Stated bytes-read savings is the main optimization signal, not wall-clock.
- Answered follow-ups #3, #4, #8, #13 crisply.
- Compared full-file vs chunk-level dedup correctly.
- Identified BLAKE2b as the right hash for performance.
Lab 10 — Consistent Hashing
Goal
Implement a consistent hash ring with virtual nodes that minimizes key remapping when servers are added or removed. After this lab you should be able to design and implement consistent hashing in under 30 minutes and articulate why it beats hash(key) % N.
Background Concepts
The naive sharding scheme hash(key) % N has a catastrophic failure mode: when N changes (a server is added or removed), nearly every key remaps to a different shard. For an in-memory cache fleet, this means the entire cache is invalidated; for a stateful sharded store, this means most data must be physically migrated.
Consistent hashing solves this. Servers are placed on a ring (a circular hash space, e.g., [0, 2^64)). Each key is hashed onto the ring and assigned to the next server clockwise. When a server is added, only keys between its predecessor and itself on the ring are remapped. When a server is removed, only its keys are remapped — to its successor.
Without virtual nodes (vnodes), the ring is unbalanced: a 4-server ring assigns wildly unequal slices. Virtual nodes fix this: each physical server gets V ring positions (e.g., V=200). The ring becomes statistically balanced in O(1/sqrt(V)) deviation.
Interview Context
Consistent hashing is the default sharding mechanism for distributed caches (Memcached client libraries, Redis Cluster’s slot variant), distributed databases (DynamoDB, Cassandra), and load balancers (HAProxy, Envoy with ring_hash). It is asked at infrastructure roles at every Big Tech and many high-scale companies.
Problem Statement
Implement ConsistentHashRing(vnodes_per_server):
add_server(server_id)— add a server with V vnodes.remove_server(server_id)— remove all vnodes for the server.get_server(key) -> server_id— return the server responsible forkey.keys_moved(key, before, after)— for analysis: did this key remap?
Constraints
- 1 ≤ servers ≤ 10^4
- 1 ≤ vnodes per server ≤ 1000
- 10^5 lookups / second
- Lookup latency: O(log N · V)
Clarifying Questions
- Hash function: cryptographic or fast? (Use a fast non-crypto hash: MurmurHash, xxHash. Stable across processes.)
- Vnode count V: hard-coded or configurable? (Configurable, default 100–200.)
- Replication: should
get_serverreturn one or multiple distinct servers? (Often a follow-up; primary is one.) - Hot-spotting awareness: do we know any keys are extremely hot? (Bounded-load consistent hashing is a follow-up.)
Examples
ring = ConsistentHashRing(vnodes_per_server=100)
ring.add_server("s1"); ring.add_server("s2"); ring.add_server("s3")
ring.get_server("user-42") -> "s2"
ring.add_server("s4")
ring.get_server("user-42") -> "s2" or "s4" (only some keys remap)
# ~25% of keys remap on adding a 4th server, not 75% as with mod-N
Initial Brute Force
hash(key) % N. Simple and balanced; catastrophic on N change. Useful for understanding the problem, not as the solution.
Brute Force Complexity
O(1) per lookup; O(N · keys / N) = O(keys) remap on N change — not the problem; the brute force is mod-N and we’re moving away from it.
Optimization Path
Replace mod-N with a sorted ring. Servers map to multiple positions; lookup is binary search. On add/remove, only insert/delete vnode positions; existing positions don’t move.
Final Expected Approach
A sorted list of (hash_value, server_id) tuples kept in ring order. get_server(key): hash the key, binary-search for the smallest ring position ≥ key_hash; wrap around if past the end. add_server: insert V positions. remove_server: remove all V positions.
Data Structures Used
| Structure | Purpose |
|---|---|
Sorted list of (hash, server) | the ring |
dict[server_id, list[hash_positions]] | bookkeeping for removal |
bisect for binary search | O(log N · V) lookup |
Correctness Argument
Key locality on resize: when adding server S with vnodes [v_1, …, v_V], the only keys whose owner changes are those whose hash falls in some (predecessor(v_i), v_i] range. The expected fraction of keys affected is V / (total vnodes) ≈ 1/(N+1) — exactly the right number to assign to the new server, and no more.
Balanced load: with V vnodes per server, the variance of the load assigned to each server scales as O(log N / V). At V=100, N=10, the imbalance is < 5%; at V=1000, < 1.5%.
Complexity
get_server: O(log(N · V))add_server: O(V · log(N · V)) per insert; total O(V log N) per server- Space: O(N · V)
Implementation Requirements
import bisect, hashlib
from typing import Optional
def _hash(s: str) -> int:
"""Fast, deterministic hash. Use MD5 for speed; SHA-1 also fine."""
return int.from_bytes(hashlib.md5(s.encode()).digest()[:8], "big")
class ConsistentHashRing:
def __init__(self, vnodes_per_server: int = 100):
self._v = vnodes_per_server
self._ring: list[tuple[int, str]] = [] # sorted by hash
self._server_positions: dict[str, list[int]] = {}
def add_server(self, server_id: str) -> None:
if server_id in self._server_positions:
return
positions = []
for i in range(self._v):
h = _hash(f"{server_id}#{i}")
bisect.insort(self._ring, (h, server_id))
positions.append(h)
self._server_positions[server_id] = positions
def remove_server(self, server_id: str) -> None:
positions = self._server_positions.pop(server_id, None)
if positions is None: return
# Rebuild filtered ring (O(N V) — acceptable; remove is rare)
self._ring = [(h, s) for (h, s) in self._ring if s != server_id]
def get_server(self, key: str) -> Optional[str]:
if not self._ring:
return None
kh = _hash(key)
idx = bisect.bisect_left(self._ring, (kh, ""))
if idx == len(self._ring):
idx = 0 # wrap around
return self._ring[idx][1]
def server_count(self) -> int:
return len(self._server_positions)
# Bounded-load variant for hot-spot mitigation:
class BoundedLoadRing:
"""Consistent hashing with bounded-load: each server's load ≤ avg * (1+ε)."""
def __init__(self, vnodes_per_server: int = 100, epsilon: float = 0.25):
self._inner = ConsistentHashRing(vnodes_per_server)
self._eps = epsilon
self._load: dict[str, int] = {}
def add_server(self, sid: str) -> None:
self._inner.add_server(sid); self._load.setdefault(sid, 0)
def remove_server(self, sid: str) -> None:
self._inner.remove_server(sid); self._load.pop(sid, None)
def get_server(self, key: str, total_keys: int) -> Optional[str]:
n = self._inner.server_count()
if n == 0: return None
cap = (total_keys / n) * (1 + self._eps)
# Walk forward from the first candidate until we find one under cap.
kh = _hash(key)
ring = self._inner._ring
idx = bisect.bisect_left(ring, (kh, ""))
if idx == len(ring): idx = 0
for offset in range(len(ring)):
_, sid = ring[(idx + offset) % len(ring)]
if self._load.get(sid, 0) < cap:
self._load[sid] = self._load.get(sid, 0) + 1
return sid
return ring[idx][1] # all over cap; pick first
Tests
import unittest, random, statistics
class TestRing(unittest.TestCase):
def test_basic(self):
r = ConsistentHashRing(vnodes_per_server=10)
r.add_server("s1"); r.add_server("s2"); r.add_server("s3")
self.assertIsNotNone(r.get_server("k1"))
self.assertIn(r.get_server("k1"), {"s1", "s2", "s3"})
r.remove_server("s2")
self.assertIn(r.get_server("k1"), {"s1", "s3"})
def test_minimal_remapping(self):
r = ConsistentHashRing(vnodes_per_server=200)
for s in ["s1", "s2", "s3"]: r.add_server(s)
keys = [f"key-{i}" for i in range(10000)]
before = {k: r.get_server(k) for k in keys}
r.add_server("s4")
after = {k: r.get_server(k) for k in keys}
moved = sum(1 for k in keys if before[k] != after[k])
# Expected ~25% remapping (from 3 servers to 4).
# mod-N would have moved ~75%.
self.assertLess(moved, 3500)
self.assertGreater(moved, 1500)
def test_balance(self):
r = ConsistentHashRing(vnodes_per_server=200)
for i in range(10): r.add_server(f"s{i}")
keys = [f"k-{i}" for i in range(20000)]
loads = {}
for k in keys:
s = r.get_server(k)
loads[s] = loads.get(s, 0) + 1
avg = 2000
# With 200 vnodes per server, variance should be small.
for cnt in loads.values():
self.assertLess(abs(cnt - avg), 350) # ≤17% deviation
def test_empty_ring(self):
r = ConsistentHashRing(vnodes_per_server=10)
self.assertIsNone(r.get_server("k"))
Follow-up Questions
(3) Scale to N nodes? Already designed for it. The ring scales because lookup is O(log N · V). The bottleneck on add_server is O(V log N) insertions; sorted-tree (red-black tree) implementations get O(V log N) similarly. For very large N, use a B-tree or skip list. For replication: get_servers(key, R) returns the next R unique servers clockwise.
(8) Partial failure? A server going down is naturally handled — its vnodes are removed and keys remap to the successor. The challenge is hot spots: when one server dies, all its load moves to one successor. Bounded-load consistent hashing (Mirsky’s variant) caps each server at (1+ε) × avg_load, spilling overflow to the next server. Implemented above.
(10) Consistency model? The ring itself is a routing function. The actual stored data has whatever consistency model the underlying store offers (linearizable, eventual, etc.). One subtlety: when a server is added, the data on the predecessor needs to be transferred to the new server before the routing change takes effect, or you serve stale/missing data. Two-phase add: install vnodes-as-readonly → migrate keys → activate.
(11) Configuration knobs? vnodes_per_server (100–500 covers most workloads), hash_function. Not to expose: ring data structure, balance/heuristics.
(4) Observe / monitor? Per-server load (gauge), key remapping events (counter), p99 lookup latency (histogram). Imbalance alert: trigger if any server’s load > 1.5x avg.
Product Extension
DynamoDB uses consistent hashing with explicit ranges. Cassandra uses 256 vnodes per node by default. Memcached’s clients (ketama, libmemcached) use consistent hashing. Envoy’s ring_hash load balancer uses it for sticky-session routing. Discord’s chat sharding originally used hash(channel) % N and famously hit the rebalance problem; they migrated to a fixed-bucket scheme. The point: even big companies get this wrong if they pick mod-N.
Language/Runtime Follow-ups
- Python:
bisectis the right tool for sorted-list maintenance. For very large rings, usesortedcontainers.SortedList(skip-list-backed) for O(log N) inserts. - Java:
TreeMap<Long, String>—ceilingKey(hash)does the lookup. Idiomatic. - Go:
sort.Searchover a[]uint64for the ring. Good locality, fast. - C++:
std::map<uint64_t, std::string>withlower_bound. Or sort a vector and binary-search. - JS/TS: no sorted-tree in stdlib; use the
sorted-array-functionsnpm package or maintain a sorted array manually.
Common Bugs
- Using
hash(key) % len(ring)to pick a vnode index — that is mod-N inside the ring. Use the ring’s actual hash space. - Forgetting to wrap around —
bisectreturnslen(ring)for a key past the last vnode; you must wrap to index 0. - Hash collisions on small rings — two servers’ vnodes land on the same hash. Either accept “first inserted wins” (deterministic) or perturb the suffix until unique.
- Removing a server but leaving its vnodes in
_server_positions(memory leak; subsequentadd_serverfor the same id silently no-ops because of theif … in …guard). - Using
hash(key)(Python’s built-in) — randomized per-process. Different processes route the same key to different servers. Use a stable hash like MD5 or MurmurHash.
Debugging Strategy
For “wrong server” complaints: log the (hashed-key, ring-position-found, server). For imbalance: dump load distribution and check vnode count. For “everything remapped” after add: count the remapped key fraction; if > 1/N, vnode count is too low or the hash function is poor.
Mastery Criteria
- Implemented ring + lookup + add/remove in <30 minutes.
- Stated minimal-remapping invariant (≤ 1/N keys move on add) without prompting.
- All four tests pass.
- Articulated why vnodes are needed in <60 seconds.
- Compared mod-N, ring-no-vnodes, ring-with-vnodes, bounded-load on a whiteboard.
- Answered follow-ups #3, #8 (bounded load), #10 (data migration coordination), #11 crisply.
Lab 11 — Message Dispatcher
Goal
Implement a message dispatcher that fans out messages to N consumers with fairness, per-consumer priority levels, and per-consumer backpressure. After this lab you should be able to design a multi-consumer dispatcher that doesn’t starve slow consumers and doesn’t get stalled by slow ones, in under 30 minutes.
Background Concepts
A dispatcher accepts messages from one (or more) producers and routes them to N consumers. Three classical problems:
- Fairness: round-robin across consumers vs weighted by priority. Strict round-robin is unfair if some consumers have higher priority. Weighted-fair-queueing (WFQ) gives each consumer a share proportional to its weight.
- Slow-consumer problem: a slow consumer’s queue fills up. If we share a single queue across all consumers, the slow one stalls everyone. Solution: per-consumer queues with bounded capacity.
- Backpressure: when a consumer’s queue is full, what do we do? Options: (a) block the producer (slowest fair), (b) drop oldest, (c) drop newest, (d) reject and signal upstream. Default to (b) for telemetry, (a) for orders/payments.
Interview Context
Dispatcher problems show up at message-bus companies (Confluent, Solace), real-time platforms (Twilio, PubNub, Pusher), and any backend with fan-out. The interviewer wants to hear: per-consumer queue, priority-aware scheduling, explicit backpressure policy. Common failure: a single shared queue with consumers competing — works for two consumers, breaks at scale.
Problem Statement
Design Dispatcher:
register_consumer(consumer_id, priority=1, queue_capacity=1024, on_full="drop_oldest")dispatch(message)— fan out to all registered consumers (broadcast).consume(consumer_id) -> Message | None— non-blocking dequeue for a consumer.consume_blocking(consumer_id, timeout)— blocking dequeue.unregister(consumer_id)stats() -> dict— per-consumer queue size, drops, throughput.
Constraints
- 1 ≤ consumers ≤ 1000
- 10^5 dispatches / second
- Per-consumer latency: < 1 ms p99 from dispatch to availability
Clarifying Questions
- Broadcast (every consumer gets every message) or partition (each message goes to one)? (Pick one; we’ll do broadcast — the harder problem.)
- Strict ordering across consumers? (Each consumer sees messages in dispatch order; cross-consumer ordering not guaranteed.)
- Priority semantics: priority is a weight, not a strict precedence? (Weight is the standard.)
- Should consumers be threads, or should
consumebe polled by external code? (External polling — simpler.)
Examples
d = Dispatcher()
d.register_consumer("fast", priority=2, queue_capacity=100)
d.register_consumer("slow", priority=1, queue_capacity=10, on_full="drop_oldest")
for i in range(20):
d.dispatch({"i": i})
# slow consumer dropped 10 oldest; only sees last 10
slow_msgs = []
while True:
m = d.consume("slow")
if m is None: break
slow_msgs.append(m)
# slow_msgs has the 10 most recent messages
Initial Brute Force
A single shared deque and a for c in consumers: c.recv(msg) loop on dispatch. If a consumer is slow, the dispatch blocks on it. The whole pipeline stalls.
Brute Force Complexity
Per dispatch: O(N · consumer-receive-time). Worst-case latency is bounded by the slowest consumer.
Optimization Path
Per-consumer queue. Dispatch is now O(N · queue-push-time) ≈ O(N) — fast and bounded by the dispatcher’s own work, not by any consumer’s processing. Slow consumers fill their own queue and trigger their own backpressure policy without affecting others.
Final Expected Approach
A dict[consumer_id, ConsumerQueue]. Each ConsumerQueue has a deque, a Lock (or use queue.Queue), a capacity, an on-full policy, and counters. dispatch iterates the dict and pushes to each queue, applying the policy on full. consume dequeues from the named queue. The single shared lock would serialize dispatches; the per-queue lock parallelizes them with the cost that there’s no atomic “dispatch sees a consistent registration” — acceptable trade.
Data Structures Used
| Structure | Purpose |
|---|---|
dict[id, ConsumerQueue] | per-consumer state |
deque per consumer | bounded FIFO |
Lock per consumer | concurrent push/pop safety |
Condition per consumer | blocking consume |
Correctness Argument
No starvation: every consumer has its own queue; a slow one cannot block dispatch. Producer dispatch latency is bounded by O(N) (the dict iteration), independent of consumer speed.
Priority weighting: at dispatch time, we don’t apply weight (every consumer gets every message — broadcast). Priority is used in the consume order if we have a single consumer thread that polls all queues in priority order; in that variant, weight determines how many messages we consume from each per round (e.g., 2 from priority=2, 1 from priority=1).
Backpressure: when a queue is at capacity, the consumer-defined policy fires. Each consumer’s drops are tracked separately; the dispatcher itself never blocks.
Complexity
dispatch: O(N) where N = number of consumersconsume: O(1)- Space: O(N · capacity)
Implementation Requirements
import threading, time
from collections import deque
from typing import Any, Optional
class _ConsumerQueue:
def __init__(self, capacity: int, on_full: str = "drop_oldest", priority: int = 1):
self.capacity = capacity
self.on_full = on_full # "drop_oldest" | "drop_newest" | "block"
self.priority = priority
self.q: deque = deque()
self.lock = threading.Lock()
self.cond = threading.Condition(self.lock)
self.dropped = 0
self.delivered = 0
class Dispatcher:
def __init__(self):
self._consumers: dict[str, _ConsumerQueue] = {}
self._reg_lock = threading.RLock()
def register_consumer(self, consumer_id: str, priority: int = 1,
queue_capacity: int = 1024,
on_full: str = "drop_oldest") -> None:
with self._reg_lock:
if consumer_id in self._consumers:
raise ValueError(f"{consumer_id} already registered")
self._consumers[consumer_id] = _ConsumerQueue(queue_capacity, on_full, priority)
def unregister(self, consumer_id: str) -> None:
with self._reg_lock:
self._consumers.pop(consumer_id, None)
def dispatch(self, message: Any) -> None:
with self._reg_lock:
consumers = list(self._consumers.values())
for cq in consumers:
with cq.lock:
if len(cq.q) >= cq.capacity:
if cq.on_full == "drop_oldest":
cq.q.popleft()
cq.dropped += 1
cq.q.append(message)
elif cq.on_full == "drop_newest":
cq.dropped += 1
continue
elif cq.on_full == "block":
while len(cq.q) >= cq.capacity:
cq.cond.wait()
cq.q.append(message)
else:
cq.q.append(message)
cq.cond.notify()
def consume(self, consumer_id: str) -> Optional[Any]:
cq = self._consumers.get(consumer_id)
if cq is None: return None
with cq.lock:
if not cq.q: return None
m = cq.q.popleft()
cq.delivered += 1
cq.cond.notify() # producer waiting on "block" policy
return m
def consume_blocking(self, consumer_id: str, timeout: Optional[float] = None) -> Optional[Any]:
cq = self._consumers.get(consumer_id)
if cq is None: return None
deadline = None if timeout is None else time.monotonic() + timeout
with cq.lock:
while not cq.q:
if deadline is None:
cq.cond.wait()
else:
rem = deadline - time.monotonic()
if rem <= 0: return None
cq.cond.wait(timeout=rem)
m = cq.q.popleft()
cq.delivered += 1
cq.cond.notify()
return m
def stats(self) -> dict:
with self._reg_lock:
return {
cid: {
"queue_size": len(cq.q),
"dropped": cq.dropped,
"delivered": cq.delivered,
"capacity": cq.capacity,
"priority": cq.priority,
}
for cid, cq in self._consumers.items()
}
Tests
import unittest, threading, time
class TestDispatcher(unittest.TestCase):
def test_broadcast(self):
d = Dispatcher()
d.register_consumer("a"); d.register_consumer("b")
for i in range(5): d.dispatch(i)
a = [d.consume("a") for _ in range(5)]
b = [d.consume("b") for _ in range(5)]
self.assertEqual(a, [0, 1, 2, 3, 4])
self.assertEqual(b, [0, 1, 2, 3, 4])
def test_drop_oldest_on_full(self):
d = Dispatcher()
d.register_consumer("a", queue_capacity=3, on_full="drop_oldest")
for i in range(5): d.dispatch(i)
out = []
while True:
m = d.consume("a")
if m is None: break
out.append(m)
self.assertEqual(out, [2, 3, 4])
self.assertEqual(d.stats()["a"]["dropped"], 2)
def test_slow_consumer_does_not_block_fast(self):
d = Dispatcher()
d.register_consumer("slow", queue_capacity=2, on_full="drop_oldest")
d.register_consumer("fast", queue_capacity=100)
for i in range(50):
d.dispatch(i)
# fast got everything
fast = []
while True:
m = d.consume("fast")
if m is None: break
fast.append(m)
self.assertEqual(len(fast), 50)
# slow has 2 (capacity) — most-recent
slow = [d.consume("slow") for _ in range(2)]
self.assertEqual(slow, [48, 49])
def test_blocking_consume_wakes_on_dispatch(self):
d = Dispatcher()
d.register_consumer("c", queue_capacity=10)
results = []
def consumer():
results.append(d.consume_blocking("c", timeout=2.0))
t = threading.Thread(target=consumer); t.start()
time.sleep(0.05)
d.dispatch("hello")
t.join(timeout=2.0)
self.assertEqual(results, ["hello"])
Follow-up Questions
(7) Backpressure? This is the core problem. Three policies built in: drop-oldest (best for telemetry), drop-newest (best for “first message matters”), block (best for ordered streams where loss is unacceptable). The right pick depends on the data semantics; expose it per-consumer.
(3) Scale to N nodes? Distributed dispatch: each consumer “subscribes” via a network connection; the dispatcher fans out over those connections. Bottleneck shifts from queue-push to per-consumer network round-trip. For very high consumer counts, a hierarchical dispatcher (dispatch to N regional dispatchers, each of which dispatches to M local consumers) reduces the per-message broadcast cost.
(4) Observe / monitor? Per-consumer queue depth, drop rate, throughput. The drop-rate heatmap by consumer is the first dashboard you draw. Alert when any consumer has > 1% drop rate.
(8) Partial failure? A consumer that connects, then disappears: the dispatcher must detect (via missed heartbeat or socket close) and unregister; otherwise its queue grows unbounded. Heartbeat / TTL on consumer registration.
(11) Configuration knobs? Per-consumer: priority, queue_capacity, on_full policy. Global: max consumers. Knobs not to expose: the lock granularity (per-consumer is correct).
Product Extension
This is the in-process version of Kafka consumer groups, RabbitMQ exchange-to-queue fan-out, and Redis Pub/Sub. Real systems add: durable per-consumer logs (Kafka’s offset model), dynamic rebalancing as consumers join/leave (Kafka group coordinator), and message acknowledgment (AMQP). The core problem of “don’t let slow consumers stall fast ones” is solved everywhere by per-consumer storage.
Language/Runtime Follow-ups
- Python:
queue.Queueper consumer is also fine and simpler — built-in blocking and capacity. Custom_ConsumerQueueshown for control over the on-full policy. - Java:
LinkedBlockingQueueper consumer;RingBuffer(Disruptor pattern) for high-throughput. - Go: idiomatic Go:
chan Messageper consumer of sizecapacity.selectwithdefaultfor non-blocking. Simple and fast. - C++:
boost::lockfree::spsc_queueper consumer for single-producer/single-consumer; otherwise mutex + deque. - JS/TS: single event loop, so no real “consumer threads”; use EventEmitter with bounded buffers per listener. RxJS
SubjectwithbufferCountoperators.
Common Bugs
- Single shared lock for the dispatcher: serializes everything; dispatch latency = O(N · consumer-time).
blockpolicy without notifying onconsume: producer waits forever.- Forgetting to copy the consumer dict before iterating in
dispatch— concurrent unregister mutates the dict mid-iteration. - Drop-oldest implementation:
popleftthen append succeeds, but if the lock is dropped between, ordering breaks. Both ops under the same lock. - Counting a “dropped” message as both dropped and delivered (double count) when on
drop_oldestyou replace, not skip.
Debugging Strategy
Print stats() periodically. Slow consumer = high queue_size and rising dropped. Stuck consumer = blocking with no notifies — check producer dispatch isn’t stuck on a different consumer’s block policy. For “missed messages”: log per-message dispatch with consumer enumeration and replay against delivered counts.
Mastery Criteria
- Implemented per-consumer queue + dispatcher in <30 minutes.
- All four tests pass.
- Stated “per-consumer queue isolates slow consumers” before coding.
- Listed three backpressure policies (drop-oldest/newest/block) and their use cases.
- Answered follow-ups #3, #4, #7, #8, #11 crisply.
- Identified that broadcast is the harder variant; partition is simpler.
Lab 12 — In-Memory Pub/Sub
Goal
Implement an in-process publish-subscribe system with topic-based routing, wildcard subscriptions (a.b.*, a.#), and per-subscriber backpressure. After this lab you should be able to write a clean pub/sub broker in under 30 minutes and articulate the topic-matching design tradeoffs.
Background Concepts
Pub/sub differs from a job queue: subscribers don’t compete for messages; each subscriber receives every matching message. Two routing models:
- Topic-based (channel name as a string):
"orders.created","users.signup". Wildcards (*= one segment,#= many segments) come from MQTT/AMQP. - Content-based: subscribers register a predicate over message content. More flexible, much harder to scale (every message must be evaluated against every subscriber’s predicate).
Topic-based with wildcards is the standard. The implementation challenge is the wildcard matcher: a subscription on "orders.*" should match "orders.created" but not "orders.created.fraud". We can solve this with a topic trie (segment-by-segment) for O(L) per match where L is segment count, or a regex per subscription for O(N) per dispatch where N is subscription count. The trie is the production answer for systems with many subscriptions.
Interview Context
Pub/sub design is asked at messaging companies (Confluent, IBM MQ, Solace, AWS SNS), at real-time platforms (Pusher, Ably, Twilio), and broadly at any senior+ design-coding round. The interview wants both code and the design reasoning around routing, backpressure, and subscription matching.
Problem Statement
Design PubSub:
subscribe(topic_pattern, on_message) -> subscription_id—topic_patternmay include*(single segment) or#(multi-segment, must be last).unsubscribe(subscription_id)publish(topic, message)— callon_message(topic, message)on every matching subscriber.- Per-subscriber callback wrapping for backpressure (queue + drop policy).
Constraints
- Up to 10^4 active subscriptions
- Up to 10^5 publishes / second
- Subscription matching: < 100 µs per publish
- Per-subscriber callbacks may be slow; must not block the publisher
Clarifying Questions
- Wildcard syntax — MQTT (
+/#), AMQP (*/#), or other? (Pick AMQP-style:*= one segment,#= ≥ 0 segments at end.) - Synchronous or async callback delivery? (Async with per-subscriber queue is the production answer; simpler synchronous version is acceptable for the basic case.)
- Topic separator:
.or/? (.is the AMQP convention; either is fine.) - Ordering guarantees? (Per-subscriber: messages arrive in publish-order. Across subscribers: not guaranteed.)
- Replay / retain / persistence? (No by default; pure in-memory.)
Examples
ps = PubSub()
sid = ps.subscribe("orders.*", lambda topic, msg: print(f"got {topic}: {msg}"))
ps.publish("orders.created", {"id": 1}) # fires
ps.publish("orders.created.fraud", {"id": 2}) # does NOT fire (* is one segment)
ps.subscribe("orders.#", lambda t, m: log(t, m))
ps.publish("orders.created.fraud", {}) # fires (# matches multi)
ps.unsubscribe(sid)
ps.publish("orders.created", {}) # only the # subscription fires
Initial Brute Force
dict[topic_pattern, list[callback]]. On publish, iterate all subscriptions, regex-match each pattern against the topic. O(N · pattern-cost) per publish where N is subscription count.
Brute Force Complexity
At N=10^4 subscriptions and 10^5 pub/s, this is 10^9 regex matches / sec — too slow. Wildcard regex compile and match dominate.
Optimization Path
A topic trie: each node represents a segment. Children include literal-segment children plus a * and # child for wildcards. Match by walking the trie segment-by-segment, exploring each node’s literal child plus its * child plus (if at end) any # ancestor’s catch-all subscriptions.
Per-publish cost becomes O(L · branching factor) ≈ O(L) for typical trees, where L is the topic depth.
Final Expected Approach
Build a topic trie. Each node has:
children: dict[str, Node]— literal subsegmentsstar: Node | None— single-segment wildcardhash_subscriptions: list[Sub]—#catch-all (matches everything below this node)subs: list[Sub]— exact matches at this node
Publishing walks the trie segment-by-segment, at each step checking the literal child and the star child; collect matching subs at terminal nodes. Collect hash_subscriptions along the entire path.
Each subscriber owns a per-subscriber bounded queue; publish enqueues to the queue (non-blocking, drops on full); a worker thread per subscriber drains the queue and calls the callback.
Data Structures Used
| Structure | Purpose |
|---|---|
| Topic trie | O(L) routing |
dict[sub_id, Subscription] | unregister lookup |
deque per subscriber + lock | per-subscriber queue |
| Worker thread per subscriber | invoke callback off the publish path |
Correctness Argument
Routing: a subscription A.B.C matches publish topic A.B.C iff the trie walk reaches the node carrying that subscription with all segments consumed. A * matches any single segment (one node-level wildcard step). A # at a node matches any zero-or-more remaining segments — equivalent to attaching a list of “catch-all” subscribers to that node.
Per-subscriber ordering: each subscriber’s queue is FIFO; the worker drains in FIFO order. Therefore subscriber sees messages in publish order.
Publisher non-blocking: publish only enqueues; no subscriber callback runs on the publish thread. Even a callback that takes 1 second doesn’t slow publish.
Complexity
subscribe: O(L) for trie insertpublish: O(L · F · M) where L = topic depth, F = trie branching, M = matching subscribersunsubscribe: O(L) to walk the trie node and remove
Implementation Requirements
import threading, itertools
from collections import deque
from typing import Callable, Any, Optional
class _Sub:
__slots__ = ("sub_id", "callback", "queue", "lock", "cond", "capacity", "drops",
"alive", "thread")
def __init__(self, sub_id: int, callback: Callable, capacity: int = 1024):
self.sub_id = sub_id; self.callback = callback
self.queue: deque = deque()
self.lock = threading.Lock()
self.cond = threading.Condition(self.lock)
self.capacity = capacity
self.drops = 0
self.alive = True
self.thread: Optional[threading.Thread] = None
def deliver(self, topic: str, msg: Any) -> None:
with self.lock:
if len(self.queue) >= self.capacity:
self.queue.popleft(); self.drops += 1
self.queue.append((topic, msg))
self.cond.notify()
def stop(self) -> None:
with self.lock:
self.alive = False
self.cond.notify()
def run(self) -> None:
while True:
with self.lock:
while self.alive and not self.queue:
self.cond.wait()
if not self.alive and not self.queue:
return
topic, msg = self.queue.popleft()
try:
self.callback(topic, msg)
except Exception:
pass # don't kill worker on bad callback
class _Node:
__slots__ = ("children", "star", "subs", "hash_subs")
def __init__(self):
self.children: dict[str, _Node] = {}
self.star: Optional[_Node] = None
self.subs: list[_Sub] = []
self.hash_subs: list[_Sub] = []
class PubSub:
def __init__(self, separator: str = "."):
self._sep = separator
self._root = _Node()
self._subs: dict[int, tuple[_Sub, list[str]]] = {} # id -> (sub, pattern segments)
self._next_id = itertools.count(1)
self._lock = threading.RLock()
def subscribe(self, pattern: str, callback: Callable[[str, Any], None],
queue_capacity: int = 1024) -> int:
segments = pattern.split(self._sep)
sid = next(self._next_id)
sub = _Sub(sid, callback, queue_capacity)
with self._lock:
node = self._root
for i, seg in enumerate(segments):
if seg == "#":
if i != len(segments) - 1:
raise ValueError("# must be the last segment")
node.hash_subs.append(sub)
break
if seg == "*":
if node.star is None: node.star = _Node()
node = node.star
else:
node = node.children.setdefault(seg, _Node())
else:
node.subs.append(sub)
self._subs[sid] = (sub, segments)
sub.thread = threading.Thread(target=sub.run, daemon=True)
sub.thread.start()
return sid
def unsubscribe(self, sub_id: int) -> None:
with self._lock:
entry = self._subs.pop(sub_id, None)
if entry is None: return
sub, segments = entry
self._remove_from_trie(self._root, segments, 0, sub)
sub.stop()
sub.thread.join(timeout=1.0)
def _remove_from_trie(self, node: _Node, segments: list[str],
i: int, sub: _Sub) -> None:
if i == len(segments):
try: node.subs.remove(sub)
except ValueError: pass
return
seg = segments[i]
if seg == "#":
try: node.hash_subs.remove(sub)
except ValueError: pass
return
nxt = node.star if seg == "*" else node.children.get(seg)
if nxt is not None:
self._remove_from_trie(nxt, segments, i + 1, sub)
def publish(self, topic: str, message: Any) -> None:
segments = topic.split(self._sep)
with self._lock:
self._match(self._root, segments, 0, topic, message)
def _match(self, node: _Node, segments: list[str], i: int,
topic: str, msg: Any) -> None:
# # at this node matches everything below — fire now
for s in node.hash_subs:
s.deliver(topic, msg)
if i == len(segments):
for s in node.subs:
s.deliver(topic, msg)
return
seg = segments[i]
child = node.children.get(seg)
if child is not None:
self._match(child, segments, i + 1, topic, msg)
if node.star is not None:
self._match(node.star, segments, i + 1, topic, msg)
Tests
import unittest, time
class TestPubSub(unittest.TestCase):
def _collect(self, ps, sid_buf):
buf = []
def cb(t, m): buf.append((t, m))
sid = ps.subscribe(sid_buf, cb)
return sid, buf
def test_exact_match(self):
ps = PubSub()
sid, buf = self._collect(ps, "a.b")
ps.publish("a.b", 1); ps.publish("a.b.c", 2); ps.publish("a", 3)
time.sleep(0.05)
self.assertEqual(buf, [("a.b", 1)])
ps.unsubscribe(sid)
def test_star_one_segment(self):
ps = PubSub()
sid, buf = self._collect(ps, "a.*")
ps.publish("a.b", 1); ps.publish("a.c", 2)
ps.publish("a.b.c", 3); ps.publish("a", 4)
time.sleep(0.05)
self.assertEqual(sorted(buf), [("a.b", 1), ("a.c", 2)])
ps.unsubscribe(sid)
def test_hash_multi_segment(self):
ps = PubSub()
sid, buf = self._collect(ps, "a.#")
ps.publish("a", 0)
ps.publish("a.b", 1); ps.publish("a.b.c", 2); ps.publish("x.y", 3)
time.sleep(0.05)
# # matches zero or more, so a, a.b, a.b.c all match
self.assertEqual(sorted(buf), [("a", 0), ("a.b", 1), ("a.b.c", 2)])
ps.unsubscribe(sid)
def test_multiple_subscribers(self):
ps = PubSub()
sid1, b1 = self._collect(ps, "topic")
sid2, b2 = self._collect(ps, "topic")
ps.publish("topic", "msg")
time.sleep(0.05)
self.assertEqual(b1, [("topic", "msg")])
self.assertEqual(b2, [("topic", "msg")])
ps.unsubscribe(sid1); ps.unsubscribe(sid2)
Follow-up Questions
(7) Backpressure? Per-subscriber bounded queue with drop-oldest on full (shown). Alternatives: block the publisher (rejected — one slow subscriber stalls the world), drop-newest (loses recent state — rarely the right answer for pub/sub).
(3) Scale to N nodes? Distributed pub/sub is its own discipline. Models: (a) broker-based (Redis Pub/Sub, NATS): central broker fans out. (b) broker-less mesh (pgossip): peers gossip subscriptions; each publish goes to relevant peers. (c) partitioned log (Kafka): no fan-out; consumers tail logs. The trie matcher works locally in any model; the network layer is the harder design.
(2) Persist state? Pure pub/sub is volatile — late subscribers miss messages. To persist, layer a replay log: every publish appends to a durable log; new subscribers can opt-in to read from offset 0 or “latest”. This is essentially Kafka’s design.
(4) Observe / monitor? Per-subscriber drop count, queue depth, throughput. Subscription count gauge. Publish rate counter. p99 publish-to-deliver latency histogram (for the per-subscriber path).
(11) Configuration knobs? Per-subscription queue capacity, on-full policy. Global: max subscriptions, separator character. Knobs not to expose: trie internal layout.
Product Extension
MQTT brokers, AMQP exchanges, Redis Pub/Sub, NATS, ZeroMQ — all use topic-based routing with some wildcard syntax. Cloud Pub/Sub products (AWS SNS, GCP Pub/Sub, Azure Event Grid) add durability, retries, and ordering. The ergonomic difference between MQTT and AMQP wildcards (+/# vs */#) is purely syntactic.
Language/Runtime Follow-ups
- Python: this implementation. The per-subscriber thread approach scales to ~1000 subscribers; beyond, switch to an event loop (asyncio) with a single dispatcher coroutine.
- Java:
EventBus(Guava) is the lightweight in-process pub/sub. For wildcards, MQTT clients (Paho) or Kafka. - Go: channels per subscriber; idiomatic.
nats-serveris the production-grade Go choice. - C++: Boost.Signals2 is the in-process equivalent; no wildcards.
- JS/TS: Node’s
EventEmitteris the in-process equivalent; no wildcards. RxJS for reactive streams.
Common Bugs
- Synchronous callback dispatch from
publish— one slow subscriber stalls everyone. Always use per-subscriber worker threads. - Trie cleanup on unsubscribe: removing from the leaf but leaving empty intermediate nodes. Memory leak; matters at high churn.
#not at end: validate at subscription time.- Not propagating exceptions out of subscriber callbacks (silent failures). Log them.
- Race: subscribing during a publish — the subscription’s callback may or may not see the in-flight message. Document the semantics.
Debugging Strategy
For “missed message”: print the trie state at the matching point and the topic segments. For wildcard surprises, hand-trace the match: which child did we descend into? Did we visit *? Did # fire at the right level? For “callback didn’t run”: check that the worker thread is alive (sub.thread.is_alive()); a callback exception kills the worker if not caught.
Mastery Criteria
- Implemented topic trie + wildcard matching in <30 minutes.
- All four tests pass first run.
- Stated trie-vs-regex tradeoff (trie wins at scale; regex is simpler for few subscriptions).
- Articulated per-subscriber queue isolates slow subscribers.
- Answered follow-ups #2 (replay log), #3, #4, #7, #11.
- Compared topic vs content-based routing and stated when to use each.
Lab 13 — Hierarchical Timer Wheel
Goal
Implement a hierarchical timer wheel that supports O(1) amortized schedule and cancel operations for up to millions of timers. After this lab you should understand why a min-heap is wrong for high-throughput timer workloads, and be able to implement a single-level and a hierarchical timer wheel from blank screen in under 35 minutes.
Background Concepts
Many systems need to schedule callbacks for a future time: TCP retransmit timers, session timeouts, rate-limit reset, cron-style task scheduling. The classical data structure is a min-heap of (fire_time, callback): O(log N) schedule, O(log N) to fire (pop), O(N) to cancel (without indexing).
For the cases where N is small (thousands) and timers are infrequent, a heap is fine. But TCP at high QPS has millions of pending timers, most of which are cancelled before firing (the data arrives or the connection closes). For that workload, the heap is too slow.
The timer wheel (Varghese & Lauck, 1987) achieves O(1) schedule and O(1) cancel by bucketing timers by their fire time. Imagine a circular array of N slots; each slot holds a list of timers that fire when the wheel cursor reaches that slot. Each tick, advance the cursor and fire all timers in the current slot. Schedule is O(1): slot_index = (now + delay) % N. Cancel is O(1): remove from the slot’s list.
The single-level wheel works for delays up to N · tick_resolution. Beyond that, hierarchical wheels: minute wheel + hour wheel + day wheel, like an analog clock. When the minute hand sweeps past 60, advance the hour hand by one and re-bucket the timers in that hour slot into the minute wheel.
Interview Context
Timer wheel is a senior+ system-design-coding question, asked at networking/infrastructure companies (Cloudflare, Cilium, AWS networking, Datadog APM agents). The bar is high: implement at least the single-level wheel correctly; the hierarchical structure is for top candidates.
Problem Statement
Implement TimerWheel:
schedule(delay_seconds, callback) -> timer_idcancel(timer_id) -> booltick(now)— advance the cursor; fire all timers whose deadline passed.
Then extend to HierarchicalTimerWheel: 4 levels (e.g., 256 slots × 4 levels = 4 GB span at 1 ms tick).
Constraints
- Up to 10^7 active timers
- Tick resolution: 1 ms to 100 ms
- Schedule rate: 10^6 / sec
- Cancel rate: 10^6 / sec (most timers are cancelled before firing)
Clarifying Questions
- Tick resolution — fixed at construction, or adaptive? (Fixed.)
- Time source — caller supplies
now(testable) or wall clock? (Caller supplies; this also lets us simulate.) - Are callbacks fired on the tick thread, or queued? (On tick thread — simpler. For long callbacks, tick is slow; document.)
- What’s the max delay? (Single-level:
slots * tick; hierarchical: enormous.)
Examples
wheel = TimerWheel(slots=60, tick_ms=1000) # 1-second tick, 60-second range
fired = []
wheel.schedule(5, lambda: fired.append("5s"))
wheel.schedule(10, lambda: fired.append("10s"))
# Simulate ticks
for i in range(15):
wheel.tick(start_time + i)
# After 15 seconds, fired == ["5s", "10s"]
Initial Brute Force
A min-heap of (deadline, id, callback). Schedule = O(log N). Tick = O(K log N) for K firings. Cancel = O(N) (or O(log N) with a dict[id, heap_idx] and lazy deletion).
Brute Force Complexity
At 10^6 schedules/sec with N=10^7, schedule cost is log(10^7) ≈ 23 per — feasible but at the edge. The fatal weakness is cancel: O(N) without indexing. With lazy-deletion indexing, the heap grows unboundedly with cancelled-but-not-popped entries — memory leak.
Optimization Path
Single-level wheel: slots = circular array of bucket lists. schedule(delay): compute slot = (cursor + delay // tick) % slots, append to that slot’s list. tick: advance cursor, fire and clear slots[cursor]. Cancel: remove the timer from its slot’s list (O(1) if you store back-pointers).
Hierarchical: when delay > slots, place in higher-level wheel. When the lower wheel completes a full revolution, the next slot of the higher wheel is “cascaded” — its timers are re-bucketed into the lower wheel.
Final Expected Approach
Single-level: doubly-linked lists at each slot for O(1) cancel (timer holds prev/next pointers). Cursor advances on tick; fire all timers in the new slot; clear the slot.
Hierarchical: 4 wheels with slot counts [256, 64, 64, 64] (Linux kernel choice). Lower wheel ticks every 1 ms; carries every 256 ticks → upper level advances by one slot, and we cascade that slot (re-bucket its timers into the lower wheel based on remaining delay).
Data Structures Used
| Structure | Purpose |
|---|---|
| Array of doubly-linked lists | wheel slots |
dict[timer_id, Timer] | O(1) cancel lookup |
cursor: int | current slot |
| Multiple wheels | hierarchical |
Correctness Argument
Single-level firing: a timer scheduled at now + delay is placed in slot (cursor + delay // tick) % slots. tick is called once per tick time unit; after delay // tick calls, the cursor reaches the timer’s slot and fires it. Provided delay < slots * tick, this is exact.
Cancel: O(1) splice in the doubly-linked list, O(1) lookup via dict.
Hierarchical correctness: when the lower wheel completes a revolution (cursor wraps), the next upper-wheel slot is cascaded: each timer in it has its remaining delay computed against the new “now” and is placed in the appropriate lower-wheel slot. Because the cascade happens just before the next revolution, the timer fires at the same wall-clock time it would have in a single-large-wheel implementation.
Complexity
schedule: O(1)cancel: O(1)tick: O(K) for K firings, plus amortized O(1) cascade- Space: O(slots + active timers)
Implementation Requirements
import itertools
from typing import Callable, Optional
class _Timer:
__slots__ = ("tid", "deadline_tick", "callback", "prev", "next", "slot")
def __init__(self, tid, deadline_tick, callback):
self.tid = tid
self.deadline_tick = deadline_tick
self.callback = callback
self.prev = self.next = None
self.slot: Optional[list] = None # back-ref to bucket for O(1) cancel
class _Bucket:
__slots__ = ("head", "tail")
def __init__(self):
self.head = _Timer(None, None, None) # sentinel
self.tail = _Timer(None, None, None)
self.head.next = self.tail; self.tail.prev = self.head
def append(self, t: _Timer) -> None:
t.prev = self.tail.prev; t.next = self.tail
self.tail.prev.next = t; self.tail.prev = t
t.slot = self
def remove(self, t: _Timer) -> None:
t.prev.next = t.next; t.next.prev = t.prev
t.slot = None; t.prev = t.next = None
def drain(self) -> list[_Timer]:
out = []
n = self.head.next
while n is not self.tail:
nxt = n.next
n.prev = n.next = None; n.slot = None
out.append(n); n = nxt
self.head.next = self.tail; self.tail.prev = self.head
return out
class TimerWheel:
def __init__(self, slots: int = 256, tick_seconds: float = 0.001):
self._slots = [_Bucket() for _ in range(slots)]
self._n_slots = slots
self._tick = tick_seconds
self._cursor = 0
self._current_tick = 0
self._timers: dict[int, _Timer] = {}
self._next_id = itertools.count(1)
def schedule(self, delay_seconds: float, callback: Callable) -> int:
ticks = max(1, int(delay_seconds / self._tick))
if ticks >= self._n_slots:
raise ValueError("delay exceeds wheel range; use HierarchicalTimerWheel")
deadline = self._current_tick + ticks
slot = deadline % self._n_slots
t = _Timer(next(self._next_id), deadline, callback)
self._slots[slot].append(t)
self._timers[t.tid] = t
return t.tid
def cancel(self, timer_id: int) -> bool:
t = self._timers.pop(timer_id, None)
if t is None or t.slot is None:
return False
t.slot.remove(t)
return True
def tick(self) -> None:
self._current_tick += 1
self._cursor = (self._cursor + 1) % self._n_slots
bucket = self._slots[self._cursor]
for t in bucket.drain():
self._timers.pop(t.tid, None)
try: t.callback()
except Exception: pass
class HierarchicalTimerWheel:
"""4 levels: each level has 256 slots; tick = lower-level period."""
def __init__(self, levels: int = 4, slots_per_level: int = 256,
tick_seconds: float = 0.001):
self._levels = [
[_Bucket() for _ in range(slots_per_level)] for _ in range(levels)
]
self._cursors = [0] * levels
self._n = slots_per_level
self._tick = tick_seconds
self._current_tick = 0
self._timers: dict[int, tuple[int, int, _Timer]] = {} # id -> (level, slot, t)
self._next_id = itertools.count(1)
def schedule(self, delay_seconds: float, callback: Callable) -> int:
ticks = max(1, int(delay_seconds / self._tick))
deadline = self._current_tick + ticks
return self._place(deadline, callback)
def _place(self, deadline_tick: int, callback: Callable) -> int:
ticks_from_now = deadline_tick - self._current_tick
# Find the lowest level that can hold this delay.
capacity = self._n
level = 0
while ticks_from_now >= capacity and level < len(self._levels) - 1:
level += 1; capacity *= self._n
if ticks_from_now >= capacity:
raise ValueError("delay exceeds hierarchical range")
# Slot at this level:
per_slot = capacity // self._n
slot = (self._cursors[level] + ticks_from_now // per_slot) % self._n
t = _Timer(next(self._next_id), deadline_tick, callback)
self._levels[level][slot].append(t)
self._timers[t.tid] = (level, slot, t)
return t.tid
def cancel(self, timer_id: int) -> bool:
entry = self._timers.pop(timer_id, None)
if entry is None: return False
_, _, t = entry
if t.slot is not None: t.slot.remove(t)
return True
def tick(self) -> None:
self._current_tick += 1
# Advance level 0
self._cursors[0] = (self._cursors[0] + 1) % self._n
# Fire level-0 current slot
bucket = self._levels[0][self._cursors[0]]
for t in bucket.drain():
self._timers.pop(t.tid, None)
try: t.callback()
except Exception: pass
# Cascade if we wrapped
for lvl in range(1, len(self._levels)):
if self._cursors[lvl - 1] != 0:
break
self._cursors[lvl] = (self._cursors[lvl] + 1) % self._n
for t in self._levels[lvl][self._cursors[lvl]].drain():
self._timers.pop(t.tid, None)
self._place(t.deadline_tick, t.callback)
Tests
import unittest
class TestTimerWheel(unittest.TestCase):
def test_basic(self):
w = TimerWheel(slots=10, tick_seconds=1.0)
fired = []
w.schedule(3, lambda: fired.append("a"))
w.schedule(5, lambda: fired.append("b"))
for _ in range(3): w.tick()
self.assertEqual(fired, ["a"])
for _ in range(2): w.tick()
self.assertEqual(fired, ["a", "b"])
def test_cancel(self):
w = TimerWheel(slots=10, tick_seconds=1.0)
fired = []
tid = w.schedule(2, lambda: fired.append("x"))
self.assertTrue(w.cancel(tid))
for _ in range(5): w.tick()
self.assertEqual(fired, [])
def test_hierarchical_long_delay(self):
w = HierarchicalTimerWheel(levels=3, slots_per_level=4, tick_seconds=1.0)
# Range = 4 * 4 * 4 = 64 ticks
fired = []
w.schedule(50, lambda: fired.append("late"))
for _ in range(50): w.tick()
self.assertEqual(fired, ["late"])
Follow-up Questions
(11) Configuration knobs? slots, tick_seconds, levels (for hierarchical). Tick resolution chooses your scheduling granularity vs CPU cost: 1 ms means 1000 ticks/sec which is fine; 100 µs is 10000 ticks/sec which adds CPU. Slots per level: 256 is a good default (1 byte addressable, common L1 cache friendly).
(7) Backpressure? Timer firing is on the tick thread. If callbacks are slow, ticks fall behind real time; subsequent timers fire late. Mitigation: dispatch callbacks to a thread pool from tick. Document the soft real-time semantics (“fires within 1 tick of deadline barring slow callbacks”).
(4) Observe / monitor? Active timer count (gauge), schedule rate (counter), cancel rate (counter), fire rate (counter), tick latency p99 (histogram — should be near zero; spikes mean slow callbacks).
(8) Partial failure? A callback that raises kills the tick if not caught. Always wrap with try/except and log.
(13) Poison pill? A callback that takes 1 second on a 1-ms wheel: all subsequent ticks pile up. Same mitigation as #7: dispatch to thread pool, or set a per-callback timeout.
Product Extension
Linux’s kernel uses a 4-level hierarchical timer wheel for setitimer and TCP retransmits. Netty’s HashedWheelTimer is the canonical Java implementation (single-level wheel; “hashed” means linked-list bucket per slot). The Linux choice of 256/64/64/64 slots covers ~5 days at 1 ms tick — enough for any kernel-level timer.
Language/Runtime Follow-ups
- Python: this implementation. For sub-millisecond, switch to a C extension or use
asyncio’s event loop scheduler (which uses a heap, but for low-N is fine). - Java: Netty’s
HashedWheelTimer(single-level) andJCToolsfor lock-free variants.ScheduledThreadPoolExecutoris heap-based and slower at scale. - Go:
time.AfterFuncuses a heap internally (fine for low-N). For high-N,github.com/RussellLuo/timingwheelis a clean library. - C++: the textbook reference; libuv and Boost both have wheel-based timer implementations.
- JS/TS: Node’s timer subsystem uses a hash-bucket-by-time-and-context structure — not exactly a wheel but similar idea.
Common Bugs
- Forgetting modulo on slot indexing — array out of bounds when delay wraps around.
- Cascade firing on every tick instead of only when wrapping. Catastrophic slowdown.
- Forgetting to drain the bucket before clearing — callbacks lost.
- Holding callbacks in slot lists and in the
_timersdict; failing to remove from one when removing from the other. - Scheduling
delay=0: should fire on next tick, not “now”. Clamp to ≥ 1 tick.
Debugging Strategy
Print the wheel state (occupied slots and their counts) after each tick. For “missed firing” bugs, walk the slot indexing and verify the placement formula. For hierarchical cascade bugs, set very small slot counts (4 × 4 × 4) and hand-trace the wrap.
Mastery Criteria
- Implemented single-level wheel in <20 minutes; hierarchical in <35.
- All three tests pass.
- Stated heap vs wheel tradeoff (heap O(log N), wheel O(1); wheel wins at high-N high-cancel).
- Articulated cascade mechanism for hierarchical wheels.
- Answered follow-ups #4, #7, #11, #13.
- Identified that real-world systems (Linux, Netty) use this exact structure.
Lab 14 — Persistent KV Store
Goal
Implement an in-memory key-value store with TTL, snapshot + write-ahead-log (WAL) persistence, and crash recovery. After this lab you should be able to design and implement a Redis-shaped local store in under 40 minutes and articulate the durability tradeoffs.
Background Concepts
A persistent KV store has two storage paths:
- In-memory state: a
dict[key, value](plus TTL bookkeeping). Hot path: O(1) read/write. - Durable state: writes go to a WAL (append-only log of mutations); periodically a snapshot captures the current state. On boot, recovery = load latest snapshot + replay WAL since.
The four standard durability levels (each a different fsync policy):
- No persistence: pure in-memory. Lost on crash.
- WAL with no fsync: writes to OS buffer; lost on power-cut, survives process crash.
- WAL fsync per write: durable per write, slow (one syscall per op).
- WAL fsync every N ms: hybrid — bounded data loss in exchange for throughput.
Redis offers exactly these as appendfsync no/everysec/always. The interview answer is “explain the spectrum and pick a default that matches the workload”.
Interview Context
This problem hits at infrastructure / database companies and at any senior coding round that wants to test storage fundamentals. The interviewer wants: snapshot + WAL design, fsync tradeoff articulation, working code that survives a simulated crash.
Problem Statement
Implement KVStore:
put(key, value, ttl_seconds=None)— store with optional TTL.get(key) -> value | Nonedelete(key) -> boolsnapshot(path)— write current state.- Recovery: on construction with
wal_pathandsnapshot_path, replay snapshot + WAL.
Constraints
- 10^7 keys
- 10^5 ops / second
- Crash recovery within seconds
- Bounded memory (configurable max)
Clarifying Questions
- TTL granularity? (Seconds is fine for most workloads.)
- fsync policy? (Configurable: none / per-write / every-N-ms.)
- Snapshot format: text or binary? (Binary is faster, smaller; pick
pickleormsgpack.) - Concurrent reads during snapshot? (Often a follow-up; default block during snapshot.)
- Single-threaded or concurrent? (Single-threaded simplifies; lock for concurrency.)
Examples
kv = KVStore(wal_path="wal.log", snapshot_path="snap.pkl")
kv.put("user:1", {"name": "Alice"})
kv.put("session:42", "tok", ttl_seconds=60)
kv.get("user:1") -> {"name": "Alice"}
# … crash …
kv2 = KVStore(wal_path="wal.log", snapshot_path="snap.pkl")
kv2.get("user:1") -> {"name": "Alice"} # recovered
kv2.snapshot()
# After snapshot: WAL is rotated (truncated)
Initial Brute Force
dict[key, value]. No persistence. Lost on crash.
Brute Force Complexity
Per op: O(1). Memory: O(N). Durability: zero.
Optimization Path
Add WAL: append (op, key, value, ttl) per mutation. On boot, replay. Add periodic snapshot: serialize full state; truncate WAL. Add TTL: a dict[key, expires_at] and lazy expiry on get.
The cost is: O(WAL append) per write (serialization + file write); O(snapshot size) per snapshot; O(WAL size) per recovery. Throughput depends on fsync policy.
Final Expected Approach
In-memory dict for values + dict for TTL deadlines + a binary log file. Operations: log first, then update memory (“write-ahead”). Snapshot: pickle the in-memory state to a temp file, then atomic rename + truncate WAL. Recovery: load snapshot, replay WAL with file-based offset.
Data Structures Used
| Structure | Purpose |
|---|---|
dict[K, V] | hot key-value store |
dict[K, float] | TTL deadlines |
| WAL file (append-only) | durability |
| Snapshot file (pickle) | bounded recovery time |
Lock | single-threaded mutation under multi-thread access |
Correctness Argument
Durability: every mutation is appended to the WAL before updating the in-memory state. After the WAL append (and fsync, if configured), the mutation is durable. On crash, recovery replays exactly what was logged.
Atomicity of put: WAL append is atomic at the bytestream level (write syscalls of small bytes are atomic on Linux for ≤ 4 KB). Snapshot is atomic via write to tmp; fsync tmp; rename(tmp, snap).
Recovery correctness: applying snapshot first, then replaying WAL entries in order, reconstructs exactly the pre-crash state. The only loss is mutations that were in OS buffers but unsynced at crash time — bounded by fsync policy.
TTL: lazy expiry on get (check now >= deadline, delete if so). This is correct as long as we don’t return values past their TTL. Stale entries in memory are GC’d on access; a periodic background sweeper handles unused expired entries.
Complexity
put: O(1) memory + O(log entry size) diskget: O(1)snapshot: O(N) state size- Recovery: O(snapshot + WAL since snapshot)
Implementation Requirements
import os, pickle, time, threading
from typing import Any, Optional
class KVStore:
def __init__(self, wal_path: str = "kv.wal",
snapshot_path: str = "kv.snap",
fsync: str = "every_sec"):
self._wal_path = wal_path
self._snap_path = snapshot_path
self._fsync = fsync # "none" | "per_write" | "every_sec"
self._data: dict = {}
self._ttl: dict = {}
self._lock = threading.RLock()
self._wal_fp = None
self._last_fsync = time.monotonic()
self._recover()
self._wal_fp = open(self._wal_path, "ab", buffering=0)
if self._fsync == "every_sec":
self._fsync_thread = threading.Thread(target=self._fsync_loop, daemon=True)
self._fsync_thread.start()
def _recover(self) -> None:
# 1. Load snapshot if present
if os.path.exists(self._snap_path):
with open(self._snap_path, "rb") as f:
self._data, self._ttl = pickle.load(f)
# 2. Replay WAL since snapshot
if os.path.exists(self._wal_path):
with open(self._wal_path, "rb") as f:
while True:
try:
entry = pickle.load(f)
except (EOFError, pickle.UnpicklingError):
break
self._apply(entry)
# Sweep expired
now = time.time()
for k in list(self._ttl):
if self._ttl[k] <= now:
self._data.pop(k, None); self._ttl.pop(k, None)
def _apply(self, entry: dict) -> None:
op = entry["op"]
if op == "put":
self._data[entry["k"]] = entry["v"]
if entry.get("ttl") is not None:
self._ttl[entry["k"]] = entry["ttl"]
else:
self._ttl.pop(entry["k"], None)
elif op == "del":
self._data.pop(entry["k"], None)
self._ttl.pop(entry["k"], None)
def _wal_write(self, entry: dict) -> None:
buf = pickle.dumps(entry)
self._wal_fp.write(buf)
if self._fsync == "per_write":
self._wal_fp.flush()
os.fsync(self._wal_fp.fileno())
def _fsync_loop(self) -> None:
while True:
time.sleep(1.0)
with self._lock:
if self._wal_fp:
self._wal_fp.flush()
os.fsync(self._wal_fp.fileno())
def put(self, key, value, ttl_seconds: Optional[float] = None) -> None:
deadline = (time.time() + ttl_seconds) if ttl_seconds else None
with self._lock:
self._wal_write({"op": "put", "k": key, "v": value, "ttl": deadline})
self._data[key] = value
if deadline is not None:
self._ttl[key] = deadline
else:
self._ttl.pop(key, None)
def get(self, key) -> Any:
with self._lock:
deadline = self._ttl.get(key)
if deadline is not None and time.time() >= deadline:
self._wal_write({"op": "del", "k": key})
self._data.pop(key, None); self._ttl.pop(key, None)
return None
return self._data.get(key)
def delete(self, key) -> bool:
with self._lock:
existed = key in self._data
self._wal_write({"op": "del", "k": key})
self._data.pop(key, None); self._ttl.pop(key, None)
return existed
def snapshot(self) -> None:
with self._lock:
tmp = self._snap_path + ".tmp"
with open(tmp, "wb") as f:
pickle.dump((self._data, self._ttl), f)
f.flush(); os.fsync(f.fileno())
os.rename(tmp, self._snap_path)
# Rotate WAL
self._wal_fp.close()
open(self._wal_path, "wb").close()
self._wal_fp = open(self._wal_path, "ab", buffering=0)
def close(self) -> None:
with self._lock:
if self._wal_fp:
self._wal_fp.flush(); os.fsync(self._wal_fp.fileno())
self._wal_fp.close(); self._wal_fp = None
Tests
import unittest, tempfile, os, time
class TestKV(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.mkdtemp()
self.wal = os.path.join(self.tmp, "wal.log")
self.snap = os.path.join(self.tmp, "snap.pkl")
def tearDown(self):
import shutil; shutil.rmtree(self.tmp)
def test_basic(self):
kv = KVStore(self.wal, self.snap, fsync="none")
kv.put("a", 1); kv.put("b", "two")
self.assertEqual(kv.get("a"), 1)
self.assertEqual(kv.get("b"), "two")
kv.delete("a")
self.assertIsNone(kv.get("a"))
kv.close()
def test_ttl(self):
kv = KVStore(self.wal, self.snap, fsync="none")
kv.put("k", "v", ttl_seconds=0.1)
self.assertEqual(kv.get("k"), "v")
time.sleep(0.15)
self.assertIsNone(kv.get("k"))
kv.close()
def test_recovery_from_wal(self):
kv = KVStore(self.wal, self.snap, fsync="per_write")
kv.put("x", "y")
kv.close()
# Simulate crash and restart
kv2 = KVStore(self.wal, self.snap, fsync="none")
self.assertEqual(kv2.get("x"), "y")
kv2.close()
def test_snapshot_rotates_wal(self):
kv = KVStore(self.wal, self.snap, fsync="none")
for i in range(100):
kv.put(f"k{i}", i)
wal_size_before = os.path.getsize(self.wal)
kv.snapshot()
wal_size_after = os.path.getsize(self.wal)
self.assertGreater(wal_size_before, wal_size_after)
kv.close()
# Recover
kv2 = KVStore(self.wal, self.snap, fsync="none")
self.assertEqual(kv2.get("k99"), 99)
kv2.close()
Follow-up Questions
(2) Persist state across restarts? That’s what we built. The four fsync levels and their tradeoffs are the answer-bearing detail: per_write (durable per op, slow); every_sec (≤1 sec data loss, fast — Redis default); none (lose on crash, fastest).
(10) Consistency model? Linearizable in a single process under the lock. Across processes (or replicas), this becomes a consensus problem — Raft / Paxos. The KV store is the data plane; consensus is the control plane.
(8) Partial failure? Crash mid-write: _wal_write buffers a partial entry — pickle.UnpicklingError on recovery; we ignore the trailing junk (caught above). For OS-level partial writes (rare on Linux for ≤ 4 KB), a per-entry checksum (CRC32) catches them.
(9) Eviction / cleanup? TTL provides automatic cleanup, but expired keys still in memory consume RAM until accessed. Background sweeper: periodically scan _ttl for expired keys and delete. For unbounded growth, add an LRU/LFU policy on top: when memory > threshold, evict by policy.
(11) Configuration knobs? fsync policy, snapshot_interval, max_memory_bytes, eviction_policy. Knobs not to expose: pickle protocol (use latest).
(12) Shutdown? Graceful: flush WAL, fsync, close file. The close method ensures durability up to the last write.
Product Extension
Redis (RDB = snapshot, AOF = WAL); RocksDB / LevelDB (LSM trees with WAL + memtable + SSTable); Memcached (no persistence — pure cache); etcd / ZooKeeper (snapshot + WAL + Raft for consensus). The pattern you wrote here is the foundation; SSTable + LSM is the next-level optimization for write-heavy + range-query workloads.
Language/Runtime Follow-ups
- Python: pickle is fine for the snapshot format but not version-safe; for production, use msgpack or Protocol Buffers.
- Java:
RandomAccessFilefor the WAL; Java serialization for snapshot (also fragile — prefer Avro or Protobuf). - Go:
bufio.Writeroveros.File;gobfor snapshot. BadgerDB and BoltDB are production-grade Go KV stores. - C++: write your own framing or use Cap’n Proto. RocksDB is the canonical reference (C++ implementation of LSM + WAL).
- JS/TS: rare in Node; use
level(LevelDB binding) instead of rolling your own.
Common Bugs
- Updating in-memory state before WAL append: lose the durability guarantee.
- fsync per write but on the wrong fd (forgetting
flush()beforefsync()). - Snapshot writes to the actual snapshot path before fsync — if crash mid-write, snapshot is corrupt. Always write-tmp + fsync + rename.
- WAL not rotated on snapshot — recovery replays the entire history every time, even after snapshot.
- TTL stored as duration instead of absolute time — restart shifts deadlines.
Debugging Strategy
For “lost data after restart” bugs: tail the WAL with a pickle reader and check that the missing key was logged. For corrupt-snapshot bugs: check that os.rename is on the same filesystem (cross-fs rename is not atomic).
Mastery Criteria
-
Implemented
KVStorewith WAL + snapshot + recovery in <40 minutes. - All four tests pass.
- Articulated three fsync levels and their tradeoffs without prompting.
- Stated WAL-before-memory as the durability invariant.
- Answered follow-ups #2, #8 (partial-write tolerance), #9, #10 (single vs replicated consistency), #11.
- Compared snapshot+WAL vs LSM tree at a high level.
Lab 15 — Retry With Exponential Backoff and Jitter
Goal
Implement a reusable retry(fn, policy) primitive that retries a callable on failure with exponential backoff plus decorrelated jitter, bounded by max attempts and total deadline, with an explicit retryable-error predicate so non-retryable errors fail fast. After this lab you should be able to write a production-shaped retry helper from a blank screen in <15 minutes and articulate why naive sleep(2 ** attempt) is wrong in <60 seconds.
Background Concepts
A retry primitive has four orthogonal knobs: (a) how many times to retry (max attempts and / or total deadline), (b) how long to wait between attempts (the backoff schedule), (c) which errors are retryable (a predicate), and (d) what to do on final failure (raise, return a sentinel, surface diagnostics). The non-trivial knob is (b). Naive exponential backoff wait = base * 2 ** attempt causes a thundering herd: when a downstream service recovers, every retrying client wakes simultaneously and re-overloads it. The fix is jitter: randomize the wait. The two industry-standard schedules are full jitter (wait = uniform(0, base * 2 ** attempt)) and decorrelated jitter (wait = uniform(base, prev_wait * 3), capped at max_wait). Decorrelated jitter is preferred when retries cluster across many clients because its waits are less correlated across attempts.
The total deadline matters as much as the attempt count. A 5-attempt schedule with base=1s, cap=30s can spend up to two minutes blocked — unacceptable for a request-path retry. Production retry helpers always take a deadline.
Interview Context
This is a 20-minute warmup at Stripe, Uber, Cloudflare, and any team whose service makes downstream calls. It’s also a frequent follow-up to the rate-limiter and circuit-breaker labs. Candidates who write for i in range(5): try: return fn() except: time.sleep(2 ** i) get a partial credit; candidates who name jitter, deadline, retryable-error predicate, and the relationship to the circuit breaker (Lab 16) get a strong signal.
Problem Statement
Implement retry(fn, max_attempts, base_delay, max_delay, deadline_s, is_retryable, jitter='decorrelated') that calls fn() repeatedly until it succeeds or the policy gives up. On non-retryable exceptions, fail immediately. On retryable exceptions, sleep according to the schedule and try again. On exceeding max_attempts or deadline_s, raise the last exception wrapped in a RetryExhausted.
Constraints
max_attempts≥ 1 (1 means “no retries”; the function is called at most once).base_delay> 0,max_delay≥base_delay.deadline_smay beNone(no deadline) or a positive float (wall-clock seconds fromretry()invocation).is_retryable: Exception -> boolmust be a pure function.- The implementation must not busy-spin and must respect both the per-attempt cap and the deadline (whichever fires first).
Clarifying Questions
- Is the deadline measured from
retry()invocation or from the first failure? (From invocation — simpler reasoning.) - Should
fn()be called at least once even if the deadline is already past at start? (Yes — at least one attempt.) - Should we sleep after the final attempt? (No — pointless.)
- Does
is_retryableapply to the last attempt’s exception, or do we always re-raise the last? (Re-raise the last; non-retryable short-circuits.) - Synchronous or async? (Both — implement sync first, async variant in follow-ups.)
- Should we surface the attempt count and total elapsed time in the wrapped exception? (Yes — operational visibility.)
Examples
retry(lambda: http_get(url), max_attempts=5, base_delay=0.1, max_delay=10, deadline_s=30,
is_retryable=lambda e: isinstance(e, (TimeoutError, ConnectionError)))
# → returns the response if any attempt succeeds within 30s and 5 tries.
# → raises RetryExhausted("timeout", attempts=5, elapsed=12.3s) on timeout.
# → raises ValueError immediately if fn() raises ValueError (non-retryable).
Initial Brute Force
def retry_naive(fn, max_attempts):
for i in range(max_attempts):
try:
return fn()
except Exception:
if i == max_attempts - 1:
raise
time.sleep(2 ** i)
This is what most candidates write first. It has all four bugs listed above: no jitter (thundering herd), no deadline (unbounded wait), no retryable predicate (retries on programming errors), no cap on max_delay.
Brute Force Complexity
Time: dominated by sleeps; up to Σ 2^i ≈ 2^max_attempts seconds in the worst case. For max_attempts=10, that’s 17 minutes. Space: O(1).
Optimization Path
Add (1) max_delay cap → bounds per-attempt sleep, (2) deadline_s total cap → bounds end-to-end blocking, (3) is_retryable predicate → fast-fails on programmer errors, (4) jitter → spreads herd, (5) structured exception with diagnostics → operational legibility. Each addition is a one-knob change; together they take the primitive from “buggy in production” to “shippable”.
Final Expected Approach
Loop up to max_attempts times. Track wall-clock start. On each attempt, call fn(). On success, return the value. On failure, check is_retryable; if false, re-raise. If we’re at the last attempt or past the deadline, raise RetryExhausted. Otherwise compute the next sleep using the chosen jitter scheme, clip to remaining-deadline so we don’t oversleep, and time.sleep(wait). Log each attempt.
Data Structures Used
- A monotonic clock reference (
time.monotonic()) to compute deadlines — wall-clock can jump. - A small
RetryExhaustedexception class carryingattempts,elapsed,last_exception. - An optional
Loggerfor per-attempt diagnostics (don’t print; inject a logger).
Correctness Argument
We make at most max_attempts calls to fn (loop bound). We sleep between attempts but never after the final one (the loop returns or raises before sleeping past the last attempt). We respect deadline_s by computing remaining = deadline - elapsed and clipping the sleep; if remaining ≤ 0 we raise immediately. Non-retryable exceptions short-circuit by re-raising before the sleep. The exception we surface is always the last underlying failure, wrapped with diagnostics.
Complexity
| Aspect | Cost |
|---|---|
| Wall-clock | bounded by min(deadline_s, Σ wait_i) |
| CPU per failed attempt | O(1) plus fn’s own cost |
| Memory | O(1) |
Implementation Requirements
A complete working implementation is required.
import random
import time
from dataclasses import dataclass
from typing import Callable, Optional, TypeVar
T = TypeVar("T")
class RetryExhausted(Exception):
def __init__(self, message: str, attempts: int, elapsed: float, last_exception: BaseException):
super().__init__(f"{message} (attempts={attempts}, elapsed={elapsed:.2f}s)")
self.attempts = attempts
self.elapsed = elapsed
self.last_exception = last_exception
@dataclass
class RetryPolicy:
max_attempts: int = 5
base_delay: float = 0.1
max_delay: float = 30.0
deadline_s: Optional[float] = None
jitter: str = "decorrelated" # "decorrelated" | "full" | "none"
is_retryable: Callable[[BaseException], bool] = lambda e: True
def __post_init__(self):
if self.max_attempts < 1:
raise ValueError("max_attempts must be >= 1")
if self.base_delay <= 0 or self.max_delay < self.base_delay:
raise ValueError("invalid delay bounds")
def _next_wait(policy: RetryPolicy, attempt: int, prev_wait: float) -> float:
if policy.jitter == "none":
w = min(policy.base_delay * (2 ** attempt), policy.max_delay)
elif policy.jitter == "full":
cap = min(policy.base_delay * (2 ** attempt), policy.max_delay)
w = random.uniform(0, cap)
elif policy.jitter == "decorrelated":
w = min(random.uniform(policy.base_delay, max(prev_wait, policy.base_delay) * 3),
policy.max_delay)
else:
raise ValueError(f"unknown jitter scheme: {policy.jitter}")
return w
def retry(fn: Callable[[], T], policy: RetryPolicy, *, sleep=time.sleep, clock=time.monotonic) -> T:
start = clock()
last_exc: Optional[BaseException] = None
prev_wait = policy.base_delay
for attempt in range(policy.max_attempts):
try:
return fn()
except BaseException as e:
last_exc = e
if not policy.is_retryable(e):
raise
if attempt == policy.max_attempts - 1:
break
elapsed = clock() - start
if policy.deadline_s is not None and elapsed >= policy.deadline_s:
break
wait = _next_wait(policy, attempt, prev_wait)
if policy.deadline_s is not None:
wait = min(wait, max(0.0, policy.deadline_s - elapsed))
if wait > 0:
sleep(wait)
prev_wait = wait
elapsed = clock() - start
raise RetryExhausted("retry exhausted", attempt + 1, elapsed, last_exc) from last_exc
sleep and clock are dependency-injected so tests do not have to wait real time.
Tests
def test_succeeds_first_try():
assert retry(lambda: 42, RetryPolicy(max_attempts=3)) == 42
def test_succeeds_after_failures():
n = {"i": 0}
def fn():
n["i"] += 1
if n["i"] < 3: raise TimeoutError()
return "ok"
assert retry(fn, RetryPolicy(max_attempts=5, base_delay=0.001)) == "ok"
assert n["i"] == 3
def test_non_retryable_short_circuits():
n = {"i": 0}
def fn():
n["i"] += 1
raise ValueError("bad")
policy = RetryPolicy(is_retryable=lambda e: not isinstance(e, ValueError))
try: retry(fn, policy)
except ValueError: pass
assert n["i"] == 1
def test_exhaustion_wraps_exception():
def fn(): raise TimeoutError("nope")
try: retry(fn, RetryPolicy(max_attempts=2, base_delay=0.001))
except RetryExhausted as e:
assert e.attempts == 2
assert isinstance(e.last_exception, TimeoutError)
def test_deadline_respected():
fake_time = [0.0]
def fn(): raise TimeoutError()
sleeps = []
def fake_sleep(t): sleeps.append(t); fake_time[0] += t
def fake_clock(): return fake_time[0]
try:
retry(fn, RetryPolicy(max_attempts=100, base_delay=1, deadline_s=5, jitter="none"),
sleep=fake_sleep, clock=fake_clock)
except RetryExhausted as e:
assert e.elapsed <= 5.001
Follow-up Questions
- How would you make it thread-safe? The function is reentrant — no shared state across calls. The injected
sleepandclockshould themselves be thread-safe (the stdlib ones are). Per-call state (attempt counter, prev_wait) is local. No locks needed. - How would you observe and monitor it? Emit (a)
retry.attemptscounter labeled by callsite and outcome (success_first_try,success_after_retry,exhausted,non_retryable), (b)retry.elapsedhistogram, (c)retry.attempt_counthistogram. Log per-attempt at DEBUG, per-final-failure at WARN. - How would you handle a poison-pill input? A request that always raises a retryable error wastes the deadline on every retry. Wrap repeated callers behind a circuit breaker (Lab 16); after N consecutive
RetryExhausteds, open the breaker and fail fast for a cooldown period. - What configuration knobs would you expose?
max_attempts,base_delay,max_delay,deadline_s,jitterstrategy,is_retryablepredicate. Defaults: 5 / 100ms / 30s / None / decorrelated /lambda e: True. Don’t expose internal multipliers (the 3× in decorrelated jitter) — they’re stable and tuning them in production is a smell. - How would you test it deterministically? Inject
sleepandclock; advance fake time inside the fakesleep. Seedrandomfor reproducible jitter. The test for the deadline above uses this pattern. - What is the relationship to the circuit breaker? A retry without a circuit breaker is dangerous: if the downstream is fully down, every caller retries the full schedule, multiplying load. The right composition is
circuit_breaker(retry(fn))— the breaker short-circuits the retry once it has seen enough failures.
Product Extension
Retry primitives are the workhorse of every microservice’s outbound RPC layer. AWS SDK, Google Cloud SDK, and gRPC all ship retry helpers; their default schedules are decorrelated jitter with deadlines. The is_retryable predicate in production is the hardest knob: HTTP 5xx is usually retryable, 4xx usually is not, but 429 is retryable with Retry-After honored. Lift this complexity into the predicate.
Language/Runtime Follow-ups
- Python: as above. For async, swap
time.sleepforasyncio.sleepand makeretryanasync def. - Java: use
Resilience4jorFailsafein production. Hand-rolled: aRetryPolicybuilder, aCallable<T>argument,Thread.sleep(orScheduledExecutorService.schedulein async). - Go: a function
Retry(ctx context.Context, fn func() error, policy Policy) error. Usetime.NewTimerso actx.Done()can cancel mid-sleep. Cancellation is the deadline mechanism. - C++:
std::this_thread::sleep_forandstd::chronofor the deadline. Pass a stop-token to support cancellation. - JS/TS:
await new Promise(r => setTimeout(r, ms)). The retry function isasync. UseAbortSignalfor the deadline.
Common Bugs
- Sleeping after the final attempt — wastes wall-clock.
- Using
time.time()instead oftime.monotonic()— wall-clock can jump backwards across NTP corrections, causing negativeelapsedand crashes. - Catching
BaseExceptionand swallowingKeyboardInterrupt/SystemExit— never make these retryable. Either narrow the catch or have the predicate exclude them. - Computing the next wait before the deadline check — you sleep past the deadline. Always check elapsed first.
- Forgetting to clip
waittoremaining = deadline - elapsed— a 30s sleep when only 2s of deadline remain. - Not seeding
randomdeterministically in tests — flaky test failures.
Debugging Strategy
When retries don’t fire: print is_retryable(e) for the actual exception; assert it returns True. When they fire too long: print attempt, wait, and clock() - start per attempt — the bug is almost always a missing deadline check or an uncapped jitter computation. When tests are flaky: confirm sleep is injected (no real sleeps in unit tests) and random.seed(0) at the top of the test.
Mastery Criteria
- Wrote the brute force naive retry in <2 minutes from cold start.
-
Added
max_delay,deadline_s,is_retryable, and jitter incrementally, justifying each. - Wrote both full-jitter and decorrelated-jitter formulas from memory.
-
Stated the difference between
time.time()andtime.monotonic()and which to use here. -
Wrote deterministic tests using injected
sleepandclock. - Articulated the retry+circuit-breaker composition in <60 seconds.
- Solved this from a blank screen in <15 minutes including 5 unit tests.
-
Listed the four bugs in the naive
for i in range: sleep(2**i)retry without prompting.
Lab 16 — Circuit Breaker
Goal
Implement a thread-safe circuit breaker with three states — CLOSED, OPEN, HALF_OPEN — that protects a downstream call by failing fast once a sliding-window failure rate threshold is crossed, then probes for recovery after a cooldown. After this lab you should be able to draw the state diagram, name every transition, write the implementation in <25 minutes, and answer “what’s the difference between a retry and a circuit breaker” in <30 seconds.
Background Concepts
A circuit breaker is the operational dual of a retry. A retry keeps trying until the downstream is probably up; a circuit breaker stops trying once the downstream is probably down. Without a breaker, every caller retries the full schedule and amplifies the outage. With one, callers fail fast for a cooldown window and only a single probe call is sent during recovery — preventing the retry storm that otherwise prolongs outages.
The three states:
- CLOSED — normal operation; calls go through; failures are counted in a sliding window.
- OPEN — the failure threshold was crossed; all calls are short-circuited with
CircuitOpenErrorforcooldown_sseconds. - HALF_OPEN — cooldown elapsed; a single probe call is allowed. If it succeeds, transition to
CLOSEDand reset counters. If it fails, transition back toOPENand start a fresh cooldown.
Two failure-counting windows are common: count-based (last N calls) and time-based (last T seconds). Time-based is preferred for low-traffic services because count-based windows can stay stale indefinitely. Both are easy to implement on top of a deque of timestamps.
Interview Context
This is the canonical follow-up to Lab 15 (retry) and a top-15 practical problem at Stripe, Netflix, Uber, and any team with a microservice mesh. The Hystrix library popularized this pattern; its successor Resilience4j is the modern reference. Candidates often hand-roll only the state transitions and miss the half-open single-probe constraint — a clear signal of “knows the diagram, hasn’t operated one in production”.
Problem Statement
Implement CircuitBreaker(failure_threshold, window_s, cooldown_s) with method call(fn) that either calls fn() (and updates the breaker state from the result) or raises CircuitOpenError if the breaker is open. Internally track failure count over the last window_s seconds; transition to OPEN when the count reaches failure_threshold. After cooldown_s in OPEN, the next call enters HALF_OPEN and is the sole probe; success → CLOSED, failure → OPEN again.
Constraints
- Thread-safe; multiple goroutines/threads may call
call()concurrently. - In
HALF_OPEN, exactly one probe is in flight. Concurrent callers seeCircuitOpenErroruntil the probe completes. failure_threshold≥ 1,window_s> 0,cooldown_s> 0.- Successful calls in
CLOSEDdecrement (or do not affect) the failure window — choose and document.
Clarifying Questions
- Are timeouts counted as failures? (Default yes — they almost always indicate downstream unhealth.)
- Are application errors (4xx vs 5xx) treated identically? (No — 4xx is the caller’s fault; only 5xx and timeouts should trip. Inject a
is_failure(exception)predicate.) - What’s the recovery semantics — strict half-open (single probe) or “let N requests through”? (Single probe by default; named
RECOVERY_QUOTAif needed.) - Do we need per-resource breakers or a global one? (Per-resource is correct — a breaker per downstream identity.)
- Should successes in
CLOSEDreset the failure count? (Most implementations don’t reset; only the sliding window aging removes failures. Tunable.)
Examples
breaker = CircuitBreaker(failure_threshold=5, window_s=10, cooldown_s=30)
breaker.call(lambda: http_get(url)) # raises if downstream raises
# After 5 failures within 10s: state -> OPEN
breaker.call(...) # raises CircuitOpenError immediately for 30s
# After 30s cooldown: next call -> HALF_OPEN probe
# Probe success -> CLOSED, fresh window
# Probe failure -> OPEN, fresh 30s cooldown
Initial Brute Force
class NaiveBreaker:
def __init__(self, threshold, cooldown_s):
self.failures = 0
self.opened_at = None
self.threshold = threshold
self.cooldown_s = cooldown_s
def call(self, fn):
if self.opened_at and time.time() - self.opened_at < self.cooldown_s:
raise CircuitOpenError()
try:
r = fn()
self.failures = 0
return r
except Exception:
self.failures += 1
if self.failures >= self.threshold:
self.opened_at = time.time()
raise
This naive version has six bugs: not thread-safe; counts forever (no window aging); no half-open state (multiple probes after cooldown); resets failures on any success even if breaker just opened; uses wall-clock; treats every exception as a failure.
Brute Force Complexity
call() is O(1). Failure window is unbounded — fails over long traffic patterns where intermittent failures should not trip.
Optimization Path
(1) Add a sliding window — deque of failure timestamps, age out on each call. (2) Add HALF_OPEN state with a probe_in_flight flag. (3) Add a Lock to serialize state transitions. (4) Inject the failure predicate so only real failures count. (5) Switch to monotonic(). (6) Emit metrics on each transition.
Final Expected Approach
State machine guarded by a threading.Lock. On each call(): under the lock, read state. If OPEN and cooldown elapsed → transition to HALF_OPEN and grant the probe to this caller (set probe_in_flight=True). If OPEN and not elapsed → raise. If HALF_OPEN and probe in flight → raise (concurrent callers see open). If CLOSED → proceed. Release the lock, call fn(), reacquire the lock to record the result. On success in HALF_OPEN → transition to CLOSED, clear failures. On failure → record (or transition to OPEN).
Data Structures Used
deque[float]for failure timestamps in the sliding window.threading.Lockfor state transitions.- An
enumforState. - A monotonic clock for all time reads.
Correctness Argument
The state diagram is a closed graph: CLOSED → OPEN → HALF_OPEN → {CLOSED | OPEN}. Every transition is guarded by the lock, so two threads cannot disagree on the current state. The half-open invariant is enforced by probe_in_flight: only the thread that flipped the state from OPEN to HALF_OPEN holds the probe right; all others see CircuitOpenError. The sliding window is monotonically aged on each call, so failures older than window_s are guaranteed evicted before being counted.
Complexity
| Operation | Time | Space |
|---|---|---|
call (CLOSED, success) | O(1) amortized | O(window) for deque |
call (OPEN, fast-fail) | O(1) | O(1) |
call (HALF_OPEN probe) | O(1) plus fn | O(1) |
Window aging is amortized O(1) per call.
Implementation Requirements
import threading
import time
from collections import deque
from enum import Enum
from typing import Callable, Optional, TypeVar
T = TypeVar("T")
class State(Enum):
CLOSED = "CLOSED"
OPEN = "OPEN"
HALF_OPEN = "HALF_OPEN"
class CircuitOpenError(Exception):
pass
class CircuitBreaker:
def __init__(self,
failure_threshold: int = 5,
window_s: float = 10.0,
cooldown_s: float = 30.0,
is_failure: Callable[[BaseException], bool] = lambda e: True,
*,
clock=time.monotonic):
if failure_threshold < 1:
raise ValueError("failure_threshold must be >= 1")
self._threshold = failure_threshold
self._window_s = window_s
self._cooldown_s = cooldown_s
self._is_failure = is_failure
self._clock = clock
self._lock = threading.Lock()
self._state = State.CLOSED
self._failures: deque[float] = deque()
self._opened_at: Optional[float] = None
self._probe_in_flight = False
# observability
self._transitions: list[tuple[float, State, State]] = []
def _age_failures(self, now: float):
cutoff = now - self._window_s
while self._failures and self._failures[0] < cutoff:
self._failures.popleft()
def _transition(self, new: State, now: float):
self._transitions.append((now, self._state, new))
self._state = new
def _try_acquire_probe(self, now: float) -> bool:
"""Called under lock. True if this caller becomes the probe."""
if self._state == State.OPEN and self._opened_at is not None \
and now - self._opened_at >= self._cooldown_s:
self._transition(State.HALF_OPEN, now)
self._probe_in_flight = True
return True
return False
def call(self, fn: Callable[[], T]) -> T:
now = self._clock()
is_probe = False
with self._lock:
if self._state == State.CLOSED:
self._age_failures(now)
elif self._state == State.OPEN:
if not self._try_acquire_probe(now):
raise CircuitOpenError("breaker is OPEN")
is_probe = True
elif self._state == State.HALF_OPEN:
if not self._probe_in_flight:
# rare race: cooldown re-elapsed during a transient state
self._probe_in_flight = True
is_probe = True
else:
raise CircuitOpenError("probe in flight")
# invoke without holding the lock
try:
result = fn()
except BaseException as e:
failed = self._is_failure(e)
with self._lock:
now = self._clock()
if is_probe:
self._probe_in_flight = False
self._transition(State.OPEN, now)
self._opened_at = now
elif failed:
self._failures.append(now)
self._age_failures(now)
if len(self._failures) >= self._threshold and self._state == State.CLOSED:
self._transition(State.OPEN, now)
self._opened_at = now
self._failures.clear()
raise
with self._lock:
if is_probe:
self._probe_in_flight = False
self._transition(State.CLOSED, self._clock())
self._failures.clear()
self._opened_at = None
return result
def state(self) -> State:
with self._lock:
return self._state
Tests
def test_closed_passes_through():
b = CircuitBreaker(failure_threshold=3, window_s=10, cooldown_s=5)
assert b.call(lambda: 42) == 42
assert b.state() == State.CLOSED
def test_opens_after_threshold():
b = CircuitBreaker(failure_threshold=3, window_s=10, cooldown_s=5)
for _ in range(3):
try: b.call(lambda: (_ for _ in ()).throw(RuntimeError()))
except RuntimeError: pass
assert b.state() == State.OPEN
try: b.call(lambda: 42)
except CircuitOpenError: pass
def test_half_open_success_closes():
fake = [0.0]
b = CircuitBreaker(failure_threshold=2, window_s=10, cooldown_s=5, clock=lambda: fake[0])
for _ in range(2):
try: b.call(lambda: (_ for _ in ()).throw(RuntimeError()))
except RuntimeError: pass
assert b.state() == State.OPEN
fake[0] = 6
assert b.call(lambda: "ok") == "ok"
assert b.state() == State.CLOSED
def test_half_open_failure_reopens():
fake = [0.0]
b = CircuitBreaker(failure_threshold=2, window_s=10, cooldown_s=5, clock=lambda: fake[0])
for _ in range(2):
try: b.call(lambda: (_ for _ in ()).throw(RuntimeError()))
except RuntimeError: pass
fake[0] = 6
try: b.call(lambda: (_ for _ in ()).throw(RuntimeError()))
except RuntimeError: pass
assert b.state() == State.OPEN
def test_concurrent_only_one_probe():
import threading
fake = [0.0]
b = CircuitBreaker(failure_threshold=1, window_s=10, cooldown_s=5, clock=lambda: fake[0])
try: b.call(lambda: (_ for _ in ()).throw(RuntimeError()))
except RuntimeError: pass
fake[0] = 6
seen_states = []
barrier = threading.Barrier(10)
def worker():
barrier.wait()
try:
b.call(lambda: time.sleep(0.05) or "ok")
seen_states.append("ok")
except CircuitOpenError:
seen_states.append("open")
threads = [threading.Thread(target=worker) for _ in range(10)]
for t in threads: t.start()
for t in threads: t.join()
assert seen_states.count("ok") == 1
assert seen_states.count("open") == 9
Follow-up Questions
- How would you make it thread-safe? A single
threading.Lockaround state transitions and counter updates is sufficient and is what the implementation does. Thefn()call is invoked outside the lock so a slow downstream does not block other callers from seeingCircuitOpenError. The half-open probe race is resolved byprobe_in_flightflipping atomically under the lock. - What metrics would you emit? State-transition counter (labels:
from_state,to_state); current state gauge; per-call outcome counter (success,failure,short_circuit,probe_success,probe_failure); failure-window gauge (current count); time-in-state histogram. - What is the consistency model? Linearizable on the breaker’s state — all
state()reads observe transitions in a total order consistent with the lock acquisition order. The probe invariant (“at most one probe in flight at any time”) is strict. - How would you handle a poison-pill input? A request that always raises a retryable failure will trip the breaker quickly — that’s the breaker’s job. The risk is the opposite: a probe with a poison input perpetually fails the half-open probe and never recovers. Mitigation: pick the probe payload from a known-safe traffic pool (synthetic health-check), or use a periodic health probe instead of in-line traffic.
- What configuration knobs would you expose?
failure_threshold,window_s,cooldown_s,is_failurepredicate. Don’t expose the half-open probe quota — keep it 1 unless you have a strong reason. Defaults: 5 failures / 10s window / 30s cooldown. - How would you scale to N nodes? Per-process breakers are local — each instance learns about downstream health independently. This is correct for most use cases (each instance’s view of latency varies) but expensive if downstream collapse is sudden. The next step is a coordinated breaker via a shared registry, but only at very high scale.
Product Extension
Real-world breakers (Hystrix, Resilience4j, Polly) layer on top of this core: bulkheads (concurrent-call limit), rate limiters, fallbacks (return cached value when open), and metric emission to Prometheus / StatsD. The state machine is the same; the bookkeeping around it varies by framework.
Language/Runtime Follow-ups
- Python: as above. For async, replace
Lockwithasyncio.Lockand makecallanasync def. - Java: prefer Resilience4j in production. Hand-rolled:
AtomicReference<State>,LongAdderfor counters,ScheduledExecutorServicefor cooldown timeouts. - Go: a struct guarded by
sync.Mutex. The probe flag is a bool. Usetime.Now()(monotonic on Go 1.9+). - C++:
std::mutex+std::condition_variableif you want concurrent callers to wait for the probe rather than fail fast (a different policy, called “blocking breaker”). - JS/TS: in single-threaded Node, no lock is needed — the state-transition logic is naturally atomic across awaits as long as you do not
awaitin the middle of a transition block.
Common Bugs
- Holding the lock while calling
fn()— a slow downstream blocks every other caller. - Forgetting to clear
probe_in_flighton probe failure — breaker stays inHALF_OPENforever, all calls fail. - Using
time.time()— wall-clock skew can makenow - opened_atnegative and the cooldown effectively infinite. - Counting non-failure exceptions (
KeyboardInterrupt,ValueErrorfrom caller side) toward the threshold. - Resetting
failureson every successful call — masks intermittent failures. - Aging the window only on failure —
state()queries report stale counts.
Debugging Strategy
When the breaker won’t open: log the failure count after each call; check is_failure(e) returns True for the actual exception. When it won’t close after recovery: log the state and probe_in_flight flag — almost always the probe-flag-stuck-True bug. When concurrent tests are flaky: add a barrier so all callers race in lockstep, then assert exactly one probe-success and N-1 short-circuits.
Mastery Criteria
- Drew the three-state diagram from memory in <30 seconds.
- Listed every transition trigger (failure-threshold, cooldown-elapsed, probe-success, probe-failure) without prompting.
- Wrote a thread-safe implementation in <30 minutes from a blank screen.
- Wrote a concurrent test that catches the multiple-probe bug.
- Articulated the retry × circuit-breaker composition in <60 seconds.
- Named four metrics you’d emit for a production breaker.
-
Explained why
fn()must not be called under the lock.
Lab 17 — Metrics Collector (Counter / Gauge / Histogram)
Goal
Implement a thread-safe in-process metrics registry that supports the three canonical metric types — counter, gauge, histogram — with bounded memory, label support, and an export format suitable for Prometheus scraping. After this lab you should be able to write the registry from a blank screen in <25 minutes and articulate the difference between summary and histogram in <60 seconds.
Background Concepts
The four metric types in the Prometheus / OpenMetrics ecosystem are counter (monotonic non-decreasing total — requests, errors, bytes), gauge (a current value — queue depth, active connections, memory in use), histogram (a count of observations bucketed by upper bound, used to compute quantiles server-side), and summary (client-side quantiles, harder to aggregate). The first three cover ~95% of production needs. Counters answer “how many?”, gauges answer “how much right now?”, histograms answer “what’s the distribution and the p99?”.
A metric is identified by (name, label_set). The same name with different labels (e.g., http_requests_total{method="GET"} vs http_requests_total{method="POST"}) is a different time series. The number of label combinations is the metric’s cardinality. Unbounded cardinality (e.g., a label per user_id) is the most common production memory leak in metric systems — protect against it.
Histograms are tricky. Two-pass naive implementations (store all observations, sort on export) explode memory. The Prometheus model: pre-declare a fixed set of bucket upper bounds ([0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10] is the default) and increment a counter per bucket. Quantile estimation happens at the query layer using bucket interpolation. Memory is O(buckets) per series — cheap.
Interview Context
A staple practical-engineering question at Datadog, Grafana, Honeycomb, and any observability-aware team (which is most of them now). Also asked at Stripe and Uber as a “we want to see how you reason about cardinality” question. Candidates fail this round when they reach for “store every observation in a list” — that betrays a lack of production exposure.
Problem Statement
Implement MetricsRegistry with:
counter(name, labels=None).inc(by=1)— monotonically incrementing.gauge(name, labels=None).set(value)/.inc(by=1)/.dec(by=1).histogram(name, labels=None, buckets=...).observe(value).registry.snapshot()returns a list of(name, labels, type, value)tuples or the OpenMetrics text format.- Thread-safe; bounded cardinality (configurable maximum number of label combinations per metric name).
Constraints
- Must support concurrent observations from many threads.
- Histogram bucket increments must be atomic (no torn reads of
bucket_countandsum). - Cardinality cap: when a new label combination would exceed
max_labels_per_metric, drop and emit an internal counter (metrics_drops_total). - Counter and gauge: O(1) per operation; histogram: O(log buckets) per observation (binary search the bucket).
Clarifying Questions
- Do we need timestamped exposition (Prometheus exposition format)? (Yes, but the timestamp can be implicit — Prometheus assigns the scrape timestamp.)
- Are histogram buckets shared across all label combinations or per combination? (Per combination — different label values may have different distributions.)
- Are we exposing percentiles client-side or letting the server compute them? (Server-side — the histogram type is exactly this.)
- Should counters reset on process restart? (Yes — Prometheus handles this with the
rate()function and the reset detection in counter math.) - What’s the maximum
max_labels_per_metricwe should default to? (1000 is generous; 100 is conservative. Make it configurable.)
Examples
reg = MetricsRegistry()
reg.counter("http_requests_total", {"method": "GET", "status": "200"}).inc()
reg.gauge("queue_depth", {"queue": "ingest"}).set(42)
reg.histogram("request_latency_s", {"endpoint": "/api"}, buckets=[0.01, 0.1, 1, 10]).observe(0.04)
print(reg.snapshot_openmetrics())
# # HELP http_requests_total
# # TYPE http_requests_total counter
# http_requests_total{method="GET",status="200"} 1
# # TYPE queue_depth gauge
# queue_depth{queue="ingest"} 42
# ...
Initial Brute Force
class NaiveMetrics:
def __init__(self):
self.metrics = {}
def counter(self, name, labels=None):
key = (name, frozenset((labels or {}).items()))
self.metrics.setdefault(key, 0)
self.metrics[key] += 1
This conflates increment and registration, has no thread safety, no histogram (impossible to compute p99 from a counter), no cardinality cap, and uses one global dict so every metric type collides on key shape.
Brute Force Complexity
O(1) per increment under no contention. With concurrent writers, races on dict.__setitem__ and += corrupt counts. Memory unbounded.
Optimization Path
Separate types into separate sub-registries (Counter, Gauge, Histogram) to avoid type-pun bugs. Add a per-metric Lock (or atomic primitive). Use bisect_left on a sorted bucket array to find the histogram bucket. Cap cardinality with a per-metric-name combination counter. Define an exposition format.
Final Expected Approach
A MetricsRegistry holds a dict name → MetricFamily. A MetricFamily stores the metric type, the bucket schedule (for histograms), and a dict labels_tuple → MetricInstance. Each MetricInstance is a small thread-safe object: Counter has an int and a Lock; Gauge has a float and a Lock; Histogram has a list[int] of bucket counts, a float sum, an int count, and a Lock. On increment, hash the label tuple, look up or create the instance (with cardinality check), acquire its Lock, mutate. Snapshot iterates families and instances under their locks and emits an exposition string.
Data Structures Used
dict[str, MetricFamily]for the registry.dict[tuple[tuple[str, str], ...], MetricInstance]per family for label combinations.list[float](sorted) for histogram bucket boundaries.list[int]for histogram bucket counts.threading.Lockper instance.
Correctness Argument
Counter: monotonic by construction (only inc(by) with by ≥ 0 allowed). Gauge: set is the last writer’s value; inc/dec are atomic under the lock. Histogram: each observation falls into exactly one bucket (the smallest bucket whose upper bound is ≥ observation; the last bucket is +Inf). Sum and count are incremented under the same lock as the bucket count, so a snapshot sees consistent values.
Complexity
| Op | Time |
|---|---|
counter.inc | O(1) lock-and-increment |
gauge.set/inc/dec | O(1) |
histogram.observe | O(log B) for bucket lookup |
snapshot | O(N · B) where N is total instances |
Space: O(name + label-cardinality · (1 for counter/gauge or B+2 for histogram)).
Implementation Requirements
import threading
from bisect import bisect_left
from typing import Optional
DEFAULT_BUCKETS = (0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10)
def _label_key(labels: Optional[dict]) -> tuple:
if not labels:
return ()
return tuple(sorted(labels.items()))
class Counter:
__slots__ = ("_v", "_lock")
def __init__(self):
self._v = 0
self._lock = threading.Lock()
def inc(self, by: float = 1):
if by < 0:
raise ValueError("counter cannot decrease")
with self._lock:
self._v += by
def value(self) -> float:
with self._lock:
return self._v
class Gauge:
__slots__ = ("_v", "_lock")
def __init__(self):
self._v = 0.0
self._lock = threading.Lock()
def set(self, v: float):
with self._lock:
self._v = v
def inc(self, by: float = 1):
with self._lock:
self._v += by
def dec(self, by: float = 1):
with self._lock:
self._v -= by
def value(self) -> float:
with self._lock:
return self._v
class Histogram:
__slots__ = ("_buckets", "_counts", "_sum", "_count", "_lock")
def __init__(self, buckets):
self._buckets = tuple(sorted(buckets))
self._counts = [0] * (len(self._buckets) + 1) # +1 for +Inf
self._sum = 0.0
self._count = 0
self._lock = threading.Lock()
def observe(self, v: float):
idx = bisect_left(self._buckets, v)
if idx == len(self._buckets) and v > self._buckets[-1]:
idx = len(self._buckets) # +Inf bucket
with self._lock:
self._counts[idx] += 1
self._sum += v
self._count += 1
def snapshot(self) -> dict:
with self._lock:
cumulative = []
running = 0
for i, b in enumerate(self._buckets):
running += self._counts[i]
cumulative.append((b, running))
running += self._counts[-1]
cumulative.append((float("inf"), running))
return {"buckets": cumulative, "sum": self._sum, "count": self._count}
class MetricFamily:
def __init__(self, name: str, kind: str, buckets=None, max_labels: int = 1000):
self.name = name
self.kind = kind # "counter" | "gauge" | "histogram"
self.buckets = buckets
self.instances: dict = {}
self.max_labels = max_labels
self.dropped = 0
self.lock = threading.Lock()
def get(self, labels_key: tuple):
with self.lock:
inst = self.instances.get(labels_key)
if inst is not None:
return inst
if len(self.instances) >= self.max_labels:
self.dropped += 1
return None
if self.kind == "counter":
inst = Counter()
elif self.kind == "gauge":
inst = Gauge()
else:
inst = Histogram(self.buckets or DEFAULT_BUCKETS)
self.instances[labels_key] = inst
return inst
class _NullCounter:
def inc(self, by=1): pass
class MetricsRegistry:
def __init__(self, max_labels_per_metric: int = 1000):
self._families: dict[str, MetricFamily] = {}
self._lock = threading.Lock()
self._max_labels = max_labels_per_metric
def _family(self, name: str, kind: str, buckets=None) -> MetricFamily:
with self._lock:
f = self._families.get(name)
if f is None:
f = MetricFamily(name, kind, buckets, self._max_labels)
self._families[name] = f
elif f.kind != kind:
raise ValueError(f"metric {name} already registered as {f.kind}")
return f
def counter(self, name: str, labels: Optional[dict] = None) -> Counter:
f = self._family(name, "counter")
inst = f.get(_label_key(labels))
return inst if inst is not None else _NullCounter()
def gauge(self, name: str, labels: Optional[dict] = None) -> Gauge:
f = self._family(name, "gauge")
return f.get(_label_key(labels)) or Gauge() # caller-detached fallback
def histogram(self, name: str, labels: Optional[dict] = None, buckets=None) -> Histogram:
f = self._family(name, "histogram", buckets)
return f.get(_label_key(labels)) or Histogram(buckets or DEFAULT_BUCKETS)
def snapshot_openmetrics(self) -> str:
lines = []
with self._lock:
families = list(self._families.values())
for f in families:
lines.append(f"# TYPE {f.name} {f.kind}")
with f.lock:
instances = list(f.instances.items())
for labels_key, inst in instances:
lbl = "{" + ",".join(f'{k}="{v}"' for k, v in labels_key) + "}" if labels_key else ""
if isinstance(inst, (Counter, Gauge)):
lines.append(f"{f.name}{lbl} {inst.value()}")
elif isinstance(inst, Histogram):
snap = inst.snapshot()
for b, cum in snap["buckets"]:
b_str = "+Inf" if b == float("inf") else f"{b}"
bucket_lbl = labels_key + (("le", b_str),)
bl = "{" + ",".join(f'{k}="{v}"' for k, v in bucket_lbl) + "}"
lines.append(f"{f.name}_bucket{bl} {cum}")
lines.append(f"{f.name}_sum{lbl} {snap['sum']}")
lines.append(f"{f.name}_count{lbl} {snap['count']}")
return "\n".join(lines)
Tests
def test_counter_increments():
r = MetricsRegistry()
c = r.counter("hits")
c.inc(); c.inc(2)
assert c.value() == 3
def test_counter_rejects_negative():
r = MetricsRegistry()
try: r.counter("x").inc(-1)
except ValueError: pass
else: assert False
def test_gauge_set_inc_dec():
g = MetricsRegistry().gauge("depth")
g.set(10); g.inc(5); g.dec(3)
assert g.value() == 12
def test_histogram_buckets():
h = MetricsRegistry().histogram("lat", buckets=[0.1, 1, 10])
for v in [0.05, 0.5, 1.5, 100]: h.observe(v)
snap = h.snapshot()
assert snap["count"] == 4
assert snap["sum"] == 102.05
assert snap["buckets"][0] == (0.1, 1) # ≤ 0.1
assert snap["buckets"][1] == (1, 2) # ≤ 1
assert snap["buckets"][2] == (10, 3) # ≤ 10
assert snap["buckets"][3][1] == 4 # +Inf
def test_concurrent_counter():
import threading
r = MetricsRegistry()
c = r.counter("racy")
def inc():
for _ in range(10_000): c.inc()
threads = [threading.Thread(target=inc) for _ in range(8)]
for t in threads: t.start()
for t in threads: t.join()
assert c.value() == 80_000
def test_cardinality_cap():
r = MetricsRegistry(max_labels_per_metric=2)
r.counter("uid", {"id": "a"}).inc()
r.counter("uid", {"id": "b"}).inc()
r.counter("uid", {"id": "c"}).inc() # dropped
assert len(r._families["uid"].instances) == 2
assert r._families["uid"].dropped == 1
def test_type_conflict():
r = MetricsRegistry()
r.counter("x")
try: r.gauge("x")
except ValueError: pass
else: assert False
Follow-up Questions
- How would you make it thread-safe? Per-instance locks (counters and gauges have one each; histograms have one per
(name, labels)instance). The registry-level lock only guards family creation. Result: counter increments from different label tuples never block each other, which is the expected hot path. - What metrics would you emit (about the metrics system itself)?
metrics_dropped_total{name=...}(cardinality drops);metrics_active_series(gauge of total instances);metrics_scrape_duration_seconds(histogram ofsnapshot_openmetricslatency). Self-instrumentation is a sign of mature instrumentation. - What is the eviction policy? None for active series — they live forever. For TTL’d metrics (rare), an external sweeper deletes instances unobserved for N minutes. Design so the sweeper is optional, not on the hot path.
- What configuration knobs would you expose?
max_labels_per_metric(cardinality cap), default histogram buckets, the registry’s exposition format. Don’t expose the per-instance lock granularity — it’s an implementation detail. - How would you handle backpressure? The hot path is lock-bounded; a write that contends waits microseconds. If a histogram’s lock becomes hot, switch to per-bucket atomics or sharded counters (8 shards keyed by
thread_id % 8, summed at scrape). - What’s the difference between summary and histogram? Summary computes quantiles client-side using a streaming algorithm (Greenwald-Khanna). Pros: exact-ish percentiles per series. Cons: cannot aggregate across series. Histogram pushes the work to the query layer; aggregation across series is just bucket-wise addition. Histograms are the right default in modern observability.
Product Extension
This is exactly the data model exposed by the Prometheus client libraries. Real implementations add: gauge track_inprogress (decorator/context manager that increments on enter, decrements on exit), summary type, exemplars (trace IDs attached to histogram observations), native histograms (sparse representation that auto-tunes bucket boundaries).
Language/Runtime Follow-ups
- Python: as above. The
prometheus_clientlibrary is the production reference. Be aware of GIL implications: counter increments aren’t atomic at the bytecode level for floats, but the explicit lock makes them so. - Java: use
LongAdderfor counters (avoids contention via per-thread cells);DoubleAdderfor histogram sums. Micrometer is the production reference. - Go: counters and gauges as
atomic.Int64/atomic.Float64. Histograms withsync.Mutexper metric. Theprometheus/client_golanglibrary does exactly this. - C++:
std::atomic<uint64_t>for counter;std::mutexfor histogram. Cardinality maps require careful design —tbb::concurrent_hash_mapis one option. - JS/TS: single-threaded — no locks needed in Node. The
prom-clientpackage is the production reference.
Common Bugs
- Histogram bucket lookup off-by-one —
bisect_leftis correct only if buckets are pre-sorted. - Sharing histogram buckets across label combinations — different distributions have different optimal buckets. Each instance gets its own bucket array.
- Forgetting the
+Infbucket — observations larger than the largest bucket are silently dropped. - Using
intfor histogram sum — overflows for high-throughput histograms after a few hours. Usefloat. - Type confusion — registering the same name as both counter and gauge corrupts exposition. The registry must reject this.
- Unbounded cardinality — a label per
request_idcreates a new series per request. The cardinality cap is the safety net.
Debugging Strategy
When totals look wrong: confirm the counter.inc(by) has by ≥ 0 and the operation is under the lock. When percentiles are off: dump the bucket cumulative counts; the math should be cumulative[i] = sum(counts[0..i]). When concurrency tests are flaky: add time.sleep(0) between increments to expose races; if the test passes, your locking is correct.
Mastery Criteria
- Wrote the three metric types with correct semantics in <25 minutes.
- Articulated histogram vs summary in <60 seconds.
- Stated the cardinality risk without prompting and showed the cap in code.
- Wrote a concurrent counter test that verifies no lost updates.
- Produced a Prometheus-compatible exposition string.
- Listed three metrics-about-metrics to emit (drops, active series, scrape duration).
- Explained why per-instance locks scale better than a global registry lock.
Lab 18 — Concurrent Web Crawler
Goal
Implement a concurrent web crawler that BFS-traverses a starting URL, respects a depth limit, per-host politeness (max in-flight requests per domain + minimum inter-request delay), dedup (visit each URL once), and a bounded worker pool. After this lab you should be able to write the crawler from a blank screen in <40 minutes and answer the politeness/backpressure follow-ups crisply.
Background Concepts
A web crawler is a BFS over the web graph where nodes are URLs and edges are anchor links extracted from the HTML. The interesting engineering is not the BFS — it’s the constraints layered on top:
- Politeness: never overload a single host. The classic rule is “no more than
kconcurrent requests per host” plus “at leastdelayseconds between consecutive requests to the same host”. Both rules must be enforced even when many workers race to crawl the same domain. - Dedup: the web has cycles. A
seenset keyed on canonicalized URL prevents enqueueing the same page twice. - Depth limit: domains can have effectively infinite reachable URLs (calendars, faceted search). Hard-cap depth.
- Domain restriction: crawl only within a configured allowlist of domains; otherwise the crawler immediately drifts off-topic.
- Bounded workers: limit total concurrency to
N. Without this, the crawler will saturate the host network and crash with file-descriptor exhaustion. - Backpressure: the URL frontier (queue of pending URLs) must be bounded — otherwise a fan-out page with 10,000 links allocates 10,000 entries and pushes more discovery on top.
This is the canonical “build something concurrent” interview question at companies like Google (Search), Cloudflare, Datadog, and any team that does any kind of scraping.
Interview Context
A 40-to-60-minute round at senior+ practical interviews. The interviewer almost always extends the basic problem with politeness, then with persistence (resume after restart), then with distributed scaling. Candidates who write a single-threaded loop with no per-host politeness fail; candidates who reach for a thread pool and a Lock around seen plus a per-host counter pass.
Problem Statement
Implement crawl(start_url, *, max_depth, max_workers, per_host_concurrency, per_host_delay_s, allow_domains, http_get) that returns a list (or yields a stream) of (url, depth, content) tuples. Visit each canonical URL at most once. Never have more than per_host_concurrency requests in flight to a single host. Wait at least per_host_delay_s seconds since the last completed request to that host before starting a new one. Stop when the frontier is empty.
Constraints
- Thread-safe; many workers race for URLs from the frontier.
- Bounded memory: frontier capped,
seenset is the only unbounded structure (acceptable — proportional to corpus). - Graceful shutdown on
Ctrl-Cor external cancellation. http_getis injected so tests don’t hit the network.
Clarifying Questions
- URL canonicalization rules? (Lowercase host, drop fragment, sort query params, default port elision.)
- Should
robots.txtbe honored? (Yes in production; mock it in this lab viais_allowedpredicate.) - What’s a “host”? (Registered domain —
example.com, notwww.example.comvsimages.example.com. Or just hostname; document the choice.) - Should depth-0 (the start URL) count toward the depth limit? (No — depth-0 is always crawled.)
- Should we follow redirects? (Yes, but the redirect target counts as the visited URL.)
- Output order — does it need to be deterministic? (No — concurrency makes determinism hard. Document.)
Examples
results = crawl(
"https://example.com/",
max_depth=3,
max_workers=8,
per_host_concurrency=2,
per_host_delay_s=0.5,
allow_domains={"example.com"},
http_get=fake_http_get,
)
# returns ~30 (url, depth, body) tuples, never more than 2 in-flight to example.com.
Initial Brute Force
def crawl_naive(url, max_depth):
seen = {url}
frontier = [(url, 0)]
out = []
while frontier:
u, d = frontier.pop(0)
body = http_get(u)
out.append((u, d, body))
if d < max_depth:
for link in extract_links(body):
if link not in seen:
seen.add(link)
frontier.append((link, d + 1))
return out
This is single-threaded (slow), has no politeness (will get IP-banned), and grows the frontier unboundedly.
Brute Force Complexity
Time: O(V) HTTP requests serially, where V is the number of unique pages. With 1s/page and 10k pages, ~3 hours.
Optimization Path
Add a thread pool of max_workers. Add a Lock-guarded seen set. Add per-host concurrency via a Semaphore keyed on host. Add per-host last-request-time via a dict-of-(timestamp, lock). Cap the frontier with a BoundedQueue. Add a stop_event for graceful shutdown.
Final Expected Approach
A ThreadPoolExecutor(max_workers) runs crawl_one(url, depth). The frontier is a queue.Queue(maxsize=...). A seen set guarded by a Lock ensures each URL is enqueued once. A HostLimiter class encapsulates per-host concurrency (a Semaphore) and per-host delay (a Lock + last-completed timestamp). Workers pull URLs, acquire the host limiter (which may block on the semaphore or sleep for the delay), call http_get, extract links, check the depth limit and the seen set under the lock, enqueue new URLs.
Data Structures Used
queue.Queuefor the frontier (bounded).set[str]forseen(guarded byLock).dict[str, _HostState]for per-host limiters._HostState: aSemaphore(per_host_concurrency)and a(Lock, last_completed_ts).ThreadPoolExecutorfor workers.
Correctness Argument
Each URL is enqueued at most once (the seen set is checked under the global lock atomically with the add). Each URL is dequeued and crawled at most once (the queue is FIFO, items are not re-enqueued). The depth limit is enforced before enqueueing children, not before crawling parents — this matches the natural BFS semantics. Per-host politeness: a worker holds the host’s semaphore for the duration of the request, and the inter-request delay is enforced by checking now - last_completed >= delay under the host’s lock; this guarantees no two completed requests are closer than delay for the same host, even with N workers.
Complexity
| Aspect | Cost |
|---|---|
| HTTP requests | O(V) total, parallel by max_workers |
seen lookup | O(1) average |
| Per-host serialization | bounded by per_host_concurrency and per_host_delay_s |
| Memory | O(V) for seen plus frontier_capacity for the queue |
Implementation Requirements
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from queue import Queue, Empty
from urllib.parse import urlparse, urldefrag
def canonicalize(url: str) -> str:
u, _ = urldefrag(url)
p = urlparse(u)
host = p.hostname.lower() if p.hostname else ""
port = f":{p.port}" if p.port else ""
path = p.path or "/"
return f"{p.scheme}://{host}{port}{path}" + (f"?{p.query}" if p.query else "")
def host_of(url: str) -> str:
return (urlparse(url).hostname or "").lower()
class _HostState:
__slots__ = ("sem", "lock", "last_completed")
def __init__(self, concurrency: int):
self.sem = threading.Semaphore(concurrency)
self.lock = threading.Lock()
self.last_completed = 0.0
class HostLimiter:
def __init__(self, per_host_concurrency: int, per_host_delay_s: float, *, clock=time.monotonic, sleep=time.sleep):
self._concurrency = per_host_concurrency
self._delay = per_host_delay_s
self._states: dict[str, _HostState] = {}
self._guard = threading.Lock()
self._clock = clock
self._sleep = sleep
def _state(self, host: str) -> _HostState:
with self._guard:
s = self._states.get(host)
if s is None:
s = _HostState(self._concurrency)
self._states[host] = s
return s
def acquire(self, host: str):
s = self._state(host)
s.sem.acquire()
with s.lock:
wait = s.last_completed + self._delay - self._clock()
if wait > 0:
self._sleep(wait)
return s
def release(self, s: _HostState):
with s.lock:
s.last_completed = self._clock()
s.sem.release()
def crawl(start_url: str, *,
max_depth: int = 3,
max_workers: int = 8,
per_host_concurrency: int = 2,
per_host_delay_s: float = 0.0,
allow_domains: set[str] | None = None,
frontier_capacity: int = 10_000,
http_get,
extract_links,
is_allowed=lambda url: True):
seen: set[str] = set()
seen_lock = threading.Lock()
frontier: Queue = Queue(maxsize=frontier_capacity)
in_flight = 0
in_flight_lock = threading.Lock()
inflight_zero = threading.Event()
inflight_zero.set()
stop_event = threading.Event()
results: list[tuple[str, int, str]] = []
results_lock = threading.Lock()
limiter = HostLimiter(per_host_concurrency, per_host_delay_s)
def _allow(url: str) -> bool:
if not is_allowed(url):
return False
if allow_domains is None:
return True
h = host_of(url)
return any(h == d or h.endswith("." + d) for d in allow_domains)
def _enqueue(url: str, depth: int):
canon = canonicalize(url)
if not _allow(canon):
return
with seen_lock:
if canon in seen:
return
seen.add(canon)
with in_flight_lock:
nonlocal_in_flight = None # placate linters
# frontier put outside the lock; bounded queue applies backpressure
frontier.put((canon, depth))
def _worker():
nonlocal in_flight
while not stop_event.is_set():
try:
url, depth = frontier.get(timeout=0.1)
except Empty:
with in_flight_lock:
if in_flight == 0:
return
continue
with in_flight_lock:
in_flight += 1
inflight_zero.clear()
try:
state = limiter.acquire(host_of(url))
try:
body = http_get(url)
finally:
limiter.release(state)
if body is None:
continue
with results_lock:
results.append((url, depth, body))
if depth < max_depth:
for link in extract_links(body, base=url):
_enqueue(link, depth + 1)
except Exception:
# in production: emit a metric, maybe DLQ; here we just continue
pass
finally:
frontier.task_done()
with in_flight_lock:
in_flight -= 1
if in_flight == 0 and frontier.empty():
inflight_zero.set()
_enqueue(start_url, 0)
with ThreadPoolExecutor(max_workers=max_workers) as pool:
futures = [pool.submit(_worker) for _ in range(max_workers)]
try:
while True:
if inflight_zero.wait(timeout=0.5) and frontier.empty():
with in_flight_lock:
if in_flight == 0:
break
except KeyboardInterrupt:
stop_event.set()
stop_event.set()
for f in futures:
f.result()
return results
Tests
def make_site(graph: dict[str, list[str]]):
def http_get(url): return graph.get(url)
def extract_links(body, base): return body if isinstance(body, list) else []
return http_get, extract_links
def test_basic_bfs():
graph = {
"https://e.com/a": ["https://e.com/b", "https://e.com/c"],
"https://e.com/b": ["https://e.com/d"],
"https://e.com/c": ["https://e.com/d"],
"https://e.com/d": [],
}
http_get, extract_links = make_site(graph)
out = crawl("https://e.com/a", max_depth=5, max_workers=4,
per_host_concurrency=4, allow_domains={"e.com"},
http_get=http_get, extract_links=extract_links)
visited = {u for u, _, _ in out}
assert visited == set(graph.keys())
def test_depth_limit():
chain = {f"https://e.com/{i}": [f"https://e.com/{i+1}"] for i in range(10)}
chain["https://e.com/10"] = []
http_get, extract_links = make_site(chain)
out = crawl("https://e.com/0", max_depth=2, max_workers=2,
per_host_concurrency=2, allow_domains={"e.com"},
http_get=http_get, extract_links=extract_links)
assert len({u for u, _, _ in out}) == 3 # depths 0, 1, 2
def test_dedup_on_cycle():
g = {"https://e.com/a": ["https://e.com/b"],
"https://e.com/b": ["https://e.com/a", "https://e.com/c"],
"https://e.com/c": []}
http_get, extract_links = make_site(g)
out = crawl("https://e.com/a", max_depth=10, max_workers=4,
per_host_concurrency=4, allow_domains={"e.com"},
http_get=http_get, extract_links=extract_links)
urls = [u for u, _, _ in out]
assert len(urls) == len(set(urls)) == 3
def test_domain_restriction():
g = {"https://e.com/a": ["https://other.com/x"], "https://other.com/x": []}
http_get, extract_links = make_site(g)
out = crawl("https://e.com/a", max_depth=5, max_workers=2,
per_host_concurrency=2, allow_domains={"e.com"},
http_get=http_get, extract_links=extract_links)
urls = {u for u, _, _ in out}
assert "https://other.com/x" not in urls
def test_per_host_concurrency():
import threading
in_flight = {"max": 0, "now": 0}
lock = threading.Lock()
def http_get(url):
with lock:
in_flight["now"] += 1
in_flight["max"] = max(in_flight["max"], in_flight["now"])
time.sleep(0.05)
with lock:
in_flight["now"] -= 1
return []
crawl("https://e.com/a", max_depth=0, max_workers=10,
per_host_concurrency=3, allow_domains={"e.com"},
http_get=http_get, extract_links=lambda b, base: [])
# depth 0 -> only the start url is fetched. Need a bigger frontier:
g = {f"https://e.com/{i}": [f"https://e.com/{i+1}" for _ in range(1)] for i in range(20)}
# ... but the assertion shape is: in_flight["max"] <= 3 in any test variant.
Follow-up Questions
- How would you make it thread-safe? The implementation uses three locks:
seen_lock(guards the URL set),in_flight_lock(guards the in-flight counter and frontier-empty signaling), and per-host locks insideHostLimiter. TheQueueis internally thread-safe. The frontier-capacity bound provides backpressure when discovery outpaces processing. - How would you persist state across restarts? Periodic snapshot of
seento disk (or push to Redis/RocksDB on every visit). On restart, loadseenfrom disk; restart with all known URLs, optionally re-enqueue any URLs that were in-flight but not completed (track via a separatependingset). - How would you scale to N nodes? Shard the URL space by
hash(host) % N— each node owns a fixed slice. Cross-node enqueues go via a message bus. Theseenset is replicated or sharded the same way. Per-host politeness becomes per-(node, host) — no cross-node coordination needed since each host is owned by exactly one node. - How would you handle backpressure? The bounded frontier (
Queue(maxsize=...)) blocks workers that try to enqueue when full. This naturally throttles fast-discovery pages — they wait for the consumers to drain. Drop-on-overflow is wrong for crawlers (you’d lose URLs); blocking is right. - What is the shutdown / draining behavior? On
stop_event, workers stop pulling from the queue. The main loop waits for currenthttp_getcalls to complete (no forced cancellation). Any URLs in the frontier are abandoned but their canonical forms remain inseen, so a restart with the sameseensnapshot will re-enqueue them on demand. - How would you handle a poison-pill input? A URL whose response triggers an infinite link-extraction loop (e.g., calendar with year=∞). Mitigations: depth limit (already there), per-host hit cap (max 100 URLs per host), URL length cap, link-extraction time cap with
signal.alarmor a sub-thread timeout, and a content-size cap onhttp_get.
Product Extension
Real crawlers (Googlebot, Bingbot) layer on top: robots.txt parsing per host, sitemap.xml ingestion, content fingerprinting (SimHash) to dedup near-duplicates, freshness scheduling (re-crawl frequently changing pages sooner), priority scoring (PageRank-like), and distributed coordination via Bigtable / DynamoDB / Cassandra.
Language/Runtime Follow-ups
- Python: as above. For very high concurrency switch to
asynciowithaiohttpandasyncio.Semaphoreper host — single-threaded, no GIL contention, 1k+ concurrent requests are realistic. - Java:
ExecutorServicewith boundedBlockingQueue;ConcurrentHashMap.newKeySet()forseen;Semaphoreper host. OrCompletableFuturechains with virtual threads (Project Loom) for high concurrency. - Go: a worker-pool of goroutines reading from a buffered channel;
sync.Mapforseen; per-hostchan struct{}of sizeconcurrencyas a semaphore. The idiom is exceptionally clean in Go. - C++:
std::threadpool;std::unordered_set+std::shared_mutexforseen; per-hoststd::counting_semaphore(C++20). - JS/TS: Node +
p-limitper host; single-threaded so no locks. The “global” concurrency is enforced by an outerp-limit.
Common Bugs
- Adding to
seenafterhttp_getreturns — multiple workers crawl the same URL. - Holding
seen_lockwhile callinghttp_get— blocks every other worker. - Per-host semaphore allocated per request instead of memoized — concurrency limit not enforced.
- Per-host delay measured from request start instead of completion — fast pages still violate politeness.
- Forgetting
urldefragin canonicalization —?#section1and?#section2count as different URLs. - The “main loop” exits before all workers drain — items in the frontier are silently lost. Use a counter + condition or
Queue.join().
Debugging Strategy
When dedup fails: log every seen.add(canon) call with the canon string; the bug is almost always a canonicalization difference. When politeness fails: log per-host completed timestamps and confirm consecutive ones are at least delay apart. When the crawler hangs at end: print in_flight and frontier.qsize() periodically — if both are 0 but the main loop hasn’t exited, your termination signal is broken.
Mastery Criteria
- Wrote a crawler with thread pool, bounded frontier, dedup, and depth limit in <40 minutes.
- Implemented per-host concurrency limit and per-host inter-request delay correctly under stress.
- Stated the four reasons politeness matters (overload, IP ban, robots violation, cost) without prompting.
- Articulated the sharding strategy for scaling to N nodes.
- Listed two metrics you’d emit (per-host request rate, frontier depth gauge).
- Identified the canonicalization bug class (fragment / param order / case).
- Explained why blocking-on-full-queue is the right backpressure choice for crawlers.
Lab 19 — In-Memory Filesystem
Goal
Implement an in-memory filesystem that supports ls, mkdir, addContentToFile, readContentFromFile over a tree of inode-like directory and file nodes. After this lab you should be able to design and implement the filesystem from a blank screen in <30 minutes and answer the standard follow-ups (concurrency, persistence, paths/permissions) crisply.
Background Concepts
A filesystem is a tree where internal nodes are directories and leaves are files. Every Unix-style filesystem reduces to the same three operations: navigate (walk a path to a node), mutate (create / delete / write), and read (list / cat). The interview problem strips out permissions, hard links, symbolic links, journaling, and on-disk layout, leaving the core tree structure + path-walking — which is enough to test naming, encapsulation, and correctness.
The two design choices that matter:
- One node class or two? A
Directoryhas adict[name, Node]of children; aFilehas a content string. They share path-walking but diverge in storage. Use either an inheritance hierarchy (DirectoryandFileextendNode) or a singleNodewith akinddiscriminator. Inheritance is cleaner for this problem. - Path-walk encapsulation. Every operation needs
_walk(path)that returns the target node (or raises). Centralize it; do not duplicate the path-split logic in each method.
This problem appears as LC 588 — “Design In-Memory File System” — and is asked at Amazon, Google, and Bloomberg as an OOD warmup.
Interview Context
A 30-to-45-minute round at the senior level. The interviewer wants to see (a) clean class decomposition, (b) correct path-handling (absolute paths, edge cases like / and trailing slash), (c) sensible API, (d) tests. Common failure: stuffing everything into one class with one dict[str, str] keyed on full paths — works for small inputs, fails the “ls a directory” requirement, and looks like LeetCode glue rather than production code.
Problem Statement
Implement FileSystem with:
ls(path) -> list[str]: ifpathis a file, return[filename]; if a directory, return sorted list of children.mkdir(path) -> None: create the directory and any missing intermediate directories (mkdir -p semantics).addContentToFile(path, content) -> None: create the file if missing; appendcontentto it. Intermediate directories are created.readContentFromFile(path) -> str: return the file’s content. Raise if not a file.
All paths are absolute, start with /, components separated by /. The root is /.
Constraints
- Path components are non-empty alphanumeric (no
.,.., no slashes-in-names). - Directories and files in the same directory must have distinct names.
addContentToFileon an existing directory should raise.ls("/")returns the children of root.
Clarifying Questions
- Is
addContentToFileappend or overwrite? (LC 588 is append; confirm.) - Are path components case-sensitive? (Yes, like Unix.)
- What’s the error mode for missing files in
readContentFromFile? (Raise — caller should not silently get an empty string.) - Can
mkdir("/a")succeed if/aalready exists as a directory? (Yes — idempotent. As a file? No — raise.) - Does
ls("/a/b")where/a/bdoesn’t exist raise or return[]? (Raise.) - Concurrency? (Single-threaded by default; thread-safety as a follow-up.)
Examples
fs = FileSystem()
fs.ls("/") # []
fs.mkdir("/a/b/c")
fs.addContentToFile("/a/b/c/d.txt", "hello")
fs.ls("/") # ["a"]
fs.ls("/a/b/c") # ["d.txt"]
fs.readContentFromFile("/a/b/c/d.txt") # "hello"
fs.addContentToFile("/a/b/c/d.txt", " world")
fs.readContentFromFile("/a/b/c/d.txt") # "hello world"
Initial Brute Force
class FlatFS:
def __init__(self): self.store = {} # full path -> content (None for dirs)
def mkdir(self, p): self.store[p] = None
def addContentToFile(self, p, c): self.store[p] = self.store.get(p, "") or "" ; self.store[p] += c
def readContentFromFile(self, p): return self.store[p]
def ls(self, p):
if self.store.get(p) is not None: return [p.rsplit("/", 1)[1]]
prefix = p.rstrip("/") + "/"
return sorted({k[len(prefix):].split("/")[0] for k in self.store if k.startswith(prefix)})
This works on small inputs but is O(N) per ls (scans every key) and conflates the “file vs directory” type via a None-or-string trick. It also can’t represent an empty directory unambiguously.
Brute Force Complexity
ls: O(N · L) where N is total entries and L is average path length. addContentToFile: O(L) for hashing. Memory: O(total path text + content).
Optimization Path
Replace the flat dict with a tree of nodes. Each Directory has dict[str, Node] children — ls is now O(K log K) where K is the number of children of the queried directory, not the whole filesystem. mkdir and addContentToFile walk the path once, creating intermediate directories on demand. The walk is O(path_depth).
Final Expected Approach
Node is the base class. Directory(Node) has children: dict[str, Node]. File(Node) has content: list[str] (a list of chunks; append by chunks.append(c); read by "".join(chunks)). The list-of-chunks representation makes append O(1) regardless of total size. _walk(path, create_dirs=False) returns the terminal node, raising or creating intermediates as configured. Each public method is a thin wrapper around _walk.
Data Structures Used
dict[str, Node]for directory children — O(1) lookup, sorted on demand forls.list[str]for file content chunks — O(1) append.- A path-split helper that handles
/correctly.
Correctness Argument
Every node has exactly one parent (the dict entry that points to it). The root is the only node with no parent. _walk is the single source of truth for path resolution; it raises if it encounters a missing intermediate (create_dirs=False) or auto-creates one (create_dirs=True). addContentToFile resolves the parent directory, then either fetches the existing File (and asserts it’s a file, not a directory) or creates a new File. The “name collision” case (a directory exists where a file is being created) is detected at this exact step.
Complexity
| Operation | Time | Space |
|---|---|---|
mkdir | O(L) where L = path components | O(L) new nodes |
addContentToFile | O(L) for walk + O(1) append | O(content) |
readContentFromFile | O(L + content_size) for join | O(content) |
ls (directory) | O(K log K) for sort | O(K) |
ls (file) | O(L) | O(1) |
Implementation Requirements
class Node:
pass
class File(Node):
__slots__ = ("chunks",)
def __init__(self):
self.chunks: list[str] = []
def read(self) -> str:
return "".join(self.chunks)
def append(self, content: str):
if content:
self.chunks.append(content)
class Directory(Node):
__slots__ = ("children",)
def __init__(self):
self.children: dict[str, Node] = {}
def _split(path: str) -> list[str]:
if not path or path[0] != "/":
raise ValueError(f"path must be absolute: {path!r}")
return [p for p in path.split("/") if p]
class FileSystem:
def __init__(self):
self._root = Directory()
def _walk(self, parts: list[str], *, create_dirs: bool = False) -> Node:
node: Node = self._root
for i, p in enumerate(parts):
if not isinstance(node, Directory):
raise NotADirectoryError("/".join(parts[:i]) or "/")
child = node.children.get(p)
if child is None:
if not create_dirs:
raise FileNotFoundError("/" + "/".join(parts[: i + 1]))
child = Directory()
node.children[p] = child
node = child
return node
def ls(self, path: str) -> list[str]:
parts = _split(path)
node = self._walk(parts)
if isinstance(node, File):
return [parts[-1]]
return sorted(node.children.keys())
def mkdir(self, path: str) -> None:
parts = _split(path)
if not parts:
return # mkdir '/' is a no-op
# Walk-with-create, but reject a final-segment that exists as a file
parent = self._walk(parts[:-1], create_dirs=True)
if not isinstance(parent, Directory):
raise NotADirectoryError("/".join(parts[:-1]))
last = parts[-1]
existing = parent.children.get(last)
if existing is None:
parent.children[last] = Directory()
elif isinstance(existing, File):
raise FileExistsError(path + " is a file")
# else: existing directory; idempotent
def addContentToFile(self, path: str, content: str) -> None:
parts = _split(path)
if not parts:
raise IsADirectoryError("/")
parent = self._walk(parts[:-1], create_dirs=True)
if not isinstance(parent, Directory):
raise NotADirectoryError("/".join(parts[:-1]))
last = parts[-1]
node = parent.children.get(last)
if node is None:
node = File()
parent.children[last] = node
elif isinstance(node, Directory):
raise IsADirectoryError(path)
node.append(content)
def readContentFromFile(self, path: str) -> str:
parts = _split(path)
node = self._walk(parts)
if not isinstance(node, File):
raise IsADirectoryError(path)
return node.read()
Tests
def test_empty_root():
fs = FileSystem()
assert fs.ls("/") == []
def test_mkdir_p():
fs = FileSystem()
fs.mkdir("/a/b/c")
assert fs.ls("/") == ["a"]
assert fs.ls("/a") == ["b"]
assert fs.ls("/a/b") == ["c"]
assert fs.ls("/a/b/c") == []
def test_add_and_read_file():
fs = FileSystem()
fs.addContentToFile("/x/y/z.txt", "hello")
fs.addContentToFile("/x/y/z.txt", " world")
assert fs.readContentFromFile("/x/y/z.txt") == "hello world"
assert fs.ls("/x/y") == ["z.txt"]
assert fs.ls("/x/y/z.txt") == ["z.txt"]
def test_mkdir_idempotent():
fs = FileSystem()
fs.mkdir("/a")
fs.mkdir("/a")
assert fs.ls("/") == ["a"]
def test_mkdir_over_file_fails():
fs = FileSystem()
fs.addContentToFile("/a", "x")
try: fs.mkdir("/a")
except FileExistsError: pass
else: assert False
def test_read_nonexistent():
fs = FileSystem()
try: fs.readContentFromFile("/nope")
except FileNotFoundError: pass
else: assert False
def test_ls_sorts():
fs = FileSystem()
for n in ["zeta", "alpha", "mu"]: fs.mkdir(f"/{n}")
assert fs.ls("/") == ["alpha", "mu", "zeta"]
def test_root_path():
fs = FileSystem()
fs.addContentToFile("/a.txt", "x")
assert fs.ls("/a.txt") == ["a.txt"]
Follow-up Questions
- How would you make it thread-safe? Two options. (a) Single coarse
RLockon the whole FileSystem — every operation acquires it. Simple, fine for low write rates. (b) Per-directory lock; acquire locks along the path during a walk. Avoids serializing readers from disjoint subtrees, but care is needed to acquire in path-order to avoid deadlock. For an interview answer, name both and pick (a) unless write contention is the explicit follow-up. - How would you persist state across restarts? Two layers. (i) Snapshot: serialize the tree (DFS, emit
(path, kind, content_or_empty)); on boot, replay. (ii) Write-ahead log: append every mutation as a record (mkdir /a,add /a/b "hello"); periodic checkpoint. Tradeoff: pure snapshot loses recent writes; pure log replays slowly; combine for production. - What configuration knobs would you expose?
max_filesize,max_path_depth,max_filename_length. Don’t expose the lock granularity — implementation detail. Reject paths exceeding the caps with a typed error. - How would you handle a poison-pill input? A path with millions of components, or a single file with multi-gigabyte content. Cap path depth, cap filename length, cap per-file content size, and surface metric counters for rejected requests.
- How would you test it? Unit tests on each method’s contract. Property-based tests: random sequence of
mkdir/addContentoperations followed by anls/readthat asserts consistency with a simple oracle (e.g., a flat dict). Concurrency tests: many threads each writing to disjoint subtrees should produce identical state regardless of interleaving. - What metrics would you emit? Operation counters (per method), latency histograms,
total_files,total_directories,bytes_storedgauges, error counters by type.
Product Extension
Variants in real systems: S3-style flat namespace with / as a virtual delimiter; in-memory FUSE filesystems for tests; Kubernetes ConfigMap/Secret mounting (a tiny in-memory FS exposed to a pod). The data structure is the same; the API surface and persistence vary.
Language/Runtime Follow-ups
- Python: as above. Use
__slots__forFileandDirectoryto cut per-node memory. - Java:
Map<String, Node>(HashMap or TreeMap). For sortedls, TreeMap is natural and avoids the per-call sort. - Go:
type Node interface { ... }withDirectoryandFilestructs. For sortedls, sort.Strings on the keys. - C++:
std::variant<Directory, File>or a tagged union.std::map<std::string, std::unique_ptr<Node>>for ordered children. - JS/TS:
Map<string, Node>(insertion-ordered; sort onls). Use a discriminated union forNode.
Common Bugs
- Using
path.split("/")without filtering empty strings —["", "a", "b"]for"/a/b". - Treating
/differently from non-/paths inconsistently; a single_splithelper avoids this. - Not detecting directory-vs-file at the final path component —
addContentToFile("/a")where/ais a directory must raise. mkdiroverwriting an existing file silently.- Storing file content as a single growing string —
s += contentis O(N) per append. Use a list of chunks. - Returning unsorted
ls— the spec usually requires sorted output for determinism.
Debugging Strategy
When ls is wrong: print the children dict at the resolved node — almost always a wrong-path-walk bug. When append seems to overwrite: check that addContentToFile calls node.append, not node.chunks = [content]. When concurrency tests fail: log every operation in order with a thread ID; the bug is usually a missing lock around parent.children[last] = ....
Mastery Criteria
-
Decomposed
Node/File/Directorycleanly in <5 minutes. -
Wrote a single
_walkhelper used by every public method. -
Handled
mkdiridempotency and the file-vs-directory collision case correctly. - Used the list-of-chunks pattern for O(1) append.
-
Wrote tests for every error mode (
FileNotFoundError,IsADirectoryError,FileExistsError). - Articulated the snapshot+WAL persistence strategy in <60 seconds.
- Implemented from a blank screen in <30 minutes.
Lab 20 — Snake Game
Goal
Implement the game logic for Snake (LC 353) — a snake moves on a grid, eats food, grows by one each meal, and dies on collision with a wall or itself. Each move(direction) returns the current score or -1 on game-over. After this lab you should be able to write the implementation from a blank screen in <25 minutes with O(1) per move.
Background Concepts
Snake is a classic OOD warmup that hides a single non-obvious data-structure decision: representing the snake as a deque of cells (head at one end, tail at the other) and using a set for O(1) self-collision check. The naive representation — a list scanned linearly every move — is O(N) per move and TLE-prone at large grids.
A nuance: when the snake moves and doesn’t eat food, the tail moves out of its old cell before the head moves into the new cell. So the head’s new cell could be the old tail’s cell — that’s not a collision. The standard bug is to check self-collision before removing the old tail, producing a false-positive death.
Interview Context
A 30-minute round at Amazon, Microsoft, and Bloomberg. The setup is clear; the interviewer is grading on (a) data-structure choice (deque + set), (b) correct ordering of tail-removal vs head-addition, (c) edge cases (food at head’s new cell, food consumed in order from a queue), (d) clean class design.
Problem Statement
A snake starts at (0, 0) on a width × height grid (top-left is origin, x grows right, y grows down). Food is given as a queue of [row, col] positions consumed in order. On move(direction) where direction ∈ {U, D, L, R}:
- The head advances one cell in that direction.
- If the new head is out of bounds → game over, return
-1. - If the new head collides with the snake’s body (excluding the cell the tail vacates this turn) → game over, return
-1. - If the new head equals the next food position → consume the food (advance the food queue), grow by one (do NOT remove the tail), score += 1.
- Otherwise → remove the tail.
Return the current score (number of foods eaten).
Constraints
- 1 ≤
width, height≤ 10^4. - 0 ≤
food.length≤ 50. - Food positions are inside the grid and never on
(0, 0). moveis called up to 10^4 times.
Clarifying Questions
- Is the head or the tail at index 0 of the snake list? (Convention: head at index 0; tail at the end. Document.)
- Can the snake move backwards onto itself in one move? (Length 1: yes — that’s just a turn. Length > 1: that’s a self-collision.)
- Is food consumed FIFO from the queue? (Yes.)
- Does the game continue after an illegal move? (No —
-1is terminal; subsequent calls should also return-1or be undefined. Document.) - Can two food items occupy the same cell? (Spec says no; assume distinct.)
Examples
g = SnakeGame(width=3, height=2, food=[[1,2],[0,1]])
g.move("R") # head: (0,1) -> 0
g.move("D") # head: (1,1) -> 0
g.move("R") # head: (1,2) eats food[0] -> 1
g.move("U") # head: (0,2) -> 1
g.move("L") # head: (0,1) eats food[1] -> 2
g.move("U") # out of bounds -> -1
Initial Brute Force
class SnakeNaive:
def __init__(self, w, h, food):
self.w, self.h = w, h
self.food = food
self.snake = [(0, 0)]
def move(self, d):
dr, dc = {"U":(-1,0),"D":(1,0),"L":(0,-1),"R":(0,1)}[d]
r, c = self.snake[0]
nr, nc = r + dr, c + dc
if not (0 <= nr < self.h and 0 <= nc < self.w): return -1
if self.food and [nr, nc] == self.food[0]:
self.food.pop(0)
self.snake.insert(0, (nr, nc))
else:
self.snake.pop()
if (nr, nc) in self.snake: return -1
self.snake.insert(0, (nr, nc))
return len(self.snake) - 1
Two bugs and one performance issue: (nr, nc) in self.snake is O(N); self.snake.insert(0, ...) is O(N) for a list; self.food.pop(0) is O(F).
Brute Force Complexity
move: O(N) per call. Across M moves: O(M · N). At N = 10^4 and M = 10^4: 10^8 — borderline.
Optimization Path
Replace list with collections.deque (O(1) append/pop both ends). Add a set of body cells for O(1) collision detection. Keep an integer food_idx instead of pop(0)-ing the food list. Now every move is O(1).
Final Expected Approach
State: body: deque[(r, c)] with head at the right (body[-1]), body_set: set[(r, c)] mirroring it, food_idx: int, plus width, height, food. On move: compute new head, check bounds, decide grow-or-shift. If grow: append new head to deque and set; advance food_idx. If shift: remove old tail from set first, then check collision with body_set, then add new head. The order matters — exactly the “tail vacates then head moves” semantics.
Data Structures Used
collections.dequefor the snake body (O(1) head/tail append/pop).set[tuple[int, int]]for membership (O(1) collision check).intfood_idxto avoid mutating the food list.- A
dict[str, tuple[int, int]]for direction deltas.
Correctness Argument
The body deque represents the snake as a sequence from tail to head. The body_set is the membership oracle. Invariant: set(body) == body_set is maintained at every operation. On move-without-eating, we pop the tail from both before testing the new head — this models tail-vacates-first. On move-with-eating, the tail stays, so the snake grows by one. Bounds are checked first because a head outside the grid is definitely game over regardless of body. The score equals food consumed, which equals food_idx.
Complexity
| Operation | Time | Space |
|---|---|---|
move | O(1) amortized | O(N) for body |
Implementation Requirements
from collections import deque
from typing import List
class SnakeGame:
DIRS = {
"U": (-1, 0),
"D": (1, 0),
"L": (0, -1),
"R": (0, 1),
}
def __init__(self, width: int, height: int, food: List[List[int]]):
if width <= 0 or height <= 0:
raise ValueError("width and height must be positive")
self._w = width
self._h = height
self._food = [tuple(f) for f in food]
self._food_idx = 0
self._body: deque[tuple[int, int]] = deque([(0, 0)])
self._body_set: set[tuple[int, int]] = {(0, 0)}
self._game_over = False
def move(self, direction: str) -> int:
if self._game_over:
return -1
if direction not in self.DIRS:
raise ValueError(f"invalid direction: {direction!r}")
dr, dc = self.DIRS[direction]
head_r, head_c = self._body[-1]
nr, nc = head_r + dr, head_c + dc
# 1. bounds
if not (0 <= nr < self._h and 0 <= nc < self._w):
self._game_over = True
return -1
# 2. eat-or-shift decision
new_head = (nr, nc)
eats = (
self._food_idx < len(self._food)
and self._food[self._food_idx] == new_head
)
if eats:
self._food_idx += 1
# grow: head added; tail stays
if new_head in self._body_set:
# the new head landed on the body (rare but possible: food
# placed on a cell the snake currently occupies)
self._game_over = True
return -1
self._body.append(new_head)
self._body_set.add(new_head)
return self._food_idx
# shift: tail vacates first
old_tail = self._body.popleft()
self._body_set.remove(old_tail)
if new_head in self._body_set:
self._game_over = True
return -1
self._body.append(new_head)
self._body_set.add(new_head)
return self._food_idx
Tests
def test_basic_path():
g = SnakeGame(3, 2, [[1, 2], [0, 1]])
assert g.move("R") == 0
assert g.move("D") == 0
assert g.move("R") == 1
assert g.move("U") == 1
assert g.move("L") == 2
assert g.move("U") == -1
def test_immediate_wall():
g = SnakeGame(3, 3, [])
assert g.move("U") == -1
assert g.move("R") == -1 # idempotent terminal
def test_self_collision_after_growth():
# Grow to length 4, then turn into self.
g = SnakeGame(4, 4, [[0, 1], [0, 2], [0, 3]])
assert g.move("R") == 1
assert g.move("R") == 2
assert g.move("R") == 3
assert g.move("D") == 3
assert g.move("L") == 3
assert g.move("U") == 3 # no collision yet
# body is at (0,3),(0,2),(0,1),(0,0) wait, careful — let's just sanity check non-trivial case
def test_tail_cell_is_safe():
# Length 2 snake, move into the cell its tail just vacated -> not collision.
g = SnakeGame(3, 3, [[0, 1]]) # eat once at (0,1)
assert g.move("R") == 1 # body: (0,0)->(0,1), length 2
assert g.move("D") == 1 # body: (0,1)->(1,1), tail (0,0) vacated
assert g.move("L") == 1 # body: (1,1)->(1,0)
assert g.move("U") == 1 # body: (1,0)->(0,0). Old tail vacated this turn.
def test_food_consumed_in_order():
g = SnakeGame(5, 5, [[0, 1], [0, 2]])
assert g.move("R") == 1
assert g.move("R") == 2
def test_terminal_state_persists():
g = SnakeGame(2, 2, [])
assert g.move("U") == -1
assert g.move("D") == -1
assert g.move("L") == -1
Follow-up Questions
- How would you test it? Unit tests on each path: bounds, eat, shift, self-collision, tail-cell-safe. Property test: random direction sequences with random food; oracle re-implements the naive O(N) version; assert outputs match. Smoke test: a long random run that doesn’t crash.
- What configuration knobs would you expose? Grid size, initial position, direction key bindings, optional “wrap-around” mode (snake exits one wall, enters the opposite). Don’t expose the data-structure choices.
- How would you handle a poison-pill input? Invalid direction strings → raise. Negative grid dimensions → raise at construction. Food positions outside the grid → raise at construction. After the game ends, calls to
moveare idempotent (return-1). - How would you make it thread-safe? Wrap
movein aLock. Snake game state has no natural concurrency benefit (a single player), but if multiple callers (e.g., network clients in a multiplayer variant) race, the lock prevents torn updates. - What metrics would you emit?
moves_per_gamehistogram,score_at_game_overhistogram,game_over_reasoncounter (wall vs self-collision). Useful to compare difficulty levels. - How would you scale to N players (multiplayer Snake)? Each player has their own
body/body_set. Thebody_sets must be merged for collision detection:forbidden = self.body_set | sum(other.body_set). The food queue is shared. Lock per shared-state structure or use a STM-style atomic transaction per tick.
Product Extension
Multiplayer variants (Slither.io, Agar.io descendants) keep this exact data structure but add: (a) a server-authoritative tick clock, (b) state diff broadcasts, (c) interpolation on the client. The interview-relevant primitive is unchanged.
Language/Runtime Follow-ups
- Python: as above.
collections.dequeis the right primitive —list.pop(0)is O(N). - Java:
ArrayDeque<int[]>for the body;HashSet<Long>for collision (encode(r, c)as(long)r * width + c). - Go: a slice for the body (use ring-buffer indices for O(1) ends, or accept linear shifts for small N);
map[[2]int]struct{}for the set. - C++:
std::deque<std::pair<int,int>>andstd::unordered_set<int64_t>with a(r, c)encoding. - JS/TS: array as a deque is fine for small N; for performance, use head/tail pointers in a fixed array.
Setwith a string-key"r,c"for collision.
Common Bugs
- Removing the tail after checking collision — the just-vacated cell falsely flags as collision.
- Using
list.insert(0, ...)andlist.pop(0)— both O(N), defeats the data-structure choice. - Not advancing
food_idxcorrectly — eating the same food twice or skipping food. - Comparing
food[idx](a list) to(nr, nc)(a tuple) —[0,1] == (0,1)isFalsein Python. Normalize types at construction. - Allowing
moveafter game over without returning-1— undefined behavior. Set a_game_overflag and short-circuit. - Computing direction deltas inside the function instead of as a class constant — minor, but inelegant.
Debugging Strategy
When the snake “dies” on a legal move: print the body, body_set, new_head, and the comparison being made just before returning -1. The bug is almost always the order of tail-vacate vs collision-check. When the score is wrong: print food_idx after each call. When move “succeeds” through a wall: print nr, nc, self._h, self._w — the bounds check is off-by-one.
Mastery Criteria
- Picked deque + set in <30 seconds, justified the choice.
- Stated the tail-vacate-first invariant unprompted.
-
Wrote O(1)
movefrom a blank screen in <20 minutes. - Wrote the tail-cell-is-safe test from memory.
- Listed at least three game-over reasons (wall, self-collision, food-on-body — rare).
- Articulated the multiplayer extension in <60 seconds.
- Solved LC 353 in <25 minutes total with all tests passing.
Lab 21 — Tic-Tac-Toe (Streaming Winner Detection)
Goal
Implement Tic-Tac-Toe (LC 348) where players alternate moves on an N × N board and move(row, col, player) returns 0 (no winner yet) or the player number on a winning move. The naive O(N²) per-move full-scan is unacceptable; achieve O(1) per move by maintaining row, column, and diagonal counters. After this lab you should write the implementation in <15 minutes.
Background Concepts
The non-trivial bit of Tic-Tac-Toe-as-a-data-structure-problem is the per-move winner check. Each cell affects exactly one row, one column, and (if on a diagonal) at most one or two diagonals. By incrementing player-1’s counter by +1 and player-2’s by -1 on the same axes, a counter that hits +N means player 1 won that axis, -N means player 2 won. This collapses the O(N²) scan to O(1).
Diagonals: the main diagonal is the line where row == col; the anti-diagonal is where row + col == N - 1. A cell is on the main diagonal iff row == col; on the anti-diagonal iff row + col == N - 1. The center cell of an odd-N board sits on both.
This is the cleanest real example of “exchange a redundant scan for a maintained counter” — a recurring pattern in real code (running averages, sliding maxes, materialized aggregates in databases).
Interview Context
A 20-minute warmup at Amazon, Google, and Microsoft. Often paired with the LRU lab as a phone-screen double-feature. The interviewer wants O(1) per move, clean class API, and at least one or two follow-ups about extending to N-in-a-row Connect Four-style games (where the winning condition is more complex).
Problem Statement
Implement TicTacToe(n) and move(row, col, player) -> int:
- The board is
n × nand starts empty. - Players alternate (caller manages turn order; you don’t validate it for this version).
playeris 1 or 2.- Each move places the player’s mark at
(row, col). Assume the cell is empty. - Return the player’s number if this move results in a win (full row, column, main diagonal, or anti-diagonal of that player); otherwise return 0.
- Once a player has won, the game ends; further moves are not part of the spec but should be defensively handled.
Constraints
- 1 ≤
n≤ 100. - Each call to
moveis O(1) target. - Up to 10^6 moves across the lifetime of an instance.
Clarifying Questions
- Are
rowandcol0-indexed? (Yes.) - Is the cell guaranteed empty? (Per LC 348: yes. In practice, validate defensively.)
- Do we need to detect a draw? (Not in LC 348; doable as
move_count == n*n.) - Once a player wins, are further moves undefined? (Yes; either short-circuit to that winner or raise.)
- Can the same player call
movetwice in a row? (Spec assumes alternating; we do not enforce.)
Examples
g = TicTacToe(3)
g.move(0, 0, 1) # 0 player 1 at (0,0)
g.move(0, 2, 2) # 0 player 2 at (0,2)
g.move(2, 2, 1) # 0
g.move(1, 1, 2) # 0
g.move(2, 0, 1) # 0
g.move(1, 0, 2) # 0
g.move(2, 1, 1) # 1 player 1 wins via row 2 (0,0 main diag was already two of 1's)
Initial Brute Force
class TicTacToeNaive:
def __init__(self, n):
self.n = n
self.b = [[0] * n for _ in range(n)]
def move(self, r, c, p):
self.b[r][c] = p
# check row
if all(self.b[r][j] == p for j in range(self.n)): return p
if all(self.b[i][c] == p for i in range(self.n)): return p
if r == c and all(self.b[i][i] == p for i in range(self.n)): return p
if r + c == self.n - 1 and all(self.b[i][self.n - 1 - i] == p for i in range(self.n)): return p
return 0
This is O(N) per move. For N = 100 and 10^6 moves: 10^8 — slow but passing. The point is structural: it scans the whole row/column/diagonal every time even though the move only changed one cell.
Brute Force Complexity
move: O(N) per call. Total: O(M · N).
Optimization Path
Replace each row/col/diagonal full-scan with a maintained counter. Use +1 for player 1, -1 for player 2; a counter at ±N is a win. The diagonals are special-cased by the row == col and row + col == N - 1 predicates — we only update them when the cell is on the diagonal. Now every check is a single integer comparison.
Final Expected Approach
State: rows[N], cols[N], diag (scalar), anti (scalar). Each is an integer counter. On move(r, c, player): compute delta = +1 if player == 1 else -1. Increment rows[r], cols[c], and conditionally diag and anti. If any of the four updated counters has absolute value N → that player wins.
Data Structures Used
list[int]of size N for rows.list[int]of size N for columns.intfor the main diagonal counter.intfor the anti-diagonal counter.- (Optional)
list[list[int]]board for defensive duplicate-move detection.
Correctness Argument
A row of N copies of player 1 produces a counter of +N exactly when all N cells are player 1, because every player-1 move on that row contributes +1 and no player-2 move contributes there (since the cell is occupied by player 1). Symmetric for player 2 → -N. Same argument for columns and the two diagonals. The diagonal counter is only updated for cells on the diagonal, so it correctly counts only diagonal cells.
Complexity
| Operation | Time | Space |
|---|---|---|
move | O(1) | O(N) for row/col counters |
Implementation Requirements
class TicTacToe:
def __init__(self, n: int):
if n < 1:
raise ValueError("n must be >= 1")
self._n = n
self._rows = [0] * n
self._cols = [0] * n
self._diag = 0
self._anti = 0
self._winner = 0 # 0 = no winner yet
def move(self, row: int, col: int, player: int) -> int:
if self._winner:
return self._winner
if not (0 <= row < self._n and 0 <= col < self._n):
raise IndexError(f"({row}, {col}) out of bounds for n={self._n}")
if player not in (1, 2):
raise ValueError(f"player must be 1 or 2, got {player}")
delta = 1 if player == 1 else -1
target = self._n if player == 1 else -self._n
self._rows[row] += delta
self._cols[col] += delta
if row == col:
self._diag += delta
if row + col == self._n - 1:
self._anti += delta
if (self._rows[row] == target
or self._cols[col] == target
or self._diag == target
or self._anti == target):
self._winner = player
return player
return 0
Tests
def test_row_win_player1():
g = TicTacToe(3)
assert g.move(0, 0, 1) == 0
assert g.move(1, 0, 2) == 0
assert g.move(0, 1, 1) == 0
assert g.move(1, 1, 2) == 0
assert g.move(0, 2, 1) == 1
def test_col_win_player2():
g = TicTacToe(3)
g.move(0, 0, 1); g.move(0, 1, 2)
g.move(1, 0, 1); g.move(1, 1, 2)
g.move(2, 2, 1);
assert g.move(2, 1, 2) == 2
def test_diagonal_win():
g = TicTacToe(3)
g.move(0, 0, 1); g.move(0, 1, 2)
g.move(1, 1, 1); g.move(0, 2, 2)
assert g.move(2, 2, 1) == 1
def test_anti_diagonal_win():
g = TicTacToe(3)
g.move(0, 2, 1); g.move(0, 0, 2)
g.move(1, 1, 1); g.move(0, 1, 2)
assert g.move(2, 0, 1) == 1
def test_no_winner_on_partial():
g = TicTacToe(3)
assert g.move(0, 0, 1) == 0
assert g.move(1, 1, 2) == 0
def test_n_equals_one():
g = TicTacToe(1)
assert g.move(0, 0, 1) == 1
def test_invalid_player():
g = TicTacToe(3)
try: g.move(0, 0, 3)
except ValueError: pass
else: assert False
def test_move_after_winner():
g = TicTacToe(3)
for c in range(3): g.move(0, c, 1)
# subsequent moves still report the winner
assert g.move(1, 1, 2) == 1
def test_large_n_no_win():
g = TicTacToe(100)
# Fill 99 of player 1's row 0 — should not win.
for c in range(99):
assert g.move(0, c, 1) == 0
Follow-up Questions
- How would you test it? Unit tests for each axis (row, col, both diagonals, by both players). Property test: random move sequences; oracle is the naive O(N) scan; assert outputs match. Edge:
n=1(any move wins). Edge: anti-diagonal at the corners only. - What is the consistency model? Single-threaded, linearizable trivially. If multiple threads race on
move, the counters can interleave and a player can falsely fail to win. Wrap with aLockif concurrent. - What configuration knobs would you expose? Just
n. Resist adding “win condition = K-in-a-row instead of N” — that’s a different problem (Connect Four / Gomoku). If asked, see Connect Four extension below. - How would you handle a poison-pill input? Out-of-bounds coords (
IndexError), invalid player (ValueError), repeated cell (defensive: trackboardand reject). The current implementation rejects bounds and player; cell-overwrite detection is an explicit follow-up. - How would you extend to K-in-a-row on an N×N board (Gomoku, Connect Four)? Counters no longer suffice — you need to find any window of K consecutive same-player cells. Two options: (a) on each move, scan the row, column, and both diagonals through the cell looking for K-in-a-row centered on the move (O(K) per move), or (b) maintain run-length encodings per axis (more memory, O(1) per move). For interview-time, (a) is the right answer — clean and O(K), not O(N).
- What metrics would you emit?
moves_total,wins_total{player=1|2},time_to_winhistogram. Game-balance metrics for product analytics; otherwise sparse.
Product Extension
The “maintained counter instead of full scan” pattern shows up everywhere: real-time sports scores (a goal updates a single team total instead of recomputing from a play log), database materialized views (incrementally maintained, not recomputed), Prometheus counters (the rate() function avoids re-scanning the whole series). Tic-Tac-Toe is the simplest possible illustration.
Language/Runtime Follow-ups
- Python: as above. Lists are 8 bytes per int reference; for very large N,
array.array("i", [0]*n)is denser. - Java:
int[] rows,int[] cols,int diag,int anti. No autoboxing in the hot path. - Go: same — value types throughout, no allocations after construction.
- C++:
std::vector<int>. Makemovenon-virtual; this is hot-path code. - JS/TS:
Int32Array(n)for rows and cols — denser than a regular Array.
Common Bugs
- Off-by-one in the anti-diagonal predicate:
row + col == n - 1, notnorn + 1. - Using
+1for both players (and checkingcount == Nandcount == -N) — mistake. Use opposite-sign deltas. - Forgetting to update the diagonal counter when the move is on the diagonal — counter stays stuck.
- Scaling the win threshold incorrectly (
target = n if player == 1 else -n). A cleaner version usesabs(counter) == n and sign(counter) == sign(delta). - Not guarding against repeated cells — same
(r, c)updated twice can spuriously win for one player or unjustly cancel out. - Concurrent calls without a lock: counters become inconsistent and the win condition fires on the wrong player.
Debugging Strategy
When wins are missed: print (rows, cols, diag, anti) after each move; trace which counter should have hit ±N. When wins fire spuriously: same trace — usually the diagonal predicate is wrong. When tests pass for player 1 but not player 2: confirm delta = -1 and target = -N for player 2; sign errors are common.
Mastery Criteria
- Stated the O(1)-per-move counter approach in <30 seconds.
-
Wrote the diagonal predicates (
r == c,r + c == n - 1) without prompting. - Implemented from a blank screen in <15 minutes with all tests passing.
- Listed the K-in-a-row extension and named the right scan strategy.
- Articulated why this is a “maintained counter” pattern in <30 seconds.
-
Wrote tests covering both diagonals and
n=1.
Lab 22 — Text Editor Buffer (Gap Buffer / Piece Table)
Goal
Implement a text editor data structure that supports cursor-local insert, delete (backspace), left/right cursor movement, and substring read with O(1) amortized cursor-local edits. The reference implementation is a gap buffer; the follow-up is a piece table. After this lab you should articulate why a flat str or list[char] is wrong and produce a working gap buffer in <30 minutes.
Background Concepts
A naive editor representation — a single string — makes every insert and delete O(N): the character data after the cursor must shift. For a million-character document, every keystroke is millions of operations. Real editors avoid this with one of three data structures:
- Gap buffer: a single contiguous array with a “gap” of unused slots positioned at the cursor. Insert at cursor = O(1) (write into the gap, shrink it). Move cursor = O(distance moved) — the gap moves with the cursor by shifting characters across it. Used by Emacs.
- Piece table: an immutable original buffer + an append-only “added” buffer + a list of “pieces” describing the visible document as concatenated slices. Insert anywhere = O(1) amortized (append to the added buffer, splice a piece into the piece list). Used by VS Code, Word.
- Rope / balanced tree of strings: O(log N) for all operations, the most general. Used by Xi-editor and several research editors.
The interview almost always wants the gap buffer because it is the simplest correct answer with the right asymptotics for the locality assumption (most edits happen near the cursor). The piece table is the right follow-up answer when the interviewer asks “what if edits aren’t local?” or “what if you want O(1) amortized regardless of cursor position?”.
Interview Context
A 30-to-45-minute round at Google (Docs), Microsoft (VS Code, Word), JetBrains, and any team that builds editor-like UI. Most candidates default to a list[char] and accept O(N) per insert; that’s a partial answer. Reaching for a gap buffer immediately demonstrates that you’ve thought about real editor performance.
Problem Statement
Implement TextEditor with:
insert(text: str): inserttextat the current cursor position; cursor moves to the end of inserted text.delete_left(n: int) -> int: delete up toncharacters to the left of the cursor; return the actual number deleted (capped by left content).move_left(n: int) -> str: move cursornpositions left (capped at start); return the last 10 characters to the left of the new cursor (or fewer if not available).move_right(n: int) -> str: symmetric on the right.text() -> str: return the full document (debug helper; not on the hot path).
This is the LC 2296 “Design a Text Editor” interface, with the read-back-10 affordance.
Constraints
- Up to 10^4 calls in total.
- Each
textargument up to 40 characters; total inserted up to ~10^4 characters. - Insert and
delete_leftmust be O(amortized 1) plus O(text length). Move operations are O(distance moved).
Clarifying Questions
- Is the cursor between characters (column-style) or at a character (cell-style)? (Between — like every editor.)
- Does
delete_left(n)with n > available delete only what’s available and return that count? (Yes.) - What does
move_leftreturn when the cursor is at the start? (Empty string.) - Is the buffer Unicode-aware? (For LC 2296: ASCII suffices. For real editors: must handle code points and grapheme clusters; out of scope here.)
- Are inserts at any cursor position guaranteed local (i.e., does the interviewer want gap buffer or piece table)? (Default gap buffer — most edits are local.)
Examples
ed = TextEditor()
ed.insert("leetcode") # cursor at end; text = "leetcode"
ed.delete_left(4) # 4
ed.text() # "leet"
ed.insert("practice") # text = "leetpractice"
ed.move_right(3) # "etpractice" -> last 10 to left
ed.move_left(8) # "leet"
ed.delete_left(10) # 4
ed.text() # "practice"
Initial Brute Force
class StringEditor:
def __init__(self): self.s = ""; self.c = 0
def insert(self, text): self.s = self.s[:self.c] + text + self.s[self.c:]; self.c += len(text)
def delete_left(self, n):
d = min(n, self.c); self.s = self.s[:self.c - d] + self.s[self.c:]; self.c -= d; return d
def move_left(self, n): self.c = max(0, self.c - n); return self.s[max(0, self.c - 10):self.c]
def move_right(self, n): self.c = min(len(self.s), self.c + n); return self.s[max(0, self.c - 10):self.c]
def text(self): return self.s
Correct, but every insert and delete is O(N) due to slice + concatenation. At 10^4 ops on a 10^4-char document: 10^8 char-shifts. Borderline-TLE on LC 2296.
Brute Force Complexity
insert: O(N + |text|). delete_left: O(N). move_*: O(1) for the text return (slice). Total worst case: O(N · operations).
Optimization Path
Switch to a gap buffer: a single bytearray (or list[str]) of length capacity, with two indices gap_start and gap_end. Characters before gap_start and after gap_end are real content; the range [gap_start, gap_end) is unused. The cursor position is gap_start. Insert at cursor: write into the gap, advance gap_start. Delete left: rewind gap_start (the deleted characters are now in the gap, no copying needed). Move left by k: shift k characters from before gap_start to after gap_end - 1 (the gap moves toward the start). Move right by k: symmetric. Resize when the gap shrinks to zero — double the capacity.
Final Expected Approach
A bytearray buf of size capacity. Indices gap_start (left edge of gap) and gap_end (right edge, exclusive). Invariants: 0 ≤ gap_start ≤ gap_end ≤ capacity. Document length = capacity - (gap_end - gap_start). Cursor = gap_start. Operations manipulate the indices and copy small ranges of bytes; total work for cursor-local edits is bounded by the edit size, not the document size.
Data Structures Used
bytearray(orlist[str]) for the storage buffer.- Two
intindicesgap_startandgap_end. - A
capacitybookkeeping value. - A
_growhelper that doubles capacity when the gap is exhausted.
Correctness Argument
After every operation: the document is buf[:gap_start] + buf[gap_end:capacity] decoded. insert(text): ensure the gap holds len(text) slots (grow if needed); copy text into buf[gap_start:gap_start + len(text)]; advance gap_start by len(text). The document grows by exactly len(text) and the cursor moves to the end of the insertion. delete_left(n): cap n by gap_start (cursor is gap_start, so the leftmost left-deletable count is gap_start); rewind gap_start by n. The document shrinks by exactly n. move_left(k): shift min(k, gap_start) bytes from buf[gap_start - k:gap_start] to buf[gap_end - k:gap_end]; subtract k from both indices. The visible document is unchanged, only the gap moved.
Complexity
| Operation | Time | Space |
|---|---|---|
insert(t) | O( | t |
delete_left(n) | O(1) | O(1) extra |
move_left(k) / move_right(k) | O(k) | O(1) |
text() | O(N) | O(N) |
Implementation Requirements
class TextEditor:
def __init__(self, initial_capacity: int = 16):
self._buf = bytearray(initial_capacity)
self._gap_start = 0
self._gap_end = initial_capacity
@property
def _capacity(self) -> int:
return len(self._buf)
@property
def _length(self) -> int:
return self._capacity - (self._gap_end - self._gap_start)
def _grow(self, needed: int):
new_cap = max(self._capacity * 2, self._capacity + needed)
new_buf = bytearray(new_cap)
# left segment unchanged, right segment shifted to end of new buffer
new_buf[: self._gap_start] = self._buf[: self._gap_start]
right_size = self._capacity - self._gap_end
new_buf[new_cap - right_size :] = self._buf[self._gap_end :]
self._buf = new_buf
self._gap_end = new_cap - right_size
def insert(self, text: str):
b = text.encode("utf-8")
if self._gap_end - self._gap_start < len(b):
self._grow(len(b))
self._buf[self._gap_start : self._gap_start + len(b)] = b
self._gap_start += len(b)
def delete_left(self, n: int) -> int:
d = min(n, self._gap_start)
self._gap_start -= d
return d
def _move_left(self, k: int):
k = min(k, self._gap_start)
if k == 0:
return
# copy k bytes from before gap to after gap (right side)
src_end = self._gap_start
src_start = src_end - k
dst_end = self._gap_end
dst_start = dst_end - k
# work right-to-left to handle overlap
for i in range(k - 1, -1, -1):
self._buf[dst_start + i] = self._buf[src_start + i]
self._gap_start -= k
self._gap_end -= k
def _move_right(self, k: int):
right_avail = self._capacity - self._gap_end
k = min(k, right_avail)
if k == 0:
return
# copy k bytes from after gap (right side) to before gap (left side)
for i in range(k):
self._buf[self._gap_start + i] = self._buf[self._gap_end + i]
self._gap_start += k
self._gap_end += k
def _last_10_left(self) -> str:
start = max(0, self._gap_start - 10)
return self._buf[start : self._gap_start].decode("utf-8", errors="replace")
def move_left(self, k: int) -> str:
self._move_left(k)
return self._last_10_left()
def move_right(self, k: int) -> str:
self._move_right(k)
return self._last_10_left()
def text(self) -> str:
left = self._buf[: self._gap_start]
right = self._buf[self._gap_end :]
return (left + right).decode("utf-8", errors="replace")
Tests
def test_basic_insert_delete():
ed = TextEditor()
ed.insert("leetcode")
assert ed.text() == "leetcode"
assert ed.delete_left(4) == 4
assert ed.text() == "leet"
ed.insert("practice")
assert ed.text() == "leetpractice"
def test_cursor_movement_returns_last_10():
ed = TextEditor()
ed.insert("practice")
assert ed.move_right(3) == "practice" # cursor at end already; last 10 left = "practice"
assert ed.move_left(8) == ""
assert ed.delete_left(10) == 0
ed.insert("leet")
assert ed.text() == "leetpractice"
assert ed.move_left(2) == "le"
def test_lc_2296_canonical():
ed = TextEditor()
ed.insert("leetcode")
assert ed.delete_left(4) == 4
ed.insert("practice")
assert ed.move_right(3) == "etpractice"
assert ed.move_left(8) == "leet"
assert ed.delete_left(10) == 4
assert ed.move_left(2) == ""
def test_grow_buffer():
ed = TextEditor(initial_capacity=4)
ed.insert("a" * 100)
assert ed.text() == "a" * 100
def test_delete_more_than_left():
ed = TextEditor()
ed.insert("ab")
assert ed.delete_left(10) == 2
assert ed.text() == ""
def test_move_clamps():
ed = TextEditor()
ed.insert("hello")
ed.move_left(100) # clamped to 0
ed.move_right(100) # back to end
assert ed.text() == "hello"
Follow-up Questions
- What is the relationship to a piece table? Gap buffer is one contiguous buffer with one gap; piece table is two buffers (original + append-only) and a list of pieces. Insert at cursor in piece table = append to “added” buffer, splice the piece list — O(1) amortized regardless of cursor position. The downside: random-access reads are O(log P) where P is the number of pieces (binary-search the piece list). Use a piece table when edits are non-local; use a gap buffer when most edits cluster.
- How would you make it thread-safe? Wrap public methods with a
Lock(or use a single-writer model — most editors are single-threaded on the editing buffer for exactly this reason; rendering and saving happen on background threads with snapshots). - How would you persist state across restarts? On every K seconds or after every N keystrokes, write the current text to a temporary file, then atomically rename it. For more granular crash recovery, append every operation to a log; replay on boot.
- What configuration knobs would you expose?
initial_capacity,max_document_size. The growth factor (currently 2×) is a sensible default; don’t expose unless you’ve measured. - How would you handle a poison-pill input? A multi-megabyte single
insert(text). Reject text longer thanmax_insert(e.g., 1 MiB). Total document size capped bymax_document_size. Return errors, don’t OOM. - What metrics would you emit?
inserts_total,deletes_total,cursor_moves_total,buffer_grows_total,document_size_bytesgauge,gap_size_bytesgauge. Useful for tracking edit patterns and tuning capacity defaults.
Product Extension
Real editors layer many things on top: undo/redo (each operation pushes an inverse onto a stack), syntax highlighting (incremental tree-sitter passes), multi-cursor (a list of gap-buffer-style cursors), collaborative editing (operational transforms or CRDTs over the same buffer). The buffer is the bottom; the rest is composition.
Language/Runtime Follow-ups
- Python: as above.
bytearrayis the right primitive; avoid string concatenation inside the hot path. - Java:
char[]plusint gapStart,int gapEnd.StringBuilderis internally achar[]but lacks gap-buffer semantics. - Go:
[]byte(or[]runefor Unicode-aware editors). The growth pattern matches Go’s slice append. - C++:
std::vector<char>. For piece tables,std::vector<Piece>of(buffer_id, offset, length). - JS/TS:
Uint8Arrayis the dense representation;stringconcatenation is O(N) and should be avoided.
Common Bugs
- Forgetting to grow when the gap is exhausted — silent overwrite of right-segment data.
- Off-by-one when shifting bytes during cursor moves — left-to-right copy on overlapping ranges loses data; copy right-to-left on left-shifts.
- Using
len(self._buf)after grow without updating cached references — always recomputecapacitypost-grow. - Returning the full
text()on everymove_*call when the spec only wants the last 10 characters left of cursor. - Encoding inconsistencies — mixing
strandbytearray. Pick one (here we use UTF-8 bytes; document the choice; reject mid-codepoint splits in real implementations). - Initializing the buffer too small (e.g., capacity 1) — every keystroke triggers a regrow. Default capacity 16 amortizes well.
Debugging Strategy
When text() is wrong: print (buf, gap_start, gap_end, capacity) after each operation; the bug is almost always a forgotten index update. When cursor moves leak data: print the bytes copied and the index ranges; right-to-left vs left-to-right copy direction is the most common bug. When grow fails: assert len(buf) == capacity after every operation.
Mastery Criteria
- Stated the gap-buffer invariant in <30 seconds.
- Named when piece table is preferred (non-local edits) without prompting.
- Implemented gap buffer with insert/delete/move/text in <30 minutes from blank screen.
- Wrote a regrow test that exercises the doubling.
- Articulated the overlap-direction bug for cursor moves and named the fix.
- Solved LC 2296 unaided in <40 minutes including all tests.
- Listed three real editors (Emacs, VS Code, Word) and which structure each uses.
Lab 23 — Toy SQL-Like Engine
Goal
Implement a tiny SQL-like engine that can parse and execute SELECT col1, col2 FROM t WHERE expr [JOIN u ON expr] [ORDER BY col [DESC]] [LIMIT n] over in-memory tables. The engine has three layers: tokenizer, parser (produces an AST), executor (interprets the AST). After this lab you should be able to scope a 60-minute version of this in <5 minutes and produce a working subset (no joins, no order-by) in <40 minutes.
Background Concepts
A SQL engine — even a toy — is the cleanest interview-friendly example of the frontend / backend / interpreter trilogy that runs every real query engine, compiler, and DSL:
- Lexer / tokenizer: converts a string into a stream of typed tokens (
SELECT, identifiername,=, integer42, etc.). Skips whitespace, recognizes keywords, classifies punctuation. - Parser: consumes the token stream and produces an AST:
Select(columns, from_table, where_expr, joins, order_by, limit). Recursive descent is the right tool for this problem class — top-down, predictable, fits on a whiteboard. - Executor: walks the AST and produces rows. For
WHERE, evaluate the expression against each row. ForJOIN, nested-loop the two tables and concatenate matching rows. ForORDER BY, sort by the named column withDESCflag. ForLIMIT, slice the result.
The interviewer is not testing whether you can build a real query optimizer (you can’t, in 60 minutes). They are testing whether you can decompose the problem into the three layers, write each cleanly, and connect them through a typed AST. Candidates who try to do everything inline in one function fail; candidates who name Token, Expr, Select types and split functions per layer pass.
Interview Context
A 60-minute round at Snowflake, Databricks, MongoDB, Neon, PlanetScale, and any database / data-platform company. Often paired with a smaller warmup. The supported subset varies by interviewer — at minimum SELECT cols FROM t WHERE expr is expected; joins and order-by are stretch goals; aggregates (COUNT, SUM) are extras for strong candidates.
Problem Statement
Implement Engine with:
register_table(name: str, columns: list[str], rows: list[list]): store an in-memory table.query(sql: str) -> list[list]: parse and execute the SQL, return rows.
Supported grammar:
SELECT col_list FROM table_name
[ JOIN table_name ON cond_expr ]
[ WHERE cond_expr ]
[ ORDER BY column_ref [ASC | DESC] ]
[ LIMIT integer ]
col_list is * or comma-separated column references (qualified table.col or bare col). cond_expr is a small expression language: literals (int, string), column refs, and the operators = != < <= > >= AND OR NOT.
Constraints
- Identifiers are alphanumeric (and underscore). Keywords are case-insensitive (
SELECT==select). - String literals use single quotes (
'foo'). - Tables fit in memory; nested-loop join is acceptable.
- Up to 10^3 rows per table; query must finish in well under a second.
Clarifying Questions
- Are aggregates (
COUNT,SUM) required? (No for the base; stretch goal.) - Are subqueries supported? (No.)
- Are NULLs supported? (No — undefined column = error; missing field = treat as None and
NULLpropagation rules elided.) - Are types coerced? (No — comparing
'5'and5returnsFalseor raises; document.) - Is column resolution case-sensitive? (Yes — keywords case-insensitive, identifiers case-sensitive. Document.)
- Are joins inner-only? (Yes —
INNER JOINsemantics; noLEFT/RIGHT/FULLfor the base.)
Examples
-- users(id, name, age); orders(id, user_id, total)
SELECT name FROM users WHERE age >= 18
SELECT u.name, o.total FROM users u JOIN orders o ON u.id = o.user_id WHERE o.total > 100
SELECT name FROM users ORDER BY age DESC LIMIT 3
Initial Brute Force
Skip the parser and tokenize-execute in a single big regex-soup function. This is what most candidates produce when panicked. It works for two or three test cases and breaks instantly on any extension.
Brute Force Complexity
Roughly O(rows × cols × query-length) and bug-prone.
Optimization Path
Properly separate lexer / parser / executor. Tokenizer scans the string once: O(N). Parser is recursive descent: O(tokens). Executor: O(rows × predicate cost) for WHERE; O(left × right) for nested-loop joins; O(rows log rows) for ORDER BY. Each layer is independently testable.
Final Expected Approach
Three layers connected by typed values:
tokenize(sql) -> list[Token]—Token = (kind, value)where kind isKEYWORD,IDENT,INT,STRING,OP,PUNC,EOF.Parser(tokens).parse_select() -> Select— recursive-descent. Each non-terminal is a method.Selectis a dataclass withcolumns,from_table,joins,where,order_by,limit.Engine.execute(select) -> rows— fetch base rows, apply joins, applywhere, project columns, order, limit.
Expr is a small algebraic datatype: Literal(value), Column(table_or_None, name), BinOp(op, left, right), UnaryOp(op, operand). Evaluation: eval_expr(expr, row, schema) -> value.
Data Structures Used
list[tuple]per table (rows).dict[str, int]per table (column → index).- AST: small dataclasses or named tuples.
- Token list:
list[Token]. dict[str, callable]for operator dispatch.
Correctness Argument
Each layer’s correctness is independent of the others. Tokenizer correctness: every input character is consumed exactly once and emitted as exactly one token (or skipped if whitespace). Parser correctness: a recursive-descent parser for an LL(1) grammar accepts the language exactly when the grammar is LL(1) and the parser’s lookahead matches. The grammar above is trivially LL(1). Executor correctness: each clause is a transformation on a row stream. WHERE filters; JOIN cross-products and filters; projection picks columns; ORDER BY sorts; LIMIT truncates. Each transformation preserves the well-typed-row invariant.
Complexity
| Stage | Time |
|---|---|
| Tokenize | O(N) |
| Parse | O(T) where T = tokens |
| WHERE filter | O(R · |
| Inner JOIN (nested loop) | O(R₁ · R₂ · |
| ORDER BY | O(R log R) |
| LIMIT | O(L) |
For larger data: indices, hash joins, query optimizers — out of scope.
Implementation Requirements
import re
from dataclasses import dataclass
from typing import Any, Optional
# ---------------- Tokenizer ----------------
KEYWORDS = {"SELECT", "FROM", "WHERE", "JOIN", "ON",
"ORDER", "BY", "ASC", "DESC", "LIMIT",
"AND", "OR", "NOT"}
@dataclass
class Token:
kind: str
value: Any
_TOKEN_RE = re.compile(r"""
\s+ | # whitespace
'([^']*)' | # string literal
(\d+) | # int
(==|!=|<=|>=|=|<|>) | # ops
([A-Za-z_][A-Za-z0-9_]*) | # identifier or keyword
(,|\(|\)|\.|\*) # punctuation
""", re.VERBOSE)
def tokenize(sql: str) -> list[Token]:
tokens: list[Token] = []
i = 0
while i < len(sql):
m = _TOKEN_RE.match(sql, i)
if not m:
raise SyntaxError(f"unexpected char at {i}: {sql[i]!r}")
s, ival, op, ident, punc = m.groups()
if m.group(0).isspace():
pass
elif s is not None:
tokens.append(Token("STRING", s))
elif ival is not None:
tokens.append(Token("INT", int(ival)))
elif op is not None:
tokens.append(Token("OP", "=" if op == "==" else op))
elif ident is not None:
up = ident.upper()
if up in KEYWORDS:
tokens.append(Token("KEYWORD", up))
else:
tokens.append(Token("IDENT", ident))
elif punc is not None:
tokens.append(Token("PUNC", punc))
i = m.end()
tokens.append(Token("EOF", None))
return tokens
# ---------------- AST ----------------
@dataclass
class Column:
table: Optional[str]
name: str
@dataclass
class Literal:
value: Any
@dataclass
class BinOp:
op: str
left: Any
right: Any
@dataclass
class UnaryOp:
op: str
operand: Any
@dataclass
class Join:
table: str
alias: Optional[str]
on: Any
@dataclass
class Select:
columns: list # list[Column] or ["*"]
from_table: str
from_alias: Optional[str]
joins: list # list[Join]
where: Optional[Any]
order_by: Optional[tuple] # (Column, "ASC" | "DESC")
limit: Optional[int]
# ---------------- Parser (recursive descent) ----------------
class Parser:
def __init__(self, tokens: list[Token]):
self._t = tokens
self._i = 0
def _peek(self) -> Token: return self._t[self._i]
def _eat(self, kind, value=None) -> Token:
tok = self._t[self._i]
if tok.kind != kind or (value is not None and tok.value != value):
raise SyntaxError(f"expected {kind} {value}, got {tok}")
self._i += 1
return tok
def _accept(self, kind, value=None) -> bool:
tok = self._t[self._i]
if tok.kind == kind and (value is None or tok.value == value):
self._i += 1
return True
return False
def parse_select(self) -> Select:
self._eat("KEYWORD", "SELECT")
cols = self._parse_columns()
self._eat("KEYWORD", "FROM")
ftable, falias = self._parse_table_alias()
joins = []
while self._accept("KEYWORD", "JOIN"):
jt, ja = self._parse_table_alias()
self._eat("KEYWORD", "ON")
joins.append(Join(jt, ja, self._parse_expr()))
where = self._parse_expr() if self._accept("KEYWORD", "WHERE") else None
order_by = None
if self._accept("KEYWORD", "ORDER"):
self._eat("KEYWORD", "BY")
ob_col = self._parse_column_ref()
direction = "ASC"
if self._accept("KEYWORD", "DESC"): direction = "DESC"
elif self._accept("KEYWORD", "ASC"): direction = "ASC"
order_by = (ob_col, direction)
limit = None
if self._accept("KEYWORD", "LIMIT"):
limit = self._eat("INT").value
self._eat("EOF")
return Select(cols, ftable, falias, joins, where, order_by, limit)
def _parse_columns(self):
if self._accept("PUNC", "*"):
return ["*"]
cols = [self._parse_column_ref()]
while self._accept("PUNC", ","):
cols.append(self._parse_column_ref())
return cols
def _parse_column_ref(self) -> Column:
ident = self._eat("IDENT").value
if self._accept("PUNC", "."):
name = self._eat("IDENT").value
return Column(ident, name)
return Column(None, ident)
def _parse_table_alias(self) -> tuple[str, Optional[str]]:
name = self._eat("IDENT").value
alias = None
if self._peek().kind == "IDENT":
alias = self._eat("IDENT").value
return name, alias
def _parse_expr(self):
return self._parse_or()
def _parse_or(self):
left = self._parse_and()
while self._accept("KEYWORD", "OR"):
left = BinOp("OR", left, self._parse_and())
return left
def _parse_and(self):
left = self._parse_not()
while self._accept("KEYWORD", "AND"):
left = BinOp("AND", left, self._parse_not())
return left
def _parse_not(self):
if self._accept("KEYWORD", "NOT"):
return UnaryOp("NOT", self._parse_not())
return self._parse_cmp()
def _parse_cmp(self):
left = self._parse_atom()
if self._peek().kind == "OP":
op = self._eat("OP").value
return BinOp(op, left, self._parse_atom())
return left
def _parse_atom(self):
tok = self._peek()
if tok.kind == "INT": self._i += 1; return Literal(tok.value)
if tok.kind == "STRING": self._i += 1; return Literal(tok.value)
if tok.kind == "PUNC" and tok.value == "(":
self._i += 1
e = self._parse_expr()
self._eat("PUNC", ")")
return e
return self._parse_column_ref()
# ---------------- Executor ----------------
class Engine:
def __init__(self):
self._tables: dict[str, tuple[list[str], list[list]]] = {}
def register_table(self, name: str, columns: list[str], rows: list[list]):
self._tables[name] = (columns, [list(r) for r in rows])
def query(self, sql: str) -> list[list]:
ast = Parser(tokenize(sql)).parse_select()
return self._execute(ast)
def _execute(self, sel: Select) -> list[list]:
# 1. base
base_cols, base_rows = self._fetch(sel.from_table)
alias = sel.from_alias or sel.from_table
rows = [(r, {alias: (base_cols, r)}) for r in base_rows]
# 2. joins (nested loop)
for j in sel.joins:
jcols, jrows = self._fetch(j.table)
j_alias = j.alias or j.table
new_rows = []
for left_row, env in rows:
for jr in jrows:
new_env = dict(env)
new_env[j_alias] = (jcols, jr)
if self._eval(j.on, new_env):
new_rows.append((left_row + jr, new_env))
rows = new_rows
# 3. where
if sel.where is not None:
rows = [(r, env) for r, env in rows if self._eval(sel.where, env)]
# 4. project
if sel.columns == ["*"]:
projected = [r for r, _ in rows]
else:
projected = [[self._eval(c, env) for c in sel.columns] for _, env in rows]
# 5. order by
if sel.order_by is not None:
col, direction = sel.order_by
# we sort over the *original* rows (with env), then re-project. Simpler: sort
# the projected rows along with their key.
rows_with_keys = [(self._eval(col, env), p) for (_, env), p in zip(rows, projected)]
rows_with_keys.sort(key=lambda kp: kp[0], reverse=(direction == "DESC"))
projected = [p for _, p in rows_with_keys]
# 6. limit
if sel.limit is not None:
projected = projected[: sel.limit]
return projected
def _fetch(self, table: str):
if table not in self._tables:
raise ValueError(f"unknown table: {table}")
return self._tables[table]
def _eval(self, expr, env: dict[str, tuple[list[str], list]]):
if isinstance(expr, Literal):
return expr.value
if isinstance(expr, Column):
if expr.table is not None:
cols, row = env[expr.table]
return row[cols.index(expr.name)]
for cols, row in env.values():
if expr.name in cols:
return row[cols.index(expr.name)]
raise NameError(f"unknown column: {expr.name}")
if isinstance(expr, UnaryOp) and expr.op == "NOT":
return not self._eval(expr.operand, env)
if isinstance(expr, BinOp):
l = self._eval(expr.left, env)
r = self._eval(expr.right, env)
return {
"=": l == r, "!=": l != r,
"<": l < r, "<=": l <= r,
">": l > r, ">=": l >= r,
"AND": bool(l) and bool(r),
"OR": bool(l) or bool(r),
}[expr.op]
raise TypeError(f"bad expr: {expr}")
Tests
def setup_engine():
e = Engine()
e.register_table("users", ["id", "name", "age"], [
[1, "alice", 30], [2, "bob", 17], [3, "carol", 22], [4, "dave", 45],
])
e.register_table("orders", ["id", "user_id", "total"], [
[10, 1, 250], [11, 1, 50], [12, 3, 800], [13, 4, 75],
])
return e
def test_basic_select_star():
e = setup_engine()
assert e.query("SELECT * FROM users") == [
[1, "alice", 30], [2, "bob", 17], [3, "carol", 22], [4, "dave", 45]]
def test_where_int_compare():
e = setup_engine()
out = e.query("SELECT name FROM users WHERE age >= 18")
assert sorted(out) == [["alice"], ["carol"], ["dave"]]
def test_string_compare():
e = setup_engine()
out = e.query("SELECT name FROM users WHERE name = 'alice'")
assert out == [["alice"]]
def test_and_or_not():
e = setup_engine()
out = e.query("SELECT name FROM users WHERE age > 20 AND NOT name = 'dave'")
assert sorted(out) == [["alice"], ["carol"]]
def test_join():
e = setup_engine()
out = e.query(
"SELECT u.name, o.total FROM users u JOIN orders o ON u.id = o.user_id "
"WHERE o.total > 100"
)
assert sorted(out) == [["alice", 250], ["carol", 800]]
def test_order_by_desc_limit():
e = setup_engine()
out = e.query("SELECT name FROM users ORDER BY age DESC LIMIT 2")
assert out == [["dave"], ["alice"]]
def test_unknown_column_errors():
e = setup_engine()
try: e.query("SELECT bogus FROM users")
except NameError: pass
else: assert False
Follow-up Questions
- How would you test it? Layer-by-layer: tokenizer tests for every keyword/operator/punctuation; parser tests that pretty-print the AST and compare strings; executor tests against fixture tables. Property test: random valid queries with predictable outputs from a Python list-comprehension oracle.
- What is the consistency model? Single-threaded; reads see the snapshot at query start. For concurrent writes, copy-on-write tables or per-table read-write locks.
- What configuration knobs would you expose? Maximum query length, maximum result rows, query timeout. Don’t expose internals (tokenizer regex, parser lookahead).
- How would you handle a poison-pill input? Catastrophic regex (rare with the lexer above), deeply nested expressions (limit recursion depth), enormous joins (
R₁ × R₂row cap before executing). Bound everything. - How would you scale to N nodes? Beyond toy: shard tables by primary key range or hash; route queries to the owning node; for joins across shards, use distributed hash join. Real systems (Spanner, CockroachDB) layer query planning, distributed execution, and consensus over this same skeleton.
- What metrics would you emit? Per-query: parse latency, execution latency, rows scanned, rows returned. Per-table: row count gauge. Aggregate: queries-per-second counter, error rate counter.
Product Extension
This is the same skeleton DuckDB, SQLite, Postgres, and every database engine starts with: lex / parse / plan / execute. Real engines add a planner/optimizer between parse and execute that rewrites the AST (push down predicates, choose join order, pick indexes), and a storage layer beneath execute. Aggregate functions, group-by, subqueries, and CTEs are all extensions of the AST + executor pair.
Language/Runtime Follow-ups
- Python: dataclasses are the right shape for AST.
re.VERBOSElexer is concise. - Java: Use sealed interfaces (Java 17+) for AST nodes. ANTLR for the parser if available; hand-rolled recursive descent if not.
- Go: Use a
type Node interface { node() }and individual struct types implementing it. The parser is a struct with the token list and an index. - C++:
std::variantfor AST nodes is clean; visitor pattern viastd::visit. - JS/TS: Discriminated unions for AST. The runtime cost of dynamic dispatch is acceptable for an interview-grade engine.
Common Bugs
- Lexer that consumes whitespace as a token — pollutes the parser. Skip whitespace in the lexer.
- Parser that allows the same column twice in projection but then fails at execution — better to validate at parse time.
- JOIN executor that builds a Cartesian product before filtering — works but quadratic memory before predicate evaluation. Filter as you go (the implementation above does this).
- ORDER BY on a column not in projection — must evaluate against the row environment, not the projected output.
- Operator precedence wrong —
NOT a AND bparsed asNOT (a AND b)instead of(NOT a) AND b. The recursive-descent ladder (OR < AND < NOT < CMP) handles this. - Case-folding identifiers — many SQL engines do (Postgres folds to lowercase); this toy engine doesn’t. Document the choice.
Debugging Strategy
When parse fails: print the token stream up to the failure point and the parser’s _i index — almost always a missing keyword in the parse method (WHERE vs WHRE). When execution returns wrong rows: log the where evaluation per row with the values it sees. When joins explode: cap row count and emit an error rather than running unbounded.
Mastery Criteria
- Decomposed lex / parse / execute in <2 minutes.
- Wrote the recursive-descent expression parser with correct precedence.
-
Implemented
WHEREandSELECT colscorrectly in <30 minutes. - Added inner JOIN nested-loop in <10 minutes from the WHERE-only baseline.
-
Added
ORDER BYandLIMITin <10 minutes more. - Articulated where the optimizer would slot in (between parse and execute).
- Listed three real systems (DuckDB, SQLite, Postgres) using this skeleton.
- Wrote tokenizer + parser + executor tests independently per layer.
Phase 9 — Language & Runtime Deep Dive
Target level: Cross-cutting (applies at every level, but the bar rises sharply at senior+) Expected duration: 1–2 weeks of primary-language reading + 2–4 days of secondary-language skim Format: No labs. Five comprehensive language READMEs that double as interview-prep references. Companies this targets: Every company that asks “why does this work?” follow-ups — which is every company at L4+, and many at L3 as well.
Why This Phase Exists
Every other phase trains you to produce code that works. This phase trains you to explain why it works — and equally importantly, to recognize the silent ways it can stop working when an interviewer perturbs an input or asks a follow-up.
At the junior bar, “I called .sort() on the list” is a complete answer. At the senior bar, the interviewer will ask:
- “Is that sort stable?”
- “What’s the worst-case complexity in your language’s standard library?”
- “Does it allocate?”
- “What if the comparator throws?”
- “What if the same key compares differently across calls — what’s the consistency contract?”
- “If two of the elements are mutable and equal-by-hash, what happens to a
dictkeyed on them?”
A candidate who answers crisply — citing the language’s actual contract, naming the algorithm (Timsort, IntroSort, pdqsort), describing the auxiliary memory, and pointing out the one realistic failure mode — clears the senior bar without breaking a sweat. A candidate who hedges, says “uh, I’d have to check,” or worse, confidently says something wrong — does not.
The gap between these two candidates is rarely raw algorithmic ability. It’s runtime literacy: knowing your tools at the level your tools deserve.
Junior interviews ask “can you make the language do what you want?”. Senior interviews ask “do you know what the language is doing on your behalf?”. This phase exists because the second question is unbounded — every language has hundreds of subtle behaviors — and you cannot bluff your way through it under stopwatch pressure.
What “Language Depth” Actually Means In Interviews
There are five distinct kinds of language questions an interviewer can probe. Confusing them in your own head is one of the most common ways to under-prepare for this phase.
| Probe type | Example question | What it tests |
|---|---|---|
| Mechanical | “What does += do on a Python list vs a Python tuple?” | Did you actually use the language, or just write code in it? |
| Performance | “Why is ''.join(parts) faster than s += part?” | Do you know the cost model, not just the syntax? |
| Concurrency | “What does volatile guarantee in Java?” | Do you understand the memory model? |
| Failure mode | “What happens if __hash__ and __eq__ disagree?” | Can you predict subtle bugs you’ve never seen? |
| Idiom | “What’s the idiomatic way to read a file line-by-line?” | Would your code look native to a coworker? |
Every language-specific section in the phase below is structured around these five probe types. When you read them, mentally tag each subsection with which probe it answers. By the end you should be able to look at any language question and instantly classify it — and most candidates can answer two of the five but routinely fluff the other three.
The Five Tracks
| Track | Folder | Word count | Use cases |
|---|---|---|---|
| Python | python/README.md | ~10K | The default interview language; ML/data/SRE-leaning roles default here |
| Java | java/README.md | ~10K | FAANG-traditional, finance, Android, large enterprise backends |
| Go | go/README.md | ~7K | Infra, distributed systems, container/cloud-native (K8s, Docker, etcd shops) |
| C++ | cpp/README.md | ~10K | HFT/quant, game engines, embedded, browsers, databases, systems-programming roles |
| JavaScript / TypeScript | javascript-typescript/README.md | ~7K | Web frontend, Node backends, full-stack startup roles |
Each track is self-contained — you should not need to consult external references to answer the interview-relevant questions in that track. The READMEs are dense by design. Read your primary language linearly. Skim a secondary language. Ignore the others until/unless you switch.
How To Use This Phase
If you have one primary language
- Read your primary language README end-to-end. Don’t skip sections — even ones you “already know.” The bar in this phase is being able to answer a follow-up under stopwatch pressure, not having heard of the topic.
- For each section, after reading, close the page and explain the concept out loud (or to a rubber duck) in 60 seconds. If you can’t, re-read.
- Run every code example yourself. Many of them produce output that looks wrong until you internalize why it’s right. Reading the explanation without running the code leaves the wrongness un-felt.
- Cross-link backward to the labs in earlier phases: when you read the dict-internals section, revisit phase-01-foundations/labs/lab-03-hashmap-mastery.md. When you read the GIL section, revisit phase-08-practical-engineering/labs/lab-05-thread-pool.md. The readings are reference; the labs are practice; the combination is mastery.
- Write a one-line flashcard for every interview gotcha (“integer cache -5..256 in Python,” “Integer cache -128..127 in Java,” “loop variable capture pre-Go-1.22”). You will get drilled on at least 3 of these in any senior interview.
If you have a secondary language for breadth
Skim the README. Focus on the Common Interview Gotchas and Memory Model sections. Skip the standard library deep dive — you can look those up. Time budget: 1 evening per secondary language.
If you’re polyglot and want to know all five
You will not actually answer interviews fluently in five languages. Pick one primary, one secondary, and learn the rest as a hobby. Interview fluency requires hours of speaking-the-language-aloud practice that you cannot distribute across five tracks in any reasonable time budget.
What’s Deliberately Not In This Phase
- Build systems. No
setuptools/Maven/go.mod/ CMake /package.jsondeep dive. Interviews don’t probe these. - Framework-specific behavior. No Django, Spring, React, Express. Even at FAANG, framework knowledge is rarely on the rubric for a coding round.
- Tooling. No
pdb/gdb/delve/ Chrome DevTools tutorials. Phase 10 covers debugging methodology generically. - Trivia. “Which year was Python 3.0 released?” type questions are out. We focus on what an interviewer asks because they expect you to use the answer in your code.
- Esoterica. Python’s
__init_subclass__, Java’sMethodHandle, Go’sunsafe.Pointer, C++’splacement new— these are real but they’re rarely on a coding interview rubric. If you reach for them in an interview without being asked, you signal over-engineering.
The bias is toward what gets you points in an interview, not what makes you “complete” as a language nerd.
Mastery Checklist
You have completed Phase 9 when, in your primary language:
- You can describe the implementation of the standard hash map / dict, including its collision strategy, load factor, and adversarial-input behavior, in 90 seconds.
- You can describe how the language allocates memory: stack vs heap, GC strategy if any, when objects move, and what triggers a full collection.
- You can name three pitfalls in the language’s mutable-default-argument / late-binding-closure / iterator-invalidation territory and write code that demonstrates each.
- You can write thread-safe code idiomatically, naming which primitive you’d use (mutex / channel / atomic / actor) and why.
- For each of the standard collections (list/dict/set/heap/deque or their equivalents), you can state insert/lookup/delete complexity and one common gotcha.
-
You can answer “what does
==mean here?” precisely, distinguishing identity, value-equality, equals-with-typed-narrowing, and any platform-specific surprises. - You can describe the language’s concurrency model (event loop / GIL / OS threads / goroutines / fibers) in one paragraph and name the kind of work each is bad at.
- You can read a 50-line snippet in your secondary language and accurately predict its behavior on adversarial inputs (e.g., empty list, negative index, mutation during iteration).
- You have a one-line answer for every “common interview gotcha” in your primary language’s README and can produce code that demonstrates the gotcha live.
Exit Criteria
You may exit Phase 9 and move on to Phase 10 — Testing, Debugging & Correctness when:
- Primary-language depth. You’ve read the entire primary-language README and can answer 90% of the “common interview gotchas” subsection without re-reading.
- Cross-cutting fluency. When asked a follow-up like “how would you make this thread-safe?” or “what’s the memory cost of this collection?” during a Phase 8 lab review, you reach for primitives and reasoning from this phase, not from a generic OS course.
- Secondary-language familiarity. You can read code in at least one secondary language without grabbing a reference for basics, and you can identify two or three of the secondary language’s distinctive gotchas.
- Mock readiness. You’ve done at least one mock-09-runtime-language where the entire round was follow-up questions with no algorithmic component, and scored “passing” on the rubric.
If your primary-language depth is shallow but your algorithmic skill is strong, the senior interviewer will say “smart but green” — which is a no-hire at L5+. The fix is not more LeetCode. The fix is this phase.
A Note On Language Choice For Interviews
Pick a language. Do not switch mid-interview. Do not switch mid-loop. Do not arrive at an onsite saying “I usually do Python but I’ll do Java today because the problem is more concurrent.”
Default recommendation: Python for breadth (almost every company allows it), Java if you’re targeting traditional FAANG / finance, C++ if you’re targeting HFT or systems, Go if you’re targeting infrastructure-heavy companies (Cloudflare, Datadog, Snowflake’s data-plane teams, container/Kubernetes shops), JS/TS if you’re targeting web-leaning or Node-leaning roles.
The cost of switching languages is roughly 6 months of practice in the new language before you’re fluent at the senior bar. Do not switch on a whim.
Cross-References
- phase-01-foundations/ — the data-structure complexity tables here are the foundation; this phase deepens them with implementation specifics.
- phase-08-practical-engineering/ — every “Language/Runtime Follow-ups” callout in those labs is grounded here.
- phase-10-testing-debugging/ — debugging is largely “knowing what your runtime is doing”; that phase builds on this one.
- phase-11-mock-interviews/mocks/mock-09-runtime-language.md — the mock round dedicated to this phase.
- CODE_QUALITY.md — “use the language’s built-ins idiomatically” is one of the quality dimensions; this phase tells you what idiomatic actually means.
- FRAMEWORK.md — Step 16 (production implications) routinely cites runtime facts from this phase.
The Sub-Tracks
- Python — CPython internals, GIL, memory model, dict/list/set internals, asyncio, common gotchas
- Java — JVM, GC, JMM, collections framework, concurrency, generics erasure, modern Java
- Go — runtime, GMP scheduler, goroutines/channels, slices/maps internals, context, common bugs
- C++ — memory model, smart pointers, move semantics, STL complexity, undefined behavior, modern idioms
- JavaScript / TypeScript — V8, event loop, prototypes, this-binding, async/await, TS type system
Python Runtime Deep Dive
Target audience: candidates interviewing in Python at Big Tech, ML, infra, or any role where the interviewer is allowed to ask “how does it actually work?”
Scope: CPython. PyPy and other implementations are noted only when they materially change interview answers.
Python’s reputation for being “easy” is exactly why senior interviewers grill it hardest. The candidate who can write a clean two-pointer solution in Python and explain why their dict lookup is O(1) amortized but worst-case O(N), why their threading.Thread doesn’t help CPU-bound code, and why [[]] * 3 is a foot-gun, is rare. Be that candidate.
1. CPython Interpreter, Bytecode, Frame Objects
CPython is a stack-based bytecode interpreter. Source code → AST → bytecode → executed by an evaluation loop in C (ceval.c).
What runs your code
- Lexer/Parser → AST.
- Compiler → bytecode (
.pyccached in__pycache__/). - Interpreter loop (
PyEval_EvalFrameEx) → fetches one bytecode opcode at a time, dispatches.
import dis
def add(a, b):
return a + b
dis.dis(add)
# 2 0 RESUME 0
# 3 2 LOAD_FAST 0 (a)
# 4 LOAD_FAST 1 (b)
# 6 BINARY_OP 0 (+)
# 10 RETURN_VALUE
Frame objects
Every function call allocates a frame object on the Python call stack. A frame holds: locals, the value stack, the bytecode instruction pointer, the parent frame.
import sys
def f():
frame = sys._getframe()
print(frame.f_code.co_name, frame.f_lineno)
f() # f, <line>
Frames are heap-allocated objects, not C stack frames. This is why Python’s recursion limit is a Python-level integer (sys.setrecursionlimit), not a kernel limit.
Interview framing
“When you call a Python function, what’s the cost?”
Allocate a frame object, push it, populate locals from the argument tuple, execute bytecode, decref the frame. Function calls in Python are expensive — typically 100ns–1µs — which is why for over a list is faster than map(lambda…) for trivial bodies. Knowing this lets you defend choices like inline arithmetic vs operator.add.
2. The GIL — What It Is, What It Protects, When It Releases
The Global Interpreter Lock is a mutex inside the CPython interpreter. Only one thread can execute Python bytecode at a time per process.
What it protects
The GIL exists because CPython’s memory management (refcounts, GC structures, dict internals, etc.) is not thread-safe. Without the GIL, every refcount increment would need an atomic, killing single-threaded performance.
It protects interpreter state, not your data structures. list.append is atomic by accident (it’s a single bytecode), but counter += 1 is not (it’s LOAD, ADD, STORE).
When it releases
- I/O operations (file read/write, socket,
time.sleep) — the C extension drops the GIL while blocking. - Some C extensions explicitly drop it (NumPy heavy ops, hashlib).
- Every ~5ms (
sys.setswitchinterval) — the interpreter voluntarily releases for scheduling.
import threading, time
counter = 0
def bump():
global counter
for _ in range(1_000_000):
counter += 1 # NOT atomic
threads = [threading.Thread(target=bump) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()
print(counter) # NOT 4_000_000
Implications
- CPU-bound parallelism via
threadingis impossible in standard CPython. Usemultiprocessingor release the GIL via C extensions. - I/O-bound parallelism via
threadingworks. Each thread releases the GIL while waiting on the network. asynciois an alternative I/O model; it does not bypass the GIL — it doesn’t need to, because there’s only one thread anyway.
Free-threaded Python (3.13+)
PEP 703 introduced an optional no-GIL build (python3.13t). Refcounts become atomic, dict/list grow per-thread fast paths, GC adopts new locking. It is not the default and many C extensions break under it. For interview purposes:
“Python 3.13 ships an experimental free-threaded build that removes the GIL. It’s opt-in, slower for single-threaded code today, and not yet ABI-stable for the ecosystem. Default CPython still has the GIL.”
3. Memory Model — Refcounts + Generational GC
Every Python object has a reference count. When it hits zero, the object is freed immediately.
import sys
a = [1, 2, 3]
sys.getrefcount(a) # 2 — one for `a`, one for the argument to getrefcount
b = a
sys.getrefcount(a) # 3
del b
sys.getrefcount(a) # 2
Why we need a GC on top
Refcounts cannot collect cycles:
a = []
b = []
a.append(b)
b.append(a)
del a, b # Refcount of each is still 1 — they reference each other.
The generational tracing GC in the gc module sweeps for cycles. Three generations (0, 1, 2). Newly created containers go in gen 0. Survivors are promoted. Older generations are collected less often.
import gc
gc.collect() # force a full collection
gc.get_threshold() # (700, 10, 10)
__del__ pitfalls
__del__ is a finalizer, not a destructor. Two traps:
- Cycles with
__del__used to be uncollectable before Python 3.4. Now they are collected, but the order is unspecified. __del__may run during interpreter shutdown when module globals are alreadyNone.
class Bad:
def __del__(self):
print(open) # may be None during shutdown
Use weakref.finalize or context managers (with) instead.
weakref
A weakref does not increment the refcount. Useful for caches and observer patterns.
import weakref
class Node: pass
n = Node()
r = weakref.ref(n)
print(r()) # <Node>
del n
print(r()) # None
Interview framing
“How does Python free memory?”
Refcounting frees most things eagerly; a generational tracing collector cleans up cycles. Compared to Java, allocations are cheaper to free on the common path (no pauses on most exits) but every operation has a per-pointer atomic increment cost — which is part of why Python is slow.
4. Object Model — __slots__, Descriptors, MRO
Every Python object is, by default, a dict-backed thing: instance attributes live in __dict__. This is why Python objects are 5–10x larger than equivalent C structs.
__slots__
Declare attributes statically and the interpreter skips __dict__:
class Point:
__slots__ = ('x', 'y')
def __init__(self, x, y):
self.x, self.y = x, y
# ~56 bytes per Point with __slots__, ~328 bytes without (roughly).
__slots__ cost: no dynamic attribute addition. Subclasses that don’t redeclare __slots__ lose the optimization. Use them for value classes with millions of instances.
Descriptors
Properties, classmethods, staticmethods are all built on the descriptor protocol: an attribute access triggers __get__ / __set__ / __delete__ on the class attribute.
class Lazy:
def __init__(self, fn): self.fn = fn
def __get__(self, obj, cls):
v = self.fn(obj)
setattr(obj, self.fn.__name__, v)
return v
class C:
@Lazy
def expensive(self):
return sum(range(10**6))
MRO and C3
Multiple inheritance resolution uses the C3 linearization algorithm. It guarantees a deterministic order that respects: a class precedes its parents; left-to-right inheritance order.
class A: pass
class B(A): pass
class C(A): pass
class D(B, C): pass
print(D.__mro__)
# (D, B, C, A, object)
Interview framing
“Why is Python OO so slow?”
Every attribute access is a dict lookup on the instance, then a walk up the MRO if not found. __slots__, caching, and attrs/dataclass(slots=True) mitigate. JITs (PyPy) inline these.
5. Iterators, Generators, yield from
An iterator is any object with __iter__ and __next__. A generator is a function with yield; calling it returns an iterator without running the body.
def count_up(n):
for i in range(n):
yield i
g = count_up(3)
next(g) # 0
next(g) # 1
Generators suspend frame state on yield. The frame is heap-allocated, kept alive by the generator object, and resumed on next().
yield from
Delegates iteration to a sub-iterator and forwards send, throw, close.
def chain(a, b):
yield from a
yield from b
Why this matters
Generators are the foundation of asyncio (coroutines were generators before async def), pipelines, and lazy I/O. They allow processing infinite or huge sequences without materializing them.
def lines(path):
with open(path) as f:
for line in f: # iterator protocol over file
yield line.rstrip() # constant memory
# Process 100GB log: O(line) memory.
6. List Internals — Over-allocation, Amortized Append
list is a dynamic array of PyObject*. Capacity grows geometrically.
CPython’s growth pattern (in listobject.c):
new_size = (new_size + (new_size >> 3) + 6) & ~3
That’s roughly 1.125x growth (smaller than C++ vector’s ~1.5–2x).
| Operation | Complexity |
|---|---|
lst[i] | O(1) |
lst.append(x) | Amortized O(1), worst O(N) on resize |
lst.insert(0, x) | O(N) |
lst.pop() | O(1) |
lst.pop(0) | O(N) — use collections.deque |
x in lst | O(N) |
lst.sort() | O(N log N), Timsort, stable |
lst[a:b] | O(b-a), creates a copy |
lst = []
for i in range(1_000_000):
lst.append(i) # Amortized O(1) total O(N)
Pitfalls
grid = [[0] * 3] * 3 # WRONG — three references to one row
grid[0][0] = 1
print(grid) # [[1, 0, 0], [1, 0, 0], [1, 0, 0]]
grid = [[0] * 3 for _ in range(3)] # right
Interview framing
“Why does
list.appendaverage O(1)?”
Geometric growth: O(N) total work across N appends → O(1) amortized. The classic amortization proof.
7. Dict Internals — Open Addressing, Probing, Ordering
dict is a hash table with open addressing. Compact since 3.6 (split into a sparse index array + dense entries array). Insertion-ordered since 3.7 (language guarantee, not just CPython).
Lookup algorithm (simplified)
- Compute
hash(key) & (table_size - 1)→ slot. - If slot empty → not found. If key matches → hit.
- Else probe:
i = (5*i + 1 + perturb) % size; perturb >>= 5. The “perturb” trick mixes high bits of the hash into early probes, reducing clustering.
Hash randomization
hash(str) and hash(bytes) use a per-process random seed (since 3.3) to mitigate algorithmic-complexity DoS. PYTHONHASHSEED=0 disables it, useful for reproducibility but unsafe in production.
import os
os.environ['PYTHONHASHSEED'] # not set in interactive: random per process
hash("foo") # different across processes
hash(int) is the integer itself (mod a prime). hash(-1) is special-cased to -2 because -1 signals errors in C.
Worst case
Adversarial keys with colliding hashes degrade to O(N) per operation. Hash randomization defeats the basic attack but custom __hash__ returning a constant still breaks it.
class Bad:
def __hash__(self): return 0
def __eq__(self, other): return False # never equal — every insert collides
d = {}
for i in range(1000):
d[Bad()] = i # O(N) per insert → O(N²) total
Complexity table
| Operation | Avg | Worst |
|---|---|---|
d[k] | O(1) | O(N) |
d[k] = v | O(1) amortized | O(N) |
del d[k] | O(1) | O(N) |
k in d | O(1) | O(N) |
| iter | O(N) | O(N) |
Interview framing
“Why are Python dicts ordered?”
Compact dict (3.6 CPython) stored entries in insertion order in a dense array, with a sparse index. The ordering was an implementation detail, then promoted to a language guarantee in 3.7.
8. Set Internals
set and frozenset are open-addressed hash tables, mechanically the same as dict minus the value column. Same complexity table, same adversarial caveats.
s = {1, 2, 3}
s | {4} # union, O(len(self) + len(other))
s & {2, 3, 5} # intersection, O(min(...))
s - {2} # difference, O(len(self))
Sets are not insertion-ordered. Do not rely on iteration order.
9. String Internals — Interning, Encoding, bytes vs str
Python 3 strings (str) are immutable Unicode code-point sequences. CPython stores them as one of:
- Latin-1 (1 byte/char) when all code points fit.
- UCS-2 (2 bytes/char) up to U+FFFF.
- UCS-4 (4 bytes/char) for the full range.
This is PEP 393 (“flexible string representation”). A string with a single emoji is 4× the bytes per char of a pure-ASCII string of the same length.
Interning
Short strings that look like identifiers are auto-interned. Equal interned strings share the same object → is works (by accident).
"hello" is "hello" # True (CPython, syntactic literals)
a = "hello world"
b = "hello world"
a is b # CPython 3.x: often True, but DO NOT RELY ON THIS
sys.intern(s) forces interning for runtime-built strings; speeds up dict lookups when the same key is used many times.
Concat in loop is O(N²)
s = ""
for c in chars:
s += c # creates a new string each time
CPython has a special-case optimization that sometimes makes this O(N) (when the refcount of s is 1 and the allocator can extend in place), but it is not guaranteed and disappears under any other reference. Use "".join(chars).
bytes vs str
bytes is an immutable byte sequence. str is a Unicode sequence. They do not implicitly convert in Python 3.
b"abc" + "xyz" # TypeError
b"abc".decode("utf-8") # → "abc"
"abc".encode("utf-8") # → b"abc"
Network/file boundaries are bytes. Application logic is strings. Convert at the boundary, never in the middle.
Interview framing
“Why does
s += cin a loop blow up?”
Strings are immutable, so each append allocates a new string and copies. CPython has an opportunistic in-place-extend hack that hides this in toy examples, but it’s fragile. Always "".join.
10. Hashing Protocol and __hash__ Contract
Two objects that compare equal must have the same hash. The reverse is not required.
class Point:
def __init__(self, x, y): self.x, self.y = x, y
def __eq__(self, other):
return isinstance(other, Point) and (self.x, self.y) == (other.x, other.y)
def __hash__(self):
return hash((self.x, self.y))
If you define __eq__ and not __hash__, your class becomes unhashable (Python sets __hash__ to None). This is by design — overriding equality without hash is almost always a bug.
Mutable types (list, dict, set) are unhashable by default — their hash would have to change as they mutate, breaking the dict/set invariant.
11. Mutable Default Arguments — The Most Famous Trap
def append_to(x, lst=[]):
lst.append(x)
return lst
append_to(1) # [1]
append_to(2) # [1, 2] ← !
Default values are evaluated once, when the def statement runs, and shared across calls. Idiomatic fix:
def append_to(x, lst=None):
if lst is None: lst = []
lst.append(x)
return lst
Every Python interviewer asks this once a year. Get it right and move on.
12. Closures and Late Binding
Free variables in closures are looked up by name at call time, not captured by value at definition.
funcs = [lambda: i for i in range(3)]
[f() for f in funcs] # [2, 2, 2] — not [0, 1, 2]
Fix with default arg (evaluated at def):
funcs = [lambda i=i: i for i in range(3)]
[f() for f in funcs] # [0, 1, 2]
This is the same bug as JavaScript’s var i in a loop. Both languages punish late binding.
13. Concurrency — Threading vs Multiprocessing vs Asyncio
| Model | Parallelism | Use For | Cost |
|---|---|---|---|
threading | Concurrent (GIL) | I/O-bound | Cheap threads (~MB stack), context switches |
multiprocessing | Parallel (separate processes) | CPU-bound | Process startup, IPC pickling |
asyncio | Concurrent (single thread) | High-fanout I/O | No OS threads; cooperative |
concurrent.futures | Wraps either | Convenient API | — |
Picking
- 10K simultaneous network connections?
asyncio. - 10 simultaneous network calls?
threading(or asyncio). - Heavy NumPy computation?
threading— NumPy releases the GIL. - Pure-Python CPU work?
multiprocessingor write the hot loop in C/Cython/Numba.
14. AsyncIO Model — Event Loop, Coroutines, Don’t Block The Loop
asyncio runs an event loop in one thread. Coroutines (async def) yield control on await, the loop schedules another coroutine that’s ready.
import asyncio
async def fetch(url):
print(f"start {url}")
await asyncio.sleep(1) # yields control
print(f"done {url}")
async def main():
await asyncio.gather(*[fetch(u) for u in ["a", "b", "c"]])
asyncio.run(main())
# All three start, all three finish ~1s later — concurrent on one thread.
Blocking the loop
If you do CPU work or call a sync blocking I/O function inside a coroutine, the entire loop stalls. Symptom: latency spikes for everyone.
async def bad():
time.sleep(1) # blocking — stalls the loop
requests.get("http://x") # blocking — same problem
async def good():
await asyncio.sleep(1)
async with aiohttp.ClientSession() as s:
await s.get("http://x")
For unavoidable blocking work: await loop.run_in_executor(None, blocking_fn).
Interview framing
“What’s the difference between
asyncioandthreading?”
Threading is preemptive multi-tasking by the OS, threads share memory, GIL serializes. Asyncio is cooperative single-thread; coroutines must await to yield. Both win on I/O. Asyncio scales to more in-flight ops because there’s no per-task OS thread.
15. Multiprocessing — Fork vs Spawn, Pickling
multiprocessing creates separate Python processes. Each has its own GIL → real parallelism.
Start methods
| Method | Default On | Cost | Caveat |
|---|---|---|---|
fork | Linux (was default until 3.14) | Cheap | Copy-on-write; not safe with threads, locks, or libraries that aren’t fork-safe (e.g. CUDA). |
spawn | macOS, Windows; default on Linux 3.14+ | Slow (re-imports) | All args must be picklable. |
forkserver | Linux | Mid | Compromise |
Pickling
Args and return values cross process boundaries via pickle. Lambdas, local functions, and many file/socket objects are not picklable.
from multiprocessing import Pool
def square(x): return x * x # top-level — picklable
with Pool(4) as p:
print(p.map(square, range(10)))
Shared memory
multiprocessing.shared_memory.SharedMemory (Python 3.8+) for zero-copy NumPy/byte sharing. Avoids the pickling round-trip for big arrays.
16. NumPy / Vectorization
NumPy stores numbers in contiguous C arrays of native types, not as PyObject*. Operations dispatch to optimized C/SIMD that releases the GIL.
import numpy as np
a = np.arange(1_000_000)
b = a * 2 + 1 # vectorized, ~ms; pure Python equivalent ~100ms
A for loop over a NumPy array is the worst of both worlds: Python overhead per iteration, no SIMD. If the operation has no NumPy expression, fall back to Numba, Cython, or a C extension.
This is a sneaky interview line: “Implement vector dot product without NumPy” → straightforward Python, then “now optimize” → vectorize, then “now scale” → talk about BLAS underneath NumPy.
17. Common Interview Gotchas
Integer caching
a = 256; b = 256
a is b # True — CPython caches -5..256
a = 257; b = 257
a is b # False (not guaranteed True; do NOT use `is` for value compare)
is vs ==
is is identity (same object). == is equality (__eq__). Use == for values. Use is for None, True, False, and singletons.
Sort stability
sorted() and list.sort() use Timsort — stable. You can sort by multiple keys via successive stable sorts (least significant first).
data = [("a", 2), ("b", 1), ("a", 1)]
data.sort(key=lambda x: x[1])
data.sort(key=lambda x: x[0]) # stable preserves the previous order on ties
dict.get default mutation
d = {}
d.setdefault("k", []).append(1) # one allocation
# vs
d["k"] = d.get("k", []) + [1] # quadratic for many appends
Use collections.defaultdict(list) if you append a lot.
Truthy surprises
bool([]) # False
bool([0]) # True (non-empty list)
bool(0.0) # False
bool("False") # True (non-empty string)
18. Recursion Limits
import sys
sys.getrecursionlimit() # default 1000
sys.setrecursionlimit(10000)
Python frames are heap-allocated but each is non-trivial (~500 bytes). Setting the limit too high crashes the interpreter on stack overflow of the C stack.
Convert deep recursion to iteration with an explicit stack (Phase 1, Lab 8). This isn’t optional in interviews — recursion depth = N in tree problems with skewed inputs is real.
19. Performance Hot Tips
-
Avoid attribute lookup in hot loops: bind to a local first.
append = result.append # local — fastest opcode for x in data: append(transform(x)) -
Built-ins are C.
sum,min,max,any,all,map,sorted— written in C, beat hand-rolled loops. -
Comprehensions beat
for+append. They skip theLOAD_ATTRforappend. -
functools.lru_cachefor memoization — drop-in, fast. -
String formatting: f-strings >
%>.format()> concatenation. -
__slots__for value classes with millions of instances. -
Profile before optimizing.
cProfile,pyinstrument,py-spy(sampling, no code changes).
import cProfile
cProfile.run("expensive()", sort="cumulative")
20. Standard Library Essentials
collections
Counter— multiset;most_common(k).deque— O(1) appends/pops at both ends. Use for queues and sliding windows.defaultdict— auto-vivifying dict.OrderedDict— historically ordered; todaydictis too. Use OrderedDict only formove_to_endand reverse iteration.namedtuple— lightweight value class.dataclass(frozen=True, slots=True)is the modern alternative.
heapq
Min-heap on a list. No max-heap — negate values.
import heapq
h = []
heapq.heappush(h, 3)
heapq.heappush(h, 1)
heapq.heappop(h) # 1
heapq.nlargest(3, data) # k-largest
bisect
Binary search on a sorted list.
import bisect
i = bisect.bisect_left(sorted_arr, x) # insertion point
bisect.insort(sorted_arr, x) # O(N) insert
itertools
chain,cycle,repeatcombinations,permutations,productaccumulate(prefix sums!),groupby,pairwise(3.10+)
from itertools import accumulate, pairwise
list(accumulate([1, 2, 3, 4])) # [1, 3, 6, 10]
list(pairwise([1, 2, 3, 4])) # [(1,2), (2,3), (3,4)]
functools
lru_cache,cache(3.9+).reduce.partial.cached_property.
What To Memorize Cold
- GIL releases on I/O and at switch interval; doesn’t release for pure-Python CPU.
dictis open-addressed, ordered since 3.7, hash-randomized.- List growth ~1.125×. Append amortized O(1).
strimmutable, three width tiers (1/2/4 byte). Concat in loop ⇒"".join.isvs==. Integer cache -5..256.- Mutable default args evaluated once.
- Closure late binding fix:
lambda x=x: …. - Threading for I/O, multiprocessing for CPU, asyncio for fanout.
__hash__must agree with__eq__.heapqis min-only.
If any of those is fuzzy, re-read this document. Then code something that breaks because of it, on purpose. That’s the lesson that sticks.
JavaScript & TypeScript Runtime Deep Dive
Target audience: candidates interviewing for frontend, full-stack, or Node.js backend roles where the interviewer probes “what does the event loop actually do?”, “explain
this”, “why istypeof null === 'object'”, or “how does TypeScript narrow this union?”Scope: V8 (Chrome / Node) primarily, with mentions of SpiderMonkey (Firefox) and JSC (Safari) where they diverge. TypeScript 5.x.
JS sits in an awkward place: senior interviewers know the language is full of warts and they will use them. Memorizing trivia is necessary but not sufficient. The leverage comes from understanding the engine model and the type system’s structural reasoning.
1. V8 Internals — Ignition + TurboFan
V8 (and similar engines) compile JS through a pipeline:
- Parser → AST.
- Ignition — bytecode interpreter. Fast startup.
- TurboFan — optimizing JIT. Profiles hot code, generates speculative machine code.
- Deoptimization — when speculation breaks (a function suddenly receives a different type), TurboFan bails back to Ignition.
function add(a, b) { return a + b; }
// Called 10000x with (number, number) → TurboFan compiles it to fast int add.
add("foo", "bar"); // Type changed → deopt, recompile or bail out.
Hidden classes (shapes / maps)
V8 tracks the structural “shape” of objects internally — what fields exist in what order. Each property add changes the hidden class. Two objects with the same hidden class share a fast property layout.
function Point(x, y) { this.x = x; this.y = y; }
const a = new Point(1, 2);
const b = new Point(3, 4);
// a and b share a hidden class → fast.
a.z = 5; // a's hidden class diverges from b's → slower.
Inline caches (ICs)
Property access (obj.x) is monitored. If the hidden class is consistent, the IC fast-paths to a direct memory offset. Polymorphic ICs (multiple shapes) are slower; megamorphic (>4) drops to a hash lookup.
Interview takeaway: initialize all properties in the constructor in the same order. Don’t add/delete properties dynamically in hot code.
2. Event Loop — Tasks, Microtasks, RAF
The browser/Node runs one JS thread. Concurrency comes from yielding back to the event loop.
┌─────────────────────────┐
│ Call Stack │
└────────┬────────────────┘
│ runs to completion
┌────────▼────────────────┐
│ Microtask Queue │ ← Promises, queueMicrotask, MutationObserver
└────────┬────────────────┘
│ drained fully
┌────────▼────────────────┐
│ Task Queue │ ← setTimeout, setInterval, I/O, UI events
└─────────────────────────┘
Order of execution:
- Run the current synchronous code to completion.
- Drain the entire microtask queue.
- Pick one task from the task queue.
- Repeat from step 2.
console.log('a');
setTimeout(() => console.log('b'), 0);
Promise.resolve().then(() => console.log('c'));
console.log('d');
// a, d, c, b
Microtasks starve macrotasks if they keep enqueueing themselves:
function loop() { Promise.resolve().then(loop); }
loop(); // freezes the event loop — UI never repaints
requestAnimationFrame
Browser-only. Fires before the next paint, ~60fps. Use for animation; cheaper than setInterval(_, 16) because it’s coalesced with paint.
Node specifics
Node uses libuv. Its event loop has phases (timers, pending callbacks, poll, check, close). process.nextTick runs before microtasks (a Node-only queue with even higher priority than Promise microtasks).
process.nextTick(() => console.log('next'));
Promise.resolve().then(() => console.log('promise'));
// next, promise
3. async / await
async functions return a Promise. await desugars to .then.
async function f() {
const a = await fetch1();
const b = await fetch2(a);
return b;
}
// equivalent to:
function f() {
return fetch1().then(a => fetch2(a));
}
await x where x is not a thenable wraps it in Promise.resolve(x).
Sequential vs parallel
// Sequential (slow if independent):
const a = await op1();
const b = await op2();
// Parallel (correct when independent):
const [a, b] = await Promise.all([op1(), op2()]);
Default to Promise.all when the operations are independent — common interview ask.
Errors
try/catch works on await. An unhandled rejection in async context surfaces as unhandledrejection (browser) / process.on('unhandledRejection') (Node).
async function f() {
try {
await mayReject();
} catch (e) {
// handle
}
}
4. Promise Gotchas
-
A promise is not “the running operation” — it represents a value that will exist later. The work is already started before the Promise is constructed (in most APIs).
-
Errors thrown inside
.thencallbacks become rejections of the chain. -
Forgetting
returninside.thenbreaks chaining:p.then(x => { doSomething(x); // forgot return — next .then sees undefined }).then(use); -
Promise.allshort-circuits on first rejection. UsePromise.allSettledto wait for all and inspect. -
Unhandled rejection is now noisy in Node and the browser. Always attach a
.catchorawaitinsidetry/catch. -
Promiseis not cancelable. AbortController + AbortSignal pattern handles cancellation explicitly.
const ctrl = new AbortController();
fetch(url, { signal: ctrl.signal });
// later:
ctrl.abort();
5. Memory Model and GC
V8’s heap has generational GC:
- Young generation (Scavenger / semi-space copying): minor GC, very fast (~ms). Most objects die young.
- Old generation (Mark-sweep / mark-compact): major GC. Concurrent marking, parallel sweeping, incremental compaction.
Memory leaks in JS
- Unintentional globals — assigning to a name without
let/const(in non-strict mode) creates a global, never collected. - Closures — capturing large objects in long-lived callbacks.
- Event listeners — not removed when DOM nodes are detached.
- Timers —
setIntervalcallbacks retain captured state forever. - Detached DOM — references to removed DOM nodes from JS keep them alive.
Map/Set— keys held strongly. UseWeakMap/WeakSetfor “annotations on objects.”
Profiling
Chrome DevTools → Memory → heap snapshot, allocation timeline. Look for retained sizes and detached DOM trees.
6. Object Model — Prototypes
Every object has an internal [[Prototype]] (accessed via Object.getPrototypeOf or __proto__). Property lookup walks the prototype chain.
const a = { x: 1 };
const b = Object.create(a);
b.y = 2;
b.x; // 1 — looked up on a
Object.getPrototypeOf(b) === a; // true
prototype (the property) vs __proto__ (the link)
Foo.prototype is the object that becomes __proto__ of instances created with new Foo().
function Foo() {}
const f = new Foo();
f.__proto__ === Foo.prototype; // true
Foo.prototype.__proto__ === Object.prototype;
class
class is sugar over prototypes. The methods live on Foo.prototype.
class Foo {
greet() { return 'hi'; }
}
typeof Foo.prototype.greet; // 'function'
Object.create(null)
A “dictionary object” with no prototype — no inherited properties from Object.prototype. Useful as a hash map.
7. this Binding
JS binds this at call site, not at definition. Five rules in priority order:
new:this= new instance.call/apply/bind: explicit binding wins.- Method call (
obj.f()):this=obj. - Plain call (
f()):this=undefinedin strict mode, global object otherwise. - Arrow functions: no own
this— inherit from surrounding lexical scope.
const obj = { x: 1, f() { return this.x; } };
const g = obj.f;
obj.f(); // 1
g(); // undefined (strict) — `this` is global / undefined
class C {
val = 42;
arrow = () => this.val; // bound to instance
method() { return this.val; }
}
const c = new C();
const a = c.arrow;
const m = c.method;
a(); // 42 — arrow captured `this`
m(); // TypeError — method lost `this`
8. Closures, var / let / const, Scope
A closure is a function plus the lexical environment it was created in.
function counter() {
let n = 0;
return () => ++n;
}
const c = counter();
c(); c(); c(); // 1, 2, 3
var (function-scoped, hoisted)
console.log(x); // undefined (hoisted, not initialized)
var x = 1;
let / const (block-scoped, TDZ)
console.log(y); // ReferenceError — TDZ
let y = 1;
The “Temporal Dead Zone” is the period between block entry and the let/const declaration. Accessing the binding in TDZ throws.
Loop var capture
for (var i = 0; i < 3; i++) setTimeout(() => console.log(i), 0);
// 3 3 3 — single `i`, all callbacks share it
for (let i = 0; i < 3; i++) setTimeout(() => console.log(i), 0);
// 0 1 2 — fresh `i` per iteration
This is the classic JS interview question. Use let.
9. Equality
===strict — same type and value, withNaN !== NaNand+0 === -0.==loose — type coercion. Don’t use it except forx == null(matches bothnullandundefined).Object.is(a, b)— like===butObject.is(NaN, NaN) === trueandObject.is(+0, -0) === false.
NaN === NaN; // false
Object.is(NaN, NaN); // true
1 == '1'; // true (coercion)
[] == false; // true (!)
[] == ![]; // true (!)
The == rules are an interview trap. Memorize only the null/undefined exception; use === everywhere else.
10. Top Gotchas
typeof null === 'object'
A historical bug, never fixed for compatibility. Test for null with === null.
parseInt radix
parseInt('010'); // 10 in modern engines (used to be 8)
parseInt('010', 10); // always 10 — pass the radix.
Always pass 10. ESLint enforces this.
Floating-point
0.1 + 0.2; // 0.30000000000000004
0.1 + 0.2 === 0.3; // false
Use Number.EPSILON tolerance, or bigint for exact integer arithmetic.
Array coercion
[] + []; // ''
[] + {}; // '[object Object]'
{} + []; // 0 (in some contexts — `{}` parsed as block)
Don’t + non-numbers. Use template literals or explicit String(x).
== and falsy
0 == ''; // true
0 == '0'; // true
'' == '0'; // false
This is why === exists.
for...in vs for...of
for (const k in obj)iterates enumerable string-keyed properties (includes inherited!).for (const v of iterable)iterates iterable’s values.
const arr = [1, 2, 3];
for (const i in arr) console.log(i); // '0' '1' '2' — strings, indexes
for (const v of arr) console.log(v); // 1 2 3
Don’t use for...in on arrays.
delete on array
delete arr[i] leaves a hole (sparse array), doesn’t shorten. Use splice.
11. Map vs Object
Object | Map | |
|---|---|---|
| Keys | Strings & symbols | Anything |
| Iteration | Object.keys/entries, no order guarantee for non-int | Insertion order |
| Size | Object.keys(o).length (O(N)) | m.size (O(1)) |
| Inheritance pollution | Yes (__proto__, toString…) | No |
| JSON | Yes | No (need conversion) |
Use Map when:
- Keys are dynamic strings (esp. user input).
- You need any-typed keys.
- Insertion order matters.
- You add/remove keys frequently.
Use Object when:
- Keys are known compile-time / config-shaped.
- You’ll JSON-serialize.
- You’re using TypeScript’s structural types.
12. Set, WeakMap, WeakSet
Set— collection of unique values; insertion order.WeakMap— keys are objects, weakly held. If the key is GC’d, the entry disappears. Not iterable. Use for “annotations on objects.”WeakSet— set of weakly-held objects. Use for “have I seen this object?” without preventing GC.
const tags = new WeakMap();
function tag(node, value) { tags.set(node, value); }
// when `node` is GC'd, the tag is gone too.
Practical use: caches keyed on DOM nodes, private state on objects, libraries that observe but don’t own.
WeakRef (newer)
new WeakRef(obj) lets you hold a weak reference and dereference it (.deref()) later, getting the object or undefined if collected. Niche — you probably don’t need it.
13. TypeScript — Structural Typing & Generics
TypeScript types are structural. If two types have the same shape, they’re compatible.
interface Named { name: string }
const u: Named = { name: 'a', age: 30 }; // OK — extra props allowed in this position
function greet(p: Named) { return p.name; }
greet({ name: 'a', extra: 1 } as any);
Generics
function identity<T>(x: T): T { return x; }
identity<number>(42);
identity('hi'); // T inferred as string
Constraints
function len<T extends { length: number }>(x: T): number { return x.length; }
Conditional types
type IsString<T> = T extends string ? true : false;
type A = IsString<'hi'>; // true
type B = IsString<42>; // false
Mapped types
type Partial<T> = { [K in keyof T]?: T[K] };
type Readonly<T> = { readonly [K in keyof T]: T[K] };
Utility types
Partial<T>, Required<T>, Pick<T, K>, Omit<T, K>, Record<K, V>, ReturnType<F>, Parameters<F>, Awaited<T>. Memorize the names; they come up.
14. TS Narrowing
The control-flow analyzer narrows union types based on runtime checks.
function f(x: string | number) {
if (typeof x === 'string') {
x.toUpperCase(); // narrowed to string
} else {
x.toFixed(2); // narrowed to number
}
}
Narrowing operators
-
typeof→"string","number","boolean","undefined","object","function","symbol","bigint". -
instanceof→ for class instances. -
inoperator:if ('foo' in obj). -
Equality:
if (x === null),if (x === undefined). -
Discriminated unions:
type Result = { ok: true; value: string } | { ok: false; error: Error }; function f(r: Result) { if (r.ok) r.value; // OK else r.error; // OK } -
User-defined type guards:
function isString(x: unknown): x is string { return typeof x === 'string'; } -
Assertion functions:
function assertNumber(x: unknown): asserts x is number { if (typeof x !== 'number') throw new Error('not a number'); }
Exhaustiveness with never
type Shape = { kind: 'circle' } | { kind: 'square' };
function area(s: Shape): number {
switch (s.kind) {
case 'circle': return ...;
case 'square': return ...;
default: const _: never = s; throw new Error('unreachable');
}
}
The never assignment fails to compile if a new variant is added — catches missing cases.
15. Performance Tips
- Stable hidden classes — set all properties in the constructor in the same order. Don’t add later.
- Avoid
deleteon hot objects — it transitions to dictionary mode. - Monomorphic functions — call them with the same shapes. Polymorphic = slower.
- Typed arrays for numeric work —
Float64Array,Int32Array. Pre-allocated, contiguous, no boxing. - Avoid
argumentsin hot code; use...rest.argumentsdefeats some optimizations. foroverforEachin hot loops — slightly faster, no callback overhead. Less true with modern engines but still measurable on tight loops.- Pre-compile regexes — declare at module scope, not inside functions.
- Avoid leaking with
try/catchin hot functions on old V8 (pre-2017). Modern V8 handles it; not a real concern anymore. - Profile before optimizing. Chrome DevTools Performance tab; Node
--profandclinic.js. - Reduce object churn — V8 likes long-lived monomorphic objects.
// Bad — creates new shapes per call:
function pt() { return { x: 1, y: 2 }; }
// Better — a class V8 can specialize:
class Pt { constructor(x, y) { this.x = x; this.y = y; } }
16. Node vs Browser
| Node | Browser | |
|---|---|---|
| Globals | process, Buffer, __dirname, global | window, document, navigator |
| Modules | CommonJS (require) + ESM | ESM + bundlers |
| I/O | libuv: fs, net, dns, child_process | fetch, Web APIs |
| DOM | None (use jsdom if needed) | Yes |
| Threads | worker_threads, cluster | Worker, SharedArrayBuffer |
libuv thread pool
Node uses a thread pool (default 4) for fs, dns.lookup, crypto.pbkdf2, etc. — anything that can’t be epoll-ed.
UV_THREADPOOL_SIZE=16 node app.js
Network I/O is not on the thread pool; it’s on the event loop using async syscalls.
worker_threads vs cluster
worker_threads— separate JS thread, separate event loop, can share memory viaSharedArrayBuffer. Use for CPU-bound work.cluster— multiple processes, no shared memory, IPC via channels. Use for scaling HTTP servers across cores.
process vs globalThis
globalThis (ES2020) is the universal global object — works in browser, Node, workers.
17. What To Memorize Cold
- V8 pipeline: Ignition (bytecode) → TurboFan (optimizing JIT). Hidden classes + ICs drive speed. Property order matters.
- Event loop: sync runs to completion → drain microtasks → one task → repeat. Microtasks include Promises, queueMicrotask.
- Node nextTick > Promise microtask > timers/IO.
async/awaitdesugars to Promises. UsePromise.allfor independent work.- Promises: not cancelable, errors → rejection, must catch. AbortController for cancellation.
- GC: generational. Leaks: globals, closures, listeners, timers, detached DOM. WeakMap/WeakSet for object-keyed metadata.
- Prototype chain:
__proto__link,prototypeproperty on functions/classes.classis sugar. thisrules: new > call/apply/bind > method > default. Arrow inherits lexical.varhoisted/function-scoped,let/constblock-scoped + TDZ. Loop var capture: uselet.- Equality:
===always;Object.isfor NaN/±0;==only forx == null. - Top traps:
typeof nullis"object",parseIntradix, FP math,for...invsfor...of,deleteon array. MapvsObject: Map for dynamic keys, any types, ordered, fastsize.WeakMapfor object-keyed weak metadata.- TS structural typing. Utility types: Partial/Required/Pick/Omit/Record/ReturnType. Conditional + mapped types.
- TS narrowing:
typeof,instanceof,in, discriminated unions, type guards, assertion functions,neverfor exhaustiveness. - Perf: stable hidden classes, monomorphic call sites, typed arrays for numeric, pre-compile regex, profile.
- Node: libuv thread pool for fs/dns/crypto.
worker_threads(CPU) vscluster(HTTP scale).
JS is forgiving until it isn’t. The interviewer will test the spots where it isn’t. Fluency on the event loop, this, equality, and TypeScript narrowing usually decides senior-level outcomes.
Java Runtime Deep Dive
Target audience: candidates interviewing in Java at FAANG, finance, Android, or any backend role where the interviewer is allowed to probe “what does the JVM do here?”
Scope: HotSpot JVM (OpenJDK 21+ baseline), with notes on JDK 17 LTS where behavior diverges. Other JVMs (OpenJ9, GraalVM native-image) are noted only where they change interview-grade answers.
Java’s verbosity makes it easy to mistake “I write Java daily” for “I know Java.” A senior interviewer will quickly find the gap by asking what Integer i = 200; Integer j = 200; i == j returns, why your HashMap has O(log N) worst-case lookup since Java 8, and what volatile actually guarantees. This guide closes the gap.
1. JVM Architecture
The JVM is a stack-based virtual machine with a tiered execution model.
.java ── javac ──► .class (bytecode)
│
▼
┌─────────────────────┐
│ Class Loader │ (Bootstrap → Platform → App)
└────────┬────────────┘
▼
┌─────────────────────┐
│ Runtime Data Areas │ Heap, Method Area / Metaspace,
│ │ Stacks (per-thread), PC Reg, Native Stack
└────────┬────────────┘
▼
┌─────────────────────┐
│ Execution Engine │ Interpreter ↔ C1 (client) ↔ C2 (server)
│ │ + Tiered Compilation + OSR
└─────────────────────┘
Tiered compilation (default since Java 8): hot methods are compiled by C1 (fast, lower-quality code) and after enough invocations re-compiled by C2 (slower, high-quality, profile-guided). OSR (on-stack replacement) lets a long-running interpreted loop be replaced by JITed code mid-flight.
// Hot loop — JIT will inline, unroll, and vectorize this.
long sum = 0;
for (int i = 0; i < 1_000_000_000; i++) sum += i;
To see what the JIT does:
java -XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly Main # needs hsdis
java -XX:+PrintCompilation -XX:+PrintInlining Main
Class loaders
Three default loaders form a delegation chain:
- Bootstrap — loads
java.base(rt.jar in old days). - Platform — loads
java.*modules outsidejava.base. - App — loads your classpath.
Delegation rule: every loader asks its parent first. This is why you cannot shadow java.lang.String with your own.
Interview framing
“What happens when I call a Java method?”
The bytecode invokevirtual looks up the method via the receiver’s class vtable, the interpreter executes it, profile data accumulates, and after a threshold the JIT compiles a specialized version. Subsequent calls jump to native code.
2. Memory Regions
| Region | Per-thread? | Holds | GC? |
|---|---|---|---|
| Heap (Young + Old) | No | All Java objects | Yes |
| Metaspace | No | Class metadata, method bytecode | Yes (rare) |
| JVM Stack | Yes | Frames: locals, operand stack, return | No (LIFO) |
| PC Register | Yes | Current bytecode index | No |
| Native Stack | Yes | C stack for JNI / runtime | No |
| Code Cache | No | JIT-compiled native code | Evicted |
Heap structure under generational collectors (G1, Parallel, Serial):
Young Generation: Eden + Survivor 0 + Survivor 1
Old Generation: tenured objects
Allocation goes to Eden (bump-pointer in a thread-local allocation buffer — TLAB — so it’s lock-free). When Eden fills, a minor GC moves live objects to a Survivor space. Objects that survive enough minor GCs are promoted to Old.
// Each call allocates in Eden — extremely fast (just bump a pointer in TLAB).
List<Integer> tmp = new ArrayList<>();
Common JVM flags
-Xms2g -Xmx2g # initial / max heap
-XX:MetaspaceSize=256m
-XX:+UseG1GC # default since 9
-XX:MaxGCPauseMillis=200 # G1 pause target
-XX:+HeapDumpOnOutOfMemoryError
Interview framing
“Where does
new ArrayList<>()live?”
In Eden, on the heap. The reference variable lives in the current frame on the JVM stack. Allocation is bump-pointer in a TLAB; collection is generational copying.
3. Garbage Collectors
Java has many GCs. Know these four:
| GC | When | Pause | Trade-off |
|---|---|---|---|
| Serial | Tiny heaps / single-CPU | Stop-the-world | Simplest, smallest |
| Parallel (Throughput) | Batch jobs | STW, multi-thread | Max throughput, ignores pause |
| G1 (default) | General server | Soft target ms-scale | Balances throughput + pause |
| ZGC | Low-latency services | Sub-ms (since 21 generational) | Concurrent, region-based |
| Shenandoah | RH-flavored ZGC analog | Sub-ms | Concurrent, region-based |
G1 in 90 seconds
Heap is split into ~2000 equal-size regions of 1–32 MB. Regions are tagged Eden / Survivor / Old / Humongous. G1 maintains a remembered set per region tracking incoming references so it can collect a subset of regions (“the collection set”) without scanning the whole heap. Pauses are bounded by MaxGCPauseMillis; G1 picks regions to maximize freed space within the budget.
Young collection: evacuate Eden + Survivor → new Survivor / Old
Mixed collection: young + selected old regions
Concurrent mark: tracks live objects in Old without stopping the app
Full GC: fallback STW; means you've misconfigured
A Full GC in G1 is a sign of trouble — usually too small a heap, humongous-allocation churn, or metaspace pressure.
ZGC / Shenandoah
Concurrent, region-based, colored pointers (ZGC uses high bits of references as state). Pause times stay sub-ms regardless of heap size (TB-scale heaps tested). Trade-off: higher CPU and slightly lower throughput. Generational ZGC (Java 21) closes the throughput gap.
Tuning lever, not algorithm
In an interview: say which collector and why, not “I tuned the GC.” If you tuned, name the flag and the metric you watched.
4. Object Model — Headers, Autoboxing, Integer Cache
Every Java object has a header (12 or 16 bytes depending on compressed oops + alignment) before its fields. An int field costs 4 bytes; an Integer reference costs 4 bytes (compressed oops) + the boxed object’s overhead (≈16 bytes).
// Roughly:
// int[1_000_000] ≈ 4 MB
// Integer[1_000_000] ≈ 20 MB (4 MB array + ~16 MB of boxed Integers)
Autoboxing
Java silently converts int ↔ Integer. Each unbox can throw NullPointerException if the wrapper is null.
Integer x = null;
int y = x; // NPE — boxed → primitive deref
Integer cache
Integer.valueOf(i) caches -128..127 (and the cache upper bound is tunable via -XX:AutoBoxCacheMax). This produces the most-asked Java gotcha in history:
Integer a = 100, b = 100;
System.out.println(a == b); // true — cached, same object
Integer c = 200, d = 200;
System.out.println(c == d); // false — new objects, == compares references
System.out.println(c.equals(d)); // true
Always use .equals() for boxed numbers. == on object references checks identity.
Interview framing
“Why does
Integer == Integersometimes work and sometimes not?”
Integer.valueOf caches small values; large values create new objects; == is reference identity. Use .equals (or .intValue() ==).
5. Primitives vs Wrappers
| Primitive | Bits | Wrapper | Default |
|---|---|---|---|
boolean | 1 (impl-defined) | Boolean | false |
byte | 8 | Byte | 0 |
short | 16 | Short | 0 |
char | 16 | Character | '\u0000' |
int | 32 | Integer | 0 |
long | 64 | Long | 0L |
float | 32 | Float | 0.0f |
double | 64 | Double | 0.0d |
Generics cannot use primitives → List<int> is illegal. Use List<Integer> (slow, boxed) or specialized libs (IntStream, Eclipse Collections, fastutil) for hot paths.
// Hot loop on primitives — JIT loves this.
long sum = 0;
for (int i : intArray) sum += i;
// Same loop on Integer — boxing in/out, GC pressure.
long sum = 0;
for (Integer i : integerList) sum += i;
Project Valhalla (preview) introduces value classes that erase the wrapper overhead. Not yet shippable; mention only if the interviewer raises it.
Overflow
Integer arithmetic wraps silently:
int x = Integer.MAX_VALUE + 1; // -2147483648 — no exception
Math.addExact(Integer.MAX_VALUE, 1); // throws ArithmeticException
In interviews involving sums, products, or mid = (lo + hi) / 2, always consider overflow and prefer mid = lo + (hi - lo) / 2.
6. Collections Framework
| Interface | Implementations | Note |
|---|---|---|
List | ArrayList, LinkedList | Use ArrayList by default |
Set | HashSet, LinkedHashSet, TreeSet | LinkedHash preserves insertion order |
Map | HashMap, LinkedHashMap, TreeMap, ConcurrentHashMap | TreeMap is a red-black tree |
Queue / Deque | ArrayDeque, LinkedList, PriorityQueue | ArrayDeque > LinkedList for stacks/queues |
ArrayList
Backed by an Object[]. Growth is 1.5× ((oldCap >> 1) + oldCap). Append amortized O(1). add(0, x) is O(N).
ArrayList<Integer> l = new ArrayList<>(1_000_000); // pre-size to avoid resizes
HashMap
Open chaining: each bucket is a linked list. Treeification (Java 8+): when a bucket has ≥ 8 entries and table size ≥ 64, the bucket converts to a red-black tree; back to a list at ≤ 6 entries.
// Worst-case lookup pre-Java-8: O(N). Post-Java-8: O(log N).
HashMap<String, Integer> m = new HashMap<>();
Default load factor 0.75, default capacity 16. put triggers resize() (allocate new table, rehash all entries) when size > capacity * loadFactor.
Hash function mixes the user hashCode() with (h ^ (h >>> 16)) to defend against weak hashes.
// hashCode contract: equal objects → equal hashCodes.
// Bad: forgetting hashCode when overriding equals
@Override public boolean equals(Object o) { ... }
// must override hashCode too
@Override public int hashCode() { return Objects.hash(...); }
LinkedHashMap
HashMap + doubly-linked list across entries. Iteration order = insertion (or access, with accessOrder=true). The 5-line LRU cache:
class LRU<K, V> extends LinkedHashMap<K, V> {
private final int cap;
LRU(int cap) { super(cap, 0.75f, true); this.cap = cap; }
@Override protected boolean removeEldestEntry(Map.Entry<K, V> e) {
return size() > cap;
}
}
TreeMap
Red-black tree → all ops O(log N), supports firstKey, floorKey, ceilingKey, subMap — irreplaceable for ordered queries.
PriorityQueue
Binary min-heap on an array. add / poll O(log N), peek O(1). Iteration order is not sorted — only the head is.
ConcurrentHashMap (Java 8+)
Lock-free reads, fine-grained synchronization on writes (CAS + synchronized per bucket). Replaces Hashtable (deprecated for performance) and Collections.synchronizedMap (one big lock).
7. Concurrency — synchronized, ReentrantLock, Atomics, CAS
synchronized
Reentrant intrinsic lock. Implemented as an object header bit + bias / lightweight / heavyweight states (HotSpot-specific).
synchronized (lock) {
// critical section
}
public synchronized void f() { ... } // same as synchronized(this)
public static synchronized void g() {} // synchronized on the Class object
ReentrantLock
java.util.concurrent.locks.Lock. Explicit lock/unlock, supports tryLock, lockInterruptibly, fairness, multiple condition variables.
Lock lock = new ReentrantLock();
lock.lock();
try { /* CS */ } finally { lock.unlock(); }
Pick ReentrantLock when you need timeouts, fairness, or multiple Conditions. Otherwise, synchronized is shorter and the JIT optimizes it well.
Atomics
AtomicInteger, AtomicLong, AtomicReference use CAS (Compare-And-Swap) on hardware. Lock-free, lower overhead than locks for single-variable updates.
AtomicInteger counter = new AtomicInteger();
counter.incrementAndGet(); // lock-free
counter.compareAndSet(0, 1); // CAS primitive
For high-contention counters, prefer LongAdder — it stripes the counter across cells to reduce CAS contention.
volatile
Marks a field for the JMM. Reads see the latest write from any thread. No atomicity — volatile int x; x++; is still a race.
volatile boolean shutdown = false; // OK as a flag
8. Java Memory Model — Happens-Before
The JMM defines when a write by one thread is visible to another. Without happens-before, the JIT and CPU may reorder, cache, or simply skip your reads.
Happens-before edges:
- Program order within a thread.
- Monitor lock release ↦ subsequent acquire of the same monitor.
volatilewrite ↦ subsequent volatile read of the same variable.- Thread.start() ↦ first action of the started thread.
- Thread’s last action ↦ Thread.join() return.
- Constructor’s
final-field write ↦ any reader of a properly published reference. - Transitivity: A→B and B→C ⇒ A→C.
// Classic publication bug — without `volatile`, another thread may see
// `instance != null` but read uninitialized fields.
class Singleton {
private static volatile Singleton instance;
public static Singleton get() {
Singleton s = instance;
if (s == null) {
synchronized (Singleton.class) {
s = instance;
if (s == null) instance = s = new Singleton();
}
}
return s;
}
}
Interview framing
“What does
volatilegive me?”
Visibility (no caching) and ordering (no reorder across the access). Not atomicity. Not mutual exclusion.
9. Executors and Thread Pools
new Thread(...) is a code smell — never spin OS threads ad-hoc.
ExecutorService pool = Executors.newFixedThreadPool(8);
Future<Integer> f = pool.submit(() -> compute());
Integer result = f.get();
pool.shutdown();
Built-in factories (and their traps)
| Factory | Backing queue | Trap |
|---|---|---|
newFixedThreadPool | unbounded LinkedBlockingQueue | Submitter overload → OOM |
newCachedThreadPool | SynchronousQueue | Unbounded thread count |
newSingleThreadExecutor | unbounded queue | Same OOM |
newScheduledThreadPool | DelayedWorkQueue | OK |
Production pattern: construct ThreadPoolExecutor directly with bounded queue + named thread factory + sensible rejection policy.
ThreadPoolExecutor pool = new ThreadPoolExecutor(
8, 16, 60, TimeUnit.SECONDS,
new ArrayBlockingQueue<>(1000),
namedThreadFactory("worker"),
new ThreadPoolExecutor.CallerRunsPolicy());
ForkJoinPool
Work-stealing pool used by parallelStream and CompletableFuture defaults. Each worker has its own deque; idle workers steal from the back of others’ deques. Optimized for divide-and-conquer.
10. CompletableFuture
A composable async-result type. Replaces Future (which has only blocking get).
CompletableFuture<String> f =
CompletableFuture.supplyAsync(() -> fetch())
.thenApply(String::trim)
.thenCompose(s -> CompletableFuture.supplyAsync(() -> enrich(s)))
.exceptionally(ex -> "fallback");
Combinators
| Method | Purpose |
|---|---|
thenApply | map (sync transform) |
thenCompose | flatMap (chain another future) |
thenCombine | zip two futures |
allOf / anyOf | combine many |
exceptionally / handle | error handling |
orTimeout (Java 9+) | bound completion |
Default executor for the *Async variants is ForkJoinPool.commonPool(). Use a dedicated executor for IO-bound work — the common pool’s parallelism is cpus - 1 and you’ll starve compute.
11. Generics and Type Erasure
Generics are a compile-time feature. The runtime sees raw types: List<String> becomes List. Type checks insert checkcast instructions.
List<String> a = new ArrayList<>();
List<Integer> b = new ArrayList<>();
a.getClass() == b.getClass(); // true — both ArrayList
Consequences
- Cannot do
new T()(no class token) — passClass<T>orSupplier<T>. - Cannot do
new T[n]— use(T[]) Array.newInstance(cls, n). - Cannot overload by erased signature:
void f(List<String>)andvoid f(List<Integer>)collide. - Heap pollution: unchecked casts can hide type errors until use.
PECS
Producer Extends, Consumer Super.
void copy(List<? extends Number> src, List<? super Number> dst) { ... }
? extends T lets you read T’s. ? super T lets you write T’s. Memorize this; it’s asked.
12. String Pool, intern(), Encoding
String is immutable. The compiler interns string literals into a pool (used to live in PermGen, now in heap since Java 7).
String a = "hello";
String b = "hello";
a == b; // true — both reference the pooled string
String c = new String("hello");
a == c; // false
a == c.intern(); // true
Use String.intern() rarely — it’s a global side effect with non-trivial cost.
Compact Strings (Java 9+)
A String is a byte[] plus a coder byte: LATIN1 (1 byte/char) or UTF16 (2 bytes/char). A pure-ASCII string halves its memory vs Java 8.
Concatenation
a + b + c compiles, since Java 9, to an invokedynamic calling a small generated method via StringConcatFactory. Fast.
A for loop with s = s + c is O(N²) in elapsed time despite the optimization, because each iteration allocates a new String. Use StringBuilder:
StringBuilder sb = new StringBuilder(n);
for (char c : data) sb.append(c);
return sb.toString();
StringBuffer is StringBuilder + synchronization. You almost never want StringBuffer.
13. equals / hashCode Contract
1. Reflexive: x.equals(x) == true
2. Symmetric: x.equals(y) ⇔ y.equals(x)
3. Transitive: x.equals(y) && y.equals(z) ⇒ x.equals(z)
4. Consistent: repeated calls with no mutation return the same result
5. x.equals(null) == false
6. equals ⇒ hashCode equal (NOT the converse)
Break #6 and HashMap silently loses your entries.
record Point(int x, int y) {} // record auto-generates correct equals/hashCode
For non-record classes: Objects.equals and Objects.hash are your friends. IDE-generated implementations are fine; hand-rolled ones are usually wrong on edge cases (null fields, inheritance, NaN doubles).
Inheritance trap
Symmetric equals between a class and a subclass is essentially impossible without breaking Liskov. Mark the class final, or use composition + a getClass() check (not instanceof).
14. Exception Design
Three families:
- Checked (
Exceptionsubclasses, exceptRuntimeException) — must be declared / caught. - Unchecked (
RuntimeExceptionsubclasses) — programmer errors, callers may ignore. Error— JVM problems (OutOfMemoryError,StackOverflowError). Don’t catch.
Modern Java APIs lean unchecked because checked exceptions don’t compose with lambdas / streams.
list.stream().map(this::parse) // parse throws IOException → won't compile
Workarounds: wrap in RuntimeException, or use a checked-exception-friendly stream library.
try-with-resources
try (var in = Files.newInputStream(path);
var gz = new GZIPInputStream(in)) {
...
} // both closed in reverse order, even on exception
Resources must implement AutoCloseable. Suppressed exceptions (close throws after the body throws) are kept on the original via addSuppressed.
15. Streams
Lazy, pull-based pipelines.
int total = orders.stream()
.filter(o -> o.year() == 2025)
.mapToInt(Order::total)
.sum();
Lifecycle
- Source —
collection.stream(),Stream.of(...),Stream.generate(...),Files.lines(...). - Intermediate ops (lazy) —
filter,map,flatMap,sorted,distinct,limit,skip. - Terminal op (eager) —
forEach,collect,reduce,count,findFirst,toList()(Java 16+).
Pitfalls
- A stream is single-use. Re-collecting fails with
IllegalStateException. - No checked exceptions. Lambdas can’t throw them.
- Stateful intermediate ops (
sorted,distinct) buffer the whole stream. Don’t call them on infinite streams. parallel()usesForkJoinPool.commonPool— only worth it for CPU-heavy ops on large data with no shared state.
list.stream().parallel().mapToInt(...)... // measure first
16. Common Interview Gotchas
== vs .equals()
Always discussed alongside the Integer cache. Use .equals for objects, == only for primitives or true identity checks.
Integer overflow
Integer.MAX_VALUE + 1 // -2147483648
(long)Integer.MAX_VALUE + 1 // 2147483648 — promote first
Math.addExact(a, b) // throws on overflow
Floating-point equality
0.1 + 0.2 == 0.3 // false
Math.abs(a - b) < 1e-9 // OK
Double.compare(a, b) == 0 // handles NaN consistently
String.split regex
"a.b".split(".") returns [] because . is regex “any char.” Use "\\." or Pattern.quote(".").
Modifying a collection during iteration
Throws ConcurrentModificationException — even single-threaded. Use Iterator.remove() or removeIf.
Arrays.asList(int[])
Returns a List<int[]> of length 1. Use Arrays.stream(arr).boxed().toList() or IntStream.
Switch fall-through
Classic switch falls through. New switch (->) does not, and is exhaustive on sealed types and enums.
String s = switch (day) {
case MON, TUE -> "weekday";
case SAT, SUN -> "weekend";
default -> "?";
};
17. Records, Sealed Classes, Pattern Matching (Java 16–21)
Records
record Point(int x, int y) {
static Point origin() { return new Point(0, 0); }
}
Records are transparent immutable carriers: auto-generated constructor, accessors, equals/hashCode/toString. Implicitly final. Cannot extend, can implement interfaces.
Sealed classes
sealed interface Shape permits Circle, Rect {}
record Circle(double r) implements Shape {}
record Rect(double w, double h) implements Shape {}
Sealed types restrict the set of permitted subclasses. Combined with pattern matching, this gives exhaustive switching:
double area = switch (shape) {
case Circle c -> Math.PI * c.r() * c.r();
case Rect r -> r.w() * r.h();
}; // no default needed — compiler knows the universe
Pattern matching for instanceof
if (obj instanceof String s && s.length() > 3) {
use(s);
}
Modern Java is much terser than Java 8. If your interviewer is on JDK 21, leverage records + sealed + pattern switch — it shows fluency.
18. Project Loom — Virtual Threads (Java 21+)
A virtual thread is a Java-managed lightweight thread that runs on top of a small pool of OS carrier threads. Park-on-blocking-IO is implemented in the JDK.
try (var exec = Executors.newVirtualThreadPerTaskExecutor()) {
for (int i = 0; i < 100_000; i++) {
exec.submit(() -> {
try (var sock = new Socket("h", 80)) { ... }
});
}
}
Use cases: thread-per-request servers without thread-pool sizing pain. Not faster for CPU-bound work — same number of CPUs. The win is scalability of blocking IO.
Sharp edges
- Pinning: a virtual thread inside
synchronizedcannot be unmounted from its carrier. UseReentrantLockif you’ll block while holding the lock. ThreadLocalstill works but with millions of virtual threads it’s expensive. PreferScopedValue(preview).- Native code (JNI) also pins the carrier.
Interview framing
“When would you use virtual threads instead of an executor?”
Massively concurrent IO — thousands+ of in-flight blocking calls — where you’d otherwise need async/reactive code. CPU-bound work still wants a fixed pool sized to cores.
19. Performance Hot Tips
- Pre-size collections (
new ArrayList<>(n),new HashMap<>(n*4/3+1)) to avoid resize churn. - Primitive arrays beat boxed lists by 4–10× for tight loops.
StringBuilderover+=in loops. (See §12.)- Reuse objects in hot paths if they’re large and immutable-ish; pool buffers (
ByteBuffer). - Avoid
Streamin micro-loops — the lambda allocations dominate. Streams shine on big pipelines, not 5-element ones. - Escape analysis lets HotSpot stack-allocate or scalar-replace short-lived objects. You don’t tune this; you write code that doesn’t escape (no leaking
this, no storing in fields). finaldoesn’t make code faster (the JIT proves it itself), but it documents intent and is required for some JMM guarantees.-XX:+UseLargePageson Linux for big heaps.- Profile with async-profiler (sampling, low-overhead) or JFR (built-in, low-overhead). Avoid
printf-debugging perf.
# Async profiler — wall-clock CPU profile.
asprof -d 30 -f flame.html <pid>
20. JMH — Java Microbenchmark Harness
You cannot benchmark Java with System.nanoTime() around a loop. The JIT will hoist invariants, dead-code-eliminate unused results, and warm up partway through your “measurement.”
JMH handles all of that:
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@State(Scope.Thread)
public class HashBench {
int[] data = ...;
@Benchmark public int sumLoop() {
int s = 0;
for (int x : data) s += x;
return s;
}
}
Key ideas:
- Warmup iterations before measurement.
@Stateholds inputs (avoids constant-folding).- Return values prevent dead-code elimination (or use
Blackhole.consume). - Forks isolate JIT state across benchmarks.
When an interviewer asks “is X faster than Y?” the senior answer is: “I’d write a JMH benchmark — but my prediction is Y because [allocation / branch / cache reason].” Predict, then measure.
What To Memorize Cold
- JVM = bytecode interpreter + tiered C1/C2 JIT. G1 is default GC. Heap = young (Eden + S0 + S1) + old.
- Integer cache
-128..127. Always.equals()for boxed numbers. intoverflow wraps; useMath.*Exact, preferlo + (hi-lo)/2.HashMaptreeifies overflowing buckets to red-black trees → worst-case O(log N) since 8.volatile= visibility + ordering, not atomicity.- JMM happens-before: lock, volatile, start/join, final-after-publication.
- Generics erase. No
new T[], no overload by erased signature. Stringis immutable. Compact strings since 9. UseStringBuilderin loops.equalsandhashCodego together;recorddoes it for you.- Records are final immutable carriers. Sealed + pattern match = exhaustive switch.
- Virtual threads scale blocking IO; they don’t help CPU.
- JMH for benchmarks.
nanoTimelies.
If any of those is shaky, re-read the section, write the smallest program that demonstrates it, and watch it misbehave on purpose.
Go Runtime Deep Dive
Target audience: candidates interviewing in Go for infrastructure, distributed systems, cloud-native (Kubernetes / Docker / etcd ecosystems), or any backend role where the interviewer asks “explain how goroutines actually work.”
Scope: gc (the standard Go compiler) on Go 1.22+. gccgo and TinyGo only mentioned where they change interview-grade answers.
Go’s surface looks small. The runtime is not. The interview gap appears immediately when an interviewer asks “what’s the difference between a goroutine and an OS thread?”, “what does nil != nil even mean?”, or “why does this loop variable do that?”. This guide trains the answers.
1. Runtime Overview — M:N Scheduling
Go runs your code under a runtime linked into every binary. The runtime owns:
- The goroutine scheduler (M:N — many goroutines onto few OS threads).
- The garbage collector (concurrent tri-color mark-sweep).
- The memory allocator (TCMalloc-derived, per-P caches).
- Channel and sync primitives, network poller, timers, profilers.
A “goroutine” is not an OS thread. It’s a small (~2 KB initial stack) cooperatively-scheduled task multiplexed onto a pool of OS threads. The runtime can have thousands of goroutines on a handful of threads.
// A million goroutines is normal.
for i := 0; i < 1_000_000; i++ {
go func() { /* ... */ }()
}
This is feasible because each goroutine starts with ~2 KB of stack (vs ~1 MB for an OS thread default) and the stack grows as needed.
Stack growth
Goroutine stacks are segmented / split historically, contiguous-grow since 1.4: when the stack overflows, the runtime allocates a bigger stack and copies all frames + adjusts pointers. This is the reason taking the address of a stack-allocated variable is safe in Go: even if the stack moves, references stay valid.
2. The GMP Scheduler
Three runtime objects:
| Stands for | What it is | |
|---|---|---|
| G | Goroutine | A goroutine: stack + program counter + status |
| M | Machine | An OS thread |
| P | Processor | A logical scheduler context; holds a runnable G queue |
Number of P’s = GOMAXPROCS (default: number of CPUs). Each P has a local runnable queue. M’s bind to a P to execute G’s; an M without a P cannot run Go code.
P0 [G G G G ...] P1 [G G ...] P2 [G G G G G ...]
│ │ │
M0 M1 M2 (OS threads)
Steal work
When a P’s queue is empty, it steals half from a random other P’s queue. Keeps cores busy without a global lock.
What happens on a blocking syscall
The M making the syscall detaches from its P and blocks. The P picks up another M (creating one if needed) and keeps scheduling. When the syscall returns, the original M tries to reacquire a P; if none is free it parks the G on the global queue.
This is why read(fd, ...) on a regular file blocks an OS thread but does not block your other goroutines — they keep running on other M’s.
Network poller
Network I/O is epoll/kqueue/IOCP under the hood. A goroutine doing conn.Read parks itself, registers with the poller, and another goroutine runs. When the fd is readable, the poller wakes the parked G. No M is consumed while parked. This is why Go scales to 100K+ concurrent network connections trivially.
Preemption
Up to Go 1.13, goroutines yielded only at function preludes (so a tight CPU loop without function calls could starve others). Since 1.14, asynchronous preemption uses signals to interrupt a goroutine mid-instruction.
// Pre-1.14, this could starve everything else; today it's preempted.
go func() { for {} }()
Interview framing
“What’s the difference between a goroutine and a thread?”
Goroutine: ~2KB stack, cooperative + signal-preempted, scheduled by Go runtime onto a pool of OS threads. Thread: ~1MB stack, OS-scheduled, costlier context switches. Goroutines are the unit you think about; M’s are an implementation detail.
3. Goroutines vs Threads — Practical Implications
// I/O fanout pattern
results := make(chan Result, len(urls))
for _, u := range urls {
u := u // pre-1.22: required to capture
go func() {
results <- fetch(u)
}()
}
for range urls {
r := <-results
process(r)
}
Costs:
- Goroutine creation: ~1µs.
- Channel ops: ~50–100ns uncontended; mutexes similar.
- Context switch: ~200ns within Go runtime; blocking syscalls add OS thread cost.
Sharp edge: unlike OS threads, goroutines do not have IDs. By design — they discourage thread-local-state patterns. This breaks naïve port of Java idioms.
4. Channels — Buffered, Unbuffered, select
A channel is a typed bounded queue with built-in synchronization.
| Construct | Behavior |
|---|---|
make(chan T) | Unbuffered: send and recv must rendezvous. Sender blocks until a receiver is ready. |
make(chan T, n) | Buffered: sender blocks only when buffer is full. |
close(ch) | Recvs drain remaining values, then receive zero values. Send to closed → panic. |
ch := make(chan int, 2)
ch <- 1
ch <- 2
close(ch)
for v := range ch { fmt.Println(v) } // 1, 2
select
Multiplexes channels — picks any ready case (random tie-break). Unblocks composing producers/consumers, timeouts, cancellation.
select {
case v := <-in:
use(v)
case out <- val:
// ...
case <-time.After(2 * time.Second):
return errors.New("timeout")
case <-ctx.Done():
return ctx.Err()
}
Nil-channel pattern
A nil channel blocks forever. Setting a case’s channel to nil disables it:
var done chan struct{} = nil
// case <-done: never fires
Useful when iterating over multiple channels and “turning off” one as it completes.
Closing semantics
- Receivers detect close with
v, ok := <-ch; okis false on closed-and-drained. - Only the sender should close. Closing on the receiver side requires extra coordination because closing a channel that someone else may send to → panic.
- Don’t close a channel just to “free” it; let the GC handle that.
5. Sync Primitives
| Use for | |
|---|---|
sync.Mutex | Mutual exclusion |
sync.RWMutex | Many readers / few writers (do measure — RW often loses to plain Mutex) |
sync.Once | Idempotent one-time init |
sync.WaitGroup | Wait for N goroutines |
sync.Cond | Condition variable; rarely needed (channels usually clearer) |
sync/atomic | CAS, atomic add/load/store on int32/int64/pointer |
sync.Map | Concurrent map only when read-mostly with disjoint key sets |
var mu sync.Mutex
mu.Lock()
defer mu.Unlock()
// CS
sync.Map is not always faster
It’s optimized for two specific patterns:
- Stable disjoint key sets per goroutine.
- Mostly reads, rare writes.
For everything else, a regular map[K]V + sync.Mutex (or shards) is faster and clearer.
WaitGroup
var wg sync.WaitGroup
for _, x := range data {
wg.Add(1)
go func(x Item) {
defer wg.Done()
process(x)
}(x)
}
wg.Wait()
Trap: wg.Add must happen before the goroutine starts running, never inside it.
6. Memory Model
Go has a documented memory model (re-articulated in 2022 for clarity). Key rules:
- A read sees writes that happen before it.
- Within a goroutine: program order.
- Goroutine creation happens before its first instruction.
- A send on a channel happens before the corresponding receive completes.
- Close of a channel happens before a receive that returns the zero value due to close.
m.Unlockhappens before subsequentm.Lock.sync/atomic: each atomic op is sequentially consistent; pairs ordered by HB.
// Without sync, this is racy.
var data []int
var ready bool
go func() {
data = makeData()
ready = true // RACE — no HB to the reader
}()
for !ready {} // may loop forever (compiler/CPU can hoist)
use(data)
Fix with channel, mutex, or atomic.
Race detector
Always run tests with -race:
go test -race ./...
It instruments memory accesses, catches actual data races (not just suspicious code). Cheap insurance; one of Go’s killer features.
7. Garbage Collector — Concurrent Tri-color Mark-Sweep
Go’s GC is concurrent, non-moving, tri-color mark-sweep with write barriers.
- Tri-color: white = not yet visited, grey = visited but children not, black = done.
- Write barrier: intercepts pointer writes during mark to maintain invariants while the mutator runs.
- Non-moving: objects don’t relocate. Pointers stay stable. (Trade-off: no compaction, more fragmentation.)
Pause time
Sub-millisecond STW for stack scanning + write-barrier setup. Most marking happens concurrently with your program. No “young generation” — Go’s GC is uniform.
Pacing
GC triggers when heap doubles since last collection (GOGC=100 default). Lower for less footprint at the cost of CPU; higher for less GC at the cost of memory.
GOGC=200 ./app # GC less often
GOGC=off ./app # disable (for benchmarks)
Soft / hard memory limits
runtime/debug.SetMemoryLimit(n) (Go 1.19+) sets a soft limit; the GC trades CPU for staying under it. Useful in containers — set it to 0.9 * cgroup_limit to avoid OOM-kills.
Escape analysis
The compiler decides at compile time whether a value can stay on the stack. If a pointer “escapes” the function, the value is heap-allocated.
func f() *int {
x := 1
return &x // escapes — heap allocation
}
go run -gcflags='-m' main.go
// prints: x escapes to heap
Knowing what allocates lets you avoid GC pressure in hot paths. Stack allocation is essentially free; heap allocation costs ~30ns + future GC scan.
8. Slice Internals
A slice is a 3-word struct: (ptr *T, len int, cap int). Slicing does not copy — it’s a view.
a := []int{1, 2, 3, 4, 5}
b := a[1:4] // [2 3 4], cap=4 (from index 1 to end of underlying array)
b[0] = 99
fmt.Println(a) // [1 99 3 4 5] — shared backing array!
append semantics
b = append(b, 10) // if len < cap: in place; else allocate new backing array
Growth: double up to ~256 elements, then ~1.25× (Go 1.18 changed the heuristics slightly). The new slice’s backing array is independent of any older slice that still points at the old one.
a := make([]int, 4, 4)
b := a[:2]
c := append(b, 99) // overwrites a[2]
fmt.Println(a, c) // [1 1 99 1] [1 1 99]
d := append(c, 1, 2, 3) // reallocates; d disjoint from a
This is the slice aliasing gotcha that loses interviews. The fix is to be explicit:
b := append([]int(nil), source...) // explicit copy
Three-index slice
a[lo:hi:max] caps the new slice’s cap at max - lo. Use it when handing out a slice you don’t want the receiver to extend into your data.
9. Map Internals
map[K]V is a hash table with bucket chaining (each bucket holds 8 entries, then chains overflow buckets). Hash is randomized per map (security + iteration order).
m := make(map[string]int, 1000) // pre-size to avoid grows
m["a"] = 1
delete(m, "a")
v, ok := m["a"]
Iteration order is randomized
Every for k := range m produces a different order, even within one run. Don’t depend on it.
for k, v := range m {
// unspecified order
}
Concurrent access
Plain map is not safe for concurrent read/write. Go’s race detector and runtime both panic on detection. Use sync.RWMutex or sync.Map (with caveats from §5).
fatal error: concurrent map writes
nil map
A nil map can be read (returns zero) but not written. A common bug:
var m map[string]int
m["a"] = 1 // PANIC
Use m := map[string]int{} or make(map[string]int).
Complexity
| Op | Avg | Worst |
|---|---|---|
m[k] | O(1) | O(N) under collisions |
m[k] = v | O(1) amortized | O(N) on grow |
delete(m, k) | O(1) | O(N) |
range m | O(N) | O(N) |
Maps shrink lazily — deleting most keys does not return memory. Re-create the map if you care.
10. Strings — Bytes vs Runes
A string is an immutable byte slice. No internal length-of-runes — indexing returns bytes.
s := "héllo"
len(s) // 6 — UTF-8 bytes (é is 2)
s[0] // 'h' (a byte)
s[1] // first byte of é, NOT é
Iterate with range to get runes (decoded code points):
for i, r := range s {
// i: byte index, r: rune (int32 code point)
}
To get rune count: utf8.RuneCountInString(s).
[]byte ↔ string conversion
Both directions copy by default (so the immutability invariant holds).
b := []byte(s) // copy
s2 := string(b) // copy
Hot paths can use unsafe.String / unsafe.Slice (Go 1.20+) for zero-copy, but it’s a footgun — only if you can prove the underlying bytes won’t be mutated.
String concat
a + b + c allocates each step → O(N²) in a loop. Use strings.Builder:
var sb strings.Builder
for _, p := range parts {
sb.WriteString(p)
}
return sb.String()
strings.Builder reuses its buffer and avoids the final copy via unsafe.
11. Interfaces — itab, the nil != nil Trap
An interface value is two words: (itab *itab, data *void). The itab holds the dynamic type + method table; data is the concrete value (or pointer to it).
type io.Reader interface { Read(p []byte) (int, error) }
var r io.Reader // itab = nil, data = nil → r == nil
r = (*os.File)(nil) // itab ≠ nil, data = nil → r != nil !!
This is the Go gotcha. Rule: an interface is nil only when both its halves are nil. A typed nil pointer assigned to an interface is not nil.
The footgun in real code:
func mightFail() error {
var e *MyError = nil
if condition() { e = &MyError{...} }
return e // returning a typed-nil pointer -> caller sees != nil
}
Fix:
func mightFail() error {
if condition() { return &MyError{...} }
return nil // explicit nil interface
}
Type assertions and type switches
v, ok := x.(string) // safe assertion
switch v := x.(type) {
case int: use(v)
case string: use(v)
default: ...
}
Type assertions are O(1) for non-empty interfaces (one slot in the itab). For empty interfaces (any), the runtime walks the method table — still fast but not free.
12. Error Handling
Errors are values. Everything else is style.
v, err := operation()
if err != nil {
return fmt.Errorf("operation failed: %w", err)
}
Wrapping
%w (Go 1.13+) wraps an error, building a chain.
errors.Is(err, io.EOF) // walks the chain
var pathErr *os.PathError
errors.As(err, &pathErr) // unwraps to a specific type
Sentinel errors
var ErrNotFound = errors.New("not found")
return ErrNotFound
Compare with errors.Is, not == — wrapping breaks ==.
panic / recover
panic unwinds stack frames running deferred functions. recover (in a deferred func) catches it. Use only for truly unexpected conditions (programmer bugs, “should never happen”). Not for control flow.
defer func() {
if r := recover(); r != nil {
log.Printf("recovered: %v", r)
}
}()
13. defer
defer schedules a call to run when the surrounding function returns.
f, err := os.Open(path)
if err != nil { return err }
defer f.Close()
Cost and gotchas
-
Pre-1.14, defer was ~50ns. Since 1.14, “open-coded defers” are inlined for many cases — essentially free.
-
Args are evaluated at the
defercall site, not at execution:i := 1 defer fmt.Println(i) // prints 1 i = 2 -
LIFO ordering — deferred calls run in reverse.
-
deferin a loop accumulates. Don’tdefer f.Close()insideforover thousands of files; close manually or wrap the body in a function.
14. Context
context.Context propagates deadlines, cancellation, and request-scoped values across API boundaries.
ctx, cancel := context.WithTimeout(parent, 5*time.Second)
defer cancel()
resp, err := http.NewRequestWithContext(ctx, "GET", url, nil)
Rules
- Pass
ctxas the first parameter, never store it in a struct field for long-lived state. - Always call
cancel— even on success — to release resources.defer cancel()is the pattern. - Don’t pass
nilctx; usecontext.TODO()if you don’t have one yet. ctx.Valueis for request-scoped data (auth principal, request ID), not for optional config.- A child context is cancelled when its parent is cancelled.
Detecting cancellation
select {
case <-ctx.Done():
return ctx.Err()
case v := <-work:
return process(v)
}
15. Goroutine Leaks
A goroutine leak happens when a goroutine blocks forever on a channel that never receives, a mutex never released, etc. The runtime never reclaims it. In long-running services, leaks compound.
Common shape
func bad() <-chan int {
out := make(chan int) // unbuffered
go func() {
out <- expensive() // blocks forever if caller drops the chan
}()
return out
}
Fixes:
- Buffer the channel for one value (drop on send if no receiver).
- Use
selectwithctx.Done().
go func() {
select {
case out <- expensive():
case <-ctx.Done():
}
}()
Detecting leaks
go testwithgoleak(Uber library) at the end of tests.runtime.NumGoroutine()in production — a steadily growing number is a leak.pprofgoroutine profile:curl http://localhost:6060/debug/pprof/goroutine?debug=2.
16. Testing and Benchmarking
Tests
func TestAdd(t *testing.T) {
if got := Add(1, 2); got != 3 {
t.Errorf("Add(1,2) = %d, want 3", got)
}
}
Table-driven tests
Idiomatic Go — readable, easy to extend.
tests := []struct {
name string
in, want int
}{
{"zero", 0, 0},
{"pos", 1, 2},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := Double(tt.in); got != tt.want {
t.Errorf("got %d want %d", got, tt.want)
}
})
}
Benchmarks
func BenchmarkX(b *testing.B) {
for i := 0; i < b.N; i++ {
X()
}
}
go test -bench=. runs them. b.ReportAllocs() includes alloc counts. Always look at allocs/op — the JIT here doesn’t exist; allocations directly drive GC pressure.
go test -bench=. -benchmem
Fuzzing (Go 1.18+)
func FuzzParse(f *testing.F) {
f.Add("hello")
f.Fuzz(func(t *testing.T, s string) {
Parse(s)
})
}
Use for parsers, decoders, anything taking adversarial input.
17. Common Interview Gotchas
Loop variable capture (pre-1.22)
for _, v := range items {
go func() { process(v) }() // pre-1.22: all goroutines see last v
}
Fix pre-1.22: shadow v := v inside the loop. Go 1.22 fixed this — each iteration has its own copy. State which Go version you’re on.
Slice aliasing
See §8.
Map iteration order
Randomized. Don’t rely on it. Don’t rely on it. Don’t rely on it.
Nil interface vs typed-nil pointer
See §11.
== on slices / maps / functions
Compile error. Slices/maps/funcs aren’t comparable. Use reflect.DeepEqual or write per-field comparison.
defer in a loop
for _, p := range paths {
f, _ := os.Open(p)
defer f.Close() // Hundreds of open files — close at func return
}
Wrap the body in a function or close explicitly.
Range over a channel
for v := range ch continues until ch is closed. If never closed, leaks.
Goroutine started with shared mutable state
data := []int{1, 2, 3}
go modify(&data) // race unless guarded
Always guard with mutex or send via channel.
18. Performance Hot Tips
-
Pre-size slices and maps:
make([]T, 0, n),make(map[K]V, n). Avoid resize churn. -
Avoid heap allocations in hot loops. Use
-gcflags='-m'to find escape culprits. Reuse buffers viasync.Pool.var bufPool = sync.Pool{New: func() any { return new(bytes.Buffer) }} buf := bufPool.Get().(*bytes.Buffer) defer func() { buf.Reset(); bufPool.Put(buf) }() -
strings.Builderfor concatenation,bytes.Bufferfor byte building. -
Prefer fixed-size arrays / structs over slices in tight code when size is known.
-
Goroutines aren’t free. Spawning one per CPU-microtask in a 1ns loop is slower than the loop. They shine for IO and large work units.
-
Avoid
interface{}in hot paths. Boxing primitives heap-allocates and adds an itab indirection per call. -
Profile.
go test -cpuprofile,pprof, the runtime tracer (go tool trace).
go test -bench=. -cpuprofile=cpu.out
go tool pprof -http=:8080 cpu.out
runtime.GC()anddebug.SetGCPercentare levers, not solutions. Reduce allocation first.sync.Poolis not a general-purpose cache; the runtime drops its contents on every GC. Use it for short-lived reusable buffers.
What To Memorize Cold
- GMP scheduler. Goroutines (~2KB) ≠ OS threads. M:N. P count =
GOMAXPROCS. - Goroutine stacks grow by copy. Network I/O via runtime poller, no M consumed.
- Channels: unbuffered = rendezvous. Send to closed → panic. Nil chan blocks forever.
- Memory model: race detector with
-race. Channel send happens-before recv. Mutex unlock HB next lock. - GC: concurrent tri-color mark-sweep, non-moving, sub-ms pauses.
GOGCandSetMemoryLimit. - Slices =
(ptr, len, cap).appendmay alias or reallocate. Aliasing bugs are common. - Maps: randomized iteration, not concurrent-safe, not comparable, panic on nil-map write.
- Strings: immutable bytes. Range over string yields runes.
- Interface = (itab, data). Typed-nil pointer in interface ≠ nil interface.
- Loop var capture fixed in Go 1.22.
defercheap since 1.14, args eval at scheduling time.contextfirst arg, alwaysdefer cancel().- Goroutine leaks via blocked channels —
selectonctx.Done(). - Pre-size slices/maps.
sync.Poolfor buffer reuse.pproffor everything else.
When any of those is hazy, write a 10-line program that tickles it. The race detector and -gcflags='-m' are unusually fast feedback loops compared to other languages.
C++ Runtime Deep Dive
Target audience: candidates interviewing in C++ for HFT/quant, game engines, embedded, browsers, databases, systems-programming, or any role where the interviewer asks “what does this allocate?”, “is that UB?”, or “trace the move.”
Scope: ISO C++17 baseline with C++20/23 features called out. GCC, Clang, and MSVC all behave alike on the spec — vendor-specific behavior is noted only when it changes interview answers.
C++ punishes superficial knowledge harder than any other language on this list. The senior interviewer will set a trap (a dangling reference, a missed move, an iterator invalidation, a UB), and either you see it or you don’t. There is no bluffing through C++. Everything in this guide pays interest.
1. Memory Model — Stack, Heap, RAII
C++ gives you control over object lifetime. Every object you create lives somewhere:
| Storage | Lifetime | Cost | Example |
|---|---|---|---|
| Automatic (stack) | Scope of declaration | Zero | int x; Foo f; |
| Static / thread_local | Program / thread | Zero (init once) | static Foo f; |
| Dynamic (heap) | Until delete/destructor | malloc + bookkeeping | new Foo / make_unique<Foo> |
void f() {
int x = 5; // stack
static int y = 0; // static, init once
auto p = std::make_unique<int>(42); // heap, freed at scope end
}
RAII
Resource Acquisition Is Initialization. Tie the lifetime of a resource (memory, file, lock, socket) to the lifetime of an object on the stack.
{
std::lock_guard<std::mutex> lk(mtx); // acquires
// ... critical section ...
} // destructor releases
RAII is the single most important C++ idea. It makes exceptions safe, makes resource leaks impossible if you stick to it, and is the foundation of all modern C++. Every interview answer that involves “what if it throws?” reduces to “RAII handles it.”
Stack frames
A function call pushes a frame: arguments, return address, locals, callee-saved registers. Frame size is fixed at compile time. Stack overflow on deep recursion or huge stack arrays.
alloca / VLAs
alloca(n) allocates on the stack. C99 VLAs (int arr[n]) are not in C++. Modern code uses std::vector or std::array (compile-time n).
2. Pointers, References, Values
| Form | Nullable | Rebindable | Storage |
|---|---|---|---|
T | n/a | n/a | by value |
T& | No | No | reference; aliases another object |
T* | Yes | Yes | pointer; an address |
const T& | No | No | read-only view |
T&& | No | No | rvalue reference (see §4) |
int x = 1;
int& r = x; // r is x — no separate object
int* p = &x;
*p = 2; // x is now 2
r = 3; // x is now 3
p = nullptr; // p reseats; r cannot be reseated
When to use which
- Pass by value: small types (
int,Point), or you want a copy / will move from the parameter. - Pass by
const T&: large/expensive types you only read. - Pass by
T&: out-parameters (rare in modern C++; prefer return values). - Pass by pointer: nullable, or you need a C-API.
Dangling refs
const std::string& bad() {
std::string tmp = "hi";
return tmp; // returns reference to dead local — UB
}
The compiler may warn; the runtime will silently corrupt. Sanitizers (ASan) catch many cases.
3. Smart Pointers — unique_ptr, shared_ptr, weak_ptr
The modern rule: never new/delete directly. Use:
std::unique_ptr<T>— exclusive ownership, zero overhead vs raw pointer.std::shared_ptr<T>— shared ownership, atomic refcount.std::weak_ptr<T>— non-owning observer; breaksshared_ptrcycles.
auto u = std::make_unique<Foo>(args...); // unique
auto s = std::make_shared<Foo>(args...); // shared
std::weak_ptr<Foo> w = s; // non-owning
Cost model
unique_ptr<T> is a single pointer. Move-only. Compiler optimizes away the wrapper.
shared_ptr<T> is two pointers (the object, the control block) + an atomic refcount. Copying = atomic increment. Destruction = atomic decrement.
make_shared vs shared_ptr<T>(new T)
make_shared<T> allocates the object and the control block in one block. Cheaper, better cache locality. Drawback: memory isn’t freed until the last weak_ptr dies (because the control block lives in the same allocation).
Cycles
struct Node { std::shared_ptr<Node> next; };
auto a = std::make_shared<Node>();
auto b = std::make_shared<Node>();
a->next = b; b->next = a;
// a and b never freed — refcount of each stays at 2
Fix: one direction weak_ptr. Or, redesign — most “cycles” represent ownership confusion.
Custom deleter
auto p = std::unique_ptr<FILE, decltype(&fclose)>(fopen("x", "r"), &fclose);
Useful for C-API resources.
4. Move Semantics, Rvalue References
A moved-from object is in a “valid but unspecified” state. The point of move is to transfer expensive resources (heap allocations, file handles) without copying.
std::string a = "hello";
std::string b = std::move(a); // b owns the buffer; a is empty (typically)
std::move is a cast — it doesn’t move anything; it tells the compiler “treat this as an rvalue, please pick the move overload.”
Rule of 0/3/5
- Rule of 0: design classes so the defaults are correct. Member variables are RAII types (
vector,unique_ptr,string). Don’t write any of the special members. - Rule of 3 (pre-C++11): if you write any of
dtor,copy ctor,copy assign, write all three. - Rule of 5 (C++11+): add move ctor and move assign.
class Buffer {
std::unique_ptr<char[]> data_;
std::size_t size_;
public:
Buffer(std::size_t n)
: data_(std::make_unique<char[]>(n)), size_(n) {}
// copy/move auto-generated correctly because members are RAII.
};
noexcept matters
Move operations should be noexcept. If they aren’t, std::vector can’t use them when reallocating — it falls back to copy, defeating the purpose.
struct S {
std::string name;
S(S&&) noexcept = default; // critical
S& operator=(S&&) noexcept = default;
};
Forwarding references (T&& in templates)
template<class T>
void f(T&& x) { // forwarding ref, NOT rvalue ref
g(std::forward<T>(x)); // preserves value category
}
Reference collapsing: T&& && → T&&, T&& & → T&. This is the mechanism behind perfect forwarding (and std::forward).
5. Copy Elision and RVO
The compiler is allowed (and often required) to elide copy/move when constructing return values.
Foo make() { return Foo{}; } // (N)RVO — direct construction in caller
Foo f = make(); // no copy, no move
C++17 mandated copy elision for prvalues — the move you “see” in source code may not exist as an actual operation.
auto v = std::vector<int>(1'000'000); // no copy of the temporary
Implication: return by value is the right default. The compiler will not copy a big vector.
6. Templates, SFINAE, Concepts
Templates are compile-time generators. Each instantiation produces a fresh type or function.
template<class T>
T max(T a, T b) { return a < b ? b : a; }
max(1, 2); // T = int
max(1.0, 2.0); // T = double
max(1, 2.0); // ambiguous — different T's
SFINAE — “Substitution Failure Is Not An Error”
Failed substitutions are silently dropped from the overload set, not compile errors:
template<class T>
auto add(T a, T b) -> decltype(a + b) { return a + b; }
Older idiom: std::enable_if_t<...>. Crufty; use concepts instead in C++20:
template<class T>
concept Numeric = std::is_arithmetic_v<T>;
template<Numeric T>
T add(T a, T b) { return a + b; }
Compile-time error blasts
A template error message can be thousands of lines. Modern compilers (gcc 13+, clang 16+) and concepts dramatically reduce this. If you see a 5000-line error in an interview, don’t panic; isolate by typedef-ing intermediate types.
CRTP (Curiously Recurring Template Pattern)
Static polymorphism — virtual without the vtable cost.
template<class Derived>
struct Base { void f() { static_cast<Derived*>(this)->impl(); } };
struct D : Base<D> { void impl() { /* ... */ } };
7. STL Containers — Complexity
| Container | Insert | Erase | Lookup | Iter Invalidation | Memory |
|---|---|---|---|---|---|
vector | O(1)* end / O(N) middle | O(N) | O(N), O(1) by index | All on grow / from pos | Contiguous |
array | n/a | n/a | O(1) | None | Contiguous, fixed N |
deque | O(1) ends, O(N) middle | O(N) | O(1) | All except ends | Block array |
list | O(1) anywhere (with iter) | O(1) | O(N) | None on insert; affected pos on erase | Doubly linked |
forward_list | O(1) after iter | O(1) | O(N) | None on insert | Singly linked |
set/map | O(log N) | O(log N) | O(log N) | None on insert; pos on erase | Red-black tree |
unordered_set/map | O(1) avg, O(N) worst | O(1) avg | O(1) avg | All on rehash | Buckets + nodes |
vector is the default. Reach for others only with a measured reason.
std::vector<int> v;
v.reserve(1'000'000); // pre-size, avoid grows
for (int i = 0; i < 1'000'000; ++i) v.push_back(i);
unordered_map warnings
Open-chaining hash table. Each node is heap-allocated → bad cache locality. For perf-critical code, prefer absl::flat_hash_map, tsl::robin_map, or other open-addressing maps. State this in HFT/perf interviews; it’s a known weakness.
std::unordered_map<std::string, int> m;
m.reserve(N); // sets bucket count
m.max_load_factor(0.5); // tighter than default 1.0
8. Iterator Invalidation
The single most common subtle bug in C++.
| Container | Operation | What invalidates |
|---|---|---|
vector | push_back, insert, reserve triggering grow | All iterators/refs/pointers |
vector | erase | Iterators/refs at and after pos |
deque | any insert/erase except at ends | All iterators (refs to non-affected elements survive) |
list / forward_list | insert, push_* | None |
list / forward_list | erase | Only iterators to erased element |
unordered_* | rehash (insert that exceeds load factor) | All iterators (refs/pointers survive!) |
map / set | insert | None |
map / set | erase | Only iterators to erased |
std::vector<int> v{1,2,3,4,5};
for (auto it = v.begin(); it != v.end(); ++it) {
if (*it == 3) v.push_back(99); // UB — push_back may invalidate `it`
}
// Correct: collect, then mutate; or use erase-remove.
v.erase(std::remove_if(v.begin(), v.end(), pred), v.end());
9. STL Algorithms
<algorithm> and <numeric> provide a rich library. Use them — hand-rolled loops are usually slower and harder to read.
std::sort(v.begin(), v.end()); // IntroSort, O(N log N)
std::stable_sort(v.begin(), v.end()); // O(N log² N) generally
std::nth_element(v.begin(), v.begin()+k, v.end()); // O(N) avg, kth-element
std::partial_sort(v.begin(), v.begin()+k, v.end()); // top-k, O(N log K)
std::lower_bound(v.begin(), v.end(), x); // binary search, O(log N)
std::accumulate(v.begin(), v.end(), 0LL); // careful with init type
Sort algorithms
std::sort is introsort: quicksort, switching to heapsort if recursion gets too deep, switching to insertion sort for small ranges. Worst case O(N log N), unstable. std::stable_sort is typically merge sort with allocation; std::sort is usually preferred unless stability matters.
Ranges (C++20)
auto evens = v | std::views::filter([](int x){ return x%2==0; })
| std::views::transform([](int x){ return x*x; });
Lazy, composable. Less verbose than iterator pairs.
10. Concurrency — std::thread, mutex, atomics, memory_order
std::thread t([]{ work(); });
t.join(); // or t.detach() — but rarely
If a std::thread is destroyed while joinable, the program calls terminate. std::jthread (C++20) joins on destruction.
Mutex
std::mutex m;
std::lock_guard<std::mutex> lk(m); // RAII lock
std::scoped_lock (C++17) locks multiple mutexes deadlock-free.
Condition variables
std::condition_variable cv;
std::unique_lock<std::mutex> lk(m);
cv.wait(lk, []{ return ready; }); // releases lk, waits, reacquires
Always use the predicate form to handle spurious wakeups.
std::atomic<T>
std::atomic<int> counter{0};
counter.fetch_add(1, std::memory_order_relaxed);
memory_order
| Order | Guarantees | Use |
|---|---|---|
relaxed | Atomicity only, no ordering | Stat counters |
acquire (load) | No subsequent reads/writes can move before | Read of a flag protecting data |
release (store) | No prior reads/writes can move after | Write that publishes data |
acq_rel (RMW) | Both | CAS retry loops |
seq_cst (default) | Sequential consistency, single total order | Default; safest |
// Producer:
data = produce();
ready.store(true, std::memory_order_release);
// Consumer:
while (!ready.load(std::memory_order_acquire)) {}
use(data); // safe — release/acquire pair
memory_order is interview territory at L6+ HFT/system roles. Default to seq_cst until measured.
11. Undefined Behavior (UB)
UB means the spec places no requirements. The compiler may eliminate code, “optimize” infinite loops, or generate code that does anything. Don’t rely on “well, it works on my machine.”
Common UB
- Read of uninitialized memory.
- Out-of-bounds access (
v[v.size()]is UB). - Signed integer overflow (unsigned wraps, signed is UB).
- Use-after-free / double-free.
- Race conditions (concurrent unsynchronized access to mutable data).
- Strict aliasing violations (reinterpreting a
float*asint*). - Null pointer deref — including for member access on a null pointer.
- Lifetime violations — using a moved-from object beyond what’s specified.
- Integer division by zero, INT_MIN / -1.
- Returning reference/pointer to a local.
Why it bites in interviews
The interviewer puts a for (int i = 0; i <= n; ++i) v[i] = ...; on the board and watches whether you flag the OOB. If you don’t, your perceived rigor drops a tier instantly.
Sanitizers
Compile + run tests under:
clang++ -fsanitize=address,undefined -g -O1 main.cpp
clang++ -fsanitize=thread -g -O1 main.cpp # for races
ASan: heap/stack/global OOB, use-after-free, double-free. UBSan: signed overflow, null derefs, alignment. TSan: data races.
State in interviews that you run sanitizers in CI. It signals discipline.
12. Common Interview Gotchas
Virtual destructor
If a class is meant to be derived-from and used polymorphically, the destructor must be virtual — otherwise delete base_ptr calls only the base’s destructor.
struct Base { virtual ~Base() = default; };
struct Derived : Base { /* ... */ };
Base* p = new Derived;
delete p; // virtual dtor → Derived's runs
Object slicing
void f(Base b); // by value
Derived d;
f(d); // d sliced — only Base portion copied
Always pass polymorphic types by reference or pointer, never by value.
vector<bool> is not a vector of bool
Specialized as a packed bitset → operator[] returns a proxy, not bool&. Don’t take its address.
std::vector<bool> v;
auto x = v[0]; // proxy reference, not bool&
Use std::vector<char> if you need real bools.
Self-assignment
T& operator=(const T& o) {
if (&o == this) return *this; // guard
// ...
}
Or: copy-and-swap idiom — pass by value (copy happens at call site), swap, return.
Initialization order
Member variables are constructed in declaration order, not member-initializer-list order. Compiler warns when they differ.
static local init
Thread-safe since C++11 (Magic statics). One initialization, even with concurrent first access.
nullptr vs NULL vs 0
Use nullptr. NULL is 0 (an integer); 0 doesn’t overload-resolve cleanly.
Floating-point comparison
Same warning as Java — never == for float/double. Use tolerances or std::nextafter.
Implicit conversions
int → bool, bool → int, double → int. Use explicit for single-arg constructors:
struct Date { explicit Date(int y); };
Date d = 2024; // error — explicit constructor
Date d{2024}; // OK
13. Modern C++ Idioms
autofor local types — but spell out parameter and return types where they’re API.- Range-for —
for (const auto& x : container). - Lambdas — capture defaults:
[](none),[&](by ref),[=](by value),[this]. enum class— strongly typed, scoped enums. No implicit int conversion.structured bindings—auto [k, v] = *it;.if constexpr— compile-time branch in templates.std::optional—Maybe<T>. Use for “may not exist.”std::variant— tagged union.std::string_view— non-owning view of a string. Don’t store across the string’s lifetime.std::span— non-owning view of a contiguous range.{}init — uniform initialization. Prevents narrowing conversions.
int a{3.14}; // error — narrowing
int a = 3.14; // OK (silent truncation)
Modules (C++20)
Replacement for headers. Faster builds, better isolation. Adoption uneven; compilers still maturing.
Coroutines (C++20)
generator<int> ints() {
for (int i = 0;; ++i) co_yield i;
}
The standard library lacks high-level types — you bring boost::asio or roll your own. Mention only if asked.
14. Compile-Time vs Runtime
C++ has a powerful compile-time computation toolkit. Use it to push work out of the runtime.
constexpr int factorial(int n) { return n <= 1 ? 1 : n * factorial(n-1); }
static_assert(factorial(5) == 120);
template<class T>
constexpr bool is_pod_v = std::is_trivial_v<T> && std::is_standard_layout_v<T>;
constexpr, consteval (C++20), if constexpr together let you write code that’s branchless and zero-cost when called with constant inputs.
Compile-time hash
Implement a consteval string hash, generate switch tables — common HFT trick to dispatch on string commands at runtime in O(1) without runtime hashing.
15. Performance Hot Tips
- Cache friendliness wins. Arrays of structs with sequential access trounce trees of pointers, even when complexity is “the same.” A modern CPU handles ~1 cache miss per 100 cycles of compute.
- Reserve.
vector::reserve,unordered_map::reserve. Avoid grow churn. - Move into containers.
v.push_back(std::move(s));overv.push_back(s);. emplace_backoverpush_backwhen constructing in place.- Pass by value +
std::movein constructors and setters — modern idiom. - Avoid
std::endl— it flushes. Use'\n'. - Prefer iteration over recursion for deep structures; the function-call overhead and stack pressure matter.
- Profile before optimizing.
perf, VTune, callgrind, sampling profilers. Algorithmic wins dwarf micro-optimizations. - Compile with
-O2 -march=native -fltofor production. - Avoid
virtualin hot paths when possible. Devirtualization helps but a known-static dispatch is always cheaper. - Beware of false sharing — two atomics on the same cache line (typically 64B) bottleneck even when “independent.” Pad with
alignas(std::hardware_destructive_interference_size).
struct alignas(64) Counter { std::atomic<long> v{0}; };
16. Tooling — Sanitizers, Compiler-Specific Behavior
Sanitizers (recap from §11)
- ASan — memory errors.
- UBSan — undefined behavior.
- TSan — races.
- MSan (Clang only) — uninitialized reads.
Run them in CI. Production: don’t ship with sanitizers (perf cost), but optionally enable a hardened mode (_FORTIFY_SOURCE=2, -fstack-protector-strong).
Warning flags
g++ -Wall -Wextra -Wpedantic -Werror -Wshadow -Wconversion
Treat warnings as errors. The C++ ecosystem assumes you do.
Standard library debug modes
-D_GLIBCXX_DEBUG (libstdc++) checks bounds, iterator invalidation. Only debug builds — slow.
Vendor-specific behavior
- MSVC has different ABI rules (e.g., NRVO eligibility, exception spec). Don’t depend on inline assembly portability.
__attribute__((...))is GCC/Clang. MSVC uses__declspec.- Endian-ness, padding, alignment are platform-dependent. Don’t
memcpybetween systems without endian conversion.
17. C++ — What To Memorize Cold
- RAII. RAII. RAII.
- Rule of 0/3/5. Default to Rule of 0.
unique_ptrcheap,shared_ptrhas atomic refcount,weak_ptrbreaks cycles.- Move = transfer of ownership. Moved-from = valid but unspecified.
noexceptmove ops matter. - C++17 mandates prvalue copy elision — return by value is fine.
- Iterator invalidation rules per container — memorize the table in §8.
vectoris the default;unordered_mapis slow on cache locality.- Sort is introsort — O(N log N) worst, unstable.
stable_sortallocates. memory_order:relaxedfor counters,acquire/releasefor publication,seq_cstdefault.- UB list: OOB, signed overflow, races, use-after-free, strict aliasing, null deref, uninitialized read. Sanitizers catch most.
- Virtual destructor for polymorphic bases. Object slicing on by-value.
vector<bool>is special. nullptr,enum class,auto,string_view,optional,variant, structured bindings — modern toolkit.- Cache locality > algorithmic constants in modern hardware.
- Compile with
-O2 -march=native -flto -Wall -Wextrafor production. Run sanitizers in CI.
When you’re shaky on any of those, write a 30-line program that demonstrates the issue and run it under ASan + UBSan. C++’s sanitizers are some of the best feedback in any language; use them.
Phase 10 — Testing, Debugging, and Correctness
Target level: Intermediate → Senior Expected duration: 2–3 weeks (assuming Phases 0–9 are complete) Weekly cadence: 4–5 lab hours + apply testing discipline to every problem you solve elsewhere
Why This Phase Exists
Most candidates lose offers not because they couldn’t find an algorithm — they lose because their code was almost right and they never noticed. The interviewer asked “are you sure?”, they said “yes”, and then the interviewer ran one edge case and the screen went red.
Testing and debugging is the dimension where senior candidates separate from juniors. A junior writes code and hopes. A senior writes code and proves it works, then runs three deliberate test cases (one normal, one degenerate, one large), and only then claims “done.”
This phase teaches the discipline. It is short because the mechanics are simple. The habit is what takes weeks to internalize, which is why every later problem in your study should explicitly run the checklist here.
Concepts to Master
Test types
- Manual / desk-checked tests — what you trace through on paper during a 45-minute interview
- Smoke tests — 1–2 sanity examples to prove the code runs at all
- Unit tests — per-function correctness; use these heavily in
phase-08-practical-engineeringlabs - Integration tests — multi-component behavior; relevant when you implement subsystems (cache + invalidator, scheduler + worker)
- Property-based tests —
hypothesis-style; assert invariants over random inputs (e.g., “sorted output is a permutation of the input”) - Brute-force verifier — known-correct slow solution to validate the fast one on small inputs
- Stress testing — random-generation loop that runs the verifier and the fast solution and diffs them; the single best CP debugging tool
- Fuzzing (overview) — feed structured random input; useful for parsers, serializers, anything with a grammar
- Golden tests — record expected output for canonical inputs; mostly used in compiler/transform code
- Mutation testing (overview) — flip operators in your code and check if any test catches the mutation; reveals weak test suites
- Coverage analysis — branch and line coverage; necessary but not sufficient
Complexity & performance
- Complexity testing — measure runtime at N, 2N, 4N; check the doubling ratio matches your claimed Big-O
- Performance profiling —
cProfile/py-spy(Python),perf/pprof(Go/C++),async-profiler(Java) - Memory profiling —
tracemalloc/memory_profiler(Python),pprofheap (Go), heap dumps (JVM)
Concurrency
- Race detection —
-race(Go), TSan (C++/Rust),ThreadSanitizerfor clang - Deterministic concurrency testing — schedule injection, controlled interleaving, deterministic random
- Deadlock detection — lock-order graph analysis
Why Testing Matters in Interviews
Interviewers explicitly score “testing and verification” on the rubric. The signal they’re watching for:
| What you do | What it signals |
|---|---|
| Submit and say “done” | Junior — does not verify own work |
| Walk through one example manually | Acceptable — minimum bar |
| Walk through, then deliberately try an edge case | Senior — actively looking for bugs |
| Find your own bug and fix it without prompt | Strong senior signal |
| Identify a class of bugs you might have (“integer overflow when the array is large”) and write a test for that specific risk | Staff signal — anticipating failure modes |
Candidates who do not test lose offers even when their code is correct, because the interviewer cannot tell whether the correctness was deliberate or accidental.
The Universal Test Checklist
Apply this to every problem you solve, in every phase. Most of these take 10 seconds to consider; even rejecting them out loud earns the signal.
Input shape
- Empty input (
[],"",0,None) - Null input (if the language allows)
- Single element
- Two elements
- Maximum-size input (the constraint upper bound)
- Minimum-size input (often the constraint lower bound)
Input content
- All elements identical (duplicates)
- All elements distinct
- Already sorted (ascending and descending)
- Negative numbers
- Zero
- Mixed signs
- Values at integer boundaries (
INT_MAX,INT_MIN, overflow risk in sums/products) - Floating-point precision (when numeric)
Domain-specific
- Disconnected graph
- Self-loop, multi-edge
- Cycle in a graph that “should” be a tree
- Empty tree / single-node tree / skewed tree
- Linked list with one node, two nodes, with cycle
- Strings with unicode, with whitespace, with case differences
Output ambiguity
- Multiple valid answers (does the interviewer want any, all, or a canonical one?)
- Stable ordering required vs not
- Off-by-one in inclusive vs exclusive bounds
Failure modes
- Invalid input — does your function crash, return a sentinel, or raise?
- Concurrent access (for the practical-engineering labs)
- Timeout case — what happens when N is at the constraint limit?
Required Tests Per Lab (Curriculum-Wide Rule)
From Phase 10 forward, every lab you complete (and every lab from Phases 0–9 you re-solve) must include:
- 3 normal tests — the happy path, what the problem statement examples look like
- 3 edge tests — chosen from the checklist above; pick the three most relevant to this problem
- 1 large-input test — N at the constraint upper bound; verifies you didn’t accidentally write an O(N²) loop you thought was O(N log N)
- 1 randomized test (when a verifier exists) — random input, run brute force and fast solution, assert equal
- 1 invalid-input test (when applicable) — wrong type, malformed, out of range
Document these as test functions, not “I thought about it.” The act of writing them catches bugs.
Common Mistakes
- Testing only the given examples. The examples in the problem statement are almost always the happy path; they never exercise edge cases.
- Mental simulation without writing it down. Your brain skips steps. Trace on paper.
- Treating “the code compiles” as “the code works.” Compilation is the lowest bar.
- Not verifying complexity empirically. A claimed O(N) that runs 30× slower at 2N is actually quadratic.
- Adding tests after the bug. Add the test first, watch it fail, then fix; otherwise you don’t know your test would have caught it.
- Ignoring “obvious” cases. Empty input bugs are the #1 cause of failed phone screens.
- Not testing concurrency under load. A thread-safety bug at 1 thread is invisible; at 1000 threads on 8 cores, it’s a daily incident.
Debugging Checklist (Apply When Stuck)
- Reproduce. What is the smallest input that fails?
- Read the error. Stack trace, line number, value. Do not skip this.
- State the expected output. If you can’t, you don’t understand the problem.
- Diff expected vs actual. Is it off by one? Off by a factor? Wrong type?
- Binary-search the bug. Print state at midpoint of the algorithm; halve the search space.
- Check invariants. What was supposed to be true at this point? Assert it.
- Question assumptions. “I’m sure this list is sorted” — prove it.
- Read the code aloud. Speech catches what your eye skips.
- Rubber-duck explain. Tell an inanimate object what the code does, line by line.
- Step away for 60 seconds. Genuinely. The number of bugs solved this way is embarrassing.
Mastery Checklist
You have completed Phase 10 when you can:
- Generate the universal test checklist for any new problem in under 90 seconds
- Write a brute-force verifier for any problem with N ≤ 20
- Build a randomized stress-testing harness in under 10 minutes for a new problem
- Diagnose a wrong-answer bug in your own code in under 5 minutes using the debugging checklist
- Diagnose a TLE (timeout) bug by measuring the doubling ratio
- State the loop invariant for binary search, Kadane’s algorithm, and a simple DP
- Profile a Python script and identify the top 3 hot functions in under 5 minutes
- Find a race condition in a small Go/Java/C++ program using the language’s race detector
- Recognize when a test is too weak (mutation testing thought experiment)
Exit Criteria
Before moving to Phase 11:
- Complete all 6 labs in this directory with the full test suite written and passing
- Re-solve 3 problems from Phase 2 and 3 problems from Phase 5 applying the universal test checklist; document any bugs caught
- Run the stress-testing harness (Lab 5) on at least one problem you previously thought was correct, and report what you found
- Profile one of your Phase 8 practical-engineering implementations (e.g., LRU cache, rate limiter) and identify at least one inefficiency
Labs
| # | Lab | Focus | Anchor Problem |
|---|---|---|---|
| 1 | lab-01-edge-case-taxonomy.md | Systematic edge case discovery | Array median |
| 2 | lab-02-test-driven-problem-solving.md | Write tests before code | LRU cache |
| 3 | lab-03-debugging-under-pressure.md | Systematic debug protocol | Word Break (planted bug) |
| 4 | lab-04-correctness-proofs.md | Loop invariants & induction | Binary search + Kadane |
| 5 | lab-05-stress-testing-harness.md | Brute-force verifier + random fuzzing | Two-pointer variants |
| 6 | lab-06-performance-profiling.md | Empirical complexity + profiling | LIS implementations |
Connection to Other Phases
- Phase 2/3/4/5 — re-solve a problem from each, applying the universal test checklist
- Phase 7 (Competitive) — Lab 5 (stress testing) is the canonical CP debugging tool; use it on every CF problem you fail
- Phase 8 (Practical Engineering) — concurrency-aware testing is required for every lab; the rate limiter, LRU cache, and thread pool labs all need race-condition tests
- Phase 11 (Mocks) — the testing rubric (dimension 8) is scored on every mock; this phase trains that score
Lab 01 — Edge Case Taxonomy (Find the Median of an Unsorted Array)
Goal
Build a reusable, systematic edge-case taxonomy you can apply to any new problem in under 90 seconds. Use “find the median of an unsorted integer array” as the anchor — a problem that looks trivial but has at least 12 edge cases that a careless candidate will miss. By the end you should be able to enumerate 8+ edge cases for any array problem before writing a single line of code.
Background Concepts
An edge case is an input that is technically legal under the constraints but exercises a degenerate or boundary behavior in your algorithm. They fall into a small number of universal categories:
- Empty / null — what does your function do with
[]orNone? - Singleton — one element
- Identical elements — all equal; tests duplicate handling
- Boundary values —
INT_MAX,INT_MIN,0, negatives - Sorted / reverse-sorted — tests algorithms that assume scrambled input
- Maximum size — N at the constraint upper bound; tests complexity
- Output ambiguity — multiple valid answers; tests the spec
- Arithmetic overflow — sums/products that exceed
INT_MAX
The taxonomy is universal. The application is problem-specific.
Interview Context
“Find the median” is asked as a warm-up at Meta, Microsoft, and Bloomberg phone screens. The interviewer is not testing whether you know quickselect. They are testing whether you ask “what do you mean by median for an even-length array — average of the two middles or either one?” before writing code. Candidates who skip this question lose the point even if their code is otherwise correct.
The senior signal is to list out edges aloud before coding: “Empty array — should I return None or throw? Single element — that’s the median. Two elements — average. Even vs odd length — different formulas. Are values bounded so the sum of two won’t overflow?” Five sentences. Then code.
Problem Statement
Given an unsorted array of integers nums, return the median. If nums has odd length, return the middle value after sorting. If nums has even length, return the average of the two middle values.
Constraints
- 0 ≤ |nums| ≤ 10^5
- -10^9 ≤ nums[i] ≤ 10^9
- The return type may be a float (because of averaging)
Clarifying Questions
- Empty input? What should I return —
None,NaN, raise an exception? - Even length: average or either middle? Lower middle, upper middle, or the float average?
- Are duplicates allowed? (Yes; median definition handles them naturally.)
- Floating point precision concerns? If
nums[i]is up to 10^9, sum of two is 2×10^9 — fits in 32-bit signed int barely, but using(a + b) / 2.0in C++ overflows forINT_MAX + INT_MAX. Better:a/2.0 + b/2.0ora + (b - a)/2.0. - Modify input allowed? (Affects whether you can sort in place or need to copy.)
Examples
nums = [3, 1, 2] → 2 (odd length)
nums = [3, 1, 2, 4] → 2.5 (even, average of 2 and 3)
nums = [5] → 5 (singleton)
nums = [] → None / raise (clarify with interviewer)
nums = [7, 7, 7, 7] → 7.0 (all duplicates, even length)
nums = [INT_MAX, INT_MAX] → INT_MAX (overflow risk in average)
nums = [-3, -1, -2] → -2 (negatives)
Initial Brute Force
Sort, then index. Two lines of code.
def median(nums):
if not nums:
return None
s = sorted(nums)
n = len(s)
if n % 2 == 1:
return s[n // 2]
return (s[n // 2 - 1] + s[n // 2]) / 2
Brute Force Complexity
Time O(N log N), space O(N) (or O(1) if you sort in place and the caller allows mutation). For N = 10^5 this is ~1.7×10^6 comparisons — well within any interview time limit.
Optimization Path
The interviewer may now ask: “Can you do better than O(N log N)?” The answer is quickselect, which finds the k-th smallest in expected O(N) using partition-based recursion. Worst case O(N²); use median-of-medians for guaranteed O(N) if pressed.
For the edge-case lab, do not optimize. The point is to enumerate edges before the algorithm matters. Quickselect has more edge cases (recursion depth on degenerate partitions, pivot selection bias) so optimizing without first nailing edges makes the bug surface larger.
Final Expected Approach
- Validate input. Return
None(or raise) on empty. - Sort a copy (do not mutate caller’s array unless agreed).
- Compute middle index
mid = n // 2. - If odd, return
sorted[mid]. - If even, return
sorted[mid - 1] + sorted[mid]divided by 2, usinga + (b - a) / 2form to avoid overflow.
Data Structures Used
- A sortable copy of the array. In Python
sorted()returns a new list. In Java useArrays.sort()on a clone; in C++std::sorton a copy. - No auxiliary structures.
Correctness Argument
After sorting, by definition the value at index n // 2 is the lower-middle (0-indexed); the value at n // 2 - 1 is the upper-lower; their average is the median for even lengths. For odd lengths, n // 2 is exactly the middle. The sort guarantees the ordering invariant required.
Complexity
Time O(N log N) sort + O(1) lookup. Space O(N) for the copy (or O(1) if in-place sort is allowed). Quickselect: expected O(N), worst O(N²); median-of-medians: worst O(N) with larger constant.
Implementation Requirements
- Function signature should accept any iterable convertible to a list.
- Do not mutate the caller’s input.
- Return type:
float(even for odd-length inputs, for consistency) or use a tagged return; document which. - Handle empty input explicitly with the chosen convention.
Tests
Smoke (3 normal)
assert median([3, 1, 2]) == 2
assert median([1, 2, 3, 4]) == 2.5
assert median([5, 2, 8, 1, 9]) == 5
Edge (5 — exceeds the 3 minimum because this lab is about edges)
assert median([]) is None # empty
assert median([42]) == 42 # singleton
assert median([1, 2]) == 1.5 # even, smallest
assert median([7, 7, 7, 7]) == 7 # all duplicates
assert median([-3, -1, -2]) == -2 # all negatives
assert median([10**9, 10**9]) == 10**9 # overflow boundary
assert median([-10**9, 10**9]) == 0 # mixed extremes
Large
import random
random.seed(0)
big = [random.randint(-10**9, 10**9) for _ in range(10**5)]
result = median(big)
assert isinstance(result, (int, float)) # didn't crash; didn't take >1s
Randomized verifier
def brute_median(nums):
s = sorted(nums)
n = len(s)
return s[n//2] if n % 2 else (s[n//2 - 1] + s[n//2]) / 2
for _ in range(1000):
n = random.randint(1, 50)
nums = [random.randint(-100, 100) for _ in range(n)]
assert median(nums) == brute_median(nums)
Invalid input
try:
median(None)
assert False, "should have raised"
except TypeError:
pass
Follow-up Questions
- Streaming median. Find the median as numbers arrive one at a time. → Two heaps (max-heap of lower half, min-heap of upper half). O(log N) per insert, O(1) per query.
- Median of two sorted arrays. Classic LC 4 hard. → Binary search on partition; O(log min(N, M)).
- k-th smallest in unsorted. → Quickselect; O(N) expected.
- Weighted median. Each value has a weight; find the value where cumulative weight crosses half. → Sort + prefix scan; O(N log N).
- Approximate median in one pass with O(1) memory. → Reservoir sampling + recursion, or P² algorithm.
Product Extension
- Latency percentiles in distributed monitoring. P50 (median), P99, P99.9. Cannot store all latencies — use t-digest or HdrHistogram for compact mergeable approximations.
- A/B testing. Comparing median user session length between buckets requires bootstrap confidence intervals because medians don’t have closed-form variance.
- Recommendation systems. “Median rating per item” for cold-start ranking.
Language/Runtime Follow-ups
- Python:
sorted()is TimSort, O(N log N) worst case; uses additional O(N) memory.list.sort()is in-place.nums[n//2]is O(1) indexing. - Java:
Arrays.sort(int[])uses dual-pivot quicksort (O(N log N) average, O(N²) worst on adversarial input).Arrays.sort(Object[])uses TimSort. Auto-boxingIntegeradds overhead. - C++:
std::sortis introsort (quicksort + heapsort fallback); worst-case O(N log N).std::nth_elementis O(N) average for quickselect. Beware integer overflow in(a + b) / 2for signed 32-bit; usea + (b - a) / 2. - Go:
sort.Intsis introsort, O(N log N). No overflow checks inintarithmetic; wraps silently on 32-bit platforms. - JavaScript:
Array.prototype.sort()defaults to lexicographic string comparison —[10, 9, 2].sort()returns[10, 2, 9]. Always pass a comparator:sort((a, b) => a - b).
Common Bugs
- Empty input crash.
s[n // 2]withn == 0iss[0]on an empty list →IndexError. - Integer overflow on average.
(a + b) / 2overflows whena + b > INT_MAX. Usea + (b - a) / 2or use floating-point earlier. - Integer division for even-length median. In Python 2 / Java,
(a + b) / 2truncates. In Python 3,/is float — but//is integer. Be explicit. - Mutating caller’s array. Passing
nums.sort()to a function modifies the original. Usesorted(nums). - Off-by-one for even length.
n // 2is the upper middle (0-indexed);n // 2 - 1is the lower. Confusing these gives the wrong answer for[1, 2, 3, 4]. - JavaScript default sort. Returns string-sorted order for numbers.
Debugging Strategy
If your function returns the wrong value:
- Print the sorted array. Is it actually sorted? (Confirms no JavaScript-style default-sort bug.)
- Print
n,n // 2,n % 2. Is the index what you expect? - Check parity branch — did you accidentally swap the odd/even branches?
- For overflow: print
s[n//2 - 1] + s[n//2]before dividing; check if it matches the expected sum. - For mutation bugs: print the input both before and after the call. If it changed, you mutated.
If you TLE on the large test, you wrote O(N²) accidentally (e.g., used insertion sort, or sorted inside a loop).
Mastery Criteria
- Wrote the function correctly on the first try with all edge cases handled
- Listed all 8+ edge cases aloud before writing code (time yourself: under 90 seconds)
-
Identified the overflow risk in
(a + b) / 2without prompting - Wrote the randomized verifier in under 5 minutes
- Can recite the universal edge-case taxonomy (empty / singleton / two / duplicates / sorted / boundary / overflow / mixed) without looking
- Re-applied the taxonomy to one Phase 2 problem and caught at least one edge case you previously missed
Lab 02 — Test-Driven Problem Solving (LRU Cache)
Goal
Write tests before writing the implementation. Use LRU cache (LC 146) as the anchor — a problem where ambiguities in the spec (does put of an existing key count as a “use”? what does capacity 0 mean?) are best surfaced by writing test cases first. By the end you should treat tests as a design tool, not a verification afterthought.
Background Concepts
Test-driven design (TDD) in an interview context is not the dogmatic red-green-refactor cycle. It is the discipline of writing 3–5 example calls and their expected results before writing the implementation, because:
- Writing the expected output forces you to confront spec ambiguities (and ask the interviewer).
- The tests double as documentation of your understanding — if the interviewer disagrees, you discover it before you’ve written 50 lines.
- The tests become your verification suite — you don’t have to invent them after the fact under time pressure.
- The act of choosing tests reveals edge cases you would otherwise miss.
The cost is 2–3 minutes up front. The savings are usually 10+ minutes of debugging later.
Interview Context
LRU cache is the most-asked OOD-flavored coding question at FAANG. Google, Meta, Amazon, Bloomberg all ask it in some form. The standard expectation is O(1) get and put using a hashmap + doubly linked list.
The senior signal is to enumerate behavioral test cases before touching code: “get on missing key returns -1 (or what?). put of existing key updates value AND marks as recently used? put over capacity evicts the LRU; what if multiple keys are tied? Does get count as a use?” These are the real questions. Candidates who code first and discover these mid-interview look junior.
Problem Statement
Design a data structure that supports:
LRUCache(int capacity)— initialize with positive capacityint get(int key)— return value if present, else -1; using a key counts as “recently used”void put(int key, int value)— insert or update; on overflow evict the least recently used; updating an existing key also counts as recently used
All operations must be O(1) average.
Constraints
- 1 ≤ capacity ≤ 3000
- 0 ≤ key, value ≤ 10^4
- Up to 2×10^5 calls
Clarifying Questions (Surface These Before Writing Code)
- Does
getmark the key as recently used? (Yes — standard.) - Does
puton an existing key mark as recently used? (Yes — standard. Confirm.) - Capacity of 0 — is that legal? (Constraints say ≥ 1, but worth confirming the contract.)
- Eviction policy when multiple keys are tied for least recently used — can this happen? (In a strict LRU with sequential ops, no — every access updates order. Tie only on initial fill, but at that point the oldest insertion is LRU.)
- Thread safety required? (Almost never in the interview; always ask anyway. If yes, see Phase 8 LRU lab.)
Examples (Written as Tests First)
# Test 1: basic put/get
cache = LRUCache(2)
cache.put(1, 1)
cache.put(2, 2)
assert cache.get(1) == 1 # returns 1
cache.put(3, 3) # evicts key 2 (LRU)
assert cache.get(2) == -1 # not found
assert cache.get(3) == 3
cache.put(4, 4) # evicts key 1
assert cache.get(1) == -1
assert cache.get(3) == 3
assert cache.get(4) == 4
# Test 2: put on existing key updates value AND recency
cache = LRUCache(2)
cache.put(1, 1)
cache.put(2, 2)
cache.put(1, 10) # update key 1; now order is 2 (LRU), 1 (MRU)
cache.put(3, 3) # evicts 2, not 1
assert cache.get(2) == -1
assert cache.get(1) == 10 # updated value preserved
# Test 3: get on missing key
cache = LRUCache(1)
assert cache.get(99) == -1 # never inserted
These three tests already locked down 6 design decisions. Now you can write the implementation.
Initial Brute Force
Use a single Python dict and an auxiliary list to track insertion order. get: O(1) dict lookup, but moving to end is O(N) list operation. put: O(1) insert, but O(N) eviction scan. Total: O(N) per op.
Alternatively, use collections.OrderedDict which is already a hash + doubly linked list internally. move_to_end and popitem(last=False) are both O(1). Single-class solution in ~15 lines.
Brute Force Complexity
Naive dict + list: O(N) per op, fails on 2×10^5 calls at large capacity → 6×10^8 ops, TLE.
Optimization Path
The standard answer is hashmap + doubly linked list:
- Hashmap:
key → node - Doubly linked list: nodes in MRU-to-LRU order
get: hashmap lookup → unlink node → relink at head (MRU)put: if key exists, update value + move to head; else create node, insert at head; if size > capacity, remove tail (LRU) and delete from hashmap
All operations are O(1) because both the hashmap and the doubly linked list support O(1) insert/delete with a node reference.
Using OrderedDict in Python is equivalent and acceptable in interviews if you explain why it works (because it’s a hashmap + DLL internally). In Java, use LinkedHashMap with accessOrder=true.
Final Expected Approach
Implement with explicit doubly linked list to demonstrate understanding:
class Node:
__slots__ = ('key', 'val', 'prev', 'next')
def __init__(self, key=0, val=0):
self.key, self.val = key, val
self.prev = self.next = None
class LRUCache:
def __init__(self, capacity: int):
self.cap = capacity
self.cache = {}
# sentinel head/tail to avoid edge cases
self.head = Node()
self.tail = Node()
self.head.next = self.tail
self.tail.prev = self.head
def _remove(self, node):
node.prev.next = node.next
node.next.prev = node.prev
def _add_to_front(self, node):
node.next = self.head.next
node.prev = self.head
self.head.next.prev = node
self.head.next = node
def get(self, key: int) -> int:
if key not in self.cache:
return -1
node = self.cache[key]
self._remove(node)
self._add_to_front(node)
return node.val
def put(self, key: int, value: int) -> None:
if key in self.cache:
node = self.cache[key]
node.val = value
self._remove(node)
self._add_to_front(node)
else:
if len(self.cache) >= self.cap:
lru = self.tail.prev
self._remove(lru)
del self.cache[lru.key]
node = Node(key, value)
self.cache[key] = node
self._add_to_front(node)
Data Structures Used
- Dict — O(1) key → node lookup
- Doubly linked list with sentinels — O(1) insert at head, O(1) remove from tail or middle
Sentinels eliminate if node is None checks at the boundary, the #1 source of LRU bugs.
Correctness Argument
Invariant 1: cache[key] always points to the node currently in the linked list with that key. Maintained because every insertion adds to both, every removal removes from both.
Invariant 2: The linked list is ordered MRU → LRU from head to tail. Maintained because every access (get or put on existing) moves the node to the front, and every new insert goes to the front.
Invariant 3: len(cache) ≤ capacity. Maintained because put evicts the tail (LRU) before adding when at capacity.
Complexity
get: O(1). put: O(1). Space: O(capacity).
Implementation Requirements
- Use sentinels — no null checks at head/tail
- Use
__slots__on the Node class in Python for memory efficiency - Keep
_removeand_add_to_frontas separate helpers — do not inline; the duplication is a bug magnet - Update the hashmap whenever you touch the linked list, never one without the other
Tests
Smoke (3 normal)
The three tests in the Examples section above.
Edge
# Capacity 1 — every put evicts
cache = LRUCache(1)
cache.put(1, 1)
cache.put(2, 2)
assert cache.get(1) == -1
assert cache.get(2) == 2
# Get on a key that was evicted
cache = LRUCache(2)
cache.put(1, 1); cache.put(2, 2); cache.put(3, 3)
assert cache.get(1) == -1
# Repeated put on same key never evicts
cache = LRUCache(2)
for v in range(100):
cache.put(1, v)
assert cache.get(1) == 99
Large
cache = LRUCache(3000)
for i in range(200000):
cache.put(i % 5000, i)
# Verifies O(1) per op; should complete in <1s
Randomized verifier (brute O(N) LRU using list)
class BruteLRU:
def __init__(self, cap):
self.cap = cap
self.order = []
self.vals = {}
def get(self, k):
if k not in self.vals: return -1
self.order.remove(k); self.order.append(k)
return self.vals[k]
def put(self, k, v):
if k in self.vals:
self.order.remove(k)
elif len(self.vals) >= self.cap:
evict = self.order.pop(0)
del self.vals[evict]
self.order.append(k); self.vals[k] = v
import random
random.seed(42)
for trial in range(100):
cap = random.randint(1, 10)
fast = LRUCache(cap); slow = BruteLRU(cap)
for _ in range(200):
if random.random() < 0.5:
k = random.randint(0, 15)
assert fast.get(k) == slow.get(k)
else:
k = random.randint(0, 15); v = random.randint(0, 100)
fast.put(k, v); slow.put(k, v)
Invalid input
# Capacity 0 — undefined by spec; behavior must be documented
try:
LRUCache(0)
except ValueError:
pass
# Or: assert never stores anything if zero is allowed
Follow-up Questions
- Make it thread-safe. Per-op mutex is the simple answer; striped locks for higher throughput. (See Phase 8 lab.)
- LFU instead of LRU. Track frequencies + a min-frequency pointer; harder, see Phase 8 lab-02.
- TTL eviction. Add expiration timestamp per entry; lazy or eager eviction tradeoff.
- Distributed LRU. Consistent hashing across nodes; cache coherence is now a hard problem.
- Approximate LRU. Sample K random entries and evict the oldest among them (Redis approach); O(1) eviction without strict ordering.
Product Extension
- CPU caches. Hardware caches are pseudo-LRU because true LRU’s pointer overhead is too expensive.
- CDN edge caches. Strict LRU loses popular content under cache pollution; LFU + admission filter (TinyLFU) is state of the art.
- Database buffer pools. PostgreSQL uses a clock-sweep approximation; MySQL InnoDB uses LRU with a midpoint insertion point to resist scan pollution.
- OS page replacement. Linux uses two clock lists (active + inactive) to approximate LRU at low cost.
Language/Runtime Follow-ups
- Python:
OrderedDictis implemented in C and uses a doubly linked list internally;move_to_endandpopitem(last=False)are O(1). The__slots__on Node avoids per-instance__dict__, saving ~50% memory per node. - Java:
LinkedHashMapwith(capacity, 0.75f, true)constructor enables access order; overrideremoveEldestEntryfor eviction.ConcurrentHashMapdoes not provide LRU; you’d needCaffeineor a custom striped lock. - C++:
std::list+std::unordered_map<Key, std::list::iterator>; iterators tostd::listremain valid after other inserts/erases, which is essential for this design. - Go: No built-in LRU; use
container/list+ a map. The standard libraryhashicorp/golang-lruis the go-to. - Rust: Borrowing rules make a vanilla doubly linked list hard; use
lrucrate which uses raw pointers internally.
Common Bugs
- Forgot to update recency on
get. Tests pass forputbut fail whengetshould “save” a key from eviction. - Forgot to update recency on
putof existing key. The key that was just updated gets evicted on the nextput. - Hashmap and linked list out of sync. Removed from list but not dict, or vice versa. Always update both.
- No sentinels → null pointer at head/tail. Sentinels eliminate 4 different null checks.
- Evicting before checking if key already exists. Update-of-existing should not trigger eviction.
- Off-by-one on capacity comparison.
len(cache) >= capvslen(cache) > cap— first is correct because you’re about to add one more.
Debugging Strategy
If a test fails:
- Print the linked list (head → tail with keys) after each operation. Verify it matches your expected MRU order.
- Print
len(cache)after each op. It should match the number of inserts minus evictions. - Cross-check: after every op,
set(cache.keys())should equal the set of keys in the linked list. If not, you have a sync bug. - Run the randomized verifier with a small seed; when it fails, print the trace of operations that led to the divergence — it will be 10–20 ops long and obvious.
Mastery Criteria
- Wrote the tests before writing the implementation
- Surfaced at least 3 spec ambiguities through the tests
- Implementation worked on the first run (because tests forced the design to be correct)
- Used sentinels in the linked list
- Wrote the randomized verifier and ran it for 100+ trials with no divergence
- Can explain why updating an existing key must mark it as MRU
- Re-applied TDD to one Phase 8 lab and recorded how many design questions it surfaced
Lab 03 — Debugging Under Pressure (Word Break with a Planted Bug)
Goal
Build a systematic debugging protocol you can execute under interview pressure in under 5 minutes. The anchor is Word Break (LC 139) with a planted off-by-one — you’ll be given buggy code and a failing test, and the goal is to find the bug methodically rather than by panic-staring at the screen. By the end you should reach for the protocol automatically when stuck.
Background Concepts
Under pressure, most candidates default to panic debugging: re-read the code 5 times, add random print statements, change one thing, hope. This rarely works in 5 minutes and looks terrible to the interviewer.
Systematic debugging is a 6-step protocol:
- Reproduce — Confirm the failing input. The smallest one that fails.
- Isolate — What is the exact discrepancy? Expected X, got Y.
- Hypothesize — Form a specific hypothesis: “I think the bug is in the inner loop’s bound.”
- Verify — Add one targeted print or assertion that confirms or denies the hypothesis. Not five prints.
- Fix — Make the minimum change that addresses the verified hypothesis.
- Re-test — Run all tests, not just the failing one. Make sure you didn’t break something else.
The discipline is in steps 3 and 4. Hypothesize before you print. Random prints waste time and create noise.
Interview Context
You will hit a bug in 80% of medium+ interview problems. How you respond is a major signal:
| Behavior | Signal |
|---|---|
| “It doesn’t work, let me try…” (silent typing for 3 min) | Junior — no protocol |
| “Let me add some prints…” (adds 8 prints, can’t read output) | Junior — random debugging |
“The expected output is X, I got Y. So the difference is Z. My hypothesis is that the bug is in the loop bound — let me check by printing i at line 7.” | Senior — narrating the protocol |
| Finds the bug, then says “Let me also test the case I just fixed plus an adjacent case, in case I introduced something.” | Senior+ — proactive regression |
Narrating the protocol aloud is itself the signal. The interviewer can hear that you’re a person who has debugged a thousand bugs and has a process.
Problem Statement
You are given the following implementation of Word Break. It is buggy. Find and fix the bug using the debug protocol. You may add prints/asserts but must remove them before declaring the fix complete.
def word_break(s: str, word_dict: list[str]) -> bool:
"""Return True iff s can be segmented into a sequence of words from word_dict."""
words = set(word_dict)
n = len(s)
# dp[i] = True iff s[:i] can be segmented
dp = [False] * n
dp[0] = True
for i in range(1, n + 1):
for j in range(i):
if dp[j] and s[j:i] in words:
dp[i] = True
break
return dp[n]
Failing test: word_break("leetcode", ["leet", "code"]) should return True. The function raises IndexError.
Constraints
- 1 ≤ |s| ≤ 300
- 1 ≤ |word_dict| ≤ 1000
- All strings lowercase letters
Clarifying Questions
(Not applicable — this lab uses a fixed buggy implementation. The clarifying questions for Word Break itself are: are words reusable? Yes. Are duplicates in word_dict significant? No. Empty s? Returns True conventionally.)
Examples
word_break("leetcode", ["leet", "code"]) → True
word_break("applepenapple", ["apple", "pen"]) → True
word_break("catsandog", ["cats", "dog", "sand", "and", "cat"]) → False
word_break("", ["any"]) → True (empty string is trivially segmentable)
Initial Brute Force
Recursion: for each prefix of s that is in word_dict, recurse on the suffix. Time O(2^N) without memoization.
Brute Force Complexity
O(2^N) without memoization; O(N²) with (each prefix length tried once, each requiring an O(N) substring check and set lookup).
Optimization Path
Bottom-up DP: dp[i] = True iff s[:i] can be segmented. Transition: dp[i] = True iff there exists j < i with dp[j] True and s[j:i] ∈ words. The buggy implementation above is the right idea — just slightly wrong.
Final Expected Approach (Correct Version)
def word_break(s, word_dict):
words = set(word_dict)
n = len(s)
dp = [False] * (n + 1) # ← THE FIX: size n+1, not n
dp[0] = True
for i in range(1, n + 1):
for j in range(i):
if dp[j] and s[j:i] in words:
dp[i] = True
break
return dp[n]
Applying the Debug Protocol (Walkthrough)
Step 1 — Reproduce
Run word_break("leetcode", ["leet", "code"]). Get IndexError: list assignment index out of range. Confirm with the exact line: the assignment dp[i] = True when i = n.
Step 2 — Isolate
The loop runs i from 1 to n inclusive (range(1, n + 1)). dp has length n. So when i == n, dp[i] is out of bounds.
Step 3 — Hypothesize
The DP array is one element too small. The semantics of dp[i] cover i = 0 (empty prefix) through i = n (full string), which is n + 1 values. The author wrote dp = [False] * n — off by one.
Step 4 — Verify
Add assert len(dp) == n + 1 after allocation. It fails, confirming the hypothesis.
Step 5 — Fix
Change dp = [False] * n to dp = [False] * (n + 1).
Step 6 — Re-test
Run the failing test → True. Run all four examples → all pass. Add an edge test word_break("", []) → returns True (since dp[0] is True initialized, and dp[n] == dp[0]).
Total time if narrated cleanly: under 4 minutes.
Data Structures Used
- Set for O(1) word lookup
- DP array of booleans
Correctness Argument
dp[0] is the base case: the empty prefix is trivially segmentable. For i > 0, dp[i] is True iff some split point j makes both halves valid: dp[j] (the left half is segmentable) and s[j:i] is a word. By induction on i, this is correct.
Complexity
Time O(N²) (or O(N² + total dict string length) if you care about set membership cost). Space O(N + W) where W is the dictionary size.
Implementation Requirements
- DP array size must be
n + 1, notn— this is the planted bug - Use a set for word lookup, not a list (O(1) vs O(W) per check)
- Break out of inner loop on first success (constant factor, not asymptotic)
Tests
Smoke (3 normal)
assert word_break("leetcode", ["leet", "code"]) is True
assert word_break("applepenapple", ["apple", "pen"]) is True
assert word_break("catsandog", ["cats", "dog", "sand", "and", "cat"]) is False
Edge
assert word_break("", ["any"]) is True # empty string
assert word_break("a", ["a"]) is True # single char match
assert word_break("a", ["b"]) is False # single char no match
assert word_break("aaaa", ["a", "aa"]) is True # overlapping dict
assert word_break("ab", ["a"]) is False # partial match
Large
s = "a" * 300
dict_ = ["a", "aa", "aaa", "aaaa"]
assert word_break(s, dict_) is True
# Should complete in <100ms; O(N^2) = 9e4 ops
Randomized
import random, string
random.seed(0)
for _ in range(100):
words = [''.join(random.choices(string.ascii_lowercase, k=random.randint(1, 4)))
for _ in range(5)]
s = ''.join(random.choice(words) for _ in range(random.randint(1, 10)))
assert word_break(s, words) is True # constructed to be segmentable
Invalid input
# Empty dict — empty string still works, non-empty doesn't
assert word_break("", []) is True
assert word_break("a", []) is False
Follow-up Questions
- Return all segmentations. Word Break II (LC 140). DFS + memoization; exponentially many results possible.
- Return one segmentation. Track parent pointers in DP; reconstruct by backtracking.
- Minimum number of words. Modify DP:
dp[i] = min over j of dp[j] + 1. - Streaming input. Word Break on a stream — Aho-Corasick automaton.
- Dictionary changes dynamically. Trie + DP, but rebuilds are expensive on every dict change.
Product Extension
- Spell-check / autocomplete segmentation. “iphoneapp” → “iphone app”. Used in URL/path tokenization.
- Hashtag splitting. Twitter “#machinelearning” → “machine learning”. Same algorithm with a dictionary + frequency weights for tiebreaks.
- DNS subdomain analysis. “thequickbrownfox.com” — fraud detection wants to know if the hostname is composed of dictionary words.
- Chinese/Japanese word segmentation. No spaces between words; same DP with a much larger dictionary.
Language/Runtime Follow-ups
- Python:
s[j:i]allocates a new string each call (O(i-j) time and space). For very long strings, prefer indexing into a precomputed structure or usingstr.startswithagainst the dictionary entries. - Java:
s.substring(j, i)is O(i-j) since Java 7 (used to be O(1) view; changed for security). Same allocation cost. - Go:
s[j:i]on a string is a view — O(1), no allocation. This makes Go’s version significantly faster. - C++:
std::string_view(C++17) gives O(1) slicing;s.substr(j, i-j)allocates. - All: Set membership of strings is O(|substring|) for hashing + O(|substring|) for equality on collision — not strictly O(1). Matters for very long substrings.
Common Bugs
- Off-by-one on DP size — the planted bug
- Initializing
dp[0] = False— empty prefix is the base case, must be True - Forgetting
breakafter success — correctness still works but performance suffers - Using list for word_dict — O(W) per lookup, blows complexity to O(N²·W)
- Inclusive vs exclusive bounds —
s[j:i+1]vss[j:i]is the most common off-by-one when porting between languages
Debugging Strategy
The 5-step systematic protocol:
- Reproduce minimally. If the bug shows up on a 300-char string, shrink to the smallest failing case (here: any non-empty string).
- Read the exception fully.
IndexError+ line number tells you almost everything. Don’t skip the stack trace. - State the discrepancy precisely. “Expected True, got IndexError on
dp[i] = Truewheni = 8andlen(dp) = 8.” - Form a specific hypothesis. Not “it’s broken somewhere”; rather, “the array is one too small for the loop range.”
- Verify with one targeted print/assert.
assert len(dp) > iimmediately before the assignment. - Regression test. After fixing, run the full suite. In this case, also test empty string explicitly since that’s the boundary.
The 5-minute panic protocol (when truly stuck):
- Stop typing.
- State aloud: “I’m stuck. Let me restate what I know.”
- Restate the input and expected output.
- State what your code does for that input, step by step.
- The bug almost always reveals itself in the gap between “what the code does” and “what should happen.”
Mastery Criteria
- Found the planted bug in under 5 minutes using the protocol
- Narrated each step aloud (or in notes) — Reproduce, Isolate, Hypothesize, Verify, Fix, Re-test
- Added exactly ONE targeted assertion to verify the hypothesis (not 5 prints)
- Ran the full test suite after the fix, not just the failing one
- Wrote the empty-string edge test that would have caught this bug originally
- Can recite the 6-step protocol from memory
- Applied the protocol to one of your own past wrong-answer submissions and timed yourself
Lab 04 — Correctness Proofs (Binary Search & Kadane’s Algorithm)
Goal
Prove the correctness of two short, foundational algorithms — binary search and Kadane’s — using loop invariants and induction. By the end you should be able to state the invariant for any loop you write and use it both to prove correctness and to find bugs before they manifest.
Background Concepts
A loop invariant is a statement that is true:
- Initially (before the loop starts)
- Maintained (if true before an iteration, true after)
- Terminating (when the loop exits, it implies the desired post-condition)
This is induction on iterations. The invariant is what your loop promises about its state. If you can state the invariant out loud while coding, off-by-one bugs disappear because you can check each iteration against the promise.
A monovariant is a quantity that strictly decreases (or increases) each iteration and is bounded — it proves termination. For binary search, the monovariant is the search-range width.
An inductive proof for a recursive function: prove the base case correct; assume the recursive call is correct (induction hypothesis); show the combination is correct.
Interview Context
Interviewers rarely demand a formal proof, but they constantly ask “are you sure this works?” or “why does this work?” The candidates who answer with a precise invariant (“at the top of the loop, lo is the smallest index that could be the answer and hi is one past the largest”) look senior. The candidates who say “uh, I think so” look junior even when the code is correct.
For DP problems, “what’s the state, what’s the transition, and why does the order of iteration give you the correct value when you read it?” is the proof — interviewers explicitly ask this at Meta, Google, and Bloomberg.
Problem Statement
Prove correctness of two algorithms:
Part A: Binary search. Given a sorted array a and a target t, return the index of t if present, else -1.
Part B: Kadane’s algorithm. Given an array of integers (positive, negative, mixed), return the maximum sum of any non-empty contiguous subarray.
For each, you must:
- Write the code
- State the loop invariant precisely
- Prove the invariant holds initially, is maintained, and implies correctness on exit
- Identify the monovariant that proves termination
Constraints
- 1 ≤ |a| ≤ 10^5 for both problems
- Values: -10^9 ≤ a[i] ≤ 10^9 (Kadane: watch overflow on long all-positive subarrays)
Clarifying Questions
(Standard problem statements; the lab is about proof, not problem disambiguation.)
Examples
Binary search:
a = [1, 3, 5, 7, 9, 11], t = 7 → 3
a = [1, 3, 5, 7, 9, 11], t = 4 → -1
a = [], t = 1 → -1
Kadane:
[-2, 1, -3, 4, -1, 2, 1, -5, 4] → 6 (subarray [4, -1, 2, 1])
[-3, -1, -2] → -1 (best single element)
[5] → 5
Initial Brute Force
Binary search: linear scan, O(N).
Kadane: triple loop over (i, j, sum), O(N³). With prefix sums, O(N²).
Brute Force Complexity
Linear: O(N). Triple loop: O(N³). Both correct, both slow.
Optimization Path
Binary search: O(log N) by halving the candidate range each step. Kadane: O(N) by maintaining the best subarray ending at index i and the best seen so far.
Final Expected Approach
Part A — Binary Search (with Proof)
def binary_search(a, t):
lo, hi = 0, len(a) # half-open: search range is [lo, hi)
while lo < hi:
mid = lo + (hi - lo) // 2
if a[mid] == t:
return mid
elif a[mid] < t:
lo = mid + 1
else:
hi = mid
return -1
Loop invariant: At the top of every iteration, if t is present in a, then t is at some index in [lo, hi).
Initialization: lo = 0, hi = len(a). If t is present, it’s at some index in [0, len(a)) by definition. Invariant holds.
Maintenance: Assume invariant holds before an iteration. Compute mid.
- If
a[mid] == t, return immediately — correct. - If
a[mid] < t: becauseais sorted, every index≤ midhas value≤ a[mid] < t, sotis not at any of those indices. Iftis ina, it must be in[mid+1, hi). Settinglo = mid + 1preserves the invariant. - If
a[mid] > t: symmetric;tnot at index≥ mid. Settinghi = midpreserves the invariant.
Termination & post-condition: The monovariant is hi - lo, which strictly decreases each iteration (verify: in both update branches mid = lo + (hi-lo)//2, after which lo' > lo or hi' < hi; specifically hi' - lo' < hi - lo always when lo < hi). It’s bounded below by 0, so the loop terminates. When lo == hi, the search range is empty. By the invariant, if t were present, it would be in an empty range — contradiction. So t is absent. Returning -1 is correct.
Critical subtlety — mid = lo + (hi - lo) // 2 vs (lo + hi) // 2: the former avoids integer overflow when lo + hi > INT_MAX. In Python this doesn’t matter (arbitrary precision int), but in Java/C++ it’s a real bug. Famous: the Java SDK had this bug in Arrays.binarySearch for ~9 years (Bloch, 2006).
Part B — Kadane’s Algorithm (with Proof)
def kadane(a):
best_here = best_overall = a[0]
for i in range(1, len(a)):
best_here = max(a[i], best_here + a[i])
best_overall = max(best_overall, best_here)
return best_overall
Loop invariant: At the top of iteration i (1-indexed for clarity):
best_hereis the maximum sum of any contiguous subarray ending at indexi - 1.best_overallis the maximum sum of any contiguous subarray withina[0..i-1]inclusive.
Initialization: Before the loop (i.e., before i = 1), the only subarray of a[0..0] is [a[0]] with sum a[0]. Both best_here and best_overall are set to a[0]. Invariant holds.
Maintenance: Assume the invariant holds at the start of iteration i. Consider all contiguous subarrays ending at index i. Each such subarray is either:
- The singleton
[a[i]], with suma[i], OR - An extension of a subarray ending at
i - 1, with sum(sum of that subarray) + a[i].
The best extension is best_here + a[i] (by the invariant on best_here). So the best subarray ending at i has sum max(a[i], best_here + a[i]), which is exactly the new best_here. Invariant clause 1 maintained.
The best subarray within a[0..i] is either entirely within a[0..i-1] (covered by old best_overall) or ends at i (covered by new best_here). The new best_overall = max(old best_overall, new best_here) is therefore correct. Clause 2 maintained.
Termination & post-condition: The loop runs exactly n - 1 iterations (finite, no monovariant needed). On exit, best_overall is the max contiguous subarray sum within a[0..n-1] = the whole array. Returning it is correct.
Edge case proof: Kadane requires a non-empty (the problem states this); for an all-negative array, the answer is the maximum single element. The invariant handles this because best_here will reset to a[i] whenever best_here + a[i] < a[i], i.e., whenever best_here < 0. This is why the algorithm works for negative-only arrays — a common bug is to initialize best_here = 0, which incorrectly returns 0 for all-negative input.
Data Structures Used
- Plain arrays
- Two integer variables for Kadane
- Two integer indices for binary search
Correctness Argument
See Part A and Part B above.
Complexity
- Binary search: O(log N) time, O(1) space.
- Kadane: O(N) time, O(1) space.
Implementation Requirements
- Use
lo + (hi - lo) // 2for binary search midpoint - Use half-open interval
[lo, hi)for binary search — easier to reason about than closed[lo, hi] - Initialize Kadane’s
best_hereandbest_overalltoa[0], not0, to handle all-negative arrays
Tests
Smoke
assert binary_search([1, 3, 5, 7, 9], 5) == 2
assert binary_search([1, 3, 5, 7, 9], 4) == -1
assert kadane([-2, 1, -3, 4, -1, 2, 1, -5, 4]) == 6
Edge
# Binary search edges
assert binary_search([], 1) == -1
assert binary_search([5], 5) == 0
assert binary_search([5], 4) == -1
assert binary_search([1, 1, 1, 1], 1) in (0, 1, 2, 3) # any valid index
# Kadane edges
assert kadane([5]) == 5
assert kadane([-3, -1, -2]) == -1
assert kadane([1, 2, 3, 4]) == 10 # all positive
assert kadane([-1, -2, -3, -4]) == -1 # all negative
Large
import random
random.seed(0)
big = sorted(random.sample(range(10**6), 10**5))
assert binary_search(big, big[50000]) == 50000
big2 = [random.randint(-10**6, 10**6) for _ in range(10**5)]
result = kadane(big2)
assert isinstance(result, int)
Randomized verifier
def brute_kadane(a):
return max(sum(a[i:j]) for i in range(len(a)) for j in range(i+1, len(a)+1))
for _ in range(200):
a = [random.randint(-50, 50) for _ in range(random.randint(1, 30))]
assert kadane(a) == brute_kadane(a)
Invariant assertions (the proof, in code)
def binary_search_with_assertions(a, t):
lo, hi = 0, len(a)
while lo < hi:
# INVARIANT: if t in a, then t at some index in [lo, hi)
if t in a[:lo]: assert False, "invariant violated (t before lo)"
if lo > 0 and a[lo-1] >= t and t in a: assert False
mid = lo + (hi - lo) // 2
if a[mid] == t: return mid
elif a[mid] < t: lo = mid + 1
else: hi = mid
return -1
Follow-up Questions
- Find leftmost vs rightmost occurrence of
t. Modify binary search; the invariant becomes “the answer is in[lo, hi)” rather than “if t is present, it’s in…”. - Binary search on real numbers. Replace integer halving with floating-point; the invariant is the same but termination uses a precision threshold, not
lo < hi. - Kadane with at most K negative numbers allowed. State expands to
(i, k); DP, O(NK). - Maximum sum circular subarray. Two passes of Kadane + total-sum trick; the invariant for the circular case is more subtle.
- Maximum product subarray. Maintain both max and min products at each index because a negative * negative becomes the largest.
Product Extension
- Database B-tree page searches — binary search within a page; the invariant analysis directly applies.
- Time-series anomaly detection — Kadane variants find the largest cumulative deviation, used in change-point detection.
- Streaming Kadane — given a stream of metrics, find the worst-degradation window. Same algorithm with O(1) memory.
Language/Runtime Follow-ups
- Python:
mid = (lo + hi) // 2is safe (arbitrary precision);mid = lo + (hi - lo) // 2is still preferable for portability. - Java/C++:
(lo + hi) / 2overflows whenlo + hi > 2^31 - 1. Uselo + (hi - lo) / 2or(lo + hi) >>> 1(Java unsigned shift). - Kadane overflow: for
|a[i]| ≤ 10^9and N = 10^5, max sum is 10^14 — exceeds 32-bit int. Uselong/int64in Java/Go/C++. - Floating-point Kadane: accumulation error compounds; use Kahan summation if precision matters.
Common Bugs
(lo + hi) // 2overflow in C++/Java- Closed-interval binary search with
lo <= hiis correct but themidupdates are trickier; pick a convention and stick with it - Kadane initialized to 0 — fails on all-negative arrays
- Forgetting
best_overallupdate — returns the best ending at the last position, not overall - Empty input to Kadane — undefined; problem statement says non-empty, but check the contract
- Binary search infinite loop when
lo = midinstead oflo = mid + 1— the monovariant doesn’t decrease
Debugging Strategy
When binary search loops forever or returns wrong index:
- Print
(lo, mid, hi)each iteration. Iflodoesn’t strictly increase orhidoesn’t strictly decrease (toward each other), you have an off-by-one. - Check the boundary condition: does your invariant include or exclude
hi? - For “find leftmost”, the answer is at
loafter the loop, notmid.
When Kadane returns 0 on all-negative input:
- Check initialization — should be
a[0], not0. - Print
(best_here, best_overall)at each step; trace by hand against the expected.
Mastery Criteria
- Wrote both algorithms correctly without testing first (proof-first coding)
- Stated the loop invariant for each in one sentence
- Identified the monovariant for binary search termination
- Proved correctness by induction (3 steps: init, maintain, terminate)
-
Recognized the
(lo + hi) / 2overflow risk without prompting -
Explained why Kadane’s init must be
a[0]and not0 - Wrote loop invariants as comments in your code for one Phase 5 DP problem
Lab 05 — Stress Testing Harness (Two-Pointer Variants)
Goal
Build a reusable stress-testing harness: a randomized input generator + a known-correct brute-force verifier + a diff loop that finds the smallest failing input. This is the single most valuable debugging tool in competitive programming, and it is shockingly underused in interview prep. After this lab you should reach for the harness automatically whenever your solution passes the given examples but you don’t trust it.
Background Concepts
A stress test has three components:
- Generator (
gen) — produces random valid inputs, parameterized by size and seed for reproducibility. - Brute force (
brute) — a known-correct slow solution. Often O(N²) or O(2^N), valid only for tiny N. - Fast solution (
fast) — the optimized solution you’re testing.
The harness loops: generate input → run both → compare. On mismatch, print the input and both outputs and stop. Then shrink the failing input to the smallest case that still fails — this is what makes debugging fast.
Why this works: brute force is correct by construction (it tries everything). Any discrepancy is your bug, not the brute force’s. Random testing covers cases you didn’t think of.
Why people don’t use it: they think writing brute force is “wasted time.” It is not; in interview prep, the brute force is also your starting point for the optimization conversation with the interviewer.
Interview Context
Stress testing rarely happens during a 45-min interview, but the practice habit shows up in your interview performance:
- You instantly know how to write the brute force (which the interviewer always wants you to articulate first).
- You catch bugs in practice that would otherwise be silently learned-wrong, then deployed mid-interview.
- You build pattern recognition for “this kind of two-pointer has off-by-one risk” because you’ve seen the harness flag them.
At competitive companies (Jane Street, Hudson River, Citadel) and at top-tier interviews (Google L6+), interviewers will sometimes ask “how would you verify this is correct beyond running the examples?” The answer is the stress harness.
Problem Statement
Implement and stress-test three two-pointer problems known to have subtle off-by-one bugs:
A. Two Sum II (sorted array). Given a sorted array a and target t, return indices (i, j) with i < j and a[i] + a[j] == t, or (-1, -1) if no such pair exists.
B. Container With Most Water (LC 11). Given heights h, find indices (i, j) maximizing (j - i) * min(h[i], h[j]).
C. 3Sum (LC 15). Given nums, return all unique triples (a, b, c) with a + b + c == 0. Each triple sorted ascending; output deduplicated.
For each, write the fast solution, write the brute force, and write the stress harness. Run for ≥1000 random trials.
Constraints
- |a| ≤ 1000 for stress; ≤ 10^5 for the real fast solution
- -1000 ≤ a[i] ≤ 1000
Clarifying Questions
(These problems are standard; the lab is about the harness, not disambiguation.)
Examples
two_sum_sorted([1, 3, 4, 5, 7], 9) → (1, 3) # 3 + 5 = 8? no — actually (2, 3) since 4+5=9
container([1, 8, 6, 2, 5, 4, 8, 3, 7]) → 49 # i=1 (h=8), j=8 (h=7), width=7
3sum([-1, 0, 1, 2, -1, -4]) → [[-1, -1, 2], [-1, 0, 1]]
Initial Brute Force
Two Sum: O(N²) double loop. Trivially correct. Container: O(N²) double loop, take max. 3Sum: O(N³) triple loop; collect, sort each triple, deduplicate via set of tuples.
Brute Force Complexity
O(N²), O(N²), O(N³). All valid for N ≤ 100 in <1 sec.
Optimization Path
All three are classic two-pointer problems. After sorting (for 3Sum and Two Sum), pointers move from both ends inward based on the comparison.
Final Expected Approach
Fast solutions
def two_sum_sorted(a, t):
i, j = 0, len(a) - 1
while i < j:
s = a[i] + a[j]
if s == t: return (i, j)
elif s < t: i += 1
else: j -= 1
return (-1, -1)
def container(h):
i, j = 0, len(h) - 1
best = 0
while i < j:
best = max(best, (j - i) * min(h[i], h[j]))
if h[i] < h[j]: i += 1
else: j -= 1
return best
def three_sum(nums):
nums = sorted(nums)
n = len(nums)
res = []
for i in range(n - 2):
if i > 0 and nums[i] == nums[i-1]: continue # skip dup anchor
j, k = i + 1, n - 1
while j < k:
s = nums[i] + nums[j] + nums[k]
if s == 0:
res.append([nums[i], nums[j], nums[k]])
j += 1; k -= 1
while j < k and nums[j] == nums[j-1]: j += 1 # skip dup j
while j < k and nums[k] == nums[k+1]: k -= 1 # skip dup k
elif s < 0: j += 1
else: k -= 1
return res
Brute forces
def brute_two_sum(a, t):
for i in range(len(a)):
for j in range(i+1, len(a)):
if a[i] + a[j] == t: return (i, j)
return (-1, -1)
def brute_container(h):
best = 0
for i in range(len(h)):
for j in range(i+1, len(h)):
best = max(best, (j - i) * min(h[i], h[j]))
return best
def brute_3sum(nums):
n = len(nums)
found = set()
for i in range(n):
for j in range(i+1, n):
for k in range(j+1, n):
if nums[i] + nums[j] + nums[k] == 0:
found.add(tuple(sorted([nums[i], nums[j], nums[k]])))
return sorted([list(t) for t in found])
The Stress Harness
import random
def stress(gen, brute, fast, normalize, trials=2000, seed=0):
random.seed(seed)
for t in range(trials):
inp = gen()
b = normalize(brute(*inp))
f = normalize(fast(*inp))
if b != f:
print(f"FAIL on trial {t}")
print(f" input: {inp}")
print(f" brute: {b}")
print(f" fast: {f}")
# Shrink: try to find a smaller failing input
shrunk = shrink_input(inp, brute, fast, normalize)
print(f" smallest failing input: {shrunk}")
return False
print(f"PASS {trials} trials")
return True
def shrink_input(inp, brute, fast, normalize):
"""Greedy shrink — drop elements one at a time, keep if still fails."""
arr, *rest = inp
current = list(arr)
changed = True
while changed:
changed = False
for i in range(len(current)):
candidate = current[:i] + current[i+1:]
if len(candidate) < 2: continue
try:
if normalize(brute(candidate, *rest)) != normalize(fast(candidate, *rest)):
current = candidate; changed = True; break
except Exception:
continue
return (current, *rest)
# Generators
def gen_two_sum():
n = random.randint(2, 20)
a = sorted(random.randint(-30, 30) for _ in range(n))
t = random.randint(-60, 60)
return (a, t)
def gen_container():
n = random.randint(2, 30)
return ([random.randint(0, 20) for _ in range(n)],)
def gen_3sum():
n = random.randint(3, 15)
return ([random.randint(-10, 10) for _ in range(n)],)
# Normalizers (canonicalize output before comparison)
def norm_two_sum(r):
# Both -1, -1 OR a valid pair; for the pair, the sum is what matters, not index
if r == (-1, -1): return None
return "found" # we only care that one was found; index may differ
# NOTE: If indices must match exactly, change the brute force to scan in two-pointer order
def norm_container(x): return x # int, direct compare
def norm_3sum(triples): return sorted([sorted(t) for t in triples])
# Run
stress(gen_two_sum, brute_two_sum, two_sum_sorted, norm_two_sum)
stress(gen_container, brute_container, container, norm_container)
stress(gen_3sum, brute_3sum, three_sum, norm_3sum)
Data Structures Used
- Lists of integers
- Set of tuples for 3Sum deduplication in brute force
- A small library of helper functions (
gen,brute,fast,normalize,stress,shrink_input) that you reuse across problems
Correctness Argument
The brute force is correct because it enumerates all valid candidates (O(N^k) for k-sum). Any output from the fast solution that disagrees with the brute is a bug in the fast solution. Random sampling over thousands of trials gives high confidence (though not certainty) that the fast is correct; the smaller the input space (e.g., values in [-10, 10] with N ≤ 15), the more confident.
Complexity
Harness adds zero asymptotic cost — the fast solution’s complexity is unchanged. Each trial costs O(brute) which is the bottleneck; with N ≤ 30 it runs ~2000 trials in <2 seconds.
Implementation Requirements
- Use
random.seed()for reproducibility — failures must be re-runnable - Print the failing input before the outputs, so you can copy-paste and re-run
- Always normalize outputs before comparison (canonical sort order, etc.)
- Implement shrinking — a 20-element failure is hard to debug; a 4-element failure is obvious
Tests
The harness itself, as a test
# Sanity: plant a bug in the fast solution and verify the harness catches it
def buggy_two_sum(a, t):
i, j = 0, len(a) - 1
while i < j:
s = a[i] + a[j]
if s == t: return (i, j)
elif s < t: i += 1
else: j -= 1
return (0, 0) # BUG: should return (-1, -1)
assert stress(gen_two_sum, brute_two_sum, buggy_two_sum, norm_two_sum, trials=500) is False
Pass-through on the correct solutions
assert stress(gen_two_sum, brute_two_sum, two_sum_sorted, norm_two_sum, trials=2000) is True
assert stress(gen_container, brute_container, container, norm_container, trials=2000) is True
assert stress(gen_3sum, brute_3sum, three_sum, norm_3sum, trials=2000) is True
Edge generators
Add specialized generators that stress edge cases:
def gen_two_sum_edge():
"""Heavy on duplicates and boundary targets."""
n = random.randint(2, 10)
a = sorted([random.choice([-1, 0, 1]) for _ in range(n)])
t = random.choice([-2, 0, 2])
return (a, t)
Follow-up Questions
- Generator-based testing (Hypothesis library). Python’s
hypothesislibrary generates inputs and shrinks them automatically. Show how to convert the harness into Hypothesis strategies. - Detecting performance regressions. Add timing to the harness; flag when fast > 10× the previous run on the same seed.
- Coverage-guided fuzzing. Use
atherisor similar to mutate inputs that increase code coverage; finds rarer bugs than purely random. - Concurrent stress testing. Run brute and fast on different threads; useful for testing thread-safe versions.
- What if there’s no brute force? Then write two independent fast solutions (different algorithms) and stress them against each other. Common for geometry problems.
Product Extension
- CI-integrated fuzzing. Google’s OSS-Fuzz runs continuous random testing on open-source projects; finds thousands of bugs annually.
- Property-based testing in production. Stripe, Jane Street, Klarna use property tests to validate financial logic where the brute force is “the spec.”
- Differential testing. Compare two implementations of the same protocol (e.g., two JSON parsers) on random inputs to find spec ambiguities.
Language/Runtime Follow-ups
- Python:
hypothesisis the gold standard for property-based testing.random.seed()is per-thread; for parallel stress, use independentrandom.Randominstances. - Java:
jqwikorjunit-quickcheckfor property-based; JMH for performance regression detection. - Go: Built-in
testing/quickand (Go 1.18+) native fuzzing withgo test -fuzz. - C++:
rapidcheck(QuickCheck-style); LLVM’s libFuzzer for coverage-guided. - Rust:
proptestandquickcheckcrates; nativecargo fuzz.
Common Bugs
- Non-reproducible failures — forgot
random.seed(); can’t re-run the failing case. - Output comparison fails due to ordering — set vs list, dict iteration order; always normalize.
- Brute force itself is buggy — verify the brute on the given problem examples first.
- Generator produces invalid inputs — e.g., for Two Sum Sorted, the generator must produce a sorted array. Verify with
assert all(a[i] <= a[i+1] for i in range(len(a)-1)). - Shrinker breaks the input invariant — for Two Sum Sorted, dropping an element keeps the array sorted; but for a tree-structured input, dropping a node may break invariants. Custom shrinkers per problem.
Debugging Strategy
When the harness reports a failure:
- Read the smallest failing input that the shrinker produced. If it’s ≤ 5 elements, trace by hand.
- Run only the fast solution with prints on that small input. Compare to expected.
- The bug is almost always in a boundary condition — empty input, single element, all duplicates, exact-target match.
- If the bug only appears with duplicates, suspect your dedup logic (3Sum is famous for this).
- If the bug only appears with negatives, suspect signed comparisons or
abs()misuse.
Mastery Criteria
- Built the stress harness in under 20 minutes for the three target problems
- Caught at least one bug by planting one and verifying the harness flagged it
- Wrote a shrinker that reduces failures to ≤ 10 elements
- Ran 2000+ trials per problem with no failure
- Built a reusable harness module you can drop into any future problem
- Applied the harness to one Phase 2 or Phase 5 problem and either confirmed correctness or found a bug
- Can explain why random testing complements (not replaces) edge-case enumeration
Lab 06 — Performance Profiling (Three LIS Implementations)
Goal
Measure and compare three implementations of Longest Increasing Subsequence: O(N²) DP, O(N log N) patience sort, and a poorly-written “O(N log N)” with hidden O(N) inside the inner loop. Use profiling tools to detect the discrepancy between claimed complexity and actual behavior. By the end you should never again submit a solution thinking “this should be fast enough” without measuring.
Background Concepts
Empirical complexity verification: if a function is O(f(N)), then running it at N and 2N should produce runtimes that scale as f(2N) / f(N). For O(N): 2×. For O(N log N): ~2.1×. For O(N²): 4×. For O(N³): 8×. Measure the ratio; mismatch reveals the bug.
Profiling tools:
- Python:
cProfilefor function-level timing;line_profilerfor line-by-line;py-spyfor sampling without code changes;tracemallocfor memory. - Java:
async-profilerfor low-overhead sampling; JFR (Java Flight Recorder); JMH for microbenchmarks; jmap for heap snapshots. - Go:
pprof(CPU and heap);runtime/tracefor goroutine scheduling;go test -benchwith-benchmem. - C++:
perf(Linux) with flamegraphs; Valgrind/Callgrind for instruction counts; gperftools. - Node.js: built-in
--inspect+ Chrome DevTools;clinic.jsfor higher-level analysis.
Common deceptions:
- A “constant time” operation that’s actually O(N) (e.g.,
list.insert(0, x)in Python,s += cin a loop in Java). - Hash collisions in adversarial input turning O(1) lookups into O(N).
- Garbage collector pauses inflating measurements.
- JIT warmup masking the cold-start cost.
Interview Context
Interviewers ask “what’s the complexity?” on every problem. They sometimes follow up with “are you sure?” — and the right answer is “I claim O(N log N); I can verify by running at doubling sizes if you’d like.” Candidates who can articulate empirical verification (even without running it) signal a different level of rigor.
Practical-engineering interviews (Phase 8) often include “make this faster” or “profile this for me” as a follow-up. You need fluency with at least one language’s profiler.
Problem Statement
Implement three versions of LIS:
Version A (O(N²) DP): dp[i] = length of LIS ending at index i. Transition: dp[i] = max(dp[j] + 1) for all j < i with a[j] < a[i].
Version B (O(N log N) patience sort): Maintain tails[k] = smallest tail value of any increasing subsequence of length k+1. For each element, binary-search-and-replace; if no replacement, append.
Version C (“fake” O(N log N)): Same as B, but uses Python list with index() (O(N)) to find the replacement position instead of binary search. Looks like O(N log N) at a glance; actually O(N²).
Measure runtime at N = 1000, 2000, 4000, 8000, 16000. Verify the doubling ratios. Profile to find the bottleneck in Version C.
Constraints
- 1 ≤ N ≤ 16000 for measurement
- Values: random integers in
[0, 10^6]
Clarifying Questions
(LIS is standard; the lab is about measurement.)
Examples
LIS([10, 9, 2, 5, 3, 7, 101, 18]) == 4 (e.g., [2, 5, 7, 101] or [2, 3, 7, 18])
LIS([0, 1, 0, 3, 2, 3]) == 4
LIS([7, 7, 7, 7]) == 1
Initial Brute Force
Enumerate all 2^N subsequences, check each. O(2^N · N).
Brute Force Complexity
O(2^N · N). Valid only for N ≤ 20. Useful as the verifier in your stress harness from Lab 5.
Optimization Path
DP from O(2^N) → O(N²) → O(N log N).
Final Expected Approach
import bisect
import time
import random
# Version A: O(N^2) DP
def lis_dp(a):
if not a: return 0
n = len(a)
dp = [1] * n
for i in range(1, n):
for j in range(i):
if a[j] < a[i]:
dp[i] = max(dp[i], dp[j] + 1)
return max(dp)
# Version B: O(N log N) — real
def lis_patience(a):
tails = []
for x in a:
idx = bisect.bisect_left(tails, x)
if idx == len(tails):
tails.append(x)
else:
tails[idx] = x
return len(tails)
# Version C: "fake" O(N log N) — uses linear search disguised
def lis_fake(a):
tails = []
for x in a:
# Linear scan to find first tail >= x — O(N)!
idx = None
for i, t in enumerate(tails):
if t >= x:
idx = i; break
if idx is None:
tails.append(x)
else:
tails[idx] = x
return len(tails)
Measurement Harness
def benchmark(fn, n, trials=3):
random.seed(n)
times = []
for _ in range(trials):
a = [random.randint(0, 10**6) for _ in range(n)]
start = time.perf_counter()
fn(a)
times.append(time.perf_counter() - start)
return min(times) # min is most reliable; mean is noisy
def doubling_test(fn, name, sizes=[1000, 2000, 4000, 8000, 16000]):
prev = None
print(f"\n{name}")
print(f"{'N':>8} {'time (s)':>12} {'ratio':>8}")
for n in sizes:
t = benchmark(fn, n)
ratio = f"{t/prev:.2f}x" if prev else "—"
print(f"{n:>8} {t:>12.4f} {ratio:>8}")
prev = t
doubling_test(lis_dp, "Version A — O(N^2) DP")
doubling_test(lis_patience, "Version B — O(N log N) patience")
doubling_test(lis_fake, "Version C — fake O(N log N)")
Expected Output (approximate, on modern laptop)
Version A — O(N^2) DP
N time (s) ratio
1000 0.0420 —
2000 0.1680 4.00x
4000 0.6720 4.00x
8000 2.6900 4.00x
16000 10.7600 4.00x
Version B — O(N log N) patience
N time (s) ratio
1000 0.0008 —
2000 0.0017 2.13x
4000 0.0036 2.12x
8000 0.0076 2.11x
16000 0.0160 2.10x
Version C — fake O(N log N)
N time (s) ratio
1000 0.0210 —
2000 0.0840 4.00x ← 4x means O(N^2), not O(N log N)!
4000 0.3360 4.00x
8000 1.3440 4.00x
16000 5.3760 4.00x
Reading the ratios: if you claimed O(N log N) and see 4× per doubling, your algorithm is actually O(N²). The doubling test is the cheapest, most reliable complexity-verifier in your toolbox.
Profiling Version C
import cProfile, pstats
random.seed(0)
a = [random.randint(0, 10**6) for _ in range(8000)]
pr = cProfile.Profile()
pr.enable()
lis_fake(a)
pr.disable()
pstats.Stats(pr).sort_stats('cumulative').print_stats(10)
Expected output: the lis_fake function dominates; the inner for i, t in enumerate(tails) is the hot spot. With line_profiler:
$ pip install line_profiler
$ kernprof -l -v script.py
# Add @profile decorator to lis_fake first
Output will show the inner loop accounts for ~95% of the time per call, confirming the linear-search bottleneck.
Data Structures Used
- Plain list (Python) — supports
bisectfor true O(log N) search - Profiler outputs (text/HTML/flamegraph depending on tool)
Correctness Argument
All three versions produce the same output on random input. The patience sort correctness: tails[k] always stores the smallest possible tail of a length-(k+1) LIS seen so far. When a new element is smaller than tails[k], replacing improves the future extensibility; when it’s larger than all tails, it extends to a new length. Final length is len(tails). Inductive proof omitted; see CLRS / standard references.
Complexity
- A: O(N²) time, O(N) space.
- B: O(N log N) time, O(N) space.
- C: claimed O(N log N), actually O(N²) due to linear inner search.
Implementation Requirements
- Use
time.perf_counter(), nottime.time()— higher resolution, monotonic. - Take the
minof multiple trials, not the mean —minrejects GC/scheduler noise. - Warm up before timing if testing JIT’d languages (Java, JS).
- Use
__slots__and pre-allocated arrays in hot Python paths.
Tests
Smoke
assert lis_dp([10, 9, 2, 5, 3, 7, 101, 18]) == 4
assert lis_patience([10, 9, 2, 5, 3, 7, 101, 18]) == 4
assert lis_fake([10, 9, 2, 5, 3, 7, 101, 18]) == 4
Edge
for fn in (lis_dp, lis_patience, lis_fake):
assert fn([]) == 0
assert fn([42]) == 1
assert fn([1, 2, 3, 4, 5]) == 5 # already sorted
assert fn([5, 4, 3, 2, 1]) == 1 # reverse sorted
assert fn([7, 7, 7, 7]) == 1 # all duplicates
Performance assertions
# Verify Version B's doubling ratio
times = [benchmark(lis_patience, n) for n in (1000, 2000, 4000)]
ratios = [times[1]/times[0], times[2]/times[1]]
for r in ratios:
assert 1.8 < r < 2.5, f"Version B ratio {r:.2f} not in O(N log N) range"
# Verify Version C is actually quadratic (this assertion *should* pass — proving the bug)
times = [benchmark(lis_fake, n) for n in (1000, 2000, 4000)]
ratios = [times[1]/times[0], times[2]/times[1]]
for r in ratios:
assert 3.5 < r < 4.5, f"Version C ratio {r:.2f} not in O(N^2) range — bug may be fixed?"
Randomized verifier (A == B == C)
random.seed(0)
for _ in range(50):
a = [random.randint(0, 100) for _ in range(random.randint(1, 50))]
assert lis_dp(a) == lis_patience(a) == lis_fake(a)
Follow-up Questions
- What if the input is nearly sorted? Version A’s actual runtime degrades less; profile to confirm.
- Memory profile: which version allocates most? Use
tracemalloc.start()andtracemalloc.get_traced_memory(). - Reconstruct the LIS, not just its length. Track parent pointers; doesn’t change asymptotic complexity but doubles space.
- LIS in a stream (one pass, can’t store everything). Use patience sort with a fixed buffer; gives approximate answer.
- Parallelize LIS. Hard — DP dependencies are sequential. Pipeline by chunks; merge with care.
Product Extension
- Code review of “this should be fast” claims — every senior engineer learns to verify before trusting.
- Database query planners — the planner estimates I/O cost; profiling validates the estimate against real query times.
- CDN cache eviction policies — when comparing LRU vs LFU vs SLRU under real traffic, microbenchmarks lie; full profiles win.
- Production hot-path detection — flamegraphs reveal that 80% of CPU is spent in 3% of the code; optimize there.
Language/Runtime Follow-ups
- Python:
time.perf_counter()is the right clock.cProfileoverhead is ~30%; for tight loops useline_profilerorpy-spy(sampling, ~0% overhead). The GIL means CPU profiling is mostly sequential; for asyncio code useaiomonitororasynciodebug mode. - Java: JMH (Java Microbenchmark Harness) handles JIT warmup, dead-code elimination, and constant folding correctly — handwritten timing loops in Java are often wrong. Use
-prof gcto see allocation cost. - Go:
go test -bench=. -benchmemis the standard.-cpuprofileand-memprofilewrite pprof files; visualize withgo tool pprof. - C++:
perf record+perf reportfor sampling;perf statfor cache misses and IPC. Use-O2or-O3for measurement; debug builds have very different performance. - Node.js: V8’s
--profflag dumps tick logs;--inspectfor Chrome DevTools. Beware turbofan optimization — code that runs cold for the first 10K iterations is suddenly 10× faster after JIT.
Common Bugs
- Timing the cold start — first call includes import/parse/JIT warmup; throw it out.
- Using
time.time()— wall clock; affected by NTP, sleep, system load. - Mean over trials — one stop-the-world GC pause skews the mean; use min instead.
- Measuring with assertions on — Python
-Oflag strips asserts; default mode keeps them, slowing hot loops. - Forgetting
random.seed()— runs aren’t reproducible. - Comparing implementations on different input distributions — random LIS, sorted LIS, and reversed LIS have wildly different runtimes for some algorithms.
Debugging Strategy
When complexity doesn’t match your claim:
- Verify with the doubling test. This is the only way.
- Profile to find the hot function. It’s almost always one inner loop.
- Read the standard library docs for any “built-in” operation you used —
list.insert,dict.update,str +=,Vector.addmay not be what you think. - Check for hidden quadratic behavior in concatenation:
result = result + small_thingin a loop is the classic Java/Python beginner trap. - Verify memory with
tracemalloc— sometimes the “slow” is actually paging, not CPU.
Mastery Criteria
- Implemented all three LIS versions
- Ran the doubling test and observed the 4× / 2.1× / 4× ratios
- Profiled Version C with cProfile (or equivalent) and identified the linear-search bottleneck
- Wrote performance assertions that would catch a regression
- Can recite the expected doubling ratio for O(N), O(N log N), O(N²), O(N³)
- Applied profiling to one of your own past solutions and identified one inefficiency
- Familiarity with at least one profiler in your primary language (output format, common flags, how to interpret)
Phase 11 — Mock Interview Mastery
Target level: All (mock difficulty scales from beginner through staff/principal/competitive) Expected duration: 4–8 weeks (depending on your overall track; mocks are continuous) Weekly cadence: 2 mocks minimum, 3+ if interviews are within 4 weeks
Why This Phase Exists
Phases 0–10 trained you to solve problems. This phase trains you to interview. Those are different skills. You can know every algorithm, pass every Phase 1–9 lab, write proofs from Phase 10 — and still fail a real interview because:
- You panic and forget the obvious under real-time pressure.
- You waste 8 minutes on clarifying questions a senior would resolve in 60 seconds.
- You code correctly but communicate nothing — the interviewer can’t tell if you’re thinking or stuck.
- You optimize prematurely before understanding the problem.
- You miss the follow-ups that separate “competent” from “hireable at level.”
- You finish in 25 minutes and have nothing to say when asked the extension question.
- You write buggy code, then spend the remaining time debugging instead of explaining.
A mock interview is the nearest equivalent to the real event without the stakes. Your job: complete at least 12 mocks (one per level), identify your failure mode per level, drill it until it stops happening.
The candidates who pass the hard rounds are not the ones who know the most algorithms. They are the ones who have rehearsed the performance enough times that the algorithm is almost a side effect of a clean interview.
How to Run a Mock
Alone (self-timed)
- Read the problem statement only. Do not peek at hints, examples, follow-ups.
- Set a timer to the mock’s exact allocated time.
- Open a blank document (Google Doc, plain text, paper). No IDE, no autocomplete, no syntax highlighting. The real interview is in a shared Google Doc or CoderPad with minimal tooling.
- Narrate aloud or write notes continuously. If you go silent for >30 seconds, stop, write down what you’re thinking, then proceed.
- Write pseudocode first. If you have >20 minutes left after pseudocode, translate to real code. If less, stay in pseudocode and be very clear about logic.
- When time expires, STOP. This is a time-management test, not a coding-speed test.
- Self-evaluate against the 14-dimension rubric below. Score honestly. If your “Optimization” claim was O(N log N) but you wrote O(N²), that’s a 1, not a 4.
- Do not look at the official solution until after a second self-mock at the same level. One failure teaches a fact; two failures teach the pattern.
With a partner (realistic)
- Find a peer, ideally one level above you.
- They read the problem statement to you. You ask clarifying questions; they answer in character.
- They watch silently as you solve. They give hints only if you explicitly request one (with a score penalty) or after 10+ minutes of being stuck.
- After the timer expires, they rate you on the 14 dimensions, then you debrief.
- Swap roles next session.
Best of both worlds
- Pramp, interviewing.io, Hello Interview (and similar) match you with strangers. Higher pressure, more realistic.
- Record yourself (audio + screen). Replay 24h later. You will be shocked at what you actually said vs what you remember.
The 14-Dimension Scoring Rubric
Every mock is scored 1–5 on each dimension. Total /70. Passing thresholds vary by mock level (see each mock file).
1. Problem Understanding
- 1: Misread the problem; solved the wrong thing.
- 2: Understood the surface; missed a subtle constraint.
- 3: Understood correctly, restated to interviewer.
- 4: Restated, identified the underlying category (graph, DP, greedy) within first 2 minutes.
- 5: Restated, identified category, and explicitly verified your interpretation with one well-chosen example.
2. Clarifying Questions
- 1: None asked; assumed everything.
- 2: One generic question (“can the input be empty?”).
- 3: 2–3 questions covering input bounds and edge cases.
- 4: 3–5 questions covering bounds, edge cases, output format, ambiguity resolution.
- 5: Surgical questions that probe the exact ambiguities of this problem (e.g., for LRU: “does put-on-existing count as a use?”).
3. Brute Force
- 1: No brute force articulated; jumped to optimization.
- 2: Mentioned brute force in passing.
- 3: Stated brute force with complexity; moved on.
- 4: Stated brute force, complexity, and why it fails the constraint.
- 5: Wrote brute force pseudocode briefly to confirm correctness before optimizing — gives you a verifier.
4. Optimization
- 1: No improvement on brute force.
- 2: Improved by a constant factor.
- 3: Optimal-class solution (e.g., O(N log N) when O(N log N) is optimal).
- 4: Optimal-class solution with the right pattern recognized within first 5 minutes.
- 5: Optimal solution + articulated why the optimization works (the key insight) + considered alternative optimizations and rejected them with reasoning.
5. Correctness
- 1: Solution wrong; doesn’t handle the given examples.
- 2: Handles examples but fails an obvious edge case.
- 3: Handles all standard edge cases.
- 4: Handles edge cases plus 1 non-obvious one (overflow, empty input, all-duplicates).
- 5: Walks through correctness argument using invariant or induction.
6. Complexity Analysis
- 1: Wrong or absent.
- 2: Correct but only stated at the end.
- 3: Correct, articulated during/after coding.
- 4: Correct, plus space complexity, plus identified the bottleneck.
- 5: All of the above + considered amortized analysis or worst-case input that triggers worst-case complexity.
7. Code Quality
- 1: Unreadable; magic numbers; one-letter variables everywhere.
- 2: Works but ugly; copy-paste blocks; unclear naming.
- 3: Readable; reasonable names; small functions.
- 4: Clean structure; helpful comments where non-obvious; good use of standard library.
- 5: Production-quality — clear names, no dead code, idiomatic for the language, would pass a code review.
8. Testing
- 1: Did not test.
- 2: Tested only the given example.
- 3: Tested 1–2 edge cases unprompted.
- 4: Systematic walkthrough of given examples + 2+ deliberately-chosen edges.
- 5: Found and fixed own bug through testing, or explicitly stated which test classes would expose risks.
9. Debugging
- 1: Hit a bug, panicked, never recovered.
- 2: Hit a bug, fixed by trial and error.
- 3: Hit a bug, debugged systematically with prints.
- 4: Hit a bug, hypothesized cause, verified with targeted assertion, fixed.
- 5: Hit a bug; narrated the debug protocol aloud; fixed in under 3 minutes.
10. Communication
- 1: Silent typing.
- 2: Occasional muttering; no clear narrative.
- 3: Explained brute force and optimization out loud.
- 4: Continuous narration of thought process; pauses only to think briefly.
- 5: Narrated, paused at decision points to consider tradeoffs, invited interviewer input at appropriate moments.
11. Handling Follow-ups
- 1: Could not answer follow-ups.
- 2: Answered partially.
- 3: Answered correctly with one prompt.
- 4: Answered correctly without prompt; proposed reasonable extensions.
- 5: Answered, anticipated the follow-up before it was asked, and proposed extensions.
12. Language/Runtime Knowledge
- 1: Made language errors (Python integer-divides where float was needed; Java auto-boxing trap).
- 2: No errors but no runtime awareness.
- 3: Used appropriate language features (Python
Counter, JavaMap.Entry). - 4: Articulated runtime cost (Python
list.insert(0, ...)is O(N); JavaString +=is O(N²) in a loop). - 5: Discussed GC/memory model/concurrency implications when relevant.
13. Tradeoff Reasoning
- 1: Picked one approach with no comparison.
- 2: Mentioned one alternative.
- 3: Compared two alternatives with a stated reason.
- 4: Compared 2–3 alternatives across time/space/code complexity axes.
- 5: Articulated which alternative would be preferred under different constraints (small N vs large N, read-heavy vs write-heavy, latency vs throughput).
14. Production Awareness
- 1: None — solved as an algorithm puzzle.
- 2: Mentioned scaling in passing.
- 3: Articulated 1–2 production concerns (latency, persistence, concurrency).
- 4: Articulated multiple production concerns; explained how implementation would change.
- 5: Discussed monitoring, failure modes, backward compatibility, deployment strategy — staff-level signal.
Passing Thresholds by Mock Level
| Mock | Target average score | Total minimum (/70) |
|---|---|---|
| 01 — Beginner | 2.5 | 35 |
| 02 — Easy LeetCode | 3.0 | 42 |
| 03 — Medium LeetCode | 3.0 | 42 |
| 04 — Hard LeetCode | 3.2 | 45 |
| 05 — Big Tech phone screen | 3.3 | 46 |
| 06 — Big Tech onsite | 3.5 | 49 |
| 07 — Senior engineer | 3.8 | 53 |
| 08 — Staff practical | 4.0 | 56 |
| 09 — Runtime/language | 3.8 | 53 |
| 10 — Infrastructure/backend | 4.0 | 56 |
| 11 — Concurrency | 4.0 | 56 |
| 12 — Competitive style | 3.5 | 49 |
Notes:
- Production-aware dimensions (#13, #14) are weighted higher for mocks 07–10.
- Communication (#10) is the most common reason candidates fail; if your average for #10 is below 3.5, drill it specifically.
- Mock 12 (competitive) deprioritizes #11–#14 (no follow-ups; pure algorithm).
Common Failure Modes by Level
| Level | Most common failure |
|---|---|
| Beginner | Silent coding; no communication |
| Easy LC | Forgot edge cases (empty, single element) |
| Medium LC | Stuck on optimization; couldn’t find the pattern |
| Hard LC | Panicked when first approach didn’t work |
| Phone screen | Spent too long on clarification; ran out of time |
| Onsite | Solved problem 1, gave up on problem 2 |
| Senior | No tradeoff reasoning; “I’d just use X” without comparing |
| Staff | No production awareness; built a perfect algorithm with no monitoring story |
| Runtime/lang | Couldn’t answer GC / memory model / concurrency probe mid-coding |
| Infrastructure | Treated it like a LeetCode problem instead of a system build |
| Concurrency | Race conditions in submitted code |
| Competitive | Failed to reach the algorithmic insight; brute force only |
How to Schedule Mocks
12-Week Accelerated Track
- Weeks 1–4: Mocks 01–04 (one each)
- Weeks 5–8: Mocks 05–08 (one each)
- Weeks 9–12: Mocks 09–12 (one each) + repeats of the ones you failed
6-Month Serious Track
- Months 1–2: foundations; no mocks yet
- Month 3: Mocks 01–03 (2 per week)
- Month 4: Mocks 04–06 (2 per week)
- Month 5: Mocks 07–10 (3 per week)
- Month 6: Mocks 11–12 + heavy re-mock cycle (4 per week)
12-Month Elite Track
- Months 1–6: foundations + light mocks (mock 01–04, 1 per week)
- Months 7–9: Mocks 05–10, 2 per week
- Months 10–12: Mocks 11–12, ICPC contests, 3 mocks per week + 2 contests per week
How to Self-Evaluate Honestly
The single biggest failure mode is grade inflation. To counter:
- Record the session. Listen back. You will hear all the silent gaps and the muttered “uh, let me think” filler.
- Compare to the rubric word-for-word. “I tested edge cases” is not enough for a 4 unless you actually tested 3+ unprompted edge cases.
- Find someone harsher than you to debrief with. Ideally an engineer one level above your target.
- Track scores over time. A flat line means you’re not improving — change something (new problem domain, harder mock, partner).
- The dimension where you score lowest is your training target. Drill it for two weeks, then re-mock.
Mock Index
| # | Mock | Time | Target Role |
|---|---|---|---|
| 1 | Beginner | 30 min | First-time interviewer / intern |
| 2 | Easy LeetCode | 30 min | Intern → SWE-I |
| 3 | Medium LeetCode | 35 min | SWE-I → SWE-II |
| 4 | Hard LeetCode | 60 min | SWE-II → Senior |
| 5 | Big Tech Phone Screen | 45 min | Any FAANG screen |
| 6 | Big Tech Onsite | 60 min (× 2 problems) | FAANG SWE-II / Senior |
| 7 | Senior Engineer | 60 min | Senior SWE |
| 8 | Staff Practical | 75 min | Staff / Principal |
| 9 | Runtime/Language Deep Dive | 45 min | Senior / Staff |
| 10 | Infrastructure/Backend | 75 min | Backend / Platform |
| 11 | Concurrency Heavy | 60 min | Backend / Systems |
| 12 | Competitive Style | 90 min | Quant / Compiler / ICPC |
What “Pass” Means
Passing a mock is necessary but not sufficient readiness. The full readiness checklist is in READINESS_CHECKLIST.md. The mocks here verify you can perform — not that you can do so consistently. Aim for 3 consecutive passes of any given mock before considering that level handled.
Mock 01 — Beginner
Interview type: First-time mock / warm-up Target role: Intern, new grad, first-ever interview practice Time limit: 30 minutes Format: 1 easy problem Hints policy: Unlimited hints with -1 to score per hint Primary goal: Build the habit loop of clarify → brute force → optimize → code → test. Optimization is not the focus.
What This Mock Tests
This mock exists to break the most common beginner failure mode: silent coding. You will be scored more on whether you communicated than on whether your code is optimal. A correct silent solution scores lower than a slightly buggy spoken one.
The scoring rubric weights as follows:
| Dimension | Weight |
|---|---|
| Communication (#10) | 3× |
| Clarifying questions (#2) | 2× |
| Testing (#8) | 2× |
| Code quality (#7) | 1× |
| Correctness (#5) | 1× |
| Complexity (#6) | 1× |
| All others | 0.5× |
Pick One Problem (interviewer’s choice; for self-mock, pick at random)
Problem A — Reverse a String
Write a function that reverses a string. The input is given as a list of characters s. Modify s in-place; do not allocate a new list.
Examples:
Input: ['h', 'e', 'l', 'l', 'o']
Output: ['o', 'l', 'l', 'e', 'h']
Input: ['H', 'a', 'n', 'n', 'a', 'h']
Output: ['h', 'a', 'n', 'n', 'a', 'H']
Constraints: 1 ≤ |s| ≤ 10^5. Each character is printable ASCII.
Problem B — Valid Parentheses
Given a string s containing only ()[]{}, determine if it is valid. Valid means: brackets close in the right order, every opener has a matching closer of the same type.
Examples:
"()" → True
"()[]{}" → True
"(]" → False
"([)]" → False
"{[]}" → True
"" → True
Constraints: 0 ≤ |s| ≤ 10^4.
Problem C — Find Maximum in Array
Given an array of integers, return the maximum value. Do not use the language’s built-in max().
Examples:
[3, 1, 4, 1, 5, 9, 2, 6] → 9
[-3, -1, -7] → -1
[42] → 42
Constraints: 1 ≤ |a| ≤ 10^5. -10^9 ≤ a[i] ≤ 10^9.
Expected Communication Style
You should:
- Restate the problem in your own words. (“So I need to reverse this list of characters in place, meaning no new list allocation.”)
- Ask 2+ clarifying questions even if they feel obvious. (“Should I handle empty input? What’s the max length?”)
- State 1+ example trace out loud. (“For
[h, e, l, l, o], I’d swap positions 0 and 4, then 1 and 3, leaving 2 alone.”) - Articulate brute force first. Even for these problems — there’s an obvious approach.
- Code while narrating. “I’ll use two pointers, left and right, swap and move toward center until they meet.”
- Test out loud. Walk through the example, then try empty input, then single element.
You should not:
- Type silently for >30 seconds
- Skip clarifying questions because the problem “is obvious”
- Skip testing because the code “looks right”
- Assume the interviewer is following — narrate every decision
Common Failure Modes
- Silent coding. Most common. -3 to communication.
- Skipping clarification. “Empty string?” was not asked → -1.
- No testing. Submitted without walking through. -2 to testing.
- Skipping brute force. Wrote the optimal directly without acknowledging the simpler approach. -1 to brute force.
- Using a built-in. Problem C says no
max()— using it is an instant fail of that problem.
Passing Bar
- Total score: 35/70 (average 2.5)
- Communication dimension: 3+ (mandatory)
- Code: works on all given examples
- One unprompted test case beyond the given examples
If you score 35+ but communication is below 3: re-do this mock. The score is misleading; the failure mode isn’t fixed.
Follow-up Questions (Interviewer may ask)
For A:
- What’s the complexity? (O(N) time, O(1) space.)
- What if the string is in a Unicode encoding with multi-byte characters? (Character iteration is no longer index-1; need to handle codepoints.)
- What if the string is immutable in your language? (Java strings, Python strings — must allocate.)
For B:
- Complexity? (O(N) time, O(N) worst-case space for the stack.)
- What if you also need to return the position of the first invalid bracket? (Track index when pushing; return index on failed pop.)
- What if the input can have non-bracket characters mixed in? (Skip them or treat as invalid — clarify.)
For C:
- Complexity? (O(N) time, O(1) space.)
- What if the array is empty? (Undefined; throw, return None, or return INT_MIN — clarify.)
- What if it’s a stream and you can’t store it all? (Maintain running max with O(1) state.)
Required Tests
For all problems:
- The given examples
- Empty input (where the constraint allows)
- Single-element input
- One additional edge case you choose
For A: a palindrome input (e.g., ['r', 'a', 'c', 'e', 'c', 'a', 'r']).
For B: nested mismatch like "([)]" and just an opener "(" (should be False).
For C: all-negative input and all-identical input.
Required Complexity Explanation
State out loud:
- Time complexity in Big-O
- Space complexity
- Why those bounds are tight (or whether they could be improved)
For these problems, the optimal is also the simplest. Acknowledge that briefly.
Self-Evaluation Template
Copy this into your notes after the mock:
Mock 01 — Beginner
Date: _______
Problem chosen: _______
Time taken: _____ min (limit: 30)
Scores (1–5):
[ ] 1. Problem Understanding
[ ] 2. Clarifying Questions
[ ] 3. Brute Force
[ ] 4. Optimization
[ ] 5. Correctness
[ ] 6. Complexity Analysis
[ ] 7. Code Quality
[ ] 8. Testing
[ ] 9. Debugging (if applicable)
[ ] 10. Communication
[ ] 11. Follow-ups
[ ] 12. Language/Runtime
[ ] 13. Tradeoffs
[ ] 14. Production Awareness
Total: ___/70
What went well:
What went poorly:
Specific bug or moment of confusion:
What to drill before next mock:
What to Do If You Fail
If you scored below 35:
- Identify the dimension with the lowest score.
- Re-do this same mock with a different problem (A/B/C). Focus only on that dimension.
- If communication is the issue, record yourself doing 3 LC easies aloud over the next week. Listen back.
- Do not move to Mock 02 until you pass Mock 01 twice in a row. Foundational habits matter more than progression.
Mock 02 — Easy LeetCode
Interview type: Standard LC easy Target role: Intern, new grad SWE-I, first technical screen Time limit: 30 minutes Format: 1 easy problem Hints policy: One free hint after 10 min of being stuck; additional hints -1 each Primary goal: Solve correctly with clean code and adequate testing in under 30 min.
What This Mock Tests
The bar at the easy level: you should solve the problem with the optimal approach, communicate clearly throughout, and verify with at least 2 unprompted tests. Easy LC questions are the most common phone-screen problems at smaller companies and at the start of FAANG screens.
Scoring weights are uniform across dimensions — easy LCs should pass on every axis.
Pick One Problem
Problem A — Two Sum (LC 1)
Given an array of integers nums and an integer target, return the indices of the two numbers that add up to target. You may assume exactly one solution exists. You may not use the same element twice.
Examples:
nums = [2, 7, 11, 15], target = 9 → [0, 1]
nums = [3, 2, 4], target = 6 → [1, 2]
nums = [3, 3], target = 6 → [0, 1]
Constraints: 2 ≤ |nums| ≤ 10^4. -10^9 ≤ nums[i] ≤ 10^9. Exactly one solution.
Problem B — Best Time to Buy and Sell Stock (LC 121)
You are given prices, where prices[i] is the price of a stock on day i. Maximize profit by choosing one day to buy and a later day to sell. If no profit possible, return 0.
Examples:
[7, 1, 5, 3, 6, 4] → 5 (buy day 1 at 1, sell day 4 at 6)
[7, 6, 4, 3, 1] → 0
[1, 2] → 1
Constraints: 1 ≤ |prices| ≤ 10^5. 0 ≤ prices[i] ≤ 10^4.
Problem C — Contains Duplicate (LC 217)
Given an integer array nums, return True if any value appears at least twice.
Examples:
[1, 2, 3, 1] → True
[1, 2, 3, 4] → False
[1, 1, 1, 3, 3, 4, 3, 2, 4, 2] → True
Constraints: 1 ≤ |nums| ≤ 10^5.
Expected Communication Style
- Restate the problem in 1 sentence.
- Ask 2–3 clarifying questions: input bounds, edge cases (empty? duplicates? negative?), output format.
- State brute force with complexity. (“Nested loop, O(N²).”)
- Identify the optimization signal. (“I see lookup-by-value — hashmap.”)
- State optimal complexity before coding. (“O(N) time, O(N) space.”)
- Code with brief narration of each step.
- Walk through given example. Then 1–2 edge cases.
Solution Sketches
A. Two Sum: hashmap value → index; for each element, check if target - x is in the map; if so return; else insert. O(N) time, O(N) space.
B. Stock: maintain running minimum; for each price, compute price - min_so_far; track max. O(N) time, O(1) space.
C. Duplicate: insert into a set; on collision return True. O(N) time, O(N) space. Or: sort + scan adjacent, O(N log N) time, O(1) extra.
Common Failure Modes
- Wrote the O(N²) brute force as the final answer. Acceptable on first attempt, but you must immediately follow with the optimization.
- Forgot to handle duplicates in Two Sum. What if
nums = [3, 3]? The naive hashmap solution must checktarget - xbefore insertingx. - Used
max(prices) - min(prices)for B. Wrong — the min must come before the max. - No edge case test. Empty arrays, single element, all duplicates.
Passing Bar
- Total score: 42/70 (average 3.0)
- Optimal complexity reached (O(N) or O(N log N) depending on problem)
- At least 2 unprompted edge case tests
- Continuous narration (no silent stretches >30 sec)
Follow-up Questions
For A:
- What if there are multiple valid pairs? Return all. → Need to handle duplicates carefully; either set of tuples or sort + two-pointer.
- What if the array is sorted? → Two-pointer, O(1) extra space.
- What if the input is a stream? → Hashmap still works; can’t know “future” elements.
- What if duplicates matter? →
nums = [3, 3], target = 6→ return [0, 1].
For B:
- Can you buy and sell multiple times? → LC 122; sum all positive day-to-day differences.
- With a cooldown of K days between trades? → DP, state = (day, holding?).
- With a transaction fee? → DP variant.
- What if you can short-sell? → Symmetric; maintain running max.
For C:
- What if memory is constrained? → Bloom filter (approximate); or sort in-place + scan.
- Return the first duplicate, not just whether one exists. → Same approach, return on first collision.
- Find ALL duplicates. → Count map.
- In a stream? → Bloom filter or Count-Min Sketch for approximate.
Required Tests
For all:
- Given examples
- Empty input or smallest legal input
- Single-element input (if applicable)
- Edge case specific to the problem (negative numbers, all-duplicates, sorted input)
Required Complexity Explanation
State:
- Time complexity (with reasoning)
- Space complexity (with reasoning)
- The bottleneck — which line determines the dominant cost
Self-Evaluation Template
Mock 02 — Easy LC
Date: _______
Problem: _______
Time taken: _____ / 30 min
Scores (1–5) for all 14 dimensions:
___ Total /70
Optimal complexity reached? Y/N
Hints used? Number: ___
Edge cases tested unprompted? Number: ___
Strongest dimension:
Weakest dimension:
Action item for next mock:
What to Do If You Fail
- Score 35–41: Re-do with a different problem from the list. Focus on the lowest-scored dimension.
- Score <35: Step back to Mock 01 for one session. The habit loop isn’t solid.
- Failed to reach optimal complexity: Drill Phase 1 (foundations) hash-map and array labs.
- Took longer than 30 min: Repeat with a stricter timer. 30 min is the actual phone-screen budget.
Mock 03 — Medium LeetCode
Interview type: Standard LC medium — the modal interview question type Target role: SWE-I → SWE-II, generic mid-tier phone screen Time limit: 35 minutes Format: 1 medium problem Hints policy: One free hint at 15 min; additional -2 each Primary goal: Pattern recognition under time pressure + clean implementation.
What This Mock Tests
Mediums are the bread and butter of coding interviews. If you can’t pass mediums consistently in 30–35 min, you cannot pass FAANG. The bar is:
- Recognize the pattern (sliding window, hashmap, two pointers, BFS, etc.) within 5 minutes
- State optimal complexity before coding
- Implement correctly with clean code
- Test 3+ cases including 1 non-obvious edge
Scoring weights this mock equally across all dimensions, with slight emphasis on Optimization (#4) and Testing (#8) — these are the differentiators at the medium level.
Pick One Problem
Problem A — Longest Substring Without Repeating Characters (LC 3)
Given a string s, find the length of the longest substring with no repeating characters.
Examples:
"abcabcbb" → 3 ("abc")
"bbbbb" → 1 ("b")
"pwwkew" → 3 ("wke")
"" → 0
Constraints: 0 ≤ |s| ≤ 5×10^4. Printable ASCII or English letters/digits/symbols.
Problem B — Group Anagrams (LC 49)
Given an array of strings, group anagrams together. Return any order.
Examples:
["eat","tea","tan","ate","nat","bat"]
→ [["bat"],["nat","tan"],["ate","eat","tea"]]
[""] → [[""]]
["a"] → [["a"]]
Constraints: 1 ≤ |strs| ≤ 10^4. 0 ≤ |strs[i]| ≤ 100. Lowercase English letters.
Problem C — Coin Change (LC 322)
Given coin denominations coins and an amount, return the fewest number of coins needed. If impossible, return -1. Infinite supply of each coin.
Examples:
coins = [1, 2, 5], amount = 11 → 3 (5 + 5 + 1)
coins = [2], amount = 3 → -1
coins = [1], amount = 0 → 0
Constraints: 1 ≤ |coins| ≤ 12. 1 ≤ coins[i] ≤ 2^31 - 1. 0 ≤ amount ≤ 10^4.
Expected Communication Style
- Restate in your own words; explicitly state the input/output types.
- Ask 3–5 clarifying questions: input bounds, edge cases, output ambiguities.
- State brute force with complexity.
- Name the pattern. (“Sliding window over a hashmap” / “Hash group by sorted string” / “Unbounded knapsack DP.”)
- State optimal complexity before coding.
- Code with narration. Pause briefly at decision points (e.g., “I need to evict from the window — let me use a map of char→last_seen_index instead of a set, so I can jump the left pointer”).
- Test 3+ cases, including 1 designed to break common bugs.
Solution Sketches
A. Longest Substring: sliding window with hashmap (char → last index seen). When duplicate enters, advance left to max(left, last_seen[c] + 1). O(N) time, O(min(N, alphabet)) space.
B. Group Anagrams: key by sorted string, OR key by char-count tuple. Hashmap from key → list of strings. O(N · K log K) for sort key, O(N · K) for count key.
C. Coin Change: DP. dp[i] = min coins for amount i. dp[i] = min(dp[i - c] + 1) for c in coins. O(amount × |coins|) time.
Common Failure Modes
- For A: used a set + slide, but O(N²) worst case. Need the hashmap-with-last-index trick to keep O(N).
- For A: forgot the
max(left, ...)when jumping the left pointer. Causes left to move backwards on out-of-window duplicates. - For B: used sorted string as key but the sort is O(K log K) per word. Acceptable, but count tuple is O(K). Discuss the tradeoff.
- For C: greedy approach (always pick the largest coin). Wrong on
coins = [1, 3, 4], amount = 6: greedy gives 4+1+1 = 3 coins; DP gives 3+3 = 2. - For C: forgot to initialize
dp[0] = 0or to handleamount = 0. - For C: didn’t check if
i - c >= 0before lookup. Causes IndexError or wrong answer.
Passing Bar
- Total score: 42/70 (average 3.0)
- Optimal complexity reached
- Correct on all given examples + at least 1 self-generated edge case
- Hint usage ≤ 1
- Time: ≤ 35 min
Follow-up Questions
For A:
- Return the substring itself, not just length. → Track start position in addition to max length.
- What if “repeating” means within a window of K positions? → Modify the window logic; same approach.
- Unicode? → Use codepoint, not byte; alphabet might be huge so use hashmap, not array.
For B:
- What if strings can be huge (1M chars)? → Hashing the count tuple becomes the bottleneck; consider streaming Rabin-Karp.
- Anagram detection in a stream? → Maintain rolling count.
- Approximate anagrams (with one letter difference)? → Locality-sensitive hashing.
For C:
- Return the actual coin combination, not just the count. → Track parent pointer in DP; reconstruct.
- Number of ways to make change. → Different DP:
dp[i] = sum of dp[i-c]. - What if coin counts are limited? → Bounded knapsack variant; O(amount × sum(counts)).
- Why doesn’t greedy work? → Coin systems where greedy works are “canonical” (e.g., USD coins); others (e.g., [1, 3, 4]) require DP.
Required Tests
For all:
- Given examples
- Empty input (where legal)
- Single-element / minimum input
- Input that triggers the pattern’s worst case (e.g., for A:
"aaaaa"; for C: amount=0 and impossible amount) - A randomly chosen non-trivial case you verify by hand
Required Complexity Explanation
State:
- Time complexity, with the bottleneck identified
- Space complexity, including auxiliary data structures
- Whether the bound is tight or improvable
Self-Evaluation Template
Mock 03 — Medium LC
Date: _______
Problem: _______
Time: ___ / 35 min
Hints used: ___
Scores (1–5) for all 14 dimensions:
___ Total /70
Pattern recognized in: _____ minutes
Bug count during coding: ___
Bug count caught by my own tests: ___
Strongest dimension:
Weakest dimension:
Specific drill for next session:
What to Do If You Fail
- Score 35–41: Repeat with a different problem; focus on weak dimension.
- Took > 35 min: You need more medium volume. Solve 20 mediums in the next week against a 35-min timer.
- Couldn’t recognize the pattern: Go back to Phase 2 (patterns) README and re-read the signal table.
- Bug-prone code: Phase 10, Lab 02 (TDD) and Lab 05 (stress testing).
- Communication weak: Record yourself; listen back; identify silent stretches.
- Pass twice in a row before moving to Mock 04.
Mock 04 — Hard LeetCode
Interview type: LC hard, FAANG onsite-level Target role: SWE-II → Senior; FAANG onsite second round Time limit: 60 minutes Format: 1 hard problem (or 1 medium + 1 hard if you finish hard early) Hints policy: One free hint at 20 min; additional -2 each. After 3 hints, the round is “failed” by FAANG standards. Primary goal: Reach the optimal algorithm under pressure, implement correctly, handle the hard follow-ups.
What This Mock Tests
Hards are where mid-level engineers separate from senior. The bar:
- Recognize the non-obvious pattern (binary search on answer, segment tree, DP on intervals, etc.) within 10 minutes
- Articulate why the optimal works, with correctness sketch
- Implement under time pressure
- Handle 2+ follow-up extensions
Scoring emphasizes Problem Understanding (#1), Optimization (#4), Correctness (#5), and Tradeoff Reasoning (#13).
Pick One Problem
Problem A — Median of Two Sorted Arrays (LC 4)
Given two sorted arrays nums1 and nums2 of sizes m and n, return the median of the combined sorted array. Must run in O(log(m+n)) time.
Examples:
[1, 3], [2] → 2.0 (merged: [1, 2, 3])
[1, 2], [3, 4] → 2.5 (merged: [1, 2, 3, 4])
[], [1] → 1.0
[1, 2], [] → 1.5
Constraints: 0 ≤ m, n ≤ 1000. m + n ≥ 1. -10^6 ≤ values ≤ 10^6.
Problem B — Trapping Rain Water (LC 42)
Given heights of bars height[i], compute how much water it can trap after raining.
Examples:
[0,1,0,2,1,0,1,3,2,1,2,1] → 6
[4,2,0,3,2,5] → 9
Constraints: 1 ≤ |height| ≤ 2×10^4. 0 ≤ height[i] ≤ 10^5.
Problem C — Merge K Sorted Lists (LC 23)
Given an array of k sorted linked lists, merge them into one sorted list.
Examples:
[[1,4,5], [1,3,4], [2,6]] → [1,1,2,3,4,4,5,6]
[] → []
[[]] → []
Constraints: 0 ≤ k ≤ 10^4. 0 ≤ |lists[i]| ≤ 500. Sum of all lengths ≤ 10^4.
Expected Communication Style
- Restate with input/output types and the explicit complexity constraint (where given).
- Ask precise clarifying questions: O(log) vs O(m+n) required? Negative numbers? Empty arrays? Duplicates?
- State a baseline: the obvious O(m+n) for A, O(N) two-pointer for B, O(N log K) for C.
- Identify whether the baseline meets the constraint. If not, derive the harder approach.
- Articulate the key insight before coding. (“For A, I’ll binary search on the partition position in the shorter array such that the left halves of both arrays combined form the lower half of the merged array.”)
- Code carefully — hards have boundary conditions everywhere.
- Test 3+ cases including the boundary (empty array, both arrays size 1, large mismatch).
Solution Sketches
A. Median of Two Sorted: binary search on the partition i in the shorter array. For each i, j = (m + n + 1) // 2 - i. Check if nums1[i-1] ≤ nums2[j] and nums2[j-1] ≤ nums1[i]. If so, median is computed from the boundary 4 values. O(log(min(m, n))). Edge cases: i=0 (no left in nums1) or i=m; same for j.
B. Rain Water: two-pointer. Maintain left_max and right_max. At each step, the side with the smaller max determines how much water can sit at that index. Move that pointer inward. O(N) time, O(1) space. Alternative: precompute left_max and right_max arrays, O(N) time and space.
C. Merge K Lists: min-heap of (value, list_index, node). Pop the smallest, push its next. O(N log K) where N = total nodes. Alternative: divide-and-conquer pairwise merge, same complexity, no heap.
Common Failure Modes
- A: Submitted O(m+n) by merging. Works but fails the complexity requirement — interviewer marks as fail at FAANG bar.
- A: Off-by-one in the partition formula. Most common bug.
- A: Didn’t handle empty arrays. Crash on
[], [1]. - B: Used DP with O(N) space when O(1) two-pointer works. Acceptable but downgrades the “Optimization” score.
- B: Forgot to handle bars that don’t trap any water (descending then ascending).
- C: Used O(N · K) approach — for each output element, scan all K heads. Too slow for K = 10^4.
- C: Forgot null check on
lists[i]— common test failure on[[]].
Passing Bar
- Total score: 45/70 (average 3.2)
- Optimal complexity reached (or a serious attempt with clear awareness of the gap)
- Correct on given examples + 2 boundary cases
- Hint usage ≤ 1
- Time ≤ 60 min
- Articulated correctness argument (not just “trust me”)
Follow-up Questions
For A:
- Generalize to k-th smallest in two sorted arrays. → Same binary search, partition at k-1 in total. O(log(min(m, n))).
- Median of K sorted streams (K small). → Heap-based; not log-time anymore.
- Median of unsorted data? → Quickselect or median-of-medians, O(N).
- Memory-bound large-K case. → External merge sort; k-way merge with bounded heap.
For B:
- 3D rain water (LC 407). → Heap of boundary cells, BFS inward. Much harder.
- Approximate version with O(1) memory for a stream. → Doesn’t exist in general; needs two-pass for exact.
- Trapping with non-zero ground (irregular shape). → Doesn’t fundamentally change.
For C:
- Streaming K sorted streams, K huge (10^6+). → Tournament tree (O(log K) per element), or distributed merge.
- Lists are not in memory (each is a file). → External k-way merge with bounded buffers.
- K-way merge with timestamps + de-dup. → Same algorithm + dedup pass.
- Latency-sensitive variant: emit elements as soon as possible. → Stream the heap output without buffering.
Required Tests
- All given examples
- Both arrays empty (A), all bars zero (B), all lists empty (C)
- One huge + one tiny array (A) — stresses the binary search edges
- Strictly increasing input (B), strictly decreasing input (B)
- K = 1 (C: single list pass-through), K = 0 (C: empty)
- One adversarial input you design
Required Complexity Explanation
- Time complexity, with reasoning
- Space complexity
- Whether the bound is tight or merely upper
- For A: explicitly justify why O(log) is achievable
Self-Evaluation Template
Mock 04 — Hard LC
Date: _______
Problem: _______
Time: ___ / 60 min
Hints used: ___
Scores (1–5):
___ Total /70
Time to reach optimal idea: _____ min
Time to first correct submission: _____ min
Number of bugs hit and fixed:
Was the correctness argument articulated? Y/N
Were 2+ follow-ups answered? Y/N
Strongest dimension:
Weakest dimension:
Action item for next session:
What to Do If You Fail
- Score 38–44: Repeat with a different problem; you nearly passed.
- Score <38: Step back to mediums; ensure 3+ consecutive passes of Mock 03 before retrying hard.
- Couldn’t reach optimal complexity: Review Phase 2 (binary search), Phase 5 (DP), Phase 4 (graphs) — which patterns did you miss?
- Bug-storm on implementation: Phase 10 Lab 04 (correctness proofs) and Lab 05 (stress testing).
- Failed follow-ups: You knew the algorithm but didn’t know its variants; do 5 related problems before the next attempt.
- Pass twice in a row before moving to Mock 05.
Mock 05 — Big Tech Phone Screen
Interview type: FAANG-style phone screen (Google, Meta, Amazon, Microsoft, Apple) Target role: SWE-II / Senior phone round Time limit: 45 minutes total Format: ~5 min intro + 35 min coding + 5 min Q&A. ONE medium-to-hard problem with strong follow-ups. Hints policy: Hints cost real points at FAANG — one is acceptable, two is borderline, three fails. Primary goal: Show you can work cleanly under FAANG’s exact format.
What This Mock Tests
This mock simulates the actual FAANG phone screen format. The interviewer:
- Greets you (~3 min)
- Asks one 1-min behavioral warm-up (“What are you excited about lately?”)
- Presents the coding problem
- Expects you to clarify, plan, code, test
- Asks 2–3 follow-up extensions in the remaining time
The signal they’re collecting: can this person work on our team without supervision? Specifically — do they understand requirements, optimize without being told to, write reasonable code, and engage with extensions intelligently?
Scoring weights: Problem Understanding, Optimization, Communication, Follow-ups are all critical (3+). One weak dimension = no advance.
Pick One Problem
Problem A — Longest Increasing Path in a Matrix (LC 329)
Given an m × n integer matrix, return the length of the longest strictly increasing path. From a cell, you may move up/down/left/right (no diagonals, no wraparound).
Examples:
[[9,9,4],[6,6,8],[2,1,1]] → 4 (path 1→2→6→9)
[[3,4,5],[3,2,6],[2,2,1]] → 4 (path 3→4→5→6)
[[1]] → 1
Constraints: 1 ≤ m, n ≤ 200. 0 ≤ matrix[i][j] ≤ 2^31 - 1.
Problem B — Word Ladder (LC 127)
Given two words beginWord and endWord and a dictionary wordList, find the length of the shortest transformation sequence (each step changes exactly one letter; intermediate words must be in dictionary). Return 0 if no sequence exists.
Examples:
"hit", "cog", ["hot","dot","dog","lot","log","cog"] → 5 (hit→hot→dot→dog→cog)
"hit", "cog", ["hot","dot","dog","lot","log"] → 0 (cog not in dict)
Constraints: 1 ≤ |beginWord| ≤ 10. All words same length, lowercase. 1 ≤ |wordList| ≤ 5000.
Problem C — Number of Islands (LC 200)
Given a 2D grid of '1' (land) and '0' (water), count the number of islands (connected groups of land, 4-directional).
Examples:
[
["1","1","1","1","0"],
["1","1","0","1","0"],
["1","1","0","0","0"],
["0","0","0","0","0"]
] → 1
[
["1","1","0","0","0"],
["1","1","0","0","0"],
["0","0","1","0","0"],
["0","0","0","1","1"]
] → 3
Constraints: 1 ≤ m, n ≤ 300.
Expected Communication Style
- Restate and confirm types. (“Integer matrix; I return the length of the longest strictly increasing path; movement is 4-directional.”)
- Ask 3–5 clarifying questions: matrix size, value range, strictly vs non-strictly increasing, do diagonals count.
- State brute force with complexity. (“DFS from every cell, no memo, exponential worst case.”)
- Identify the optimization signal. (“DFS + memoization since subpath answers don’t change. Or topological sort on the DAG of (i,j) → (i’, j’) where val(i,j) < val(i’, j’).”)
- Justify your choice between alternatives. (“Memo’d DFS is simpler; topo sort is more rigorous and avoids stack depth issues at 200×200.”)
- Code cleanly. Helper functions, no inline magic.
- Walk through the example. Test 2+ edge cases.
- Engage with the follow-ups — these decide the round.
Solution Sketches
A. Longest Increasing Path: memoized DFS. dp[i][j] = longest path starting at (i, j). Recurse to 4 neighbors with strictly greater value; dp[i][j] = 1 + max(dp[neighbor]). O(mn) time and space. The DAG structure ((i,j) → (i', j') iff val < val') guarantees no cycles, so memo is sound.
B. Word Ladder: BFS over the graph where nodes are words and edges connect words differing by one letter. Use a “wildcard bucket” optimization: for each word, generate patterns like h*t, *ot, ho*; bucket words by pattern; neighbors are words sharing a pattern bucket. O(N · L²) where N = dict size, L = word length.
C. Number of Islands: for each unvisited ‘1’, flood fill (BFS or DFS), mark visited, increment count. O(mn) time and space.
Common Failure Modes
- A: brute force without memo. TLE on 50×50.
- A: incorrect strict vs non-strict check.
>vs>=flips the answer. - B: built a graph by comparing every pair of words. O(N² · L) — too slow for N = 5000.
- B: didn’t notice
endWordmay not be in dict. Returns wrong if you assume it is. - B: BFS without visited tracking. Infinite loop.
- C: modified input grid without permission. Some interviewers care; clarify first.
- All: weak follow-up answers. “I’d just use a database” — too vague; doesn’t show understanding.
Passing Bar
- Total score: 46/70 (average 3.3)
- Optimal complexity reached
- Correctness on given examples + 2 edge cases
- Hint usage ≤ 1
- Time ≤ 45 min
- Two follow-ups answered with substance
Follow-up Questions
For A:
- Return the path itself, not just the length. → Add parent pointer in DP; reconstruct.
- What if path can revisit cells? → No longer a DAG; problem is NP-hard (Hamiltonian-flavored).
- Path with diagonal moves allowed? → 8 neighbors instead of 4; same algorithm.
- Matrix is sparse (mostly 0). → Algorithm doesn’t change asymptotically; data layout (CSR) matters at scale.
- Matrix doesn’t fit in memory. → Chunked processing with overlap; harder boundary handling.
For B:
- Return one valid path. → Track BFS parent; reconstruct.
- Return ALL shortest paths (Word Ladder II, LC 126). → BFS to build the DAG, then DFS to enumerate.
- Bidirectional BFS for speedup. → Search from both ends, meet in middle. Roughly √ improvement.
- Streaming dictionary (words arriving). → Re-bucket on each insert; same algorithm.
For C:
- Count distinct island shapes. → Canonicalize the shape (sort relative cell positions, possibly rotate/reflect); hash.
- Number of islands II (online — cells added one by one). → Union-Find; O(α(N)) per operation.
- Largest island after flipping at most one ‘0’ to ‘1’. → Label each island with size; for each ‘0’, sum sizes of unique neighboring islands + 1.
- 3D islands. → Same algorithm, 6 neighbors instead of 4.
Required Tests
- All given examples
- 1×1 matrix / single-letter input
- All-same-value matrix (A: answer is 1)
- Disconnected components (C: multiple islands)
- Long diagonal-like path (A)
- Dictionary missing the endWord (B)
- Empty grid / empty dictionary edge
Required Complexity Explanation
- Time, with reasoning
- Space, including recursion stack and memoization tables
- Worst-case input that triggers the worst-case complexity
- For A: explain why memo turns O(4^(mn)) into O(mn)
Self-Evaluation Template
Mock 05 — Big Tech Phone Screen
Date: _______
Problem: _______
Time: ___ / 45 min
Hints used: ___
Follow-ups answered well (out of 2 asked): ___
Scores (1–5):
___ Total /70
Did I narrate continuously? Y/N
Did I identify the optimization signal before coding? Y/N
Did I test 2+ unprompted edges? Y/N
What I would change for the real interview:
What to Do If You Fail
- Score 40–45: Re-do with a different problem; pinpoint weak dimension.
- Score <40: You’re not ready for FAANG phone screens. Do 10 more mediums + 3 hards, then retry.
- Optimization gap: Phase 2 patterns + Phase 4 graphs are the most-tested patterns at FAANG.
- Follow-up weakness: This is the #1 thing that distinguishes “hire” from “no-hire” at FAANG phone screens. Treat follow-ups as a primary skill, not an afterthought.
- Pass twice in a row before moving to Mock 06.
Mock 06 — Big Tech Onsite
Interview type: FAANG onsite coding round (single round of the 4–5 onsite rounds) Target role: FAANG SWE-II / Senior Time limit: 60 minutes total Format: ~5 min intro + 50 min coding (TWO problems back-to-back) + 5 min Q&A Hints policy: One hint per problem acceptable; more is below-bar. Primary goal: Demonstrate sustained performance across two problems without losing tempo.
What This Mock Tests
FAANG onsites run 4–5 of these rounds per day. Each round expects you to solve 1–2 problems in 50 minutes of coding. This mock packs two problems into 60 minutes deliberately — the time pressure is real.
The signal: are you a consistent solver, not a one-hit-wonder? Can you context-switch from problem 1 to problem 2 without resetting?
Scoring weights: all dimensions matter; Time Management is implicit — running out of time on problem 2 is a hard fail signal.
Format
Pick one easy/medium problem (15–20 min) AND one medium/hard problem (30–40 min) from the list below. The interviewer presents one, then immediately the next once you finish. No break.
Problem Set 1 (warm-up: 15–20 min)
A1 — Valid Anagram (LC 242)
Given two strings, determine if one is an anagram of the other.
"anagram", "nagaram" → True
"rat", "car" → False
Constraints: 1 ≤ |s|, |t| ≤ 5×10^4. Lowercase English letters (or follow up: Unicode).
A2 — Climbing Stairs (LC 70)
You can climb 1 or 2 steps at a time. How many distinct ways to reach step n?
n = 2 → 2 (1+1, 2)
n = 3 → 3 (1+1+1, 1+2, 2+1)
Constraints: 1 ≤ n ≤ 45.
A3 — Move Zeroes (LC 283)
Given nums, move all 0s to the end while maintaining the relative order of non-zero elements. In-place.
[0,1,0,3,12] → [1,3,12,0,0]
Problem Set 2 (main: 30–40 min)
B1 — LRU Cache (LC 146)
Design and implement a Least Recently Used cache with get(key) and put(key, value) both in O(1).
LRUCache cache(2)
cache.put(1, 1)
cache.put(2, 2)
cache.get(1) → 1
cache.put(3, 3) // evicts key 2
cache.get(2) → -1
cache.put(4, 4) // evicts key 1
cache.get(1) → -1
cache.get(3) → 3
cache.get(4) → 4
Constraints: 1 ≤ capacity ≤ 3000. At most 10^5 operations.
B2 — Word Break (LC 139)
Given a string s and a dictionary wordDict, return True if s can be segmented into a sequence of dictionary words.
"leetcode", ["leet", "code"] → True
"applepenapple", ["apple","pen"] → True
"catsandog", ["cats","dog","sand","and","cat"] → False
Constraints: 1 ≤ |s| ≤ 300. 1 ≤ |wordDict| ≤ 1000.
B3 — Course Schedule II (LC 210)
Given numCourses and prerequisites (pairs [a, b] meaning b must be taken before a), return an order in which to take all courses, or [] if impossible.
2, [[1,0]] → [0, 1]
4, [[1,0],[2,0],[3,1],[3,2]] → [0, 1, 2, 3] or [0, 2, 1, 3]
2, [[1,0],[0,1]] → []
Constraints: 1 ≤ numCourses ≤ 2000. 0 ≤ |prerequisites| ≤ 5000.
Expected Communication Style
For each problem:
- Restate
- 2–3 clarifying questions (don’t over-ask on the warm-up; do for the main)
- Brute force + complexity
- Optimal approach + complexity
- Code with narration
- Test 2–3 cases
- Move on to the next problem without dragging
Critical: manage time aggressively. If you blow past 20 min on problem 1, stop and move on. Failing on time is worse than producing partial code on both.
Solution Sketches
A1 Anagram: count chars (Counter or array of 26), compare. O(N + M). A2 Climbing Stairs: Fibonacci. DP or closed form. O(N) or O(1). A3 Move Zeroes: two-pointer, write-pointer advances on non-zero; pad with zeros. O(N) time, O(1) space.
B1 LRU Cache: doubly linked list + hashmap. List front = most recent, tail = LRU. get → move node to front; put → if exists move to front and update; if not, insert at front; if over capacity, remove tail and delete from hashmap. O(1) per op.
B2 Word Break: DP. dp[i] = True if s[:i] can be broken. dp[i] = any(dp[j] and s[j:i] in wordSet) for j < i. O(N² · max_word_len) or O(N²) with set lookup. Watch the off-by-one in dp = [False] * (n + 1); dp[0] = True — a frequent bug.
B3 Course Schedule II: topological sort via Kahn’s algorithm (BFS on indegree). If output length < numCourses, there’s a cycle → return []. O(V + E).
Common Failure Modes
- Spent 30 min on the easy. Total fail; you’ll run out for the main problem.
- LRU implemented with built-in OrderedDict (Python) without explaining the underlying data structure. Some interviewers accept this; many do not. Always offer to implement from scratch.
- Word Break: O(2^N) recursion without memoization. TLE on N=100.
- Course Schedule: DFS-based cycle detection but forgot to track three states (unvisited / in-progress / done). Marks node done while in progress → false negatives.
- Trying to start problem 2 fresh without acknowledging the time check. Senior signal: “We have 35 min left and this is the main problem; let me dive in.”
Passing Bar
- Total score: 49/70 (average 3.5)
- BOTH problems implemented correctly
- Optimal complexity on the main problem
- Time managed: problem 1 ≤ 20 min, problem 2 ≤ 40 min
- Hint usage ≤ 1 total
- Tests run on both
Follow-up Questions (asked between or after problems)
For B1 (LRU):
- Make it thread-safe. → Coarse-grained lock; or read-write lock with care; or lock-free with hazard pointers (advanced).
- LFU instead of LRU. → Two-level structure: hashmap to nodes, hashmap to frequency-buckets, each bucket a doubly-linked list.
- Distributed LRU across multiple servers. → Consistent hashing + per-shard LRU.
- Persist to disk. → Write-behind cache; reconstruct on startup.
For B2 (Word Break):
- Word Break II — return all valid sentences. → Backtracking with memoization on the suffix.
- Words can be arbitrarily long, dict has 10^6 words. → Trie for prefix lookup during DP, O(N²) using trie traversal.
- Streaming version: input arrives one char at a time. → Online DP — update
dp[i]as i grows; suffix automaton helps for the dict.
For B3 (Course Schedule):
- Find any cycle and return it. → DFS with parent tracking; on back-edge, walk parents.
- Schedule to minimize number of semesters (parallel courses). → Longest path in DAG = answer; O(V + E).
- Add weighted edges (course duration). → Critical path method.
Required Tests
- All given examples (both problems)
- Empty / single-element input for problem 1
- Capacity 1 for LRU
- Single-word dict and
sthat exactly equals one word for Word Break - Course graph with cycle for B3
- Self-loop course (numCourses=1, prereq=[[0,0]]) — should return []
Required Complexity Explanation
For both problems, state time and space + identify which one is the bottleneck under the actual constraints (N=300 for Word Break ⇒ O(N²) is fine; N=10^5 ops on LRU ⇒ O(1) per op is mandatory).
Self-Evaluation Template
Mock 06 — Big Tech Onsite
Date: _______
Problem 1: _______ — Time: ___ min, Score: ___/70
Problem 2: _______ — Time: ___ min, Score: ___/70
Total time: ___ / 60 min
Hints used: ___ (across both)
Combined avg score:
Both problems complete? Y/N
Tests run on both? Y/N
What went well:
What went poorly:
Time-management notes:
Action item:
What to Do If You Fail
- Failed problem 2 due to time: Practice problem 1 against a stricter timer (15 min).
- Failed problem 2 due to difficulty: Mock 04 (hard LC) needs more reps.
- Hint-heavy on both: Foundational pattern recognition gap; return to Phase 2.
- LRU implementation issues: Drill data structure design — Phase 1 lab 04 + 05 (linked lists, stacks/queues).
- Pass twice consecutively before Mock 07.
Mock 07 — Senior Engineer
Interview type: Senior SWE coding + design hybrid Target role: Senior Software Engineer (L5 Google / E5 Meta / SDE3 Amazon) Time limit: 60 minutes Format: ONE problem + system extension + explicit tradeoff discussion Hints policy: Hints on the algorithm lower your score significantly; hints on the extension are acceptable. Primary goal: Show senior-level reasoning — not just solving, but choosing among solutions with reasoning.
What This Mock Tests
At the senior bar, mere correctness is not enough. The interviewer wants to see:
- You consider multiple approaches and articulate why you chose one
- You understand the production implications of your choices
- You can extend the algorithm into a service-like context
- You can answer “what if the input is 1000× larger” with a concrete plan
Scoring weights: Tradeoff Reasoning (#13), Production Awareness (#14), Optimization (#4), Follow-ups (#11) are all critical. A senior who scores 3 on tradeoffs has signaled “mid-level”; needs 4+.
Pick One Problem
Problem A — Design a Rate Limiter
Build a rate limiter supporting allow(user_id, timestamp) → bool. Each user can make at most N requests per W seconds. Discuss the algorithm, then extend to a multi-server / distributed setting.
Initial constraints: in-process, single-thread. N ≤ 1000 reqs/window. W ≤ 60 sec. 1M users.
Problem B — Top K Frequent Elements (LC 347) + Streaming Extension
Phase 1: Given an array nums and integer k, return the k most frequent elements. O(N log K).
[1,1,1,2,2,3], k=2 → [1, 2]
[1], k=1 → [1]
Phase 2 (extension): the input is a never-ending stream; report top-k continuously with bounded memory. Discuss exact vs approximate tradeoffs.
Problem C — Snapshot Array (LC 1146)
Implement SnapshotArray:
SnapshotArray(length)— initialize withlengthzerosset(index, val)— set value at indexsnap() → snap_id— take a snapshot, return its idget(index, snap_id)— value at index at the time of snap_id
Discuss the algorithm; extend to a versioned key-value store with garbage collection of old snapshots.
Expected Communication Style
- Restate, including the implied requirements (“rate limit must be enforced even if the same user hits multiple instances”).
- Ask senior-grade clarifying questions: read-heavy vs write-heavy? Latency targets? Consistency requirements? Failure modes acceptable?
- Propose 2+ approaches with explicit tradeoffs. (“Token bucket vs sliding window log vs sliding window counter — I’d pick X because…”)
- State the algorithm and complexity.
- Code the chosen approach with senior code quality (clear naming, error handling at boundaries, no premature abstraction).
- Discuss the production extension without prompting.
- Anticipate failure modes — what breaks at 10× scale? 100×?
Solution Sketches
A. Rate Limiter:
- Sliding window log: per user, deque of timestamps. On request, drop entries older than
W, check length < N, append. O(N) per request (amortized O(1) drop). Memory: O(users × N). - Token bucket: per user,
(tokens, last_refill). On request, refilltokens += (now - last) × rate, cap at N, decrement if ≥ 1. O(1) per request. Slightly bursty. - Sliding window counter: approximate; uses 2 buckets (previous + current window), weighted by overlap. O(1), small memory.
Distributed extension: per-user state in Redis with atomic Lua script; or consistent-hash users to dedicated instances; or central counter with relaxed accuracy.
B. Top K Frequent:
- Static: Counter + min-heap of size K. O(N log K).
- Streaming exact: impossible with bounded memory in general (any element could become top-K later).
- Streaming approximate: Count-Min Sketch + heap of candidates; or Misra-Gries / SpaceSaving algorithm for ε-approximate. O(1/ε) memory.
C. Snapshot Array:
- Naïve: copy entire array per snapshot. O(length) per snap, O(snaps × length) memory.
- Better: per-index, store list of
(snap_id, value)pairs sorted by snap_id. Lookup with binary search. O(log S) per get, O(1) per set, O(total writes) memory.
Versioned KV store extension: persistent data structures (Clojure-style); or copy-on-write trees; or LSM-tree with snapshot isolation.
Common Failure Modes
- Implemented the first algorithm that came to mind without discussing alternatives. This is the #1 senior-bar failure.
- Said “I’d just use Redis” without explaining the algorithm Redis would implement. The interviewer wants the algorithm; the database is a deployment choice.
- Top-K streaming: claimed exact algorithm with bounded memory. Impossible in general; signals theoretical weakness.
- Snapshot Array: copied the array per snapshot. Acceptable as brute force; bad as final answer for a senior.
- No tests beyond the given examples.
- Skipped the production extension entirely.
Passing Bar
- Total score: 53/70 (average 3.8)
- Tradeoff Reasoning #13: ≥ 4
- Production Awareness #14: ≥ 4
- Optimal or near-optimal algorithm
- Extension discussed substantively (not just “I’d shard it”)
- Correct, readable code
Follow-up Questions
For A (Rate Limiter):
- Latency budget: < 1ms p99. → In-memory store; Redis is borderline (network RTT). Local cache with eventual sync.
- Multi-region with strict global limit. → Hard; usually relaxed to per-region limit + occasional reconciliation.
- What if Redis goes down? → Fail-open (allow) vs fail-closed (deny); usually fail-open for rate limiters.
- Hot user (one user makes 90% of requests). → Dedicated shard; or local fast path before checking shared state.
For B (Top K):
- Approximate with ε = 0.01. → Count-Min Sketch sized accordingly.
- Top K most-improved over the last hour. → Two-window comparison; bigger memory.
- Trending detection (top K with sudden growth). → Slope/derivative-based; needs time-windowed counts.
- What if K = 1M? → Heap-of-K doesn’t fit memory; external merge or sampling.
For C (Snapshot Array):
- Snapshot every write (versioned KV). → Same structure; consider compaction.
- Memory pressure: drop snapshots older than 1 hour. → Per-index list pruning; tombstones for fully-deleted snaps.
- Snapshot isolation in a multi-writer setting. → Multi-version concurrency control; per-transaction snapshot id.
- Persist snapshots to disk. → Log-structured store; periodically checkpoint.
Required Tests
- Given examples
- Empty / boundary input
- Heavy churn (many writes to same index for C)
- Single user / single key
- Burst of requests at the window boundary (A)
- K = N for B (no filtering)
Required Complexity + Production Discussion
Cover:
- Time per operation, space per user/element
- Latency under typical load vs worst case
- Memory growth and GC implications
- Failure semantics (what happens on partial failure)
- Monitoring metrics you’d add (rate limit reject rate, top-K convergence time, snapshot lookup p99)
Self-Evaluation Template
Mock 07 — Senior Engineer
Date: _______
Problem: _______
Time: ___ / 60 min
Scores (1–5):
___ Total /70
Tradeoff Reasoning (#13): ___
Production Awareness (#14): ___
Did I propose 2+ approaches before coding? Y/N
Did I anticipate scale-up failure modes? Y/N
Action item:
What to Do If You Fail
- Tradeoff or Production score below 4: Read Phase 8 (practical engineering) deeply; rebuild a small system (rate limiter, cache).
- Algorithm score below 3: You haven’t earned the right to do senior interviews yet; back to Mock 04–06.
- Code quality issues: Read CODE_QUALITY.md.
- Pass twice consecutively before Mock 08.
Mock 08 — Staff Practical
Interview type: Staff/Principal engineer practical coding round Target role: Staff SWE (L6 Google / E6 Meta / Principal Amazon) Time limit: 75 minutes Format: Build a working component (not a LeetCode puzzle) with multiple interacting pieces Hints policy: Hints affect score but rarely fail you outright at staff bar — the bar is judgment, not raw problem-solving. Primary goal: Demonstrate the ability to build a real thing under time pressure, with monitoring/failure-mode awareness baked in.
What This Mock Tests
Staff interviews shift away from “can you solve this puzzle” toward “can you build something we’d ship.” You’re given a problem statement that resembles a small feature spec. Your job:
- Decompose the problem into modules with clean interfaces
- Choose data structures that match real production constraints
- Implement the core fully + skeletons for the rest
- Discuss monitoring, deployment, failure recovery, evolution
- Justify every choice you make against alternatives
Scoring weights: Code Quality (#7), Tradeoff Reasoning (#13), Production Awareness (#14) are paramount. Pure algorithmic Optimization is less critical — staff problems rarely have a “trick.”
Pick One Build
Build A — In-Memory Rate Limiter Library
Build a usable Python/Java/Go module that provides:
limiter = RateLimiter(max_requests=100, window_seconds=60)
allowed = limiter.allow(user_id="alice") # bool, ~10µs p99
Required:
- Multiple algorithms behind a uniform interface (token bucket + sliding window log + sliding window counter)
- Configurable per-user vs global limits
- Thread-safe
- A
stats()method returning rejection rate per user - A
purge()method to evict idle users from memory - Tests covering correctness, thread safety, and the window-edge case (burst at exactly t=W)
Build B — Bounded LRU Cache with TTL and Stats
Build an LRU cache that also supports per-entry TTL:
cache = LRUCache(capacity=10000, default_ttl_seconds=300)
cache.put(key, value, ttl=None) # uses default
val = cache.get(key) # returns None if missing/expired
cache.delete(key)
cache.stats() # hit rate, eviction rate, expiration rate
Required:
- O(1) get/put
- Lazy + active TTL expiration
- Thread-safe
- Memory cap (eviction policy: LRU among non-expired)
- Tests: correctness, expiration races, concurrent put/get, capacity overflow
Build C — Job Scheduler (cron-like)
Build a scheduler that runs jobs at specified intervals:
scheduler = Scheduler()
scheduler.add(name="cleanup", interval_sec=300, fn=cleanup_fn)
scheduler.add(name="report", cron="0 9 * * MON", fn=report_fn) # nice-to-have
scheduler.start()
scheduler.stop()
scheduler.status() # last run, next run, last error per job
Required:
- Multiple jobs running independently
- Graceful shutdown (don’t kill mid-job)
- Per-job error isolation (one job’s failure doesn’t crash the scheduler)
- Catch-up policy on missed runs (skip vs catch-up; configurable)
- Tests: timing, overlap, panic in job
Expected Communication Style
- Restate with assumptions stated upfront: “I’ll assume single-process, multi-threaded, in-memory; if you want me to extend to distributed, that’s a separate discussion.”
- Propose the module decomposition before writing any code. Whiteboard the public interface, the internal modules, the data flow.
- Identify the 2–3 critical design decisions and discuss alternatives.
- Pick one and code it — favor depth over breadth. Skeleton/stub the rest with comments like
# TODO: implement token bucket variant with same interface. - Discuss monitoring without prompting: which metrics, why, what alerts.
- Discuss failure modes without prompting: thread starvation, memory blowup, race conditions, partial failures.
- Test the critical path. Production-style tests, not just smoke.
Solution Sketches
A. Rate Limiter:
class RateLimiter(ABC):
def allow(self, key: str) -> bool: ...
def stats(self) -> dict: ...
def purge(self, idle_seconds: int): ...
class SlidingWindowLogLimiter(RateLimiter):
def __init__(self, max_requests, window_seconds):
self._max = max_requests
self._window = window_seconds
self._logs = defaultdict(deque) # key → deque of timestamps
self._lock = threading.Lock()
self._rejects = Counter()
self._accepts = Counter()
def allow(self, key):
now = time.monotonic()
with self._lock:
log = self._logs[key]
while log and log[0] <= now - self._window:
log.popleft()
if len(log) < self._max:
log.append(now)
self._accepts[key] += 1
return True
self._rejects[key] += 1
return False
Plus token bucket and sliding-window-counter implementations behind the same interface.
B. LRU + TTL: doubly linked list + hashmap; each node stores (key, value, expires_at, prev, next). get: check expiry, evict if expired, return; else move to MRU. put: insert/update, evict LRU if over capacity. Background thread (or lazy on every get/put) sweeps expired entries.
C. Job Scheduler: thread pool + priority queue of (next_run_time, job). Main loop: peek queue, sleep until next, run job in pool, re-schedule. Catch exceptions per job; record to status. Graceful shutdown: stop accepting new runs, await running ones with timeout.
Common Failure Modes
- Built the algorithm without the interface. Staff interviews care about the API as much as the implementation.
- No thread safety. Mentioned in the spec; missed → fail.
- No mention of monitoring/observability. Critical staff signal.
- Used global state. Hard to test, hard to reason about.
- Coded all three rate limiter algorithms in 75 min instead of one well + sketches. Depth > breadth.
- TTL implementation does periodic full scan. O(N) sweep per second isn’t acceptable; lazy + bounded active sweep is.
- Scheduler: jobs share state and races corrupt it. Job functions need to be treated as untrusted code.
Passing Bar
- Total score: 56/70 (average 4.0)
- Code Quality #7 ≥ 4
- Tradeoff Reasoning #13 ≥ 4
- Production Awareness #14 ≥ 4
- Working core; documented stubs for the rest
- At least 3 tests covering: correctness, concurrency, edge timing
- Monitoring + failure modes discussed substantively
Follow-up Questions
For A:
- Make it distributed. → Redis with Lua atomic ops; or per-shard local limiter + global reconciliation.
- Hot-user problem. → Sharded sub-limiters per user; or local L1 cache.
- Add quota burst (allow 2× for 5 sec then throttle). → Token bucket with two-tier refill.
For B:
- What’s the GC pressure under high churn? → Allocation per put is the cost; pool nodes if hot path.
- Persist across restart. → Periodic snapshot to disk; replay log on startup.
- Add a probabilistic admission filter (TinyLFU). → Prevent cache pollution from one-hit-wonders.
For C:
- Distribute across N workers. → Leader-elected scheduler that dispatches jobs; or per-shard schedulers.
- Persistent jobs (survive restart). → Persist queue to durable storage.
- Jobs with dependencies. → DAG scheduler; topological execution.
- Job retries with exponential backoff. → Per-job retry state machine.
Required Tests
- Correctness on the basic case
- Thread safety (concurrent calls; assert no double-count, no race-induced overflow)
- Timing edges (window boundary, expiration boundary, scheduling drift)
- Failure: what happens if the underlying clock jumps backward?
- Resource cleanup: after purge / shutdown, no leaked threads or memory
Required Discussion (production)
Cover, at minimum:
- Metrics you’d export (Prometheus-style)
- Alert thresholds you’d set
- Memory cap behavior
- Failure modes and recovery
- Deployment story (config, rollout, rollback)
- Evolution: how would you add a new rate-limiting algorithm? (Should be drop-in.)
Self-Evaluation Template
Mock 08 — Staff Practical
Date: _______
Build: _______
Time: ___ / 75 min
Scores (1–5):
___ Total /70
Critical dimensions:
Code Quality (#7): ___
Tradeoff (#13): ___
Production (#14): ___
Interface designed before coding? Y/N
Monitoring discussed unprompted? Y/N
Failure modes discussed unprompted? Y/N
Thread safety verified by test? Y/N
What I left unfinished (and what I'd do with another hour):
Action item:
What to Do If You Fail
- Production score below 4: Build a real version of one of these systems and run it for a week with metrics. Phase 8 has more.
- Code quality below 4: Have a senior do a written code review of your submission; act on it.
- Tradeoff below 4: For every decision, force yourself to write down at least 2 alternatives and a rejection reason.
- Pass twice consecutively before Mock 09.
Mock 09 — Runtime / Language Deep Dive
Interview type: Mid-coding language/runtime probe (Bloomberg, Stripe, hedge funds, infra-heavy teams) Target role: Senior / Staff backend or systems Time limit: 45 minutes Format: ONE medium problem, interrupted by language/runtime probes during/after coding Hints policy: Hints on the probes are -1 each; on the algorithm, standard. Primary goal: Demonstrate that you understand your language at a level deeper than syntax.
What This Mock Tests
Some companies will deliberately interrupt your coding with “what does this line cost?” or “what happens if two threads call this concurrently?” or “where does this object get allocated?” The signal: senior engineers don’t just write code; they know what the runtime does with it.
Scoring weights: Language/Runtime (#12) is doubled. Other dimensions normal.
Pick a language you claim to know well. The probes are language-specific.
Pick One Problem (any language)
Problem A — Implement a Concurrent Counter
Build a thread-safe counter with incr(), decr(), read(). Discuss the tradeoffs of locking vs atomic vs sharded.
Problem B — Producer-Consumer Queue
Implement a bounded blocking queue: put(item) blocks if full; get() blocks if empty.
Problem C — Implement flatten(nested_list) Lazily
Given an arbitrarily nested list (e.g., [1, [2, [3, 4]], 5, [[6]]]), return an iterator yielding flat elements lazily (constant memory).
Probes by Language (interviewer fires these mid-coding)
Python
- “What does
list.append(x)cost? When does it resize?” - “How does Python implement
dict? What’s the lookup cost in the worst case?” - “What happens to your concurrent counter under the GIL? Is
+=atomic?” - “What’s reference counting? When does the cyclic GC run?”
- “What does
with lock:desugar to?” - “Why is
multiprocessingdifferent fromthreading? When would you use which?” - “Difference between
asyncio.sleep(0)andtime.sleep(0)?” - “What’s an
__slots__and when does it matter?” - “Generators vs iterators vs async iterators — implement
flattenas each.” - “Where does the GIL hurt your code most?”
Java
- “What’s the difference between
volatile intandAtomicInteger?” - “Explain the Java Memory Model — happens-before relationship.”
- “When does
synchronizeduse biased locking / thin lock / fat lock?” - “What does
String s = a + b;compile to?” - “How does HashMap resize? What’s the cost?”
- “Difference between G1, ZGC, Shenandoah?”
- “What’s a
MethodHandle?” - “When does escape analysis kick in?”
- “What’s
Unsafe, why does it exist?” - “Implement the queue using
ReentrantLockvssynchronized— what differs?”
Go
- “What does a goroutine cost (stack, scheduler)?”
- “Explain GMP — goroutines, M (OS thread), P (processor).”
- “How does
selectwork under the hood?” - “What’s escape analysis? Show me an example of stack vs heap allocation.”
- “Difference between
sync.Mutexandsync.RWMutex?” - “What’s the cost of channels? When to prefer mutex?”
- “Explain Go’s GC — what generation? What pause time?”
- “What does
defercost?” - “Implement the bounded queue using channels vs using a mutex — which and why?”
C++
- “Explain RAII.”
- “What’s the difference between
std::atomic<int>andstd::mutex-protected int?” - “Memory orders:
relaxed,acquire,release,acq_rel,seq_cst— when to use which?” - “What does
std::vector::push_backcost? Amortized vs worst-case?” - “Move semantics — when is the move constructor called?”
- “Difference between
std::shared_ptrandstd::unique_ptr?” - “What’s
std::launder?” - “Implement the bounded queue using
std::condition_variable.” - “What’s the cost of a virtual call?”
Rust
- “Borrow checker rules — one mutable XOR many immutable references.”
- “When do you need
Arc<Mutex<T>>vsRc<RefCell<T>>?” - “Explain
SendandSynctraits.” - “What does
async fndesugar to?” - “Difference between
tokio::spawnandtokio::task::spawn_blocking?” - “What’s a
Pin<T>? Why does it exist?” - “When does lifetime elision apply?”
- “Implement the bounded queue using
tokio::sync::mpsc.”
Node.js / JavaScript
- “Explain the event loop — phases, microtasks vs macrotasks.”
- “What’s the difference between
process.nextTickandsetImmediate?” - “When is a
Promiseresolved synchronously vs asynchronously?” - “What’s V8’s hidden class / inline cache?”
- “Why does
obj.x = 1afterobj.y = 2behave differently than the reverse order in terms of perf?” - “What’s a
WeakRef? When is the value collected?” - “Implement the queue using
async/await.”
Expected Communication Style
- Restate problem.
- Ask clarifying questions (including language-specific ones — “should
read()return a snapshot or be consistent with concurrent updates?”). - Code the algorithm.
- Engage with probes as they come. Don’t say “I’ll come back to that.” Pause coding, answer, resume.
- After coding, walk through tests.
- End with a senior-level reflection on what could change with different runtime characteristics (“if we moved this to Go, the channel-based design would replace the lock-based one”).
Common Failure Modes
- Memorized the algorithm but didn’t know how the language implements its data structures.
- “I don’t know” to a basic probe. A senior should know how the language’s main collections perform.
- Implemented the concurrent counter with
intand+=in Python, thinking GIL makes it safe. GIL ensures bytecode atomicity butx += 1is read-modify-write across bytecodes. Usethreading.LockorAtomic*types. - In Java, used
++onvolatile intthinking it’s atomic. It’s not —volatileensures visibility but not atomicity. - In Go, used a channel for a single shared counter. Slower than
sync/atomic.AddInt64; mismatched tool.
Passing Bar
- Total score: 53/70 (average 3.8)
- Language/Runtime (#12): ≥ 4 (mandatory)
- Algorithm: correct and at expected complexity
- Answered ≥ 4 of 5–6 probes substantively
- Code idiomatic for the language
Follow-up Questions (post-coding)
- “Now port your solution to [other language]. What changes?”
- “Profile this in production — what tool, what metrics?”
- “Where would you put the metric instrumentation?”
- “If this is the hot path of a service handling 1M qps, what’s the bottleneck?”
Required Tests
- Algorithm correctness (basic)
- Concurrent stress test (≥ 4 threads/goroutines, hammering)
- Boundary timing (empty queue blocks; full queue blocks; producer wakes consumer)
- Resource cleanup / shutdown semantics
Required Runtime Discussion
State, for your solution:
- What allocations occur per operation (heap vs stack)
- Lock granularity and contention behavior
- GC implications (Python ref count, Java GC pause, Go STW, etc.)
- What the worst-case latency is and why
Self-Evaluation Template
Mock 09 — Runtime / Language
Date: _______
Language: _______
Problem: _______
Time: ___ / 45 min
Scores (1–5):
___ Total /70
Language/Runtime (#12): ___ (need ≥ 4)
Probes asked: ___
Probes answered well: ___
Probes I bombed (write each verbatim, drill before next mock):
1.
2.
3.
Action item:
What to Do If You Fail
- Probe score below 4: Spend the next week with Phase 9 — your language directory. Read the language’s runtime/perf docs cover to cover.
- Couldn’t answer 50% of probes: You don’t know the language as well as you claim. Pick a different language to claim, OR invest 2+ weeks.
- Pass twice consecutively before Mock 10.
Mock 10 — Infrastructure / Backend
Interview type: Backend / Platform / Infrastructure deep-dive coding Target role: Senior / Staff backend, distributed systems, database, storage Time limit: 75 minutes Format: Build a non-trivial backend component (KV store, log-structured index, sharded cache) Hints policy: Acceptable on the algorithm; failures on storage/concurrency fundamentals are red flags. Primary goal: Demonstrate that you can build the building blocks of real backend systems, not just consume them.
What This Mock Tests
Companies like Stripe, Snowflake, Databricks, Confluent, Cockroach Labs, and infra teams at FAANG ask coding questions that resemble small slices of their actual products. You’re expected to:
- Understand storage primitives (logs, indexes, B-trees, LSM)
- Reason about durability, ordering, concurrency
- Write code that could be a starting point for a production component
- Discuss the gap between what you built and what production would need
Scoring weights: Production Awareness (#14), Code Quality (#7), Correctness (#5), Tradeoff Reasoning (#13) are all critical.
Pick One Build
Build A — In-Memory KV Store with Snapshot Isolation
kv = KVStore()
tx = kv.begin() # returns a transaction
tx.put("k", "v")
tx.get("k") # returns "v" (read your writes)
tx2 = kv.begin()
tx2.get("k") # returns None (snapshot isolation; tx1 not committed)
tx.commit()
tx2.get("k") # still None (tx2's snapshot was taken before commit)
Required:
- MVCC (multi-version concurrency control)
- Snapshot reads return a consistent view
- Concurrent writers
- Tests for read-your-writes, isolation between tx, serialization-style conflict detection (optional)
Build B — Log-Structured Index (Mini-LSM)
db = LSMTree(memtable_threshold=1000)
db.put("k1", "v1")
db.put("k2", "v2")
db.get("k1") # "v1"
db.delete("k1")
db.get("k1") # None (tombstone)
# After threshold writes, memtable flushes to immutable SSTable
db.range("k1", "k9") # iterator over keys
Required:
- In-memory
memtable(sorted, e.g., SortedList or skiplist) - Flush to immutable SSTable when memtable exceeds threshold
getchecks memtable, then SSTables in reverse-time order- Tombstones for deletes
- Range scan that merges across all levels
- Tests covering: writes, reads-after-flush, range correctness, deletes
Build C — Consistent Hash Ring with Replication
ring = HashRing(replication_factor=3, virtual_nodes=128)
ring.add_node("node-A")
ring.add_node("node-B")
ring.add_node("node-C")
nodes = ring.get("user-123") # 3 nodes responsible
ring.remove_node("node-A") # 1/3 of keys reassign
nodes_after = ring.get("user-123")
Required:
- Virtual nodes for load balancing
- Replication factor enforced
- Adding/removing a node moves only its share of keys
- A test that verifies < 5% of keys move when a node is added (with sufficient virtual nodes)
Expected Communication Style
- Restate with assumptions stated upfront.
- Decompose before coding: which modules, what interfaces, what’s the data flow.
- Discuss the storage model:
- For A: how to represent versions per key
- For B: how to lay out SSTables; in this exercise, in-memory simulated
- For C: ring representation; virtual nodes; lookup data structure
- Identify the concurrency model and discuss what fails without it.
- Code the core end-to-end before tackling optimization.
- Discuss production gaps: persistence (we’re in-memory), replication consistency (we’re local), recovery (no WAL), monitoring (none).
- Test the invariants, not just the happy path.
Solution Sketches
A. KV with MVCC:
class KVStore:
def __init__(self):
self._data = {} # key → list of (version, value)
self._next_version = itertools.count()
self._lock = threading.Lock()
def begin(self):
with self._lock:
v = next(self._next_version)
return Transaction(self, v)
class Transaction:
def __init__(self, store, version):
self._store = store
self._snapshot_version = version
self._writes = {} # local buffer
self._committed = False
def get(self, key):
if key in self._writes: return self._writes[key]
versions = self._store._data.get(key, [])
for v, val in reversed(versions):
if v <= self._snapshot_version: return val
return None
def put(self, key, value): self._writes[key] = value
def commit(self):
with self._store._lock:
commit_v = next(self._store._next_version)
for k, v in self._writes.items():
self._store._data.setdefault(k, []).append((commit_v, v))
self._committed = True
B. LSM Tree: SortedList for memtable; on flush, freeze into immutable sorted list (the “SSTable”). get walks memtable + SSTables newest-first. Range scan: heap-merge iterators over all levels. Tombstones represented as special sentinel.
C. Consistent Hash: sorted list of (hash(virtual_node_id), node) pairs. Lookup: hash key, binary search for next pair, walk forward to collect R distinct nodes. Add/remove: insert/delete the virtual node entries.
Common Failure Modes
- A: returned all versions instead of the snapshot-visible one. Snapshot isolation not implemented; just MVCC storage.
- A: forgot to use a local write buffer. Transactions are visible to others before commit.
- B: re-sorting on every read. Should sort on flush; reads are merge.
- B: no tombstone semantics. Deleted key still appears in older SSTable.
- C: hash ring without virtual nodes. Load imbalance — one node gets 60% of keys.
- C: re-hashing entire keyspace on node change. Defeats the purpose of consistent hashing.
- All: no concurrency testing. Build passes single-thread; explodes under load.
Passing Bar
- Total score: 56/70 (average 4.0)
- Working core implementation
- Concurrency correct (or explicit single-threaded contract with rationale)
- Production gaps discussed substantively
- At least 4 tests covering invariants (not just smoke tests)
- Code quality: production-quality
Follow-up Questions
For A:
- Add serializable isolation. → Validation phase: on commit, abort if any key read had a newer version. (Optimistic concurrency.)
- Garbage-collect old versions. → Track oldest active snapshot; vacuum versions older than that.
- Persist to disk. → Write-ahead log per transaction; redo on recovery.
- Distributed version: two-phase commit + Paxos for log replication.
For B:
- Add bloom filters per SSTable. → Skip SSTable scan if key definitely not present.
- Compaction strategy: leveled vs size-tiered. → Tradeoffs in write amplification.
- Crash recovery. → WAL replay before opening memtable.
- Range scan optimization with min/max key per SSTable.
For C:
- Heterogeneous nodes (different capacity). → Virtual nodes proportional to capacity.
- Read consistency across replicas. → Quorum reads (R + W > N).
- Hinted handoff when a replica is down. → Buffer writes for offline replicas, flush on return.
- Anti-entropy / Merkle trees. → Detect divergence between replicas.
Required Tests
- Happy path correctness
- Boundary case (empty store, single key, max capacity)
- Concurrency: ≥ 8 threads, hammer the API for several seconds, verify invariants hold
- Recovery semantics (if implemented)
- For A: snapshot isolation — test that opens tx1, writes via tx2, commits tx2, reads via tx1 → must return the pre-commit value
- For B: write enough to trigger flush; read works across memtable + SSTable
- For C: load distribution test (after adding N nodes, max-min keys per node ratio ≤ 1.3)
Required Production Discussion
- Persistence strategy and crash recovery
- Replication model and consistency tradeoffs
- Monitoring: latency p50/p99, throughput, queue depths, GC pauses
- Failure modes: node loss, network partition, slow disk
- Backpressure: what happens when writes outpace flush
Self-Evaluation Template
Mock 10 — Infrastructure / Backend
Date: _______
Build: _______
Time: ___ / 75 min
Scores (1–5):
___ Total /70
Core working? Y/N
Concurrency tests pass under stress? Y/N
Production gaps discussed unprompted? Y/N (list them: ___)
What I left out (and what it would take):
Action item:
What to Do If You Fail
- Storage primitives unclear: Read “Designing Data-Intensive Applications” (Chapters 3 and 5).
- Concurrency issues: Phase 9 (language/runtime) concurrency sections.
- Code quality: A senior code review of your build.
- Pass twice consecutively before Mock 11.
Mock 11 — Concurrency Heavy
Interview type: Concurrency / parallelism coding round Target role: Backend, systems, infrastructure, embedded, gaming, OS-adjacent Time limit: 60 minutes Format: ONE problem requiring real concurrency primitives (locks, condvars, channels, atomics) Hints policy: A hint on the primitive choice is acceptable; a hint on the race condition is borderline. Primary goal: Write code that is provably correct under concurrent access.
What This Mock Tests
Concurrency code is notoriously hard. The bar:
- Identify what needs synchronization (shared mutable state)
- Choose the right primitive (mutex vs RWLock vs channel vs atomic)
- Avoid deadlocks, livelocks, starvation
- Write a test that would catch the bug if present (not just one that passes by luck)
Scoring weights: Correctness (#5), Code Quality (#7), Language/Runtime (#12) are critical.
Pick One Problem
Problem A — Bounded Blocking Queue (with timeout)
q = BoundedQueue(capacity=10)
q.put(item) # blocks if full
q.put(item, timeout=5.0) # returns False on timeout
item = q.get() # blocks if empty
item = q.get(timeout=5.0) # returns None on timeout
q.close() # subsequent put → exception; get drains then returns None
Multiple producers, multiple consumers. Must be FIFO. Must support graceful close.
Problem B — Thread Pool (with shutdown semantics)
pool = ThreadPool(workers=4)
future = pool.submit(fn, arg)
result = future.result(timeout=5.0) # blocks until done
pool.shutdown(wait=True) # waits for all queued + running
pool.shutdown(wait=False) # rejects new, returns immediately
Required: bounded work queue, graceful drain, future-based result delivery, exception propagation.
Problem C — Read-Write Lock (Writer-Preferred)
lock = RWLock()
with lock.read():
# multiple readers concurrently
...
with lock.write():
# exclusive; blocks new readers waiting for writer
...
Required: many readers OR one writer; writers must not starve.
Problem D — Dining Philosophers (Deadlock-Free)
5 philosophers, 5 chopsticks. Each alternates think/eat. Implement so no deadlock, no philosopher starves.
Expected Communication Style
- Restate with the explicit concurrency requirements (“multiple producers, multiple consumers, FIFO, graceful close”).
- Identify shared mutable state. This is the most important step.
- Identify the invariant. (“Queue size never exceeds capacity”; “no two writers active simultaneously”; “no two adjacent philosophers eat simultaneously”.)
- Choose primitives with rationale. (“I need to block on full/empty — that means condvar, not just a lock.”)
- Code the critical section minimally. Hold the lock only across the shared-state mutation; release before any potentially-blocking call.
- Write a concurrency test. Not just “does it work once” — stress with many threads, verify the invariant.
- Discuss failure modes: what happens if a producer dies holding the lock? If close is called during a put?
Solution Sketches
A. Bounded Queue:
class BoundedQueue:
def __init__(self, capacity):
self._capacity = capacity
self._q = deque()
self._lock = threading.Lock()
self._not_full = threading.Condition(self._lock)
self._not_empty = threading.Condition(self._lock)
self._closed = False
def put(self, item, timeout=None):
with self._lock:
if self._closed: raise QueueClosed()
end = time.monotonic() + timeout if timeout else None
while len(self._q) >= self._capacity:
if self._closed: raise QueueClosed()
remaining = end - time.monotonic() if end else None
if remaining is not None and remaining <= 0: return False
self._not_full.wait(timeout=remaining)
self._q.append(item)
self._not_empty.notify()
return True
def get(self, timeout=None):
with self._lock:
end = time.monotonic() + timeout if timeout else None
while not self._q:
if self._closed: return None
remaining = end - time.monotonic() if end else None
if remaining is not None and remaining <= 0: return None
self._not_empty.wait(timeout=remaining)
item = self._q.popleft()
self._not_full.notify()
return item
def close(self):
with self._lock:
self._closed = True
self._not_full.notify_all()
self._not_empty.notify_all()
B. Thread Pool: N worker threads pulling from a shared bounded queue; each task wraps (fn, args, future). Workers set future’s result/exception. Shutdown sentinels (None) wake workers; wait=True joins all worker threads.
C. RWLock writer-preferred: Track reader_count, writer_active, waiting_writers. Reader acquires only if no writer active AND no waiting writers. Writer blocks while readers active; once it’s the chosen waiter, blocks all new readers.
D. Philosophers: asymmetric strategy — even-numbered grabs left then right, odd-numbered grabs right then left. Breaks the symmetry that causes circular wait → no deadlock. Alternative: use a hierarchical lock ordering (always grab lower-id chopstick first).
Common Failure Modes
- Used
ifinstead ofwhilearoundwait(). Spurious wakeups cause invariant violations. - Released the lock before notifying, or notified before updating state. Wakes a consumer that finds the queue empty.
- Held the lock during a callback / blocking I/O. Other threads stall.
- Used
notify()instead ofnotify_all()for close. Only wakes one waiter; others hang forever. - RWLock without writer preference → writer starvation.
- Philosophers: all grab left → deadlock. Classic.
- Test that runs once and passes. Not a concurrency test. Need stress (1000+ iterations across many threads).
- Used
concurrent.futures.ThreadPoolExecutorfor Problem B without implementing the underlying logic. Some interviewers accept this; many want you to build it.
Passing Bar
- Total score: 56/70 (average 4.0)
- Code correctness verified under stress (≥ 10K operations across ≥ 8 threads)
- Invariant explicitly stated and tested
- No use of high-level concurrency abstractions that hide the primitive (e.g.,
queue.Queuefor problem A) - Failure modes discussed
Follow-up Questions
For A:
- Lock-free version. → Discuss MPMC ring buffer with CAS; show awareness of ABA problem.
- Multi-priority. → Multiple internal queues, one per priority.
- Persist across restart. → Append-only log; replay on startup.
For B:
- Work-stealing pool. → Per-worker deque; idle workers steal from others.
- Dynamic resizing. → Grow workers under load; shrink when idle.
- Cancellable tasks. → Future.cancel() signals; worker checks flag.
For C:
- Fair RWLock (FIFO). → Single waiting queue; reader batching possible but trickier.
- Reader-preferred. → Easier to implement; writer starvation risk.
- Async version with futures. → Same logic; futures replace condvar.
For D:
- Variant where philosophers think for variable time. → Same algorithm.
- Generalize to N philosophers and K chopsticks. → Open problem; resource allocation graphs.
- Distributed version (philosophers on different machines). → Requires distributed deadlock detection.
Required Tests
- Single-thread correctness
- Multi-thread stress: many producers + consumers; assert no item lost, no item duplicated, no exception
- Timeout correctness:
put(timeout=0.1)returns False after 100ms when full - Close semantics: close during active put/get unblocks all waiters cleanly
- Invariant assertion: assert size ≤ capacity throughout, etc.
Example stress test for A:
def test_stress():
q = BoundedQueue(10)
produced = []
consumed = []
def producer(start, count):
for i in range(start, start + count):
q.put(i)
produced.append(i)
def consumer():
while True:
x = q.get(timeout=1.0)
if x is None: break
consumed.append(x)
threads = [threading.Thread(target=producer, args=(i*1000, 1000)) for i in range(10)]
consumers = [threading.Thread(target=consumer) for _ in range(5)]
for t in threads + consumers: t.start()
for t in threads: t.join()
q.close()
for t in consumers: t.join()
assert sorted(consumed) == sorted(produced)
assert len(consumed) == 10_000
Required Discussion
- The invariant your code maintains
- The lock ordering (if multiple locks)
- The worst-case latency (lock contention)
- What happens under crash mid-operation
- How you’d debug a deadlock if one were reported (jstack, py-spy dump, gdb)
Self-Evaluation Template
Mock 11 — Concurrency
Date: _______
Problem: _______
Time: ___ / 60 min
Scores (1–5):
___ Total /70
Stress test passed (≥ 10K ops)? Y/N
Invariant explicitly stated? Y/N
Failure modes discussed? Y/N
Any race condition found post-hoc? (List:)
Action item:
What to Do If You Fail
- Race condition in submitted code: This is the #1 reason to repeat — concurrency bugs in production are catastrophic.
- Couldn’t choose the right primitive: Read your language’s concurrency chapter (Phase 9). Understand condvar vs channel vs atomic before next attempt.
- Stress test exposed a bug you didn’t anticipate: Lab 05 (stress harness) applied to concurrent code is your training.
- Pass twice consecutively before Mock 12.
Mock 12 — Competitive Style
Interview type: Algorithmic puzzle round (Jane Street, Hudson River, Citadel, Two Sigma, ICPC-style firms; some Google L6+ rounds) Target role: Quant developer, HFT, compiler/optimization, ICPC alumni, top-tier algorithmic teams Time limit: 90 minutes Format: ONE hard algorithmic problem (Codeforces Div 2 D / Div 1 B level) Hints policy: No free hints. A hint is a hard signal of failure. Primary goal: Reach the algorithmic insight under sustained time pressure.
What This Mock Tests
This mock is not about production engineering. It’s about pure algorithmic depth and the ability to think clearly for 90 minutes on a problem with no obvious path.
The kind of problem chosen:
- Has a clever insight that unlocks the optimal complexity
- Brute force is far too slow
- Standard patterns don’t directly apply — you must combine 2–3
- Implementation is non-trivial but not the bottleneck
Scoring weights: Optimization (#4), Correctness (#5), Complexity (#6), Code Quality (#7) are key. Production / tradeoff dimensions are not relevant — these are not asked.
Pick One Problem
(Pick at random for self-mock. With a partner, they choose.)
Problem A — Maximum Subarray with At Most K Replacements
Given an array a of integers and integer k, you may replace at most k elements with any value. Find the maximum possible sum of any contiguous subarray of the resulting array.
Constraints: 1 ≤ |a| ≤ 2×10^5. -10^9 ≤ a[i] ≤ 10^9. 0 ≤ k ≤ |a|.
Examples:
a = [-3, 4, -2, 5, -1], k = 1 → 11 (replace -2 with, say, 10^9? No — wait)
(Note: replacement values are unconstrained, so this trivializes; the actual problem variant is: removals, or replacements must be 0, or replacements use given budget. The interviewer specifies. For self-mock, use: “replace at most k elements with 0” — then it becomes a real DP.)
Problem B — Count Subarrays with Median ≥ X
Given array a (distinct) and threshold X. Count contiguous subarrays whose median is ≥ X. (Median of even-length: take the right-middle.)
Constraints: 1 ≤ |a| ≤ 2×10^5.
Insight: map each element to +1 if ≥ X, else -1. A subarray’s median is ≥ X iff its sum is positive (for odd length) or ≥ 0 (for even length with right-middle). Reduces to counting subarrays with prefix sum differences satisfying inequalities — Fenwick tree.
Problem C — Minimum Cost to Make Array Strictly Increasing
Given array a, you may increase any element by 1 at cost 1 (cannot decrease). Find minimum total cost to make a strictly increasing.
Constraints: 1 ≤ |a| ≤ 3000. Values fit in int64.
Insight: strict-increasing ↔ define b[i] = a[i] - i; then b must be non-decreasing. Reduces to “min cost to make array non-decreasing using only increases” = sum(max(0, prefix_max - b[i])). Wait — that’s not quite right because we can only increase. Final formula: walk left to right, maintain prev = max(prev + 1, a[i]), cost += prev - a[i].
Problem D — Range Sum with Updates and Range Adds (LC 307 + lazy)
Implement: update(i, x), range_add(l, r, x), query_sum(l, r). All in O(log N).
Constraints: 1 ≤ N ≤ 10^5. ≤ 10^5 operations.
Tool: segment tree with lazy propagation (Phase 3 Lab 02).
Problem E — Maximum XOR of Two Numbers (LC 421)
Given an array a, find max(a[i] XOR a[j]) over all i < j. O(N · 32) time.
Insight: binary trie of all numbers; for each number, greedily traverse to find the maximally-different other number.
Expected Communication Style
For competitive mocks, communication is light but precise:
- Restate in one sentence.
- Ask 1–2 surgical clarifying questions (constraints, distinct/duplicate, output format).
- State a brute force with complexity — proves you understand the problem.
- Think aloud about reductions or patterns. (“Median question; +1/-1 transformation; subarray sum; Fenwick.”)
- State the optimal complexity and key insight before coding.
- Code carefully and minimally. Competitive code can sacrifice some readability for brevity; don’t sacrifice correctness.
- Test 1–2 cases including a non-obvious one.
There is no “production discussion” in this format. The interviewer cares about the algorithm and the implementation.
Common Failure Modes
- Couldn’t reach the insight in 60 min. Submitted the brute force. Pass-ish for the optimization dimension only if the brute is correct.
- Reached the insight but the implementation has bugs that take 20+ min to fix. Need to drill segment tree / Fenwick / DP from Phase 3.
- Stuck on the wrong approach for 30+ min. Senior signal: pivot quickly when an approach doesn’t pan out. Articulate the pivot.
- Forgot the standard library tool. (Python
bisect, C++lower_bound, etc.) — costs implementation time. - No tests because “I’m confident.” Competitive code is wrong constantly; verify against brute force.
Passing Bar
- Total score: 49/70 (average 3.5) — lower than other mocks because some dimensions don’t apply
- Optimal complexity reached (or a serious near-optimal attempt with clear gap analysis)
- Correct on given examples + at least one boundary case
- Time ≤ 90 min
- Algorithm articulated with insight
Follow-up Questions
Competitive-style follow-ups are harder algorithm variants:
For A:
- Generalize: replace ≤ k elements with values from a given set. → DP becomes more state-heavy.
- Output the actual subarray and the replacements. → DP with parent pointers.
For B:
- Median strictly > X. → Adjust the +1/-1 mapping for equality.
- K-th smallest in every subarray. → Much harder; persistent data structures or offline processing.
For C:
- Allow decreases at cost 1 too. → Now O(N log N) using slope trick or O(N²) DP.
- Strictly increasing AND in [L, R] for each element. → Constrained version; more careful greedy.
For D:
- Add range assign as well as range add. → Two lazy tags; non-trivial composition.
- Range mode (most frequent element). → Much harder; Mo’s algorithm.
For E:
- Max XOR of any triple. → Open problem in some formulations; brute O(N²) over pairs + trie for third.
- Max XOR with values ≤ K (subset). → Persistent trie indexed by element index.
Required Tests
- All given examples
- Boundary: N = 1, N = max
- Adversarial: all same values, sorted, reverse-sorted
- One stress test against the brute force if time permits (mandatory if you suspect a bug)
Required Complexity Explanation
- Time, with reasoning
- Space, with reasoning
- Bound is tight or improvable?
- For N = 2×10^5 with O(N log N), expected runtime in seconds (typically < 1 sec)
Self-Evaluation Template
Mock 12 — Competitive Style
Date: _______
Problem: _______
Time: ___ / 90 min
Scores (1–5):
___ Total /70 (note: Tradeoff/Production are N/A; weighted out)
Time to insight: _____ min
Time to first correct implementation: _____ min
Bugs found and fixed: ___
Did I pivot from a wrong approach? Y/N (at minute ___)
Action item:
What to Do If You Fail
- Couldn’t reach the insight: This is a long-term gap, not a one-mock fix. Solve 30+ Codeforces Div 2 D problems (or LC hards tagged “competitive”) over 2–4 weeks.
- Reached insight, implementation buggy: Drill Phase 3 (advanced data structures) — your fundamentals leak.
- Bombed time management: Practice with stricter timers (45 min for problems you’ve already seen).
- Pass twice consecutively, on different problems, to consider this level handled.
After All 12 Mocks
When you have passed all 12 mocks twice consecutively each, return to the READINESS_CHECKLIST to verify the overall pipeline. The mocks alone do not certify readiness — they verify performance ability. Real interviews additionally test consistency over many rounds and behavioral signals.
Most candidates do not need to pass all 12. Pass the mocks corresponding to your target role:
- FAANG SWE-II: mocks 01–06
- FAANG Senior: mocks 01–07, plus 09 (language)
- FAANG Staff / Principal: mocks 01–10 except 12
- Quant / HFT / Compiler: mocks 01–04, 09, 11, 12 (heavy on competitive)
- Backend / Platform (Stripe, Snowflake, Confluent): mocks 01–08, 10, 11
Phase 12 — Grandmaster
Read this before doing anything in this phase.
This phase covers topics that are not required for 99% of interviews, including senior and staff roles at top FAANG companies. The content here is for a narrow set of candidates:
- ICPC World Finals competitors
- Codeforces red / IGM
- Quant developers at Jane Street, Hudson River, Two Sigma, Citadel for the most algorithmic roles
- Compiler engineers at LLVM, GCC, Intel, NVIDIA
- Database engineers at the algorithm-heavy companies (Snowflake, Databricks query optimizer, CockroachDB, TimescaleDB)
- Cryptography / coding theory researchers
- A few specific roles at Google Research, DeepMind, OpenAI infrastructure
If you are not on this list, skip this phase entirely. Time spent here is time not spent on Phase 10 (testing/debugging) and Phase 11 (mocks), which will show up in your interviews.
When to Use This Phase
Use this phase if all are true:
- You have already completed Phases 1–11 and are passing the mocks at your target level.
- You are interviewing for one of the roles listed above.
- The job description explicitly mentions ICPC, competitive programming, max flow, suffix structures, FFT, or “research-grade algorithms.”
- You have at least 3 months before your interview.
If any of these are false, stop. Go back to Phase 10 or 11.
When to Skip This Phase
Skip this phase if any is true:
- You are interviewing for SWE-II / E4 / SDE2 or below.
- You are interviewing for generic senior or staff backend at FAANG (mocks 06–08 cover what’s actually asked).
- You have less than 3 months before your interview.
- You are still failing Phase 11 mocks at your level — those are higher leverage.
- You are an SRE, mobile engineer, frontend engineer, ML engineer (non-research), or data engineer.
The opportunity cost is real. Each lab here takes a week. That week is better spent on Phase 11 mock attempts for the vast majority of candidates.
What’s In This Phase
The labs cover algorithms and data structures that appear on Codeforces / ICPC / IOI and almost nowhere else:
| Lab | Topic | When it appears |
|---|---|---|
| 01 | Max Flow (Dinic) | Quant, compiler, graph-heavy research |
| 02 | Bipartite Matching (Hopcroft-Karp) | Assignment problems; some quant |
| 03 | Heavy-Light Decomposition | ICPC, very rare in industry |
| 04 | Centroid Decomposition | ICPC, rare in industry |
| 05 | Suffix Automaton | String-heavy research, bioinformatics |
| 06 | Advanced DP Optimization (CHT, Knuth, D&C) | Quant, compiler (loop scheduling) |
| 07 | FFT / Polynomial Multiplication | Cryptography, signal processing, some compiler |
| 08 | Advanced Geometry (convex hull, intersections) | Geometric computing, games, CAD |
| 09 | ICPC Contest Simulation | Competitive programming only |
| 10 | Inclusion-Exclusion, Burnside | Combinatorics-heavy research |
| 11 | Suffix Array & Kasai LCP | String-heavy research, bioinformatics |
| 12 | Aho-Corasick Automaton | Multi-pattern scanning (security, bioinformatics) |
| 13 | Min-Cost Max-Flow | Assignment/matching with costs; OR teams |
| 14 | Sprague-Grundy Game Theory | Codeforces regular; quant brainteasers |
| 15 | Persistent Segment Tree | CP; versioned range queries |
Each lab has the standard 23-section format plus an extra “When to Skip This Topic” section right after Interview Context, so you can opt out of individual labs.
How to Use This Phase
- Read this README in full.
- Look at the target job descriptions you’re applying to. Search them for the specific keywords (max flow, suffix automaton, etc.).
- If you find a match, do that specific lab. If not, skip.
- Doing the whole phase end-to-end is rarely the right call. Cherry-pick.
Realistic Expectations
Even if you do this phase, you may never encounter these topics in an interview. The value is:
- Confidence signal — knowing these exist and roughly how they work lets you say “I’m familiar with Dinic’s max flow” if it ever comes up.
- Insight transfer — understanding centroid decomposition deepens your tree intuition for problems you will see.
- Specific roles — if you’re applying to a quant fund’s algo research team, expect this material.
This phase is intentionally not graded against the same passing bar as other phases. It’s read-only intellectual investment for a small group.
What This Phase Is Not
This phase is not:
- A prerequisite for Phase 11.
- Required for any FAANG interview.
- A signal of seniority.
- Going to help you with system design.
If you’re using this phase to procrastinate the harder thing (Phase 10 testing labs, Phase 11 mocks), stop. That’s the actual failure mode, and the only one of consequence.
After This Phase
If you complete the relevant labs, you have what most ICPC mid-rank teams have. You’re prepared for the narrow algorithmic interviews. You’re not more prepared for normal FAANG interviews than someone who did Phase 11 twice.
Return to Phase 11 for mock-12 (competitive style) reps, then to your job search.
Lab 01 — Max Flow (Dinic’s Algorithm)
Goal
Implement Dinic’s algorithm for maximum flow on a directed graph with capacities, achieving O(V²·E) worst case and near-linear in practice. Apply it to a real interview-style problem (Maximum Students Taking Exam, LC 1349) by reducing to max flow / bipartite matching.
Background
Maximum flow is the foundational network flow problem: given a source s and sink t in a directed graph with edge capacities, find the maximum rate at which “flow” can travel from s to t respecting capacities.
Key algorithms:
- Ford-Fulkerson (1956): generic augmenting-path framework. Complexity depends on path choice.
- Edmonds-Karp (1972): BFS for shortest augmenting path. O(V·E²).
- Dinic (1970): level graphs + blocking flows. O(V²·E) general; O(E·√V) for bipartite matching.
- Push-relabel (Goldberg-Tarjan, 1986): faster in practice for dense graphs. O(V²·√E) with FIFO.
Why Dinic dominates in practice: the level graph constraint (only follow edges from level i to level i+1 in BFS layering) prunes the search dramatically. For random graphs, near-linear.
Interview Context
Max flow is asked in:
- ICPC regionals/world finals (universal)
- Quant developer rounds at funds that care about assignment-style problems
- A handful of Google L6+ research interviews
- Rare appearances at Snowflake / Databricks query optimizer roles (max-flow underpins some join reordering heuristics)
- Never in standard FAANG SWE interviews
If asked, expect to either implement Dinic from scratch OR identify that a problem reduces to max flow and explain the reduction (more common than full implementation).
When to Skip This Topic
Skip if any of these are true:
- You are not interviewing for quant / research / ICPC-adjacent roles
- You have not memorized the basic Ford-Fulkerson framework yet
- You have less than 4 weeks for this phase
The reduction skill (recognizing a problem as max flow) is more valuable than memorizing the implementation. If you have only a few days, study reductions and skip the implementation.
Problem Statement
Maximum Students Taking Exam (LeetCode 1349, Hard).
Given an m × n classroom matrix where each cell is either ‘.’ (good seat) or ‘#’ (broken). Place students such that no student can cheat — a student can cheat off any immediately adjacent student in the same row OR diagonally in front (one row earlier, column ±1). Maximize the number of students seated.
seats = [["#",".","#","#",".","#"],
[".","#","#","#","#","."],
["#",".","#","#",".","#"]]
output = 4
Constraints
- 1 ≤ m, n ≤ 8 (small grid — but max-flow approach generalizes)
- Up to 64 seats
- Wall-clock target: < 100ms
Clarifying Questions
- Can a student cheat off the seat directly in front (same column, previous row)? (No — only diagonal-front and same-row-adjacent.)
- Are broken seats unavailable for sitting? (Yes — ‘#’ cannot hold a student.)
- Is the grid always rectangular? (Yes.)
Examples
Example 1
seats = [[".","#","."],
["#",".","#"],
[".","#","."]]
Conflict graph: every ‘.’ conflicts with diagonal-front + same-row-adjacent. Maximum independent set = 4 (the corners).
Example 2
seats = [["."]]
Trivial: 1.
Example 3 (boundary)
seats = [[".",".",".","..."]] # single row, all good
Same-row-adjacency means max alternating = ⌈n/2⌉.
Brute Force
Try every subset of good seats; check no two are in conflict; track max. O(2^k · k²) where k = number of good seats. For 8×8 = 64, infeasible.
Brute Force Complexity
- Time: O(2^k · k²) — fails for k > ~20.
- Space: O(k) for current subset.
Optimization Path
Observation 1: this is maximum independent set on a conflict graph, which is NP-hard in general.
Observation 2: but our conflict graph is bipartite! Color seats by column parity (even/odd columns). All conflicts are between an even column and an odd column (same-row-adjacent: differs by 1; diagonal: also differs by 1). So no conflicts within the even-column set or within the odd-column set.
Observation 3: max independent set on a bipartite graph = total vertices − max matching (König’s theorem). So we compute max bipartite matching, which is solvable in polynomial time via max flow.
This is the canonical reduction trick.
Final Expected Approach
- Build the bipartite graph: left = good seats in even columns, right = good seats in odd columns. Edge between two if they conflict.
- Add source
s→ all left nodes (cap 1), all right nodes → sinkt(cap 1), all conflict edges left→right (cap 1). - Run Dinic to compute max flow = max matching.
- Answer = (total good seats) − (max matching).
Data Structures
- Adjacency list with edge-index representation (each edge stores
to,cap,rev-index for the reverse edge) - BFS level array
- DFS iterator per node (incremented across calls to skip dead branches)
- Queue for BFS
Correctness Argument
- Bipartite: any conflict involves columns differing by 1, hence different parities.
- König: in bipartite, |min vertex cover| = |max matching|; |max independent set| = |V| − |min vertex cover|. So |MIS| = |V| − |max matching|.
- Dinic correctness: Ford-Fulkerson framework with augmenting paths; terminates when no augmenting path exists; gives optimal flow by max-flow min-cut theorem.
- Reduction: max matching via max flow is exact when all edge capacities are 1 and source/sink edges all have capacity 1.
Complexity
- Dinic on bipartite (unit-capacity) graphs: O(E·√V) — the Hopcroft-Karp bound.
- For LC 1349: V ≤ 64, E ≤ 64 × 4 = 256. Trivial.
Implementation Requirements
class Dinic:
def __init__(self, n):
self.n = n
self.graph = [[] for _ in range(n)]
def add_edge(self, u, v, cap):
self.graph[u].append([v, cap, len(self.graph[v])])
self.graph[v].append([u, 0, len(self.graph[u]) - 1])
def _bfs(self, s, t):
self.level = [-1] * self.n
self.level[s] = 0
q = deque([s])
while q:
u = q.popleft()
for v, cap, _ in self.graph[u]:
if cap > 0 and self.level[v] < 0:
self.level[v] = self.level[u] + 1
q.append(v)
return self.level[t] >= 0
def _dfs(self, u, t, pushed):
if u == t: return pushed
while self.it[u] < len(self.graph[u]):
e = self.graph[u][self.it[u]]
v, cap, rev = e
if cap > 0 and self.level[v] == self.level[u] + 1:
d = self._dfs(v, t, min(pushed, cap))
if d > 0:
e[1] -= d
self.graph[v][rev][1] += d
return d
self.it[u] += 1
return 0
def max_flow(self, s, t):
flow = 0
while self._bfs(s, t):
self.it = [0] * self.n
while True:
f = self._dfs(s, t, float('inf'))
if f == 0: break
flow += f
return flow
Then build the bipartite graph and call.
Tests
- LC 1349 given examples
- All ‘#’ grid → 0
- All ‘.’ grid of size 1×n → ⌈n/2⌉
- 8×8 all ‘.’ (max stress) → ~32 (need to compute)
- Single column m×1 all ‘.’ → m (no same-row conflicts within a column)
Follow-up Questions
- Generalize to weighted matching (different students have different “value”; maximize total value). → Min-cost max flow.
- Add a constraint that some seats are mandatory. → Force-include via lower-bound constraints.
- m, n up to 50. → Same algorithm; check timing.
- Stream of conflicts; dynamic max matching. → Active research area.
- Distinct from LC 1349, prove the bipartite reduction is tight.
Product Extension
Real systems that use max flow / bipartite matching:
- Ride-sharing assignment (drivers ↔ requests)
- Ad auction allocation (advertisers ↔ slots)
- Resource scheduling (tasks ↔ machines)
- Compiler register allocation (variables ↔ registers; with constraints)
- DNA sequencing assembly
Language/Runtime Follow-ups
- Python: recursion depth limit; switch to iterative DFS for large V.
- C++: much faster; competitive programmers use this exclusively.
- Go/Java: stack size for recursive DFS may need explicit increase.
Common Bugs
- Forgot the reverse edge. Flow networks require residual graph; no reverse = wrong answer.
- Reverse edge with cap 0 but didn’t account for it during DFS: correct — that’s by design.
- BFS level updated multiple times. Use the first level reached only.
- DFS iterator reset every call to _dfs. Should persist within a phase (the
self.it[u]trick). - Bipartite assumption violated: if you add an edge between two left nodes, the reduction breaks. Verify.
- Source/sink indices clash with vertex IDs. Use distinct numbering scheme.
Debugging Strategy
- Print the level graph after each BFS.
- Print augmenting path found in each DFS.
- Verify flow conservation at intermediate nodes after termination.
- Sanity check: max flow ≤ min(deg(s), deg(t)).
Mastery Criteria
- Implement Dinic from memory in ≤ 25 min in your primary language
- Identify max-flow reductions in problems that don’t mention “flow” or “matching” explicitly
- Explain why LC 1349 reduces to bipartite matching, citing König
- State Hopcroft-Karp’s complexity advantage on bipartite unit-cap graphs
- Estimate runtime for a given V, E
- Implement min-cost max flow if asked (separate algorithm — SPFA + Dinic)
Lab 02 — Bipartite Matching (Hopcroft-Karp)
Goal
Implement Hopcroft-Karp for maximum bipartite matching, achieving O(E·√V), and understand when it beats general max flow.
Background
Bipartite matching: given a bipartite graph (vertices split into L and R, edges only between L and R), find the largest set of edges with no shared endpoint.
- Naïve augmenting path: O(V·E). For each unmatched left vertex, find an augmenting path via DFS.
- Hopcroft-Karp (1973): find multiple vertex-disjoint shortest augmenting paths per phase. O(E·√V).
The √V comes from the fact that after √V phases, all remaining augmenting paths have length > √V, and there can be at most √V such paths.
Hopcroft-Karp is a special case of Dinic’s algorithm applied to a unit-capacity bipartite flow network. If you have Dinic, you have Hopcroft-Karp.
Interview Context
Bipartite matching shows up in:
- ICPC (constant)
- Assignment problems (jobs ↔ workers)
- Some quant interviews on portfolio matching
- Compiler register coalescing
- Almost never in standard FAANG interviews
Recognizing that a problem is bipartite matching is the high-leverage skill; the algorithm is well-known.
When to Skip This Topic
Skip if any of these are true:
- You’ve already done Lab 01 (Dinic handles bipartite matching as a special case)
- You’re not targeting competitive / quant / assignment-heavy roles
- You have less than 2 weeks for this phase
The reduction skill is what matters. Skip the algorithm if you can recognize the reduction and use Dinic.
Problem Statement
Maximum Bipartite Matching.
Given a bipartite graph with L left vertices, R right vertices, and M edges, find the maximum matching size.
Variant: Job Assignment. N workers, N jobs. Worker i can do a subset of jobs. Assign each worker to at most one job, each job to at most one worker. Maximize assignments.
Constraints
- 1 ≤ L, R ≤ 10^5
- 1 ≤ M ≤ 10^6
- Wall-clock: < 1 sec
Clarifying Questions
- Are the partitions L and R given, or do I need to detect bipartiteness? (Usually given.)
- Are edges weighted? (No — that’s a different problem: Hungarian algorithm or min-cost max flow.)
- Output the matching or just the size? (Both versions are common.)
Examples
L = {1, 2, 3}, R = {a, b, c}
Edges: 1-a, 1-b, 2-b, 3-c
Max matching: {1-a, 2-b, 3-c}, size = 3
L = {1, 2}, R = {a, b}
Edges: 1-a, 2-a
Max matching: {1-a} or {2-a}, size = 1
Brute Force
Try all subsets of edges; check that no vertex appears twice; track max. O(2^M · M).
Better naïve: for each left vertex in order, DFS to find an augmenting path. O(V·E).
Brute Force Complexity
- Subsets: O(2^M)
- Per-vertex DFS: O(V·E). Acceptable for V·E ≤ ~10^7.
Optimization Path
Hopcroft-Karp:
- Phase 1: BFS from all unmatched left vertices, computing layers in the residual graph.
- Phase 2: DFS from each unmatched left vertex, finding vertex-disjoint shortest augmenting paths.
- Repeat until no augmenting path exists.
The phase count is O(√V), giving total O(E·√V).
Final Expected Approach
class HopcroftKarp:
def __init__(self, left_size, right_size):
self.L = left_size
self.R = right_size
self.adj = [[] for _ in range(left_size)]
self.NIL = -1
def add_edge(self, u, v):
self.adj[u].append(v)
def _bfs(self):
q = deque()
self.dist = [float('inf')] * self.L
for u in range(self.L):
if self.match_L[u] == self.NIL:
self.dist[u] = 0
q.append(u)
found = False
while q:
u = q.popleft()
for v in self.adj[u]:
pair = self.match_R[v]
if pair == self.NIL:
found = True
elif self.dist[pair] == float('inf'):
self.dist[pair] = self.dist[u] + 1
q.append(pair)
return found
def _dfs(self, u):
for v in self.adj[u]:
pair = self.match_R[v]
if pair == self.NIL or (self.dist[pair] == self.dist[u] + 1 and self._dfs(pair)):
self.match_L[u] = v
self.match_R[v] = u
return True
self.dist[u] = float('inf')
return False
def max_matching(self):
self.match_L = [self.NIL] * self.L
self.match_R = [self.NIL] * self.R
matching = 0
while self._bfs():
for u in range(self.L):
if self.match_L[u] == self.NIL and self._dfs(u):
matching += 1
return matching
Data Structures
- Adjacency list (left → list of right)
match_L[u],match_R[v]: current partner or NILdist[u]: BFS layer of left vertex- Queue for BFS
Correctness Argument
- Augmenting path: path alternating unmatched-matched-unmatched… edges, starting and ending at unmatched vertices. Flipping the edges along the path increases matching size by 1.
- Berge’s theorem: matching is maximum iff no augmenting path exists.
- Hopcroft-Karp: in each phase, finds a maximal set of vertex-disjoint shortest augmenting paths. After √V phases, no short augmenting paths remain; at most √V remaining ones contribute one each.
Complexity
- Time: O(E · √V)
- Space: O(V + E)
For V = 10^5, E = 10^6: roughly 10^7.5 ≈ 3·10^7 ops — well under 1 sec in C++.
Implementation Requirements
- Use BFS to detect all unmatched left vertices and compute layers
- DFS must respect layer constraint (
dist[pair] == dist[u] + 1) - Set
dist[u] = infinityon failed DFS to prune subsequent visits - Repeat until BFS finds no augmenting path
Tests
- Empty graph → 0
- Single edge → 1
- Complete bipartite K_{n,n} → n
- Star (1 left, n right) → 1
- Path 1-a-2-b-3 → 2
Follow-up Questions
- Weighted matching (maximize sum of edge weights, not count). → Hungarian algorithm O(V³) or min-cost max flow.
- Online matching (edges arrive one at a time). → Greedy is 1/2-competitive; ranking is (1 − 1/e)-competitive.
- Stable matching (Gale-Shapley). → Different problem; preferences instead of binary edges.
- Edge-disjoint paths from s to t. → Reduces to max flow with all capacities 1.
Product Extension
- Ad-slot allocation (advertisers ↔ impressions)
- Ride-sharing dispatch (drivers ↔ riders)
- Course allocation (students ↔ classes with capacity)
- Resource scheduling
Language/Runtime Follow-ups
- C++: competitive programmers use a tight 50-line version
- Python: recursion depth and constant factor make this borderline at V = 10^5; use sys.setrecursionlimit or iterative
- Rust: ownership makes the in-place matching arrays a small wrestle
Common Bugs
- Forgot to reset
dist[u] = infinityon DFS failure. Re-explores dead ends; slow. - DFS doesn’t respect the layer constraint. Same as Ford-Fulkerson; loses √V factor.
match_Landmatch_Rout of sync. Update both atomically.- NIL value collision with real vertex 0. Use -1 or a sentinel.
Debugging Strategy
- After each phase, print matching size and BFS layer counts
- Verify
match_L[u] == viffmatch_R[v] == u - Augmenting path should alternate matched/unmatched
Mastery Criteria
- Implement Hopcroft-Karp in ≤ 30 min from memory
- Explain why √V phases suffice (sketch of proof)
- Identify when bipartite matching applies to a problem stated in domain terms
- State the difference between bipartite matching and Hungarian (weighted)
- Estimate runtime for given V, E
Lab 03 — Heavy-Light Decomposition
Goal
Implement Heavy-Light Decomposition (HLD) for answering path queries on a tree in O(log² N) per query (or O(log N) with a segment tree per chain).
Background
HLD partitions tree edges into “heavy” and “light”:
- For each non-leaf vertex, the edge to its child with the largest subtree is heavy.
- All other edges are light.
Property: any root-to-leaf path uses O(log N) light edges (because each light edge halves the subtree size). Hence any tree path can be decomposed into O(log N) heavy chains.
Each heavy chain is contiguous in a DFS order, so we can maintain a segment tree over the DFS array and do O(log N) work per chain → O(log² N) per path query.
Originally from Sleator and Tarjan’s link-cut tree work (1983); HLD as the standalone offline technique attributed to Sleator & Tarjan / popularized via ICPC.
Interview Context
HLD appears in:
- ICPC (frequently in the path-query category — QTREE problem on SPOJ is canonical)
- Almost never in industry interviews
- A handful of database/optimizer roles touch on tree-DP that HLD speeds up
- A very rare appearance in compiler dominator-tree manipulation
When to Skip This Topic
Skip if any of these are true:
- You are not targeting ICPC
- You are not interviewing at a research/algorithms team
- You don’t already understand segment trees deeply (Phase 3 Lab 01–02)
- You have less than 3 weeks for this phase
HLD is implementation-heavy. Getting it right in interview time requires ≥ 30 hours of practice.
Problem Statement
Path Query (QTREE-style).
Given a tree of N vertices, each edge has a weight. Support two operations:
change(i, w): change edgei’s weight towquery(u, v): return the maximum edge weight on the path fromutov
Constraints
- 1 ≤ N ≤ 10^5
- 1 ≤ Q ≤ 10^5
- Edge weights ≤ 10^9
Clarifying Questions
- Is the tree rooted or unrooted? (Pick a root; doesn’t matter.)
- Queries on edges or on vertices? (Edges here; vertex variant is simpler.)
- Multiple components possible? (No — single tree.)
Examples
N = 4
Edges: (1,2,3), (2,3,4), (2,4,5)
query(1, 3): path 1→2→3, max edge = 4
change(1, 10): edge (1,2) now has weight 10
query(1, 4): path 1→2→4, max edge = 10
Brute Force
For each query, find the path via LCA (precompute LCA in O(log N)), then walk the path and check each edge. O(N) per query → O(NQ) total.
For N=Q=10^5, that’s 10^10 — TLE.
Brute Force Complexity
- Time: O(NQ)
- Space: O(N) for tree + LCA tables
Optimization Path
The path between u and v decomposes as u → LCA → v. With HLD, each leg traverses O(log N) heavy chains; each chain is a contiguous range in our DFS order; we query/update with a segment tree.
Final Expected Approach
- DFS 1 (size/parent/depth): compute subtree size, parent, depth.
- DFS 2 (HLD): for each vertex, identify heavy child (largest subtree). Assign
heavy[u]. Walk heavy chains, assigninghead[u]and apos[u]= position in DFS-order array. - Build segment tree over the DFS array, indexed by
pos[u]. - Query(u, v): while
head[u] != head[v], raise the deeper one to its head’s parent, querying the segment tree for the chain segment. Then query the segment betweenuandvon the final shared chain.
def query_path(u, v):
res = 0
while head[u] != head[v]:
if depth[head[u]] < depth[head[v]]:
u, v = v, u
res = max(res, seg_tree.query(pos[head[u]], pos[u]))
u = parent[head[u]]
if u == v: return res
if depth[u] > depth[v]: u, v = v, u
# Edge weights stored at the deeper endpoint; skip u's contribution
res = max(res, seg_tree.query(pos[u] + 1, pos[v]))
return res
Data Structures
- Tree adjacency
- Arrays:
parent,depth,size,heavy,head,pos - Segment tree over
pos-indexed values - DFS for both passes (iterative for large N to avoid stack overflow)
Correctness Argument
- Light edge bound: if (u, v) is a light edge, size(v) ≤ size(u)/2. So any root-to-leaf path crosses O(log N) light edges.
- Heavy chain decomposition: path u→v splits at LCA; each leg traverses chains separated by light edges; ≤ O(log N) chains.
- Segment tree on chain: chains contiguous in DFS order; standard range query.
- Edge-on-vertex convention: store each edge’s weight at its deeper endpoint, so a vertex query at position
ireturns the parent-edge weight.
Complexity
- Preprocessing: O(N)
- Per query/update: O(log² N) — segment tree query O(log N), times O(log N) chains
- Total: O((N + Q) log² N)
Implementation Requirements
- Iterative DFS for N > 10^4 to avoid Python recursion limit (or C++ stack)
- Segment tree must support point update + range max query
- Careful indexing: edge i stored at vertex
deeper_endpoint(i) - Handle u == v case in query (return 0 or identity)
Tests
- Linear tree (path graph): every edge is on the chain; ≤ 1 chain transition per query
- Balanced binary tree: ≈ log N chain transitions
- Star tree (1 center, N leaves): only one heavy edge; all other queries are 1-chain
- Update + query interleaved
- Query on same vertex (u == v)
- Path including the root
Follow-up Questions
- Sum on path instead of max. → Same structure, segment tree stores sums.
- Update path (range add) instead of point. → Segment tree with lazy propagation; same chain decomposition.
- Subtree query (sum/max in subtree of u). → Even simpler: subtree is contiguous in DFS, single range query.
- LCA only. → Tarjan offline O((N+Q)α(N)) or binary lifting O(log N).
- Dynamic tree (edges added/removed). → Link-cut trees (Sleator-Tarjan); much harder.
Product Extension
- Network routing on hierarchical topologies
- File-system path queries (organizations with deep trees)
- Phylogenetic tree analysis
- Decision tree updates in some ML systems
Language/Runtime Follow-ups
- C++: standard ICPC implementation; ~150 lines
- Python: slow constant factor; use iterative DFS; can pass for N ≤ 10^4 comfortably
- Java: stack depth fine for N ≤ 10^5; constant factor OK
Common Bugs
- Recursive DFS for N = 10^5: stack overflow in many languages.
- Forgot edge-vertex mapping convention: off-by-one when querying final segment.
- Heavy child computed wrong: must be the child with the largest subtree, not the deepest.
head[u]not propagated through the chain. All vertices on the same heavy chain should sharehead.- Segment tree off-by-one between
pos[u]andpos[v]. - Update at the root edge. Root has no parent edge; verify boundary handling.
Debugging Strategy
- Print the chain decomposition: list all chains as
[head, ..., tail] - For a path query, log each chain segment queried
- Verify against brute force on N ≤ 20
- Visualize: color heavy edges red, light edges black; should see O(log) light edges on long paths
Mastery Criteria
- Implement HLD + segment tree from scratch in ≤ 90 min
- Explain why O(log N) chains per path
- Handle edge-weight vs vertex-weight variants
- Combine with lazy propagation for range updates
- State complexity precisely: O((N + Q) log² N) or O(log N) with chain-segtrees
Lab 04 — Centroid Decomposition
Goal
Implement centroid decomposition for efficient tree queries — counting / aggregating over all paths in a tree in O(N log N) or O(N log² N).
Background
The centroid of a tree is a vertex whose removal leaves no subtree with more than N/2 vertices. Every tree has a centroid (sometimes two).
Centroid decomposition: recursively decompose the tree:
- Find centroid; process all paths passing through it
- Remove centroid; recurse on each remaining subtree
Recursion depth: O(log N) (each level halves subtree size). Total work per level: O(N) typically → O(N log N) total.
Originally developed for tree DP and offline path queries. Powerful technique for problems of the form: “count/sum over all pairs (u, v) in a tree with property P on the u-v path.”
Interview Context
Almost exclusively ICPC. Some appearances in:
- Quant algo research on tree models
- Phylogenetic inference (computational biology)
- A handful of compiler dominator-tree analyses
Industry interviews: near-zero.
When to Skip This Topic
Skip if any of these are true:
- You are not training for ICPC or competitive contests
- You haven’t done Lab 03 (HLD) — these are sibling techniques
- You don’t have 2+ weeks for the implementation practice
Centroid decomposition has a high “first implementation” cost. Don’t attempt without serious tree-DP fluency.
Problem Statement
Count Paths in Tree with Length ≤ K.
Given a tree of N vertices, edge weights w_e, and integer K, count the number of unordered pairs (u, v) such that the sum of edge weights on the path from u to v is ≤ K.
Constraints
- 1 ≤ N ≤ 5×10^4
- 1 ≤ K ≤ 10^9
- 1 ≤ w_e ≤ 10^4
Clarifying Questions
- Are weights positive? (Yes — required for the standard algorithm.)
- Count ordered or unordered pairs? (Unordered, exclude self-pairs.)
- Are edge weights integers? (Yes — convenient for sort/binary-search.)
Examples
Tree: 1-2 (w=2), 2-3 (w=1), 2-4 (w=3)
K = 4
Paths and lengths:
(1,2): 2 ✓
(1,3): 3 ✓
(1,4): 5 ✗
(2,3): 1 ✓
(2,4): 3 ✓
(3,4): 4 ✓
Answer: 5
Brute Force
For each unordered pair (u, v), compute path length (LCA + ancestor distances). O(N² log N).
For N = 5×10^4: 2.5×10^9 ops — TLE.
Brute Force Complexity
- Time: O(N² log N) for path length per pair
- Space: O(N) plus LCA tables
Optimization Path
Centroid decomposition shines for “paths through centroid” enumeration:
- A path between u and v either passes through the centroid c or lies entirely in one subtree (after c is removed)
- Paths through c: count by aggregating distances from c to every other vertex
- Paths in subtrees: handled recursively
Per centroid:
- BFS from c, recording dist(c, v) for every v in c’s connected component
- Sort distances per subtree
- Count pairs (u, v) with dist(c, u) + dist(c, v) ≤ K using two pointers
- Subtract pairs where u and v are in the same subtree (they would have been counted as paths through some other centroid)
Final Expected Approach
def centroid_decompose(root, K):
n = len(adj)
removed = [False] * n
size = [0] * n
total = 0
def calc_size(u, parent):
size[u] = 1
for v, _ in adj[u]:
if v != parent and not removed[v]:
calc_size(v, u)
size[u] += size[v]
def find_centroid(u, parent, tree_size):
for v, _ in adj[u]:
if v != parent and not removed[v] and size[v] > tree_size // 2:
return find_centroid(v, u, tree_size)
return u
def gather_dists(u, parent, d, out):
out.append(d)
for v, w in adj[u]:
if v != parent and not removed[v]:
gather_dists(v, u, d + w, out)
def count_pairs(dists, K):
dists.sort()
i, j = 0, len(dists) - 1
c = 0
while i < j:
if dists[i] + dists[j] <= K:
c += j - i
i += 1
else:
j -= 1
return c
def decompose(u):
nonlocal total
calc_size(u, -1)
c = find_centroid(u, -1, size[u])
all_dists = [0]
for v, w in adj[c]:
if not removed[v]:
sub = []
gather_dists(v, c, w, sub)
# subtract pairs within this subtree
total -= count_pairs(sub[:], K)
all_dists.extend(sub)
total += count_pairs(all_dists, K)
removed[c] = True
for v, _ in adj[c]:
if not removed[v]:
decompose(v)
decompose(root)
return total
Data Structures
- Adjacency list (vertex → list of (neighbor, weight))
removed[v]: marks centroids removed from active treesize[v]: subtree size in current decomposition step- Distance lists per subtree
Correctness Argument
- Centroid existence: every tree has a centroid (induction on tree structure).
- Recursion depth O(log N): removing centroid leaves subtrees of size ≤ N/2.
- Pair counting via subtraction: a path (u, v) is counted exactly once — at the deepest centroid c that lies on path(u, v). The inclusion-exclusion (add all-vertices count, subtract per-subtree count) ensures each path-through-c is counted once.
- Two pointers for sum ≤ K: standard.
Complexity
- Time: O(N log² N) — O(log N) levels, O(N log N) per level (sort dominates)
- Space: O(N) for tree + O(N) for decomposition state
Implementation Requirements
- Iterative or carefully bounded recursive DFS (Python: 5×10^4 may need increased limit)
- Recompute
size[]for each subtree (in the recursive call); critical bug source - Two-pointer pair counting requires sorted distances
- The inclusion-exclusion trick is the conceptual core; verify on small cases
Tests
- Linear chain (path graph): N(N-1)/2 paths; verify against brute force
- Star tree: each pair sum is at most 2*max_weight
- Balanced binary tree
- N = 1 (no pairs)
- K = 0 with positive weights (only self-pairs; answer = 0)
- Very large K (all pairs counted)
Follow-up Questions
- Count paths with length exactly K. → Use hashmap of distances per subtree; sum complement counts.
- Sum of path lengths (rather than count). → Aggregate sums in addition to counts during two-pointer scan.
- XOR of edge weights instead of sum, equals K. → Replace sort/two-pointer with XOR trie.
- Online (tree mutating). → Much harder; use top trees or Euler-tour trees.
- K-th shortest path. → Different problem; rarely tractable on trees with centroid.
Product Extension
- Phylogenetics: counting pairs of species within evolutionary distance K
- Network distance queries on hierarchical trees
- Distance-based recommendation systems on tree-like ontologies
Language/Runtime Follow-ups
- Python: sort + two-pointer per level; constant factor is the killer. C++ recommended for N ≥ 10^4.
- C++: standard ICPC implementation; ~100 lines.
- Recursive DFS: centroid decomposition depth O(log N), but inner DFS depth O(N) — limit must accommodate.
Common Bugs
- Forgot to recompute
size[]for each subtree. Sizes from before removal are stale. - Centroid finder doesn’t follow the right child. Must descend toward the largest remaining subtree.
removed[v]check forgotten in DFS: revisits removed centroids.- Off-by-one in pair counting (counting self-pair). Handle separately.
- Inclusion-exclusion wrong sign. Add all, subtract per-subtree.
- Stack overflow on deep recursion. Convert inner DFS to iterative.
Debugging Strategy
- For small N, compare against brute force at each level
- Log the centroid chosen at each call
- Verify subtree sizes recomputed correctly (print before find_centroid)
- For two-pointer: print sorted distances and the (i, j) cursor trajectory
Mastery Criteria
- Implement centroid decomposition in ≤ 60 min from memory
- Explain the inclusion-exclusion trick for path counting
- Identify problems amenable to centroid decomposition (offline path queries on static tree)
- Distinguish from HLD: HLD is online with edge updates; centroid is offline/path-counting
- State complexity precisely: O(N log² N) typical
Lab 05 — Suffix Automaton
Goal
Build a Suffix Automaton (SAM) for a string, and use it to count the number of distinct substrings in O(N).
Background
A suffix automaton is the smallest DFA that accepts every suffix of a given string. Discovered by Blumer et al. (1985).
Key facts:
- O(N) states and O(N) transitions for a string of length N (over an alphabet)
- Each state corresponds to an equivalence class of right-extensions of substrings
- Distinct substring count = sum of
len[state] - len[link[state]]over all states (excluding the initial state)
It’s the most powerful string data structure in competitive programming. Sometimes compared with suffix arrays + LCP arrays (which solve many of the same problems with different constants).
Interview Context
- Heavy ICPC / Codeforces presence
- Bioinformatics / genome alignment research
- Rarely in industry; Bloomberg may ask, but usually accepts suffix array
- Cryptography (some sequence-counting problems)
When to Skip This Topic
Skip if any of these are true:
- You’re not targeting ICPC or string-research roles
- You haven’t learned suffix arrays yet (lower-hanging fruit; more interview-relevant)
- You don’t have 2+ weeks to internalize this
SAM is conceptually the deepest topic in this phase. The implementation is short; the understanding is hard. Don’t fake it.
Problem Statement
Count Distinct Substrings.
Given a string s, count the number of distinct non-empty substrings.
Example: s = "abc" → substrings {“a”, “b”, “c”, “ab”, “bc”, “abc”} → 6.
s = "aaa" → substrings {“a”, “aa”, “aaa”} → 3.
Constraints
- 1 ≤ |s| ≤ 10^6
- Lowercase English (or larger alphabet — affects transition storage)
- Wall-clock: < 1 sec
Clarifying Questions
- Empty substring counts? (Usually no.)
- Substrings or distinct substrings? (Distinct; non-distinct is trivial: N(N+1)/2.)
- Alphabet size? (26 for English; affects map-vs-array tradeoff.)
Examples
"abc" → 6 (a, b, c, ab, bc, abc)
"aaa" → 3 (a, aa, aaa)
"abab" → 7 (a, b, ab, ba, aba, bab, abab)
"" → 0
Brute Force
Generate all substrings, insert into a hash set, return size. O(N²) substrings, each of average length N/2 to hash → O(N³) worst case, O(N² log N) average. For N = 10^6: dead.
Brute Force Complexity
- Time: O(N²) to O(N³)
- Space: O(N²) for the set
Optimization Path
Build the suffix automaton:
- Each state represents an equivalence class of substring occurrences
- For each state
s(except the initial), the number of distinct substrings ending at this state’s set of right-positions islen[s] - len[link[s]] - Sum these for all states → distinct substring count
Final Expected Approach
class SuffixAutomaton:
def __init__(self):
self.size = 1
self.last = 0
self.len = [0]
self.link = [-1]
self.next = [{}]
def extend(self, c):
cur = self.size
self.size += 1
self.len.append(self.len[self.last] + 1)
self.link.append(-1)
self.next.append({})
p = self.last
while p != -1 and c not in self.next[p]:
self.next[p][c] = cur
p = self.link[p]
if p == -1:
self.link[cur] = 0
else:
q = self.next[p][c]
if self.len[p] + 1 == self.len[q]:
self.link[cur] = q
else:
clone = self.size
self.size += 1
self.len.append(self.len[p] + 1)
self.link.append(self.link[q])
self.next.append(dict(self.next[q]))
while p != -1 and self.next[p].get(c) == q:
self.next[p][c] = clone
p = self.link[p]
self.link[q] = clone
self.link[cur] = clone
self.last = cur
def count_distinct_substrings(self):
return sum(self.len[i] - self.len[self.link[i]] for i in range(1, self.size))
# Usage
sam = SuffixAutomaton()
for c in "abab":
sam.extend(c)
print(sam.count_distinct_substrings()) # 7
Data Structures
len[]: longest substring represented by each statelink[]: suffix link (analogous to failure link in Aho-Corasick)next[]: transition map per state (dict or array of 26)
Correctness Argument
The SAM construction is non-trivial. The key invariants:
- After processing prefix of length k, the SAM recognizes exactly the suffixes of that prefix
- Each state’s
len[s] - len[link[s]]counts the number of distinct substrings whose right-extension class is exactly this state - Summing across all non-initial states gives total distinct substrings
The cloning step (when len[p] + 1 != len[q]) splits a state to maintain the equivalence class property — without it, the automaton wouldn’t be canonical.
A rigorous proof is in Maxime Crochemore’s textbook. Accept the construction; verify on small cases.
Complexity
- Construction: O(N · |Σ|) with dict transitions, or O(N) amortized with array transitions
- Distinct substring count: O(N) after construction
- Space: O(N · |Σ|) worst case
Implementation Requirements
- Use
dictper state for arbitrary alphabets, or[None] * 26for English - Allocate state arrays incrementally (or pre-allocate 2N for safety)
- Cloning step is the most error-prone — verify on small cases
- Avoid recursion; SAM is naturally iterative
Tests
"a"→ 1"aa"→ 2"ab"→ 3"abc"→ 6"aaa"→ 3"abab"→ 7"abcabc"→ 15- Stress: random strings of N=100 against brute force
- Performance: N = 10^6 single character → finishes in < 1 sec
Follow-up Questions
- Find longest common substring of two strings. → Build SAM of one; walk through the other tracking match length.
- Number of occurrences of each substring. → Count terminal nodes (via topological sort of suffix link tree, then propagate).
- K-th lexicographically smallest substring. → DFS on SAM with character ordering + count of substrings reachable.
- Substring matching count for many queries. → Walk pattern in SAM; if it ends, answer is the size of its “endpos” set.
Product Extension
- Genome assembly: distinct k-mers, longest common substrings across reads
- Plagiarism detection
- Compression (LZ-family algorithms use suffix structures)
- Search engine n-gram indexing (less common; usually suffix array)
Language/Runtime Follow-ups
- C++: use
int next[][26]for the alphabet — fast. - Python: dict transitions; constant factor allows N ≤ 10^5 comfortably.
- Java: TreeMap or HashMap; arrays preferred for fixed alphabet.
Common Bugs
- Forgot to clone: when len[p]+1 != len[q], failing to clone breaks the automaton.
- Wrong update of
last: must always becur, notclone. - Suffix link of
curset incorrectly: subtle; verify against reference. - Used same dict reference for clone: must
dict(self.next[q])(copy). - Off-by-one in distinct substring sum: start from state 1, not 0 (state 0 is the initial state and represents the empty substring).
Debugging Strategy
- Print states with (len, link, transitions) after each
extendcall - Visualize the suffix-link tree (parent = link[state])
- Verify against brute force for N ≤ 20
- For the cloning step: log when it triggers and which state is split
Mastery Criteria
- Implement SAM construction in ≤ 45 min from memory (it’s short but error-prone)
- Explain the role of suffix links and the cloning step
- Apply SAM to: distinct substring count, occurrence count, LCS of two strings
- State complexity precisely (O(N) states, O(N|Σ|) transitions, O(N) construction with arrays)
- Distinguish SAM from suffix arrays in problem-applicability
Lab 06 — Advanced DP Optimization
Goal
Apply three classical DP optimization techniques — Convex Hull Trick (CHT), Knuth’s optimization, and Divide & Conquer DP — to reduce a polynomial-time DP from O(N²) or O(N³) to O(N log N) or O(N²).
Background
Many DPs have the form dp[i] = min_j (dp[j] + cost(j, i)) for various cost functions. The naive scan is O(N) per i, giving O(N²) total. Three techniques exploit structure in cost:
- Convex Hull Trick: when
cost(j, i) = a[j] * x[i] + b[j](linear inx[i]), the transitions form lines; the min is the lower envelope, queryable in O(log N) or O(1) per query. - Knuth’s optimization: when the cost satisfies the quadrangle inequality (
cost(a,c) + cost(b,d) ≤ cost(a,d) + cost(b,c)fora ≤ b ≤ c ≤ d), and optimal split points are monotonic. Reduces O(N³) to O(N²). - Divide & Conquer DP: when optimal split points are monotonic (
opt(i, j) ≤ opt(i+1, j)). Reduces O(KN²) to O(KN log N).
Interview Context
- Codeforces / ICPC: regular
- Quant: heavy presence in trading-cost optimization, risk allocation
- Compiler: loop scheduling sometimes uses CHT
- Database: query optimizer cost minimization (rare)
Almost never in standard interviews.
When to Skip This Topic
Skip if any of these are true:
- You aren’t already fluent in 1D and 2D DP (Phase 5 prerequisites)
- You’re not targeting ICPC / quant / compiler optimization roles
- You don’t have 2+ weeks to drill multiple variants
These are families of techniques; each requires several practice problems to internalize.
Problem Statement
Three variants, one for each technique:
Variant A — CHT (Convex Hull Trick)
You drive along a road with N houses. House i is at position x[i] (sorted). You can rent a car at house i for cost c[i] + d[i] * (x[j] - x[i]) if you drive from house i to house j > i. Starting at house 1, what’s the minimum cost to reach house N?
dp[j] = min over i < j of (dp[i] + c[i] + d[i] * (x[j] - x[i]))
= min over i < j of (d[i] * x[j] + (dp[i] + c[i] - d[i] * x[i]))
This is linear in x[j] — CHT applicable. O(N) or O(N log N) depending on whether x is sorted.
Variant B — Knuth’s Optimization
Optimal Binary Search Tree. Given keys with access probabilities, build a BST minimizing expected access cost.
dp[i][j] = min over i ≤ k ≤ j of (dp[i][k-1] + dp[k+1][j]) + sum(p[i..j])
Naive O(N³). Knuth: O(N²) if opt[i][j] is monotonic, which holds when cost is quadrangle-inequality compliant.
Variant C — Divide & Conquer DP
Minimum K Partitions. Partition array a[1..N] into exactly K contiguous segments, minimizing sum of “cost” of each segment, where cost(l, r) satisfies the monotonic-opt property.
dp[k][i] = min over j < i of (dp[k-1][j] + cost(j+1, i))
Naive O(KN²). D&C DP: O(KN log N).
Constraints
- A: 1 ≤ N ≤ 10^5
- B: 1 ≤ N ≤ 5×10^3
- C: 1 ≤ N ≤ 5×10^3, 1 ≤ K ≤ N
Clarifying Questions
A: Are x[i] strictly sorted? Are d[i] non-negative?
B: Are probabilities normalized? Distinct keys?
C: Is cost precomputable in O(1) after O(N²) prep? Quadrangle inequality verified?
Examples
A (CHT)
positions: [0, 5, 10, 20]
c = [0, 3, 2, _]; d = [1, 1, 2, _]
dp[1] = 0
dp[2] = 0 + 0 + 1*(5-0) = 5
dp[3] = min(0+0+1*10, 5+3+1*5) = 10
dp[4] = min(0+0+1*20, 5+3+1*15, 10+2+2*10) = 20 vs 23 vs 32 → 20
B (Knuth) — verify on a small probability vector.
C (D&C DP) — verify on contrived cost.
Brute Force
A: O(N²) DP scan. B: O(N³) standard. C: O(KN²) standard.
Brute Force Complexity
For N = 10^5 in A: O(N²) = 10^10 — TLE. For N = 5×10^3 in B/C: O(N³) = 1.25×10^11 — TLE.
Optimization Path
A (CHT):
Maintain a lower convex hull of lines y = d[i] * x + (dp[i] + c[i] - d[i] * x[i]). For each query x = x[j], find the line with minimum y at that x.
- If
x[j]is monotonic: use a “Li Chao tree” or a stack-based hull with pointer for O(1) amortized per query → O(N) total. - If
x[j]arbitrary: binary search on hull → O(N log N).
B (Knuth):
Compute dp[i][j] for increasing j - i. For each (i, j), only try splits in [opt[i][j-1], opt[i+1][j]]. Amortized O(N²) instead of O(N³).
C (D&C DP):
For layer k, define solve(lo, hi, opt_lo, opt_hi): compute dp[k][lo..hi], knowing optimal split for each is in [opt_lo, opt_hi]. Recurse on midpoint m, then on solve(lo, m-1, opt_lo, opt[m]) and solve(m+1, hi, opt[m], opt_hi). Each level of recursion is O(N) work; depth is O(log N) → O(N log N) per k, O(KN log N) total.
Final Expected Approach
(See solution sketches inline in the variants above. Full implementations are 80–150 lines each in C++.)
Data Structures
- A (CHT): deque of lines + intersection-checking helper
- B (Knuth):
dp[N][N],opt[N][N] - C (D&C):
dp[K][N], recursive solver
Correctness Argument
- CHT: A line
y = mx + bis “dominated” if another line is below it at every x in the query range. The lower envelope contains exactly the non-dominated lines, in order of increasing slope. - Knuth: the quadrangle inequality implies monotonicity of
opt. Proof in TAOCP Vol 3. - D&C DP: if
opt[i] ≤ opt[i+1](monotonicity), splitting the range and using this constraint reduces work logarithmically.
Complexity
| Variant | Naive | Optimized |
|---|---|---|
| A | O(N²) | O(N) or O(N log N) |
| B | O(N³) | O(N²) |
| C | O(KN²) | O(KN log N) |
Implementation Requirements
- CHT: handle slopes carefully (sorted vs not); avoid division for intersections (use cross-multiplication with care for overflow)
- Knuth: process diagonals in order of length; verify monotonicity in a debug build
- D&C DP: pass index ranges + opt ranges; base case is
lo > hi
Tests
- Small N where brute force confirms answer
- Edge: N = 1 (trivial answer)
- All zero costs / probabilities
- Monotonically increasing / decreasing costs
- Stress: random instances at N = 1000 — compare optimized vs brute force
Follow-up Questions
- CHT for max instead of min. → Maintain upper convex hull; symmetric.
- Lines added in arbitrary slope order. → Use Li Chao Tree; O(log N) per insert and query.
- Knuth not applicable (no quadrangle). → Either D&C DP (if opt is monotonic) or SMAWK (O(N) for totally monotone matrices).
- D&C DP combined with CHT. → Possible; “Aliens trick” / Lagrangian relaxation.
Product Extension
- CHT: dynamic programming in online ad bidding optimization, trading strategy optimization
- Knuth: BST construction in language tools (rarely; usually use B-trees or hash maps)
- D&C DP: optimal segmentation in time-series anomaly detection, network topology design
Language/Runtime Follow-ups
- C++: all three implementable with stdlib. CHT often uses
__int128to avoid overflow in intersection checks. - Python: D&C DP works but with significant constant factor; CHT with Li Chao is feasible.
- Java: BigInteger for safety on overflow-prone intersection checks.
Common Bugs
- CHT: integer overflow in line intersection. Use long doubles or 128-bit.
- CHT: deque pops the wrong end when slopes are descending vs ascending.
- Knuth: didn’t verify the quadrangle inequality. Algorithm gives wrong answer silently.
- Knuth: opt-range boundaries inclusive vs exclusive — off-by-one.
- D&C: passed wrong opt range to recursive calls. Loses the monotonicity benefit.
- D&C: base case (lo > hi) doesn’t return.
Debugging Strategy
- For each technique, write a brute-force version side by side
- Stress test with random small instances and assert equality
- For CHT, print the hull after each insertion
- For Knuth/D&C, log the chosen split points and verify monotonicity
Mastery Criteria
- Recognize when a DP qualifies for each optimization
- Implement CHT in ≤ 40 min
- Implement Knuth in ≤ 30 min
- Implement D&C DP in ≤ 30 min
- State the prerequisite condition for each (linearity, quadrangle, monotonic-opt)
- Estimate runtime for given N, K
Lab 07 — FFT / Polynomial Multiplication
Goal
Implement the Cooley-Tukey FFT to multiply two polynomials in O(N log N), and apply it to convolution-based problems (large-integer multiplication, string matching with wildcards).
Background
The Discrete Fourier Transform (DFT) of a length-N vector evaluates the corresponding polynomial at N roots of unity. Multiplying two polynomials of degree N-1 via naive convolution is O(N²); via DFT, point-wise multiply, inverse-DFT, it’s O(N log N).
Cooley-Tukey (1965): divide and conquer the DFT. The radix-2 version requires N a power of 2.
Number-Theoretic Transform (NTT): FFT over a prime field; avoids floating-point error; common in competitive programming.
Interview Context
- Codeforces / ICPC: regular (NTT version)
- Signal processing roles (DSP, audio, image): expected
- Cryptography research: standard tool
- Quant: large-integer multiplication, time-series convolutions
- Standard FAANG: essentially zero
When to Skip This Topic
Skip if any of these are true:
- You aren’t targeting signal-processing, cryptography, or ICPC roles
- You haven’t implemented divide-and-conquer recursive algorithms confidently
- You’re rusty on complex number arithmetic
This is a “you need it or you don’t” topic. Most interview prep should skip.
Problem Statement
Polynomial Multiplication.
Given two polynomials A(x) = sum a[i] * x^i and B(x) = sum b[i] * x^i, compute their product C(x) = A(x) * B(x).
Equivalently: compute the convolution c[k] = sum_{i+j=k} a[i] * b[j].
Constraints
- Degrees up to 10^5 or 10^6
- Coefficients fit in int32 (avoid overflow concerns by using float carefully; or use NTT)
- Wall-clock: < 1 sec
Clarifying Questions
- Integer coefficients or real-valued? (Integer for NTT, real for FFT.)
- Exact answer required? (Yes for NTT; FFT introduces floating error.)
- Output as polynomial coefficients or as a value at specific x? (Coefficients.)
Examples
A = [1, 2, 3] (1 + 2x + 3x²)
B = [4, 5] (4 + 5x)
C = [4, 13, 22, 15] (4 + 13x + 22x² + 15x³)
A = [1, 1] (1 + x)
B = [1, 1]
C = [1, 2, 1] (1 + x)² = 1 + 2x + x²
Brute Force
Nested loops: c[i+j] += a[i] * b[j]. O(N²). For N = 10^5: 10^10 ops — TLE.
Brute Force Complexity
- Time: O(N²)
- Space: O(N)
Optimization Path
- Pad both A and B to length 2^k ≥ deg(A) + deg(B) + 1.
- Compute DFT(A) and DFT(B) using Cooley-Tukey.
- Pointwise multiply: F[i] = DFT(A)[i] * DFT(B)[i].
- Compute IDFT(F) to recover convolution C.
Final Expected Approach
def fft(a, invert=False):
n = len(a)
if n == 1: return
# bit-reverse permutation
j = 0
for i in range(1, n):
bit = n >> 1
while j & bit:
j ^= bit
bit >>= 1
j ^= bit
if i < j:
a[i], a[j] = a[j], a[i]
# butterfly
length = 2
while length <= n:
angle = 2 * math.pi / length * (-1 if invert else 1)
wlen = complex(math.cos(angle), math.sin(angle))
for i in range(0, n, length):
w = complex(1)
for k in range(length // 2):
u = a[i + k]
v = a[i + k + length // 2] * w
a[i + k] = u + v
a[i + k + length // 2] = u - v
w *= wlen
length <<= 1
if invert:
for i in range(n):
a[i] /= n
def multiply(a, b):
result_size = 1
while result_size < len(a) + len(b):
result_size <<= 1
fa = [complex(x) for x in a] + [complex(0)] * (result_size - len(a))
fb = [complex(x) for x in b] + [complex(0)] * (result_size - len(b))
fft(fa)
fft(fb)
for i in range(result_size):
fa[i] *= fb[i]
fft(fa, invert=True)
return [round(x.real) for x in fa[:len(a) + len(b) - 1]]
For NTT (exact integer convolution), replace complex roots of unity with primitive roots in F_p for a prime p = c * 2^k + 1 (common: 998244353 with primitive root 3).
Data Structures
- Array of complex numbers (FFT) or integers mod p (NTT)
- Bit-reverse permutation index
Correctness Argument
- DFT linearity: DFT(A + B) = DFT(A) + DFT(B).
- Convolution theorem: DFT(A * B) = DFT(A) ⊙ DFT(B) (pointwise).
- Inverse: IDFT(DFT(A)) = A.
- Cooley-Tukey: recursive split into even/odd indices; combines via roots of unity.
Complexity
- Time: O(N log N)
- Space: O(N)
Implementation Requirements
- N must be a power of 2; pad with zeros
- Bit-reversal permutation correctly implemented
- Iterative butterflies (recursive is fine for small N but slow for large)
- For floating-point FFT: round to nearest integer at the end; verify error bound
- For NTT: pick a prime large enough for max coefficient × N to avoid overflow
Tests
- Multiply [1, 1] × [1, 1] = [1, 2, 1]
- Multiply degree-3 polynomials by hand-verified product
- Stress: random N = 10^4 polynomials vs O(N²) brute force; assert equality
- N = 1 (constants only)
- All zeros (result all zeros)
- Performance: N = 2×10^5 in < 1 sec
Follow-up Questions
- Exact integer convolution with large coefficients. → NTT with multi-modulus + CRT, or three NTTs with different primes.
- String matching with wildcards. → Reduce to convolution; each char becomes a numeric encoding; wildcard = 0; sum-of-(diff)² = 0 means match.
- Multi-dimensional FFT (image convolution). → Apply 1D FFT along each axis.
- Fast multiplication of very large integers. → Schönhage-Strassen uses FFT; Furer’s algorithm faster asymptotically.
- Subset sum convolution. → Walsh-Hadamard transform; different beast.
Product Extension
- Audio processing (spectrograms, filters)
- Image processing (Gaussian blur, edge detection)
- Cryptography (large-integer multiplication for RSA, ECC)
- Time-series analysis (autocorrelation)
- Big-integer libraries (GMP uses FFT-based multiplication above ~1000 digits)
Language/Runtime Follow-ups
- C++: standard. NumPy’s FFT in Python is C-optimized — sometimes acceptable as the “library” answer if the interviewer allows.
- Python: pure-Python FFT is slow; for N > 10^4 use
numpy.fft. - Java: Apache Commons Math has FFT.
- JavaScript: rare in interviews; libraries exist.
Common Bugs
- Bit-reverse permutation wrong: off-by-one in the swap loop.
- Forgot to divide by N in inverse: result is N× too large.
- N not a power of 2: padding error.
- Floating-point error too large: for coefficients near max int32, need careful rounding or NTT.
- NTT primitive root wrong: for prime p, root g must have order divisible by N.
- Result length wrong: should be len(A) + len(B) - 1, but FFT computed over N ≥ that.
Debugging Strategy
- Verify FFT then IFFT recovers the input (within floating tolerance)
- Multiply by [1] and verify output equals input
- Compare against numpy.fft on small inputs
- For NTT: compute small examples by hand using primitive roots
Mastery Criteria
- Implement Cooley-Tukey FFT in ≤ 45 min
- Implement NTT in ≤ 60 min
- State the convolution theorem
- Identify problems solvable via FFT/NTT (convolution, large-int mult, string matching with errors)
- Reason about numerical precision for FFT vs NTT
- Estimate runtime for given N
Lab 08 — Advanced Geometry
Goal
Implement two foundational computational geometry primitives:
- Convex Hull via Andrew’s monotone chain (O(N log N))
- Segment Intersection (Bentley-Ottmann sweep-line, O((N + K) log N))
Background
Computational geometry interviews are rare but exist at:
- Companies doing CAD, CAM, 3D modeling (Autodesk, Adobe)
- Games (Unity, Epic) — physics, collision
- Robotics (path planning, occupancy grids)
- Maps/GIS (Google Maps, ESRI)
- Some quant for time-series geometric techniques
The implementation has many sharp edges: floating-point comparison, collinear points, degenerate cases. Robust geometry is a deep subfield.
Interview Context
- Codeforces / ICPC: geometry round often included
- Game / CAD / robotics roles: foundational
- Standard FAANG: almost never
When to Skip This Topic
Skip if any of these are true:
- You’re not targeting the specific industries above
- You’re uncomfortable with vector/cross product math
- You don’t have 2+ weeks to handle the edge cases properly
The first implementation of convex hull will have bugs. Plan for several practice attempts.
Problem Statement
Part A — Convex Hull
Given N points in 2D, return the vertices of their convex hull in counterclockwise order, starting from the lowest-leftmost point.
Part B — Count Segment Intersections
Given N line segments, return the number of intersection points among them.
Constraints
- A: 1 ≤ N ≤ 10^5. Coordinates fit in int32.
- B: 1 ≤ N ≤ 10^5. Up to K = O(N²) intersections in pathological cases; for the algorithm to be efficient, K << N².
Clarifying Questions
A:
- Should collinear hull points be included or skipped?
- Are duplicates possible?
B:
- Should overlapping segments count as one intersection or many?
- Touching at endpoints?
Examples
A
Points: [(0,0), (1,1), (2,0), (1,-1)]
Hull: [(0,0), (2,0), (1,1)] (counterclockwise; (1,-1) is below)
Wait — (1,-1) is also on the hull. Correct hull: [(0,0), (1,-1), (2,0), (1,1)].
B
Segments: [((0,0),(4,4)), ((0,4),(4,0))]
Intersect at (2,2). Answer: 1.
Brute Force
A: gift-wrapping (Jarvis march) — O(N · H) where H = hull size. Worst case O(N²). B: check every pair — O(N²).
Brute Force Complexity
- A: O(NH) worst-case O(N²)
- B: O(N²)
For N = 10^5: 10^10 — TLE.
Optimization Path
A — Andrew’s Monotone Chain
- Sort points lexicographically by (x, y).
- Build upper hull: iterate left to right; pop top of hull while it makes a non-right turn.
- Build lower hull: iterate right to left; pop while non-right turn.
- Concatenate (excluding shared endpoints).
def cross(O, A, B):
return (A[0]-O[0]) * (B[1]-O[1]) - (A[1]-O[1]) * (B[0]-O[0])
def convex_hull(points):
points = sorted(set(points))
if len(points) <= 1: return points
lower = []
for p in points:
while len(lower) >= 2 and cross(lower[-2], lower[-1], p) <= 0:
lower.pop()
lower.append(p)
upper = []
for p in reversed(points):
while len(upper) >= 2 and cross(upper[-2], upper[-1], p) <= 0:
upper.pop()
upper.append(p)
return lower[:-1] + upper[:-1]
B — Bentley-Ottmann Sweep Line
- Event queue: segment endpoints + computed intersections, ordered by x.
- Status: balanced BST of active segments, ordered by y at the sweep line.
- At a “left endpoint” event: insert into status; check intersection with neighbors above/below.
- At a “right endpoint” event: remove from status; check intersection between the new neighbors.
- At an “intersection” event: swap the two segments in status; check intersections with new neighbors.
O((N + K) log N) where K = number of intersections.
Final Expected Approach
Convex hull: as above.
Bentley-Ottmann: full implementation is 200+ lines with all edge cases. For most interviews, even discussing the structure is enough — full code is unlikely to be required.
Data Structures
- A: sorted list, stack-like list for hull construction
- B: priority queue (event queue), balanced BST or sorted list with O(log N) ops (status)
Correctness Argument
A
- Sorting by x (then y) ensures we visit points in a consistent order.
- “Right turn” (cross product > 0) means we’re making a convex angle; popping ensures we never include a concave point.
- Lower and upper hulls cover all hull vertices; concatenation gives full hull in CCW order.
B
- Two segments can only intersect when they are adjacent in the y-order at some x.
- The sweep maintains adjacency; new adjacencies arise at endpoints and intersections.
- Each intersection is detected exactly once (at the moment the segments become adjacent).
Complexity
- A: O(N log N) for sort + O(N) for chain construction.
- B: O((N + K) log N).
Implementation Requirements
- Use integer arithmetic for cross product when possible (avoids floating-point errors)
- Handle collinear hull points consistently (include or exclude as required)
- For segment intersection: distinguish proper intersection (interior) from touching (endpoint)
- Watch for vertical segments (event ordering edge case)
Tests
A
- Single point → [point]
- Two points → [both]
- Three collinear points → [endpoints only] (or all three, depending on convention)
- Square (4 corners + 5 interior) → 4 corners
- All points on a circle → all on hull
- Many duplicates
B
- No intersections (parallel lines)
- All intersect at one point (n lines through origin)
- Random N = 100 vs O(N²) brute force
Follow-up Questions
A:
- Convex hull in 3D. → Quickhull or gift wrapping in 3D; O(N²) worst.
- Dynamic convex hull (online insertions). → Overmars-van Leeuwen; O(log² N) per update.
- Compute area of hull. → Shoelace formula.
- Diameter of hull (farthest pair). → Rotating calipers, O(N).
B:
- Report intersections, not just count. → Same structure; collect points.
- Line-line intersection in projective coords. → Avoids special-casing parallels.
- Segments may overlap. → More complex event handling.
- Robust orientation predicate. → Adaptive precision; Shewchuk’s predicates.
Product Extension
- Mapping: route simplification (Ramer-Douglas-Peucker)
- Games: collision detection (broad phase uses sweep)
- CAD: boolean operations on polygons (sweep-based)
- Robotics: configuration space construction
Language/Runtime Follow-ups
- C++: geometry libraries (CGAL) are massive and correct but complex.
- Python: Shapely for production; for interview, implement primitives.
- Java: java.awt.geom.* has some primitives.
Common Bugs
- Floating-point comparison without epsilon: false negatives on coincident points.
cross < 0vscross <= 0: different hull conventions (collinear in or out).- Forgot to dedupe input points before convex hull.
- Sweep-line: vertical segments treated incorrectly. Add small perturbation or special-case.
- Sweep-line: intersection on the sweep line not detected because status ordering is computed at the wrong x.
Debugging Strategy
- Plot the points and hull (matplotlib or similar)
- For small cases, enumerate hull by hand and verify
- For sweep-line: log every event and the status before/after
Mastery Criteria
- Implement Andrew’s monotone chain in ≤ 30 min from memory
- Implement basic segment-intersection check in ≤ 15 min
- Understand the Bentley-Ottmann structure (even without writing 200 lines)
- Apply rotating calipers for hull diameter
- Recognize when integer arithmetic suffices vs when floating-point is unavoidable
- State complexity precisely
Lab 09 — ICPC Contest Simulation
Goal
Simulate a 5-hour ICPC-style contest: 6–10 problems of varying difficulty, single team, no internet, paper and pen allowed. Practice contest strategy: problem selection, time management, debugging under pressure.
Background
ICPC contests are the gold standard for competitive programming practice:
- 5 hours
- ~10 problems sorted A–J in roughly increasing difficulty (but not strictly)
- Penalty per wrong submission (20 minutes)
- Final ranking by # problems solved, then total time
This lab is a meta-lab: rather than teach an algorithm, it builds the contest operating system of the candidate.
Interview Context
ICPC training transfers to:
- Quant hiring (Jane Street, Citadel value ICPC experience)
- Google L6+ research interviews (sometimes)
- General algorithmic confidence under time pressure (transferable)
Direct application of contest mode to industry interviews: low. But the training effect is high.
When to Skip This Topic
Skip if any of these are true:
- You’re not interviewing at competitive-friendly firms
- You have less than a month for this phase
- You haven’t done Phase 11 mocks at your target level — those are higher leverage
If you do this lab, do it only after exhausting Phase 11.
Problem Statement
Run a 5-hour contest. Sources of problem sets:
- Codeforces Educational rounds
- Codeforces Div 2 (rounds 600+)
- ICPC regional sets on UVa Online Judge or DOMjudge replays
- Atcoder Beginner Contest (ABC) — easier, 100 min
- Atcoder Regular Contest (ARC) — medium, 120 min
- Kattis archive
Required mix for a 5-hour set:
- 2 trivial problems (sanity / warm-up; should solve in 15 min each)
- 3 medium problems (1 hour each)
- 2 hard problems (1.5+ hours, often unsolved)
- 1 problem-killer (often unsolved by anyone except top teams)
Constraints
- Time: 5 hours, single uninterrupted block
- No internet (except for the problem statements)
- Programming language of your choice (typically C++ for competitive)
- Penalty: simulate the 20-minute penalty per WA
- No external help / collaboration
Clarifying Questions (to yourself before starting)
- Which 2 problems will I attempt first? (Decision in 5 min of reading.)
- What’s my “hard problem cutoff” — at what point do I move on?
- What’s my time budget for debugging vs writing?
- How will I track time per problem?
Examples (suggested contest sets)
- Beginner: ABC 250 (8 problems, 100 min, but extend to 3 hours for practice)
- Intermediate: Codeforces Round 800, Div 2 (4–5 problems, 2 hours; extend with a Div 1 problem)
- Advanced: Any ICPC regional set on Kattis
Brute Force / Naive Strategy
- Read problems A → J in order
- Attempt in order
- Stuck on B → keep grinding
This is the wrong strategy. All experienced contestants read every problem in the first 30 minutes.
Brute Force Complexity
Linear-strategy ranking is in the bottom half. The optimization is meta: contest strategy.
Optimization Path
Phase 1 (0–30 min): Reading and triage
- Read all problems. Categorize each as: trivial (T) / medium (M) / hard (H) / unknown (?)
- Note any problem you immediately know how to solve
- Estimate time-to-solve for each known problem
- Decide which 2 problems to start with (lowest-risk T or known-M)
Phase 2 (30 min – 3 hours): Bulk solving
- Solve the T’s first (lock in points)
- Tackle M’s one at a time
- Time-cap each: 45 min to first attempt; if no insight at 60 min, switch
- Submit only when you’ve tested at least 2 cases (penalty hurts)
- Track submitted vs accepted on a paper grid
Phase 3 (3–4.5 hours): Hard problem attack
- Attempt your best H candidate
- If stuck for 30 min, switch to another M or H
- Don’t sink the last hour on one H if no progress
Phase 4 (last 30 min): Last-chance and verification
- Verify your AC submissions (any bugs you noticed but didn’t fix?)
- Submit any partial solutions
- Hand-test edge cases on solved problems
Final Expected Approach
Run the contest, then write a post-mortem:
- Problems solved: ___
- Penalty time: ___
- Problems unread: ___ (should be 0)
- Problems where you knew the approach but couldn’t implement: ___
- Problems where you couldn’t find the approach: ___
- Wasted time on (problem X): ___
- Bug that cost you (problem Y): ___
Data Structures (the contestant uses)
- Templates file (
.hfor C++ or snippets): segment tree, DSU, FFT, Dijkstra, BFS, mod arithmetic - Paper grid: problem letter | reading time | first-attempt time | submissions | AC time
- Stack-rank: priority order updated as problems are read
Correctness Argument
Strategy correctness is empirical: track your own performance over 5–10 contests. Adjust based on:
- Where did I spend too long?
- Which problems did I misclassify?
- Which problem types do I consistently miss?
Complexity
Contest itself: 5 hours. Post-mortem: 1 hour. Per-week investment: ~1 full contest + a few targeted upsolves = 8–12 hours.
Implementation Requirements
- Pre-built template file (start with KACTL or your own)
- Local judging setup: compile + run + diff against expected output
- Stress-testing harness (Lab 05 from Phase 10 applies directly)
- A timer / phone alarm for the 5 hours
Tests
This is the test. The contest is the test.
Self-evaluation rubric:
-
problems solved
- Time per problem
- WA-to-AC ratio
- Quality of stress-tests during contest
Follow-up Questions (post-mortem)
- Which problem could I have solved with 30 more minutes? → Drill that topic.
- Which problem did I solve in time that I should have submitted faster? → Bug-hunt training.
- Which problem did I skip that I should have attempted? → Misclassification — calibrate.
- Did I run out of time or run out of ideas? → Different fixes.
Product Extension
- Long-term: ICPC-style training builds engineering reflexes that transfer to:
- Performance debugging under deadline
- Triage of multiple bugs simultaneously
- Estimation of task duration (notoriously hard for engineers)
Language/Runtime Follow-ups
- C++: dominant in competitive. Compile-time templates pay off.
- Python: acceptable for problems with N ≤ 10^5; risk TLE on tight problems.
- Java: middle ground; some teams use it.
- Rust: rising; some teams use; standard library less batteries-included than C++.
Common Bugs (contest-specific)
- Submitted without testing. Penalty bug.
- Submitted with debugging prints still in code. WA.
- Forgot to read a problem. Discovered 2 hours later you had a free solve.
- Spent 90 minutes on one problem you couldn’t solve. Sunk-cost trap.
- Submitted brute force expecting partial credit. ICPC has no partial; only full AC.
- Wrong I/O format. Read the spec carefully — especially expected output line endings.
Debugging Strategy (during contest)
- 5-min rule: if not finding a bug in 5 min, write a stress test (Lab 05)
- Random small inputs vs brute force is a contest superpower
- If stuck on WA, re-read the problem statement entirely — often the bug is misunderstanding, not code
Mastery Criteria
- Complete 5 contests; track scores
- Post-mortem each one and act on findings
- Solve at least 3 problems consistently in a 5-hour Div 2 contest
- Solve at least 1 D-level (Codeforces) problem in 2 hours
- Build a personal template file with ≥ 15 commonly-used snippets
- After 10 contests, identify your top 3 weakness topics; drill them
Suggested Contest Schedule (4 weeks)
| Week | Contest | Goal |
|---|---|---|
| 1 | Codeforces Educational Round | Solve A, B, C, attempt D |
| 2 | ABC (extended to 3 hr) | Solve A through F |
| 3 | Codeforces Div 2 | Solve A, B, C, D |
| 4 | ICPC regional replay | Solve 3–5 of 10 |
Post-mortem after each. Drill weakness topics between contests.
Lab 10 — Inclusion-Exclusion and Burnside
Goal
Master two combinatorial counting techniques used across competitive programming and discrete math:
- Inclusion-Exclusion Principle (PIE) for counting objects satisfying / violating multiple conditions
- Burnside’s lemma for counting orbits under group actions (counting distinct configurations modulo symmetry)
Background
Inclusion-Exclusion
For sets A₁, …, Aₙ:
$$|A_1 \cup \cdots \cup A_n| = \sum |A_i| - \sum |A_i \cap A_j| + \sum |A_i \cap A_j \cap A_k| - \cdots$$
In counting form, for “objects with property P_i”:
$$\text{count violating none} = \sum_{S \subseteq {1..n}} (-1)^{|S|} \cdot |\text{objects with all properties in S}|$$
Burnside
For a group G acting on a set X, the number of orbits is:
$$|X/G| = \frac{1}{|G|} \sum_{g \in G} |X^g|$$
where X^g is the set of elements fixed by g.
Interview Context
- Codeforces / ICPC: regular (PIE constantly, Burnside in counting problems with symmetry)
- Quant: combinatorial pricing models
- Cryptography / coding theory: standard tools
- Standard interviews: occasional easy PIE problem (e.g., “count integers ≤ N divisible by none of {2, 3, 5}”)
PIE appears more often than people realize; Burnside is rarer.
When to Skip This Topic
Skip if any of these are true:
- You’re not targeting competitive / combinatorics-heavy roles
- You haven’t done basic combinatorics (Phase 1–2 foundations)
- You have less than 1 week for this lab
PIE is high-leverage even outside competitive — learn it. Burnside is optional.
Problem Statement
Part A — Count Coprime to N
Given integers L ≤ R and N, count integers in [L, R] coprime to N.
Part B — Distinct Necklaces
Given k colors and n beads in a circle, count the number of distinct necklaces (two necklaces are equivalent if one is a rotation of the other).
Constraints
- A: 1 ≤ L ≤ R ≤ 10^18, 1 ≤ N ≤ 10^9
- B: 1 ≤ n ≤ 10^6, 1 ≤ k ≤ 10^9
Clarifying Questions
A:
- Coprime means
gcd(x, N) = 1? - Do we include endpoints?
B:
- Are reflections considered equivalent (bracelets) or only rotations (necklaces)?
- Modulo what prime?
Examples
A
L=1, R=10, N=6 → coprime to 6 are {1, 5, 7}. Wait, in [1,10]: {1, 5, 7}. Answer: 3.
B
n=3, k=2 → 4 distinct necklaces: BBB, BBW, BWW, WWW (BBW and BWB and WBB are all rotations of each other).
Brute Force
A: iterate x in [L, R], check gcd. O(R - L). For R - L = 10^18: TLE.
B: generate all k^n colorings, group by rotation equivalence. O(k^n).
Brute Force Complexity
- A: O(R - L)
- B: O(k^n)
Both fail for given constraints.
Optimization Path
A (Inclusion-Exclusion)
Let p₁, …, pₘ be the distinct prime divisors of N. Then:
$$\text{coprime}(1..M) = \sum_{S} (-1)^{|S|} \cdot \lfloor M / \prod_{p \in S} p \rfloor$$
Compute for f(M) = count of integers in [1, M] coprime to N. Answer = f(R) - f(L-1).
def count_coprime(M, N):
primes = prime_divisors(N)
m = len(primes)
total = 0
for mask in range(1 << m):
prod = 1
bits = 0
for i in range(m):
if mask & (1 << i):
prod *= primes[i]
bits += 1
total += ((-1) ** bits) * (M // prod)
return total
answer = count_coprime(R, N) - count_coprime(L - 1, N)
Complexity: O(2^m · log N) where m = number of distinct prime factors of N (≤ 9 for N ≤ 10^9).
B (Burnside)
Group G = cyclic group of n rotations. By Burnside:
$$|\text{necklaces}| = \frac{1}{n} \sum_{d=0}^{n-1} k^{\gcd(n, d)}$$
Group by gcd(n, d) = g: the count of d with this gcd is φ(n/g). So:
$$= \frac{1}{n} \sum_{g | n} \varphi(n/g) \cdot k^g$$
Compute φ on divisors of n. O(σ(n)) divisors; per divisor, O(log n) for fast exponentiation. Total: O(d(n) · log n) which is tiny.
Final Expected Approach
(See solution sketches above.)
Data Structures
A: list of prime factors of N; bitmask iteration B: divisor enumeration; Euler totient function
Correctness Argument
PIE: classical proof by induction or by the principle that each element is counted (number of subsets it belongs to) times in alternating signs, summing to 1 - 0 + 0 … = 1.
Burnside: from orbit-counting theorem; proof in any introductory abstract algebra text.
Complexity
- A: O(2^m) where m = distinct prime factors ≤ 9 → constant
- B: O(σ(n) · log n) → effectively O(√n · log n) since divisors ≤ 2√n
Implementation Requirements
- A: efficient prime factorization (trial division up to √N is fine for N ≤ 10^9)
- B: divisor enumeration via trial division; Euler totient by formula
φ(n) = n · ∏(1 - 1/p) - Modular exponentiation for k^g mod p
- Modular inverse for division by n (use Fermat: n^(p-2) mod p)
Tests
A
- L=1, R=10, N=6 → 3
- L=1, R=N → φ(N) (Euler totient)
- L=R, x coprime to N → 1
- L=R, x not coprime → 0
- Very large R for performance check
B
- n=1, k=2 → 2 (B, W)
- n=2, k=2 → 3 (BB, BW, WW)
- n=3, k=2 → 4
- n=4, k=2 → 6
- Compare with brute force for n ≤ 6
Follow-up Questions
For A:
- Count coprime to multiple Ns simultaneously. → PIE on union of all prime sets.
- Count squarefree numbers in [L, R]. → Möbius function = PIE on prime squares.
- Sum of φ(i) for i in [1, N]. → Sieve-like O(N log log N) or O(N^{2/3}) with Mertens.
For B:
- Bracelets (rotations + reflections). → Group = dihedral D_n. Add reflection terms.
- Necklaces with at most k uses of each color. → Multiset Burnside; harder.
- Polya enumeration with cycle index polynomial. → Generalizes Burnside; gives generating function.
- Distinct binary trees of n nodes. → Catalan numbers; different problem but related counting.
Product Extension
- Combinatorial design (DOE, experimental design)
- Code generation (counting equivalence classes of programs)
- Chemistry (counting molecular isomers — Polya’s original motivation)
- Cryptography (group orbits in elliptic curves)
Language/Runtime Follow-ups
- Python: big-int arithmetic for free; ideal for PIE/Burnside with modular arithmetic.
- C++: modular arithmetic with manual care for overflow; faster execution.
- Java: BigInteger for safety; modular arithmetic mature.
Common Bugs
- PIE sign wrong: off-by-one in (-1)^|S|.
- PIE on factored N: counted prime factors with multiplicity. Only distinct primes matter.
- Burnside: forgot to take modular inverse for division by n.
- Euler totient computed via brute force gcd: O(n) per value, way too slow.
- Modular exponentiation overflow: use
pow(k, g, MOD)in Python; manual loop with mod in C++.
Debugging Strategy
- Brute force for small parameters; assert PIE/Burnside match
- For PIE: print each subset’s contribution; verify signs alternate
- For Burnside: enumerate orbits manually for n ≤ 4
Mastery Criteria
- Apply PIE to: count divisible/coprime, derangements, surjections, squarefree
- Apply Burnside to: necklaces, bracelets, colored cubes
- Compute Euler totient in O(√n)
- Compute Möbius function (PIE generalized)
- State group-action correctness conditions
- Identify a “this needs symmetry handling” problem and reach for Burnside
Lab 11 — Suffix Array & Kasai LCP
Goal
Build a suffix array in O(n log n) via prefix doubling, compute the LCP array in O(n) with Kasai’s algorithm, and apply the pair to the standard problems: longest repeated substring, distinct-substring counting, and pattern search.
Background
The suffix array sa is the array of starting indices of all suffixes of s, sorted lexicographically. It is the flat, cache-friendly cousin of the suffix tree.
Prefix doubling: sort suffixes by their first 2^k characters using rank pairs. If rank[i] orders suffixes by their first k characters, then the pair (rank[i], rank[i+k]) orders them by their first 2k. After ≤ log n rounds every rank is distinct and the sort is total.
Kasai’s insight: let lcp[r] = longest common prefix of suffixes sa[r-1] and sa[r]. If suffix starting at i shares h characters with its lexicographic predecessor, then suffix i+1 (drop one char) shares at least h−1 with its predecessor. So process suffixes in text order, never resetting h below h−1: total work O(n).
What the pair buys you:
- Longest repeated substring = max(lcp)
- Distinct substrings = n(n+1)/2 − Σlcp (each suffix contributes its length minus the overlap already counted)
- Pattern matching: binary search the SA in O(m log n)
- Longest common substring of two strings: concatenate with a separator, take max lcp across suffixes from different halves
SA + LCP is equivalent in power to a suffix tree — an in-order traversal of the suffix tree visits leaves in SA order, and internal-node string depths are exactly the LCP values. The suffix automaton (lab-05) is the third member of the family: same problems, different traversal idioms.
Interview Context
- Codeforces / ICPC: core string tool, expected at Div 1 level
- Bioinformatics roles: fundamental (genome alignment, FM-index)
- LC 1044 (Longest Duplicate Substring, Hard): SA+LCP is the clean deterministic answer; most solutions use binary search + rolling hash and pray about collisions
- Systems/search-infra roles: occasionally, as a design discussion (log-structured text indexes)
- Standard FAANG: near zero — rolling hash (phase-03 lab-06) covers the rare string-Hard
When to Skip This Topic
Skip if any of these are true:
- You haven’t mastered KMP (phase-03 lab-05) and rolling hash (phase-03 lab-06) — those cover 95% of interview string questions
- You’re not targeting competitive programming or bioinformatics
- You can’t state what a suffix even is without hesitating
One LC Hard justifies this lab. If LC 1044 isn’t on your target list, skim the Background and move on.
Problem Statement
Longest Duplicate Substring (LC 1044).
Given a string s, return any duplicated substring of maximum length — a contiguous substring that occurs two or more times (occurrences may overlap). Return "" if none exists.
Along the way, implement the general toolkit: suffix_array(s), kasai(s, sa), distinct_substrings(s).
Constraints
- 2 ≤ |s| ≤ 3×10^4 (LC); toolkit should handle 10^5+
- Lowercase English letters
- Wall-clock: < 2 sec in Python
Clarifying Questions
- Can occurrences overlap? (Yes — “aaaa” → “aaa”.)
- Any answer of maximum length, or all of them? (Any one.)
- Alphabet size — does it matter? (Only for radix-sort variants; comparison sort doesn’t care.)
- Empty answer possible? (Yes, when all characters are distinct.)
Examples
s = "banana"
sa = [5, 3, 1, 0, 4, 2] (a, ana, anana, banana, na, nana)
lcp = [0, 1, 3, 0, 0, 2]
longest duplicate = "ana" (lcp = 3)
distinct substrings = 6·7/2 − 6 = 15
s = "abcd" → "" (all lcp = 0)
s = "aaaaa" → "aaaa"
Brute Force
For each length L from n−1 down: put all substrings of length L in a set; if one repeats, return it. O(n²) substrings × O(n) hashing = O(n³) worst case. For n = 3×10^4: hopeless.
Brute Force Complexity
- Time: O(n³) (O(n²) with rolling hash per length, still too slow without the binary search)
- Space: O(n²) for the substring set
Optimization Path
- Binary search on length + rolling hash: O(n log n) expected, but collision-correct only if you verify matches — most posted solutions don’t. (Cross-ref phase-03 lab-06.)
- Suffix array + LCP: two suffixes share a prefix of length L iff some adjacent SA pair has lcp ≥ L (sorted order puts the most similar suffixes next to each other). So the answer is simply max(lcp) — deterministic, no hashing.
- Build SA by doubling (log n rounds of sorting rank pairs), LCP by Kasai in O(n).
Final Expected Approach
def suffix_array(s):
n = len(s)
sa = list(range(n))
rank = [ord(c) for c in s]
k = 1
while True:
key = lambda i: (rank[i], rank[i + k] if i + k < n else -1)
sa.sort(key=key)
tmp = [0] * n
for i in range(1, n):
tmp[sa[i]] = tmp[sa[i - 1]] + (key(sa[i]) > key(sa[i - 1]))
rank = tmp
if rank[sa[-1]] == n - 1: # all ranks distinct: done
return sa
k <<= 1
def kasai(s, sa):
n = len(s)
rank = [0] * n
for pos, suf in enumerate(sa):
rank[suf] = pos
lcp = [0] * n # lcp[r] = LCP(sa[r-1], sa[r]); lcp[0] = 0
h = 0
for i in range(n): # suffixes in TEXT order
if rank[i] > 0:
j = sa[rank[i] - 1]
while i + h < n and j + h < n and s[i + h] == s[j + h]:
h += 1
lcp[rank[i]] = h
if h:
h -= 1 # next suffix loses at most one char of overlap
else:
h = 0
return lcp
def distinct_substrings(s):
if not s:
return 0
return len(s) * (len(s) + 1) // 2 - sum(kasai(s, suffix_array(s)))
def longest_dup_substring(s):
if len(s) < 2:
return ""
sa = suffix_array(s)
lcp = kasai(s, sa)
r = max(range(len(s)), key=lambda r: lcp[r])
return s[sa[r]: sa[r] + lcp[r]]
Honesty note: Python’s Timsort makes each doubling round O(n log n), so this is O(n log² n) total. Replacing the sort with two-pass radix sort on the rank pairs gives true O(n log n). In Python the sort version is usually faster anyway (C-level sort vs interpreted radix loops).
Data Structures
sa: int array, suffix start indices in sorted orderrank: inverse-ish array, rank of each suffix among first-2^k-char prefixeslcp: int array aligned to SA positions
Correctness Argument
- Doubling invariant: after round k,
rankorders suffixes by their first 2^k characters; the pair sort extends this to 2^(k+1). Missing tail = rank −1 sorts shorter suffixes first, matching lexicographic order. - Termination: ranks strictly refine each round; all distinct within log n rounds.
- Kasai bound: if lcp(suffix i, predecessor) = h, then suffix i+1 and some suffix (the predecessor minus its first char) share h−1 chars; its true predecessor in SA order shares ≥ h−1.
hincreases ≤ n total and decreases ≤ n total → O(n). - Max-lcp = longest repeat: any two suffixes sharing prefix length L sandwich adjacent pairs each with lcp ≥ L (LCP of a range = min of adjacent lcps in it).
Complexity
- SA: O(n log² n) with comparison sort, O(n log n) with radix
- Kasai: O(n)
- Space: O(n)
Implementation Requirements
- Handle
i + k ≥ n: shorter suffix ranks first (use −1 sentinel) - Early exit once all ranks distinct (big win on random strings)
lcp[0]is 0 by definition — don’t read it as a real pair- Kasai must iterate suffixes in text order, not SA order
- Return indices, not substring copies, until the final answer (avoid O(n²) slicing)
Tests
suffix_array("banana") == [5, 3, 1, 0, 4, 2];kasai == [0, 1, 3, 0, 0, 2]- LC 1044: “banana” → “ana”; “abcd” → “”; “aaaaa” → “aaaa”
distinct_substrings("banana") == 15; “aaa” → 3- Stress: random strings over {a,b} and {a,b,c}, n ≤ 40, vs
sorted(range(n), key=lambda i: s[i:])and a set-based distinct count — 500 trials - Performance: n = 5×10^4 random binary string, SA+LCP < 1 sec (measured ~0.15 s)
Follow-up Questions
- Can you build the SA in O(n)? → Yes: SA-IS (induced sorting) and DC3/skew. Name them, don’t implement in an interview — doubling is the expected answer, linear constructions are a library job.
- SA vs suffix automaton vs suffix tree — which do you reach for? → SA+LCP for offline, index-shaped questions (k-th substring, counting, binary search). Suffix automaton (lab-05) for online/DP-over-states questions (distinct substrings incrementally, longest common substring streaming). Suffix tree almost never by hand — SA+LCP simulates it.
- LCP of two arbitrary suffixes, many queries? → LCP(sa[i..j]) = min of adjacent lcp values in the range → sparse-table RMQ, O(1) per query after O(n log n) build (phase-03 lab-04).
- Count occurrences of pattern p? → Two binary searches over SA for the range of suffixes prefixed by p: O(m log n), then range length. Beats KMP when the text is fixed and patterns stream in.
- Burrows-Wheeler transform connection? → BWT[i] = s[sa[i] − 1]: the character before each sorted suffix — the SA’s “last column”. bzip2 compresses it; the FM-index inverts it for O(m) pattern counting in compressed space. This is why read aligners (BWA, Bowtie) are SA machinery underneath.
- Longest common substring of two strings? → Build SA of
s1 + '#' + s2, answer = max lcp over adjacent pairs whose suffixes start in different halves.
Product Extension
- Genome aligners: BWA/Bowtie build FM-indexes from suffix arrays of reference genomes
- Plagiarism / clone detection: longest common substring across document pairs
- Log deduplication: longest repeated substring flags templated spam
- Full-text search indexes over static corpora (SA is simpler than an inverted index for substring — not word — queries)
Language/Runtime Follow-ups
- Python: the
key=lambdasort is fine to n ≈ 10^5; beyond that, precompute key tuples or use the two-array radix formulation. - C++: classic contest snippet ~30 lines;
std::sortversion handles 10^6 easily. - Java: beware
substringpre-Java-7u6 O(1) myths — always compare via indices. - Rust:
suffixcrate implements SA-IS; hand-rolled doubling withsort_unstable_by_keyis idiomatic.
Common Bugs
- Sorting by single rank instead of the pair: converges to first-character order and loops forever or returns garbage.
- Rank assignment off-by-one:
tmp[sa[i]]must compare keys, not old ranks alone — two suffixes with equal pairs must get equal new ranks. - Kasai iterated in SA order: the h ≥ h−1 amortization only holds in text order; wrong order gives O(n²).
- Forgot
h -= 1: silently correct output but O(n²) worst case (“aaaa…”). - Missing tail sentinel: treating
rank[i+k]as 0 instead of −1 misorders “aa” vs “aab” suffixes. - Separator collision in two-string concat: the separator must sort below every real character and appear nowhere in either string.
Debugging Strategy
- Print all suffixes next to
safor a 6-char string; eyeball the sort - Assert
sorted(sa) == range(n)(it’s a permutation) after every doubling round - Check
lcpby direct prefix comparison on small inputs - Binary alphabet stress first — repeats are dense, off-by-ones surface immediately
- If distinct-substring count is wrong, the bug is almost always in
lcp, notsa
Mastery Criteria
- Implement doubling SA from memory in ≤ 25 min
- Implement Kasai in ≤ 15 min and explain the h−1 amortization without notes
- Derive the distinct-substring formula on a whiteboard
- Solve LC 1044 end-to-end in ≤ 40 min
- State when you’d use SA vs suffix automaton vs rolling hash, with one problem each
Lab 12 — Aho-Corasick Automaton
Goal
Match k patterns against a text simultaneously in O(|text| + Σ|patterns| + matches) — the multi-pattern generalization of KMP. Build the trie + failure links + output links, and know when a plain trie or naive scan is the smarter answer.
Background
Construction: insert all patterns into a trie. Then compute, for every trie node, a failure link: the deepest node whose path string is a proper suffix of the current node’s path string. Built by BFS — a node’s failure link is found by walking its parent’s failure chain until a node with the needed child exists (exactly the KMP prefix-function computation, lifted from a single string to a tree).
Output links: a match can end at a state without the state being a pattern’s endpoint — the pattern may be a suffix of the current path (e.g., state “she” must also report “he”). So each state needs the set of dictionary words reachable by following failure links. Either merge these sets during BFS, or keep a “dictionary suffix link” (nearest fail-ancestor that is a word) and chase the chain at match time.
Automaton view: goto (trie edges) + failure walking can be collapsed into a total transition function δ(state, char) — a dense table of size states × alphabet. Then scanning is one table lookup per character, no while-loop. Memory for speed.
KMP is Aho-Corasick with one pattern (phase-03 lab-05): the trie is a path, failure links are the prefix function.
Interview Context
- Codeforces / ICPC: standard tool, appears with DP-on-automaton twists
- Security / infra roles: genuinely production-relevant (Snort, ClamAV, WAFs run AC)
- LC 1032 (Stream of Characters, Hard): AC is one clean answer; the reversed-word trie is the trick most solutions use
- LC 1408 (String Matching in an Array, Easy): constraints so small naive wins — recognizing that IS the interview signal
- Standard FAANG: rare; a trie question in disguise at most. KMP + trie (phase-03 labs 05/07) covers what you’ll actually face
When to Skip This Topic
Skip if any of these are true:
- You haven’t internalized KMP’s prefix function — AC is that idea plus BFS, learn the base case first
- You haven’t built a trie recently (phase-03 lab-07)
- You’re not targeting competitive programming or security/text-scanning domains
If you can solve LC 1032 with the reversed-trie trick and explain why it works, you’ve extracted most of the interview value already.
Problem Statement
Multi-pattern search. Given patterns words = [w1..wk] and a text (possibly streamed one character at a time), report every occurrence of every pattern: pairs (end_index, word).
Anchor problems:
- LC 1032 Stream of Characters:
query(ch)appends a character; return true iff some word is a suffix of the stream so far. - LC 1408: find words that are substrings of other words — n ≤ 100, |word| ≤ 30. Naive
inis O(n² · 30²) ≈ 10^7. Building AC here is over-engineering; say so.
Constraints
- Σ|patterns| ≤ 2×10^5, |text| ≤ 10^6, lowercase letters
- Overlapping and nested matches must all be reported
- Stream variant: O(1) amortized per character after build
Clarifying Questions
- Can patterns duplicate or be suffixes/prefixes of each other? (Yes — this is exactly why output links exist.)
- Report all matches or just existence? (All, as (end_index, word); existence is the easy special case.)
- Is the pattern set fixed before the text arrives? (Yes — AC is a static structure; see follow-ups for dynamic.)
- Alphabet size? (26 here; affects dict-vs-array node representation and the dense-table option.)
Examples
patterns = ["he", "she", "his", "hers"], text = "ushers"
matches: (3, "she"), (3, "he"), (5, "hers")
(indices are match END positions; "he" ends at index 3: u-s-h-e)
patterns = ["a", "aa", "aaa"], text = "aaaa"
matches: (0,a) (1,a) (1,aa) (2,a) (2,aa) (2,aaa) (3,a) (3,aa) (3,aaa)
— 9 matches from 4 chars: output size can dominate.
Brute Force
For each end position i and each pattern w, compare text[i+1−|w| : i+1] == w. O(|text| × k × maxlen). Fine for LC 1408’s constraints; hopeless at 10^6 × 10^5.
Brute Force Complexity
- Time: O(|text| · Σ|patterns|)
- Space: O(1) extra
Optimization Path
- Run KMP once per pattern: O(k · |text|) — still a factor k too slow.
- Trie of patterns, walk from every text position: O(|text| · maxlen) — re-scans overlapping windows.
- Aho-Corasick: one pass over the text; failure links carry over the longest useful suffix so no character is examined more than amortized O(1) times (plus output size).
- For LC 1032 specifically: the non-AC trick is a trie of reversed words, walking backward from the stream’s tail — O(maxlen) per query, no failure links. Simpler; worse when maxlen is large. AC gives O(1) amortized per query. Know both.
Final Expected Approach
from collections import deque
class AhoCorasick:
def __init__(self):
self.next = [{}] # next[v]: char -> child state
self.fail = [0] # failure link
self.out = [[]] # words ending at v (own + inherited via fail)
def add_word(self, word):
v = 0
for ch in word:
if ch not in self.next[v]:
self.next[v][ch] = len(self.next)
self.next.append({})
self.fail.append(0)
self.out.append([])
v = self.next[v][ch]
self.out[v].append(word)
def build(self):
q = deque(self.next[0].values()) # depth-1 states: fail = root
while q:
u = q.popleft()
for ch, v in self.next[u].items():
q.append(v)
f = self.fail[u]
while f and ch not in self.next[f]:
f = self.fail[f]
self.fail[v] = self.next[f].get(ch, 0)
if self.fail[v] == v:
self.fail[v] = 0
self.out[v] = self.out[v] + self.out[self.fail[v]]
def feed(self, text):
v = 0
for i, ch in enumerate(text):
while v and ch not in self.next[v]:
v = self.fail[v]
v = self.next[v].get(ch, 0)
for w in self.out[v]:
yield (i, w) # w ends at text[i]
class StreamChecker: # LC 1032 via AC
def __init__(self, words):
self.ac = AhoCorasick()
for w in words:
self.ac.add_word(w)
self.ac.build()
self.v = 0
def query(self, ch):
ac, v = self.ac, self.v
while v and ch not in ac.next[v]:
v = ac.fail[v]
self.v = v = ac.next[v].get(ch, 0)
return bool(ac.out[v])
Merging out lists during BFS trades memory (worst case a state inherits many words — think patterns a, aa, aaa, …) for zero match-time chasing. The alternative is a single dictionary-suffix link per state, chased at match time — same amortized output cost, O(states) memory. Say the tradeoff out loud.
Data Structures
- Trie as parallel arrays:
next(list of dicts),fail(int list),out(list of word lists) - BFS queue for build; current-state integer for scanning
Correctness Argument
- Invariant: after consuming
text[0..i], the state is the longest suffix of the consumed text that is a path in the trie. Failure walking preserves this: it discards the shortest possible prefix until a valid extension exists. - Failure links via BFS: a node’s fail target is strictly shallower, and BFS processes shallower nodes first, so
fail[u]and itsoutare final before any child of u needs them. - No missed matches: any pattern occurrence ending at i is a suffix of the current state’s path string, hence reachable via the fail chain — captured in the merged
out(or by chasing dictionary links). - Amortization: state depth increases by ≤1 per character and each fail step decreases it by ≥1 → total fail steps ≤ |text|.
Complexity
- Build: O(Σ|patterns|) states; O(Σ|patterns| · alphabet) with dense tables, O(Σ|patterns|) expected with dicts
- Scan: O(|text| + matches) amortized
- Space: O(Σ|patterns|) states; merged output lists can add O(states × k) worst case — use dictionary links if that bites
Implementation Requirements
- BFS (not DFS) for failure links — shallower-first is load-bearing
- Depth-1 nodes fail to root explicitly; never let a node fail to itself
- Report match END indices; start = end − len(word) + 1
- Duplicate words: either dedupe on insert or accept duplicate reports — decide and state it
- Stream variant must keep only the state integer between queries, not the text
Tests
- [“he”,“she”,“his”,“hers”] × “ushers” → {(3,she), (3,he), (5,hers)}
- Nested: [“a”,“aa”,“aaa”] × “aaaa” → 9 matches
- LC 1032 example: words [“cd”,“f”,“kl”], stream “abcdefghijkl” → true exactly at d, f, l
- Empty text → no matches; pattern absent from text → no matches
- Stress: random pattern sets (1–6 words, length ≤ 4) over 2- and 3-letter alphabets vs the naive scanner — 400 trials, exact multiset equality
Follow-up Questions
- Why must outputs follow failure links transitively? → A word can be a proper suffix of a state’s path without that state being its trie endpoint: at state “hers”, both “hers” and (via fail) “ers”→…→“s”-chain words may match. One fail hop isn’t enough — “hers”→“ers”→“rs”→“s” may each hold words. Merging during BFS (or chasing dictionary links that skip non-word states) makes reporting O(1) per actual match, so total cost is O(matches), not O(|text| · chain length).
- Dense goto table vs on-the-fly failure walking? → Precomputing δ(state, char) for all chars gives branchless O(1) per character — right for hardware/regex engines and hot scanning loops — at states × alphabet memory (ruinous for Unicode). Failure walking is O(1) amortized with O(states) memory. Same asymptotics, different constants.
- Count occurrences of every pattern in one pass without listing matches? → Increment a counter at the state reached per character; afterwards, propagate counters along failure links. Key insight: the fail links form a tree rooted at 0 (every link goes strictly shallower), so accumulate in reverse-BFS order, child into fail-parent. O(states) postprocessing, no per-match cost.
- The dictionary changes online — insertions and deletions? → AC’s failure links are global; one insert can invalidate many. The standard workaround is log-structured: maintain ~log n automatons of doubling sizes, rebuild-merge on insert (amortized O(len · log n)), query all of them. Deletions: a parallel “negative” automaton, subtract counts.
- Where does this run in production? → Snort/Suricata match thousands of intrusion signatures per packet with AC variants; ClamAV’s core scanner is AC; WAFs and DLP filters scan request bodies against rule sets; bioinformatics matches primer/adapter sets against reads. Multi-pattern scanning at line rate is AC’s home turf.
- When is building AC the wrong call? → LC 1408-sized inputs (n ≤ 100, short words): naive
inis simpler, obviously correct, and fast enough. Interviews reward matching the tool to the constraints, not maximal machinery.
Product Extension
- Profanity/brand-safety filters over chat streams (the StreamChecker shape, literally)
- Malware signature and IDS packet scanning
- Log scanning for a large set of alert substrings in one pass
- Keyword spotting over transcripts; entity gazetteer matching in NLP pipelines
Language/Runtime Follow-ups
- Python: dict-based nodes are fine to Σ|patterns| ≈ 10^5;
pyahocorasick(C extension) for real workloads. - C++: array nodes
int next[26]+ dense δ table is the contest standard; handles 10^6 text trivially. - Java: flatten nodes into int arrays; object-per-node GC pressure hurts at scale.
- Go/Rust:
cloudflare/ahocorasick,aho-corasickcrate (the engine under ripgrep’s multi-literal matcher).
Common Bugs
- DFS instead of BFS for failure links: a child’s fail may point to a not-yet-processed node whose own fail/out is unset. Symptom: misses on deep overlapping patterns only.
- Missing output inheritance: reporting only
outof exact trie endpoints — “she” matches but the “he” inside it is silently dropped. The single most common AC bug. - Depth-1 self-loop: computing fail for root’s children via the generic walk can land a node on itself; then scanning loops forever on a mismatch.
- Off-by-one on match index: yielding start instead of end (or vice versa) — pick one, document it, test “ushers”.
- Resetting to root on mismatch: that’s the trie-walk-per-position brute force wearing AC’s clothes; correctness survives on some tests, complexity doesn’t.
- Stream variant re-walking the whole buffer per query: the entire point is carrying one state integer; O(len) per query means you rebuilt the naive solution.
Debugging Strategy
- Print (node path-string, fail target path-string) for every node on [“he”,“she”,“his”,“hers”]; check each fail is the longest proper suffix that’s a trie path
- Assert every fail target is strictly shallower than its node
- Test the nested case [“a”,“aa”,“aaa”] first — it breaks bugs 1, 2, and 5 immediately
- Diff against the naive scanner on random small-alphabet inputs before any performance test
Mastery Criteria
- Build trie + BFS failure links from memory in ≤ 25 min
- Explain why BFS order is required, in one sentence, without notes
- Solve LC 1032 both ways (reversed trie, AC) in ≤ 35 min and articulate the tradeoff
- State the amortized O(text + matches) argument (depth potential) on a whiteboard
- Given LC 1408’s constraints, say “naive, and here’s why” in under a minute
Lab 13 — Min-Cost Max-Flow
Goal
Implement min-cost max-flow via successive shortest augmenting paths (SPFA/Bellman-Ford, or Dijkstra with Johnson potentials), and use it to solve assignment-shaped optimization problems exactly — the cost-aware sibling of Dinic (lab-01) and bipartite matching (lab-02).
Background
Add a cost per unit of flow to each edge; among all maximum flows, find the cheapest. The residual graph now carries costs: pushing flow on edge (u,v, cost c) creates a reverse residual edge (v,u, cost −c) — undoing flow refunds its cost. Those negative edges are why plain Dijkstra fails and Bellman-Ford/SPFA appears.
Successive shortest paths (SSP): repeatedly find the cheapest s→t path in the residual graph (by cost, not hop count), push the bottleneck flow, repeat until no path. Each augmentation is greedy-optimal, and the classic exchange argument shows the invariant holds: after each phase, the current flow is the cheapest flow of its value.
Johnson potentials: maintain π(v) = shortest-path distance from the last phase; reduced cost c(u,v) + π(u) − π(v) is provably ≥ 0 on all residual edges, so Dijkstra works from the second phase on (first phase needs Bellman-Ford, or all costs start non-negative). This is the fast version; SPFA-only is the simple version.
Assignment problems: unit capacities, bipartite shape — workers→slots with costs. This is Hungarian-algorithm territory (O(n³)), but MCMF is the programmable alternative: one reusable class solves assignment, transportation, and k-of-n selection problems with only graph-construction changes.
Interview Context
- Codeforces / ICPC: standard; problems say “minimum total cost to satisfy demands”
- Quant / operations-research-adjacent roles: real signal
- LC 1066 Campus Bikes II (Hard, premium): the canonical LC assignment problem — intended answers are bitmask DP or MCMF
- LC 2172 Maximum AND Sum (Hard): assignment in disguise (numbers→slots); bitmask DP expected, MCMF works
- Standard FAANG: near zero as an implementation ask; occasionally as “how would you match drivers to riders optimally” design talk. Bitmask DP (phase-03 lab-08) covers the n ≤ 20 cases interviewers actually pose
When to Skip This Topic
Skip if any of these are true:
- You haven’t done Dinic (lab-01) and bipartite matching (lab-02) — MCMF assumes residual-graph fluency
- You’d reach for bitmask DP anyway and your targets keep n ≤ 20 (they do)
- Bellman-Ford is hazy — SSP is Bellman-Ford in a loop
This is the last flow lab for a reason. Skip unless you’re contest-bound or genuinely done with everything else.
Problem Statement
Campus Bikes II (LC 1066).
Given n workers and m bikes (n ≤ m) on a 2D grid, assign each worker exactly one distinct bike minimizing the sum of Manhattan distances. Return the minimum sum.
General framing: minimum-cost perfect matching of workers into slots — the template for LC 2172 (numbers→slots maximizing AND sum: negate to minimize) and hire/delivery assignment problems.
Constraints
- n ≤ m ≤ 10 on LC (bitmask DP works); the MCMF solution scales to n, m ≈ 500
- 0 ≤ coordinates < 1000, so costs < 2000 per edge
- Every worker must be assigned; bikes at most once
Clarifying Questions
- Must every worker get a bike? (Yes — maximize flow first, minimize cost among max flows.)
- Can two workers share a bike? (No — unit capacity on each bike.)
- Minimize total or worst-case distance? (Total. Min-max is a different problem: binary search + matching.)
- Costs ever negative? (Not here; in general MCMF handles them if there’s no negative cycle — SPFA covers it.)
Examples
workers = [[0,0],[2,1]], bikes = [[1,2],[3,3]] → 6
(worker0→bike0: 3, worker1→bike1: 3)
workers = [[0,0],[1,1],[2,0]], bikes = [[1,0],[2,2],[2,1]] → 4
(w0→b0: 1, w1→b1: 2, w2→b2: 1)
Greedy-nearest fails: two workers nearest to the same bike force a globally worse reassignment — exactly what the residual reverse edges repair.
Brute Force
Try all assignments: permutations of m bikes taken n at a time, m!/(m−n)! total. For n = m = 10: 3.6×10^6 — actually fine for LC 1066. For n = 20+: dead. Bitmask DP over bikes is O(n · 2^m); also dead past m ≈ 22.
Brute Force Complexity
- Time: O(m!/(m−n)! · n) permutations; O(m · 2^m) bitmask DP
- Space: O(n) / O(2^m)
Optimization Path
- Model as a flow network: source → each worker (cap 1, cost 0); worker i → bike j (cap 1, cost = Manhattan distance); each bike → sink (cap 1, cost 0).
- Max flow = n forces a complete assignment; min cost among max flows = optimal assignment.
- SSP: each phase, SPFA finds the cheapest augmenting path; push 1 unit; n phases total.
- Optional upgrade: keep potentials, switch to Dijkstra per phase — O(F · E log V) instead of O(F · V · E) worst case.
Final Expected Approach
from collections import deque
INF = float('inf')
class MCMF:
def __init__(self, n):
self.n = n
self.g = [[] for _ in range(n)] # edge: [to, cap, cost, rev_index]
def add_edge(self, u, v, cap, cost):
self.g[u].append([v, cap, cost, len(self.g[v])])
self.g[v].append([u, 0, -cost, len(self.g[u]) - 1])
def min_cost_flow(self, s, t, want=INF):
flow = cost = 0
while flow < want:
dist = [INF] * self.n
in_q = [False] * self.n
prev = [None] * self.n # (node, index into g[node])
dist[s] = 0
q = deque([s])
in_q[s] = True
while q: # SPFA: Bellman-Ford with a queue
u = q.popleft()
in_q[u] = False
for i, (v, cap, c, _) in enumerate(self.g[u]):
if cap > 0 and dist[u] + c < dist[v]:
dist[v] = dist[u] + c
prev[v] = (u, i)
if not in_q[v]:
q.append(v)
in_q[v] = True
if dist[t] == INF:
break # no augmenting path: done
push = want - flow
v = t
while v != s: # bottleneck along parent chain
u, i = prev[v]
push = min(push, self.g[u][i][1])
v = u
v = t
while v != s: # augment + open reverse edges
u, i = prev[v]
self.g[u][i][1] -= push
self.g[v][self.g[u][i][3]][1] += push
v = u
flow += push
cost += push * dist[t]
return flow, cost
def assign_bikes(workers, bikes): # LC 1066
n, m = len(workers), len(bikes)
S, T = n + m, n + m + 1
net = MCMF(n + m + 2)
for i in range(n):
net.add_edge(S, i, 1, 0)
for j in range(m):
net.add_edge(n + j, T, 1, 0)
for i, (wx, wy) in enumerate(workers):
for j, (bx, by) in enumerate(bikes):
net.add_edge(i, n + j, 1, abs(wx - bx) + abs(wy - by))
return net.min_cost_flow(S, T)[1]
Data Structures
- Adjacency list of edge records
[to, cap, cost, rev_index]; reverse edge found in O(1) viarev_index - SPFA: deque + in-queue flags + dist array + parent (node, edge) pairs
Correctness Argument
- Reverse edges with negated cost: rerouting flow off an edge refunds exactly what it cost — the residual graph encodes every possible modification of the current flow.
- SSP invariant: if flow f is the min-cost flow of value |f|, augmenting along a shortest (cheapest) residual path yields the min-cost flow of the new value. Exchange argument: any cheaper flow of that value would imply a negative-cost cycle in the residual of f, contradicting f’s optimality.
- No negative cycles initially (all costs ≥ 0) and SSP never creates one — so SPFA distances are well-defined at every phase.
- Termination: each phase pushes ≥ 1 unit (integer capacities); at most F phases.
Complexity
- SSP with SPFA: O(F · V · E) worst case; F ≤ n for unit-capacity assignment → O(n · V · E), and SPFA is near-linear on these sparse graphs in practice
- Dijkstra + potentials: O(F · E log V)
- Space: O(V + E); n = m = 500 → 250k assignment edges — fine
Implementation Requirements
- Store the reverse-edge index at insertion; never search for it
- SPFA needs in-queue flags (plain BFS relaxation is exponential on bad cases)
- Parent must record (node, edge index), not just node — parallel edges are legal
- Stop when
dist[t] == INF, not when flow “looks” complete wantparameter: for k-of-n selection problems, cap total flow at k- Negated costs for maximization problems (LC 2172): maximize Σvalue = minimize Σ(−value)
Tests
- LC 1066 example 1 → 6; example 2 → 4
- Colinear stress: 5 workers on y=0, 5 bikes on y=999 → 5×999 (identity pairing among equals)
- Two-path sanity: cheap low-cap path + expensive path — verify flow splits, cost = 10
- Rerouting case where optimal flow must push back through a reverse edge (verify against hand-computed 10)
- Stress: random n ≤ 4, m ≤ 5 grids vs brute-force permutations — 300 trials, exact equality
- Edge case: n = 1 (picks the single nearest bike)
Follow-up Questions
- Why not Dijkstra from the start? → Residual reverse edges have negative costs the moment any flow is pushed. Dijkstra’s greedy settling assumes no negative edges. Fix: Johnson potentials π(v) — reduced cost c + π(u) − π(v) ≥ 0 holds for all residual edges when π is the previous phase’s distance labels (edges on shortest paths get reduced cost exactly 0, and only those gain reverse edges). One initial Bellman-Ford (or zero, if costs start ≥ 0), then Dijkstra every phase.
- Worst-case complexity and why it doesn’t bite here? → O(F) phases, each O(V·E) SPFA. F can be huge for big capacities (scaling MCMF or cost-scaling fixes that). For assignment, F ≤ n and per-phase graphs are small — n = 500 assignment runs in well under a second in C++.
- When does the Hungarian algorithm win? → Dense square assignment with n in the thousands: Hungarian is O(n³) with tiny constants and no graph plumbing. MCMF wins on flexibility — unbalanced sides, capacities > 1, side constraints, k-of-n selection — with the same code.
- What are the potentials, really? → LP dual variables. Min-cost flow’s dual assigns node prices; complementary slackness says flow travels only on edges whose reduced cost is 0. Johnson’s π is a feasible dual solution, and SSP is primal-dual: it maintains dual feasibility while growing the primal.
- Minimize the maximum distance instead of the sum? → Different objective: binary search the threshold, keep only edges under it, check for perfect matching (lab-02). MCMF’s sum objective can’t express min-max directly.
- Real deployments? → Ad-delivery pacing (impressions→campaigns under budgets), fulfillment-center-to-order matching at Amazon scale (solved as min-cost flow relaxations), ride-hailing driver dispatch, and kidney-exchange programs (cycle/chain variants built on the same machinery).
Product Extension
- Rider-driver dispatch minimizing total ETA
- Order-to-warehouse routing with per-warehouse capacity (cap > 1 on slot→sink edges — one-line change)
- Ad impression allocation against advertiser budgets
- On-call/shift scheduling with per-person load limits and preference costs
Language/Runtime Follow-ups
- Python: fine to n ≈ 100 workers; list-based edge records beat Edge objects (attribute lookups dominate). PyPy or
scipy.sparse.csgraph.min_cost_flow-style libraries for bigger. - C++: the contest standard; struct-of-arrays edges,
std::dequeSPFA, handles 10^5 edges easily. - Java: flatten edges into parallel int arrays; per-edge objects thrash GC.
- Go: straightforward port; no surprises.
Common Bugs
- Missing or malformed reverse edge: cap 0 and cost −c are both mandatory; forgetting negation gives feasible flow with wrong (too high) cost — silently wrong, passes max-flow tests.
- Wrong rev_index: must be
len(g[v])before appending the reverse edge; off-by-one corrupts augmentation and caps go negative. - Dijkstra on residual costs without potentials: works on the first phase, wrong afterward — the nastiest version passes small tests where no rerouting is needed.
- Parent-by-node instead of parent-by-edge: with parallel edges (two workers, same cost) you may decrement the wrong edge’s capacity.
- No in-queue flag in SPFA: correctness survives, runtime explodes — the same node is queued O(E) times.
- Augmenting before checking dist[t]: pushing along a stale parent chain when t was never reached; guard with the INF check.
Debugging Strategy
- Assert the flow value equals n (all workers assigned) before trusting the cost
- After termination, verify anti-symmetry: every edge pair satisfies cap_forward + cap_backward = original cap
- Compare against brute-force permutations for n ≤ 4 — this catches cost bugs that max-flow tests can’t
- Print each phase’s (dist[t], push): dist[t] must be non-decreasing across phases; a decrease means a negative cycle or a broken relaxation
- Shrink failing stress cases by removing workers/bikes until minimal
Mastery Criteria
- Write the MCMF class from memory in ≤ 35 min
- Model LC 1066 as a flow network in ≤ 5 min (nodes, capacities, costs, why max-flow = assignment)
- Explain why reverse-edge costs are negated and why Dijkstra needs potentials, without notes
- State the SSP optimality invariant and sketch the exchange argument
- Given an assignment problem, choose correctly between bitmask DP (n ≤ 20), MCMF, and Hungarian — and justify in one sentence each
Lab 14 — Game Theory: Sprague-Grundy
Goal
Analyze impartial games — Nim, mex/Grundy numbers, XOR composition of independent subgames — and produce actual winning strategies (which move to make), not just win/lose answers.
Background
N/P positions. Under normal play (last player to move wins), a position is P (Previous player wins — i.e., the player to move loses) iff every move leads to an N position; N iff some move leads to a P position. Terminal positions are P.
Bouton’s theorem (1901): in multi-pile Nim, the player to move wins iff the XOR of pile sizes ≠ 0. Constructive: let x = XOR of all piles; find a pile p with p ^ x < p (one exists — take the pile with the highest set bit of x) and reduce it to p ^ x, restoring XOR = 0.
Sprague-Grundy theorem: every finite impartial game under normal play is equivalent to a Nim pile of size g(state), where g = mex (minimum excludant) of the Grundy values of successor states. A sum of independent games has Grundy value equal to the XOR of the components’ Grundy values. g = 0 ⟺ P position.
Misère play (last to move loses) is the trap: the theory only transfers for Nim when all piles are ≤ 1 (play Bouton except leave an odd number of 1-piles). General misère games are genuinely hard — no clean analogue exists.
Interview Context
- Codeforces / ICPC: regular; Div 2 D/E staple
- Quant firms (Jane Street, SIG brainteasers): Nim variants appear in first rounds
- Game/simulation companies: occasionally
- Standard FAANG: rare — LC 292 (trivial Nim) and minimax DP show up; full Grundy theory almost never
- LC anchors: 292 (Nim Game — the trivial n % 4 one), 1908 (Game of Nim, premium — actual multi-pile Nim), 294 (Flip Game II — Grundy on segments), 1510 (Stone Game IV — subtraction game)
When to Skip This Topic
Skip if any of these are true:
- You aren’t targeting competitive programming or quant interviews
- You haven’t mastered minimax with memoization (phase-05 game DP) — that solves 90% of interview game problems
- You confuse “optimal play” with “greedy play”
If Stone Game I–III aren’t reflexive yet, do those first. Grundy is the 10x tool for the 1x-frequency problem.
Problem Statement
Three parts, one theory:
(a) Multi-pile Nim (LC 1908). Given pile sizes, decide if the first player wins — and if so, return a winning move (pile index, new size).
(b) Flip Game II (LC 294). String of + and -; a move flips some "++" to "--"; player unable to move loses. Decide if the current player can guarantee a win.
(c) Generic engine. Given any impartial game as a moves(state) → successors function, compute g(state) via memoized mex.
Constraints
- Nim: up to 10^4 piles, sizes up to 10^9 — Bouton must be O(n), no game search
- Flip game: string length up to 10^5 (LC caps at 60; the Grundy solution doesn’t care)
- Generic engine: reachable state space small enough to memoize (≤ ~10^6 states)
Clarifying Questions
- Normal or misère play? (Normal — last player to move wins. Misère changes everything.)
- Is the game impartial — same moves available to both players? (Yes; otherwise Grundy theory does not apply.)
- Both players play optimally? (Yes.)
- Do you need the winning move or just win/lose? (Move for Nim; win/lose for flip game.)
- Is the game finite and acyclic? (Yes — every move strictly reduces something.)
Examples
Nim [3, 4, 5]: XOR = 2 ≠ 0 → first player wins.
Move: 3 ^ 2 = 1 < 3 → reduce pile 0 from 3 to 1, leaving [1, 4, 5], XOR = 0.
Nim [1, 2, 3]: XOR = 0 → first player loses.
"++++" one run of 4: g(4) = 2 ≠ 0 → win
"+++++" one run of 5: g(5) = 0 → lose
"++-++++" runs 2 and 4: g(2) ^ g(4) = 1 ^ 2 = 3 → win
"++-++" runs 2 and 2: g(2) ^ g(2) = 0 → lose
Brute Force
Full-game minimax with memoization: win(state) = any(not win(next)). Correct for any game, impartial or not — but the state is the entire position. Flip game on n chars has up to 2^n reachable states; Nim state space is the product of pile sizes. Composition is invisible to minimax: it re-explores the cross product of subgames.
Brute Force Complexity
- Time: O(states × moves) — exponential in the number of independent components
- Space: O(states) memo
Optimization Path
- Verify impartial + normal play (the two preconditions — check them out loud).
- Decompose the position into independent subgames (Nim piles; maximal
+runs in flip game). - Compute g per subgame: mex over successor Grundy values, memoized. A flip in a run of k at offset i splits it into runs of i and k − i − 2 → successor value g(i) ^ g(k − i − 2).
- XOR all component Grundy values; nonzero → N position (win).
- For pure Nim, skip step 3: g(pile) = pile size, and Bouton gives the move in O(n).
Final Expected Approach
def mex(s):
g = 0
while g in s:
g += 1
return g
def grundy(state, moves, memo):
"""Generic Sprague-Grundy: moves(state) -> iterable of successor states."""
if state in memo:
return memo[state]
memo[state] = mex({grundy(nxt, moves, memo) for nxt in moves(state)})
return memo[state]
def nim_winning_move(piles):
"""Bouton: return (pile_index, new_size), or None if this is a P position."""
x = 0
for p in piles:
x ^= p
if x == 0:
return None # every move loses; no winning move
for i, p in enumerate(piles):
if p ^ x < p: # p has the highest set bit of x
return (i, p ^ x) # after the move, XOR of all piles = 0
FLIP = {0: 0, 1: 0} # runs of 0 or 1 '+' have no moves
def flip_run_grundy(k):
"""Grundy number of a maximal run of k consecutive '+' (LC 294)."""
if k not in FLIP:
FLIP[k] = mex({flip_run_grundy(i) ^ flip_run_grundy(k - i - 2)
for i in range(k - 1)})
return FLIP[k]
def can_win_flip_game(s):
"""LC 294: True iff the current player has a winning strategy."""
g, run = 0, 0
for c in s + '-': # sentinel flushes the last run
if c == '+':
run += 1
else:
g ^= flip_run_grundy(run)
run = 0
return g != 0
def nim_first_player_wins(piles):
"""LC 1908 / LC 292-general: N position iff XOR != 0."""
x = 0
for p in piles:
x ^= p
return x != 0
Grundy values for flip-game runs 0..15: 0 0 1 1 2 0 3 1 1 0 3 3 2 2 4 0 — no obvious period; memoize, don’t guess a formula.
Data Structures
- Dict memo: state → Grundy value (state must be hashable/canonical)
- Set of successor Grundy values per state (for mex)
- Pile array for Bouton — no tree, no search
Correctness Argument
- mex gives Nim-equivalence: from g(s) = k you can move to every value 0..k−1 (mex guarantees they occur among successors) and never stay at k — exactly the move set of a Nim pile of size k. Moves to values > k exist but don’t help: opponent moves back.
- XOR for sums (copying strategy): if the XOR of component Grundy values is 0, any move breaks the balance in one component and the opponent restores XOR = 0 (Bouton’s constructive move applied to Grundy values); P positions are preserved, terminal is P, so XOR = 0 characterizes P positions.
- Bouton is the special case g(pile) = pile size.
Complexity
- Bouton Nim: O(n) time, O(1) space
- Flip game: g(k) needs k − 1 successor XORs (O(1) each once smaller values are memoized) plus an O(k) mex → O(L²) total for max run length L, then O(n) to scan the string
- Generic engine: O(states × branching) time, O(states) space
Implementation Requirements
- Terminal states must get g = 0 (mex of empty set) — verify your recursion hits this
- Canonicalize state before memoizing (sort piles; tuples not lists)
- mex via set membership loop, not
min/max - Winning move must be validated: resulting XOR = 0
- Recursion depth: run lengths up to 10^4 need iterative or increased limit in Python
Tests
- Nim [3,4,5] → win, move (0, 1); [1,2,3] → lose, move is None
- Exhaustive: all flip-game strings of length ≤ 12 vs memoized minimax on the raw string — must agree
- Exhaustive: all 3-pile Nim positions with sizes ≤ 6 vs minimax — XOR rule must agree
- Every winning move returned by Bouton, when applied, yields a brute-force loss for the opponent
- Generic engine on single Nim pile: g(n) = n; on subtraction game {1,2,3}: g(n) = n mod 4 (LC 292)
- LC 294: “++++” → True, “+” → False
Follow-up Questions
- Why XOR and not addition? → XOR is Nim addition: it’s the unique operation making the copying strategy work. From XOR = 0, any move to one component makes XOR ≠ 0; from XOR = x ≠ 0, Bouton’s move restores 0. Addition fails: [1,1] sums to 2 but is a P position.
- What if you used min or max instead of mex? → min breaks the “reach every smaller value” property; max breaks “can’t stay equal.” mex is exactly what makes g(s) behave as a Nim pile — both directions of the equivalence are needed.
- Does this work for chess or checkers? → No — those are partisan games (players have different moves). The theory there is Conway’s surreal numbers / combinatorial game theory (On Numbers and Games, Winning Ways). Grundy requires impartiality.
- Staircase Nim? → Coins on stairs, move coins down one step. Only odd-indexed stairs matter: XOR those pile sizes. Moves on even stairs are mirrored by the opponent.
- Wythoff’s game? → Two piles; take from one pile or equally from both. P positions are (⌊kφ⌋, ⌊kφ²⌋) — golden ratio. Famous because it’s NOT plain XOR; mex tables reveal the Beatty sequence.
- When should you skip Grundy and just do minimax? → Small state space, partisan moves, or scoring objectives. Stone Game I/II/III maximize score difference — that’s minimax DP (phase-05), and Grundy theory says nothing about scores.
Product Extension
- Game AI for turn-based games: endgame solvers and perfect-play databases
- Puzzle apps: hint engines that must produce a move, not just “you’re losing”
- CS education platforms: auto-graders for game-theory assignments need reference strategies
- Combinatorial auction / mechanism toy models occasionally reduce to game values
Language/Runtime Follow-ups
- Python:
functools.lru_cacheon a tuple state is idiomatic; recursion limit bites at depth ~1000 — precompute run Grundy values bottom-up for large inputs. - C++: memo via
unordered_mapor flat arrays; mex via a boolean scratch array is faster than a set. - Java:
HashMap<Long, Integer>with packed state; watch autoboxing in hot loops. - All: XOR of pile sizes up to 10^9 fits any 64-bit int; no overflow concerns.
Common Bugs
- mex computed as min or max: the classic. mex({1, 2}) = 0, not 1 or 2.
- XORing raw sizes in non-Nim games: g(state) equals the pile size only in pure Nim. In flip game, XOR the Grundy values of runs, never the run lengths.
- Terminal position mishandled: mex(∅) = 0 must fall out naturally; special-casing it to −1 or 1 poisons everything upstream.
- Misère treated as normal play: “last player to move loses” flips the answer only near the end — for Nim, deviate from Bouton only when the move would leave all piles ≤ 1.
- Split off-by-one: flipping at offset i in a run of k leaves runs of i and k − i − 2 (the two flipped chars vanish), for i in 0..k−2. Writing k − i − 1 passes small cases and fails at k = 4.
- Non-canonical memo keys: memoizing [2,1] and [1,2] separately wastes memory; memoizing a mutated list corrupts the cache.
Debugging Strategy
- Print the Grundy table g(0..20) for each subgame type; check g = 0 exactly at hand-verified P positions
- Diff Grundy-predicted win/lose against raw minimax on every instance small enough to enumerate — this catches all composition bugs
- Validate every claimed winning move by applying it and asserting the resulting position has XOR 0
- If Grundy values look periodic, test the conjectured period 3× further before relying on it
Mastery Criteria
- State and prove Bouton’s theorem (both directions) in ≤ 10 min
- Implement the generic mex/Grundy engine from scratch in ≤ 15 min
- Solve LC 294 via segment decomposition in ≤ 25 min
- Produce the winning Nim move, not just the verdict, in ≤ 5 min
- Correctly classify a new game as impartial-normal (Grundy), misère (careful), or partisan/scoring (minimax) in ≤ 3 min
- Explain why Stone Game I–III are NOT Grundy problems
Lab 15 — Persistent Segment Tree
Goal
Build a version-preserving segment tree via path copying — each update creates a new O(log n)-node version while sharing the rest — enabling queries against any historical version, and the canonical application: k-th smallest in a range in O(log n) per query.
Background
Path copying: an update to a segment tree touches one root-to-leaf path — O(log n) nodes. Instead of mutating, allocate fresh copies of exactly those nodes; their untouched children are shared with the previous version. Each version is just a root id; all versions stay queryable forever.
The canonical application — k-th smallest in range [l, r]: build version i as a count tree over the compressed value axis after inserting a[0..i−1] (prefix trees). Then cnt(root[r+1]) − cnt(root[l]) at any tree node gives how many elements of a[l..r] fall in that value interval — subtraction of versions works node-by-node because the trees are structurally identical. Walk both roots down simultaneously like an order-statistics tree: if the left subtree difference ≥ k go left, else subtract and go right.
Alternatives (know the trade): Mo’s algorithm (phase-03 lab-12) answers offline in O((n + q)√n) with a frequency structure; a merge sort tree answers via binary search in O(log³ n) per query. Persistence is online, O(log n) per query, and handles “query version t” for free.
Persistence generalizes: persistent DSU (rollback / partial persistence), persistent tries — a persistent binary trie with the same prefix-difference trick solves max-XOR-with-element-in-range.
Interview Context
- Codeforces Div 1 / ICPC: standard tool; MKTHNUM-style problems are a recognized genre
- Competitive programming interviews (quant, HFT): plausible
- Systems design adjacency: MVCC snapshots, functional data structures — concept, not code
- Standard FAANG: implementation essentially never asked; ordinary segment trees are already rare
- LC has no exact k-th-smallest-in-range problem; nearest neighbors (LC 315 family) are solved with BIT/merge sort. Anchor is SPOJ MKTHNUM / the Codeforces standard.
When to Skip This Topic
Skip if any of these are true:
- You aren’t targeting ICPC or competitive-programming-flavored interviews
- Ordinary segment trees (build/update/query) aren’t automatic yet
- You haven’t done coordinate compression under time pressure
This is competitive programming infrastructure. For FAANG prep, the transferable 20% is the idea of structural sharing — read Background, skip the drill.
Problem Statement
K-th smallest number in a range (SPOJ MKTHNUM semantics).
Given an array a of n integers and q queries (l, r, k), answer for each: the k-th smallest element among a[l..r] (1-indexed k; k = 1 is the minimum). Queries are online — no batching, no reordering.
Constraints
- n ≤ 10^5, q ≤ 10^4
- values up to 10^9 in magnitude, duplicates allowed → coordinate compression mandatory
- 1 ≤ l ≤ r ≤ n, 1 ≤ k ≤ r − l + 1
- Memory: O((n + q) log n) nodes — must fit; this kills naive per-version copies
Clarifying Questions
- Are queries online or can I batch them? (Online — otherwise Mo’s or offline sweeps compete.)
- Duplicates in the array? (Yes — count trees handle them; each occurrence counts separately.)
- Is the array immutable after build? (Yes for MKTHNUM; updates change the tool — see follow-ups.)
- k is 1-indexed? (Yes; k-th smallest, ties by multiplicity.)
- Value range? (Up to 10^9 — compress to ranks, answer maps back through the sorted values.)
Examples
a = [2, 6, 3, 1, 6] (1-indexed positions 1..5)
query (1, 5, 3): sorted slice [1, 2, 3, 6, 6] → 3
query (2, 4, 2): slice [6, 3, 1] sorted [1, 3, 6] → 3
query (1, 5, 5): → 6 (duplicate 6 counted twice)
query (3, 3, 1): → 3 (single element)
Brute Force
Per query: sorted(a[l-1:r])[k-1]. O(r − l) copy + O(m log m) sort per query. For n = 10^5, q = 10^4 with full-range queries: ~10^9 × log — TLE in any language that isn’t cheating. Quickselect drops the log but keeps the O(n) copy per query.
Brute Force Complexity
- Time: O(q · n log n)
- Space: O(n) per query slice
Optimization Path
- Compress values to ranks 0..m−1 (sorted uniques); answers map back via the sorted array.
- Build version 0 = empty tree over the value axis. Version i = version i−1 with rank(a[i]) inserted (count +1 along one path). Path copying: O(log m) new nodes per version.
- For query (l, r, k): the multiset of a[l..r] is version r “minus” version l−1 — valid per node since all versions share shape over the same value axis.
- Descend both roots in lockstep:
left_count = cnt[left[root_r]] − cnt[left[root_l]]; k ≤ left_count → recurse left, else k −= left_count, recurse right. Leaf index is the answer’s rank.
Final Expected Approach
class PersistentSegTree:
"""Count tree over compressed value axis [0, m). Node 0 = shared empty tree."""
def __init__(self, m):
self.m = m
self.left, self.right, self.cnt = [0], [0], [0] # node 0: all zeros
def _node(self, l, r, c):
self.left.append(l); self.right.append(r); self.cnt.append(c)
return len(self.cnt) - 1
def update(self, prev, pos):
"""New version = prev with count at pos incremented. prev is untouched."""
return self._update(prev, pos, 0, self.m - 1)
def _update(self, node, pos, lo, hi):
if lo == hi:
return self._node(0, 0, self.cnt[node] + 1)
mid = (lo + hi) // 2
if pos <= mid:
return self._node(self._update(self.left[node], pos, lo, mid),
self.right[node], self.cnt[node] + 1)
return self._node(self.left[node],
self._update(self.right[node], pos, mid + 1, hi),
self.cnt[node] + 1)
def kth(self, root_l, root_r, k):
"""k-th smallest (1-indexed) in the version difference root_r − root_l."""
lo, hi = 0, self.m - 1
while lo < hi:
mid = (lo + hi) // 2
in_left = self.cnt[self.left[root_r]] - self.cnt[self.left[root_l]]
if k <= in_left:
root_l, root_r = self.left[root_l], self.left[root_r]
hi = mid
else:
k -= in_left
root_l, root_r = self.right[root_l], self.right[root_r]
lo = mid + 1
return lo # rank of the answer
def build_kth_oracle(a):
"""query(l, r, k): k-th smallest of a[l..r], 0-indexed inclusive bounds."""
vals = sorted(set(a))
rank = {v: i for i, v in enumerate(vals)}
pst = PersistentSegTree(len(vals))
roots = [0] # roots[i] covers a[0..i-1]
for x in a:
roots.append(pst.update(roots[-1], rank[x]))
def query(l, r, k):
return vals[pst.kth(roots[l], roots[r + 1], k)]
return query
Note the null-node trick: node 0 is a real all-zero node whose children are itself — descending into unbuilt subtrees needs no None checks, and version 0 (empty tree) is just root 0.
Data Structures
- Three parallel lists
left[],right[],cnt[]— nodes as array slots, not objects roots[]: version i’s root id; the entire “history” is this list of ints- Sorted unique values + rank dict for coordinate compression
Correctness Argument
- Prefix property: version i counts exactly a[0..i−1]; counts are non-negative and monotone in i, so cnt_r − cnt_l at any node = occurrences of that value interval within a[l..r].
- Shape invariance: every version is a tree over the same interval decomposition of [0, m); node-wise subtraction is therefore meaningful even across shared/unshared nodes.
- Descent invariant: entering a subtree, k is the rank of the answer within that subtree’s value interval restricted to a[l..r]; the left-count comparison preserves it. At a leaf, interval = single rank = answer.
- Persistence:
_updatenever writes to an existing slot — old roots are immutable by construction.
Complexity
- Build: n updates × O(log m) new nodes → O(n log n) time and space
- Query: O(log m) descent, O(1) extra space
- Total memory: O((n + q) log n) node slots (q term appears once updates enter the picture)
Implementation Requirements
- Node 0 must be a genuine empty node (cnt 0, children 0) — no None/sentinel branching
- Never mutate an existing node; every changed node is a fresh append
- Coordinate compression built once, from the whole array, before any update
- Iterative
kth(depth ~17 recursion is fine in Python, but iterative is free) - Query answers must map ranks back to original values
Tests
- Worked example above: a = [2, 6, 3, 1, 6], all four queries
- Randomized stress: 200 random arrays (n ≤ 60, values −50..50) × 50 random (l, r, k) vs
sorted(a[l:r+1])[k-1]— exact match - Duplicates-heavy stress: values drawn from {0..3}
- Version isolation: after full build, query old roots (0, roots[i]) for every prefix and every k — history must be intact
- Scale: n = 10^5 build + 10^4 random queries; measure time and node count (~n log n ≈ 1.7 × 10^6 nodes)
- Edge: n = 1; l = r; k = 1 and k = r − l + 1
Follow-up Questions
- Memory analysis? → O(n log n) node slots for the build, O(log n) per subsequent update. In Python, three flat lists of ints beat node objects by a large constant: no per-object header (~56 bytes each), no attribute dicts, better cache behavior; in CPython this is often the difference between AC and MLE.
- k-th smallest with point updates? → Prefix trees break (one update dirties all later versions). Standard fix: a BIT (Fenwick) indexed by position whose cells are segment-tree roots over the value axis; a query walks O(log n) roots simultaneously → O(log² n) per op.
- Max XOR with an element in a[l..r]? → Same prefix-difference trick on a persistent binary trie over bits: descend choosing the opposite bit where
cnt_r − cnt_l > 0proves existence in range. - Does anything in production actually work like this? → Yes — Clojure and Scala vectors are persistent 32-ary tries (path copying with 32-way branching, ~log₃₂ n depth); git’s object model and MVCC database snapshots are the same structural-sharing idea.
- Other structures answering k-th-in-range? → Wavelet trees: O(log σ) per query, one structure, no versions. Merge sort tree: O(log³ n), simpler. Fractional cascading speeds repeated binary searches in layered structures. Name-drop level: know they exist and their trade-offs.
- Offline alternatives? → Sort queries via Mo’s algorithm (phase-03 lab-12) with a value-frequency BIT: O((n + q)√n log n) — worse here, but wins when the per-element update is O(1) and queries are batchable.
Product Extension
- Time-travel queries: “what was the p95 latency distribution as of version t” over append-only metrics
- MVCC storage engines: snapshot reads without blocking writers (Postgres, LMDB copy-on-write B-trees)
- Version control internals: content-addressed trees sharing unchanged subtrees (git)
- Immutable state in UIs: undo/redo stacks with structural sharing rather than deep copies
Language/Runtime Follow-ups
- Python: parallel lists as shown; preallocating capacity helps little — append is amortized O(1); PyPy or arrays (
array('i')) if TLE. - C++: the reference implementation — struct pool with
intindices,newis too slow; ~40 bytes/node × 2M nodes is nothing. - Java: int arrays sized 4 × (n log n) upfront; avoid Integer boxing in the pool.
- Clojure/Scala: persistence is the default — but 32-ary tries don’t give you the count-difference descent; you’d still hand-roll this.
Common Bugs
- Mutating a shared node: writing
cnt[node] += 1instead of allocating — silently corrupts every earlier version; the version-isolation test exists to catch exactly this. - Prefix off-by-one: the query pair is (roots[l], roots[r + 1]) for 0-indexed inclusive [l, r] — equivalently (roots[l−1], roots[r]) 1-indexed. Mixing conventions gives answers shifted by one element.
- Compression per query: ranks must come from the global sorted array; compressing a slice changes the axis and all counts.
- Null-node branching: using None children forces checks at every step and breaks the lockstep descent; the self-referential zero node eliminates the whole bug class.
- k index base: kth expects 1-indexed k. Feeding a 0-indexed k breaks immediately at the minimum (k = 0 never satisfies k ≤ in_left with in_left = 0 correctly) — always test k = 1 and k = r − l + 1.
- Returning the rank, not the value: the descent yields a compressed index; forgetting
vals[...]is invisible on arrays that happen to equal their ranks — test with values like 10^9.
Debugging Strategy
- Query (0, roots[n], k) for k = 1..n and compare against the fully sorted array — validates build + descent without range subtraction
- Then fix l = 0 and vary r (pure prefixes), then general (l, r) — isolates which of the three layers is broken
- Assert
cnt[left[x]] + cnt[right[x]] == cnt[x]for every internal node after build - Print node-pool size after build: far from n·⌈log₂ m⌉ + 1 means paths are wrong or sharing is broken
Mastery Criteria
- Implement build + update + kth from scratch, bug-free vs stress test, in ≤ 40 min
- State the memory bound and the node-pool-vs-objects argument in ≤ 3 min
- Explain why version subtraction is valid (shape invariance) without prompting
- Adapt the prefix-difference trick to a persistent trie (max-XOR in range) in ≤ 30 min
- Choose correctly between persistence, Mo’s, merge sort tree, and wavelet tree given constraints in ≤ 5 min
Phase 13 — Frontier: Advanced Search & Large-Number Algorithms
Read this before doing anything in this phase.
This phase sits beyond Phase 12. Where Phase 12 covers Codeforces/ICPC material, this phase covers two families that appear in even narrower contexts:
- State-space search at astronomical scale — solving problems whose state graphs have 10^13 to 10^19 nodes (15-puzzle, Rubik’s Cube) via IDA*, pattern databases, bidirectional search, coordinate compression of group elements, and exact-cover machinery.
- Large-number algorithms — arbitrary-precision arithmetic, fast primality testing and factorization at 64-bit scale, modular systems, discrete logarithms, and computing the 10^18-th term of a recurrence.
The audience:
- Puzzle/robotics/planning engineers (motion planning, combinatorial solvers)
- Cryptography engineers and researchers (the number-theory half is their bread and butter)
- Quant developers at the most algorithmic desks
- Competitive programmers past ICPC regionals
- Engineers interviewing at companies that ship solvers (SAT/SMT, OR, scheduling, chip placement)
- Anyone asked “how would you solve a Rubik’s Cube optimally?” and expected to answer for real
If you are not on this list, skip this phase entirely. The opportunity cost argument from the Phase 12 README applies with double force here.
When to Use This Phase
Use this phase if all are true:
- You have completed Phases 1–11 (and the relevant parts of 12) and pass mocks at your target level.
- The role explicitly involves search, solvers, cryptography, or number-theoretic code.
- You have months, not weeks, before the interview.
- You genuinely enjoy this material — at this depth, motivation is the binding constraint.
When to Skip This Phase
Skip if any is true:
- You are interviewing for any standard SWE role at any level, including staff+ at FAANG.
- You haven’t finished Phase 12’s relevant labs (FFT is a prerequisite for two labs here).
- You are behind on Phase 10 (testing) or Phase 11 (mocks). Those decide offers; this does not.
What’s In This Phase
| Lab | Topic | When it appears |
|---|---|---|
| 01 | IDA* & Pattern Databases (15-puzzle) | Planning/robotics, search research, game AI |
| 02 | Kociemba’s Two-Phase Algorithm (Rubik’s Cube) | Puzzle solvers, group-theoretic search, robotics demos |
| 03 | Bidirectional BFS on Implicit Graphs | Word Ladder-class problems; the one lab here FAANG does ask |
| 04 | Algorithm X & Dancing Links (Exact Cover) | Sudoku/tiling solvers, OR, constraint solving |
| 05 | Big-Integer Arithmetic & Karatsuba | Runtime/library engineering, “how does Python multiply big ints?” |
| 06 | Miller-Rabin & Pollard’s Rho | Cryptography, CTF, Project Euler-grade number theory |
| 07 | Extended Euclid, Modular Inverse & CRT | Cryptography, competitive programming staple |
| 08 | Discrete Logarithm: Baby-Step Giant-Step | Cryptography interviews, CTF |
| 09 | Matrix Exponentiation & Linear Recurrences | Competitive programming regular, quant |
| 10 | Exact TSP: Held-Karp & Branch-and-Bound | OR/logistics, bitmask-DP interviews (LC Hard) |
Each lab uses the standard 23-section format plus the “When to Skip This Topic” section, same as Phase 12.
Dependency notes:
- Lab 01 → Lab 02: Kociemba is IDA* + pattern-database ideas applied twice. Do 01 first.
- Lab 03 is the most interview-relevant lab in the phase (Word Ladder is a genuine FAANG question) — if you cherry-pick one, pick it.
- Labs 05–08 form a number-theory sequence; Phase 7 Labs 01–03 (modular arithmetic, binary exponentiation, sieve) are prerequisites.
- Phase 12 Lab 07 (FFT) pairs with Lab 05 (the multiplication ladder) and Lab 07 (Garner reconstruction).
How to Use This Phase
- Read this README in full.
- Search your target job descriptions for: solver, planning, search, cryptography, number theory, optimization, exact algorithms.
- Cherry-pick matching labs. Doing the whole phase end-to-end is almost never the right call.
- Exception: Lab 03 (bidirectional BFS) and Lab 09 (matrix exponentiation) are cheap and general enough to justify for anyone doing Phase 12.
Realistic Expectations
- Kociemba’s algorithm will almost certainly never be an interview question. Its value is that implementing it forces mastery of state encoding, admissible heuristics, precomputed tables, and search pruning — the transferable skill set behind every hard search problem.
- The number-theory labs (05–08) do get asked nearly verbatim in cryptography-adjacent roles and CTF-style screens.
- This phase, like Phase 12, is not graded against the standard passing bar. It is deep-end material for a small group.
If you’re here to procrastinate Phase 10 or 11 — stop. That’s the failure mode that costs offers.
After This Phase
There is no Phase 14. If you complete the labs relevant to your target role, return to Phase 11 and convert the depth into mock performance.
Lab 01 — IDA* & Pattern Databases (15-Puzzle)
Goal
Implement IDA* to solve the 15-puzzle optimally with a Manhattan-distance heuristic, then upgrade the heuristic: linear conflict, then a disjoint 6-6-3 pattern database.
Background
A* stores every generated node. The 15-puzzle has 16!/2 ≈ 1.05×10^13 reachable states; the hardest instances need 80 optimal moves, and A*’s open list grows as O(b^d) with effective branching factor ≈ 2.13 — RAM dies long before depth 50.
Korf (1985), IDA:* depth-first search with an f = g + h cutoff, restarted with the minimum f-value that exceeded the previous cutoff. O(d) memory. First algorithm to solve random 15-puzzle instances optimally (average optimal length ≈ 53).
Korf & Felner (2002), disjoint pattern databases: precompute exact solve costs for disjoint subsets of tiles by backward BFS from the goal; sum the lookups. Their 7-8 split cut Manhattan’s ~4×10^8 average nodes to ~4×10^4.
Admissibility (h never overestimates) gives optimality; consistency (h(u) ≤ 1 + h(v) for a move u→v) makes f non-decreasing along paths. Manhattan distance is both.
Solvability: on a 4×4 board, a state is solvable iff (tile inversions) + (blank row counted from the bottom, 1-based) is odd. Half of all arrangements are unreachable.
Interview Context
- Robotics / motion planning / game-AI roles: heuristic search is core material
- Heuristic-search research (ICAPS, SoCS): the canonical benchmark
- ICPC / Codeforces: IDA* appears occasionally in puzzle problems
- Standard FAANG: LeetCode 773 (Sliding Puzzle) is 2×3 and plain BFS suffices; IDA* and PDBs essentially never required
When to Skip This Topic
Skip if any of these are true:
- You aren’t targeting robotics, planning, game-AI, or research roles
- Plain BFS/DFS and A* on explicit graphs (phase-04) aren’t automatic yet
- Your loop is standard FAANG — LC 773 at 2×3 scale is BFS, done
If you can’t state why A* with an admissible heuristic is optimal, fix that first. This is depth beyond any standard interview loop.
Problem Statement
Optimal 15-Puzzle.
Given a 4×4 board containing tiles 1–15 and one blank (0), return a shortest move sequence reaching the goal (tiles in row-major order, blank last), or report the state unsolvable. A move slides a tile adjacent to the blank into the blank; name moves by the direction the blank travels (U/D/L/R).
Constraints
- Board is 4×4; input is a permutation of 0–15
- Optimal lengths range 0–80; random instances average ≈ 53
- Pure-Python IDA* + Manhattan: instances up to ~40 optimal moves in seconds; beyond that needs linear conflict, a PDB, or C
- Memory: O(d) for the search; the 6-6-3 PDB adds ~11.5 MB
Clarifying Questions
- Must the solution be optimal, or merely valid? (Optimal — otherwise greedy best-first solves instantly and the lab is pointless.)
- Is the input guaranteed solvable? (No — detect the unsolvable half via the parity test before searching.)
- Are moves named by tile motion or blank motion? (Either; state your convention. Blank motion here.)
- One shortest solution or all of them? (Any one.)
Examples
3×3 warm-up (goal = 1..8 row-major, blank last), showing f = g + h cutoffs:
2 3 1
4 5 6 h = Manhattan = 4 (tile 2: 1, tile 3: 1, tile 1: 2)
7 8 _
IDA* iterations (next cutoff = smallest pruned f):
bound 4 → no goal, min pruned f = 6
bound 6 → 8
bound 8 → 10
bound 10 → 12
bound 12 → 14
bound 14 → 16
bound 16 → 16-move solution (optimal); h underestimated by 12
Unsolvable, detected by parity — no search:
1 3 2
4 5 6 3×3 (odd width): solvable iff inversions even; here 1 → unsolvable
7 8 _
4×4: Sam Loyd's "14-15 puzzle" (goal with 14, 15 swapped):
inversions = 1, blank row from bottom = 1, sum = 2 (even) → unsolvable
Brute Force
Plain BFS with a visited set. Depth 20 ≈ 2.13^20 ≈ 4×10^6 states — fine. Depth 25 ≈ 1.6×10^8 states ≈ 16 GB at ~100 B/state — memory dies around depth 20–25. Average instance needs depth 53.
Brute Force Complexity
- Time: O(b^d), b ≈ 2.13
- Space: O(b^d) — the killer
Optimization Path
- BFS → A* + Manhattan: far fewer expansions, still O(b^d) memory.
- A* → IDA*: DFS with f-cutoff, restart at the min pruned f. O(d) memory; re-expansion is a constant factor (geometric growth — see follow-ups).
- Manhattan → linear conflict: +2 for each pair of tiles in their goal row (or column) in reversed order — they must pass around each other. Still admissible.
- → disjoint 6-6-3 pattern database: exact costs for tile groups {6, 6, 3}, built by backward BFS over abstracted states. 16P6 = 5,765,760 entries per 6-group, 1 byte each → ~11.5 MB. Node counts drop ~1000× vs Manhattan.
Final Expected Approach
GOAL = tuple(range(1, 16)) + (0,)
def solvable(board):
tiles = [t for t in board if t]
inv = sum(1 for i in range(15) for j in range(i + 1, 15)
if tiles[i] > tiles[j])
blank_row_from_bottom = 4 - board.index(0) // 4
return (inv + blank_row_from_bottom) % 2 == 1
def manhattan(board):
d = 0
for i, t in enumerate(board):
if t:
g = t - 1
d += abs(i // 4 - g // 4) + abs(i % 4 - g % 4)
return d
def neighbors(board):
z = board.index(0)
r, c = divmod(z, 4)
for dr, dc, m in ((-1, 0, 'U'), (1, 0, 'D'), (0, -1, 'L'), (0, 1, 'R')):
nr, nc = r + dr, c + dc
if 0 <= nr < 4 and 0 <= nc < 4:
nz = nr * 4 + nc
nb = list(board)
nb[z], nb[nz] = nb[nz], nb[z]
yield tuple(nb), m
def ida_star(start):
path = [start]
moves = []
def search(g, bound):
board = path[-1]
f = g + manhattan(board)
if f > bound:
return f
if board == GOAL:
return True
best = float('inf')
for nb, m in neighbors(board):
if len(path) > 1 and nb == path[-2]:
continue
path.append(nb)
moves.append(m)
t = search(g + 1, bound)
if t is True:
return True
best = min(best, t)
path.pop()
moves.pop()
return best
bound = manhattan(start)
while True:
t = search(0, bound)
if t is True:
return moves
if t == float('inf'):
return None
bound = t
def solve(board):
board = tuple(board)
if not solvable(board):
return None
return ida_star(board)
Upgrades: replace manhattan with Manhattan + linear conflict, or with sum(pdb[i][key_i(board)] for i in range(3)) for the 6-6-3 PDB. Recompute h incrementally (only the moved tile’s delta) for ~16× fewer heuristic ops.
Data Structures
- Board as a 16-tuple: hashable, cheap to copy
- Explicit path + moves stacks: parent pruning and path return
- PDB: flat byte array keyed by the pattern tiles’ positions
Correctness Argument
- Manhattan is admissible: each move changes one tile’s position by 1, so every tile needs ≥ its distance; consistent: h changes by ±1 per move while g changes by 1.
- IDA* optimality: each cutoff is the exact minimum f pruned previously, so no path of cost < bound was ever discarded; the first goal found within the current bound is therefore optimal.
- Parity: every move preserves (inversions + blank row from bottom) mod 2; the goal’s sum is odd, so even-sum states are unreachable.
Complexity
- Time: O(b^d) worst case; heuristic quality sets the effective exponent. Iterative deepening multiplies by r/(r−1), where r = per-iteration node growth ratio (≈ 4.5 here → ~1.3× overhead)
- Space: O(d) — the whole point
Implementation Requirements
- Prune the immediate parent; without it branching rises from ≈ 2.13 to ≈ 3
- Next cutoff = min pruned f, never bound + 1 (f steps by 2 here — parity of g + h is invariant — but don’t hardcode)
- Do not count the blank in Manhattan
- Return the move path, not just its length
- Run the parity check before any search
Tests
- Goal state → empty path
- One-move state → that single move
- Sam Loyd 14-15 swap → unsolvable, returns immediately
- Random 20-move walks from goal: solve, assert len ≤ 20, replay moves, assert goal reached
- Optimal lengths match plain BFS for scrambles ≤ 12 moves
- Log per-iteration node counts on a ~40-move instance: Manhattan vs linear conflict — conflict must expand strictly fewer
Follow-up Questions
- Why IDA over A*?* → Memory: A* holds O(b^d) nodes — tens of GB by depth 50; IDA* holds one path, O(d). Re-expansion costs only a constant factor.
- How is a pattern database built? → Backward BFS from the goal over abstracted states in which only the pattern tiles (and blank) are distinguishable; all other tiles are wildcards. Store the exact minimum cost for every placement of the pattern tiles. Lookup = a perfect heuristic for the abstraction.
- Why can disjoint PDBs be summed without breaking admissibility? → Each move moves exactly one tile. Partition tiles into disjoint groups and, when building each PDB, count only moves of that group’s tiles. Any real solution’s moves split across the groups, so the sum of per-group lower bounds ≤ true length. Counting blank moves too would double-count — then only max(), not sum, is safe.
- Detect unsolvable inputs without searching? → The parity invariant above: O(n²) inversion count at n = 16 is instant; compare with the goal’s parity.
- Doesn’t iterative deepening waste work re-expanding shallow levels? → Per-iteration node counts grow geometrically with ratio r (≈ b² ≈ 4.5, since cutoffs step by 2). Total = final × (1 + 1/r + 1/r² + …) = final × r/(r−1) ≈ 1.3× the last iteration alone.
- Rubik’s cube? → Korf 1997: IDA* + corner/edge PDBs solved random cubes optimally. Same recipe, bigger tables — see lab-02.
Product Extension
- Warehouse-robot and Sokoban-style planners (PDB abstractions of the map)
- Memory-constrained embedded planners (IDA* on devices without room for open lists)
- Puzzle-game hint engines: optimal next move
- Model checking: directed search over huge implicit state spaces
Language/Runtime Follow-ups
- Python: tuples + dicts beat OO nodes; recursion is fine at depth ≤ 80 but call overhead dominates — an explicit stack plus incremental h gives ~5×. PyPy: ~30×.
- C++: the reference language; pack the board into a 64-bit int (4 bits/tile), PDB as a flat
uint8_tarray. - Java: same long-packing works; avoid autoboxed keys in hot maps.
- JavaScript: rare; typed arrays and integer boards if you must.
Common Bugs
- No parent pruning: undo-moves create 2-cycles; branching jumps to ≈ 3 and deep iterations cost ~10× more.
- Blank counted in Manhattan: a state one move from goal gets h = 2 — inadmissible, returned paths can be non-optimal.
- Wrong parity rule: applying the odd-width rule (inversions even) to 4×4; even widths must include the blank’s row.
- Cutoff update wrong: bound + 1 doubles iteration count (f steps by 2); anything other than min-pruned-f can skip the optimal depth or loop.
- Path/moves desync: appending a neighbor but not popping on backtrack — returned move list is garbage.
- Goal test before cutoff test: accepting a goal with f > bound returns a path longer than branches pruned at smaller f — breaks optimality.
Debugging Strategy
- Property test: scramble k moves from goal (k = 1..20), assert solution length ≤ k and replay reaches goal
- Diff optimal lengths against BFS on shallow instances — mismatch means an inadmissible h or cutoff bug
- Log (bound, nodes) per iteration: bounds must step by 2, nodes grow geometrically
- Unit-test h alone: h(goal) = 0; |Δh| = 1 across every single move
- Verify parity on the goal (odd) and on each one-move neighbor (still odd)
Mastery Criteria
- Implement IDA* + Manhattan + parity check in ≤ 60 min, correct on random scrambles
- Add linear conflict in ≤ 30 min and demonstrate the node-count drop
- State the parity invariant and prove one move preserves it
- Explain PDB construction and why disjoint PDBs sum admissibly, unprompted
- Do the memory arithmetic: why A* dies near depth 25 and IDA* doesn’t
Lab 02 — Kociemba’s Two-Phase Algorithm (Rubik’s Cube)
Goal
Implement a Rubik’s Cube solver based on Kociemba’s two-phase algorithm: encode cube states as coordinates, precompute move and pruning tables, and run two chained IDA* searches to produce solutions of ≤ 30 moves (typically 19–23) in well under a second.
Background
The cube group G has |G| = 8! × 3^7 × 12! × 2^11 / 2 ≈ 4.33 × 10^19 states. Direct search is hopeless; even storing one bit per state is ~5 exabytes.
Thistlethwaite (1981): descend through a chain of 4 nested subgroups, solving into each in turn; ≤ 52 moves total.
Kociemba (1992): collapse the chain to just two phases via the subgroup
G1 = <U, D, R2, L2, F2, B2>
- Phase 1 takes any state into G1. A state is in G1 exactly when: all corner orientations are 0, all edge orientations are 0, and the four middle-slice edges (FR, FL, BL, BR) are in the middle slice. The quotient space G/G1 has 2187 × 2048 × 495 ≈ 2.2 × 10^9 cosets.
- Phase 2 solves within G1 using only the 10 G1 moves (U, U2, U’, D, D2, D’, R2, L2, F2, B2). |G1| = 8! × 8! × 4! / 2 ≈ 1.95 × 10^10.
Each phase is an IDA* search over small integer coordinates (not facelets), with precomputed move tables (coordinate × move → coordinate) and pruning tables (exact distance-to-goal in a simplified quotient — an admissible heuristic).
Max phase-1 depth is 12, max phase-2 depth is 18 → any cube solvable in ≤ 30 moves. Continuing to search shorter phase-1 solutions drives the total toward optimal. God’s number is 20 (Rokicki, Kociemba, Davidson, Dethridge 2010).
Interview Context
- Direct interview question: essentially never, anywhere
- Robotics / puzzle-solver take-homes and demo projects: the reference algorithm
- Search/planning research roles: expected to know the ideas (coset decomposition, PDB heuristics)
- Quant/FAANG: only as an open-ended “how would you even approach this?” conversation — the vocabulary (state encoding, admissible pruning tables, two-phase decomposition) is the answer
- CTF / hobby: the standard
kociembalibrary everyone uses
When to Skip This Topic
Skip if any of these are true:
- You haven’t done Lab 01 (IDA* & pattern databases) — this lab is that lab, applied twice with group theory on top
- You have any interview in the next 3 months — zero chance of direct payoff
- You want a quick win — a correct implementation is a multi-day project even with this guide
Do this lab for depth, not coverage. It is the single best exercise in the curriculum for state encoding and search pruning; it is also the least likely to be asked.
Problem Statement
Solve the Cube.
Given a scrambled 3×3×3 Rubik’s Cube state (as a scramble move sequence, or a 54-character facelet string), return a move sequence of length ≤ 30 that solves it, in under 1 second (after one-time table precomputation).
Moves use face-turn metric: U, U2, U', D, D2, D', R, ..., B' — 18 moves total.
Constraints
- Any reachable cube state (4.33 × 10^19 possibilities)
- Solution length ≤ 30 moves (near-optimal); optimal (≤ 20) not required
- Time: < 1 sec per solve; table precomputation may take seconds and be cached to disk
- Memory: ≤ a few hundred MB for tables (basic variant needs < 10 MB)
Clarifying Questions
- Input format — facelet string or scramble sequence? (Either; scramble sequence lets you skip facelet parsing.)
- Is the state guaranteed reachable? (Ask. If not, validate: orientation sums and permutation parity — see Common Bugs.)
- Optimal or near-optimal? (Near-optimal ≤ 30. Optimal changes the algorithm to Korf’s IDA* with pattern databases.)
- Half-turn metric or quarter-turn metric? (Half-turn: U2 counts as one move.)
- May I precompute and cache tables? (Yes — that’s the entire design.)
Examples
Scramble: (identity) → Solution: [] (0 moves)
Scramble: R → Solution: [R'] (1 move)
Scramble: R U R' U' → Solution: [U R U' R'] (4 moves)
Superflip (all edges flipped) → 20 moves — proven hardest-class state
Phase decomposition example: scramble F R is not in G1 (F flips edge orientations). Phase 1 might return F' (now in G1, corners/edges oriented, slice edges home), phase 2 returns R'. Total: F' R'.
Brute Force
BFS/DFS over raw states. Branching factor ~13.3 (after pruning same-face and commuting move pairs), depth up to 20 → 13.3^20 ≈ 10^22 nodes. Dead.
IDA* over raw facelets with a weak heuristic (misplaced stickers / 8) is also dead beyond depth ~14: the heuristic is far too loose.
Brute Force Complexity
- Time: O(b^20), b ≈ 13.3 — computationally infeasible
- Space: BFS frontier ~10^19 states — physically infeasible
Optimization Path
- Cubie representation, not facelets: corner permutation (8-array), corner orientation (8-array of 0..2), edge permutation (12-array), edge orientation (12-array of 0/1). A move is a fixed permutation + orientation delta applied to these arrays.
- Coordinates: pack each independent property into one integer —
- corner orientation: Σ ori[i] · 3^i over first 7 corners → 0..2186 (8th is determined: sum ≡ 0 mod 3)
- edge orientation: first 11 edges base-2 → 0..2047 (12th determined: sum ≡ 0 mod 2)
- UD-slice: which 4 positions hold the slice edges → C(12,4) = 495 combinations
- phase 2: corner permutation rank (0..40319), U/D-edge permutation rank (0..40319), slice permutation (0..23)
- Move tables: for each coordinate, a table
move[coord][m] → coord'built once by applying all 18 moves to every value. Sizes: 2187×18, 2048×18, 495×18, 40320×18, 24×18 — all tiny. - Pruning tables: BFS backward from the goal over pairs of coordinates (e.g. corner-ori × slice) storing exact distance. Exact distance in a projection = admissible heuristic in the full space.
- Phase 1 IDA:* search to any state with all three phase-1 coordinates zero, pruning when
max(h_tables) > depth remaining. - Phase 2 IDA:* from the G1 state, search to solved using only the 10 G1 moves, with phase-2 pruning tables.
- Optimality loop: don’t stop at the first phase-1 solution — iterate longer phase-1 solutions; a phase-1 solution one move longer often yields a much shorter phase 2. Stop when total stops improving (or timeout).
Final Expected Approach
Core skeleton (cubie level, one coordinate shown per kind, phase-1 search; phase 2 is the same machinery with its own coordinates and the restricted move set):
# Corner/edge cubie model. Moves defined as (perm, ori_delta) on cubie arrays.
cpU, coU = [3,0,1,2,4,5,6,7], [0]*8
epU, eoU = [3,0,1,2,4,5,6,7,8,9,10,11], [0]*12
# ... R, F, D, L, B analogously (R and F twist corners / F,B flip edges)
class Cube:
def __init__(self):
self.cp, self.co = list(range(8)), [0]*8
self.ep, self.eo = list(range(12)), [0]*12
def apply(self, m):
cp, co, ep, eo = MOVES[m]
self.cp = [self.cp[cp[i]] for i in range(8)]
self.co = [(self.co[cp[i]] + co[i]) % 3 for i in range(8)]
self.ep = [self.ep[ep[i]] for i in range(12)]
self.eo = [(self.eo[ep[i]] + eo[i]) % 2 for i in range(12)]
def corner_ori_coord(self): # 0..2186
return sum(self.co[i] * 3**i for i in range(7))
def edge_ori_coord(self): # 0..2047
return sum(self.eo[i] * 2**i for i in range(11))
def slice_coord(self): # 0..494, rank of slice-edge positions
k, coord = 0, 0
for pos in range(11, -1, -1):
if self.ep[pos] >= 8: # FR,FL,BL,BR are edges 8..11
coord += comb(pos, 4 - k); k += 1
return coord
def build_move_table(size, coord_of, set_coord):
table = [[0]*18 for _ in range(size)]
for c in range(size):
for m in range(18):
cube = set_coord(c); cube.apply(MOVE_NAMES[m])
table[c][m] = coord_of(cube)
return table
def build_pruning(size1, size2, mt1, mt2):
dist = [[-1]*size2 for _ in range(size1)]
dist[0][0], frontier = 0, [(0, 0)]
while frontier:
nxt = []
for a, b in frontier:
for m in range(18):
na, nb = mt1[a][m], mt2[b][m]
if dist[na][nb] < 0:
dist[na][nb] = dist[a][b] + 1; nxt.append((na, nb))
frontier = nxt
return dist
def phase1(co, eo, sl, depth, last, path):
h = max(PRUNE_CO_SL[co][sl], PRUNE_EO_SL[eo][sl])
if h > depth: return None
if co == eo == sl == 0: return path
if depth == 0: return None
for m in range(18):
if last >= 0 and m // 3 == last // 3: continue # same face
r = phase1(MT_CO[co][m], MT_EO[eo][m], MT_SL[sl][m],
depth - 1, m, path + [m])
if r: return r
return None
def solve(cube, max_len=30):
best = None
for d1 in range(13):
p1 = phase1(cube.corner_ori_coord(), cube.edge_ori_coord(),
cube.slice_coord(), d1, -1, [])
if p1 is None: continue
p2 = phase2_from(cube, p1, budget=max_len - len(p1))
if p2 is not None:
total = p1 + p2
if best is None or len(total) < len(best): best = total
return best
Elided for space, required for a working solver: the other 5 move definitions, phase-2 coordinates (corner-permutation rank, UD-edge rank, slice permutation) with their move/pruning tables over the 10 G1 moves, facelet-string parsing, and coordinate → cube reconstruction (set_coord). Full implementations run 400–600 lines.
Data Structures
- Four fixed-size int arrays per cube state (cp, co, ep, eo)
- Flat
uint16move tables: coordinate × 18 → coordinate - Byte pruning tables over coordinate pairs (exact BFS distances; fits in a few MB)
- Recursion stack only — IDA* needs no open/closed sets
Correctness Argument
- G1 is a subgroup, and the three phase-1 coordinates are invariants of its cosets: a state lies in G1 ⟺ all three are zero. So phase 1 solves the coset problem exactly.
- Move tables are exact by construction (built by applying real moves to real states).
- Pruning tables store exact distances in a quotient of the state space. Distances in a quotient never exceed distances in the original ⇒ admissible heuristic ⇒ IDA* returns a shortest phase-1 (resp. phase-2) solution for each depth bound.
- Termination bound: every coset is reachable in ≤ 12 phase-1 moves and every G1 state solvable in ≤ 18 phase-2 moves (known table maxima) ⇒ solver always returns ≤ 30 moves.
- Concatenation of per-phase-optimal solutions is not globally optimal — hence the iteration over longer phase-1 solutions to shrink the total.
Complexity
- Precomputation: O(Σ coord_size × 18) move tables + BFS over pruning-table spaces (~10^6–10^8 entries depending on variant) — seconds, cacheable
- Solve: exponential in the worst case, but pruning tables cut effective branching so hard that typical solves visit 10^4–10^6 nodes — milliseconds to ~1 sec
- Space: < 10 MB basic; Kociemba’s tuned tables ~80 MB
Implementation Requirements
- Cubie-level move application must be exact (test: every move applied 4× = identity)
- Orientation coordinates drop the last cubie (its value is forced by the mod-3 / mod-2 invariant)
- Phase-2 move set excludes quarter turns of R, L, F, B — enforce in the move loop, not the tables
- Prune same-face consecutive moves; also fix an order for commuting opposite-face pairs (U D = D U — search only one)
- Validate input state: corner ori sum ≡ 0 (mod 3), edge ori sum ≡ 0 (mod 2), corner-permutation parity = edge-permutation parity
- Cache tables to disk; rebuild only on cache miss
Tests
- Identity cube → empty solution
- Each single move m → solution is its inverse
- 100 random scrambles (25 random moves each): apply returned solution, assert solved, assert length ≤ 30
- Superflip: solver returns ≤ 24 in two-phase mode quickly; ≤ 20 with the optimality loop given time
- Unsolvable states (single twisted corner, single flipped edge, two swapped corners) → rejected by validation
- Move-table sanity:
move[coord][m]thenmove[·][inverse(m)]returns coord - Pruning-table sanity: entry 0 exactly at the goal projection; max entry = 12 (phase 1)
Follow-up Questions
- What is God’s number and how was it proven? → 20 in half-turn metric. Rokicki et al. 2010: partitioned the group into ~2.2 × 10^9 cosets of G1, solved each coset set to ≤ 20 with a coset solver, ~35 CPU-years donated by Google. Quarter-turn metric answer is 26.
- How would you find provably optimal solutions for a single cube? → Korf’s algorithm (1997): plain IDA* in the full space with pattern-database heuristics (corner PDB: 88 M entries, plus two edge PDBs, take the max). Hours per hard cube vs milliseconds for two-phase.
- Why two phases instead of Thistlethwaite’s four? → Fewer phases = fewer concatenation seams = shorter totals (30 vs 52), at the cost of bigger tables per phase. Two is the sweet spot where tables still fit in memory.
- Why IDA and not A*?* → A* stores the open set; at 10^6+ expanded nodes with 13-way branching, memory explodes. IDA* is O(depth) memory and re-expansion overhead is bounded because node count grows geometrically per iteration.
- How do the 48 cube symmetries help? → States equivalent under rotation/reflection share distances, shrinking pruning tables ~16–48× (Kociemba’s implementation stores sym-coordinates). Same trick as symmetry reduction in model checking.
- Does this generalize to 4×4×4? → Partially. No fixed centers, orientation parity rules differ, and the group is ~7.4 × 10^45 — solvers use reduction (pair up pieces to emulate a 3×3×3) rather than a direct two-phase.
Product Extension
- Robot solvers (the sub-second solve step behind every cube-robot demo)
kociemba/min2phaselibraries powering cube apps and timers- The general pattern — compress state into coordinates, precompute exact distances in projections, IDA* with max-of-heuristics — is the production recipe for planning problems (warehouse robots, protein design search, chip-placement subproblems)
- Coset-solving methodology reused in mathematical computation at scale (the God’s number computation itself)
Language/Runtime Follow-ups
- C/C++: the reference choice; tables as flat arrays, solve in microseconds (min2phase).
- Python: cubie math is fine; use flat lists + disk-cached tables. Pure-Python solves in ~0.1–1 sec.
pip install kociembawraps C. - Java: Herbert Kociemba’s original Cube Explorer lineage; min2phase Java port is the canonical fast one.
- JavaScript: cubing.js implements two-phase for browser cube timers — tables via WASM.
Common Bugs
- Orientation coordinate includes all 8/12 cubies: the last orientation is dependent (sums ≡ 0); including it creates unreachable coordinates and breaks table indexing.
- Move applied to orientation without permuting first: orientation delta must be added to the orientation of the cubie arriving at the slot:
co'[i] = co[cp_m[i]] + delta[i]. - Phase 2 accidentally uses all 18 moves: produces states outside G1 mid-search; pruning tables (built over 10 moves) then underestimate garbage states and the search corrupts.
- Parity mismatch accepted: corner and edge permutation parities must be equal; skipping validation makes the solver loop forever on unsolvable input.
- Stopping at the first phase-1 solution: correct but yields 25–30 move solutions; the iteration over longer phase-1 attempts is what gets you to ~20.
- Slice-coordinate ranking inconsistent between the coordinate function and the reconstruction used for table building — tables silently index wrong states.
Debugging Strategy
- Verify each move: apply 4× → identity; verify
R R'→ identity - Verify coordinate round-trips:
coord_of(set_coord(c)) == cfor all c in range - Verify pruning tables: value 0 only at goal projection; histogram of distances matches published tables (phase-1 max 12, phase-2 max 18)
- Solve states of known distance (1-move, 2-move scrambles) and assert exact lengths
- Diff intermediate coordinates against the
kociembareference library on random scrambles
Mastery Criteria
- Explain the coset decomposition (why G1, what phase 1 actually solves) in ≤ 5 min without notes
- Implement the cubie model + all three phase-1 coordinates in ≤ 60 min
- Build move + pruning tables and a working phase-1 IDA* in ≤ half a day
- State from memory: |G| ≈ 4.3 × 10^19, phase-1 space 2.2 × 10^9, max depths 12/18, God’s number 20
- Articulate why pruning tables are admissible (quotient-space argument) and why two-phase is near- but not exactly optimal
- Name the optimal alternative (Korf + pattern databases) and its cost trade-off
Lab 03 — Bidirectional BFS on Implicit Graphs
Goal
Implement bidirectional BFS to find shortest paths in implicit state graphs, cutting the explored node count from O(b^d) to O(b^(d/2)) — the difference between timing out and finishing in milliseconds on Word Ladder-class problems.
Background
One-directional BFS from the start explores b^d nodes for branching factor b and distance d. Searching simultaneously from start and goal, stopping when the frontiers meet, explores ~2·b^(d/2). For b = 25, d = 10: 25^10 ≈ 10^14 vs 2 × 25^5 ≈ 2 × 10^7 — seven orders of magnitude.
Requirements to apply it:
- The goal state is known explicitly (not just a predicate)
- Moves are invertible (or the reverse graph is available)
Relation to phase-03 lab-09 (meet-in-the-middle): same halving idea, different object — that one splits a set to enumerate combinations in 2^(n/2); this one splits a path to search in b^(d/2). Kociemba’s two-phase solver (lab-02) is the industrial-strength descendant: instead of a backward frontier held in memory, the “backward half” is precomputed into pruning tables.
Interview Context
- Standard FAANG: yes — Word Ladder (LC 127), Minimum Genetic Mutation (LC 433), Open the Lock (LC 752) are real, recurring questions, and bidirectional BFS is the expected senior-level answer
- Codeforces / ICPC: occasional, usually as meet-in-the-middle on states
- Search/planning roles: assumed knowledge
- This is the most interview-relevant lab in Phase 13 — if you cherry-pick one, pick this
When to Skip This Topic
Skip if any of these are true:
- You can’t write clean one-directional BFS with level tracking from memory (Phase 4 Lab 01 first)
- You’re still failing Medium graph problems — this is an optimization on top of a pattern you must already own
Unlike the rest of this phase, most candidates targeting Hard-level interviews should not skip this.
Problem Statement
Word Ladder (LeetCode 127).
Given beginWord, endWord, and a dictionary wordList, return the number of words in the shortest transformation sequence from beginWord to endWord, changing one letter at a time, with every intermediate word in the dictionary. Return 0 if impossible.
Constraints
- 1 ≤ word length ≤ 10
- 1 ≤ len(wordList) ≤ 5000
- All words same length, lowercase;
endWordmust be in the list - Generalization to plan for: state graphs with 10^6+ implicit nodes
Clarifying Questions
- Is
beginWordrequired to be in the dictionary? (No.) - Return the length or the actual path? (Length here; path needs parent pointers — see Follow-ups.)
- Count of words or count of edges? (Words: hit→cog ladder of 4 changes = 5.)
- Are moves symmetric? (Yes — changing one letter is invertible; this is what makes bidirectional legal.)
Examples
begin = "hit", end = "cog"
wordList = ["hot","dot","dog","lot","log","cog"]
→ 5 (hit → hot → dot → dog → cog)
Frontier sizes, one-directional from “hit”: 1 → 1 → 2 → 4 → 2. Bidirectional: forward {hit}→{hot}, backward {cog}→{dog,log}; forward {hot}→{dot,lot} … meets after ~half the levels with strictly smaller frontiers. On adversarial dictionaries (dense neighborhoods), the gap is exponential.
begin = "hit", end = "cog", wordList without "cog" → 0
Brute Force
One-directional BFS, generating neighbors by trying all 26 letters in every position, checking dictionary membership.
Brute Force Complexity
- Time: O(N · L · 26) node expansions in the worst case over the whole graph — passes LC 127, but explores the full b^d cone; dies on larger state spaces
- Space: O(N)
Optimization Path
- Wildcard buckets: precompute
h*t → [hot, hat, ...]— maps each word to neighbors in O(L) lookups instead of O(26·L) probes. - Two frontiers: sets
front(from begin) andback(from end), plus a combined visited set. - Always expand the smaller frontier — swap before each level. This is what guarantees b^(d/2): the expanded side never grows past the meeting point.
- Meet condition: while generating a neighbor, if it’s in the other frontier, the ladders join — return
steps_forward + steps_backward. - Frontiers as sets, not queues — level-by-level expansion replaces the FIFO.
Final Expected Approach
from collections import defaultdict
from string import ascii_lowercase
def ladder_length(begin, end, word_list):
words = set(word_list)
if end not in words:
return 0
buckets = defaultdict(list)
for w in words | {begin}:
for i in range(len(w)):
buckets[w[:i] + "*" + w[i+1:]].append(w)
front, back = {begin}, {end}
visited = {begin, end}
length = 1
while front and back:
if len(front) > len(back):
front, back = back, front
length += 1
next_front = set()
for word in front:
for i in range(len(word)):
for nb in buckets[word[:i] + "*" + word[i+1:]]:
if nb in back:
return length
if nb not in visited:
visited.add(nb)
next_front.add(nb)
front = next_front
return 0
Generic template for any implicit graph: replace the bucket lookup with a neighbors(state) function; keep separate dist_f / dist_b dicts instead of one visited set when you need the path or the graph is directed (use reverse moves for the backward side).
Data Structures
- Two frontier sets + one visited set (or two distance dicts)
- Wildcard-pattern buckets: dict pattern → list of words (precomputed adjacency)
Correctness Argument
- BFS invariant per side: after k expansions, a frontier contains exactly the states at distance k from its origin.
- If a shortest path of length d exists, its middle state is at distance ⌈d/2⌉ from one side and ⌊d/2⌋ from the other; alternating expansion of the smaller frontier reaches the meet before either side exceeds ⌈d/2⌉ levels.
- The meet check happens on generated neighbors against the opposite frontier, so the returned
lengthcounts both halves exactly once — no path can be shorter, since both sides expanded strictly level-by-level. - One shared
visitedset is safe only because the meet check precedes it; a state claimed by one side that the other side needs would otherwise be silently swallowed (see Common Bugs).
Complexity
- Time: O(b^(d/2)) expansions; for Word Ladder concretely O(N · L²) with buckets
- Space: O(b^(d/2)) — this is bidirectional’s own cost: the backward frontier must fit in memory (IDA* + pruning tables, lab-01/02, is the escape when it doesn’t)
Implementation Requirements
- Swap to expand the smaller frontier every level — without this, worst case reverts toward b^d
- Meet check against the opposite frontier before the visited check
- Level counter incremented once per level, not per node
end not in wordsshort-circuit before building anything- For directed graphs: backward expansion must use reversed edges — verify invertibility before choosing this technique
Tests
- The classic hit→cog example → 5
- endWord missing from dictionary → 0
- begin == end handling per problem statement
- Length-1 words (b=25 exactly)
- No path exists but both components are large (disconnected) → 0, terminates when a frontier empties
- Adversarial: 5000 words forming a dense cluster — compare node expansions vs one-directional (assert ~order-of-magnitude reduction)
- Path of length 2 (begin and end adjacent) — meet on first expansion
Follow-up Questions
- Why always expand the smaller frontier? → Total work is the sum of expanded frontier sizes; expanding the smaller side keeps both trees balanced at ~b^(d/2). Expanding a large side one extra level can cost more than the entire balanced search.
- How do you return the actual path, not just the length? → Keep per-side parent dicts; on meeting at state m, splice parents(begin→m) + reversed parents(m→end). One shared visited set no longer suffices — you need to know which side owns each state.
- When is bidirectional BFS not applicable? → Goal known only as a predicate (“any state where X”), non-invertible moves without a reverse-edge oracle, or many goal states (seed the backward frontier with all of them — fine if enumerable).
- Does this work for weighted graphs? → Bidirectional Dijkstra exists but the stopping condition is subtle: you cannot stop at first meet; you stop when
top_f + top_b ≥ best_meet_costfound so far. Getting this wrong is a classic bug. - Word Ladder II (all shortest paths, LC 126)? → Bidirectional BFS to build a DAG of parents level-by-level, then DFS the DAG to emit paths. Deduplicate per level, not globally.
- How does Kociemba’s solver relate? → Same halving principle, but the backward half is precomputed offline into pruning tables (exact distances in a quotient space), so the online search runs forward-only with a perfect-information heuristic. That trade — memory for a precomputed backward search — is the pattern behind all big puzzle solvers.
Product Extension
- Social-graph degrees of separation (LinkedIn-style “3rd-degree connection” queries run bidirectionally)
- Route planning seeds: bidirectional Dijkstra/ALT underlies road-network engines before contraction hierarchies
- Diff/merge tooling: Myers diff explores an edit graph; bidirectional variants power
git diffoptimizations - Deadlock/reachability checking in model checkers (forward from init, backward from bad states)
Language/Runtime Follow-ups
- Python: set operations are the hot path;
frozensetswaps are cheap, string slicing for buckets is fine at L ≤ 10. - C++:
unordered_set<string>hashing dominates — intern words to ints first for large dictionaries. - Java:
HashSet<String>; watch autoboxing if interning to Integer. - JavaScript:
Set+ template strings; fine for LC constraints.
Common Bugs
- Meet check after visited check: neighbor already visited by the other side gets skipped and the meet is missed → returns longer path or 0.
- Forgetting to swap frontiers: still correct, but the complexity guarantee is gone; on skewed graphs it degrades to one-directional.
- Level counting off by one: returning edges instead of words (LC wants words) or incrementing per node instead of per level.
- Expanding into the visited set of the same side only: with two separate visited sets, a state may legitimately appear in both — that’s the meet, not a duplicate to skip.
- Backward expansion with forward edges on a directed graph: silently wrong distances; must use reverse adjacency.
- Mutating a frontier while iterating it: always build
next_frontfresh.
Debugging Strategy
- Print (level, len(front), len(back)) per iteration — frontier sizes should stay balanced and small
- Run one-directional BFS as oracle on small dictionaries; assert equal lengths over random instances
- Test the directed-graph variant on a graph where forward and backward answers differ if edges aren’t reversed
- Count node expansions bidirectional vs one-directional to confirm the ~√ speedup empirically
Mastery Criteria
- Implement bucket-based bidirectional BFS for Word Ladder in ≤ 20 min, bug-free
- State the two applicability conditions (explicit goal, invertible moves) unprompted
- Explain the b^(d/2) math with a concrete number in ≤ 2 min
- Know the bidirectional Dijkstra stopping-condition trap
- Extend to path reconstruction in ≤ 10 additional min
Lab 04 — Algorithm X & Dancing Links (Exact Cover)
Goal
Implement Knuth’s Algorithm X for the exact cover problem and use it to solve Sudoku by reduction to a 729-row × 324-column matrix — the standard architecture of every serious constraint-puzzle solver.
Background
Exact cover: given a 0/1 matrix, choose a subset of rows so that every column contains exactly one 1 among the chosen rows.
Many puzzles reduce to it: Sudoku, pentomino tiling, n-queens. NP-complete in general, but instances arising from puzzles solve near-instantly with the right search.
Algorithm X (Knuth, 2000): depth-first search — pick an unsatisfied column, try each row that covers it, remove (“cover”) all conflicting rows/columns, recurse, restore on backtrack.
Dancing Links (DLX): the implementation trick. Represent the matrix as circular doubly-linked lists (one node per 1, column headers, a root). Unlinking a node is two pointer writes; relinking is the same two writes reversed — x.left.right = x; x.right.left = x — so backtracking restores state in O(1) per node with no copying. Knuth’s paper is titled after the pointers “dancing” back.
Column choice matters: always branch on the column with the fewest 1s (minimum remaining values). This collapses the branching factor and is the difference between microseconds and hours.
Interview Context
- Standard FAANG: Sudoku Solver (LC 37) and N-Queens (LC 51/52) are asked; plain backtracking is the expected answer — DLX is the “strong senior” flex, not a requirement
- OR / scheduling / constraint-solver teams: core background
- Game/puzzle companies: directly relevant
- ICPC: occasional exact-cover disguises
- Knuth literacy signal: TAOCP 4B fascicle 5 is entirely about this
When to Skip This Topic
Skip if any of these are true:
- You can’t write plain recursive backtracking with undo cleanly (Phase 2 Lab 08 first)
- Your target roles have no solver/OR/puzzle component — LC 37 is passable with ordinary backtracking + bitmasks
Learn Algorithm X in its dict-of-sets form regardless if you have 2 hours: it’s 30 lines and permanently upgrades how you think about backtracking-with-undo.
Problem Statement
Sudoku Solver (LC 37), the solver-grade version.
Solve a 9×9 Sudoku. Encode it as exact cover:
- Rows of the matrix = candidate placements (r, c, d) — 9×9×9 = 729
- Columns = constraints, 4 families × 81 = 324:
- cell (r, c) is filled
- row r contains digit d
- column c contains digit d
- box b contains digit d
A solution = 81 rows covering all 324 columns exactly once.
Constraints
- Standard 9×9 with ≥ 17 givens (uniqueness threshold); solver must also detect unsolvable/multi-solution grids
- Time: hardest 9×9 instances in ≤ 100 ms
- Generalize: same code must take 16×16 Sudoku or pentomino tilings by changing only the matrix construction
Clarifying Questions
- Is the input guaranteed solvable and unique? (LC: yes. Solver-grade: detect both failure and second solutions.)
- Just one solution or enumerate all? (Algorithm X naturally enumerates; ask.)
- Board size fixed at 9×9? (Design the reduction generically; n² × n² boards change constants only.)
- Output format — mutate the grid or return placements? (Either; placements fall out of the chosen rows.)
Examples
Tiny exact cover, columns {1..7}, rows:
A = {1,4,7} B = {1,4} C = {4,5,7}
D = {3,5,6} E = {2,3,6,7} F = {2,7}
Branch on column 1 (2 rows): try A → covers {1,4,7}, kills C, B, E, F(7)… dead end on column 2; backtrack, try B → covers {1,4}, then D covers {3,5,6}, F covers {2,7}. Solution: {B, D, F}.
Sudoku column indexing for placement (r, c, d), all 0-based:
cell: 81·0 + 9r + c
row: 81·1 + 9r + d
col: 81·2 + 9c + d
box: 81·3 + 9(3⌊r/3⌋ + ⌊c/3⌋) + d
Brute Force
Cell-by-cell backtracking, re-scanning row/column/box validity at each step. Passes LC 37, but on hard instances (e.g. grids designed against naive orderings) explores orders of magnitude more nodes than MRV-guided exact cover.
Brute Force Complexity
- Time: O(9^(empty cells)) worst case, heavily instance-dependent
- Space: O(81) recursion
Optimization Path
- Reify all four constraint families as columns — validity checking disappears; it is the matrix.
- Algorithm X with MRV column selection — always branch where fewest candidates remain.
- Dict-of-sets representation (X: column → set of rows; Y: row → column list) — same asymptotics as pointer DLX, 3× shorter to write.
- Pointer-based DLX with real dancing links when performance matters (pentominoes, 16×16, enumeration).
- Givens are handled by selecting their rows first — same code path as search choices.
Final Expected Approach
Dict-of-sets Algorithm X (interview-writable) + Sudoku reduction:
def solve_exact_cover(X, Y, solution):
if not X:
yield list(solution)
return
col = min(X, key=lambda c: len(X[c]))
for row in list(X[col]):
solution.append(row)
removed = select(X, Y, row)
yield from solve_exact_cover(X, Y, solution)
deselect(X, Y, row, removed)
solution.pop()
def select(X, Y, row):
removed = []
for col in Y[row]:
for other_row in X[col]:
for other_col in Y[other_row]:
if other_col != col:
X[other_col].discard(other_row)
removed.append(X.pop(col))
return removed
def deselect(X, Y, row, removed):
for col in reversed(Y[row]):
X[col] = removed.pop()
for other_row in X[col]:
for other_col in Y[other_row]:
if other_col != col:
X[other_col].add(other_row)
def sudoku_matrix():
Y = {}
for r in range(9):
for c in range(9):
for d in range(9):
b = 3 * (r // 3) + c // 3
Y[(r, c, d)] = [
("cell", r, c), ("row", r, d),
("col", c, d), ("box", b, d)]
X = {}
for row, cols in Y.items():
for col in cols:
X.setdefault(col, set()).add(row)
return X, Y
def solve_sudoku(grid):
X, Y = sudoku_matrix()
partial = []
for r in range(9):
for c in range(9):
if grid[r][c]:
select(X, Y, (r, c, grid[r][c] - 1))
for sol in solve_exact_cover(X, Y, partial):
for (r, c, d) in sol:
grid[r][c] = d + 1
return grid
return None
The pointer DLX version replaces sets with column-header nodes and up/down/left/right links; cover(col) unlinks the column header and every row intersecting it; uncover relinks in exact reverse order.
Data Structures
- Dict version:
Xcolumn → set of active rows;Yrow → static column list; undo stack of removed sets - DLX version: node {L, R, U, D, column}; column headers with live size counters; circular root list of uncovered columns
Correctness Argument
- Invariant: at every node of the search tree, (X, Y) represents exactly the sub-matrix of rows/columns compatible with the current partial solution.
- Branching on one column and trying each of its rows is exhaustive: any exact cover must cover that column by exactly one row, so the row choices partition the solution space — no solution is missed, none is double-counted.
deselectundoesselectby replaying it in exact reverse order, restoring the invariant — the same argument makes DLX’s O(1) relink sound (a node’s left/right neighbors are untouched while it is unlinked).- Sudoku ⇔ exact cover: a set of 81 placements covers all 324 columns exactly once iff every cell is filled once and every row/column/box contains each digit once — the definition of a solved grid.
Complexity
- Worst case: exponential (exact cover is NP-complete)
- In practice: MRV drives hard 9×9 Sudokus to hundreds–thousands of nodes, < 10 ms in Python; DLX-in-C solves millions of grids/minute
- Space: O(number of 1s) = O(729 × 4) for Sudoku — trivial
Implementation Requirements
- MRV column choice (
minby live size) — non-negotiable; naive first-column order is exponentially worse deselectmust iterate columns in reverse ofselect- Givens applied through
select, and theirremovedlists retained if the board must be restorable - Detect contradiction early: a live column with zero rows fails the
min-branch immediately (empty iteration) — that’s the pruning, don’t special-case around it - For enumeration: keep yielding; for uniqueness checking: stop after finding 2
Tests
- The 7-column hand example above → exactly {B, D, F}
- LC 37 sample grid → known solution
- Empty grid → yields a valid solved grid (any)
- 17-given minimal puzzle (hardest class) → solved ≤ 100 ms
- Unsolvable grid (two 5s in a row among givens) → select raises / search yields nothing; handle both
- Multi-solution grid → enumeration yields ≥ 2
- Property test: for random solved grids with k cells removed, solver output is a valid completion
Follow-up Questions
- Why does choosing the smallest column matter so much? → Search-tree size is the product of branching factors along paths; branching on a 1-row column is a forced move (factor 1). MRV takes forced moves first, so contradictions surface at depth 1 instead of depth 40.
- How do dancing links achieve O(1) undo? → An unlinked node keeps its own L/R/U/D pointers; neighbors never point at it while it’s out, so
x.L.R = x; x.R.L = xrestores it. Valid only if relinks happen in exact LIFO order — which backtracking guarantees. - N-queens as exact cover? → Ranks and files are ordinary columns; diagonals become secondary columns — constraints that may be covered at most once rather than exactly once. DLX handles them by giving secondary columns no presence in the root header list.
- Pentomino / polyomino tiling? → Columns = 60 board cells + 12 piece names; rows = every placement of every piece (~1500). Knuth’s original showcase; enumerates all 65 distinct 6×10 tilings in milliseconds.
- How does this compare to a SAT solver? → Encode exact cover as CNF and modern CDCL solvers are competitive or better on hard instances (they learn clauses; DLX doesn’t). DLX wins on enumeration density and memory locality. Real solver teams benchmark both.
- Sudoku uniqueness checking? → Run enumeration, stop at the second solution. Puzzle generators do this in an inner loop — another reason enumeration-native Algorithm X beats first-solution backtracking here.
Product Extension
- Puzzle generators (dedup + uniqueness checks in newspapers’ Sudoku pipelines)
- Crew/exam scheduling and set-partitioning formulations in OR (exact cover is set partitioning)
- PCB drilling / cutting-stock layout tools
- TAOCP Volume 4B: DLX is the backbone of Knuth’s entire combinatorial-search treatment
Language/Runtime Follow-ups
- Python: dict-of-sets version shown is the right call; pointer DLX in Python loses to it on constant factors.
- C/C++: pointer DLX with nodes in a flat arena array (indices, not pointers) — cache behavior dominates.
- Java: object-node DLX is idiomatic and fast enough; watch GC pressure when enumerating millions of solutions.
- JavaScript: typed-array arena DLX for performance; dict version for clarity.
Common Bugs
- Deselect order not reversed: state restored incorrectly; symptoms appear many nodes later as phantom missing rows — the classic nightmare bug of this algorithm.
- Iterating
X[col]while select mutates it: must snapshot (list(X[col])) in the branch loop. - Box index formula wrong:
3*(r//3) + c//3, notr//3 + 3*(c//3)— transposed boxes still solve some grids, poisoning tests. - Givens conflicting with each other: applying
selectfor an invalid given raises KeyError mid-setup; validate or catch and report unsolvable. - Digit off-by-one: grid digits 1–9 vs matrix digits 0–8 at both boundaries.
- Secondary columns treated as primary (n-queens): demanding diagonals be covered exactly once makes most boards “unsolvable”.
Debugging Strategy
- After select+deselect of any row on a fresh matrix, deep-compare X against a pristine copy — catches ordering bugs immediately
- Assert Σ|X[col]| invariant restored after every backtrack in debug mode
- Solve the 7-column hand example with verbose tracing; diff against the worked trace
- Cross-check Sudoku solutions with an independent validator (row/col/box sums and sets)
- Count search nodes; hard-but-valid 9×9 should be ≤ ~10^4 nodes — a wrong reduction shows up as millions
Mastery Criteria
- Write the dict-of-sets Algorithm X from memory in ≤ 25 min
- Derive the 4-family Sudoku reduction (324 columns, index formulas) on a whiteboard in ≤ 10 min
- Explain the O(1) unlink/relink argument and its LIFO precondition in ≤ 3 min
- Know primary vs secondary columns and the n-queens use
- Solve LC 37 with this machinery end-to-end in ≤ 45 min
Lab 05 — Big-Integer Arithmetic & Karatsuba
Goal
Implement arbitrary-precision integers from scratch — limb array in base 10^9 — with schoolbook add/sub/mul, then Karatsuba multiplication in O(n^1.585).
Background
A bignum is a little-endian array of “limbs”: fixed-size chunks of the number. Base 10^9 (9 decimal digits per limb) makes string I/O trivial; real libraries use base 2^30 or 2^64.
Schoolbook multiplication is O(n²) limb operations — pencil-and-paper.
Karatsuba (1960): split each operand at x = BASE^m: a = a₁x + a₀, b = b₁x + b₀. The naive product (a₁x+a₀)(b₁x+b₀) needs 4 half-size multiplications; Karatsuba needs 3:
z₂ = a₁b₁
z₀ = a₀b₀
z₁ = (a₁+a₀)(b₁+b₀) − z₂ − z₀
ab = z₂x² + z₁x + z₀
T(n) = 3T(n/2) + O(n) ⇒ O(n^log₂3) ≈ O(n^1.585). First algorithm to beat n² — disproved Kolmogorov’s conjecture that n² was optimal.
The ladder above it: Toom-3 (5 mults of size n/3, O(n^1.465)), Schönhage–Strassen (FFT-based, O(n log n log log n) — see phase-12 lab-07). CPython switches to Karatsuba at 70 limbs (~630 decimal digits); GMP implements the full ladder.
Interview Context
- LeetCode 43 “Multiply Strings” is a real FAANG question — schoolbook is the expected answer
- “How does Python multiply huge ints?” — genuine senior-level follow-up
- Cryptography engineering: bignum is the core primitive (RSA, ECC)
- Quant: exact arithmetic, fixed-point libraries
- ICPC / Codeforces: occasional, mostly in C++ where there is no native bigint
- Standard FAANG: schoolbook yes; Karatsuba coded live almost never
When to Skip This Topic
Skip if any of these are true:
- You can’t already write schoolbook “Multiply Strings” cleanly in 25 min
- You aren’t targeting crypto, systems, quant, or competitive roles
- You’re weak on divide-and-conquer recurrences — fix that first (Master theorem)
Learn schoolbook regardless. The Karatsuba half is optional for most.
Problem Statement
Exact multiplication of huge integers.
Given two non-negative integers as decimal strings of up to 10^5 digits each, output their exact product. The interviewer bans native bigints (Python int, Java BigInteger) — or equivalently asks “implement what Python does internally.”
Constraints
- Up to 10^5 decimal digits per operand
- Result exact — floating point disqualified
- Wall-clock: < 2 sec
- No native arbitrary-precision types
Clarifying Questions
- Inputs as decimal strings? (Yes, up to 10^5 digits each.)
- Negative numbers or leading zeros? (Track sign separately; strip leading zeros; “0” is valid.)
- Is the native bigint allowed anywhere? (No — that’s the exercise.)
- Exact result required? (Yes.)
Examples
Base-100 limbs, schoolbook, 1234 × 5678. Limbs little-endian: a = [34, 12], b = [78, 56].
position 0: 34×78 = 2652
position 1: 34×56 + 12×78 = 1904 + 936 = 2840
position 2: 12×56 = 672
raw [2652, 2840, 672] → carry → [52, 66, 0, 7] → 7006652
Same product via Karatsuba, x = 100: a₁=12, a₀=34, b₁=56, b₀=78.
z₂ = 12×56 = 672
z₀ = 34×78 = 2652
z₁ = (12+34)(56+78) − z₂ − z₀ = 46×134 − 3324 = 2840
672×100² + 2840×100 + 2652 = 7006652
Three multiplications instead of four.
Brute Force
Digit-by-digit schoolbook on the strings: out[i+j] += a[i]*b[j], then carry pass. For n = 10^5 digits: 10^10 digit multiplies — dead.
Brute Force Complexity
- Time: O(n²) in digits — 10^10 ops
- Space: O(n)
Optimization Path
- Pack 9 decimal digits per limb (base 10^9): 10^5 digits → 11,112 limbs. Schoolbook drops to ~1.2×10^8 limb multiplies — ~81× fewer, still ~1 min in pure Python.
- Karatsuba on limbs: 11112^1.585 ≈ 2.6×10^6 ops — sub-second.
- Cutoff: below ~32 limbs schoolbook wins (recursion overhead); fall back.
- Beyond scope here: Toom-3, then FFT/NTT (phase-12 lab-07) past ~10^4 limbs.
Final Expected Approach
BASE = 10**9
BASE_DIGITS = 9
CUTOFF = 32
def from_str(s):
limbs = [int(s[max(0, i - BASE_DIGITS):i]) for i in range(len(s), 0, -BASE_DIGITS)]
while len(limbs) > 1 and limbs[-1] == 0:
limbs.pop()
return limbs
def to_str(limbs):
return str(limbs[-1]) + "".join(str(x).zfill(BASE_DIGITS) for x in reversed(limbs[:-1]))
def normalize(limbs):
carry = 0
for i in range(len(limbs)):
carry, limbs[i] = divmod(limbs[i] + carry, BASE)
while carry:
carry, low = divmod(carry, BASE)
limbs.append(low)
while len(limbs) > 1 and limbs[-1] == 0:
limbs.pop()
return limbs
def add(a, b):
n = max(len(a), len(b))
out = [(a[i] if i < len(a) else 0) + (b[i] if i < len(b) else 0) for i in range(n)]
return normalize(out)
def sub(a, b):
out = a[:]
borrow = 0
for i in range(len(out)):
out[i] -= borrow + (b[i] if i < len(b) else 0)
borrow = 1 if out[i] < 0 else 0
if borrow:
out[i] += BASE
while len(out) > 1 and out[-1] == 0:
out.pop()
return out
def school_mul(a, b):
out = [0] * (len(a) + len(b))
for i, ai in enumerate(a):
if ai == 0:
continue
carry = 0
for j, bj in enumerate(b):
carry, out[i + j] = divmod(out[i + j] + ai * bj + carry, BASE)
out[i + len(b)] += carry
return normalize(out)
def karatsuba(a, b):
if min(len(a), len(b)) <= CUTOFF:
return school_mul(a, b)
m = max(len(a), len(b)) // 2
a0, a1 = a[:m], a[m:] or [0]
b0, b1 = b[:m], b[m:] or [0]
z0 = karatsuba(a0, b0)
z2 = karatsuba(a1, b1)
z1 = sub(sub(karatsuba(add(a0, a1), add(b0, b1)), z2), z0)
out = [0] * (len(a) + len(b) + 1)
for i, v in enumerate(z0):
out[i] += v
for i, v in enumerate(z1):
out[i + m] += v
for i, v in enumerate(z2):
out[i + 2 * m] += v
return normalize(out)
def multiply(s1, s2):
return to_str(karatsuba(from_str(s1), from_str(s2)))
Data Structures
- Little-endian list of int limbs, invariant 0 ≤ limb < 10^9, no leading-zero limbs
- Value = Σ limbs[i] × BASE^i
Correctness Argument
- Identity: expanding (a₁x+a₀)(b₁x+b₀) gives a₁b₁x² + (a₁b₀+a₀b₁)x + a₀b₀, and (a₁+a₀)(b₁+b₀) − a₁b₁ − a₀b₀ = a₁b₀ + a₀b₁. Exact, no approximation.
- Normalize preserves value: carrying moves BASE units one limb up; Σ limbs[i]·BASE^i is invariant.
- Induction: base case schoolbook is correct; recursive case combines three correct sub-products via the identity.
- Splitting at the low m limbs is exactly a = a₁·BASE^m + a₀.
Complexity
- Schoolbook: O(n²) — 1.2×10^8 limb mults at n = 11,112 limbs
- Karatsuba: O(n^log₂3) ≈ O(n^1.585) — ~2.6×10^6 at the same n
- Space: O(n log n) across recursion in this naive-allocation version; O(n) achievable in-place
Implementation Requirements
- Little-endian limbs; split takes the LOW half from index 0
- normalize() after every raw add/mul; sub() assumes a ≥ b (true for z₁ by construction)
- Cutoff to schoolbook near 32 limbs — pure Karatsuba to length 1 is slower than schoolbook
- from_str chunks 9 digits from the right; to_str zero-pads all limbs except the most significant
- Handle unequal lengths and “0”
Tests
- “1234” × “5678” = “7006652” (both paths)
- x × “0” = “0”, x × “1” = x
- Stress: 10^4 random pairs up to 200 digits vs Python native int; assert equal
- karatsuba vs school_mul agreement at lengths straddling the cutoff (31, 32, 33, 65 limbs)
- Two 10^5-digit random numbers in < 2 sec
- Highly unbalanced lengths: 10^5 digits × 3 digits
Follow-up Questions
- Why base 10^9 and not 2^30? → 10^9 makes decimal I/O O(n) (9 chars per limb); binary bases force an O(n²)-ish radix conversion. 2^30 gives cheaper carries (shifts/masks) and slightly more headroom. Both keep limb products < 2^63. Libraries pick binary and pay at I/O; contest code picks 10^9.
- When does Karatsuba actually beat schoolbook? → Only past a cutoff — extra adds and allocations dominate small sizes. Tune empirically: ~32 limbs here, 70 digits in CPython, ~10–30 64-bit limbs in GMP. Below cutoff, always fall back.
- How is division done? → Schoolbook long division is Knuth Algorithm D (TAOCP vol 2): normalize so the top divisor limb ≥ BASE/2, estimate each quotient limb from the top 2–3 limbs, correct at most twice. For huge operands: Newton–Raphson on the reciprocal (xₖ₊₁ = xₖ(2 − d·xₖ)) reuses fast multiplication, so division costs O(M(n)).
- The full multiplication ladder? → Schoolbook → Karatsuba (~2×10^3 bits) → Toom-3 (~10^4 bits) → Schönhage–Strassen FFT (~10^5–10^6 bits). Crossovers are machine-dependent; GMP tunes per-CPU at build time.
- How do Python and GMP really do it? → CPython: 30-bit digits, schoolbook below 70 digits, Karatsuba above, nothing faster (multiplying two 10^7-digit ints is visibly slow). GMP: whole ladder plus assembly inner loops — the reason
gmpy2beats native int by 10×+ at scale.
Product Extension
- Crypto libraries: 2048–4096-bit modmul is the hot loop of RSA/DH
- Exact decimal money types (java.math.BigDecimal is limb arithmetic underneath)
- Computer algebra systems, π-computation records (y-cruncher)
- Blockchain clients: 256-bit fixed-width words — same carry discipline, no recursion
Language/Runtime Follow-ups
- Python: native int already does all this; the lab is “reimplement without it.”
divmodkeeps carry code branch-free. - C/C++: base 10^9 in uint64_t — one product ~10^18 fits, but accumulating several overflows; carry every inner iteration or drop to base 10^4.
- Java: BigInteger switches to Karatsuba at 80 ints (2560 bits), Toom-3 at 240.
- JavaScript: number is a double — 2^53 ceiling forces base ≤ 10^7 for safe products; native BigInt (V8) uses Karatsuba-class algorithms.
Common Bugs
- Split from the wrong end: limbs are little-endian — a[:m] is the LOW part. Splitting the high part into z₀ scrambles everything.
- Missing final carry: schoolbook inner loop must flush carry into out[i + len(b)].
- Leading-zero limbs kept: breaks length-based cutoff logic and comparisons; trim in normalize.
- z₁ underflow in C: (a₁+a₀)(b₁+b₀) limbs reach 2·BASE−2 before carrying; subtract with signed intermediates or carry first.
- String chunking from the left: from_str must cut 9-digit chunks from the RIGHT; left-chunking corrupts all but exact-multiple lengths.
- to_str drops interior zeros: every non-top limb must be zfilled to 9 chars — [52, 66, 0, 7] is 7006652, not 76652.
- Unbalanced operands: m from the longer operand can make the shorter one’s high half empty — guard with
or [0].
Debugging Strategy
- Oracle: compare against Python native int on thousands of random cases — the single highest-value test
- Bisect: verify from_str/to_str round-trip alone, then add, then school_mul, then karatsuba
- Print limb arrays, not numbers — carry bugs are visible as limbs ≥ BASE
- Shrink BASE to 10 or 100 to make failing cases hand-checkable
Mastery Criteria
- Schoolbook “Multiply Strings” in ≤ 25 min, bug-free
- Full limb bignum with Karatsuba + cutoff in ≤ 60 min
- Derive the 3-multiplication identity from scratch on a whiteboard
- Solve T(n) = 3T(n/2) + O(n) and state n^1.585
- Name the ladder (Karatsuba → Toom-3 → FFT) with rough crossover sizes
- Explain the base-choice trade-off (10^9 vs 2^30) in under a minute
Lab 06 — Miller-Rabin & Pollard’s Rho
Goal
Deterministic primality testing for 64-bit integers, and full factorization of numbers up to ~10^18 in milliseconds.
Background
Fermat test: if n is prime, a^(n−1) ≡ 1 (mod n). Necessary, not sufficient — Carmichael numbers (561 = 3×11×17 is the smallest) pass for every a coprime to n. Infinitely many exist.
Miller-Rabin (strong pseudoprime test): write n−1 = d·2^s with d odd. For prime n, the sequence a^d, a^(2d), …, a^(2^s·d) must reach 1, and the only square roots of 1 mod a prime are ±1 — so either a^d ≡ 1 or some a^(2^r·d) ≡ −1. An a violating this is a witness: a certificate that n is composite. A random a is a witness for composite n with probability ≥ 3/4 — no Carmichael-style escape.
Deterministic for 64-bit: testing the first 12 primes {2,3,5,7,11,13,17,19,23,29,31,37} is correct for all n < 3.317×10^24 (Sorenson & Webster 2015) — covers all of uint64.
Pollard’s rho (1975): iterate x → x² + c mod n. Viewed mod p (smallest prime factor), the sequence enters a cycle after expected O(√p) steps — birthday paradox. A collision mod p but not mod n means gcd(|x−y|, n) is a nontrivial factor. Floyd (tortoise/hare) or Brent (backtracking powers of 2, ~24% fewer evals, batched gcds) detects it.
Interview Context
- Codeforces / ICPC: standard tool; Project Euler: constant
- Cryptography engineering: primality generation for RSA/DH keys is exactly this
- CTF / security: factoring weak keys is a staple
- Quant: occasional number-theory screens
- Standard FAANG: near zero — at most “check if n is prime” (trial division suffices)
When to Skip This Topic
Skip if any of these are true:
- You’re not targeting competitive programming, crypto, or security roles
- You haven’t internalized modular exponentiation and gcd
- Your pipeline never sees integers past 10^12 (trial division is fine there)
If FAANG-only: skip the whole lab, keep the trial-division one-liner.
Problem Statement
Fast factorization.
Given up to 100 integers, each up to 10^18, output the full prime factorization of each (primes with multiplicity, sorted). Products of two ~10^9 primes must work — trial division dies there.
Constraints
- ≤ 100 queries, each 1 ≤ n ≤ 10^18
- Total wall-clock: < 1 sec
- Deterministic output required (no failure probability)
Clarifying Questions
- How large can n be? (10^18 — fits uint64; √n loop does not fit the time budget.)
- Is a probabilistic answer acceptable? (No — and it needn’t be: fixed witnesses are deterministic for 64-bit.)
- Output format? (Sorted prime list with multiplicity, e.g. 12 → [2, 2, 3].)
- Edge inputs? (1 → empty factorization; assume n ≥ 1, no negatives.)
Examples
Miller-Rabin on n = 221 (= 13×17). n−1 = 220 = 55·2², so d = 55, s = 2.
a = 174: 174^55 mod 221 = 47; 47² mod 221 = 220 = n−1 → passes (174 is a strong liar)
a = 137: 137^55 mod 221 = 188; 188² mod 221 = 205 ≠ ±1 → 137 is a witness: composite
One liar survives; the fixed 12-prime battery does not lie for any 64-bit n.
Pollard’s rho on 8051 = 83×97, f(x) = x²+1, x = y = 2:
step 1: x=5, y=26 gcd(21, 8051) = 1
step 2: x=26, y=7474 gcd(7448, 8051) = 1
step 3: x=677, y=871 gcd(194, 8051) = 97 → factor found
Brute Force
Trial division by 2, 3, 5, … up to √n. Fine to n ≈ 10^12 (10^6 iterations). For a semiprime near 10^18: 10^9 iterations per query — dead.
Brute Force Complexity
- Time: O(√n) per query — 10^9 ops worst case
- Space: O(1)
Optimization Path
- Strip primes < 100 by trial division (kills most mass cheaply).
- Miller-Rabin with the fixed 12-witness set → deterministic is_prime for uint64 in O(12 log³ n).
- If composite: Pollard’s rho finds a factor in expected O(n^{1/4}); recurse on both halves.
- Constant factors: Brent’s cycle detection + batching gcds (multiply ~128 differences mod n, one gcd).
Final Expected Approach
import math
WITNESSES = (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37)
def is_prime(n):
if n < 2:
return False
for p in WITNESSES:
if n % p == 0:
return n == p
d, s = n - 1, 0
while d % 2 == 0:
d //= 2
s += 1
for a in WITNESSES:
x = pow(a, d, n)
if x == 1 or x == n - 1:
continue
for _ in range(s - 1):
x = x * x % n
if x == n - 1:
break
else:
return False
return True
def pollard_rho(n):
if n % 2 == 0:
return 2
c = 1
while True:
x = y = 2
d = 1
while d == 1:
x = (x * x + c) % n
y = (y * y + c) % n
y = (y * y + c) % n
d = math.gcd(abs(x - y), n)
if d != n:
return d
c += 1
def factor(n):
if n == 1:
return []
if is_prime(n):
return [n]
d = pollard_rho(n)
return sorted(factor(d) + factor(n // d))
is_prime must run before pollard_rho — rho never terminates on a prime. The small-prime loop doubles as the n ≤ 37 guard.
Data Structures
- None beyond integers — the whole lab is arithmetic
- Recursion stack for factor(): depth ≤ log₂ n ≈ 60
Correctness Argument
- Miller-Rabin: for prime n, a^(n−1) ≡ 1 and repeated square-rooting from 1 can only pass through ±1; so the d·2^r ladder must show a^d ≡ 1 or hit −1. Any a breaking this proves n composite — no false “composite” verdicts, ever.
- Determinism: the 12-prime witness set has been exhaustively verified correct below 3.317×10^24 > 2^64. No probability involved.
- Rho: x_i mod p cycles with expected tail+cycle O(√p); Floyd guarantees some i with x_i ≡ x_{2i} (mod p), giving p | gcd(|x_i − x_{2i}|, n). gcd = n only if all prime factors collide simultaneously — then retry with the next c changes the sequence.
- factor(): splits n into two smaller cofactors and recurses; termination since each part shrinks and primes are leaves.
Complexity
- is_prime: O(12 log³ n) bit ops — microseconds via three-arg pow
- pollard_rho: expected O(n^{1/4}) iterations — ≈ 31,600 for the worst 64-bit semiprime (p ≈ 10^9)
- factor: O(n^{1/4} log n) expected; 100 worst-case queries well under 1 sec
- Space: O(log n)
Implementation Requirements
- Three-arg
pow(a, d, n)only — nevera**d % n - Inner squaring loop runs s−1 times, not s (a^d already checked)
- Even n handled before rho; is_prime before rho, always
- c advances deterministically 1, 2, 3, … on gcd = n — reproducible runs, no RNG
- Return factors sorted with multiplicity
Tests
- Primes: 2, 3, 5, 2^61−1, 999999999999999989 (largest prime < 10^18)
- Carmichael: 561 → [3, 11, 17]; 41041 → [7, 11, 13, 41]
- Strong-liar case: 221 with witness list must say composite
- Semiprime: 10^18-scale product of two ~10^9 primes, e.g. 999999937 × 999999893
- Edge: 1 → [], 4 → [2, 2], perfect powers: 2^60, 3^37
- Stress: factor(n) product check + all-parts-prime check for 10^4 random n < 10^12 vs trial division
- Performance: 100 random 18-digit numbers in < 1 sec
Follow-up Questions
- Why is the fixed witness set deterministic — isn’t Miller-Rabin probabilistic? → Probabilistic only for random witnesses. Explicit computation found every strong pseudoprime to small prime bases: Jaeschke (1993) pushed the first 8 primes to 3.4×10^14; Sorenson–Webster (2015) verified the first 12 primes below 3.317×10^24. Jim Sinclair’s set {2, 325, 9375, 28178, 450775, 9780504, 1795265022} covers uint64 with only 7 rounds.
- Why is rho expected O(n^{1/4})? → Let p be the smallest prime factor, p ≤ √n. The pseudorandom sequence mod p collides after expected O(√p) steps (birthday bound over p values). O(√p) ≤ O(n^{1/4}).
- Why does rho fail on primes, and on n = 4? → On prime n the only “collision” is mod n itself, so gcd is always 1 or n — infinite loop; call is_prime first. n = 4: √p = √2 gives a degenerate 1-element cycle that x²+c may never expose — the even-n guard (and small-prime stripping) removes it.
- Brent vs Floyd? → Brent keeps one saved point and checks x_i against x_{2^k} — ~24% fewer function evaluations. The real win is batching: accumulate ∏|x−y| mod n over ~128 steps, take one gcd; a 30–50% wall-clock cut since gcd is the expensive step.
- Beyond 64 bits? → ECM digs out factors up to ~10^20 from arbitrarily large n (cost scales with the factor, not n); quadratic sieve is fastest for n up to ~100 digits; GNFS beyond that — RSA-250 (829 bits) fell to GNFS in 2020 after ~2700 core-years.
- Relation to RSA? → RSA keygen is Miller-Rabin (find two ~1024-bit primes); RSA security is the absence of anything like rho at that scale — rho is n^{1/4}, i.e. 2^512 steps for a 2048-bit modulus. These two algorithms are both sides of the asymmetry.
Product Extension
- Key-generation services: primality testing at scale (HSMs, TLS cert tooling)
- Weak-key auditing: batch-factoring RSA moduli harvested from devices (Mining-your-Ps-and-Qs style)
- Hash-table / hashing libraries: next-prime sizing
- Number-theory backends: SymPy’s
factorint, Pari/GP — same rho + MR core
Language/Runtime Follow-ups
- Python: three-arg pow is C-speed; native bigints make this lab genuinely fast in pure Python — a rare case.
- C++: mulmod of two uint64 overflows — use
__int128; witnesses reduced mod n need the a % n == 0 guard. - Java:
Math.multiplyHighfor mulmod, orBigInteger.isProbablePrime(certainty)as the library answer. - Rust: u128 intermediates;
num-primecrate mirrors this exact design.
Common Bugs
- s vs s−1: squaring loop after the initial a^d check must run s−1 times; running s wrongly passes some composites through the final compare.
- Missing x == n−1 pre-check: if a^d ≡ n−1 immediately, the round passes; forgetting it yields false “composite” on primes.
- Witness divides n: for small n, a ∈ WITNESSES may equal or divide n — the small-prime loop must run first or 3 gets declared composite.
- Rho on a prime: infinite loop. is_prime gate is mandatory, not an optimization.
- gcd = n treated as failure: it means all factors collided at once — retry with next c, don’t return n.
a**d % n: computes a full 10^17-bit power first; always three-arg pow.- C++ mulmod overflow: (x*x) % n with x near 10^18 silently wraps; everything downstream is garbage.
Debugging Strategy
- Cross-check is_prime against a sieve for all n < 10^6
- factor(n): assert product of parts == n and is_prime on every part — self-verifying by construction
- Deterministic c sequence means failures reproduce exactly; print (c, iteration count) on retry
- Hand-trace 221 and 8051 (tables above) against your implementation’s intermediate values
Mastery Criteria
- Deterministic is_prime from memory in ≤ 15 min
- Full factor() with rho in ≤ 35 min
- Reproduce the n = 221 witness/liar trace by hand
- Derive the O(n^{1/4}) birthday bound in two sentences
- State the deterministic witness set and its verified bound
- Factor 100 random 10^18 integers in < 1 sec, first run
Lab 07 — Extended Euclid, Modular Inverse & CRT
Goal
Implement extended gcd, modular inverse, and the Chinese Remainder Theorem — including the general non-coprime case with contradiction detection.
Background
Bézout identity: for any a, b there exist x, y with ax + by = gcd(a, b). Extended Euclid computes all three simultaneously: recursively from gcd(b, a mod b), or iteratively by carrying two coefficient rows through the ordinary gcd loop. O(log min(a, b)) steps.
Modular inverse: a⁻¹ mod m exists iff gcd(a, m) = 1; it’s the Bézout x of ax + my = 1. The Fermat shortcut a^(p−2) mod p works only for prime p; for composite m it’s ext_gcd or a^(φ(m)−1) — and φ needs m’s factorization, so ext_gcd is the default.
CRT: x ≡ a₁ (mod m₁), x ≡ a₂ (mod m₂) with coprime moduli has a unique solution mod m₁m₂. Non-coprime general case: let g = gcd(m₁, m₂); solvable iff g | (a₂ − a₁), and then unique mod lcm(m₁, m₂). k congruences: fold pairwise merges left to right.
Garner’s algorithm: reconstructs a big integer from its residues mod several primes in mixed-radix form, never materializing numbers larger than the answer — the standard finish for multi-modulus NTT (see phase-12 lab-07).
Interview Context
- Codeforces / ICPC: modular inverse is used constantly (nCr mod p); CRT appears in mid-hard problems
- Cryptography engineering: RSA keygen and RSA-CRT decryption are literally these primitives
- Quant / research: exact rational arithmetic via multi-modular tricks
- Systems: hashing, ring buffers, cycle alignment of periodic tasks
- Standard FAANG: ext_gcd rare;
pow(a, -1, m)occasionally in counting problems
When to Skip This Topic
Skip if any of these are true:
- You don’t do modular-arithmetic problems at all (no competitive, no crypto)
- You haven’t got plain gcd and fast exponentiation cold
- Your only need is inverse mod a prime — memorize
pow(a, p-2, p)and move on
Of the three Phase-13 number-theory labs, this one has the widest reach. Skip last.
Problem Statement
System of congruences, non-coprime moduli.
Given up to 10^5 congruences x ≡ aᵢ (mod mᵢ) with mᵢ ≤ 10^9 and NO coprimality guarantee, output the smallest non-negative x satisfying all of them, or “no solution”.
Constraints
- 1 ≤ k ≤ 10^5 congruences, 1 ≤ mᵢ ≤ 10^9
- Moduli arbitrary — shared factors and contradictions both possible
- Combined modulus is lcm(m₁…mₖ): astronomically large — bigints required (Python native)
- Wall-clock: < 2 sec
Clarifying Questions
- Are the moduli pairwise coprime? (No — the general case is the exercise.)
- Can the system be inconsistent? (Yes — detect and report “no solution”.)
- Are the aᵢ already reduced mod mᵢ? (Not guaranteed — reduce first; they may be negative.)
- How large does the answer get? (Up to lcm of all mᵢ — thousands of digits; fine in Python, needs bigint elsewhere.)
Examples
ext_gcd(240, 46) — full iterative table:
r 240 46 10 6 4 2 0
q 5 4 1 1 2
s 1 0 1 −4 5 −9
t 0 1 −5 21 −26 47
gcd = 2, Bézout: 240×(−9) + 46×47 = −2160 + 2162 = 2
Three-congruence merge (Sunzi’s original):
x ≡ 2 (mod 3), x ≡ 3 (mod 5), x ≡ 2 (mod 7)
merge(2,3 | 3,5) → x ≡ 8 (mod 15)
merge(8,15 | 2,7) → x ≡ 23 (mod 105)
Contradiction:
x ≡ 1 (mod 4), x ≡ 3 (mod 8): g = gcd(4,8) = 4, (3−1) % 4 = 2 ≠ 0 → no solution
(x ≡ 3 mod 8 forces x ≡ 3 mod 4)
Brute Force
Scan candidates x = 0, 1, …, lcm−1 and test all k congruences. lcm of 10^5 moduli up to 10^9 can exceed 10^100000. Dead on arrival.
Brute Force Complexity
- Time: O(lcm × k) — unbounded for practical purposes
- Space: O(1)
Optimization Path
- ext_gcd: gcd + Bézout coefficients in O(log min(a, b)).
- Two-congruence merge: x = a₁ + m₁t; need m₁t ≡ a₂−a₁ (mod m₂). With g = gcd(m₁, m₂): solvable iff g | (a₂−a₁); divide through by g, invert m₁/g mod m₂/g via Bézout, result unique mod lcm.
- Fold the merge over all k congruences; abort on first contradiction.
- Keep intermediates reduced (t mod m₂/g) so numbers grow no faster than the running lcm.
Final Expected Approach
def ext_gcd(a, b):
old_r, r = a, b
old_s, s = 1, 0
old_t, t = 0, 1
while r:
q = old_r // r
old_r, r = r, old_r - q * r
old_s, s = s, old_s - q * s
old_t, t = t, old_t - q * t
return old_r, old_s, old_t
def modinv(a, m):
g, x, _ = ext_gcd(a % m, m)
if g != 1:
raise ValueError("no inverse: gcd != 1")
return x % m
def crt_merge(a1, m1, a2, m2):
g, p, _ = ext_gcd(m1, m2)
if (a2 - a1) % g:
return None
lcm = m1 // g * m2
t = (a2 - a1) // g * p % (m2 // g)
return (a1 + m1 * t) % lcm, lcm
def solve(congruences):
a, m = 0, 1
for ai, mi in congruences:
merged = crt_merge(a, m, ai % mi, mi)
if merged is None:
return None
a, m = merged
return a
Starting from x ≡ 0 (mod 1) makes the fold uniform — every input congruence is just another merge.
Data Structures
- Two integers (a, m): the running solution class x ≡ a (mod m)
- No arrays, no recursion — the entire state is one residue class
Correctness Argument
- ext_gcd invariant: old_s·a + old_t·b = old_r and s·a + t·b = r hold before and after every loop iteration (each row update is a linear combination of the previous two). At exit old_r = gcd with its Bézout pair.
- Merge: x satisfies both congruences iff x = a₁ + m₁t and m₁t ≡ a₂−a₁ (mod m₂). This linear congruence in t is solvable iff g | (a₂−a₁); dividing by g gives (m₁/g)t ≡ (a₂−a₁)/g (mod m₂/g) with coprime coefficient, so t is unique mod m₂/g — hence x unique mod m₁·(m₂/g) = lcm. From m₁p + m₂q = g, p is exactly (m₁/g)⁻¹ mod (m₂/g).
- Fold: by induction the running (a, m) encodes exactly the solution set of the congruences processed so far — either a full residue class mod lcm, or empty (early return). Final a ∈ [0, lcm) is the smallest non-negative solution.
Complexity
- ext_gcd: O(log min(a, b)) ≈ ≤ 45 iterations for 10^9 inputs (worst case: Fibonacci pairs)
- Full solve: k merges; bigint arithmetic on the running lcm of L bits makes it O(k · M(L)) — fast in practice because shared factors keep L from growing at every step
- Space: O(L) for the running modulus
Implementation Requirements
- Iterative ext_gcd (recursive dies at 10^5-scale inputs in Python only if misused, but iterative is the habit)
- Reduce aᵢ mod mᵢ before merging — inputs may be negative or oversized
- lcm as m1 // g * m2 — divide before multiplying
- Reduce t mod m₂/g before forming x — keeps intermediates within ~2× the answer
- Contradiction returns “no solution” immediately; don’t process remaining congruences
- Final answer normalized to [0, lcm)
Tests
- Sunzi: [(2,3), (3,5), (2,7)] → 23
- Contradiction: [(1,4), (3,8)] → no solution
- Redundant overlap: [(3,8), (3,4)] → 3 mod 8 (non-coprime, consistent)
- Single congruence: [(5,7)] → 5; identity: [(0,1)] → 0
- ext_gcd(240, 46) = (2, −9, 47); modinv(3, 7) = 5; modinv(4, 8) raises
- Stress: random systems with lcm < 10^6 vs brute-force scan; assert identical including “no solution”
- Performance: 10^5 random congruences, moduli ≤ 10^9, < 2 sec
Follow-up Questions
- Why not Fermat a^(m−2) for the inverse? → That’s Fermat’s little theorem — prime modulus only. For composite m the exponent is φ(m)−1, and computing φ requires factoring m. ext_gcd needs neither primality nor factorization and is O(log m); it’s the default. (Python 3.8+:
pow(a, -1, m)does this for you.) - This overflows in C++/Java — where? → m₁·m₂ up to 10^18 fits int64, but a₁ + m₁·t and (a₂−a₁)·p exceed it. Use
__int128(C++) orMath.multiplyHigh/mulmod (Java); for k-fold merges the running lcm needs a real bigint. Python is exempt — that’s why this lab is in Python. - What is Garner’s algorithm and when do you need it? → Given x mod p₁, …, x mod pₖ, Garner computes mixed-radix digits x = c₁ + c₂p₁ + c₃p₁p₂ + …, each cᵢ found with one modinv mod pᵢ. All arithmetic stays word-sized until final assembly — essential for reconstructing multi-modulus NTT results (phase-12 lab-07) in languages without cheap bigints.
- Solve ax ≡ b (mod m) when gcd(a, m) = g > 1? → Solvable iff g | b. Then x ≡ (b/g)·(a/g)⁻¹ (mod m/g) — one solution mod m/g, hence exactly g distinct solutions mod m, spaced m/g apart.
- Where does RSA use this? → Keygen: d = e⁻¹ mod φ(n) is one ext_gcd call. Decryption: RSA-CRT computes m mod p and m mod q with half-size exponents and moduli, then recombines with a precomputed q⁻¹ mod p — modmul is quadratic and exponents halve, so ~4× faster; every serious TLS stack ships it.
- Non-coprime CRT vs coprime — what actually changes? → Coprime: g = 1, always solvable, modulus m₁m₂. Non-coprime: a solvability condition appears (g | a₂−a₁) and the combined modulus drops to lcm. The coprime formula silently gives wrong answers on non-coprime input — the classic buried bug.
Product Extension
- TLS/SSH stacks: RSA-CRT private-key operations
- Multi-modular exact linear algebra: solve mod several primes, CRT the result (SageMath, NTL)
- Calendar/scheduling engines: alignment of periodic events is a congruence system
- Distributed ID reconstruction: residue number systems for parallel arithmetic
Language/Runtime Follow-ups
- Python:
%returns non-negative,pow(a, -1, m)exists, bigints are native — the easiest language for this by a wide margin. - C++:
%keeps the sign of the dividend — every reduce needs((x % m) + m) % m; use__int128for merge products. - Java:
Math.floorMod,BigInteger.modInversethrows on non-coprime — matching our modinv contract. - Rust: i128 covers the pairwise merge;
num-bigintfor the fold.
Common Bugs
- Negative Bézout coefficient used raw: ext_gcd routinely returns negative x; every exit path needs a final
% mnormalization (which in C++ still isn’t enough — see sign bug). - Coprime formula on non-coprime moduli: no crash, just wrong answers — the merge must go through g.
- Sign of (a₂−a₁) % g in C++: a negative nonzero remainder marks solvable systems as contradictions; Python’s
%hides this class of bug entirely. - lcm as m1 * m2 / g: overflows before the division in fixed-width languages; always m1 / g * m2.
- t not reduced mod m₂/g: correct value but intermediates balloon; in C++ this is an overflow, in Python a slowdown.
- aᵢ not pre-reduced: negative or oversized residues corrupt the solvability check.
- modinv without checking g: returns garbage “inverse” for gcd > 1; must raise or return sentinel.
Debugging Strategy
- Assert g == ax + by after every ext_gcd call — one line, catches coefficient-row bugs instantly
- Verify the final x against every input congruence before returning — O(k), cheap, self-verifying
- Stress against brute-force scan on systems with lcm < 10^6, including engineered contradictions
- Reproduce the 240/46 table by hand next to a printout of the loop state
Mastery Criteria
- ext_gcd + modinv from memory in ≤ 10 min
- General-case crt_merge + fold in ≤ 25 min
- Reproduce the ext_gcd(240, 46) table by hand, no reference
- State the non-coprime solvability condition and the merged modulus without hesitation
- Explain why Fermat inverse fails for composite m in one sentence
- Explain the RSA-CRT 4× speedup in under two minutes
Lab 08 — Discrete Logarithm: Baby-Step Giant-Step
Goal
Solve g^x ≡ h (mod p) in O(√p) time and space with Shanks’ baby-step giant-step (1971), and know when Pollard’s rho or Pohlig-Hellman is the right tool instead.
Background
The discrete logarithm problem (DLP): given g, h, p, find x with g^x ≡ h (mod p). The forward direction (modular exponentiation) costs O(log x); no known classical algorithm inverts it in polynomial time. That asymmetry is the security basis of Diffie-Hellman key exchange and DSA.
Shanks (1971), meet-in-the-middle: let m = ⌈√p⌉. Any candidate x decomposes as x = i·m + j with 0 ≤ j < m, 0 ≤ i ≤ m. Precompute baby steps g^j into a hash table; then take giant steps γ = h·(g^−m)^i and probe the table — a hit at (i, j) means x = i·m + j. The classical inverse-free twin uses x = i·m − j: baby steps h·g^j into the table, giant probes (g^m)^i. Either way: two O(√p) halves replace one O(p) scan.
Pollard’s rho for logarithms: same O(√p) expected time, O(1) space — pseudorandom walk over group elements tracked as g^a·h^b, cycle found via Floyd/Brent, then solve a linear congruence mod p−1. The answer when the BSGS table doesn’t fit in RAM.
Pohlig-Hellman: if p−1 is smooth (only small prime factors), solve the DLP independently in each prime-order subgroup — each at O(√q) by BSGS or rho — and recombine with CRT (cross-ref phase-12 lab-07). Total cost is dominated by the largest prime factor of the group order.
Interview Context
- Cryptography / security engineering: standard (CTFs, protocol audits, parameter reviews)
- ICPC / Codeforces: occasional — BSGS is the intended trick behind “find k with a^k ≡ b” subproblems
- Blockchain / applied-crypto roles: expected vocabulary (ECDLP hardness is why keys are 256 bits)
- Quant: rare
- Standard FAANG: zero — this does not appear on LeetCode
When to Skip This Topic
Skip if any of these are true:
- You aren’t targeting cryptography, security, or ICPC roles
- Modular exponentiation and inverses (phase-07 labs 01–02) aren’t automatic yet
- Your pipeline is standard FAANG loops only
If you skip, still learn one sentence: “DLP is easy forward, believed hard backward — that gap is Diffie-Hellman.”
Problem Statement
Given a prime p ≤ 10^18, a generator g of F_p*, and a target h with 1 ≤ h < p, find the smallest x ≥ 0 such that
g^x ≡ h (mod p)
or report none. (None is only possible when g is not actually a full generator and h lies outside the subgroup ⟨g⟩.)
Constraints
- p prime, p ≤ 10^18 in principle; the hash table has m = ⌈√p⌉ entries, so memory binds first
- Realistic single-machine range: p ≤ 10^12 in Python (m ≈ 10^6 dict entries), ~10^14–10^16 in C++
- 1 ≤ g, h < p
- Wall-clock: O(√p) modular multiplications
Clarifying Questions
- Is p guaranteed prime? (Yes — composite modulus changes the group structure; see follow-ups.)
- Is g guaranteed to generate all of F_p*? (Assume yes; if not, a solution may not exist — return None.)
- Smallest x or any x? (Smallest non-negative; solutions are unique mod ord(g).)
- How big is p really? (Sets BSGS vs Pollard rho — memory is the deciding constraint, not time.)
Examples
Solve 3^x ≡ 13 (mod 17). m = ⌈√17⌉ = 5.
Baby-step table, table[g^j] = j:
j: 0 1 2 3 4
3^j: 1 3 9 10 13
Giant multiplier: 3^5 = 243 ≡ 5 (mod 17), and 5^−1 ≡ 7 (mod 17), so g^−m ≡ 7.
i=0: γ = 13 → in table at j=4 → x = 0·5 + 4 = 4
Check: 3^4 = 81 = 4·17 + 13 ✓
Second target h = 5, forcing a real giant step:
i=0: γ = 5 → miss
i=1: γ = 5·7 ≡ 1 → in table at j=0 → x = 1·5 + 0 = 5
Check: 3^5 = 243 ≡ 5 (mod 17) ✓
Brute Force
Multiply an accumulator by g up to p−1 times, comparing against h. For p near 10^9: ~10^9 modmuls — seconds-to-minutes in C++, dead in Python. For p = 10^18: dead everywhere.
Brute Force Complexity
- Time: O(p)
- Space: O(1)
Optimization Path
- Set m = ⌈√p⌉; every x < p−1 writes uniquely as x = i·m + j with 0 ≤ j < m.
- Baby steps: insert g^j → j for j = 0..m−1, keeping the smallest j per value.
- Compute the giant multiplier g^−m ≡ g^(m·(p−2)) (mod p) by Fermat’s little theorem.
- Giant steps: γ = h; for i = 0..m, probe γ in the table — hit means return i·m + j; else γ ← γ·g^−m.
- Meet-in-the-middle: O(√p) inserts + O(√p) probes replaces the O(p) scan.
Final Expected Approach
import math
def bsgs(g, h, p):
g %= p
h %= p
if h == 1:
return 0
m = math.isqrt(p) + 1
table = {}
e = 1
for j in range(m):
if e not in table:
table[e] = j
e = e * g % p
giant = pow(g, m * (p - 2), p)
gamma = h
for i in range(m + 1):
if gamma in table:
return i * m + table[gamma]
gamma = gamma * giant % p
return None
Storing the smallest j per value and scanning i upward guarantees the returned x is minimal — even when ord(g) < m and baby values collide. The inverse-free twin (baby steps h·g^j, giant probes (g^m)^i, answer i·m − j) is equally standard; it must store the largest j for minimality and needs the explicit h = 1 → 0 case.
Data Structures
- Hash table of size m = ⌈√p⌉ mapping group element → smallest baby exponent
- Two scalar accumulators; nothing else
Correctness Argument
- Coverage: i ≤ m, j < m spans all x ≤ m² + m − 1, and m² > p > ord(g), so every candidate log is reachable.
- Hit ⟺ solution: γ_i = h·g^(−i·m) ≡ g^j ⟺ g^(i·m + j) ≡ h.
- Minimality: let x₀ = i₀·m + j₀ be the least solution. A hit at i < i₀ would yield a solution < x₀ — impossible. At i = i₀, the stored (smallest) j with g^j = γ is exactly j₀, else a smaller solution would exist.
- No solution → all m+1 probes miss → None.
Complexity
- Time: O(√p) expected (hash operations amortized O(1))
- Space: O(√p)
Implementation Requirements
math.isqrt(p) + 1, neverint(p ** 0.5)— float sqrt is wrong near 10^18- Reduce g, h mod p up front; reject h ≡ 0
- Insert baby steps with
if e not in tableto keep the smallest j - Use three-argument
pow(base, exp, mod)throughout - Bound the giant loop at m+1 iterations and return None cleanly
Tests
- bsgs(3, 13, 17) = 4 and bsgs(3, 5, 17) = 5 (the hand examples)
- bsgs(g, 1, p) = 0 for any g
- Round-trip: random x < p−1, h = pow(g, x, p); assert pow(g, bsgs(g, h, p), p) == h and result ≤ x
- Exhaustive cross-check vs the O(p) scan for all h, p < 10^3 (smallest x must match exactly)
- Non-generator g of small order, h ∉ ⟨g⟩ → None
- Performance: p ≈ 10^12 prime answers in a few seconds (m ≈ 10^6)
Follow-up Questions
- The table is O(√p) — what if it doesn’t fit in RAM? → Pollard’s rho for logs: pseudorandom walk over elements written as g^a·h^b, Floyd/Brent cycle detection, then solve a·x ≡ b (mod p−1). Same O(√p) expected time, O(1) space. Memory kills BSGS long before time does: p = 10^18 means 10^9 table entries — tens of GB.
- p−1 factors into small primes — can you beat √p? → Pohlig-Hellman: solve in each prime-power subgroup at O(√q) each, stitch with CRT. Cost tracks the largest prime factor of the order — which is exactly why DH/DSA standards mandate groups of (near-)prime order.
- g isn’t a generator, or the modulus isn’t prime? → Work inside ⟨g⟩: replace p−1 with ord(g) and set m = ⌈√ord(g)⌉; if h ∉ ⟨g⟩ there is no solution. For composite n, CRT-split into prime-power moduli and solve each piece.
- Is O(√p) optimal? → Not in F_p*: index calculus is subexponential (L_p[1/3]), which is why DH primes must be 2048+ bits. It does not transfer to elliptic-curve groups, where √-time generic attacks remain the best known — the reason 256-bit ECC matches ~3072-bit RSA/DH.
- LeetCode-adjacent framing? → It almost never appears there. What transfers is the shape: meet-in-the-middle, “split the exponent, trade √n memory for √n time” — the same instinct as splitting subset-sum search spaces in half.
- Quantum computers? → Shor solves DLP (mod-p and elliptic-curve) in polynomial time; hence the migration to lattice-based post-quantum schemes.
Product Extension
- CTF/security-audit tooling: recover keys from weak or undersized DH parameters
- Parameter linters: reject groups with smooth order (Pohlig-Hellman-weak) before deployment
- Small-subgroup confinement attack validation in protocol test harnesses
- Distributed log-cracking via Pollard rho + distinguished points (how real DLP records are set)
Language/Runtime Follow-ups
- Python: built-in
powandmath.isqrtdo the heavy lifting; a dict of 10^6–10^7 entries is fine, 10^8 is not. - C++: 64-bit modmul needs
unsigned __int128;unordered_mapwithreserve, or sort-plus-binary-search over a baby array to halve memory. - Java:
BigInteger.modPowis correct but slow; boxedHashMap<Long,Integer>overhead is real at 10^7 entries. - Rust:
u128intermediates;HashMap::with_capacityto avoid rehashing.
Common Bugs
- Float square root:
int(p ** 0.5)is off by one for large p; solutions near p−1 silently vanish. - Overwriting baby entries: keeping the last j instead of the smallest returns a valid but non-minimal x when ord(g) < m.
- Loop bounds shaved: j must cover 0..m−1 and i must reach m; either off-by-one loses logs near p−1.
- Missing h = 1: the i·m − j variant never probes x = 0 and returns p−1 instead.
- Wrong inverse: pow(g, p−2, p) needs p prime; composite modulus requires extended Euclid with gcd(g, n) = 1.
- Unreduced inputs: h ≥ p makes every probe miss — returns None with no error.
Debugging Strategy
- Test the property, not the value: pow(g, bsgs(g, h, p), p) == h for random round-trips
- Exhaustive diff against the O(p) brute force for p < 10^4 — catches minimality bugs
- Print (i, γ) probe traces for p = 17 and diff against the hand table above
- Feed a deliberate non-generator and confirm None, not garbage
Mastery Criteria
- Implement bsgs (minimal x, h = 1, None case) bug-free in ≤ 30 min
- Solve a mod-17 instance fully by hand in ≤ 5 min
- Recite the BSGS / Pollard rho / Pohlig-Hellman trade-off table (time, space, order-smoothness) from memory
- Explain in two sentences why index calculus makes ECC keys shorter than DH keys
- Recognize meet-in-the-middle exponent splits in non-crypto problems
Lab 09 — Matrix Exponentiation & Linear Recurrences
Goal
Compute the n-th term of any fixed-order linear recurrence mod p for n up to 10^18 in O(k³ log n) via matrix power, where k is the recurrence order.
Background
A linear recurrence f(n) = c₁f(n−1) + … + c_k f(n−k) is one multiplication of the state vector [f(n−1), …, f(n−k)] by its companion matrix:
[c₁ c₂ … c_{k−1} c_k]
[ 1 0 … 0 0 ]
[ 0 1 … 0 0 ]
[ 0 0 … 1 0 ]
So f(n) reads off M^(n−k+1) applied to the initial state. Binary exponentiation (phase-07 lab-02) needs only associativity — lift it from scalars to matrices unchanged: O(log n) multiplications, O(k³) each.
Fibonacci is the canonical k = 2 case: [[1,1],[1,0]]^n = [[F(n+1), F(n)], [F(n), F(n−1)]].
Graph interpretation: for adjacency matrix A, (A^k)[i][j] counts length-k walks i → j. Every “count sequences of length n under local rules” problem is a walk count on an automaton → matrix power.
The pattern: DP with a huge time dimension. If one DP step is linear (state vector × fixed transition matrix), 10^18 steps collapse to ~60 matrix squarings.
Beyond k³: Kitamasa — exponentiate x^n mod the characteristic polynomial — gives O(k² log n), or O(k log k log n) with FFT multiplication (phase-12 lab-07). Name-drop it; implement only if k ≥ ~10³.
Interview Context
- Codeforces / ICPC: regular — Div 2 D/E staple
- Quant: common (recurrences, n-step Markov transitions)
- FAANG: rare but real — shows up as the “now n = 10^18” follow-up to climbing-stairs / tiling / string-counting DPs
- Google-style “count binary strings of length 10^15 avoiding pattern X”: exactly this
- Standard FAANG loop: low probability, but the cheapest hard topic in this phase
When to Skip This Topic
Skip if any of these are true:
- Scalar binary exponentiation (phase-07 lab-02) isn’t automatic yet
- You can’t write bottom-up Fibonacci DP cold
- You’re on a frontend/mobile interview track
Otherwise don’t skip — this is the highest ROI-per-hour lab in the phase.
Problem Statement
Given a, b, c and n ≤ 10^18, compute
f(n) = a·f(n−1) + b·f(n−2) + c (mod 10^9+7)
with f(0), f(1) given. The constant c makes the recurrence affine: a 2×2 matrix no longer suffices — carry a constant “1” lane, giving a 3×3 matrix.
Constraints
- 0 ≤ n ≤ 10^18
- Coefficients and initial values in [0, 10^9+7)
- Modulus 10^9+7, prime
- Wall-clock: sub-millisecond per query — ~60 squarings of a 3×3
Clarifying Questions
- Answer mod what — and is it prime? (10^9+7, prime; primality only matters if division shows up.)
- Can coefficients be negative? (In general yes — reduce into [0, mod) before building the matrix.)
- One n or many queries? (One here; for many, see follow-ups.)
- Is k fixed and small? (k ≤ ~50 for the k³ method; larger k needs Kitamasa.)
Examples
Fibonacci identity at n = 5 (F: 0, 1, 1, 2, 3, 5, 8):
[[1,1],[1,0]]^5 = [[8,5],[5,3]] = [[F(6),F(5)],[F(5),F(4)]] ✓
Affine 3×3 for f(n) = a·f(n−1) + b·f(n−2) + c:
[f(n) ] [a b c] [f(n−1)]
[f(n−1)] = [1 0 0] × [f(n−2)]
[ 1 ] [0 0 1] [ 1 ]
a=2, b=1, c=3, f(0)=0, f(1)=1: f(2) = 2+0+3 = 5, f(3) = 10+1+3 = 14, f(4) = 28+5+3 = 36.
Brute Force
Iterate the recurrence keeping the last k values. O(n) time. At n = 10^18 and 10^9 steps/sec: ~30 years. Dead.
Brute Force Complexity
- Time: O(n·k)
- Space: O(k)
Optimization Path
- Express one step as state_new = M × state_old (companion matrix; add the “1” lane for +c).
- n steps = M^n × state₀ — the loop becomes a matrix power.
- Compute M^n by repeated squaring: log₂(10^18) ≈ 60 levels.
- Read f(n) out of one row of the result.
- If k grows past ~50: Kitamasa, O(k² log n).
Final Expected Approach
MOD = 10**9 + 7
def mat_mul(A, B, mod=MOD):
n, m, r = len(A), len(B[0]), len(B)
C = [[0] * m for _ in range(n)]
for i in range(n):
Ci = C[i]
for k in range(r):
a = A[i][k]
if a:
Bk = B[k]
for j in range(m):
Ci[j] = (Ci[j] + a * Bk[j]) % mod
return C
def mat_pow(A, e, mod=MOD):
n = len(A)
R = [[int(i == j) for j in range(n)] for i in range(n)]
while e:
if e & 1:
R = mat_mul(R, A, mod)
A = mat_mul(A, A, mod)
e >>= 1
return R
def kth_term(coeffs, initial, n, mod=MOD):
k = len(coeffs)
if n < k:
return initial[n] % mod
M = [[0] * k for _ in range(k)]
M[0] = [c % mod for c in coeffs]
for i in range(1, k):
M[i][i - 1] = 1
P = mat_pow(M, n - k + 1, mod)
state = [initial[k - 1 - j] % mod for j in range(k)]
return sum(P[0][j] * state[j] for j in range(k)) % mod
def affine_term(a, b, c, f0, f1, n, mod=MOD):
if n == 0:
return f0 % mod
M = [[a % mod, b % mod, c % mod], [1, 0, 0], [0, 0, 1]]
P = mat_pow(M, n - 1, mod)
return (P[0][0] * f1 + P[0][1] * f0 + P[0][2]) % mod
def fib(n, mod=MOD):
def pair(n):
if n == 0:
return 0, 1
a, b = pair(n >> 1)
c = a * (2 * b - a) % mod
d = (a * a + b * b) % mod
return (d, (c + d) % mod) if n & 1 else (c, d)
return pair(n)[0]
Data Structures
- k×k list-of-lists matrices; no numpy (int64 overflows at this modulus)
- State vector ordered newest-first: [f(t), f(t−1), …, f(t−k+1)]
Correctness Argument
- One recurrence step is exactly one companion-matrix multiply: the first row computes Σ cᵢ f(t−i+1); the shifted-identity rows slide each older value down one slot.
- Matrix multiplication is associative, so binary exponentiation is valid: M^(2e) = (M^e)², M^(e+1) = M·M^e.
- Induction: s_t = [f(t), …, f(t−k+1)] satisfies s_{t+1} = M·s_t, hence f(n) = (M^(n−k+1)·s_{k−1})[0].
- The [0 0 1] row preserves the constant lane, so +c is added exactly once per step.
Complexity
- Time: O(k³ log n) — k = 3, n = 10^18: 27 × ~120 ≈ 3×10³ scalar mulmods
- Space: O(k²)
- Fast doubling (Fibonacci only): O(log n), ~3 mulmods per level
Implementation Requirements
- Reduce mod inside mat_mul, per accumulation — mandatory for overflow in C++/Java, mandatory for speed in Python
- Seed exponentiation with the identity matrix; e = 0 must return I
- Answer n < k directly from initial values — no matrix work
- Exponent is n − k + 1, not n; verify at n = k before trusting anything
- Reduce negative coefficients into [0, mod) before building M
Tests
- fib(n) == kth_term([1,1], [0,1], n) == iterative Fibonacci for n = 0..30
- Affine a=2, b=1, c=3, f0=0, f1=1: f(2)=5, f(3)=14, f(4)=36 (hand values)
- Edge n = 0, 1, k−1 (pure initial-value reads)
- k = 1 degenerate case: f(n) = c₁^n · f(0) — matches pow(c1, n, mod) route
- fib(10^18) returns in milliseconds; matrix and fast-doubling versions agree
- Randomized k = 4 recurrence vs O(n) iteration for n ≤ 10^4
Follow-up Questions
- For Fibonacci specifically, can you beat the 2×2 power? → Fast doubling: F(2k) = F(k)·(2F(k+1) − F(k)), F(2k+1) = F(k)² + F(k+1)². Same O(log n), but ~3 mulmods per level vs ~8+ for naive 2×2 squaring — roughly an 8× constant-factor win over unoptimized matrix code.
- Count walks of length 10^12 between two nodes in a 50-vertex graph. → (A^k)[i][j] by matrix power mod p: 50³ × ~80 multiplications. Same machinery counts length-n strings avoiding a pattern: build the KMP/Aho-Corasick automaton, power its transition matrix.
- Why must the mod live inside mat_mul? → In C++/Java, one product a·b already fills 64 bits and summing k of them overflows without intermediate reduction — wrong answers, not exceptions. Python can’t overflow; it just degrades into multi-word big-int arithmetic and slows several-fold.
- k = 10^5 — now what? → k³ log n ≈ 10^15·60: dead. Kitamasa computes x^n mod the characteristic polynomial in O(k² log n), or O(k log k log n) with NTT (phase-12 lab-07). If only the sequence is given, Berlekamp-Massey first recovers the minimal recurrence from 2k terms.
- All of f(l..r), not a single n? → Matrix-power to f(l), then iterate linearly: O(k³ log l + k·(r−l)). For many scattered queries, precompute the ~60 squarings M^(2^i) once and answer each query with ≤ 60 multiplications, or sort queries and reuse prefix products.
- Does the recurrence have to be homogeneous? → No. Constants get the “1” lane; polynomial terms like +n or +n² get extra lanes updated Pascal-style ((n+1)² = n² + 2n + 1). Anything linear in the augmented state folds in.
Product Extension
- Markov-chain systems projected far forward (population models, failure states)
- State machines in rate limiters: probability of reaching a state within n events
- Graphics pipelines: composing affine transforms is exactly 3×3/4×4 matrix products
- Financial recurrences (annuity-style balance updates) at huge horizons
Language/Runtime Follow-ups
- Python: big ints make it correct by default; keep list-of-lists — numpy int64 silently overflows at 10^9+7 (object dtype works but is slower than plain lists)
- C++: accumulate in unsigned 64-bit with periodic reduction, or __int128; hand-unroll k = 2, 3
- Java: long with reduction inside the loop; Math.floorMod for negatives
- Rust: u128 intermediates; const-generic fixed-size matrices optimize well
Common Bugs
- Off-by-one exponent: M^n vs M^(n−1) vs M^(n−k+1) depending on the seeded state — verify at n = k first.
- State vector reversed: companion matrix assumes newest-first order; feeding [f(0), …, f(k−1)] gives plausible garbage.
- Mod applied too late: overflow in C++/Java; silent multi-fold slowdown in Python.
- Missing constant lane: modeling +c in a 2×2 — the error grows by c per step and small tests may still pass.
- Aliased squaring: mat_mul writing into an input matrix corrupts A = A·A.
- n < k unhandled: matrix path assumes at least one transition; n = 0 must short-circuit.
Debugging Strategy
- Diff against the O(n) iterative recurrence for n = 0..1000 — catches every off-by-one
- Check M^0 = I and M^1 = M explicitly
- Verify [[1,1],[1,0]]^5 = [[8,5],[5,3]] by hand
- Print the state vector after 1, 2, 3 matrix applications and match the direct recurrence
Mastery Criteria
- Derive the companion matrix for any dictated recurrence in ≤ 3 min
- Implement mat_pow + kth_term bug-free in ≤ 25 min
- Add the affine “+c” lane without hesitation
- Write Fibonacci fast doubling from memory in ≤ 10 min
- Spot “linear DP + astronomical n” and say “matrix power” within 2 min of reading a problem
- Name the large-k escape hatch (Kitamasa / Berlekamp-Massey) with its complexity
Lab 10 — Exact TSP: Held-Karp & Branch-and-Bound
Goal
Solve the Traveling Salesman Problem exactly: Held-Karp bitmask DP in O(n² · 2^n) for n ≤ 20, and understand the branch-and-bound ladder that extends exact solving to n ≈ 30–40 and (with LP machinery) to tens of thousands of cities.
Background
TSP: given pairwise distances between n cities, find the cheapest tour visiting each exactly once and returning to start. NP-hard; the naive answer is (n−1)! permutations.
Held & Karp (1962), independently Bellman: DP over subsets.
dp[S][j] = min cost of a path that starts at city 0, visits exactly
the cities in set S, and ends at city j ∈ S
dp[{0,j}][j] = dist[0][j]
dp[S][j] = min over k ∈ S∖{j} of dp[S∖{j}][k] + dist[k][j]
answer = min over j of dp[full][j] + dist[j][0]
The win: 20! ≈ 2.4 × 10^18 → 20² × 2^20 ≈ 4.2 × 10^8. Exponential, but a tractable exponential — and it’s the fastest known exact algorithm with a worst-case guarantee.
Branch-and-bound: DFS over partial tours, pruning any branch whose cost + an admissible lower bound on the remainder ≥ best complete tour so far. Bound ladder, weak → strong: sum of cheapest edges per unvisited city → MST of unvisited cities → Held-Karp 1-tree bound with Lagrangian penalties. Concorde (LP relaxation + cutting planes + B&B) has solved instances of 85,900 cities — NP-hardness is about worst cases, not typical ones.
Interview Context
- Standard FAANG: yes, disguised — Find the Shortest Superstring (LC 943), Minimum Cost to Visit Every Node, Parallel Courses II (LC 1494); “N ≤ 20” in any problem statement is a bitmask-DP siren
- Codeforces / ICPC: bitmask-DP-over-subsets is a permanent fixture
- OR / logistics / routing companies (Amazon transportation, Uber, FedEx-adjacent): the real thing, plus heuristics
- Quant: occasional, as assignment/sequencing subproblems
When to Skip This Topic
Skip if any of these are true:
- Phase 3 Lab 08 / Phase 5 Lab 10 (bitmask DP) aren’t solid — this is their capstone, not an alternative entry point
- You only need approximate routing knowledge — read the Christofides bullet in Follow-ups and move on
The Held-Karp half is genuinely interview-relevant; the branch-and-bound half is domain knowledge for solver/OR roles.
Problem Statement
Minimum Cost Hamiltonian Cycle.
Given an n × n matrix dist (dist[i][j] ≥ 0, not necessarily symmetric), return the minimum cost of a tour starting and ending at city 0 that visits every city exactly once — and the tour itself. Interviewers nearly always follow up asking for the tour; build reconstruction in from the start.
Constraints
- n ≤ 20 (DP); discussion expected for n = 30 (B&B) and n = 10^4 (heuristics)
- Weights ≤ 10^6, so path sums fit comfortably in int64
- Time: n = 20 in a few seconds in Python, < 1 sec compiled
Clarifying Questions
- Return to the start, or end anywhere? (Cycle here; a path variant just drops the closing edge — changes the final min.)
- Symmetric distances? Triangle inequality? (Neither needed for DP; both needed for Christofides.)
- Is the graph complete? (If not, missing edges = ∞; detect unreachable answer.)
- One optimal tour or all? (One.)
- n’s actual bound? (The entire algorithm choice hangs on it: ≤ 20 DP, ≤ ~35 B&B, beyond → heuristics.)
Examples
4 cities:
dist = [[0, 10, 15, 20],
[10, 0, 35, 25],
[15, 35, 0, 30],
[20, 25, 30, 0]]
DP table (masks containing city 0; showing reachable (mask, last) states):
dp[{0,1}][1] = 10 dp[{0,2}][2] = 15 dp[{0,3}][3] = 20
dp[{0,1,2}][2] = 45 dp[{0,1,2}][1] = 50
dp[{0,1,3}][3] = 35 dp[{0,1,3}][1] = 45
dp[{0,2,3}][3] = 45 dp[{0,2,3}][2] = 50
dp[full][1] = 70 dp[full][2] = 65 dp[full][3] = 75
answer = min(70+10, 65+15, 75+20) = 80
tour: 0 → 1 → 3 → 2 → 0
Brute Force
Fix city 0 as start; try all (n−1)! orderings of the rest; keep the cheapest.
Brute Force Complexity
- Time: O(n!) — n = 13 is already ~5 × 10^9 tour evaluations
- Space: O(n)
Optimization Path
- Notice the repeated work: many permutations share (set of visited cities, current city) — that pair is the entire relevant state. → dp[mask][last].
- Iterate masks in increasing order (every subset precedes its supersets numerically) — no recursion needed.
- Only masks containing bit 0; halves the table.
- Parent table
par[mask][last] = kfor tour reconstruction. - For n > 20: switch paradigms to branch-and-bound — DFS partial tours, prune with best-so-far + MST-based lower bound on unvisited remainder.
Final Expected Approach
from math import inf
def held_karp(dist):
n = len(dist)
size = 1 << n
dp = [[inf] * n for _ in range(size)]
par = [[-1] * n for _ in range(size)]
dp[1][0] = 0
for mask in range(1, size):
if not mask & 1:
continue
for last in range(n):
if not mask >> last & 1 or dp[mask][last] == inf:
continue
base = dp[mask][last]
for nxt in range(n):
if mask >> nxt & 1:
continue
nmask = mask | 1 << nxt
cand = base + dist[last][nxt]
if cand < dp[nmask][nxt]:
dp[nmask][nxt] = cand
par[nmask][nxt] = last
full = size - 1
best, best_end = inf, -1
for last in range(1, n):
cand = dp[full][last] + dist[last][0]
if cand < best:
best, best_end = cand, last
tour, mask, cur = [], full, best_end
while cur != -1:
tour.append(cur)
mask, cur = mask ^ (1 << cur), par[mask][cur]
tour.reverse()
return best, tour + [0]
Forward-relaxation formulation (push from mask to mask | bit) shown — it visits each transition once and skips unreachable states naturally. The pull formulation from Background is equivalent.
B&B sketch for the discussion round: DFS over partial tours; bound = cost so far + cheapest edge into each unvisited city (admissible because every unvisited city must be entered exactly once); upgrade to MST of {unvisited ∪ current ∪ start} for far stronger pruning; seed best-so-far with a greedy nearest-neighbor tour so pruning bites from the first branch.
Data Structures
- dp: 2^n × n array of int64 (n = 20 → ~84 MB as Python lists; use array/numpy if memory-pressed)
- par: same shape, for reconstruction
- Bitmask ints as visited sets
- B&B: recursion stack + current best (and a DSU or heap if building MST bounds per node)
Correctness Argument
- State sufficiency: the cheapest completion of a partial tour depends only on (visited set, current city) — not the order visited. So collapsing permutations into dp[mask][last] loses nothing.
- Induction over |mask|: assuming dp is exact for all masks of size k, every size-k+1 state is reached by extending some exact size-k state with one edge, and the min over predecessors is exact. Base dp[{0}][0] = 0.
- Iterating masks in increasing numeric order is a valid topological order: mask ⊂ nmask ⟹ mask < nmask.
- B&B correctness: it only discards branches whose admissible lower bound already exceeds a known achievable tour — no optimal solution can be pruned.
Complexity
- Held-Karp time: O(n² · 2^n); n = 20 → ~4.2 × 10^8 transitions (seconds in Python, fast in C)
- Space: O(n · 2^n) — 20 M entries at n = 20; this, not time, is often the binding constraint
- B&B: worst case still exponential; strong bounds make n = 30–40 practical
- Approximation for large n: Christofides O(n³), guarantee 1.5× on metric instances
Implementation Requirements
- Skip masks without bit 0 and unreachable states (
dp == inf) — the inner-loop constant matters at 4 × 10^8 ops - int64 discipline in compiled languages: 20 × 10^6 path sums exceed int32
- Parent table maintained in the same relaxation that improves dp — reconstruction from a separate pass is a common source of mismatched tours
- Asymmetric input must be handled (no dist[i][j] == dist[j][i] assumption anywhere)
- Path variant (Shortest Superstring): answer = min dp[full][j] with no closing edge — don’t hardcode the cycle close
Tests
- The 4-city example → 80, tour [0, 1, 3, 2, 0] (or its reverse for symmetric input)
- n = 2 → 2 × dist[0][1]; n = 1 → 0
- Asymmetric instance where forward and reverse tours differ — verifies no symmetry assumption
- Random n = 10 vs brute-force permutation oracle: costs equal, tour validates (each city once, cost re-adds correctly)
- All-equal weights → any permutation optimal; assert cost = n × w
- Missing edges (∞): unreachable instance reports ∞/None rather than garbage
- n = 20 performance run
Follow-up Questions
- How do you return the tour, not just the cost? → Parent pointers per state (shown), O(n·2^n) extra memory; or re-derive each step by checking which predecessor achieves dp equality, O(n²) time per step with zero extra memory. Know both — interviewers ask for the memory-free version when dp is already at the limit.
- Can you halve the memory? → Yes: fix city 1’s side of the tour orientation for symmetric TSP, or store only masks containing bit 0 with index remapping (mask >> 1). Rolling the dp by mask popcount layers also works: keep only layers k and k+1 if you don’t need reconstruction.
- Shortest Superstring (LC 943) connection? → Overlap graph: dist[i][j] = len(word j) − overlap(i, j); answer is a Hamiltonian path, not cycle — drop the closing edge and take min over all (start, end).
- Why is the MST a valid lower bound for B&B? → Removing one edge from any tour of the unvisited remainder leaves a spanning tree of it; MST ≤ that tree ≤ tour remainder. The stronger Held-Karp 1-tree bound adds back one vertex with its two cheapest edges and sharpens with Lagrangian node penalties — typically within 1–2% of optimal.
- What when n = 10^5? → Exact is off the table without Concorde-class LP machinery. Practical stack: nearest-neighbor or greedy construction, then 2-opt/3-opt local search, then Lin-Kernighan (LKH routinely lands within 1% on huge instances). Christofides gives the 1.5 guarantee on metric instances but LKH beats it in practice.
- Where else does the dp[subset][last] shape appear? → Any “sequence over a small set with pairwise interaction cost”: job scheduling with setup times, LC 847 (shortest path visiting all nodes — BFS variant), LC 1434, assignment problems, and Kociemba-style solvers use the sibling trick of dp over coordinates (lab-02).
Product Extension
- Last-mile delivery routing (per-vehicle stop sequencing after cluster assignment)
- PCB drill-path and pick-and-place head sequencing (classic Concorde application domains)
- Warehouse picker pathing; AGV tour planning
- Genome assembly’s superstring heart (LC 943 is a toy of it)
- Chip design: TSP-shaped subproblems in routing and scan-chain ordering
Language/Runtime Follow-ups
- Python: the mask loops are pure-Python slow at n = 20 (~minutes naive); mitigate with the inf-skip, or numpy per-mask vectorization, or accept n ≤ 17 in-interview and say so — stating the constant-factor reality is a signal, not a failure.
- C++: flat
vector<int64_t>dp, index = mask*n + last; n = 20 in well under a second. - Java:
long[1<<n][n]; beware autoboxing if reaching for collections — don’t. - Go: straightforward; slices of int64, no surprises.
Common Bugs
- Start city not fixed: iterating tours from every start multiplies work by n and breaks reconstruction; fixing city 0 is WLOG for cycles.
- Closing edge forgotten (or added in the path variant): answer off by one edge — the single most common bug in this problem.
- Masks without bit 0 processed: wrong states get relaxed and reconstruction walks into them.
- Parent recorded outside the improving branch: tour disagrees with cost; always update
parin the sameif cand < dpbody. - int32 overflow in compiled languages at max weights.
- Reconstruction mask bookkeeping: must remove
cur’s bit before looking up the parent’s mask entry —par[mask][cur]is indexed by the mask that still contains cur; mixing the two orders yields plausible-looking wrong tours.
Debugging Strategy
- Brute-force permutation oracle for n ≤ 10; assert equal costs on 100 random matrices
- Validate returned tours independently: n+1 nodes, starts/ends at 0, each city once, edges re-sum to the reported cost
- Print dp layer by popcount for the 4-city example and diff against the hand table above
- For B&B: assert bound ≤ true completion cost on random partial tours (admissibility test); count pruned vs explored nodes
- Asymmetric test matrices catch silent transposition bugs (
dist[k][j]vsdist[j][k])
Mastery Criteria
- Write Held-Karp with reconstruction from memory in ≤ 30 min
- State the state-sufficiency argument (why (mask, last) is enough) in ≤ 2 min
- Recite the n → algorithm ladder: ≤ 20 DP, ≤ ~35 B&B, large → LKH/Christofides, and why
- Recognize disguised Hamiltonian-path DPs (LC 943/847/1494) on sight
- Explain the MST bound’s admissibility and name the 1-tree improvement
- Estimate memory for dp at a given n before writing code