Big O notation describes the upper bound on how an algorithm's runtime or memory usage grows relative to input size n, ignoring constant factors and lower-order terms. It matters because it lets you compare algorithms independent of hardware or implementation details — O(n²) will always lose to O(n log n) for large n regardless of the CPU speed. Engineers use it to make architectural decisions before writing a single line of code, avoiding solutions that are fast on toy inputs but catastrophically slow in production.
python
# O(n²) — nested loops, unacceptable for n=1,000,000
for i in range(n):
for j in range(n):
process(i, j)
O(1) — constant time regardless of input size; e.g. array index access `arr[i]`. O(log n) — halves the search space each step; e.g. binary search. O(n) — one pass through all elements; e.g. finding the max in an unsorted array. O(n log n) — divide-and-conquer with a linear merge; e.g. merge sort. O(n²) — all pairs; e.g. bubble sort or checking every pair in an array. O(2^n) — exponential explosion; e.g. generating all subsets of a set or naive recursive Fibonacci. The practical cutoff is roughly: n≤10 any complexity is fine; n≤10⁴ O(n²) works; n≤10⁶ needs O(n log n) or better; n≤10⁸ requires O(n).
python
# O(log n): binary search
def bsearch(arr, t):
lo, hi = 0, len(arr)-1
while lo <= hi:
mid = (lo+hi)//2
if arr[mid]==t: return mid
elif arr[mid]<t: lo=mid+1
else: hi=mid-1
return -1
An array is a contiguous block of memory where each element is stored at a fixed offset from the base address, enabling O(1) random access via index arithmetic (`base + i * element_size`). Searching an unsorted array is O(n) (linear scan); a sorted array can be searched in O(log n) with binary search. Inserting or deleting at an arbitrary index is O(n) because all subsequent elements must shift. Appending to the end of a static array is O(1) if capacity allows; otherwise the array must be reallocated.
A dynamic array wraps a fixed-size array and automatically grows when capacity is exhausted. When the array is full it allocates a new buffer — typically 2× the current capacity — copies all existing elements, then frees the old buffer. This doubling strategy ensures that n total pushes cost O(n) total work: the series of copies (1 + 2 + 4 + … + n) sums to O(n), giving O(1) amortized cost per append. Python's `list` uses a growth factor of ~1.125 for small sizes and ~1.5–2× for larger ones.
A linked list is a sequence of nodes where each node stores a value and a pointer to the next node; nodes are not necessarily contiguous in memory. A singly linked list has one pointer per node (`next`), enabling O(n) traversal forward only. A doubly linked list adds a `prev` pointer, enabling O(1) deletion of a node given a direct reference (no need to find the predecessor) and bidirectional traversal, at the cost of 2× pointer overhead. The head (and optionally tail) pointer is stored separately.
Arrays offer O(1) random access and excellent cache locality (contiguous memory means prefetcher hits), but insertions/deletions in the middle are O(n) due to shifting, and resizing requires copying. Linked lists support O(1) insert/delete at a known node and have no wasted capacity, but access is O(n) and pointer chasing is cache-unfriendly (each node is a separate heap allocation). In practice, arrays outperform linked lists for most workloads due to cache effects; linked lists shine when you have a pointer to the target node and need frequent mid-list mutations (e.g. LRU eviction in a cache).
Arrays offer O(1) random access and excellent cache locality (contiguous memory means prefetcher hits), but insertions/deletions in the middle are O(n) due to shifting, and resizing requires copying. Linked lists support O(1) insert/delete at a known node and have no wasted capacity, but access is O(n) and pointer chasing is cache-unfriendly (each node is a separate heap allocation). In practice, arrays outperform linked lists for most workloads due to cache effects; linked lists shine when you have a pointer to the target node and need frequent mid-list mutations (e.g. LRU eviction in a cache).
A stack is a Last-In-First-Out (LIFO) abstract data type. `push(x)` adds x to the top; `pop()` removes and returns the top element; `peek()` returns the top without removing it. All three are O(1) when implemented over a dynamic array (using the array's end as the top) or a singly linked list (using the head as the top). Classic uses: function call stack, undo history, expression parsing, and DFS iterative traversal.
python
stack = []
stack.append(1) # push
stack.append(2)
top = stack[-1] # peek
stack.pop() # pop -> 2
A stack is a Last-In-First-Out (LIFO) abstract data type. `push(x)` adds x to the top; `pop()` removes and returns the top element; `peek()` returns the top without removing it. All three are O(1) when implemented over a dynamic array (using the array's end as the top) or a singly linked list (using the head as the top). Classic uses: function call stack, undo history, expression parsing, and DFS iterative traversal.
```python
stack = []
stack.append(1) # push
stack.append(2)
top = stack[-1] # peek
stack.pop() # pop -> 2
```
A queue is a First-In-First-Out (FIFO) abstract data type. `enqueue(x)` appends to the back; `dequeue()` removes from the front. Both are O(1) with a doubly linked list or a circular buffer. Implementing with a plain dynamic array makes dequeue O(n) due to shifting — avoid it. Python's `collections.deque` gives O(1) on both ends. Classic uses: BFS, task scheduling, print spooling, and producer-consumer pipelines.
A queue is a First-In-First-Out (FIFO) abstract data type. `enqueue(x)` appends to the back; `dequeue()` removes from the front. Both are O(1) with a doubly linked list or a circular buffer. Implementing with a plain dynamic array makes dequeue O(n) due to shifting — avoid it. Python's `collections.deque` gives O(1) on both ends. Classic uses: BFS, task scheduling, print spooling, and producer-consumer pipelines.
```python
from collections import deque
q = deque()
q.append(1) # enqueue
q.append(2)
q.popleft() # dequeue -> 1
```
A deque (pronounced "deck") supports O(1) insert and delete at both ends. It generalises both stack and queue. Typical implementations use a doubly linked list or a circular buffer of fixed-size blocks. Python's `collections.deque` uses a block-linked list giving O(1) amortized on all four operations (`appendleft`, `append`, `popleft`, `pop`) but O(n) random access. Deques are used for sliding-window problems (monotonic deque for window maximum), work-stealing schedulers, and palindrome checking.
A deque (pronounced "deck") supports O(1) insert and delete at both ends. It generalises both stack and queue. Typical implementations use a doubly linked list or a circular buffer of fixed-size blocks. Python's `collections.deque` uses a block-linked list giving O(1) amortized on all four operations (`appendleft`, `append`, `popleft`, `pop`) but O(n) random access. Deques are used for sliding-window problems (monotonic deque for window maximum), work-stealing schedulers, and palindrome checking.
A hash table maps keys to values via a hash function that converts a key to a bucket index. Given bucket index h = hash(key) % capacity, lookup, insert, and delete are O(1) average because each operation touches only the element(s) in that one bucket. The guarantee relies on a good hash function distributing keys uniformly and a low load factor (α = n/capacity < ~0.7). Worst case degrades to O(n) when all keys hash to the same bucket. Python's `dict` and Java's `HashMap` are hash table implementations.
A hash table maps keys to values via a hash function that converts a key to a bucket index. Given bucket index h = hash(key) % capacity, lookup, insert, and delete are O(1) average because each operation touches only the element(s) in that one bucket. The guarantee relies on a good hash function distributing keys uniformly and a low load factor (α = n/capacity < ~0.7). Worst case degrades to O(n) when all keys hash to the same bucket. Python's `dict` and Java's `HashMap` are hash table implementations.
A hash function maps keys of arbitrary type/size to a fixed-range integer. A good hash function is (1) deterministic — same key always yields same hash; (2) uniform — output bits are evenly distributed, minimising collisions; (3) fast to compute — typically O(key length); (4) avalanche effect — a single bit change in the input flips ~50% of output bits (critical for security-grade hashes like SHA-256). For hash tables, non-cryptographic hashes like MurmurHash3 or xxHash are preferred for speed. For string keys the FNV-1a or djb2 are popular simple choices.
python
# djb2 for strings
def djb2(s):
h = 5381
for c in s:
h = ((h << 5) + h) + ord(c) # h*33 + c
return h & 0xFFFFFFFF
A hash function maps keys of arbitrary type/size to a fixed-range integer. A good hash function is (1) deterministic — same key always yields same hash; (2) uniform — output bits are evenly distributed, minimising collisions; (3) fast to compute — typically O(key length); (4) avalanche effect — a single bit change in the input flips ~50% of output bits (critical for security-grade hashes like SHA-256). For hash tables, non-cryptographic hashes like MurmurHash3 or xxHash are preferred for speed. For string keys the FNV-1a or djb2 are popular simple choices.
```python
# djb2 for strings
def djb2(s):
h = 5381
for c in s:
h = ((h << 5) + h) + ord(c) # h*33 + c
return h & 0xFFFFFFFF
```
A collision occurs when two distinct keys hash to the same bucket index. Separate chaining stores each bucket as a linked list (or small array); colliding entries are appended. Lookup is O(1 + α) average where α is the load factor. Open addressing stores all entries in the array itself; on collision, it probes for the next empty slot. Linear probing checks indices h, h+1, h+2, … — simple and cache-friendly but suffers from primary clustering (long runs of filled slots). Separate chaining degrades more gracefully at high load factors (α > 0.7) but has pointer overhead per node.
A collision occurs when two distinct keys hash to the same bucket index. Separate chaining stores each bucket as a linked list (or small array); colliding entries are appended. Lookup is O(1 + α) average where α is the load factor. Open addressing stores all entries in the array itself; on collision, it probes for the next empty slot. Linear probing checks indices h, h+1, h+2, … — simple and cache-friendly but suffers from primary clustering (long runs of filled slots). Separate chaining degrades more gracefully at high load factors (α > 0.7) but has pointer overhead per node.
Load factor α = n / capacity where n is the number of stored keys. As α rises, collision probability increases and average lookup time grows. Standard practice is to resize (rehash) when α exceeds a threshold — typically 0.75 for chaining (Java's default) and 0.5–0.6 for open addressing to avoid severe clustering. Resizing allocates a new array (usually 2× size), recomputes all hashes, and reinserts every entry — an O(n) operation, but amortised O(1) per insert. Shrinking on delete is optional but prevents wasted memory (resize down when α < 0.25).
Load factor α = n / capacity where n is the number of stored keys. As α rises, collision probability increases and average lookup time grows. Standard practice is to resize (rehash) when α exceeds a threshold — typically 0.75 for chaining (Java's default) and 0.5–0.6 for open addressing to avoid severe clustering. Resizing allocates a new array (usually 2× size), recomputes all hashes, and reinserts every entry — an O(n) operation, but amortised O(1) per insert. Shrinking on delete is optional but prevents wasted memory (resize down when α < 0.25).
A tree is an acyclic connected graph. The root is the unique node with no parent. Every other node has exactly one parent; nodes it points to are its children. A leaf has no children. Depth of a node = number of edges from root to that node. Height of a node = length of the longest path from that node to a leaf; height of the tree = height of the root. A tree with n nodes has exactly n-1 edges. Trees model hierarchies: file systems, DOM, corporate org charts.
A tree is an acyclic connected graph. The root is the unique node with no parent. Every other node has exactly one parent; nodes it points to are its children. A leaf has no children. Depth of a node = number of edges from root to that node. Height of a node = length of the longest path from that node to a leaf; height of the tree = height of the root. A tree with n nodes has exactly n-1 edges. Trees model hierarchies: file systems, DOM, corporate org charts.
A binary tree is a tree where each node has at most two children, typically called left and right. There are no ordering constraints on a plain binary tree. A BST adds the invariant: for every node N, all values in N's left subtree are strictly less than N's value, and all values in N's right subtree are strictly greater. This invariant enables O(h) search, insert, and delete where h is the tree height — O(log n) for balanced trees, O(n) for degenerate (linked-list shaped) trees.
A binary tree is a tree where each node has at most two children, typically called left and right. There are no ordering constraints on a plain binary tree. A BST adds the invariant: for every node N, all values in N's left subtree are strictly less than N's value, and all values in N's right subtree are strictly greater. This invariant enables O(h) search, insert, and delete where h is the tree height — O(log n) for balanced trees, O(n) for degenerate (linked-list shaped) trees.
All three operations traverse one root-to-leaf path, taking O(h) time where h is the height. For a balanced BST with n nodes h = O(log n), giving O(log n) average. For a sorted insertion sequence the BST degenerates into a linked list with h = n-1, giving O(n) worst case for all operations. Delete is the trickiest: removing a node with two children requires finding its in-order successor (leftmost node of right subtree), swapping values, then deleting the successor (which has at most one child).
All three operations traverse one root-to-leaf path, taking O(h) time where h is the height. For a balanced BST with n nodes h = O(log n), giving O(log n) average. For a sorted insertion sequence the BST degenerates into a linked list with h = n-1, giving O(n) worst case for all operations. Delete is the trickiest: removing a node with two children requires finding its in-order successor (leftmost node of right subtree), swapping values, then deleting the successor (which has at most one child).
A balanced BST constrains height to O(log n) so that search, insert, and delete are guaranteed O(log n) rather than O(n). Without balancing, sequential insertions create a right-skewed chain indistinguishable from a linked list. Self-balancing BSTs like AVL trees and red-black trees perform rotations after insertions/deletions to restore balance invariants automatically. Practical implementations: Java's `TreeMap` and `TreeSet` use red-black trees; Python's `sortedcontainers.SortedList` uses a B-tree variant.
A balanced BST constrains height to O(log n) so that search, insert, and delete are guaranteed O(log n) rather than O(n). Without balancing, sequential insertions create a right-skewed chain indistinguishable from a linked list. Self-balancing BSTs like AVL trees and red-black trees perform rotations after insertions/deletions to restore balance invariants automatically. Practical implementations: Java's `TreeMap` and `TreeSet` use red-black trees; Python's `sortedcontainers.SortedList` uses a B-tree variant.
A heap is a complete binary tree stored in an array satisfying the heap property. Max-heap: every node's value ≥ its children's values — the root holds the maximum. Min-heap: every node's value ≤ its children's values — the root holds the minimum. The array representation uses 1-based indices: children of node i are at 2i and 2i+1; parent of i is at ⌊i/2⌋. Heaps are the canonical implementation of priority queues. Python's `heapq` is a min-heap; negate values for max-heap behaviour.
A heap is a complete binary tree stored in an array satisfying the heap property. Max-heap: every node's value ≥ its children's values — the root holds the maximum. Min-heap: every node's value ≤ its children's values — the root holds the minimum. The array representation uses 1-based indices: children of node i are at 2i and 2i+1; parent of i is at ⌊i/2⌋. Heaps are the canonical implementation of priority queues. Python's `heapq` is a min-heap; negate values for max-heap behaviour.
Insert (push): append the new element at the end of the array (O(1)), then bubble it up (heapify-up) by repeatedly swapping with its parent while the heap property is violated. The element travels at most h = O(log n) levels, so insert is O(log n). Extract-min (pop): swap the root with the last element (O(1)), shrink the array by one, then push the new root down (heapify-down) by swapping with the smaller child until the heap property holds — also O(log n). Peek (min without removing) is always O(1) since the root is the minimum.
Insert (push): append the new element at the end of the array (O(1)), then bubble it up (heapify-up) by repeatedly swapping with its parent while the heap property is violated. The element travels at most h = O(log n) levels, so insert is O(log n). Extract-min (pop): swap the root with the last element (O(1)), shrink the array by one, then push the new root down (heapify-down) by swapping with the smaller child until the heap property holds — also O(log n). Peek (min without removing) is always O(1) since the root is the minimum.
```python
import heapq
heap = []
heapq.heappush(heap, 5)
heapq.heappush(heap, 1)
print(heapq.heappop(heap)) # 1
```
A graph G = (V, E) consists of a set of vertices V and edges E connecting pairs of vertices. In an undirected graph edges are bidirectional — edge (u,v) implies (v,u). In a directed graph (digraph) edges have direction: (u→v) does not imply (v→u). A weighted graph assigns a numeric weight to each edge; unweighted graphs treat all edges equally (weight = 1). Combinations: directed weighted (road network with tolls), undirected unweighted (friendship network), directed unweighted (web links).
A graph G = (V, E) consists of a set of vertices V and edges E connecting pairs of vertices. In an undirected graph edges are bidirectional — edge (u,v) implies (v,u). In a directed graph (digraph) edges have direction: (u→v) does not imply (v→u). A weighted graph assigns a numeric weight to each edge; unweighted graphs treat all edges equally (weight = 1). Combinations: directed weighted (road network with tolls), undirected unweighted (friendship network), directed unweighted (web links).
An adjacency matrix uses a V×V boolean (or weight) matrix where matrix[u][v] = 1 if edge (u,v) exists. Space is O(V²), edge lookup is O(1), but iterating all neighbours of a node takes O(V). An adjacency list stores, for each vertex, a list of its neighbours. Space is O(V + E), neighbour iteration is O(degree(v)), and edge lookup is O(degree(v)). Use an adjacency matrix for dense graphs (E ≈ V²) or when you need O(1) edge existence checks. Use adjacency lists for sparse graphs (most real-world graphs), which dominate in practice.
An adjacency matrix uses a V×V boolean (or weight) matrix where matrix[u][v] = 1 if edge (u,v) exists. Space is O(V²), edge lookup is O(1), but iterating all neighbours of a node takes O(V). An adjacency list stores, for each vertex, a list of its neighbours. Space is O(V + E), neighbour iteration is O(degree(v)), and edge lookup is O(degree(v)). Use an adjacency matrix for dense graphs (E ≈ V²) or when you need O(1) edge existence checks. Use adjacency lists for sparse graphs (most real-world graphs), which dominate in practice.
BFS explores a graph level by level, visiting all neighbours of a node before moving to their neighbours. It uses a queue, starting from a source vertex, marking nodes visited on enqueue to avoid revisits. Time complexity O(V + E), space O(V) for the visited set and queue. BFS finds the shortest path (in terms of number of edges) between source and any reachable vertex in an unweighted graph. It also detects connected components and can check bipartiteness.
python
from collections import deque
def bfs(graph, src):
q, visited = deque([src]), {src}
while q:
node = q.popleft()
for nb in graph[node]:
if nb not in visited:
visited.add(nb); q.append(nb)
BFS explores a graph level by level, visiting all neighbours of a node before moving to their neighbours. It uses a queue, starting from a source vertex, marking nodes visited on enqueue to avoid revisits. Time complexity O(V + E), space O(V) for the visited set and queue. BFS finds the shortest path (in terms of number of edges) between source and any reachable vertex in an unweighted graph. It also detects connected components and can check bipartiteness.
```python
from collections import deque
def bfs(graph, src):
q, visited = deque([src]), {src}
while q:
node = q.popleft()
for nb in graph[node]:
if nb not in visited:
visited.add(nb); q.append(nb)
```
DFS explores as far as possible along each branch before backtracking. Recursively: visit a node, mark it, then recurse on each unvisited neighbour. Iteratively: replace the call stack with an explicit stack. Time O(V + E), space O(V) for the recursion stack. DFS is the basis for topological sort, cycle detection, strongly connected components (Kosaraju/Tarjan), and solving mazes. DFS does not guarantee shortest paths in unweighted graphs.
python
def dfs(graph, node, visited=None):
if visited is None: visited = set()
visited.add(node)
for nb in graph[node]:
if nb not in visited:
dfs(graph, nb, visited)
return visited
DFS explores as far as possible along each branch before backtracking. Recursively: visit a node, mark it, then recurse on each unvisited neighbour. Iteratively: replace the call stack with an explicit stack. Time O(V + E), space O(V) for the recursion stack. DFS is the basis for topological sort, cycle detection, strongly connected components (Kosaraju/Tarjan), and solving mazes. DFS does not guarantee shortest paths in unweighted graphs.
```python
def dfs(graph, node, visited=None):
if visited is None: visited = set()
visited.add(node)
for nb in graph[node]:
if nb not in visited:
dfs(graph, nb, visited)
return visited
```
Recursion is when a function calls itself to solve a smaller subproblem of the same shape. The base case is a condition where the function returns immediately without a recursive call, preventing infinite loops. The recursive case breaks the problem down and calls itself with a strictly smaller input, converging toward the base case. Every recursive algorithm has an equivalent iterative version (using an explicit stack). Recursive solutions are often cleaner for tree/graph traversal, divide-and-conquer, and backtracking.
python
def factorial(n):
if n <= 1: return 1 # base case
return n * factorial(n-1) # recursive case
Recursion is when a function calls itself to solve a smaller subproblem of the same shape. The base case is a condition where the function returns immediately without a recursive call, preventing infinite loops. The recursive case breaks the problem down and calls itself with a strictly smaller input, converging toward the base case. Every recursive algorithm has an equivalent iterative version (using an explicit stack). Recursive solutions are often cleaner for tree/graph traversal, divide-and-conquer, and backtracking.
```python
def factorial(n):
if n <= 1: return 1 # base case
return n * factorial(n-1) # recursive case
```
A trie is a tree where each path from root to a node represents a prefix of stored strings. Each node has up to 26 children (for lowercase English) and an `is_end` flag. Insert and search are O(L) where L is the string length — independent of the number of stored strings. Tries excel at prefix queries (autocomplete), are used in spell checkers and IP routing tables (binary tries over bit prefixes). Space can be O(ALPHABET_SIZE × L × N) which is worse than a hash table for large sparse key sets; compressed tries (Patricia/radix trees) mitigate this.
A trie is a tree where each path from root to a node represents a prefix of stored strings. Each node has up to 26 children (for lowercase English) and an `is_end` flag. Insert and search are O(L) where L is the string length — independent of the number of stored strings. Tries excel at prefix queries (autocomplete), are used in spell checkers and IP routing tables (binary tries over bit prefixes). Space can be O(ALPHABET_SIZE × L × N) which is worse than a hash table for large sparse key sets; compressed tries (Patricia/radix trees) mitigate this.
A set stores unique keys with no associated values; its core operations are add, remove, and contains (membership test), all O(1) average for a hash set or O(log n) for a tree-based sorted set. A map (dictionary) stores key-value pairs; it is essentially a set of keys with an associated value for each. Internally, a hash set is often implemented as a hash map where values are discarded (or replaced with a sentinel). Sets are used for deduplication and membership checking; maps for key-based retrieval.
A set stores unique keys with no associated values; its core operations are add, remove, and contains (membership test), all O(1) average for a hash set or O(log n) for a tree-based sorted set. A map (dictionary) stores key-value pairs; it is essentially a set of keys with an associated value for each. Internally, a hash set is often implemented as a hash map where values are discarded (or replaced with a sentinel). Sets are used for deduplication and membership checking; maps for key-based retrieval.
Memoization is an optimisation that caches the return value of a function call keyed by its arguments, so subsequent calls with the same arguments return the cached result in O(1) instead of recomputing. It converts certain exponential-time recursive algorithms (e.g. naive Fibonacci O(2^n)) into polynomial-time by ensuring each unique subproblem is solved only once. Memoization is top-down dynamic programming — you write the natural recursion and add a cache.
python
from functools import lru_cache
@lru_cache(maxsize=None)
def fib(n):
if n < 2: return n
return fib(n-1) + fib(n-2) # O(n) with memo
Memoization is an optimisation that caches the return value of a function call keyed by its arguments, so subsequent calls with the same arguments return the cached result in O(1) instead of recomputing. It converts certain exponential-time recursive algorithms (e.g. naive Fibonacci O(2^n)) into polynomial-time by ensuring each unique subproblem is solved only once. Memoization is top-down dynamic programming — you write the natural recursion and add a cache.
```python
from functools import lru_cache
@lru_cache(maxsize=None)
def fib(n):
if n < 2: return n
return fib(n-1) + fib(n-2) # O(n) with memo
```
Dynamic programming (DP) solves problems by breaking them into overlapping subproblems, solving each exactly once, and storing results to avoid redundant recomputation. DP applies when a problem has optimal substructure (optimal solution built from optimal sub-solutions) and overlapping subproblems (same sub-inputs recur). Two approaches: top-down (recursive + memoization) and bottom-up (iterative tabulation, usually more space-efficient). Classic DP problems: Fibonacci, knapsack, LCS, edit distance, shortest paths (Bellman-Ford, Floyd-Warshall).
Dynamic programming (DP) solves problems by breaking them into overlapping subproblems, solving each exactly once, and storing results to avoid redundant recomputation. DP applies when a problem has optimal substructure (optimal solution built from optimal sub-solutions) and overlapping subproblems (same sub-inputs recur). Two approaches: top-down (recursive + memoization) and bottom-up (iterative tabulation, usually more space-efficient). Classic DP problems: Fibonacci, knapsack, LCS, edit distance, shortest paths (Bellman-Ford, Floyd-Warshall).
Greedy algorithms make the locally optimal choice at each step with no backtracking, hoping to reach a global optimum. They are O(n log n) or O(n) and simple to implement but only correct for problems with the greedy-choice property (each local optimum leads to global optimum). DP explores all possible subproblem combinations via tabulation/memoization, guaranteeing the global optimum at higher time/space cost. Greedy works for: interval scheduling, Huffman coding, Dijkstra's, Prim's. DP is needed when greedy fails: 0/1 knapsack, edit distance, matrix chain multiplication.
Greedy algorithms make the locally optimal choice at each step with no backtracking, hoping to reach a global optimum. They are O(n log n) or O(n) and simple to implement but only correct for problems with the greedy-choice property (each local optimum leads to global optimum). DP explores all possible subproblem combinations via tabulation/memoization, guaranteeing the global optimum at higher time/space cost. Greedy works for: interval scheduling, Huffman coding, Dijkstra's, Prim's. DP is needed when greedy fails: 0/1 knapsack, edit distance, matrix chain multiplication.
A sorting algorithm reorders a sequence into ascending or descending order. Key properties: stable (equal elements keep original order), in-place (O(1) extra space), and comparison-based (lower bound O(n log n) by decision tree argument). Five common ones: (1) Bubble sort O(n²) time, O(1) space, stable. (2) Merge sort O(n log n) time, O(n) space, stable. (3) Quick sort O(n log n) average / O(n²) worst time, O(log n) space, unstable. (4) Heap sort O(n log n) time, O(1) space, unstable. (5) Insertion sort O(n²) time, O(1) space, stable — optimal for nearly-sorted data (O(n) best case).
A sorting algorithm reorders a sequence into ascending or descending order. Key properties: stable (equal elements keep original order), in-place (O(1) extra space), and comparison-based (lower bound O(n log n) by decision tree argument). Five common ones: (1) Bubble sort O(n²) time, O(1) space, stable. (2) Merge sort O(n log n) time, O(n) space, stable. (3) Quick sort O(n log n) average / O(n²) worst time, O(log n) space, unstable. (4) Heap sort O(n log n) time, O(1) space, unstable. (5) Insertion sort O(n²) time, O(1) space, stable — optimal for nearly-sorted data (O(n) best case).
Amortized analysis spreads the cost of occasional expensive operations over a sequence of cheaper ones to give a per-operation average. For a dynamic array that doubles on full capacity: most pushes cost O(1); the occasional doubling copy costs O(n). Using the accounting (or aggregate) method: over n pushes, the total number of element copies is 1 + 2 + 4 + … + n ≤ 2n, so total work is O(n) and cost per push is O(1) amortized even though individual doubling steps are O(n). This is why Python's `list.append` is described as O(1) amortized despite periodic reallocations.
python
# Total copies for n=8 appends (capacity 1→2→4→8):
# copies = 1 + 2 + 4 = 7 < 2*8 = 16 => O(n) total
Amortized analysis spreads the cost of occasional expensive operations over a sequence of cheaper ones to give a per-operation average. For a dynamic array that doubles on full capacity: most pushes cost O(1); the occasional doubling copy costs O(n). Using the accounting (or aggregate) method: over n pushes, the total number of element copies is 1 + 2 + 4 + … + n ≤ 2n, so total work is O(n) and cost per push is O(1) amortized even though individual doubling steps are O(n). This is why Python's `list.append` is described as O(1) amortized despite periodic reallocations.
```python
# Total copies for n=8 appends (capacity 1→2→4→8):
# copies = 1 + 2 + 4 = 7 < 2*8 = 16 => O(n) total
```
The sliding window technique maintains a window [left, right] over a sequence, efficiently adding the element entering on the right and removing the element leaving on the left in O(1) each, giving O(n) total instead of O(nk) for the naive approach. For maximum sum of size k: compute the initial window sum, then slide right, adding the new element and subtracting the outgoing element, tracking the running maximum. Two-pointer (for variable-size windows) expands right until a condition is met, then shrinks left until it's violated again — both pointers only move forward, giving O(n).
python
def max_sum_k(arr, k):
window = sum(arr[:k])
best = window
for i in range(k, len(arr)):
window += arr[i] - arr[i-k]
best = max(best, window)
return best
The sliding window technique maintains a window [left, right] over a sequence, efficiently adding the element entering on the right and removing the element leaving on the left in O(1) each, giving O(n) total instead of O(nk) for the naive approach. For maximum sum of size k: compute the initial window sum, then slide right, adding the new element and subtracting the outgoing element, tracking the running maximum. Two-pointer (for variable-size windows) expands right until a condition is met, then shrinks left until it's violated again — both pointers only move forward, giving O(n).
```python
def max_sum_k(arr, k):
window = sum(arr[:k])
best = window
for i in range(k, len(arr)):
window += arr[i] - arr[i-k]
best = max(best, window)
return best
```
Binary search halves the search interval each iteration, finding a target in O(log n) time on a sorted array. The two canonical bugs: (1) using `mid = (lo + hi) / 2` can overflow for large indices in languages with fixed-width integers — use `lo + (hi - lo) / 2`. (2) Loop condition `lo < hi` vs `lo <= hi` — use `lo <= hi` with the half-open convention to handle single-element arrays and ensure every index is reachable. When searching for a boundary (first true, last false), use `lo <= hi` and carefully update `lo` or `hi` without returning early.
python
def binary_search(arr, target):
lo, hi = 0, len(arr) - 1
while lo <= hi:
mid = lo + (hi - lo) // 2
if arr[mid] == target: return mid
elif arr[mid] < target: lo = mid + 1
else: hi = mid - 1
return -1 # not found
Binary search halves the search interval each iteration, finding a target in O(log n) time on a sorted array. The two canonical bugs: (1) using `mid = (lo + hi) / 2` can overflow for large indices in languages with fixed-width integers — use `lo + (hi - lo) / 2`. (2) Loop condition `lo < hi` vs `lo <= hi` — use `lo <= hi` with the half-open convention to handle single-element arrays and ensure every index is reachable. When searching for a boundary (first true, last false), use `lo <= hi` and carefully update `lo` or `hi` without returning early.
```python
def binary_search(arr, target):
lo, hi = 0, len(arr) - 1
while lo <= hi:
mid = lo + (hi - lo) // 2
if arr[mid] == target: return mid
elif arr[mid] < target: lo = mid + 1
else: hi = mid - 1
return -1 # not found
```
Binary search on the answer applies when you can frame a problem as: "find the minimum (or maximum) value x such that condition(x) is true," and condition is monotone (once true, always true for larger x). Instead of searching an array, you binary search over the answer space. A classic example: "given n tasks with times and m workers, what is the minimum possible maximum load?" Check feasibility for a given load cap in O(n), then binary search the cap in O(log(sum)). Total O(n log(sum)) vs O(n²) brute force.
python
def can_split(times, m, cap):
workers, cur = 1, 0
for t in times:
if cur + t > cap: workers += 1; cur = 0
cur += t
return workers <= m
# binary search lo=max(times), hi=sum(times)
Binary search on the answer applies when you can frame a problem as: "find the minimum (or maximum) value x such that condition(x) is true," and condition is monotone (once true, always true for larger x). Instead of searching an array, you binary search over the answer space. A classic example: "given n tasks with times and m workers, what is the minimum possible maximum load?" Check feasibility for a given load cap in O(n), then binary search the cap in O(log(sum)). Total O(n log(sum)) vs O(n²) brute force.
```python
def can_split(times, m, cap):
workers, cur = 1, 0
for t in times:
if cur + t > cap: workers += 1; cur = 0
cur += t
return workers <= m
# binary search lo=max(times), hi=sum(times)
```
Robin Hood hashing is an open-addressing collision resolution scheme that minimises the variance in probe lengths by "stealing" slots from rich entries to give to poor ones. When inserting an element, if its current probe length exceeds the probe length of the element already in that slot (the "rich" element), they swap. The displaced element continues looking for its slot. This guarantees no element has a probe length greater than O(log n) with high probability and achieves better average performance than linear probing at the same load factor. Rust's `HashMap` (before switching to SwissTable) and some game engines use Robin Hood hashing.
Robin Hood hashing is an open-addressing collision resolution scheme that minimises the variance in probe lengths by "stealing" slots from rich entries to give to poor ones. When inserting an element, if its current probe length exceeds the probe length of the element already in that slot (the "rich" element), they swap. The displaced element continues looking for its slot. This guarantees no element has a probe length greater than O(log n) with high probability and achieves better average performance than linear probing at the same load factor. Rust's `HashMap` (before switching to SwissTable) and some game engines use Robin Hood hashing.
Quadratic probing resolves collisions in open addressing by probing at offsets h, h+1², h+2², h+3², … (mod capacity) instead of consecutive slots. This eliminates primary clustering (long consecutive runs caused by linear probing) because different keys that hash to the same initial slot take different probe sequences. However, it introduces secondary clustering: all keys with the same initial hash h follow the exact same probe sequence, so they still compete with each other. Additionally, quadratic probing does not guarantee hitting all slots unless the table size is a prime and load factor < 0.5. Double hashing is an alternative that eliminates secondary clustering entirely.
Quadratic probing resolves collisions in open addressing by probing at offsets h, h+1², h+2², h+3², … (mod capacity) instead of consecutive slots. This eliminates primary clustering (long consecutive runs caused by linear probing) because different keys that hash to the same initial slot take different probe sequences. However, it introduces secondary clustering: all keys with the same initial hash h follow the exact same probe sequence, so they still compete with each other. Additionally, quadratic probing does not guarantee hitting all slots unless the table size is a prime and load factor < 0.5. Double hashing is an alternative that eliminates secondary clustering entirely.
AVL trees maintain |height(left) - height(right)| ≤ 1 at every node, enforced after every insert/delete via rotations. An LL imbalance (left-left) occurs when the left child's left subtree is too tall — fix with a right rotation at the unbalanced node. RR imbalance: right child's right subtree too tall — left rotation. LR imbalance: left child's right subtree too tall — left-rotate the left child first, then right-rotate the node. RL: right child's left subtree too tall — right-rotate the right child, then left-rotate the node. Each rotation is O(1). After balancing, height is O(log n) guaranteeing O(log n) operations.
LL case: RR case:
z z
/ \
y →[R-rot]→ y → [L-rot]
/ \
x x
AVL trees maintain |height(left) - height(right)| ≤ 1 at every node, enforced after every insert/delete via rotations. An LL imbalance (left-left) occurs when the left child's left subtree is too tall — fix with a right rotation at the unbalanced node. RR imbalance: right child's right subtree too tall — left rotation. LR imbalance: left child's right subtree too tall — left-rotate the left child first, then right-rotate the node. RL: right child's left subtree too tall — right-rotate the right child, then left-rotate the node. Each rotation is O(1). After balancing, height is O(log n) guaranteeing O(log n) operations.
```
LL case: RR case:
z z
/ \
y →[R-rot]→ y → [L-rot]
/ \
x x
```
Red-black trees augment each node with a color (red or black) and enforce five invariants: (1) every node is red or black; (2) the root is black; (3) leaves (NIL sentinels) are black; (4) a red node's children are both black (no two consecutive reds); (5) every root-to-leaf path has the same number of black nodes (black-height). These invariants bound height: the black-height bh ≤ h ≤ 2·bh because red nodes can at most double the black path. Since a tree of black-height bh has at least 2^bh - 1 nodes, h ≤ 2 log₂(n+1), guaranteeing O(log n). Rebalancing after insert/delete requires at most O(1) rotations (unlike AVL which may rotate up the whole path).
Red-black trees augment each node with a color (red or black) and enforce five invariants: (1) every node is red or black; (2) the root is black; (3) leaves (NIL sentinels) are black; (4) a red node's children are both black (no two consecutive reds); (5) every root-to-leaf path has the same number of black nodes (black-height). These invariants bound height: the black-height bh ≤ h ≤ 2·bh because red nodes can at most double the black path. Since a tree of black-height bh has at least 2^bh - 1 nodes, h ≤ 2 log₂(n+1), guaranteeing O(log n). Rebalancing after insert/delete requires at most O(1) rotations (unlike AVL which may rotate up the whole path).
A B-tree of order m stores up to m-1 keys per node and up to m child pointers, keeping all data at every level — both internal and leaf nodes hold records. A B+ tree stores data only in leaf nodes; internal nodes hold separator keys only, increasing the branching factor significantly (more keys fit per internal page). B+ tree leaves are linked in a doubly-linked list enabling efficient O(n) range scans (critical for SQL range queries). For a 4KB page with 8-byte keys, a B+ tree internal node can hold ~500 keys vs maybe 100 for a B-tree that also stores data. This raises the tree to 3-4 levels for billions of records, giving 3-4 disk I/Os per lookup. PostgreSQL, MySQL InnoDB, and SQLite all use B+ trees.
A B-tree of order m stores up to m-1 keys per node and up to m child pointers, keeping all data at every level — both internal and leaf nodes hold records. A B+ tree stores data only in leaf nodes; internal nodes hold separator keys only, increasing the branching factor significantly (more keys fit per internal page). B+ tree leaves are linked in a doubly-linked list enabling efficient O(n) range scans (critical for SQL range queries). For a 4KB page with 8-byte keys, a B+ tree internal node can hold ~500 keys vs maybe 100 for a B-tree that also stores data. This raises the tree to 3-4 levels for billions of records, giving 3-4 disk I/Os per lookup. PostgreSQL, MySQL InnoDB, and SQLite all use B+ trees.
Heapify-up (sift-up) restores the heap invariant after inserting at the end: compare the node at index i with its parent at ⌊i/2⌋; if the heap property is violated, swap them, and repeat at the parent index. This runs in O(log n) worst case. Heapify-down (sift-down) restores the heap after removing the root (replaced by the last leaf): compare the node at index i with its children at 2i and 2i+1; swap with the smaller (min-heap) or larger (max-heap) child if the property is violated; repeat at that child's index. Also O(log n).
python
def sift_up(h, i):
while i > 0 and h[(i-1)//2] > h[i]:
h[i], h[(i-1)//2] = h[(i-1)//2], h[i]
i = (i-1)//2
Heapify-up (sift-up) restores the heap invariant after inserting at the end: compare the node at index i with its parent at ⌊i/2⌋; if the heap property is violated, swap them, and repeat at the parent index. This runs in O(log n) worst case. Heapify-down (sift-down) restores the heap after removing the root (replaced by the last leaf): compare the node at index i with its children at 2i and 2i+1; swap with the smaller (min-heap) or larger (max-heap) child if the property is violated; repeat at that child's index. Also O(log n).
```python
def sift_up(h, i):
while i > 0 and h[(i-1)//2] > h[i]:
h[i], h[(i-1)//2] = h[(i-1)//2], h[i]
i = (i-1)//2
```
The naive approach of inserting n elements one by one costs O(n log n). Bottom-up heap construction achieves O(n): treat the input array as a complete binary tree, then call heapify-down on every non-leaf node from bottom to top (indices ⌊n/2⌋-1 down to 0). The key insight is that most nodes are near the bottom and travel only a few levels down — the exact sum of work is Σ(h × nodes at depth d) = O(n). Python's `heapq.heapify` uses this O(n) algorithm.
python
def build_heap(arr):
n = len(arr)
for i in range(n//2 - 1, -1, -1):
sift_down(arr, i, n)
return arr # now a valid min-heap
The naive approach of inserting n elements one by one costs O(n log n). Bottom-up heap construction achieves O(n): treat the input array as a complete binary tree, then call heapify-down on every non-leaf node from bottom to top (indices ⌊n/2⌋-1 down to 0). The key insight is that most nodes are near the bottom and travel only a few levels down — the exact sum of work is Σ(h × nodes at depth d) = O(n). Python's `heapq.heapify` uses this O(n) algorithm.
```python
def build_heap(arr):
n = len(arr)
for i in range(n//2 - 1, -1, -1):
sift_down(arr, i, n)
return arr # now a valid min-heap
```
A priority queue supports insert and extract-min (or extract-max) with the minimum element always accessible. Implementation options: (1) Binary heap — insert O(log n), extract-min O(log n), peek O(1); simplest and most cache-friendly. (2) Sorted array — insert O(n), extract O(1); good only for nearly-static collections. (3) Fibonacci heap — insert O(1), decrease-key O(1) amortized, extract-min O(log n) amortized; theoretically optimal for Dijkstra (O(E + V log V)) but complex with high constants, rarely used in practice. (4) Pairing heap — simpler than Fibonacci, similar amortized bounds in practice. Binary heaps dominate real implementations.
A priority queue supports insert and extract-min (or extract-max) with the minimum element always accessible. Implementation options: (1) Binary heap — insert O(log n), extract-min O(log n), peek O(1); simplest and most cache-friendly. (2) Sorted array — insert O(n), extract O(1); good only for nearly-static collections. (3) Fibonacci heap — insert O(1), decrease-key O(1) amortized, extract-min O(log n) amortized; theoretically optimal for Dijkstra (O(E + V log V)) but complex with high constants, rarely used in practice. (4) Pairing heap — simpler than Fibonacci, similar amortized bounds in practice. Binary heaps dominate real implementations.
Topological sort orders vertices of a DAG so every directed edge u→v has u before v. Kahn's algorithm: compute in-degrees; enqueue all zero-in-degree vertices; repeatedly dequeue a vertex, add it to the result, decrement neighbours' in-degrees, and enqueue any that reach zero. If the result has fewer than V vertices, the graph has a cycle. DFS post-order: run DFS; when a vertex finishes (all descendants visited), push it onto a stack; the reverse of the stack is a topological order. Both are O(V + E). Kahn's naturally detects cycles; DFS requires tracking grey/white/black states. Kahn's is easier to parallelize.
Topological sort orders vertices of a DAG so every directed edge u→v has u before v. Kahn's algorithm: compute in-degrees; enqueue all zero-in-degree vertices; repeatedly dequeue a vertex, add it to the result, decrement neighbours' in-degrees, and enqueue any that reach zero. If the result has fewer than V vertices, the graph has a cycle. DFS post-order: run DFS; when a vertex finishes (all descendants visited), push it onto a stack; the reverse of the stack is a topological order. Both are O(V + E). Kahn's naturally detects cycles; DFS requires tracking grey/white/black states. Kahn's is easier to parallelize.
Dijkstra's finds shortest paths from a source in a graph with non-negative edge weights. It maintains a tentative distance array (initially ∞, source = 0) and a min-heap of (distance, vertex) pairs. Greedily extract the minimum-distance unvisited vertex u; for each neighbour v, if dist[u] + w(u,v) < dist[v], update dist[v] and push (dist[v], v) to the heap. Correctness relies on non-negative weights: when u is extracted, no future path can improve dist[u] because all future extractions have ≥ dist[u]. Time complexity: O((V + E) log V) with a binary min-heap (each edge may trigger a push, each vertex is extracted once). With a Fibonacci heap: O(E + V log V).
Dijkstra's finds shortest paths from a source in a graph with non-negative edge weights. It maintains a tentative distance array (initially ∞, source = 0) and a min-heap of (distance, vertex) pairs. Greedily extract the minimum-distance unvisited vertex u; for each neighbour v, if dist[u] + w(u,v) < dist[v], update dist[v] and push (dist[v], v) to the heap. Correctness relies on non-negative weights: when u is extracted, no future path can improve dist[u] because all future extractions have ≥ dist[u]. Time complexity: O((V + E) log V) with a binary min-heap (each edge may trigger a push, each vertex is extracted once). With a Fibonacci heap: O(E + V log V).
Use Bellman-Ford when the graph contains negative edge weights. Dijkstra's greedy extraction is incorrect with negative weights because a later path through a negative edge could improve an already-finalized distance. Bellman-Ford relaxes all V-1 edges in V-1 rounds, guaranteeing shortest paths. It also detects negative-weight cycles: if any distance can still be improved after V-1 rounds, a negative cycle exists (no well-defined shortest path). Time O(VE) vs Dijkstra's O((V+E) log V). Practical use: routing protocols (RIP uses Bellman-Ford), currency arbitrage detection, and any graph with negative weights (e.g. gain edges rewritten as negative costs).
Use Bellman-Ford when the graph contains negative edge weights. Dijkstra's greedy extraction is incorrect with negative weights because a later path through a negative edge could improve an already-finalized distance. Bellman-Ford relaxes all V-1 edges in V-1 rounds, guaranteeing shortest paths. It also detects negative-weight cycles: if any distance can still be improved after V-1 rounds, a negative cycle exists (no well-defined shortest path). Time O(VE) vs Dijkstra's O((V+E) log V). Practical use: routing protocols (RIP uses Bellman-Ford), currency arbitrage detection, and any graph with negative weights (e.g. gain edges rewritten as negative costs).
Floyd-Warshall computes shortest paths between every pair of vertices in O(V³) time and O(V²) space. It uses dynamic programming: dist[i][j][k] = shortest path from i to j using only vertices {1..k} as intermediates. The recurrence: dist[i][j][k] = min(dist[i][j][k-1], dist[i][k][k-1] + dist[k][j][k-1]). The k dimension is eliminated in-place, reducing to a V×V matrix updated in V passes. Initialize with adjacency matrix weights (∞ for non-edges, 0 on diagonal). Detects negative cycles: if dist[i][i] < 0 after running, a negative cycle exists. Preferred over running Dijkstra V times when the graph is dense or has negative weights (but no negative cycles).
Floyd-Warshall computes shortest paths between every pair of vertices in O(V³) time and O(V²) space. It uses dynamic programming: dist[i][j][k] = shortest path from i to j using only vertices {1..k} as intermediates. The recurrence: dist[i][j][k] = min(dist[i][j][k-1], dist[i][k][k-1] + dist[k][j][k-1]). The k dimension is eliminated in-place, reducing to a V×V matrix updated in V passes. Initialize with adjacency matrix weights (∞ for non-edges, 0 on diagonal). Detects negative cycles: if dist[i][i] < 0 after running, a negative cycle exists. Preferred over running Dijkstra V times when the graph is dense or has negative weights (but no negative cycles).
Union-Find maintains a partition of n elements into disjoint sets supporting two operations: find (which set does x belong to?) and union (merge the sets containing x and y). Each set is a tree; find traverses to the root. Path compression flattens the tree during find: each node on the root-path points directly to the root after the call. Union by rank always attaches the shorter tree under the taller, keeping trees shallow. Together, find and union run in O(α(n)) amortized — inverse Ackermann function, practically O(1) for all real n. Used in Kruskal's MST, cycle detection, and network connectivity.
python
parent = list(range(n)); rank = [0]*n
def find(x):
if parent[x]!=x: parent[x]=find(parent[x])
return parent[x]
def union(x,y):
rx,ry=find(x),find(y)
if rank[rx]<rank[ry]: rx,ry=ry,rx
parent[ry]=rx
if rank[rx]==rank[ry]: rank[rx]+=1
Union-Find maintains a partition of n elements into disjoint sets supporting two operations: find (which set does x belong to?) and union (merge the sets containing x and y). Each set is a tree; find traverses to the root. Path compression flattens the tree during find: each node on the root-path points directly to the root after the call. Union by rank always attaches the shorter tree under the taller, keeping trees shallow. Together, find and union run in O(α(n)) amortized — inverse Ackermann function, practically O(1) for all real n. Used in Kruskal's MST, cycle detection, and network connectivity.
```python
parent = list(range(n)); rank = [0]*n
def find(x):
if parent[x]!=x: parent[x]=find(parent[x])
return parent[x]
def union(x,y):
rx,ry=find(x),find(y)
if rank[rx]<rank[ry]: rx,ry=ry,rx
parent[ry]=rx
if rank[rx]==rank[ry]: rank[rx]+=1
```
A segment tree is a binary tree over an array of size n where each node stores the aggregate (sum, min, max) of a contiguous subarray. The root covers [0, n-1]; each node covers half its parent's range; leaves hold individual elements. Build is O(n). Point update: update the leaf, then recompute ancestors bottom-up — O(log n). Range query: recursively combine nodes whose ranges are fully inside the query range, splitting at at most O(log n) nodes — O(log n). Total space O(4n) using an array representation (1-indexed, children at 2i and 2i+1).
A segment tree is a binary tree over an array of size n where each node stores the aggregate (sum, min, max) of a contiguous subarray. The root covers [0, n-1]; each node covers half its parent's range; leaves hold individual elements. Build is O(n). Point update: update the leaf, then recompute ancestors bottom-up — O(log n). Range query: recursively combine nodes whose ranges are fully inside the query range, splitting at at most O(log n) nodes — O(log n). Total space O(4n) using an array representation (1-indexed, children at 2i and 2i+1).
```python
def build(node, lo, hi):
if lo==hi: tree[node]=arr[lo]; return
mid=(lo+hi)//2
build(2*node,lo,mid); build(2*node+1,mid+1,hi)
tree[node]=tree[2*node]+tree[2*node+1]
```
A Fenwick tree (BIT) stores partial sums in a compact array bit[] where bit[i] covers a range of length lowbit(i) = i & (-i) ending at i. Prefix sum query: accumulate bit[i], then jump to i - lowbit(i), repeating until i = 0 — O(log n). Point update: add delta to bit[i], then jump to i + lowbit(i), propagating up — O(log n). Space O(n). Compared to segment trees: BIT is simpler to code, uses 2× less memory, and has smaller constants, but supports fewer operations (no range update + range query without extra tricks). Ideal for order statistics, inversion count, and competitive programming.
python
def update(i, d):
while i < len(bit):
bit[i] += d; i += i & -i
def prefix(i):
s = 0
while i > 0:
s += bit[i]; i -= i & -i
return s
A Fenwick tree (BIT) stores partial sums in a compact array bit[] where bit[i] covers a range of length lowbit(i) = i & (-i) ending at i. Prefix sum query: accumulate bit[i], then jump to i - lowbit(i), repeating until i = 0 — O(log n). Point update: add delta to bit[i], then jump to i + lowbit(i), propagating up — O(log n). Space O(n). Compared to segment trees: BIT is simpler to code, uses 2× less memory, and has smaller constants, but supports fewer operations (no range update + range query without extra tricks). Ideal for order statistics, inversion count, and competitive programming.
```python
def update(i, d):
while i < len(bit):
bit[i] += d; i += i & -i
def prefix(i):
s = 0
while i > 0:
s += bit[i]; i -= i & -i
return s
```
All three traverse the trie character by character, taking O(L) time where L is the length of the input string — independent of the number of stored strings. Insert: for each character, if the child node doesn't exist, create it; mark the last node as end-of-word. Search: traverse; return True only if the last node exists and is marked end-of-word. startsWith: same as search but return True even if the last node is not end-of-word. Space per inserted string is O(L × ALPHABET_SIZE) in the worst case. Trie outperforms a hash set for prefix queries because hash sets can't answer "all strings starting with 'pre'" in sub-O(n) time.
All three traverse the trie character by character, taking O(L) time where L is the length of the input string — independent of the number of stored strings. Insert: for each character, if the child node doesn't exist, create it; mark the last node as end-of-word. Search: traverse; return True only if the last node exists and is marked end-of-word. startsWith: same as search but return True even if the last node is not end-of-word. Space per inserted string is O(L × ALPHABET_SIZE) in the worst case. Trie outperforms a hash set for prefix queries because hash sets can't answer "all strings starting with 'pre'" in sub-O(n) time.
```python
class TrieNode:
def __init__(self):
self.children = {}
self.is_end = False
```
A suffix tree is a compressed trie of all n suffixes of a string, built in O(n) via Ukkonen's algorithm. It enables O(m) substring search (m = pattern length), O(n) LCS (longest common substring), and supports arbitrary complex pattern queries. Space is O(n) but with a large constant (~20× the string). A suffix array is the sorted array of suffix start indices, built in O(n) with O(n) LCP array, using ~4× string space. Most suffix tree queries can be answered with suffix array + LCP in the same asymptotic time. In practice, suffix arrays are preferred for large texts (genome assembly, full-text search) due to cache efficiency; suffix trees for online string matching and complex structural queries.
A suffix tree is a compressed trie of all n suffixes of a string, built in O(n) via Ukkonen's algorithm. It enables O(m) substring search (m = pattern length), O(n) LCS (longest common substring), and supports arbitrary complex pattern queries. Space is O(n) but with a large constant (~20× the string). A suffix array is the sorted array of suffix start indices, built in O(n) with O(n) LCP array, using ~4× string space. Most suffix tree queries can be answered with suffix array + LCP in the same asymptotic time. In practice, suffix arrays are preferred for large texts (genome assembly, full-text search) due to cache efficiency; suffix trees for online string matching and complex structural queries.
An LRU (Least Recently Used) cache evicts the least recently accessed entry when at capacity. Use a hash map (key → node) for O(1) lookup and a doubly-linked list for O(1) eviction order maintenance. The list's tail is the LRU; head is the MRU. On get: move the accessed node to the head, return its value. On put: if key exists, update value and move to head; if at capacity, remove the tail node, delete from hash map, insert new node at head. All operations O(1). Python's `functools.lru_cache` and `collections.OrderedDict` implement this pattern.
python
from collections import OrderedDict
class LRUCache:
def __init__(self, cap):
self.cap = cap; self.cache = OrderedDict()
def get(self, key):
if key not in self.cache: return -1
self.cache.move_to_end(key); return self.cache[key]
def put(self, key, val):
if key in self.cache: self.cache.move_to_end(key)
self.cache[key] = val
if len(self.cache) > self.cap: self.cache.popitem(last=False)
An LRU (Least Recently Used) cache evicts the least recently accessed entry when at capacity. Use a hash map (key → node) for O(1) lookup and a doubly-linked list for O(1) eviction order maintenance. The list's tail is the LRU; head is the MRU. On get: move the accessed node to the head, return its value. On put: if key exists, update value and move to head; if at capacity, remove the tail node, delete from hash map, insert new node at head. All operations O(1). Python's `functools.lru_cache` and `collections.OrderedDict` implement this pattern.
```python
from collections import OrderedDict
class LRUCache:
def __init__(self, cap):
self.cap = cap; self.cache = OrderedDict()
def get(self, key):
if key not in self.cache: return -1
self.cache.move_to_end(key); return self.cache[key]
def put(self, key, val):
if key in self.cache: self.cache.move_to_end(key)
self.cache[key] = val
if len(self.cache) > self.cap: self.cache.popitem(last=False)
```
A monotonic stack maintains elements in strictly increasing or decreasing order from bottom to top. For "next greater element to the right": iterate left to right, maintaining a decreasing stack. When processing element x, pop all stack elements smaller than x — x is their next greater element. Push x. After processing all elements, remaining stack entries have no greater element to the right. Total work is O(n) because each element is pushed and popped at most once. Other uses: largest rectangle in histogram, daily temperatures, online stock span.
python
def next_greater(nums):
res = [-1] * len(nums)
stack = [] # indices, decreasing by value
for i, x in enumerate(nums):
while stack and nums[stack[-1]] < x:
res[stack.pop()] = x
stack.append(i)
return res
A monotonic stack maintains elements in strictly increasing or decreasing order from bottom to top. For "next greater element to the right": iterate left to right, maintaining a decreasing stack. When processing element x, pop all stack elements smaller than x — x is their next greater element. Push x. After processing all elements, remaining stack entries have no greater element to the right. Total work is O(n) because each element is pushed and popped at most once. Other uses: largest rectangle in histogram, daily temperatures, online stock span.
```python
def next_greater(nums):
res = [-1] * len(nums)
stack = [] # indices, decreasing by value
for i, x in enumerate(nums):
while stack and nums[stack[-1]] < x:
res[stack.pop()] = x
stack.append(i)
return res
```
A monotonic deque maintains indices such that values are strictly decreasing from front to back. For sliding window max of size k: iterate through the array; before processing index i, (1) remove front if it's outside the window (index ≤ i-k); (2) remove all indices from the back whose values are ≤ current value (they can never be the max while the current element is in the window); (3) append i; the front is always the max of the current window. Total O(n) time — each index is enqueued and dequeued at most once.
python
from collections import deque
def max_sliding_window(nums, k):
dq, res = deque(), []
for i, x in enumerate(nums):
while dq and dq[0] < i-k+1: dq.popleft()
while dq and nums[dq[-1]] <= x: dq.pop()
dq.append(i)
if i >= k-1: res.append(nums[dq[0]])
return res
A monotonic deque maintains indices such that values are strictly decreasing from front to back. For sliding window max of size k: iterate through the array; before processing index i, (1) remove front if it's outside the window (index ≤ i-k); (2) remove all indices from the back whose values are ≤ current value (they can never be the max while the current element is in the window); (3) append i; the front is always the max of the current window. Total O(n) time — each index is enqueued and dequeued at most once.
```python
from collections import deque
def max_sliding_window(nums, k):
dq, res = deque(), []
for i, x in enumerate(nums):
while dq and dq[0] < i-k+1: dq.popleft()
while dq and nums[dq[-1]] <= x: dq.pop()
dq.append(i)
if i >= k-1: res.append(nums[dq[0]])
return res
```
Floyd's (tortoise-and-hare) algorithm detects cycles using two pointers: slow moves one step at a time, fast moves two. If a cycle exists, fast will eventually lap slow and they meet inside the cycle — this is guaranteed within O(n) steps. If fast reaches null, no cycle. To find the cycle start: after detection, reset one pointer to the head; move both one step at a time; they meet at the cycle's entrance. This works because the distance from head to cycle start equals the distance from meeting point to cycle start (modulo cycle length). Space O(1). The alternative (hash set of visited nodes) uses O(n) space.
python
def detect_cycle(head):
slow = fast = head
while fast and fast.next:
slow = slow.next; fast = fast.next.next
if slow == fast: return True
return False
Floyd's (tortoise-and-hare) algorithm detects cycles using two pointers: slow moves one step at a time, fast moves two. If a cycle exists, fast will eventually lap slow and they meet inside the cycle — this is guaranteed within O(n) steps. If fast reaches null, no cycle. To find the cycle start: after detection, reset one pointer to the head; move both one step at a time; they meet at the cycle's entrance. This works because the distance from head to cycle start equals the distance from meeting point to cycle start (modulo cycle length). Space O(1). The alternative (hash set of visited nodes) uses O(n) space.
```python
def detect_cycle(head):
slow = fast = head
while fast and fast.next:
slow = slow.next; fast = fast.next.next
if slow == fast: return True
return False
```
Iterate with three pointers: prev (starts null), curr (starts head), and next. At each step, save curr.next, point curr.next to prev, advance prev to curr, and advance curr to saved next. When curr is null, prev points to the new head. This is O(n) time and O(1) space — no recursion, no extra memory. The recursive approach is conceptually clean but uses O(n) call stack space and risks stack overflow on very long lists.
python
def reverse_list(head):
prev, curr = None, head
while curr:
nxt = curr.next
curr.next = prev
prev = curr
curr = nxt
return prev # new head
Iterate with three pointers: prev (starts null), curr (starts head), and next. At each step, save curr.next, point curr.next to prev, advance prev to curr, and advance curr to saved next. When curr is null, prev points to the new head. This is O(n) time and O(1) space — no recursion, no extra memory. The recursive approach is conceptually clean but uses O(n) call stack space and risks stack overflow on very long lists.
```python
def reverse_list(head):
prev, curr = None, head
while curr:
nxt = curr.next
curr.next = prev
prev = curr
curr = nxt
return prev # new head
```
The naive approach (store root-to-node paths, find last common node) is O(n) time and O(n) space. For a BST, LCA is the first node where values split (one key ≤ node < other). For a general binary tree with preprocessing: Euler tour flattens the tree into a sequence of 2n-1 nodes recording each node when first entered and when returning from each child. LCA(u,v) = node with minimum depth between the first occurrences of u and v in the tour — a range minimum query (RMQ). Building a sparse table for RMQ takes O(n log n) preprocessing, then each LCA query is O(1). Without preprocessing, recursive O(n): if node equals either target return it; recurse left and right; if both return non-null, current node is LCA.
The naive approach (store root-to-node paths, find last common node) is O(n) time and O(n) space. For a BST, LCA is the first node where values split (one key ≤ node < other). For a general binary tree with preprocessing: Euler tour flattens the tree into a sequence of 2n-1 nodes recording each node when first entered and when returning from each child. LCA(u,v) = node with minimum depth between the first occurrences of u and v in the tour — a range minimum query (RMQ). Building a sparse table for RMQ takes O(n log n) preprocessing, then each LCA query is O(1). Without preprocessing, recursive O(n): if node equals either target return it; recurse left and right; if both return non-null, current node is LCA.
Serialization converts a tree to a string/array; deserialization reconstructs it. BFS-based: level-order traversal, encoding null children explicitly. Deserialization reads values level by level, pairing each non-null node with the next two values as children — O(n) time and space. Preorder + null markers also works: recursively write node value then recurse left then right, encoding null as "#". Deserialization reads one token at a time, returning null on "#". This is the approach used in LeetCode's TreeNode codec and handles arbitrary trees including non-complete ones.
python
def serialize(root):
res=[]; dfs=lambda n: (res.append(str(n.val) if n else "#"),
dfs(n.left) if n else None, dfs(n.right) if n else None)
dfs(root); return ",".join(res)
Serialization converts a tree to a string/array; deserialization reconstructs it. BFS-based: level-order traversal, encoding null children explicitly. Deserialization reads values level by level, pairing each non-null node with the next two values as children — O(n) time and space. Preorder + null markers also works: recursively write node value then recurse left then right, encoding null as "#". Deserialization reads one token at a time, returning null on "#". This is the approach used in LeetCode's TreeNode codec and handles arbitrary trees including non-complete ones.
```python
def serialize(root):
res=[]; dfs=lambda n: (res.append(str(n.val) if n else "#"),
dfs(n.left) if n else None, dfs(n.right) if n else None)
dfs(root); return ",".join(res)
```
Three-color DFS assigns each vertex a state: white (unvisited), grey (in current DFS path / recursion stack), or black (fully processed). Start DFS from every white vertex. When exploring a white neighbour, mark it grey and recurse; when done, mark it black. If you encounter a grey neighbour, you've found a back edge — a cycle exists. Black neighbours are safe (already fully explored, no cycle through them). This is O(V + E) and detects all cycles. Note: for undirected graphs, simply tracking visited is enough (a revisited non-parent node means a cycle).
python
def has_cycle(graph, n):
color = [0]*n # 0=white,1=grey,2=black
def dfs(u):
color[u]=1
for v in graph[u]:
if color[v]==1: return True
if color[v]==0 and dfs(v): return True
color[u]=2; return False
return any(dfs(i) for i in range(n) if color[i]==0)
Three-color DFS assigns each vertex a state: white (unvisited), grey (in current DFS path / recursion stack), or black (fully processed). Start DFS from every white vertex. When exploring a white neighbour, mark it grey and recurse; when done, mark it black. If you encounter a grey neighbour, you've found a back edge — a cycle exists. Black neighbours are safe (already fully explored, no cycle through them). This is O(V + E) and detects all cycles. Note: for undirected graphs, simply tracking visited is enough (a revisited non-parent node means a cycle).
```python
def has_cycle(graph, n):
color = [0]*n # 0=white,1=grey,2=black
def dfs(u):
color[u]=1
for v in graph[u]:
if color[v]==1: return True
if color[v]==0 and dfs(v): return True
color[u]=2; return False
return any(dfs(i) for i in range(n) if color[i]==0)
```
Kosaraju's is two-pass: (1) run DFS on the original graph, push vertices to a stack in finish order; (2) transpose all edges; (3) pop vertices from the stack, running DFS on the transposed graph — each DFS tree is an SCC. Time O(V + E), space O(V + E). Tarjan's is single-pass: DFS maintaining a discovery time (disc[]) and low[] (lowest disc reachable via back/cross edges). A node u is the root of an SCC when low[u] == disc[u]; pop the stack to that point. Also O(V + E). Tarjan's is preferred in practice (one pass, no graph transposition), while Kosaraju's is easier to reason about for correctness proofs.
Kosaraju's is two-pass: (1) run DFS on the original graph, push vertices to a stack in finish order; (2) transpose all edges; (3) pop vertices from the stack, running DFS on the transposed graph — each DFS tree is an SCC. Time O(V + E), space O(V + E). Tarjan's is single-pass: DFS maintaining a discovery time (disc[]) and low[] (lowest disc reachable via back/cross edges). A node u is the root of an SCC when low[u] == disc[u]; pop the stack to that point. Also O(V + E). Tarjan's is preferred in practice (one pass, no graph transposition), while Kosaraju's is easier to reason about for correctness proofs.
Given n items with weights w[i] and values v[i] and a capacity W, the goal is to maximise value without exceeding W. Recurrence: dp[i][c] = max(dp[i-1][c], dp[i-1][c-w[i]] + v[i]) if c ≥ w[i], else dp[i-1][c]. The 2D table is O(nW) time and space. Space optimization: since row i only depends on row i-1, use a single 1D array and iterate c from W down to w[i] (right-to-left prevents reusing item i twice). This gives O(W) space. Note: the "fractional knapsack" (items can be split) is greedily solvable in O(n log n); the 0/1 variant requires DP because greedy fails.
python
def knapsack(weights, values, W):
dp = [0] * (W+1)
for w, v in zip(weights, values):
for c in range(W, w-1, -1): # reverse!
dp[c] = max(dp[c], dp[c-w]+v)
return dp[W]
Given n items with weights w[i] and values v[i] and a capacity W, the goal is to maximise value without exceeding W. Recurrence: dp[i][c] = max(dp[i-1][c], dp[i-1][c-w[i]] + v[i]) if c ≥ w[i], else dp[i-1][c]. The 2D table is O(nW) time and space. Space optimization: since row i only depends on row i-1, use a single 1D array and iterate c from W down to w[i] (right-to-left prevents reusing item i twice). This gives O(W) space. Note: the "fractional knapsack" (items can be split) is greedily solvable in O(n log n); the 0/1 variant requires DP because greedy fails.
```python
def knapsack(weights, values, W):
dp = [0] * (W+1)
for w, v in zip(weights, values):
for c in range(W, w-1, -1): # reverse!
dp[c] = max(dp[c], dp[c-w]+v)
return dp[W]
```
LCS finds the longest sequence of characters (not necessarily contiguous) that appears in both strings in the same order. DP recurrence: if s1[i] == s2[j], dp[i][j] = dp[i-1][j-1] + 1; else dp[i][j] = max(dp[i-1][j], dp[i][j-1]). Base case: dp[0][*] = dp[*][0] = 0. Fill the table in O(mn) time and O(mn) space; optimize to O(min(m,n)) by keeping only two rows. Backtrack through the table to reconstruct the actual subsequence. LCS is the basis of `diff` tools, DNA alignment, and plagiarism detection.
python
def lcs(a, b):
m,n=len(a),len(b); dp=[[0]*(n+1) for _ in range(m+1)]
for i in range(1,m+1):
for j in range(1,n+1):
dp[i][j]=dp[i-1][j-1]+1 if a[i-1]==b[j-1] else max(dp[i-1][j],dp[i][j-1])
return dp[m][n]
LCS finds the longest sequence of characters (not necessarily contiguous) that appears in both strings in the same order. DP recurrence: if s1[i] == s2[j], dp[i][j] = dp[i-1][j-1] + 1; else dp[i][j] = max(dp[i-1][j], dp[i][j-1]). Base case: dp[0][*] = dp[*][0] = 0. Fill the table in O(mn) time and O(mn) space; optimize to O(min(m,n)) by keeping only two rows. Backtrack through the table to reconstruct the actual subsequence. LCS is the basis of `diff` tools, DNA alignment, and plagiarism detection.
```python
def lcs(a, b):
m,n=len(a),len(b); dp=[[0]*(n+1) for _ in range(m+1)]
for i in range(1,m+1):
for j in range(1,n+1):
dp[i][j]=dp[i-1][j-1]+1 if a[i-1]==b[j-1] else max(dp[i-1][j],dp[i][j-1])
return dp[m][n]
```
Edit distance between strings a and b is the minimum number of single-character insertions, deletions, or substitutions to transform a into b. DP recurrence: dp[i][j] = cost of transforming a[0..i-1] to b[0..j-1]. If a[i-1]==b[j-1]: dp[i][j] = dp[i-1][j-1]; else dp[i][j] = 1 + min(dp[i-1][j-1] (substitute), dp[i-1][j] (delete from a), dp[i][j-1] (insert into a)). Time O(mn), space O(min(m,n)) with rolling array. Used in spell checking, DNA alignment, fuzzy search, and autocorrect.
python
def edit_dist(a, b):
m,n=len(a),len(b); dp=list(range(n+1))
for i in range(1,m+1):
prev=dp[:]; dp[0]=i
for j in range(1,n+1):
dp[j]=prev[j-1] if a[i-1]==b[j-1] else 1+min(prev[j-1],prev[j],dp[j-1])
return dp[n]
Edit distance between strings a and b is the minimum number of single-character insertions, deletions, or substitutions to transform a into b. DP recurrence: dp[i][j] = cost of transforming a[0..i-1] to b[0..j-1]. If a[i-1]==b[j-1]: dp[i][j] = dp[i-1][j-1]; else dp[i][j] = 1 + min(dp[i-1][j-1] (substitute), dp[i-1][j] (delete from a), dp[i][j-1] (insert into a)). Time O(mn), space O(min(m,n)) with rolling array. Used in spell checking, DNA alignment, fuzzy search, and autocorrect.
```python
def edit_dist(a, b):
m,n=len(a),len(b); dp=list(range(n+1))
for i in range(1,m+1):
prev=dp[:]; dp[0]=i
for j in range(1,n+1):
dp[j]=prev[j-1] if a[i-1]==b[j-1] else 1+min(prev[j-1],prev[j],dp[j-1])
return dp[n]
```
Given coin denominations and a target amount, find the minimum number of coins. DP: dp[0] = 0; dp[i] = min(dp[i - coin] + 1) for all coins ≤ i; initialize dp[1..amount] = ∞. Fill bottom-up in O(amount × coins) time and O(amount) space. This is unbounded knapsack (coins can be reused). Greedy fails on arbitrary coin sets (e.g. coins = {1,3,4}, amount=6: greedy picks 4+1+1=3 coins, DP finds 3+3=2). The "can you make exact change" variant is a boolean DP.
python
def coin_change(coins, amount):
dp = [float('inf')] * (amount+1); dp[0]=0
for i in range(1, amount+1):
for c in coins:
if c<=i: dp[i]=min(dp[i], dp[i-c]+1)
return dp[amount] if dp[amount]!=float('inf') else -1
Given coin denominations and a target amount, find the minimum number of coins. DP: dp[0] = 0; dp[i] = min(dp[i - coin] + 1) for all coins ≤ i; initialize dp[1..amount] = ∞. Fill bottom-up in O(amount × coins) time and O(amount) space. This is unbounded knapsack (coins can be reused). Greedy fails on arbitrary coin sets (e.g. coins = {1,3,4}, amount=6: greedy picks 4+1+1=3 coins, DP finds 3+3=2). The "can you make exact change" variant is a boolean DP.
```python
def coin_change(coins, amount):
dp = [float('inf')] * (amount+1); dp[0]=0
for i in range(1, amount+1):
for c in coins:
if c<=i: dp[i]=min(dp[i], dp[i-c]+1)
return dp[amount] if dp[amount]!=float('inf') else -1
```
Merge sort divides the array in half recursively until subarrays of size 1 are reached, then merges sorted pairs back together. The merge step linearly scans two sorted arrays into one in O(n). The recurrence is T(n) = 2T(n/2) + O(n). By the Master Theorem (case 2: a=2, b=2, f(n)=n, n^(log_b a)=n^1): T(n) = O(n log n). Merge sort is stable, comparison-based, and optimal for linked lists (no random access needed). Its O(n) auxiliary space is a drawback for large in-memory arrays.
python
def merge_sort(arr):
if len(arr)<=1: return arr
mid=len(arr)//2
L,R=merge_sort(arr[:mid]),merge_sort(arr[mid:])
i=j=0; res=[]
while i<len(L) and j<len(R):
if L[i]<=R[j]: res.append(L[i]); i+=1
else: res.append(R[j]); j+=1
return res+L[i:]+R[j:]
Merge sort divides the array in half recursively until subarrays of size 1 are reached, then merges sorted pairs back together. The merge step linearly scans two sorted arrays into one in O(n). The recurrence is T(n) = 2T(n/2) + O(n). By the Master Theorem (case 2: a=2, b=2, f(n)=n, n^(log_b a)=n^1): T(n) = O(n log n). Merge sort is stable, comparison-based, and optimal for linked lists (no random access needed). Its O(n) auxiliary space is a drawback for large in-memory arrays.
```python
def merge_sort(arr):
if len(arr)<=1: return arr
mid=len(arr)//2
L,R=merge_sort(arr[:mid]),merge_sort(arr[mid:])
i=j=0; res=[]
while i<len(L) and j<len(R):
if L[i]<=R[j]: res.append(L[i]); i+=1
else: res.append(R[j]); j+=1
return res+L[i:]+R[j:]
```
Quicksort partitions an array around a pivot: elements smaller go left, larger go right, then recursively sort each partition. Lomuto partition selects the last element as pivot; Hoare uses two converging pointers. Average case O(n log n): each level of recursion does O(n) work and on average creates balanced partitions. Worst case O(n²): always picking the min or max pivot (sorted input with last-element pivot) creates partitions of size 0 and n-1. Randomized pivot: swap a random element with the last before partitioning, reducing probability of worst case to negligibly small (O(n²) with probability 1/n!). In practice quicksort outperforms merge sort due to better cache behaviour and O(log n) stack space.
python
import random
def quicksort(arr, lo, hi):
if lo<hi:
r=random.randint(lo,hi); arr[r],arr[hi]=arr[hi],arr[r]
p=partition(arr,lo,hi)
quicksort(arr,lo,p-1); quicksort(arr,p+1,hi)
Quicksort partitions an array around a pivot: elements smaller go left, larger go right, then recursively sort each partition. Lomuto partition selects the last element as pivot; Hoare uses two converging pointers. Average case O(n log n): each level of recursion does O(n) work and on average creates balanced partitions. Worst case O(n²): always picking the min or max pivot (sorted input with last-element pivot) creates partitions of size 0 and n-1. Randomized pivot: swap a random element with the last before partitioning, reducing probability of worst case to negligibly small (O(n²) with probability 1/n!). In practice quicksort outperforms merge sort due to better cache behaviour and O(log n) stack space.
```python
import random
def quicksort(arr, lo, hi):
if lo<hi:
r=random.randint(lo,hi); arr[r],arr[hi]=arr[hi],arr[r]
p=partition(arr,lo,hi)
quicksort(arr,lo,p-1); quicksort(arr,p+1,hi)
```
Counting sort counts occurrences of each value (works for integers in range [0, k]): O(n + k) time and space. Radix sort applies counting sort digit-by-digit from least significant to most significant: O(d(n + b)) where d is digits and b is the base; O(n) for fixed-width integers. Bucket sort distributes elements into k equally-spaced buckets, sorts each bucket, and concatenates: O(n + k) average when input is uniformly distributed. Use non-comparison sorts when: integers with bounded range (counting sort), fixed-width keys like 32-bit integers (radix sort), or uniformly distributed floats (bucket sort). They break the Ω(n log n) lower bound for comparison-based sorting because they exploit key structure.
Counting sort counts occurrences of each value (works for integers in range [0, k]): O(n + k) time and space. Radix sort applies counting sort digit-by-digit from least significant to most significant: O(d(n + b)) where d is digits and b is the base; O(n) for fixed-width integers. Bucket sort distributes elements into k equally-spaced buckets, sorts each bucket, and concatenates: O(n + k) average when input is uniformly distributed. Use non-comparison sorts when: integers with bounded range (counting sort), fixed-width keys like 32-bit integers (radix sort), or uniformly distributed floats (bucket sort). They break the Ω(n log n) lower bound for comparison-based sorting because they exploit key structure.
Heap sort: (1) build a max-heap in O(n) using bottom-up heapify; (2) repeatedly swap the root (maximum) with the last heap element, shrink the heap by one, and heapify-down — each extraction is O(log n), for n extractions: O(n log n). It is in-place (O(1) extra space, unlike merge sort's O(n)). It is unstable because the heapify-down swap can change the relative order of equal elements. Heap sort has poor cache performance (random access pattern in heapify-down) making it slower in practice than introsort (quicksort + heap sort hybrid). Prefer heap sort when: worst-case O(n log n) and O(1) space are both required and you can accept unstable sorting (e.g. embedded systems with tight memory).
Heap sort: (1) build a max-heap in O(n) using bottom-up heapify; (2) repeatedly swap the root (maximum) with the last heap element, shrink the heap by one, and heapify-down — each extraction is O(log n), for n extractions: O(n log n). It is in-place (O(1) extra space, unlike merge sort's O(n)). It is unstable because the heapify-down swap can change the relative order of equal elements. Heap sort has poor cache performance (random access pattern in heapify-down) making it slower in practice than introsort (quicksort + heap sort hybrid). Prefer heap sort when: worst-case O(n log n) and O(1) space are both required and you can accept unstable sorting (e.g. embedded systems with tight memory).
Reservoir sampling allows you to uniformly sample k items from a stream of unknown length n in O(n) time and O(k) space without knowing n in advance. Algorithm: fill the reservoir with the first k elements. For each subsequent element i (i > k), generate a random integer j in [1, i]; if j ≤ k, replace reservoir[j] with the new element. Proof of uniformity (by induction): after seeing i elements, each of the first i has probability k/i of being in the reservoir. Python's `random.sample` uses this. Extended by Algorithm L for faster O(k log(n/k)) expected skips.
Reservoir sampling allows you to uniformly sample k items from a stream of unknown length n in O(n) time and O(k) space without knowing n in advance. Algorithm: fill the reservoir with the first k elements. For each subsequent element i (i > k), generate a random integer j in [1, i]; if j ≤ k, replace reservoir[j] with the new element. Proof of uniformity (by induction): after seeing i elements, each of the first i has probability k/i of being in the reservoir. Python's `random.sample` uses this. Extended by Algorithm L for faster O(k log(n/k)) expected skips.
A skip list is a probabilistic data structure with multiple levels of sorted linked lists. Level 0 is the base (all elements); each higher level is a random subset of the level below, where each element is promoted with probability p (typically 0.5). Search: start at the top-left, move right until the next value exceeds target, then drop a level — skipping O(log n) elements per level on average. Total expected O(log n) search, insert, and delete. Space O(n log n) expected. Compared to BST: skip list is lock-friendly (concurrent insertions affect only a constant number of pointers, enabling fine-grained locking), simpler to implement correctly, but uses more memory. Redis uses skip lists for its sorted sets.
A skip list is a probabilistic data structure with multiple levels of sorted linked lists. Level 0 is the base (all elements); each higher level is a random subset of the level below, where each element is promoted with probability p (typically 0.5). Search: start at the top-left, move right until the next value exceeds target, then drop a level — skipping O(log n) elements per level on average. Total expected O(log n) search, insert, and delete. Space O(n log n) expected. Compared to BST: skip list is lock-friendly (concurrent insertions affect only a constant number of pointers, enabling fine-grained locking), simpler to implement correctly, but uses more memory. Redis uses skip lists for its sorted sets.
Cache-oblivious algorithms are designed without knowledge of cache line size B or cache capacity M, yet automatically optimise cache usage at every level of the memory hierarchy simultaneously. By contrast, cache-aware algorithms must be tuned for a specific cache configuration. The key idea is recursive divide-and-conquer: when a subproblem fits in cache, it stays there for all subsequent accesses. Example: cache-oblivious matrix multiplication recursively divides matrices until they fit in cache, achieving O(n³ / B√M) cache misses — matching the optimal cache-aware tiled algorithm. This means the same code performs optimally on L1, L2, L3, and RAM without manual tuning. The van Emde Boas layout for B-trees is a classic cache-oblivious tree arrangement that achieves O(log_B n) I/Os per search.
Cache-oblivious algorithms are designed without knowledge of cache line size B or cache capacity M, yet automatically optimise cache usage at every level of the memory hierarchy simultaneously. By contrast, cache-aware algorithms must be tuned for a specific cache configuration. The key idea is recursive divide-and-conquer: when a subproblem fits in cache, it stays there for all subsequent accesses. Example: cache-oblivious matrix multiplication recursively divides matrices until they fit in cache, achieving O(n³ / B√M) cache misses — matching the optimal cache-aware tiled algorithm. This means the same code performs optimally on L1, L2, L3, and RAM without manual tuning. The van Emde Boas layout for B-trees is a classic cache-oblivious tree arrangement that achieves O(log_B n) I/Os per search.
Van Emde Boas (vEB) trees support insert, delete, successor, predecessor, min, and max in O(log log U) time, where U is the universe size of integer keys. Structure: recursively divide the key space of size U into √U clusters of size √U; maintain a summary vEB over the cluster minimums. Recurrence: T(U) = T(√U) + O(1), solving to T(U) = O(log log U). Space is O(U) in the naïve form — impractical for large U. With hash maps for sparse children, space drops to O(n) but operations become O(log log U) expected. Practical when: universe size U is known and bounded (e.g. 32-bit integers, U = 2³²), n is large enough that O(log log U) ≈ 5 beats O(log n) ≈ 25, and in applications like network routing (IP lookup), priority queues in simulations, or when you need predecessor queries that balanced BSTs don't support in sub-O(log n).
Van Emde Boas (vEB) trees support insert, delete, successor, predecessor, min, and max in O(log log U) time, where U is the universe size of integer keys. Structure: recursively divide the key space of size U into √U clusters of size √U; maintain a summary vEB over the cluster minimums. Recurrence: T(U) = T(√U) + O(1), solving to T(U) = O(log log U). Space is O(U) in the naïve form — impractical for large U. With hash maps for sparse children, space drops to O(n) but operations become O(log log U) expected. Practical when: universe size U is known and bounded (e.g. 32-bit integers, U = 2³²), n is large enough that O(log log U) ≈ 5 beats O(log n) ≈ 25, and in applications like network routing (IP lookup), priority queues in simulations, or when you need predecessor queries that balanced BSTs don't support in sub-O(log n).
A Fibonacci heap is a collection of min-heap-ordered trees linked in a circular doubly-linked list. Insert and decrease-key are O(1) amortized by lazily adding new trees without consolidation. Decrease-key: simply cut the node from its parent and add to the root list (O(1)); if the parent has already lost one child (marked), cascade-cut up the tree recursively. The potential function Φ = (root list size) + 2·(marked nodes) bounds amortized costs. Extract-min is O(log n) amortized (consolidation merges trees of equal rank). In Dijkstra's, relaxing an edge calls decrease-key; with a binary heap, each relaxation is O(log V) → total O(E log V). With a Fibonacci heap, E decrease-keys cost O(E) total and V extract-mins cost O(V log V) → O(E + V log V). For dense graphs (E = V²) this is O(V²) vs O(V² log V) — asymptotically significant.
A Fibonacci heap is a collection of min-heap-ordered trees linked in a circular doubly-linked list. Insert and decrease-key are O(1) amortized by lazily adding new trees without consolidation. Decrease-key: simply cut the node from its parent and add to the root list (O(1)); if the parent has already lost one child (marked), cascade-cut up the tree recursively. The potential function Φ = (root list size) + 2·(marked nodes) bounds amortized costs. Extract-min is O(log n) amortized (consolidation merges trees of equal rank). In Dijkstra's, relaxing an edge calls decrease-key; with a binary heap, each relaxation is O(log V) → total O(E log V). With a Fibonacci heap, E decrease-keys cost O(E) total and V extract-mins cost O(V log V) → O(E + V log V). For dense graphs (E = V²) this is O(V²) vs O(V² log V) — asymptotically significant.
A persistent data structure preserves all previous versions after updates, enabling queries on any historical version. Full persistence allows updates on any version; partial persistence only allows updates on the latest. Path copying is the simplest technique: when modifying a BST node, create a new copy of that node and new copies of all ancestors on the path to the root, sharing all other nodes with the previous version. Each update creates O(h) new nodes (O(log n) for balanced trees) and a new root pointer. The old root still gives access to the previous version. Space per update: O(log n). Used in functional programming languages (Clojure's persistent maps), version control internals, and offline range tree queries.
A persistent data structure preserves all previous versions after updates, enabling queries on any historical version. Full persistence allows updates on any version; partial persistence only allows updates on the latest. Path copying is the simplest technique: when modifying a BST node, create a new copy of that node and new copies of all ancestors on the path to the root, sharing all other nodes with the previous version. Each update creates O(h) new nodes (O(log n) for balanced trees) and a new root pointer. The old root still gives access to the previous version. Space per update: O(log n). Used in functional programming languages (Clojure's persistent maps), version control internals, and offline range tree queries.
Chris Okasaki's purely functional red-black trees (from "Purely Functional Data Structures", 1999) implement insert in O(log n) using structural pattern matching to enumerate and collapse all four rotation cases into a single `balance` function operating on algebraic data types. Since no node is mutated, the old tree is still valid after insert — persistence is free via structural sharing. Okasaki's balance merges the LL, LR, RL, and RR rotation cases into one elegant rule: whenever a black node has a red child with a red child (in any of four configurations), rewrite to a red node with two black children. In lazy functional languages (Haskell), tree rebuilding is deferred until forced, enabling O(1) amortized operations for some workloads via lazy evaluation. This design influenced Haskell's `Data.Map`.
Chris Okasaki's purely functional red-black trees (from "Purely Functional Data Structures", 1999) implement insert in O(log n) using structural pattern matching to enumerate and collapse all four rotation cases into a single `balance` function operating on algebraic data types. Since no node is mutated, the old tree is still valid after insert — persistence is free via structural sharing. Okasaki's balance merges the LL, LR, RL, and RR rotation cases into one elegant rule: whenever a black node has a red child with a red child (in any of four configurations), rewrite to a red node with two black children. In lazy functional languages (Haskell), tree rebuilding is deferred until forced, enabling O(1) amortized operations for some workloads via lazy evaluation. This design influenced Haskell's `Data.Map`.
A Rope represents a string as a binary tree of shorter string chunks at the leaves. Each internal node stores the total length of its left subtree (the "weight"). Concatenation of two ropes: create a new root node with the two ropes as children — O(1) if no rebalancing, O(log n) with rebalancing. Substring lookup: traverse the tree using weights to locate the correct leaf — O(log n). Split: walk down using weights, creating new nodes along the split path — O(log n). Ropes avoid the O(n) copy cost of string concatenation (e.g. repeated `str +=` in Python/Java is O(n²)). Used in text editors (Atom, VS Code internals), version-control diff engines, and anywhere with frequent large string mutations. Tree height must be kept O(log n) via rebalancing (AVL or weight-balanced criteria).
A Rope represents a string as a binary tree of shorter string chunks at the leaves. Each internal node stores the total length of its left subtree (the "weight"). Concatenation of two ropes: create a new root node with the two ropes as children — O(1) if no rebalancing, O(log n) with rebalancing. Substring lookup: traverse the tree using weights to locate the correct leaf — O(log n). Split: walk down using weights, creating new nodes along the split path — O(log n). Ropes avoid the O(n) copy cost of string concatenation (e.g. repeated `str +=` in Python/Java is O(n²)). Used in text editors (Atom, VS Code internals), version-control diff engines, and anywhere with frequent large string mutations. Tree height must be kept O(log n) via rebalancing (AVL or weight-balanced criteria).
A suffix automaton (SAM) is the smallest DFA accepting all suffixes of a string S of length n. It has at most 2n-1 states and 3n-4 transitions. Online construction (Ukkonen's SAM algorithm) processes one character at a time in O(n) total time and space. Each state in the SAM corresponds to an equivalence class of ending positions (endpos sets); transitions encode extensions of substrings. The suffix link tree of the SAM is equivalent to the suffix tree's internal nodes. Applications: counting distinct substrings in O(n), finding all occurrences of a pattern in O(m + occurrences), longest common substring of two strings, and string factorisation. SAM is a remarkably compact representation compared to suffix trees, making it dominant in competitive programming and bioinformatics.
A suffix automaton (SAM) is the smallest DFA accepting all suffixes of a string S of length n. It has at most 2n-1 states and 3n-4 transitions. Online construction (Ukkonen's SAM algorithm) processes one character at a time in O(n) total time and space. Each state in the SAM corresponds to an equivalence class of ending positions (endpos sets); transitions encode extensions of substrings. The suffix link tree of the SAM is equivalent to the suffix tree's internal nodes. Applications: counting distinct substrings in O(n), finding all occurrences of a pattern in O(m + occurrences), longest common substring of two strings, and string factorisation. SAM is a remarkably compact representation compared to suffix trees, making it dominant in competitive programming and bioinformatics.
Aho-Corasick builds a finite automaton from a set of patterns (total length m) enabling simultaneous matching of all patterns in a text of length n. Phase 1 (O(m)): insert all patterns into a trie. Phase 2 (O(m)): BFS to compute failure links (longest proper suffix of the current state's string that is also a prefix of some pattern — same as KMP's failure function). Also build output links: follow failure links to collect all patterns ending at a state. Phase 3 (O(n + z)): run the automaton on the text; transitions use goto (trie edge) or failure link. Each character is processed in O(1) amortized. Total O(m + n + z) where z is the number of matches. The original 1975 paper solves keyword-in-text search; modern uses include network intrusion detection (Snort), antivirus signature matching, and DNA database search.
Aho-Corasick builds a finite automaton from a set of patterns (total length m) enabling simultaneous matching of all patterns in a text of length n. Phase 1 (O(m)): insert all patterns into a trie. Phase 2 (O(m)): BFS to compute failure links (longest proper suffix of the current state's string that is also a prefix of some pattern — same as KMP's failure function). Also build output links: follow failure links to collect all patterns ending at a state. Phase 3 (O(n + z)): run the automaton on the text; transitions use goto (trie edge) or failure link. Each character is processed in O(1) amortized. Total O(m + n + z) where z is the number of matches. The original 1975 paper solves keyword-in-text search; modern uses include network intrusion detection (Snort), antivirus signature matching, and DNA database search.
A wavelet tree is built by recursively splitting an array's value range in half and partitioning elements into left (below median) and right (above median) children. Each node stores a bitmap indicating which elements went left. For range k-th smallest query [l, r, k]: at each level, count how many elements in [l, r] went left (using the prefix sum of the node's bitmap); if k ≤ that count, recurse left; else k -= count, recurse right. This traverses O(log V) levels (V = value range) with O(1) work per level using precomputed bitmap rank/select. Total O(log V) per query, O(n log V) build time and space. Also supports range frequency, range k-th element, and range wavelet decomposition for signal processing. A practical alternative to merge sort trees for offline range quantile queries.
A wavelet tree is built by recursively splitting an array's value range in half and partitioning elements into left (below median) and right (above median) children. Each node stores a bitmap indicating which elements went left. For range k-th smallest query [l, r, k]: at each level, count how many elements in [l, r] went left (using the prefix sum of the node's bitmap); if k ≤ that count, recurse left; else k -= count, recurse right. This traverses O(log V) levels (V = value range) with O(1) work per level using precomputed bitmap rank/select. Total O(log V) per query, O(n log V) build time and space. Also supports range frequency, range k-th element, and range wavelet decomposition for signal processing. A practical alternative to merge sort trees for offline range quantile queries.
HLD decomposes a tree into chains of "heavy edges" (connecting a node to its child with the largest subtree). Each node belongs to exactly one chain; there are O(log n) chains on any root-to-leaf path (light edges are taken at most O(log n) times because crossing a light edge at least doubles the subtree size). Map each chain to a contiguous segment of a 1D array, then build a segment tree over this array. A path query from u to v: traverse up to the LCA, processing O(log n) chain segments, each requiring one segment tree query (O(log n)) → total O(log² n). HLD supports path sum, path maximum, path update in O(log² n) and is used in competitive programming for problems on tree paths. With a Fenwick tree instead of a segment tree, constants are smaller but range updates require extra care.
HLD decomposes a tree into chains of "heavy edges" (connecting a node to its child with the largest subtree). Each node belongs to exactly one chain; there are O(log n) chains on any root-to-leaf path (light edges are taken at most O(log n) times because crossing a light edge at least doubles the subtree size). Map each chain to a contiguous segment of a 1D array, then build a segment tree over this array. A path query from u to v: traverse up to the LCA, processing O(log n) chain segments, each requiring one segment tree query (O(log n)) → total O(log² n). HLD supports path sum, path maximum, path update in O(log² n) and is used in competitive programming for problems on tree paths. With a Fenwick tree instead of a segment tree, constants are smaller but range updates require extra care.
Link-Cut trees (Sleator & Tarjan, 1983) are a forest of splay trees representing a dynamic forest that supports link (add edge), cut (remove edge), and path queries (sum/max on root path) in O(log n) amortized per operation. The key idea is "preferred paths": each node has at most one preferred child, forming preferred paths stored as splay trees. access(v) splays v within its preferred path and re-roots to expose the path to the root. Link and cut manipulate preferred paths in O(log n) amortized. Applications: dynamic connectivity, dynamic minimum spanning tree (together with Euler Tour Trees), offline LCA, and computing maximum flows (via dynamic tree augmentation in Sleator-Tarjan O(nm log n) max-flow). More powerful than HLD but also significantly more complex to implement.
Link-Cut trees (Sleator & Tarjan, 1983) are a forest of splay trees representing a dynamic forest that supports link (add edge), cut (remove edge), and path queries (sum/max on root path) in O(log n) amortized per operation. The key idea is "preferred paths": each node has at most one preferred child, forming preferred paths stored as splay trees. access(v) splays v within its preferred path and re-roots to expose the path to the root. Link and cut manipulate preferred paths in O(log n) amortized. Applications: dynamic connectivity, dynamic minimum spanning tree (together with Euler Tour Trees), offline LCA, and computing maximum flows (via dynamic tree augmentation in Sleator-Tarjan O(nm log n) max-flow). More powerful than HLD but also significantly more complex to implement.
Sqrt decomposition partitions an array of n elements into blocks of size √n ≈ √n. Each block maintains a precomputed aggregate (sum, min, max). Range query [l, r]: directly iterate partial left block, sum precomputed values for full middle blocks, iterate partial right block — O(√n). Point update: update the element and recompute its block aggregate — O(√n) or O(1) with lazy tags. For range updates with lazy propagation: mark fully covered blocks with a lazy delta and apply it at query time for partial blocks — O(√n) per operation. Easier to implement than segment trees, competitive in practice for n ≤ 10⁵, and the basis for Mo's algorithm. Block size can be tuned: for heavy updates use larger blocks; for heavy queries use smaller ones.
Sqrt decomposition partitions an array of n elements into blocks of size √n ≈ √n. Each block maintains a precomputed aggregate (sum, min, max). Range query [l, r]: directly iterate partial left block, sum precomputed values for full middle blocks, iterate partial right block — O(√n). Point update: update the element and recompute its block aggregate — O(√n) or O(1) with lazy tags. For range updates with lazy propagation: mark fully covered blocks with a lazy delta and apply it at query time for partial blocks — O(√n) per operation. Easier to implement than segment trees, competitive in practice for n ≤ 10⁵, and the basis for Mo's algorithm. Block size can be tuned: for heavy updates use larger blocks; for heavy queries use smaller ones.
Mo's algorithm answers a batch of range queries [l, r] offline by sorting them to minimise total pointer movement. Divide the array into blocks of size √n. Sort queries by (block of l, r if same block with ascending r, else alternating r direction for even/odd blocks to reduce constant). Process queries in this order, maintaining a running answer as l and r pointers move. Add/remove an element at each step (O(1) per element). Total pointer moves: O(q√n) for left pointer (jumps between blocks) and O(n√n) for right pointer (sweeps monotonically within each block group). Total O((n+q)√n). Used for offline range mode, range distinct count, range inversions — anything where incremental add/remove is O(1) but random-access computation is expensive. Mo on trees extends this to tree paths via Euler tour.
Mo's algorithm answers a batch of range queries [l, r] offline by sorting them to minimise total pointer movement. Divide the array into blocks of size √n. Sort queries by (block of l, r if same block with ascending r, else alternating r direction for even/odd blocks to reduce constant). Process queries in this order, maintaining a running answer as l and r pointers move. Add/remove an element at each step (O(1) per element). Total pointer moves: O(q√n) for left pointer (jumps between blocks) and O(n√n) for right pointer (sweeps monotonically within each block group). Total O((n+q)√n). Used for offline range mode, range distinct count, range inversions — anything where incremental add/remove is O(1) but random-access computation is expensive. Mo on trees extends this to tree paths via Euler tour.
Kinetic data structures maintain combinatorial properties (e.g. convex hull, maximum, k-d tree) over a set of objects moving continuously as a function of time. They use a priority queue of "certificates" — conditions that, while true, ensure the data structure is valid. When a certificate expires (an event), the structure is locally updated and new certificates are scheduled. For kinetic maximum over n points moving with linear trajectories: each maximum changes when lines intersect. The certificate queue holds intersection events; each event triggers O(log n) work. Total events: O(n²) in the worst case but O(n polylog n) for random motion. Kinetic data structures are used in computational geometry, collision detection in game physics, and GPS-based fleet management systems.
Kinetic data structures maintain combinatorial properties (e.g. convex hull, maximum, k-d tree) over a set of objects moving continuously as a function of time. They use a priority queue of "certificates" — conditions that, while true, ensure the data structure is valid. When a certificate expires (an event), the structure is locally updated and new certificates are scheduled. For kinetic maximum over n points moving with linear trajectories: each maximum changes when lines intersect. The certificate queue holds intersection events; each event triggers O(log n) work. Total events: O(n²) in the worst case but O(n polylog n) for random motion. Kinetic data structures are used in computational geometry, collision detection in game physics, and GPS-based fleet management systems.
A Bloom filter uses k independent hash functions mapping items into an m-bit array. Insert: set bits at all k hash positions. Query: if any bit is 0, definitely not present; if all 1s, probably present. False positive probability: p ≈ (1 - e^(-kn/m))^k, minimised by choosing k = (m/n) ln 2, giving p ≈ (0.6185)^(m/n). Space is O(m) bits — 10 bits/element yields ~1% FPR. False negatives are impossible. A counting Bloom filter replaces each bit with a small counter (typically 4 bits), enabling deletion by decrementing. The downside is false delete cascades if overflow occurs (4-bit counters overflow at 16 insertions to the same cell — rare). Used in: database query optimisers (avoid disk reads for definitely-absent keys), CDN caches, distributed systems (distributed deduplication), and spell checkers.
A Bloom filter uses k independent hash functions mapping items into an m-bit array. Insert: set bits at all k hash positions. Query: if any bit is 0, definitely not present; if all 1s, probably present. False positive probability: p ≈ (1 - e^(-kn/m))^k, minimised by choosing k = (m/n) ln 2, giving p ≈ (0.6185)^(m/n). Space is O(m) bits — 10 bits/element yields ~1% FPR. False negatives are impossible. A counting Bloom filter replaces each bit with a small counter (typically 4 bits), enabling deletion by decrementing. The downside is false delete cascades if overflow occurs (4-bit counters overflow at 16 insertions to the same cell — rare). Used in: database query optimisers (avoid disk reads for definitely-absent keys), CDN caches, distributed systems (distributed deduplication), and spell checkers.
Count-Min sketch (Cormode & Muthukrishnan, 2003) estimates the frequency of any item in a stream with (ε, δ) guarantees. Use d = ⌈ln(1/δ)⌉ hash functions and w = ⌈e/ε⌉ counters per row (total d×w array). Increment: for each row i, increment count[i][h_i(x)]. Query: return min over all rows of count[i][h_i(x)]. The estimate always overestimates true frequency; error ≤ ε·N with probability 1-δ (N = stream length). Space: O((e/ε) log(1/δ)) — e.g. ε=0.01, δ=0.01 → ~500 counters, independent of stream size. Used in network traffic analysis (top-k heavy hitters), approximate frequency counting in databases, and real-time analytics where exact counts are too memory-expensive.
Count-Min sketch (Cormode & Muthukrishnan, 2003) estimates the frequency of any item in a stream with (ε, δ) guarantees. Use d = ⌈ln(1/δ)⌉ hash functions and w = ⌈e/ε⌉ counters per row (total d×w array). Increment: for each row i, increment count[i][h_i(x)]. Query: return min over all rows of count[i][h_i(x)]. The estimate always overestimates true frequency; error ≤ ε·N with probability 1-δ (N = stream length). Space: O((e/ε) log(1/δ)) — e.g. ε=0.01, δ=0.01 → ~500 counters, independent of stream size. Used in network traffic analysis (top-k heavy hitters), approximate frequency counting in databases, and real-time analytics where exact counts are too memory-expensive.
HyperLogLog estimates the number of distinct elements in a stream using O(log log n + log(1/ε)) bits. Hash each element to a uniform binary string. Split the hash: the leading b bits select one of 2^b registers; the remaining bits determine the position of the leftmost 1 (max run of leading zeros + 1), stored in the register as max(register, rank). Estimation: harmonic mean of 2^register[i] values × constant α × (2^b)². Error: ~1.04/√(2^b) — 1.6% error with b=10 (1024 registers, ~5KB). The harmonic mean corrects for overestimation from hash collisions. Corrections for small counts (linear counting) and large counts (near-full hash space) are applied. Redis uses HyperLogLog for `PFCOUNT`. The algorithm achieves the theoretically optimal memory-accuracy tradeoff for cardinality estimation.
HyperLogLog estimates the number of distinct elements in a stream using O(log log n + log(1/ε)) bits. Hash each element to a uniform binary string. Split the hash: the leading b bits select one of 2^b registers; the remaining bits determine the position of the leftmost 1 (max run of leading zeros + 1), stored in the register as max(register, rank). Estimation: harmonic mean of 2^register[i] values × constant α × (2^b)². Error: ~1.04/√(2^b) — 1.6% error with b=10 (1024 registers, ~5KB). The harmonic mean corrects for overestimation from hash collisions. Corrections for small counts (linear counting) and large counts (near-full hash space) are applied. Redis uses HyperLogLog for `PFCOUNT`. The algorithm achieves the theoretically optimal memory-accuracy tradeoff for cardinality estimation.
An LSM-tree (Log-Structured Merge-tree) optimises write throughput by turning random writes into sequential I/O. Write path: (1) Append to an in-memory write-ahead log (WAL) for durability. (2) Insert into an in-memory sorted structure (MemTable, typically a skip list or red-black tree). (3) When MemTable reaches a size threshold (~64MB), flush it as an immutable SSTable file to Level-0 on disk. (4) Background compaction merges Level-L SSTables into Level-(L+1), maintaining sorted order and discarding overwritten/deleted keys. Read path: check MemTable → L0 files (may overlap, search all) → L1+ (non-overlapping, binary search for key). Bloom filters per SSTable avoid reading files that can't contain the key. Trade-off: write amplification (a key may be rewritten O(L) times during compaction) vs dramatically higher write throughput than B-trees for write-heavy workloads.
An LSM-tree (Log-Structured Merge-tree) optimises write throughput by turning random writes into sequential I/O. Write path: (1) Append to an in-memory write-ahead log (WAL) for durability. (2) Insert into an in-memory sorted structure (MemTable, typically a skip list or red-black tree). (3) When MemTable reaches a size threshold (~64MB), flush it as an immutable SSTable file to Level-0 on disk. (4) Background compaction merges Level-L SSTables into Level-(L+1), maintaining sorted order and discarding overwritten/deleted keys. Read path: check MemTable → L0 files (may overlap, search all) → L1+ (non-overlapping, binary search for key). Bloom filters per SSTable avoid reading files that can't contain the key. Trade-off: write amplification (a key may be rewritten O(L) times during compaction) vs dramatically higher write throughput than B-trees for write-heavy workloads.
B-epsilon trees (Brodal & Fagerberg, 1996; popularised by Bender et al.) are write-optimised trees where internal nodes have B keys but store only ε·B outgoing pointers (ε < 1), leaving space for a message buffer of (1-ε)·B messages. Insert/delete operations are applied as "messages" buffered in the root and flushed down lazily during background compaction. A flush sends a buffer of Θ(B^ε) messages in one I/O, amortising write cost. Insert cost: O(log_B n / B^(1-ε)) I/Os — much cheaper than B-tree O(log_B n). Read cost: O(log_B n) + buffer traversal overhead. TokuDB (now in MariaDB) and BetrFS implement B-epsilon trees. They can achieve 10-100× write throughput improvement over B-trees at the same hardware, at the cost of slightly higher read latency during heavy write bursts.
B-epsilon trees (Brodal & Fagerberg, 1996; popularised by Bender et al.) are write-optimised trees where internal nodes have B keys but store only ε·B outgoing pointers (ε < 1), leaving space for a message buffer of (1-ε)·B messages. Insert/delete operations are applied as "messages" buffered in the root and flushed down lazily during background compaction. A flush sends a buffer of Θ(B^ε) messages in one I/O, amortising write cost. Insert cost: O(log_B n / B^(1-ε)) I/Os — much cheaper than B-tree O(log_B n). Read cost: O(log_B n) + buffer traversal overhead. TokuDB (now in MariaDB) and BetrFS implement B-epsilon trees. They can achieve 10-100× write throughput improvement over B-trees at the same hardware, at the cost of slightly higher read latency during heavy write bursts.
A KD-tree (k-dimensional tree) partitions a k-dimensional space by alternating split dimensions. Build: at each level, choose the dimension with the widest spread, split at the median, and recurse — O(n log n). Nearest-neighbor query: recurse into the closer subtree first; backtrack and check the other subtree only if the hypersphere of radius (current_best) intersects the splitting hyperplane. In low dimensions (k ≤ 10), average query cost is O(log n). The curse of dimensionality: as k grows, all points become approximately equidistant in high-dimensional spaces. The backtracking condition is almost never satisfied, making query time approach O(n). For k > 20, approximate methods (locality-sensitive hashing, ball trees, HNSW for embeddings) outperform KD-trees. KD-trees are used in 2D/3D collision detection, PCL (Point Cloud Library), and k-NN in low-dimensional feature spaces.
A KD-tree (k-dimensional tree) partitions a k-dimensional space by alternating split dimensions. Build: at each level, choose the dimension with the widest spread, split at the median, and recurse — O(n log n). Nearest-neighbor query: recurse into the closer subtree first; backtrack and check the other subtree only if the hypersphere of radius (current_best) intersects the splitting hyperplane. In low dimensions (k ≤ 10), average query cost is O(log n). The curse of dimensionality: as k grows, all points become approximately equidistant in high-dimensional spaces. The backtracking condition is almost never satisfied, making query time approach O(n). For k > 20, approximate methods (locality-sensitive hashing, ball trees, HNSW for embeddings) outperform KD-trees. KD-trees are used in 2D/3D collision detection, PCL (Point Cloud Library), and k-NN in low-dimensional feature spaces.
An R-tree indexes multi-dimensional rectangles (or other bounding shapes) by grouping nearby objects into minimum bounding rectangles (MBRs) arranged in a balanced tree. Each node contains up to M entries; each entry is (MBR, pointer to child/object). Search: traverse nodes whose MBR overlaps the query region — unlike B-trees, multiple subtrees may need to be explored (MBRs can overlap). Insert: choose the subtree that requires the least MBR enlargement; if a node overflows (M+1 entries), split it using a heuristic (quadratic split, R*-tree linear split). Query performance is O(log n) on average for non-overlapping data but degrades with high MBR overlap. PostgreSQL's PostGIS, SQLite's SpatiaLite, and MySQL spatial indexes use R-trees. The R*-tree variant improves on R-tree by minimising overlap during insertion.
An R-tree indexes multi-dimensional rectangles (or other bounding shapes) by grouping nearby objects into minimum bounding rectangles (MBRs) arranged in a balanced tree. Each node contains up to M entries; each entry is (MBR, pointer to child/object). Search: traverse nodes whose MBR overlaps the query region — unlike B-trees, multiple subtrees may need to be explored (MBRs can overlap). Insert: choose the subtree that requires the least MBR enlargement; if a node overflows (M+1 entries), split it using a heuristic (quadratic split, R*-tree linear split). Query performance is O(log n) on average for non-overlapping data but degrades with high MBR overlap. PostgreSQL's PostGIS, SQLite's SpatiaLite, and MySQL spatial indexes use R-trees. The R*-tree variant improves on R-tree by minimising overlap during insertion.
Consistent hashing maps both keys and servers to points on a 360° ring using the same hash function. A key is assigned to the first server clockwise. Adding/removing a server only remaps O(n/S) keys (n keys, S servers) instead of O(n) in classic modulo hashing. Virtual nodes: each physical server is assigned V = O(log S) virtual positions on the ring. This ensures that when a server fails, its keys are distributed evenly across all remaining servers rather than all going to one successor. Load distribution proof: with V virtual nodes per server, by the Chernoff bound, the maximum load on any server is O((n/S) log S) with high probability — within a constant factor of ideal n/S. Used by Amazon DynamoDB, Apache Cassandra, and Riak for sharding and replication.
Consistent hashing maps both keys and servers to points on a 360° ring using the same hash function. A key is assigned to the first server clockwise. Adding/removing a server only remaps O(n/S) keys (n keys, S servers) instead of O(n) in classic modulo hashing. Virtual nodes: each physical server is assigned V = O(log S) virtual positions on the ring. This ensures that when a server fails, its keys are distributed evenly across all remaining servers rather than all going to one successor. Load distribution proof: with V virtual nodes per server, by the Chernoff bound, the maximum load on any server is O((n/S) log S) with high probability — within a constant factor of ideal n/S. Used by Amazon DynamoDB, Apache Cassandra, and Riak for sharding and replication.
A Treap is a BST where each node has a key (satisfying BST order) and a random priority (satisfying max-heap order). By uniqueness: for any set of (key, priority) pairs, there is exactly one Treap shape. Since priorities are random, the Treap has the same shape as a BST built by inserting keys in random priority order, which has expected height O(log n) (same argument as randomised quicksort). Insert: BST insert, then rotate up until heap property is restored — O(log n) expected. Delete: rotate node down until it becomes a leaf, then remove — O(log n) expected. Treaps are simpler to implement than AVL/red-black trees and support split/merge in O(log n), making them useful for persistent and functional implementations.
A Treap is a BST where each node has a key (satisfying BST order) and a random priority (satisfying max-heap order). By uniqueness: for any set of (key, priority) pairs, there is exactly one Treap shape. Since priorities are random, the Treap has the same shape as a BST built by inserting keys in random priority order, which has expected height O(log n) (same argument as randomised quicksort). Insert: BST insert, then rotate up until heap property is restored — O(log n) expected. Delete: rotate node down until it becomes a leaf, then remove — O(log n) expected. Treaps are simpler to implement than AVL/red-black trees and support split/merge in O(log n), making them useful for persistent and functional implementations.
A splay tree (Sleator & Tarjan, 1985) is a self-adjusting BST where every access operation splays the accessed node to the root via a sequence of rotations (zig, zig-zig, zig-zag). No balance invariant is explicitly maintained. Amortized O(log n) per operation is proven via a potential function Φ = Σ log(size(v)) (sum of log-sizes). Each zig-zig rotation increases the potential by at most the decrease in rotated subtree size, bounding total work. The working set theorem: if item x was last accessed t operations ago, accessing it costs O(log t) amortized — splay trees adapt to the access pattern automatically. This makes them excellent for non-uniform access patterns (caches, recently-accessed data). Drawback: O(n) worst-case single operation, and not cache-friendly due to rotations changing tree shape on every access.
A splay tree (Sleator & Tarjan, 1985) is a self-adjusting BST where every access operation splays the accessed node to the root via a sequence of rotations (zig, zig-zig, zig-zag). No balance invariant is explicitly maintained. Amortized O(log n) per operation is proven via a potential function Φ = Σ log(size(v)) (sum of log-sizes). Each zig-zig rotation increases the potential by at most the decrease in rotated subtree size, bounding total work. The working set theorem: if item x was last accessed t operations ago, accessing it costs O(log t) amortized — splay trees adapt to the access pattern automatically. This makes them excellent for non-uniform access patterns (caches, recently-accessed data). Drawback: O(n) worst-case single operation, and not cache-friendly due to rotations changing tree shape on every access.
X-fast trie stores all prefixes of keys in a hash table at each level of a binary trie over a universe of size U. Predecessor query: binary search over the O(log U) levels of the trie to find the deepest level where a prefix of the query exists — O(log log U) via binary search on levels. Space O(n log U) — expensive. Y-fast trie improves space to O(n) by using a two-level design: an X-fast trie over O(n / log U) representative keys (one per group of log U keys), with each group stored in a balanced BST. Predecessor query: O(log log U) in the X-fast layer plus O(log log U) in the BST layer = O(log log U) total. Space O(n). These structures are theoretically important — they break the Ω(log n) BST lower bound using the integer key structure — but rarely used in practice due to high constants and complexity.
X-fast trie stores all prefixes of keys in a hash table at each level of a binary trie over a universe of size U. Predecessor query: binary search over the O(log U) levels of the trie to find the deepest level where a prefix of the query exists — O(log log U) via binary search on levels. Space O(n log U) — expensive. Y-fast trie improves space to O(n) by using a two-level design: an X-fast trie over O(n / log U) representative keys (one per group of log U keys), with each group stored in a balanced BST. Predecessor query: O(log log U) in the X-fast layer plus O(log log U) in the BST layer = O(log log U) total. Space O(n). These structures are theoretically important — they break the Ω(log n) BST lower bound using the integer key structure — but rarely used in practice due to high constants and complexity.
Static connectivity asks "are u and v connected?" in a fixed graph — answerable in O(α(n)) with DSU. Dynamic connectivity maintains a forest under link (add edge) and cut (remove edge) queries, answering connectivity between any two nodes after each update. Link-cut trees support link and cut in O(log n) amortized, and connectivity queries in O(log n). For undirected dynamic graphs (not just forests), the Holm-de Lichtenberg-Thorup (HDT) algorithm uses a hierarchy of spanning forests across O(log n) levels, handling edge deletions in O(log² n) amortized. Offline dynamic connectivity (all queries known in advance) uses divide-and-conquer on the time axis with DSU rollback in O((n + q) log n log α(n)). Applications: dynamic road networks, social graph connectivity under friend add/remove.
Static connectivity asks "are u and v connected?" in a fixed graph — answerable in O(α(n)) with DSU. Dynamic connectivity maintains a forest under link (add edge) and cut (remove edge) queries, answering connectivity between any two nodes after each update. Link-cut trees support link and cut in O(log n) amortized, and connectivity queries in O(log n). For undirected dynamic graphs (not just forests), the Holm-de Lichtenberg-Thorup (HDT) algorithm uses a hierarchy of spanning forests across O(log n) levels, handling edge deletions in O(log² n) amortized. Offline dynamic connectivity (all queries known in advance) uses divide-and-conquer on the time axis with DSU rollback in O((n + q) log n log α(n)). Applications: dynamic road networks, social graph connectivity under friend add/remove.
Union-Find with rollback supports undo of the last union operation by using union by rank without path compression (path compression destroys undo history). Keep a stack of (node, old_parent, old_rank) tuples. Each union pushes its changes; rollback pops them. Union by rank alone (no path compression) guarantees O(log n) per find, giving O(log n) per operation. For offline dynamic connectivity using divide-and-conquer on time: build a segment tree over the time interval [0, T]; each edge is active during an interval, added to O(log T) segment tree nodes. DFS the segment tree, unioning edges at each node and rolling back on exit. A connectivity query at time t is answered when the DFS reaches the leaf for t. Total: O((n + q) log T log n). This avoids the complexity of link-cut trees for offline scenarios.
Union-Find with rollback supports undo of the last union operation by using union by rank without path compression (path compression destroys undo history). Keep a stack of (node, old_parent, old_rank) tuples. Each union pushes its changes; rollback pops them. Union by rank alone (no path compression) guarantees O(log n) per find, giving O(log n) per operation. For offline dynamic connectivity using divide-and-conquer on time: build a segment tree over the time interval [0, T]; each edge is active during an interval, added to O(log T) segment tree nodes. DFS the segment tree, unioning edges at each node and rolling back on exit. A connectivity query at time t is answered when the DFS reaches the leaf for t. Total: O((n + q) log T log n). This avoids the complexity of link-cut trees for offline scenarios.
The external memory (EM) model (Aggarwal & Vitter, 1988) counts disk block transfers as the cost metric, where each I/O reads or writes B consecutive words, the disk has unlimited capacity, and main memory holds M words. I/O complexity of sorting n elements: Θ((n/B) log_{M/B}(n/B)) I/Os — achieved by merge sort with M/B-way merging, far better than the O((n/B) log(n/B)) of binary merge sort. This is the optimal sorting bound in the EM model. B-tree search: O(log_B n) I/Os — each node is exactly one block, so each level consumes exactly one I/O. Compare to a binary BST: O(log n) I/Os (each comparison is a separate block read). This O(log n / log B) speedup (factor ~1000 for B=1000) explains why B-trees dominate database indexing.
The external memory (EM) model (Aggarwal & Vitter, 1988) counts disk block transfers as the cost metric, where each I/O reads or writes B consecutive words, the disk has unlimited capacity, and main memory holds M words. I/O complexity of sorting n elements: Θ((n/B) log_{M/B}(n/B)) I/Os — achieved by merge sort with M/B-way merging, far better than the O((n/B) log(n/B)) of binary merge sort. This is the optimal sorting bound in the EM model. B-tree search: O(log_B n) I/Os — each node is exactly one block, so each level consumes exactly one I/O. Compare to a binary BST: O(log n) I/Os (each comparison is a separate block read). This O(log n / log B) speedup (factor ~1000 for B=1000) explains why B-trees dominate database indexing.
The PRAM (Parallel RAM) model has P processors sharing a global memory. Work W = total operations across all processors; Span S (depth) = longest chain of sequential dependencies (critical path). By Brent's scheduling theorem, actual time ≤ W/P + S. Parallel merge sort: split arrays in half recursively (O(1) span per level), then parallel merge using a binary-search-based merge. Parallel merge of two arrays of total size n: binary search to find the split point of one array relative to the other — O(log n) span; recurse on two halves in parallel. Span recurrence: S(n) = S(n/2) + O(log n) → S(n) = O(log² n). Work: W(n) = O(n log n) — same as sequential. Total time with P processors: O(n log n / P + log² n). For P = n/log n, time is O(log³ n).
The PRAM (Parallel RAM) model has P processors sharing a global memory. Work W = total operations across all processors; Span S (depth) = longest chain of sequential dependencies (critical path). By Brent's scheduling theorem, actual time ≤ W/P + S. Parallel merge sort: split arrays in half recursively (O(1) span per level), then parallel merge using a binary-search-based merge. Parallel merge of two arrays of total size n: binary search to find the split point of one array relative to the other — O(log n) span; recurse on two halves in parallel. Span recurrence: S(n) = S(n/2) + O(log n) → S(n) = O(log² n). Work: W(n) = O(n log n) — same as sequential. Total time with P processors: O(n log n / P + log² n). For P = n/log n, time is O(log³ n).
Classical unstructured search requires Ω(n) queries in the worst case — every element must be checked. Grover's algorithm (1996) searches an unsorted database of n items using O(√n) quantum oracle calls with high probability. Intuition: initialise a superposition over all n states with equal amplitude 1/√n. Each Grover iteration (oracle query + diffusion step) amplifies the amplitude of the target state by O(1/√n) and slightly reduces all others. After O(√n) iterations, the target state has amplitude ~1 (probability ~1). Measuring collapses the superposition to the target. The diffusion operator is an inversion about the mean of all amplitudes — a geometric rotation in the Hilbert space by angle θ ≈ 2/√n; after √n rotations, total angle ≈ π/2 (pointing at the target). Grover's is provably optimal — no quantum algorithm can do better than O(√n) for unstructured search (Bennett et al., 1997). Applications: speedup for NP-complete problems (not exponential, only quadratic), and cryptographic key search.
Classical unstructured search requires Ω(n) queries in the worst case — every element must be checked. Grover's algorithm (1996) searches an unsorted database of n items using O(√n) quantum oracle calls with high probability. Intuition: initialise a superposition over all n states with equal amplitude 1/√n. Each Grover iteration (oracle query + diffusion step) amplifies the amplitude of the target state by O(1/√n) and slightly reduces all others. After O(√n) iterations, the target state has amplitude ~1 (probability ~1). Measuring collapses the superposition to the target. The diffusion operator is an inversion about the mean of all amplitudes — a geometric rotation in the Hilbert space by angle θ ≈ 2/√n; after √n rotations, total angle ≈ π/2 (pointing at the target). Grover's is provably optimal — no quantum algorithm can do better than O(√n) for unstructured search (Bennett et al., 1997). Applications: speedup for NP-complete problems (not exponential, only quadratic), and cryptographic key search.
Frequently Asked Questions
Which data structures are asked most?
Arrays, hashmaps, strings, linked lists, trees (especially BST and tries), heaps, and graphs. Master these in your preferred language.
How important are Big-O tradeoffs?
Essential. You should be able to state time and space complexity for every solution and justify trade-offs.
Do I need to implement every DS from scratch?
Know implementations of linked list, BST, heap, trie, and union-find. For others, knowing the interface and complexity is usually enough.
How much LeetCode?
For FAANG targets, 200–400 problems across all patterns. Prioritize quality over quantity — understand one problem deeply before moving on.
What if I blank during an interview?
Verbalize your thought process, start with brute force, think out loud, and ask clarifying questions. Interviewers value process over perfect answers.