Divide-and-conquer theorem

In general, the time complexity of divide-and-conquer algorithms is determined by 3 things: the number and size of the subproblems, and the amount of work needed to bring them together.

If a problem of size n is divided into a problems each of size n/c, and it takes bn work to bring together the solutions of the subproblems, then

T(n) = aT(n/c) + bn, when n > 1
T(n) = b, when n = 1

For real problems, a,b,c are non-negative. In this case, when n is a power of c, we have...

D&C Theorem

	T(n) = Θ( n ) ,	when a < c
	T(n) = Θ( n log n ) ,	when a = c
	T(n) = Θ( n^log_ca ),	when a > c

Proof of the D&C Theorem

Let n = c^m, and therefore m = log_cn. By expanding the recurrence, we get:

T(n) = aT(n/c) + bn
= a[aT(n/c²) + b(n/c)] + bn
= a²T(n/c²) + bn[1 + a/c]
= a²[aT(n/c³) + b(n/c²)] + bn[1 + a/c]
= a³T(n/c³) + bn[1 + a/c + (a/c)²]
. . .
= a^mT(n/c^m) + bn[1 + a/c + (a/c)² + ... + (a/c)^m-1]
( Note that T(1) = b and that a^mb = a^mb(n/c^m) = bn(a/c)^m. )
= bn[1 + a/c + (a/c)² + ... + (a/c)^m].

Therefore, T(n) = bn ∑_{i = 0 to log_cn} (a/c)ⁱ.

When a < c, the summation converges to be less than 2, and so T(n) = O(n).
When a = c, each term in the summation is 1 and there are O(log n) terms. Therefore, T(n) = O(n log n).
When a > c, we have a finite divergent series.
T(n) = [ bn - bn(a/c)^m+1 ] /( 1 - a/c )
= [ bn(a/c)^m+1 - bn ] /( a/c - 1 )
< [ (a/c)bn(a/c)^m ] /( a/c - 1 ) = [ b / (1 - c/a) ]a^m
Thus, T(n) = O(a^m). Also, from above we have that T(n) = Ω(a^m).
Therefore, T(n) = Θ(a^m).
Since a^m = a^log_cn = n^log_ca, we conclude that T(n) = Θ( n^log_ca ).

D&C Theorem when n is not a power of c

The underlying problem can be padded to make a larger-sized problem having the same solution.
We will show that
T(n) ≤ a(n^k), for some constant a, when n = c^m (for integer m)
implies
T(n) ≤ b(n^k), for some constant b, for any n (not just integer powers of c).

Proof. Let n = r c^m, for 1 < r < c.
Then T(n) ≤ T(c^m+1) ≤ a(c^m+1)^k = a((c/r)n)^k = a(c/r)^kn^k
Finally, note that b = a(c/r)^k is a constant.

The Master D&C Theorem

(generalized slightly from CLR page 94)

If T(n) = aT(n/b) + f(n),
where a ≥ 1, b > 1, f(n) asymptotically > 0,
and n/b means either the floor or ceiling of (n/b)

then T(n) =

Θ( n^log_ba ), when f(n) = O ( n^log_ba-ε )
Θ( n^log_ba log^k+1n ), when f(n) = Θ( n^log_ba log^kn ), for k ≥ 0
Θ( f(n) ), when f(n) = Ω( n^log_ba+ε )
and af(n/b) ≤ cf(n), for some c < 1, for all n > n₀

Dan Hirschberg
Last modified: Feb 19, 2004

	Θ( n^log_ba ),	when f(n) = O ( n^log_ba-ε )
	Θ( n^log_ba log^k+1n ),	when f(n) = Θ( n^log_ba log^kn ), for k ≥ 0
	Θ( f(n) ),	when f(n) = Ω( n^log_ba+ε ) and af(n/b) ≤ cf(n), for some c < 1, for all n > n₀