Dynamic programming can be thought of as being the reverse of recursion. Recursion is a top-down mechanism -- we take a problem, split it up, and solve the smaller problems that are created. Dynamic programming is a bottom-up mechanism -- we solve all possible small problems and then combine them to obtain solutions for bigger problems.

The reason that this may be better is that, using recursion, it is possible that we may solve the same small subproblem many times. Using dynamic programming, we solve it once.

The order of which matrices are multiplied together first can significantly affect the time required.

To multiply M×N,
where matrix M is *p×q* and matrix N is *q×r*,
takes *pqr* operations if we use the "normal" matrix
multiplication algorithm.
Note that the matrices have a common dimension value of *q*.
This makes the matrices have the property of *compatibility*,
without which it would not be possible for them to be multiplied.
Also note that, while matrix multiplication is associative,
matrix multiplication is not commutative.
That is, N×M might not equal M×N and, in fact,
N×M might not even be defined because of a lack of compatibility.

__ EXAMPLE__:

Calculate M = M

where the dimensions of the matrices are

M

Calculating M = M_{1} × ( M_{2}
× ( M_{3} × M_{4} ) )
requires 125000 operations.

Calculating M = ( M_{1} × ( M_{2}
× M_{3} ) ) × M_{4}
requires 2200 operations.

We could figure out how many operations each possible
order will take and then use the one having the minimum
number of operations,
but there are there are an exponential number of orderings.
Any of the *n-*1 multiplications could be first
and then any of the remaining *n-*2 multiplications could be next
and so on, leading to a total of (*n-*1)! orderings.

We can find the best order in time *O*(*n*^{3})
by using dynamic programming.

If *m _{i ,j}* is the minimum cost of evaluating
the product M

__The algorithm__:

fori := 1tondom_{i,i}:= 0forlength := 1ton-1dofori := 1ton-lengthdoj := i + length m_{i,j}= MIN_{i≤k<j}{m_{i,k}+ m_{k+1,j}+ r_{i-1}r_{k}r_{j}}

In the above listing, `length` refers to the number of
matrix multiplications in a subproblem.
An alternative approach would be to use `size`
(equal to `length+1`)
as the number of matrices in the subproblem.

For the example given above, we would calculate:

*m*_{1,1} = 0,
*m*_{2,2} = 0,
*m*_{3,3} = 0,
*m*_{4,4} = 0

*m*_{1,2} = 10000,
*m*_{2,3} = 1000,
*m*_{3,4} = 5000

*m*_{1,3} = 1200,
*m*_{2,4} = 3000

*m*_{1,4} = 2200

find the longest contiguous sequence appearing in both.

__One solution__:
(assume strings have lengths *m* and *n*)

For each of the *m* starting points of *A*,
check for the longest common string starting at each of the
*n* starting points of *B*.

The checks could average Θ(*m*) time
→ a total of Θ(*m*^{2}*n*) time.

__Dynamic programming solution__:

Let L_{i, j} = maximum length of common strings
that end at *A*[*i*] & *B*[*j*].
Then,

*A*[*i*] = *B*[*j*] →
L_{i, j} = 1 + L_{i-1, j-1}

*A*[*i*] ≠ *B*[*j*] →
L_{i, j} = 0

LONGEST COMMON SUBSTRING(A,m,B,n)fori := 0tomdoL_{i,0}:= 0forj := 0tondoL_{0,j}:= 0 len := 0 answer := <0,0>fori := 1tomdoforj := 1tondoifA_{i}≠ B_{j}thenL_{i,j}:= 0elseL_{i,j}:= 1 + L_{i-1,j-1}ifL_{i,j}> lenthenlen := L_{i,j}answer = <i,j>

__Example__:

A L O H A H 0 0 0 1 0 E 0 0 0 0 0 L 0 1 0 0 0 L 0 1 0 0 0 O 0 0 2 0 0

__Example__:

`houseboat`

`houseboat`

`ousbo` is a subsequence of `houseboat`

String *C* is a *common subsequence* of strings
*A* and *B* if *C* is a subsequence of *A*
and also *C* is a subsequence of *B*.

__Example__:

`houseboat`

`computer`

`houseboat`

`computer`

`out` is a common subsequence of `houseboat` and `computer`

String *C* is a *longest common subsequence* (LCS)
of strings *A* and *B* if
*C* is a common subsequence of *A* & *B*
and there is no other common subsequence of *A* & *B*
that has greater length.

Let L_{i, j} be the length of an LCS of
*A*[1...*i*] & *B*[1...*j*],
*i.e.*, the prefixes of strings *A* & *B*
of lengths *i* and *j*.

L_{i, j} =
L_{i-1, j-1} + 1,
if *A _{i}* =

L

LONGEST COMMON SUBSEQUENCE(A,m,B,n)fori := 0tomdoL_{i,0}:= 0forj := 0tondoL_{0,j}:= 0fori := 1tomdoforj := 1tondoifA_{i}= B_{j}thenL_{i,j}:= 1 + L_{i-1,j-1}elseL_{i,j}:= Max{ L_{i-1,j}, L_{i,j-1}} length := L_{m,n}

Dan Hirschberg

Last modified: Oct 14, 2005