The Knapsack problem

0/1 Knapsack Problem

Given n objects, each having a weight wi and a value vi, we wish to fill a knapsack with objects so as to maximize the total value, but under the constraint that the total weight is at most maxw. That is,

      maximize ∑i = 1 to n ( xi vi )
      such that  ∑i = 1 to n ( xi wi ) ≤ maxw,   where xi in {0, 1}

Note that this is an NP-complete problem and so, probably, there is no polynomial-time solution.

However, here is a pseudo-polynomial-time solution (term explained later), using dynamic programming.

Let V[i, w] be the maximum value obtainable when choosing from objects 1 through i and which has total weight at most w,
where 0 ≤ i ≤ n and 0 ≤ w ≤ maxw.

The solution value for the problem is V[n, maxw].

Recurrence:
if w < wi then V[i, w] = V[i-1, w] can't use object i
else V[i, w] = max{ V[i-1, w],   don't use object i
V[i-1, w - wi] + vi }   use object i

Boundary conditions:
for 0 ≤ w ≤ maxw, V[0, w] = 0   no value with no objects
for 0 ≤ i ≤ n, V[i, 0] = 0   no value if can't carry any weight 

    Knapsack
       for w := 0 to maxw do
          V[0,w] := 0
       for i := 1 to n do
          for w := 0 to maxw do
             if w < wi then
                V[i,w] := V[i-1,w]
             else
                V[i,w] := max{ V[i-1,w],  V[i-1,w-wi] + vi }
       ans := V[n,maxw]

The time complexity is O(n maxw).
The space complexity is O(n maxw).
Note that since maxw can be quite large, in fact exponential in the size of the input, this is not a polynomial-time solution.   If, as in this case, the complexity is polynomial in the size of the input and numbers appearing within the input then it said that the complexity is a pseudo-polynomial of the input.

In practice, if n = 1000 and maxw = 106 then we might be able to afford the time but not the space.   Here is a way to save space, resulting in time O(n maxw) and space O(maxw). 

    Knapsack
       for w := 0 to maxw do
          V[w] := 0
       for i := 1 to n do
          for w := maxw downto wi do
             V[w] := max{ V[w],  V[w-wi] + vi }
       ans := V[maxw]

Integer Knapsack Problem

Same as before, but now we may take any non-negative integer number of copies of each object.

      maximize ∑i = 1 to n ( xi vi )
      such that  ∑i = 1 to n ( xi wi ) ≤ maxw,   where xi in {0, 1, 2, ...} 

    IntKnapsack
       for w := 0 to maxw do
          V[0,w] := 0
       for i := 1 to n do
          for w := 0 to maxw do
             V[i,w] := V[i-1,w]
             if w ≥ wi then
                for k := 1 to w/wi do
                   V[i,w] := max{ V[i,w],  V[i-1,w-k*wi] + k*vi }
       ans := V[n,maxw]

The time complexity is O(n maxw2).
The space complexity is O(n maxw).

We can reduce the time and space complexities. 

    IntKnapsack
       for w := 0 to maxw do
          V[w] := 0
       for i := 1 to n do
          for w := wi to maxw do
                V[w] := max{ V[w],  V[w-wi] + vi }
       ans := V[maxw]

The time complexity is O(n maxw).
The space complexity is O(maxw).

Dan Hirschberg
Last modified: Aug 24, 2006