Excerpt from wiki (http://en.wikipedia.org/wiki/Knapsack_problem):
The knapsack problem or rucksack problem is a problem in combinatorial optimization: Given a set of items, each with a mass and a value, determine the number of each item to include in a collection so that the total weight is less than or equal to a given limit and the total value is as large as possible. It derives its name from the problem faced by someone who is constrained by a fixed-size knapsack and must fill it with the most valuable items.
say f[i, w] is the maximum value that can be obtained using up-to 'i' elements so that sum of their weights is at most 'w'
f[i, w] = 0 if i <= 0
0 if w == 0
f[i-1, w] if weights[i] > w
max (f[i-1, w], f[i-1, w-weights[i]] + values[i])
Below is the corresponding code -
The knapsack problem or rucksack problem is a problem in combinatorial optimization: Given a set of items, each with a mass and a value, determine the number of each item to include in a collection so that the total weight is less than or equal to a given limit and the total value is as large as possible. It derives its name from the problem faced by someone who is constrained by a fixed-size knapsack and must fill it with the most valuable items.
Knapsack 0/1
Basically the constraint is an item can be selected at most only once.say f[i, w] is the maximum value that can be obtained using up-to 'i' elements so that sum of their weights is at most 'w'
f[i, w] = 0 if i <= 0
0 if w == 0
f[i-1, w] if weights[i] > w
max (f[i-1, w], f[i-1, w-weights[i]] + values[i])
Unbound Knapsack
basically there is no bound on the number of times an item can be selected
f[i, w] = 0 if i <= 0
0 if w == 0
f[i-1, w] if weights[i] > w
max (f[i-1, w], f[i, w-weights[i]]+ vi)
Using Memoization
Code Snippet
- public int Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy(int[] weights, int[] values, int maxWeight)
- {
- this.ValidataInput_Knapsack_0_1(weights, values, maxWeight);
- int?[][] DP_Memoization_Max_Value_Cache = new int?[values.Length+1][];
- for(int i = 0; i<=values.Length; i++)
- {
- DP_Memoization_Max_Value_Cache[i] = new int?[maxWeight+1];
- for(int j = 0; j <= maxWeight; j++)
- {
- DP_Memoization_Max_Value_Cache[i][j] = null; //yes, its default - just to be defensive
- if(i == 0)
- {
- DP_Memoization_Max_Value_Cache[0][j] = 0;
- }
- }
- //value for weight zero is zero :)
- DP_Memoization_Max_Value_Cache[i][0] = 0;
- }
- var v = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy_Recursive(weights, values,
- index: values.Length, weight: maxWeight,
- DP_Memoization_Max_Value_Cache: DP_Memoization_Max_Value_Cache);
- Assert.IsTrue(v == DP_Memoization_Max_Value_Cache[values.Length][maxWeight]);
- return v;
- }
- /// <summary>
- /// f(i, w) = f(i-1, w) if Wi > w
- ///Or, max (f(i-1, w), f(i-1, w-Wi) + Vi
- ///Or 0 if i < 0
- /// </summary>
- int Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy_Recursive(int[] weights, int[] values,
- int index, int weight, int?[][] DP_Memoization_Max_Value_Cache)
- {
- if (index < 0)
- {
- return 0;
- }
- Debug.Assert(weight >= 0);
- #if DEBUG
- if ((index == 0) || (weight == 0))
- {
- Debug.Assert(DP_Memoization_Max_Value_Cache[index][weight] == 0);
- }
- #endif
- //value is cached, so return
- if(DP_Memoization_Max_Value_Cache[index][weight] != null)
- {
- return DP_Memoization_Max_Value_Cache[index][weight].Value;
- }
- Debug.Assert(index > 0);
- Debug.Assert(weight > 0);
- int maxValueWithoutIncludingCurrentItem = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy_Recursive
- (weights, values, index - 1, weight, DP_Memoization_Max_Value_Cache);
- if (weights[index-1] > weight)
- {
- DP_Memoization_Max_Value_Cache[index][weight] = maxValueWithoutIncludingCurrentItem;
- //current item weight is more, so we cant include - so, just return
- return maxValueWithoutIncludingCurrentItem;
- }
- int overallMaxValue = maxValueWithoutIncludingCurrentItem;
- int maxValueIncludingCurrentItem = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy_Recursive
- (weights, values, index - 1, weight - weights[index-1],
- DP_Memoization_Max_Value_Cache) + values[index - 1];
- if(maxValueIncludingCurrentItem > overallMaxValue)
- {
- overallMaxValue = maxValueIncludingCurrentItem;
- }
- DP_Memoization_Max_Value_Cache[index][weight] = overallMaxValue;
- return overallMaxValue;
- }
Bottom Up (Tabular) - Eager approach
Code Snippet
- public int Knapsack_0_1_DP_Tabular_Bottom_Up(int[] weights, int[] values, int maxWeight)
- {
- this.ValidataInput_Knapsack_0_1(weights, values, maxWeight);
- int[][] DP_Tabular_Eager_Max_Value_Cache = new int[values.Length + 1][];
- for (int i = 0; i <= values.Length; i++)
- {
- DP_Tabular_Eager_Max_Value_Cache[i] = new int[maxWeight + 1];
- for (int j = 0; j <= maxWeight; j++)
- {
- DP_Tabular_Eager_Max_Value_Cache[i][j] = 0; //yes, its default -
- }
- }
- /// f(i, w) = f(i-1, w) if Wi > w
- ///Or, max (f(i-1, w), f(i-1, w-Wi) + Vi
- ///Or 0 if i < 0
- for(int i = 1; i<=values.Length; i++)
- {
- for(int w = 1; w <= maxWeight; w++)
- {
- //below code can be refined - intentional as i just want it
- //look similar to other 2 versions (top_to_bottom_momoization
- //and recursive_without_resuing_subproblem_solution
- int maxValueWithoutIncludingCurrentItem =
- DP_Tabular_Eager_Max_Value_Cache[i - 1][w];
- if (weights[i - 1] > w)
- {
- DP_Tabular_Eager_Max_Value_Cache[i][w] = maxValueWithoutIncludingCurrentItem;
- }
- else
- {
- int maxValueByIncludingCurrentItem =
- DP_Tabular_Eager_Max_Value_Cache[i - 1][w - weights[i - 1]]
- + values[i-1];
- int overAllMax = maxValueWithoutIncludingCurrentItem;
- if(maxValueByIncludingCurrentItem > overAllMax)
- {
- overAllMax = maxValueByIncludingCurrentItem;
- }
- DP_Tabular_Eager_Max_Value_Cache[i][w] = overAllMax;
- }
- }
- }
- return DP_Tabular_Eager_Max_Value_Cache[values.Length][maxWeight];
- }
Recursion which has optimal sub structure and overlapping sub problems
Code Snippet
- public int Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure(int[] weights, int[] values, int maxWeight)
- {
- this.ValidataInput_Knapsack_0_1(weights, values, maxWeight);
- int v = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure_Recursive(weights, values, index: weights.Length-1, weight: maxWeight);
- return v;
- }
- /// <summary>
- /// f(i, w) = f(i-1, w) if Wi > w
- ///Or, max (f(i-1, w), f(i-1, w-Wi) + Vi
- ///Or 0 if i < 0
- /// </summary>
- int Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure_Recursive(int[] weights, int[] values, int index, int weight)
- {
- if (index < 0)
- {
- return 0;
- }
- Debug.Assert(weight >= 0);
- int maxValueWithoutIncludingCurrentItem = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure_Recursive(weights,
- values, index - 1, weight);
- if(weights[index] > weight)
- {
- //current item weight is more, so we cant include - so, just return
- return maxValueWithoutIncludingCurrentItem;
- }
- int overallMaxValue = maxValueWithoutIncludingCurrentItem;
- int maxValueIncludingCurrentItem = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure_Recursive(weights,
- values, index - 1, weight - weights[index]) + values[index];
- if(maxValueIncludingCurrentItem > overallMaxValue)
- {
- overallMaxValue = maxValueIncludingCurrentItem;
- }
- return overallMaxValue;
- }
Another recursive method which is also Brute Force
(but note that this recursive method don't have above properties - optimal substructure and overlapping solutions)
Code Snippet
- private int _maxValue = int.MinValue;
- private int[] _valueIndices = null;
- public void Knapsack_0_1_BruteForce_2_Power_N(int[] weights, int[] values, int maxWeight)
- {
- this.ValidataInput_Knapsack_0_1(weights, values, maxWeight);
- this._maxValue = int.MinValue;
- this._valueIndices = null;
- this.Knapsack_0_1_BruteForce_2_Power_N_Rcursive(weights, values, maxWeight, 0, 0, 0, new List<int>());
- }
- private void Knapsack_0_1_BruteForce_2_Power_N_Rcursive(int[] weights, int[] values, int maxWeight, int index, int currentWeight, int currentValue, List<int> currentValueIndices)
- {
- if(currentWeight > maxWeight)
- {
- return;
- }
- if(currentValue > this._maxValue)
- {
- this._maxValue = currentValue;
- this._valueIndices = currentValueIndices.ToArray();
- }
- if(index == weights.Length)
- {
- return;
- }
- Debug.Assert(index < weights.Length);
- var w = weights[index];
- var v = values[index];
- //see if the current value, conributes to max value
- currentValueIndices.Add(index);
- Knapsack_0_1_BruteForce_2_Power_N_Rcursive(weights, values, maxWeight, index + 1, currentWeight + w,
- currentValue + v, currentValueIndices);
- currentValueIndices.Remove(index);
- //see if current value, does not contribute to max value
- Knapsack_0_1_BruteForce_2_Power_N_Rcursive(weights, values, maxWeight, index + 1, currentWeight, currentValue,
- currentValueIndices);
- }
- private void ValidataInput_Knapsack_0_1(int[] weights, int[] values, int maxWeight)
- {
- #region inputvalidations
- weights.ThrowIfNull("weights");
- values.ThrowIfNull("values");
- weights.Throw("weights", w => (w.Length == 0)
- || w.Any(wi => wi < 0));
- values.Throw("values", v => (v.Length == 0)
- || v.Any(vi => vi < 0)
- || (v.Length != weights.Length));
- maxWeight.Throw("maxWeight", mw => mw <= 0);
- #endregion
- }
Unit Tests
Code Snippet
- [TestCategory(Constants.DynamicProgramming)]
- [TestMethod]
- public void Knapsack_0_1_Tests()
- {
- int[] benefits = new int[] { 60, 100, 120 };
- int[] weights = new int[] { 10, 20, 30 };
- this.Knapsack_0_1_BruteForce_2_Power_N(weights, values: benefits, maxWeight: 50);
- Assert.IsTrue(this._maxValue == 220);
- int v = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure(weights,
- values: benefits, maxWeight: 50);
- Assert.IsTrue(v == 220);
- v = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy(weights,
- values: benefits, maxWeight: 50);
- Assert.IsTrue(v == 220);
- v = this.Knapsack_0_1_DP_Tabular_Bottom_Up(weights,
- values: benefits, maxWeight: 50);
- Assert.IsTrue(v == 220);
- benefits = new int[] { 3, 4, 5, 8, 10 };
- weights = new int[] { 2, 3, 4, 5, 9 };
- this.Knapsack_0_1_BruteForce_2_Power_N(weights, values: benefits, maxWeight: 20);
- Assert.IsTrue(this._maxValue == 26);
- v = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure(weights, values: benefits,
- maxWeight: 20);
- Assert.IsTrue(v == 26);
- v = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy(weights,
- values: benefits, maxWeight: 20);
- Assert.IsTrue(v == 26);
- v = this.Knapsack_0_1_DP_Tabular_Bottom_Up(weights,
- values: benefits, maxWeight: 20);
- Assert.IsTrue(v == 26);
- benefits = new int[] { 3, 4, 5, 6};
- weights = new int[] { 2, 3, 4, 5 };
- this.Knapsack_0_1_BruteForce_2_Power_N(weights, values: benefits, maxWeight: 5);
- Assert.IsTrue(this._maxValue == 7);
- v = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure(weights, values: benefits,
- maxWeight: 5);
- Assert.IsTrue(v == 7);
- v = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy(weights,
- values: benefits, maxWeight: 5);
- Assert.IsTrue(v == 7);
- v = this.Knapsack_0_1_DP_Tabular_Bottom_Up(weights,
- values: benefits, maxWeight: 5);
- Assert.IsTrue(v == 7);
- }
Code Snippet
- [TestCategory(Constants.DynamicProgramming)]
- [TestMethod]
- public void Knapsack_0_1_Brute_Force_Tests()
- {
- int[] benefits = new int[] { 60, 100, 120 };
- int[] weights = new int[] { 10, 20, 30 };
- this.Knapsack_0_1_BruteForce_2_Power_N(weights, values: benefits, maxWeight: 50);
- Assert.IsTrue(this._maxValue == 220);
- Assert.IsTrue(this._valueIndices.Contains(1));
- Assert.IsTrue(this._valueIndices.Contains(2));
- Assert.IsTrue(this._valueIndices.Length == 2);
- this._maxValue = int.MinValue;
- this._valueIndices = null;
- benefits = new int[] { 3, 4, 5, 8, 10 };
- weights = new int[] { 2, 3, 4, 5, 9 };
- this.Knapsack_0_1_BruteForce_2_Power_N(weights, values: benefits, maxWeight: 20);
- Assert.IsTrue(this._maxValue == 26);
- Assert.IsTrue(this._valueIndices.Contains(0));
- Assert.IsTrue(this._valueIndices.Contains(2));
- Assert.IsTrue(this._valueIndices.Contains(3));
- Assert.IsTrue(this._valueIndices.Contains(4));
- Assert.IsTrue(this._valueIndices.Length == 4);
- }