Optimization in Python

Solving the candy model

Now let’s see how can we write a program to solve the simple candy model, and we will be using: Pandas, a popular python package for data wrangling, Cplex, an interface to IBM’s cplex solver, and docplex: a modeling package. Solving large scale optimization problem often relies on specialized software, called solvers, and IBM provides a community version of cplex free of charge. If you use pip:

pip install pandas cplex docplex

Or if you use anaconda python distribution:

conda install pandas
conda install -c ibmdecisionoptimization cplex docplex

First, let’s set up the data into Pandas Series and DataFrames.

from docplex.mp.model import Model
import pandas as pd

index = ['Chocolate', 'M&M']
col = ['bundle0', 'bundle1']
A = pd.DataFrame([[2, 4], [3, 2]], index=index, columns=col).T
L = pd.Series([40, 50], index=index)
P = pd.Series([3, 3], index=col)

They look like these:

In [2]: A
Out[2]:
         Chocolate  M&M
bundle0          2    3
bundle1          4    2

In [3]: L
Out[3]:
Chocolate    40
M&M          50
dtype: int64

In [4]: P
Out[4]:
bundle0    3
bundle1    3
dtype: int64

Then we go ahead building the model. The decision we are trying to make, and the unknown we are trying to solve, is for each candy bundle how many we should buy, which should be integer. Therefore we call it decision variable, and in this particular case integer decision variable (line 2). Further more, they has to be non-negative. Therefore we specify some lower bound (lb) for each of the decision variables.
In the end we need to have >=40 chocolate and >=50 M&M. To do that we specify some constraints, using Python generate expression in line 3. Next we need to specify our objective of minimizing cost (line 4).
All done, let’s solve the problem via a .solve call.

There are many guides around showing model construction using Python build-in types such as list, tuple, dictionary etc. Since in real world applications we will use Pandas everyday anyway, we might just as well using Pandas in building model.

mdl = Model('Candies')
X = pd.Series(mdl.integer_var_dict(P.index, lb=0))
C = mdl.add_constraints(mdl.dot(X, A[i]) >= L[i] for i in index)
mdl.minimize(mdl.dot(X, P))
sol = mdl.solve()

Looks like we need to buy 15 bundle0 and 3 bundle1 to reach the cost-minimizing target, while satisfying both of our constraints.

In [5]: sol
Out[5]: docplex.mp.solution.SolveSolution(obj=54,values={_x1:15,_x2:3})

As discussed previously, if the decision variables are allowed to be continuous, we might be able to get even better numerical solution. We can solve the same problem with the integrality removed, which is called solving linear programming (LP) relaxation.

for v in X:
    mdl.set_var_type(v, mdl.continuous_vartype)
    v.set_lb(0)
slp = mdl.solve()

Looks like we now only need to spend 52.5 cents instead of 54 cents from the previous integer solution. However for bundle1 we will buy 2.5 instead of 3.

In [6]: slp
Out[6]: docplex.mp.solution.SolveSolution(obj=52.5,values={_x1:15,_x2:2.5})

_x1 and _x2 in the solution are probably not the most intuitive labels. This one-liner will convert the solution into a nice Pandas Series.

In [7]: pd.Series(sol.get_value_dict(X))
Out[7]:
bundle0    15.0
bundle1     3.0
dtype: float64

Geometric intro in Python

import matplotlib.pyplot as plt
import numpy as np


# The problem in array form
A = np.array([[2, 4], [3, 2]])
b = np.array([40, 50])
obj = np.array([3, 3])

# The problem in Mathtex
T = '''$Min. 3x + 3y$\n
$s.t.$ (Subject to)\n
$2x + 4y \geq 40$
Greater than 40 once projected to $\\vec {(2,4)}$\n
$3x + 2y \geq 50$
Greater than 50 once projected to $\\vec {(3,2)}$'''

# Intercept, feasiblity cutoff and axes angles
c = (b / A.T).T
f = (A.T * b / (A * A).sum(1)).T
r = np.arctan(np.divide(*A.T)) * 180 / np.pi

# Feasiblity region for both continous and integer cases
X = np.dstack(np.meshgrid(np.linspace(0, 20, 400), np.linspace(0, 25, 500)))
Xi = np.dstack(np.meshgrid(np.arange(20), np.arange(25)))
F = (np.dot(X, A.T) > b).all(2)
Fi = (np.dot(Xi, A.T) > b).all(2)


def plot_constraint0(ax):
    ax.annotate("",
                xy=f[0], xycoords='data',
                xytext=(0, 0), textcoords='data',
                arrowprops=dict(arrowstyle="->", color='r',
                                connectionstyle="arc3"),
                )

def plot_constraint1(ax):
    ax.annotate("",
                xy=f[1], xycoords='data',
                xytext=(0, 0), textcoords='data',
                arrowprops=dict(arrowstyle="->", color='b',
                                connectionstyle="arc3"),
                )

def plot_objective(ax):
    ax.annotate("",
                xy=(12, 12), xycoords='data',
                xytext=(0, 0), textcoords='data',
                arrowprops=dict(arrowstyle="->", color='g',
                                connectionstyle="arc3"),
                )

def plot_boundnlabel(ax):
    for i, color in enumerate('rb'):
        ax.plot([c[i, 0], 0], [0, c[i, 1]], c=color)
        ax.text(*(f[i] * 0.5), s='Constraint %s' % i,
                ha='center', va='center', rotation=90-r[i])

def plot_objectnlabel(ax):
    ax.text(6, 6, s='Object', ha='center', va='center', rotation=45)
    for i in np.linspace(15, 20, 10):
        ax.plot([0, i], [i, 0], 'g--', alpha=0.2)

def plot_lpfeasiblity(ax):
    ax.contourf(X[:, :, 0], X[:, :, 1], 1/F, alpha=0.1)

def plot_ipfeasiblity(ax):
    ax.scatter(*np.where(Fi.T), marker='+')

plt.rcParams['font.family'] = ['Arial']
fig, axes = plt.subplots(1, 5, sharex=True, sharey=True)
axes = axes.ravel()

axes[0].text(-2, 20, T, ha='left', va='top', fontsize=12)
axes[0].axis('off')

plot_constraint0(axes[1])
plot_constraint1(axes[1])
plot_boundnlabel(axes[1])
axes[1].set_title('''Legit solutions must cross the
cutoff lines once projected on to\ncontraint axes''')

plot_constraint0(axes[2])
plot_constraint1(axes[2])
plot_boundnlabel(axes[2])
plot_lpfeasiblity(axes[2])
axes[2].set_title('''Legit solutions must satisfy
both constraints\n''')

plot_constraint0(axes[3])
plot_constraint1(axes[3])
plot_boundnlabel(axes[3])
plot_objective(axes[3])
plot_objectnlabel(axes[3])
plot_lpfeasiblity(axes[3])
axes[3].set_title('''Need to find the minimus among
legit solutions once projected to\nobject axis''')

plot_constraint0(axes[4])
plot_constraint1(axes[4])
plot_boundnlabel(axes[4])
plot_objective(axes[4])
plot_objectnlabel(axes[4])
plot_lpfeasiblity(axes[4])
plot_ipfeasiblity(axes[4])
axes[4].set_title('''For integer problem\nlegit solutions are further
confined to the integer grid''')

for ax in axes:
    ax.set_aspect('equal')

plt.tight_layout()