Advanced Features¶

This section of the tutorial covers features of CVXPY intended for users with advanced knowledge of convex optimization. We recommend Convex Optimization by Boyd and Vandenberghe as a reference for any terms you are unfamiliar with.

N-dimensional expressions¶

CVXPY now supports N-dimensional expressions. This allows one to define variables, parameters, and constants with arbitrary number of dimensions. This new feature enables users to model problems with multi-dimensional data in a more natural way.

In the example below, we consider a problem where the goal is to optimize the usage of a resource across multiple locations, days, and hours. We are now able to easily form constraints on any combination of dimensions.

# create a 3-dimensional variable (locations, days, hours)
x = cp.Variable((12, 10, 24))

constraints = [
  cp.sum(x, axis=(0, 2)) <= 2000, # constrain the daily usage across all locations
  x[:, :, :12] <= 100, # constrain the first 12 hours of each day at every location
  x[:, 3, :] == 0,] # constrain the usage on the fourth day to be zero

obj = cp.Minimize(cp.sum_squares(x))
prob = cp.Problem(obj, constraints)
prob.solve()

Please refer to NumPy’s excellent reference on N-dimensional arrays and the array API standard for more details on how to manipulate N-dimensional arrays. Our goal is to match the NumPy API as closely as possible.

Dual variables¶

You can use CVXPY to find the optimal dual variables for a problem. When you call prob.solve() each dual variable in the solution is stored in the dual_value field of the constraint it corresponds to.

import cvxpy as cp

# Create two scalar optimization variables.
x = cp.Variable()
y = cp.Variable()

# Create two constraints.
constraints = [x + y == 1,
               x - y >= 1]

# Form objective.
obj = cp.Minimize((x - y)**2)

# Form and solve problem.
prob = cp.Problem(obj, constraints)
prob.solve()

# The optimal dual variable (Lagrange multiplier) for
# a constraint is stored in constraint.dual_value.
print("optimal (x + y == 1) dual variable", constraints[0].dual_value)
print("optimal (x - y >= 1) dual variable", constraints[1].dual_value)
print("x - y value:", (x - y).value)

optimal (x + y == 1) dual variable 6.47610300459e-18
optimal (x - y >= 1) dual variable 2.00025244976
x - y value: 0.999999986374

The dual variable for x - y >= 1 is 2. By complementarity this implies that x - y is 1, which we can see is true. The fact that the dual variable is non-zero also tells us that if we tighten x - y >= 1, (i.e., increase the right-hand side), the optimal value of the problem will increase.

Transforms¶

Transforms provide additional ways of manipulating CVXPY objects beyond the atomic functions. For example, the indicator transform converts a list of constraints into an expression representing the convex function that takes value 0 when the constraints hold and \(\infty\) when they are violated.

x = cp.Variable()
constraints = [0 <= x, x <= 1]
expr = cp.transforms.indicator(constraints)
x.value = .5
print("expr.value = ", expr.value)
x.value = 2
print("expr.value = ", expr.value)

expr.value = 0.0
expr.value = inf

The full set of transforms available is discussed in Transforms.

Problem arithmetic¶

For convenience, arithmetic operations have been overloaded for problems and objectives. Problem arithmetic is useful because it allows you to write a problem as a sum of smaller problems. The rules for adding, subtracting, and multiplying objectives are given below.

# Addition and subtraction.

Minimize(expr1) + Minimize(expr2) == Minimize(expr1 + expr2)

Maximize(expr1) + Maximize(expr2) == Maximize(expr1 + expr2)

Minimize(expr1) + Maximize(expr2) # Not allowed.

Minimize(expr1) - Maximize(expr2) == Minimize(expr1 - expr2)

# Multiplication (alpha is a positive scalar).

alpha*Minimize(expr) == Minimize(alpha*expr)

alpha*Maximize(expr) == Maximize(alpha*expr)

-alpha*Minimize(expr) == Maximize(-alpha*expr)

-alpha*Maximize(expr) == Minimize(-alpha*expr)

The rules for adding and multiplying problems are equally straightforward:

# Addition and subtraction.

prob1 + prob2 == Problem(prob1.objective + prob2.objective,
                         prob1.constraints + prob2.constraints)

prob1 - prob2 == Problem(prob1.objective - prob2.objective,
                         prob1.constraints + prob2.constraints)

# Multiplication (alpha is any scalar).

alpha*prob == Problem(alpha*prob.objective, prob.constraints)

Note that the + operator concatenates lists of constraints, since this is the default behavior for Python lists. The in-place operators +=, -=, and *= are also supported for objectives and problems and follow the same rules as above.

Getting the standard form¶

If you are interested in getting the standard form that CVXPY produces for a problem, you can use the get_problem_data method. When a problem is solved, a SolvingChain passes a low-level representation that is compatible with the targeted solver to a solver, which solves the problem. This method returns that low-level representation, along with a SolvingChain and metadata for unpacking a solution into the problem. This low-level representation closely resembles, but is not identical to, the arguments supplied to the solver.

A solution to the equivalent low-level problem can be obtained via the data by invoking the solve_via_data method of the returned solving chain, a thin wrapper around the code external to CVXPY that further processes and solves the problem. Invoke the unpack_results method to recover a solution to the original problem.

For example:

problem = cp.Problem(objective, constraints)
data, chain, inverse_data = problem.get_problem_data(cp.SCS)
# calls SCS using `data`
soln = chain.solve_via_data(problem, data)
# unpacks the solution returned by SCS into `problem`
problem.unpack_results(soln, chain, inverse_data)

Alternatively, the data dictionary returned by this method contains enough information to bypass CVXPY and call the solver directly.

For example:

problem = cp.Problem(objective, constraints)
probdata, _, _ = problem.get_problem_data(cp.SCS)

import scs
data = {
  'A': probdata['A'],
  'b': probdata['b'],
  'c': probdata['c'],
}
cone_dims = probdata['dims']
cones = {
    "f": cone_dims.zero,
    "l": cone_dims.nonpos,
    "q": cone_dims.soc,
    "ep": cone_dims.exp,
    "s": cone_dims.psd,
}
soln = scs.solve(data, cones)

The structure of the data dict that CVXPY returns depends on the solver. For details, print the dictionary, or consult the solver interfaces in cvxpy/reductions/solvers.

Canonicalization backends¶

Users can select from multiple canonicalization backends by adding the canon_backend keyword argument to the .solve() call, e.g. problem.solve(canon_backend=cp.SCIPY_CANON_BACKEND) (Introduced in CVXPY 1.3). This can speed up the canonicalization time significantly for some problems. Currently, the following canonicalization backends are supported:

• CPP (default): The original C++ implementation, also referred to as CVXCORE.

• SCIPY: A pure Python implementation based on the SciPy sparse module.

  Generally fast for problems that are already vectorized.

• NUMPY: Reference implementation in pure NumPy. Fast for some small or dense problems.