Atomic Functions

This section of the tutorial describes the atomic functions that can be applied to CVXPY expressions. CVXPY uses the function information in this section and the DCP rules to mark expressions with a sign and curvature.

Operators

The infix operators +, -, *, / are treated as functions. + and - are affine functions. The expression expr1*expr2 is are affine in CVXPY when one of the expressions is constant, and expr1/expr2 is affine when expr2 is a scalar constant.

Note that in CVXPY, expr1 * expr2 denotes matrix multiplication when expr1 and expr2 are matrices; if you’re running Python 3, you can alternatively use the @ operator for matrix multiplication. Regardless of your Python version, you can also use the function :function:`~cvxpy.atoms.affine.binary_operators.matmul` to multiply two matrices. To multiply two arrays or matrices elementwise, use :function:`~cvxpy.atoms.affine.binary_operators.multiply`_.

Indexing and slicing

Indexing in CVXPY follows exactly the same semantics as NumPy ndarrays. For example, if expr has shape (5,) then expr[1] gives the second entry. More generally, expr[i:j:k] selects every kth element of expr, starting at i and ending at j-1. If expr is a matrix, then expr[i:j:k] selects rows, while expr[i:j:k, r:s:t] selects both rows and columns. Indexing drops dimensions while slicing preserves dimenions. For example,

x = cvx.Variable(5)
print("0 dimensional", x[0].shape)
print("1 dimensional", x[0:1].shape)
O dimensional: ()
1 dimensional: (1,)

Transpose

The transpose of any expression can be obtained using the syntax expr.T. Transpose is an affine function.

Power

For any CVXPY expression expr, the power operator expr**p is equivalent to the function power(expr, p).

Scalar functions

A scalar function takes one or more scalars, vectors, or matrices as arguments and returns a scalar.

Function Meaning Domain Sign Curvature  Monotonicity

geo_mean(x)

geo_mean(x, p)

\(p \in \mathbf{R}^n_{+}\)

\(p \neq 0\)

\(x_1^{1/n} \cdots x_n^{1/n}\)

\(\left(x_1^{p_1} \cdots x_n^{p_n}\right)^{\frac{1}{\mathbf{1}^T p}}\)

\(x \in \mathbf{R}^n_{+}\) positive positive concave concave incr incr.
harmonic_mean(x) \(\frac{n}{\frac{1}{x_1} + \cdots + \frac{1}{x_n}}\) \(x \in \mathbf{R}^n_{+}\) positive positive concave concave incr incr.
lambda_max(X) \(\lambda_{\max}(X)\) \(X \in \mathbf{S}^n\) unknown unknown convex convex None
lambda_min(X) \(\lambda_{\min}(X)\) \(X \in \mathbf{S}^n\) unknown unknown concave concave None

lambda_sum_largest(X,k)

\(k = 1,\ldots, n\)

\(\text{sum of $k$ largest}\\ \text{eigenvalues of $X$}\) \(X \in\mathbf{S}^{n}\) unknown unknown convex convex None

lambda_sum_smallest(X,k)

\(k = 1,\ldots, n\)

\(\text{sum of $k$ smallest}\\ \text{eigenvalues of $X$}\) \(X \in\mathbf{S}^{n}\) unknown unknown concave concave None
log_det(X) \(\log \left(\det (X)\right)\) \(X \in \mathbf{S}^n_+\) unknown unknown concave concave None
log_sum_exp(X) \(\log \left(\sum_{ij}e^{X_{ij}}\right)\) \(X \in\mathbf{R}^{m \times n}\) unknown unknown convex convex incr incr.
matrix_frac(x, P) \(x^T P^{-1} x\)

\(x \in \mathbf{R}^n\)

\(P \in\mathbf{S}^n_{++}\)

positive positive convex convex None
max(X) \(\max_{ij}\left\{ X_{ij}\right\}\) \(X \in\mathbf{R}^{m \times n}\) same as X convex convex incr incr.
min(X) \(\min_{ij}\left\{ X_{ij}\right\}\) \(X \in\mathbf{R}^{m \times n}\) same as X concave concave incr incr.
mixed_norm(X, p, q) \(\left(\sum_k\left(\sum_l\lvert x_{k,l}\rvert^p\right)^{q/p}\right)^{1/q}\) \(X \in\mathbf{R}^{n \times n}\) positive positive convex convex None

norm(x)

norm(x, 2)

\(\sqrt{\sum_{i} \lvert x_{i} \rvert^2 }\) \(X \in\mathbf{R}^{n}\) positive positive convex convex

incr for \(x_{i} \geq 0\)

decr for \(x_{i} \leq 0\)

norm(X, “fro”) \(\sqrt{\sum_{ij}X_{ij}^2 }\) \(X \in\mathbf{R}^{m \times n}\) positive positive convex convex

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

norm(X, 1) \(\sum_{ij}\lvert X_{ij} \rvert\) \(X \in\mathbf{R}^{m \times n}\) positive positive convex convex

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

norm(X, “inf”) \(\max_{ij} \{\lvert X_{ij} \rvert\}\) \(X \in\mathbf{R}^{m \times n}\) positive positive convex convex

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

norm(X, “nuc”) \(\mathrm{tr}\left(\left(X^T X\right)^{1/2}\right)\) \(X \in\mathbf{R}^{m \times n}\) positive positive convex convex None

norm(X)

norm(X, 2)

\(\sqrt{\lambda_{\max}\left(X^T X\right)}\) \(X \in\mathbf{R}^{m \times n}\) positive positive convex convex None

pnorm(X, p)

\(p \geq 1\)

or p = 'inf'

\(\|X\|_p = \left(\sum_{ij} |X_{ij}|^p \right)^{1/p}\) \(X \in \mathbf{R}^{m \times n}\) positive positive convex convex

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

pnorm(X, p)

\(p < 1\), \(p \neq 0\)

\(\|X\|_p = \left(\sum_{ij} X_{ij}^p \right)^{1/p}\) \(X \in \mathbf{R}^{m \times n}_+\) positive positive concave concave incr incr.

quad_form(x, P)

constant \(P \in \mathbf{S}^n_+\)

\(x^T P x\) \(x \in \mathbf{R}^n\) positive positive convex convex

incr for \(x_i \geq 0\)

decr for \(x_i \leq 0\)

quad_form(x, P)

constant \(P \in \mathbf{S}^n_-\)

\(x^T P x\) \(x \in \mathbf{R}^n\) negative negative concave concave

decr for \(x_i \geq 0\)

incr for \(x_i \leq 0\)

quad_form(c, X)

constant \(c \in \mathbf{R}^n\)

\(c^T X c\) \(X \in\mathbf{R}^{n \times n}\) depends on c, X affine affine depends on c
quad_over_lin(X, y) \(\left(\sum_{ij}X_{ij}^2\right)/y\)

\(x \in \mathbf{R}^n\)

\(y > 0\)

positive positive convex convex

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

decr decr. in \(y\)

sum(X) \(\sum_{ij}X_{ij}\) \(X \in\mathbf{R}^{m \times n}\) same as X affine affine incr incr.

sum_largest(X, k)

\(k = 1,2,\ldots\)

\(\text{sum of } k\text{ largest }X_{ij}\) \(X \in\mathbf{R}^{m \times n}\) same as X convex convex incr incr.

sum_smallest(X, k)

\(k = 1,2,\ldots\)

\(\text{sum of } k\text{ smallest }X_{ij}\) \(X \in\mathbf{R}^{m \times n}\) same as X concave concave incr incr.
sum_squares(X) \(\sum_{ij}X_{ij}^2\) \(X \in\mathbf{R}^{m \times n}\) positive positive convex convex

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

trace(X) \(\mathrm{tr}\left(X \right)\) \(X \in\mathbf{R}^{n \times n}\) same as X affine affine incr incr.
tv(x) \(\sum_{i}|x_{i+1} - x_i|\) \(x \in \mathbf{R}^n\) positive positive convex convex None
tv(x) \(\sum_{ij}\left\| \left[\begin{matrix} X_{i+1,j} - X_{ij} \\ X_{i,j+1} -X_{ij} \end{matrix}\right] \right\|_2\) \(X \in \mathbf{R}^{m \times n}\) positive positive convex convex None
tv([X1,…,Xk]) \(\sum_{ij}\left\| \left[\begin{matrix} X_{i+1,j}^{(1)} - X_{ij}^{(1)} \\ X_{i,j+1}^{(1)} -X_{ij}^{(1)} \\ \vdots \\ X_{i+1,j}^{(k)} - X_{ij}^{(k)} \\ X_{i,j+1}^{(k)} -X_{ij}^{(k)} \end{matrix}\right] \right\|_2\) \(X^{(i)} \in\mathbf{R}^{m \times n}\) positive positive convex convex None

Clarifications

The domain \(\mathbf{S}^n\) refers to the set of symmetric matrices. The domains \(\mathbf{S}^n_+\) and \(\mathbf{S}^n_-\) refer to the set of positive semi-definite and negative semi-definite matrices, respectively. Similarly, \(\mathbf{S}^n_{++}\) and \(\mathbf{S}^n_{--}\) refer to the set of positive definite and negative definite matrices, respectively.

For a vector expression x, norm(x) and norm(x, 2) give the Euclidean norm. For a matrix expression X, however, norm(X) and norm(X, 2) give the spectral norm.

The function norm(X, "fro") is called the Frobenius norm and norm(X, "nuc") the nuclear norm. The nuclear norm can also be defined as the sum of X’s singular values.

The functions max and min give the largest and smallest entry, respectively, in a single expression. These functions should not be confused with maximum and minimum (see Elementwise functions). Use maximum and minimum to find the max or min of a list of scalar expressions.

The CVXPY function sum sums all the entries in a single expression. The built-in Python sum should be used to add together a list of expressions. For example, the following code sums a list of three expressions:

expr_list = [expr1, expr2, expr3]
expr_sum = sum(expr_list)

Functions along an axis

The functions sum, norm, max, and min can be applied along an axis. Given an m by n expression expr, the syntax func(expr, axis=0, keepdims=True) applies func to each column, returning a 1 by n expression. The syntax func(expr, axis=1, keepdims=True) applies func to each row, returning an m by 1 expression. By default keepdims=False, which means dimensions of length 1 are dropped. For example, the following code sums along the columns and rows of a matrix variable: .. code:: python

X = cvx.Variable((5, 4)) col_sums = cvx.sum(X, axis=0, keepdims=True) # Has size (1, 4) col_sums = cvx.sum(X, axis=0) # Has size (4,) row_sums = cvx.sum(X, axis=1) # Has size (5,)

Elementwise functions

These functions operate on each element of their arguments. For example, if X is a 5 by 4 matrix variable, then abs(X) is a 5 by 4 matrix expression. abs(X)[1, 2] is equivalent to abs(X[1, 2]).

Elementwise functions that take multiple arguments, such as maximum and multiply, operate on the corresponding elements of each argument. For example, if X and Y are both 3 by 3 matrix variables, then maximum(X, Y) is a 3 by 3 matrix expression. maximum(X, Y)[2, 0] is equivalent to maximum(X[2, 0], Y[2, 0]). This means all arguments must have the same dimensions or be scalars, which are promoted.

Function Meaning Domain Sign Curvature  Monotonicity
abs(x) \(\lvert x \rvert\) \(x \in \mathbf{R}\) positive positive convex convex

incr for \(x \geq 0\)

decr for \(x \leq 0\)

entr(x) \(-x \log (x)\) \(x > 0\) unknown unknown concave concave None
exp(x) \(e^x\) \(x \in \mathbf{R}\) positive positive convex convex incr incr.

huber(x, M=1)

\(M \geq 0\)

\(\begin{cases}x^2 &|x| \leq M \\2M|x| - M^2&|x| >M\end{cases}\) \(x \in \mathbf{R}\) positive positive convex convex

incr for \(x \geq 0\)

decr for \(x \leq 0\)

inv_pos(x) \(1/x\) \(x > 0\) positive positive convex convex decr decr.
kl_div(x, y) \(x \log(x/y) - x + y\)

\(x > 0\)

\(y > 0\)

positive positive convex convex None
log(x) \(\log(x)\) \(x > 0\) unknown unknown concave concave incr incr.
log1p(x) \(\log(x+1)\) \(x > -1\) same as x concave concave incr incr.
logistic(x) \(\log(1 + e^{x})\) \(x \in \mathbf{R}\) positive positive convex convex incr incr.
maximum(x, y) \(\max \left\{x, y\right\}\) \(x,y \in \mathbf{R}\) depends on x,y convex convex incr incr.
minimum(x, y) \(\min \left\{x, y\right\}\) \(x, y \in \mathbf{R}\) depends on x,y concave concave incr incr.

multiply(c, x)

\(c \in \mathbf{R}\)

c*x \(x \in\mathbf{R}\) \(\mathrm{sign}(cx)\) affine affine depends on c
neg(x) \(\max \left\{-x, 0 \right\}\) \(x \in \mathbf{R}\) positive positive convex convex decr decr.
pos(x) \(\max \left\{x, 0 \right\}\) \(x \in \mathbf{R}\) positive positive convex convex incr incr.
power(x, 0) \(1\) \(x \in \mathbf{R}\) positive positive constant  
power(x, 1) \(x\) \(x \in \mathbf{R}\) same as x affine affine incr incr.

power(x, p)

\(p = 2, 4, 8, \ldots\)

\(x^p\) \(x \in \mathbf{R}\) positive positive convex convex

incr for \(x \geq 0\)

decr for \(x \leq 0\)

power(x, p)

\(p < 0\)

\(x^p\) \(x > 0\) positive positive convex convex decr decr.

power(x, p)

\(0 < p < 1\)

\(x^p\) \(x \geq 0\) positive positive concave concave incr incr.

power(x, p)

\(p > 1,\ p \neq 2, 4, 8, \ldots\)

\(x^p\) \(x \geq 0\) positive positive convex convex incr incr.

scalene(x, alpha, beta)

\(\text{alpha} \geq 0\)

\(\text{beta} \geq 0\)

\(\alpha\mathrm{pos}(x)+ \beta\mathrm{neg}(x)\) \(x \in \mathbf{R}\) positive positive convex convex

incr for \(x \geq 0\)

decr for \(x \leq 0\)

sqrt(x) \(\sqrt x\) \(x \geq 0\) positive positive concave concave incr incr.
square(x) \(x^2\) \(x \in \mathbf{R}\) positive positive convex convex

incr for \(x \geq 0\)

decr for \(x \leq 0\)

Vector/matrix functions

A vector/matrix function takes one or more scalars, vectors, or matrices as arguments and returns a vector or matrix.

Function Meaning Domain Sign Curvature  Monotonicity
bmat([[X11,…,X1q], …, [Xp1,…,Xpq]]) \(\left[\begin{matrix} X^{(1,1)} & \cdots & X^{(1,q)} \\ \vdots & & \vdots \\ X^{(p,1)} & \cdots & X^{(p,q)} \end{matrix}\right]\) \(X^{(i,j)} \in\mathbf{R}^{m_i \times n_j}\) \(\mathrm{sign}\left(\sum_{ij} X^{(i,j)}_{11}\right)\) affine affine incr incr.

conv(c, x)

\(c\in\mathbf{R}^m\)

\(c*x\) \(x\in \mathbf{R}^n\) \(\mathrm{sign}\left(c_{1}x_{1}\right)\) affine affine depends on c
cumsum(X, axis=0) cumulative sum along given axis. \(X \in \mathbf{R}^{m \times n}\) same as X affine affine incr incr.
diag(x) \(\left[\begin{matrix}x_1 & & \\& \ddots & \\& & x_n\end{matrix}\right]\) \(x \in\mathbf{R}^{n}\) same as x affine affine incr incr.
diag(X) \(\left[\begin{matrix}X_{11} \\\vdots \\X_{nn}\end{matrix}\right]\) \(X \in\mathbf{R}^{n \times n}\) same as X affine affine incr incr.

diff(X, k=1, axis=0)

\(k \in 0,1,2,\ldots\)

kth order differences along given axis \(X \in\mathbf{R}^{m \times n}\) same as X affine affine incr incr.
hstack([X1, …, Xk]) \(\left[\begin{matrix}X^{(1)} \cdots X^{(k)}\end{matrix}\right]\) \(X^{(i)} \in\mathbf{R}^{m \times n_i}\) \(\mathrm{sign}\left(\sum_i X^{(i)}_{11}\right)\) affine affine incr incr.

kron(C, X)

\(C\in\mathbf{R}^{p \times q}\)

\(\left[\begin{matrix}C_{11}X & \cdots & C_{1q}X \\ \vdots & & \vdots \\ C_{p1}X & \cdots & C_{pq}X \end{matrix}\right]\) \(X \in\mathbf{R}^{m \times n}\) \(\mathrm{sign}\left(C_{11}X_{11}\right)\) affine affine depends on C
reshape(X, (n’, m’)) \(X' \in\mathbf{R}^{m' \times n'}\)

\(X \in\mathbf{R}^{m \times n}\)

\(m'n' = mn\)

same as X affine affine incr incr.
vec(X) \(x' \in\mathbf{R}^{mn}\) \(X \in\mathbf{R}^{m \times n}\) same as X affine affine incr incr.
vstack([X1, …, Xk]) \(\left[\begin{matrix}X^{(1)} \\ \vdots \\X^{(k)}\end{matrix}\right]\) \(X^{(i)} \in\mathbf{R}^{m_i \times n}\) \(\mathrm{sign}\left(\sum_i X^{(i)}_{11}\right)\) affine affine incr incr.

Clarifications

The input to bmat is a list of lists of CVXPY expressions. It constructs a block matrix. The elements of each inner list are stacked horizontally and then the resulting block matrices are stacked vertically.

The output \(y\) of conv(c, x) has size \(n+m-1\) and is defined as \(y[k]=\sum_{j=0}^k c[j]x[k-j]\).

The output \(x'\) of vec(X) is the matrix \(X\) flattened in column-major order into a vector. Formally, \(x'_i = X_{i \bmod{m}, \left \lfloor{i/m}\right \rfloor }\).

The output \(X'\) of reshape(X, (m', n')) is the matrix \(X\) cast into an \(m' \times n'\) matrix. The entries are taken from \(X\) in column-major order and stored in \(X'\) in column-major order. Formally, \(X'_{ij} = \mathbf{vec}(X)_{m'j + i}\).