Atomic Functions

This section of the tutorial describes the atomic functions that can be applied to CVXPY expressions. CVXPY uses the function information in this section and the DCP rules to mark expressions with a sign and curvature.

Operators

The infix operators +, -, *, /, @ are treated as functions. The operators + and - are always affine functions. The expression expr1*expr2 is affine in CVXPY when one of the expressions is constant, and expr1/expr2 is affine when expr2 is a scalar constant.

Historically, CVXPY used expr1 * expr2 to denote matrix multiplication. This is now deprecated. Starting with Python 3.5, users can write expr1 @ expr2 for matrix multiplication and dot products. As of CVXPY version 1.1, we are adopting a new standard:

  • @ should be used for matrix-matrix and matrix-vector multiplication,

  • * should be matrix-scalar and vector-scalar multiplication

Elementwise multiplication can be applied with the multiply function.

Indexing and slicing

Indexing in CVXPY follows exactly the same semantics as NumPy ndarrays. For example, if expr has shape (5,) then expr[1] gives the second entry. More generally, expr[i:j:k] selects every kth element of expr, starting at i and ending at j-1. If expr is a matrix, then expr[i:j:k] selects rows, while expr[i:j:k, r:s:t] selects both rows and columns. Indexing drops dimensions while slicing preserves dimensions. For example,

x = cvxpy.Variable(5)
print("0 dimensional", x[0].shape)
print("1 dimensional", x[0:1].shape)
O dimensional: ()
1 dimensional: (1,)

Transpose

The transpose of any expression can be obtained using the syntax expr.T. Transpose is an affine function.

Power

For any CVXPY expression expr, the power operator expr**p is equivalent to the function power(expr, p).

Scalar functions

A scalar function takes one or more scalars, vectors, or matrices as arguments and returns a scalar.

Function

Meaning

Domain

DCP Properties

Curvature 

cvar(x, beta)

average of the \((1-\beta)\)

fraction of largest values in \(x\)

\(x \in \mathbf{R}^m\)

\(\beta \in (0,1)\)

sign depends on \(x\)

incr incr.

convex convex

dotsort(X,W)

constant \(W \in \mathbf{R}^{o \times p}\)

\(\text{dot product of}\) \(\operatorname{sort}\operatorname{vec}(X) \text{ and}\) \(\operatorname{sort}\operatorname{vec}(W)\)

\(X \in \mathbf{R}^{m \times n}\)

sign depends on

\(X\) and \(W\)

incr for \(\min(W) \geq 0\)

decr for \(\max(W) \leq 0\)

convex convex

geo_mean(x)

geo_mean(x, p)

\(p \in \mathbf{R}^n_{+}\)

\(p \neq 0\)

\(x_1^{1/n} \cdots x_n^{1/n}\)

\(\left(x_1^{p_1} \cdots x_n^{p_n}\right)^{\frac{1}{\mathbf{1}^T p}}\)

\(x \in \mathbf{R}^n_{+}\)

positive positive

incr incr.

concave concave

harmonic_mean(x)

\(\frac{n}{\frac{1}{x_1} + \cdots + \frac{1}{x_n}}\)

\(x \in \mathbf{R}^n_{+}\)

positive positive

incr incr.

concave concave

inv_prod(x)

\((x_1\cdots x_n)^{-1}\)

\(x \in \mathbf{R}^n_+\)

positive positive

decr decr.

convex convex

lambda_max(X)

\(\lambda_{\max}(X)\)

\(X \in \mathbf{S}^n\)

unknown unknown sign

convex convex

lambda_min(X)

\(\lambda_{\min}(X)\)

\(X \in \mathbf{S}^n\)

unknown unknown sign

concave concave

lambda_sum_largest(X,k)

\(k = 1,\ldots, n\)

\(\text{sum of $k$ largest}\) \(\text{eigenvalues of $X$}\)

\(X \in\mathbf{S}^{n}\)

unknown unknown sign

convex convex

lambda_sum_smallest(X,k)

\(k = 1,\ldots, n\)

\(\text{sum of $k$ smallest}\) \(\text{eigenvalues of $X$}\)

\(X \in\mathbf{S}^{n}\)

unknown unknown sign

concave concave

log_det(X)

\(\log \left(\det (X)\right)\)

\(X \in \mathbf{S}^n_+\)

unknown unknown sign

concave concave

log_sum_exp(X)

\(\log \left(\sum_{ij}e^{X_{ij}}\right)\)

\(X \in\mathbf{R}^{m \times n}\)

unknown unknown sign

incr incr.

convex convex

matrix_frac(x, P)

\(x^T P^{-1} x\)

\(x \in \mathbf{R}^n\)

\(P \in\mathbf{S}^n_{++}\)

positive positive

convex convex

max(X)

\(\max_{ij}\left\{ X_{ij}\right\}\)

\(X \in\mathbf{R}^{m \times n}\)

same sign as X

incr incr.

convex convex

mean(X)

\(\frac{1}{m n}\sum_{ij}\left\{ X_{ij}\right\}\)

\(X \in\mathbf{R}^{m \times n}\)

same sign as X

incr incr.

affine affine

min(X)

\(\min_{ij}\left\{ X_{ij}\right\}\)

\(X \in\mathbf{R}^{m \times n}\)

same sign as X

incr incr.

concave concave

mixed_norm(X, p, q)

\(Y = \left(\sum_l\lvert x_{k,l}\rvert^p\right)\)

\(\left(\sum_k Y^{q/p}\right)^{1/q}\)

\(X \in\mathbf{R}^{n \times n}\)

positive positive

convex convex

norm(x)

norm(x, 2)

\(\sqrt{\sum_{i} \lvert x_{i} \rvert^2 }\)

\(X \in\mathbf{R}^{n}\)

positive positive

incr for \(x_{i} \geq 0\)

decr for \(x_{i} \leq 0\)

convex convex

norm(x, 1)

\(\sum_{i}\lvert x_{i} \rvert\)

\(x \in\mathbf{R}^{n}\)

positive positive

incr for \(x_{i} \geq 0\)

decr for \(x_{i} \leq 0\)

convex convex

norm(x, “inf”)

\(\max_{i} \{\lvert x_{i} \rvert\}\)

\(x \in\mathbf{R}^{n}\)

positive positive

incr for \(x_{i} \geq 0\)

decr for \(x_{i} \leq 0\)

convex convex

norm(X, “fro”)

\(\sqrt{\sum_{ij}X_{ij}^2 }\)

\(X \in\mathbf{R}^{m \times n}\)

positive positive

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

convex convex

norm(X, 1)

\(\max_{j} \|X_{:,j}\|_1\)

\(X \in\mathbf{R}^{m \times n}\)

positive positive

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

convex convex

norm(X, “inf”)

\(\max_{i} \|X_{i,:}\|_1\)

\(X \in\mathbf{R}^{m \times n}\)

positive positive

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

convex convex

norm(X, “nuc”)

\(\mathrm{tr}\left(\left(X^T X\right)^{1/2}\right)\)

\(X \in\mathbf{R}^{m \times n}\)

positive positive

convex convex

norm(X)

norm(X, 2)

\(\sqrt{\lambda_{\max}\left(X^T X\right)}\)

\(X \in\mathbf{R}^{m \times n}\)

positive positive

convex convex

perspective(f(x),s)

\(sf(x/s)\)

\(x \in \mathop{\bf dom} f\)

\(s \geq 0\)

same sign as f

convex / concave

same as \(f\)

pnorm(X, p)

\(p \geq 1\)

or p = 'inf'

\(\left(\sum_{ij} |X_{ij}|^p \right)^{1/p}\)

\(X \in \mathbf{R}^{m \times n}\)

positive positive

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

convex convex

pnorm(X, p)

\(p < 1\), \(p \neq 0\)

\(\left(\sum_{ij} X_{ij}^p \right)^{1/p}\)

\(X \in \mathbf{R}^{m \times n}_+\)

positive positive

incr incr.

concave concave

ptp(X)

\(\max_{ij} X_{ij}\)

\(- \min_{ij} X_{ij}\)

\(X \in \mathbf{R}^{m \times n}\)

positive positive

convex convex

quad_form(x, P)

constant \(P \in \mathbf{S}^n_+\)

\(x^T P x\)

\(x \in \mathbf{R}^n\)

positive positive

incr for \(x_i \geq 0\)

decr for \(x_i \leq 0\)

convex convex

quad_form(x, P)

constant \(P \in \mathbf{S}^n_-\)

\(x^T P x\)

\(x \in \mathbf{R}^n\)

negative negative

decr for \(x_i \geq 0\)

incr for \(x_i \leq 0\)

concave concave

quad_form(c, X)

constant \(c \in \mathbf{R}^n\)

\(c^T X c\)

\(X \in\mathbf{R}^{n \times n}\)

sign depends on c, X

monotonicity depends on c

affine affine

quad_over_lin(X, y)

\(\left(\sum_{ij}X_{ij}^2\right)/y\)

\(x \in \mathbf{R}^n\)

\(y > 0\)

positive positive

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

decr decr. in \(y\)

convex convex

std(X)

analog to numpy.std

\(X \in\mathbf{R}^{m \times n}\)

positive positive

convex convex

sum(X)

\(\sum_{ij}X_{ij}\)

\(X \in\mathbf{R}^{m \times n}\)

same sign as X

incr incr.

affine affine

sum_largest(X, k)

\(k = 1,2,\ldots\)

\(\text{sum of } k\text{ largest }X_{ij}\)

\(X \in\mathbf{R}^{m \times n}\)

same sign as X

incr incr.

convex convex

sum_smallest(X, k)

\(k = 1,2,\ldots\)

\(\text{sum of } k\text{ smallest }X_{ij}\)

\(X \in\mathbf{R}^{m \times n}\)

same sign as X

incr incr.

concave concave

sum_squares(X)

\(\sum_{ij}X_{ij}^2\)

\(X \in\mathbf{R}^{m \times n}\)

positive positive

incr for \(X_{ij} \geq 0\)

decr for \(X_{ij} \leq 0\)

convex convex

trace(X)

\(\mathrm{tr}\left(X \right)\)

\(X \in\mathbf{R}^{n \times n}\)

same sign as X

incr incr.

affine affine

tr_inv(X)

\(\mathrm{tr}\left(X^{-1} \right)\)

\(X \in\mathbf{S}^n_{++}\)

positive positive

convex convex

tv(x)

\(\sum_{i}|x_{i+1} - x_i|\)

\(x \in \mathbf{R}^n\)

positive positive

convex convex

tv(X) \(Y = \left[\begin{matrix} X_{i+1,j} - X_{ij} \\ X_{i,j+1} -X_{ij} \end{matrix}\right]\)

\(\sum_{ij}\left\| Y \right\|_2\)

\(X \in \mathbf{R}^{m \times n}\)

positive positive

convex convex

tv([X1,…,Xk]) \(Y = \left[\begin{matrix} X_{i+1,j}^{(1)} - X_{ij}^{(1)} \\ X_{i,j+1}^{(1)} -X_{ij}^{(1)} \\ \vdots \\ X_{i+1,j}^{(k)} - X_{ij}^{(k)} \\ X_{i,j+1}^{(k)} -X_{ij}^{(k)} \end{matrix}\right]\)

\(\sum_{ij}\left\| Y \right\|_2\)

\(X^{(i)} \in\mathbf{R}^{m \times n}\)

positive positive

convex convex

var(X)

analog to numpy.var

\(X \in\mathbf{R}^{m \times n}\)

positive positive

convex convex

von_neumann_entr(X)

\(-\operatorname{tr}(X\operatorname{logm}(X))\)

\(X \in \mathbf{S}^{n}_+\)

unknown unknown sign

concave concave

Clarifications for scalar functions

The domain \(\mathbf{S}^n\) refers to the set of symmetric matrices. The domains \(\mathbf{S}^n_+\) and \(\mathbf{S}^n_-\) refer to the set of positive semi-definite and negative semi-definite matrices, respectively. Similarly, \(\mathbf{S}^n_{++}\) and \(\mathbf{S}^n_{--}\) refer to the set of positive definite and negative definite matrices, respectively.

For a vector expression x, norm(x) and norm(x, 2) give the Euclidean norm. For a matrix expression X, however, norm(X) and norm(X, 2) give the spectral norm.

The function norm(X, "fro") is called the Frobenius norm and norm(X, "nuc") the nuclear norm. The nuclear norm can also be defined as the sum of X’s singular values.

The functions max and min give the largest and smallest entry, respectively, in a single expression. These functions should not be confused with maximum and minimum (see Elementwise functions). Use maximum and minimum to find the max or min of a list of scalar expressions.

The CVXPY function sum sums all the entries in a single expression. The built-in Python sum should be used to add together a list of expressions. For example, the following code sums a list of three expressions:

expr_list = [expr1, expr2, expr3]
expr_sum = sum(expr_list)

Functions along an axis

The functions sum, norm, max, min, mean, std, var, and ptp can be applied along an axis. Given an m by n expression expr, the syntax func(expr, axis=0, keepdims=True) applies func to each column, returning a 1 by n expression. The syntax func(expr, axis=1, keepdims=True) applies func to each row, returning an m by 1 expression. By default keepdims=False, which means dimensions of length 1 are dropped. For example, the following code sums along the columns and rows of a matrix variable:

X = cvxpy.Variable((5, 4))
col_sums = cvxpy.sum(X, axis=0, keepdims=True) # Has size (1, 4)
col_sums = cvxpy.sum(X, axis=0) # Has size (4,)
row_sums = cvxpy.sum(X, axis=1) # Has size (5,)

Elementwise functions

These functions operate on each element of their arguments. For example, if X is a 5 by 4 matrix variable, then abs(X) is a 5 by 4 matrix expression. abs(X)[1, 2] is equivalent to abs(X[1, 2]).

Elementwise functions that take multiple arguments, such as maximum and multiply, operate on the corresponding elements of each argument. For example, if X and Y are both 3 by 3 matrix variables, then maximum(X, Y) is a 3 by 3 matrix expression. maximum(X, Y)[2, 0] is equivalent to maximum(X[2, 0], Y[2, 0]). This means all arguments must have the same dimensions or be scalars, which are promoted.

Function

Meaning

Domain

DCP Properties

Curvature 

abs(x)

\(\lvert x \rvert\)

\(x \in \mathbf{C}\)

positive positive

incr for \(x \geq 0\)

decr for \(x \leq 0\)

convex convex

conj(x)

complex conjugate

\(x \in \mathbf{C}\)

unknown unknown sign

affine affine

entr(x)

\(-x \log (x)\)

\(x > 0\)

unknown unknown sign

concave concave

exp(x)

\(e^x\)

\(x \in \mathbf{R}\)

positive positive

incr incr.

convex convex

huber(x, M=1)

\(M \geq 0\)

\(\begin{cases}x^2 &|x| \leq M \\2M|x| - M^2&|x| >M\end{cases}\)

\(x \in \mathbf{R}\)

positive positive

incr for \(x \geq 0\)

decr for \(x \leq 0\)

convex convex

imag(x)

imaginary part of a complex number

\(x \in \mathbf{C}\)

unknown unknown sign

affine affine

inv_pos(x)

\(1/x\)

\(x > 0\)

positive positive

decr decr.

convex convex

kl_div(x, y)

\(x \log(x/y) - x + y\)

\(x > 0\)

\(y > 0\)

positive positive

convex convex

log(x)

\(\log(x)\)

\(x > 0\)

unknown unknown sign

incr incr.

concave concave

log_normcdf(x)

approximate log of the standard normal CDF

\(x \in \mathbf{R}\)

negative negative

incr incr.

concave concave

log1p(x)

\(\log(x+1)\)

\(x > -1\)

same sign as x

incr incr.

concave concave

loggamma(x)

approximate log of the Gamma function

\(x > 0\)

unknown unknown sign

convex convex

logistic(x)

\(\log(1 + e^{x})\)

\(x \in \mathbf{R}\)

positive positive

incr incr.

convex convex

maximum(x, y)

\(\max \left\{x, y\right\}\)

\(x,y \in \mathbf{R}\)

sign depends on x,y

incr incr.

convex convex

minimum(x, y)

\(\min \left\{x, y\right\}\)

\(x, y \in \mathbf{R}\)

sign depends on x,y

incr incr.

concave concave

multiply(c, x)

\(c \in \mathbf{R}\)

c*x

\(x \in\mathbf{R}\)

\(\mathrm{sign}(cx)\)

monotonicity depends on c

affine affine

neg(x)

\(\max \left\{-x, 0 \right\}\)

\(x \in \mathbf{R}\)

positive positive

decr decr.

convex convex

pos(x)

\(\max \left\{x, 0 \right\}\)

\(x \in \mathbf{R}\)

positive positive

incr incr.

convex convex

power(x, 0)

\(1\)

\(x \in \mathbf{R}\)

positive positive

constant

power(x, 1)

\(x\)

\(x \in \mathbf{R}\)

same sign as x

incr incr.

affine affine

power(x, p)

\(p = 2, 4, 8, \ldots\)

\(x^p\)

\(x \in \mathbf{R}\)

positive positive

incr for \(x \geq 0\)

decr for \(x \leq 0\)

convex convex

power(x, p)

\(p < 0\)

\(x^p\)

\(x > 0\)

positive positive

decr decr.

convex convex

power(x, p)

\(0 < p < 1\)

\(x^p\)

\(x \geq 0\)

positive positive

incr incr.

concave concave

power(x, p)

\(p > 1,\ p \neq 2, 4, 8, \ldots\)

\(x^p\)

\(x \geq 0\)

positive positive

incr incr.

convex convex

real(x)

real part of a complex number

\(x \in \mathbf{C}\)

unknown unknown

incr incr.

affine affine

rel_entr(x, y)

\(x \log(x/y)\)

\(x > 0\)

\(y > 0\)

unknown unknown sign

decr in \(y\)

convex convex

scalene(x, alpha, beta)

\(\text{alpha} \geq 0\)

\(\text{beta} \geq 0\)

\(\alpha\mathrm{pos}(x)+ \beta\mathrm{neg}(x)\)

\(x \in \mathbf{R}\)

positive positive

incr for \(x \geq 0\)

decr for \(x \leq 0\)

convex convex

sqrt(x)

\(\sqrt x\)

\(x \geq 0\)

positive positive

incr incr.

concave concave

square(x)

\(x^2\)

\(x \in \mathbf{R}\)

positive positive

incr for \(x \geq 0\)

decr for \(x \leq 0\)

convex convex

xexp(x)

\(x e^x\)

\(x \geq 0\)

positive positive

incr incr.

convex convex

Clarifications on elementwise functions

The functions log_normcdf and loggamma are defined via approximations. log_normcdf has highest accuracy over the range -4 to 4, while loggamma has similar accuracy over all positive reals. See CVXPY GitHub PR #1224 and CVXPY GitHub Issue #228 for details on the approximations.

Vector/matrix functions

A vector/matrix function takes one or more scalars, vectors, or matrices as arguments and returns a vector or matrix.

CVXPY is conservative when it determines the sign of an Expression returned by one of these functions. If any argument to one of these functions has unknown sign, then the returned Expression will also have unknown sign. If all arguments have known sign but CVXPY can determine that the returned Expression would have different signs in different entries (for example, when stacking a positive Expression and a negative Expression) then the returned Expression will have unknown sign.

Function

Meaning

Domain

Curvature 

Monotonicity

bmat([[X11,…,X1q], …, [Xp1,…,Xpq]])

\(\left[\begin{matrix} X^{(1,1)} & \cdots & X^{(1,q)} \\ \vdots & & \vdots \\ X^{(p,1)} & \cdots & X^{(p,q)} \end{matrix}\right]\)

\(X^{(i,j)} \in\mathbf{R}^{m_i \times n_j}\)

affine affine

incr incr.

convolve(c, x)

\(c\in\mathbf{R}^m\)

\(c*x\)

\(x\in \mathbf{R}^n\)

affine affine

depends on c

cumsum(X, axis=0)

cumulative sum along given axis.

\(X \in \mathbf{R}^{m \times n}\)

affine affine

incr incr.

diag(x)

\(\left[\begin{matrix}x_1 & & \\& \ddots & \\& & x_n\end{matrix}\right]\)

\(x \in\mathbf{R}^{n}\)

affine affine

incr incr.

diag(X)

\(\left[\begin{matrix}X_{11} \\\vdots \\X_{nn}\end{matrix}\right]\)

\(X \in\mathbf{R}^{n \times n}\)

affine affine

incr incr.

diff(X, k=1, axis=0)

\(k \in 0,1,2,\ldots\)

kth order differences along given axis

\(X \in\mathbf{R}^{m \times n}\)

affine affine

incr incr.

hstack([X1, …, Xk])

\(\left[\begin{matrix}X^{(1)} \cdots X^{(k)}\end{matrix}\right]\)

\(X^{(i)} \in\mathbf{R}^{m \times n_i}\)

affine affine

incr incr.

kron(X, Y)

constant \(X\in\mathbf{R}^{p \times q}\)

\(\left[\begin{matrix}X_{11}Y & \cdots & X_{1q}Y \\ \vdots & & \vdots \\ X_{p1}Y & \cdots & X_{pq}Y \end{matrix}\right]\)

\(Y \in \mathbf{R}^{m \times n}\)

affine affine

depends on \(X\)

kron(X, Y)

constant \(Y\in\mathbf{R}^{m \times n}\)

\(\left[\begin{matrix}X_{11}Y & \cdots & X_{1q}Y \\ \vdots & & \vdots \\ X_{p1}Y & \cdots & X_{pq}Y \end{matrix}\right]\)

\(X \in \mathbf{R}^{p \times q}\)

affine affine

depends on \(Y\)

outer(x, y)

constant \(y \in \mathbf{R}^m\)

\(x y^T\)

\(x \in \mathbf{R}^n\)

affine affine

depends on \(y\)

partial_trace(X, dims, axis=0)

partial trace

\(X \in\mathbf{R}^{n \times n}\)

affine affine

incr incr.

partial_transpose(X, dims, axis=0)

partial transpose

\(X \in\mathbf{R}^{n \times n}\)

affine affine

incr incr.

reshape(X, (m’, n’), order=’F’)

\(X' \in\mathbf{R}^{m' \times n'}\)

\(X \in\mathbf{R}^{m \times n}\)

\(m'n' = mn\)

affine affine

incr incr.

upper_tri(X)

flatten the strictly upper-triangular part of \(X\)

\(X \in \mathbf{R}^{n \times n}\)

affine affine

incr incr.

vec(X)

\(x' \in\mathbf{R}^{mn}\)

\(X \in\mathbf{R}^{m \times n}\)

affine affine

incr incr.

vec_to_upper_tri(X, strict=False)

\(x' \in\mathbf{R}^{n(n-1)/2}\) for strict=True

\(x' \in\mathbf{R}^{n(n+1)/2}\) for strict=False

\(X \in\mathbf{R}^{n \times n}\)

affine affine

incr incr.

vstack([X1, …, Xk])

\(\left[\begin{matrix}X^{(1)} \\ \vdots \\X^{(k)}\end{matrix}\right]\)

\(X^{(i)} \in\mathbf{R}^{m_i \times n}\)

affine affine

incr incr.

Clarifications on vector and matrix functions

The input to \(\texttt{bmat}\) is a list of lists of CVXPY expressions. It constructs a block matrix. The elements of each inner list are stacked horizontally and then the resulting block matrices are stacked vertically.

The output \(y = \texttt{convolve}(c, x)\) has size \(n+m-1\) and is defined as \(y_k =\sum_{j=0}^{k} c[j]x[k-j]\).

The output \(y = \texttt{vec}(X)\) is the matrix \(X\) flattened in column-major order into a vector. Formally, \(y_i = X_{i \bmod{m}, \left \lfloor{i/m}\right \rfloor }\).

The output \(Y = \texttt{reshape}(X, (m', n'), \text{order='F'})\) is the matrix \(X\) cast into an \(m' \times n'\) matrix. The entries are taken from \(X\) in column-major order and stored in \(Y\) in column-major order. Formally, \(Y_{ij} = \texttt{vec}(X)_{m'j + i}\). If order=’C’ then \(X\) will be read in row-major order and \(Y\) will be written to in row-major order.

The output \(y = \texttt{upper_tri}(X)\) is formed by concatenating partial rows of \(X\). I.e., \(y = (X[0,1{:}],\, X[1, 2{:}],\, \ldots, X[n-1, n])\).