API Reference

Problem

Module containing pymanopt problem class. Use this to build a problem object to feed to one of the solvers.

class pymanopt.core.problem.Problem(manifold, cost, egrad=None, ehess=None, grad=None, hess=None, precon=None, verbosity=2)[source]

Bases: object

Problem class for setting up a problem to feed to one of the pymanopt solvers.

Attributes:
  • manifold

    Manifold to optimize over.

  • cost

    A callable which takes an element of manifold and returns a real number, or a symbolic Theano or TensorFlow expression. In case of a symbolic expression, the gradient (and if necessary the Hessian) are computed automatically if they are not explicitly given. We recommend you take this approach rather than calculating gradients and Hessians by hand.

  • grad

    grad(x) is the gradient of cost at x. This must take an element X of manifold and return an element of the tangent space to manifold at X. This is usually computed automatically and doesn’t need to be set by the user.

  • hess

    hess(x, a) is the directional derivative of grad at x, in direction a. It should return an element of the tangent space to manifold at x.

  • egrad

    The ‘Euclidean gradient’, egrad(x) should return the grad of cost in the usual sense, i.e. egrad(x) need not lie in the tangent space.

  • ehess

    The ‘Euclidean Hessian’, ehess(x, a) should return the directional derivative of egrad at x in direction a. This need not lie in the tangent space.

  • verbosity (2)

    Level of information printed by the solver while it operates, 0 is silent, 2 is most information.

property cost
property egrad
property grad
property ehess
property hess

Manifolds

Manifold

class pymanopt.manifolds.manifold.Manifold(name, dimension, point_layout=1)[source]

Bases: object

Abstract base class setting out a template for manifold classes. If you would like to extend Pymanopt with a new manifold, then your manifold should inherit from this class.

Not all methods are required by all solvers. In particular, first order gradient based solvers such as pymanopt.solvers.steepest_descent and pymanopt.solvers.conjugate_gradient require egrad2rgrad() to be implemented but not ehess2rhess(). Second order solvers such as pymanopt.solvers.trust_regions will require ehess2rhess().

All of these methods correspond closely to methods in Manopt. See http://www.manopt.org/tutorial.html#manifolds for more details on manifolds in Manopt, which are effectively identical to those in Pymanopt (all of the methods in this class have equivalents in Manopt with the same name).

property dim

The dimension of the manifold

property point_layout

The number of elements a point on a manifold consists of.

For most manifolds, which represent points as (potentially multi-dimensional) arrays, this will be 1, but other manifolds might represent points as tuples or lists of arrays. In this case, point_layout describes how many elements such tuples/lists contain.

property typicaldist

Returns the “scale” of the manifold. This is used by the trust-regions solver to determine default initial and maximal trust-region radii.

abstract inner(X, G, H)[source]

Returns the inner product (i.e., the Riemannian metric) between two tangent vectors G and H in the tangent space at X.

abstract proj(X, G)[source]

Projects a vector G in the ambient space on the tangent space at X.

abstract norm(X, G)[source]

Computes the norm of a tangent vector G in the tangent space at X.

abstract rand()[source]

Returns a random point on the manifold.

abstract randvec(X)[source]

Returns a random vector in the tangent space at X. This does not follow a specific distribution.

abstract zerovec(X)[source]

Returns the zero vector in the tangent space at X.

dist(X, Y)[source]

Returns the geodesic distance between two points X and Y on the manifold.

egrad2rgrad(X, G)[source]

Maps the Euclidean gradient G in the ambient space on the tangent space of the manifold at X. For embedded submanifolds, this is simply the projection of G on the tangent space at X.

ehess2rhess(X, G, H, U)[source]

Converts the Euclidean gradient G and Hessian H of a function at a point X along a tangent vector U to the Riemannian Hessian of X along U on the manifold.

retr(X, G)[source]

Computes a retraction mapping a vector G in the tangent space at X to the manifold.

exp(X, U)[source]

Computes the Lie-theoretic exponential map of a tangent vector U at X.

log(X, Y)[source]

Computes the Lie-theoretic logarithm of Y. This is the inverse of exp.

transp(X1, X2, G)[source]

Computes a vector transport which transports a vector G in the tangent space at X1 to the tangent space at X2.

pairmean(X, Y)[source]

Returns the intrinsic mean of two points X and Y on the manifold, i.e., a point that lies mid-way between X and Y on the geodesic arc joining them.

class pymanopt.manifolds.manifold.EuclideanEmbeddedSubmanifold(name, dimension, point_layout=1)[source]

Bases: pymanopt.manifolds.manifold.Manifold

A class to model embedded submanifolds of a Euclidean space. It provides a generic way to project Euclidean gradients to their Riemannian counterparts via the egrad2rgrad method. Similarly, if the Weingarten map (also known as shape operator) is provided via implementing the ‘weingarten’ method, the class provides a generic implementation of the ‘ehess2rhess’ method required by second-order solvers to translate Euclidean Hessian-vector products to their Riemannian counterparts.

Refer to [1]_ for the exact definition of the Weingarten map.

1

Absil, P-A., Robert Mahony, and Jochen Trumpf. “An extrinsic look at the Riemannian Hessian.” International Conference on Geometric Science of Information. Springer, Berlin, Heidelberg, 2013.

egrad2rgrad(X, G)[source]

Maps the Euclidean gradient G in the ambient space on the tangent space of the manifold at X. For embedded submanifolds, this is simply the projection of G on the tangent space at X.

ehess2rhess(X, G, H, U)[source]

Converts the Euclidean gradient G and Hessian H of a function at a point X along a tangent vector U to the Riemannian Hessian of X along U on the manifold. This uses the Weingarten map

weingarten(X, U, V)[source]

Evaluates the Weingarten map of the manifold. This map takes a vector U in the tangent space at X and a vector V in the normal space at X to produce another tangent vector.

Euclidean Space

class pymanopt.manifolds.euclidean.Euclidean(*shape)[source]

Bases: pymanopt.manifolds.euclidean._Euclidean

Euclidean manifold of shape n1 x n2 x … x nk tensors. Useful for unconstrained optimization problems or for unconstrained hyperparameters, as part of a product manifold.

Examples: Create a manifold of vectors of length n: manifold = Euclidean(n)

Create a manifold of m x n matrices: manifold = Euclidean(m, n)

class pymanopt.manifolds.euclidean.Symmetric(n, k=1)[source]

Bases: pymanopt.manifolds.euclidean._Euclidean

Manifold of n x n symmetric matrices, as a Riemannian submanifold of Euclidean space.

If k > 1 then this is an array of shape (k, n, n) (product manifold) containing k (n x n) matrices.

proj(X, U)[source]

Projects a vector G in the ambient space on the tangent space at X.

ehess2rhess(X, egrad, ehess, H)[source]

Converts the Euclidean gradient G and Hessian H of a function at a point X along a tangent vector U to the Riemannian Hessian of X along U on the manifold. This uses the Weingarten map

rand()[source]

Returns a random point on the manifold.

randvec(X)[source]

Returns a random vector in the tangent space at X. This does not follow a specific distribution.

class pymanopt.manifolds.euclidean.SkewSymmetric(n, k=1)[source]

Bases: pymanopt.manifolds.euclidean._Euclidean

The Euclidean space of n-by-n skew-symmetric matrices.

If k > 1 then this is an array of shape (k, n, n) (product manifold) containing k (n x n) matrices.

proj(X, U)[source]

Projects a vector G in the ambient space on the tangent space at X.

ehess2rhess(X, egrad, ehess, H)[source]

Converts the Euclidean gradient G and Hessian H of a function at a point X along a tangent vector U to the Riemannian Hessian of X along U on the manifold. This uses the Weingarten map

rand()[source]

Returns a random point on the manifold.

randvec(X)[source]

Returns a random vector in the tangent space at X. This does not follow a specific distribution.

The Grassmann Manifold

class pymanopt.manifolds.grassmann.Grassmann(n, p, k=1)[source]

Bases: pymanopt.manifolds.manifold.Manifold

Factory class for the Grassmann manifold. This is the manifold of p- dimensional subspaces of n dimensional real vector space. Initiation requires the dimensions n, p to be specified. Optional argument k allows the user to optimize over the product of k Grassmanns.

Elements are represented as n x p matrices (if k == 1), and as k x n x p matrices if k > 1 (Note that this is different to manopt!).

property typicaldist

Returns the “scale” of the manifold. This is used by the trust-regions solver to determine default initial and maximal trust-region radii.

dist(X, Y)[source]

Returns the geodesic distance between two points X and Y on the manifold.

inner(X, G, H)[source]

Returns the inner product (i.e., the Riemannian metric) between two tangent vectors G and H in the tangent space at X.

proj(X, U)[source]

Projects a vector G in the ambient space on the tangent space at X.

egrad2rgrad(X, U)

Projects a vector G in the ambient space on the tangent space at X.

ehess2rhess(X, egrad, ehess, H)[source]

Converts the Euclidean gradient G and Hessian H of a function at a point X along a tangent vector U to the Riemannian Hessian of X along U on the manifold.

retr(X, G)[source]

Computes a retraction mapping a vector G in the tangent space at X to the manifold.

norm(X, G)[source]

Computes the norm of a tangent vector G in the tangent space at X.

rand()[source]

Returns a random point on the manifold.

randvec(X)[source]

Returns a random vector in the tangent space at X. This does not follow a specific distribution.

transp(x1, x2, d)[source]

Computes a vector transport which transports a vector G in the tangent space at X1 to the tangent space at X2.

exp(X, U)[source]

Computes the Lie-theoretic exponential map of a tangent vector U at X.

log(X, Y)[source]

Computes the Lie-theoretic logarithm of Y. This is the inverse of exp.

zerovec(X)[source]

Returns the zero vector in the tangent space at X.

The Oblique Manifold

class pymanopt.manifolds.oblique.Oblique(m, n)[source]

Bases: pymanopt.manifolds.manifold.EuclideanEmbeddedSubmanifold

Manifold of matrices w/ unit-norm columns.

Oblique manifold: deals with matrices of size m-by-n such that each column has unit 2-norm, i.e., is a point on the unit sphere in R^m. The metric is such that the oblique manifold is a Riemannian submanifold of the space of m-by-n matrices with the usual trace inner product, i.e., the usual metric.

property typicaldist

Returns the “scale” of the manifold. This is used by the trust-regions solver to determine default initial and maximal trust-region radii.

inner(X, U, V)[source]

Returns the inner product (i.e., the Riemannian metric) between two tangent vectors G and H in the tangent space at X.

norm(X, U)[source]

Computes the norm of a tangent vector G in the tangent space at X.

dist(X, Y)[source]

Returns the geodesic distance between two points X and Y on the manifold.

proj(X, H)[source]

Projects a vector G in the ambient space on the tangent space at X.

ehess2rhess(X, egrad, ehess, U)[source]

Converts the Euclidean gradient G and Hessian H of a function at a point X along a tangent vector U to the Riemannian Hessian of X along U on the manifold. This uses the Weingarten map

exp(X, U)[source]

Computes the Lie-theoretic exponential map of a tangent vector U at X.

retr(X, U)[source]

Computes a retraction mapping a vector G in the tangent space at X to the manifold.

log(X, Y)[source]

Computes the Lie-theoretic logarithm of Y. This is the inverse of exp.

rand()[source]

Returns a random point on the manifold.

randvec(X)[source]

Returns a random vector in the tangent space at X. This does not follow a specific distribution.

transp(X, Y, U)[source]

Computes a vector transport which transports a vector G in the tangent space at X1 to the tangent space at X2.

pairmean(X, Y)[source]

Returns the intrinsic mean of two points X and Y on the manifold, i.e., a point that lies mid-way between X and Y on the geodesic arc joining them.

zerovec(X)[source]

Returns the zero vector in the tangent space at X.

The Product Manifold

class pymanopt.manifolds.product.Product(manifolds)[source]

Bases: pymanopt.manifolds.manifold.Manifold

Product manifold, i.e., the cartesian product of multiple manifolds.

property typicaldist

Returns the “scale” of the manifold. This is used by the trust-regions solver to determine default initial and maximal trust-region radii.

inner(X, G, H)[source]

Returns the inner product (i.e., the Riemannian metric) between two tangent vectors G and H in the tangent space at X.

norm(X, G)[source]

Computes the norm of a tangent vector G in the tangent space at X.

dist(X, Y)[source]

Returns the geodesic distance between two points X and Y on the manifold.

proj(X, U)[source]

Projects a vector G in the ambient space on the tangent space at X.

egrad2rgrad(X, U)[source]

Maps the Euclidean gradient G in the ambient space on the tangent space of the manifold at X. For embedded submanifolds, this is simply the projection of G on the tangent space at X.

ehess2rhess(X, egrad, ehess, H)[source]

Converts the Euclidean gradient G and Hessian H of a function at a point X along a tangent vector U to the Riemannian Hessian of X along U on the manifold.

exp(X, U)[source]

Computes the Lie-theoretic exponential map of a tangent vector U at X.

retr(X, U)[source]

Computes a retraction mapping a vector G in the tangent space at X to the manifold.

log(X, U)[source]

Computes the Lie-theoretic logarithm of Y. This is the inverse of exp.

rand()[source]

Returns a random point on the manifold.

randvec(X)[source]

Returns a random vector in the tangent space at X. This does not follow a specific distribution.

transp(X1, X2, G)[source]

Computes a vector transport which transports a vector G in the tangent space at X1 to the tangent space at X2.

pairmean(X, Y)[source]

Returns the intrinsic mean of two points X and Y on the manifold, i.e., a point that lies mid-way between X and Y on the geodesic arc joining them.

zerovec(X)[source]

Returns the zero vector in the tangent space at X.

The Manifolds of Positive Semidefinite Matrices

class pymanopt.manifolds.psd.SymmetricPositiveDefinite(n, k=1)[source]

Bases: pymanopt.manifolds.manifold.EuclideanEmbeddedSubmanifold

Manifold of (n x n)^k symmetric positive definite matrices, based on the geometry discussed in Chapter 6 of Positive Definite Matrices (Bhatia 2007). Some of the implementation is based on sympositivedefinitefactory.m from the Manopt MATLAB package. Also see “Conic geometric optimisation on the manifold of positive definite matrices” (Sra & Hosseini 2013) for more details.

property typicaldist

Returns the “scale” of the manifold. This is used by the trust-regions solver to determine default initial and maximal trust-region radii.

dist(x, y)[source]

Returns the geodesic distance between two points X and Y on the manifold.

inner(x, u, v)[source]

Returns the inner product (i.e., the Riemannian metric) between two tangent vectors G and H in the tangent space at X.

proj(X, G)[source]

Projects a vector G in the ambient space on the tangent space at X.

egrad2rgrad(x, u)[source]

Maps the Euclidean gradient G in the ambient space on the tangent space of the manifold at X. For embedded submanifolds, this is simply the projection of G on the tangent space at X.

ehess2rhess(x, egrad, ehess, u)[source]

Converts the Euclidean gradient G and Hessian H of a function at a point X along a tangent vector U to the Riemannian Hessian of X along U on the manifold. This uses the Weingarten map

norm(x, u)[source]

Computes the norm of a tangent vector G in the tangent space at X.

rand()[source]

Returns a random point on the manifold.

randvec(x)[source]

Returns a random vector in the tangent space at X. This does not follow a specific distribution.

transp(x1, x2, d)[source]

Computes a vector transport which transports a vector G in the tangent space at X1 to the tangent space at X2.

exp(x, u)[source]

Computes the Lie-theoretic exponential map of a tangent vector U at X.

retr(x, u)

Computes the Lie-theoretic exponential map of a tangent vector U at X.

log(x, y)[source]

Computes the Lie-theoretic logarithm of Y. This is the inverse of exp.

zerovec(x)[source]

Returns the zero vector in the tangent space at X.

class pymanopt.manifolds.psd.PSDFixedRank(n, k)[source]

Bases: pymanopt.manifolds.psd._PSDFixedRank

Manifold of n-by-n symmetric positive semidefinite matrices of rank k.

A point X on the manifold is parameterized as YY^T where Y is a matrix of size nxk. As such, X is symmetric, positive semidefinite. We restrict to full-rank Y’s, such that X has rank exactly k. The point X is numerically represented by Y (this is more efficient than working with X, which may be big). Tangent vectors are represented as matrices of the same size as Y, call them Ydot, so that Xdot = Y Ydot’ + Ydot Y. The metric is the canonical Euclidean metric on Y.

Since for any orthogonal Q of size k, it holds that (YQ)(YQ)’ = YY’, we “group” all matrices of the form YQ in an equivalence class. The set of equivalence classes is a Riemannian quotient manifold, implemented here.

Notice that this manifold is not complete: if optimization leads Y to be rank-deficient, the geometry will break down. Hence, this geometry should only be used if it is expected that the points of interest will have rank exactly k. Reduce k if that is not the case.

An alternative, complete, geometry for positive semidefinite matrices of rank k is described in Bonnabel and Sepulchre 2009, “Riemannian Metric and Geometric Mean for Positive Semidefinite Matrices of Fixed Rank”, SIAM Journal on Matrix Analysis and Applications.

The geometry implemented here is the simplest case of the 2010 paper: M. Journee, P.-A. Absil, F. Bach and R. Sepulchre, “Low-Rank Optimization on the Cone of Positive Semidefinite Matrices”. Paper link: http://www.di.ens.fr/~fbach/journee2010_sdp.pdf

class pymanopt.manifolds.psd.PSDFixedRankComplex(n, k)[source]

Bases: pymanopt.manifolds.psd._PSDFixedRank

Manifold of n x n complex Hermitian pos. semidefinite matrices of rank k.

Manifold of n-by-n complex Hermitian positive semidefinite matrices of fixed rank k. This follows the quotient geometry described in Sarod Yatawatta’s 2013 paper: “Radio interferometric calibration using a Riemannian manifold”, ICASSP.

Paper link: http://dx.doi.org/10.1109/ICASSP.2013.6638382.

A point X on the manifold M is parameterized as YY^*, where Y is a complex matrix of size nxk of full rank. For any point Y on the manifold M, given any kxk complex unitary matrix U, we say Y*U is equivalent to Y, i.e., YY^* does not change. Therefore, M is the set of equivalence classes and is a Riemannian quotient manifold C^{nk}/U(k) where C^{nk} is the set of all complex matrix of size nxk of full rank. The metric is the usual real-trace inner product, that is, it is the usual metric for the complex plane identified with R^2.

Notice that this manifold is not complete: if optimization leads Y to be rank-deficient, the geometry will break down. Hence, this geometry should only be used if it is expected that the points of interest will have rank exactly k. Reduce k if that is not the case.

inner(Y, U, V)[source]

Returns the inner product (i.e., the Riemannian metric) between two tangent vectors G and H in the tangent space at X.

norm(Y, U)[source]

Computes the norm of a tangent vector G in the tangent space at X.

dist(U, V)[source]

Returns the geodesic distance between two points X and Y on the manifold.

rand()[source]

Returns a random point on the manifold.

class pymanopt.manifolds.psd.Elliptope(n, k)[source]

Bases: pymanopt.manifolds.manifold.Manifold, pymanopt.manifolds.psd._RetrAsExpMixin

Manifold of n-by-n psd matrices of rank k with unit diagonal elements.

A point X on the manifold is parameterized as YY^T where Y is a matrix of size nxk. As such, X is symmetric, positive semidefinite. We restrict to full-rank Y’s, such that X has rank exactly k. The point X is numerically represented by Y (this is more efficient than working with X, which may be big). Tangent vectors are represented as matrices of the same size as Y, call them Ydot, so that Xdot = Y Ydot’ + Ydot Y and diag(Xdot) == 0. The metric is the canonical Euclidean metric on Y.

The diagonal constraints on X (X(i, i) == 1 for all i) translate to unit-norm constraints on the rows of Y: norm(Y(i, :)) == 1 for all i. The set of such Y’s forms the oblique manifold. But because for any orthogonal Q of size k, it holds that (YQ)(YQ)’ = YY’, we “group” all matrices of the form YQ in an equivalence class. The set of equivalence classes is a Riemannian quotient manifold, implemented here.

Note that this geometry formally breaks down at rank-deficient Y’s. This does not appear to be a major issue in practice when optimization algorithms converge to rank-deficient Y’s, but convergence theorems no longer hold. As an alternative, you may use the oblique manifold (it has larger dimension, but does not break down at rank drop.)

The geometry is taken from the 2010 paper: M. Journee, P.-A. Absil, F. Bach and R. Sepulchre, “Low-Rank Optimization on the Cone of Positive Semidefinite Matrices”. Paper link: http://www.di.ens.fr/~fbach/journee2010_sdp.pdf

property typicaldist

Returns the “scale” of the manifold. This is used by the trust-regions solver to determine default initial and maximal trust-region radii.

inner(Y, U, V)[source]

Returns the inner product (i.e., the Riemannian metric) between two tangent vectors G and H in the tangent space at X.

norm(Y, U)[source]

Computes the norm of a tangent vector G in the tangent space at X.

proj(Y, H)[source]

Projects a vector G in the ambient space on the tangent space at X.

retr(Y, U)[source]

Computes a retraction mapping a vector G in the tangent space at X to the manifold.

egrad2rgrad(Y, egrad)[source]

Maps the Euclidean gradient G in the ambient space on the tangent space of the manifold at X. For embedded submanifolds, this is simply the projection of G on the tangent space at X.

ehess2rhess(Y, egrad, ehess, U)[source]

Converts the Euclidean gradient G and Hessian H of a function at a point X along a tangent vector U to the Riemannian Hessian of X along U on the manifold.

rand()[source]

Returns a random point on the manifold.

randvec(Y)[source]

Returns a random vector in the tangent space at X. This does not follow a specific distribution.

transp(Y, Z, U)[source]

Computes a vector transport which transports a vector G in the tangent space at X1 to the tangent space at X2.

zerovec(X)[source]

Returns the zero vector in the tangent space at X.

The Sphere Manifold

class pymanopt.manifolds.sphere.Sphere(*shape)[source]

Bases: pymanopt.manifolds.sphere._Sphere

Manifold of shape n1 x n2 x … x nk tensors with unit 2-norm. The metric is such that the sphere is a Riemannian submanifold of Euclidean space.

The implementation of the Weingarten map is taken from [1]_.

1

Absil, P-A., Robert Mahony, and Jochen Trumpf. “An extrinsic look at the Riemannian Hessian.” International Conference on Geometric Science of Information. Springer, Berlin, Heidelberg, 2013.

class pymanopt.manifolds.sphere.SphereSubspaceIntersection(U)[source]

Bases: pymanopt.manifolds.sphere._SphereSubspaceIntersectionManifold

Manifold of n-dimensional unit 2-norm vectors intersecting the r-dimensional subspace of R^n spanned by the columns of the matrix U. This implementation is based on spheresubspacefactory.m from the Manopt MATLAB package.

class pymanopt.manifolds.sphere.SphereSubspaceComplementIntersection(U)[source]

Bases: pymanopt.manifolds.sphere._SphereSubspaceIntersectionManifold

Manifold of n-dimensional unit 2-norm vectors which are orthogonal to the r-dimensional subspace of R^n spanned by columns of the matrix U. This implementation is based on spheresubspacefactory.m from the Manopt MATLAB package.

The Stiefel Manifold

class pymanopt.manifolds.stiefel.Stiefel(n, p, k=1)[source]

Bases: pymanopt.manifolds.manifold.EuclideanEmbeddedSubmanifold

Factory class for the Stiefel manifold. Instantiation requires the dimensions n, p to be specified. Optional argument k allows the user to optimize over the product of k Stiefels.

Elements are represented as n x p matrices (if k == 1), and as k x n x p matrices if k > 1 (Note that this is different to manopt!).

property typicaldist

Returns the “scale” of the manifold. This is used by the trust-regions solver to determine default initial and maximal trust-region radii.

inner(X, G, H)[source]

Returns the inner product (i.e., the Riemannian metric) between two tangent vectors G and H in the tangent space at X.

proj(X, U)[source]

Projects a vector G in the ambient space on the tangent space at X.

ehess2rhess(X, egrad, ehess, H)[source]

Converts the Euclidean gradient G and Hessian H of a function at a point X along a tangent vector U to the Riemannian Hessian of X along U on the manifold. This uses the Weingarten map

retr(X, G)[source]

Computes a retraction mapping a vector G in the tangent space at X to the manifold.

norm(X, G)[source]

Computes the norm of a tangent vector G in the tangent space at X.

rand()[source]

Returns a random point on the manifold.

randvec(X)[source]

Returns a random vector in the tangent space at X. This does not follow a specific distribution.

transp(x1, x2, d)[source]

Computes a vector transport which transports a vector G in the tangent space at X1 to the tangent space at X2.

exp(X, U)[source]

Computes the Lie-theoretic exponential map of a tangent vector U at X.

zerovec(X)[source]

Returns the zero vector in the tangent space at X.

Solvers

Solver

class pymanopt.solvers.solver.Solver(maxtime=1000, maxiter=1000, mingradnorm=1e-06, minstepsize=1e-10, maxcostevals=5000, logverbosity=0)[source]

Bases: object

Abstract base class setting out template for solver classes.

abstract solve(problem, x=None)[source]

Solve the given pymanopt.core.problem.Problem (starting from a random initial guess if the optional argument x is not provided).

Riemannian Conjugate Gradients

class pymanopt.solvers.conjugate_gradient.ConjugateGradient(beta_type=2, orth_value=inf, linesearch=None, *args, **kwargs)[source]

Bases: pymanopt.solvers.solver.Solver

Module containing conjugate gradient algorithm based on conjugategradient.m from the manopt MATLAB package.

solve(problem, x=None, reuselinesearch=False)[source]

Perform optimization using nonlinear conjugate gradient method with linesearch. This method first computes the gradient of obj w.r.t. arg, and then optimizes by moving in a direction that is conjugate to all previous search directions. Arguments:

  • problem

    Pymanopt problem setup using the Problem class, this must have a .manifold attribute specifying the manifold to optimize over, as well as a cost and enough information to compute the gradient of that cost.

  • x=None

    Optional parameter. Starting point on the manifold. If none then a starting point will be randomly generated.

  • reuselinesearch=False

    Whether to reuse the previous linesearch object. Allows to use information from a previous solve run.

Returns:
  • x

    Local minimum of obj, or if algorithm terminated before convergence x will be the point at which it terminated.

The Nelder-Mead Algorithm

pymanopt.solvers.nelder_mead.compute_centroid(manifold, points)[source]

Compute the centroid of points on the manifold as Karcher mean.

class pymanopt.solvers.nelder_mead.NelderMead(maxcostevals=None, maxiter=None, reflection=1, expansion=2, contraction=0.5, *args, **kwargs)[source]

Bases: pymanopt.solvers.solver.Solver

Nelder-Mead minimization alglorithm for derivative-free minimization based on neldermead.m and centroid.m from the manopt MATLAB package.

solve(problem, x=None)[source]

Perform optimization using a Nelder-Mead minimization algorithm. Arguments:

  • problem

    Pymanopt problem setup using the Problem class, this must have a .manifold attribute specifying the manifold to optimize over, as well as a cost and enough information to compute the gradient of that cost.

  • x=None

    Optional parameter. Initial population of elements on the manifold. If None then an initial population will be randomly generated

Returns:
  • x

    Local minimum of obj, or if algorithm terminated before convergence x will be the point at which it terminated

Particle Swarms

class pymanopt.solvers.particle_swarm.ParticleSwarm(maxcostevals=None, maxiter=None, populationsize=None, nostalgia=1.4, social=1.4, *args, **kwargs)[source]

Bases: pymanopt.solvers.solver.Solver

Particle swarm optimization method based on pso.m from the manopt MATLAB package.

solve(problem, x=None)[source]

Perform optimization using the particle swarm optimization algorithm. Arguments:

  • problem

    Pymanopt problem setup using the Problem class, this must have a .manifold attribute specifying the manifold to optimize over, as well as a cost (specified using a theano graph or as a python function).

  • x=None

    Optional parameter. Initial population of elements on the manifold. If None then an initial population will be randomly generated

Returns:
  • x

    Local minimum of obj, or if algorithm terminated before convergence x will be the point at which it terminated

The Riemannian Steepest Descent

class pymanopt.solvers.steepest_descent.SteepestDescent(linesearch=None, *args, **kwargs)[source]

Bases: pymanopt.solvers.solver.Solver

Steepest descent (gradient descent) algorithm based on steepestdescent.m from the manopt MATLAB package.

solve(problem, x=None, reuselinesearch=False)[source]

Perform optimization using gradient descent with linesearch. This method first computes the gradient (derivative) of obj w.r.t. arg, and then optimizes by moving in the direction of steepest descent (which is the opposite direction to the gradient). Arguments:

  • problem

    Pymanopt problem setup using the Problem class, this must have a .manifold attribute specifying the manifold to optimize over, as well as a cost and enough information to compute the gradient of that cost.

  • x=None

    Optional parameter. Starting point on the manifold. If none then a starting point will be randomly generated.

  • reuselinesearch=False

    Whether to reuse the previous linesearch object. Allows to use information from a previous solve run.

Returns:
  • x

    Local minimum of obj, or if algorithm terminated before convergence x will be the point at which it terminated.

Second-Order Riemannian Trust Regions

class pymanopt.solvers.trust_regions.TrustRegions(miniter=3, kappa=0.1, theta=1.0, rho_prime=0.1, use_rand=False, rho_regularization=1000.0, *args, **kwargs)[source]

Bases: pymanopt.solvers.solver.Solver

NEGATIVE_CURVATURE = 0
EXCEEDED_TR = 1
REACHED_TARGET_LINEAR = 2
REACHED_TARGET_SUPERLINEAR = 3
MAX_INNER_ITER = 4
MODEL_INCREASED = 5
TCG_STOP_REASONS = {0: 'negative curvature', 1: 'exceeded trust region', 2: 'reached target residual-kappa (linear)', 3: 'reached target residual-theta (superlinear)', 4: 'maximum inner iterations', 5: 'model increased'}
solve(problem, x=None, mininner=1, maxinner=None, Delta_bar=None, Delta0=None)[source]

Solve the given pymanopt.core.problem.Problem (starting from a random initial guess if the optional argument x is not provided).

Line-Search Methods

class pymanopt.solvers.linesearch.LineSearchBackTracking(contraction_factor=0.5, optimism=2, suff_decr=0.0001, maxiter=25, initial_stepsize=1)[source]

Bases: object

Back-tracking line-search based on linesearch.m in the manopt MATLAB package.

search(objective, manifold, x, d, f0, df0)[source]

Function to perform backtracking line-search. Arguments:

  • objective

    objective function to optimise

  • manifold

    manifold to optimise over

  • x

    starting point on the manifold

  • d

    tangent vector at x (descent direction)

  • df0

    directional derivative at x along d

Returns:
  • stepsize

    norm of the vector retracted to reach newx from x

  • newx

    next iterate suggested by the line-search

class pymanopt.solvers.linesearch.LineSearchAdaptive(contraction_factor=0.5, suff_decr=0.5, maxiter=10, initial_stepsize=1)[source]

Bases: object

Adaptive line-search

search(objective, man, x, d, f0, df0)[source]

Autodiff Backends

Tools

pymanopt.tools.multi.multiprod(A, B)[source]

Inspired by MATLAB multiprod function by Paolo de Leva. A and B are assumed to be arrays containing M matrices, that is, A and B have dimensions A: (M, N, P), B:(M, P, Q). multiprod multiplies each matrix in A with the corresponding matrix in B, using matrix multiplication. so multiprod(A, B) has dimensions (M, N, Q).

pymanopt.tools.multi.multitransp(A)[source]

Inspired by MATLAB multitransp function by Paolo de Leva. A is assumed to be an array containing M matrices, each of which has dimension N x P. That is, A is an M x N x P array. Multitransp then returns an array containing the M matrix transposes of the matrices in A, each of which will be P x N.

pymanopt.tools.multi.multisym(A)[source]
pymanopt.tools.multi.multiskew(A)[source]
pymanopt.tools.multi.multieye(k, n)[source]
pymanopt.tools.multi.multilog(A, pos_def=False)[source]
pymanopt.tools.multi.multiexp(A, sym=False)[source]

Testing

Module containing tools for testing correctness in Pymanopt. Note, these currently require autograd.

Note: the methods for generating rgrad, egrad2rgrad, ehess and ehess2rhess will only be correct if the manifold is a submanifold of Euclidean space, that is if the projection is an orthogonal projection onto the tangent space.

pymanopt.tools.testing.rgrad(cost, proj)[source]

Generates the Riemannain gradient of cost. Cost must be defined using autograd.numpy.

pymanopt.tools.testing.egrad2rgrad(proj)[source]

Generates an egrad2rgrad function.

pymanopt.tools.testing.rhess(cost, proj)[source]

Generates the Riemannian hessian of the cost. Specifically, rhess(cost, proj)(x, u) is the directional derivatative of cost at point X on the manifold, in direction u. cost and proj must be defined using autograd.numpy. See http://sites.uclouvain.be/absil/2013-01/Weingarten_07PA_techrep.pdf for some discussion. proj and cost must be defined using autograd. Currently this is correct but not efficient, because of the jacobian- vector product. Hopefully this can be fixed in future.

pymanopt.tools.testing.ehess2rhess(proj)[source]

Generates an ehess2rhess function for a manifold which is a sub-manifold of Euclidean space. ehess2rhess(proj)(x, egrad, ehess, u) converts the Euclidean hessian ehess at the point x to a Riemannian hessian. That is the directional derivatative of the gradient in the direction u. proj must be defined using autograd.numpy. This will not be an efficient implementation because of missing support for efficient jacobian-vector products in autograd.