pybop.optimisers.pints_optimisers#

Classes#

AdamW

Adapter for adaptive moment estimation with weight decay (AdamW), a variant of the Adam

CMAES

Adapter for the covariance matrix adaptation evolution strategy (CMA-ES), an evolutionary

CuckooSearch

Adapter for cuckoo search, a population-based optimisation algorithm inspired by the brood

GradientDescent

Adapter for gradient descent, a canonical method that takes steps in the opposite direction

IRPropMin

Adapter for improved resilient backpropagation (without weight-backtracking), an optimisation

IRPropPlus

Adapter for improved resilient backpropagation with weight-backtracking, an optimisation

NelderMead

Adpater for the Nelder-Mead downhill simplex method, a deterministic local optimiser that does

PSO

Adapter for particle swarm optimisation (PSO), a metaheuristic optimisation method inspired by

RandomSearch

Adapter for random search, a simple algorithm which samples parameter values randomly and

SNES

Adapter for the stochastic natural evolution strategy (SNES), an evolutionary algorithm that

SimulatedAnnealing

Adapter for simulated annealing, a probabilistic optimisation method inspired by the annealing

XNES

Adapter for the exponential natural evolution strategy (XNES), an evolutionary algorithm that

Module Contents#

class pybop.optimisers.pints_optimisers.AdamW(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for adaptive moment estimation with weight decay (AdamW), a variant of the Adam optimiser which does not support boundary constraints.

This optimiser is designed to be more robust and stable for training deep neural networks, particularly when using larger learning rates.

Parameters:

See also

pybop.AdamWImpl

The PyBOP implementation this class is based on.

class pybop.optimisers.pints_optimisers.CMAES(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for the covariance matrix adaptation evolution strategy (CMA-ES), an evolutionary algorithm for difficult non-linear non-convex optimisation problems.

It adapts the covariance matrix of a multivariate normal distribution to capture the shape of the cost landscape.

Parameters:

See also

pints.CMAES

PINTS implementation of CMA-ES algorithm.

class pybop.optimisers.pints_optimisers.CuckooSearch(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for cuckoo search, a population-based optimisation algorithm inspired by the brood parasitism of some cuckoo species which is suitable for global optimisation problems.

Cuckoo search is designed to be simple, efficient, and robust. It explores the search space by randomly suggesting candidate “nests” and abandoning poorly performing “nests” throughout the process.

Parameters:

See also

pybop.CuckooSearchImpl

PyBOP implementation of Cuckoo Search algorithm.

class pybop.optimisers.pints_optimisers.GradientDescent(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for gradient descent, a canonical method that takes steps in the opposite direction of the cost gradient with respect to the parameters (does not support boundary constraints).

Gradient descent is designed to minimise a scalar function of one or more variables. Due to the fixed step-size, the convergence rate commonly decreases as the gradient shrinks when approaching a local minima.

Parameters:

See also

pints.GradientDescent

The PINTS implementation this class is based on.

class pybop.optimisers.pints_optimisers.IRPropMin(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for improved resilient backpropagation (without weight-backtracking), an optimisation algorithm designed to handle problems with large plateaus, noisy gradients, and local minima.

This method uses gradient information for the proposal direction with a separated step-size.

Parameters:

See also

pints.IRPropMin

The PINTS implementation this class is based on.

class pybop.optimisers.pints_optimisers.IRPropPlus(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for improved resilient backpropagation with weight-backtracking, an optimisation algorithm designed to handle problems with large plateaus, noisy gradients, and local minima.

Parameters:

See also

pints.IRPropMin

The PINTS implementation this class is based on.

class pybop.optimisers.pints_optimisers.NelderMead(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adpater for the Nelder-Mead downhill simplex method, a deterministic local optimiser that does not use gradient information or support boundary constraints.

In most update steps, it performs either one evaluation, or two sequential evaluations, so that it will not typically benefit from parallelisation.

Parameters:

See also

pints.NelderMead

PINTS implementation of Nelder-Mead algorithm.

class pybop.optimisers.pints_optimisers.PSO(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for particle swarm optimisation (PSO), a metaheuristic optimisation method inspired by the social behavior of birds flocking or fish schooling, suitable for global optimisation problems.

The method considers “particles” moving around the search space. Global optima convergence is guaranteed in the infinite limit for the number of optimiser iterations.

Parameters:

See also

pints.PSO

The PINTS implementation this class is based on.

class pybop.optimisers.pints_optimisers.RandomSearch(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for random search, a simple algorithm which samples parameter values randomly and stores the current best proposal based on fitness (not recommended for optimisation).

This optimiser has been implemented for benchmarking and comparisons, convergence will be better with one of other optimisers in the majority of cases.

Parameters:

See also

pybop.RandomSearchImpl

PyBOP implementation of Random Search algorithm.

class pybop.optimisers.pints_optimisers.SNES(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for the stochastic natural evolution strategy (SNES), an evolutionary algorithm that evolves a probability distribution on the parameter space, guiding the search for the optimum based on the natural gradient of expected fitness.

Parameters:

See also

pints.SNES

The PINTS implementation this class is based on.

property name: str[source]#

Overwrite misspelling of separable from Pints 0.5.1.

class pybop.optimisers.pints_optimisers.SimulatedAnnealing(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for simulated annealing, a probabilistic optimisation method inspired by the annealing process in metallurgy which is suitable for global optimisation problems.

It works by iteratively proposing new solutions and accepting them based on both their fitness and a temperature parameter that decreases over time. This allows the algorithm to initially explore broadly and gradually focus on local optimisation as the temperature decreases.

Parameters:

See also

pybop.SimulatedAnnealingImpl

PyBOP implementation of Simulated Annealing algorithm.

class pybop.optimisers.pints_optimisers.XNES(problem: pybop.Problem, options: pybop.PintsOptions | None = None)[source]#

Bases: pybop.optimisers.base_pints_optimiser.BasePintsOptimiser

Adapter for the exponential natural evolution strategy (XNES), an evolutionary algorithm that samples from a multivariate normal distribution, which is updated iteratively to fit the distribution of successful solutions.

Parameters:

See also

pints.XNES

PINTS implementation of XNES algorithm.