Optimisers#

PyBOP has a range of optimisers and samplers available for solving optimisation problems. Different optimisers have different strengths and weaknesses. See the tables below for summaries, and follow the links for full documentation on each.

Pints Optimisers#

GradientDescent(problem[, options])

Adapter for gradient descent, a canonical method that takes steps in the opposite direction of the cost gradient with respect to the parameters (does not support boundary constraints).

AdamW(problem[, options])

Adapter for adaptive moment estimation with weight decay (AdamW), a variant of the Adam optimiser which does not support boundary constraints.

IRPropMin(problem[, options])

Adapter for improved resilient backpropagation (without weight-backtracking), an optimisation algorithm designed to handle problems with large plateaus, noisy gradients, and local minima.

IRPropPlus(problem[, options])

Adapter for improved resilient backpropagation with weight-backtracking, an optimisation algorithm designed to handle problems with large plateaus, noisy gradients, and local minima.

PSO(problem[, options])

Adapter for particle swarm optimisation (PSO), a metaheuristic optimisation method inspired by the social behavior of birds flocking or fish schooling, suitable for global optimisation problems.

SNES(problem[, options])

Adapter for the stochastic natural evolution strategy (SNES), an evolutionary algorithm that evolves a probability distribution on the parameter space, guiding the search for the optimum based on the natural gradient of expected fitness.

XNES(problem[, options])

Adapter for the exponential natural evolution strategy (XNES), an evolutionary algorithm that samples from a multivariate normal distribution, which is updated iteratively to fit the distribution of successful solutions.

NelderMead(problem[, options])

Adpater for the Nelder-Mead downhill simplex method, a deterministic local optimiser that does not use gradient information or support boundary constraints.

CMAES(problem[, options])

Adapter for the covariance matrix adaptation evolution strategy (CMA-ES), an evolutionary algorithm for difficult non-linear non-convex optimisation problems.

CuckooSearch(problem[, options])

Adapter for cuckoo search, a population-based optimisation algorithm inspired by the brood parasitism of some cuckoo species which is suitable for global optimisation problems.

RandomSearch(problem[, options])

Adapter for random search, a simple algorithm which samples parameter values randomly and stores the current best proposal based on fitness (not recommended for optimisation).

SimulatedAnnealing(problem[, options])

Adapter for simulated annealing, a probabilistic optimisation method inspired by the annealing process in metallurgy which is suitable for global optimisation problems.

SciPy Optimisers#

SciPyMinimize(problem[, options])

Adapter for various scalar minimisation algorithms implemented in SciPy, allowing fine-tuning of the optimisation process through method selection and option configuration.

SciPyDifferentialEvolution(problem[, options])

Adapter for SciPy's differential_evolution function for global optimisation, useful for problems involving continuous parameters and potentially multiple local minima.

Pints Samplers#

NUTS(log_pdf[, options])

Implements the No-U-Turn Sampler (NUTS) algorithm.

DREAM(log_pdf[, options])

Implements the DiffeRential Evolution Adaptive Metropolis (DREAM) algorithm.

DifferentialEvolutionMCMC(log_pdf[, options])

Implements the Differential Evolution Markov Chain Monte Carlo (MCMC) algorithm.

DramACMC(log_pdf[, options])

Implements the Delayed Rejection Adaptive Metropolis (DRAM) Adaptive Covariance MCMC algorithm.

EmceeHammerMCMC(log_pdf[, options])

Implements the Emcee Hammer Markov Chain Monte Carlo (MCMC) algorithm.

HaarioACMC(log_pdf[, options])

Implements the Haario Adaptive Covariance Markov Chain Monte Carlo (MCMC) algorithm.

HaarioBardenetACMC(log_pdf[, options])

Implements the Haario-Bardenet Adaptive Covariance Markov Chain Monte Carlo (MCMC) algorithm.

HamiltonianMCMC(log_pdf[, options])

Implements the Hamiltonian Markov Chain Monte Carlo (MCMC) algorithm.

MALAMCMC(log_pdf[, options])

Implements the Metropolis Adjusted Langevin Algorithm (MALA) Markov Chain Monte Carlo (MCMC) algorithm.

MetropolisRandomWalkMCMC(log_pdf[, options])

Implements the Metropolis Random Walk Markov Chain Monte Carlo (MCMC) algorithm.

MonomialGammaHamiltonianMCMC(log_pdf[, options])

Implements the Monomial Gamma Hamiltonian Markov Chain Monte Carlo (MCMC) algorithm.

PopulationMCMC(log_pdf[, options])

Implements the Population Markov Chain Monte Carlo (MCMC) algorithm.

RaoBlackwellACMC(log_pdf[, options])

Implements the Rao-Blackwell Adaptive Covariance Markov Chain Monte Carlo (MCMC) algorithm.

RelativisticMCMC(log_pdf[, options])

Implements the Relativistic Markov Chain Monte Carlo (MCMC) algorithm.

SliceDoublingMCMC(log_pdf[, options])

Implements the Slice Doubling Markov Chain Monte Carlo (MCMC) algorithm.

SliceRankShrinkingMCMC(log_pdf[, options])

Implements the Slice Rank Shrinking Markov Chain Monte Carlo (MCMC) algorithm.

SliceStepoutMCMC(log_pdf[, options])

Implements the Slice Stepout Markov Chain Monte Carlo (MCMC) algorithm.