Optimizer Methods¶

Parameters of phenology models have a complex search space and are commonly fit with global optimization algorithms. To estimate parameters pyPhenology uses optimizers built-in to scipy. Optimizers available are:

Differential evolution (the default)
Basin hopping
Brute force

Changing the optimizer¶

The optimizer can be specified in the fit method by using the codes DE, BH, or BF for differential evolution, basin hopping, or brute force, respectively:

from pyPhenology import models, utils
observations, temp = utils.load_test_data(name='vaccinium')

model = models.ThermalTime(parameters={})
model.fit(observations, temp, method='DE')

Optimizer Arguments¶

Arguments to the optimization algorithm are crucial to model fitting. These control things like the maximimum number of iterations and how to specify convergence. Ideally one should choose arguments which find the “true” solution, yet this is a tradeoff of time and resources available. Models with a large number of parameters (such as the Unichill model) can take several hours to days to fit if the optimizer arguments are set liberally.

Optimizer arguments can be set two ways. The first is using some preset defaults:

model.fit(observations, temp, method='DE', optimizer_params='practical')

testing Designed to be quick for testing code. Results from this should not be used for analysis.
practical Default. Should produce realistic results on desktop systems in a relatively short period.
intensive Designed to find the absolute optimal solution. Can potentially take hours to days on large datasets.

The 2nd is using a dictionary for customized optimizer arguments:

model.fit(observations, temp, method='DE',optimizer_params={'popsize': 50,
                                                            'maxiter': 5000})

All the arguments in the scipy optimizer functions are available via the optimizer_params argument in fit. The important ones are described below, but also look at the available options in the scipy documentation. Any arguments not set will be set to the default specifed in the scipy package. Note that the three presets can be used with all optimizer methods, but using custimized methods will only work for that specific method. For example the popsize argument above will only work with method DE.

Optimizers¶

Differential Evolution¶

Differential evolution uses a population of models each randomly initialized to different parameter values within the respective search spaces. Each “member” is adjusted slightly based on the performance of the best model. This process is repeated until the maxiter value is reached or a convergence threshold is met.

Differential evolution Scipy documentation

Differential Evolution Key arguments¶

See the official documentation for more in depth details.

maxiter : int

the maximum number of itations. higher means potentially longer fitting times.
popsize : int

total population size of randomly initialized models. higher means longer fitting times.
mutation : float, or tuple

mutation constant. must be 0 < x < 2. Can be a constant (float) or a range (tuple). Higher mean longer fitting times.
recombination : float

probability of a member progressing to the next generation. must be 0 < x < 1. Lower means longer fitting times.
disp : boolean

Display output as the model is fit.

Differential Evolution Presets¶

testing:

{'maxiter':5,
 'popsize':10,
 'mutation':(0.5,1),
 'recombination':0.25,
 'disp':False}

practical:

{'maxiter':1000,
 'popsize':50,
 'mutation':(0.5,1),
 'recombination':0.25,
 'disp':False}

intensive:

{'maxiter':10000,
 'popsize':100,
 'mutation':(0.1,1),
 'recombination':0.25,
 'disp':False}

Basin Hopping¶

Basin hopping first makes a random guess at the parameters, then finds the local minima using L-BFGS-B. From the local minima the parameters are then randomly perturbed and minimized again, with this new estimate accepted using a Metropolis criterion. This is repeated until niter is reached. The parameters with the best score are selected. Basin hopping is essentially the same as simulated annealing, but with the added step of finding the local minima between random perturbations. Simulated annealing was retired from the Scipy packge in favor of basin hopping, see the note here.

Basin Hopping Scipy documentation

Basin Hopping Key arguments¶

See the official documentation for more in depth details.

niter : int

the number of itations. higher means potentially longer fitting times.
T : float

The “temperature” parameter for the accept or reject criterion.
stepsize : float

The size of the random perturbations
disp : boolean

Display output as the model is fit.

Basin Hopping Presets¶

testing:

{'niter': 100,
 'T': 0.5,
 'stepsize': 0.5,
 'disp': False}

practical:

{'niter': 50000,
 'T': 0.5,
 'stepsize': 0.5,
 'disp': False}

intensive:

{'niter': 500000,
 'T': 0.5,
 'stepsize': 0.5,
 'disp': False}

Brute Force¶

Brute force is a comprehensive search within predefined parameter ranges.

Brute force Scipy documentation

Brute Force Key Arguments¶

See the official documentation for more details

Ns : int

Number of grid points within search space to search over. See below.
finish : function

Function to find the local best solution from the best search space solution. This is set to optimize.fmin_bfgs in the presets, which is the scipy bfgs minimizer. See more options here.
disp : boolean

Display output as the model is fit.

Brute Force Presets¶

testing:

{'Ns':2,
 'finish':optimize.fmin_bfgs,
 'disp':False}

practical:

{'Ns':20,
 'finish':optimize.fmin_bfgs,
 'disp':False}

intensive:

{'Ns':40,
 'finish':optimize.fmin_bfgs,
 'disp':False}

Brute Force Notes¶

The Ns argument defines the number of points to test with each search parameter. For example consider the following search spaces for a three parameter model:

{'t1': (-10,10), 'T':(0,10), 'F': (0,1000),}

Using Ns=20 will search all combinations of:

t1=[-10, -9, -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
T=[0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5, 9.0, 9.5]
F=[0, 50, 100, 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700, 750, 800, 850, 900, 950]

which results in \(20^3\) model evalutaions. In this way model fitting time increases exponentially with the number of parameters in a model.

Alternatively you can set the search range using slices of (low, high, step) instead of (low,high). This allows for more control over the search space for each paramter. For example:

{'t1': slice(-10, 10, 1),'T': slice(0,10, 1),'F': slice(0,1000, 5)}

Note that using slices this way only works for the brute force method. This can create more realistic search space for each parameter. But in this example the number of evalutaions is still high, \(20*10*200=40000\). It’s recommended that Brute Force is only used for models with a low number of parameters, otherwise Differential Evolution is quicker and more robust.