Gaussian Process

The Gaussian process regression represents the first step of our Gaussian process implementation and is done using the sklearn.gaussian_process.GaussianProcessRegressor class from the scikit-learn package.

../_images/gp_flowchart.png

Main Optimization Method

hyppo.surrogate.gp_opt. gp ( config , log_dir , step , rank , xlow , xup , mult , names , loops = 1000 , istop = None , nhof = 10 , ngen = 100 , mu = 100 , cxpb = 0.75 , indpb = 0.1 , default = {} , ** kwargs ) [source]

Gaussian Process based surrogate optimization algorithm.

Parameters :
config dict

Configuration dictionary object to be used when evaluating subsequent sample sets.

log_dir str

Relative path where log files are stored

loops int

Number of iterations to do surrogate modeling over.

step int

Current SLURM step

rank int

Current processor rank

xlow list [ int ]

Lower boundaries of hyperparameters to evaluate.

xup list [ int ]

Upper boundaries of hyperparameters to evaluate.

mult list [ int ]

Scaling factor for each hyperparameter

names list [ str ]

Name of each hyperparameter

nbof int

Number of best individuals to keep in the hall of fame

ngen int

Number of generations to run during genetic process

mu int

Number of individuals in the genetic population

cxpb float

Cross-over probability

indpb float

Independent probability for each individual to be exchanged

default dict

Dictionary of default hyperparameter values

Expected Improvement

hyppo.surrogate.gp_opt. Expected_improvement ( x , samples , fvals , gpr ) [source]

Calculate expected improvement.

Parameters :
x deap.creator.Individual

Trial set of hyperparameters

opti hyppo.surrogate.utility.Optimization

Object containing all information

References

This blog about Bayesian Optimization

Generational Process

Selection of Individuals

Crossover Operator

Crossover is an operator used to vary chromosomes from one generator to the next

Mutation Operator

The mutation operator is used to maintain genetic diversity from one generation of a population to the next.

Euclidean Distance