Documentation¶
File: bo/bo_main.py¶
-
class
boss.bo.bo_main.
BoMain
(STS, mainOutput, rstManager)¶ Class for handling Bayesian Optimization
-
_acqnext
(i_iter)¶ Selects the acquisition function to use and returns its xnext location as well as the used acquisition function.
-
_add_convergence
()¶ Updates self.convergence with a new row containing bestx, besty, xhat, muhat, nuhat (in this order).
-
_add_xy
(xnew, ynew)¶ Internal functionality for saving a new acquisition (x, y), accounting for model mean shift. Initializes the GP model when the number of acquisitions meets the pre-specified number of initialization points.
-
_evaluate
(i, xnext)¶ Get a new evaluation either from the rst-file or, in case it doesn’t contain the corresponding evaluation, by evaluating the user function
-
_evaluate_xnew
(xnew)¶ Evaluates user function at given location ‘xnew’ to get the observation scalar ‘ynew’. Later also gradient ‘ydnew’ should be made possible.
-
_get_xnext
(i_iter)¶ Get a new point to evaluate by either reading it from the rst-file or, in case it doesn’t contain the next point to evaluate, by obtaining a new initial point (when run is in initialization stage) or minimizing the acquisition function (when the run is in BO stage).
-
_has_converged
(i_iter)¶ Checks whether dxhat has been within tolerance for long enough TODO: should use dxmuhat instead?
-
_init_model
()¶ Initializes the GP model. This should be called when all initialization points have been evaluated. Further acquisition locations can then be queried by optimizing an acquisition function.
-
_location_overconfident
(xnext)¶ Checks is model variance is lower than tolerance at suggested xnext.
-
_minimize_acqfn
(acqfn)¶ Minimizes the acquisition function to find the next sampling location ‘xnext’.
-
_optimize_model
(i)¶ Optimize the GP model or, if the next hyperparameters are contained in the rst-file, just use them.
-
_should_optimize
(i)¶ Returns True if the model should be optimized at iteration i.
-
add_xy_list
(xnew, ynew)¶ Saves multiple acquisitions, accounting for model mean shift. Initializes the GP model when the number of acquisitions meets the pre-specified number of initialization points.
-
get_grad
(x)¶ Returns the predictive gradients. If the implemented mean shift is a constant, should just wrap self.model.predictive_gradients.
-
get_mu
(x)¶ Returns the model prediction at point x. This method should always be called instead of BoMain.model.mu for any use outside of the BoMain class.
-
get_nu
(x)¶ Returns the variance of the model prediction at point x, with added noise (model variance).
-
get_x
()¶ Returns the points where the objective function has been evaluated, in order of acquisition.
-
get_y
()¶ Returns the evaluated values of the objective function, in order of acquistiion. This method should always be called instead of BoMain.model.Y for any use outside of the BoMain class.
-
run_optimization
()¶ The Bayesian optimization main loop. Evaluates first the initialization points, then creates a GP model and uses it and an acquisition function to locate the next points where to evaluate. Stops when a pre-specified number of initialization points and BO points have been acquired or a convergence criterion is met.
-
File: bo/model.py¶
-
class
boss.bo.model.
Model
(X, Y, bounds, min_dist_acqs, minzacc, kerntype, kernel, dim, noise)¶ Functionality for creating, refitting and optimizing a GP model
-
add_data
(newX, newY)¶ Updates the model evidence (observations) dataset appending.
-
get_all_params
()¶ Returns the parameters of the model in format: noise, variance, lengthscales, periods where the last two are 1D lists. There exists a period only for those dimensions which are using a periodic kernel.
-
get_bestxy
()¶ Returns the lowest energy acquisition (x, y). Note that this does not account for the model mean shift introduced by the BoMain class.
-
get_unfixed_params
()¶ Returns the unfixed parameters of the model in an array where the first element is the variance and the rest are lenghtscales. Note that this function assumes that only variances and lengthscales are free hyperparameters.
-
min_prediction
()¶ Find and returns the model global minimum prediction xhat and the model mean (muhat) and variance (nuhat) at that point.
-
mu
(x)¶ Returns model prediction mean at point x.
-
mu_with_grad
(x)¶ Returns model mean and its gradient at point x.
-
nu
(x)¶ Returns model prediction variance at point x, taking into account the model variance (noise).
-
optimize_controlrstr
(restarts=1)¶ Updates the model hyperparameters by maximizing marginal likelihood.
-
predictive_m_s_grad
(x)¶ Returns the model prediction mean, standard deviation and their gradients at point x, taking into account the model variance (noise).
-
redefine_data
(X, Y)¶ Updates the model evidence (observations) dataset overwriting.
-
set_unfixed_params
(params)¶ Sets the unfixed parameters of the model (variance, lengthscales) to given values.
-
File: bo/kernel_factory.py¶
-
class
boss.bo.kernel_factory.
KernelFactory
¶ This class contains the construction of the kernel.
-
static
_select_kernels
(kerns, STS, forced_hypers)¶ Selects and creates kernel objects for each dimension. Hyperparameters are set to their initial values and default constraints removed.
-
static
_set_constraints
(kerns, STS)¶ Sets hyperparameter constraints on kernels.
-
static
_set_priors
(kerns, STS)¶ Sets hyperparameter priors on kernels.
-
static
construct_kernel
(STS, forced_hypers=None)¶ Creates the kernel.
-
static
Directory: bo/acq/¶
-
boss.bo.acq.
explore
(x, model, params)¶ Purely exploring acquisition function
Doesn’t take any parameters (apart from the model).
-
boss.bo.acq.
exploit
(x, model, params)¶ Purely exploiting acquisition function
Doesn’t take any parameters (apart from the model).
-
boss.bo.acq.
elcb
(x, model, params)¶ GP-Lower Confidence Bound acquisition function with increasing exploration
Doesn’t take any parameters (apart from the model). explr_weight = sqrt( 2*log[ ( i^((dim/2) + 2)*pi^(2) ) / ( 3*0.1 ) ] )
File: io/settings.py¶
-
class
boss.io.settings.
Settings
(ipfile, timer)¶ Reads, interprets and defines the code internal settings based on input.
-
__init__
(ipfile, timer)¶ Sets all settings by reading first the input file, filling in defaults and calculating implicit settings.
-
__weakref__
¶ list of weak references to the object (if defined)
-
_convertstr
(value, type_)¶ Converts the parameter value into the requested type type_ and returns it. Supported types are python’s str, bool, int, float as well as custom types ‘intlist’, ‘floatlist’, ‘strlist’, ‘boollist’ and ‘2Dfloatarray’, where the last is a numpy 2D array.
-
_read_ipfile
()¶ Reads the input file and saves the found keywords to self.keywds dict. Additionally kicks off handling of an rst file
-
_read_iterations
(ipf)¶ Reads the ensebles (x, y, theta) given in an rst-file.
-
_setopt
(keyword, default, type_)¶ Returns the value of the setting/option variable by first looking for it in self.keywds dictionary and otherwise using default.
-
File: io/main_output.py¶
-
class
boss.io.main_output.
MainOutput
(STS)¶ Functionality to write to the main output (*.out) file.
-
convergence_stop
()¶ Announces BO stop due to global minimum convergence
Writes a footer to main output file
-
header
()¶ Writes a header to main output file overwriting a possibly existing old output file at the same filepath.
-
ipfile_repeat
(is_rst)¶ Repeats the input file near the beggining of the main output file.
-
iteration_start
(itNo, initpts)¶ Output section header for the new iteration.
-
iteration_summary
(datasize, newXs, newYs, convergence, xnext, est_yrange, model_params, timer)¶ Outputs info about one BO iteration to main output file
-
mep_result
(mep)¶ Writes the results of MEP
-
mep_start
(mep)¶ Writes MEP options and local minima
-
progress_msg
(msg, priority, preceding_bl=False, nospace=False)¶ Announce progress message to main output file depending on verbosity.
-
section_header
(text)¶ Writes a section header to main output file.
-
settings
(STS)¶ Outputs the interpreted code variable settings to main output file.
-
File: io/rst_output.py¶
-
class
boss.utils.rstmanager.
RstManager
(STS)¶ - A class that handles restart-files (rst-files). These files can be used to
introduce acquisition data from another source as initial values,
continue a run that has been interrupted for some reason,
continue a finished run by acquiring more points or
keep the acquisitions but change the model or settings for a rerun.
-
get_theta
(i, n)¶ Returns the model paramters at iteration i from the rst-data or None if they can’t be found.
-
get_x
(i)¶ Returns the i:th acquisition location from the rst-data or None if it can’t be found.
-
get_y
(i)¶ Returns the i:th acquisition evaluation (energy and gradient) from the rst-data or None if it can’t be found.
-
ipfile_repeat
()¶ Repeats the input file contents at the beginning of the NEW rst file. If the input file is a restart file itself, the parts starting from ‘RESULTS:’ are not repeated at the beginning of the new rst file. Will add the ‘RESULTS:’ line to the new rst file.
-
new_data
(x, y)¶ Outputs a new data point (x,y) to rst file.
-
new_model_params
(mod_param)¶ Outputs a new set of model parameters to rst file. Format is variance, lengthscales
File: io/data_output.py¶
-
class
boss.io.data_output.
DataOutput
¶ Functionality to output raw data on request to files outside of the main output (.out) file and the restart (.rst) file.
-
static
dump_acqfn
(STS, dest_file, bo, acqfn, defs)¶ Outputs acquisition function slice (up to 2D) in a grid to acqfns/it#.dat
-
static
dump_mep
(path)¶ Outputs the coordinates of each minimum energy path into files
-
static
dump_model
(STS, dest_file, bo, mod_params, xhat)¶ Outputs model slice (up to 2D) mean and variance in a grid to models/it#.dat
-
static
dump_truef
(STS, dest_file, last_xhat)¶ Outputs true function slice (up to 2D) in a grid
-
static
File: io/ioutils.py¶
-
class
boss.io.ioutils.
IOutils
¶ Reading/writing files and converting strings into Python variable types.
-
static
append_write
(filepath, text)¶ Writes the text into the given file appending.
-
static
data_line
(X, Q=None, fstr='%15.7E', separ=' ')¶ Returns a line of data. X and Q (optional) should be 1D arrays/lists.
-
static
overwrite
(filepath, text)¶ Writes the text into the given file overwriting.
-
static
readCols
(filepath, skiprows=1)¶ Reads data columns from a file into a numpy array, where each column is separated like col1 = arr[:,0], col2 = arr[:,1] etc.
-
static
writeCols
(filepath, vectorList, space=' ', titleLine=None, formatString='%15.7E')¶ Needs as input a vector of form [ [column1 elements], [column2 elements], … ]
-
static
File: io/parse.py¶
-
class
boss.io.parse.
Parse
¶ Functions to parse files produced by a BOSS optimization.
-
acqs
(filepath)¶ Extracts data acquisitions from all bo iterations in the output file. Returns a numpy array where each element is [npts, x, y]. Note that there may be several acquisitions on the same iteration - especially the initial points at the 0th iteration. Works on output files with all verbosity levels.
-
best_acqs
(filepath)¶ Extracts xbest and ybest from all bo iterations in the output file. Returns a numpy array where each element is [npts, xbest, ybest]. Works only on output files with verbosity levels of at least 1.
-
conv_measures
(filepath)¶ Extracts dx_hat and dmu_hat from all bo iterations in the output file. Returns a numpy array where each element is [npts, dxhat, dmuhat]. Works only on output files with verbosity levels of at least 1.
-
min_preds
(filepath)¶ Extracts xhat, muhat and nuhat from all bo iterations in the output file. Returns a numpy array where each element is [npts, xhat, muhat, nuhat]. Works on output files with all verbosity levels.
-
minima
()¶ Extract local minima
-
mod_params
(filepath)¶ Extracts unfixed GP model hyperparameters from all bo iterations in the output file. Returns a numpy array where each element is [npts, variances, lengthscales]. Works only on output files with verbosity levels of at least 2.
-
rst
(filepath)¶ Extracts all information below the keyword ‘RESULTS:’ from an rst file.
-
xnexts
(filepath)¶ Extracts xnext locations from all bo iterations in the output file. Returns a numpy array where each element is [npts, xnext]. Note that this means that the location ‘xnext’ is to be evaluated in the next iteration ‘iter+1’. Works only on output files with verbosity levels of at least 2.
-
File: pp/pp_main.py¶
-
boss.pp.pp_main.
PPmain
(STS, ipt_rstfile, ipt_outfile, mainOutput)¶ Controls all the post-processing features
File: pp/plot.py¶
-
class
boss.pp.plot.
Plot
¶ Plotting functionality. All plot functions make PNG images to destination file based on given source array(s) of data.
-
acq_func
(dest_file, acqf_data, acqs=None, xhat=None, xnext=None, axis_labels=None, legends=True)¶ Plots a (max 2D) slice of the acquisition function. Can include also acquisitions, xhat and xnext if those are given.
-
conv_measures
(dest_file, conv_meas, legends=True)¶ Plots quantities related to convergence as a function of iteration.
-
data_acquisitions
(dest_file, acqs, min_preds)¶ Plots minimum predictions and uncertainty, acquisition energies and locations as a function of iteration.
-
hyperparameters
(dest_file, hypers, legends=True)¶ Plots GP model unfixed hyperparameters (variances and lengthscales) as a function of iteration.
-
model
(dest_file, model_data, xhat=None, acqs=None, xnext=None, minima=None, truef=None, incl_uncert=True, axis_labels=None, legends=True, paths=None)¶ Plots a (max 2D) slice of the model.
-
truef
(dest_file, truef_data, axis_labels=None)¶ Plots a (max 2D) slice of the true function.
-
truef_hat
(dest_file, truef_hats, legends=True)¶ Plots true function value at xhat locations as a function of iteration.
-
File: utils/distributions.py¶
-
class
boss.utils.distributions.
Distributions
¶ Utilities for handling distributions.
-
static
gammaparams
(q1, q2, p1=0.1, p2=0.9)¶ A function for parametrizing gamma distributions by specifying two quantiles.
Relevant math can be found in “Determining distribution parameters from quantiles” by John D. Cook, 2010.
- Parameters
q1 – p1-quantile
q2 – p2-quantile
p1 – lower percentage (0 < p1 < 1)
p2 – higher percentage (0 < p2 < 1)
-
static
File: utils/minimization.py¶
-
class
boss.utils.minimization.
Minimization
¶ Minimization utilities
-
static
_remove_duplicates
(original, min_distance)¶ Removes duplicates from a list of found local minima given a minimum distance between distinct minima.
-
static
_run_minimizer
(func, x0, bounds, args, finite_grads=False)¶ Runs a single L-BFGS-B minimization starting from x0
-
static
minimize
(func, bounds, kerntype, acqs, min_dist_acqs, accuracy=0.3, args=(), lowest_min_only=True)¶ Tries to find global minimum of func by starting minimizers from accuracy*100 percentage of lowest acquisitions. func has to return both value and gradient given an x of same length as bounds.
-
static
minimize_from_random
(func, bounds, num_pts, shift, args=(), lowest_min_only=True)¶ Tries to find global minimum of func by starting minimizers from num_pts random points (min 10 max 100). func has to return both value and gradient given an x of same length as bounds.
-
static
minimize_from_sobol
(func, bounds, num_pts, shift, args=(), lowest_min_only=True)¶ Tries to find global minimum of func by starting minimizers from num_pts shifted sobol points. func has to return both value and gradient given an x of same length as bounds.
-
static
File: utils/timer.py¶
-
class
boss.utils.timer.
Timer
¶ Utility class for wall clock time keeping.
-
getLapTime
()¶ Returns lap time in seconds.
-
getTotalTime
()¶ Returns total time since timer creation in seconds.
-
startLap
()¶ Starts counting lap time from zero.
-
str_lapTime
(units='s')¶ Returns lap time in the given unit of time as a string.
-
str_totalTime
(units='s')¶ Returns total time since timer creation in the given unit of time as a string.
-
File: utils/initial_points.py¶
-
class
boss.utils.initmanager.
InitManager
(inittype, dim, bounds, initpts)¶ -
_grid
(bounds, npts)¶ Initial points in a grid. Total number of points returned is npts^dim.
-
_random
(dim, bounds, npts)¶ Initial points randomly
-
_sobol
(dim, bounds, npts)¶ Initial points with the quasi-random Sobol sequence
-
get_all
()¶ Returns all generated initial points
-
get_x
(i)¶ Returns the i:th initial point
-