Documentation

File: __main__.py

boss.__main__.main(args=None)

The main routine.

File: bo/bo_main.py

class boss.bo.bo_main.BoMain(STS, mainOutput, rstManager)

Class for handling Bayesian Optimization

_acqnext(i_iter)

Selects the acquisition function to use and returns its xnext location as well as the used acquisition function.

_add_convergence()

Updates self.convergence with a new row containing bestx, besty, xhat, muhat, nuhat (in this order).

_add_xy(xnew, ynew)

Internal functionality for saving a new acquisition (x, y), accounting for model mean shift. Initializes the GP model when the number of acquisitions meets the pre-specified number of initialization points.

_evaluate(i, xnext)

Get a new evaluation either from the rst-file or, in case it doesn’t contain the corresponding evaluation, by evaluating the user function

_evaluate_xnew(xnew)

Evaluates user function at given location ‘xnew’ to get the observation scalar ‘ynew’. Later also gradient ‘ydnew’ should be made possible.

_get_xnext(i_iter)

Get a new point to evaluate by either reading it from the rst-file or, in case it doesn’t contain the next point to evaluate, by obtaining a new initial point (when run is in initialization stage) or minimizing the acquisition function (when the run is in BO stage).

_has_converged(i_iter)

Checks whether dxhat has been within tolerance for long enough TODO: should use dxmuhat instead?

_init_model()

Initializes the GP model. This should be called when all initialization points have been evaluated. Further acquisition locations can then be queried by optimizing an acquisition function.

_location_overconfident(xnext)

Checks is model variance is lower than tolerance at suggested xnext.

_minimize_acqfn(acqfn)

Minimizes the acquisition function to find the next sampling location ‘xnext’.

_optimize_model(i)

Optimize the GP model or, if the next hyperparameters are contained in the rst-file, just use them.

_should_optimize(i)

Returns True if the model should be optimized at iteration i.

add_xy_list(xnew, ynew)

Saves multiple acquisitions, accounting for model mean shift. Initializes the GP model when the number of acquisitions meets the pre-specified number of initialization points.

get_grad(x)

Returns the predictive gradients. If the implemented mean shift is a constant, should just wrap self.model.predictive_gradients.

get_mu(x)

Returns the model prediction at point x. This method should always be called instead of BoMain.model.mu for any use outside of the BoMain class.

get_nu(x)

Returns the variance of the model prediction at point x, with added noise (model variance).

get_x()

Returns the points where the objective function has been evaluated, in order of acquisition.

get_y()

Returns the evaluated values of the objective function, in order of acquistiion. This method should always be called instead of BoMain.model.Y for any use outside of the BoMain class.

run_optimization()

The Bayesian optimization main loop. Evaluates first the initialization points, then creates a GP model and uses it and an acquisition function to locate the next points where to evaluate. Stops when a pre-specified number of initialization points and BO points have been acquired or a convergence criterion is met.

File: bo/model.py

class boss.bo.model.Model(X, Y, bounds, min_dist_acqs, minzacc, kerntype, kernel, dim, noise)

Functionality for creating, refitting and optimizing a GP model

add_data(newX, newY)

Updates the model evidence (observations) dataset appending.

get_all_params()

Returns the parameters of the model in format: noise, variance, lengthscales, periods where the last two are 1D lists. There exists a period only for those dimensions which are using a periodic kernel.

get_bestxy()

Returns the lowest energy acquisition (x, y). Note that this does not account for the model mean shift introduced by the BoMain class.

get_unfixed_params()

Returns the unfixed parameters of the model in an array where the first element is the variance and the rest are lenghtscales. Note that this function assumes that only variances and lengthscales are free hyperparameters.

min_prediction()

Find and returns the model global minimum prediction xhat and the model mean (muhat) and variance (nuhat) at that point.

mu(x)

Returns model prediction mean at point x.

mu_with_grad(x)

Returns model mean and its gradient at point x.

nu(x)

Returns model prediction variance at point x, taking into account the model variance (noise).

optimize_controlrstr(restarts=1)

Updates the model hyperparameters by maximizing marginal likelihood.

predictive_m_s_grad(x)

Returns the model prediction mean, standard deviation and their gradients at point x, taking into account the model variance (noise).

redefine_data(X, Y)

Updates the model evidence (observations) dataset overwriting.

set_unfixed_params(params)

Sets the unfixed parameters of the model (variance, lengthscales) to given values.

File: bo/kernel_factory.py

class boss.bo.kernel_factory.KernelFactory

This class contains the construction of the kernel.

static _select_kernels(kerns, STS, forced_hypers)

Selects and creates kernel objects for each dimension. Hyperparameters are set to their initial values and default constraints removed.

static _set_constraints(kerns, STS)

Sets hyperparameter constraints on kernels.

static _set_priors(kerns, STS)

Sets hyperparameter priors on kernels.

static construct_kernel(STS, forced_hypers=None)

Creates the kernel.

Directory: bo/acq/

boss.bo.acq.explore(x, model, params)

Purely exploring acquisition function

Doesn’t take any parameters (apart from the model).

boss.bo.acq.exploit(x, model, params)

Purely exploiting acquisition function

Doesn’t take any parameters (apart from the model).

boss.bo.acq.elcb(x, model, params)

GP-Lower Confidence Bound acquisition function with increasing exploration

Doesn’t take any parameters (apart from the model). explr_weight = sqrt( 2*log[ ( i^((dim/2) + 2)*pi^(2) ) / ( 3*0.1 ) ] )

File: io/settings.py

class boss.io.settings.Settings(ipfile, timer)

Reads, interprets and defines the code internal settings based on input.

__init__(ipfile, timer)

Sets all settings by reading first the input file, filling in defaults and calculating implicit settings.

__weakref__

list of weak references to the object (if defined)

_convertstr(value, type_)

Converts the parameter value into the requested type type_ and returns it. Supported types are python’s str, bool, int, float as well as custom types ‘intlist’, ‘floatlist’, ‘strlist’, ‘boollist’ and ‘2Dfloatarray’, where the last is a numpy 2D array.

_read_ipfile()

Reads the input file and saves the found keywords to self.keywds dict. Additionally kicks off handling of an rst file

_read_iterations(ipf)

Reads the ensebles (x, y, theta) given in an rst-file.

_setopt(keyword, default, type_)

Returns the value of the setting/option variable by first looking for it in self.keywds dictionary and otherwise using default.

File: io/main_output.py

class boss.io.main_output.MainOutput(STS)

Functionality to write to the main output (*.out) file.

convergence_stop()

Announces BO stop due to global minimum convergence

footer(totaltime)

Writes a footer to main output file

header()

Writes a header to main output file overwriting a possibly existing old output file at the same filepath.

ipfile_repeat(is_rst)

Repeats the input file near the beggining of the main output file.

iteration_start(itNo, initpts)

Output section header for the new iteration.

iteration_summary(datasize, newXs, newYs, convergence, xnext, est_yrange, model_params, timer)

Outputs info about one BO iteration to main output file

mep_result(mep)

Writes the results of MEP

mep_start(mep)

Writes MEP options and local minima

progress_msg(msg, priority, preceding_bl=False, nospace=False)

Announce progress message to main output file depending on verbosity.

section_header(text)

Writes a section header to main output file.

settings(STS)

Outputs the interpreted code variable settings to main output file.

File: io/rst_output.py

class boss.utils.rstmanager.RstManager(STS)
A class that handles restart-files (rst-files). These files can be used to
  1. introduce acquisition data from another source as initial values,

  2. continue a run that has been interrupted for some reason,

  3. continue a finished run by acquiring more points or

  4. keep the acquisitions but change the model or settings for a rerun.

get_theta(i, n)

Returns the model paramters at iteration i from the rst-data or None if they can’t be found.

get_x(i)

Returns the i:th acquisition location from the rst-data or None if it can’t be found.

get_y(i)

Returns the i:th acquisition evaluation (energy and gradient) from the rst-data or None if it can’t be found.

ipfile_repeat()

Repeats the input file contents at the beginning of the NEW rst file. If the input file is a restart file itself, the parts starting from ‘RESULTS:’ are not repeated at the beginning of the new rst file. Will add the ‘RESULTS:’ line to the new rst file.

new_data(x, y)

Outputs a new data point (x,y) to rst file.

new_model_params(mod_param)

Outputs a new set of model parameters to rst file. Format is variance, lengthscales

File: io/data_output.py

class boss.io.data_output.DataOutput

Functionality to output raw data on request to files outside of the main output (.out) file and the restart (.rst) file.

static dump_acqfn(STS, dest_file, bo, acqfn, defs)

Outputs acquisition function slice (up to 2D) in a grid to acqfns/it#.dat

static dump_mep(path)

Outputs the coordinates of each minimum energy path into files

static dump_model(STS, dest_file, bo, mod_params, xhat)

Outputs model slice (up to 2D) mean and variance in a grid to models/it#.dat

static dump_truef(STS, dest_file, last_xhat)

Outputs true function slice (up to 2D) in a grid

File: io/ioutils.py

class boss.io.ioutils.IOutils

Reading/writing files and converting strings into Python variable types.

static append_write(filepath, text)

Writes the text into the given file appending.

static data_line(X, Q=None, fstr='%15.7E', separ=' ')

Returns a line of data. X and Q (optional) should be 1D arrays/lists.

static overwrite(filepath, text)

Writes the text into the given file overwriting.

static readCols(filepath, skiprows=1)

Reads data columns from a file into a numpy array, where each column is separated like col1 = arr[:,0], col2 = arr[:,1] etc.

static writeCols(filepath, vectorList, space=' ', titleLine=None, formatString='%15.7E')

Needs as input a vector of form [ [column1 elements], [column2 elements], … ]

File: io/parse.py

class boss.io.parse.Parse

Functions to parse files produced by a BOSS optimization.

acqs(filepath)

Extracts data acquisitions from all bo iterations in the output file. Returns a numpy array where each element is [npts, x, y]. Note that there may be several acquisitions on the same iteration - especially the initial points at the 0th iteration. Works on output files with all verbosity levels.

best_acqs(filepath)

Extracts xbest and ybest from all bo iterations in the output file. Returns a numpy array where each element is [npts, xbest, ybest]. Works only on output files with verbosity levels of at least 1.

conv_measures(filepath)

Extracts dx_hat and dmu_hat from all bo iterations in the output file. Returns a numpy array where each element is [npts, dxhat, dmuhat]. Works only on output files with verbosity levels of at least 1.

min_preds(filepath)

Extracts xhat, muhat and nuhat from all bo iterations in the output file. Returns a numpy array where each element is [npts, xhat, muhat, nuhat]. Works on output files with all verbosity levels.

minima()

Extract local minima

mod_params(filepath)

Extracts unfixed GP model hyperparameters from all bo iterations in the output file. Returns a numpy array where each element is [npts, variances, lengthscales]. Works only on output files with verbosity levels of at least 2.

rst(filepath)

Extracts all information below the keyword ‘RESULTS:’ from an rst file.

xnexts(filepath)

Extracts xnext locations from all bo iterations in the output file. Returns a numpy array where each element is [npts, xnext]. Note that this means that the location ‘xnext’ is to be evaluated in the next iteration ‘iter+1’. Works only on output files with verbosity levels of at least 2.

File: pp/pp_main.py

boss.pp.pp_main.PPmain(STS, ipt_rstfile, ipt_outfile, mainOutput)

Controls all the post-processing features

File: pp/plot.py

class boss.pp.plot.Plot

Plotting functionality. All plot functions make PNG images to destination file based on given source array(s) of data.

acq_func(dest_file, acqf_data, acqs=None, xhat=None, xnext=None, axis_labels=None, legends=True)

Plots a (max 2D) slice of the acquisition function. Can include also acquisitions, xhat and xnext if those are given.

conv_measures(dest_file, conv_meas, legends=True)

Plots quantities related to convergence as a function of iteration.

data_acquisitions(dest_file, acqs, min_preds)

Plots minimum predictions and uncertainty, acquisition energies and locations as a function of iteration.

hyperparameters(dest_file, hypers, legends=True)

Plots GP model unfixed hyperparameters (variances and lengthscales) as a function of iteration.

model(dest_file, model_data, xhat=None, acqs=None, xnext=None, minima=None, truef=None, incl_uncert=True, axis_labels=None, legends=True, paths=None)

Plots a (max 2D) slice of the model.

truef(dest_file, truef_data, axis_labels=None)

Plots a (max 2D) slice of the true function.

truef_hat(dest_file, truef_hats, legends=True)

Plots true function value at xhat locations as a function of iteration.

File: utils/distributions.py

class boss.utils.distributions.Distributions

Utilities for handling distributions.

static gammaparams(q1, q2, p1=0.1, p2=0.9)

A function for parametrizing gamma distributions by specifying two quantiles.

Relevant math can be found in “Determining distribution parameters from quantiles” by John D. Cook, 2010.

Parameters
  • q1 – p1-quantile

  • q2 – p2-quantile

  • p1 – lower percentage (0 < p1 < 1)

  • p2 – higher percentage (0 < p2 < 1)

File: utils/minimization.py

class boss.utils.minimization.Minimization

Minimization utilities

static _remove_duplicates(original, min_distance)

Removes duplicates from a list of found local minima given a minimum distance between distinct minima.

static _run_minimizer(func, x0, bounds, args, finite_grads=False)

Runs a single L-BFGS-B minimization starting from x0

static minimize(func, bounds, kerntype, acqs, min_dist_acqs, accuracy=0.3, args=(), lowest_min_only=True)

Tries to find global minimum of func by starting minimizers from accuracy*100 percentage of lowest acquisitions. func has to return both value and gradient given an x of same length as bounds.

static minimize_from_random(func, bounds, num_pts, shift, args=(), lowest_min_only=True)

Tries to find global minimum of func by starting minimizers from num_pts random points (min 10 max 100). func has to return both value and gradient given an x of same length as bounds.

static minimize_from_sobol(func, bounds, num_pts, shift, args=(), lowest_min_only=True)

Tries to find global minimum of func by starting minimizers from num_pts shifted sobol points. func has to return both value and gradient given an x of same length as bounds.

File: utils/timer.py

class boss.utils.timer.Timer

Utility class for wall clock time keeping.

getLapTime()

Returns lap time in seconds.

getTotalTime()

Returns total time since timer creation in seconds.

startLap()

Starts counting lap time from zero.

str_lapTime(units='s')

Returns lap time in the given unit of time as a string.

str_totalTime(units='s')

Returns total time since timer creation in the given unit of time as a string.

File: utils/initial_points.py

class boss.utils.initmanager.InitManager(inittype, dim, bounds, initpts)
_grid(bounds, npts)

Initial points in a grid. Total number of points returned is npts^dim.

_random(dim, bounds, npts)

Initial points randomly

_sobol(dim, bounds, npts)

Initial points with the quasi-random Sobol sequence

get_all()

Returns all generated initial points

get_x(i)

Returns the i:th initial point