`corrfitter` - Least-Squares Fit to Correlators¶

Correlator Model Objects¶

Correlator objects describe theoretical models that are fit to correlator data by varying the models’ parameters.

A model object’s parameters are specified through priors for the fit. A model assigns labels to each of its parameters (or arrays of related parameters), and these labels are used to identify the corresponding parameters in the prior. Parameters can be shared by more than one model object.

A model object also specifies the data that it is to model. The data is identified by the data tag that labels it in the input file or gvar.dataset.Dataset.

class corrfitter.Corr2(datatag, a, b, dE, s=1.0, tp=None, tmin=None, tmax=None, tdata=None, tfit=None, reverse=False, reverseddata=[], otherdata=[])¶

Two-point correlators Gab(t) = <b(t) a(0)>.

corrfitter.Corr2 models the t dependence of a 2-point correlator Gab(t) using

Gab(t) = sn * sum_i an[i] * bn[i] * fn(En[i], t)
       + so * sum_i ao[i] * bo[i] * fo(Eo[i], t)

where sn and so are typically -1, 0, or 1 and

fn(E, t) =  exp(-E*t) + exp(-E*(tp-t)) # tp>0 -- periodic
       or   exp(-E*t) - exp(-E*(-tp-t))# tp<0 -- anti-periodic
       or   exp(-E*t)                  # if tp is None (nonperiodic)

fo(E, t) = (-1)**t * fn(E, t)

The fit parameters for the non-oscillating piece of Gab (first term) are an[i], bn[i], and dEn[i] where:

dEn[0] = En[0]
dEn[i] = En[i]-En[i-1] > 0     (for i>0)

and therefore En[i] = sum_j=0..i dEn[j]. The fit parameters for the oscillating piece are defined analogously: ao[i], bo[i], and dEo[i].

The fit parameters are specified by the keys corresponding to these parameters in a dictionary of priors supplied to corrfitter.CorrFitter. The keys are strings and are also used to access fit results. A log-normal prior can be specified for a parameter by including an entry for log(c) in the prior, rather than for c itself. See the lsqfit documentation for information about other distributions that are available. Values for both log(c) and c are included in the parameter dictionary. Log-normal distributions are useful for forcing an, bn and/or dE to be positive.

When tp is not None and positive, the correlator is assumed to be symmetrical about tp/2, with Gab(t)=Gab(tp-t). Data from t>tp/2 is averaged with the corresponding data from t<tp/2 before fitting. When tp is negative, the correlator is assumed to be anti-symetrical about -tp/2.

Parameters

datatag (str) – Key used to access correlator data in the input data dictionary (see corrfitter.CorrFitter): data[self.datatag] is a (1-d) array containing the correlator values (gvar.GVars).
a (str or tuple) – Key identifying the fit parameters for the source amplitudes an in the dictionary of priors provided to corrfitter.CorrFitter; or a two-tuple of keys for the source amplitudes (an, ao). The corresponding values in the dictionary of priors are (1-d) arrays of prior values with one term for each an[i] or ao[i]. Replacing either key by None causes the corresponding term to be dropped from the fit function. These keys are used to label the corresponding parameter arrays in the fit results as well as in the prior.
b (str or tuple) – Same as self.a but for the sinks (bn, bo) instead of the sources (an, ao).
dE (str) – Key identifying the fit parameters for the energy differences dEn in the dictionary of priors provided by corrfitter.CorrFitter; or a two-tuple of keys for the energy differences (dEn, dEo). The corresponding values in the dictionary of priors are (1-d) arrays of prior values with one term for each dEn[i] or dEo[i]. Replacing either key by None causes the corresponding term to be dropped from the fit function. These keys are used to label the corresponding parameter arrays in the fit results as well as in the prior.
s (float or tuple) – Overall factor sn for non-oscillating part of fit function, or two-tuple of overall factors (sn, so) for both pieces.
tdata (list of ints) – The ts corresponding to data entries in the input data. Note that len(self.tdata) should equal len(data[self.datatag]). If tdata is omitted, tdata=numpy.arange(tp) is assumed, or tdata=numpy.arange(tmax) if tp is not specified.
tfit (list of ints) – List of ts to use in the fit. Only data with these ts (all of which should be in tdata) is used in the fit. If tfit is omitted, it is assumed to be all t values from tdata that are larger than or equal to tmin (if specified) and smaller than or equal to tmax (if specified).
tp (int or None) – If tp is positive, the correlator is assumed to be periodic with Gab(t)=Gab(tp-t). If negative, the correlator is assumed to be anti-periodic with Gab(t)=-Gab(-tp-t). Setting tp=None implies that the correlator is not periodic, but rather continues to fall exponentially as t is increased indefinitely.
tmin (int or None) – If tfit is omitted, it is assumed to be all t values from tdata that are larger than or equal to tmin and smaller than or equal to tmax (if specified). tmin is ignored if tfit is specified.
tmax (int or None) – If tfit is omitted, it is assumed to be all t values from tdata that are larger than or equal to tmin (if specified) and smaller than or equal to tmax. tmin is ignored if tfit and tdata are specified.
ncg (int) – Width of bins used to coarse-grain the correlator before fitting. Each bin of ncg correlator values is replaced by its average. Default is ncg=1 (ie, no coarse-graining).
reverse (bool) – If True, the data associated with self.datatag is time-reversed (data -> [data[0], data[-1], data[-2]...data[1]]). Ignored otherwise.
otherdata (str or list or None) – Data tag or list of data tags for additional data that are averaged with the self.datatag data before fitting. This is useful including data from correlators with the source and sink interchanged. Default is None.
reverseddata (str or list or None) – Data tag or list of data tags for data that is time-reversed and then averaged with the self.datatag data before fitting. Default is None.

builddata(data)¶: Assemble fit data from dictionary data.

builddataset(dataset)¶: Assemble fit data from data set dictionary dataset.

buildprior(prior, nterm=None, mopt=None)¶

Create fit prior by extracting relevant pieces from prior.

This routine selects the entries in dictionary prior corresponding to the model’s fit parameters. If nterm is not None, it also adjusts the number of terms that are retained.

Parameters

prior (dictionary) – Dictionary containing priors for fit parameters.
nterm (None or int or two-tuple) – Setting nterm=(n,no) restricts the number of terms to n in the non-oscillating part and no in the oscillating part of the fit function. Replacing either or both by None keeps all terms, as does setting nterm=None. This optional argument is used to implement marginalization.

fitfcn(p, t=None)¶: Return fit function for parameters p.

class corrfitter.Corr3(datatag, T, Vnn, a, b, dEa, dEb, sa=1.0, sb=1.0, Vno=None, Von=None, Voo=None, tdata=None, tfit=None, tmin=None. reverse=False, symmetric_V=False, reverseddata=[], otherdata=[])¶

Three-point correlators Gavb(t, T) = <b(T) V(t) a(0)>.

corrfitter.Corr3 models the t dependence of a 3-point correlator Gavb(t, T) using

Gavb(t, T) =
 sum_i,j san * an[i] * fn(Ean[i],t) * Vnn[i,j] * sbn * bn[j] * fn(Ebn[j],T-t)
+sum_i,j san * an[i] * fn(Ean[i],t) * Vno[i,j] * sbo * bo[j] * fo(Ebo[j],T-t)
+sum_i,j sao * ao[i] * fo(Eao[i],t) * Von[i,j] * sbn * bn[j] * fn(Ebn[j],T-t)
+sum_i,j sao * ao[i] * fo(Eao[i],t) * Voo[i,j] * sbo * bo[j] * fo(Ebo[j],T-t)

where

fn(E, t) =  exp(-E*t)
fo(E, t) = (-1)**t * exp(-E*t)

The fit parameters for the non-oscillating piece of Gavb (first term) are Vnn[i,j], an[i], bn[j], dEan[i] and dEbn[j] where, for example:

dEan[0] = Ean[0]
dEan[i] = Ean[i] - Ean[i-1] > 0     (for i>0)

and therefore Ean[i] = sum_j=0..i dEan[j]. The parameters for the other terms are similarly defined.

Parameters

datatag (str) – Tag used to label correlator in the input data.
a (str or tuple) – Key identifying the fit parameters for the source amplitudes an, for a->V, in the dictionary of priors provided to corrfitter.CorrFitter; or a two-tuple of keys for the source amplitudes (an, ao). The corresponding values in the dictionary of priors are (1-d) arrays of prior values with one term for each an[i] or ao[i]. Replacing either key by None causes the corresponding term to be dropped from the fit function. These keys are used to label the corresponding parameter arrays in the fit results as well as in the prior.
b (str or tuple) – Same as self.a but for the V->b sink amplitudes (bn, bo).
dEa (str or tuple) – Fit-parameter label for a->V intermediate-state energy differences dEan, or two-tuple of labels for the differences (dEan, dEao). Each label represents an array of energy differences. Replacing either label by None causes the corresponding term in the correlator function to be dropped. These keys are used to label the corresponding parameter arrays in the fit results as well as in the prior.
dEb (str or tuple) – Same as self.dEa but for V->b sink energies (dEbn, dEbo).
sa (float or tuple) – Overall factor san for the non-oscillating a->V terms in the correlator, or two-tuple containing the overall factors (san, sao) for the non-oscillating and oscillating terms. Default is (1,-1).
sb (float or tuple) – Same as self.sa but for V->b sink overall factors (sbn, sbo).
Vnn (str or None) –
Fit-parameter label for the matrix of current matrix elements Vnn[i,j] connecting non-oscillating states. The matrix must be square and symmetric if symmetric_V=True, and only the elements V[i,j] for j>=i are specified, using a 1-d array V_sym with the following layout:
```
[V[0,0],V[0,1],V[0,2]...V[0,N],
        V[1,1],V[1,2]...V[1,N],
               V[2,2]...V[2,N],
                     .
                      .
                       .
                        V[N,N]]
```
Note that V[i,j] = V_symm[i*N + j - i * (i+1) / 2] for j>=i. Set Vnn=None to omit it.
Vno (str or None) – Fit-parameter label for the matrix of current matrix elements Vno[i,j] connecting non-oscillating to oscillating states. Only one of Von and Vno can be specified if symmetric_V=True; the other is defined to be its transform. Set Vno=None to omit it.
Von (str or None) – Fit-parameter label for the matrix of current matrix elements Vno[i,j] connecting oscillating to non- oscillating states. Only one of Von and Vno can be specified if symmetric_V=True; the other is defined to be its transform. Set Von=None to omit it.
Voo (str or None) –
Fit-parameter label for the matrix of current matrix elements Voo[i,j] connecting oscillating states. The matrix must be square and symmetric if symmetric_V=True, and only the elements V[i,j] for j>=i are specified, using a 1-d array V_sym with the following layout:
```
[V[0,0],V[0,1],V[0,2]...V[0,N],
        V[1,1],V[1,2]...V[1,N],
               V[2,2]...V[2,N],
                     .
                      .
                       .
                        V[N,N]]
```
Note that V[i,j] = V_symm[i*N + j - i * (i+1) / 2] for j>=i. Set Voo=None to omit it.

reverse (bool) –

If True, the data associated with self.datatag is time-reversed before fitting (interchanging t=0 with t=T). This is useful for doing simultaneous fits to a->V->b and b->V->a, where one is time-reversed relative to the other: e.g.,

models = [ ...
    Corr3(
        datatag='a->V->b', tmin=3, T=15,
        a=('a', 'ao'), dEa=('dEa', 'dEao'),
        b=('b', 'bo'), dEb=('dEb', 'dEbo'),
        Vnn='Vnn', Vno='Vno', Von='Von', Voo='Voo',
        ),
    Corr3(
        datatag='b->V->a', tmin=3, T=15,
        a=('a', 'ao'), dEa=('dEa', 'dEao'),
        b=('b', 'bo'), dEb=('dEb', 'dEbo'),
        Vnn='Vnn', Vno='Vno', Von='Von', Voo='Voo',
        reverse=True,
        ),
    ...
    ]

Another (faster) strategy for such situations is to average data from the second process with that from the first, before fitting, using keyword reverseddata. Default is False.

symmetric_V (bool) – If True, the fit function for a->V->b is unchanged (symmetrical) under the the interchange of a and b. Then Vnn and Voo are square, symmetric matrices and their priors are one-dimensional arrays containing only elements V[i,j] with j>=i, as discussed above. Only one of Von and Vno can be specified if symmetric_V=True; the other is defined to be its transform.
T (int) – Separation between source and sink.
tdata (list of ints) – The ts corresponding to data entries in the input data. If omitted, is assumed equal to numpy.arange(T + 1).
tfit (list of ints) – List of ts to use in the fit. Only data with these ts (all of which should be in tdata) is used in the fit. If omitted, is assumed equal to numpy.arange(tmin, T - tmin + 1).
tmin (int or None) – If tfit is omitted, it is set equal to numpy.arange(tmin, T - tmin + 1). tmin is ignored if tfit is specified.
ncg (int) – Width of bins used to coarse-grain the correlator before fitting. Each bin of ncg correlator values is replaced by its average. Default is ncg=1 (ie, no coarse-graining).
reverseddata (str or list or None) –
Data tag or list of data tags for additional data that are time-reversed and then averaged with the self.datatag data before fitting. This is useful for folding data from b->V->a into a fit for a->V->b: e.g.,
```
Corr3(
    datatag='a->V->b',
    a=('a', 'ao'), dEa=('dEa', 'dEao'),
    b=('b', 'bo'), dEb=('dEb', 'dEbo'),
    Vnn='Vnn', Vno='Vno', Von='Von', Voo='Voo',
    tmin=3, T=15, reverseddata='b->V->a'
    ),
```
This is faster than using a separate model with reverse=True. Default is None.
otherdata (str or list or None) – Data tag or list of data tags for additional data that are averaged with the self.datatag data before fitting. Default is None.

builddata(data)¶

Extract fit data corresponding to this model from data set data.

The fit data is returned in a 1-dimensional array; the fitfcn must return arrays of the same length.

Parameters: data – Data set containing the fit data for all models. This is typically a dictionary, whose keys are the datatags of the models.

builddataset(dataset)¶

Extract fit dataset from gvar.dataset.Dataset dataset.

The code

import gvar as gv

data = gv.dataset.avg_data(m.builddataset(dataset))

that builds data for model m should be functionally equivalent to

import gvar as gv

data = m.builddata(gv.dataset.avg_data(dataset))

This method is optional. It is used only by MultiFitter.process_dataset().

Parameters: dataset – gvar.dataset.Dataset (or similar dictionary) dataset containing the fit data for all models. This is typically a dictionary, whose keys are the datatags of the models.

buildprior(prior, mopt=None, nterm=None)¶

Extract fit prior from prior.

Returns a dictionary containing the part of dictionary prior that is relevant to this model’s fit. The code could be as simple as collecting the appropriate pieces: e.g.,

def buildprior(self, prior, mopt=None):
    mprior = gv.BufferDict()
    model_keys = [...]
    for k in model_keys:
        mprior[k] = prior[k]
    return mprior

where model_keys is a list of keys corresponding to the model’s parameters. Supporting non-Gaussian distributions requires a slight modification: e.g.,

def buildprior(self, prior, mopt=None):
    mprior = gv.BufferDict()
    model_keys = [...]
    for k in gv.get_dictkeys(prior, model_keys):
        mprior[k] = prior[k]
    return mprior

Marginalization involves omitting some of the fit parameters from the model’s prior. mopt=None implies no marginalization. Otherwise mopt will typically contain information about what and how much to marginalize.

Parameters

prior – Dictionary containing a priori estimates of all fit parameters.
mopt (object) – Marginalization options. Ignore if None. Otherwise marginalize fit parameters as specified by mopt. mopt can be any type of Python object; it is used only in buildprior and is passed through to it unchanged.

fitfcn(p, t=None)¶

Compute fit function fit for parameters p.

Results are returned in a 1-dimensional array the same length as (and corresponding to) the fit data returned by self.builddata(data).

If marginalization is supported, fitfcn must work with or without the marginalized parameters.

Parameters: p – Dictionary of parameter values.

`corrfitter.CorrFitter` Objects¶

corrfitter.CorrFitter objects are wrappers for lsqfit.nonlinear_fit() which is used to fit a collection of models to a collection of Monte Carlo data.

class corrfitter.CorrFitter(models, **kargs)¶

Nonlinear least-squares fitter for a collection of correlator models.

Parameters

models – List of models, derived from lsqfit.MultiFitterModel, to be fit to the data. Individual models in the list can be replaced by lists of models or tuples of models; see below.
nterm (tuple or int or None) – Number of terms fit in the non-oscillating part of fit functions; or a two-tuple of numbers indicating how many terms to fit in each of the non-oscillating and oscillating parts. Terms omitted from the fit are marginalized (i.e., included as corrections to the fit data). If set to None, all parameters in the prior are fit, and none are marginalized.
mopt – Marginalization options; alias for nterm.
ratio (bool) – If True, implement marginalization using ratios: data_marg = data * fitfcn(prior_marg) / fitfcn(prior). If False (default), implement using differences: data_marg = data + (fitfcn(prior_marg) - fitfcn(prior)).
fast (bool) – Setting fast=True (default) strips any variable not required by the fit from the prior. This speeds fits but loses information about correlations between variables in the fit and those that are not.
fitterargs – Additional arguments for the lsqfit.nonlinear_fit, such as tol, maxit, svdcut, fitter, etc., as needed.

lsqfit(data=None, prior=None, pdata=None, p0=None, **kargs)¶

Compute least-squares fit of models to data.

CorrFitter.lsqfit() fits all of the models together, in a single fit. It returns the lsqfit.nonlinear_fit object from the fit.

See documentation for lsqfit.MultiFitter.lsqfit for more information.

Parameters

data – Input data. One of data or pdata must be specified but not both.
pdata – Input data that has been processed by the models using CorrFitter.process_data() or CorrFitter.process_dataset(). One of data or pdata must be specified but not both. pdata is obtained from data by collecting the output from m.builddata(data) for each model m and storing it in a dictionary with key m.datatag.
prior – Bayesian prior for fit parameters used by the models.
p0 – Dictionary , indexed by parameter labels, containing initial values for the parameters in the fit. Setting p0=None implies that initial values are extracted from the prior. Setting p0="filename" causes the fitter to look in the file with name "filename" for initial values and to write out best-fit parameter values after the fit (for the next call to self.lsqfit()).
wavg_svdcut (float) – SVD cut used in weighted averages for parallel fits.
kargs – Arguments that override parameters specified when the CorrFitter was created. Can also include additional arguments to be passed through to the lsqfit fitter.

chained_lsqfit(data=None, prior=None, pdata=None, p0=None, **kargs)¶

Compute chained least-squares fit of models to data.

CorrFitter.chained_lsqfit() fits the models specified in models one at a time, in sequence, with the fit output from one being fed into the prior for the next.

See documentation for lsqfit.MultiFitter.chained_lsqfit for (much) more information.

Parameters

data – Input data. One of data or pdata must be specified but not both. pdata is obtained from data by collecting the output from m.builddata(data) for each model m and storing it in a dictionary with key m.datatag.
pdata – Input data that has been processed by the models using CorrFitter.process_data() or CorrFitter.process_dataset(). One of data or pdata must be specified but not both.
prior – Bayesian prior for fit parameters used by the models.
p0 – Dictionary , indexed by parameter labels, containing initial values for the parameters in the fit. Setting p0=None implies that initial values are extracted from the prior. Setting p0="filename" causes the fitter to look in the file with name "filename" for initial values and to write out best-fit parameter values after the fit (for the next call to self.lsqfit()).
kargs – Arguments that override parameters specified when the CorrFitter was created. Can also include additional arguments to be passed through to the lsqfit fitter.

set(**kargs)¶

Reset default keyword parameters.

Assigns new default values from dictionary kargs to the fitter’s keyword parameters. Keywords for the underlying lsqfit fitters can also be included (or grouped together in dictionary fitterargs).

Returns tuple (kargs, oldkargs) where kargs is a dictionary containing all lsqfit.MultiFitter keywords after they have been updated, and oldkargs contains the original values for these keywords. Use fitter.set(**oldkargs) to restore the original values.

static read_dataset(inputfiles, grep=None, keys=None, h5group='/', binsize=1, tcol=0, Gcol=1)¶

Read correlator Monte Carlo data from files into a gvar.dataset.Dataset.

Three files formats are supported by read_dataset(), depending upon inputfiles.

If inputfiles is a string ending in '.h5', it is assumed to be the name of a file in hpf5 format. The file is opened as h5file and all hpf5 datasets in h5file[h5group] are collected into a dictionary and returned.

The second file format is the text-file format supported by gvar.dataset.Dataset: each line consists of a tag or key identifying a correlator followed by data corresponding to a single Monte Carlo measurement of the correlator. This format is assumed if inputfiles is a filename or a list of filenames. It allows a single file to contain an arbitrary number of measurements for an arbitrary number of different correlators. The data can also be spread over multiple files. A typical file might look like

# this is a comment; it is ignored
aa 1.237 0.912 0.471
bb 3.214 0.535 0.125
aa 1.035 0.851 0.426
bb 2.951 0.625 0.091
...

which describes two correlators, aa and bb, each having three different t values.

The third file format is assumed when inputfiles is a dictionary. The dictionary’s keys and values identify the (one-dimensional) correlators and the files containing their Monte Carlo data, respectively. So the data for correlators aa and bb above are in separate files:

fileinputs = dict(aa='aafile', bb='bbfile')

Each line in these data files consists of an index t value followed by the corresponding value for correlator G(t). The ts increase from line to line up to their maximum value, at which point they repeat. The aafile file for correlator aa above would look like:

# this is a comment; it is ignored
1.237
0.912
0.471
1.035
0.851
0.426
...

The columns in these files containing t and G(t) are assumed to be columns 0 and 1, respectively. These can be changed by setting arguments tcol and Gcol, respectively.

corrfitter.process_dataset supports keywords binsize, grep and keys. If binsize is greater than one, random samples are binned with bins of size binsize. If grep is not None, only keys that match or partially match regular expression grep are retained; others are ignored. If keys is not None, only keys that are in list keys are retained; others are discarded.

simulated_pdata_iter(n, dataset, p_exact=None, rescale=1.0)¶

Create iterator that returns simulated fit pdata from dataset.

Simulated fit data has the same covariance matrix as pdata=self.process_dataset(dataset), but mean values that fluctuate randomly, from copy to copy, around the value of the fitter’s fit function evaluated at p=p_exact. The fluctuations are generated from bootstrap copies of dataset.

The best-fit results from a fit to such simulated copies of pdata should agree with the numbers in p_exact to within the errors specified by the fits (to the simulated data) — p_exact gives the “correct” values for the parameters. Knowing the correct value for each fit parameter ahead of a fit allows us to test the reliability of the fit’s error estimates and to explore the impact of various fit options (e.g., fitter.chained_fit versus fitter.lsqfit, choice of SVD cuts, omission of select models, etc.)

Typically one need examine only a few simulated fits in order to evaluate fit reliability, since we know the correct values for the parameters ahead of time. Consequently this method is much faster than traditional bootstrap analyses.

p_exact is usually taken from the last fit done by the fitter (self.fit.pmean) unless overridden in the function call. Typical usage is as follows:

dataset = gvar.dataset.Dataset(...)
data = gvar.dataset.avg_data(dataset)
...
fit = fitter.lsqfit(data=data, ...)
...
for spdata in fitter.simulated_pdata_iter(n=4, dataset):
    # redo fit 4 times with different simulated data each time
    # here p_exact=fit.pmean is set implicitly
    sfit = fitter.lsqfit(pdata=spdata, ...)
    ... check that sfit.p (or functions of it) agrees ...
    ... with p_exact=fit.pmean to within sfit.p's errors      ...

Parameters

n (int) – Maximum number of simulated data sets made by iterator.
dataset (dictionary) – Dataset containing Monte Carlo copies of the correlators. dataset[datatag] is a two-dimensional array for the correlator corresponding to datatag, where the first index labels the Monte Carlo copy and the second index labels time.
p_exact (dictionary or None) – Correct parameter values for fits to the simulated data — fit results should agree with p_exact to within errors. If None, uses self.fit.pmean from the last fit.
rescale (float) – Rescale errors in simulated data by rescale (i.e., multiply covariance matrix by rescale ** 2). Default is one, which implies no rescaling.

`corrfitter.EigenBasis` Objects¶

corrfitter.EigenBasis objects are useful for analyzing two-point and three-point correlators with multiplle sources and sinks. The current interface for EigenBasis is experimental. It may change in the near future, as experience accumulates from its use.

class corrfitter.EigenBasis(data, srcs, t, keyfmt='{s1}.{s2}', tdata=None, osc=False)¶

Eigen-basis of correlator sources/sinks.

Given $N$ sources/sinks and the $N \times N$ matrix $G_{ij}(t)$ of 2-point correlators created from every combination of source and sink, we can define a new basis of sources that makes the matrix correlator approximately diagonal. Each source in the new basis is associated with an eigenvector $v^{(a)}$ defined by the matrix equation

$G(t_1) v^{(a)} = \lambda^{(a)}(t_1-t_0) G(t_0) v^{(a)},$

for some $t_0, t_1$ . As $t_0, t_1$ increase, fewer and fewer states couple to $G(t)$ . In the limit where only $N$ states couple, the correlator

$\overline{G}_{ab}(t) \equiv v^{(a)T} G(t) v^{(b)}$

becomes diagonal, and each diagonal element couples to only a single state.

In practice, this condition is only approximate: that is, $\overline G(t)$ is approximately diagonal, with diagonal elements that overlap strongly with the lowest lying states, but somewhat with other states. These new sources are nevertheless useful for fits because there is an obvious prior for their amplitudes: prior[a][b] approximately equal to one when b==a, approximately zero when b!=a and b<N, and order one otherwise.

Such a prior can significantly enhance the stability of a multi-source fit, making it easier to extract reliable results for excited states. It encodes the fact that only a small number of states couple strongly to $G(t)$ by time $t_0$ , without being overly prescriptive about what their energies are. We can easilty project our correlator onto the new eigen-basis (using EigenBasis.apply()) in order to use this prior, but this is unnecessary. EigenBasis.make_prior() creates a prior of this type in the eigen-basis and then transforms it back to the original basis, thereby creating an equivalent prior for the amplitudes corresponding to the original sources.

Typical usage is straightforward. For example,

basis = EigenBasis(
    data,                           # data dictionary
    keyfmt='G.{s1}.{s2}',           # key format for dictionary entries
    srcs=['local', 'smeared'],      # names of sources/sinks
    t=(5, 6),                       # t0, t1 used for diagonalization
    )
prior = basis.make_prior(nterm=4, keyfmt='m.{s1}')

creates an eigen-prior that is optimized for fitting the 2-by-2 matrix correlator given by

[[data['G.local.local'],     data['G.local.smeared']]
 [data['G.smeared.local'],   data['G.smeared.smeared']]]

where data is a dictionary containing all the correlators. Parameter t specifies the times used for the diagonalization: $t_0=5$ and $t_1=7$ . Parameter nterm specifies the number of terms in the fit. basis.make_prior(...) creates priors prior['m.local'] and prior['m.smeared'] for the amplitudes corresponding to the local and smeared source, and a prior prior[log(m.dE)] for the logarithm of the energy differences between successive levels.

The amplitudes prior['m.local'] and prior['m.smeared'] are complicated, with strong correlations between local and smeared entries for the same state. Projecting the prior unto the eigen-basis, however, reveals its underlying structure:

p_eig = basis.apply(prior, keyfmt='m.{s1}')

implies

p_eig['m.0'] = [1.0(3), 0.0(1), 0(1), 0(1)]
p_eig['m.1'] = [0.0(1), 1.0(3), 0(1), 0(1)]

where the different entries are now uncorrelated. This structure registers our expectation that the 'm.0' source in the eigen-basis overlaps strongly with the ground state, but almost not at all with the first excited state; and vice versa for the 'm.1' source. Amplitude p_eig is noncommittal about higher states. This structure is built into prior['m.local'] and prior['smeared'].

It is easy to check that fit results are consistent with the underlying prior. This can be done by projecting the best-fit parameters unto the eigen-basis using p_eig = basis.apply(fit.p). Alternatively, a table listing the amplitudes in the new eigen-basis, together with the energies, is printed by:

print(basis.tabulate(fit.p, keyfmt='m.{s1}', eig_srcs=True))

The prior can be adjusted, if needed, using the dEfac, ampl, and states arguments in EigenBasis.make_prior().

EigenBasis.tabulate() is also useful for printing the amplitudes for the original sources:

print(basis.tabulate(fit.p, keyfmt='m.{s1}'))

The example above assumed there were no oscillating states (from staggered quarks) in the correlator. Two modifications to the code above are needed if there are oscillating states: 1) EigenBasis parameter osc must be set to osc=True; and 2) keyfmt for basis.make_prior(), basis.apply(), basis.unapply(), and basis.tabulate() needs to be a tuple where keyfmt[0] is the key format for non-oscillating states and keyfmt[1] is for oscillating states (e.g., ('m.{s1}', 'mo.{s1}')). The energies obtained from the eigenvalue analysis are collected in array basis.E where basis.E[:basis.neig[0]] are the energies for the non-oscillating states, and basis.E[basis.neig[0]:] are the energies for the oscillating states.

corrfitter.EigenBasis requires the scipy library in Python.

Parameters

data – Dictionary containing the matrix correlator using the original basis of sources and sinks.
keyfmt – Format string used to generate the keys in dictionary data corresponding to different components of the matrix of correlators. The key for $G_{ij}$ is assumed to be keyfmt.format(s1=i, s2=j) where i and j are drawn from the list of sources, srcs.
srcs – List of source names used with keyfmt to create the keys for finding correlator components $G_{ij}$ in the data dictionary.
t – t=(t0, t1) specifies the t values used to diagonalize the correlation function. Larger t values are better than smaller ones, but only if the statistics are adequate. When fitting staggered-quark correlators, with oscillating components, choose t values where the oscillating pieces are positive (typically odd t). If only one t is given, t=t0, then t1=t0+2 is used with it. Fits that use corrfitter.EigenBasis typically depend only weakly on the choice of t.
tdata – Array containing the times for which there is correlator data. tdata is set equal to numpy.arange(len(G_ij)) if it is not specified (or equals None).
osc (bool) – Set osc=True when the correlator contains states that oscillate in sign ((-1)**t); otherwise osc=False (default).

In addition to keyfmt, srcs, t and tdata above, the main attributes are:

E¶: Array of approximate energies obtained from the eigenanalysis.

eig_srcs¶: List of labels for the sources in the eigen-basis: '0', '1' …

correction¶: The sum of the gvar.regulate`() corrections added to the data by the last call to EigenBasis.regulate().

nmod¶: The number of degrees of freedom modified in the last call to EigenBasis.regulate().

v¶: v[a] is the eigenvector corresponding to source a in the new basis, where a=0,1....

v_inv¶: v_inv[i] is the inverse-eigenvector for transforming from the new basis back to the original basis.

The main methods are:

apply(data, keyfmt='{s1}')¶

Transform data to the eigen-basis.

The data to be transformed is data[k] where key k equals keyfmt.format(s1=s1) for vector data, or keyfmt.format(s1=s1, s2=s2) for matrix data with sources s1 and s2 drawn from self.srcs. A dictionary containing the transformed data is returned using the same keys but with the sources replaced by '0', '1' ... (from basis.eig_srcs).

If keyfmt is an array of formats, the transformation is applied for each format and a dictionary containing all of the results is returned. This is useful when the same sources and sinks are used for different types of correlators (e.g., in both two-point and three-point correlators).

make_prior(nterm, keyfmt=('{s1}', 'o.{s1}'), dEfac='1(1)', ampl=('1.0(3)', '0.03(10)', '0.2(1.0)'), states=None, eig_srcs=False)¶

Create prior from eigen-basis.

Parameters

nterm (int) – Number of terms in fit function.
keyfmt (str or tuple) – Format string usded to generate keys for amplitudes and energies in the prior (a dictionary): keys are obtained from keyfmt.format(s1=a) where a is one of the original sources, self.srcs, if eig_srcs=False (default), or one of the eigen-sources, self.eig_srcs, if eig_srcs=True. The key for the energy differences is generated by 'log({})'.format(keyfmt.format(s1='dE')). keyfmt should be a tuple of strings when osc==True, where keyfmt[0] is used for the non-oscillating states and keyfmt[1] for the oscillating states. keyfmt[1] is ignored (if present) when osc==False. The default is keyfmt=('{s1}', 'o.{s1}').
dEfac (str or gvar.GVar) – A string or gvar.GVar from which the priors for energy differences dE[i] are constructed. The mean value for dE[0] is set equal to the lowest energy obtained from the diagonalization. The mean values for the other dE[i]s are set equal to the difference between the lowest two energies from the diagonalization (or to the lowest energy if there is only one). These central values are then multiplied by gvar.gvar(dEfac). The default value, 1(1), sets the width equal to the mean value. The prior is the logarithm of the resulting values.
ampl (tuple) – A 3-tuple of strings or gvar.GVars from which priors are contructed for amplitudes corresponding to the eigen-sources. gvar.gvar(ampl[0]) is used for for source components where the overlap with a particular state is expected to be large; 1.0(3) is the default value. gvar.gvar(ampl[1]) is used for states that are expected to have little overlap with the source; 0.03(10) is the default value. gvar.gvar(ampl[2]) is used where there is nothing known about the overlap of a state with the source; 0(1) is the default value.
states (list or tuple) – A list of the states in the correlator corresponding to successive eigen-sources, where states[i] is the state corresponding to i-th source. The correspondence between sources and states is strong for the first sources, but can decay for subsequent sources, depending upon the quality of the data being used and the t values used in the diagonalization. In such situations one might specify fewer states than there are sources by making the length of states smaller than the number of sources. Setting states=[] assigns broad priors to the every component of every source. Parameter states can also be used to deal with situations where the order of later sources is not aligned with that of the actual states: for example, states=[0,1,3] connects the eigen-sources with the first, second and fourth states in the correlator. If states is a tuple of lists, list states[0] is used for non-oscillating states while list states[1] is used for oscillating states (if there are any). The default value, states=[0, 1 ... N-1] where N is the number of sources, assumes that sources and states are aligned.
eig_srcs (bool) – Amplitudes for the eigen-sources are tabulated if eig_srcs=True; otherwise amplitudes for the original basis of sources are tabulated (default).

regulate(data, keyfmt=None, svdcut=None, eps=None)¶

Regulate singular correlation matrices in eigen-basis.

gvar.regulate() is used to regulate singularities in data[k] where key k equals keyfmt.format(s1=s1) for vector data, or keyfmt.format(s1=s1, s2=s2) for matrix data with sources s1 and s2 drawn from self.srcs. The data are transformed to the eigen-basis of sources/sinks before a cutoff is introduced (svdcut or eps). and then transformed back to the original basis of sources. Results are returned in a dictionary containing the modified correlators.

If keyfmt is a list of formats, the cutoff is applied to the collection of data formed from each format. The defaul value for keyfmt is self.keyfmt.

Parameters

data – Eigen-basis data dictionary.
keyfmt (str) – Data keys are either keyfmt.format(s1=s1) for vector data or keyfmt.format(s1=s1, s2=s2) for matrix data, where s1 and s2 are sources drawn from self.srcs.
eps (float) – If positive, singularities in the correlation matrix for data[k] are regulated using gvar.regulate() with cutoff eps. Ignored if svdcut is specified (and not None).
svdcut (float) – If nonzero, singularities in the correlation matrix for data[k] are regulated using gvar.regulate() with an SVD cutoff svdcut. Default is svdcut=1e-12.

svd(data, keyfmt=None, svdcut=1e-15)¶

Apply SVD cut to data in the eigen-basis.

The SVD cut is applied to data[k] where key k equals keyfmt.format(s1=s1) for vector data, or keyfmt.format(s1=s1, s2=s2) for matrix data with sources s1 and s2 drawn from self.srcs. The data are transformed to the eigen-basis of sources/sinks before the cut is applied and then transformed back to the original basis of sources. Results are returned in a dictionary containing the modified correlators.

If keyfmt is a list of formats, the SVD cut is applied to the collection of data formed from each format. The defaul value for keyfmt is self.keyfmt.

tabulate(p, keyfmt='{s1}', nterm=None, nsrcs=None, eig_srcs=False, indent=' ')¶

Create table containing energies and amplitudes for nterm states.

Given a correlator-fit result fit and a corresponding EigenBasis object basis, a table listing the energies and amplitudes for the first N states in correlators can be printed using

print basis.tabulate(fit.p)

where N is the number of sources and basis is an EigenBasis object. The amplitudes are tabulated for the original sources unless parameter eig_srcs=True, in which case the amplitudes are projected onto the the eigen-basis defined by basis.

Parameters

p – Dictionary containing parameters values.
keyfmt – Parameters are p[k] where keys k are obtained from keyfmt.format(s1=s) where s is one of the original sources (basis.srcs) or one of the eigen-sources (basis.eig_srcs). The default definition is '{s1}'.
nterm – The number of states from the fit tabulated. The default sets nterm equal to the number of sources in the basis.
nsrcs – The number of sources tabulated. The default causes all sources to be tabulated.
eig_srcs – Amplitudes for the eigen-sources are tabulated if eig_srcs=True; otherwise amplitudes for the original basis of sources are tabulated (default).
indent – A string prepended to each line of the table. Default is 4 * ' '.

unapply(data, keyfmt='{s1}')¶

Transform data from the eigen-basis to the original basis.

The data to be transformed is data[k] where key k equals keyfmt.format(s1=s1) for vector data, or keyfmt.format(s1=s1, s2=s2) for matrix data with sources s1 and s2 drawn from self.eig_srcs. A dictionary containing the transformed data is returned using the same keys but with the original sources (from self.srcs).

If keyfmt is an array of formats, the transformation is applied for each format and a dictionary containing all of the results is returned. This is useful when the same sources and sinks are used for different types of correlators (e.g., in both two-point and three-point correlators).

Fast Fit Objects¶

class corrfitter.fastfit(G, ampl='0(1)', dE='1(1)', E=None, s=(1, -1), tp=None, tmin=6, svdcut=1e-06, osc=False, nterm=10, **fitterargs)¶

Fast fit of a two-point correlator.

This function class estimates E=En[0] and ampl=an[0]*bn[0] for a two-point correlator modeled by

Gab(t) = sn * sum_i an[i]*bn[i] * fn(En[i], t)
       + so * sum_i ao[i]*bo[i] * fo(Eo[i], t)

where (sn, so) is typically (1, -1) and

fn(E, t) =  exp(-E*t) + exp(-E*(tp-t)) # tp>0 -- periodic
       or   exp(-E*t) - exp(-E*(-tp-t))# tp<0 -- anti-periodic
       or   exp(-E*t)                  # if tp is None (nonperiodic)

fo(E, t) = (-1)**t * fn(E, t)

Prior estimates for the amplitudes and energies of excited states are used to remove (that is, marginalize) their contributions to give a corrected correlator Gc(t) that includes uncertainties due to the terms removed. Estimates of E are given by:

Eeff(t) = arccosh(0.5 * (Gc(t+1) + Gc(t-1)) / Gc(t)),

The final estimate is the weighted average Eeff_avg of the Eeff(t)s for different ts. Similarly, an estimate for the amplitude ampl is obtained from the weighted average of

Aeff(t) = Gc(t) / fn(Eeff_avg, t).

If osc=True, an estimate is returned for Eo[0] rather than En[0], and ao[0]*bo[0] rather than an[0]*bn[0]. These estimates are reliable when Eo[0] is smaller than En[0] (and so dominates at large t), but probably not otherwise.

Examples

The following code examines a periodic correlator (period 64) at large times (t >= tmin), where estimates for excited states don’t matter much:

>>> import corrfitter as cf
>>> print(G)
[0.305808(29) 0.079613(24) ... ]
>>> fit = cf.fastfit(G, tmin=24, tp=64)
>>> print('E =', fit.E, ' ampl =', fit.ampl)
E = 0.41618(13)  ampl = 0.047686(95)

Smaller tmin values can be used if (somewhat) realistic priors are provided for the amplitudes and energy gaps:

>>> fit = cf.fastfit(G, ampl='0(1)', dE='0.5(5)', tmin=3, tp=64)
>>> print('E =', fit.E, ' ampl =', fit.ampl)
E = 0.41624(11)  ampl = 0.047704(71)

The result here is roughly the same as from the larger tmin, but this would not be true for a correlator whose signal to noise ratio falls quickly with increasing time.

corrfitter.fastfit estimates the amplitude and energy at all times larger than tmin and then averages to get its final results. The chi-squared of the average (e.g., fit.E.chi2) gives an indication of the consistency of the estimates from different times. The chi-squared per degree of freedom is printed out for both the energy and the amplitude using

>>> print(fit)
E: 0.41624(11) ampl: 0.047704(71) chi2/dof [dof]: 0.9 0.8 [57] Q: 0.8 0.9

Large values for chi2/dof indicate an unreliable results. In such cases the priors should be adjusted, and/or tmin increased, and/or an SVD cut introduced. The averages in the example above have good values for chi2/dof.

Parameters

G – An array of gvar.GVars containing the two-point correlator. G[j] is assumed to correspond to time t=j, where j=0....
ampl – A gvar.GVar or its string representation giving an estimate for the amplitudes of the ground state and the excited states. Use ampl=(ampln, amplo) when the correlator contains oscillating states; ampln is the estimate for non-oscillating states, and amplo for oscillating states; setting one or the other to None causes the corresponding terms to be dropped. Default value is '0(1)'.
dE – A gvar.GVar or its string representation giving an estimate for the energy separation between successive states. This estimate is also used to provide an estimate for the lowest energy when parameter E is not specified. Use dE=(dEn, dEo) when the correlator contains oscillating states: dEn is the estimate for non-oscillating states, and dEo for oscillating states; setting one or the other to None causes the corresponding terms to be dropped. Default value is '1(1)'.
E – A gvar.GVar or its string representation giving an estimate for the energy of the lowest-lying state. Use E=(En, Eo) when the correlator contains oscillating states: En is the estimate for the lowest non-oscillating state, and Eo for lowest oscillating state. Setting E=None causes E to be set equal to dE. Default value is None.
s – A tuple containing overall factors (sn, so) multiplying contributions from the normal and oscillating states. Default is (1,-1).
tp (int or None) – When not None, the correlator is periodic with period tp when tp>0, or anti-periodic with period -tp when tp<0. Setting tp=None implies that the correlator is neither periodic nor anti-periodic. Default is None.
tmin (int) – Only G(t) with t >= tmin are used. Default value is 6.
svdcut (float or None) – SVD cut used in the weighted average of results from different times. (See the corrfitter.CorrFitter documentation for a discussion of SVD cuts.) Default is 1e-6.
osc (bool) – Set osc=True if the lowest-lying state is an oscillating state. Default is False.

Note that specifying a single gvar.GVar g (as opposed to a tuple) for any of parameters ampl, dE, or E is equivalent to specifying the tuple (g, None) when osc=False, or the tuple (None, g) when osc=True. A similar rule applies to parameter s.

corrfitter.fastfit objects have the following attributes:

E¶: Energy of the lowest-lying state (gvar.GVar).

ampl¶: Amplitude of the lowest-lying state (gvar.GVar).

Both E and ampl are obtained by averaging results calculated for each time larger than tmin. These are averaged to produce a final result. The consistency among results from different times is measured by the chi-squared of the average. Each of E and ampl has the following extra attributes:

chi2¶: chi-squared for the weighted average.

dof¶: The effective number of degrees of freedom in the weighted average.

Q¶: The probability that the chi-squared could have been larger, by chance, assuming that the data are all Gaussain and consistent with each other. Values smaller than 0.05 or 0.1 suggest inconsistency. (Also called the p-factor.)

An easy way to inspect these attributes is to print the fit object fit using print(fit), which lists the values of the energy and amplitude, the chi2/dof for each of these, the number of degrees of freedom, and the Q for each.

`corrfitter` - Least-Squares Fit to Correlators¶

Correlator Model Objects¶

`corrfitter.CorrFitter` Objects¶

`corrfitter.EigenBasis` Objects¶

Fast Fit Objects¶

Table of Contents

Previous topic

This Page

corrfitter - Least-Squares Fit to Correlators¶

Correlator Model Objects¶

corrfitter.CorrFitter Objects¶

corrfitter.EigenBasis Objects¶

Fast Fit Objects¶

`corrfitter` - Least-Squares Fit to Correlators¶

`corrfitter.CorrFitter` Objects¶

`corrfitter.EigenBasis` Objects¶