Precision weighting utilities
- precisions_utils.precision_modulated_energy(model, r, theta=[], predictions_flow_upward=False, gamma=<tf.Tensor: shape=(), dtype=float32, numpy=0.5>)
Energy (total squared L2 norm of precision-weighted errors + regularizer) computation and autodifferentiation with respect to representations and learnable parameters.
- Parameters
model (list of
PrecisionModulatedDense
) – description of a sequential network by a list of layers, can be generated e.g. usingtf_utils.mlp()
r (list of 3d tf.Tensor of float32) – representations
theta (list of variable tf.Tensor of float32, optional) – learnable parameters, defaults to []
predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False
gamma (float32, optional) – Regularization scalar, defaults to .5
- Returns
energy, autodiff gradient tape
- Return type
float32, tf.GradientTape
- precisions_utils.precision_modulated_inference_step_backward_predictions(e, r, w, P, ir, f, df, update_last=True, sensory_noise=0.0)
Representations update using stochastic gradient descent with analytic expressions of the gradients of energy wrt representations, only applicable to an unbiased MLP with reversed flow (predictions come from higher layer)
- Parameters
e (list of 3d tf.Tensor of float32) – prediction errors
r (list of 3d tf.Tensor of float32) – representations
w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using
tf_utils.mlp()
P (list of 2d tf.Tensor of float32) – list of precision matrices
ir (float32) – inference rate
f (function) – activation function
df (function) – derivate of the activation function
update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True
- precisions_utils.precision_modulated_inference_step_forward_predictions(e, r, w, P, ir, f, df, update_last=True, sensory_noise=0.0)
Representations update using stochastic gradient descent with analytic expressions of the gradients of energy wrt representations, only applicable to an unbiased MLP (predictions come from lower layer)
- Parameters
e (list of 3d tf.Tensor of float32) – prediction errors
r (list of 3d tf.Tensor of float32) – representations
w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using
tf_utils.mlp()
P (list of 2d tf.Tensor of float32) – list of precision matrices
ir (float32) – inference rate
f (float32) – activation function
df (function) – derivate of the activation function
update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True
- precisions_utils.precision_modulated_weight_update_backward_predictions(w, e, r, P, lr, f)
Weight update using stochastic gradient descent with analytic expressions of the gradients of energy wrt weights, only applicable to an unbiased MLP with reversed flow (predictions come from higher layer)
- Parameters
w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using
tf_utils.mlp()
e (list of 3d tf.Tensor of float32) – prediction errors
r (list of 3d tf.Tensor of float32) – representations
P (list of 2d tf.Tensor of float32) – list of precision matrices
lr (float32) – learning rate
f (function) – activation function
- precisions_utils.precision_modulated_weight_update_forward_predictions(w, e, r, P, lr, f)
Weight update using stochastic gradient descent with analytic expressions of the gradients of energy wrt weights, only applicable to an unbiased MLP (predictions come from lower layer)
- Parameters
w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using
tf_utils.mlp()
e (list of 3d tf.Tensor of float32) – prediction errors
r (list of 3d tf.Tensor of float32) – representations
P (list of 2d tf.Tensor of float32) – list of precision matrices
lr (float32) – learning rate
f (function) – activation function
- precisions_utils.precisions_update_backward_predictions(e, P, lr, diagonal=False, update_layer_indices=None, gamma=<tf.Tensor: shape=(), dtype=float32, numpy=0.05>)
Precision update using stochastic gradient descent with analytic expressions of the gradients of energy wrt precision, only applicable to an unbiased MLP (predictions come from higher layer)
- Parameters
e (list of 3d tf.Tensor of float32) – prediction errors
P (list of 2d tf.Tensor of float32) – list of precision matrices
lr (float32) – learning rate
diagonal (bool) – controls weither we use a diagonal approximation of the precision
only_update_layer_indices –
- precisions_utils.precisions_update_forward_predictions(e, P, lr, diagonal=False, update_layer_indices=None, gamma=<tf.Tensor: shape=(), dtype=float32, numpy=0.05>)
Precision update using stochastic gradient descent with analytic expressions of the gradients of energy wrt precision, only applicable to an unbiased MLP (predictions come from lower layer)
- Parameters
e (list of 3d tf.Tensor of float32) – prediction errors
P (list of 2d tf.Tensor of float32) – list of precision matrices
lr (float32) – learning rate
diagonal (bool) – controls weither we use a diagonal approximation of the precision
- precisions_utils.set_noisy_sensory_horizontal_window(representations, noise, clipmin=- 10.0, clipmax=10.0, first_pixel=196, last_pixel=392)
Obscure a part of the sensory layer by clamping pixels that are not between first_pixel and last_pixel to a random gaussian variable of standard deviation noise
- Parameters
representations (list of 3d tf.Tensor of float32) – representations
noise (float) – standard deviation of the noise mask
clipmin (float, optional) – minimum value of representations, defaults to -10
clipmax (float, optional) – maximum value of representations, defaults to 10
first_pixel (int, optional) – first index of unobstructed pixels, defaults to 196
last_pixel (int, optional) – last index of unobstructed pixels, defaults to 392