Supervised, Fully-connected, strict-hierarchy, explicit Predictive Coding with precision weighting of prediction errors
Supervised predictive coding with analytic expressions of the gradients of the energy with respect to representations and learnable parameters, including precision weighting of prediction errors.
- precision_modulated_supervised_explicit_pc.infer(weights, precisions, data, ir=0.05, T=40, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False, target_shape=None, noise=0.0, forward_pass_initialize=True, initialization_bias=<tf.Tensor: shape=(), dtype=float32, numpy=0.0>)
Implements the following logic:
Initialize representations do T times e = r - W * r r += ir * (-P * e + tranpose(W) * P * e) return r
- Parameters
weights (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using
tf_utils.mlp()
precisions – list of precision matrices, can be generated e.g. using
tf_utils.mlp()
data (3d tf.Tensor of float32) – inuput data batch
ir (float, optional) – inference rate, defaults to 0.05
T (int, optional) – number of inference steps, defaults to 40
f (function, optional) – activation function, defaults to tf.nn.relu
df (function, optional) – derivate of the activation function, defaults to relu_derivate
predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False
target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None
noise (float, optional) – standard deviation of an eventual noise mask in the sensory layer, defaults to 0 (no noise mask)
forward_pass_initialize (bool, optional) – controls weither we initialize the hidden and top representations to the predictions or zero for models with predictions flowing upwards, defaults to True (intiialize to predictions)
initialization_bias (float) – If forward_pass_initialize is False, controls the value at which representations are initialized, defaults to 0.
- Returns
latent representations
- Return type
list of 3d tf.Tensor of float32
- precision_modulated_supervised_explicit_pc.learn(weights, precisions, data, target, ir=0.05, lr=0.001, pr=0.001, T=40, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False, diagonal=False, noise=0.0, learn_precision_indices=None, gamma=<tf.Tensor: shape=(), dtype=float32, numpy=0.05>)
Implements the following logic:
Initialize representations do T times e = r - W * r r += ir * (- P * e + tranpose(W) * P * e) W += lr * (P * e * tranpose(r)) P += pr * (I - P * e * transpose(e))
- Parameters
weights (list of 2d variable tf.Tensor of float32) – list of weight matrices, can be generated e.g. using
tf_utils.mlp()
precisions – list of precision matrices, can be generated e.g. using
tf_utils.mlp()
data (3d tf.Tensor of float32) – inuput data batch
target (3d tf.Tensor of float32) – output target batch
ir (float, optional) – inference rate, defaults to 0.05
lr (float, optional) – learning rate for weights, defaults to 0.001
pr (float, optional) – learning rate for precision, defaults to 0.001
T (int, optional) – number of inference steps, defaults to 40
f (function, optional) – activation function, defaults to tf.nn.relu
df (function, optional) – derivate of the activation function, defaults to tf_utils.relu_derivate
predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False
diagonal (bool, optional) – controls weither we use a diagonal approximation of the precision, defaults to False
noise (float, optional) – standard deviation of an eventual noise mask in the sensory layer, defaults to 0 (no noise mask)
learn_precision_indices (list of int, optional) – list of indices of layers in which we want to learn precision matrices, defaults to None (all layers)