Supervised Autodiff Predictive Coding with precision weighting of prediction errors
Supervised predictive coding with implicit gradients using tensorflow’s autodifferentiation of the energy with respect to representations and learnable parameters, including precision weighting of prediction errors.
- precision_modulated_supervised_autodiff_pc.infer(model, data, ir=0.05, T=40, predictions_flow_upward=False, target_shape=None)
Implements the following logic:
Initialize representations do T times E = 0.5 * norm(r - model(r)) ^ 2 + log |P| r -= ir * dE/dr return r
- Parameters
model (list of
tf_utils.PrecisionModulatedDense
) – description of a sequential network by a list of layers, can be generated e.g. usingtf_utils.mlp()
data (3d tf.Tensor of float32) – inuput data batch
ir (float, optional) – inference rate, defaults to 0.05
T (int, optional) – number of inference steps, defaults to 40
predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False
target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None
- Returns
latent representations
- Return type
list of 3d tf.Tensor of float32
- precision_modulated_supervised_autodiff_pc.learn(model, data, target, ir=0.05, lr=0.001, pr=0.001, T=40, predictions_flow_upward=False, gamma=<tf.Tensor: shape=(), dtype=float32, numpy=0.5>)
Implements the following logic:
Initialize representations do T times E = 0.5 * norm(r - model(r)) ^ 2 + log |P| r -= ir * dE/dr W -= lr * dE/dW P -= pr * dE/dP
- Parameters
model (list of
tf_utils.PrecisionModulatedDense
) – description of a sequential network by a list of layers, can be generated e.g. usingtf_utils.mlp()
data (3d tf.Tensor of float32) – inuput data batch
target (3d tf.Tensor of float32) – output target batch
ir (float, optional) – inference rate, defaults to 0.05
lr (float, optional) – learning rate, defaults to 0.001
pr (float, optional) – learning rate for precision, defaults to 0.001
T (int, optional) – number of inference steps, defaults to 40
predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False