Supervised Autodiff Predictive Coding with precision weighting of prediction errors

Supervised predictive coding with implicit gradients using tensorflow’s autodifferentiation of the energy with respect to representations and learnable parameters, including precision weighting of prediction errors.

precision_modulated_supervised_autodiff_pc.infer(model, data, ir=0.05, T=40, predictions_flow_upward=False, target_shape=None)

Implements the following logic:

Initialize representations
do T times
    E = 0.5 * norm(r - model(r)) ^ 2 + log |P|
    r -= ir * dE/dr
return r
Parameters
  • model (list of tf_utils.PrecisionModulatedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • ir (float, optional) – inference rate, defaults to 0.05

  • T (int, optional) – number of inference steps, defaults to 40

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

Returns

latent representations

Return type

list of 3d tf.Tensor of float32

precision_modulated_supervised_autodiff_pc.learn(model, data, target, ir=0.05, lr=0.001, pr=0.001, T=40, predictions_flow_upward=False, gamma=<tf.Tensor: shape=(), dtype=float32, numpy=0.5>)

Implements the following logic:

Initialize representations
do T times
    E = 0.5 * norm(r - model(r)) ^ 2 + log |P|
    r -= ir * dE/dr
W -= lr * dE/dW
P -= pr * dE/dP
Parameters
  • model (list of tf_utils.PrecisionModulatedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • target (3d tf.Tensor of float32) – output target batch

  • ir (float, optional) – inference rate, defaults to 0.05

  • lr (float, optional) – learning rate, defaults to 0.001

  • pr (float, optional) – learning rate for precision, defaults to 0.001

  • T (int, optional) – number of inference steps, defaults to 40

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False