Supervised, Fully-connected, strict-hierarchy, explicit Predictive Coding

Supervised predictive coding with analytic expressions of the gradients of the energy with respect to representations and learnable parameters.

supervised_fullyconnected_stricthierarchy_explicit_pc.generate(weights, data, ir=0.025, T=200, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False)

Implements the following logic:

Initialize representations
do T times
    e = r - W * r
    r += ir * (-e + tranpose(W) * e)
return r
Parameters
  • weights (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • ir (float, optional) – inference rate, defaults to 0.025

  • T (int, optional) – number of inference steps, defaults to 200

  • f (function, optional) – activation function, defaults to tf.nn.relu

  • df (function, optional) – derivate of the activation function, defaults to relu_derivate

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

Returns

latent representations

Return type

list of 3d tf.Tensor of float32

supervised_fullyconnected_stricthierarchy_explicit_pc.infer(weights, data, ir=0.025, T=200, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False, target_shape=None)

Implements the following logic:

Initialize representations
do T times
    e = r - W * r
    r += ir * (-e + tranpose(W) * e)
return r
Parameters
  • weights (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • ir (float, optional) – inference rate, defaults to 0.025

  • T (int, optional) – number of inference steps, defaults to 200

  • f (function, optional) – activation function, defaults to tf.nn.relu

  • df (function, optional) – derivate of the activation function, defaults to relu_derivate

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

Returns

latent representations

Return type

list of 3d tf.Tensor of float32

supervised_fullyconnected_stricthierarchy_explicit_pc.learn(weights, data, target, ir=0.1, lr=0.001, T=20, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False)

Implements the following logic:

Initialize representations
do T times
    e = r - W * r
    r += ir * (-e + tranpose(W) * e)
W += lr * (e * tranpose(r))
Parameters
  • weights (list of 2d variable tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • target (3d tf.Tensor of float32) – output target batch

  • ir (float, optional) – inference rate, defaults to 0.1

  • lr (float, optional) – learning rate, defaults to 0.001

  • T (int, optional) – number of inference steps, defaults to 20

  • f (function, optional) – activation function, defaults to tf.nn.relu

  • df (function, optional) – derivate of the activation function, defaults to tf_utils.relu_derivate

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False