Supervised, Fully-connected, strict-hierarchy, explicit Predictive Coding
Supervised predictive coding with analytic expressions of the gradients of the energy with respect to representations and learnable parameters.
- supervised_fullyconnected_stricthierarchy_explicit_pc.generate(weights, target, ir=0.05, T=40, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False)
Implements the following logic:
Initialize representations and clamp representations in the top layer do T times e = r - W * r r += ir * (-e + tranpose(W) * e) return r
- Parameters
weights (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using
tf_utils.mlp()
target (3d tf.Tensor of float32) – inuput data batch (one-hot encoded labels for generation)
ir (float, optional) – inference rate, defaults to 0.05
T (int, optional) – number of inference steps, defaults to 40
f (function, optional) – activation function, defaults to tf.nn.relu
df (function, optional) – derivate of the activation function, defaults to relu_derivate
predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False
- Returns
latent representations
- Return type
list of 3d tf.Tensor of float32
- supervised_fullyconnected_stricthierarchy_explicit_pc.infer(weights, data, ir=0.05, T=40, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False, target_shape=None)
Implements the following logic:
Initialize representations and clamp representations in the sensory layer do T times e = r - W * r r += ir * (-e + tranpose(W) * e) return r
- Parameters
weights (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using
tf_utils.mlp()
data (3d tf.Tensor of float32) – inuput data batch
ir (float, optional) – inference rate, defaults to 0.05
T (int, optional) – number of inference steps, defaults to 40
f (function, optional) – activation function, defaults to tf.nn.relu
df (function, optional) – derivate of the activation function, defaults to relu_derivate
predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False
target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None
- Returns
latent representations
- Return type
list of 3d tf.Tensor of float32
- supervised_fullyconnected_stricthierarchy_explicit_pc.learn(weights, data, target, ir=0.05, lr=0.001, T=40, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False)
Implements the following logic:
Initialize representations do T times e = r - W * r r += ir * (-e + tranpose(W) * e) W += lr * (e * tranpose(r))
- Parameters
weights (list of 2d variable tf.Tensor of float32) – list of weight matrices, can be generated e.g. using
tf_utils.mlp()
data (3d tf.Tensor of float32) – inuput data batch
target (3d tf.Tensor of float32) – output target batch
ir (float, optional) – inference rate, defaults to 0.05
lr (float, optional) – learning rate, defaults to 0.001
T (int, optional) – number of inference steps, defaults to 40
f (function, optional) – activation function, defaults to tf.nn.relu
df (function, optional) – derivate of the activation function, defaults to tf_utils.relu_derivate
predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False