predflow package

Submodules

predflow.datasets_utils module

Dataset loading utilities, mainly relying on the tensorflow_datasets module.

Datasets:

  • MNIST

predflow.datasets_utils.load_mnist(batch_size=50)

Load the MNIST train and test dataset

Parameters

batch_size (int, optional) – minibatch size, defaults to 50

Returns

mnist dataset

Return type

tf.Dataset

predflow.pc_utils module

Utilities for predictive coding specifically.

  • Energy computation and autodifferentiated gradients of energy wrt to representations and learnable parameters

  • Explicit computation of gradients expressed with prediction errors in the nonlinear fully-connected strictly hierarchical case

  • Stochastic gradient descent steps on representations and learnable parameters

  • Representations initializers (first pass)

predflow.pc_utils.backward_initialize_representations(model, target, data=None)

Initialize representations with a backward sweep through the model

Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • target (3d tf.Tensor of float32) – output target batch

  • data (3d tf.Tensor of float32, optional) – inuput data batch, defaults to None

Returns

representations

Return type

list of 3d tf.Tensor of float32

predflow.pc_utils.energy_and_error(model, r, theta=[], predictions_flow_upward=False)

Energy (total squared L2 norm of errors) computation and autodifferentiation with respect to representations and learnable parameters

Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • r (list of 3d tf.Tensor of float32) – representations

  • theta (list of variable tf.Tensor of float32, optional) – learnable parameters, defaults to []

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

Returns

energy, autodiff gradient tape

Return type

float32, tf.GradientTape

predflow.pc_utils.forward_initialize_representations(model, data, target=None)

Initialize representations with a forward sweep through the model

Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • target (3d tf.Tensor of float32, optional) – output target batch, defaults to None

Returns

representations

Return type

list of 3d tf.Tensor of float32

predflow.pc_utils.forward_initialize_representations_explicit(w, f, data, target=None)

Initialize representations with a forward sweep through the model explictly defined as w and f

Parameters
  • w (list of 2d tf.Tensor of float32) – weight matrix

  • f (function) – activation function

  • data (3d tf.Tensor of float32) – inuput data batch

  • target (3d tf.Tensor of float32, optional) – output target batch, defaults to None

Returns

representations

Return type

list of 3d tf.Tensor of float32

predflow.pc_utils.inference_SGD_step(r, ir, g, update_last=True)

Stochastic gradient descent step on represnetations (inference) using autodifferentiated gradients

Parameters
  • r (list of 3d tf.Tensor of float32) – representations

  • ir (float32) – inference rate

  • g (tf.GradientTape) – autodiff gradient tape

  • update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True

predflow.pc_utils.inference_step_backward_predictions(e, r, w, ir, f, df, update_last=True, update_first=False)

Representations update using stochastic gradient descent with analytic expressions of the gradients of energy wrt representations, only applicable to an unbiased MLP with reversed flow (predictions come from higher layer)

Parameters
  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • ir (float32) – inference rate

  • f (function) – activation function

  • df (function) – derivate of the activation function

  • update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True

  • update_first (bool, optional) – controls weither representations in the first layer are updated, defaults to False

predflow.pc_utils.inference_step_forward_predictions(e, r, w, ir, f, df, update_last=True)

Representations update using stochastic gradient descent with analytic expressions of the gradients of energy wrt representations, only applicable to an unbiased MLP (predictions come from lower layer)

Parameters
  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • ir (float32) – inference rate

  • f (float32) – activation function

  • df (function) – derivate of the activation function

  • update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True

predflow.pc_utils.parameters_SGD_step(theta, lr, g)

Stochastic gradient descent step on learnable parameters (learning) using autodifferentiated gradients

Parameters
  • theta (list of variable tf.Tensor of float32) – learnable parameters

  • lr (float32) – learning rate

  • g (tf.GradientTape) – autodiff gradient tape

predflow.pc_utils.random_initialize_representations(model, data, stddev=0.001, predictions_flow_upward=False, target_shape=None)

Randomly initialize latent representations

Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • stddev (float, optional) – standard deviation of the normal initialization, defaults to 0.001

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

Returns

representations

Return type

list of 3d tf.Tensor of float32

predflow.pc_utils.weight_update_backward_predictions(w, e, r, lr, f)

Weight update using stochastic gradient descent with analytic expressions of the gradients of energy wrt weights, only applicable to an unbiased MLP with reversed flow (predictions come from higher layer)

Parameters
  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • lr (float32) – learning rate

  • f (function) – activation function

predflow.pc_utils.weight_update_forward_predictions(w, e, r, lr, f)

Weight update using stochastic gradient descent with analytic expressions of the gradients of energy wrt weights, only applicable to an unbiased MLP (predictions come from lower layer)

Parameters
  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • lr (float32) – learning rate

  • f (function) – activation function

predflow.pc_utils.zero_initialize_representations(model, data, predictions_flow_upward=False, target_shape=None, bias=<tf.Tensor: shape=(), dtype=float32, numpy=0.0>)

Initialize representations at zero (or a constant)

Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

  • bias (float32, optional) – initialize representation with bias rather than 0., defaults to tf.constant(0.)

Returns

representations

Return type

list of 3d tf.Tensor of float32

predflow.precision_modulated_supervised_autodiff_pc module

Supervised predictive coding with implicit gradients using tensorflow’s autodifferentiation of the energy with respect to representations and learnable parameters.

predflow.precision_modulated_supervised_autodiff_pc.infer(model, data, ir=0.025, T=200, predictions_flow_upward=False, target_shape=None)

Implements the following logic:

Initialize representations
do T times
    E = 0.5 * norm(r - model(r)) ^ 2 + log |P|
    r -= ir * dE/dr
return r
Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • ir (float, optional) – inference rate, defaults to 0.025

  • T (int, optional) – number of inference steps, defaults to 200

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

Returns

latent representations

Return type

list of 3d tf.Tensor of float32

predflow.precision_modulated_supervised_autodiff_pc.learn(model, data, target, ir=0.1, lr=0.001, pr=0.001, T=40, predictions_flow_upward=False)

Implements the following logic:

Initialize representations
do T times
    E = 0.5 * norm(r - model(r)) ^ 2 + log |P|
    r -= ir * dE/dr
W -= lr * dE/dW
P -= pr * dE/dP
Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • target (3d tf.Tensor of float32) – output target batch

  • ir (float, optional) – inference rate, defaults to 0.1

  • lr (float, optional) – learning rate, defaults to 0.001

  • pr (float, optional) – learning rate for precision, defaults to 0.001

  • T (int, optional) – number of inference steps, defaults to 40

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

predflow.precision_modulated_supervised_explicit_pc module

Supervised predictive coding with analytic expressions of the gradients of the energy with respect to representations and learnable parameters.

predflow.precision_modulated_supervised_explicit_pc.infer(weights, precisions, data, ir=0.025, T=200, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False, target_shape=None, noise=0.0, initialize=True, defective_error_ratio=0.0)

Implements the following logic:

Initialize representations
do T times
    e = r - W * r
    r += ir * (-P * e + tranpose(W) * P * e)
return r
Parameters
  • weights (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • precisions – list of precision matrices, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • ir (float, optional) – inference rate, defaults to 0.025

  • T (int, optional) – number of inference steps, defaults to 200

  • f (function, optional) – activation function, defaults to tf.nn.relu

  • df (function, optional) – derivate of the activation function, defaults to relu_derivate

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

Returns

latent representations

Return type

list of 3d tf.Tensor of float32

predflow.precision_modulated_supervised_explicit_pc.learn(weights, precisions, data, target, ir=0.1, lr=0.001, pr=0.001, T=20, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False, diagonal=False, noise=0.0, clamp_w=None, defective_error_ratio=0.0)

Implements the following logic:

Initialize representations
do T times
    e = r - W * r
    r += ir * (- P * e + tranpose(W) * P * e)
W += lr * (P * e * tranpose(r))
P += pr * (I - P * e * transpose(e))
Parameters
  • weights (list of 2d variable tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • precisions – list of precision matrices, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • target (3d tf.Tensor of float32) – output target batch

  • ir (float, optional) – inference rate, defaults to 0.1

  • lr (float, optional) – learning rate for weights, defaults to 0.001

  • pr (float, optional) – learning rate for precision, defaults to 0.001

  • T (int, optional) – number of inference steps, defaults to 20

  • f (function, optional) – activation function, defaults to tf.nn.relu

  • df (function, optional) – derivate of the activation function, defaults to tf_utils.relu_derivate

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

predflow.precisions_utils module

predflow.precisions_utils.precision_modulated_energy(model, r, theta=[], predictions_flow_upward=False, gamma=<tf.Tensor: shape=(), dtype=float32, numpy=0.5>)

Energy (total squared L2 norm of precision-weighted errors + regularizer) computation and autodifferentiation with respect to representations and learnable parameters

Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • r (list of 3d tf.Tensor of float32) – representations

  • theta (list of variable tf.Tensor of float32, optional) – learnable parameters, defaults to []

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • gamma (float32, optional) – Regularization scalar, defaults to .5

Returns

energy, autodiff gradient tape

Return type

float32, tf.GradientTape

predflow.precisions_utils.precision_modulated_inference_step_backward_predictions(e, r, w, P, ir, f, df, update_last=True, sensory_noise=0.0)

Representations update using stochastic gradient descent with analytic expressions of the gradients of energy wrt representations, only applicable to an unbiased MLP with reversed flow (predictions come from higher layer)

Parameters
  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • ir (float32) – inference rate

  • f (function) – activation function

  • df (function) – derivate of the activation function

  • update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True

predflow.precisions_utils.precision_modulated_inference_step_forward_predictions(e, r, w, P, ir, f, df, update_last=True)

Representations update using stochastic gradient descent with analytic expressions of the gradients of energy wrt representations, only applicable to an unbiased MLP (predictions come from lower layer)

Parameters
  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • ir (float32) – inference rate

  • f (float32) – activation function

  • df (function) – derivate of the activation function

  • update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True

predflow.precisions_utils.precision_modulated_weight_update_backward_predictions(w, e, r, P, lr, f)

Weight update using stochastic gradient descent with analytic expressions of the gradients of energy wrt weights, only applicable to an unbiased MLP with reversed flow (predictions come from higher layer)

Parameters
  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • lr (float32) – learning rate

  • f (function) – activation function

predflow.precisions_utils.precision_modulated_weight_update_forward_predictions(w, e, r, P, lr, f)

Weight update using stochastic gradient descent with analytic expressions of the gradients of energy wrt weights, only applicable to an unbiased MLP (predictions come from lower layer)

Parameters
  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • lr (float32) – learning rate

  • f (function) – activation function

predflow.precisions_utils.precisions_update_backward_predictions(e, P, lr, diagonal=False, only_update_layer_indices=None)

Precision update using stochastic gradient descent with analytic expressions of the gradients of energy wrt precision, only applicable to an unbiased MLP (predictions come from higher layer)

Parameters
  • e (list of 3d tf.Tensor of float32) – prediction errors

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • lr (float32) – learning rate

  • diagonal (bool) – controls weither we use a diagonal approximation of the precision

  • only_update_layer_indices

predflow.precisions_utils.precisions_update_forward_predictions(e, P, lr, diagonal=False)

Precision update using stochastic gradient descent with analytic expressions of the gradients of energy wrt precision, only applicable to an unbiased MLP (predictions come from lower layer)

Parameters
  • e (list of 3d tf.Tensor of float32) – prediction errors

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • lr (float32) – learning rate

predflow.precisions_utils.set_noisy_sensory(representations, data, noise)

predflow.supervised_autodiff_pc module

Supervised predictive coding with implicit gradients using tensorflow’s autodifferentiation of the energy with respect to representations and learnable parameters.

predflow.supervised_autodiff_pc.infer(model, data, ir=0.025, T=200, predictions_flow_upward=False, target_shape=None)

Implements the following logic:

Initialize representations
do T times
    E = 0.5 * norm(r - model(r)) ^ 2
    r -= ir * dE/dr
return r
Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • ir (float, optional) – inference rate, defaults to 0.025

  • T (int, optional) – number of inference steps, defaults to 200

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

Returns

latent representations

Return type

list of 3d tf.Tensor of float32

predflow.supervised_autodiff_pc.learn(model, data, target, ir=0.1, lr=0.001, T=40, predictions_flow_upward=False)

Implements the following logic:

Initialize representations
do T times
    E = 0.5 * norm(r - model(r)) ^ 2
    r -= ir * dE/dr
W -= lr * dE/dW
Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • target (3d tf.Tensor of float32) – output target batch

  • ir (float, optional) – inference rate, defaults to 0.1

  • lr (float, optional) – learning rate, defaults to 0.001

  • T (int, optional) – number of inference steps, defaults to 40

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

predflow.supervised_fullyconnected_stricthierarchy_explicit_pc module

Supervised predictive coding with analytic expressions of the gradients of the energy with respect to representations and learnable parameters.

predflow.supervised_fullyconnected_stricthierarchy_explicit_pc.generate(weights, data, ir=0.025, T=40, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False)

Implements the following logic:

Initialize representations and clamp representations in the top layer
do T times
    e = r - W * r
    r += ir * (-e + tranpose(W) * e)
return r
Parameters
  • weights (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • ir (float, optional) – inference rate, defaults to 0.025

  • T (int, optional) – number of inference steps, defaults to 200

  • f (function, optional) – activation function, defaults to tf.nn.relu

  • df (function, optional) – derivate of the activation function, defaults to relu_derivate

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

Returns

latent representations

Return type

list of 3d tf.Tensor of float32

predflow.supervised_fullyconnected_stricthierarchy_explicit_pc.infer(weights, data, ir=0.025, T=10, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False, target_shape=None)

Implements the following logic:

Initialize representations and clamp representations in the sensory layer
do T times
    e = r - W * r
    r += ir * (-e + tranpose(W) * e)
return r
Parameters
  • weights (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • ir (float, optional) – inference rate, defaults to 0.025

  • T (int, optional) – number of inference steps, defaults to 200

  • f (function, optional) – activation function, defaults to tf.nn.relu

  • df (function, optional) – derivate of the activation function, defaults to relu_derivate

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

Returns

latent representations

Return type

list of 3d tf.Tensor of float32

predflow.supervised_fullyconnected_stricthierarchy_explicit_pc.learn(weights, data, target, ir=0.1, lr=0.001, T=20, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False)

Implements the following logic:

Initialize representations
do T times
    e = r - W * r
    r += ir * (-e + tranpose(W) * e)
W += lr * (e * tranpose(r))
Parameters
  • weights (list of 2d variable tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • target (3d tf.Tensor of float32) – output target batch

  • ir (float, optional) – inference rate, defaults to 0.1

  • lr (float, optional) – learning rate, defaults to 0.001

  • T (int, optional) – number of inference steps, defaults to 20

  • f (function, optional) – activation function, defaults to tf.nn.relu

  • df (function, optional) – derivate of the activation function, defaults to tf_utils.relu_derivate

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

predflow.tf_utils module

General tensorflow utilities.

class predflow.tf_utils.BiasedDense(input_dim, output_size, name=None, activation=<function relu>, stddev=0.001)

Bases: tensorflow.python.module.module.Module

Biased dense layer, y = W * f(x) + b

Parameters
  • input_dim (int) – dimension of the layer’s input

  • output_size (int) – dimension of the layer’s output

  • name (str, optional) – custom name for the layer, defaults to None

  • activation (function, optional) – activation function, defaults to tf.nn.relu

  • stddev (float, optional) – standard deviation of the normal initialization, defaults to .001

class predflow.tf_utils.Dense(input_dim, output_size, name=None, activation=<function relu>, stddev=0.001)

Bases: tensorflow.python.module.module.Module

Unbiased dense layer, y = W * f(x)

Parameters
  • input_dim (int) – dimension of the layer’s input

  • output_size (int) – dimension of the layer’s output

  • name (str, optional) – custom name for the layer, defaults to None

  • activation (function, optional) – activation function, defaults to tf.nn.relu

  • stddev (float, optional) – standard deviation of the normal initialization, defaults to .001

class predflow.tf_utils.PrecisionModulatedDense(input_dim, output_size, name=None, activation=<function relu>, stddev=0.001)

Bases: tensorflow.python.module.module.Module

Unbiased dense layer additionally containing a precision matrix variable

Parameters
  • input_dim (int) – dimension of the layer’s input

  • output_size (int) – dimension of the layer’s output

  • name (str, optional) – custom name for the layer, defaults to None

  • activation (function, optional) – activation function, defaults to tf.nn.relu

  • stddev (float, optional) – standard deviation of the normal initialization, defaults to .001

predflow.tf_utils.load_tensorboard_graph(logdir, func, args, name, step=0, kwargs={})

Log the tensorboard graph trace of func(*args, **kwargs)

Parameters
  • logdir (str) – log folder path

  • func (function) – function to analyze

  • args (list) – arguments of func

  • name (str) – name of the tensorboard trace

  • step (int, optional) – tensorboard step, defaults to 0

  • kwargs (dict, optional) – kwargs of func, defaults to {}

predflow.tf_utils.mlp(*args, biased=False, reversed_flow=False, activation=<function relu>, stddev=0.01, only_return_variables=False, precision_modulated=False)

Create a multi-layer perceptron

Parameters
  • args – sequence of int representing layer sizes

  • biased (bool, optional) – controls weither we use bias in layers, defaults to False

  • reversed_flow (bool, optional) – controls weither we reverse the flow of activation (default is bottom-up), defaults to False

  • activation (function, optional) – activation function, defaults to tf.nn.relu

  • stddev (float, optional) – standard deviation of the normal initialization, defaults to 0.01

  • only_return_weights (bool, optional) – controls weither we return a list of tf.Module or 2d variable weight matrices, defaults to False

  • precision_modulated (bool, optional) – controls weither the model includes precision weighting of prediction errors, defaults to False

Returns

model

Return type

list of tf_utils.Dense, tf_utils.BiasedDense, tf_utils.PrecisionModulatedDense or 2d variable tf.Tensor of float32

predflow.tf_utils.one_hot_pred_accuracy(p, t, axis=1)

Compute the accuracy of a prediction p with respect to target t as the proportion of time argmax(p) == argmax(t)

Parameters
  • p (3d tf.Tensor) – network prediction

  • t (3d tf.Tensor) – ground truth target

  • axis (int, optional) – argmax axis, defaults to 1

Returns

accuracy

Return type

float32

predflow.tf_utils.reduced_batched_outer_product(x, y)

Compute the outer product of x and y summed over batch dimesion

Parameters
  • x (3d tf.Tensor) – first tensor

  • y (3d tf.Tensor) – second tensor

Returns

outer product summed over batch dimesion

Return type

2d tf.Tensor

predflow.tf_utils.relu_derivate(x)

Derivate of the ReLU activation function

Parameters

x (tf.Tensor) – input

Returns

output

Return type

tf.Tensor

Module contents