entropy

bcselector.information_theory.basic_approximations.entropy(vector, base=None)[source]

This estimator computes the entropy of the empirical probability distribution.

Parameters
  • vector (list or np.array) – Vector of which entropy is calculated.

  • base (int or float (default=np.e)) – Base of the logarithm in entropy approximation

Returns

vector_entropy – Approximated entropy

Return type

float

Examples

>>> from bcselector.information_theory.basic_approximations import entropy
>>> foo = [1,4,1,2,5,6,3]
>>> entropy(foo)

conditional_entropy

bcselector.information_theory.basic_approximations.conditional_entropy(vector, condition, base=None)[source]

This estimator computes the conditional entropy of the empirical probability distribution.

Parameters
  • vector (list or np.array) – Vector of which entropy is calculated.

  • condition (list or np.array) – Vector of condition for entropy.

  • base (int or float) – Base of the logarithm in entropy approximation. If None, np.e is selected and entropy is returned in nats.

Returns

vector_entropy – Approximated entropy.

Return type

float

mutual_information

bcselector.information_theory.basic_approximations.mutual_information(vector_1, vector_2, base=None)[source]

This estimator computes the mutual information of two vectors with method of the empirical probability distribution.

Parameters
  • vector_1 (list or np.array) – Vector of one variable.

  • vector_2 (list or np.array) – Vector of one variable.

  • base (int or float) – Base of the logarithm in entropy approximation. If None, np.e is selected and entropy is returned in nats.

Returns

variables_mutual_information – Approximated mutual information between variables.

Return type

float

conditional_mutual_information

bcselector.information_theory.basic_approximations.conditional_mutual_information(vector_1, vector_2, condition, base=None)[source]

This estimator computes the conditional mutual information of two vectors and condition vector with method of the empirical probability distribution.

Parameters
  • vector_1 (list or np.array) – Vector of one variable.

  • vector_2 (list or np.array) – Vector of one variable.

  • condition (list or np.array) – Vector of condition for mutual information.

  • base (int or float) – Base of the logarithm in entropy approximation. If None, np.e is selected and entropy is returned in nats.

Returns

variables_conditional_mutual_information – Approximated conditional mutual information between variables.

Return type

float