Modules
astromer
- ASTROMER.core.astromer.get_ASTROMER(num_layers=2, d_model=200, num_heads=2, dff=256, base=10000, dropout=0.1, use_leak=False, no_train=True, maxlen=100, batch_size=None)[source]
attention
- class ASTROMER.core.attention.MultiHeadAttention(*args, **kwargs)[source]
Bases:
Layer
- call(x, mask)[source]
This is where the layer’s logic lives.
The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state, including tf.Variable instances and nested Layer instances,
in __init__(), or in the build() method that is
called automatically before call() executes for the first time.
- Args:
- inputs: Input tensor, or dict/list/tuple of input tensors.
The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero
arguments, and inputs cannot be provided via the default value of a keyword argument.
NumPy array or Python scalar values in inputs get cast as tensors.
Keras mask metadata is only collected from inputs.
Layers are built (build(input_shape) method) using shape info from inputs only.
input_spec compatibility is only checked against inputs.
Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.
The SavedModel input specification is generated using inputs only.
Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.
- *args: Additional positional arguments. May contain tensors, although
this is not recommended, for the reasons above.
- **kwargs: Additional keyword arguments. May contain tensors, although
this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating
whether the call is meant for training or inference.
mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).
- Returns:
A tensor or list/tuple of tensors.
- ASTROMER.core.attention.scaled_dot_product_attention(q, k, v, mask)[source]
Calculate the attention weights. q, k, v must have matching leading dimensions. k, v must have matching penultimate dimension, i.e.: seq_len_k = seq_len_v. The mask has different shapes depending on its type(padding or look ahead) but it must be broadcastable for addition.
- Parameters:
q – query shape == (…, seq_len_q, depth)
k – key shape == (…, seq_len_k, depth)
v – value shape == (…, seq_len_v, depth_v)
mask – Float tensor with shape broadcastable to (…, seq_len_q, seq_len_k). Defaults to None.
- Returns:
output, attention_weights
classifier
- ASTROMER.core.classifier.get_lstm_attention(units, num_classes, weigths, dropout=0.5)[source]
ATT + LSTM + LSTM + FC
- ASTROMER.core.classifier.get_lstm_no_attention(units, num_classes, maxlen, dropout=0.5)[source]
LSTM + LSTM + FC
data
- ASTROMER.core.data.create_dataset(meta_df, source='data/raw_data/macho/MACHO/LCs', target='data/records/macho/', n_jobs=None, subsets_frac=(0.5, 0.25), test_subset=None, max_lcs_per_record=100, **kwargs)[source]
- ASTROMER.core.data.deserialize(sample)[source]
Read a serialized sample and convert it to tensor Context and sequence features should match with the name used when writing. Args:
sample (binary): serialized sample
- Returns:
type: decoded sample
- ASTROMER.core.data.divide_training_subset(frame, train, val, test_meta)[source]
Divide the dataset into train, validation and test subsets. Notice that:
test = 1 - (train + val)
- Args:
frame (Dataframe): Dataframe following the astro-standard format dest (string): Record destination. train (float): train fraction val (float): validation fraction
- Returns:
tuple x3 : (name of subset, subframe with metadata)
- ASTROMER.core.data.get_example(lcid, label, lightcurve)[source]
Create a record example from numpy values.
- Args:
lcid (string): object id label (int): class code lightcurve (numpy array): time, magnitudes and observational error
- Returns:
tensorflow record
- ASTROMER.core.data.load_numpy(samples, ids=None, labels=None, batch_size=1, shuffle=False, sampling=False, max_obs=100, msk_frac=0.0, rnd_frac=0.0, same_frac=0.0, repeat=1)[source]
- ASTROMER.core.data.mask_sample(x, y, i, msk_prob, rnd_prob, same_prob, max_obs)[source]
Pretraining formater
- ASTROMER.core.data.pretraining_records(source, batch_size, max_obs=100, msk_frac=0.2, rnd_frac=0.1, same_frac=0.1, sampling=False, shuffle=False, n_classes=-1, repeat=1)[source]
Pretraining data loader. This method build the ASTROMER input format. ASTROMER format is based on the BERT masking strategy.
- Args:
source (string): Record folder batch_size (int): Batch size no_shuffle (bool): Do not shuffle training and validation dataset max_obs (int): Max. number of observation per serie msk_frac (float): fraction of values to be predicted ([MASK]) rnd_frac (float): fraction of [MASKED] values to replace with random values same_frac (float): fraction of [MASKED] values to replace with true values
- Returns:
Tensorflow Dataset: Iterator withg preprocessed batches
encoder
- class ASTROMER.core.encoder.Encoder(*args, **kwargs)[source]
Bases:
Layer
- call(data, training=False)[source]
This is where the layer’s logic lives.
The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state, including tf.Variable instances and nested Layer instances,
in __init__(), or in the build() method that is
called automatically before call() executes for the first time.
- Args:
- inputs: Input tensor, or dict/list/tuple of input tensors.
The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero
arguments, and inputs cannot be provided via the default value of a keyword argument.
NumPy array or Python scalar values in inputs get cast as tensors.
Keras mask metadata is only collected from inputs.
Layers are built (build(input_shape) method) using shape info from inputs only.
input_spec compatibility is only checked against inputs.
Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.
The SavedModel input specification is generated using inputs only.
Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.
- *args: Additional positional arguments. May contain tensors, although
this is not recommended, for the reasons above.
- **kwargs: Additional keyword arguments. May contain tensors, although
this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating
whether the call is meant for training or inference.
mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).
- Returns:
A tensor or list/tuple of tensors.
- class ASTROMER.core.encoder.EncoderLayer(*args, **kwargs)[source]
Bases:
Layer
- call(x, training, mask)[source]
This is where the layer’s logic lives.
The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state, including tf.Variable instances and nested Layer instances,
in __init__(), or in the build() method that is
called automatically before call() executes for the first time.
- Args:
- inputs: Input tensor, or dict/list/tuple of input tensors.
The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero
arguments, and inputs cannot be provided via the default value of a keyword argument.
NumPy array or Python scalar values in inputs get cast as tensors.
Keras mask metadata is only collected from inputs.
Layers are built (build(input_shape) method) using shape info from inputs only.
input_spec compatibility is only checked against inputs.
Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.
The SavedModel input specification is generated using inputs only.
Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.
- *args: Additional positional arguments. May contain tensors, although
this is not recommended, for the reasons above.
- **kwargs: Additional keyword arguments. May contain tensors, although
this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating
whether the call is meant for training or inference.
mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).
- Returns:
A tensor or list/tuple of tensors.
losses
masking
- ASTROMER.core.masking.get_masked(tensor, frac=0.15)[source]
Add [MASK] values to be predicted Args:
tensor : tensor values frac (float, optional): percentage for masking [MASK]
- Returns:
binary tensor: a time-distributed mask
- ASTROMER.core.masking.get_padding_mask(steps, lengths)[source]
Create mask given a tensor and true length
- ASTROMER.core.masking.set_random(serie_1, mask_1, serie_2, rnd_frac, name='set_random')[source]
Add Random values in serie_1 Note that if serie_2 == serie_1 then it replaces the true value Args:
serie_1: current serie mask_1 : mask containing the [MASKED]-indices from serie_1 serie_2: random values to be placed within serie_1 rnd_frac (float): fraction of [MASKED] to be replaced by random
elements from serie_2
- Returns:
serie_1: serie_1 with random values
metrics
output
- class ASTROMER.core.output.RegLayer(*args, **kwargs)[source]
Bases:
Layer
- call(inputs)[source]
This is where the layer’s logic lives.
The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state, including tf.Variable instances and nested Layer instances,
in __init__(), or in the build() method that is
called automatically before call() executes for the first time.
- Args:
- inputs: Input tensor, or dict/list/tuple of input tensors.
The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero
arguments, and inputs cannot be provided via the default value of a keyword argument.
NumPy array or Python scalar values in inputs get cast as tensors.
Keras mask metadata is only collected from inputs.
Layers are built (build(input_shape) method) using shape info from inputs only.
input_spec compatibility is only checked against inputs.
Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.
The SavedModel input specification is generated using inputs only.
Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.
- *args: Additional positional arguments. May contain tensors, although
this is not recommended, for the reasons above.
- **kwargs: Additional keyword arguments. May contain tensors, although
this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating
whether the call is meant for training or inference.
mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).
- Returns:
A tensor or list/tuple of tensors.
- class ASTROMER.core.output.SauceLayer(*args, **kwargs)[source]
Bases:
Layer
- build(input_shape)[source]
Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call. It is invoked automatically before the first execution of call().
This is typically used to create the weights of Layer subclasses (at the discretion of the subclass implementer).
- Args:
- input_shape: Instance of TensorShape, or list of instances of
TensorShape if the layer expects a list of inputs (one instance per input).
- call(inputs)[source]
This is where the layer’s logic lives.
The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state, including tf.Variable instances and nested Layer instances,
in __init__(), or in the build() method that is
called automatically before call() executes for the first time.
- Args:
- inputs: Input tensor, or dict/list/tuple of input tensors.
The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero
arguments, and inputs cannot be provided via the default value of a keyword argument.
NumPy array or Python scalar values in inputs get cast as tensors.
Keras mask metadata is only collected from inputs.
Layers are built (build(input_shape) method) using shape info from inputs only.
input_spec compatibility is only checked against inputs.
Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.
The SavedModel input specification is generated using inputs only.
Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.
- *args: Additional positional arguments. May contain tensors, although
this is not recommended, for the reasons above.
- **kwargs: Additional keyword arguments. May contain tensors, although
this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating
whether the call is meant for training or inference.
mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).
- Returns:
A tensor or list/tuple of tensors.
plots
positional
scheduler
tboard
utils
- ASTROMER.core.utils.get_folder_name(path, prefix='')[source]
Look at the current path and change the name of the experiment if it is repeated
- Args:
path (string): folder path prefix (string): prefix to add
- Returns:
string: unique path to save the experiment
- ASTROMER.core.utils.standardize(tensor, axis=0, return_mean=False)[source]
Standardize a tensor subtracting the mean
- Args:
tensor (1-dim tensorflow tensor): values axis (int): axis on which we calculate the mean return_mean (bool): output the mean of the tensor
turning on the original scale
- Returns:
tensor (1-dim tensorflow tensor): standardize tensor