configuration package¶
Submodules¶
configuration.configuration module¶
configuration.configuration_dag module¶
configuration.util module¶
Module contents¶
Module to implement a Configuration parser which enhances parsing functionality of configparser
- Author(s):
Michael Skarlinski (michael.skarlinski@weightwatchers.com)
Carl Anderson (carl.anderson@weightwatchers.com)
-
class
configuration.
Configuration
(config_location, is_dict_config=False, dict_config=None)¶ Bases:
object
Stores user defined configuration for primrose job
-
check_config
()¶ check the configuration as much as we can as early as we can
- Raises
various exceptions if any checks fail –
-
check_metadata
()¶ checks some dependencies among metadata keys
- Raises
ConfigurationError is issues found –
-
check_sections
()¶ Check that all the sections in implementation are supported ones. Either the user supplied metata.section_registry, or they are using default sections
- Raises
ConfigurationError if declaring metadata.section_registry and sections from implementation were not found in metadata –
or vice versa, or if using default operations but sections found that were not supported –
-
config_for_instance
(instance_name)¶ get the configuration for a given node / instance_name
- Returns
JSON chunk for this instance
-
dict_raise_on_duplicates
(ordered_pairs)¶ Reject duplicate keys in JSON string, ie. sections and node names.
- Parameters
ordered_pairs (list) – list of key:values from the config Example: ordered_pairs [(‘class’, ‘CsvReader’), (‘filename’, ‘data/tennis.csv’), (‘destinations’, [‘write_output’])] ordered_pairs [(‘read_data’, {‘class’: ‘CsvReader’, ‘filename’: ‘data/tennis.csv’, ‘destinations’: [‘write_output’]})]
- Returns
dictionary of key (node type) and value (node name)
- Return type
dictionary (dict)
-
static
perform_any_config_fragment_substitution
(config_str)¶ Given some configuration file content string, look for subtitutions given by $$FILE=path/to/config/file/fragment.json$$ and make the replacements using the filenames provided For example: { $$FILE=/tmp/metadata.json$$ “implementation_config”: { $$FILE= config/read_write_fragment.json $$ } } will inject /tmp/metadata.json into the 2nd line of that config.
- Parameters
config_str (str) – content of some configuration file that may or may not contain substition variables
- Returns
the post-substituted configuration string
- Return type
config_str (str)
-
sections_in_order
()¶ Return list of section names in order, either explicitly from metadata or from default Enum order
Note
If there is a non-empty section_run list in metadata return that elif there is a non-empty section_registry in metadata return that otherwise return sections present from default OperationType enum.
We need this method because the config sections are a dictionary not a list so we can’t guarantee order of keys. This method imposes an expected order.
- Returns
tuple containing:
section names (list): list of sections
source (str): where did the list come from? section_run, section_registry, or default?
- Return type
(tuple)
-