primrose.dag package¶
Submodules¶
primrose.dag.config_layer_traverser module¶
Module to traverse DAG by running all readers, then all pipelines, then all models, then all postprocess, then all cleanup
- Author(s):
Carl Anderson (carl.anderson@weightwatchers.com)
-
class
primrose.dag.config_layer_traverser.
ConfigLayerTraverser
(configuration)¶ Bases:
primrose.dag.dag_traverser.DagTraverser
Module to traverse DAG by running all readers, then all pipelines, then all models, then all postprocess, then all cleanup
-
run_section_by_section
()¶ Do we want go through section by section?
Note
suppose we had 5 sections and we asked to run just 2: section1 and section2. Is it important that we run section1 and then section 2 (if so, return True) or just that we run all the nodes from section1 and section2 together in some desired order (return False)
- Returns
result (Boolean)
-
traversal_list
()¶ get list of nodes to traverse doing all readers, then all pipelines, then all models etc, or from user-defined list from metadata.section_registry
- Returns
list of node names in the order that they will be run
- Return type
sequence (list)
-
primrose.dag.dag_traverser module¶
Abstract module to traverse a DAG
- Author(s):
Carl Anderson (carl.anderson@weightwatchers.com)
-
class
primrose.dag.dag_traverser.
DagTraverser
(configuration)¶ Bases:
abc.ABC
Abstract module to traverse a DAG
-
abstract
run_section_by_section
()¶ Do we want go through section by section?
Note
suppose we had 5 sections and we asked to run just 2: section1 and section2. Is it important that we run section1 and then section 2 (if so, return True) or just that we run all the nodes from section1 and section2 together in some desired order (return False)
- Returns
result (Boolean)
-
abstract
traversal_list
()¶ get list of nodes to traverse
- Returns
list of node name in the order that they should be run
- Return type
sequence (list)
-
abstract
primrose.dag.depth_first_traverser module¶
Module to traverse DAG by running depth first algorithm
- Author(s):
Carl Anderson (carl.anderson@weightwatchers.com)
-
class
primrose.dag.depth_first_traverser.
DepthFirstTraverser
(configuration)¶ Bases:
primrose.dag.dag_traverser.DagTraverser
a traverser that traverses in depth first manner
-
run_section_by_section
()¶ Do we want go through section by section?
Note
suppose we had 5 sections and we asked to run just 2: section1 and section2. Is it important that we run section1 and then section 2 (if so, return True) or just that we run all the nodes from section1 and section2 together in some desired order (return False)
- Returns
result (Boolean)
-
traversal_list
()¶ get list of nodes to traverse doing a depth first traversal
- Returns
list of node name in the order that they will be run
- Return type
sequence (list)
-
primrose.dag.traverser_factory module¶
Singleton Factory where one can register traversers for instantiation
- Author(s):
Carl Anderson (carl.anderson@weightwatchers.com)