crikit.io.lazy5 package

Subpackages

Submodules

crikit.io.lazy5.alter module

Macros for inspection of HDF5 files

crikit.io.lazy5.alter.alter_attr(dset, attr_key, attr_val, file=None, pth=None, verbose=False, check_same_type=False, must_exist=False)[source]

Alter attribute dset[‘attr_key’] with attr_val.

Parameters
  • dset (str or h5py.Dataset) – String to or Dataset-object for dataset in HDF5 file. If string, file/fid must be provided.

  • attr_key (str) – Attribute name (key)

  • attr_val (str) – Attribute value to write (replace old)

  • file (str or h5py.File) – Filename or File-object for open HDF5 file

  • pth (str) – Path

  • verbose (bool) – Verbose output to stdout

  • check_same_type (bool) – Check that the inputs are compatible types as defined in lazy5.nonh5utils.check_type_compat or lazy5.utils.return_family_type

  • must_exist (bool) – The attribute must already exist.

Notes

None

crikit.io.lazy5.alter.alter_attr_same(dset, attr_key, attr_val, file=None, pth=None, verbose=True, must_exist=False)[source]

Alter attribute dset[‘attr_key] with attr_val checkint to make sure that the original and new attribute values are of similar type e.g., int and np.int32.

Parameters
  • dset (str or h5py.Dataset) – String to or Dataset-object for dataset in HDF5 file. If string, file/fid must be provided.

  • attr_key (str) – Attribute name (key)

  • attr_val (str) – Attribute value to write (replace old)

  • file (str or h5py.File) – Filename or File-object for open HDF5 file

  • pth (str) – Path

  • verbose (bool) – Verbose output to stdout

Notes

None

crikit.io.lazy5.alter.write_attr_dict(dset, attr_dict, fid=None, sort_attrs=False, verbose=False)[source]

Write entire dictionary of attrbutes to dataset.

Parameters
  • dset (str or h5py.Dataset) – String to or Dataset-object for dataset in HDF5 file. If string, fid must be provided.

  • attr_dict (dict) – Attribute dictionary

  • fid (h5py.File) – If dset is a string, file-object for open HDF5 file must be provided.

  • sort_attrs (bool) – Sort attribute keys alphabetically prior to writing

  • verbose (bool) – Verbose output to stdout

crikit.io.lazy5.config module

Default configuration class

class crikit.io.lazy5.config.DefaultConfig[source]

Bases: object

crikit.io.lazy5.create module

Macros for creation of HDF5 files and/or datasets

crikit.io.lazy5.create.save(file, dset, data, pth=None, attr_dict=None, mode='a', dset_overwrite=False, sort_attrs=False, chunks=True, verbose=False)[source]

Save an HDF5 file

Parameters
  • file (str or h5py.File object (fid)) – Filename

  • dset (str) – Dataset name (including groups if any)

  • data (ndarray) – Data to write

  • pth (str) – Path to file. Otherwise, will use present working directory (PWD)

  • attr_dict (dict) – Attribute dictionary. Will be Ordered.

  • mode (str) – h5py file mode.

  • dset_overwrite (bool) – If a dset already exists, overwrite or raise error?

  • sort_attrs (bool) – Sort the attribute dictionary (alphabetically) prior to saving

  • chunks (str or tuple or list) – Chunking shape or True for auto-chunking

  • verbose (bool) – Verbose output

Returns

bool

Return type

Saved with no errors

crikit.io.lazy5.inspect module

Macros for inspection of HDF5 files

crikit.io.lazy5.inspect.get_attrs_dset(file, dset, pth=None, convert_to_str=True, convert_sgl_np_to_num=False)[source]

Get dictionary of attribute values for a given dataset

Parameters
  • file (str or h5py.File) – Filename or File-object for open HDF5 file

  • dset (str) – Full dataset name with preprended group names. E.g., ‘/Group1/Dataset’

  • convert_to_str (bool) – If an attribute is a numpy.bytes_ string-like object, but not a str, try to decode into utf-8.

  • convert_sgl_np_to_num (bool) – If an attribute is a numpy array with a single entry, convert to non-numpy numeric type. E.g. np.array([1.0]) -> 1.0

Returns

OrderedDict

Return type

(key, value)

crikit.io.lazy5.inspect.get_datasets(file, pth=None, fulldsetpath=True)[source]
Parameters
  • file (str or _h5py.File) – Filename or File-object for open HDF5 file

  • fulldsetpath (bool) – Return just the dataset names with group names or not.

crikit.io.lazy5.inspect.get_groups(file, pth=None)[source]
Parameters

file (str or h5py.File) – Filename or File-object for open HDF5 file

Notes

Gets groups in a hierarchical list starting from the base ‘/’. Thus if Group2 is INSIDE Group1, it will return Group1, Group1/Group2 – NOT Group2 inidividually.

crikit.io.lazy5.inspect.get_hierarchy(file, pth=None, fulldsetpath=False, grp_w_dset=False)[source]

Return an ordered dictionary, where the keys are groups and the items are the datasets

Parameters
  • file (str or h5py.File) – Filename or File-object for open HDF5 file

  • fulldsetpath (bool) – If True, a dataset name will be prepended with the group down to the base level, ‘/’. If False, it will just be the dset name.

  • grp_w_dset (bool) – If True, only return groups that contain datasets. If False, include empty groups

Returns

OrderedDict – Group and dataset names

Return type

(group, [dataset list])

crikit.io.lazy5.inspect.valid_dsets(file, dset_list, pth=None, verbose=False)[source]

Check whether 1 or more datasets are valid

crikit.io.lazy5.inspect.valid_file(file, pth=None, verbose=False)[source]

Validate whether a file exists (or if a fid, is-open

crikit.io.lazy5.nonh5utils module

Non-HDF5 utility functions

crikit.io.lazy5.nonh5utils.check_type_compat(input_a, input_b)[source]

Check the compatibility of types. E.g. np.float32 IS compatible with float

crikit.io.lazy5.nonh5utils.filterlist(in_list, filters, keep_filtered_items=True, exclusive=True)[source]
Parameters
  • in_list (list) – List of strings to filter

  • filters (str, list, tuple) – Find filters (or entries of filters) in in_list

  • keep_filtered_items (bool) – Returns entries from in_list that DO have filters (INCLUDE filter). If False, EXCLUDE filter

  • exclusive (bool) – Filter is exclusive, i.e. includes/excludes in_list entries that have ALL filters. Otherwise, non-exclusive and any entry with A filter are excluded/included.

Returns

list

Return type

filtered list

crikit.io.lazy5.nonh5utils.return_family_type(input_a)[source]

Return family of type input_a. int, float, complex, str, bytes, bool

crikit.io.lazy5.utils module

Utility functions

class crikit.io.lazy5.utils.FidOrFile(file=None, mode='r')[source]

Bases: object

Class for opening an HDF5 file and returning a file ID (fid) or if passed and already open fid, passing it along (pass-thru). Primarily for enabling functions and methods to operate on open and closed files.

Parameters
  • file (str or h5py.File) – Filename or File-object for open HDF5 file

  • mode (str) – If opening a file, open with mode. Available: r,r+,w,w-,x,a

is_fid

Was the input file actually an fid.

Type

bool

fid

File ID

Type

h5py.File object

close_if_file_not_fid()[source]

Close the file if originally a filename (not a fid) was passed

return_fid_from_file(file, mode='r')[source]

Return an open fid (h5py.File). If provided a string, open file, else pass-thru given fid.

Parameters
  • file (str or h5py.File) – Filename or File-object for open HDF5 file

  • mode (str) – If opening a file, open with mode. Available: r,r+,w,w-,x,a

Returns

fid – File ID

Return type

h5py.File object

crikit.io.lazy5.utils.fullpath(filename, pth=None)[source]

Return a full path by joining a pth and filename

crikit.io.lazy5.utils.hdf_is_open(fid)[source]

Is an HDF file open via fid

Module contents

Macros for h5py… because I’m lazy