| |
- IndexedRedis.IndexedRedisModel(builtins.object)
-
- NetFetchFile
-
- NetFetchCompressedBzip2File
- NetFetchCompressedGzipFile
- builtins.Exception(builtins.BaseException)
-
- InvalidPasswordException
- NoSuchNetFetchFile
class InvalidPasswordException(builtins.Exception) |
|
Raised when an invalid password is provided, or some other password-related failure.
Message contains exact reason. |
|
- Method resolution order:
- InvalidPasswordException
- builtins.Exception
- builtins.BaseException
- builtins.object
Data descriptors defined here:
- __weakref__
- list of weak references to the object (if defined)
Methods inherited from builtins.Exception:
- __init__(self, /, *args, **kwargs)
- Initialize self. See help(type(self)) for accurate signature.
- __new__(*args, **kwargs) from builtins.type
- Create and return a new object. See help(type) for accurate signature.
Methods inherited from builtins.BaseException:
- __delattr__(self, name, /)
- Implement delattr(self, name).
- __getattribute__(self, name, /)
- Return getattr(self, name).
- __reduce__(...)
- helper for pickle
- __repr__(self, /)
- Return repr(self).
- __setattr__(self, name, value, /)
- Implement setattr(self, name, value).
- __setstate__(...)
- __str__(self, /)
- Return str(self).
- with_traceback(...)
- Exception.with_traceback(tb) --
set self.__traceback__ to tb and return self.
Data descriptors inherited from builtins.BaseException:
- __cause__
- exception cause
- __context__
- exception context
- __dict__
- __suppress_context__
- __traceback__
- args
|
class NetFetchCompressedBzip2File(NetFetchFile) |
|
NetFetchCompressedBzip2File - Used for bzip2 compression on a NetFetchFile |
|
- Method resolution order:
- NetFetchCompressedBzip2File
- NetFetchFile
- IndexedRedis.IndexedRedisModel
- builtins.object
Data and other attributes defined here:
- FIELDS = [IRField( "filename", valueType=str, hashIndex=F..."bz2" ) ], hashIndex=True, defaultValue=irNull )]
- NETFETCH_TYPE = 3
Methods inherited from NetFetchFile:
- encryptData(self, password)
- encryptData - Encrypts data on this object and sets 'encrypted' flag. Does not save object.
@param password <str> 4-32 characters of password, used to encrypt.
@raises - ValueError if password does not meet requirements
- getData(self, password=None)
- getData - Fetch the data associated with this file, and potentially decrypt.
@param password <str/None> - None if unencrypted, otherwise a password 4-32 chars.
@return <bytes> - file data
@raises InvalidPasswordException - The file was encrypted and a password was invalid, or the file was not encrypted and a password was provided, or the file was encrypted and no password was provided.
@raises ValueError - If password is not 4-32 characters.
- setData(self, data)
- setData - Store data on this object and calculates checksum. If a password is required, must also call encryptData. Does not save object.
@param data <bytes> - Data to store
Class methods inherited from NetFetchFile:
- create(filename, data, mode='', owner='', group='', password=None, hostnameOverride=None) from builtins.type
- create - Create and save NetFetchFile object
@param filename <str> - filename to use for storage
@param data <bytes> - Data to str
@param mode <str> - String or int of base-8 encoded chmod value. os.stat(x).st_mode returns this.
@param owner <str> - Username owner
@param group <str> - Associated group
@param password <str/None> - If provided, 4-32 characters to encrypt. If not provided, file will not be encrypted.
@param hostnameOverride <None/Str> - Override hostname with this value. Default is to use current hostname.
@return - Saved NetFetchFile object
@raises KeyError if a hostname/filename pair already exists. use createOrUpdate to conditionally update it.
@raises ValueError if provided password does not meet criteria
- createOrUpdate(filename, data, mode='', owner='', group='', password=None, hostnameOverride=None) from builtins.type
- createOrUpdate - Create and save NetFetchFile object, or update an existing one.
@param filename <str> - filename to use for storage
@param data <bytes> - Data to str
@param mode <str> - String or int of base-8 encoded chmod value. os.stat(x).st_mode returns this.
@param owner <str> - Username owner
@param group <str> - Associated group
@param password <str/None> - If provided, 4-32 characters to encrypt. If not provided, file will not be encrypted.
@param hostnameOverride <None/Str> - Override hostname with this value. Default is to use current hostname.
@return - Saved NetFetchFile object
@raises ValueError if provided password does not meet criteria
- createOrUpdateFromFile(filename, password=None, hostnameOverride=None, savePermissions=True) from builtins.type
- createOrUpdateFromFile - Create and save NetFetchFile object, or update an existing one, provided with a filename.
@param filename <str> - filename to use for storage
@param password <str/None> - If provided, 4-32 characters to encrypt. If not provided, file will not be encrypted.
@param hostnameOverride <None/Str> - Override hostname with this value. Default is to use current hostname.
@param savePermissions <bool> Default True - If True, will store owner/group/mode of file.
@return - Saved NetFetchFile object
@raises ValueError if provided password does not meet criteria
- deleteFile(hostname, filename) from builtins.type
- deleteFile - Deletes a file from NetFetch storage.
@param hostname <str> - Hostname that file was stored on
@param filename <str> - Filename to delete
@return <bool> - True if a file was deleted, otherwise False
- downloadToLocal(hostname, filename, password=None, localFilename=None, retainPermissions=True) from builtins.type
- downloadToLocal - Download file to a local filename
@param hostname <str> - Hostname that file was stored on
@param filename <str> - Filename to fetch
@param password <str/None> - Try this password on potentially encrpyted file.
@param localFilename <str/None> - If defined, saves at this location. Otherwise, saves at #filename
@param retainPermissions <bool> Default True - If True, tries to retain owner/group/mode. If owner/group, you must be root. Silently fails if can't apply.
@raises NoSuchNetFetchFile - If no hostname/filename match exists
@raises InvalidPasswordException - If password was invalid, see getData for all conditions.
- downloadToStr(hostname, filename, password=None) from builtins.type
- downloadToStr - Download a hostname/filename pair and return as a string
@param hostname <str> - Hostname that file was stored on
@param filename <str> - Filename to fetch
@param password <str/None> - Try this password on potentially encrpyted file.
@return <bytes> - Data that has been downloaded
@raises NoSuchNetFetchFile - If no hostname/filename match exists
@raises InvalidPasswordException - If password was invalid, see getData for all conditions.
- exists(hostname, filename) from builtins.type
- exists - Check if a hostname/filename pair exists
@param hostname <str> - Hostname field
@param filename <str> - Filename field
@return <bool> - If a file is stored under the hostname/filename key pair
Static methods inherited from NetFetchFile:
- calculateChecksum(data)
- calculateChecksum - Calculates a checksum from given data
@param data <bytes> - Data to check
- getNetFetchClassForKey(primaryKey)
Data and other attributes inherited from NetFetchFile:
- INDEXED_FIELDS = ['filename', 'hostname']
- KEY_NAME = 'NetFetchFile'
Methods inherited from IndexedRedis.IndexedRedisModel:
- __copy__(self)
- __copy__ - Used by the "copy" module to make a copy,
which will NOT be linked to the original entry in the database, but will contain the same data
@return <IndexedRedisModel> - Copy of this object, per above
- __deepcopy__(self, *args, **kwargs)
- __deepcopy__ - Used by the "copy" module to make a deepcopy.
Will perform a deepcopy of all attributes, which will NOT be linked to the original entry in the database.
If you need a copy that IS linked, @see IndexedRedisModel.copy
@return <IndexedRedisModel> - Deep copy of this object, per above
- __eq__(self, other)
- __eq__ - Check if two IndexedRedisModels are equal.
They are equal if they have the same type and same field values (including id).
To check if two models have the same values (but can have different ids), use #hasSameValues method.
- __getstate__(self)
- pickle uses this
- __init__(self, *args, **kwargs)
- __init__ - Set the values on this object. MAKE SURE YOU CALL THE SUPER HERE, or else things will not work.
- __ne__(self, other)
- __ne__ - Check if two IndexedRedisModels are NOT equal.
@see IndexedRedisModel.__eq__
- __repr__(self)
- __repr__ - Returns a string of the constructor/params to recreate this object.
Example: objCopy = eval(repr(obj))
@return - String of python init call to recreate this object
- __setattr__(self, keyName, value)
- __setattr__ - Will be used to set an attribute on this object.
If the attribute is a field (in self.FIELDS), it will be converted via the field type's #fromInput method.
Otherwise, it will just set the attribute on this object.
- __setstate__(self, stateDict)
- pickle uses this
- __str__(self)
- __str__ - Returns a string representation of this object's state.
See implementation.
@return <str>-
Some samples:
(Pdb) str(z)
'<Song obj _id=24 at 0x7f3c6a3a4490>'
(Pdb) z.artist = 'New Artist'
(Pdb) str(z)
'<Song obj _id=24 (Unsaved Changes) at 0x7f3c6a3a4490>'
- asDict(self, includeMeta=False, forStorage=False, strKeys=False)
- toDict / asDict - Get a dictionary representation of this model.
@param includeMeta - Include metadata in return. For now, this is only pk stored as "_id"
@param convertValueTypes <bool> - default True. If False, fields with fieldValue defined will be converted to that type.
Use True when saving, etc, as native type is always either str or bytes.
@param strKeys <bool> Default False - If True, just the string value of the field name will be used as the key.
Otherwise, the IRField itself will be (although represented and indexed by string)
@return - Dictionary reprensetation of this object and all fields
- connect = _deprecated_wrapper(*args, **kwargs)
- copy(self, copyPrimaryKey=False, copyValues=False)
- copy - Copies this object.
@param copyPrimaryKey <bool> default False - If True, any changes to the copy will save over-top the existing entry in Redis.
If False, only the data is copied, and nothing is saved.
@param copyValues <bool> default False - If True, every field value on this object will be explicitly copied. If False,
an object will be created with the same values, and depending on the type may share the same reference.
This is the difference between a copy and a deepcopy.
@return <IndexedRedisModel> - Copy of this object, per above
If you need a copy that IS linked, @see IndexedRedisModel.copy
- delete(self)
- delete - Delete this object
- getPk(self)
- getPk - Gets the internal primary key associated with this object
- getUpdatedFields(self)
- getUpdatedFields - See changed fields.
@return - a dictionary of fieldName : tuple(old, new).
fieldName may be a string or may implement IRField (which implements string, and can be used just like a string)
- hasSameValues(self, other)
- hasSameValues - Check if this and another model have the same fields and values.
This does NOT include id, so the models can have the same values but be different objects in the database.
@param other <IndexedRedisModel> - Another model
@return <bool> - True if all fields have the same value, otherwise False
- hasUnsavedChanges(self)
- hasUnsavedChanges - Check if any unsaved changes are present in this model, or if it has never been saved.
@return <bool> - True if any fields have changed since last fetch, or if never saved. Otherwise, False
- pprint(self, stream=None)
- pprint - Pretty-print a dict representation of this object.
@param stream <file/None> - Either a stream to output, or None to default to sys.stdout
- reload(self)
- reload - Reload this object from the database, overriding any local changes and merging in any updates.
@raises KeyError - if this object has not been saved (no primary key)
@return - Dict with the keys that were updated. Key is field name that was updated,
and value is tuple of (old value, new value).
- save(self)
- save - Save this object.
Will perform an "insert" if this object had not been saved before,
otherwise will update JUST the fields changed on THIS INSTANCE of the model.
i.e. If you have two processes fetch the same object and change different fields, they will not overwrite
eachother, but only save the ones each process changed.
If you want to save multiple objects of type MyModel in a single transaction,
and you have those objects in a list, myObjs, you can do the following:
MyModel.saver.save(myObjs)
@see #IndexedRedisSave.save
@return <list> - Single element list, id of saved object (if successful)
- saveToExternal(self, redisCon)
- saveToExternal - Saves this object to a different Redis than that specified by REDIS_CONNECTION_PARAMS on this model.
@param redisCon <dict/redis.Redis> - Either a dict of connection params, a la REDIS_CONNECTION_PARAMS, or an existing Redis connection.
If you are doing a lot of bulk copies, it is recommended that you create a Redis connection and pass it in rather than establish a new
connection with each call.
@note - You will generate a new primary key relative to the external Redis environment. If you need to reference a "shared" primary key, it is better
to use an indexed field than the internal pk.
- toDict = asDict(self, includeMeta=False, forStorage=False, strKeys=False)
- toDict / asDict - Get a dictionary representation of this model.
@param includeMeta - Include metadata in return. For now, this is only pk stored as "_id"
@param convertValueTypes <bool> - default True. If False, fields with fieldValue defined will be converted to that type.
Use True when saving, etc, as native type is always either str or bytes.
@param strKeys <bool> Default False - If True, just the string value of the field name will be used as the key.
Otherwise, the IRField itself will be (although represented and indexed by string)
@return - Dictionary reprensetation of this object and all fields
Class methods inherited from IndexedRedis.IndexedRedisModel:
- connectAlt(redisConnectionParams) from builtins.type
- connectAlt - Create a class of this model which will use an alternate connection than the one specified by REDIS_CONNECTION_PARAMS on this model.
@param redisConnectionParams <dict> - Dictionary of arguments to redis.Redis, same as REDIS_CONNECTION_PARAMS.
@return - A class that can be used in all the same ways as the existing IndexedRedisModel, but that connects to a different instance.
The fields and key will be the same here, but the connection will be different. use #copyModel if you want an independent class for the model
- copyModel() from builtins.type
- copyModel - Copy this model, and return that copy.
The copied model will have all the same data, but will have a fresh instance of the FIELDS array and all members,
and the INDEXED_FIELDS array.
This is useful for converting, like changing field types or whatever, where you can load from one model and save into the other.
@return <IndexedRedisModel> - A copy class of this model class with a unique name.
- reset(newObjs) from builtins.type
- reset - Remove all stored data associated with this model (i.e. all objects of this type),
and then save all the provided objects in #newObjs , all in one atomic transaction.
Use this method to move from one complete set of objects to another, where any querying applications
will only see the complete before or complete after.
@param newObjs list<IndexedRedisModel objs> - A list of objects that will replace the current dataset
To just replace a specific subset of objects in a single transaction, you can do MyModel.saver.save(objs)
and just the objs in "objs" will be inserted/updated in one atomic step.
This method, on the other hand, will delete all previous objects and add the newly provided objects in a single atomic step,
and also reset the primary key ID generator
@return list<int> - The new primary keys associated with each object (same order as provided #newObjs list)
- validateModel() from builtins.type
- validateModel - Class method that validates a given model is implemented correctly. Will only be validated once, on first model instantiation.
@param model - Implicit of own class
@return - True
@raises - InvalidModelException if there is a problem with the model, and the message contains relevant information.
Data descriptors inherited from IndexedRedis.IndexedRedisModel:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
Data and other attributes inherited from IndexedRedis.IndexedRedisModel:
- REDIS_CONNECTION_PARAMS = {}
- __hash__ = None
- deleter = <IndexedRedis.IndexedRedisDelete object>
- objects = <IndexedRedis.IndexedRedisQuery object>
- saver = <IndexedRedis.IndexedRedisSave object>
|
class NetFetchCompressedGzipFile(NetFetchFile) |
|
NetFetchCompressedGzipFile - Used for gzip compression on a NetFetchFile |
|
- Method resolution order:
- NetFetchCompressedGzipFile
- NetFetchFile
- IndexedRedis.IndexedRedisModel
- builtins.object
Data and other attributes defined here:
- FIELDS = [IRField( "filename", valueType=str, hashIndex=F...zlib" ) ], hashIndex=True, defaultValue=irNull )]
- NETFETCH_TYPE = 2
Methods inherited from NetFetchFile:
- encryptData(self, password)
- encryptData - Encrypts data on this object and sets 'encrypted' flag. Does not save object.
@param password <str> 4-32 characters of password, used to encrypt.
@raises - ValueError if password does not meet requirements
- getData(self, password=None)
- getData - Fetch the data associated with this file, and potentially decrypt.
@param password <str/None> - None if unencrypted, otherwise a password 4-32 chars.
@return <bytes> - file data
@raises InvalidPasswordException - The file was encrypted and a password was invalid, or the file was not encrypted and a password was provided, or the file was encrypted and no password was provided.
@raises ValueError - If password is not 4-32 characters.
- setData(self, data)
- setData - Store data on this object and calculates checksum. If a password is required, must also call encryptData. Does not save object.
@param data <bytes> - Data to store
Class methods inherited from NetFetchFile:
- create(filename, data, mode='', owner='', group='', password=None, hostnameOverride=None) from builtins.type
- create - Create and save NetFetchFile object
@param filename <str> - filename to use for storage
@param data <bytes> - Data to str
@param mode <str> - String or int of base-8 encoded chmod value. os.stat(x).st_mode returns this.
@param owner <str> - Username owner
@param group <str> - Associated group
@param password <str/None> - If provided, 4-32 characters to encrypt. If not provided, file will not be encrypted.
@param hostnameOverride <None/Str> - Override hostname with this value. Default is to use current hostname.
@return - Saved NetFetchFile object
@raises KeyError if a hostname/filename pair already exists. use createOrUpdate to conditionally update it.
@raises ValueError if provided password does not meet criteria
- createOrUpdate(filename, data, mode='', owner='', group='', password=None, hostnameOverride=None) from builtins.type
- createOrUpdate - Create and save NetFetchFile object, or update an existing one.
@param filename <str> - filename to use for storage
@param data <bytes> - Data to str
@param mode <str> - String or int of base-8 encoded chmod value. os.stat(x).st_mode returns this.
@param owner <str> - Username owner
@param group <str> - Associated group
@param password <str/None> - If provided, 4-32 characters to encrypt. If not provided, file will not be encrypted.
@param hostnameOverride <None/Str> - Override hostname with this value. Default is to use current hostname.
@return - Saved NetFetchFile object
@raises ValueError if provided password does not meet criteria
- createOrUpdateFromFile(filename, password=None, hostnameOverride=None, savePermissions=True) from builtins.type
- createOrUpdateFromFile - Create and save NetFetchFile object, or update an existing one, provided with a filename.
@param filename <str> - filename to use for storage
@param password <str/None> - If provided, 4-32 characters to encrypt. If not provided, file will not be encrypted.
@param hostnameOverride <None/Str> - Override hostname with this value. Default is to use current hostname.
@param savePermissions <bool> Default True - If True, will store owner/group/mode of file.
@return - Saved NetFetchFile object
@raises ValueError if provided password does not meet criteria
- deleteFile(hostname, filename) from builtins.type
- deleteFile - Deletes a file from NetFetch storage.
@param hostname <str> - Hostname that file was stored on
@param filename <str> - Filename to delete
@return <bool> - True if a file was deleted, otherwise False
- downloadToLocal(hostname, filename, password=None, localFilename=None, retainPermissions=True) from builtins.type
- downloadToLocal - Download file to a local filename
@param hostname <str> - Hostname that file was stored on
@param filename <str> - Filename to fetch
@param password <str/None> - Try this password on potentially encrpyted file.
@param localFilename <str/None> - If defined, saves at this location. Otherwise, saves at #filename
@param retainPermissions <bool> Default True - If True, tries to retain owner/group/mode. If owner/group, you must be root. Silently fails if can't apply.
@raises NoSuchNetFetchFile - If no hostname/filename match exists
@raises InvalidPasswordException - If password was invalid, see getData for all conditions.
- downloadToStr(hostname, filename, password=None) from builtins.type
- downloadToStr - Download a hostname/filename pair and return as a string
@param hostname <str> - Hostname that file was stored on
@param filename <str> - Filename to fetch
@param password <str/None> - Try this password on potentially encrpyted file.
@return <bytes> - Data that has been downloaded
@raises NoSuchNetFetchFile - If no hostname/filename match exists
@raises InvalidPasswordException - If password was invalid, see getData for all conditions.
- exists(hostname, filename) from builtins.type
- exists - Check if a hostname/filename pair exists
@param hostname <str> - Hostname field
@param filename <str> - Filename field
@return <bool> - If a file is stored under the hostname/filename key pair
Static methods inherited from NetFetchFile:
- calculateChecksum(data)
- calculateChecksum - Calculates a checksum from given data
@param data <bytes> - Data to check
- getNetFetchClassForKey(primaryKey)
Data and other attributes inherited from NetFetchFile:
- INDEXED_FIELDS = ['filename', 'hostname']
- KEY_NAME = 'NetFetchFile'
Methods inherited from IndexedRedis.IndexedRedisModel:
- __copy__(self)
- __copy__ - Used by the "copy" module to make a copy,
which will NOT be linked to the original entry in the database, but will contain the same data
@return <IndexedRedisModel> - Copy of this object, per above
- __deepcopy__(self, *args, **kwargs)
- __deepcopy__ - Used by the "copy" module to make a deepcopy.
Will perform a deepcopy of all attributes, which will NOT be linked to the original entry in the database.
If you need a copy that IS linked, @see IndexedRedisModel.copy
@return <IndexedRedisModel> - Deep copy of this object, per above
- __eq__(self, other)
- __eq__ - Check if two IndexedRedisModels are equal.
They are equal if they have the same type and same field values (including id).
To check if two models have the same values (but can have different ids), use #hasSameValues method.
- __getstate__(self)
- pickle uses this
- __init__(self, *args, **kwargs)
- __init__ - Set the values on this object. MAKE SURE YOU CALL THE SUPER HERE, or else things will not work.
- __ne__(self, other)
- __ne__ - Check if two IndexedRedisModels are NOT equal.
@see IndexedRedisModel.__eq__
- __repr__(self)
- __repr__ - Returns a string of the constructor/params to recreate this object.
Example: objCopy = eval(repr(obj))
@return - String of python init call to recreate this object
- __setattr__(self, keyName, value)
- __setattr__ - Will be used to set an attribute on this object.
If the attribute is a field (in self.FIELDS), it will be converted via the field type's #fromInput method.
Otherwise, it will just set the attribute on this object.
- __setstate__(self, stateDict)
- pickle uses this
- __str__(self)
- __str__ - Returns a string representation of this object's state.
See implementation.
@return <str>-
Some samples:
(Pdb) str(z)
'<Song obj _id=24 at 0x7f3c6a3a4490>'
(Pdb) z.artist = 'New Artist'
(Pdb) str(z)
'<Song obj _id=24 (Unsaved Changes) at 0x7f3c6a3a4490>'
- asDict(self, includeMeta=False, forStorage=False, strKeys=False)
- toDict / asDict - Get a dictionary representation of this model.
@param includeMeta - Include metadata in return. For now, this is only pk stored as "_id"
@param convertValueTypes <bool> - default True. If False, fields with fieldValue defined will be converted to that type.
Use True when saving, etc, as native type is always either str or bytes.
@param strKeys <bool> Default False - If True, just the string value of the field name will be used as the key.
Otherwise, the IRField itself will be (although represented and indexed by string)
@return - Dictionary reprensetation of this object and all fields
- connect = _deprecated_wrapper(*args, **kwargs)
- copy(self, copyPrimaryKey=False, copyValues=False)
- copy - Copies this object.
@param copyPrimaryKey <bool> default False - If True, any changes to the copy will save over-top the existing entry in Redis.
If False, only the data is copied, and nothing is saved.
@param copyValues <bool> default False - If True, every field value on this object will be explicitly copied. If False,
an object will be created with the same values, and depending on the type may share the same reference.
This is the difference between a copy and a deepcopy.
@return <IndexedRedisModel> - Copy of this object, per above
If you need a copy that IS linked, @see IndexedRedisModel.copy
- delete(self)
- delete - Delete this object
- getPk(self)
- getPk - Gets the internal primary key associated with this object
- getUpdatedFields(self)
- getUpdatedFields - See changed fields.
@return - a dictionary of fieldName : tuple(old, new).
fieldName may be a string or may implement IRField (which implements string, and can be used just like a string)
- hasSameValues(self, other)
- hasSameValues - Check if this and another model have the same fields and values.
This does NOT include id, so the models can have the same values but be different objects in the database.
@param other <IndexedRedisModel> - Another model
@return <bool> - True if all fields have the same value, otherwise False
- hasUnsavedChanges(self)
- hasUnsavedChanges - Check if any unsaved changes are present in this model, or if it has never been saved.
@return <bool> - True if any fields have changed since last fetch, or if never saved. Otherwise, False
- pprint(self, stream=None)
- pprint - Pretty-print a dict representation of this object.
@param stream <file/None> - Either a stream to output, or None to default to sys.stdout
- reload(self)
- reload - Reload this object from the database, overriding any local changes and merging in any updates.
@raises KeyError - if this object has not been saved (no primary key)
@return - Dict with the keys that were updated. Key is field name that was updated,
and value is tuple of (old value, new value).
- save(self)
- save - Save this object.
Will perform an "insert" if this object had not been saved before,
otherwise will update JUST the fields changed on THIS INSTANCE of the model.
i.e. If you have two processes fetch the same object and change different fields, they will not overwrite
eachother, but only save the ones each process changed.
If you want to save multiple objects of type MyModel in a single transaction,
and you have those objects in a list, myObjs, you can do the following:
MyModel.saver.save(myObjs)
@see #IndexedRedisSave.save
@return <list> - Single element list, id of saved object (if successful)
- saveToExternal(self, redisCon)
- saveToExternal - Saves this object to a different Redis than that specified by REDIS_CONNECTION_PARAMS on this model.
@param redisCon <dict/redis.Redis> - Either a dict of connection params, a la REDIS_CONNECTION_PARAMS, or an existing Redis connection.
If you are doing a lot of bulk copies, it is recommended that you create a Redis connection and pass it in rather than establish a new
connection with each call.
@note - You will generate a new primary key relative to the external Redis environment. If you need to reference a "shared" primary key, it is better
to use an indexed field than the internal pk.
- toDict = asDict(self, includeMeta=False, forStorage=False, strKeys=False)
- toDict / asDict - Get a dictionary representation of this model.
@param includeMeta - Include metadata in return. For now, this is only pk stored as "_id"
@param convertValueTypes <bool> - default True. If False, fields with fieldValue defined will be converted to that type.
Use True when saving, etc, as native type is always either str or bytes.
@param strKeys <bool> Default False - If True, just the string value of the field name will be used as the key.
Otherwise, the IRField itself will be (although represented and indexed by string)
@return - Dictionary reprensetation of this object and all fields
Class methods inherited from IndexedRedis.IndexedRedisModel:
- connectAlt(redisConnectionParams) from builtins.type
- connectAlt - Create a class of this model which will use an alternate connection than the one specified by REDIS_CONNECTION_PARAMS on this model.
@param redisConnectionParams <dict> - Dictionary of arguments to redis.Redis, same as REDIS_CONNECTION_PARAMS.
@return - A class that can be used in all the same ways as the existing IndexedRedisModel, but that connects to a different instance.
The fields and key will be the same here, but the connection will be different. use #copyModel if you want an independent class for the model
- copyModel() from builtins.type
- copyModel - Copy this model, and return that copy.
The copied model will have all the same data, but will have a fresh instance of the FIELDS array and all members,
and the INDEXED_FIELDS array.
This is useful for converting, like changing field types or whatever, where you can load from one model and save into the other.
@return <IndexedRedisModel> - A copy class of this model class with a unique name.
- reset(newObjs) from builtins.type
- reset - Remove all stored data associated with this model (i.e. all objects of this type),
and then save all the provided objects in #newObjs , all in one atomic transaction.
Use this method to move from one complete set of objects to another, where any querying applications
will only see the complete before or complete after.
@param newObjs list<IndexedRedisModel objs> - A list of objects that will replace the current dataset
To just replace a specific subset of objects in a single transaction, you can do MyModel.saver.save(objs)
and just the objs in "objs" will be inserted/updated in one atomic step.
This method, on the other hand, will delete all previous objects and add the newly provided objects in a single atomic step,
and also reset the primary key ID generator
@return list<int> - The new primary keys associated with each object (same order as provided #newObjs list)
- validateModel() from builtins.type
- validateModel - Class method that validates a given model is implemented correctly. Will only be validated once, on first model instantiation.
@param model - Implicit of own class
@return - True
@raises - InvalidModelException if there is a problem with the model, and the message contains relevant information.
Data descriptors inherited from IndexedRedis.IndexedRedisModel:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
Data and other attributes inherited from IndexedRedis.IndexedRedisModel:
- REDIS_CONNECTION_PARAMS = {}
- __hash__ = None
- deleter = <IndexedRedis.IndexedRedisDelete object>
- objects = <IndexedRedis.IndexedRedisQuery object>
- saver = <IndexedRedis.IndexedRedisSave object>
|
class NetFetchFile(IndexedRedis.IndexedRedisModel) |
|
NetFetchFile - Represents a File stored in NetFetch.
A file is keyed based on hostname and absolute filename.
May optionally have mode,owner,group filled-in.
May be encrypted with a password, in which case "encrypted" is set to "1". |
|
- Method resolution order:
- NetFetchFile
- IndexedRedis.IndexedRedisModel
- builtins.object
Methods defined here:
- encryptData(self, password)
- encryptData - Encrypts data on this object and sets 'encrypted' flag. Does not save object.
@param password <str> 4-32 characters of password, used to encrypt.
@raises - ValueError if password does not meet requirements
- getData(self, password=None)
- getData - Fetch the data associated with this file, and potentially decrypt.
@param password <str/None> - None if unencrypted, otherwise a password 4-32 chars.
@return <bytes> - file data
@raises InvalidPasswordException - The file was encrypted and a password was invalid, or the file was not encrypted and a password was provided, or the file was encrypted and no password was provided.
@raises ValueError - If password is not 4-32 characters.
- setData(self, data)
- setData - Store data on this object and calculates checksum. If a password is required, must also call encryptData. Does not save object.
@param data <bytes> - Data to store
Class methods defined here:
- create(filename, data, mode='', owner='', group='', password=None, hostnameOverride=None) from builtins.type
- create - Create and save NetFetchFile object
@param filename <str> - filename to use for storage
@param data <bytes> - Data to str
@param mode <str> - String or int of base-8 encoded chmod value. os.stat(x).st_mode returns this.
@param owner <str> - Username owner
@param group <str> - Associated group
@param password <str/None> - If provided, 4-32 characters to encrypt. If not provided, file will not be encrypted.
@param hostnameOverride <None/Str> - Override hostname with this value. Default is to use current hostname.
@return - Saved NetFetchFile object
@raises KeyError if a hostname/filename pair already exists. use createOrUpdate to conditionally update it.
@raises ValueError if provided password does not meet criteria
- createOrUpdate(filename, data, mode='', owner='', group='', password=None, hostnameOverride=None) from builtins.type
- createOrUpdate - Create and save NetFetchFile object, or update an existing one.
@param filename <str> - filename to use for storage
@param data <bytes> - Data to str
@param mode <str> - String or int of base-8 encoded chmod value. os.stat(x).st_mode returns this.
@param owner <str> - Username owner
@param group <str> - Associated group
@param password <str/None> - If provided, 4-32 characters to encrypt. If not provided, file will not be encrypted.
@param hostnameOverride <None/Str> - Override hostname with this value. Default is to use current hostname.
@return - Saved NetFetchFile object
@raises ValueError if provided password does not meet criteria
- createOrUpdateFromFile(filename, password=None, hostnameOverride=None, savePermissions=True) from builtins.type
- createOrUpdateFromFile - Create and save NetFetchFile object, or update an existing one, provided with a filename.
@param filename <str> - filename to use for storage
@param password <str/None> - If provided, 4-32 characters to encrypt. If not provided, file will not be encrypted.
@param hostnameOverride <None/Str> - Override hostname with this value. Default is to use current hostname.
@param savePermissions <bool> Default True - If True, will store owner/group/mode of file.
@return - Saved NetFetchFile object
@raises ValueError if provided password does not meet criteria
- deleteFile(hostname, filename) from builtins.type
- deleteFile - Deletes a file from NetFetch storage.
@param hostname <str> - Hostname that file was stored on
@param filename <str> - Filename to delete
@return <bool> - True if a file was deleted, otherwise False
- downloadToLocal(hostname, filename, password=None, localFilename=None, retainPermissions=True) from builtins.type
- downloadToLocal - Download file to a local filename
@param hostname <str> - Hostname that file was stored on
@param filename <str> - Filename to fetch
@param password <str/None> - Try this password on potentially encrpyted file.
@param localFilename <str/None> - If defined, saves at this location. Otherwise, saves at #filename
@param retainPermissions <bool> Default True - If True, tries to retain owner/group/mode. If owner/group, you must be root. Silently fails if can't apply.
@raises NoSuchNetFetchFile - If no hostname/filename match exists
@raises InvalidPasswordException - If password was invalid, see getData for all conditions.
- downloadToStr(hostname, filename, password=None) from builtins.type
- downloadToStr - Download a hostname/filename pair and return as a string
@param hostname <str> - Hostname that file was stored on
@param filename <str> - Filename to fetch
@param password <str/None> - Try this password on potentially encrpyted file.
@return <bytes> - Data that has been downloaded
@raises NoSuchNetFetchFile - If no hostname/filename match exists
@raises InvalidPasswordException - If password was invalid, see getData for all conditions.
- exists(hostname, filename) from builtins.type
- exists - Check if a hostname/filename pair exists
@param hostname <str> - Hostname field
@param filename <str> - Filename field
@return <bool> - If a file is stored under the hostname/filename key pair
Static methods defined here:
- calculateChecksum(data)
- calculateChecksum - Calculates a checksum from given data
@param data <bytes> - Data to check
- getNetFetchClassForKey(primaryKey)
Data and other attributes defined here:
- FIELDS = [IRField( "filename", valueType=str, hashIndex=F...d( "data", hashIndex=True, defaultValue=irNull )]
- INDEXED_FIELDS = ['filename', 'hostname']
- KEY_NAME = 'NetFetchFile'
- NETFETCH_TYPE = 0
Methods inherited from IndexedRedis.IndexedRedisModel:
- __copy__(self)
- __copy__ - Used by the "copy" module to make a copy,
which will NOT be linked to the original entry in the database, but will contain the same data
@return <IndexedRedisModel> - Copy of this object, per above
- __deepcopy__(self, *args, **kwargs)
- __deepcopy__ - Used by the "copy" module to make a deepcopy.
Will perform a deepcopy of all attributes, which will NOT be linked to the original entry in the database.
If you need a copy that IS linked, @see IndexedRedisModel.copy
@return <IndexedRedisModel> - Deep copy of this object, per above
- __eq__(self, other)
- __eq__ - Check if two IndexedRedisModels are equal.
They are equal if they have the same type and same field values (including id).
To check if two models have the same values (but can have different ids), use #hasSameValues method.
- __getstate__(self)
- pickle uses this
- __init__(self, *args, **kwargs)
- __init__ - Set the values on this object. MAKE SURE YOU CALL THE SUPER HERE, or else things will not work.
- __ne__(self, other)
- __ne__ - Check if two IndexedRedisModels are NOT equal.
@see IndexedRedisModel.__eq__
- __repr__(self)
- __repr__ - Returns a string of the constructor/params to recreate this object.
Example: objCopy = eval(repr(obj))
@return - String of python init call to recreate this object
- __setattr__(self, keyName, value)
- __setattr__ - Will be used to set an attribute on this object.
If the attribute is a field (in self.FIELDS), it will be converted via the field type's #fromInput method.
Otherwise, it will just set the attribute on this object.
- __setstate__(self, stateDict)
- pickle uses this
- __str__(self)
- __str__ - Returns a string representation of this object's state.
See implementation.
@return <str>-
Some samples:
(Pdb) str(z)
'<Song obj _id=24 at 0x7f3c6a3a4490>'
(Pdb) z.artist = 'New Artist'
(Pdb) str(z)
'<Song obj _id=24 (Unsaved Changes) at 0x7f3c6a3a4490>'
- asDict(self, includeMeta=False, forStorage=False, strKeys=False)
- toDict / asDict - Get a dictionary representation of this model.
@param includeMeta - Include metadata in return. For now, this is only pk stored as "_id"
@param convertValueTypes <bool> - default True. If False, fields with fieldValue defined will be converted to that type.
Use True when saving, etc, as native type is always either str or bytes.
@param strKeys <bool> Default False - If True, just the string value of the field name will be used as the key.
Otherwise, the IRField itself will be (although represented and indexed by string)
@return - Dictionary reprensetation of this object and all fields
- connect = _deprecated_wrapper(*args, **kwargs)
- copy(self, copyPrimaryKey=False, copyValues=False)
- copy - Copies this object.
@param copyPrimaryKey <bool> default False - If True, any changes to the copy will save over-top the existing entry in Redis.
If False, only the data is copied, and nothing is saved.
@param copyValues <bool> default False - If True, every field value on this object will be explicitly copied. If False,
an object will be created with the same values, and depending on the type may share the same reference.
This is the difference between a copy and a deepcopy.
@return <IndexedRedisModel> - Copy of this object, per above
If you need a copy that IS linked, @see IndexedRedisModel.copy
- delete(self)
- delete - Delete this object
- getPk(self)
- getPk - Gets the internal primary key associated with this object
- getUpdatedFields(self)
- getUpdatedFields - See changed fields.
@return - a dictionary of fieldName : tuple(old, new).
fieldName may be a string or may implement IRField (which implements string, and can be used just like a string)
- hasSameValues(self, other)
- hasSameValues - Check if this and another model have the same fields and values.
This does NOT include id, so the models can have the same values but be different objects in the database.
@param other <IndexedRedisModel> - Another model
@return <bool> - True if all fields have the same value, otherwise False
- hasUnsavedChanges(self)
- hasUnsavedChanges - Check if any unsaved changes are present in this model, or if it has never been saved.
@return <bool> - True if any fields have changed since last fetch, or if never saved. Otherwise, False
- pprint(self, stream=None)
- pprint - Pretty-print a dict representation of this object.
@param stream <file/None> - Either a stream to output, or None to default to sys.stdout
- reload(self)
- reload - Reload this object from the database, overriding any local changes and merging in any updates.
@raises KeyError - if this object has not been saved (no primary key)
@return - Dict with the keys that were updated. Key is field name that was updated,
and value is tuple of (old value, new value).
- save(self)
- save - Save this object.
Will perform an "insert" if this object had not been saved before,
otherwise will update JUST the fields changed on THIS INSTANCE of the model.
i.e. If you have two processes fetch the same object and change different fields, they will not overwrite
eachother, but only save the ones each process changed.
If you want to save multiple objects of type MyModel in a single transaction,
and you have those objects in a list, myObjs, you can do the following:
MyModel.saver.save(myObjs)
@see #IndexedRedisSave.save
@return <list> - Single element list, id of saved object (if successful)
- saveToExternal(self, redisCon)
- saveToExternal - Saves this object to a different Redis than that specified by REDIS_CONNECTION_PARAMS on this model.
@param redisCon <dict/redis.Redis> - Either a dict of connection params, a la REDIS_CONNECTION_PARAMS, or an existing Redis connection.
If you are doing a lot of bulk copies, it is recommended that you create a Redis connection and pass it in rather than establish a new
connection with each call.
@note - You will generate a new primary key relative to the external Redis environment. If you need to reference a "shared" primary key, it is better
to use an indexed field than the internal pk.
- toDict = asDict(self, includeMeta=False, forStorage=False, strKeys=False)
- toDict / asDict - Get a dictionary representation of this model.
@param includeMeta - Include metadata in return. For now, this is only pk stored as "_id"
@param convertValueTypes <bool> - default True. If False, fields with fieldValue defined will be converted to that type.
Use True when saving, etc, as native type is always either str or bytes.
@param strKeys <bool> Default False - If True, just the string value of the field name will be used as the key.
Otherwise, the IRField itself will be (although represented and indexed by string)
@return - Dictionary reprensetation of this object and all fields
Class methods inherited from IndexedRedis.IndexedRedisModel:
- connectAlt(redisConnectionParams) from builtins.type
- connectAlt - Create a class of this model which will use an alternate connection than the one specified by REDIS_CONNECTION_PARAMS on this model.
@param redisConnectionParams <dict> - Dictionary of arguments to redis.Redis, same as REDIS_CONNECTION_PARAMS.
@return - A class that can be used in all the same ways as the existing IndexedRedisModel, but that connects to a different instance.
The fields and key will be the same here, but the connection will be different. use #copyModel if you want an independent class for the model
- copyModel() from builtins.type
- copyModel - Copy this model, and return that copy.
The copied model will have all the same data, but will have a fresh instance of the FIELDS array and all members,
and the INDEXED_FIELDS array.
This is useful for converting, like changing field types or whatever, where you can load from one model and save into the other.
@return <IndexedRedisModel> - A copy class of this model class with a unique name.
- reset(newObjs) from builtins.type
- reset - Remove all stored data associated with this model (i.e. all objects of this type),
and then save all the provided objects in #newObjs , all in one atomic transaction.
Use this method to move from one complete set of objects to another, where any querying applications
will only see the complete before or complete after.
@param newObjs list<IndexedRedisModel objs> - A list of objects that will replace the current dataset
To just replace a specific subset of objects in a single transaction, you can do MyModel.saver.save(objs)
and just the objs in "objs" will be inserted/updated in one atomic step.
This method, on the other hand, will delete all previous objects and add the newly provided objects in a single atomic step,
and also reset the primary key ID generator
@return list<int> - The new primary keys associated with each object (same order as provided #newObjs list)
- validateModel() from builtins.type
- validateModel - Class method that validates a given model is implemented correctly. Will only be validated once, on first model instantiation.
@param model - Implicit of own class
@return - True
@raises - InvalidModelException if there is a problem with the model, and the message contains relevant information.
Data descriptors inherited from IndexedRedis.IndexedRedisModel:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
Data and other attributes inherited from IndexedRedis.IndexedRedisModel:
- REDIS_CONNECTION_PARAMS = {}
- __hash__ = None
- deleter = <IndexedRedis.IndexedRedisDelete object>
- objects = <IndexedRedis.IndexedRedisQuery object>
- saver = <IndexedRedis.IndexedRedisSave object>
|
class NoSuchNetFetchFile(builtins.Exception) |
|
Raised when a file attempted to be fetched does not exist |
|
- Method resolution order:
- NoSuchNetFetchFile
- builtins.Exception
- builtins.BaseException
- builtins.object
Data descriptors defined here:
- __weakref__
- list of weak references to the object (if defined)
Methods inherited from builtins.Exception:
- __init__(self, /, *args, **kwargs)
- Initialize self. See help(type(self)) for accurate signature.
- __new__(*args, **kwargs) from builtins.type
- Create and return a new object. See help(type) for accurate signature.
Methods inherited from builtins.BaseException:
- __delattr__(self, name, /)
- Implement delattr(self, name).
- __getattribute__(self, name, /)
- Return getattr(self, name).
- __reduce__(...)
- helper for pickle
- __repr__(self, /)
- Return repr(self).
- __setattr__(self, name, value, /)
- Implement setattr(self, name, value).
- __setstate__(...)
- __str__(self, /)
- Return str(self).
- with_traceback(...)
- Exception.with_traceback(tb) --
set self.__traceback__ to tb and return self.
Data descriptors inherited from builtins.BaseException:
- __cause__
- exception cause
- __context__
- exception context
- __dict__
- __suppress_context__
- __traceback__
- args
| |