djmessenger.utils package

Submodules

djmessenger.utils.decorators module

djmessenger.utils.decorators.run_async(wrapped)

function decorator, intended to make “func” run in a separate thread (asynchronously). Returns the created Thread object

E.g.: @run_async def task1():

do_something

@run_async def task2():

do_something_too

t1 = task1() t2 = task2() ... t1.join() t2.join()

djmessenger.utils.decorators.synchronized(wrapped)

A simple decorator to synchronize an instance method. The RLock object is saved into the wrapped function itself

djmessenger.utils.default_routing_policy module

This is a sample routing policy to demonstrate how to define a routing policy.

From the following sample, we can observe that

  1. A policy has a rules and it contains a list of djmessenger.routing.Rule

  2. Each djmessenger.routing.Rule defines

    2.1 type: djmessenger.receiving.ReceivingType indicates what kinds of djmessenger.receiving.Messaging this Rule should be applied upon

    2.2 name: Rule name

    2.3 filters: A list of subclass of djmessenger.filtering.BaseFilter that will be applied to the message, if any of the filters fails, the whole Rule will be skipped

    2.4 handlers: A list of subclass of djmessenger.handling.BaseHandler that will be applied to the message

    2.5 repliers: A list of subclass of djmessenger.replying.CommonReplier that will take a message and reply something back

  3. Filter, Handler and Repliers can take arguments, for example,

class SimpleMessageReplier(CommonReplier):
    custom_obj_map = {
        "localized_string": (LocalizedString, object)
    }
    def __init__(self, localized_string):
        super().__init__()
        self.localized_string = localized_string

djmessenger.replying.SimplyMessageReplier requires an instance of djmessenger.utils.utils.LocalizedString as argument. Therefore, in the policy we can see

{
    "name": "djmessenger.replying.SimpleMessageReplier",
    "args": {
        "localized_string": {
            "base_string": "Thank you for your thumb!!!",
            "translations": {
                "zh_TW": "謝謝您的讚",
            }
        }
    }
},

which indicates that the replier requires an argument and provide the data

djmessenger.utils.filelock module

Cross platform mechanism to acquire and release a file lock. The module uses additional lock file to mark if the file is currently used.

Example: from filelock import FileLock with FileLock(‘path_to_my_file’):

# My operations with this files comes here
class djmessenger.utils.filelock.FileLock(filePath, delay=0.05, timeout=60, strict=False)

Bases: object

Implementation of cross platform file lock mechanism.

acquire()

Acquires a file lock. If file lock has been already created, then waits until the file lock is released. If the file lock is not released and timeout elapsed then either throw Exception or ignore locking a file based on the value of strict parameter.

release()

Releases file lock created by acquire method.

exception djmessenger.utils.filelock.FileLockException

Bases: Exception

Thrown if cannot acquire a file lock.

class djmessenger.utils.filelock.SecureOpen(filePath, openFileFunction=<built-in function open>, strict=False, *openFileArgs, **openFileKwArgs)

Bases: object

Implementation of cross platform secure file writer logic. Use this class for write file operations only. The class is responsible to securely persist the file content into a file. In case we are running on Windows we may fail because the file could be opened by another process. If this is the case, then we have to:

  1. move preliminary known file path to temporary location
  2. delete preliminary known file path
  3. Write the file content to preliminary known file path.
close()

Releases the self.newFileDescriptor and move the file content to preliminary set file location.

static moveFile(srcFile, dstFile)

Moves the source file to destination file. If destination file exists then the file will be overwritten.

@param srcFile: Path to the source file which has to be renamed @type srcFile: str

@param dstFile: Destination file path @type dstFile: str

@raise IOError: If move operation failed to be performed.

open()

Opens the file. Actually it will open self.newFile and will write the content there. We do that to decrease the chance two processes to use the same file at the same time.

static renameFile(srcFile, dstFile, helperFile)

Tries to rename the source file to destination file. If the rename operation failed(perhaps because the destination file is currently in use), then try to: 1/ Rename destination file to helper file in order to release the

destination file

2/ Retry to rename the source file to destination file.

@param srcFile: Path to the source file which has to be renamed @type srcFile: str

@param dstFile: Destination file path @type dstFile: str

@param helperFile: File used for intermediate rename transaction @type helperFile: str

@raise IOError: If rename operation failed to be performed.

djmessenger.utils.geocoding module

djmessenger.utils.geocoding.calculate_distance(latitude_1, longitude_1, latitude_2, longitude_2)

Given 2 sets of coordinates, calculate the distance in kilometers and return an int and discard decimal points

Parameters:
  • latitude_1 (float) – -90 to 90
  • longitude_1 – -180 to 180
  • latitude_2
  • longitude_2
Returns:

the distance between the 2 coordinates in meters

Return type:

float

Raises:

ValueError – if any of the coordinates is invalid

djmessenger.utils.geocoding.calculate_distance_coordinates(coordinate1, coordinate2)
Parameters:
  • coordinate1 (tuple) – (latitude, longitude)
  • coordinate2 (tuple) – (latitude, longitude)
Returns:

the distance between the 2 coordinates in meters

Return type:

float

djmessenger.utils.geocoding.geocoding(location)

geocode a location

Parameters:location – a string
Returns:a tuple of (latitude, longitude)
djmessenger.utils.geocoding.get_geocoder()

djmessenger.utils.serializable module

class djmessenger.utils.serializable.Serializable

Bases: object

Serializable provides an implementation to make an object able to be serialized into json as in dict or in string format recursively.

# Notes

  1. When you subclass another Serializable, think about whether you’d like to call superclass’s constructor. If you did, you need to take care of defining includes and excludes for the attributes defined by superclass

  2. includes and excludes can exist at the same time, but you can’t define the same attribute in both

  3. Always define custom_obj_map, if nothing should be there, put empty dict.

    Otherwise your class might be inheriting it from superclass and you don’t it which might lead to unexpected result

  4. Always define includes and excludes, if nothing should be there, put them as empty tuple, otherwise your class might be inheriting them from superclass and you don’t know it which might lead to unexpected result

# Assumptions

  1. All nested custom objects must implement Serializable

  2. If any attribute is custom object and you need to deserialize it later, you need to define it in custom_obj_map so that we know the exactly class

    and if it is contained in either list or dict

Serializable supports list and dict:

  1. Serialization if the attribute is a list: json() will loop through the list and construct another list of dict by calling json() for each item

  2. Serialization if the attribute is a dict: json() will loop through the dict and construct another dict which key remains exactly the same but calling value’s json()

  3. Deserialization of custom objects and custom_obj_map was configured for attribute name: deserialize() will first identify the attribute type from

    the loaded json dict. If it was a list of custom objects, we de them one by one, put them in a list and assign back to the attribute; if it was a dict, construct another dict where key remains the same but de the value; if it is an object, just de it

# Configurations

## custom_obj_map

custom_obj_map provides a way to properly deserialize arbitrary json data into this class, since this class itself must know what classes its instance variables are, this approach makes sense. You only need to put the instance variable whose type is not primitive. Follow this format:

``` custom_obj_map = {

‘instance variable name’: <class>,

This way we know exactly the class of the instance variable and whether it is a single object or a list of <class>, either way, deserialzation can handle it.

The <class> can be either a class or a string which matches the class name, but the class MUST extends Serializable

## excludes

exclude any attributes that are listed in this tuple.

includes and excludes can exist at the same time but you can’t have the same attribute name listed in both

## includes

only include attributes that are listed in this tuple

includes and excludes can exist at the same time but you can’t have the same attribute name listed in both

## exclude_underscore

exclude any attributes that starts with an underscore

## ordering

If you want to have certain ordering for the class attributes to display
in the result of serialize(), you can define this list.

If ordering was not defined, the order is not guaranteed.

If ordering is defined, the output of serialization will be ordered by it.

You are not required to include all attributes in ordering, but those which are defined in ordering will be put before those which are not

# Quick example:

``` class TestDevice(Serializable):

def __init__(self, id, type):
# id and type are both primitive, so no need to use _obj_map self.id = id self.type = type
class TestUser(Serializable):
_obj_map = {
‘devices’: [‘TestDevice’, object]

} excludes = [‘password’] exclude_underscore = True

def __init__(self, username, password, devices=[], log_location=””):
super(TestUser, self).__init__(log_location) self.username = username self.password = password self.devices = devices
def add_device(self, device):
if isinstance(device, TestDevice):
self.devices.append(device)

```

custom_obj_map = {}
classmethod deserialize(json_data)
Provide default implementation of how to deserialize json representation
to python object.

If you have any instance variables that are custom objects and you did not define them in custom_obj_map, you need to override deserialize() otherwise that instance variable will be deserialized to a dict instead of the object itself.

@param json_data: a dict which represents a json object; or a string
which is a valid json format

@type json_data: dict, str

@return: an instance of this class

exclude_underscode = True
excludes = ()
includes = ()
json()

Returns an OrderedDict representation of the object, after applying includes and excludes policies

This method takes ordering into account

@return: a dict represents this object recursively @rtype: OrderedDict

json_unordered()

This method does not consider ordering

@return:

ordering = []
classmethod read_json(json_file)

Given an absolute file path, try to decode it as json representation to an instance of this class

@param json_file: absolute file path for the .json file @type json_file: str

@return: an instance of this class @raise ValueError: if the json representation can not be deserialized @raise IOError: if the file can not be opened @raise ValueError: if not able to deserialize the json data into cls

serialize(pretty=False)

Serialize the object into a valid json-formatted string after applying includes and excludes policies

@return: json representation for this instance @rtype: str

write_json(destination)

Securely opens the log_location and dumps json from self to it, this method is synchronized

@raise ValueError if log_location was not set @raise IOError if log_location can not be opened

class djmessenger.utils.serializable.SerializableEnum(name, description='')

Bases: djmessenger.utils.serializable.Serializable

An Enum-like class, which serializes to a simple string. In json serialization, this will be treated just like a string, instead of an object

Usage:

>>> class Severity(SerializableEnum):
...    pass
>>> Severity.INFO = Severity('INFO', 'INFO')
>>> Severity.WARNING = Severity('WARNING', 'WARNING')
>>> Severity.ERROR = Severity('ERROR', 'ERROR')
>>> Severity.INFO.name == 'INFO'
True
>>> Severity.value_of('info') == Severity.INFO
True
>>> Severity.value_of('ERROR') == Severity.ERROR
True
>>> Severity.value_of('ERROR') == Severity.WARNING
False
>>> test = Severity.value_of('TEST')
Traceback (most recent call last):
...
KeyError: 'name TEST not defined for Severity'
>>> Severity.INFO.serialize() == 'INFO'
True
classmethod deserialize(json_data)

@param json_data: this must be a single string @return: An instance of Status @rtype: Status

classmethod get_members()

@return: A map maps from enum name to the actual enum instance

json()
serialize(pretty=False)
classmethod value_of(name)

Given enum name, find and return the corresponding enum object

@param name: A string for the status code @param name: str

@return: An instance of Status @rtype: Status @raise KeyError: if name is not defined for the enum

djmessenger.utils.utils module

class djmessenger.utils.utils.LocalizedString(base_string, translations)

Bases: djmessenger.utils.serializable.Serializable

We are using this LocalizedString class to implement i18n support, so every string that needs to be localized(including variables in settings), needs to be able to deserialize back to this class

The json format is

{
  "base_string": "default string, in any language, if locale not found, then use this",
  "translations": {
    "en_US": "English translation",
    "zh_TW": "Traditional Chinese translation",
    ...
  }
}

The key in translations dict is locale defined by Facebook and can be found in facebook_locales.py

get_text_by_locale(locale)
Parameters:locale (FacebookLocale) –
Returns:
djmessenger.utils.utils.get_class_name(obj)

Given a type of an instance, return its fully qualified class name

Parameters:obj (type or object) –
Returns:fully qualified class name
Return type:str
djmessenger.utils.utils.load_class(fully_qualified_name)

Load class by its fully qualified name

Parameters:

fully_qualified_name – if given fully qualified name is a.b.c, try to load the module from a.b and load the class c in a.b module

Returns:

A class

Raises:
  • ImportError – not able to load module
  • AttributeError – not able to load class

Module contents