CityIQ API¶
This module and command line tool provides access to the CityIQ API, with particular focus on getting metadata and parking events. The interface includes a basic access object for getting metadata and events, a scraper object for mass downloading events, and a command line tool for downloading and processing events.
Configuration¶
The CityIq module and programs require a configuration file that hold credentials and urls. You can generate a default configuration with
ciq_config -w
The generated file is configured for the San Diego system. To you this system , you will just need to add your client id and secret to the file.
The Config
object can be constructed can constructed on a path where the config file is location. If none
is specified it will look for this file in several places, in this order:
- The path specified in the constructor
- The path specified by the
CITYIQ_CONFIG
env var .city-iq.yaml
in the current dircity-iq.yaml
in the current dir.city-iq.yaml
in the user’s home dir
Each of the configuration files can be overridden with a keywork in the Config
object constructor, and each value
can be accessed as an attribute or an index:
# Load from well-known file and override cache_dir
c = Config(cache_dir='/tmp')
print(c.cache_dir)
print(c['cache_dir'])
Basic Use¶
API objects are the primary way to get access to assets and events. CityIq
is the top level access
object. The API offers access to Locations, Events and Events.
Typically, you will construct CityIq
from a Config
. If a configuration is not specific,
the system will look for the file in default locations. You can also override individual configuration parameters
with keyword arguments to the constructor.
Metadata Access¶
Metadata, for both locations and assets, can be fetched with property accessors. The bounding box for the queries can
be set in the configuration, or on the CityIq
constructor.
The asset metadata properties are:
CityIq.assets
: All assetsCityIq.nodes
: Nodes, the parents for other assets on a poleCityIq.cameras
: All assetsCityIq.em_sensors
: ?CityIq.env_sensors
: Environmental sensors
The location metadata properties are:
CityIq.locations
: All locationsCityIq.walkways
:CityIq.parking_zones
:CityIq.traffic_lane
:
Events can be fetched with cityiq.api.CityIq.events()
Each of these acessor properties or functions returns a generator that generates objects of a specific type, one base class for each of Locations, Assets or Events:
bbox = '32.718987:-117.174244,32.707356:-117.154850'
c = CityIq(bbox=bbox) # Use default config, override bbox
# Get Locations
locations = list(c.locations)
# Get the assets at this location:
for location in locations:
do_something_with(location.assets
Scraping¶
The ciq_events program is used to scrape and process events. To properly run the program, two directories should be specified in the configuration:
events_cache
: The directory where hourly event fiels will be specifiedcache_dir
: The directory where processed event files will be written.
To scrape events from the API, run ciq_events --scrape --start_time
. If <isotime>
is omitted, the program will start from the
start_time
specified in the config. This will download events from the
start time in hourly batches, and save one file per hour.
To process events, first break up the hourly event files by location with
ciq_events --split
command. This will create one CSV file per
location per month. Then, re-combine and renormalize the data with
ciq_events --normalize
, which will write a CSV file to the local directory.
The final output file will have columns for delta, which is the number of cars that went into or out of a parking zone per 15 minute interval. However, there are a lot of suprious events, so the delta_norm has a normalized value that tries to remove the spurious events.
These programs can produce a lot of data. For the San Diego system, the extracted PKIN and PKOUT events for September 2018 through Feb 2019 is 21GB, and the download process takes several days. The final processed CSV file, with records at 15 minute intervals, is about 81MB and akes about an hour to process.
For instance:
$ ciq_events -s -e PKIN -e PKOUT -t 20190901
$ ciq_events -S
$ ciq_events -n