Basic tutorial 4: Live preview with analytics and overlay

This tutorial will walk you through steps that are required to start the analytics and see the results i.e. get preview, analytics, and overlay working together. This example highlights usage of configure_overlay, set_overlay_state, and set_analytics_state API.

Warning

We assume the correct analytics engine is already installed and for ML based engine you have installed required model and related files. If you are not clear on what is required, please get in touch with QTI’s support team. We also assume that you have already checked Basic tutorial 1: Live preview. If not, please go through it before proceeding as this tutorial only explains the additional code i.e. overlay and analytics configuration and usage.

Code

test-preview-inference-overlay.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
import argparse
import sys
import socket
import time

from sdk.camera import CameraClient

def getWlanIp():
    #if(os.name == "nt") :
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    try:
        # doesn't even have to be reachable
        s.connect(('10.255.255.255', 1))
        IP = s.getsockname()[0]
        if IP.split('.')[0] == '172':
            print("Ip address detected is :: " + IP )
            IP = '127.0.0.1'
            print("Ip address changed to :: " + IP + "to avoid docker interface")
        print("Ip address detected is :: " + IP )
        
    except:
        IP = '127.0.0.1'
    finally:
        s.close()
    return IP

def main(protocol=None):
    print("\nPython %s\n" % sys.version)
    parser = argparse.ArgumentParser()
    parser.add_argument('--ip', help='ip address of the camera', default='127.0.0.1')
    parser.add_argument('--username', help='username of the camera', default='admin')
    parser.add_argument('--password', help='password of the camera', default='admin')
    args = parser.parse_args()
    ip_addr = args.ip
    username = args.username
    password = args.password

    with CameraClient.connect(ip_address=ip_addr, username=username, password=password) as camera_client:

        print(camera_client.configure_preview(resolution="1080P", display_out=1))

        camera_client.set_preview_state("on")

        print(camera_client.preview_url)

        camera_client.set_analytics_state("on")

        print(camera_client.vam_url)

        camera_client.configure_overlay("inference")

        camera_client.set_overlay_state("on")

        try:
            with camera_client.get_inferences() as results:
                print_inferences(results)
        except:
            print("Stopping")

def print_inferences(results=None):
    print("")
    for result in results:
        if result is not None and result.objects is not None and len(result.objects):
            timestamp = result.timestamp
            if timestamp:
                print("timestamp={}".format(timestamp))
            else:
                print("timestamp= " + "None")
            for object in result.objects:
                id = object.id
                print("id={}".format(id))
                label = object.label
                print("label={}".format(label))
                confidence = object.confidence
                print("confidence={}".format(confidence))
                x = object.position.x
                y = object.position.y
                w = object.position.width
                h = object.position.height
                print("Position(x,y,w,h)=({},{},{},{})".format(x, y, w, h))
                print("")
        else:
            print("No results")

if __name__ == '__main__':
    main()

Note

This script is present in the tests/ directory of the SDK.

Walkthrough

As you would have noticed the beginning of the code is the same as the Basic tutorial 1: Live preview. Which means we now have things in place for starting preview both live and on HDMI. We can start the analytics using the set_analytics_state API. And if the request was successful then vam_url will contain the RTSP url where the analytics result will be available.

camera_client.set_analytics_state("on")
print(camera_client.vam_url)

Next, we will configure and start overlay so that we can see the results from the analytics engine when they are available on the preview. Overlay can be configured using the configure_overlay API and it can be turned on using the set_overlay_state API.

camera_client.configure_overlay("inference")
camera_client.set_overlay_state("on")

All is good till now. You have started preview, are able to see the analytics results on screen. Wouldn’t it have been great if you can get access to the result of the analytics engine so that you can take actions based on it, if required? Don’t worry, we have get_inferences API for that which will parse the analytics RTSP stream and give you results. In the above example we print every result.

try:
    with camera_client.get_inferences() as results:
        print_inferences(results)
except:
    print("Stopping")

We print till results are available or an exception occurs.

Please refer to frame for more info on the contents of the result.

The implementation of print_inferences method shows how to extract the parameters from the result obtained from get_inferences API.

def print_inferences(results=None):
    print("")
    for result in results:
        if result is not None and result.objects is not None and len(result.objects):
            timestamp = result.timestamp
            if timestamp:
                print("timestamp={}".format(timestamp))
            else:
                print("timestamp= " + "None")
            for object in result.objects:
                id = object.id
                print("id={}".format(id))
                label = object.label
                print("label={}".format(label))
                confidence = object.confidence
                print("confidence={}".format(confidence))
                x = object.position.x
                y = object.position.y
                w = object.position.width
                h = object.position.height
                print("Position(x,y,w,h)=({},{},{},{})".format(x, y, w, h))
                print("")
        else:
            print("No results")

It’s time to test

Once you have the application/script ready. You can test it by following the steps mentioned in Test section of the Getting started page.

Warning

This example script will not exit until there is an exception. If you wish to stop it you can do so by pressing Ctrl+C key on your keyboard.