Creating an OpenCV module for IoT Edge

Recently, I had a need to do image classification on the Edge using Azure IoT Edge.

To start, there are 2 challenges:

  1. No image detection module
  2. No image classification module

No problem, Azure IoT Edge support many languages, including Python, and with OpenCV we can connect to RTSP cameras and classify images.

This blog walks you through the process of setting up the image / video ingestion module.  Here is the image classification Machine Learning module .  It assumes you have Azure IoT Edge running.  New to IoT Edge?  Start here: https://docs.microsoft.com/en-us/azure/iot-edge/tutorial-python-module

Our Architecture will look something like this:

Drawing1

Note how openCV is a module used for image (or video stream) ingestion.  We later score or interact with the images.  Also note how the openCV module allows the Machine Learning module to fetch the image for scoring and processing.  This interaction allows for images or streams of any size to be scored and overcomes any message limitations.

To get started, open a new folder in Visual Studio Code and select View –> Command Palette

1

Select Edge: New IoT Edge Solution

2

Name your solution

3

Select Python Module Language

4

Name your Module

5

Select your Container Registry

6

Open the file main.py and replace the contents from: https://github.com/ksaye/IoTDemonstrations/blob/master/IoTEdgeOpenCV/main.py.  It is worth looking through the code to really understand what is happening.  Close and save this file.

Open the Dockerfile and replace the contents from: https://github.com/ksaye/IoTDemonstrations/blob/master/IoTEdgeOpenCV/Dockerfile note this file actually downloads opencv and builds the binaries.  Close and save this file.

Right click on module.json and click Build and Push IoT Edge Module Image

7

Select amd64 as your platform.  As of this writing, the Python module is only supported on 64 bit Linux.

8

Assuming you have login credentials to your container registry, it should upload and you can view the image via the Azure Portal

9

Next we will configure an IoT deployment, with settings needed.  Note, you can add other modules and routes later, which are not the focus of this blog.

In the Azure Portal, goto an existing IoT Edge device.  Select Set Modules

10

Select Add IoT Edge Module

11

At the module dialog, name the module, type your URI and (optionally) include the container create options that allow remote viewing of the images and include the desired properties of a valid RTSP URL as shown below.  You can copy and past the following JSON to make it simpler.

12

You can copy and past the following JSON to make it simpler.

{ 
	"ExposedPorts": {
		"8080/tcp": {}
	},
	"HostConfig": {
		"PortBindings": {
			"8080/tcp": [
				{
					"HostPort": "8080"
				}
			]
		}
	}
}

{ 
	"properties.desired": {
 		"imageProcessingInterval": 15,
 		"camaraArray" : {
 			"MyCamera": "rtsp://myUser:myPassword@192.168.15.253:88/videoMain"
 		}
 	}
}

Note, the JSON above assumes you are using a remote camera.  If you are using a local camera on the Linux host, you need to use the following JSON as the create options to give access to the /dev/video0 device:

{
  "ExposedPorts": {
    "8080/tcp": {}
  },
  "HostConfig": {
    "Privileged": true, 
    "Binds": [
      "/dev/video0:/dev/video0"
    ],
    "PortBindings": {
      "8080/tcp": [
        {
          "HostPort": "8080"
        }
      ]
    }
  }
}

Click Save and Next for the default route.

Finally click submit for to submit the task.

Assuming no typos, you should be able to type “docker ps” to see the image running and that it is visible from port 8080.

13

Viewing the docker logs (docker logs opencv) you will see the following logs in the opencv module:

14

You can also web browse to the docker container to see the image:

15

Lastly, you can see the message from Device Explorer:

16

Note we see the image size, name, URL, and changes from the prior image.  To understand the change methods, look to: https://stackoverflow.com/questions/189943/how-can-i-quantify-difference-between-two-images

Putting it all together and using Azure Stream Analytics + Power BI, we can see real-time the changes happening:

19

Also, looking at it from a hardware perspective, without any GPU’s it can consume CPU and, as shown below:

20

This is running on a 4 core “Intel(R) Core(TM) i5-2467M CPU @ 1.60GHz” machine.

Advertisements

One thought on “Creating an OpenCV module for IoT Edge

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s