Computer Vision or Visual Recognition is part of cognitive computing (CC) aka Artificial Intelligence. One of the main concepts is to extract information out of unstructured data. For example you have a webcam pointing on a highway. As a human you see if there is a traffic jam or not. For a computer it’s only 640x480x3x8 (7.372.800) bit. Visual Recognition helps you to extract information out of this data. For example “This is a highway”. Out of the box systems like Watson are able to give you information what do you see on the picture. You can try it here https://visual-recognition-demo.mybluemix.net. The result can be seen on the left picture. So Watson knows it is a highway and even it’s a divided highway but it does not tell you there is a traffic jam or even a blocked road. Fortunately Watson is always eager to learn, let us see how we can teach him what is a traffic jam. This article only focuses on the picture preparation part not the train Watson part. See next postings for the Watson part.
There are many traffic cameras all around but I am not sure about the licence, so it is hard to use it here as a demo. But let us assume we can take pictures like this one from Wikimedia: Cars in I-70.If you live in south Germany there are nice traffic cameras from Strassenverkehrszentrale BaWue. Unfortunately they don’t offer the pictures with the right licence for my blog. If you know a great source for traffic cameras with the right licence please let me know.
Prepare pictures for training
Visual Recognition works a little bit like magic. You give watson 100 pictures of a traffic jam and 100 without traffic jam and he learns the difference. But how do we make sure he really learns traffic jam and not the weather or the light conditions. And furthermore only one lane in case the camera shows both lanes? So first we need to make sure we find enough different pictures of the road with traffic jam under different weather and light conditions. The second part can be done with OpenCV. OpenCV stands for open computer vision and helps you to manipulate images. The idea is to mask out parts we don’t want Watson to learn. In our case the second part of the lane and the sky. We can use GIMP to create a mask we can apply with openCV automatically to each picture.
First step is obvious to load the image in GIMP. Then open the layers dialog. It’s located under Windows/Dockable Dialogs/Layers or cmd-L. Here we add a new layer and select this one to paint on. Then we select in the tools menu the Paintbrush Tool and just paint the parts black we don’t want Watson to learn.
Then we hide the original image by pressing the eye symbol in the layer dialog. This should leave us with only the black painting we did before. This will be our mask for openCV to be applied to all pictures. Under File/Export you can save it as mask.jpg. Make sure it is only the black mask and not the picture with the black painting.
Use openCV in docker
As openCV is quite a lot to install, we could easily use it within docker to work with our pictures. We can mount host directories inside a docker container, so in this case our directory with pictures:
docker run --name opencv --rm -it -v $(pwd):/host victorhcm/opencv /bin/bash
This brings up the openCV docker container from victorhcm and opens a shell with our current directory mounted under /host. As soon es you exit the container it will be removed because of the “–rm” parameter. Don’t worry only the docker container will be deleted, everything under /host is mounted from the host system and will remain. Everything you save in other directories will be deleted.
How to mask out part of the picture
The python program to use openCV to mask out all pictures in a directory is then really easy to use:
import cv2 import glob mask = cv2.imread("mask.jpg", cv2.IMREAD_GRAYSCALE) for fullname in glob.glob("pics/*.jpg"): filename = fullname.split('/')[-1] image = cv2.imread(fullname, cv2.IMREAD_COLOR) dst = cv2.bitwise_and(image, image, mask=mask) cv2.imwrite("masked/" + filename, dst) print filename
Basically the program iterate through all “jpg” pictures in the subfolder “pics” and saves the masked pictures with the same name in the “masked” folder. Both directories have to exists before you start the script. In order to keep the script reduced to the important parts I left the create and check directory part out of this script.
Loads the mask images as a grayscale image.
Loads the image to work on as a colour image.
Here is the real work done, this applies the mask with bitwise add of all pixels. Therefore the blank will win and the transparent will let the normal picture gets through.
Saves the new masked picture in the “maksed” folder.
For the learning process we need to sort the pictures by hand. One bucked with traffic jam and the other with ok.