New AI Features Notes

New Features

From Radu Email 3/20/2018

Control Center 3.5.5

Hello all, In a few days we hope to make an internal release for our new amazing firmware...There are a lot of new cool stuffs and I think that you guys (especially US team) should be familiarized in advanced with them. In this email I would like to show you how these new features are reflected in Control Center.

You can find a new CC (3.5.5) in Idrive Downloads section. There is also a new Idrive Player (3.5.10) able to play a new format of our existing events + 3 new types of events: - face recognized event (extension .137) - face unrecognized event (.136) - safe distance warning event (.135)

Facial Recognition

As you can easily deduce, the face recognized event is trigger when a driver is recognized by camera, a face unrecognized event is triggered when the camera is unable to recognize the driver for an amount of time and the safe distance warning event is triggered when the vehicle is following too close other vehicle (the front vehicle). However, the face unrecognized event is not implemented yet in the firmware and it will be available in a future version.

Below is a small description about how the facial recognition is implemented from the Control Center's point of view. In a future email I will explain in detail how the camera is making the recognition (with some help from Idrive Cloud).

From the CC perspective the facial recognition is very similar with the driver key identification. The corespondent of the driver key is a global unique identification field (I will name it "guid") automatically generated by CC when a driver is added in the system (the existing drivers have this field). As you can imagine, this field is synced together with the other driver's fields, so Idrive Cloud (this is a better term for Global Center or GC) also knows the guid for each driver.

If you want to learn more about guid (or uuid), take a look at: https://en.wikipedia.org/wiki/Universally_unique_identifier

During the facial recognition process, the Idrive Cloud is sending the guid of the recognized driver to the camera. The camera is triggering the "face recognized" event and it is also placing the guid in all next events and in the gps file (just as it does with the driver key).

The recognition process in CC is also similar with the driver key identification: during the transfer, the CC is searching the guid field in all events / gps files and placing the recognized driver, like below:

Events & Reviews:

(the yellow icon notify the user that the driver has been identified during the facial recognition process).

Gps data and the map:

There is also a small change in Fleet Manager (see the driving side field from the image below):

The driving side field is uploaded on the camera during the Wi-Fi transfer and it is used by the camera in order to identify the position of the driver. "Middle" option should be used for inter-modal clients when only one person can appear in the image (usually a small vehicle or a cabin).

And now the cool stuff: Take a look at the attached event. You have 2 options to play it:

- with Idrive Player (not recommended): when the event is loaded, click on the video->click on "Face recognition info"

- with CC (recommended): make sure that your location belongs to Idrive Inc Company (not Demo Idrive or other), use Import Viewer->Browse and Import immediately; go to Events&Reviews section, play the event and click on the face pictogram (see below)

PArt 2

From Radu Email 3/23/2018

Hello guys, It is time to share with you the "Part 2" information of the facial recognition feature. In Part 1 I have explained how this feature is implemented in CC. I will explain now how this feature works in firmware...

===Driver Face Detection

Driver face detection The detection of the driver face is based on the "driving side" field from CC->Fleet Manager->Edit Vehicle

- Left driving side (considered default). Camera will perform a face detection over the whole image and will analyze the most right face. That face is considered driver face if the median of the face is placed in the right portion of image (see below)

- Right driving side. This is an opposite case and the camera will analyze the most left face from the image. On the above example the second image will be considered the valid driver face. - Middle: This is a particular case when the camera will consider the first detected face as the driver face.

Face recognition The face recognition process requires a cooperation from the fleet managers until the system learns all drivers faces but is does NOT require cooperation from the drivers. Like in the image below, the camera will upload the detected face in Idrive Cloud. During a session (Ignition On / Ignition Off), the camera will upload 3 faces in cloud.

The managers will have an interface in the new Global Center where they will be able to assign each face to a driver:

At the next request, the cloud will be able to recognize the driver. The camera will not upload the face anymore; instead will add the detected face in the camera storage together with the driver information obtained from cloud.

At the next ignition ON, the camera will try to recognize itself the driver by using the faces from the local storage (Idrive Cloud will become the second option).

Based on the above mechanism, each camera will be able to recognize all persons that are usually driving the car (in different conditions) and the cloud will not be used anymore. From some technical reasons, the first firmware release will not include the recognition on the camera (only the recognition on the cloud will be performed). However, the system has been successfully tested with all people from Idrive RO in both environment (local + cloud). I expect less drivers in a car than people in RO, so the camera should be fine to perform recognition itself.

The face recognition will be performed in the following circumstances: - At Ignition ON - When someone will make a request from LTI Interface - When the speed becomes higher than a threshold and the vehicle was parking (with engine ON) for an amount of time (this feature is not implemented yet)

When the driver is recognized, camera will trigger a "face recognized" event (see previous email) and the LTI will show the driver in the LTI interface.

Licenses The facial recognition process is using a third part component provided by http://www.neurotechnology.com/ These guys are the only company that are providing APIs usable for our camera. They have also a good reputation http://www.neurotechnology.com/cgi-bin/customers.cgi

The facial recognition in Idrive Cloud involves a face recognition matching server (http://www.neurotechnology.com/cgi-bin/biometric-components.cgi?ref=vl&component=NServer) This server is already installed in one of our Amazon machines and does not require a future license.

For enabling the face recognition feature on our camera, we have to use 2 licenses on every camera where we want to use this feature (ONLY column 2 + 3).

http://www.neurotechnology.com/prices-verilook.html A license is a small text (400-500 bytes) like below:

and these guys are sending the licenses via email.

We have developed a license manager in Admin Center. I'm not sure if has been released yet but Stefan will give you more information about it soon... In order to use the Face Recognition APIs, camera will need a pair license (one face extractor + one face matcher). These licenses will be uploaded on camera via 3G (by Server) or via Wi-Fi (by CC).

That's it !

Regards, Radu