View source for New AI Features Notes

==New Control Center Features==

From Radu Email 3/20/2018

<br><br>

===Control Center 3.5.5===

Hello all,
In a few days we hope to make an internal release for our new amazing firmware...There are a lot of new cool stuffs and I think that you guys (especially US team) should be familiarized in advanced with them. In this email I would like to show you how these new features are reflected in Control Center. 

'''CC (3.5.5)or higher required''' in Idrive Downloads section. 

also a new Idrive Player (3.5.10) to play a new format of our existing events + 3 new types of events: 

====New Event Types:====

::* face recognized event (extension .137) (the face recognized event is trigger when a driver is recognized by camera)

::* face unrecognized event (.136) (face unrecognized event is triggered when the camera is unable to recognize the driver for an amount of time)

:::<span style="color:#ff0000">(the face unrecognized event is  not implemented yet in the firmware and it will be available in a future version)

::* safe distance warning event (.135) (the safe distance warning event is triggered when the vehicle is following too close other vehicle (the front vehicle))

<br><br>

==Facial Recognition==

<br><br>

===Facial Recognition - Control Center===

Below is a small description about how the facial recognition is implemented from the Control Center's point of view. In a future email I will explain in detail how the camera is making the recognition (with some help from Idrive Cloud). 

From the CC perspective the facial recognition is very similar with the driver key identification. The corespondent of the driver key is a global unique identification field (I will name it "guid") automatically generated by CC when a driver is added in the system (the existing drivers have this field). As you can imagine, this field is synced together with the other driver's fields, so Idrive Cloud (this is a better term for Global Center or GC) also knows the guid for each driver.
 
If you want to learn more about guid (or uuid), take a look at: https://en.wikipedia.org/wiki/Universally_unique_identifier

During the facial recognition process, the Idrive Cloud is sending the guid of the recognized driver to the camera. The camera is triggering the "face recognized" event and it is also placing the guid in all next events and in the gps file (just as it does with the driver key).

The recognition process in CC is also similar with the driver key identification: during the transfer, the CC is searching the guid field in all events / gps files and placing the recognized driver, like below:

Events & Reviews:

[[File:picture goes here.jpg|left|400px|]]
 
(the yellow icon notify the user that the driver has been identified during the facial recognition process). 

<br><br>

Gps data and the map:

[[File:picture goes here.jpg|left|400px|]]

[[File:picture goes here.jpg|left|400px|]]
 
<br><br>
 

There is also a small change in Fleet Manager (see the driving side field from the image below):

[[File:picture goes here.jpg|left|400px|]]
 
The driving side field is uploaded on the camera during the Wi-Fi transfer and it is used by the camera in order to identify the position of the driver. "Middle" option should be used for inter-modal clients when only one person can appear in the image (usually a small vehicle or a cabin).

<br><br>

And now the cool stuff:

====Play Event====
Take a look at the attached event. You have 2 options to play it:

- with Idrive Player (not recommended): when the event is loaded, click on the video->click on "Face recognition info"

- with CC (recommended): make sure that your location belongs to Idrive Inc Company (not Demo Idrive or other), use Import Viewer->Browse and Import immediately; go to Events&Reviews section, play the event and click on the face pictogram (see below)

[[File:picture goes here.jpg|left|400px|]]
 
<br><br>

===Facial Recognition - Firmware Functionality===

'''The face recognition process requires a cooperation from the fleet managers until the system learns all drivers faces but is does NOT require cooperation from the drivers.'''

====Driver Face Detection====


=====Driving Side Settings=====

The detection of the driver face is based on the '''"driving side"''' field from CC->Fleet Manager->Edit Vehicle

[[File:driver side pulldown.jpg|left|400px|]]

<br><br><br><br><br><br><br><br><br><br><br><br><br>
 
'''Left driving side''' (considered default). Camera will perform a face detection over the whole image and will analyze the most right face. That face is considered driver face if the median of the face is placed in the right portion of image (see below)

[[File:valid Driver face.jpg|left|400px|]][[File:invalid Driver face.jpg|400px|]]

<br><br><br><br><br><br><br><br><br><br><br><br><br>
                                                                                                                                      
'''Right driving side'''. This is an opposite case and the camera will analyze the most left face from the image. On the above example the second image will be considered the valid driver face.

'''Middle''' In this particular case  the camera will consider the first detected face as the driver face.


<br><br>

=====Triggering Facial Recognition=====

The face recognition will be performed in the following circumstances:

::*At Ignition ON

::*When someone will make a request from LTI Interface

::*When the speed becomes higher than a threshold and the vehicle was parking (with engine ON) for an amount of time (this feature is not implemented yet)

<br><br>

When the driver is recognized, camera will trigger a "face recognized" event (see previous email) and the LTI will show the driver in the LTI interface.

<br><br>


=====Face image captured and uploaded to cloud=====

During a session (Ignition On / Ignition Off) the camera will upload 3 detected face images to the in Idrive Cloud (GC).

[[File:save face to cloud.jpg|left|400px|]]


<br>

===Facial Recognition - Global Center Interface===
 
The managers will have an interface in the new Global Center where they will be able to assign each face to a driver:

[[File:facial driver assignment.jpg|left|400px|]]

[[File:facial driver assignment2.jpg|left|400px|]]


<br><br>
                                                    

[[File:facial recognized driver.jpg|left|400px|]]


At the next request, the cloud will be able to recognize the driver. The camera will not upload the face anymore; instead will add the detected face in the camera storage together with the driver information obtained from cloud.

<br><br><br><br>

[[File:facial recognized on camera.jpg|left|400px|]]

At the next ignition ON, the camera will try to recognize itself the driver by using the faces from the local storage (Idrive Cloud will become the second option). Based on this mechanism, each camera will be able to recognize all persons that are usually driving the car (in different conditions) and the cloud will not be used anymore. 

<br><br>

<span style="color:#ff0000">From some technical reasons, the first firmware release will not include the recognition on the camera (only the recognition on the cloud will be performed).  However, the system has been successfully tested with all people from Idrive RO in both environment (local + cloud). I expect less drivers in a car than people in RO, so the camera should be fine to perform recognition itself. 



===Licenses===

<br><br>

====Individual License Provider====

The facial recognition process is using a third part component provided by http://www.neurotechnology.com

These guys are the only company that are providing APIs usable for our camera. They have also a good reputation http://www.neurotechnology.com/cgi-bin/customers.cgi

<br><br>

====Server License Provider====

The facial recognition in Idrive Cloud involves a face recognition matching server (http://www.neurotechnology.com/cgi-bin/biometric-components.cgi?ref=vl&component=NServer)

This server is already installed in one of our Amazon machines and does not require a future license.

<br><br>

====Enabling Facial Recognition on our Camera====

[[File:facial licensing pricing.jpg|left|400px|]]

we have to use 2 licenses on every camera where we want to use this feature (ONLY column 2 + 3).
 
http://www.neurotechnology.com/prices-verilook.html

[[File:facial sample license.jpg|left|400px|]]

A license is a small text (400-500 bytes) like below:
 
and these guys are sending the licenses via email. 

<br><br>

====License Management====

We have developed a license manager in Admin Center. Settings > Licenses

https://admincenter.idriveglobal.com/?Module=Support&Action=Licences&trace=Settings

I'm not sure if has been released yet but Stefan will give you more information about it soon...
In order to use the Face Recognition APIs, camera will need a pair license (one face extractor + one face matcher). These licenses will be uploaded on camera via 3G (by Server) or via Wi-Fi (by CC).

That's it !

Regards,
Radu

==Safe Distance Warning==

===From Vlad on 3/21/2018===

<br><br>

Hi guys,
  
SDW (safe distance warning) is a feature that will be available on the new firmware. 
The applicability goes beyond having another event type, soon the reports algorithm will take into considerations SDW events for calculating scores
We are using computer vision and artificial intelligence techniques to make this feature available directly on our X2 camera device (without any extra external add-ons devices or cloud processing)


====Quick Description==== 
The main steps of how this feature works are:

1.	SDW algorithm is scanning the road ahead and is detecting the locations of other cars

2.	Due to the calibration procedure we are accurately detecting which of the detected cars are in a frontal position related to your car (on the same lane) and filter out cars placed on other lanes.

3.	We approximate the distance to the closest car in front of you into 4 categories: extremely close frontal vehicle, close frontal vehicle, midrange frontal vehicle, far-mid range frontal vehicle.

4.	Knowing your car current speed (from GPS) and distance to the closest car in front we can determine if you are keeping a safe distance to the car in front or not.

<br><br>
 
====Trigger Conditions====

Most of the data regarding speed and safe distance in the literature are highly exaggerated to be applied in a real traffic environment (most of them consider that the vehicle in front will suddenly freeze and calculate the distance that you have to keep considering your speed so that your braking time will be enough (driver reaction time + maximum braking distance). Also, there can be taken into account many car characteristics like cars mass, braking system, tire type, tire conditions, state of the road (icy, wet, dry, etc.). In the future we can implement a system that will use multiple parameters to calculate the speed-distance condition.  Now we are using a trigger system based on real-life tests. The beta release of the firmware will allow us to improve the distance-speed conditions so we can optimize the trigger threshold.

[[File:picture goes here.jpg|left|400px|]]

<br><br>
 
Distance class Category	Distance in m and feet	Speed threshold on your car for making a SDW event
Extremely close frontal vehicle 	0-3 m  (0 feet -9 feet and 10)	 40 km / h (24.8 mph)
Close frontal vehicle 	3-6 m (9 feet and 10 - 19 feet and 8)	 60 km / h (37.2 mph)
Midrange frontal vehicle 	6-10 m (19 feet and 8- 32 feet and 9)	 80 km / h  (49.7 mph)
Far-mid range frontal vehicle 	10-15 m (32 feet and 9 - 49 feet and 2)	 100 km / h (62.1 mph)
 
<br><br>
 
 
====Calibration Procedure==== 
 
The '''calibration procedure is mandatory (!!!)''' in order to have this feature available. 
Because the X2 camera can be mounted in any place on the windshield or even at different inclination angles and each car has its own characteristics (height, windshield inclination, windshield-front of the car distance, etc.), the designed algorithm will need to be calibrated to that specifically position of the camera mounted on car. 
After the calibration is done, the X2 camera should not be moved from that specific location and orientation. 
If the movement of the camera is necessary (RMA, moving the camera to another car, etc..) the calibration will need to be done again.
Our calibration system allows us to do a very quickly and non-invasive calibration comparative with the other companies that are having this feature. Want we need for calibration is that the car that we want to be calibrated to be put between two traffic lanes (lanes width aprox 3.7 m (12 feet)). 
Considering that it will be a nightmare to put the cars in a road in order to calibrate them or to start drawing lanes in garages or parking lots, we are designing a roll-up carpet that is simulating the lanes. The only thing necessary is to unfold the roll-up carpet and put it in front of the vehicle, wherever the vehicle is placed. More details about this roll-up carpet will be given as soon as we will have some prototypes.

====Calibration Interface====
There are a few easy steps which need to be adhered to in order to successfully calibrate a device:

1.Open the new Globalcenter 3 interface, on the Configure tab;

2.Click on the Safe Distance Warning Calibration ‘Enter here’ button

[[File:picture goes here.jpg|left|400px|]]
 

 
3.Select the location and the vehicle to be calibrated. Only compatible online vehicles will be shown in the list. (You can alternatively scan the barcode on the device or upload a picture of the barcode to identify the device – useful on mobile devices if the client has a lot of online devices)

[[File:picture goes here.jpg|left|400px|]]
 
 
4. Click on ‘Start calibration’. A photo will be fetched real-time from the device, showing the guidelines (the image is from our calibration studio improvised in our garage 😊 , you can notice the 2 white lanes)
 
[[File:picture goes here.jpg|left|400px|]]
 
5. Click on ‘Start’ and place the calibration points as instructed (top and bottom of each visible line, they are 4 total points, the left lane first visible starting  point (lower) and last point (upper) and the right lane first visible starting point(lower) and the last one (upper))

[[File:picture goes here.jpg|left|400px|]]
 
 
6. After placing the 4 interest points, the vanishing point triangle will be shown in green.
 
[[File:picture goes here.jpg|left|400px|]]

<br><br>

You can zoom in on the picture using the mouse wheel (or pinch-zoom on mobile) to adjust the position of the points.
If there are errors, the triangle will be red and the submit button will be disabled.
 
[[File:picture goes here.jpg|left|400px|]]

<br><br>
 
 
7. If the vanishing point triangle is green, a valid configuration can be sent to the device (via the ‘Submit’ button). Configuration will be sent and applied on the device in real-time.

[[File:picture goes here.jpg|left|400px|]]

<br><br> 

===How to view the SDW events===

The SDW events are having a .135 extension.

For this kind of events in the Control Center and also on the IdrivePlayer when you right click on the image you will have a Safe Distance Warning  info box. 

[[File:picture goes here.jpg|left|400px|]]

<br><br>

When you click on it, it will open a pop-up image with the event details. With the red circle is marked the car that you were too close to, with green are projected the lanes in front of your car. Also in the top left corner will be information’s like the distance to the other car, your speed and the detection coefficient (the graphical interface will be improved).

[[File:picture goes here.jpg|left|400px|]]

<br><br>
 
I will come back with information’s and updates regarding the speed-distance threshold and the calibration. 
I attached a SDW event to this e-mail (the speeds triggers  were custom made very low for tests) and also a pdf file with the text written here

Vlad

<br><br>