Toyota Technological Institute: Smart Vehicle Research Center
Smart Vehicle Research Center of Toyota Technological Institute is promoting R & D on driving assistance systems that are safe for elderly people and beginner drivers and have less burden on the environment. Specifically, the computer detects various objects such as a pedestrian, a vehicle, a road, a white line and information obtained from a sensor such as a TV camera, a laser scanner, a GPS (global positioning system).We are developing technologies that can recognize the running environment.In this research and development, RoboVision 2 is utilized as a tool for developing nighttime Image Recognition technology with cameras.
Study on development of robust stereo camera technology to changes in driving environment
1. Outline of stereo vision
Mr. Mita's is conducting research on distance calculation algorithms that are resistant to various environmental changes such as rain, snow, and nighttime as a method of constructing a robust recognition system in the running environment using RoboVision 2. Professor Mita of Toyota Technological Institute has been introducing RoboVision 2 from the past and is conducting research on sensor fusion technology during running.
Experimental vehicle and sensor configuration
Stereo vision uses measurement of two cameras to measure distances. There are several methods of distance measurement of stereo vision, but in this research we extract features, extract matching and calculate parallax images.
There is also a problem of calculation load in matching, but it calculates it by taking correlation between right and left camera images.
From professor`s research results, it calculates by taking the correlation with the product of the three kinds of correlation patterns of vertical, horizontal, diagonal of the right and left images.
Also, as a matter of the stereo camera, we will use Gradient's information of gradient to fill in information about how to calculate the distance in a texture (feature) -free region such as white wall, night sky and road surface.
In computation, computation speed of about 60 fps is secured by dynamically computing necessary places.
2. Responding to changes in the driving environment
As a change in the driving environment, which can include sunny, rain, snow, other bad weather conditions.
When the driving environment becomes of snow or rain, the wiper is reflected on one of the cameras, the distance cannot be calculated well, snow and rain are calculated as obstacles, calculation of the distance in front becomes unreliable value.
Also, in terms of sunshine conditions, noise occurs even in spaces and the situation may occur where accurate distance cannot be calculated.
Therefore, we calculate the index of reliability when matching the left and right pictures to the existing stereo matching computation as introduced above as a new method and calculate the distance in the area of high reliability .
In the calculation of reliability, reliability in each area of the acquired image is calculated by incorporating the matching cost into the calculation.
In running such as during night time, the CMOS sensor equipped with RoboVision 2 is highly sensitive and is affected by noise at night, so cases where the distance is not accurately calculated due to the influence of noise or cases where the distance such as the road surface is abnormal can occur.
Therefore, we use a median filter to handle smooth processing results.