Autonomous Driving and ADAS

On-vehicle camera in ADAS (Advanced Driving Support System)

In the advanced driving support system (ADAS), the camera is an important device that supports various functions, including parking assistance by assisting the driver's visibility and monitoring the rear image.

In recent years, due to technological innovation related to Image Recognition , it has come to play a sensor function for directly controlling automobiles as well as visual assistance. This page thoroughly explains the role, structure, kind, etc. of ADAS camera about its equipment. We will also introduce camera related products provided by ZMP.

1. On the role of in-vehicle camera in ADAS

Conventional on-board cameras were mainly used as a "view camera" that compensates for blind spots during driving such as a back monitor.
In recent years, the Image Recognition technology has greatly improved, so recognizing vehicles, pedestrians, traffic signs, etc. based on digital image data photographed by a camera, encouraging the driver to warn or give a warning.In some cases, it has become a function as an input sensor for directly controlling a car, that is, as a "sensing camera".

In ADAS, in-vehicle cameras are used for the following functions.

· Function as surrounding visibility assistance system: Around view · monitor, driver confirms the blind spot of the car with video,
. Backward image is displayed on the monitor, parking assistance
· Detect lane markers on road with monocular camera
· Stereo camera detects forward obstacles and predecessors
· Recognition of road signs such as speed indication
· Pedestrian detection · Collision warning
· Driver like drive recorder · Automobile monitoring
· Driver monitoring technology

2. Background and future of the spread of cameras used for ADAS

To evaluate the safety performance of the car in order for the driver to select safer cars is called automobile assessment. There are institutions that carry out assessment in each country. In the United States, it is called NCAP (New Car Assessment Program), in Europe it is called Euro NCAP and in Japan it is called JNCAP. Based on this automobile assessment, the movement to discount automobile insurance etc. is expanding mainly in Europe, and introduction of cameras is expanding among ADAS products.



Due to these system improvements, demand for in-vehicle cameras is expanding rapidly worldwide.

3. On the image sensor (image sensor) of the in-vehicle camera used for ADAS

Typical imaging devices (image sensors) for in-vehicle cameras are CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor). For in-vehicle cameras for automobiles used as monitors, CMOS which is lower in price than CCD is used in many cases.
However, CMOS has many disadvantages such as high noise and distortion at high speed movement, and CCD is said to be suitable for Image Recognition . For example, in "LKAS: Lane departure prevention support system", the image sensor of the camera for recognizing the white line is mainly CCD.

4. Types of cameras for ADAS

In-vehicle cameras used for ADAS are roughly divided into monocular cameras (mono camera) and stereo cameras. In this section, we will explain the features and techniques of monocular cameras and stereo cameras.

4-1. Monocular camera (mono camera)

A monocular camera (mono camera) is a camera that we often see in everyday life. Monocular cameras are the mainstream camera that can be used as a monitor function.
4-1-1. Usage of Monocular Camera in ADAS
In ADAS, monocular cameras are used in the following scenes.
. View camera that compensates for the blind spots behind and sideways.
· Acquisition of image of parking assistance round view monitor
· Lane recognition for lane keeping
· Warning function to the driver by recognizing traffic signs such as speed limit
· Crosswalk recognition
4-1-2. Advantages and Disadvantages of Monocular Camera
In the case of a monocular camera, when distance measurement is performed, it is calculated from the pixel position in the vertical direction of the recognized image coordinates, but the error generally increases. The main advantage is that it can be installed at low cost and its size is small, so the degree of freedom of the installation place is high. Calibration can be done relatively easily. The downside is that the objects to be recognized are limited.
4-1-3. Method of detecting vehicles, pedestrians, traffic signs, etc. using monocular cameras
Vehicles and pedestrians using monocular cameras, traffic signs, etc. are generally detected through the following four processes.
(1) Feature extraction
(2) Object modeling
(3) Learning
(4) Object tracking

For details, please refer to our book "Automobile driving environment recognition technology and its applications".
4-1-4. Future of monocular camera technology
The ADAS function "professional pilot" installed in Nissan's Serena etc. realizes distance measurement with only a single-lens camera. In addition, technology development such as Image Recognition technology using NVIDIA GPU etc. is progressing, and monocular camera recognition technology is expected to further evolve from now on.

4-2. Stereo camera

In addition to ADAS (advanced driving support system), three-dimensional recognition of the running environment of the vehicle is becoming important to realize AD ( Autonomous Driving ). For this reason, it is necessary to accurately extract the depth information of the shooting scene. Stereo cameras can grasp the depth information more accurately by its structure.
4-2-1. Stereo camera principle
A stereo camera is an object that measures objects with two cameras like a human eye, and its parallax makes it possible to measure distances with high institution.

For details on the mechanism and principle of the stereo camera, refer to "On the mechanism and principle of stereo camera used in ADAS (advanced driving support system).

4-2-2. Advantages and Disadvantages of Stereo Camera
From the principle of the stereo camera, it becomes possible to generate not only a stationary object such as a traffic sign or a white line, but also distance information of a moving object.
Although stereo cameras are difficult to calibrate and increase computational complexity, the range of use is advanced due to the development of car electronics technology, that is, in-vehicle ECU and semiconductor innovation.
4-2-3. Example of stereo camera

Examples of practical application of stereo cameras include Subaru's​ ​Eye site​ ​and Daihatsu's​ ​Smart Assist 3​ ​and so on. It is mainly used for preventing collision with a forward vehicle.

5. On-vehicle camera issues and sensor fusion

In-vehicle cameras are principally equipment that obtains optical information on the exterior of the car. Therefore, depending on its characteristics, it is impossible to properly receive the light source during bad weather such as day and night, tunnel entrance and exit, sunshine and shade, morning sun and sunrise, wet road, white line faint, glare, halation, It is more susceptible to disturbance.
Technical innovation such as image sensor, exposure control, and lens barrel is important for optically more accurate external information acquisition, but sensor innovation such as millimeter wave radar and laser radar, combination of sensor and camera, sensor fusion technology has appeared. For example, when realizing the collision avoidance function, the sensor fusion technology combined with the camera and millimeter wave radar can calculate the possibility of collision more accurately, so it is possible to control the brake optimally.

6. ZMP ADAS camera field products

ZMP offers abundant products for camera solutions in the ADAS field.

6-1. Stereo camera RoboVision series

RoboVision 2s: Super high sensitivity stereo camera system compatible with ranging and object detection software

RoboVison 3: A stereo camera capable of sensing distance and field of view up to 150 m, horizontal 100 °

6-2. Image processing software

RoboVision2s object detection option: Software capable of displaying the size and position coordinate data of the detected object in real time

RoboVision2s CarTrack option: Tracking of peripheral objects · Detecting relative speed & Software capable of CAN output

In addition, we are preparing a number of products suitable for development and design of ADAS function in ZMP, please feel free to inquire.

7. ADAS camera related information

Also, links on our company's ADAS camera related press release are be posted below.

8. Inquiry, document request

For inquiries about the above contents and products, please contact us for the information requested from the following.