Adopting the stereo vision algorithm developed by Dr. Saneyoshi, who is a leading researcher on stereo camera for vehicles.
To realize a stereo camera, we needed a high-performance computing environment with high speed processor before. Fujisoft has developed Stereo Vision IP Suite aiming at introducing this technology for embedded devices that have limited processing speed, development time and cost.
We chose an algorithm developed by Dr. Keiji Saneyoshi, and associate Professor of Tokyo Institute of Technology, who is a leading researcher on stereo camera technologies for vehicles. The system calibrates the images with photos taken with two cameras placed in parallel by restoring to 3D shape with epipolar geometry. With this calibration, it can measure the distance to objects with high accuracy.
Using FPGA we realized a customizable system with high-performance, low-power, and compact design. Our Stereo Vision IP suite will make it easy for engineers to apply with a reasonable cost. That will lower research and development workload significantly.
A stereo camera is one of sensors that measures distance as well as infrared and millimeter wave radar. It uses triangulation with stereo camera same as when people see things. Shooting images with two cameras on the left and right produces disparity data. The system can calculate the distance from cameras to the object surface with the disparity data. The distance information can be used to detect unevenness of surrounding area, to measure the actual size of the object and the location. Furthermore, it can accurately identify marks on the road surface such as white line, yellow line, etc., and side walls and road shoulder, which are boundaries of the driving area, which are difficult with infrared and millimeter wave radar. In this way, stereo cameras can be widely applied as an alternative to visual recognition of objects by a human.
With the stereo vision function, the system can measure the distance from the camera to a particular object quickly. It realizes real-time 3D object detection and tracking with the range data.
Images with two cameras on the left and right produces disparity data, followed by pattern matching with a sub-pixel resolution which delivers highly accurate disparity data. Then calculate the distance to the object with it.
With the distance data to the object, the system detects location and size of the object in 3D coordinate.
By using 3D-coordinate data and motion vector data, the algorithm predicts the object’s location in the next following frame, and then the object can be tracked in the next frame accurately.
A typical use case of a stereo camera is Advanced Driving Assistance System (ADAS). Also, Stereo Vision IP Suite is used in various fields such as control of robot arm, crime prevention and security measures.
Automotive makers use the stereo camera for Advanced Driver Assistance System (ADAS) such as automatic braking function to prevent collisions by detecting and predicting pedestrians and obstacles with the in-vehicle camera. It is making a big contribution to the safe and comfortable driving.
In industrial fields such as production line and distribution, a robotic arm can be controlled more accurately and quickly by taking advantage of 3D distance and object data calculated with stereo vision.
With stereo vision camera in a retail store, customer movement data can be collected and analyzed. The results of the data analysis are useful for improving the layout of products.
By combining stereo vision’s 3D data and standard surveillance camera image, it can lower detection errors for crime prevention. Furthermore, detecting entrance into prohibited area can be realized with the 3D data.
The stereo camera can detect objects in 3D same as human eyes. With distortion correction and the exact distance calculated with disparity data, it realizes object detection and object tracking.
After the calibration, first Rectification Distortion Correction function removes distortion of an image which is produced by lens distortion and then corrects location of two cameras.
Stereo matching is the process to extract 3D information from the disparity which is produced by two cameras. Our system adopts SAD (Sum of Absolute Difference) to calculate disparity. SAD obtains accurate stereo matching data, necessary to do pattern matching at sub-pixel resolution.
With the distance data calculated with disparity, it detects objects and calculates its size. Then using 3D-coordinate data and motion vector data, the algorithm predicts and tracks the object’s location in the next following frame.
Due to the effect of temperature, vibration and so on, the position of the camera shifts as time goes by. It affects the accuracy of the disparity information and the distance information. Automatic calibration helps to keep the accurate results.
* This functions for stationary applications such as surveillance cameras, etc.
You can start evaluation immediately with our all-in-one evaluation kit at reasonable cost, which includes Altera® Cyclone® V SoC evaluation board, Stereo Vision IP Suite evaluation version, and 5 megapixel CMOS stereo camera. The kit also includes software for computer and stereo image viewer.
Resolution: 1280 (H) × 720 (V) @ 30 fps
You can choose baseline length from below.
62 mm, 100 mm, 149 mm, 198 mm
Please feel free to contact us.
For urgent issues please contact our Helpline