The Ibeo.Reference system allows our customers to evaluate and improve the performance of their ADAS/AD sensors and their AD functions. The Validation Toolchain from Ibeo enables the efficiency of the sensor or sensor fusion to be determined precisely. The core function of this toolchain is the comparison of the sensor’s perception results with reference objects. The strengths and weaknesses of the sensor can thus be evaluated, which in turn means that the customer can improve specific aspects of the sensor. The Ibeo.Reference system is also used in development to test various algorithmic approaches, compare parameters and evaluate the effect of each modification throughout the development period on performance.
The Validation Toolchain begins by recording situations that are relevant to the sensor which is currently under development. By means of Ibeo recording software, these scenarios are recorded by the Device under Test (DuT) and reference sensors. Ibeo.Reference offers various solutions for generating the raw data for subsequent reference objects with 2D and 3D sensors.
With the Evaluation Suite (EVS), Ibeo.Reference provides a software solution which automatically generates the ground truth data required for postprocessing.
The customer therefore no longer has to create the reference objects manually. This is the great advantage of the Ibeo.Reference system: as a result of automatic labelling, the customer saves time and money. High-quality results are also provided: time-independent processing (known as forward-backward tracking) in the Ibeo Evaluation Suite increases performance when tracking and classifying objects.
Ground truth data can also be used in other areas of application, such as simulation tools, to depict realistic AD/ADAS scenarios for the development of new technologies.
Meaningful evaluation is possible by comparing the outputs of the DuT sensor with the reference data. Ibeo.Reference provides “DuT Matching” as an expansion module for this purpose. The objects of the DuT are assigned to the reference objects in relation to both trajectories and are compared with one another. The recording frequency of the DuT and the Ibeo.Reference system can also be different. By means of timestamp matching, the Ibeo Evaluation Suite makes it possible to synchronize the time stamps of the two systems retrospectively.
The key performance indicator includes the detection rates of the DuT sensor, the positional accuracy and the precision of correct classification. In addition to these core values, a detailed consideration of strengths and weaknesses and the various situations and scenarios is essential to reduce weaknesses effectively. From 2020 Ibeo.Reference is offering an “Algorithm Performance” validation tool which includes these functions, for example KPI clustering according to automatically generated scenarios. This makes it possible for customers to evaluate the precise efficiency of the sensor and therefore to determine market readiness. In addition, developers can obtain direct performance feedback on a development iteration. Through the precise characterization of efficiency, they can improve specific weaknesses and evaluate the impact of each change in the software on efficiency.
Ibeo.Reference is a combination of LiDAR sensor technology and the corresponding Ibeo evaluation software. The algorithms of the Ibeo Evaluation Suite generate reference scenarios to create a benchmark for the evaluation of ADAS and HAD applications. In addition, they serve as the basis for the development of new applications. Use cases for the Ibeo.Reference system include, for example, blind spot assistants, lane detection and ACC (adaptive cruise control).
Ibeo.Reference ILV is a visualization software package for offline visualization of raw and processed Ibeo sensor data. It can be used to play, evaluate and correct object data. It also offers an assistant for display of the sensor connection.
The Ibeo Evaluation Suite is a modular software package for automatic object labelling, the postprocessing of recorded data for large test runs and the generation of a lane marking map.