Blog

On May 14, 2025, the fourth seminar was held to present the results of the DARTS-PL project, entitled “Database of Autonomous Vehicles Road Testing Scenarios”, commissioned by the Ministry of Infrastructure and financed by the National Center for Research and Development under the GOSPOSTRATEG program. The meeting was held on-site at the Motor Transport Institute. The subject of the seminar was issues related to the recording of environmental data using the measurement set built as part of the DARTS-PL project and technical aspects related to the annotation of data intended for implementation in the algorithms of autonomous vehicle perception systems.

The seminar was officially opened by Prof. Marcin Ślęzak, Ph.D., D.Sc., Director of the Motor Transport Institute, DARTS-PL Project Manager, who outlined the goals and significance of the project as an introduction, and also recalled the division of tasks and responsibilities between the consortium members. The aim of the project is to develop an original database of 840 test scenarios for autonomous vehicles (AVs), which will be the basis for designing, creating, testing and evaluating L3-L5 vehicle perception systems according to SAE J3016. Different types of roads and objects will be recorded several times, in different environmental conditions (e.g. air temperature), taking into account different vehicle-oriented data (e.g. speed), with the participation of different road users, including vehicles, pedestrians and animals. The scenarios will contain both “raw” data and data with annotations. The implementation of the project also includes the preparation of an internet service enabling downloading and filtering of scenarios, as well as validation of object detection algorithms. The developed scenarios will take into account road conditions characteristic of Poland and will be available free of charge, which is aimed at strengthening the potential of the Polish economy.

As part of the DARTS-PL project, the Motor Transport Institute is responsible for the methodology of selecting measurement locations (also taking into account accident rates, based on accident databases collected since 1990 by the Polish Road Safety Observatory), performing measurements, selecting scenarios and providing comprehensive metadata, as well as promoting the project. The tasks of the Warsaw University of Technology include: processing of recorded scenario data, data filtration and fusion, implementation of semi-automatic annotation generation algorithms, scenario annotation and preparation of infrastructure and software for disseminating the scenario database.

After the ceremonial opening of the seminar, in the first presentation entitled “Spatial road mapping using a measurement set in the DARTS-PL research vehicle”, the authors – Mikołaj Kruszewski, PhD Eng., Anna Niedzicka and Aleksandra Rodak, Msc. Eng. from the Motor Transport Institute presented the methodology for selecting sections, types of metadata in the DARTS-PL project, sensory diversity of the measurement set and the sensor layout at the DARTS-PL station.

As part of the project, 100 sections will be selected throughout Poland (current map of measurement points: https://darts-database.com/en/map-2). Driving along each section will be recorded 8 times – in 4 different seasons and both at night and during the day, so that the same route can be checked for reference in different weather conditions, depending on lighting, on different types of roads, in different road traffic conditions (with the participation of different types of vehicles, pedestrians, people in wheelchairs, animals). Some of the sections were selected due to their level of safety – as a result of research on places with high accident rates on roads (based on data from the Police Headquarters, Polish State Railways and the Polish Road Safety Observatory), and some sections were selected due to their infrastructure diversity. The database will also be an excellent tool for testing road sign detection algorithms. Initially, it was planned to map approximately 12 km of roads, but at this stage of work, it was noticed that it was necessary to extend the measurement sections. The previously adopted scenario length imposed by the ordering party was to be no longer than 10 seconds. However, it turned out that driving in real road traffic will take longer, also due to the scenario start and end phases. The scenarios included in the database will be provided with a rich set of metadata, which will allow them to be filtered among available resources. The current set of sensors used for the project includes 4 lidars (in various configurations, of different types, for different applications), 7 cameras, 6 radars, 1 IMU-GPS-RTK module and 1 thermal imaging camera. Each of these sensors provides a different type of data. The recordings included in the DARTS-PL database will allow you to check in how many scenarios the individual sensors proved useful. However, the limited usability of a specific sensor does not mean that the research has failed, but may mean that the data obtained from such a device may be useful for accident protection purposes in very specific situations.

The author of the next presentation, entitled: “Data annotation in the DARTS-PL project: a pipeline using machine learning models and expert information” was Krystian Radlak, PhD Eng. from the Warsaw University of Technology. The concept proposed by the Warsaw University of Technology aims to automate the data annotation process in order to reduce human involvement in the time-consuming recognition of registered objects as much as possible. It was assumed that at the initial stage of data annotation, automatic machine learning algorithms would be used. Then, it is planned to use visualization software so that the person responsible for the annotation could approve or correct the annotation. Dr. Radlak drew attention to the need to work out a solution to the problem of incorrect detection of object outlines. It is mainly a consequence of incorrectly defined rules for applying outlines in the form of cuboids for 3D data and rectangular outlines in the case of data occurring in 2D format, e.g. taking into account or omitting the importance of mirrors in vehicles. The problem with annotation is significant primarily due to the high cost of manual data processing. Currently, the price of a single annotation of one object is on average about 14 cents USD per object. On average, one lidar frame contains about 20-70 vehicle data. The annotation time of a single 3D object is on average 23 seconds. Annotation of a single frame in a sequence is on average 40 minutes. The use of algorithms would allow for faster annotation. The current object detection efficiency is about 80% on lidar data, 20% of the data requires human correction. Work is currently underway to improve detection efficiency in order to minimize manual work of annotators. The presentation also indicated the need to adapt the work to the safety standards provided for autonomous vehicles and for artificial intelligence algorithms, where such topics are widely discussed, including: indications of how databases should be prepared to comply with security standards and to be used for the needs of the automotive industry (ASPICE 4.0, SUP.11, ISO 8800).

The special guest of the event, Mr. Wojciech Konieczny, President of Envibra Sp. z o.o., presented the beginnings of the company in a presentation entitled “ENVIBRA – Projects from the inside”, paying particular attention to the people who were involved in its establishment and the team developing the company. The speech presented the company’s activities and examples of completed projects. Envibra designs and manufactures specialist research stations, including for passive safety tests, drive system tests, seat tests, control system tests, vibration or fatigue tests, battery tests. The company provides technologically advanced solutions used by laboratories, universities and academic centers.

Mr. Sebastian Graboń and Mr. Mikołaj Małecki from Dewesoft Polska presented “Innovative Dewesoft measurement tools”. Dewesoft is a leading manufacturer of devices enabling measurement and analysis of data and recording data from various types of sensors, as well as a manufacturer of various types of software. During the speech, technical details and wide research possibilities of the products supplied by the company were presented.

The last point of the seminar was the presentation of the vehicle, which was part of Dewesoft Tech DAYS 2025. The participants of the event took part in a demonstration of the capabilities of the mobile laboratory – the Dewesoft measuring vehicle. The demonstration was led by engineers responsible for the construction of the measurement set in the BMW Demo Car. Inside the vehicle, for example, a vehicle power sensor and an industrial computer were installed, which allows for recording data, including from the cylinder pressure sensor and the crankshaft sensor and many other sensors. It is also possible to analyse the operation of automated driving systems installed in the vehicle. The system is connected to a battery module that is integrated with the vehicle’s power supply system.

During the 4th DARTS-PL Seminar, Aleksandra Rodak, Msc. Eng. (Motor Transport Institute), acting as the meeting moderator, invited to participate in the AV-POLAND Conference, which will be held on June 12, 2025 at the headquarters of the Institute of Motor Transport.

The DARTS project – Database of Autonomous vehicles Road Testing Scenarios is financed by the National Centre for Research and Development under contract no. GOSPOSTRATEG-VIII/0001/2022