From smart city model to self-driving vehicle. Sylvie Daniel, full Professor at the Université Laval discusses the role of 3d data mapping…
The past few years have seen remarkable development in mobile laser scanning (MLS) to accommodate the need for large area and high-resolution 3d data acquisition involved in building smart city model (cf. https://www.openaccessgovernment.org/situated-augmented-3d-cities/25452/). Such data collection systems provide the required level of detail and precision to build large scale 3d city models and run simulation applications, such as building shadowing or traffic sound pollution studies. These 3d urban maps enable a high degree of understanding of the complete environment and facilitate an enhanced ability to plan and manage events while providing solid decision making.
Beyond the framework of smart cities, there is a growing interest for high resolution 3d urban maps. Indeed, the emerging world of self-driving vehicles need high resolution maps to show autonomous vehicles the way. Self-driving vehicles work by relying on a combination of detailed pre-made maps, as well as sensors that see obstacles in real time. The vehicles have to process inputs from multiple sensors to precisely understand their environments. Relying on detailed maps simplifies this task. Self-driving vehicles concern not only the car industry market (ex. Tesla, Volvo, Google). Various prototypes have been designed and tested for public transportation as well like the self-driving, fully electric shuttle (ex. European project CityMobil 2). Also, it is anticipated self-driving trucks could rewrite the rules for transporting freight.
There are still many challenges to overcome from the standpoint of high resolution 3d mapping based on MLS data before self-driving vehicles become a reality. First, self-driving vehicles rely on a combination of sensors to understand their environment (ex. LiDAR, fisheye cameras). The data from these sensors have to be brought together in order to combine their individual perception of their surrounding. Mapping of heterogeneous data is an active research area. This is a complex task since the data sources have not the same structure (ex. continuous image versus discrete point clouds), the same representation of the physical world (ex. radiometry versus geometry), the same viewpoint (ex. short range image viewpoint versus mid to long-range LiDAR viewpoint) and are not collected at the same frequency.
Also, creating and maintaining such maps is difficult work. Advancements in processing LiDAR point clouds are required to improve the automation of objects classification and identification. So far, people pore over the data in order to categorise different features such as intersection, light poles etc. It is still very difficult for a computer to recognise an object as that object, especially if it has to rely on a series of points with no connection among them. To be able to do so, the computer needs to be trained with examples of features it has to identify. A vast amount of examples, covering a breadth of contexts and situations, has to be provided. Much like a human learns through experiences, the computer need to be exposed to vast quantities of information. The more data we collect, the smarter the system becomes.
Thus there is a need for large databases allowing training of algorithms on board the self-driving vehicles. Availability of public and documented databases will also enable comparison among various approaches and determination of the best classification methods. This is the rationale behind the French National Mapping Agency (IGN) contest (http://data.ign.fr/benchmarks/UrbanAnalysis/). The database contains 3d MLS data from a dense urban environment in Paris (France). Each individual object has been manually annotated in the 3d dataset. All the 3d points belonging to the same object have the same object identifier (id). From the classification standpoint, a category is assigned to each segmented object. Each class represents an urban semantic entity.
Availability of rich and public database of 3d MLS data can also be a catalyst of new applications or usages of this technology. It may favor collaboration among various communities (ex. creators, developers and users) and stimulate researchers from different fields (ex. computer graphics, robotics, geomatics) to propose innovative solutions to 3d MLS data processing issues.
Another issue related to high resolution 3d mapping for self-driving vehicles is the need to collect beforehand MLS data in new cities or territories. So far, this task is completed manually by driving vehicles in the targeted locations. Eventually, autonomous vehicles will be used to collect the data to build high resolution 3d maps. However, in such a context, smarter data acquisition approaches will be required. Indeed, currently, all the environment is digitised even if most of the collected points do not correspond to valuable information for building the 3d maps. Thus there is a vast amount of data collected. This is an issue not only from the storage standpoint but also from the processing standpoint. Indeed, some of the 3d point cloud processing has to be carried out in real time onboard the vehicle. Relying on machine intelligence, new acquisition strategies need to be devised in order to tune the data collection according to the vehicle context (ex. narrow streets versus avenues: close range versus mid-range acquisition settings) and information need (ex. more details about a potential obstacle).
As a conclusion, MLS offer great advantages not only for building smart 3d city models, but also for guiding the future generation of vehicles. As dense and documented 3d MLS databases are becoming more and more available publicly, a diversity of applications and usages (ex. digital arts, engineering, immersive intelligence,) can be anticipated. This context induces 2 major stakes that would need to be addressed: 1) documentation of 3d MLS data fitness for use: users will need guidelines to better understand the application accuracy and precision achievable with the databases; 2) crowdsourcing feature extraction and classification: if millions of kilometers of streets are to be digitised, mapping teams won’t be sufficient to process them. Similarly as OpenStreetmap initiative (https://www.openstreetmap.org), volunteers could contribute to the 3d high resolution map construction. Automatic quality assurance and quality control tools would have to be developed accordingly.
Dr Sylvie Daniel
Full Professor
Université Laval
Sylvie.daniel@scg.ulaval.ca
http://www.crg.ulaval.ca/?page_id=851&idFonction=96
https://www.scg.ulaval.ca/sylvie-daniel
Please note: this is a commercial profile
