The H2020 Project EUROBENCH and GET2EXCEL are creating the first benchmarking and standardisation framework for robotics innovation worldwide, much more of which we find out about here
Robotics has entered a new exciting era. For the first time, robots are facing the challenge to work in close interaction with humans in everyday life activities.
This new field, called Interactive Robotics, is receiving particular attention in the research community, with an increasing number of projects and initiatives. Most of the current research & development efforts are focused on improving robots’ abilities, e.g. stability, robustness, and safety in out-of-the-lab environments.
To allow these new machines to be perceived as a real value by the society and survive in this rapidly evolving technological landscape, one question is yet to be answered: “To what extent can robots meet real users’ needs?”. Without a concrete and quantitative answer to this question, the entire robotics innovation chain is put into danger. Benchmarking can, therefore, be the key solution.
What is benchmarking?
Benchmarking is the process of comparing different systems by using a common standard or point of reference. Benchmarking aims to generate quantitative and clear performance scores that help costumers quickly identify which products better fit their needs. Benchmarks can be eventually translated into international standards, to drive policymakers in ensuring a smooth and safe introduction of novel technologies into existing market domains. Benchmarking can be also of great help to scientists and technical developers, establishing clear technical targets to demonstrate a certain readiness level of the technology.
Benchmarks are well established in many markets and scientific domains, e.g. mobile phone technology, electrical appliances, automobile and internet connections. Unfortunately, when it comes to robotics, these aspects have been largely overlooked. With the advent of Interactive Robots, which are going out of the classic industrial environment entering the consumer market at an exponential pace, the need for benchmarks and standards is now evident.
Nevertheless, a generalised, well-accepted and comprehensive benchmarking methodology does not exist yet. The EUROBENCH and GET2EXCEL projects aim to fill this gap.
The EUROBENCH project
The H2020 Project EUROBENCH is creating the first framework for the benchmarking of Interactive Robots worldwide. The project is currently focusing on bipedal robotics technologies, i.e. wearable robots (exoskeletons and prostheses) and humanoid robots, aiming to expand towards a wide range of robotic technologies. The EUROBENCH Consortium, with the help of 17 Consortia funded by the first EUROBENCH Open Call (FSTP-1, now closed), is currently developing the three main pillars of the benchmarking framework:
Two testing facilities. EUROBENCH is creating two testing facilities (one for wearable robots and one for humanoids) equipped with state-of-the-art testbeds, which replicate the main conditions encountered by humans in daily life locomotion, such as slopes, stairs, obstacles, unstable terrains, as well as external disturbances, e.g. pushes. These testbeds will allow testing the performance of the robotic machines in typical human-like environments.
One Benchmarking Software. The project will release a Benchmarking Software suite to allow any researchers, developers or users worldwide to replicate the testing experiments in their own laboratory settings and automatically calculate relevant indicators on robot performance.
One unified database. EUROBENCH will gather and organise all the data recorded by the EUROBENCH users into a unified database. This data, combined with AI techniques, will produce efficient comparison algorithms and reliable predictors to allow users to identify the best match between their needs and available technologies.
With the community, for the community: The GET2EXCEL Center of Excellence
The ultimate goal of EUROBENCH is creating a sustainable “benchmarking infrastructure” and to make it available to the general public, as well as the research and industrial community worldwide. For this reason, we are now creating a portfolio of services for future users of the framework. To involve scientific and industrial experts in validating and improving these services, we will allow third parties to become beta testers of the EUROBENCH framework, at zero costs.
Through the Second competitive Open Call (FSTP-2, open from June-August 2020), we will offer financial and technical support to a selected number of participants to exploit the EUROBENCH tools (facilities, software and datasets) to test their robotic technologies, either commercial or prototypal. All the testing experiments will be developed in total respect of confidentiality and industrial secret requirements of participants.
This is not all. EUROBENCH is at the core of a new initiative, the GET2EXCEL Center of Excellence (CoE), a global network including all relevant stakeholders and organised in five cluster areas: North America Cluster, Europe Cluster, Asia Cluster, Pacific Cluster and South and Central America Cluster. Each of the cluster areas is coordinated around physical sectoral Hubs, where testing, benchmarking and R&D infrastructure are available.
Therefore, the EUROBENCH reach has been broadened through this CoE for global leadership, including collaborations with several international standard organisations (e.g. ISO, ASTM) to boost the process of converting the current and future benchmarking procedures and methodologies into accepted standards for the evaluation of the different facets of robotic performance. This will facilitate the efficient introduction of these new technologies into the different domains, e.g. healthcare, manufacturing, logistic, consumer, each of them characterised by the needs, requirements and regulations of diverse users and applications.
In conclusion, EUROBENCH and GET2EXCEL are working hard to prepare the Society to improve and accelerate the introduction of Interactive Robotics in people’s lives, by means of new tools that will help: 1) Developers to identify the key technological factors to improve current robot abilities; 2) End-users to identify whether a specific robot can meet a real user’s need’ 3) Researchers to “stand on giants’ shoulders”, replicating scientific experiments across laboratories and robotic platforms, achieving true comparison of results and 4) stakeholders, to be fully involved in (and able to influence) the standardisation process worldwide.
Both initiatives are now welcoming new participants. Be part of it!
*Please note: This is a commercial profile
