Professor Stan Gielen, President of NWO discusses whether changes are needed to the peer review process and how these can be implemented successfully
In the Netherlands, as in many other countries, peer review has been the standard procedure for selecting the best research proposals for decades. But, in recent decades, the system has come under pressure for various reasons. First of all, the fierce international competition for research excellence has triggered researchers to write more proposals. This has led to an increase in the number of research proposals in most countries, whereas the funding has not increased by the same amount. Consequently, more and more academics write and evaluate proposals that have very little chance of being awarded funding. In several research grant competitions, the Netherlands Organisation for Scientific Research (NWO) has experienced a misbalance between a large amount of time spent in writing and evaluating research proposals and the relatively few proposals that will be funded.
Many scientists spend lots of time preparing their own research proposals and being involved in the peer-review process of their colleagues. We have to achieve a more acceptable balance between available budgets – which will always be limited – the success rates, and the time spent on funding acquisition. Another aspect is that over the years, peer review has proven to be an academically effective route to selecting proposals with the highest scientific impact. However, to predict societal impact – an important criterion for funding – it has proven to be a less useful selection method. So we should definitely redefine how we prepare and evaluate research proposals.
Investigating alternative methods
Based on these observations, the Dutch Minister of Education, Culture, & Science recently asked NWO to consider alternative methods for evaluating proposals and to explore alternative evaluation strategies. This is why NWO has organised a series of conferences in 2017 dedicated to discussing the consequences of the high application pressure on the peer-review procedure, considering what potential adjustments to the peer review process are needed, and investigating which alternative review methods could be used.
NWO started off with a national conference on 4 April 2017 in Amsterdam. For this conference, we invited scientists based in the Netherlands as well as managers from universities and from NWO’s partners and policymakers to discuss possible improvements to how we allocate funds for research. Subsequently, on 21 May 2017 during an internal conference, employees of NWO explored the possibilities of mitigating application pressure due to a large number of proposals. On 29 and 30 June, we invited delegates from research councils and granting organisations around the world, but also researchers and peer reviewers, to an international conference.
With this international conference – which was attended by representatives from 20 countries – NOW aimed to jointly discuss various topics regarding application pressure, applicant workload, and the cost and efficacy of peer review in its current form. Are there any alternatives we could take into consideration? How can we implement them? Building on our joint experiences: how can we solve the problems we have come to identify and how can we work together to improve the peer review system? Is the solution to be found in alternative methods of evaluating research proposals? If so, which methods are the most appropriate?
The conferences were a great success, and many researchers appreciated the opportunity to share their experiences and explain their ideas. It became clear that many research councils worldwide are facing the same problems as NWO in the Netherlands regarding a high number of applications and success rates. Ideas for alternatives within the peer review system and experiments with new evaluation methods were discussed.
With the input gathered during the conferences, NOW will make a proposal to the Dutch Minister of Education, Culture, and Science by the autumn. Since this will be an ongoing challenge for us and for most research councils, we will simultaneously investigate how to build a strong international network where research council members and other stakeholders can continue sharing their ideas and lessons learned. This will ensure that the issue remains high on the agenda so that we can keep working on further improvements to our methods of evaluation.
Professor Stan Gielen
President
The Netherlands Organisation for Scientific Research (NWO)
