Toma Susi, Monica Heintz, Eva Hnatkova, Wolfram Koch, Maria Leptin, Martin Andler, Marco Masi, Michele Garfinkel
Open Science is a broad approach to improve the reproducibility, transparency, and robustness of research. By enabling broader access to data, code, methods and publications, it has the potential to increase the efficiency and impact of public funding of research, and societal engagement. Although some aspects are being implemented by specific funders and organisations as well as many individual researchers, Open Science overall is still far from being fully embraced by the research community. At the same time, the ways in which research assessment is carried out at present have become an acute issue both for researchers and organisations. A key factor for overcoming these systemic challenges is to reform academic evaluation and reward systems to include Open Science practices.
This report explores how and at which levels change can happen, and which routes can be taken to reach a comprehensive change that could be applied across the research system while respecting valid disciplinary or other relevant sectoral differences. Several policy options for each stakeholder are proposed. Overall, the pressure by the European Commission and other actors should be welcomed to drive much-needed changes, but this approach may risk neglecting the quality of research in favour of how it is performed. This is why we feel it is crucial for researchers to drive the transition to Open Science and is the main motivation for this work.
We believe that all stakeholders must recognize four essential principles for the successful reform of research assessment, and these require coordination both within and between stakeholder groups:
For research communities, it is urgent and vital to concretely consider how they wish evaluation systems to be adapted to eliminate pernicious incentives and to reward pertinent Open Science practices in their diverse circumstances. There is a serious risk that if they cannot make concrete proposals on how to replace currently prevalent prestige indicators such as journal impact factors and quartile ranks, these will either continue to be (mis)used or new indicators will be imposed without the communities’participation.
Similarly important and critical is the engagement of all concerned parties in the larger discussion of how to move towards new forms of research assessment, with an appropriate balance of qualitative and quantitative evaluation. This engagement requires the involvement of researchers themselves and their community representatives, such as learned societies. By positively articulating what the goals of research assessment should be, research communities can help build better systems of assessment and credit reflecting those goals.
All options are in the context of four framing questions to be kept in mind:
The options are in service of key policy goals for an improved research assessment and reward system:
The options are in the context of four essential principles:
Initiative for Science in Europe
1 quai Lezay-Marnesia
This report can be downloaded at: https://initiative-se.eu/paper-research-assessment/
Graphics and report design by Sandra Krahl