Evaluating the overall research results

News from the Committee
04/12/2019

“There is an urgent need to improve the methods of evaluation of scientific research results by funding agencies, educational and research organizations and other actors.”(DORA, 2012)

The National Plan for Open Science follows the spirit of the Leiden Manifesto and the San Francisco Declaration on Research Assessment (DORA) and emphasizes the need to change the evaluation system for researchers and institutions to reduce the quantitative dimension in favour of more qualitative evaluation.

However the system must go further and not consider scientific publications to be the only output requiring evaluation. One of the DORA recommendations specifies that evaluation of research must take into account the value and impact of all research results (including data sets and software programmes) in addition to scientific publications. The implementation of open science also requires all research processes and activities to be taken into account in the evaluation.

In Types of documents, productions and activities promoted by open science which are eligible for evaluation, the Committee for Open Science’s “Evaluation” group presents the full range of research productions and activities which are eligible for evaluation, as the (translated) title suggests.

The first table lists 17 types of documents and productions created during the entire research cycle and their open dissemination modes. They can only be considered fully open if they are made public under the conditions of a free or open license.

The second table lists 12 types of activity promoted by open science which are eligible for evaluation.

The recommendations of the Open Science Committee are based on those of the Open Science Platform Policy (OSPP) which federates the European Union’s work and strategies around eight priority pillars, two of which are specific to evaluation issues – firstly encouraging open practices and their recognition and secondly rolling out a new generation of research indicators.