This paper looks at the effect of multiple reviewers and their behavior on the quality and efficiency of peer review. By extending a previous model, we tested various reviewer behavior, fair, random and strategic, and examined the impact of selecting multiple reviewers for the same author submission. We found that, when reviewer reliability is random or reviewers behave strategically, involving more than one reviewer per submission reduces evaluation bias. However, if scientists review scrupulously, multiple reviewers require an abnormal resource drain at the system level from research activities towards reviewing. This implies that reviewer selection mechanisms that protect the quality of the process against reviewer misbehavior might be economically unsustainable.
Is three better than one? simulating the effect of reviewer selection and behavior on the quality and efficiency of peer review
BIANCHI, Federico;SQUAZZONI, Flaminio
2016-01-01
Abstract
This paper looks at the effect of multiple reviewers and their behavior on the quality and efficiency of peer review. By extending a previous model, we tested various reviewer behavior, fair, random and strategic, and examined the impact of selecting multiple reviewers for the same author submission. We found that, when reviewer reliability is random or reviewers behave strategically, involving more than one reviewer per submission reduces evaluation bias. However, if scientists review scrupulously, multiple reviewers require an abnormal resource drain at the system level from research activities towards reviewing. This implies that reviewer selection mechanisms that protect the quality of the process against reviewer misbehavior might be economically unsustainable.File | Dimensione | Formato | |
---|---|---|---|
BianchiSquazzoni2015WSCProceedings.pdf
accesso aperto
Tipologia:
Full Text
Licenza:
PUBBLICO - Pubblico con Copyright
Dimensione
89.02 kB
Formato
Adobe PDF
|
89.02 kB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.