Saint Matthews Strikes Again. An Agent-Based Model of Peer Review and the Scientific Community Structure

This paper investigates the impact of referee reliability on the quality and efficiency of peer review. We modeled peer review as a process based on knowledge asymmetries and subject to evaluation bias. We tested various levels of referee reliability and different mechanisms of reviewing effort distribution among agents. We also tested different scientific community structures (cohesive vs. parochial) and competitive science environments (high vs. low competition). We found that referee behavior drastically affects peer review and an equal distribution of the reviewing effort is beneficial only if the scientific community is homogeneous and referee reliability is the rule. We also found that the Matthew effect in the allocation of resources and credit is inherent to a ‘winner takes all’ well functioning science system, more than a consequence of evaluation bias.