So getting back to the paper ...
Quote:
After running the model with 1000 scientists over 500 time-steps, Thurner and Hanel find that even a small presence of rational or random referees can significantly reduce the quality of published papers. When just 10% of referees do not behave "correctly" the quality of accepted papers drops by one standard deviation. If the fractions of rational, random and correct referees are about 1/3 each, the quality selection aspect of peer review practically vanished altogether.
"Our message is clear: if it can not be guaranteed that the fraction of rational and random referees is confined to a very small number, the peer-review system will not perform much better than by accepting papers by throwing (an unbiased!) coin," explain the researchers.
|
From the first post;
"
rational": those who reject papers that might draw attention away from their own work;
"
random" who are not qualified to judge the quality of a paper because of incompetence or lack of time.
and the types who do the best job (in the mix) are:
"
correct" who accept the good papers and reject the bad and;
"
altruists" who accept all papers and;
"
misanthropists", who reject all papers.
So, maintaining the quality of the peer review process is critically dependent on keeping career-oriented, busy and incompetent types out of the process.
Seems logical ... (almost self-evident, really) but difficult to achieve in practice (?).
We need empirical evidence to take this one further !!
Cheers
PS: This could be good ammo for you David. Ie: That journal who published the work that's flawed ?