Transforming one million lives through literacy

How we ensure fairness of peer review 

All valid applications will be reviewed and scored by five peer reviewers. These reviewers will offer scores and comments against five scoring criteria on a 0-5 point scale. 

Because people differ, not all reviewers will score submitted applications in the same way—some reviewers will be more generous and others stingier. For example, one reviewer may score all applications with scores between 0 to 3, another will score all applications between 1 and 5. Both of these reviewers may look favorably upon a single submission, but give a different scores. 

Likewise, reviewers’ spread of scores may differ. One reviewer may have a more binary scoring approach, giving only 1’s and 5’s, while another reviewer may spread scores around more evenly across the range. 

To ensure fairness for all applicants, we will normalize all reviewers’ scores by measuring each individual reviewer’s mean and standard deviation of scores, as well as measuring the mean and standard deviation of all scores across all reviewers. We will then rescale the mean and standard deviation of each individual reviewer to match that of the entire population of reviewers. 

This way, no matter which reviewer is scoring your application, your submission is treated fairly.

Scores from these peer-reviewers will be used to select up to 100 applications that will advance to the expert judging review round. These experts judges will then select up to 50 milestone prize awardees who will each receive an equal share of $500,000. 

We are happy to answer any questions you have about the scoring process. Please register and submit an application today. You can learn more in the FAQ section.