OSF | Crowdsourcing Analytics – Twenty-nine teams use same dataset, find contradicting results


Crowdsourcing Analytics – Final Manuscript.pdf ShareDownloadViewRevisions

Twenty-nine teams involving 61 analysts used the same dataset to address the same research question: whether soccer referees are more likely to give red cards to dark skin toned players than light skin toned players. Analytic approaches varied widely across teams, and estimated effect sizes ranged from 0.89 to 2.93 in odds ratio units, with a median of 1.31. Twenty teams (69%) found a statistically significant positive effect and nine teams (31%) observed a non-significant relationship. Crowdsourcing data analysis, a strategy by which numerous research teams are recruited to simultaneously investigate the same research question, makes transparent how variations in analytical choices affect results.

Source: OSF | Crowdsourcing Analytics – Final Manuscript.pdf