Grade Inflation in Music Ratings? Statistical Analysis of Professional Ratings of Albums on Wikipedia

With so many supposed experts on the internet, who can be trusted about perceptions of artistic merit? All professionals’ album ratings on Wikipedia were tabulated every five years over 50 years, yielding a sample of 531 raters giving 16,614 ratings of 4,448 albums in 17 genres. For all critic pairs rating at least 30 albums, ratings were moderately in sync, with 72% of 230 pairs showing significant positive correlations (average r  = +.35). In addition, shared aesthetic consensus regarding album quality explained more than twice the variance in ratings compared to individual critic idiosyncrasies. As predicted, the critics’ rating distribution was mound-shaped but negatively skewed (−.62), while individuals varied widely in rating behavior. Inflated ratings were particularly salient (46.5% of ratings at 80/100 or higher). Consistent with Martindale, who argued that quality should decrease in older artistic areas, critics tended to perceive weakening average album quality between 1960 and 2010.