Thoughts On Ratings

by John McFerrin

“What grade should I give this album?” This question has been asked by all of the members of the Discord & Rhyme family at some point in our lives; as mentioned many times in our episodes, we all first became aware of each other through a collection of Web 1.0 music review sites where the writers gave grades to various albums with supporting text to accompany them. Some of these sites graded on a scale of 1-10, some used star ratings (up to *****), and some used letter grades. Since these sites generally accepted reader comments, it became regular practice to include our own proposed rating for a given album in addition to our own written thoughts. Later, at some point or another, most of us went through the process of writing our own reviews and providing our own grades, some within the context of a collective site (i.e. Music Junkies Anonymous), and some within the context of our very own review sites (e.g. CosmicBen’s Record Reviews, The Disclaimer Music Review Archive, John McFerrin’s Reviews of Music). Each of us in turn chose our own scale to use; most followed the typical 1-10 approach; Will used letter grades; and I eventually shifted to using hexadecimal (base-16), a rating system that makes perfect sense to me but is an acquired taste for others.

While we all cut our teeth as music reviewers in a framework that involved ratings, our assessments of albums in the context of our podcast episodes do not involve assigning a rating, and this makes sense; if we had ever conceived of a section in a standard episode where each of the hosts would provide a proposed rating for the album in question, we would have realized almost immediately the perfunctory nature of such a section and would have scrapped it for good. While we don’t include ratings in our episodes, though, the topic of ratings pops up frequently in our various Slack channel discussions; not about the ratings for specific albums, but about the very nature and usefulness of ratings as a concept. As one might infer from reading my site, I am the staunchest defender of the usefulness of ratings among our group, yet when others in our group suggest mild objections to the necessity of them, I generally find myself in agreement with their points. Considering these points further led me to realize, “Hey, this could be a blog post,” and here we are.

The best way I can think to address the question of “What’s the use of ratings at all?” is to address the question of how I use them in the context of my own review site. A central guiding principle of my site is that the review must always be the main thing; sometimes my reviews are long, and sometimes my reviews are short, but in every case I seek to say as much as I feel is pertinent to expressing my thoughts and impressions of the album and where it fits in the general narrative of the artist’s career. In turn, the rating given for the album should always be plausible in the context of the provided review; there may be cases where the provided rating initially seems a shade too high or too low given the review, but as often as possible a way to reconcile the two should present itself. Furthermore, this match between rating and review, if I am doing things correctly, should occur with enough consistency that, as the number of reviews I write increases and the various quirks and eccentricities of my tastes reveal themselves, an astute reader should be able to predict with reasonable accuracy the ratings I will give to the albums by a given artist (in fact this actually happens; on the message board accompanying my site, it’s commonplace for people to post predictions of ratings that I will give albums from a particular artist, and people who have read my site for years can consistently guess every album grade I will give for that artist, +/- one rating point). Similarly, if somebody visits my page and clicks through the page that sorts the different reviews by rating (my reviews sorted by rating), they should find, within a given ratings tier, a consistency in language and tone between the different reviews.

For me, then, ratings are useful precisely because, despite how prominently I feature them, I try to use them as a complementary tool and nothing more. Unfortunately, I have found over the years that ratings are not typically used in such a way, and I find much of how they are often used somewhat troubling. Case in point: many years ago, my fellow online reviewer George Starostin posted a new review of the Aerosmith album Rocks on his site, after years of resisting the band, and gave it a surprisingly high rating considering his previous attitudes regarding the band. On the message board for his site, the discussion regarding this new review took an unfortunate (though sadly not unexpected) turn when one particular hot-head used this as an opportunity to lecture George on his stupidity, on the grounds that this was a higher grade than he had given to any albums by Soft Machine (the influential Canterbury scene group that later became a jazz fusion outfit). Obviously, Soft Machine and Aerosmith have little in common, but all that mattered to this man in this moment was the rating, and the discussion quickly degraded as many involving him did.

This example is extreme, yet it is typical of how ratings often bring unwanted dimensions to discussions regarding music. A common problem, for instance, regards the fundamental problem of “what does this rating even mean” that occurs when people discuss ratings; for example, what should a 5/10 rating mean (even if people agree that it means “mediocre” that doesn’t mean that they agree what “mediocre” means)? Another problem regards precision; for me, 10 available ratings is too little precision, while 100 available ratings (such as with Pitchfork, which grades 0-10 but where a grade like 7.6 is valid) is too much precision, but many others would disagree and would have good reason to do so. Almost inevitably, discussion of ratings leads to people disagreeing over semantics regarding rating systems rather than the work that was rated, and while this be might interesting for me (yet only to a point), most people will find such discussions unfathomably tedious. And of course, this doesn’t begin to approach the topic of what happens if one’s rating of an album is corrupted by the relation between the reviewer and the artist with whom the reviewer hopes to have continued access.

Bottom line, I will never relinquish my position that, used correctly, ratings can provide a useful framing device for a review and can help organize one’s sense of where an album approximately fits into their personal hierarchy. And yet, I also believe strongly that a rating should never supplant a review, either for the writer or for the reader, and if ratings act at cross-purposes to making sense of an album, then they should be ignored.