After my first SpinTunes competition last year, I posted a lot about both my experiences going through SpinTunes and my thoughts on songwriting contests in general. This year, I knew better what to expect and so didn’t have as many thoughts along the way. I probably wouldn’t even be writing this post if it weren’t for the fact that Travis Langworthy, who runs SpinTunes, asks everyone for feedback, with particular questions as well as an open door to whatever other comments we might have. The additional comments I had to share with him don’t seem particularly private or sensitive, so I figured I may as well post it here as well. Maybe it’ll even help drum up support for some of these ideas.
For the most part, I’ll just end up reiterating my recommendations from last year, though with some added insight that comes from added experience. Before moving onto to the meatiest stuff, I want to just mention briefly my thoughts on eliminations and voting systems, as well as the issue of qualification vs. judging.
Increasingly difficult challenges: In a competition with eliminations instead of cumulative scoring, the contest would likely be that much more rewarding to contestants and listeners alike if the challenges become, as much as possible, successively more difficult in each next round. Obviously there’s subjectivity in assessing this. But in SpinTunes 3, there was a pretty big consensus (shared by me) that Round 3′s rap challenge was the most daunting. For me, that was followed by Round 2′s song based on a newspaper article, for which even just finding an article was itself a challenge, much less finding a decent songwriting angle for it. Independent of how I might have done, the evocative and highly open-ended nature of Round 4′s song inspired by a particular photograph made it seem like perhaps the easiest challenge of the entire contest. Even if not the easiest, it seemed something of an anticlimax. If challenges become generally more difficult as the contest goes on, though, the unfolding story of the competition becomes more interesting.
Preferential voting: When it comes to ranking the contestants, a preferential voting system would be a much better way to see the combined picture across judges. Rankings are not scores, but SpinTunes tallies the rankings as if they were scores. The math just doesn’t make as much sense as treating them as what they are, which is rankings, and a preferential voting system is designed to do just that. Not to mention that it would eliminate any confusion over whether to give “1″ to your top choice or, because the ranks are treated as scores, to your bottom choice — a potential confusion that could lead mistaken results. Preferential voting doesn’t require anything different from judges than what SpinTunes already asks. It only requires that their votes be combined in a different way than they are now. And at places like DemoChoice, there are even some free tools to do all the work. Set up the poll, and judges can literally cast their votes, with results automatically determined — no further work on the part of SpinTunes. Later, though, I’m going to make a recommendation that could make both rankings and preferential voting moot…
Qualifying contenders: Since SpinTunes 1, with some controversy over certain entries being allowed through for judging, there has been some good effort made to qualify entries first. Disqualifying entries prior to judging and passing others into contention, though, is just the first step. Qualification can only really mean something if, after that point, judges accept those decisions and judge on that basis. A big controversy arose in the recent competition’s Round 2, when a judge refused to review a song, ranking it last by default. This was motivated by the fact that the song’s creator, Edric Haleen, didn’t allow it to be posted online. However, online publication was never a genuine, explicit requirement of the contest, and the official decision was against disqualifying the song simply because of the non-publication request. Rule changes have already been put into place in response to this Edric situation, and apparently more may be planned. And that’s a very good thing, because this kind of unilateral act on the part of a judge, and the inclusion of that judge’s scores in the combined rankings, dramatically reduces the integrity of the round’s scores. Only what deserves to go into contention should go into contention, but once deemed deserving, all contending entries should be judged accordingly.
So those were the more minor points I had to share. Leading up to a final key recommendation, I want to talk about a handful of things that can be seen as problematic, with one solution to address them all.
- Judges have a huge amount of work to do in evaluating and ranking songs, giving them all the attention they deserve. It would be nice if their job could be made easier.
- Case in point: Even when I’m just voting in the public poll in the early rounds, and even when I don’t bring a lot of rigor to my rankings because I know that my public poll vote doesn’t count for that much, it’s amazing how much time it takes to listen to all the songs, with even a minimally critical ear, and make decisions about them. The actual judges are asked to provide meaningful feedback about all the entries. I’ve heard mention of just how much time some of the judges spend doing their job, and how difficult it can be for them to compare so many very different songs. Their job is a big one.
- It’s hoped that both averaging judges’ scores and having a fairly big pool of judges will ameliorate extremes and glitches in judging. That’s no reason to try to eliminate extremes, glitches and other inconsistencies that make for messy results and fail to serve contestants and the contest. Subjectivity is a given, but nobody is served by misperception, extreme weighing of one factor over others, altering the weight given to different factors from song to song and round to round based on whims, or the existence of any of these kinds of extremes in one judge without counteracting extremes being present in another judge at the same time.
- Cases in point: Edric Haleen’s round 2 entry mentioned above. And judges who admit that lyrics, or music, or emotion, or some other factor is by far the most important to them. And the occasional idiotic things said by judges — no offense to anyone, but I’ve had enough conversations with enough other SpinTunes participants to know that I’m nowhere near the only person who has felt that some things said by some judges, about not only my own entries but also those of many others, have been fairly nuts.
- In a contest where bragging rights and T-shirts are the only explicit prizes, where shadow entries are wholeheartedly encouraged, and where songwriters with limited or even no experience are welcome, competition is clearly not a driving force above and beyond all other considerations. Though it may not be fully built into the formal rules, songwriters, fans, followers and even judges overtly place a tremendous focus on creative growth as a big part of what SpinTunes is about. If people want the game to remain a game instead of simply a songwriting collective, that’s fine. But when so many involved are so keen to foster the cultivation of artistic voices and not merely to see who wins, it would be nice if the contest were better designed to give songwriters as much clear, constructive feedback as possible toward that end. As it stands, though, not only the extreme/quirky judging already noted but even more forthright judging often fails to serve participants toward this end.
- Cases in point:
- When my first ever SpinTunes entry Step Back Swooperman placed 19th out of 20 songs, and when I heard what not only judges but even some fellow contestants thought of it, I figured that at best my tastes were more different from most people’s than I’d imagined, and at worst I had a lot more to learn about songwriting than I’d imagined. But while participating in the recent SpinTunes Interview on the Geeky Pleasures Radio Show, I learned something interesting from host Jules Sherred, who herself had been one of the judges of SpinTunes 1. She’d ranked that song 17th, making her pretty representative of the song’s overall reception. During the interview, though, she told me that, outside of the competition, she still listens to the song, and that it’s one of her favorites, but that, within the competition, she was really torn in judging it, because she felt I was extremely ambiguous in meeting the criteria of the challenge, and in the end that’s what won out in her judging.
- Something similar happened to me in SpinTunes 3. In fact, this situation was also somewhat similar to Edric’s round 2 unilateral disqualification. For various reasons I won’t go into detail about the what or the who. Both published evaluation and offline discussion revealed a judge to have a profound misperception of both my song and the nature of the challenge, leading to an incredibly poor ranking for me despite the judge also being downright profuse in extreme praise for the song. Once again, the rank had to do with the perception of how well the challenge was met — and as with Edric, the situation was tantamount to a judge disqualifying a song after it had been deemed a qualifying contender.
- It would have been really helpful for me to have understood these things as clearly as possible when the evaluations came out. Understanding how well one meets a challenge can lead to learning that can help you do better as a contestant, but it’s otherwise fairly useless in informing one’s artistic growth, whereas believing a song to be thought bad when maybe it’s not thought so bad after all can impact artistic growth. It can only help contestants to understand the difference between judgments that have to do with their songs-as-contest-entries vs. judgments that have to do with their songs-as-songs, because the two are obviously sometimes two very different things. Any number of other things could also be made clearer in ways that would similarly help artists.
Every one of these things can be addressed really well by one other thing I talked about last year: a scoring system. Take a few key factors, assign a particular number of possible points to each, and have the judges choose how many points to give each song for each category. A judge then simply has to total up their own scores across the categories for each song, and then they know where they stand on all the entries.
The many benefits include but may not even be limited to these:
- Judges would have a tremendously easier job. Instead of weighing things in an amorphous way, comparing songs in their totality in ways that can often seem like apples and oranges, a simple structure would give them something to go on, in effect having them answer a few fairly simple questions in fairly simple ways for each song. Boom, job done.
- Subjectivity would still be ever-present. It would not be stifled. It would merely be channeled in ways that were constructive. Perhaps the most crucial aspect of that constructive channeling: consistency.
-
- Songwriters are creating whole songs, plus it’s not just a songwriting contest. That’s an issue I take some exception with, but even if one embraces it, it’s important to account for the different things that judges really do consider beyond the writing. A scoring system ensures that all judges look at the whole package, writing and beyond — and it ensures that contestants have a really clear picture of just how their entries will be evaluated, so there’ll be no misunderstandings about the extent to which the contest is about songwriting vs. a battle of the bands, etc..
- A scoring system prevents all kinds of extremes, glitches and quirks from making too much of an impact. They’re all allowed to be there, they’re just tempered, at the source and not merely downstream when judges’ scores are combined. This even takes some pressure off the judge-finding process — it’s not as crucial to “weed out” beforehand when even the weeds would end up having their subjectivity channeled constructively. (Incidentally, this would also make a situation like Edric’s Round 2 mishap essentially impossible, since judges would have to score by category and could not simply make a unilateral and simplistic decision to rank a particular contending song last.)
- Contestants would get clear feedback on the various distinct aspects of their work. This would be helpful in general, and it would be particularly helpful in letting songwriters see clearly the extent to which the challenge itself affected each judge’s opinion. All of this would give the songwriters as much as possible to use as the basis for informing future artistic progress, in the contest and otherwise.
- If the desire is to continue to use rankings for the contest, with or without a preferential voting system like I talked about above, judges just need to sort their total scores, and the rankings pop out automatically. Judges could break their own tied scores simply based on preference. However, this kind of scoring system opens up another compelling option for combining judgments, which is to simply add all the raw scores (including judges’ own ties) together across judges. This means even less work for judges, since they wouldn’t even need to sort their own scores. More importantly, it also means that the relative weight of each song’s judgment would actually be honored in the final results. Judges seldom feel that the songs in a round are evenly spaced from first to last in terms of quality. They feel more strongly about some songs being much better, some songs being mediocre, and some songs being much worse. And they spend quite a lot of effort arriving at these nuances of opinion. But all those notions — some quirky and bizarre, some quite justified — are washed away when their evaluations are normalized as rankings. Keep the raw scores, though, and all those nuances from across all judges would be honored, combining to give a much better picture of the overall opinion of all the songs.
- SpinTunes has always done two things with judges evaluations: it has treated their rankings as not rankings but scores, and it has refrained from using a preferential voting system as an optimal way to combine those very rankings. All of this suggests that what SpinTunes is really interested in is scores as opposed to rankings. Moving to a scoring system and then combining judges’ raw scores would probably be the truest possible way for the contest to do what it seems to want to do with its results anyway.
A scoring system could take almost any shape. Here are just some suggestions, to jog thinking.
First, the number of points in each category could be varied if there was a desire to weigh certain factors more than others, but I’m inclined to suggest that each category weigh equally. I’d pose at least three points per category, since that would allow very basic poor / okay / good options for each category. More than that could be fine, e.g., adding a “very good / great,” and possibly also a “very poor / terrible.” Going no more than these five might be good to keep things relatively simple.
As for the category breakdown itself, the simplest possible scheme that could have meaning might have just three categories:
- Challenge — in which judges assess the concept and execution for meeting the given challenge
- Writing — in which judges assess the songwriting itself, i.e., lyrics and musical composition, distinct from performance and production — akin to the Grammy for Song of the Year
- Recording — in which judges assess the performances (instrumental and vocal) and the production (including all aspects of arrangement, engineering, etc.) — akin to the Grammy for Record of the Year
Since there are some very different skills rolled into the last two categories above, I think some additional worthwhile specificity would come from separating things out just a little bit more:
- Challenge
- Lyrics
- Composition — i.e., purely the writing of the music, the thing that combined with lyrics makes the Writing category above — the essential quality of the melody/melodies/countermelodies, harmonic progressions and musical form/structure, independent of arrangements or performances
- Instrumental Performance
- Vocal Performance
- Production
In all the cases above, there’s full opportunity for subjectivity and personal preference. These things aren’t eliminated. They’re just shaped to smooth out the edges of arbitrariness, inconsistency and extremity.
Not enough subjectivity for you? Not enough room for personality? Want more opportunity for judges to be subjective, even quirkily so? Fine, just design that right into the system. Add a category for Judge’s Whim. Same number of points as the other categories, but for whatever tickles a judge’s fancy. By making this an explicit category, it allows judges with admitted preferences for particular aspects to “be themselves” and strut their stuff without feeling too constrained by the structure of the other categories,. However, it allows this without making that strutting tip any scales at all too far, and it does this in two ways. First, the very existence of the other categories ameliorates the quirkiness and whimsicality, giving it some room but no opportunity to overtake anything. Second, even judges who don’t have such particular preferences would have these very same additional points to score at their discretion. I personally would not want to see this category added, because I think the scoring should be about the contestants and not about the judges, but if there was a strong feeling in favor of amplifying the judges’ voices, this is the way to do it in a sensible way.
So:
Better handling of qualification? Apparently in the works.
More obvious increasing difficulty of challenges as the rounds go on? Hopefully, as much as subjectivity allows.
Preferential voting? If there will be no scoring system, then hopefully so.
And about that scoring system? I think that’s the main thing — and use of raw scores instead of rankings would even make preferential voting moot.
So there you go. Comment/discuss below.
