Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Music Businesses Media Math Apple

Crunching the Math On iTunes 276

Posted by Zonk from the not-completely-random dept.
markmcb writes "OmniNerd has posted an interesting article about the statistical math behind iTunes. The author makes some interesting observations concerning the same song playing twice in a row during party shuffle play, the impact that star ratings have on playback, and comparisons with plain old random play (star ratings not considered)." From the article: "To test the option's preference for 5-stars, I created a short playlist of six songs: one from each different star rating and a song left un-rated. The songs were from the same genre and artist and were changed to be only one second in duration. After resetting the play count to zero, I hit play and left my desk for the weekend. To satisfy a little more curiosity, I ran the same songs once more on a different weekend without selecting the option to play higher rated songs more often. Monday morning the play counts were as shown in Table 1."
This discussion has been archived. No new comments can be posted.

Crunching the Math On iTunes More

Crunching the Math On iTunes

Comments Filter:

  • iTunes is a monopoly (Score:3, Funny)

    by Anonymous Coward on Sunday August 28, 2005 @04:49AM (#13420069)
    As a record store owner, My business faces ruin. CD sales have dropped through the floor. People aren't buying half as many CDs as they did just a year ago. Revenue is down and costs are up. My store has survived for years, but I now face the prospect of bankruptcy. Every day I ask myself why this is happening.

    I bought the store about 12 years ago. It was one of those boutique record stores that sell obscure, independent releases that no-one listens to, not even the people that buy them. I decided that to grow the business I'd need to aim for a different demographic, the family market. My store specialised in family music - stuff that the whole family could listen to. I don't sell sick stuff like Marilyn Manson or cop-killer rap, and I'm proud to have one of the most extensive Christian rock sections that I know of.

    The business strategy worked. People flocked to my store, knowing that they (and their children) could safely purchase records without profanity or violent lyrics. Over the years I expanded the business and took on more clean-cut and friendly employees. It took hard work and long hours but I had achieved my dream - owning a profitable business that I had built with my own hands, from the ground up. But now, this dream is turning into a nightmare.

    Every day, fewer and fewer customers enter my store to buy fewer and fewer CDs. Why is no one buying CDs? Are people not interested in music? Do people prefer to watch TV, see films, read books? I don't know. But there is one, inescapable truth - Internet piracy is mostly to blame. The statistics speak for themselves - one in three discs world wide is a pirate. On The Internet, you can find and download hundreds of dollars worth of music in just minutes. It has the potential to destroy the music industry, from artists, to record companies to stores like my own. Before you point to the supposed "economic downturn", I'll note that the book store just across from my store is doing great business. Unlike CDs, it's harder to copy books over The Internet.

    A week ago, an unpleasant experience with pirates gave me an idea. In my store, I overheard a teenage patron talking to his friend.

    "Dude, I'm going to put this CD on the Internet right away."

    "Yeah, dude, that's really lete [sic], you'll get lots of respect."

    I was fuming. So they were out to destroy the record industry from right under my nose? Fat chance. When they came to the counter to make their purchase, I grabbed the little shit by his shirt. "So...you're going to copy this to your friends over The Internet, punk?" I asked him in my best Clint Eastwood/Dirty Harry voice.

    "Uh y-yeh." He mumbled, shocked.

    "That's it. What's your name? You're blacklisted. Now take yourself and your little bitch friend out of my store - and don't come back." I barked. Cravenly, they complied and scampered off.

    So that's my idea - a national blacklist of pirates. If somebody cannot obey the basic rules of society, then they should be excluded from society. If pirates want to steal from the music industry, then the music industry should exclude them. It's that simple. One strike, and you're out - no reputable record store will allow you to buy another CD. If the pirates can't buy the CDS to begin with, then they won't be able to copy them over The Internet, will they? It's no different to doctors blacklisting drug dealers from buying prescription medicine.

    I have just written a letter to the RIAA outlining my proposal. Suing pirates one by one isn't going far enough. Not to mention pirates use the fact that they're being sued to unfairly portray themselves as victims. A national register of pirates would make the problem far easier to deal with. People would be encouraged to give the names of suspected pirates to a hotline, similar to TIPS. Once we know the size of the problem, the police and other law enforcement agencies will be forced to take piracy seriously. They have fought the War on Drugs with skill, so why not the War on Piracy?

    This evening, m
    • Before any idiots post saying "You're lying! don't be an idiot" - This post is a satirical post critizing the music industry, it has been posted before on slashdot, it is a joke. It is not serious!

      • Re:iTunes is a monopoly (Score:2, Insightful)

        by shish ( 588640 )
        This post is a satirical post critizing the music industry

        ... how? All I see is a tired old troll :/ Where's the funny / insightful that "satire" implies?

        • "Dude, I'm going to put this CD on the Internet right away."

          "Yeah, dude, that's really lete [sic], you'll get lots of respect."

          I was fuming. So they were out to destroy the record industry from right under my nose? Fat chance. When they came to the counter to make their purchase, I grabbed the little shit by his shirt. "So...you're going to copy this to your friends over The Internet, punk?" I asked him in my best Clint Eastwood/Dirty Harry voice.

          You don't find that hilarious? Something is wrong wi
      • Anyone who doesn't catch the inherent parody in

        They have fought the War on Drugs with skill

        clearly shouldn't be allowed online without a minder.
      • So is your post's rating of informative.

    • that sucks (Score:5, Funny)

      by Anonymous Coward on Sunday August 28, 2005 @04:59AM (#13420094)
      I can't tell you how many Christian record stores I'm permanently banned from.

    • Re:iTunes is a monopoly (Score:5, Insightful)

      by PaulusMagnus ( 797138 ) on Sunday August 28, 2005 @07:15AM (#13420362)
      Reads just like one of those anti piracy adverts the MPAA forces us to watch at the movies, or that FACT in the UK put on their DVDs.

      Piracy happens because technology happens. We pirate music because it's easy to copy and considerably less than buying it. We don't pirate books because it's frankly too expensive in photocopying charges but there's a whole collection of pirated PDFs out there, if you care to look.

      Technology changes the world we live in. I don't recall the Horse & Cart Association of America (HCAA) suing people that moved to cars which put them out of business. I also don't recall the MPAA or RIAA suing Intel, IBM or Microsoft for giving us these tools that enable us to pirate music.

      If piracy destroys the music business, so be it. Technology often destroys antiquated business models whether it's children cleaning chimneys, horse drawn carriages, coal mining or farming by hand. These people need to find a business model that works. An artist only makes around 5% from every track sold, the label and distributors cream off the rest. That's unfair, IMO.

      Why do we also need to have movie distributors for every corner of the world bidding for the distribution rights? Are we not one global market?

      I think it's about time that the movie and music industries were overhauled as they've had way too much power and too much of a monopoly for too long. After all, we're not killing people here with this technology, we're just changing lives. We're just hurting the profit margins, I thought this is what happened in a capitalist and democratic society. Why do we in the Western world create these societies with freedom to innovate and freedom to make money but then try to shackle them when it starts to backfire?

      Bring on the technology, lets keep changing the world!!!
      • How about we give the artist that right to choose whether they want to give their music away for free or not...that is what we do with our website. The artist will get more freedom with us, than with the RIAA...they really need to revamp their business model.

    • The World Changes (Score:2, Insightful)

      by xh3g ( 213494 )
      Everything changes, even how we make a living. I myself paint cars. If someone tomorrow came up with a cheap method for producing car parts that never need to be painted, I would be out of a job while the entire world heralded the new technology as a breakthrough.

      We all want to think that once we have our lives in order, they're going to stay that way. Nothing is guaranteed, not even whether yesterdays market will be here tomorrow.

      It's no one's fault but your own if you're selling something no one wants to
    • No, I'm pretty sure the song title is 2+2=5 [greenplastic.com].
    • I love your black list system. Where can I sign up. I want to be sure I'm on your list.
    • I don't sell sick stuff like Marilyn Manson or cop-killer rap, and I'm proud to have one of the most extensive Christian rock sections that I know of.

      Generally, in a business like this, you always want to cast a wide net. As soon as you start judging what music is "worth" carrying, based on values rather than economics, you are taking a pretty big risk. You aimed for a niche, and it looks like it hurt you.

      Before you point to the supposed "economic downturn", I'll note that the book store just across

  • Ok... (Score:4, Funny)

    by jim_v2000 ( 818799 ) on Sunday August 28, 2005 @04:51AM (#13420072)
    So from this we learn that the random play on iTunes really is random, and that rating a song really does have an effect. Who'd a thunk?

    Next, "iTunes really does play tunes!"

    • Re:Ok... (Score:5, Insightful)

      by QuantumG ( 50515 ) <qg@biodome.org> on Sunday August 28, 2005 @05:27AM (#13420156) Homepage Journal
      Confirming something instead of just assuming it is the case. What's that called again.. oh yeah, Science! Clearly this article is in the wrong section.

      • Re:Ok... (Score:5, Funny)

        by TheGavster ( 774657 ) on Sunday August 28, 2005 @09:52AM (#13420814) Homepage
        All lies! The play order is too complex to occur naturally; there must be some intelligence that selects the order of the songs!

        And, of course, looking into the origins of said intelligence is blasphemy.
        • Damn that's funny. Disturbingly accurate, but funny.


        • All lies! The play order is too complex to occur naturally; there must be some intelligence that selects the order of the songs!

          And, of course, looking into the origins of said intelligence is blasphemy.

          Ah but is it not manifest that this intelligence must be attributed to the FSM? May his noodly appendage touch our hearts, RAmen.
        • Yes! Yes! iTunes has been touched by His Noodly Appendage [venganza.org]! All praise be to the FSM!

          Ramen.

    • Re:Ok... (Score:2, Informative)

      by Jamu ( 852752 )

      Like most random numbers generated by computer they aren't really random. They're pseudo-random, generated by a deterministic algorithm and having the appearance of randomness. The less processing power you dedicate to the algorithm, the less random the numbers will appear on inspection. The iTune statistics can only be as good as the algorithm they used to generate their (pseudo-)random numbers.

      You can find true random numbers here [random.org] and also some more information on pseudo-random and random numbers.

      • Re:Ok... (Score:2, Insightful)

        by Mozk ( 844858 )
        Being an almost firm believer in determinism, I would say that there are no such things as (true) random numbers. If we were able to monitor the atmospheric noise (movements of atoms), and predict it perfectly, the numbers would be just as random as pseudo-random numbers generated by computers. However we are nowhere near that kind of technology, like the Googleplex Star Thinker in H2G2, so for the time, yeah, it basically is truly random.
        • How could you be a true believer in determinism with what we know now about quantum physics?

          There is almost no evidence of determinism in any walk of life. The bible, Science, and day to day experiences. People are unpredictable and so is quantum reality (which creates the lives we see). Even the philosopher would normally disagree with determinism.

          Now, if they come back and say they can predict quantum reality determinately, then you might be on to something. Until then I find it hard for anyone to believ

    • Re:Ok... (Score:2, Insightful)

      by SpinyNorman ( 33776 )
      But take a look at figure 4...

      Rating your songs has an effect, but having done so it often makes little difference whether you use random vs rating-biased play! It seems the difference between these two options is dominated by rating distribution, not by individual ratings!

  • I am not sure I see what he sees (Score:3, Interesting)

    by ReformedExCon ( 897248 ) <reformed.excon@gmail.com> on Sunday August 28, 2005 @04:52AM (#13420075)
    I'm looking at this data and it seems that iTunes does seem to pick out favorite songs more often than not-so-favorite songs. Which, I suppose, is the whole idea behind the Party Shuffle concept.

    So after analyzing all that data, how does Brian Hansen come to the conclusion that "it's simply the mind's tendency to find a pattern that makes you think iTunes has a preference". Uh, no. It's the software learning that you have a certain type of genre or style that you strongly favor and will selectively pick songs that are related, thus giving you a better-selected playlist.

    And it seems that the program has a bug in that it will play a song twice in a row. That's a real bug (if you don't like that type of thing).

    • Re:I am not sure I see what he sees (Score:5, Informative)

      by jim_v2000 ( 818799 ) on Sunday August 28, 2005 @04:56AM (#13420087)
      I think his point was that with a random order, it is possible for the same song to play twice in row. Not likely, but possible. He then goes on to say that people sometimes try to find patterns where there are none...which is correct. iTunes just happened to play the same song twice randomly.

      • You can only play the same song twice in a row if the algorithm reshuffles the songlist after every song played.

        If you do a static shuffling, i.e., a shuffle at the beginning of playback, and then trudge through the playlist that was generated then you will certainly get each song played the same number of times, and you won't get repeats. The only chance of getting a repeated song is if the last song of a shuffled playlist is the same as the first song of the next shuffled list, which is 1/n^2.

        You can comb
        • Now imagine that system if you've only got 1 song with 5 stars. Your "5 star" group has a slightly better chance of being selected than your 3 or 4 star group, which means that everytime it's "5 star song time" you'll get the same song. If your 3 and 4 star lists are 1 hundred or so songs strong each, you'll very quickly notice the repetition.

          Obviously 1 song in the group is a bit of an exaggeration, it gets the point across. That idea wouldn't work :)

          • You could simply have it avoid the "select from rating group" method the GP was referring to if it sees vastly unbalanced rating group sizes (at least, at the high end... since the rating 1 would hardly ever get played, it wouldn't matter much if there was only 1 of them).
          • Sure it would. When it selected the 5-star category, the "next song" pointer would be null, so it would pick again. You wouldn't reset the groups until the last unplayed song was played. Which could suck, your tunage would start out strong and get progressively weaker as the party went on. Oh well, I never said it was a perfect solution, just that you could make it work! ;)
        • The only chance of getting a repeated song is if the last song of a shuffled playlist is the same as the first song of the next shuffled list, which is 1/n^2.

          No, it would be 1/n^2 if that song had to be a particular song. The probability of any song repeating, it's 1/n.

        • You can only play the same song twice in a row if the algorithm reshuffles the songlist after every song played.

          I believe TFA said that he noticed that the same song was on the list twice, so obviously he could have gotten the same song twice in a row. Kind of a no duh moment there, but whatever.

  • Reminds me of... (Score:5, Funny)

    by Anonymous Coward on Sunday August 28, 2005 @04:54AM (#13420081)
    the time my 2G iPod seemed to have a liking for the Aphex Twin's Selected Ambient Works Vol. 2. It was playing a track off it pretty much every other song. Those of you who know the album can appreciate that it's not the kind of music that you'd maybe choose as everyday listening material.

    It became so annoying that I ended up removing the album from iTunes, at which point my iPod promptly died. The replacement was big on Roxy Music IIRC...

    • Re:Reminds me of... (Score:4, Interesting)

      by hattig ( 47930 ) on Sunday August 28, 2005 @05:15AM (#13420124) Journal
      Maybe songs need more than one rating.

      Rating For Morning Listening (* for Aphex Twin, Slayer, etc)
      Rating For Afternoon Listening (**)
      Rating For Evening Listening (****)
      Rating For Party Listening (**)
      Rating For ${mood} Listening

      Then instead of getting work done we can spend out entire lives rating music.

      • Re:Reminds me of... (Score:4, Interesting)

        by Gorath99 ( 746654 ) on Sunday August 28, 2005 @06:56AM (#13420322)
        I was thinking about something like this myself. Basically, what I'd like to have are two flags:

        1: Never play unless I explicitly say so.
        2: Don't include in shuffle.

        The first one I'd use to flag interviews etc. that are sometimes included on albums. Is not necessarily bad content, just something that you don't generally need to hear multiple times.

        The second one is for flagging things like Beethoven's 9th. It's really good music, but you don't want 67 minute long pieces in a random playlist.

        I currently just use the 1 and 2 star ratings for this, but it's not really ideal. It's too bad (but understandable) that iTunes has no option for looking at TXX frames [id3.org] or I could implement it in a better way.
        • If you deselect the track in iTunes, it should never play it unless you double-click on it.

          • Re:track selection (Score:2, Insightful)

            by Gorath99 ( 746654 )
            With the checkmark, you mean? True, that works, but I find the thing to be the most confusing part of the entire interface.

            The iTunes help says that:
            "A black checkmark next to a song means the song will be included in your next action. For example, only songs with black checkmarks are transferred to your iPod or imported when you click the Import button. Or, when you're playing all the songs in a list, only the checked songs play."

            But meanwhile, if I select a bunch of tracks and I set a rating via the conte
            • I'd prefer to use an explicit tag that does exactly what I want it too and no more or less, rather than this checkmark thingy that seemingly arbitrarily affects some things, but not others.

              I use the 'Comment' field for those kind of things. If Comment contains "Not for shuffle", keep out of Smart playlist which is the source for the Party shuffle. Etc.

              JP

        • Re:Reminds me of... (Score:4, Interesting)

          by Lars T. ( 470328 ) <Lars...Traeger@@@googlemail...com> on Sunday August 28, 2005 @08:49AM (#13420615) Journal
          Use unique strings (as many as you like) in the Comments tag. Like "-don't play unless I say so-" or "-don't include in shuffle-". Then build intelligent playlists accordingly (Comments doesn't include "-don't play unless I say so-"). Errm, better use something shorter like "-DPlay-".
          • Yeah, I considered going that route myself. I eventually decided to stick to stars + intelligent playlists since that works much the same. The comment route is definitely more flexible though. Probably more portable as well, so I may come to regret my decision :-)
          • Actually, there's a 'Grouping' metafield which is better used for this so you don't hose your proper comments.
        • as for number 1, ever wonder what that little check box was for?
        • I was thinking about something like this myself. Basically, what I'd like to have are two flags

          You could probably work something out using keywords in the "Grouping" field. I create smart playlists by putting keywords like "Mellow" or "Work-out" and then create "Mellow" and "Work out music" playlists by pulling songs with those words in the grouping field. This way the real genre of the song is not changed, but I create my own groupings.

      • Re:Reminds me of... (Score:5, Interesting)

        by Dixie_Flatline ( 5077 ) <vincent@jan@goh.gmail@com> on Sunday August 28, 2005 @10:51AM (#13421016) Homepage
        I use the stars to indicate how often things should be played.

        * - Never play. It's only in the list for the sake of completeness (I hate having partial albums)

        ** - Play very rarely. If I'm in the mood, I might listen to it.

        *** - I'll listen to it at least once a week. If it comes up randomly on the shuffle, I won't take it out of the list.

        **** - I can listen to this several times in a day.

        ***** - I'll listen to this song anytime, anywhere. If it comes up twice in a row, no problem. If my playlist only has this song on it, I can cope with that for at least a few hours.

        This means that I have to periodically re-rate the songs. That seems only reasonable, though. Why would songs stay at the same rating forever? As the novelty wears off, I can relegate a song to 4 or 3 stars.

        I also keep extensive smart playlists that make sure that songs that are 3 stars or less only get played once every few days.

  • Interesting (Score:5, Interesting)

    by hattig ( 47930 ) on Sunday August 28, 2005 @04:57AM (#13420089) Journal
    I wish iTunes would get ratings from some online source much like it gets tracknames from Gracenote. Can you imagine a server of user-submitted ratings? You could opt to use an average rating from all users, or a rating from users with particular tasks (i.e., if you are a metaller, then you'll probably not want raver's musical opinions affecting your ratings!).

    Why? Because I haven't got the time to go around rating my entire music library. Judging from that article, it is dangerous to only do a few because of the weighting algorithm used - surely it would be more sensible to assume that 'not rated' meant 3 stars rather than 0 stars? That way you could rate down shitty songs, and rate up excellent songs, but ignore rating the vast majority of songs.

    • Re:Interesting (Score:4, Interesting)

      by the_unknown_soldier ( 675161 ) on Sunday August 28, 2005 @05:03AM (#13420101)
      I only rate good songs... Indifference is the worst rating you can give to a song, so i think "0" fits in pretty well

      As for Gracenote: perhaps sales on the ITMS could act as a gauge of this. e.g. "This is this artist's most downloaded song and this artist compared to similar ones is bought 5x as much, so our algorithms suggest it should be rated 5" Then once you have downloaded it you can change it if you get the time.

      • Re:Interesting (Score:2, Insightful)

        by Anonymous Coward
        Problem is, then you'd end up playing what ITMS wanted you to play. And then they'd get paid by record companies to fix ratings slightly. And... well, you can probably work it out from there.

    • Re: Try last.fm (Score:5, Interesting)

      by P!Alexander ( 448903 ) on Sunday August 28, 2005 @05:33AM (#13420170)
      That's exactly why I love last.fm [www.last.fm] (formerly Audioscrobbler & Last.fm). It automatically tracks what you listen to and then allows that information to be used to give you neighbors in the music world based on what interests you have in common. You can add friends, join groups, and even tag your music. All of this is extremely useful in finding new stuff. They've got plugins for all the major media players (and even some minor ones).

      Add on top of that the ability to play a custom-built radio station, set it to play only new music or listen only to music from a particular user profile.

      Linux and BSD supported! Open source plugins and radio station player! Could it get better? ;)

      ---
      but make sure that the last line
      Generated by SlashdotRndSig [snop.com] via GreaseMonkey [mozdev.org]

    • You wonder why the music industry is mad (Score:2, Insightful)

      by Anonymous Coward
      "I haven't got the time to go around rating my entire music library"

      Much less pay for it, I assume. I'm a certifiable geezer by slashdot standards (>40), but I do know a thing or two about music. I was a really hardcore music collector in college, and one core credo was to know everything in your collection inside-out. Practically speaking, it meant that any new addition to the library (vinyl or cassette, if you've heard of either) stayed in a "just-in" quarantine section until I had listened to it en
      • Music has turned from something that you collect and treasure into something you have and listen to practically all the time. It is very rare you decide you want to listen to just song X these days (in comparison to how much music is listened to overall), and actively put it and actively spend the time solely listening to it. Large mp3 collections have replaced radios at many places, great for getting rid of the music you really dislike and the DJ.

        Would I pay to have my music rated by an external algorithm?

    • Re:Interesting (Score:3, Informative)

      by RAMMS+EIN ( 578166 )
      Some open-source music players have a system where music is automatically rated based on how often you select the song to play (and possibly on how often you interrupt it once playing). This always seemed like a good way to make the program learn what you like.

      I can't really say how well this works in practice, or which programs support it, because I don't use the feature myself. However, I suspect it would work better than an explicit rating system, much like bayesian spam filters work better than explicit
      • Windows Media Player also has this feature, and rates all "unrated" tracks at a default of three stars. Anything that gets played often gets bumped up, anything that's frequently skipped gets bumped down. Anything that you rate yourself is excluded from this auto-adjustment.

    • Re:Interesting (Score:5, Interesting)

      by Stuart Gibson ( 544632 ) on Sunday August 28, 2005 @06:21AM (#13420262) Homepage
      Select All -> Get Info -> My Rating -> Three Stars.

      Rate up and down others as necessary. OK, not the point that default should be doing this for you, but a quick fix if you want it to work that way.

      If you already have songs rated then create a 0 star smart playlist and repeat.

      Stuart

    • Re:Interesting (Score:5, Funny)

      by Feanturi ( 99866 ) on Sunday August 28, 2005 @06:30AM (#13420275)
      A public moderation system, cool. That never gets abused anywhere that I know of.

    • Re:Interesting (Score:2, Informative)

      by rimbaldi ( 596256 )
      The Music Recommendation System for iTunes [uiuc.edu] from the University of Illinois doesn't solve your problem because it requires you to rate most of your own library. However, it might be an interesting step on the way to solving the problem you raise.
    • If you are interested in something like that, check out Last.FM [www.last.fm] powered by AudioScrobbler. It will let you know other people who share your same musical interests. Instead of "rating" songs, it simply analyzes what you listen to and lets you know what others listen to.

      AudioScrobbler plug-ins are available for many popular players.
      • It's not really powered by audioscrobbler, it's a dang replacement for the audioscrobbler site. I much preferred the old audioscrobbler site. Damn shame.

  • Finally (Score:5, Funny)

    by Ed Thomson ( 704721 ) on Sunday August 28, 2005 @05:03AM (#13420099)
    Someone to show how cool mathematics is

  • Underlying formula (Score:5, Informative)

    by Pemdas ( 33265 ) * on Sunday August 28, 2005 @05:45AM (#13420191) Journal

    From their results, I'd venture a guess as to the underlying algorithm:

    Each song is given a number of points equal to (rating + 1). Then the probability of the song being played is (song rating)/(total points).

    Or, to put more succinctly:

    prob(song) = (rating)/(n + sum(i=1..n)(rating(i)))

    That yields probabilities in the given test case of:

    5 star - .285
    4 star - .238
    3 star - .190
    2 star - .143
    1 star - .095
    0 star - .048

    Which is reasonably close to what the author found. Heck, if I were implementing that feature, it's what I'd try first...

    • Re:Underlying formula (Score:5, Interesting)

      by ciroknight ( 601098 ) on Sunday August 28, 2005 @06:07AM (#13420239)
      Your decimals look more like the pricing model than the weights for playing songs..

      5 star - .285 -- $299, iPod (full?) 20gb
      4 star - .238 -- $249, iPod mini 6gb
      3 star - .190 -- $199, iPod mini 4gb
      2 star - .143 -- $149, iPod shuffle 1gb
      1 star - .095 -- $99, iPod shuffle 512mb
    • is this the algorithm MS patented? I wouldn't be surprised...
    • Which is reasonably close to what the author found.

      Your numbers may seem "reasonably close" from a casual glance, but if you do the math, even for 99% confidence intervals his numbers are accurate within +- .005. Your approximation falls well outside this. Note he had a 52,000+ sample size -- that's pretty big!

      Yours is a nice guess though, perhaps only slightly off in some way. I agree iTunes must use some simple formula or algorithm to derive the probabilities, and not the crazy equation in the link

      • the REAL underlying formula (Score:5, Interesting)

        by Bert690 ( 540293 ) on Sunday August 28, 2005 @12:37PM (#13421427)
        OK, after a bit more thinking, you were indeed very close. It appears the actual formula is:

        points(0 stars)=1
        points(1 stars)=3
        points(2 stars)=4
        points(3 stars)=5
        points(4 stars)=6
        points(5 stars)=7

        probability(X stars) = points(X stars) / 26

        This yields the following probabilities, listed along side the observed values from the article along with 95% condience intervals.

        p(5 star)=.2692 [.270 +- .0038]
        p(4 star)=.2308 [.230 +- .0036]
        p(3 star)=.1923 [.189 +- .0033]
        p(2 star)=.1538 [.154 +- .0031]
        p(1 star)=.1154 [.118 +- .0027]
        p(0 star)=.0385 [.039 +- .0016]

        As you can see each computed probability falls within the 95% confidence interval, so there's a good chance this is the correct forumla.

        Boy do I have too much time on my hands today.

  • Why Assume a Bell Curve? (Score:5, Insightful)

    by TubeSteak ( 669689 ) on Sunday August 28, 2005 @05:46AM (#13420195) Journal
    Seriously, on what basis is he assuming a bell curve of preferences?

    You'd think, with iTunes, that people would be buying music they like (a four or five rating) in a much higher proportion than music they'd rate as a three.

    Then there's music added from your own collection. Maybe its just me, but my preferences tend to be stronger than -, 1, 2, 3, 4, 5.

    I usually go through my music collection on a regular basis and delete crap that I don't listen to, which is usually anything less than a three, and definitely a - or a one.

    And is 4334 just a random arbitrary # of songs to use?
    (when you add up X0 through X5)

    • I was gonna go with an n/t here, but you have some other points that I can address as well.

      Some of us aren't music nazis. I am; I keep all of my music in a custom database, with a custom structure designed for tagging which I'm currently working on trying to design a way to serialize to ID3 without success. That being said, for the average person, they just keep whatever music in their iTunes, and if they care enough about the song, they rate it.

      Personally, I use 1-5 ratings, and leave no stars to mea
    • Most people follow a bell shaped curve for their ratings, with the 3-star rating being the most common.

      I mean, where is this statistic coming from?

      In my case the majority of rated songs are 5's, almost the same number of 4's, then some 3's, and hardly any 2's or 1's.. with perhaps 50% left unrated. I use iTunes at least several hours a day. Those of my friends who use iTunes seem to have a similar distribution.
    • Agreed. I've never rated a song lower than three, and I have very few threes. All of the songs that I like are either four or five, and the songs that I don't like are left unrated.

      I do this because I use the "My Top Rated" smart playlist and I only rate songs I want to go in there. Although I keep the crap I don't listen to.
    • Seriously, on what basis is he assuming a bell curve of preferences?

      When you look at data, particularly from modest numbers of samples, it seldom fits the bell curve nicely. Often it is multimodal.

      However, the Guassian (bell curve) distribution has one important property: it is the distribution that (given a number of stipulations like having a finite integral over any interval, and there being no set uppler or lower limit to the variable) has the highest entropy. It is, in a sense, the most random of ran
    • I usually go through my music collection on a regular basis and delete crap that I don't listen to, which is usually anything less than a three, and definitely a - or a one.

      I have a lot of stuff on my system I don't care for. Mainly because my daughter likes it (although mostly she has good taste), but partially for guests.

      I draw the line at Rush, though. No Rush is allowed to play in my house...
    • Like I mentioned in another thread, I use the system to indicate my preferred frequency of play for the song. One star means that it's just around so that I'm not missing parts of an album, and 5 means that I love the song so much I can listen to it continuously for hours on end. If you only use part of the rating system, it just means that you're missing out on some granularity. The 0-5 star system doesn't mean anything except what you decide to make it mean.

      If you own the music, you already know that you
    • You'd think, with iTunes, that people would be buying music they like (a four or five rating) in a much higher proportion than music they'd rate as a three.

      You're not accounting for whole albums ripped/downloaded. Even the average CD that I love usually will contain at least one crappy track, and several "okay" tracks. Most CD's I have on my hard drive weigh in at about 25% 4/5 star, 50% 3 star, and 25% crap (2/1 Star). Why don't I delete the crap songs? Because I like to maintain the integrity of th

  • Some calculations errors in my opinion.. (Score:5, Informative)

    by Fr4ncis ( 763671 ) on Sunday August 28, 2005 @05:48AM (#13420198) Homepage
    If you have 2000 songs and 40 of them are from the same artist, there is always a 2% chance of hearing them next with random play. So right after one of their songs finishes, odds almost guarantee they will be played again within the next 50 songs and show a 50% chance they will play again within the next 25 songs. It's simply the mind's tendency to find a pattern that makes you think iTunes has a preference.

    A way to calculate the odds that 2% will be played in the next 50 songs doesn't work 50* (2/100) = 100% as the author does, and neither 25*(2*100) = 50% is correct.

    The correct calculations are: 1-(98/100)^50 = 63% and 1-(98/100)^25 = 39%.
    This way you calculate the odds a song will be played at least once in the next 50 or 25 songs.

    If you want to calculate the odds the song will be played exactly once in the next 50 or 25 songs:

    50 * (2/100) * ((98/100)^49) = 37% or 25 * (2/100) * ((98/100)^24) = 31%.

    I guess that's all..

  • From the article...trick of the mind (Score:4, Informative)

    by djupedal ( 584558 ) on Sunday August 28, 2005 @05:50AM (#13420202)
    For those too lazy to go read for themselves...

    "Many claim to still see patterns as iTunes rambles through their music collection, but the majority of these patterns are simply multiple songs from the same artist. Think of it this way: If you have 2000 songs and 40 of them are from the same artist, there is always a 2% chance of hearing them next with random play. So right after one of their songs finishes, odds almost guarantee they will be played again within the next 50 songs and show a 50% chance they will play again within the next 25 songs. It's simply the mind's tendency to find a pattern that makes you think iTunes has a preference."
    • Moreover, following the "Birthday Paradox" [wikipedia.org], if you have N songs and the selection is completely random, then in a list of sqrt(N), there's a 50% chance a song will appear twice.
      For 4000 songs, that's around 64~ songs. So if your player chooses tracks completely randomly then 50% of the times you'll listen to 64 songs, you'll hear the same song twice from those 64.

      Even if your player doesn't play the same song twice, if you have 8000 songs from 4000 artists, 2 songs per artist, then you get a similar calcula
      • This calculation assumes that random song picking is done every time the iPod switches songs, which is not the case. When you turn on shuffle and hit play, the songs are sorted into random order and then played in that order, so a particular song is guaranteed not to show up twice until all other songs are played (all else being equal). You can see this by skipping forwards and backwards during shuffle play- it will stay in the same order until you go off the top or bottom of the list or interrupt and resta

        • Hence the shuffle terminology, not random play. It's like shuffling a deck of cards and going through it in that new random order, rather than picking songs at random from a hat and putting them back when you are done.

          B

  • Rating songs (Score:2, Insightful)

    by ptimmons ( 235569 )
    I take a small bit of exception to the author's statement:

    "Although the higher rated songs are given preference, you will not definitively hear more 5-star rated songs than all other ratings. Most people follow a bell shaped curve for their ratings, with the 3-star rating being the most common."

    In fact, I find that I tend to rate only the songs I *really* like and *really* dislike, and leave the average songs alone. I suspect that I am not alone here. It's akin to the trend of many online forums to attr

  • For anyone looking on an interesting read about stats (that is actually accurate) check out the Birthday Paradox [wikipedia.org].

    The whole "good chance that the same artist will come up in the next 50 songs" is actually the same type of math as the Birthday Paradox. (larger set)

  • Modal Music (Score:5, Interesting)

    by Johnny Mozzarella ( 655181 ) on Sunday August 28, 2005 @08:35AM (#13420572)
    A friend of mine who worked at a radio station that played a very diverse range of music told me how they select music.

    She said that research had shown that listeners would rate the same song higher if it followed other song of a similar genre. If they play songs of different genres randomly the listener does not enjoy the music as much.

    So their tendency is to play "blocks" of music.
    For example....
    4 Classic Rock songs
    3 Blues Songs
    3 Folk songs
    4 Female Rockers
    3 Grunge
    etc.

    This is common knowledge in the radio world. I wonder if Apple has incorporated this type of logic into it's iTunes algorithms?

    The radio station in question is WXPN and can be found under iTunes > Radio > Public > WXPN

    • Re:Modal Music (Score:3, Informative)

      by WebGangsta ( 717475 )
      So their tendency is to play "blocks" of music. For example....
      4 Classic Rock songs
      3 Blues Songs
      3 Folk songs
      4 Female Rockers
      3 Grunge

      Not entirely true, and it depends on the station (as you stated). Some stations make it a point *not* to put songs with female lead singers together; *not* to put songs from the same R&B/Dance/whatever genre together; *not* to put songs from solo artists next to each other. And so on. And don't forget issues with playing more than one old song after another.

      A

  • Strange math crunching (Score:2, Interesting)

    by mpiktas ( 740253 )

    After reading the article, I still do not understand the iPod's shuffling algorithm.

    The first half of the article is devoted to describing how the writer got the probabilities of rated songs and properties of these probabilities. Although these probabilities give some insight to the shuffling algorithm, they are pretty useless, since they are observed from unrealistic list of songs, i.e. 6 songs with different ratings.

    Then cames the formula in Figure 2. How it is calculated and where from the autho

  • something i've noticed (Score:5, Funny)

    by macsox ( 236590 ) on Sunday August 28, 2005 @10:37AM (#13420977) Journal
    there definitely seems to be some time-based randomness in the selection of tunes. often, i'll hear a song pop up randomly on my ipod in the car on the way into work, and then the song come up again, randomly, while being played on itunes at my desk.

  • Whisky Tango Foxtrot? (Score:4, Insightful)

    by Bug-Y2K ( 126658 ) on Sunday August 28, 2005 @11:41AM (#13421198) Homepage
    I read the headline and imagine a story about the costs and revenues from Apple iTMS. Analysis of running the datacenters: costs of electricity, bandwidth, storage, etc.

    Instead I read about some geek with way too much time on his hands. Yawn.

  • He should have measured the probabilities of 3 songs patterns for example. This way he would have detected wether iTunes implements something to prevent playing the same song twice in a row. That would be much more interesting I think. (of course to know this he needs to know a little more than just the play count)

  • Dear Apple (Score:3, Interesting)

    by NeMon'ess ( 160583 ) <flinxmid AT yahoo DOT com> on Sunday August 28, 2005 @03:06PM (#13422126) Homepage Journal
    Let me, an advanced user who knows a damn thing about computers and interfaces, change the weighting of the stars. I don't want my 5-star songs to play just twice as often as 2-star songs. I want them to 6 times as often. I want 4-stars to play 4 times, 3-stars to play 3 times, and no-stars to play 2 times.

    Why no-stars? Because that way the majority of the collection is unrated. Stared songs really stand out in a playlist. 1 and 2 star songs play less often than no-stars, while 3, 4, and 5 play more often. But I want my favorites to play much more often than your arbitrary algorithm.

  • Crunching the math on Slashdot (Score:5, Interesting)

    by AlpineR ( 32307 ) <wagnerr@umich.edu> on Sunday August 28, 2005 @06:02PM (#13423019) Homepage
    Nice analysis of iTunes. I'm somewhat surprised at the small difference in play frequency between 3, 4, and 5 stars; and disappointed that unrated songs are almost never played. In my collection, unrated means that the music is new to my collection. I think 1 star should be the kiss of death, not a blessing upon a previously unrated song.

    But all this talk of 0, 1, 2, 3, 4, 5 has me thinking of another rating system. Would anybody care to do an analysis of the ratings in Slashdot comments? What are the relative populations (I expect a ton of 2's but how about the rest)? Do comments made in the first hour after a story is posted stand a better chance of reaching +5 than comments made later in the day?

    One of my gripes about the Slashdot comment system is that it discourages contemplation and discussion. Comments made more than 24 hours after a story is posted are rarely read and almost never moderated. This is in contrast with comments system like Usenet or other bulletin boards, where threads can remain lively for weeks.

    AlpineR

  • Car mp3 CD player (Score:3, Funny)

    by suitti ( 447395 ) on Monday August 29, 2005 @02:26PM (#13429791) Homepage
    Two years ago, i replaced my car's tape player, which died after only 15 years of service, with an MP3 CD player. I cut a CD with 14 hours of my favorite stuff. I put the CD in the car, set it up to play in shuffle mode, and set out on a cross country trip. It was great. Just before i arrived, i heard a repeat. I was so disgusted i hit the eject button. Fortunately, i had another disc with me. Feh, i said. Can't even go 750 miles without having to change the CD.

Slashdot Top Deals

Mater artium necessitas. [Necessity is the mother of invention].

Close