Below is my precise illustration of how the so-called shuffle feature is broken. This is not a problem of people simply not understanding "true randomness". The shuffle feature will keep playing your most popular tracks first and it will keep putting the least-popular tracks near the bottom of the list. This means that you will rarely hear your least-popular tracks unless you listen to the entire playlist all the way through.
My newly-chosen, purposely “unpopular” track (Carnaval de Guaranda) was never listed higher than 77th in the shuffled list. Out of 50 trials, Carnaval de Guaranda was presented, on average, at the 87th position in the shuffled list. That’s not random. And that’s not what most people think of when they talk about “shuffling”.
So what difference does this make, as long as the song does come up SOMEWHERE in the list?? Well, if you don’t listen to the entire playlist, all the way through, you’ll likely never hear Carnaval de Guaranda. And if this “unpopular” track is always shown near the end of the shuffled list, that means that significantly-more popular tracks will always be shown nearer the beginning of the shuffled list. So you'll be more likely to hear them during an "average" listening session.
People rightfully complain that Spotify’s “shuffle” algorithm stinks. If you have sizable playlists, you’re usually not going to listen to the entire playlist in one session. You’ll listen for a few hours to one playlist, then for a few hours to another, and then maybe back to the original. But when you go BACK to the original, you’re starting back at the beginning of a newly faux-shuffled list. So you end up hearing a much stronger concentration of the same songs – even when you have a very large playlist.
Even if you did listen to the entire playlist all the way through, do you always want to hear the “unpopular” tracks near the end of that session and the “popular” tracks near the beginning?? If you added it to your own playlist, it’s probably “popular” in your mind.
We understand your situation @bytebodger!
We're looking into it and we're working on our shuffle algorithm. Keep an eye on any updates soon.