Your Ideas At Work: Shuffle Improvements

Spotify
Spotify

The Community Ideas Board isn’t just about requesting new features. It’s also about improving the features already in Spotify.

 

Take our shuffle algorithm for instance.

 

The Idea “Implement an actual shuffle function” by Community user @RoninTheOrigina  gathered over 850 votes.  Users were vocal in their comments about what they wanted (and didn’t want) when it came to shuffling playlists.

 

We then passed this knowledge on and our teams got to work.

 

The result: an improved shuffling algorithm that avoids playing a couple songs from an artist too close together.

 

shuffle algorithm.png

 

If you’re thinking, “that sounds kind of vague, what are these improvements” then fret no more.

 

Spotify’s @lukasP  has written an extensive blog post on how we gathered user feedback on Shuffle, analyzed their comments, took a hard look at our previous algorithm and found the best way to bring the improvements users wanted.

 

We’re aware this doesn’t fix all shuffling issues forever. Rest assured we are still working on this though.  You’ll also see the new algorithm in other clients other than desktop soon.

 

Now go hit shuffle on your favorite playlist and reap the benefits of your hard work clicking that Kudos button.

 

Enjoy!

 

105 Comments
Kaano
Newbie

I am having this same issue that everyone is reporting, shuffle is not actually random, it is prioritizing a certain artists/songs above everything else. Everytime I shuffle my playlist of 800+ songs I almost always hear this one song within the first 10 songs. I actually deleted the song because I had heard it so much I was sick of hearing it. This is a simple thing, why has it been years without a fix?

kneipe2
Regular

One of the biggest problems is that they're not randomizing, or at least not very well, their seeds which is one of the major challenges to creating a true randomizer but they seem to be completely oblivious to this fact or just don't care. As to the "Gambler's Falacy" section of their condescending and arrogant blog post they have their own fallacy in the logic they're using to (wrongly) justify their terrible ability to code and implement known algorithms. They use the example of a coin throw:

 

"If you think the coin has higher probability of deciding for Indian on Friday, you are wrong. Throwing the coin for a millionth time is the same as throwing it for the first time. After all, it is just a simple coin, it has no memory, doesn’t know who threw it, etc. So both heads and tails have the same probability on Friday – 50%."

 

The difference here is that the coin is not two sided. If you have a playlist or artists catalog with several hundred songs you are now throwing a several hundred sided coin and so the probability is no longer 1 out of 2 (50%) it's now 1 out of several hundred or, for say 300 songs, 0.33% that it will play. They use the example of a two sided coin to obfuscate the fact that their algorithm just plain sucks and obviously has built-in biases they're not telling us about. Specific artists and songs are given priority either because they wrote the algorithm that way or because they don't know how to write the program properly. Either way it shows severe incompetence on their part. They are probably also unaware of the issues involved with a mapping between continuous/sem-continuous sets of numbers and a discrete set, which is also an inherent problem with writing a good randomizer program.

 

They keep blowing smoke and throw in enough jargon in their explanation/excuse to make sure to confuse those that might not have a background in math, but it's fairly obvious they don't understand the math or computer science behind the problem very well themselves.

 

Keep up the good work spotify and then keep scratching your heads over why more people aren't signing up for premium.

Colach
Music Fan
Im not sure they are selecting music for me deliberately. Because they still play a lot of unknown strange music that i have in my list. I mean it does not matter what kind of tracks are in a list - the randomizer algorithm still sucks. So I think that its just an inability/ignorance on their part that the shuffle fails.

Shape up spotify programmers!!!
_daniel_
Music Fan

I am officially giving up hope now.

 

I have complained, I have experimented, I have sent inbox messages to Spotify staff, I have suggested, supported, kudosed... I even wrote a letter to the Spotify CEO on May 25th 2015.

 

Guess the result of all this effort?

 

Right, so here's my letter to Daniel Ek:

 

 

 

Spotify Sweden AB
To: Daniel Ek
Birger Jarlsgatan 61, 10tr
113 56 Stockholm
Sweden                                                                                                                                                               May 25th 2015

 

Dear Daniel

I love music. I really do. I listen to music when I get up. I listen to music on my way to work and on the way back. And I listen to music at home for hours and hours, every single day. I do love music.

 

And not just one song, or one artist, or one period, or one genre – I love it all. When I’m having breakfast, when I’m working out, when I have friends come over, even when I’m watching football. I love to listen to music just about the whole time.

Some time ago a friend recommended Spotify to me and I felt heaven was close. All the music was finally in one handy app and provided me with unlimited enjoyment, it seemed. It didn’t take long until I signed up for the premium version and I thought I’d never regret this decision. Despite all my love for music, however, this day has come. I started love-hating Spotify and I ask for five minutes of your time so I can tell you why.

 

It all started when I listened to a playlist on shuffle mode and recognised that some songs were repeating over and over again whereas other songs seemed to have never been played. I searched the Internet and found the Spotify community. In there I found a thread called ‘Your Ideas at Work: Shuffle Improvements.’ In this thread Meredith claims Spotify has solved a problem related to the shuffle mode. She goes on to explain how the algorithm has been improved and shuffle problems are an issue of the past.

However, if I still experienced problems after this post was written, how could it have been fixed in the first place? So I read and read and found out that Meredith was talking about a completely different problem than the users experienced. While Spotify fixed an issue where several songs of one artist might have been played in a row, users were complaining that particular songs of their playlists were played over and over again whereas others were never chosen by the shuffle mode.

Here are a few comments of your users:

 

‘The shuffle function still isn't working correctly in that it IS NOT random. I have a playlist with 400 songs and the shuffle function plays the same 50 -100 songs over and over.’ – shatkinson

 

‘I have almost 6,000 songs on my playlist and i hear the same songe over again in a single days listening’ - kjbllc 

‘…it's really annoying to have to take time to flip through 20-30 songs that I hear EVERY TIME I play the playlist, just to get to something "new".’ - mollieneff

 

Moisaras summarises it perfectly.

‘It's almost unbelievable to me that such a simple process is broken.’ - moisaras

And continues on to say,

‘It would be great if we could get some feedback from Spotify on this. It seems we are talking to ourselves here..’ - moisaras

 

But hey, Meredith returns.

‘…we're always looking for ways to improve/ gather feedback. Could you give us more specifics about what you're seeing? Is it all on devices?’ – Meredith

 

Feedback is given (pretty much the same as above) but nothing much is changing.

‘…this bug drives me insane!’ – Peter (47,000+ posts, heavy user I’d assume)

This is where I enter the stage and after a small experiment which took me hours to come up with and conduct I wrote about my experiences.

 

I created a playlist of 50 songs and conducted a little experiment: I shuffle played all 50x songs and to my surprise each song was only played ONCE until each single song of the playlist was played. After that the shuffle function started all over again and shuffle played the 50x songs all over again. In total I repeated this experiment four times and to my surprise it worked perfectly. No song was played twice within a 50x-song-cycle.

 

But because my feeling told me, something must be wrong, I continued with other experiments. I hibernated my laptop but still, the shuffle play worked perfectly.

 

Eventually I found the problem (at least what I consider to be MY problem). I listened to 25 songs out of my 50 song playlist and switched off Spotify. When I switched it back on again Spotify would still remember which song I played when I switched it off. However, Spotify forgot which 25 songs I played before so the whole shuffle play started all over again, also playing the 25 songs it had already played.

 

The same happened when I played a song from another playlist, artist or album. Spotify simply doesn’t remember the songs it had already played.

 

To be honest I’m not sure whether this means Spotify should remember the songs it has already played or not, or whether there should be an option to switch that function on or off. For me personally it means, shuffle works just fine in a lab environment. But in real life, at least in my life, where I interact and randomly choose to listen here and there and then return and forward and go back again, it means it is highly depressing and frustrating. So remembering the songs which had been played would just solve the problem.

 

It’s just a matter of time, I thought, help was on the way and the major bug that apparently no one at Spotify was aware of, could be eradicated. After all, the data I provided was well researched and could be repeated by everyone who was looking into the issue. So Spotify’s team could repeat it and come up with a solution – is what I thought. Stupid me.

 

That was on September 4th 2014.

Nothing happened.

 

Ok, I thought, maybe Meredith was too busy to check the thread so I wrote her a private message on September 29th.

 

Hi Meredith,

would you mind reading and replying my post in "Your Ideas at Work: Shuffle Improvements".  I conducted a small experiment and would be more than happy if you could confirm my results and even more, if a fix could be introduced.

 

Thanks a lot, best wishes

Daniel

 

Again I was hoping that this all was just a big misunderstanding and help was on its way. Stupid me. Again.

Days, weeks, months – nothing. No reply in the thread. No answer to my pm. No nothing – but the issue with the shuffle play. Every time when using Spotify. Every single time.

 

In the meantime users came up with really smart solutions, for example, SebastianSaint.

May I propose a solution?

Spotify Coders:
1) Create a data field for each song that stores the time I last played that song.
2) Whenever the player is on Shuffle, choose the next song from the current playlist only considering the 10% of songs on the playlist with the oldest LastPlayed value.

Here's my logic:
1) The current song will not be replayed until I have heard at least 90% of the other songs in the playlist.
2) It will work if I listen with or without interruption.
3) It will work if I switch between playlists containing some of the same songs.
4) The solution remains simple.

 

Sounds great, right? And simple, too. What else could Spotify ask for? Users spot the problem and solve it. For free. Great users, I’d say.

 

It’s pretty much one year ago when shatkinson first complained about the issue and still there is no fix. To be honest, I lost hope there’ll ever be.

 

Don’t get me wrong, if there was an alternate music player which would work for me, I was gone already. But just because there isn’t yet, this doesn’t mean there’ll never be one. And as soon as there is, I’m off (unless you start listening to us).

I also want to say I’m 32 years old, I have worked in several customer services jobs and all of that in a country to be known for its terrible service industry. But even here in Germany your behaviour would be considered unacceptable. I simply can’t understand how a global company is concentrating on new products rather than employing a few more people to moderate the forum and forward issues and suggestions to managers and developers and especially communicate with their customers.

 

Don’t you think we deserve some respect? After all, we are paying your wages. How can you ignore us?

 

And let me add one last point. Shuffle play is just one issue. Equally bad is the lack of an option which allows me to rate and tag songs. Just imagine how awesome it would be, if I could use several tags on one song and play all songs tagged, for example, with *positive* *electronic* -*dance* *120rpm* *cardio* *four and five stars* when I go running (*cardio*) because the sun is shining (*positive*), I feel like electronic but not dance music (*electronic* -*dance*), I want to go fast today (*120rpm) and I want to hear my best rated songs (*four and five stars*). Temporary playlists for each mood - that would be legendary.

 

But I don’t want to go into further detail here because, as you might expect, there is plenty of unheard threads and comments in your Spotify community you can read if you were interested.

 

I hope I made my point and I hope even more that someone at Spotify who actually cares reads this.

I still have hope left. Don’t let me down Spotify! Not again!

 

Best

 

 

Daniel

floyd831
Casual Listener
I can't create a better post than this. Spot on!
swearm
Casual Listener

so here is my suggestion for ramdom play!

 

I have a playlist with 400 songs and I select the button "RANDOM" (Not in spotify right now, but my suggesion is to implement this)

When I press play, each song is assigned a number between 1-400.

When next song is supposed to start, or I press "next song", a dice with, in this case, 400 values is digitally created and digitally thrown.

The next song that will be played is the number on the "dice"

 

If I during play add or remove songs, the dice is altered. So +20 songs means a dice with 420 sides, if 20 songs is removed the next dice to be thrown has 380 sides. This is very easy to program. Even the commodore 64 had a built in "random" funcion, accessable even from basic.

 

Anyway, by this procedure the next song is picked randomly. It also means that the chance to listen to the same song again is only 1 out of 400. You can even program this so if I have selected "repeat" all songs can be played (by 1 to 400 chance) and if unselected it can mean that this recent played song is excluded for the next throwns, means a dice with 399 sides and the number of the recent played song is not possible to be played. This procedure can be valid until I press stop, and by this the playlist is reset, or if I select repeat the list is reset. If I add or remove songs during play, the list alter and the dice equallly. If nothing is done, the playlist plays until the end and stops 400 songs later, and not a single one of the songs has been played twice.

 

I think this is what I really would like to see in spotify. For me it is of no problem that the same song can, in my above exmple, be picked twice, the chance is only 1 out of 400.

 

You can keep the advanced algorithm, but add another button labeld "SHUFFLE" for this funcion

 

//M

 

 

camport
Roadie

I have an alternative idea for shuffling without repeating songs: create a column or a right-click option to sort your playlist in a (temporary) random order. So the songs are played in a random order when the shuffle function is turned off. And if you want to create a new random order, click on that option again for a new random order.

I also submitted this idea in the Ideas subforum.

Mrdannyblob
Newbie
I don't see why it can't just put all the songs in a playlist into a random order then just playing them. It's as if after ever song it just picks another song from the playlist at random. The fact that this next song can be the same as the one that just played is a joke. I remember when I used to use iTunes and it would randomize the order of the playlist and just play it in that order over and over until you hit shuffle again, I see zero drawback to this method, aside from generating a very long playlist in a random order maybe. I've given up on using the radio function. Every time it just starts with the same songs. Out of all the songs in each album it still picks the same one from each album every time. Not only that but songs/artists I've disliked still come up over and over. There's this one band that I absolutely hate that comes up constantly on certain artist radios.

Here's a screenshot of the result of hitting the shuffle button on a Pearl Jam album:
https://www.dropbox.com/s/daq5nfyf26xudd3/Screenshot_2015-09-18-17-59-09.png?dl=0
Sirens played 3 times in a row while I was riding my bike, unable to skip the song. Future days played twice within 3 songs after that.

Seriously, how hard can this possibly be? This isn't a feature that Spotify made up and tried to perfect. Shuffle has already been done perfectly for years with literally every other audio player I've ever used. How have they **bleep**ed this up?
Harlowin
Roadie

Some of you must have some sort of bug that you're experiencing with the shuffle The way the shuffle presently works means that it shuffles the entire playlist. Yeah, you can only see 50 tracks in the queue, but the rest of the tracks are shuffled as well. I've tested this dozens of times. When you go through the playlist, it will never repeat a track until the end of the playlist because the shuffle is just an imaginary playlist in a different order. 

 

The problem is that some things reset the shuffle that probably shouldn't. If you click on a song anywhere on the playlist, it will reset the shuffle. Clicking to another playlist also resets the shuffle.

 

Also, reaching the end of the shuffle DOES NOT CURRENTLY RESET THE SHUFFLE. I have no idea why as I'm pretty sure it used to do this. So if you have a group of songs say "A B C D E F G" and you turn on shuffle and get the order "G E A B D F C", your playlist will persist in this order after you reach the end of the shuffle. With "repeat" on, it will continuously play "G E A B D F C G E A B D F C G E A B D F C" until you do something that resets the shuffle.

 

So yes, it does have its shortcomings but the algorithm itself makes it nearly impossible to repeat tracks just because of the way the shuffle is handled. It's the resetting of the shuffle that is causing the problem

FlyingPole
Newbie

So apparently spotify say their shuffle is random. The problem is with random is data will appear in clusters, it won't be evenly spread out and a piece of data could appear multiple times before others appear once. This happens because after a piece of data appears is still has the exact same probability of appearing as any other data in the set. i.e. Take the numbers 1-10. Each has a 1 in 10 chance of appearing in a random sequence. The first number could be 8. The second number could be 5. Now that numbers 8 and 5 have appeared does not mean they are less likely to be the next number in the sequence. They still have a 1 in 10 chance just like 1, 2, 3, 4, 6, 7, 9, and 10. This means after 10 numbers have been randomly put in a sequence the sequence could appear as 8, 5, 3, 7, 5, 2, 1, 9, 9, 4. Notice how there are multiple 5's and 9's but no 6's or 10's? This seems to be how Spotifies Algorithim works.

 

How it should work is after a song has been played its probability of being queued should change from being 1 in x (x being number of songs in the playlist) to 0 in x (which is just 0 chance obviously) untill the playlist has been played completely.

 

Now what I have also observed mainly while using Spotify on Android 5.x.x is just like when on shuffle it only plays the first 30-50 songs on a 50+ song playlist, while playing without shuffle (so in the intented playlist order) the playlist will randomly restart the playlist after 30-50 songs even when repeat is turned off.

 

This suggests to me that Spotify has issues with allocating RAM on Android devices. (I only specify Android as I have not experienced this on iOS or Windows). If Spotify is trying to store the playlist order in the devices RAM this could be whats limiting the number of songs it plays to 30-50 even when the playlist is 50+. If this is the case maybe move the playlist order to the devices permanent memory (sort of like virtual memory). It will be slower but could solve the issue everyone is reporting as it will allow for larger playlists so it would be worth looking at if you haven't already. The only limitation would then be the amount of storage available on the device itself.

 

This thread has been going for over a year and the problem still persists so I hope you (the Spotify Team)  see this and at least let me know where you are at with this problem.

 

On an unrelated but possibly actually is related note is how slow Spotify loads on Android devices, taking an average of 30 seconds to load menus. Again this could be how Spotify manages RAM with the Android SDK.

martmitch
Newbie

I think its fair to say shuffle sucks, it just doesnt take into account songs already played. This is such a bad and annoying issue I've cancelled my premium account and gone elsewhere, I'll only consider coming back if its fixed, properly this time

 

The final straw was last night when I skipped a song and it was back 1 skip later argh!

Taymaster
Casual Listener

Still definitely not random. I don't her the same songs in one sit down, but when i can come back to my 1000+ playlist I can expect to hear alot of the same songs i heard in my previous hour long listening session. 

 

Maybe this is what Spotify wants so they can get more skips aka adds aka revenue 😛 Sounds like a crazy theory, but i cant think of a more logical explanation for such a prolific software not being able to execute a simple random function.

 

Don't get me wrong i loooove Spotify. But shuffle is a joke still.

 

1231848368
Newbie

Over a year and a half later and I'm still having this problem. My main playlist is 1,398 songs. I maybe hear 40-50 on a regular basis when shuffling. If I don't have the playlist repeat function, spotify will stop after the 40-50. I've noticed that when I'm listening from my phone on an airplane (in offline mode) I suddenly hear WAY more of my songs. Is downloading the playlist and using offline mode really the best way to fix this? What makes the difference? 

arcdeseel
Newbie

Easy! Shuffle doesn't mean random! Shuffle means order every song randomly, then play through in order, so each will be played once before any are played twice.

sronis
Newbie

The problem wasn't whether it's a shuffle or a random.

The problem was that I have a playlist of 500 songs that I actually had on my computer and exported to spotify as a playlist, but still when I was clicking the HUGE shuffle button (clearly a very important feature in Spotify's eyes), I was getting to hear 30-50 songs from my playlist in repeat.

I'm talking in past tense because it seems the problem is partially fixed in the last month (amazing, it took 5 years), but still I see a weird issue of tending to favore specific artists. 

There are almost 200 artists in that playlist, and yet in the next 50 songs or so it is showing in the "next from" list, i see about 12 songs from the same artist, of which i have about 14 songs total in this playlist.

 

What's going on?

RolandSolc
Newbie

Hi,

 

I also found out that shuffle works quite strange (spotify web app). After some googling I have found easy but manual solution from another spotify comunity thread (comment by Stranger makred as solution )

https://community.spotify.com/t5/Help-Desktop-Linux-Windows-Web/Shuffle-repeats-songs-too-often/td-p...

 

but works only in desktop app. Solution si to create playlist in desktop app and then play it on any device with shuffle off.

 

Best regards

wilson9cl
Newbie

Shuffle still isn't working for me.  I thought this might help a little.

 

private int RandomSong(int min, int max)
{
Random song =
new song();
return song.Next(min, max);
}

wirelessneo
Newbie

this issue definitely isn't fixed...I have about 2500 songs. the same EXACT song will play multiple times in a one-two hour window for about 50 songs and everything else is ignored. Driving me nuts! I'm a premium user and all are downloaded. this seems to happen more on my PC than on my Android, but it happens on Android too

mfugel
Casual Listener

still not random shuffling....that is what is reallly needed...dont replay the same 20 songs or so...shuffle means shuffle and play all songs...PLEASE PLEASE

Yoshi900
Newbie

Um, seriously it's not that hard, please simplify the code.

just use a RNG (random number generator) + exclude if it has already played that number (song) then once all of the tracks have been exhausted then remove all the excluded numbers and start again....

 

edit: I would like to add I don't care about if an artist gets played twice, the code should be simple and using a RNG on the totally number of songs in a playlist should be all that is needed, because with a truly RNG will be random, but with excluding songs it has already played (based on number rather than artist) is better.

Labels