Hello everyone,
we'd like to share an update of the findings that we got from investigating this in greater detail. Heads up, long read upcoming.
First, some data facts!
Since we asked for your reports a month ago, we received a total of 61 reports.
Out of those, 27 were actually valid reports that looked like the algorithm isn't functioning as intended, but for all intents and purposes, if you're not happy with your RR, that means we have room for improvement and we'll count all reports as valid for the discussion at hand.
Let's dissect these numbers and put them into perspective.
There're roughly 340 million RRs generated each week. For the 1 month investigative period, that would be 1,360,000,000 Release Radar playlists generated.
That amounts to reports for 0.000004% of the playlists.
Now, we know the algorithm is not perfect so there're probably more errors. And we also know there're reports submitted elsewhere and maybe even a lot of people don't even bother, so let's boost that number by a significant amount - let's make it a factor of 100 000. We pick that number because it would already be outside of our scope to handle if we received so many reports each week, but at the same time we want to be pessimistic about the accuracy of the recommendations.
That still amounts to 0.4% of errors.
For us that is 0.4% still too many and we'd love to bring that number even closer to a total zero, even if, let's be honest, an auto generated playlist will never be 100% accurate for 100% of all the music fans out there.
So the question is what can we do to improve the experience of those affected?
Well, we think we have just two options available to us:
- Provide information and clarity on why things happen as they do.
- Commit to continuously improve our recommendations.
The 2nd one's a given, you have our word. Good recommendations is a big priority for us and we're continuously and constantly tweaking things backstage to improve those. You can do your share by feeding some data to the automatic systems using the Like and Hide features and listening to music that you like from as many different sources within the app as you find useful for your personal listening habits. If you find something new that you like, it will have a more positive effect on recommendations than established favorites.
I'll try and tackle point number 1 now and provide you with as much info as we were able to gather during the course of this investigation. I'll post part of this again in a separate post that will be pinned to the top of the thread to improve visibility, so apologies beforehand for a double post.
There're several complaints that we have to address and unravel. Let's look at them one by one.
- I receive tracks from artists I don't follow at all in my Release Radar - This is intended and we don't plan on changing this. The Release Radar doesn't only help you keep track of artists you follow, but is meant to allow you to discover new music from others we think you might enjoy. It is also one way for Spotify to help artists get their music noticed.
- I get music that I don't enjoy at all in my Release Radar - This means our algorithm doesn't know your music taste so well. There can be many reasons behind this, but the rule of thumb to improve the music you receive in the automatically generated playlists for you is to listen to music without a private session, use the like and hide features, have more intentional music listening habits rather than opportunity based and when you discover something new that you enjoy, listen to it proportionally more than other content. Following these guidelines should give you better recommendations in time. It will still not be perfect and it can vary from person to person, but as mentioned, we're constantly improving these as we go.
Now here's the big one that didn't sound right even to us.
I get tracks from artists that are named exactly the same as ones I follow and enjoy, but it turns out the track is unrelated to that artist and they only have the same name. What gives?
There's also the question if there is some sort of malicious intent behind this so that artists capitalize on others' success. Well, we'll try and answer this with concrete examples based on your reports.
For the Release Radar that was generated on the 26th of February, a lot of you reported this track:
spotify:track:4pFDWJmUVY82I4Gn0A1eBI
It appeared to those of you who follow:
spotify:artist:6kFay2DQ5aZfeu5OsrF3Pw
Another common report from the same week was
spotify:track:3E9JGbmGk1w0mQvX6KLM1v
that appeared on RRs for folks who followed
spotify:artist:64MoFdq8ORI3V98AR5SPWL
And a third one was
spotify:track:5EjufVUD5PLlw4RxxJpyBa
which landed on Release Radars for people who followed
spotify:artist:6c0mTNAxJxlp9HpKTUZwA8
Thanks to your reports we found out why this actually happened.
Here's a bit of backstage info - all of those millions of Release Radars don't get generated on the exact date. They don't look what got released exactly then, but they use data from a release data pool and the systems start compiling your playlists quite a bit in advance so that there's no delay in deliveries due to the amount of data that needs to be processed.
Turns out, "Hypercolours" was uploaded a couple of days earlier to
spotify:artist:6kFay2DQ5aZfeu5OsrF3Pw.
The distributor uploading this then spotted the error and sent a correction request. The track was then moved to a separate artist profile, but alas, it was already captured in your Release Radar.
The exact same thing has happened with the other two reports in the example and with all valid reports that were submitted. During upload, metadata submitted by the distributors erroneously attaches a track to a wrong artist, the Release Radar sees that artist that you follow has new content and captures it to present it to you and by the time the error is corrected and the tracks are associated with different artists, it's already been added to your playlist.
Mind you this can happen and actually happens more often with featured artists as well, as it seems those are more often really small and share names with more popular ones.
Based on how these errors happen and that they usually quickly get corrected afterwards, we don't suspect malicious intent. The first track has been taken down completely for a majority of markets, while the others in our example were separated from the bigger artists even before your Release Radar synced in your app. The only positive effect this would bring is a temporary boost in streams, but looking at the numbers, this is minimal and if you don't like that track, it would actually have an averse affect on capturing an audience and generating revenue. (Remember that Spotify does not pay on a "per-stream" basis.) If however you happen to like that track and decide to follow the artist, we would argue it falls under the principle of genuine discovery, even if it might've been recommended in error.
Now here's the point where we're sure we'll still receive complaints - you still don't want those tracks, even if we explained the technical reason for them appearing. We want to be honest with you - at this point we don't have a solution. Your reports allowed us to receive an incredibly valuable insight on a flaw in the system, but for now, we really can't provide a fix for it. And it's not that we don't want to, it's that the reason for this is deeply ingrained in a complex amalgam of both human error and technical limitations during the upload and distribution process. We do not have the manpower to fact-check the metadata of the tens of thousands of songs uploaded to Spotify every day. And the distributors we work with are human as well. The systems we have set in place do limit errors by requiring specific data to be submitted, but without teaching an AI to audibly differentiate two artists from another, the systems still use the metadata provided to them.
That leaves us with the 2nd best option - making corrections based on human input and reports on incorrect content. This really helps and is one of the reasons why some tracks even get removed from the platform as can be seen in the above examples. And we've read your posts that you can't know on which artist profiles these tracks belong, but here's the thing - you don't need to. The report just needs to be clear and state that you're reporting a track that is by another artist of the same name, but isn't actually the one on whose artist profile the track or album is attached to. This is often the case - a new artist profile for this same-name artist just needs to be created and you can't find that if it doesn't exist yet, so no one expects that 🙂 It will get created by the team after your report is processed.
That being said, we'd like to ask you for your understanding that processing these reports can take a while and yeah, sometimes you might get a question that doesn't seem to make sense but it's just because the person looking at your report maybe didn't understand it, so help them out to help correcting content errors for everyone.
I know this is a lot to digest, but as mentioned, at this moment this is what we can do - give you insights and information. We hope you understand but at the same time have to point out that aggressive and offensive behavior will not be tolerated on the Community and could lead to the closure of this thread. We'd think that after what it helped accomplish in terms of insights it would be really sad to see, so we're keeping our fingers crossed to receive more constructive discussions and understanding surrounding this topic.
Once more, thank you to all who participated and submitted reports to the form we posted here. The data received is set for complete deletion and we'll only keep the knowledge and non-verbatim feedback we gained from it.
All the best!
Mihail Moderator
Help others find this answer and click "Accept as Solution".
If you appreciate an answer, maybe give it a Like.
Staff / Moderator/ 3 years ago in Social & Random