Track Analysis for Hobby und Scientific Research

[pythonguy]

I want to use track infos for myself and also for scientific research.

I read in the spotipy package that i have to create an app.

Unfortunately the API does not work for me and the app gets disabled after a time.

Is it not possible for normal persons to use the spotify data?

And why is it not working? Is the extended quota mode necessary? But it says not for school, what about university?

[Ximzend]

A university is a type of school…

[pythonguy]

yes i thought about it, so its not possible to access the data? and why?

So can only companies access the data like valence scores?

But i have seen research papers with content from the api

[Ximzend]

It is possible using thia code example at the GitHub repo here: https://github.com/spotipy-dev/spotipy/blob/master/examples/audio_features_for_track.py

It assumes you have set environment variables (I sadly can't find instructions how to do this at the moment).

You can also define the variables in your code like here: https://github.com/spotipy-dev/spotipy#without-user-authentication

[pythonguy]

i already used spotipy in python, but sometimes the codes are running without an end. Even if its super easy.

Which is why i thought that its not possible to get the data for normal people.

I need valence scores to a daily top tracks list for at least one year

[pythonguy]

now i had an output: read time out

but is it possible to get data like valence scores for songs over a significant period of time?

Or not possible if no real developer?

[pythonguy]

okay now it somehow worked lol, but sometimes it does not work.

and i dont know if it works for a bigger sample

[Ximzend]

You can get the data from up to 100 tracks at a time, and I recommend you to do this.

You can use one of the methods shown at the following page to divide your list of IDs in chunks of 100: https://www.geeksforgeeks.org/break-list-chunks-size-n-python/

[Ximzend]

What do you mean with valence scores for songs over a significant period of time?

The valence of a song is fixed and doesn't change.

[pythonguy]

ah okay then its easier?

But do you think it will be possible to get track info for 200 songs per day over 2 years?

why is it not always working? is it just that the servers are bad or is there a limit?

[Ximzend]

Because your app is in Development mode, it will hit the rate limit more quickly. If you let your script sleep for 5 seconds after every request, you will hit the rate limit less quickly. (It will take more then 2 hours to get all the data, but that is better then hitting the rate limit non-stop)

You can also let the data save to a file after every request, so you will not lose any data when your script crashes.

[pythonguy]

i need valence scores for the top songs, but the top songs vary over time, so its quite big number

sleep time is a good idea, maybe also caching,

but in general: is the developement mode made for that?

and why is the api not working sometimes

[Ximzend · Accepted solution]

then you can filter the duplicate IDs out with filtered = list(set(ids_list)) before running the script.

As I said, the rate limit is lower in developer mode, so it will stop working after too much requests in a period of time.

Edit: fixed code example 

[pythonguy]

okay i used the unique function and have a way smaller Dataframe, but if i want to get the track infos (for 2 to 3000 songs)

i get the error that its to many retries

[pythonguy]

now its working again, seems like server are not that stable

[pythonguy]

now i had the problem that the codes stopped if no track info was found

after removing them it worked

But now i again get 429 errors

The rate limit is really stupid

is it only there from time to time ?

And is this considered a time ban?

So code could work again after some time ?

[Ximzend]

A couple years ago, I also noticed that a few tracks miss features. I had to write some code that can handle that situation.

The API is not made for your use case actually. With normal use, it shouldn't rate limit that much.