topic Re: Getting better API usage insights? in Spotify for Developers

Getting better API usage insights?

mherger — Sun, 03 May 2020 19:33:22 GMT

Plan

Premium

Operating System

Web Service

My Question or Issue

I have an application which generates a good amount of traffic to your API servers. There are thousandfold unique users every day (according to your dashboard).

From the dashboard I can see that something isn’t good, though: there are days where some endpoints are called 20-30x the number of regular days. In order to better understand what‘s going on I’d eg. need to know whether the ‚tracks‘ endpoint had been called for single tracks or with the ‚ids‘ parameter for multiple tracks. Is there any way to get this kind of additional information?

Recently we got an increasing number of 429 status back. But it’s totally unclear to me how the rate limiting is applied. These requests come from the users‘ computers. Thus the IP should be unique per user. Is this even taken into account? Or is the rate limiting applied to all calls using the same client ID? Any further information how to mitigate the rate limiting issue?

Michael

Re: Getting better API usage insights?

mherger — Sun, 03 May 2020 19:38:36 GMT

More helpful pieces of information would be top requests per user (is one client gone nuts, or the application overall?), longer period stats to better understand trends, or even raw logs forever request.

Re: Getting better API usage insights?

mherger — Sat, 16 May 2020 08:36:18 GMT

Unfortunately my posting didn't get any traction... maybe this screenshot can help me illustrate what I'm struggling with right now:

As you can see some queries go wild: the pink one is /v1/tracks, going from a few thousand to hundreds of thousands of hits per day. Similarly, but not as badly the green one (/v1/playlists/{id}/tracks). Quite obviously something's wrong. And I have no way to investigate it because that chart is all I have.

I have a few thousand daily users. What would be super helpful would be to understand whether these abnormal spikes were coming from single IP addresses, or just all instances going crazy. To understand whether somebody was abusing my application to do whatever or whether there is a bug in my application leading to systematic craziness.

The situation today is that we're seeing too many 429s due to this behaviour. And I have little to no means to investigate the problem.

Re: Getting better API usage insights?

spotifyjosh — Mon, 18 May 2020 20:33:42 GMT

Hey @mherger, thanks for sharing this feedback with us. I can see how more granular analytics might be useful here. Do you have any logging set up in your application? If not, that might be a good way to identify the source of any excessive API calls within your app.

Re: Getting better API usage insights?

mherger — Mon, 18 May 2020 20:36:56 GMT

Hi Josh

thanks for getting back to me. Yes, there is logging available - but the application is running on the users' devices. They would have to enable logging and send me their log files. That's not easily done without knowing who (out of the thousands of users) is having a problem...

Michael