Hello Team, I am building a personal Spotify analytics project using the Spotify Web API and Snowflake. I am currently in Development Mode and I am trying to enrich my listening history data with additional metadata. Use case I have around 40,000 unique tracks extracted from my Spotify Extended Streaming History.
My pipeline is:
Extract unique track_id values from my listening history.
Call the Track API endpoint to retrieve metadata and get the associated artist_id. Deduplicate artists.
Call the Artist API endpoint to retrieve additional information, mainly: artist genres, popularity
I am running this extraction through a Snowflake stored procedure using Python/Snowpark.
Problem During the extraction, my API calls started returning: HTTP 429 Too Many Requests Retry-After: 68167, this means Spotify is asking me to wait approximately 19 hours before retrying.
What is surprising is that: The extraction was running successfully before this happened. I am not running multiple applications using the same Spotify account. I am respecting batching where possible.
The procedure has checkpointing, so it does not restart from zero. Questions Is a Retry-After value of ~68,000 seconds expected behavior for the Spotify Web API? Does Spotify progressively increase the cooldown period if an application keeps hitting rate limits?
Is there a recommended request rate limit for Development Mode applications?
Is there a recommended architecture for this type of use case (tens of thousands of tracks → artists → genres)?
My current approach is to: cache already processed track IDs and artist IDs, batch requests, wait according to the Retry-After header, avoid unnecessary duplicate API calls.
Any advice would be appreciated. Thanks!