Everything in the /pool directory has an aggressive cache, including the directory view, but everything should expire at the same time once per day, so what you described should not happen, but it obviously did.
You should however follow the preferred instructions at: https://www.spotify.com/download/linux/ and not check the directory listing manually for deb files. apt will fetch list of updated packages in a different way, from the /dists directory which should not be affected by this kind of cache problem.
