It seems to be somewhat random.
I'm on a Moto G6 with the latest updates and since the last Spotify update all my problems have gone. I've tested two restarts of my phone and all my downloads are there along with configuration still set to the SD card.
Occasionally Spotify will download a few tracks but I am guessing they have just been updated to it refreshes them.
And about 20 minutes after posting the message saying that it seems to have been fixed my phone crashed, my entire music collection downloaded wiped and changed back to phone storage rather than SD card!
...and now...I start Spotify, it shows all my music as waiting to download again. I put my phone in airplane mode. I listen to my music successfully, playing from my SD card. Spotify doesn't know it's there, except it plays it. When I connect to WiFi again? All my music starts to download again. Probably to local storage. It seems less predictable than it was before. Is it better? Is it worse? I can't really decide.
I am sure that this software still doesn't work correctly.
Oh nice. I couldn't find a way to contact Spotify Support, which is why I started this thread. You are smarter and more resourceful than I am!
I have tried your steps. Fingers crossed. It would be great if that worked.
One thing I noticed is that after I reinstalled, the downloads went really fast. Like, it wasn't actually downloading them at all, just checking the SD card for each file. So maybe clearing the app data doesn't actually delete them from external storage? I forgot to check the volume before and after.
In any case, we will see if this persists as a fix. I will leave the post open for another week, but then I hope we can accept your answer and move on!