Windows 10 allows to assign audio output for separate apps - in Windows' Sound settings. Scroll down in there, click on 'Other Sound Options' to reveal 'App volume and device preferences'.
I think Spotify might be stuck with speakers as an output. Try setting it to default or what you're currently using, then restart Spotify.
Let me know how it goes 🙂
Help others find this answer and click "Accept as Solution".
If you appreciate my answer, maybe give me a Like.
Note: I'm not a Spotify employee.