Hey @Toplight! Welcome to the Spotify Community. ^^
That sounds not nice.
Do you've tried to reinstall your app? If not, here is a tutorial for a clean reinstallation.
This may help you. ^^
Toplight, is your phone one of the Z5 series by any chance? If so take a look at this thread...
Using a BT headset it's working as expected for me. Also activating 1 second of crossfade in the app.