Spotify Connect stops working after a few songs

[diadara]

Not using Chromecast because it doesn't scrobble to Last.fm and the shuffle isn't working properly. So I use Spotify Connect with my phone and my laptop connected to my TV. It used to work flawlessly with my really old Xperia but now I've got an Xperia XZ2 and it keeps losing connection.

After 5-6 songs I have to pause the music, switch back to "this phone", start a song and then back to the laptop. I am using the phone meanwhile but mostly only for browsing the web etc. 

I'm just curious whether this is a Spotify issue or a phone issue.