Spotify connect keeps connecting to the wrong device.

[Rymdkakor]

Plan

Premium

Country

Sweden 

Device

PC/Sony HTST-5000/OnePlus5

Operating System

Windows 10/???/Android

My Question or Issue

I have a Sony HTST-5000 soundbar at home that I use with Spotify Connect from time to time. It's a fairly powerful sound system and I seldom use more than volume level 12 for music, but when I use my PC with it I use volume level 20 and use the PC volume control to control the volume since that is more convenient. This means that I usually leave it sitting at volume level 20 when I leave home to go to work.

So far so good. My problem is that quite often when I press play in the Spotify desktop app on my PC at work Spotify Connect decides to send the music to my soundbar at home, turning it on remotely and starting playback, instead of playing the music on the PC I actually used to start the music. This also sometimes happens when I do the same on my OnePlus5 Android phone. I haven't gotten any complaints from my neighbors yet but I feel that if I'm not careful I could unwittingly leave my Spotify playing INCREDIBLY loud on my sound system at home, when I'm not even there.

How do I tell Spotify to primarily (or always) start playback on the device I'm currently using? Ergo, if I press play on my phone/pc/tablet I want the music to start playing there and nowhere else unless I manually choose to send the music somewhere else.

[Rymdkakor]

Edit: with "leave it sitting" I don't mean that I leave it turned on, I turn it off but it will remember the volume level it was on when I turned it off. Spotify Connect can turn it on remotely.