I and others have brought this up REPEATEDLY in the past. It has continously been ignored. So here's another go!
Spotify has literally the absolute WORST playback system I have ever used in any music player, including launch Winamp or the Win95 sound player. It's actually three very simple problems that could easily be fixed, but repeatedly the community managers/developers have brushed off all of the numerous complaints people have lodged at the current system.
First, the inability to remove/clear your play queue. I've seen dozens of posts requesting this, made a few myself, and it's always dismissed quickly as unwanted or impossible. That's stupid. I've never used a music player that doesn't allow you to remove songs from your queue or clear it altogether. As is there's no button to remove songs from your play queue and right-clicking a song listed in your play queue brings up a context menu with the word 'delete' ON IT, but greyed out. Why even have the useless button there? There was at one time a staff recommended workaround to create an empty playlist and then play it. That's no longer possible, so the ONE option to clear your queue is no longer valid.
That leads to the second issue Spotify has with queues: When an album or playlist finishes playing, the queue loops back to the very first song and then proceeds to play the whole thing back again. This is a horrible behavior. As a brief example, if I play an album and listen to it all the way through, then decide to listen to a single song afterwards, it'll play that single song and then start on the second song of the album, proceeding to play the entire album again. Same with a playlist. I was listening to a playlist a couple days ago and then this evening I decided to listen to a few single songs. It then, days after I initially listened to the playlist, started back into it.
The third issue isn't near as bad as the other two, but is still pretty terrible. When searching for something, double-clicking it will queue up EVERY. SINGLE. SEARCH. RESULT. This is the worst possible option considering there are often multiple songs with the same title or songs from the same artist and double-clicking is the default action 'to do stuff' on the Windows platform. This can be subverted by right-clicking and selecting 'play,' but if it's called a workaround it's a problem.
Your first two suggestions are similar to an idea (as you mention) that has also been suggested here: