I mean, that's the native Windows behaviour. If you want to close a program in Windows, you right click it in the Task bar and hit 'Close window', then the program is closed. Spotify won't do anything, but instead you have a 'Task' to 'Close Spotify', which is for whatever reason sometimes there and sometimes not (see screenshots below). Please support this idea if you want to easily close Spotify like every other Windows software by the native 'Close window' command. Thx.
'Close window' does not work, you have an extra task for that. But...
not always! Sometimes it's missing and you also can't pin it!
Please bear in mind that the Community is not an official Spotify support service. It's a place where we all help each other, whether we work for Spotify or not. So please use your discretion when using the forum.