This is a very niche issue, but I expect this to be an easy fix for Spotify's programmers.
When using Spotify Connect, I'm able to control the volume of my Android TV with my Android phone, just like I can through the "Android TV Remote" app. That's awesome! Here's the problem, though: when adjusting the volume, a slider appears on the screen.
My Android TV is set up to adjust my receiver's volume, and my receiver doesn't use a slider. Thus, when I try to adjust the volume, I'm only able to turn it up or down a limited amount. e.g. If my receiver was on volume 50% when I opened Spotify Connect, I could only adjust the volume between 40% and 60%.
By contrast, no slider appears in the "Android TV Remote" app; I'm able to turn the volume all the way up or down, whether or not Android TV is set up to adjust the receiver volume.
My idea, then: Make Spotify Connect's volume control more like that of the "Android TV Remote" app!