I noticed this too. You can remove songs that you have added to the queue, but not when you are just shuffling a playlist. Also, I think the behavior has changed recently, since I would swear I used to be able to remove songs when controlling from my iPhone and definitely from the desktop, but now I think it applies whenever I am playing on an Alexa speaker.
To be accurate, the problem is that you can't remove songs from the "Next Up" section of the queue when playing on an Alexa device. You can remove songs from the queue that you have explicitly added to the queue, but if you are just shuffling a playlist, you cannot remove or reorder songs from the shuffled list.
Ah, thanks for that update. It's strange since it definitely did work at some point in the past - though not on all devices. It never worked when controlling from the iPad app, but did on desktop or phone. One thing I notice is that Alexa devices seem to keep a separate next up queue that is not in sync with other devices. When I switch speakers from computer/phone/ipad to an Alexa speaker, the next up list will change.