To summarize - the easies way to remove a downloaded playlist/ album is to disable the green slider button Download above the playlist. When it is grayed out - the playlist will not be available for Offline play.
To clear all downloaded playlists/ albums/ podcasts from your device's memory you'd need to clear the cache on your device. You can follow the steps in this Spotify Answer for device specific instructions.
You can also add your +Vote to this Idea:
Hope you find this info helpful.
Go to settings, scrooll down to the bottom and click on 'Delete cache and saved data'. That will delete your downloaded files. If you want to delete a particular song, in the player screen, unmark the green check to the left of the song's title.