To summarize - the easies way to remove a downloaded playlist/ album is to disable the green slider button Download above the playlist. When it is grayed out - the playlist will not be available for Offline play.
To clear all downloaded playlists/ albums/ podcasts from your device's memory you'd need to clear the cache on your device. You can follow the steps in this Spotify Answer for device specific instructions.
You can also add your +Vote to this Idea:
Hope you find this info helpful.