Hi There, you can indeed do exactly that! I'll use offline playlists as an example, but the same principle works for tracks, artists, albums etc.
1. Go to 'Your Music - Playlists'
2. from this screen which shows all the different playlists you manage/subscribe to whilst at the top of the list pull down on your screen to reveal the hidden sort/filter menu.
3. Tap the newly appeared button on the top right (3 horizontal lines) and then select 'Downloads' from the filter section
Now you should only be able to see playlists that have been downloaded to your device.
You may also want to add kudos to this idea to show your support: Automatically filter offline playlists when offline
Hey there you, Yeah, you! 😁 Welcome - we're glad you joined the Spotify Community! While you here, let's have a fun game…