Need help? Check out Spotify Answers for solutions to a wide range of topics.
I use offline playlist (songs) mysefl too because I sometimes travel abroad (and data roaming is quite expensive 🙂 ).
I can asure you that when the green arrow next to the song is visible, the song is offline available.
Or if you want to test it yourself: put your phone in airplane mode and play the offline song. It should work, otherwise it is not available offline.