Okay, strangely it seems to need a specific series of events.
Didn't work = Play music through loudspeaker ans then plug in headphones while playing.
Didn't work= Restarted with headphones plugged in and tried again, system didn't even acknowledge headphones (even when unplugged and plugged in again).
WORKED= Start phone, open spotify, DON'T play music, plugin headphones, press play.
Doesn't make sense to me, but it's currently working. Sorry guys 🙂