Okay, so I've been syncing my local files and for some reason there's always been a few songs missing, like 1% or something.
It's not that they're another format than mp3 iether.
I've finally figured it out. Maybe it's old news, but I wanted to spare someone else the trouble.
Apparently Spotify for Windows
(I'm using 126.96.36.1990.g8f3b16fc (Microsoft Store-version) )
Do not recognize mp3 files if any of the file extension letters are capital.
All the mp3s that were missing were named iether xxx.MP3, or xxx.Mp3 (i guess xxx.mP3 woudn't be rcognized iether).<
My solution was to download the "search everything" software to be able to search for exact matching word ("match case") and searched for ".MP3" and ".Mp3" to find all the wrong files and then I used a renamer software to rename them all to .mp3 in one batch.
You might also want to make sure that the file extensions are showing.
Hopefully it helps someone.