Hey @turbonilsen! Welcome to the Spotify Community!
Unfortunately, that's not possible right now. I know it's not ideal but, here's what you can do:
- Move all songs from the original playlist that everyone's following to a new one.
- Remove all songs from the original playlist
- Add the songs from your new playlist to the original one
Let me know how you get on.
Have an awesome day!
Help others find this answer and click "Accept as Solution".
If you appreciate my answer, maybe give me a Like.
Note: I'm not a Spotify employee.