Playback on Mac mini works fine until the screen saver kicks in, music get interrupted after a couple of tracks (not full playlist). I log in again to my mac and music starts.
Also when playback is running and I want to change track on my iphone x, this is possible until power saver becomes enabled after that, my iphone has no idea about what track is played and I have to log on to my mac mini and the then the track on my iphone is updated and I can change track from there.
My internet speed is 200/200 mbit and there are no issues w speed.
It probably has something to do with your Mac power settings and not your Internet. I would check and see if you have the box checked for "Put Hard Disks To Sleep When Possible" and if the box for "Prevent Computer From Sleeping Automatically is unchecked. The settings I have in my screen shot allows my computer to keep playing Spotify in the background without interruption - even when my display goes to sleep. I currently do not have a screensaver enabled, but you could try extending the time it goes into screensaver mode as well to see if that stops disrupting the music.
I hope this was helpful and feel free to reply if not and I'll keep trying.