Currently, assuming the Desktop player is mirroring the behavior of the web player, it grabs the metadata for 50 albums as a time as you scroll through your library. However, it doesn't know anything about how large that library is as it goes through. This means that, as the user scrolls through their library of albums, it has to repeatedly load at the bottom of the "page" as the application loads in the next set of albums.
It would be functionally better, at the very least, for the application to know how big the user's library is. The user could scroll to any point in their library and with some quick math of how far they've scrolled the application could grab the albums for wherever they've scrolled to.
Even better, seeing as the 'Liked Songs' view of a user's library doesn't have this lazy-loading problem, would be to display the entirety of the user's library with blank/placeholder album art, and simply load the album art as they scroll through rather than the entire album cards.
This functionality is doubly necessary and would help to alleviate some of the frustrations of the desktop application not remembering where you were in your library if you click into an album and then click back.
Also, I've built scrolling behavior like this before, and you should load the next row of albums at least one row before the user gets to where the blank row would be, if not two. It would make scrolling through one's library feel much smoother.