Maybe the "icon on albums" solution works for Windows users, but on Mac I found the best way to find and play all songs for an artist is to enter the artist into the search box, and then click on Songs -> All Songs, rather than clicking on Artist or Albums.
In my experience this solution only show albums as lists of tracks, but not all artists tracks.
But I found different solution. You can search for any artist in Windows Client and when you see "Top results" and after "Songs". It's better if artist name is unique, so there will be only needed artist songs. And there is "See all" button on the right side of "Songs" block. After clicking this button the list with all artists tracks appears and you can also sort this list by play count (popularity).
It's strange this only works on search page, but not on artists page as it is should be more intuitive.
I also added screenshots to my post. Hope it helps.