As I had stated in a previous post, I realized the best first step to adjusting my color algorithm was to look at the actual input I was feeding into it! This is something I really should have done when I originally started running into issues.
Using Processing, I produced full spectrograms of three contrasting songs.
After viewing these outputs, it is obvious that I was getting muddied results due to the fact that I was using a linear distribution when averaging.
These results also pointed out how much my current color algorithm excludes out of the frequency spectrum. I was only using the first three bands, which correspond to the bottom three rows (easier to distinguish in the close-up).
I read through the Minim documentation more closely, and found a function called logAverages that will group frequency bands by octaves. The logarithmically spaced averages correlate more closely to how humans perceive sound than the linearly spaced averages (that I was using initially).
Using logarithmically spaced averages shows a more clear difference between the songs, though the latter third of the octaves are still similar. I will take this consideration when I start composing a new color algorithm.