There should be a separate toggle for ignoring the Beatmap Color palette. For instance, when playing Blue Zenith, it'll be gray and blue at some points. If ignored, it'll be any color. I'd rather keep the theme while ignoring all the other skins in my library.