Well, I'd like to have spectators, and scoreboard separately...Why? Because when playing unranked map, for the first time - there is no scoreboard (because we haven't set any local score, and there is not online scoreboard), and when there is no scoreboard, we can't hide spectators (when we're playing a map for the first time, but don't want to get distracted by 5 spectators watching you, joining / leaving).
bump again because two weeks and still no word (and dropped out of first page)
I really hate playing without being able to see whether somebody comes to spectate me, but at the same time I find the scoreboard on the left pane really distracting and prefer to play without it. The hiding of each of those two elements should be a separate setting and not the same one.
I support the above mentioned idea of shift+tab and tab as two different hotkeys for those two different behaviors.
After the leaderboard update, it's impossible to disable in-game leaderboard now (even with Local Ranking selected. See: https://osu.ppy.sh/forum/t/127372).