Any mode can have virtually infinitively hard maps; reaching the skill ceiling is hard for any mode. For any mode a top-tier map requires a lot of practice to get the skill needed to play.
Comparing difficulty based on how much time the top players have played to reach their position isn't very accurate; that parameter describes more how popular and competitive the mode is.
Based on difficulty of passing maps of median difficulty, CTB would be the easiest (a player would be able to pass over 80% of the maps available on the first day they start playing, the group of maps that require skill to pass is rather small compared to the total available maps), followed by o!mania (it's harder to play properly, but you can mash the keyboard just to survive).
Based on the difficulty of getting perfect or near-perfect performance on maps of median difficulty, the order would be: CTB (many maps where getting 100% accuracy is near-free) > Standard (median maps have relatively high leniency for getting 300s) > Taiko > o!mania (all maps needs the player to be very accurate for perfect performance).