While the current player ranking works fine in single player mode, I find it to be pretty useless in multiplayer mode, as it barely relates to the actual player’s skill.
I would love to see in osu! a multiplayer mode player rating based on skill, like the ELO rating system used in chess or something similar to Microsoft TrueSkill. My suggestion isn’t about replacing the current ranking system (it would be bad imo, as many players spent a lot of time grinding the ladder), but creating another one that could be used to look for players as (un)skilled as you are, or to develop competitive gaming.
Of course, it’s easier said than done. Being a developer myself, I couldn’t help thinking about how an ELO rating system could be tweaked to match osu’s gameplay, so I thought I’d share it, hoping that it helps starting some thinking about it (I actually don't care about THIS suggestion being accepted, I'd just like to see a skill-based rating ^^).
Note: this post being a wall of text, I used spoiler boxes to make it look like it’s not. That’s just an evil trap to make you read what you probably wouldn’t otherwise
The problem
My suggestion to get it working with osu!
What I think to be its main "selling points"
That's it! I look forward to your feedback, be it about my suggestion or the ugly English mistakes I probably made ^^
I would love to see in osu! a multiplayer mode player rating based on skill, like the ELO rating system used in chess or something similar to Microsoft TrueSkill. My suggestion isn’t about replacing the current ranking system (it would be bad imo, as many players spent a lot of time grinding the ladder), but creating another one that could be used to look for players as (un)skilled as you are, or to develop competitive gaming.
Of course, it’s easier said than done. Being a developer myself, I couldn’t help thinking about how an ELO rating system could be tweaked to match osu’s gameplay, so I thought I’d share it, hoping that it helps starting some thinking about it (I actually don't care about THIS suggestion being accepted, I'd just like to see a skill-based rating ^^).
Note: this post being a wall of text, I used spoiler boxes to make it look like it’s not. That’s just an evil trap to make you read what you probably wouldn’t otherwise
The problem
SPOILER
First, here’s what I think to be the main problems for an osu! adaptation of these algorithms:
1) Unlike chess games, there is no clearly a winner or a loser in osu multiplayer games. Here’s some examples:
- Player A gets 15.1M points and player B gets 15.2M. While the player A won, it can also be considered a draw.
- Player A misses 2 hits and get 6M points with a 98% accuracy, while player B gets 15M points with a 98% accuracy as well. No doubt player A lost, but due to the similar accuracy, there’s uncertainty about player B being better overall.
- Player A gets 6M points with a 99% accuracy and player B gets 15M points with a 95% accuracy. Cannot say for sure who’s better here…
2) Beatmap choices cannot be ignored either. Here’s some other example:
- A 4-star map isn’t relevant to rate good players, but a DT+HR 4-star map can be.
- Newcomers will obviously fail on impossible approved maps (e.g. Shotgun Symphony+). While it is possible to assess that a player who got 50% accuracy is better than another that got 40%, it’s too “borderline” to reflect the overall skill of the players.
- Player A never played a beatmap while it’s the 100th play for player B => their results cannot be compared directly.
- Some beatmaps are about streaming, other about speed or “chaos”…
3) Osu! Games are less predictable than strategy ones. Being heavily based on concentration and reflex actions, sneezing = loosing, whereas it’s not a problem in strategy games. It might have to be taken in consideration.
1) Unlike chess games, there is no clearly a winner or a loser in osu multiplayer games. Here’s some examples:
- Player A gets 15.1M points and player B gets 15.2M. While the player A won, it can also be considered a draw.
- Player A misses 2 hits and get 6M points with a 98% accuracy, while player B gets 15M points with a 98% accuracy as well. No doubt player A lost, but due to the similar accuracy, there’s uncertainty about player B being better overall.
- Player A gets 6M points with a 99% accuracy and player B gets 15M points with a 95% accuracy. Cannot say for sure who’s better here…
2) Beatmap choices cannot be ignored either. Here’s some other example:
- A 4-star map isn’t relevant to rate good players, but a DT+HR 4-star map can be.
- Newcomers will obviously fail on impossible approved maps (e.g. Shotgun Symphony+). While it is possible to assess that a player who got 50% accuracy is better than another that got 40%, it’s too “borderline” to reflect the overall skill of the players.
- Player A never played a beatmap while it’s the 100th play for player B => their results cannot be compared directly.
- Some beatmaps are about streaming, other about speed or “chaos”…
3) Osu! Games are less predictable than strategy ones. Being heavily based on concentration and reflex actions, sneezing = loosing, whereas it’s not a problem in strategy games. It might have to be taken in consideration.
My suggestion to get it working with osu!
SPOILER
Considering all these points, I thought about some tweaks to the ELO rating system that could possibly work.
The general approach might look counter-intuitive: it would be about rating beatmaps, not players. Players wouldn’t be rated based on a comparison with other players, but on their performance relative to the beatmap they played on. Basically, if you S-rank a hard beatmap your rating increase, and conversely, if you get a A-rank on an easy map you were supposed to SS, your rating decrease. The reason is actually pretty simple: osu’s multiplayer mode plays like the single mode. There’s nothing like strategies to get the upper hand, besides hacking the opponents computers or making their phone ring in the middle of a combo-breaker
As a result, beatmaps should be considered the main opponent in a multiplayer game as well.
Here’s an example of what it could be in a 4-player game results:
The beatmap: an average Insane map, with a 5-star rating, and a ELO rating of 1800 people don't know of.
The players:
- Cyborg-like player: he is rated 2300 ELO, and he achieved a S-rank (8,500,000 points, 99% accuracy).
- Newcomer A: he is rated 1100 ELO, and he failed (300,000 points, 70% accuracy).
- Newcomer B: he is rated 1200 ELO, and he achieved a A-rank (3,100,000 points, 93% accuracy).
- Average/good player: he is rated 1700 ELO, and he achieved a S-rank (8,300,000 points, 95% accuracy).
The rating would be considered this way:
- “Cyborg-like player” did what he was expected to do: he has beaten the beatmap effortlessly. The beatmap was too easy for him, so its rating won’t change.
- “Newcomer A” did what he was expected to do as well: he failed badly. His rating won’t change, and the beatmap rating won’t change either.
- “Newcomer B” did a good job in getting a A-rank on this map, considering his low rating. His rating will increase, and the beatmap rating will decrease.
- Considering his rating, “Average/good player” was expected to get something like an A-rank, but he S-ranked. His rating will increase as well, while the beatmap rating will decrease.
Some other things that might have to be considered:
- The play count on the beatmap. Some “chaotic” patterns can make even skilled players fails at FCing easy beatmaps on the first attempt. It shouldn’t be as hard on the rating as failing on the 100th attempt. Conversely, SS-ranking a beatmap at the 100th attempt shouldn’t be valued the same as doing it on the first attempt.
- A skilled player gets better score overall, but he/she also get better accuracy. Both score and accuracy should be considered.
- Mods. Calculating the impact mods have on beatmap ratings might prove to be impossible. Having accurate ratings on unplayable beatmap/mods combinations might be impossible as well. I have no satisfying solution about that.
The general approach might look counter-intuitive: it would be about rating beatmaps, not players. Players wouldn’t be rated based on a comparison with other players, but on their performance relative to the beatmap they played on. Basically, if you S-rank a hard beatmap your rating increase, and conversely, if you get a A-rank on an easy map you were supposed to SS, your rating decrease. The reason is actually pretty simple: osu’s multiplayer mode plays like the single mode. There’s nothing like strategies to get the upper hand, besides hacking the opponents computers or making their phone ring in the middle of a combo-breaker
As a result, beatmaps should be considered the main opponent in a multiplayer game as well.
Here’s an example of what it could be in a 4-player game results:
The beatmap: an average Insane map, with a 5-star rating, and a ELO rating of 1800 people don't know of.
The players:
- Cyborg-like player: he is rated 2300 ELO, and he achieved a S-rank (8,500,000 points, 99% accuracy).
- Newcomer A: he is rated 1100 ELO, and he failed (300,000 points, 70% accuracy).
- Newcomer B: he is rated 1200 ELO, and he achieved a A-rank (3,100,000 points, 93% accuracy).
- Average/good player: he is rated 1700 ELO, and he achieved a S-rank (8,300,000 points, 95% accuracy).
The rating would be considered this way:
- “Cyborg-like player” did what he was expected to do: he has beaten the beatmap effortlessly. The beatmap was too easy for him, so its rating won’t change.
- “Newcomer A” did what he was expected to do as well: he failed badly. His rating won’t change, and the beatmap rating won’t change either.
- “Newcomer B” did a good job in getting a A-rank on this map, considering his low rating. His rating will increase, and the beatmap rating will decrease.
- Considering his rating, “Average/good player” was expected to get something like an A-rank, but he S-ranked. His rating will increase as well, while the beatmap rating will decrease.
Some other things that might have to be considered:
- The play count on the beatmap. Some “chaotic” patterns can make even skilled players fails at FCing easy beatmaps on the first attempt. It shouldn’t be as hard on the rating as failing on the 100th attempt. Conversely, SS-ranking a beatmap at the 100th attempt shouldn’t be valued the same as doing it on the first attempt.
- A skilled player gets better score overall, but he/she also get better accuracy. Both score and accuracy should be considered.
- Mods. Calculating the impact mods have on beatmap ratings might prove to be impossible. Having accurate ratings on unplayable beatmap/mods combinations might be impossible as well. I have no satisfying solution about that.
What I think to be its main "selling points"
SPOILER
As you can see, players are not rated between themselves, and I think it has numerous advantages:
- First, it solves the problems related to beatmaps being too hard or too easy. For example, beating hard 5-star beatmaps like Shotgun Symphony+ would be more rewarding than beating “easy” 5-star ones that thousands people could SS on their first attempt. With the current star-rating system, there’s no way to set hard 5-star maps apart from easy ones.
- Top players wouldn’t be too much “afraid” of playing with average players who could potentially beat them on easy maps (e.g. getting 100% accuracy instead of 99.8%) and “steal” their rating points.
- Being exclusively based on player/beatmap comparisons, the rating algorithm would work in single mode as well. It would be really useful for beta testing purpose: everything could be tested in real conditions without the players even knowing it. If the first implementation sucks, no player will be hindered. Using the rating algorithm in single player mode could also make easier the rating of old beatmaps that only a few people know of.
- First, it solves the problems related to beatmaps being too hard or too easy. For example, beating hard 5-star beatmaps like Shotgun Symphony+ would be more rewarding than beating “easy” 5-star ones that thousands people could SS on their first attempt. With the current star-rating system, there’s no way to set hard 5-star maps apart from easy ones.
- Top players wouldn’t be too much “afraid” of playing with average players who could potentially beat them on easy maps (e.g. getting 100% accuracy instead of 99.8%) and “steal” their rating points.
- Being exclusively based on player/beatmap comparisons, the rating algorithm would work in single mode as well. It would be really useful for beta testing purpose: everything could be tested in real conditions without the players even knowing it. If the first implementation sucks, no player will be hindered. Using the rating algorithm in single player mode could also make easier the rating of old beatmaps that only a few people know of.
That's it! I look forward to your feedback, be it about my suggestion or the ugly English mistakes I probably made ^^
Just ignore this if you already have a solution or whatever. I do like this idea a lot, it seems a bit complex, but it's definitely possible.
