forum

ELO ranking for multiplayer

posted
Total Posts
80
This is a feature request. Feature requests can be voted up by supporters.
Current Priority: +518
Topic Starter
Hikaros
So i've been thinking about this for a while but i never posted it, i tried to look if someone posted this before but i didn't really find anything.

The feature that would be nice to have in multiplayer is the ELO ranking from chess.

Elo rating system

True, we have pp, however that can be farmed while ELO can't. Which is the main reason of seing people ranked 3XX losing every single time against someone ranked 1XXX in multiplayer.

For the ones that doesn't know how it works let me explain it in an easy way:

Everyone starts with 1200, once you play with someone if you win you get ELO according to the ELO from the other player, if the other player's ELO is higher than yours then you have a lot to gain and if you lose you still lose ELO but is just a little.

This is to make multiplayer more competitive and also fun, pp is great but is mostly meant to SOLO play and grinding, while multiplayer is just for fun and lols which is fine too, it is just sad to see like 5k players logged in, only like 100 are in multi (probably more), and most of the rooms are Easy/Normal difficulties ):

So the fun in multi doesn't get lost and everything becomes full competitivity (Cuz that is not good for everyone). There can be Ranked and NON-Ranked rooms.

Granted, the ELO rating system is usually applicable to 1 v 1 games, however i believe something can be done about it and it should work about the same. for example a solution could be to add every single value of ELO gained from each player (calculations of defeat/victory should be done individually to each player) and then get the average of it.

Quitting the game counting as a defeat.

There could also be an ELO gap specified to be able to join the room (i.e. 1000-1200)

Some may also say: "What if the other player memorized the map? It is unfair."
Well if you think about it, it is not unfair, the host makes the rules in the room but the opponent is in agreement when hitting the "Ready" button, you just won't go into a game where you think you can't win. (Force start should be disabled for ranked games).

If ELO ranking is implemented for multiplayer then the titles suggested in a thread before could be awesome.

Itd be amazing if you guys can upvote the idea <3

theowest edit:
The A skill-based player rating system request, which got implemented into PP contains a lot information about this topic.
SPOILER

Gerbator wrote:

While the current player ranking works fine in single player mode, I find it to be pretty useless in multiplayer mode, as it barely relates to the actual player’s skill.

I would love to see in osu! a multiplayer mode player rating based on skill, like the ELO rating system used in chess or something similar to Microsoft TrueSkill. My suggestion isn’t about replacing the current ranking system (it would be bad imo, as many players spent a lot of time grinding the ladder), but creating another one that could be used to look for players as (un)skilled as you are, or to develop competitive gaming.

Of course, it’s easier said than done. Being a developer myself, I couldn’t help thinking about how an ELO rating system could be tweaked to match osu’s gameplay, so I thought I’d share it, hoping that it helps starting some thinking about it (I actually don't care about THIS suggestion being accepted, I'd just like to see a skill-based rating ^^).

Note: this post being a wall of text, I used spoiler boxes to make it look like it’s not. That’s just an evil trap to make you read what you probably wouldn’t otherwise :twisted:


The problem

First, here’s what I think to be the main problems for an osu! adaptation of these algorithms:

1) Unlike chess games, there is no clearly a winner or a loser in osu multiplayer games. Here’s some examples:
- Player A gets 15.1M points and player B gets 15.2M. While the player A won, it can also be considered a draw.
- Player A misses 2 hits and get 6M points with a 98% accuracy, while player B gets 15M points with a 98% accuracy as well. No doubt player A lost, but due to the similar accuracy, there’s uncertainty about player B being better overall.
- Player A gets 6M points with a 99% accuracy and player B gets 15M points with a 95% accuracy. Cannot say for sure who’s better here…

2) Beatmap choices cannot be ignored either. Here’s some other example:
- A 4-star map isn’t relevant to rate good players, but a DT+HR 4-star map can be.
- Newcomers will obviously fail on impossible approved maps (e.g. Shotgun Symphony+). While it is possible to assess that a player who got 50% accuracy is better than another that got 40%, it’s too “borderline” to reflect the overall skill of the players.
- Player A never played a beatmap while it’s the 100th play for player B => their results cannot be compared directly.
- Some beatmaps are about streaming, other about speed or “chaos”…

3) Osu! Games are less predictable than strategy ones. Being heavily based on concentration and reflex actions, sneezing = loosing, whereas it’s not a problem in strategy games. It might have to be taken in consideration.


My suggestion to get it working with osu!

Considering all these points, I thought about some tweaks to the ELO rating system that could possibly work.

The general approach might look counter-intuitive: it would be about rating beatmaps, not players. Players wouldn’t be rated based on a comparison with other players, but on their performance relative to the beatmap they played on. Basically, if you S-rank a hard beatmap your rating increase, and conversely, if you get a A-rank on an easy map you were supposed to SS, your rating decrease. The reason is actually pretty simple: osu’s multiplayer mode plays like the single mode. There’s nothing like strategies to get the upper hand, besides hacking the opponents computers or making their phone ring in the middle of a combo-breaker :lol:
As a result, beatmaps should be considered the main opponent in a multiplayer game as well.

Here’s an example of what it could be in a 4-player game results:

The beatmap: an average Insane map, with a 5-star rating, and a ELO rating of 1800 people don't know of.

The players:
- Cyborg-like player: he is rated 2300 ELO, and he achieved a S-rank (8,500,000 points, 99% accuracy).
- Newcomer A: he is rated 1100 ELO, and he failed (300,000 points, 70% accuracy).
- Newcomer B: he is rated 1200 ELO, and he achieved a A-rank (3,100,000 points, 93% accuracy).
- Average/good player: he is rated 1700 ELO, and he achieved a S-rank (8,300,000 points, 95% accuracy).

The rating would be considered this way:
- “Cyborg-like player” did what he was expected to do: he has beaten the beatmap effortlessly. The beatmap was too easy for him, so its rating won’t change.
- “Newcomer A” did what he was expected to do as well: he failed badly. His rating won’t change, and the beatmap rating won’t change either.
- “Newcomer B” did a good job in getting a A-rank on this map, considering his low rating. His rating will increase, and the beatmap rating will decrease.
- Considering his rating, “Average/good player” was expected to get something like an A-rank, but he S-ranked. His rating will increase as well, while the beatmap rating will decrease.

Some other things that might have to be considered:

- The play count on the beatmap. Some “chaotic” patterns can make even skilled players fails at FCing easy beatmaps on the first attempt. It shouldn’t be as hard on the rating as failing on the 100th attempt. Conversely, SS-ranking a beatmap at the 100th attempt shouldn’t be valued the same as doing it on the first attempt.

- A skilled player gets better score overall, but he/she also get better accuracy. Both score and accuracy should be considered.

- Mods. Calculating the impact mods have on beatmap ratings might prove to be impossible. Having accurate ratings on unplayable beatmap/mods combinations might be impossible as well. I have no satisfying solution about that.


What I think to be its main "selling points"

As you can see, players are not rated between themselves, and I think it has numerous advantages:

- First, it solves the problems related to beatmaps being too hard or too easy. For example, beating hard 5-star beatmaps like Shotgun Symphony+ would be more rewarding than beating “easy” 5-star ones that thousands people could SS on their first attempt. With the current star-rating system, there’s no way to set hard 5-star maps apart from easy ones.

- Top players wouldn’t be too much “afraid” of playing with average players who could potentially beat them on easy maps (e.g. getting 100% accuracy instead of 99.8%) and “steal” their rating points.

- Being exclusively based on player/beatmap comparisons, the rating algorithm would work in single mode as well. It would be really useful for beta testing purpose: everything could be tested in real conditions without the players even knowing it. If the first implementation sucks, no player will be hindered. Using the rating algorithm in single player mode could also make easier the rating of old beatmaps that only a few people know of.

That's it! I look forward to your feedback, be it about my suggestion or the ugly English mistakes I probably made ^^

At the very beginning i was thinking about manually done rooms which i suggested it cuz it seems simpler and faster to do however there is room for a lot of discussions including some non logical but hey who knows, the probability is never 0 lol.

So Wishy suggested a fully automated way to do it and it is better since it makes the amount of weird scenarios be zero.
Here it is:

SPOILER

Wishy wrote:

You just need to make it automatic.

Mimic a SC2 ladder system (click a button, get matched).

Separate tiers. Like leagues on SC2, any number of players can fit in any league, top league has limited slots so being there is not that easy, you can make 8/9 separate tiers and fit players there, 6 standard ones an the top one, tier level = map difficulty, 1~2 would be easy normal, 3~4 Hards, 5~9 different insanes since the spread is gigantic, and then 10. Players should be able to get matched with people from other tiers (-1 or +1). Of course you will get a big rating boost if you defeat a higher tier player, and he would lose quite a lot. Number of tiers can be discussed, maybe you just need 7, 8, 10 or 500.

A good development for this would be the chance of meeting lower/higher tier players being dynamic. Say tier 1 = 0~99, tier 2 = 100~199, etc. Then if your rating is 590 (meaning you're tier 6 I guess) your chances of playing against a player on the tier above yours would be higher than the chances another user with a rating of 540 would have. Of course this applies the other way around. If your rating is 590 the chances of you playing against a lower tier player gonna be lower than the ones a 540 rated play has. Of course you can just "remove" the tier thing and just talk about numbers. But since map difficulty will change over time, having this "categories" thing helps sort this out. And then since you're playing a harder map than those you use to play on ladder, winning against a player who supposedly has been playing that already should give a little bonus (say 550 vs 540, 550 wins, gets 5 rating, then you get 595 vs 605, 595 wins, you get 7 rating instead of 5).

Use a proper rating/ELO system for calculations.

Rate maps by difficulty (get players to do so, it doesn't need to be 100% accurate, VERY long maps should be excluded, I don't think anyone wants to play some 10m map) so you don't get newbies playing Airman.

That's it, in a nutshell I think.
blissfulyoshi
p/1932955/

Marking as duplicate......
Topic Starter
Hikaros

blissfulyoshi wrote:

http://osu.ppy.sh/forum/p/1932955/

Marking as duplicate......
Just took a look at it, however the post is very very vague and it doesn't explain the reasons behind the request of the feature, such as cons and pros and comparison with the current pp system (which is great imo, it made me be active in Osu! again) and like i have mentioned itd only affect multiplayer not solo, that should make multi a lot more fun and competitive. I even posted the name of the ranking system and how it actually works, it is unfair to be marked as duplicated imo ):
theowest

blissfulyoshi wrote:

http://osu.ppy.sh/forum/p/1932955/

Marking as duplicate......
I'd say this is the best feature request concerning Multiplayer rankings, it has the most stars and the best concrete explanation. Unmarking this as duplicate before all is lost. The other "original" feature requests to this are even older than PP.

This ranking would significantly improve multiplayer.
James2250
This has been talked about a few times before, there is one conversation about it here p/738435 but I am sure there are more updated ones around.

It's not a bad idea, but I don't see this happening any time soon with the pp system along side it (yes I did read your reasons against that, but still). I don't ageee that pp is easy to farm especially as it's a constantly evolving system. This just seems like another set of stats to keep track of.

As for denying this as a duplicate or not, it's a hard decision based on the other threads scattered around and will wait for more opinions on the subject.
theowest

James2250 wrote:

As for denying this as a duplicate or not, it's a hard decision and will wait for more opinions on the subject.
This request does seem both better in terms of explaining and actually supporting it with money. I had a discussion with Hikaros about such potential feature and he seems to know what he's talking about.
I suggest we have this more updated (because the other one were suggested before PP), and more supported request as the new starting point for any multiplayer related ranking discussion.

This request doesn't have to be exactly as it says in the OP, it will obviously adapt to the osu! multiplayer from where it came from, chess.

SPOILER
that "original" request has this unofficial way of supporting a request

which isn't allowed nowadays.
deadbeat
this just sounds like a standard ladder system request to me(correct me if i'm wrong) which has been requested multiply times.
I think this needs some discussion about this.

Now, regarding if this should be a duplicate, it doesn't matter if you have more info the the original, you should just bump the original post with that info, or in this case, repost the OP into one of the old threads.

that's just my opinion at the moment.
blissfulyoshi

theowest wrote:

blissfulyoshi wrote:

http://osu.ppy.sh/forum/p/1932955/

Marking as duplicate......
I'd say this is the best feature request concerning Multiplayer rankings, it has the most stars and the best concrete explanation. Unmarking this as duplicate before all is lost.
If you insist... I personally think it is counter-intuitive to have a separate ranking for multiplayer. You are creating a special ranking system for the small subset of people who play multi regularly. This will exclude a lot of players from the ranking system and again it is a terrible measure of actual skill since you need to a. play a lot of multi for your rank to show, b. have dedicated ranked matches (so ppl can play for fun as well), c. get the separate mod per person thing completed (otherwise you need to find a fair way to play matches with mods), or d. some combination of the things stated above.
theowest

deadbeat wrote:

Now, regarding if this should be a duplicate, it doesn't matter if you have more info the the original, you should just bump the original post with that info, or in this case, repost the OP into one of the old threads.
We usually mark the older requests as duplicates if the newer ones is far better, and especially if it has more support.
blissfulyoshi

theowest wrote:

deadbeat wrote:

Now, regarding if this should be a duplicate, it doesn't matter if you have more info the the original, you should just bump the original post with that info, or in this case, repost the OP into one of the old threads.
We usually mark the older requests as duplicates if the newer ones is far better, and especially if it has more support.
Not when that support is made by just one person... I usually regard those ppl as ppl who don't look hard enough
theowest
09:06 theowest: I don't dislike it as much as I like it, I just wanted to quickly save it because the idea of having a worse original made my tummy twickly (whatever that means)

09:07 deadbeat: OP quailty should not affect if it should be a dupe or not
09:07 theowest: it has stars
09:07 deadbeat: words and be reposted and OP's can be updated
09:08 theowest: which we've denied requests a million times
09:08 deadbeat: from one person, kinda unfair in some ways
09:08 theowest: this feature request is also much more concrete and to the point than the other shitty requests
09:08 theowest: the idea haven't been suggested that many times, one of which it turned into what we now have: the singleplayer PP
09:09 theowest: the older one is also abandoned, not many people care for it anymore after what have happened the past year

09:10 deadbeat: i think we(mods) need to have a talk about how we manage dupes
09:10 theowest: yeah
09:10 theowest: we need an official way of dealing with them
09:10 theowest: and it has to be the BEST way of dealing with them
09:10 theowest: we currently do add info to the original if that's good
09:11 theowest: sometimes, a more updated person in charge of the OP will update the thread more and make sure it's being discuessed
09:11 theowest: compared to the original abandoned request
Loves
Multiplayer is for fun though...

and i thought pp was a good measure of skill anyways.
theowest

Lily-Kun wrote:

Multiplayer is for fun though...
You should of course be able to play "unranked" multiplayer. Where the multiplayer matches doesn't go towards this multiplayer ranking.

Let's not hurry the thinking of this idea/implementation of this ranking, inb4 denied. I just think this to be the "original" topic from now on. Currently, us mods are going to try to think of an official way of handling duplicates, etc.
Topic Starter
Hikaros

blissfulyoshi wrote:

If you insist... I personally think it is counter-intuitive to have a separate ranking for multiplayer. You are creating a special ranking system for the small subset of people who play multi regularly. This will exclude a lot of players from the ranking system and again it is a terrible measure of actual skill since you need to a. play a lot of multi for your rank to show, b. have dedicated ranked matches (so ppl can play for fun as well), c. get the separate mod per person thing completed (otherwise you need to find a fair way to play matches with mods), or d. some combination of the things stated above.
It is a small subset of people due to the same reason, multi being just for fun and nothing else. a feature like this should encourage a large amount of players in SOLO to join multi, if a player can retry a map over 1900 times (not even exaggerating, this is a real number lol some of you may know who i am talking about and what map haha) they will have no problem in doing a few ranked games.

With ELO you don't necesarely need to play thousands of times to achieve a high rank, the ELO rating system is a real skill-wise rating sytem, based in real time matches with other players not trying to beat the score it took over 1900 retries to do and just a single map.

If you see someone rated 2000 elo then you think: DAMN PRO!
If you see someone ranked 300 in pp then you think: i just might be better :O (could or not be the case)

theowest wrote:

Lily-Kun wrote:

Multiplayer is for fun though...
You should of course be able to play "unranked" multiplayer. Where the multiplayer matches doesn't go towards this multiplayer ranking.
Exactly! :D
theowest

Hikaros wrote:

theowest wrote:

You should of course be able to play "unranked" multiplayer. Where the multiplayer matches doesn't go towards this multiplayer ranking.
Exactly! :D
In that regard, it's similar to other multiplayer games with the leader boards and such. LoL, etc.
blissfulyoshi
With that same logic, how will you ensure this ELO system is any more fair than pp. How will you ensure what songs are played and what mods are used? Also, how many rounds are there per game? What happens if there is a tie? (Well tie, can just give 0)

If all mods are the same, I can just play nc + easy and win every time since basically no one knows how to play it on more difficult maps
If you choose the maps, why can't I play something super unintuitive like unreachable distance http://osu.ppy.sh/s/6509 and win all of those?
If you have a set of maps, just go prove yourself in the charts (unless you insist that mass multiplayer of these songs is obv the best way to prove your skill here)

I hear your reasoning, but what are you planning to do from there? I want some more concrete details before considering to support your request.
Topic Starter
Hikaros

blissfulyoshi wrote:

With that same logic, how will you ensure this ELO system is any more fair than pp. How will you ensure what songs are played and what mods are used? Also, how many rounds are there per game? What happens if there is a tie? (Well tie, can just give 0)

If all mods are the same, I can just play nc + easy and win every time since basically no one knows how to play it on more difficult maps
If you choose the maps, why can't I play something super unintuitive like unreachable distance http://osu.ppy.sh/s/6509 and win all of those?
If you have a set of maps, just go prove yourself in the charts (unless you insist that mass multiplayer of these songs is obv the best way to prove your skill here)

I hear your reasoning, but what are you planning to do from there? I want some more concrete details before considering to support your request.
Itd be 1 per game, it could be a set of 3 songs with breaks of 40s in between, like theowest said it can be adjusted to the need.

As per the "ifs", like i said, the creator of the ranked room sets the rules, you can make it your favourite map + DT+ HD + FL even but is the challenger that decides to accept that, Eventually you will be stucked, you can't gain elo from a 1000 elo player when you are like 1700, you may get like what? 3? and if you by any chance lose the opponent will get A LOT which also means you LOSE A LOT. If you slowly get rating by doing that you will eventually face a real pro and you will lose which will bring your rating down. You can't go up unless you beat people of the same or higher rank.

If i was the challenger, id join your room and see the setup, i know my chances of winning are very very low so i would quit the room since to me is not fair.

The challenger agrees with your rules if the game starts, no one can complain about it.

And yeah it is just for real-time matches in multiplayer. A score in the charts from a player doesn't mean he is better than you, it just means he/she spent waayyyy more time than a lot of people trying to achieve that.

The most immediate example is to compare: rrtyui vs Cookiezi, everyone knows Cookiezi is simply Osu's god lol and yet rrtyui has beaten some scores cookiezi has set but that doesn't make him better.

If youd like another example where the system is used (chess) is simple:
I like to play chess but i can't play with 10 minutes or less. Some people makes chess challenges of 5 minutes, which means i would never ever play those, not until i know i can handle it. I'm the one deciding if i take the challenge or not.
blissfulyoshi
I didn't want to bring this into chat between us 2, but I'll jsut bring up a few more points.

Under your system, most of the time, the room is going to be in the host's favor.
Next, why would people want to play under a system where we need to wait to see if our opponent accepts our rules. All I can see is that it will take forever to find a good match.

The chess example doesn't exactly work because matching making for that can be separated into a few different modes like a queue for 5min blitz players, 10min blitz players, 20min blitz, and one more for unlimited time. That can made into 4 queues that would have sufficient enough players to run. The system your propose can't. Waiting for others is just an annoying thing that no one wants to see in their queue.

On another note, I don't think multi proves skill at all. All it proves is that you can play more consistently than the other player. If you call that skill, go right ahead.
Tshemmp
I always had this idea, too. But Aqo made a good point here: p/1984310
Kuro
Well, after reading every post that came before mine I can say that I firmly support this.
I hope this doesn't kick-the-bucket anytime soon because it's well thought out and very concrete. I'd like to see more community input on this topic.
show more
Please sign in to reply.

New reply