While the current player ranking works fine in single player mode, I find it to be pretty useless in multiplayer mode, as it barely relates to the actual player’s skill.

I would love to see in osu! a multiplayer mode player rating based on skill, like the ELO rating system used in chess or something similar to Microsoft TrueSkill. My suggestion isn’t about replacing the current ranking system (it would be bad imo, as many players spent a lot of time grinding the ladder), but creating another one that could be used to look for players as (un)skilled as you are, or to develop competitive gaming.

Of course, it’s easier said than done. Being a developer myself, I couldn’t help thinking about how an ELO rating system could be tweaked to match osu’s gameplay, so I thought I’d share it, hoping that it helps starting some thinking about it (I actually don't care about THIS suggestion being accepted, I'd just like to see a skill-based rating ^^).

Note: this post being a wall of text, I used spoiler boxes to make it look like it’s not. That’s just an evil trap to make you read what you probably wouldn’t otherwise


The problem



My suggestion to get it working with osu!



What I think to be its main "selling points"


That's it! I look forward to your feedback, be it about my suggestion or the ugly English mistakes I probably made ^^

3) Osu! Games are less predictable than strategy ones. Being heavily based on concentration and reflex actions, sneezing = loosing, whereas it’s not a problem in strategy games. It might have to be taken in consideration.

Unfortunately, there is absolutely no way of knowing whether a player is just having a bad day or if he/she was genuinely distracted by something during play. There are some ways of being able to deduce it, like seeing what the average performance of a player is on a certain beatmap, then comparing it to the play when the player gets a distraction.

For example, let's say we split beatmaps in sections, then monitor where a player is at in terms of accuracy and points at the end of each section. Say the player finishes it with an accuracy of "A" and a total score of "B" on a section. Then while playing that beatmap again, for some reason, the player has a considerably lower accuracy and total score instead of usual on that same section. That could be considered being distracted instead of simply playing poorly. It should still reflect on the player's rating though, albeit much less than just simply playing poorly. That's because this mechanic can easily be manipulated if the play does not count toward the overall rating of the player. If it's not counted, a player can simply do bad (i.e. not touch anything at all) if he/she had a bad start, and it won't count against them.

Anyway, I've been hoping for something like this before. The only other way to make the ranking system work IMO is to have every player play every single ranked song. And we all know that it's never going to happen.

+1 support

Yes.

I think most people would agree that the current ranking system is not that good at representing the true skill of players. The problem is, the system for ranking beatmap difficulty is equally bad. This method seems to rely on beatmaps being ranked correctly in terms of difficulty. If that does not happen, I'm not sure how well this will work, especially given the number of 5 star songs that are extremely divergent in terms of how difficult they are.

That's just a first impression though,

It would probably be a lot more balanced if the map ranking was decided by BATs upon ranking. The big problem with that - who would want to go back and determine rankings for 15,000 maps?

waterfall wrote:

I'm not sure how well this will work, especially given the number of 5 star songs that are extremely divergent in terms of how difficult they are.

Cheer-no wrote:

The big problem with that - who would want to go back and determine rankings for 15,000 maps?

You didn't get what I was trying to say (or I explained it poorly ^^). An ELO rating of players and beatmaps wouldn't be based at all on the current beatmap ranking. A beatmap would be rated the same way players are. For example:
- if a player get a S-rank on a beatmap, it is considered the beatmap lost against this player (the beatmap rating can possibly decrease).
- if a player get a A-rank on a beatmap, the beatmap won against the player (the beatmap rating can possibly increase).

Thus, if S-rank are generally achieved by players rated > 2100 ELO, the ELO rating system will make the beatmap be rated 2100 ELO.
Conversely, if a player can barely get a S-rank on the beatmap (e.g. he/she FC with 93% accuracy), he will probably be rated about 2100 ELO as well.

There's a two-way relation between players rating and beatmaps rating, so no need to rate anything manually as this is what the ELO or TrueSkill rating systems are made for.

ps: anongos, I'll answer you later, I have to logout ^^

Don't forget some players play with mods only.

Gerbator wrote:

Thus, if S-rank are generally achieved by players rated > 2100 ELO, the ELO rating system will make the beatmap be rated 2100 ELO.
Conversely, if a player can barely get a S-rank on the beatmap (e.g. he/she FC with 93% accuracy), he will probably be rated about 2100 ELO as well.

One more question: What will be the starting/reference point? What would constitute someone to get a rating of 2100? The way I see it, your description is when people and beatmaps generally have established ratings already. Remember that we would have to first set a solid standard for this before people and beatmaps would be able to figure out what their rating is.

Don't worry about replying fast. I can wait. (Have to go to sleep anyway. lol)

The problem that I see is that there are some maps that are really, really easy to beat but have one stupid pattern where even good players can easily miss. Thus, if for example an A counts as a loss, the ELO for the map would be artificially inflated when a few highly-rated players miss in that section. Or maps like Wizards in Winter where it is literally impossible to achieve an S despite the map being rather easy.
Point is, it's really difficult to determine the exact border between "win" and "defeat" because there are so many factors. Score, Rank, Accuracy, experience with the map, different beatmaps being challenging in different areas... Of course, there is mods as well.

But in general, if a good win/defeat algorithm can be developed, this would be awesome

Gerbator wrote:

There's a two-way relation between players rating and beatmaps rating, so no need to rate anything manually as this is what the ELO or TrueSkill rating systems are made for.

Newly ranked beatmaps should, however, have an initial rating. If there is no initial rating, a large number of plays have to be devoted to "determine the initial rating" and leave no impact on player ratings. In this way, players will be discouraged from playing newly ranked maps :
Most people (especially good players who care about rating) will just wait until the map reach the rating-determining play count, and then play in order to get ELO rating. Thus players will gradually move their focus onto old, "already rated" maps, and the whole thing about mapping new stuff would be meaningless since few will play them.

Long story short:
There are too many aspects to osu! gameplay to quantify "skill" in a single rating system, and even if you COULD simplify them as such, there would be no efficient way to organize or base those ratings against other players.

JesusYamato wrote:

Don't forget some players play with mods only.

Yeah I know you love HR

Actually, I think it can be dealt with. If beatmaps would get a rating for each mods combination that have been played, it would just make the rating of mods players a bit slower as every map wouldn't be rated easily.

_Void_ wrote:

The problem that I see is that there are some maps that are really, really easy to beat but have one stupid pattern where even good players can easily miss.

You've got a point. In my original post I said S-rank = win and A-rank = lose to makes things easier, but as you said it could use accuracy as well. And while the ELO rating system is limited to win/draw/lose, it's possible to detect whether the player is a clear winner/loser or not. The rating update could be weight accordingly.

Anongos and Cirno wrote:

[...]

okay then, I’ll try to be a bit more specific about how the algorithm works, but it might look a bit weird as I am not used writing math stuff in English.

First, one of the key point of the ELO rating is that it produces ratings that looks like what’s called a Gaussian or normal distribution (here’s one). In the image I linked the curve is centered on 0 and values above 4 or below -4 are rare. In chess, ELO ratings looks the same, but the curve is centered around 1200 (it’s the ELO rating of an average player), and values above 2500 ELO are rare (grandmasters).

The point is in chess, strategy game or whatever, there is always a large majority of average players, and a few exceptions (extremely bad or good). If look at the IQ of a population, you will get the same kind of distribution (centered around ~100, with few people above 130 or below 70). Well, I was trying to say that gaussian distributions are everywhere, not only in ELO ratings.

Enough theory, here’s an example: let’s suppose we start a rating similar to chess game in osu!

Step 1: all players and beatmaps are given 1200 ELO. It’s a “provisional rating” that will change really fast in the first games.

Step 2: First games are played. When a map is beaten, the player rating increase and the map rating decrease, as I said in the original post. After a few games (it’s really fast actually) and ratings updates, ratings distribution will look like the Gaussian distribution I linked you. The players skill will be approximately known, as well as beatmaps difficulty.

Step 3: When players whose skill is known play against beatmaps whose difficulty is known, the ratings are still updated, but at a slower pace.

Now, ratings are set.

A new map is ranked! What happens?

- The beatmap is assigned 1200 ELO. It will also be a “provisional rating”. In the first games played on this map, the map rating will increase/decrease faster than usual, but the established players ratings won’t be increased or decreased (it would lead to what Cirno described).

- When the beatmap difficulty is approximately known (it would be really fast for new ranked map as only ~20 play are needed), its rating will be set to “established”. From now, players rating will be updated when they play this map.

A new players appears! What happens ?

It’s the same than beatmaps actually. As long as the player’s skill rating isn’t established, the rating of beatmaps he/she plays on won’t be increased or decreased.

Shiirn wrote:

Long story short:
There are too many aspects to osu! gameplay to quantify "skill" in a single rating system, and even if you COULD simplify them as such, there would be no efficient way to organize or base those ratings against other players.

Yeah, obviously skill cannot be perfectly represented by a number, but ELO ratings work pretty well though. You know about Starcraft II? It’s a strategy game with 3 very different races (Terran, Protoss, Zerg) and a lot of maps that alter the strategies you can use. Depending on the races combinations and the maps, the player’s skill vary greatly. Yet Starcraft II uses a modified ELO algorithm for its leagues and matchmaking system, and it works pretty well (actually, most of the game's matchmaking features are based on ELO or other similar rating systems).

nice idea +2 support

Basically what I do atm to determine one person's skill is look at their profile. the historical stats and playcount+ranked score give good clues as to how good a player is IMO.

I think this one is a pretty good feature if implemented, will give support when I'm capable to do so.

OK, now I see how it works. I didn't think it wouldn't work since there are plenty of established ways for it, we just need to tweak it to fit osu!. I was just curious since I've had no experience with any skill based rating system.

Not true everytime though, Kokizi, just look at me, my PC just broke so I'm probably not gonna get on osu! for a very long time (1 year at most) so my rank will decrease and my playcount will stay the same, while my skill won't decrease overtime (I'm not pro enough to actually lose skill if I don't play regularly).

That's why I kinda see the point of this idea and think it would be a good feature in osu!, but it seems fairly complicated to put in place.

I like where this is going, well done

Mr Color wrote:

but it seems fairly complicated to put in place.

Well, yes. It is pretty complicated, but we have the advantage of looking at how other games with skill-based ratings implement theirs. There is already a bunch that is established, we just need to see how they do it then tweak it for osu!.

I don't think I'm the only one that doesn't like the current ranking system. I can see some players that play worse than I do, and yet they are like 400 ranks above me.

Multiplayer basic rating system isn't hard to implement,but easy to abuse with current osu!
If there good solution for multi-account abuse , we will see in near future.

proxy?, only 1char from unique IP , pena for friend game abuse, song limit, class system, etc etc.


It is good to discuss how to make good system or idea for basic calculation :)

How about a special MP ranking which is like ESO based on players versus players.

dvorak wrote:

It is good to discuss how to make good system or idea for basic calculation

Yes!

anongos wrote:

Mr Color wrote:

but it seems fairly complicated to put in place.

Well, yes. It is pretty complicated, but we have the advantage of looking at how other games with skill-based ratings implement theirs. There is already a bunch that is established, we just need to see how they do it then tweak it for osu!.

I don't think I'm the only one that doesn't like the current ranking system. I can see some players that play worse than I do, and yet they are like 400 ranks above me.

Does anyone know of a rhythm game with a skill rating system?

JesusYamato wrote:

How about a special MP ranking which is like ESO based on players versus players.

While I know that's how it's done in most games, I thought about a player vs maps ratings for the following reasons:

- Rating players and maps in single player or multiplayer mode wouldn't make any difference: developers could use millions of single player games to beta test the multiplayer rating system efficiently. That wouldn't be possible with a player vs player rating system.

- Friends of very different level could play together: even with a high rating player winning every game, other players could increase their ratings as long as they beat some beatmaps.

- I think using beatmaps to rate players would make their ratings more stable. In the following example, A player vs player rating wouldn't work well:

Player A can:
- FC with 95% accuracy "easy" 5-star beatmaps.
- Get random and extremely low results on the hardest approved beatmaps (using "no fail" mod).

Player B can:
- FC with 90% accuracy "easy" 5-star beatmaps.
- Get random and extremely low results on the hardest approved beatmaps (using "no fail" mod).

Given their skills, easy 5-star beatmap plays are best suited to rate these players, and playing these maps, player A would be given an higher rating. But if they prefer playing beatmaps they can't beat:
- A player vs beatmap rating system would "know" these players aren't skilled enough to beat the maps. They would be expected to fail and they would fail => No ratings update.
- A player vs player rating system woud see random results (because it's much more random when you play beatmaps you're far from being able to beat) on a 5.00-star beatmaps like any other => the only fact that would be considered is that a player got 50,000 points while the other one got 100,000. Ratings would be updated, and that's bad imo. On an other map it could have been totally different.

I chose an extreme example to make things clear, but I think it would be a problem in any play on a too easy/hard beatmap. Any counter example?

ps : actually I think player vs player rating system would be feasible, but it would possibly be much harder on the dev team side.

Gerbator wrote:

Does anyone know of a rhythm game with a skill rating system?

That's not what I meant, but I get your point.

Really interesting idea
Too bad I ran out of my star

tl;dr I kinda agree with you, though I have no idea how to improve the ranking system of yours.

Problem is:

How do you judge how hard a map is? Some players suck at certain ARs, some others suck at streams but are incredibly pro at jumps (and the other way around). Some are really good at playing no mod but fail horribly whenever you add a mod, while there are some others that can FC maps in a single play using Hidden, HR or DT... there are lots of factors that you have to think about to see how hard a map is. And it's not like each mod is harder than the another one in every map, some maps are damn hard to do with hidden and some others (most) are just like playing no-mod, while the same can be applied with HR, there are many maps where HR actually makes the map easier since the circle size/AR is horribly low, and well DT always makes the whole thing harder but there are tons of maps that are so damn easy (even insanes) that DT just makes them fairly hard.

Your idea is quite good and stuff, but then again you have to think about what's harder, to FC an insane map or to clear it with a C using DT for example. In many cases some players can FC easily some insane maps with HR, but they get somewhat bad accuracy, but I personally think that's way harder than getting the S no-mod since it requires way more skill. Idk there are lots of factors you got to think about and it gonna end up in people discussing what's harder or not. I think the best way to solve this is to just add a ladder system like W3 or SC (I guess) already have.

Some Player Vs. Player rating system would work, since better players usually get a better score than worse players, that's a damn fact and you can see that happen in every multiplayer room you see, played a lot of multiplayer myself and the ranking is usually pretty much the same, A goes #1 or #2, B goes #1 or #2 (maybe #3), etc. Of course maybe you go sneeze and fail the map because of that, but that won't happen all the damn time, even if you are unlucky in one match, you gonna win the rest of them probably. In every game there is a factor called luck, I hate it too yeah but that's how it works, you can't remove it at all, which is why making an 1v1 system would be just fine. Having played multiplayer games with lots of top/very good/good players, I have seen how top players always (ok, 99% times) get higher scores, and with higher I mean an huge advantage, than good players, maybe they have a little fight with some very good player, but still when you're better you usually do better than a worse player, and that's a fact.

Just finished reading all the posts, and honestly I don't understand about what kind of rating system it suppose to be. Shall it be a pretty new feature or shall it replace some rating that we already have?

Have you ever played any RTS? Or are you familiar with ladder systems? Well the idea is something like that.

Ladder System i think its something like:
Is this the system you're talking about? havent heard it called ladder system before so i was wondering

Wishy22 wrote:

Have you ever played any RTS? Or are you familiar with ladder systems? Well the idea is something like that.

Is it adressed to my question? It yes, then it didn't answer my question. I do understand the rating system which is suggested here but I don't understand "where" it will work at least if it shall replace something that already exists or if it will be parallel to an exisiting "ranked" score system.

Sakura Hana wrote:

Ladder System i think its something like:

Each player starts at a certain ammount of points, winning a game against someone else will give them points depending on the difference in points between both, and losing a game will lose points depending on the difference as well:
Example 1: Player A has 140 points, and won against Player B who has 200 points, Player B has significantly more points than Player A so player A gains many points and Player B loses many points.

Example 2: same scenario but Player B wins, this time Player B wins a small ammount of points, and Player A loses a small ammount of points.

Is this the system you're talking about? havent heard it called ladder system before so i was wondering

Basically yeah, you got 1k rating the other guy has 1,1k you win you get like 30 points he loses 30 points, if the other guy had 99k instead of 1,1k and you win you earn 999999999999 and he loses 99999999999, it's basically 1v1 and it's name ladder game since that system was used by Blizzard in their RTS and got called like that (in W3 at least).

@Neruell: It doesn't matter if it does replace this ranked score system, I think both can co-exist perfectly you just got to add a tab to sort online users by rating and that's all. It could exist like a single button you can press to get a match against another player rated like you who's looking for a rated match.

it can be done with multiple ppl too like in MKWii btw

If it co-exist with the already available ranking system, then ok don't mind that one. If it should replace the already available "ranked" system then I am against it.

While most of the things that Wishy were correct, this topic doesn't request a 1on1 ladder or anything. It doesn't even want to compare players directly. Instead, it treats maps like players and pretty much becomes a 1on1 player vs map ladder. From the results of playing the map, the system decides who won (player or map) and then calculate their scores like Wishy explained. So if you FC a map that has a really high rating, your own rating goes up a lot. If you fail a map that's ranked way lower than you, you will lose a lot.

Thing is what matters is how good you are compared to other players, which can be "calculated" by rating games between players. Both would work but it'd be way more fun to play against other players instead of doing the same solo thing but now in multiplayer (or wherever you want) where you have to farm plays to get a good rating.

A good way to determine how hard something is would be to base it in player's scores, like if you see a map where in 1 out of 500 plays someone gets FC, then that map should be worth more than another one where 1 out of 100 FCs. Problem is this can be abused, then you got the retry issue (of course you gonna take data from solo! most maps are pretty much never ever played in multi rooms), and the problem of underplayed maps where you will have really few data and, from my own experience, whenever I play one of those random beatmaps that are like you do SH and get top 3, I end up quitting after 15 seconds when I see how the map goes. Though this can be solved by filtering beatmaps from 2009 or before (yes most old maps are unplayable shit) and not letting them be "rated" in "rated games". I'm saying this assuming the map being played for rated games is random and not selected by anyone.

Wishy22 wrote:

Though this can be solved by filtering beatmaps from 2009 or before (yes most old maps are unplayable shit).

No ;_;

Well not all of them but checking out every old map gonna be a pain.

A lot of times me and just 1 friend will go against each other in multiplayer mode. We'll take turns picking songs and the rule is, if you don't have the song the other person chose, you download it. Well, of course, whoever chose the song will usually win because they've played it multiple times and figured out any trick patterns, etc. And whoever just downloaded the song and is playing it for the first time might even fail/get a bad score because they've never seen the song before, and it might take them another try to get an A or whatever. Usually when I don't do so well because it's my first time, I'm like whatever, it dropped my accuracy by .02% or something. But now it will be like sacrificing my "points" if I'm doing a song for the first time, just for the fun of it >.< Even in multiplayer mode, if there's no rooms open with songs I have, I'll choose a random song in a room that seems interesting, download it, do it for the first time. This point system seems to me like it would discourage downloading a song and doing it for the first time in multiplayer. What I'm trying to say is: Will the amount of times you've played through the map effect the score? Someone who's played the map 20 times should obviously be doing better than someone of equal skill playing it the first time. Does it really matter? I don't know. Did you already discuss/find a solution to this problem? Haha, I don't know, I didn't read through this whole discussion Just ignore this if you already have a solution or whatever. I do like this idea a lot, it seems a bit complex, but it's definitely possible.