Guest
home
beatmaps
rankings
community
store
help
forum
  1. Forums
  2. osu!
  3. Development
  4. Feature Requests

A new rating system

posted
Total Posts
107
This is a feature request. Feature requests can be voted up by supporters.
Current Priority: +52
show more
Sir Minelli
524 posts
Joined September 2008
Sir Minelli
I'm trully sick of hearing new suggestions for this subject over and over.

The rating system is ok as it is.
There will ALWAYS be players that will rate your map low, even if you think that your map deserves a high rating.
You can't call them "trollers" for doing this.
Maybe they really disliked the map... or the song... Maybe they doesn't like story boards... Maybe they even dislike you.
All of these previous reasons are valid in order to rate your map as they please.

Letting everyone know who voted 1 or any other score, in my opinion, is not etic.
Votes are supposed to be personal / secret. We shouldn't change that for a whine.

61% of whines up to now.
0_o
2,155 posts
Joined December 2008
0_o
This map has been deleted on the request of its creator. It is no longer available.
Sir Minelli
524 posts
Joined September 2008
Sir Minelli
This map has been deleted on the request of its creator. It is no longer available.
anonymous_old
0 posts
Here since the beginning
anonymous_old

Sir Minelli wrote:

0_o wrote:

The main thing is that the current rating system in ambiguous; people have different perceptions of the rating scale (one person would give a map they found "average" a 6, while others would give it an 8)
I highly doubt that they think an 8 is an average score.
I am one of "them" who considers an eight to be average. That's kinda because it is average.

Sir Minelli wrote:

Those are ranked beatmaps which are supposed to be good.
Since when is "good" in the ranking criteria?

Sir Minelli wrote:

That is the score that the ranked beatmaps should be getting more or less. I don't see that as wrong, at all.
Maps should get ranks anywhere from 1~10, not 7~9 as it stands now, using the current ranking scale.
Derekku
COOL DUDE
7,270 posts
Joined March 2009
Derekku

strager wrote:

Sir Minelli wrote:

0_o wrote:

The main thing is that the current rating system in ambiguous; people have different perceptions of the rating scale (one person would give a map they found "average" a 6, while others would give it an 8)
I highly doubt that they think an 8 is an average score.
I am one of "them" who considers an eight to be average. That's kinda because it is average.

Sir Minelli wrote:

Those are ranked beatmaps which are supposed to be good.
Since when is "good" in the ranking criteria?

Sir Minelli wrote:

That is the score that the ranked beatmaps should be getting more or less. I don't see that as wrong, at all.
Maps should get ranks anywhere from 1~10, not 7~9 as it stands now, using the current ranking scale.
This. I also use 8 as average.
Gemi
osu! Alumni
685 posts
Joined March 2008
Gemi
Sir Minelli, you really missed the point. This change is not meant to combat people voting too high/too low/what ever manner of voting someone dislikes, it is meant to combat the fact that each and every player has their own view on how this system works, what should be the average, etc. With fewer choices and more clearly stated meanings for those choices the system would be closer to being the same for everyone.
Wojjan
806 posts
Joined August 2008
Wojjan
It works as is. We don't need to lower the choice of opinions if we have a working rating system as is. It isn't broken, it just seems so to people who focus too much on it. A massive spam of 1 ratings has never happened, and people will always rage with the new system and take "strongly dislike".
In the end, it would only give mappers the bliss feeling they're awesome since they say to themselves "oh, you'll always get dislikes with this kind of systems."
TL;DR: no
Gemi
osu! Alumni
685 posts
Joined March 2008
Gemi

Wojjan wrote:

A massive spam of 1 ratings has never happened
Why do you guys keep bringing up this supposed problem of "people giving 1 ratings" and telling that it is not a problem when no one is even claiming this to be a problem?
Gabi
2,554 posts
Joined December 2008
Gabi
People bitch about 1s. 1s are going to happen no matter what, but people will whine about them. Why? Because 1s are generally unused.
feels to me that you want to get rid of 1's if i read the OP's post.

if theres anyone to blame about the system right now its yourselves.
CheeseWarlock
osu! Alumni
1,185 posts
Joined May 2008
CheeseWarlock
The problem

I did a tally of the votes on the first page of ranked maps:


Actual quantities:
SPOILER
1 42
2 4
3 9
4 10
5 29
6 40
7 108
8 330
9 402
10 1353

This should be plain to see. The modal value of votes is 10, with a huge tendency towards upper votes. 59% of all votes are 10s. Are all these users really finding all these maps to be perfect? And 1 is the most effective way to bring down a map's rating, so that's what a user will do when they think a map is in any way overrated. Sure, people are opinionated sometimes, voters need to make greater use of the entire spread, maybe with 5 or 6 representing "this map is pretty average compared to other maps".

Fixing it
We need to establish that 8, 9 and 10 are not the votes to be given to maps ranging from average to good. A first step would be to tell users this on the rating screen. However, that requires faith in the voters, so it might be nice to implement a more complex calculation for the rating system. Weighted votes would be the best way to go about this, using mean to encourage varied votes, and perhaps even vote count to encourage more people to vote to avoid small sample size bias and non-response bias. I posted a similar formula in a previous thread that suggested thumbs up/thumbs down (which wouldn't really work, it would be like allowing users to only vote 1 or 10), but here it is adapted for any sort of numerical system (including the current "out-of-ten"):

vote_weight = abs((vote_maximum/2)-voter.vote_average) * (1+0.1 log10(voter.vote_count))

This still allows users to give out only 1s and 10s (or minimums and maximums) and end up with heavy vote weight, though a standard deviation component could be factored in.

While maps may not see the entire spectrum of scores from 1 to 10, a system such as this should allow for a larger range of scores. As for the old votes, they would be useless in the new system unless a bell curve approach or something similar were used to give current maps a more diverse score range.
anonymous_old
0 posts
Here since the beginning
anonymous_old
I'm against weighing votes. Everyone's vote should count as much as everyone else's.
FurukawaPan
649 posts
Joined August 2008
FurukawaPan

strager wrote:

I'm against weighing votes. Everyone's vote should count as much as everyone else's.
agreed. And people could vote in strange ways to game the system (artificially boost a 10 by 1'ing a bunch of other maps randomly or vice versa).
awp
osu! Alumni
11,869 posts
Here since the beginning
awp
The problem with the voting system is, as mentioned before, the fact that it is public. But even if you were to remove voting capability for all but an "expert panel of judges" the problem still wouldn't be resolved in whole, because people have a problem with everything and even relatively accurate beatmap ratings (that is to say, the ratings actually reflect how "good" one beatmap is to another) will offend or anger people who have irrational biases. Which is pretty much everyone.
anonymous_old
0 posts
Here since the beginning
anonymous_old
So get a panel of people with opposing opinions. =D The biases would cancel out, making every map a 5 (or 50% or whatever). Genius!
qlum
481 posts
Joined October 2008
qlum
bump because I requested a part of this
0_o
2,155 posts
Joined December 2008
0_o
Bumping this beast. What are the thoughts on this issue nowadays?

I'm in favour of a 5-star system myself. Another benefit that would come from this is that since the current ratings would be wiped, old maps and newer maps would be on the same playing field of ratings (9+ ratings were much rarer a year ago than they are now).
Ephemeral
Inland Empire
3,936 posts
Joined April 2009
Ephemeral
Thumbs fucking up, thumbs fucking down.

None of this weighting bollocks. You either like a map or you don't.
Torran
724 posts
Joined November 2008
Torran
So we'll be like YouTube and switch out the star rating system with the thumbs up or down system?

Support~!
anonymous_old
0 posts
Here since the beginning
anonymous_old

Ephemeral wrote:

Thumbs fucking up, thumbs fucking down.

None of this weighting bollocks. You either like a map or you don't.
^
ouranhshc
2,338 posts
Joined July 2009
ouranhshc

Ephemeral wrote:

Thumbs fucking up, thumbs fucking down.

None of this weighting bollocks. You either like a map or you don't.
^
Zekira
9,233 posts
Joined September 2008
Zekira

Ephemeral wrote:

Thumbs fucking up, thumbs fucking down.

None of this weighting bollocks. You either like a map or you don't.
THIS THIS THIS THIS THIS!
qlum
481 posts
Joined October 2008
qlum
Why not record everything users vote and give them a warning if there average rating is above 7 or below 5. If a player keeps having an average that is ether to high or to low ban him from voting again.
Torran
724 posts
Joined November 2008
Torran

qlum wrote:

Why not record everything users vote and give them a warning if there average rating is above 7 or below 5. If a player keeps having an average that is ether to high or to low ban him from voting again.
There's nothing wrong with giving low ratings, hm. The star system just has too many inconsistencies; what I consider a 10 is not what you would consider a 10 and vice versa. I can see why YouTube switched it with the thumbs system.
Mashley
2,042 posts
Joined September 2008
Mashley
As it is, I think it'd be nice to have a like/dislike system a la youtube, though I have started ignoring ratings because they often show a distinct dislike for originality.
Ekaru
3,617 posts
Joined July 2008
Topic Starter
Ekaru
If you count me then that makes 7 for the thumbs up, thumbs down system (starting from Ephemeral's post). Support for that, it's really close to what I originally wanted anyways.
Glass
Glass

Ephemeral wrote:

Thumbs fucking up, thumbs fucking down.

None of this weighting bollocks. You either like a map or you don't.
This.
qlum
481 posts
Joined October 2008
qlum
best for me would be thumbs up thumb sideways thumbs down this also gives the option for a neutral vote
Sakura
osu! Alumni
7,979 posts
Joined March 2010
Sakura

qlum wrote:

best for me would be thumbs up thumb sideways thumbs down this also gives the option for a neutral vote
Thumbs sideways = dont vote?

I would like to combine this and this requests for mapper ranking
ouranhshc
2,338 posts
Joined July 2009
ouranhshc
subscribing to a mapper shouldn't play a role in mapper rankings, so don't combine them
0_o
2,155 posts
Joined December 2008
0_o
I don't like the idea of having to vote the same way for a decent map as I would an awesome map. A thumbs up from someone could mean they didn't think the map sucked, or it could mean they thought it was amazing. I think a 4 or 5 star system would be small enough that the ratings aren't ambiguous, yet large enough for users to accurately portray their opinion of the map.
ouranhshc
2,338 posts
Joined July 2009
ouranhshc
i like both ideas.

For faceman's suggestion. Maybe put a confirmation box so you can't accidentally submit a vote. Maybe put a comment box so that players can give feeback about a map.

Maybe make it where only the mapper can see these comments.
anonymous_old
0 posts
Here since the beginning
anonymous_old

0_o wrote:

I don't like the idea of having to vote the same way for a decent map as I would an awesome map. A thumbs up from someone could mean they didn't think the map sucked, or it could mean they thought it was amazing. I think a 4 or 5 star system would be small enough that the ratings aren't ambiguous, yet large enough for users to accurately portray their opinion of the map.
Does it matter?

Is a person's vote significant enough to matter, really?

I can understand if you want, say, all the maps you voted as "awesome" (5/5), but that probably won't happen (think of the server load ...).
0_o
2,155 posts
Joined December 2008
0_o

strager wrote:

0_o wrote:

I don't like the idea of having to vote the same way for a decent map as I would an awesome map. A thumbs up from someone could mean they didn't think the map sucked, or it could mean they thought it was amazing. I think a 4 or 5 star system would be small enough that the ratings aren't ambiguous, yet large enough for users to accurately portray their opinion of the map.
Does it matter?

Is a person's vote significant enough to matter, really?
Sure, a single vote doesn't matter much. I wasn't referring to myself specifically, you can interpret what I said as "I don't like the idea of everyone having to vote the same way for a decent map as they would an awesome map". There are some maps that really are good, but just not great, and with the thumbs up/thumbs down system, they probably wouldn't be distinguished.

strager wrote:

I can understand if you want, say, all the maps you voted as "awesome" (5/5), but that probably won't happen (think of the server load ...).
I dunno if it's because I just woke up or not, but I'm not sure what you mean? =
anonymous_old
0 posts
Here since the beginning
anonymous_old

0_o wrote:

Sure, a single vote doesn't matter much. I wasn't referring to myself specifically, you can interpret what I said as "I don't like the idea of everyone having to vote the same way for a decent map as they would an awesome map". There are some maps that really are good, but just not great, and with the thumbs up/thumbs down system, they probably wouldn't be distinguished.
Then upvote awesome maps, don't vote good maps, downvote average bad maps.

0_o wrote:

strager wrote:

I can understand if you want, say, all the maps you voted as "awesome" (5/5), but that probably won't happen (think of the server load ...).
I dunno if it's because I just woke up or not, but I'm not sure what you mean? =
Auto-favourites?
qlum
481 posts
Joined October 2008
qlum
The real problem with the voting system is that people vote to high, give anything they like a 10 or only vote on maps they like. this problem will remain whatever scoring you use. if its thumbs up 5 stars or the current system the problem remains. the only way to solve all these problems is to remove the 10 spammers this can be done in several ways. you can use a warning system that results in voting bans, automatic score penalties to create a good average if someone votes to high or to low above 7.5 or below 6.5 lower or higher all his scores to generate an average within the boundaries. or you can just make voting a privilege to people with at least a ranked score of 500.000.000 this currently means being in the top #3282
anonymous_old
0 posts
Here since the beginning
anonymous_old

qlum wrote:

The real problem with the voting system is that people vote to high, give anything they like a 10 or only vote on maps they like. this problem will remain whatever scoring you use. if its thumbs up 5 stars or the current system the problem remains. the only way to solve all these problems is to remove the 10 spammers this can be done in several ways. you can use a warning system that results in voting bans, automatic score penalties to create a good average if someone votes to high or to low above 7.5 or below 6.5 lower or higher all his scores to generate an average within the boundaries. or you can just make voting a privilege to people with at least a ranked score of 500.000.000 this currently means being in the top #3282
That's not the only way. A rating weight was suggested earlier, I think.

But you're all noobs. Who cares about map rating anyway. =]
0_o
2,155 posts
Joined December 2008
0_o

strager wrote:

Who cares about map rating anyway. =]
Nobody cares about map rating because it's inaccurate and ambiguous, which is the exact problem we're trying to fix.

People vote differently on a 5-star scale than a 10-star scale. If the 5 (or 4 if you want to take out the neutral option) had specific labels (5=Fantastic, 4=Good, 3=OK/Average, 2=Mediocre 1=Bad), it would eliminate that ambiguity and would hopefully result in a more even ratings spread.

Also, I propose changing the poll to what we think the rating system should be changed to, seeing as it's clear that a majority agrees it should be changed.
Soaprman
1,392 posts
Joined January 2008
Soaprman
Increase it to a 1000000-point scale. On this scale, I would objectively rate Don't Say Lazy a 785263/1000000.
qlum
481 posts
Joined October 2008
qlum

0_o wrote:

strager wrote:

Who cares about map rating anyway. =]
Nobody cares about map rating because it's inaccurate and ambiguous, which is the exact problem we're trying to fix.

People vote differently on a 5-star scale than a 10-star scale. If the 5 (or 4 if you want to take out the neutral option) had specific labels (5=Fantastic, 4=Good, 3=OK/Average, 2=Mediocre 1=Bad), it would eliminate that ambiguity and would hopefully result in a more even ratings spread.

Also, I propose changing the poll to what we think the rating system should be changed to, seeing as it's clear that a majority agrees it should be changed.
still this won't change the fact that people like to vote max without really thinking about it, just taking out halve the score won't change a thing.
I also think practically no-one will vote 2 stars and only a few will use 4 stars it will be the same graph as before.
Ekaru
3,617 posts
Joined July 2008
Topic Starter
Ekaru
I like the user rating display. It's so pretty.
show more
Please sign in to reply.

New reply

0
54 / 107