forum

A new rating system

posted
Total Posts
107
This is a feature request. Feature requests can be voted up by supporters.
Current Priority: +52
show more
anonymous_old

Sir Minelli wrote:

0_o wrote:

The main thing is that the current rating system in ambiguous; people have different perceptions of the rating scale (one person would give a map they found "average" a 6, while others would give it an 8)
I highly doubt that they think an 8 is an average score.
I am one of "them" who considers an eight to be average. That's kinda because it is average.

Sir Minelli wrote:

Those are ranked beatmaps which are supposed to be good.
Since when is "good" in the ranking criteria?

Sir Minelli wrote:

That is the score that the ranked beatmaps should be getting more or less. I don't see that as wrong, at all.
Maps should get ranks anywhere from 1~10, not 7~9 as it stands now, using the current ranking scale.
Derekku

strager wrote:

Sir Minelli wrote:

0_o wrote:

The main thing is that the current rating system in ambiguous; people have different perceptions of the rating scale (one person would give a map they found "average" a 6, while others would give it an 8)
I highly doubt that they think an 8 is an average score.
I am one of "them" who considers an eight to be average. That's kinda because it is average.

Sir Minelli wrote:

Those are ranked beatmaps which are supposed to be good.
Since when is "good" in the ranking criteria?

Sir Minelli wrote:

That is the score that the ranked beatmaps should be getting more or less. I don't see that as wrong, at all.
Maps should get ranks anywhere from 1~10, not 7~9 as it stands now, using the current ranking scale.
This. I also use 8 as average.
Gemi
Sir Minelli, you really missed the point. This change is not meant to combat people voting too high/too low/what ever manner of voting someone dislikes, it is meant to combat the fact that each and every player has their own view on how this system works, what should be the average, etc. With fewer choices and more clearly stated meanings for those choices the system would be closer to being the same for everyone.
Wojjan
It works as is. We don't need to lower the choice of opinions if we have a working rating system as is. It isn't broken, it just seems so to people who focus too much on it. A massive spam of 1 ratings has never happened, and people will always rage with the new system and take "strongly dislike".
In the end, it would only give mappers the bliss feeling they're awesome since they say to themselves "oh, you'll always get dislikes with this kind of systems."
TL;DR: no
Gemi

Wojjan wrote:

A massive spam of 1 ratings has never happened
Why do you guys keep bringing up this supposed problem of "people giving 1 ratings" and telling that it is not a problem when no one is even claiming this to be a problem?
Gabi
People bitch about 1s. 1s are going to happen no matter what, but people will whine about them. Why? Because 1s are generally unused.
feels to me that you want to get rid of 1's if i read the OP's post.

if theres anyone to blame about the system right now its yourselves.
CheeseWarlock
The problem

I did a tally of the votes on the first page of ranked maps:


Actual quantities:
SPOILER
1 42
2 4
3 9
4 10
5 29
6 40
7 108
8 330
9 402
10 1353

This should be plain to see. The modal value of votes is 10, with a huge tendency towards upper votes. 59% of all votes are 10s. Are all these users really finding all these maps to be perfect? And 1 is the most effective way to bring down a map's rating, so that's what a user will do when they think a map is in any way overrated. Sure, people are opinionated sometimes, voters need to make greater use of the entire spread, maybe with 5 or 6 representing "this map is pretty average compared to other maps".

Fixing it
We need to establish that 8, 9 and 10 are not the votes to be given to maps ranging from average to good. A first step would be to tell users this on the rating screen. However, that requires faith in the voters, so it might be nice to implement a more complex calculation for the rating system. Weighted votes would be the best way to go about this, using mean to encourage varied votes, and perhaps even vote count to encourage more people to vote to avoid small sample size bias and non-response bias. I posted a similar formula in a previous thread that suggested thumbs up/thumbs down (which wouldn't really work, it would be like allowing users to only vote 1 or 10), but here it is adapted for any sort of numerical system (including the current "out-of-ten"):

vote_weight = abs((vote_maximum/2)-voter.vote_average) * (1+0.1 log10(voter.vote_count))

This still allows users to give out only 1s and 10s (or minimums and maximums) and end up with heavy vote weight, though a standard deviation component could be factored in.

While maps may not see the entire spectrum of scores from 1 to 10, a system such as this should allow for a larger range of scores. As for the old votes, they would be useless in the new system unless a bell curve approach or something similar were used to give current maps a more diverse score range.
anonymous_old
I'm against weighing votes. Everyone's vote should count as much as everyone else's.
FurukawaPan

strager wrote:

I'm against weighing votes. Everyone's vote should count as much as everyone else's.
agreed. And people could vote in strange ways to game the system (artificially boost a 10 by 1'ing a bunch of other maps randomly or vice versa).
awp
The problem with the voting system is, as mentioned before, the fact that it is public. But even if you were to remove voting capability for all but an "expert panel of judges" the problem still wouldn't be resolved in whole, because people have a problem with everything and even relatively accurate beatmap ratings (that is to say, the ratings actually reflect how "good" one beatmap is to another) will offend or anger people who have irrational biases. Which is pretty much everyone.
anonymous_old
So get a panel of people with opposing opinions. =D The biases would cancel out, making every map a 5 (or 50% or whatever). Genius!
qlum
bump because I requested a part of this
0_o
Bumping this beast. What are the thoughts on this issue nowadays?

I'm in favour of a 5-star system myself. Another benefit that would come from this is that since the current ratings would be wiped, old maps and newer maps would be on the same playing field of ratings (9+ ratings were much rarer a year ago than they are now).
Ephemeral
Thumbs fucking up, thumbs fucking down.

None of this weighting bollocks. You either like a map or you don't.
Torran
So we'll be like YouTube and switch out the star rating system with the thumbs up or down system?

Support~!
anonymous_old

Ephemeral wrote:

Thumbs fucking up, thumbs fucking down.

None of this weighting bollocks. You either like a map or you don't.
^
ouranhshc

Ephemeral wrote:

Thumbs fucking up, thumbs fucking down.

None of this weighting bollocks. You either like a map or you don't.
^
Zekira

Ephemeral wrote:

Thumbs fucking up, thumbs fucking down.

None of this weighting bollocks. You either like a map or you don't.
THIS THIS THIS THIS THIS!
qlum
Why not record everything users vote and give them a warning if there average rating is above 7 or below 5. If a player keeps having an average that is ether to high or to low ban him from voting again.
Torran

qlum wrote:

Why not record everything users vote and give them a warning if there average rating is above 7 or below 5. If a player keeps having an average that is ether to high or to low ban him from voting again.
There's nothing wrong with giving low ratings, hm. The star system just has too many inconsistencies; what I consider a 10 is not what you would consider a 10 and vice versa. I can see why YouTube switched it with the thumbs system.
Mashley
As it is, I think it'd be nice to have a like/dislike system a la youtube, though I have started ignoring ratings because they often show a distinct dislike for originality.
Topic Starter
Ekaru
If you count me then that makes 7 for the thumbs up, thumbs down system (starting from Ephemeral's post). Support for that, it's really close to what I originally wanted anyways.
Glass

Ephemeral wrote:

Thumbs fucking up, thumbs fucking down.

None of this weighting bollocks. You either like a map or you don't.
This.
qlum
best for me would be thumbs up thumb sideways thumbs down this also gives the option for a neutral vote
Sakura

qlum wrote:

best for me would be thumbs up thumb sideways thumbs down this also gives the option for a neutral vote
Thumbs sideways = dont vote?

I would like to combine this and this requests for mapper ranking
ouranhshc
subscribing to a mapper shouldn't play a role in mapper rankings, so don't combine them
0_o
I don't like the idea of having to vote the same way for a decent map as I would an awesome map. A thumbs up from someone could mean they didn't think the map sucked, or it could mean they thought it was amazing. I think a 4 or 5 star system would be small enough that the ratings aren't ambiguous, yet large enough for users to accurately portray their opinion of the map.
ouranhshc
i like both ideas.

For faceman's suggestion. Maybe put a confirmation box so you can't accidentally submit a vote. Maybe put a comment box so that players can give feeback about a map.

Maybe make it where only the mapper can see these comments.
anonymous_old

0_o wrote:

I don't like the idea of having to vote the same way for a decent map as I would an awesome map. A thumbs up from someone could mean they didn't think the map sucked, or it could mean they thought it was amazing. I think a 4 or 5 star system would be small enough that the ratings aren't ambiguous, yet large enough for users to accurately portray their opinion of the map.
Does it matter?

Is a person's vote significant enough to matter, really?

I can understand if you want, say, all the maps you voted as "awesome" (5/5), but that probably won't happen (think of the server load ...).
0_o

strager wrote:

0_o wrote:

I don't like the idea of having to vote the same way for a decent map as I would an awesome map. A thumbs up from someone could mean they didn't think the map sucked, or it could mean they thought it was amazing. I think a 4 or 5 star system would be small enough that the ratings aren't ambiguous, yet large enough for users to accurately portray their opinion of the map.
Does it matter?

Is a person's vote significant enough to matter, really?
Sure, a single vote doesn't matter much. I wasn't referring to myself specifically, you can interpret what I said as "I don't like the idea of everyone having to vote the same way for a decent map as they would an awesome map". There are some maps that really are good, but just not great, and with the thumbs up/thumbs down system, they probably wouldn't be distinguished.

strager wrote:

I can understand if you want, say, all the maps you voted as "awesome" (5/5), but that probably won't happen (think of the server load ...).
I dunno if it's because I just woke up or not, but I'm not sure what you mean? =
anonymous_old

0_o wrote:

Sure, a single vote doesn't matter much. I wasn't referring to myself specifically, you can interpret what I said as "I don't like the idea of everyone having to vote the same way for a decent map as they would an awesome map". There are some maps that really are good, but just not great, and with the thumbs up/thumbs down system, they probably wouldn't be distinguished.
Then upvote awesome maps, don't vote good maps, downvote average bad maps.

0_o wrote:

strager wrote:

I can understand if you want, say, all the maps you voted as "awesome" (5/5), but that probably won't happen (think of the server load ...).
I dunno if it's because I just woke up or not, but I'm not sure what you mean? =
Auto-favourites?
qlum
The real problem with the voting system is that people vote to high, give anything they like a 10 or only vote on maps they like. this problem will remain whatever scoring you use. if its thumbs up 5 stars or the current system the problem remains. the only way to solve all these problems is to remove the 10 spammers this can be done in several ways. you can use a warning system that results in voting bans, automatic score penalties to create a good average if someone votes to high or to low above 7.5 or below 6.5 lower or higher all his scores to generate an average within the boundaries. or you can just make voting a privilege to people with at least a ranked score of 500.000.000 this currently means being in the top #3282
anonymous_old

qlum wrote:

The real problem with the voting system is that people vote to high, give anything they like a 10 or only vote on maps they like. this problem will remain whatever scoring you use. if its thumbs up 5 stars or the current system the problem remains. the only way to solve all these problems is to remove the 10 spammers this can be done in several ways. you can use a warning system that results in voting bans, automatic score penalties to create a good average if someone votes to high or to low above 7.5 or below 6.5 lower or higher all his scores to generate an average within the boundaries. or you can just make voting a privilege to people with at least a ranked score of 500.000.000 this currently means being in the top #3282
That's not the only way. A rating weight was suggested earlier, I think.

But you're all noobs. Who cares about map rating anyway. =]
0_o

strager wrote:

Who cares about map rating anyway. =]
Nobody cares about map rating because it's inaccurate and ambiguous, which is the exact problem we're trying to fix.

People vote differently on a 5-star scale than a 10-star scale. If the 5 (or 4 if you want to take out the neutral option) had specific labels (5=Fantastic, 4=Good, 3=OK/Average, 2=Mediocre 1=Bad), it would eliminate that ambiguity and would hopefully result in a more even ratings spread.

Also, I propose changing the poll to what we think the rating system should be changed to, seeing as it's clear that a majority agrees it should be changed.
Soaprman
Increase it to a 1000000-point scale. On this scale, I would objectively rate Don't Say Lazy a 785263/1000000.
qlum

0_o wrote:

strager wrote:

Who cares about map rating anyway. =]
Nobody cares about map rating because it's inaccurate and ambiguous, which is the exact problem we're trying to fix.

People vote differently on a 5-star scale than a 10-star scale. If the 5 (or 4 if you want to take out the neutral option) had specific labels (5=Fantastic, 4=Good, 3=OK/Average, 2=Mediocre 1=Bad), it would eliminate that ambiguity and would hopefully result in a more even ratings spread.

Also, I propose changing the poll to what we think the rating system should be changed to, seeing as it's clear that a majority agrees it should be changed.
still this won't change the fact that people like to vote max without really thinking about it, just taking out halve the score won't change a thing.
I also think practically no-one will vote 2 stars and only a few will use 4 stars it will be the same graph as before.
Topic Starter
Ekaru
I like the user rating display. It's so pretty.
EvianBubble
I remember at first I rated a map, played another difficulty, and was like ... WHEN DID I RATE THIS?! So yea, it's perfectly normal to like on diff and not another, and obviously unless you play all the diffs to rate a map it's just annoying :S I don't even rate maps anymore lol

+1 support
FireballFlame
I pretty much never rated 5 or lower.
Because whenever I felt a map deserved this, I didn't play it to the end...
Jako87
Like/Dislike is the best. Neutral if you don't press nothing. It works well in Youtube.
Guy-kun
Holy crap, when was this implemented?
Gens
:O
Kitsunemimi

Gens wrote:

:O
Yay Gens!!! >ω<

<3!!
theowest
YES YES YES YES YES YES YES YES. And YES.
theowest
Bump. I want this so badly. Like and dislikes like youtube is perfect. If you're a pro youtube user, you know that likes and dislikes are better than any other 1-5/10 rating system.
Topic Starter
Ekaru
Courtesy of D33d:

Some fairly thorough discussion in #modhelp has encouraged me to request this in a much more concise and clear manner as a standalone point. It is clear that many mappers desire a much more simple rating system, which would surely prove to be more useful for on-the-spot responses. With nothing more than "I like it," "I don't care" and "I dislike it," the rating button would be much more approachable and thus possibly encourage more people to vote.

If displayed as three large, shiny buttons upon the completion of a beatmap, it would be much easier for users to give an on-the-spot reaction, in a way which wouldn't result in a horribly inconsistent manner of voting. Moreover, it would reduce the effects of ragevotes and squeeing fangirl votes, which could in turn give a fairer representation of the community's general opinion. Of course, the votes could still be averaged to a ten-point system if that is to stay, only we'd have values which would account for the accuracy of on-the-spot voting. Failing that, the beatmap listings don't even show a decimal average, instead showing a vague chart of arbitrary figures and a like:dislike meter. We might as well have this as the scope of map ratings. Please also bear in mind that a lot of people will use 6-8 to represent "average," 1-2 as "bad" and 9-10 as "good/excellent." This sort of thing varies wildly from person to person.

As there are a lot of people who care about ratings to some extent, but detest the unreliability of individual votes, a more generalised approach would be far, far more useful for what is effectively a thought process of, "Do I like it? How much do I like it? Do I like it more than this map, could it really be much better in the context?" That would be from somebody who actually wants to give a well-considered input. The rest of the users would simply think, "I like it," I don't care" or "I hate it."
": ) : | : (" is what he's asking for BTW.
Kuro

Ekaru wrote:

Courtesy of D33d:

Some fairly thorough discussion in #modhelp has encouraged me to request this in a much more concise and clear manner as a standalone point. It is clear that many mappers desire a much more simple rating system, which would surely prove to be more useful for on-the-spot responses. With nothing more than "I like it," "I don't care" and "I dislike it," the rating button would be much more approachable and thus possibly encourage more people to vote.

If displayed as three large, shiny buttons upon the completion of a beatmap, it would be much easier for users to give an on-the-spot reaction, in a way which wouldn't result in a horribly inconsistent manner of voting. Moreover, it would reduce the effects of ragevotes and squeeing fangirl votes, which could in turn give a fairer representation of the community's general opinion. Of course, the votes could still be averaged to a ten-point system if that is to stay, only we'd have values which would account for the accuracy of on-the-spot voting. Failing that, the beatmap listings don't even show a decimal average, instead showing a vague chart of arbitrary figures and a like:dislike meter. We might as well have this as the scope of map ratings. Please also bear in mind that a lot of people will use 6-8 to represent "average," 1-2 as "bad" and 9-10 as "good/excellent." This sort of thing varies wildly from person to person.

As there are a lot of people who care about ratings to some extent, but detest the unreliability of individual votes, a more generalised approach would be far, far more useful for what is effectively a thought process of, "Do I like it? How much do I like it? Do I like it more than this map, could it really be much better in the context?" That would be from somebody who actually wants to give a well-considered input. The rest of the users would simply think, "I like it," I don't care" or "I hate it."
": ) : | : (" is what he's asking for BTW.
Basically a "Thumbs up - Thumbs down" system
huehuehue, would be better than current system
D33d
Basically, keep it simple, because the average voter is simple. I didn't even realise how old this request was. Perhaps there's been too much ambiguity over the best way to add it, but I really think that my way would be the most effective and flexible. Either show the results as a bar chart or multiply by 3.33 and average the votes as values of 1, 2 and 3.

DEEDIT: Given that peppy is all about streamlining certain systems in the game, e.g. rankings, it gives me even more of a reason for suggesting this. If people really want to express a notable enjoyment or distaste of a map/mapset, then the number of favourites could be aggregated, along with a flagging system for maps which are particularly offensive. Maybe not as extreme as "flag for unrank," but that sort of thing could have a threshold if it were to have any actual impact on a map.

To get a rating out of 10, the smiley face system could represent values of 1, 2 and 3. They could be plugged into the formula (Σratings*3.33)/Number of ratings or something to that effect. Of course, a five-star system would simply need *2 instead. Everything about the proposed system would be simple, which is what this game needs. I'd say that it's also what the game deserves.
Topic Starter
Ekaru
People have generally agreed on either Like/Dislike or Like/Neutral/Dislike. I think that's the only thing left to discuss, really. I'm leaning more towards three options myself.

If people really want to go into detail on how much they like/dislike a map, well... that's why maps' threads are usually left open (unless the map's thread went to hell, of course).
Tshemmp
I'd like to have 3 options: one for maps I don't like, one for "okayish" maps, and one for maps I like.
D33d

Ekaru wrote:

People have generally agreed on either Like/Dislike or Like/Neutral/Dislike. I think that's the only thing left to discuss, really. I'm leaning more towards three options myself.

If people really want to go into detail on how much they like/dislike a map, well... that's why maps' threads are usually left open (unless the map's thread went to hell, of course).
Map thread and comments box on the listing itself. A comment box in the ranking screen could work as well.
Please sign in to reply.

New reply