Let's fix the beatmap ranking process

posted
Total Posts
21
show more
Stefan
here comes the short version of CDFA's post: Utopian suggestions that are not feasible at the moment or at any time due their unreliability and the lack of support for certain sections, such as Metadata.
Free Hong Kong

Shiguma wrote:

Me reading this post:



Also me reading this post:

hi shiguma

I like some of these ideas like:

abraker wrote:

  1. Allow beatmap metadata information to be in a separate file, allowing change after ranking
Other ideas I disagree with + reasons

abraker wrote:

  1. Replace BN with an automatic system capable of determining whether a beatmap is suitable for ranking according to the ranking criteria rules This is kind of silly because there are limited resources at to how many maps can store live scores and such. Also this would counteract the purpose of the BNG and QAT as a whole, which should be assuring quality maps are getting ranked, not just maps that are "not unrankable"


  2. Remove guidelines in ranking criteria and only keep rules
    The purpose of guidelines have been to help new mappers start/improve (though wording in the document is silly). Lower difficulty maps are meant to teach new players how to play the game, and without the lower difficulty guidelines it would be difficult to determine what exactly lower players are capable of without having it memorized and whatnot.
  3. QAT's role would remain mostly as is now - Managing ranking criteria, unranking maps that many feel should not have been ranked, and identifying what needs to change in the ranking criteria to prevent more of such maps being ranked this is unnecessary to post? or did i misread it lol

  4. Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them this goes to the same point I gave above on automated BN

  5. Make individual difficulties automatically go to ranked when it reaches a certain amount of scores same as above

  6. Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions I'm assuming this was tied into the previous ideas, though nobody really wants to update their ranked map unless there was something pretty minor, which in qualified,
    a QAT can come to help. Most people who rank maps don't want to look at their map again and would most likely rather delete them than remap/etc. (this sounds like an opinion so feel free to ask other people who have ranked maps)


  7. Fix star rating
    This is a weird one to say, but just kind of putting it out there without method by which you would solve is pretty troublesome. In a sense there is nothing wrong with star rating. There are many thing you can't account for with a computer system because players at different skill levels play maps completely differently. Even within the same skill level players still play maps pretty differently so difficulty will be relative to the player and the map.
also you should probably rework your FAQ lol
Nitrous
I didn't bother reading the whole thing so I went ahead and read the tl;dr section.

  1. Replace BN with an automatic system capable of determining whether a beatmap is suitable for ranking according to the ranking criteria rules
    While the current system is slow and manual with some degree of bias, it's still okay. Using an automated system could cause issues making bias to more generic maps than to unique and creative ones.
  2. Allow beatmap metadata information to be in a separate file, allowing change after ranking
    This would kill older beatmaps that use different audio files for each difficulty, but I definitely agree on this one.
  3. Create a group for people who do metadata checking, and verify DMCA and NSFW status of the map, that is notified of any unchecked maps which become ranked
    Unsure about this, since this is the current job of the Quality Assurance Team and Global Moderation Team.
  4. Remove guidelines in ranking criteria and only keep rules
    The guidelines is like the bible for new mappers to know what there should and shouldn't be on a map (which in my case, it did).
  5. QAT's role would remain mostly as is now - Managing ranking criteria, unranking maps that many feel should not have been ranked, and identifying what needs to change in the ranking criteria to prevent more of such maps being ranked
    This is the current job of the Quality Assurance Team. Not pointing anything to this.
  6. Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them
    Basically adding leaderboards to all pending beatmaps. I did read a post one time and peppy vetoed this idea (correct me if I'm wrong).
    Further reasoning in the next point below.

  7. Make individual difficulties automatically go to ranked when it reaches a certain amount of scores
    So it relies on popularity/playcount now? What if the song isn't popular/outdated? I disagree with this.
  8. Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions
    Definitely. A similar system like Git would be very helpful.
  9. Fix star rating
    SR is okay as it is, it just relies on formulas which apparently mappers know how to bend (in Standard mode's case).


I agree the process is slow and hard, but destroying and replacing it completely with a new one isn't definitely the solution.
Nao Tomori
SR is not ok. it's fundamentally flawed in what it values and doesn't value and as such has had an extremely noticeable and detrimental effect on mapping *diversity* as a whole.

The current system already is ridiculously biased towards generic maps, and an automated one would not be biased at all because there would be various objective criteria and that's it. No subjective things like "oh I think you should remove this long slider because the vocal line you're following with it is bad and do it my way because I'm better than you" (paraphrased ofc).
Monstrata
Replacing BN's isn't as bad of an idea as people are making it out to be.

It's far too late to enforce mapping quality at this point. Maps that are subjectively low quality can be ranked as long as proper justification is provided, meaning that there is absolutely no objective definition of "high quality" to begin with. The purpose of BN's was to push quality maps forward, but anything can be quality if you write enough bs about it, or talk about how you love it, or say it was a clever concept, and get the proper support for it.

Note also, that the majority of "quality" maps pushed forward by BN's are seen as generic and average to the player base: anime jump maps, standard ctrl+h/j and well=structured maps that just "flow well". People are looking for "creative" and "different" but the meta streamlines mapping styles that are clean, flow well, and are generally unoffensive to mappers and modders. If you want creative and different maps in the ranked section you need to be prepared to promote subjectively lower quality maps.

The questions people should really be asking is: Why should I care about mapping quality?

  1. Why does a low quality map offend you?
  2. If a low quality map is getting pushed forward, why do you bother to try and stop it? Especially if it's by a popular mapper or BN, the map will very likely get ranked eventually anyways. You are only wasting your time as well as the mapper's.
  3. Does a single map create a precedent for low quality maps? The entire argument about Alien stemmed from "if this gets ranked, we might as well rank every beginner's map". Did that end up happening?
  4. Why do you feel so entitled that you believe someone's hard work should not be recognized? By preventing a map from reaching the ranked status, you are effectively saying "Your map doesn't deserve any recognition until you change it to something that doesn't offend me".


The truth is, low quality maps (aside from CBCC, Haitai, etc...) will disappear and be forgotten. Frankly, the only people who remember subjectively low quality maps are people active in the mapping scene. And additionally, what mappers find low quality clearly doesn't align with what players find as low quality to begin with.

Talking to peppy made me realize just how different the playerbase and the mapperbase can be at times. Top players, and vocal members of the community in reality, represent the vocal minority of the community. Why did so many people vote for Highscore, or Inferno, or Hitorigoto, even though they clearly (in the perspective of a mapper) aren't creative/high quality maps? Because the average player is simply looking to have fun, and play songs that they enjoy. That is their definition of high quality: not something that is beautifully mapper and engineered, but rather, a map that simply expresses the song well, and is fun to play.

Quality is subjective, and people care too much about quality. Every day, the prestige of getting your map ranked is lowered. It's already a truth among most mappers that your map will likely be forgotten by what 95% of the playerbase within a week after the map gets ranked. It's just the truth.
Ashton
Replacing BNs with a system is a bad idea. It throws subjectivity away, which is an important factor to ranking beatmaps.

I think subjectivity is important because if the only requirements to rank a map was to: “>follow ranking criteria<“ then we’d see hundreds of low quality maps. Is “quality” subjective? Yes, but not as subjective as Monstrata or other mappers make it out to be. it is generally agreed on that a map should have good flow, aesthetics, spacing, etc... Although of course whatever is considered good flow or aesthetics is subjective, there are common and popular opinions on what makes a map good:

-Flow, how easily and comfortable you snap from one object to another.
-Aesthetics, no messy overlaps, blanketing sliders, of course these rules can be bent as long as a map is consistent. If a maps idea is to use weird abstract patterns, as long as it has some sort of solid justification and a presence of skill is apparent, it should still be considered good.
-Spacing, in intense or kiai parts of a song the spacing should be increased, in less intense parts, spacing should be lowered. Spacing should be consistent through your whole map, and if raised in high intensity parts, raised again with the same degree of the high intensity part of a song repeats.



Those three points I mentioned above cannot be checked by a system, that’s why having a real human being and community look at it is much better than a computer.


Also, drama happens in a lot less maps then you think. There are many maps being ranked each (I want to say day but I’m not sure) without any drama at all. The more popular you are the more prone you are to drama. Encouraging good manners and behaviour is a far better solution to drama than replacing human interaction all together.
Topic Starter
abraker

CDFA wrote:

You can input something into a computer and give it parameters that can be sort of like looked at (ie. "if note a is 3.5x from note b, mark unrankable, require fix") or you can give it a bunch of example maps and have it determine what is... most similar to those maps? idk but I really do think it's a stretch to be able to honestly believe that an AI system can determine how suitable a map is for ranking.

Lemme give an example:

https://osu.ppy.sh/s/723944

https://osu.ppy.sh/s/13397

These two maps are both based on distance spacing, and to an AiMod sort of thing, they're okay. How would we determine which is acceptable for ranking (obviously they both are, but still). What would the AI say to me to fix these things? Does it talk about patterns? Does it talk about note density? does it talk about how it relates to the map at all? Like I seriously don't get it lmao.
It is not the job of AImod to say how to fix things. Its job is to say what and why something can't be ranked. If you want to know how to fix things, you can ask someone with mapping experience. In this system you personally have to be willing to recognize faults within the map and have the drive to improve them in order to make ones that players will like to play.

CDFA wrote:

It's obvious from this thread that you really don't have experience in the ranking process, because you'd be able to see that a lot of these things are just plain silly, through and through.

There ARE problems with the ranking criteria, there ARE problems with things such as favortism, varying ideas of quality, inconsistent and vague rulings, but what you're proposing here doesn't fix any of that, what it does is makes a fucking joke using a literally impossible to implement system.

Like after getting 30 ranked maps and 1300+ kudosu, it really shows you what is reasonable in the ranking process, and how you can't have these fantastical ideas of fully automated systems that will somehow not rank absolutely shitty maps.
I might not have experience participating with the ranking process, but I for sure have seen enough of it to know stuff isn't right. Yes, this system will not fix favoritism globally. Popular mappers will still have maps with the most plays on scoreboards, but it does offer a chance to lesser known mappers willing to experiment with different mapping styles. Instead of prohibiting the map from getting ranked due to varying ideas of quality, the map might not get popular because most don't see the map as good. The possibility of it being ranked in the current state is what the mapper needs to think, not the inability of the map to get ranked in the current state. Nobody should tell you the thing is shit and won't be nominated. It is up to the mapper to whether ask how the map can be improved to be more enjoyable and it is up to the mapper to decide whether to apply the suggested points to make the map more enjoyable. Or perhaps the mapper has a small fanbase that is not big enough to get the number of scores to get ranked, yet the mapper sees on the scoreboard that people are playing the maps and is satisfied with that. It offers more positivism, outlook, and flexibility.

CDFA wrote:

abraker wrote:

Q: Modders mod to gain kudosu to promote their maps. What does the metadata checking group gain by doing the work?
A: They do just because they want to. Because they want correct metadata on maps.
In your perfect world they do lmfao. There HAS to be some sort of incentive to mod, that's why kudosu exists in the fucking first place lmfao. This is currently dealt with through sort of having a team that gets a special title and some special privledges, but assuming that people will just do this sort of work out of the goodness of their heart is just plain silly. It's kind of like when I have to cover
If you don't think preventing bad metadata is a good enough incentive, then, there can be something offered in exchange. I can't imagine what at the moment, but perhaps someone else can. I did not expect the entire suggestion to be bullet proof, and I expect people to provide solution/alternative suggestions.

CDFA wrote:

Also I'm not sure how you're getting the idea that in this somehow perfect system that metadata is somehow the ONLY thing that can't be automated
Care to expand on this point?

CDFA wrote:

abraker wrote:

Q: The maps are ranked so they stand out and not among piles of other shit. What you are doing defeats the whole purpose of beatmaps being ranked.
A: Living with majority of the good maps being unranked is not that bad, guys. Majority of the good mania maps are unranked. We just follow the mappers and share the good maps we find via favorites or multi. No name mappers actually have a better chance in this because they can always go to multi and put up their maps, playing and sharing them with everyone else. If the people who played the map actually like it, then they themselves would put it up next time. They no longer need to argue with someone else to get it ranked. If the map is bad, they would have a hard time getting it shared, period.
Nevermind, this is the stupidest shit I've read all day.

That's not what the ranking process was for lmfao. The ranking process was for ensuring that maps are like, you know, complete, rankable, and accessible. It's literally just a fucking approval process. You submit a map, get feedback, and then it gets ranked, and people play it. I don't know what you think modding is but that's literally what it is. It's like writing a god damn essay. "I liked this, I didn't like this, this slider isn't effective, you have a wrong BPM" like what the fuck do you think people do when they mod lmfao" It's checked to make sure the damn thing is a decent enough quality like aaaaaaaaaa. Also I have no idea what you're going off of with "no name mappers have a better chance" like uhhhhhhhh everyone can do what you fucking just said and big name mappers already have that name-recognizability. Plus it's also based on the music that people like. Like when I played in multi way back when, I didn't think "oh man this map is great" I was like "hey I like this song, I'm gonna download it"
As Monstrata said in a later post,

Monstrata wrote:

anything can be quality if you write enough bs about it, or talk about how you love it, or say it was a clever concept, and get the proper support for it.
Having people judge and tell you what is wrong and is prone subjective, arbitrary, and inconsistent results. The mapper shouldn't forced to undergo that if the mapper does not wish to. If the community likes a shit map enough to get it ranked by playing it and not complain afterwards, then that is good enough to satisfy it being complete and accessible. If somebody catches a wrongly snapped note after it is ranked, well players made enough scores on the map to not notice it. If the mapper is willing to fix it, then good, if not, then so be it. If you can only tell something is wrong by looking in the editor, then you are just nitpicking. Things need to be quality enough for player's enjoyment, not holy perfection. When I mod maps, I always mod via playtests and try my best not to look into the editor unless I am asked for opinions about patterns.


CDFA wrote:

IT'S NOT ABOUT WHAT IT MEANS TO YOU IF YOU'RE MAKING CONTENT FOR A GAME, IT'S ABOUT WHAT IT MEANS TO THE COMMUNITY. THAT'S FUCKING FANTASTIC IF YOU THINK SOMETHING WORKS REALLY WELL, BUT IF THE PEOPLE THAT ARE PLAYING YOUR MAP DON'T LIKE IT, THEN YOU SHOULD EITHER CHANGE IT SO YOUR INTENTIONS ARE BETTER TRANSLATED, OR YOU SHOULD REMOVE IT. Like if you have an artistic view of making maps then there's nothing wrong with making maps, graving them, and letting them be with cool sort of experimental stuff, or getting it Loved. But this is a fucking game, and you are producing content for the community to play. The game has standards, guidelines, and isn't really seen as an artistic medium as much as it is filling the game with multiple songs. There is individuality, and you can SEE it as art, but the importance of ranking a beatmap isn't in the artistic side of things, but on the "this is acceptable for the general community to play."
If people that are playing your map don't like it, then it will not be shared. If it is not shared, then it won't get enough scores. If it doesn't get enough scores, then it won't get ranked. The standards are in the ranking criteria which are enforced by an automatic system that determines whether the beatmap follows the criteria. Ranking criteria can be changed when sufficient amount of players complain about a map that the players believe shouldn't have passed the ranking criteria.

I believe beatmaps are artworks and should be seen as such. Games themselves are pieces of art. Anything that allows an individual's creativity to flow free is art. Another individual saying to change art because a piece of it is shit is wrong. A natural process shall dictate what art is good, and that is determined by sufficient positive community interaction with it.
Sinnoh
Hi one feasibility check

abraker wrote:

This proposes a new way to determine beatmap ranking. This does require big changes in osu! infrastructure, and by no means simple nor easy. The point is to discuss this as a possible alternative and what this might result in. TL;DR:
  1. Remove guidelines in ranking criteria and only keep rules Most guidelines should not be broken for good reason. If this became the case, most guidelines would be moved to the rules. There are almost no maps that break guidelines with good reasoning.

  2. Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them This would increase the number of loved/ranked maps by 2660%, how do you expect the servers to be able to keep up over a long period of time with the huge increase of maps.

  3. Make individual difficulties automatically go to ranked when it reaches a certain amount of scores It's just a popularity contest now. Anyone that enjoys mapping high-quality low difficulty sets will have to give up because the only maps that would be popular enough are pp farms and extra difficulties. How many E/N/H sets are in the loved section?

  4. Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions This is literally impossible. When updating a map, you make changes. When you move a circle, previous replays will miss because they clicked the old circle.

  5. Fix star rating This alone would be enough to fix the system. If it was easy enough as it is to say then it would have been done already.
If beatmap ranking was automatic, who is going to check for unsnapped objects or objectively unrankable issues? Because of the significant increase in maps, the QAT can't check all of them alone. The BN team is still required to check maps, which means the whole system changed and the outcome never changed, which is pretty redundant.

Even if this was considered to be a good idea, it's not possible to create regardless of whether or not people like it. End of discussion.
Topic Starter
abraker

abraker wrote:

Replace BN with an automatic system capable of determining whether a beatmap is suitable for ranking according to the ranking criteria rules

imbor wrote:

Also this would counteract the purpose of the BNG and QAT as a whole, which should be assuring quality maps are getting ranked, not just maps that are "not unrankable"
The playerbase will determine quality maps that would get ranked without actively saying which map should or should not be ranked. If the map attains enough scores, then the beatmap can be ranked. If a map is shit, then it can't possibly gain enough scores to be ranked. Nobody would play it, people would say to avoid it in the Disqus comments, and nobody would share it. If a shitty meme map gets ranked and players don't complain about it, then the shitty meme is a sufficiently good map. If enough players do complain about it, then the ranking criteria would be changed to prevent whatever the fault with the map is from happening again. Finally, the map is deranked and the map's scoreboard is wiped when the new criteria is applied. Wiping scoreboards serves as a deterrent against casual opinions and forces the people to think about whether they really want the new change to occur.

-------------------------------------------------------------------------------------


abraker wrote:

Remove guidelines in ranking criteria and only keep rules

imbor wrote:

The purpose of guidelines have been to help new mappers start/improve (though wording in the document is silly). Lower difficulty maps are meant to teach new players how to play the game, and without the lower difficulty guidelines it would be difficult to determine what exactly lower players are capable of without having it memorized and whatnot.
There should be guides posted on the mapping subforum for this. Putting it in the ranking criteria causes modders and bn to take it at face value.

-------------------------------------------------------------------------------------


abraker wrote:

QAT's role would remain mostly as is now - Managing ranking criteria, unranking maps that many feel should not have been ranked, and identifying what needs to change in the ranking criteria to prevent more of such maps being ranked

imbor wrote:

this is unnecessary to post? or did i misread it lol
With the amount of radical changes I am proposing, it doesn't hurt to assure QAT are still essential

-------------------------------------------------------------------------------------


abraker wrote:

Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them

imbor wrote:

This is kind of silly because there are limited resources at to how many maps can store live scores and such.
Let me do the math with worst case scenario estimates since I have not done so. You add a new score to the database, which should take up no more than 1 kb of space. Suppose 50k scores on average, that's 50 kb of space. Consider an average of 5 diffs per mapset and that there as many mapsets as there are registred user on osu! (11.7 million). This totals up to... 2.925 petabytes

Ok, an issue. Obviously right now there are not close to 11.5M maps in the pending and graveyard section combined. There are 125 pages of each, which 20 mapsets on each page, which gives us 5k mapsets total, which is considerably less. Apparently there needs to be something that tosses maps out. I would start with tossing out maps in the following order:
  • 1) Unrankable and not updated for an extend period of time
    2) Restricted user's map, had small number of scores, and not updated for an extend period of time
    3) Has small number of scores and not updated for an extend period of time
    4) Restricted user's map, and has small number of scores

There would need be further conditions if this is still not sufficient enough.

-------------------------------------------------------------------------------------


abraker wrote:

Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions

imbor wrote:

I'm assuming this was tied into the previous ideas, though nobody really wants to update their ranked map unless there was something pretty minor, which in qualified,
a QAT can come to help. Most people who rank maps don't want to look at their map again and would most likely rather delete them than remap/etc. (this sounds like an opinion so feel free to ask other people who have ranked maps)
It's to prevent scoreboard wipes from affecting the mapper's decision. Since the system relies on quantities of scores to determine ranking eligibility, wiping scoreboard by updating map when it is close to ranking is totally unwanted. So previous versions of the beatmap must exist to retain a record of scores. Part of this feature is planned, and my suggestion extends the planned to being able to interact with the past versions, not just recover past versions. The planned part is also an existing feature request.

-------------------------------------------------------------------------------------


abraker wrote:

Fix star rating

imbor wrote:

This is a weird one to say, but just kind of putting it out there without method by which you would solve is pretty troublesome. In a sense there is nothing wrong with star rating. There are many thing you can't account for with a computer system because players at different skill levels play maps completely differently. Even within the same skill level players still play maps pretty differently so difficulty will be relative to the player and the map.
Yes, I am very well aware of the issues trying to calculate reading skill. If star rating doesn't work, then replace it with a skill system, or ELO, or something. I don't know what as nothing will work to 100% satisfaction. The issue is: if that is not fixed, then we will see 100k pp scores from CtB and mania. Something at least needs to be done about that prior to the application of this entire concept.

-------------------------------------------------------------------------------------


imbor wrote:

also you should probably rework your FAQ lol
What is wrong with it?

abraker wrote:

Replace BN with an automatic system capable of determining whether a beatmap is suitable for ranking according to the ranking criteria rules

Nitrous wrote:

While the current system is slow and manual with some degree of bias, it's still okay. Using an automated system could cause issues making bias to more generic maps than to unique and creative ones.
I think the current system halts the progress of unique and creative maps. I believe modders would rather avoid weird maps they are not used to and accept ones that they are used to. There are far more debates that lead to headaches and unneeded drama when calling in someone to tell what they believe is good and bad in opposition to the mapper's opinions. Let a passive system of community deciding to play maps decide which ones are good and which ones are bad. Unique and creative maps can be promoted by the mappers or mapper's friends in multiplayer. They can would gain traction if they are fun to play.

-------------------------------------------------------------------------------------


abraker wrote:

Allow beatmap metadata information to be in a separate file, allowing change after ranking

Nitrous wrote:

This would kill older beatmaps that use different audio files for each difficulty, but I definitely agree on this one.
From a technical perspective, it's not hard to make an exception for older formats. The osu! code for reading beatmaps has if statements checking version number, and one notable example is shifting the starting time of the replay to sync with the beatmap because something was different back in the day.

-------------------------------------------------------------------------------------


abraker wrote:

Create a group for people who do metadata checking, and verify DMCA and NSFW status of the map, that is notified of any unchecked maps which become ranked

Nitrous wrote:

Unsure about this, since this is the current job of the Quality Assurance Team and Global Moderation Team.
In any which case, some kind of group would need to do it.

-------------------------------------------------------------------------------------


abraker wrote:

Remove guidelines in ranking criteria and only keep rules

Nitrous wrote:

The guidelines is like the bible for new mappers to know what there should and shouldn't be on a map (which in my case, it did).
There should be guides posted on the mapping subforum for this. Putting it in the ranking criteria causes modders and bn to take it at face value. This is abused by people that want to ruin someone's day or want to take pride in prohibiting an edge case beatmap that attempts to push creativity forward.

-------------------------------------------------------------------------------------


abraker wrote:

QAT's role would remain mostly as is now - Managing ranking criteria, unranking maps that many feel should not have been ranked, and identifying what needs to change in the ranking criteria to prevent more of such maps being ranked

Nitrous wrote:

This is the current job of the Quality Assurance Team. Not pointing anything to this.
With the amount of radical changes I am proposing, it doesn't hurt to assure QAT are still essential

-------------------------------------------------------------------------------------


abraker wrote:

Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them

Nitrous wrote:

Basically adding leaderboards to all pending beatmaps. I did read a post one time and peppy vetoed this idea (correct me if I'm wrong).
It took us years to convince peppy to implement background dimming among other things desired by the community. I don't believe peppy's word is final forever, I believe that it final for the next 5 years or so.

-------------------------------------------------------------------------------------


abraker wrote:

Make individual difficulties automatically go to ranked when it reaches a certain amount of scores

Nitrous wrote:

So it relies on popularity/playcount now? What if the song isn't popular/outdated? I disagree with this.
Maps can be promoted by the mappers or mapper's friends in multiplayer. They can would gain traction if they are fun to play.

-------------------------------------------------------------------------------------


abraker wrote:

Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions

Nitrous wrote:

Definitely. A similar system like Git would be very helpful.
Part of this feature is planned, and my suggestion extends the planned to being able to interact with the past versions, not just recover past versions. The planned part is also an existing feature request.

-------------------------------------------------------------------------------------


abraker wrote:

Fix star rating

Nitrous wrote:

SR is okay as it is, it just relies on formulas which apparently mappers know how to bend (in Standard mode's case).
In mania's case you would see 100k pp scores from top players.


-------------------------------------------------------------------------------------


Nitrous wrote:

I agree the process is slow and hard, but destroying and replacing it completely with a new one isn't definitely the solution.
Yes, maps are produced, but at the expense of a lot of unneeded headaches and drama. Mappers prefer if they can get their maps ranked despite issues and players prefer that they have new quality maps to play. The method used to balance this out right now is just too chaotic.

Sinnoh wrote:

Most guidelines should not be broken for good reason. If this became the case, most guidelines would be moved to the rules. There are almost no maps that break guidelines with good reasoning.

Sinnoh wrote:

If beatmap ranking was automatic, who is going to check for unsnapped objects or objectively unrankable issues? Because of the significant increase in maps, the QAT can't check all of them alone. The BN team is still required to check maps, which means the whole system changed and the outcome never changed, which is pretty redundant.
As Monstrata said,

Monstrata wrote:

anything can be quality if you write enough bs about it, or talk about how you love it, or say it was a clever concept, and get the proper support for it.
Having people judge and tell you what is wrong and is prone subjective, arbitrary, and inconsistent results. The mapper shouldn't forced to undergo that if the mapper does not wish to. If the community likes a shit map enough to get it ranked by playing it and not complain afterwards, then that is good enough to satisfy it being complete and accessible. If somebody catches a wrongly snapped note after it is ranked, well players made enough scores on the map to not notice it. If the mapper is willing to fix it, then good, if not, then so be it. If you can only tell something is wrong by looking in the editor, then you are just nitpicking. Things need to be quality enough for player's enjoyment, not holy perfection. When I mod maps, I always mod via playtests and try my best not to look into the editor unless I am asked for opinions about patterns.

-------------------------------------------------------------------------------------



abraker wrote:

Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them

Sinnoh wrote:

This would increase the number of loved/ranked maps by 2660%, how do you expect the servers to be able to keep up over a long period of time with the huge increase of maps.
There needs to be something that tosses maps out. I would start with tossing out maps in the following order:
  • 1) Unrankable and not updated for an extend period of time
    2) Restricted user's map, had small number of scores, and not updated for an extend period of time
    3) Has small number of scores and not updated for an extend period of time
    4) Restricted user's map, and has small number of scores


There would need be further conditions if this is still not sufficient enough, and I do encourage suggestions.

-------------------------------------------------------------------------------------


abraker wrote:

Make individual difficulties automatically go to ranked when it reaches a certain amount of scores

Sinnoh wrote:

It's just a popularity contest now. Anyone that enjoys mapping high-quality low difficulty sets will have to give up because the only maps that would be popular enough are pp farms and extra difficulties.
Unique and creative maps can be promoted by the mappers or mapper's friends in multiplayer. They can would gain traction if they are fun to play.

-------------------------------------------------------------------------------------


Sinnoh wrote:

How many E/N/H sets are in the loved section?
Loved section, in my perspective, was a solution for the lack of higher difficulty or creative ranked beatmaps. I wouldn't worry about the lack E/N/H maps getting ranked. Lower ranked players will find easier beatmaps and play them. If they complain their favorite song is not ranked, then they will be told, "tell your friends to play the maps and they will get ranked".

-------------------------------------------------------------------------------------


abraker wrote:

Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions

Sinnoh wrote:

This is literally impossible. When updating a map, you make changes. When you move a circle, previous replays will miss because they clicked the old circle.
Previous replays apply to previous version of the map and not the current version. The previous version of the map would be loaded for the replay, not latest. To avoid replays taking up too much space, perhaps make replays for past versions available if it is top 50 and is a good score, where what qualifies as a good score will need to be discussed.

-------------------------------------------------------------------------------------


abraker wrote:

Fix star rating

Sinnoh wrote:

This alone would be enough to fix the system. If it was easy enough as it is to say then it would have been done already.
This is aimed to fix modder vs mapper headaches and drama, not the pp meta. It will still exist. I included "fix star rating" because if any beatmap could be ranked in mania, you would see 100,000 pp scores.
Insp1r3
Hello,

Please fucking stop. You have no idea what really goes into mapping and ranking.

Thanks,
Insp1r3.
Adiopulse
 Replace BN with an automatic system capable of determining whether a beatmap is suitable for ranking according to the ranking criteria rules
-We don’t have that type of technology at our hands, maybe when AI is a thing, maybe. But right now humans are our best option, and you just have to deal with it

 Allow beatmap metadata information to be in a separate file, allowing change after ranking
-Imagine if book/novel authors just kept changing their story after the release, would be hella stupid.

 Create a group for people who do metadata checking, and verify DMCA and NSFW status of the map, that is notified of any unchecked maps which become ranked
-There is already one that does metadata checking, and I don’t think it should be that hard to metadata anyway. Metadata help discord: https://discord.gg/9Y4EdyM

 Remove guidelines in ranking criteria and only keep rules
-I don’t see the problem.

 QAT's role would remain mostly as is now - Managing ranking criteria, unranking maps that many feel should not have been ranked, and identifying what needs to change in the ranking criteria to prevent more of such maps being ranked
-So being QAT

 Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them
-Hell nah, just go to ripple if you care about leaderboards that much. I would like to see somewhat of a better loved system tho, RIP dobby.

 Make individual difficulties automatically go to ranked when it reaches a certain amount of scores
-is this really what you want? https://osu.ppy.sh/s/413538 and https://osu.ppy.sh/s/62395

 Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions
-defeats the purpose of ranking, like the author's example above, there should only be one final product. Fine list old versions but not with scoreboards,

 Fix star rating
-Just put SM algorithm in there and we good. And its not a SR problem, its just the algorithm
DNR
- Frontier -

abraker wrote:

The current ranking model is doomed to fail,
ok_hand
Ephemeral
To sum your points with a development perspective in mind:

  1. Replacing the BN with an automatic system (aka: aimod equivalent) is extremely unlikely to happen. While much of the RC does cover mechanical issues that can be detected at a code level, a lot of it also relies on 'common sense heuristics' which computers are terrible at.
  2. Beatmap metadata contained in a separate file has been in the cards for a very long time now and requires developer attention.
  3. This exists in the Metadata Heap server and is working pretty good as is, though the general metadata standards in osu! need some serious work done to them. I've tried to spearhead something in this regard but I'm simply pulled too thin between over a dozen projects atm to really force the issue.
  4. Guidelines serve a distinct purpose and deleting all of them is probably not very wise.
  5. The QAT react to the reports given to them by users currently, though they are considering a return to actively checking recently ranked maps, as far as I know. They've been pretty good with keeping the RC relevant.
  6. Project Loved is handling Loved maps fine. It isn't really feasible to give every map ever a scoreboard immediately upon upload for a variety of reasons - server load, gameplay paradigm, etc. Doesn't mean this will always be the case, however.
  7. The number of times a map is played has next to nothing to do with its viability for ranked.
  8. Beatmap versioning would be a useful extension to the beatmap discussions system and is something that has been suggested many many times.
  9. This is a bit of a useless thing to say. How? What needs to change to make SR better, and how do we reach that end at a solution level?


It is extraordinarily easy to peg obvious ideas as solutions for big problems. It is far harder to actually create real solutions for those problems.

While ranking could certainly be improved in a number of ways, the premise that it is 'doomed to fail' and 'failing every month' is debatable at best and flagrantly wrong at worse. In my eyes, so long as mappers have ample opportunity to get their maps ranked in a system that is relatively fair and accessible, the system is working as it should.

You will play maps that you do not like. You will find patterns and concepts and themes that challenge you and may go against everything you find enjoyable about osu! - that is fine. There are tens of thousands of beatmaps to play from thousands of unique mappers whom all take different approaches to creating beatmaps - find a handful that you like, and support their work.

The conflict between regulation and deregulation as far as mapping affairs go is as old as osu! is, and there's no real answer to it.

People in favor of regulation decry the apparent "worsening" of osu!'s mapping scene as an argument for more strictly enforcing what essentially boils down to subjective appraisals of quality, and people in favor of deregulation decry the "worsening" of osu!'s mapping scene with increasingly restrictive requirements for a map to be ranked.

In my eyes, the solution to these issues are to deregulate the ranking criteria as much as possible (which is already sort of almost the case) and permit most maps that are mechanically sound to be ranked. The 'quality' of a map is then determined by the community who play it and assign user ratings accordingly.

This would be coupled with a more prestigious curated stream of content (aka charts, spotlights, seasons) which is significantly more restrictive in what is allowed within it. This does however, return us to the ye olde regulation debate, and the inevitable enforcement of 'subjective' standards that are currently complained about now.

What we already currently have is a rough compromise that is born out of everybody's best interests - mappers want their stuff to get ranked, players want to play good maps. The loose cadre of the BN provide a rough sieve to thin out the maps that don't meet muster and the QAT tie up any loose ends that might've slipped through.
Zozimoto

Ephemeral wrote:

mappers want their stuff to get ranked, players want to play good maps
And some people play garbage maps for pp.
Please sign in to reply.

New reply