It's been a while since I posted something controversial seriously, but here we are.

-------------------------------------------------------------------------------------

This proposes a new way to determine beatmap ranking. This does require big changes in osu! infrastructure, and by no means simple nor easy. The point is to discuss this as a possible alternative and what this might result in. TL;DR:

1. Replace BN with an automatic system capable of determining whether a beatmap is suitable for ranking according to the ranking criteria rules
   
   
2. Allow beatmap metadata information to be in a separate file, allowing change after ranking
   
   
3. Create a group for people who do metadata checking, and verify DMCA and NSFW status of the map, that is notified of any unchecked maps which become ranked
   
   
4. Remove guidelines in ranking criteria and only keep rules
   
   
5. QAT's role would remain mostly as is now - Managing ranking criteria, unranking maps that many feel should not have been ranked, and identifying what needs to change in the ranking criteria to prevent more of such maps being ranked
   
   
6. Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them
   
   
7. Make individual difficulties automatically go to ranked when it reaches a certain amount of scores
   
   
8. Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions
   
   
9. Fix star rating

-------------------------------------------------------------------------------------

The current ranking model is doomed to fail, and we observe how it is failing every month or so. Yes, maps are produced, but at the expense of a lot of unneeded headaches and drama. Mappers prefer if they can get their maps ranked despite issues and players prefer that they have new quality maps to play. The method used to balance this out right now is just too chaotic.

The only solution to this is abolish the source of subjective judgment responsible for deciding which map gets to pass and which doesn't. There needs to be a system that is not subjective and is always consistent. The ranking criteria is such system, but it is interpreted by people, which introduces layers of subjectivity. There just needs to be a thing coded up that tells if the map can or cannot be ranked, which would be specified by the ranking criteria. The ranking criteria guidelines would need to go as it is impossible to interpret them without subjectivity. It is impossible to code guidelines without getting differing opinions regarding those person-to-person on various maps. They are not be needed anyway as you will see.

The next source of failure is all the arguing between modders and mappers that happen in the modding sections. Currently you need mods to get the beatmap ranked, which often introduces disagreements between how things should be done. For the map to be of "high quality" it needs to agree with a lot of people, and this often sacrifices mapper's creative intentions in the process. I can't say that there is a way to fix that completely, and I almost sure there is not, but most of it can be redirected into other points of judgments and decisions which can significantly reduce the arguing and allow mapper creativity to stay intact.

To accomplish this, all maps would need scoreboards from the get-go. So every map is basically loved. People who like the map are likely to not delete it and possibly put it up in multi to share it with others. This is how a map can get popular and gain a bigger scoreboard. When the scoreboard for a certain difficulty is sufficiently big enough, it shows people like the difficulty and it becomes almost becomes ranked if the ranking criteria are met. Almost, because the only other thing that should prevent maps from getting ranked are issues on the moddingV2 page. The mapper has the freedom to address mods by agreeing and fixing stuff, or by disagreeing. Closing all the issues would then trigger the ranking of the map.

The only thing the automated system can't cover is the metadata. That needs to be checked and approved manually. So a dedicated group for checking metadata needs to be created. Also, updating metadata right now would essentially be changing the map. It is best to avoid that as much as possible, so there needs to be a separate file for metadata. This file then can be changed to apply to the entire beatmapset without affecting the difficulties themselves. If there has been a change to the metadata online, the client would automatically grab it. An issue is automatically created when a map is triggered for ranking. This serves as a notification to the mapper and the relevant parties. The metadata checking group would receive this as a new issue on their list to go check and provide the correct metadata, make sure the beatmap is not infringing on copyright, and is suitable for all ages.

Since the entire thing is based on scores, wiping scoreboards when the beatmap qualifies for ranking is unfavorable. The issue of wiping a scoreboard should be only relevant if a difficulty is updated. However, if the ranking status is dependent on number scores, mappers will opt to not update the beatmap to not trigger a scoreboard wipe. The component needed to solve this is beatmap version control. It would allow to play past versions of the map and not worry about wiped scoreboards. While only the latest version would be suitable for ranking, ranking status should be determined by the number of unique users on scoreboard for all versions of the difficulty. Otherwise, if you consider just the latest version, it is no different than the wiped scoreboard scenario. To avoid replays taking up too much space, perhaps make replays for past versions available if it is top 50 and is a good score, where what qualifies as a good score will need to be discussed.

Finally, we have the ranking criteria management. Further changes to the ranking criteria are determined when a ranked map is identified to not be suitable for rank by the players. Yes, a ranked map and the concept of this is important. The players need to be sufficiently unhappy about it to warrant a scoreboard wipe. It will no longer be a decision where one can wave a finger and propose this and that without having something at stake. When such event does occur, QATs come in to identify whether the map is truly not suitable and proceed to compose a rule in a manner such that it can be translated in to code. For example, if the criteria mentions "blanket sliders", that would require a technical definition that describes such sliders. The code should be tested against all relevant maps to make sure it is consistent with maps acceptable for ranking. Then there would be a wait period where players would see if the new rule is what is indeed desired. New rule gets merged in, and gets applied such that AIMod and ranking trigger can then use it.

The last bit is to properly fix pp so the map don’t wreak havoc on the leaderboard. This is done by fixing star rating or implementing a new skill worth system that is good enough to remove fears of extremely overrated and underrated maps. This is easier said than done, I don’t know how it will be done, and this is a subject for an entirely different thread.


-------------------------------------------------------------------------------------

FAQ:

• Q: Then maps like Levan Polka 20 second spinner will be ranked. How is that quality?
  A: Then specify it in the ranking criteria. It serves to filter out shit like that. Simple fix would be to add the minimum required note to spinner ratio
  
  Q: Rip CtB and Taiko. There is no chance they would get as popular as other gamemodes
  A: The threshold to which the beatmap becomes ranked will need to be gamemode dependent
  
  Q: What about NSFW maps? How do you put that into code?
  A: You don't. Either metadata checking group stops them or you report inappropriate content
  
  Q: What about plagiarized maps or maps with unapproved skins?
  A: Metadata checking group stops them or you report them
  
  Q: What happens to loved section?
  A: The maps that satisfy ranking criteria are ranked. The rest remain equivalent to loved like everything else.
  
  Q: This is a horrible idea. Maps will be ranked so rarely
  A: I doubt that will happen because popular mappers will always bring about their new maps and people would generally flock to play them. If it does for Taiko or CtB, you got map leaderboards, which are everywhere. You are only missing pp, which is flawed to begin with compared to now. It's not the end of osu!
  
  Q: What if I don't want my map to be ranked?
  A: Just make an issue on moddingV2 page, halting the the process
  
  Q: So the mapper can now just ignore any issues and trigger ranking? Wtf?
  A: Actual issues will not allow the map to by the automated system that checks whether the beatmap passes the ranking criteria. If the map is ranked before metadata is fixed, it can still be fixed while ranked. For other opinionated issues, it's mapper's discretion.
  
  Q: Individual difficulties being ranked will make it a pain to download the beatmap, much less with this version control thing you mentioned.
  A: This is not something that can be done without changing osu! and the website. A fair bit of this to be done without inconveniences would require work on the dev part to get things user friendly to accommodate the change. Also beatmap preview comes in mind as a handy feature.
  
  Q: Likes and favorites should also determine ranking
  A: I don't like those because they are easy to brigade. For both, a friend or anyone can tell anyone else to like and favorite just to promote the beatmap, or even doing so without playing it. If the map has many unique plays, then the community must like it, right? Even if it is a shitty meme map, you can like a shitty meme map.
  
  Q: What happens to kudosu/hype?
  A: Obsolete
  
  Q: Modders mod to gain kudosu to promote their maps. What does the metadata checking group gain by doing the work?
  A: They do just because they want to. Because they want correct metadata on maps.
  
  Q: What is essentially doing is going back to when the ranking criteria was more strict. We recently rewrote it make it less strict
  A: Let's try this again, but less arbitrarily this time. All rules added ensure that many players are involved with the decision. It's no longer a proposal based system, but a complaint based one. A ranked map needs to be identified, and the players will really need to be unhappy with it. Unhappy enough to wipe the scoreboards and make a change in the criteria to prevent it from happening again.
  
  Q: The maps are ranked so they stand out and not among piles of other shit. What you are doing defeats the whole purpose of beatmaps being ranked.
  A: Living with majority of the good maps being unranked is not that bad, guys. Majority of the good mania maps are unranked. We just follow the mappers and share the good maps we find via favorites or multi. No name mappers actually have a better chance in this because they can always go to multi and put up their maps, playing and sharing them with everyone else. If the people who played the map actually like it, then they themselves would put it up next time. They no longer need to argue with someone else to get it ranked. If the map is bad, they would have a hard time getting it shared, period.
  
  Q: Only PP maps will get ranked then
  A: First, the intent is to solve the instability of the system. Fix all the trouble the modders and mappers cause for each other. The solution to this question is mostly a matter of fixing star rating. In terms of the modder and mapper trouble, right now modders can take pp maps and can reject possibly creative ones. In fact, modders would preferably avoid creative ones because they are sometimes so weird and fall into the grey category as far as their subjective opinions go. Instead of that, let's have: Either people like the map and make enough scores to get it ranked or they don't. This doesn't guarantee a decrease in pp maps, but it will offer a chance for more creative ones.
  
  Q: How can you talk about the ranking process when you don't even have a ranked map. You are so ignorant about things and blame bn. This is just an excuse so you can rank your shit map. This is a joke.
  A: A mapper spends time making a map and is proud of it. He/She seeks mods and a few mistakes are corrected. One modder comes around and says a slider is shit and is unrankable. Maybe it's a weird curvy slider that follows the music in a way only the mapper can see, idk. So the mapper argues it is perfectly fine and explains why. Still backlash, and even more when other modders come. The mapper really wants to keep the slider because it expresses his/her creativity. This means something to the mapper that doesn't to others. Telling that your thing is shit and needs to change imo is the worst form of respect, and what the entire modding process encourages. I opt for a system where the mapper can have his/her ways and let agreement with the mapper be reflected by how many scores are set on the map. It's better for the mind to not see the work gain traction rather than get told to change the work you are proud of.
  
  Q: What is the point of this if nothing can be done now? All of this relies on features not implemented
  A: Let’s first make sure we fix any flaws we find within this. During which time, it will probably take 5 years to convince peppy of if we happen to agree.

Me reading this post:

abraker wrote:

[*]Replace BN with an automatic system capable of determining whether a beatmap is suitable for ranking according to the ranking criteria rules

I haven't even read the rest of this thread yet but honestly this is the stupidest fucking things I've read all day.

Like god damn man are you fucking shitting me how the fuck do you suppose a system will determine something like that.

You can input something into a computer and give it parameters that can be sort of like looked at (ie. "if note a is 3.5x from note b, mark unrankable, require fix") or you can give it a bunch of example maps and have it determine what is... most similar to those maps? idk but I really do think it's a stretch to be able to honestly believe that an AI system can determine how suitable a map is for ranking.

Lemme give an example:

https://osu.ppy.sh/s/723944

https://osu.ppy.sh/s/13397

These two maps are both based on distance spacing, and to an AiMod sort of thing, they're okay. How would we determine which is acceptable for ranking (obviously they both are, but still). What would the AI say to me to fix these things? Does it talk about patterns? Does it talk about note density? does it talk about how it relates to the map at all? Like I seriously don't get it lmao.

It's obvious from this thread that you really don't have experience in the ranking process, because you'd be able to see that a lot of these things are just plain silly, through and through.

There ARE problems with the ranking criteria, there ARE problems with things such as favortism, varying ideas of quality, inconsistent and vague rulings, but what you're proposing here doesn't fix any of that, what it does is makes a fucking joke using a literally impossible to implement system.

Like after getting 30 ranked maps and 1300+ kudosu, it really shows you what is reasonable in the ranking process, and how you can't have these fantastical ideas of fully automated systems that will somehow not rank absolutely shitty maps.

Just gonna comment on a few other things,

Q: Modders mod to gain kudosu to promote their maps. What does the metadata checking group gain by doing the work?
A: They do just because they want to. Because they want correct metadata on maps.

In your perfect world they do lmfao. There HAS to be some sort of incentive to mod, that's why kudosu exists in the fucking first place lmfao. This is currently dealt with through sort of having a team that gets a special title and some special privledges, but assuming that people will just do this sort of work out of the goodness of their heart is just plain silly. It's kind of like when I have to cover shifts at work, I think in the goodness of my heart "oh man everyone does this", but you soon realize that people don't do that lmao.

Also I'm not sure how you're getting the idea that in this somehow perfect system that metadata is somehow the ONLY thing that can't be automated.

Q: The maps are ranked so they stand out and not among piles of other shit. What you are doing defeats the whole purpose of beatmaps being ranked.
A: Living with majority of the good maps being unranked is not that bad, guys. Majority of the good mania maps are unranked. We just follow the mappers and share the good maps we find via favorites or multi. No name mappers actually have a better chance in this because they can always go to multi and put up their maps, playing and sharing them with everyone else. If the people who played the map actually like it, then they themselves would put it up next time. They no longer need to argue with someone else to get it ranked. If the map is bad, they would have a hard time getting it shared, period.

Nevermind, this is the stupidest shit I've read all day.

That's not what the ranking process was for lmfao. The ranking process was for ensuring that maps are like, you know, complete, rankable, and accessible. It's literally just a fucking approval process. You submit a map, get feedback, and then it gets ranked, and people play it. I don't know what you think modding is but that's literally what it is. It's like writing a god damn essay. "I liked this, I didn't like this, this slider isn't effective, you have a wrong BPM" like what the fuck do you think people do when they mod lmfao" It's checked to make sure the damn thing is a decent enough quality like aaaaaaaaaa. Also I have no idea what you're going off of with "no name mappers have a better chance" like uhhhhhhhh everyone can do what you fucking just said and big name mappers already have that name-recognizability. Plus it's also based on the music that people like. Like when I played in multi way back when, I didn't think "oh man this map is great" I was like "hey I like this song, I'm gonna download it"

Q: How can you talk about the ranking process when you don't even have a ranked map. You are so ignorant about things and blame bn. This is just an excuse so you can rank your shit map. This is a joke.
A: A mapper spends time making a map and is proud of it. He/She seeks mods and a few mistakes are corrected. One modder comes around and says a slider is shit and is unrankable. Maybe it's a weird curvy slider that follows the music in a way only the mapper can see, idk. So the mapper argues it is perfectly fine and explains why. Still backlash, and even more when other modders come. The mapper really wants to keep the slider because it expresses his/her creativity. This means something to the mapper that doesn't to others. Telling that your thing is shit and needs to change imo is the worst form of respect, and what the entire modding process encourages. I opt for a system where the mapper can have his/her ways and let agreement with the mapper be reflected by how many scores are set on the map. It's better for the mind to not see the work gain traction rather than get told to change the work you are proud of.

IT'S NOT ABOUT WHAT IT MEANS TO YOU IF YOU'RE MAKING CONTENT FOR A GAME, IT'S ABOUT WHAT IT MEANS TO THE COMMUNITY. THAT'S FUCKING FANTASTIC IF YOU THINK SOMETHING WORKS REALLY WELL, BUT IF THE PEOPLE THAT ARE PLAYING YOUR MAP DON'T LIKE IT, THEN YOU SHOULD EITHER CHANGE IT SO YOUR INTENTIONS ARE BETTER TRANSLATED, OR YOU SHOULD REMOVE IT. Like if you have an artistic view of making maps then there's nothing wrong with making maps, graving them, and letting them be with cool sort of experimental stuff, or getting it Loved. But this is a fucking game, and you are producing content for the community to play. The game has standards, guidelines, and isn't really seen as an artistic medium as much as it is filling the game with multiple songs. There is individuality, and you can SEE it as art, but the importance of ranking a beatmap isn't in the artistic side of things, but on the "this is acceptable for the general community to play."

Like god damn I wish I didn't have work because man this is explicitly just wrong.

abraker wrote:

doomed

working just fine for 10 years, if you were more involved in you would understand this simple fact better

thats not how quality assurance works

also you dont fix sr

abraker wrote:

Make individual difficulties automatically go to ranked when it reaches a certain amount of scores

???

abraker wrote:

The current ranking model is doomed to fail, and we observe how it is failing every month or so. Yes, maps are produced, but at the expense of a lot of unneeded headaches and drama.

The actual system might cause dramas and unneeded headaches cause people doesn't know how to talk without acting like animals, but the system itself is good as it is. If you wanna avoid this, don't change the system, but people instead :/

here comes the short version of CDFA's post: Utopian suggestions that are not feasible at the moment or at any time due their unreliability and the lack of support for certain sections, such as Metadata.

Shiguma wrote:

Me reading this post:



Also me reading this post:

hi shiguma

I like some of these ideas like:

abraker wrote:

1. Allow beatmap metadata information to be in a separate file, allowing change after ranking

Other ideas I disagree with + reasons

abraker wrote:

1. Replace BN with an automatic system capable of determining whether a beatmap is suitable for ranking according to the ranking criteria rules This is kind of silly because there are limited resources at to how many maps can store live scores and such. Also this would counteract the purpose of the BNG and QAT as a whole, which should be assuring quality maps are getting ranked, not just maps that are "not unrankable"
   
   
   
2. Remove guidelines in ranking criteria and only keep rules
   The purpose of guidelines have been to help new mappers start/improve (though wording in the document is silly). Lower difficulty maps are meant to teach new players how to play the game, and without the lower difficulty guidelines it would be difficult to determine what exactly lower players are capable of without having it memorized and whatnot.
   
3. QAT's role would remain mostly as is now - Managing ranking criteria, unranking maps that many feel should not have been ranked, and identifying what needs to change in the ranking criteria to prevent more of such maps being ranked this is unnecessary to post? or did i misread it lol
   
   
4. Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them this goes to the same point I gave above on automated BN
   
   
5. Make individual difficulties automatically go to ranked when it reaches a certain amount of scores same as above
   
   
6. Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions I'm assuming this was tied into the previous ideas, though nobody really wants to update their ranked map unless there was something pretty minor, which in qualified,
   a QAT can come to help. Most people who rank maps don't want to look at their map again and would most likely rather delete them than remap/etc. (this sounds like an opinion so feel free to ask other people who have ranked maps)
   
   
7. Fix star rating
   This is a weird one to say, but just kind of putting it out there without method by which you would solve is pretty troublesome. In a sense there is nothing wrong with star rating. There are many thing you can't account for with a computer system because players at different skill levels play maps completely differently. Even within the same skill level players still play maps pretty differently so difficulty will be relative to the player and the map.

also you should probably rework your FAQ lol

I didn't bother reading the whole thing so I went ahead and read the tl;dr section.

1. Replace BN with an automatic system capable of determining whether a beatmap is suitable for ranking according to the ranking criteria rules
   While the current system is slow and manual with some degree of bias, it's still okay. Using an automated system could cause issues making bias to more generic maps than to unique and creative ones.
   
2. Allow beatmap metadata information to be in a separate file, allowing change after ranking
   This would kill older beatmaps that use different audio files for each difficulty, but I definitely agree on this one.
   
3. Create a group for people who do metadata checking, and verify DMCA and NSFW status of the map, that is notified of any unchecked maps which become ranked
   Unsure about this, since this is the current job of the Quality Assurance Team and Global Moderation Team.
   
4. Remove guidelines in ranking criteria and only keep rules
   The guidelines is like the bible for new mappers to know what there should and shouldn't be on a map (which in my case, it did).
   
5. QAT's role would remain mostly as is now - Managing ranking criteria, unranking maps that many feel should not have been ranked, and identifying what needs to change in the ranking criteria to prevent more of such maps being ranked
   This is the current job of the Quality Assurance Team. Not pointing anything to this.
   
6. Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them
   Basically adding leaderboards to all pending beatmaps. I did read a post one time and peppy vetoed this idea (correct me if I'm wrong).
   Further reasoning in the next point below.
   
7. Make individual difficulties automatically go to ranked when it reaches a certain amount of scores
   So it relies on popularity/playcount now? What if the song isn't popular/outdated? I disagree with this.
   
8. Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions
   Definitely. A similar system like Git would be very helpful.
   
9. Fix star rating
   SR is okay as it is, it just relies on formulas which apparently mappers know how to bend (in Standard mode's case).

I agree the process is slow and hard, but destroying and replacing it completely with a new one isn't definitely the solution.

SR is not ok. it's fundamentally flawed in what it values and doesn't value and as such has had an extremely noticeable and detrimental effect on mapping *diversity* as a whole.

The current system already is ridiculously biased towards generic maps, and an automated one would not be biased at all because there would be various objective criteria and that's it. No subjective things like "oh I think you should remove this long slider because the vocal line you're following with it is bad and do it my way because I'm better than you" (paraphrased ofc).

Replacing BN's isn't as bad of an idea as people are making it out to be.

It's far too late to enforce mapping quality at this point. Maps that are subjectively low quality can be ranked as long as proper justification is provided, meaning that there is absolutely no objective definition of "high quality" to begin with. The purpose of BN's was to push quality maps forward, but anything can be quality if you write enough bs about it, or talk about how you love it, or say it was a clever concept, and get the proper support for it.

Note also, that the majority of "quality" maps pushed forward by BN's are seen as generic and average to the player base: anime jump maps, standard ctrl+h/j and well=structured maps that just "flow well". People are looking for "creative" and "different" but the meta streamlines mapping styles that are clean, flow well, and are generally unoffensive to mappers and modders. If you want creative and different maps in the ranked section you need to be prepared to promote subjectively lower quality maps.

The questions people should really be asking is: Why should I care about mapping quality?

1. Why does a low quality map offend you?
2. If a low quality map is getting pushed forward, why do you bother to try and stop it? Especially if it's by a popular mapper or BN, the map will very likely get ranked eventually anyways. You are only wasting your time as well as the mapper's.
3. Does a single map create a precedent for low quality maps? The entire argument about Alien stemmed from "if this gets ranked, we might as well rank every beginner's map". Did that end up happening?
4. Why do you feel so entitled that you believe someone's hard work should not be recognized? By preventing a map from reaching the ranked status, you are effectively saying "Your map doesn't deserve any recognition until you change it to something that doesn't offend me".

The truth is, low quality maps (aside from CBCC, Haitai, etc...) will disappear and be forgotten. Frankly, the only people who remember subjectively low quality maps are people active in the mapping scene. And additionally, what mappers find low quality clearly doesn't align with what players find as low quality to begin with.

Talking to peppy made me realize just how different the playerbase and the mapperbase can be at times. Top players, and vocal members of the community in reality, represent the vocal minority of the community. Why did so many people vote for Highscore, or Inferno, or Hitorigoto, even though they clearly (in the perspective of a mapper) aren't creative/high quality maps? Because the average player is simply looking to have fun, and play songs that they enjoy. That is their definition of high quality: not something that is beautifully mapper and engineered, but rather, a map that simply expresses the song well, and is fun to play.

Quality is subjective, and people care too much about quality. Every day, the prestige of getting your map ranked is lowered. It's already a truth among most mappers that your map will likely be forgotten by what 95% of the playerbase within a week after the map gets ranked. It's just the truth.

Replacing BNs with a system is a bad idea. It throws subjectivity away, which is an important factor to ranking beatmaps.

I think subjectivity is important because if the only requirements to rank a map was to: “>follow ranking criteria<“ then we’d see hundreds of low quality maps. Is “quality” subjective? Yes, but not as subjective as Monstrata or other mappers make it out to be. it is generally agreed on that a map should have good flow, aesthetics, spacing, etc... Although of course whatever is considered good flow or aesthetics is subjective, there are common and popular opinions on what makes a map good:

-Flow, how easily and comfortable you snap from one object to another.
-Aesthetics, no messy overlaps, blanketing sliders, of course these rules can be bent as long as a map is consistent. If a maps idea is to use weird abstract patterns, as long as it has some sort of solid justification and a presence of skill is apparent, it should still be considered good.
-Spacing, in intense or kiai parts of a song the spacing should be increased, in less intense parts, spacing should be lowered. Spacing should be consistent through your whole map, and if raised in high intensity parts, raised again with the same degree of the high intensity part of a song repeats.



Those three points I mentioned above cannot be checked by a system, that’s why having a real human being and community look at it is much better than a computer.


Also, drama happens in a lot less maps then you think. There are many maps being ranked each (I want to say day but I’m not sure) without any drama at all. The more popular you are the more prone you are to drama. Encouraging good manners and behaviour is a far better solution to drama than replacing human interaction all together.

CDFA wrote:

You can input something into a computer and give it parameters that can be sort of like looked at (ie. "if note a is 3.5x from note b, mark unrankable, require fix") or you can give it a bunch of example maps and have it determine what is... most similar to those maps? idk but I really do think it's a stretch to be able to honestly believe that an AI system can determine how suitable a map is for ranking.

Lemme give an example:

https://osu.ppy.sh/s/723944

https://osu.ppy.sh/s/13397

These two maps are both based on distance spacing, and to an AiMod sort of thing, they're okay. How would we determine which is acceptable for ranking (obviously they both are, but still). What would the AI say to me to fix these things? Does it talk about patterns? Does it talk about note density? does it talk about how it relates to the map at all? Like I seriously don't get it lmao.

It is not the job of AImod to say how to fix things. Its job is to say what and why something can't be ranked. If you want to know how to fix things, you can ask someone with mapping experience. In this system you personally have to be willing to recognize faults within the map and have the drive to improve them in order to make ones that players will like to play.

CDFA wrote:

It's obvious from this thread that you really don't have experience in the ranking process, because you'd be able to see that a lot of these things are just plain silly, through and through.

There ARE problems with the ranking criteria, there ARE problems with things such as favortism, varying ideas of quality, inconsistent and vague rulings, but what you're proposing here doesn't fix any of that, what it does is makes a fucking joke using a literally impossible to implement system.

Like after getting 30 ranked maps and 1300+ kudosu, it really shows you what is reasonable in the ranking process, and how you can't have these fantastical ideas of fully automated systems that will somehow not rank absolutely shitty maps.

I might not have experience participating with the ranking process, but I for sure have seen enough of it to know stuff isn't right. Yes, this system will not fix favoritism globally. Popular mappers will still have maps with the most plays on scoreboards, but it does offer a chance to lesser known mappers willing to experiment with different mapping styles. Instead of prohibiting the map from getting ranked due to varying ideas of quality, the map might not get popular because most don't see the map as good. The possibility of it being ranked in the current state is what the mapper needs to think, not the inability of the map to get ranked in the current state. Nobody should tell you the thing is shit and won't be nominated. It is up to the mapper to whether ask how the map can be improved to be more enjoyable and it is up to the mapper to decide whether to apply the suggested points to make the map more enjoyable. Or perhaps the mapper has a small fanbase that is not big enough to get the number of scores to get ranked, yet the mapper sees on the scoreboard that people are playing the maps and is satisfied with that. It offers more positivism, outlook, and flexibility.

CDFA wrote:

abraker wrote:

Q: Modders mod to gain kudosu to promote their maps. What does the metadata checking group gain by doing the work?
A: They do just because they want to. Because they want correct metadata on maps.

In your perfect world they do lmfao. There HAS to be some sort of incentive to mod, that's why kudosu exists in the fucking first place lmfao. This is currently dealt with through sort of having a team that gets a special title and some special privledges, but assuming that people will just do this sort of work out of the goodness of their heart is just plain silly. It's kind of like when I have to cover

If you don't think preventing bad metadata is a good enough incentive, then, there can be something offered in exchange. I can't imagine what at the moment, but perhaps someone else can. I did not expect the entire suggestion to be bullet proof, and I expect people to provide solution/alternative suggestions.

CDFA wrote:

Also I'm not sure how you're getting the idea that in this somehow perfect system that metadata is somehow the ONLY thing that can't be automated

Care to expand on this point?

CDFA wrote:

abraker wrote:

Q: The maps are ranked so they stand out and not among piles of other shit. What you are doing defeats the whole purpose of beatmaps being ranked.
A: Living with majority of the good maps being unranked is not that bad, guys. Majority of the good mania maps are unranked. We just follow the mappers and share the good maps we find via favorites or multi. No name mappers actually have a better chance in this because they can always go to multi and put up their maps, playing and sharing them with everyone else. If the people who played the map actually like it, then they themselves would put it up next time. They no longer need to argue with someone else to get it ranked. If the map is bad, they would have a hard time getting it shared, period.

Nevermind, this is the stupidest shit I've read all day.

That's not what the ranking process was for lmfao. The ranking process was for ensuring that maps are like, you know, complete, rankable, and accessible. It's literally just a fucking approval process. You submit a map, get feedback, and then it gets ranked, and people play it. I don't know what you think modding is but that's literally what it is. It's like writing a god damn essay. "I liked this, I didn't like this, this slider isn't effective, you have a wrong BPM" like what the fuck do you think people do when they mod lmfao" It's checked to make sure the damn thing is a decent enough quality like aaaaaaaaaa. Also I have no idea what you're going off of with "no name mappers have a better chance" like uhhhhhhhh everyone can do what you fucking just said and big name mappers already have that name-recognizability. Plus it's also based on the music that people like. Like when I played in multi way back when, I didn't think "oh man this map is great" I was like "hey I like this song, I'm gonna download it"

As Monstrata said in a later post,

Monstrata wrote:

anything can be quality if you write enough bs about it, or talk about how you love it, or say it was a clever concept, and get the proper support for it.

Having people judge and tell you what is wrong and is prone subjective, arbitrary, and inconsistent results. The mapper shouldn't forced to undergo that if the mapper does not wish to. If the community likes a shit map enough to get it ranked by playing it and not complain afterwards, then that is good enough to satisfy it being complete and accessible. If somebody catches a wrongly snapped note after it is ranked, well players made enough scores on the map to not notice it. If the mapper is willing to fix it, then good, if not, then so be it. If you can only tell something is wrong by looking in the editor, then you are just nitpicking. Things need to be quality enough for player's enjoyment, not holy perfection. When I mod maps, I always mod via playtests and try my best not to look into the editor unless I am asked for opinions about patterns.

CDFA wrote:

IT'S NOT ABOUT WHAT IT MEANS TO YOU IF YOU'RE MAKING CONTENT FOR A GAME, IT'S ABOUT WHAT IT MEANS TO THE COMMUNITY. THAT'S FUCKING FANTASTIC IF YOU THINK SOMETHING WORKS REALLY WELL, BUT IF THE PEOPLE THAT ARE PLAYING YOUR MAP DON'T LIKE IT, THEN YOU SHOULD EITHER CHANGE IT SO YOUR INTENTIONS ARE BETTER TRANSLATED, OR YOU SHOULD REMOVE IT. Like if you have an artistic view of making maps then there's nothing wrong with making maps, graving them, and letting them be with cool sort of experimental stuff, or getting it Loved. But this is a fucking game, and you are producing content for the community to play. The game has standards, guidelines, and isn't really seen as an artistic medium as much as it is filling the game with multiple songs. There is individuality, and you can SEE it as art, but the importance of ranking a beatmap isn't in the artistic side of things, but on the "this is acceptable for the general community to play."

If people that are playing your map don't like it, then it will not be shared. If it is not shared, then it won't get enough scores. If it doesn't get enough scores, then it won't get ranked. The standards are in the ranking criteria which are enforced by an automatic system that determines whether the beatmap follows the criteria. Ranking criteria can be changed when sufficient amount of players complain about a map that the players believe shouldn't have passed the ranking criteria.

I believe beatmaps are artworks and should be seen as such. Games themselves are pieces of art. Anything that allows an individual's creativity to flow free is art. Another individual saying to change art because a piece of it is shit is wrong. A natural process shall dictate what art is good, and that is determined by sufficient positive community interaction with it.

Hi one feasibility check

abraker wrote:

This proposes a new way to determine beatmap ranking. This does require big changes in osu! infrastructure, and by no means simple nor easy. The point is to discuss this as a possible alternative and what this might result in. TL;DR:

1. Remove guidelines in ranking criteria and only keep rules Most guidelines should not be broken for good reason. If this became the case, most guidelines would be moved to the rules. There are almost no maps that break guidelines with good reasoning.
   
   
2. Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them This would increase the number of loved/ranked maps by 2660%, how do you expect the servers to be able to keep up over a long period of time with the huge increase of maps.
   
   
3. Make individual difficulties automatically go to ranked when it reaches a certain amount of scores It's just a popularity contest now. Anyone that enjoys mapping high-quality low difficulty sets will have to give up because the only maps that would be popular enough are pp farms and extra difficulties. How many E/N/H sets are in the loved section?
   
   
4. Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions This is literally impossible. When updating a map, you make changes. When you move a circle, previous replays will miss because they clicked the old circle.
   
   
5. Fix star rating This alone would be enough to fix the system. If it was easy enough as it is to say then it would have been done already.

If beatmap ranking was automatic, who is going to check for unsnapped objects or objectively unrankable issues? Because of the significant increase in maps, the QAT can't check all of them alone. The BN team is still required to check maps, which means the whole system changed and the outcome never changed, which is pretty redundant.

Even if this was considered to be a good idea, it's not possible to create regardless of whether or not people like it. End of discussion.

Hello,

Please fucking stop. You have no idea what really goes into mapping and ranking.

Thanks,
Insp1r3.

 Replace BN with an automatic system capable of determining whether a beatmap is suitable for ranking according to the ranking criteria rules
-We don’t have that type of technology at our hands, maybe when AI is a thing, maybe. But right now humans are our best option, and you just have to deal with it

 Allow beatmap metadata information to be in a separate file, allowing change after ranking
-Imagine if book/novel authors just kept changing their story after the release, would be hella stupid.

 Create a group for people who do metadata checking, and verify DMCA and NSFW status of the map, that is notified of any unchecked maps which become ranked
-There is already one that does metadata checking, and I don’t think it should be that hard to metadata anyway. Metadata help discord: https://discord.gg/9Y4EdyM

 Remove guidelines in ranking criteria and only keep rules
-I don’t see the problem.

 QAT's role would remain mostly as is now - Managing ranking criteria, unranking maps that many feel should not have been ranked, and identifying what needs to change in the ranking criteria to prevent more of such maps being ranked
-So being QAT

 Make every beatmap loved - so beatmap popularity can be accurately reflected via how many scores are on them
-Hell nah, just go to ripple if you care about leaderboards that much. I would like to see somewhat of a better loved system tho, RIP dobby.

 Make individual difficulties automatically go to ranked when it reaches a certain amount of scores
-is this really what you want? https://osu.ppy.sh/s/413538 and https://osu.ppy.sh/s/62395

 Beatmap versioning system - so beatmaps can be updated without scoreboard wipes, allowing players to play older versions
-defeats the purpose of ranking, like the author's example above, there should only be one final product. Fine list old versions but not with scoreboards,

 Fix star rating
-Just put SM algorithm in there and we good. And its not a SR problem, its just the algorithm

abraker wrote:

The current ranking model is doomed to fail,

ok_hand

To sum your points with a development perspective in mind:

1. Replacing the BN with an automatic system (aka: aimod equivalent) is extremely unlikely to happen. While much of the RC does cover mechanical issues that can be detected at a code level, a lot of it also relies on 'common sense heuristics' which computers are terrible at.
2. Beatmap metadata contained in a separate file has been in the cards for a very long time now and requires developer attention.
3. This exists in the Metadata Heap server and is working pretty good as is, though the general metadata standards in osu! need some serious work done to them. I've tried to spearhead something in this regard but I'm simply pulled too thin between over a dozen projects atm to really force the issue.
4. Guidelines serve a distinct purpose and deleting all of them is probably not very wise.
5. The QAT react to the reports given to them by users currently, though they are considering a return to actively checking recently ranked maps, as far as I know. They've been pretty good with keeping the RC relevant.
6. Project Loved is handling Loved maps fine. It isn't really feasible to give every map ever a scoreboard immediately upon upload for a variety of reasons - server load, gameplay paradigm, etc. Doesn't mean this will always be the case, however.
7. The number of times a map is played has next to nothing to do with its viability for ranked.
8. Beatmap versioning would be a useful extension to the beatmap discussions system and is something that has been suggested many many times.
9. This is a bit of a useless thing to say. How? What needs to change to make SR better, and how do we reach that end at a solution level?

It is extraordinarily easy to peg obvious ideas as solutions for big problems. It is far harder to actually create real solutions for those problems.

While ranking could certainly be improved in a number of ways, the premise that it is 'doomed to fail' and 'failing every month' is debatable at best and flagrantly wrong at worse. In my eyes, so long as mappers have ample opportunity to get their maps ranked in a system that is relatively fair and accessible, the system is working as it should.

You will play maps that you do not like. You will find patterns and concepts and themes that challenge you and may go against everything you find enjoyable about osu! - that is fine. There are tens of thousands of beatmaps to play from thousands of unique mappers whom all take different approaches to creating beatmaps - find a handful that you like, and support their work.

The conflict between regulation and deregulation as far as mapping affairs go is as old as osu! is, and there's no real answer to it.

People in favor of regulation decry the apparent "worsening" of osu!'s mapping scene as an argument for more strictly enforcing what essentially boils down to subjective appraisals of quality, and people in favor of deregulation decry the "worsening" of osu!'s mapping scene with increasingly restrictive requirements for a map to be ranked.

In my eyes, the solution to these issues are to deregulate the ranking criteria as much as possible (which is already sort of almost the case) and permit most maps that are mechanically sound to be ranked. The 'quality' of a map is then determined by the community who play it and assign user ratings accordingly.

This would be coupled with a more prestigious curated stream of content (aka charts, spotlights, seasons) which is significantly more restrictive in what is allowed within it. This does however, return us to the ye olde regulation debate, and the inevitable enforcement of 'subjective' standards that are currently complained about now.

What we already currently have is a rough compromise that is born out of everybody's best interests - mappers want their stuff to get ranked, players want to play good maps. The loose cadre of the BN provide a rough sieve to thin out the maps that don't meet muster and the QAT tie up any loose ends that might've slipped through.

Ephemeral wrote:

mappers want their stuff to get ranked, players want to play good maps

And some people play garbage maps for pp.