Guest
home
beatmaps
rankings
community
store
help
forum
  1. Forums
  2. Beatmaps
  3. Mapping Discussion
  4. Ranking Criteria
  5. Finalized/Denied Amendments

[invalid] [Discussion/Proposal] Ranking process reforms

posted
Total Posts
22
niat0004
226 posts
Joined September 2017
Topic Starter
niat0004
Hello. There have been some ideas on how to reform and improve the ranking process to make it more effective and accessible, especially for new mappers. In this thread, I will present some of my ideas to reform the process.
I'm not sure whether this goes here, but I think it does if this does.

Notice: This was mostly written before Qualified Inspectors existed.

Rankable beatmap queue
A lot of entirely rankable or near-rankable maps are ignored simply because the mapper could not or did not want to find 2 (or more) BNs. Therefore, it could be an idea to make maps with a large amount of Hype (around 25) be automatically submitted to BNs for review.
This could lead to forcing seriously unrankable maps for BN review, so some extra steps would have to be involved.

Queue reviewers and the QI
A Qualified Inspector could choose to check that the metadata is correct or close to correct, the map has no glaring unrankable flaws, the spread is reasonable, etc. In essence, they would check whether the map is worth a BN's time to mod. This should grant some fraction of an activity point (around 1/2).

Kudosu stamps
As a supplement or alternative to the former, a system of Kudosu stamps could be implemented, where regular users could give a map a Kudosu stamp at the cost of some available Kudosu (around 10). Once a map has a set number of these stamps (around 3), it could be inserted into the queue, or it could be eligible for a queue reviewer.

Beatmap Revivers and Resurrection Stamps
A Beatmap Reviver group can be created, with its members required to have some (around 60) Kudosu or a Ranked map to apply, and with the ability to give rankable Pending/WIP maps with no BN activity "Resurrection Stamps"; after a few (around 3) stamps, the map is given a Stamp Nomination, whereafter it only needs 1 nomination to be ranked. This encourages BNs to look at the maps without bypassing the current ranking system. Low-quality stamps can be reported, and if a BN or NAT agrees that it is a bad map not deserving of a stamp, the Reviver is locked out for 2 weeks or (for repeat failures) removed and must reapply.

How does the queue work exactly?
Once a map has passed a reviewer and/or received enough Kudosu stamps, the mapper will receive a notification. The mapper can update it as Pending to enter it into the queue.
Then, the map will be put into a queue where the BNs can pick from freely.
Should a map not be taken within 28 days, the NAT will assign the map to 2 BNs semi-randomly. If they do not do at least a mod of the map (as judged by the NAT), their status as a BN will be questioned as a result. Not doing that. No one likes it.
It may also be used as a queue of maps for the Trial BNs to check through and make their task of finding maps to mod easier.

If you have any ideas to improve or streamline this, or if you think the numbers are too high or low, or if you agree, please post.
Last edited by niat0004 , edited 4 times in total.
Nao Tomori
osu! Alumni
3,107 posts
Joined December 2014
Nao Tomori
Three problems:

1. Hype has 0 correlation to quality. This will just make people run around begging even more people for hypes in the misguided attempt to get their map bubbled.

2. Having a pre-BN queue is pointless because, generally speaking, people who are good enough to judge if a map is rankable or what the main issues are are good enough to get BN. So most non-BN input would not be very meaningful when evaluating if maps are good enough.

3. Forcing BNs to mod maps creates burnout and would lower activity in the long run as people procrastinate modding things that they want to because they have some giant queue of shitmaps that they have to slog through first.

I agree that rankable maps have too hard of a time getting ranked sometimes but I don't think this is a good solution.
Noch Einen
530 posts
Joined July 2018
Noch Einen
Yes for kudosu into map (just for dumping place). But how do you guys measure it? Since there are "veteran vs newcomer" (assuming the amount is vastly different) + there may/not be a thing where "force rank" (injecting free kudosu to map just for the rank sake, refer to Loved system where Loctav was actively present)
Last edited by Noch Einen , edited 1 time in total.
Ryu Sei
2,900 posts
Joined October 2013
Ryu Sei
This topic is fine, since Ranking Criteria subforum is for proposing new/modifications of RC and the process of it.

There are some opinions regarding on your proposal, entirely from my perspective:
  1. A rankable beatmap queue is easily abuse-able, so that will increase the chance of people asking for hype only to have it BN-checked. I mean, there are maps with astounding amount of 100+ hypes.
  2. Regarding QI, I don't think it's a good idea to put QIs to burden on pending maps. QIs by nature only check objective parts of the maps that is in Qualified.
  3. Kudosu stamps is basically the glorification of star priority.
  4. This is interesting. However, the idea of choosing BNs semi-randomly is kinda questionable. You also need to factor that BNs can choose to not take the maps due to personal reasons (IRL stuffs), so that kinda risks the said questioned BNs if they can't take it for legitimate reasons.
Will this proposal makes ranking easier? Maybe. Will this proposal burdens BNs further? Probably. As a mapper, I always welcome any ways to make ranking process easier, since I remember during my early ranking days the BNs/BATs would take a look at my maps even without me lifting a finger to ask for nominations/bubbles.

It's a hard truth that most BNs would reject mapper's map due to song preference, but if the patterns are rankable and there are no obvious issues, that shouldn't hinder them from ranking a map. If that is the case, can't we go back and re-read the expectations of being a BN, where they should promote variety of maps?
Shmiklak
Elite Storyboarder
3,258 posts
Joined January 2015
Shmiklak
Forcing people to check maps they aren’t going to be interested in isn’t going to be helpful and would rather destroy everything
-White
324 posts
Joined February 2020
-White
Any suggestion that implies hype should be anything other than a meaningless number is bad, imo
Drum-Hitnormal
Beatmap Nominator
1,727 posts
Joined March 2011
Drum-Hitnormal
speaking from mania

BN check 2 maps per month (there are probably 200+ created for ranked per month) , some retire from burnout

anything that reduce BN motivation (forcing to check songs/pattern they dont like) is not going to help

the way to increase ranked is to educate mapper so BN waste less time and increase number of BN from good mapper/modder

theres no way to rank all maps thats rankeable without AI BN

new mapper need to work harder, dont kill BNs with newbie maps
Last edited by Drum-Hitnormal , edited 1 time in total.
Vararaup
237 posts
Joined April 2016
Vararaup
Hypes are a waste of time. Kudosu is a bigger waste of time if you don't mod.

Another Q is useless when people can just request bns directly. Adding extra steps is the opposite of accessible.
RandomeLoL
Nomination Assessment Team
224 posts
Joined September 2015
RandomeLoL
I have several concerns making me back down from this idea:

1 - QIs aren't supposed to be BNs. The idea would basically force them to do a beatmap curation task that it's currently given to BNs. The idea was to have them assure the quality of beatmaps that BNs get to nominate, not do it before the fact.

2 - Random BN assignation. There are a few things wrong assignating beatmaps randomly, even if it's done through a preference system. Ideally, you are hoping for a BN to get a set they like, with a style they enjoy, within their capabilities to check, and do it in a timeframe they are comfortable with. It's already very hard to convince people to join the BNG. It is purely vocational. This would make the experience more of a chore than a voluntary task.

3 - Kudosu stamps/stars favour already established mappers. Not new ones. The same way hypes aren't really telling of a set's quality, it is safe to assume people will not use them in a way that would promote new mappers, but rather choose to continue ranking sets from experienced and/or popular mappers.

4 - Consent. This new system would make it possible for an inactive mapper to get a set of theirs Ranked. They would not be able to give consent if they do not want their works featured. It's not nice to expect that everyone wants their works featured, even if the community pushes for it. Loved already dealt with cases where a mapper's set was featured without consent, and replicating it in a section where they cannot get their set un-Ranked after the fact does not seem like the correct thing to promote.

Despite my negativity for the points above, I do also believe that some tweaks in the system can be beneficial! To me though, this seems like a bad step in the good direction.

I do have to say that I am a bit biased. Quaver has a system like this put into place as their only way to feature sets. If this were to be implemented or analyzed, that'd be a case study to investigate first to see how viable it is.

Edit: I could see an idea like this work well if the queue was only meant to feature osu! licensed tracks and to give priority to those in the Ranking queue. But even then, this would have to be done out of volunteering BNs. Something along those lines to incentivize pushing original content may also push newer mappers to work with our song repertoire!
Last edited by RandomeLoL , edited 1 time in total.
ikin5050
587 posts
Joined February 2014
ikin5050
I think you should keep in mind this is a volunteer program essentially. Sure it may be disheartening to see that your map doesn't get ranked. Respectfully, make another map, get better and move on.

Your idea also fails to consider that whilst some maps are rankable they are dogshit quality and forcing BNs to mod maps that they don't want to take whilst also having their refusal reflect poorly for evals is a terrible idea.
Hydria
1,247 posts
Joined April 2011
Hydria
Should a map not be taken within 28 days, the NAT will assign the map to 2 BNs semi-randomly. If they do not do at least a mod of the map (as judged by the NAT), their status as a BN will be questioned as a result.
Yeah I'd rather retire. Good luck though.
Maxim-Miau
Beatmap Nominator
106 posts
Joined July 2018
Maxim-Miau
I dont think randomly assigning BNs to maps is a good idea. Chances are big its a map/song the BN doesnt like at all and then the BN would most likely resign from BN due to burnout
Shad0wStar
Global Moderator
177 posts
Joined April 2020
Shad0wStar
This is interesting, and it does in theory reward mappers with great mapping to get their maps ranked easier, however:

1. Using hypes and kudosu to push a map into the queue is not good. There would have to be a change to both systems in order to allow this to work. I hype my friends maps without even downloading the map, and you could possibly spam the same mod such as “I think you can add these tags...” and farm kudosu on numerous sets. The amount of begging for kudosu stamps and hypes would also go up way more, bc instead of going through modding queues and ensuring your map has quality before sending to BNs, *most* people will just end up asking a lot of people for hypes.
If I can think of any solution, perhaps a new group that deals with pending maps without the mappers involvement who basically nominate the maps they think are ready for ranking. No modding is necessary at all but the people in this group should have a very good grasp of what is rankable. After X amount of noms are received, then the notification you mentioned will be sent to the mapper. This could be an alternate way to push more rankable sets to ranked without the mapper sending the maps to BN. Though I do think that most good mappers will always get their stuff ranked if they desire it and work hard enough.

2. Forcing BNs will cause people to quit. I’m sure it’s been mentioned in other people’s comments, but when I started off as BN I was ready to take everything rankable. That motivation persisted for about two months until I realized how much more fun checking a map that I personally enjoyed. I feel if this system ends up being put in place, it would only send the map to BNs who share the preferences of the map (similar to how global queue worked), and the chance that your map won’t be taken by BNs should be clear. In the end BN is a volunteering thing and thats why we have so many BN so u have multiple options and many different preferences.
Okoayu
Nomination Assessment Team / Elite Nominator
4,959 posts
Joined May 2012
Okoayu
ppl should remain in charge of what they want to push, taking that agency away from BNs is just gonna be a dumpster fire
niat0004
226 posts
Joined September 2017
Topic Starter
niat0004
Alright, I removed the bit about the forced queue and required active consent.

Which other parts need fixing?
Last edited by niat0004 , edited 1 time in total.
lewski
1,346 posts
Joined September 2014
lewski

niat0004 wrote:

Which other parts need fixing?
some points from the thread that you seem to have missed:

Nao Tomori

Nao Tomori wrote:

1. Hype has 0 correlation to quality. This will just make people run around begging even more people for hypes in the misguided attempt to get their map bubbled.

2. Having a pre-BN queue is pointless because, generally speaking, people who are good enough to judge if a map is rankable or what the main issues are are good enough to get BN. So most non-BN input would not be very meaningful when evaluating if maps are good enough.

Ryu Sei

Ryu Sei wrote:

  1. A rankable beatmap queue is easily abuse-able, so that will increase the chance of people asking for hype only to have it BN-checked. I mean, there are maps with astounding amount of 100+ hypes.
  2. Regarding QI, I don't think it's a good idea to put QIs to burden on pending maps. QIs by nature only check objective parts of the maps that is in Qualified.
  3. Kudosu stamps is basically the glorification of star priority..

RandomeLoL

RandomeLoL wrote:

1 - QIs aren't supposed to be BNs. The idea would basically force them to do a beatmap curation task that it's currently given to BNs. The idea was to have them assure the quality of beatmaps that BNs get to nominate, not do it before the fact.

3 - Kudosu stamps/stars favour already established mappers. Not new ones. The same way hypes aren't really telling of a set's quality, it is safe to assume people will not use them in a way that would promote new mappers, but rather choose to continue ranking sets from experienced and/or popular mappers.

Shad0wStar

Shad0wStar wrote:

1. Using hypes and kudosu to push a map into the queue is not good. There would have to be a change to both systems in order to allow this to work. I hype my friends maps without even downloading the map, and you could possibly spam the same mod such as “I think you can add these tags...” and farm kudosu on numerous sets. The amount of begging for kudosu stamps and hypes would also go up way more, bc instead of going through modding queues and ensuring your map has quality before sending to BNs, *most* people will just end up asking a lot of people for hypes.
If I can think of any solution, perhaps a new group that deals with pending maps without the mappers involvement who basically nominate the maps they think are ready for ranking. No modding is necessary at all but the people in this group should have a very good grasp of what is rankable. After X amount of noms are received, then the notification you mentioned will be sent to the mapper. This could be an alternate way to push more rankable sets to ranked without the mapper sending the maps to BN. Though I do think that most good mappers will always get their stuff ranked if they desire it and work hard enough.


Personally, I think the whole idea just sounds like a rehash of the global queue that used to exist on the BN site but was removed because hardly any BNs actually used it. Therefore, since the only meaningful difference between that and your idea is the ultimately meaningless hype requirement, I doubt this queue would fare much better.


response to RandomeLoL's point #4

RandomeLoL wrote:

4 - Consent. This new system would make it possible for an inactive mapper to get a set of theirs Ranked. They would not be able to give consent if they do not want their works featured. It's not nice to expect that everyone wants their works featured, even if the community pushes for it. Loved already dealt with cases where a mapper's set was featured without consent, and replicating it in a section where they cannot get their set un-Ranked after the fact does not seem like the correct thing to promote.
This is just incorrect, the proposal only includes a system for bringing maps to BNs' attention; the actual ranking process wouldn't change in any way, so "Do not qualify a beatmap directly without any communication with the mapper." would still apply.
Last edited by lewski , edited 1 time in total.
RandomeLoL
Nomination Assessment Team
224 posts
Joined September 2015
RandomeLoL
Glad that would still apply. Focused too much on their wording rather than contrasting it with pre-established norms.

But yes, the points above still need to be addressed in some capacity.
niat0004
226 posts
Joined September 2017
Topic Starter
niat0004
Nao 1: Maybe limiting Hype to mappers with at least (around) 5 minutes of (ranked/unranked) drain time in the relevant mode and/or (around) 15 kudosu could work? I know these are somewhat low bars, but I do not want to risk limiting Hype excessively; I want to limit it to mappers, new and experienced.

Nao 2: The checks are very simple - they make sure a map is not obviously unrankable and of low quality.

Ryu 1: That is the entire point. Maps with over 25 Hype are rare, and I want Hype to have a use beyond the "5 for nomination eligibility" it currently has. Although it is probably a good idea to limit it to only 1 in the queue for each mapper.

Ryu 2: You have a valid point. Unfortunately, I do not really know who else can check for a map being low-quality and obviously unrankable, and a QI should be experienced enough to know that. If you have a better idea (for e.g. a custom group), I would definitely like to hear it.

Ryu 3: I know this. In its current form, Kudosu is unusable, being no more significant to mappers than PP is to players. I wanted to bring it back in some form. (With a higher requirement as Kudosu apparently went down in value.)

LoL 1: The standard is meant to be much lower than for BNs. They essentially just check whether the map is overtly bad and unrankable.

LoL 3: You have an entirely valid point. I do not know what would work - Kudosu stamps favor established mappers too much, Hypes favor them too little.

S0S 1: This is the best idea so far. Replacing "Kudosu stamps" with "Resurrection stamps" from a new "Beatmap Reviver" team (lower than the BNs, much more open, less "official") would be a valid idea. I know this is essentially proto-bubbles, but that doesn't stop people posting Hypes/Praises that essentially serve the same function (with content like "After a thorough inspection of the Normal, Hard, and Insane, I believe this map is rankable with few flaws." or comments like "why is this stuck in Graveyard"/"rank this") At least, I used to do that.
Nao Tomori
osu! Alumni
3,107 posts
Joined December 2014
Nao Tomori
"Not unrankable and not low quality" is not a simple check. The vast majority of BNs have years of modding experience and still occasionally miss unrankables and nominate things with gaping flaws and so on. The entire BN and its evaluation system already exist to see if the maps are "not unrankable and not low quality" and standards are already quite low - adding another system below isn't gonna help.

Also I just want to add - I've literally, and I do mean literally, never opened a single map that I've hyped prior to hyping it. I would bet most people are in the same boat. It just isn't a useful system. Modding v1 had something similar with star priority. Ostensibly, maps with higher star priority were supposed to be checked by BNs. The end result was a bunch of awful maps with tons of mods sitting at the top of the list forever because they sucked and nobody wanted to explain how and why to remap them.
Last edited by Nao Tomori , edited 2 times in total.
Okoayu
Nomination Assessment Team / Elite Nominator
4,959 posts
Joined May 2012
Okoayu

Nao Tomori wrote:

[ ... words ] The end result was a bunch of awful maps with tons of mods sitting at the top of the list forever because they sucked and nobody wanted to explain how and why to remap them.
That is disregarding all the sets that were sitting there, beinig godawful and all while having a bunch of mods explaining that they should remap and also refusing to remap because they've gotten so many mods

it's the perfect catch 22

the checks you propose aren't very simple (low quality)

your statement about kudosu being no more than mapper pp is just wrong, the kudosu is just a number that gets less accurate the longer you participated in the system (the shift from modv1 to modv2 did not convert already existing kd to account for each post being their own kd thing)

a group dealing with a queue of pending maps with unknown/ambiguous/user-defined priority (resurrection stamps or whatever) implies that such a group of users wants to exist. The global bn queue on the BN site (idk usage statistics) would be that idea, except i dont think many people are using it to pick stuff to mod or check
Hivie
osu!taiko Paragon
1,002 posts
Joined March 2019
Hivie
archived considering the pushback towards the proposed changes, alongside the proposal's complexity in the first place.
niat0004
226 posts
Joined September 2017
Topic Starter
niat0004

Hivie wrote:

archived considering the pushback towards the proposed changes, alongside the proposal's complexity in the first place.
I agree with this action.
Please sign in to reply.

New reply

0
1 / 22