1. osu! forums
  2. osu!
  3. Development
posted
Hello everyone! After some discussions about the QAT rework thread, we are happy to present you our proposal for a new system.

Proposal [Updated] | Proposal


Please note that this is still a proposal and that we are taking feedback. If you want changes to be made to it, please post in this thread until Monday, February 4th 23:59 UTC+0. After that, we will try to incorporate your feedback as well as we can!
posted
kowai
posted
uh
posted
Sounds interesting, but i have a little problem with this part:

Applications for joining the Beatmap Nominators will mostly stay the same but instead of holding them every three months, they will be open at any point in time.
The applications will still be composed of two parts, a ranking criteria proficiency test and a manual evaluation by members of the “Management Team”.

If you are removed from the Beatmap Nominators, you will be put on a cooldown of three months before you can re-apply. If you leave the Beatmap Nominators on your own merit though, you will be allowed to re-join within the next two weeks after your removal.
Okay so, sounds not bad but there is one aspect not explained. What if someone applied and got rejected. Will this person be set on a cooldown before they can re-apply or not?
It's a pretty important part. If there is no cooldown, people who got rejected and "Do not accept the decision" or pretty stubborn persons would spam their applications all the time and the "Management Team" would have to re-evaluate this person all the time new.
posted
this is v nice i am pogchampioning out of my seat
posted
Give the Disqualify button to the Beatmap Nominators
The Beatmap Nominators will take on the current QAT’s job of acting on reports in the Report a Qualified beatmap here! thread.

Vetoes will be decided by the Beatmap Nominators
Instead of a small group deciding about the fate of a map, the “Management Team” will initiate a majority vote among all BNs (except for the one who placed the veto) of the respective mode.
Top 10 plans to fail within weeks, guaranteed barely anyone will be trying to dq controversial or even just subjectively bad maps by april because it won't be worth the time and effort to do so.
posted
You should clarify "If you are removed from the Beatmap Nominators, you will be put on a cooldown of three months before you can re-apply." to include "or your application is rejected" since that's presumably how the system would work.

I am concerned about 2 things. first: majority vote for all vetoes. I think this would be a lot of overhead and think that randomly selected pool of x amount of BNs would be better to avoid having to ping every BN and get their opinion on vetoed maps over and over, which would get tiring. Think of it like jury duty or something. If people are always asked "is this map which you really don't give a shit about rankable" then it will lose a sense of meaning resulting in people not looking at the maps or vetoes in detail and just saying yeah it's fine whatever.

second, and more importantly: unrankables and objective issues are not actually important or a good representation of a BN's skill. for example: BN A consistently pushes maps which have complex timing and snappings, which occasionally get DQ'd for some adjustments to timing or snapping. BN B exclusively nominates single bpm NHI 1/2 based anime maps which never get DQ'd.
BN A is at much, much, MUCH higher risk of getting kicked than BN B despite nominating more diverse or interesting content than the usual fare, and neither of the two maps getting DQ'd for being bad maps. Only counting objective DQ's does not mean that people can nominate risky maps - it just changes the definition of risky from "uses weird techniques" to "has an unquantized mp3" and doesn't address the core issue of people only wanting to nominate single bpm 1/2 based anime maps to avoid getting kicked for timing DQ's.
This is a fundamental problem with how QATs gave up on caring about map quality and only focusing on unrankables - unrankable issues are not major problems for the most part. A wrongly snapped slider takes a grand total of 2 seconds to fix. A 10ms red line adjustment is not a major problem that causes a good map to become terrible. Meanwhile people nominate all forms of complete and utter trash (Uta intro ver anyone?) which don't have unrankables and therefore suddenly they are fine.
I'm not saying to magically start giving a shit about mapping quality but this idea that a BN getting DQ's for timing is worse than a BN not getting DQ's because they nominate rankable trash is really stupid and shouldn't be in the score system. The score system should just be activity and SERIOUS unrankable issues (entire sections missing hitsoundings, large amounts of unsnaps due to fucked up green lines, 20+ms offset issues, etc.)
posted
"Disband the Quality Assurance Team"
All I needed to read.
posted
The “Quality Assurance Helpers” sub-group inside the BNs will continue checking every qualified map for unrankables and objective issues until an automated method replaces them. Moreover, the QAH will be renamed to avoid confusion with the former QAT.

The Beatmap Nominator rules will be updated so that a map may only be disqualified if unrankable or objective issues are present, the mapper requests it, or in the event of a veto.
Does this mean that probation BNs will also be able to DQ maps?

Beatmap Nominator Applications will always be open
Note that implementation of this system might take some time to prepare

Applications for joining the Beatmap Nominators will mostly stay the same but instead of holding them every three months, they will be open at any point in time.
Having the applications always open means that there would always be probation BNs, so would probation just disappear? Because I don't know how it could be done to be constantly looking over new people that don't come in a cycle or any set number, just random.

Just sounds a little confusing right now. ):
posted
You guys should add a cooldown between how long people can apply for BN - perhaps two weeks or something. People could just spam requests every other day under this proposal since it's not listed...

edit: Hi Feerum! I didn't reload my page until after I posted this and you mentioned this already.
posted
Maybe make the name something more based towards mapping? "Management Team" seems to general and reminds me of some office job lmao.

Even "Mapping Management Team" would be better imo.
posted
Main thing I took away is that it’s just the current system but everything’s appropriately labeled instead
posted
T_T
posted
Re:vetos and votes

Will every BN be required to vote?
Are BNs allowed to abstain from the vote?
If yes, and a large majority of BNs opt to abstain from the vote (so that a very small portion of BNs actually voted on the map), what will happen?
Will probation BNs vote on vetos and if so, will their votes be as evenly weighted as full BNs?
posted
Main Issues:

Majority Vote regarding Vetoes

It isn't feasible to hold a vote across the entire BNG. Something like selecting x amount of readily available BN's would be better. I think 7 is the most feasible. Selection should be random, of course, and participation in vetoes should become a mandatory responsibility of BN's. Currently, it seems as if people who don't want to participate in votes can just not. There is no penalty for not participating, but as a result the vote may be skewed. Therefore, in order to ensure participation, I feel BN's who are chosen to participate in any veto mediation vote must also provide a short opinion on their choice (yes or no). I think it only has to be 2-3 sentences, nothing major, but this is to at least acknowledge that the BN has participated in the veto and isn't just going to blindly vote yes/no to everything and be non-toxic. (Voting no to a veto mediation is NOT being toxic, let's not set any bad precedents).

What happens after a Veto?

This one's rather straightforward. After a veto has been placed, and has been maintained through voting, the mapper is then forced to change the pattern or leave the map to be graved. Who will lift the veto? I'm assuming the original BN. But what if they continue to maintain that the mapper hasn't fully addressed the issue, or the mapper continues to call the veto'ing BN back to recheck without completely fixing the issue?

I want to add two additional rules to flesh out veto mediation scenarios:

- After a veto has been maintained, If the veto'ing BN does not recheck (and/or approve) the changes made to the map one month after the mapper has called them back for a recheck, the veto will be lifted. This will resolve both issues. It allows for mappers to "escape" a veto if the BN is not willing, or has grown tired of enforcing the veto and constantly rechecking the map. However, it also allows the BN time before having to recheck. If the mapper continues to call the BN back without addressing changes adequately, the BN is also welcome to take other requests, ask the mapper to rethink their concepts, and recheck a few weeks later. I think this is fair, as some vetoes for core issues are not fixable without major reworks anyways.

- If the veto'ing BN is removed, any of the BN's who voted "no" in the veto mediation can replace the BN and uphold the veto. If no one wishes to do so, then after 1 month, the veto will be lifted.

How to get into Management Team?

I'm sure a few people (hi Ascendance) will be interested primarily in this. Is there a process to get into Management Team? And what will happen to other aspects of the mapping/modding community? I'm talking about pushing forward new amendments and changes to the Ranking Criteria mainly. (Which is another subset issue). How will people be chosen for this specialized team?

BN Acceptance

Will there be any change to the way BN's are accepted going forward, now that reviews are no longer every 3 months but on a rolling basis? Will activity count be lowered etc...? I'm also interested in changes to selection criteria. Seeing as how this can potentially mean a lot more BN's entering more frequently, (as well as exiting).

Also if someone is rejected, what is the cooldown before they can reapply again?

Another note: I feel severity of a kick should be considered too. For example, BN's kicked for missing too many unrankable elements, or are inactive etc... should not be subject to a 3 month cooldown imo. I feel that for unrankable elements, this suggests the BN needs more training, or isn't thorough enough, but 1-2 months should be enough. For inactivity, well, can't a BN just resign if they know they will get kicked for inactivity? Then they would be able to reapply 2 weeks later. On the other end, I think behavior-related kicks could be 3 months. I think having a flat # of months is not necessary, and you guys could change the cooldown to suit the specific case of the BN being removed. Basically, make cooldowns more case-by-case and not necessarily do 3 months for everyone.
posted
Another change ugh..
I'll just follow along, dunno what to say..
posted
why is BN score based on issues rather than # of maps ranked?
posted
i don't like the idea of giving the DQ button to all nominators but everything else seems okay i guess.
posted
I like where this is going at the moment.

However, I am having some concerns.

New Beatmap Nominators // Probation
How are new Beatmap Nominators and Probations going to be handled? Will they be able to participate in disqualifications, vetoes and majority votes?

Application cooldown
- Is there any cooldown for potential Beatmap Nominators to re-apply after they fail the test? If so, how long would it be?
- Just a clarifications, is there a cooldown for nominators leaving by their own? The follow sentence is a little confusing to me.

"If you leave the Beatmap Nominators on your own merit though, you will be allowed to re-join within the next two weeks after your removal."
Does it means that a nominator leaving on their own will only have two weeks to come back to the team? If so, this short period of time seems unnecessary. In this case, it would only require the nominator to send in a "Leave of Absence" for said time. Instead, give the chance to the nominator to join back in before the next 30 days after their removal.
Or does it means they can join back 2 weeks after their removal?
posted

dudehacker wrote:

why is BN score based on issues rather than # of maps ranked?
beatmap nominators are in charge of, in essence, deciding what maps get ranked. therefore, under this, a bn doesn't actually need to have a ranked map, they just need to know what is high enough quality for ranked, and be able to spot issues within a map

tl;dr - a bn doesn't need to be a good mapper, but they do need to be a good modder
show more
Please sign in to reply.