or you could not rely on a quantitative system to measure qualitative issues? lmao
Nao Tomori wrote:
or you could not rely on a quantitative system to measure qualitative issues? lmao
Kibbleru wrote:
We do take into account the number of nominations being made by said bn compared to the number of DQs they get.
Kibbleru wrote:
For the controversial factor, honestly it's a bit hard to get a good measure of. Perhaps it could be some factor of total drain time and star rating?
Kibbleru wrote:
I know star rating is not the best for judging controversy, but usually the harder the map, the more likely it would have problems.
Nao Tomori wrote:
second, and more importantly: unrankables and objective issues are not actually important or a good representation of a BN's skill. for example: BN A consistently pushes maps which have complex timing and snappings, which occasionally get DQ'd for some adjustments to timing or snapping. BN B exclusively nominates single bpm NHI 1/2 based anime maps which never get DQ'd.
BN A is at much, much, MUCH higher risk of getting kicked than BN B despite nominating more diverse or interesting content than the usual fare, and neither of the two maps getting DQ'd for being bad maps. Only counting objective DQ's does not mean that people can nominate risky maps - it just changes the definition of risky from "uses weird techniques" to "has an unquantized mp3" and doesn't address the core issue of people only wanting to nominate single bpm 1/2 based anime maps to avoid getting kicked for timing DQ's.
This is a fundamental problem with how QATs gave up on caring about map quality and only focusing on unrankables - unrankable issues are not major problems for the most part. A wrongly snapped slider takes a grand total of 2 seconds to fix. A 10ms red line adjustment is not a major problem that causes a good map to become terrible. Meanwhile people nominate all forms of complete and utter trash (Uta intro ver anyone?) which don't have unrankables and therefore suddenly they are fine.
I'm not saying to magically start giving a shit about mapping quality but this idea that a BN getting DQ's for timing is worse than a BN not getting DQ's because they nominate rankable trash is really stupid and shouldn't be in the score system. The score system should just be activity and SERIOUS unrankable issues (entire sections missing hitsoundings, large amounts of unsnaps due to fucked up green lines, 20+ms offset issues, etc.)
proposal wrote:
Disband the Quality Assurance Team
- (...) To fix this issue, we want to disband the QAT and replace it by a yet to be named “Management Team” that only focuses on promoting and evaluating Beatmap Nominators as well as making sure that the system runs smoothly.
proposal wrote:
Give the Disqualify button to the Beatmap Nominators
proposal wrote:
The Beatmap Nominator rules will be updated so that a map may only be disqualified if unrankable or objective issues are present, the mapper requests it, or in the event of a veto.
sounds good at first but has one major issue: a lot of BNs don't care about vetoes at all, they joined the BNG to push forward maps they like, not discuss vetoes on some random anime map. should only involve a part of the BNs (randomly selected group, volunteer group similar to QAH right now, idk). Point is, forcing ALL BNs do deal with every single veto seems pretty overdone and will also results in people either just not caring about their votes on a veto, or leaving BNG altogether because that's just not what they joined it for.proposal wrote:
Vetoes will be decided by the Beatmap Nominators - initiate a majority vote among all BNs (except for the one who placed the veto) of the respective mode.
proposal wrote:
Content related moderation will be handled by the Global Moderation Team
again? Didn't work before and probably won't work now unless it gets heavily adjusted (see first quote)proposal wrote:
A new score system for Beatmap Nominators will be introduced
proposal wrote:
Beatmap Nominator Applications will always be open
proposal wrote:
If you leave the Beatmap Nominators on your own merit though, you will be allowed to re-join within the next two weeks after your removal.
Vetoes will be decided by the Beatmap Nominators through majority vote
This, I agree.Lasse wrote:
sounds good at first but has one major issue: a lot of BNs don't care about vetoes at all, they joined the BNG to push forward maps they like, not discuss vetoes on some random anime map. should only involve a part of the BNs (randomly selected group, volunteer group similar to QAH right now, idk). Point is, forcing ALL BNs do deal with every single veto seems pretty overdone and will also results in people either just not caring about their votes on a veto, or leaving BNG altogether because that's just not what they joined it for.
I saw alot of my friends having good intention to nominate complex timing and snappings. But they all got punished by how many dqs they got, ofcourse there will be a lot of dqs, and I think that's inevitable. For starter, maybe reduce the punishment of these because imo many bns especially new ones are getting push to nominate stupid 1-2 jumps etc etc,Nao Tomori wrote:
second, and more importantly: unrankables and objective issues are not actually important or a good representation of a BN's skill. for example: BN A consistently pushes maps which have complex timing and snappings, which occasionally get DQ'd for some adjustments to timing or snapping. BN B exclusively nominates single bpm NHI 1/2 based anime maps which never get DQ'd.
BN A is at much, much, MUCH higher risk of getting kicked than BN B despite nominating more diverse or interesting content than the usual fare, and neither of the two maps getting DQ'd for being bad maps. Only counting objective DQ's does not mean that people can nominate risky maps - it just changes the definition of risky from "uses weird techniques" to "has an unquantized mp3" and doesn't address the core issue of people only wanting to nominate single bpm 1/2 based anime maps to avoid getting kicked for timing DQ's.
This is a fundamental problem with how QATs gave up on caring about map quality and only focusing on unrankables - unrankable issues are not major problems for the most part. A wrongly snapped slider takes a grand total of 2 seconds to fix. A 10ms red line adjustment is not a major problem that causes a good map to become terrible. Meanwhile people nominate all forms of complete and utter trash (Uta intro ver anyone?) which don't have unrankables and therefore suddenly they are fine.
I'm not saying to magically start giving a shit about mapping quality but this idea that a BN getting DQ's for timing is worse than a BN not getting DQ's because they nominate rankable trash is really stupid and shouldn't be in the score system. The score system should just be activity and SERIOUS unrankable issues (entire sections missing hitsoundings, large amounts of unsnaps due to fucked up green lines, 20+ms offset issues, etc.)
Beatmap Nominator Applications will always be open
The applications will still be composed of two parts, a ranking criteria proficiency test and a manual evaluation by members of the “Management Team”.
Nao Tomori wrote:
to be completely honest i dont think that even if the answers to the rc test are leaked it is a huge issue for 2 reasons:
1. the test itself is ridiculously easy
2. people who cheated to get in would presumably fail probation for bubbling a bunch of unrankables that they didnt know about
Nao Tomori wrote:
to be completely honest i dont think that even if the answers to the rc test are leaked it is a huge issue for 2 reasons:
1. the test itself is ridiculously easy
2. people who cheated to get in would presumably fail probation for bubbling a bunch of unrankables that they didnt know about
Uta wrote:
I think there is a reason for it to be easy. Well~ it's not really easy imo, more like tricky. especially for those whose just got into a process of being familiar with the RC test. Easy unrankables is the issue that BNs will mostly find throughtout their modding carrier. They can still learn while being a bn, myself have found some problems that I've never seen when I'm not a BN and I'm abit aware of that issues in the future. The process of knowing hard to find unrankables is still possible. Moreover, it happens rarely and only happens in complex maps. So if they are doing hard maps to nominate, they will know whats coming.
and yes, don't cheat please. if you cant pass an easy test like this. you pretty much deserve'nt becoming a BN.
Mao wrote:
Vetoes will be handled by a random jury of BNs. This random jury can be re-rolled by NAT if deemed necessary.BNs who leave the jury will be replaced as needed by another random member.
Monstrata wrote:
Some ideas: we can put some limiters to the "rng-ness" For example, NAT have the power to do a re-roll if say three members of the NAT believe that the sample group of BN's might produce a too one-sided or a biased judgement.
I have some concern here, for the minigame mode like taiko, ctb, and mania on this point.QAT Restructure Proposal [Updated] wrote:
Vetoes will be decided by the Beatmap Nominators
As the QAT will no longer exist, this task will be handed over to the Beatmap Nominators. The NAT will roll out the names of twelve BNs (can be less in other modes) that will form a jury and mediate the veto like the QAT did so far. The names of the people in the jury will be kept anonymous until the verdict is released. Participating in veto mediation is mandatory for every member of the BNs. Should a member drop out of the jury, they will be punished unless an appropriate reasoning is given. A new member is rolled in that case.
I completely agree with Monstrata on this for the 3 month cooldown for applicants. A good modder doesnt deserve to have the same cooldown with a ''pls fix blanket'' modder. We could rework the current system in a way where the NAT could give 1 extra month for a modder who was close on being a BN to prove with another 4 or more mods of the current/upcoming month that he improved on the specific aspect of his modding that hes currently lacking for. If hes gonna fail again, I guess the 3 month cooldown would be justified in that way. Right now the system is too unorganised and many modders are getting demotivated for waiting 3 months over and over again (I was one of them). So yeah, I think we could rework the BN apps in a better way than they currently are.Monstrata wrote:
Anyways,
From earlier post, that seems unaddressed even though there was quite a lot of support for it:
Being kicked from BN is honestly more severe than just failing to get into BN. 3 month cool-down for kicked BN's feels justified, but 3 month cool-down for applicants is quite harsh. I would recommend either 1 to 3 months based on how close the applicant was to becoming BN level. That way applicants who are denied also can get a sense of how much NAT members think they have yet to improve before they should try to apply for BN again.
If an applicant is only lacking perhaps in metadata checking, 1 months seems fair before they can apply again, since at that point it's a relatively simple skill that they need to improve on, and 3 months is just a lot of unnecessary time. However, if an applicant clearly only spams blanket/stack mods, clearly the maximum of 3 months is warranted. Giving out flat 3-month cooldowns can discourage promising modders from reapplying, (and also causes unnecessary social media meltdowns) and it does seem a bit unfair if someone barely missed the cut, that they still need to wait as long as someone who got denied for modding unsnapped bookmarks.
timemon wrote:
Maybe some sort of rewards/acknowledgement for being in the jury like counted activity. And as long as the rate of which the same bn is selected to be a part of the jury is reasonable.
Anyway, I assume NAT is going to organize the entire jury system.(there is obvious conflict of interest for BNs to handle this by themselves.) But it's not written in their title description other than ensuring the system runs smoothly.
The proposal also implies that QAH will be replaced by automated system eventually. I doubt you can make the system entirely automatic without human supervision due to the nature of RC (guidelines and whatnot)
To a certain degree, I feel this sort of "veto-mediation" responsibility should be expected of BN's since they are now given the power to DQ maps. I would rather have an opt-out system rather than an opt-in. Assume everyone is interested, and those who aren't can opt-out.-Mo- wrote:
I'm wondering how much interest there would be to jury if we had an opt-in system. I would rather not have such a system if we'll only have a pool of 3 people to pick from (also keep in mind that the other modes have a much smaller pool of BNs).
BNscore system will need quite a bit of work to develop, so it won't be a priority for this restructure for now and we'll offer it in a seperate proposal in the future with more detail on how it'll work.