@kibb Yep, exactly. I feel like it's not too hard to implement a 1-dq per day limit into BN functions too (similar to 3 qualifications per day etc...) so we don't have CDFA v3 lol.
Monstrata wrote:
@kibb Yep, exactly. I feel like it's not too hard to implement a 1-dq per day limit into BN functions too (similar to 3 qualifications per day etc...) so we don't have CDFA v3 lol.
Kibbleru wrote:
The one thing i want to avoid is a scoring system.
As someone who had first hand experience with the scoring system, having that number out there which determined your outcomes just put pressure on us.
While i do think a scoring system is useful for the evaluation of BNs, it's probably better to keep it private, so BNs wouldn't feel pressured by this number. The management team will simply use that scoring system for judgement of said BN.
Kibbleru wrote:
While i do think a scoring system is useful for the evaluation of BNs, it's probably better to keep it private, so BNs wouldn't feel pressured by this number. The management team will simply use that scoring system for judgement of said BN.
squirrelpascals wrote:
Kibbleru wrote:
While i do think a scoring system is useful for the evaluation of BNs, it's probably better to keep it private, so BNs wouldn't feel pressured by this number. The management team will simply use that scoring system for judgement of said BN.
This sounds like it might be unpopular, but just knowing a number exists in the first place will put pressure on me, and I feel like not knowing what it is would only make that feeling worse. I would know that there is some sort of value that determines how i rank up against other bns, but assuming that we're not able to know what value that is (because it's kept secret), I won't know what exactly I'm able to do to perform better, forcing me to be perfect in every aespect (which isn't realistic). This sounds very similar to my experience that I explained on probation here.
I think a good middle ground for this would be making the viewing of the score sheet optional, because I imagine everyone will feel differently about being able to see their score :p So if a bn wants to see their score, they can request an invite to a google sheet that shows it or whatever, but if not that's ok too.
squirrelpascals wrote:
Mao wrote:
We also still need to figure out how to handle probation exactly.
For starters, I'd like to suggest telling the bns on probation generally what Management will be looking for when they're evaluated. My readmission felt a bit like the QAT was saying "ok you're a bn now but you're also on probation, so yeah gl." There wasn't really much guidance as to how anyone on probation was being evaluated. Were the QAT looking for number of icons? bubble pops or dqs? activity? I got a lot of mixed responses and some rumors from people I asked (bns and like 2 qat i think). For myself I had the reasons from my previous probation failure to focus on improving but I didn't see any clarity as to how our performance was being measured for the other probation bns. This just put pressure on me to be perfect in every aspect of modding, which I know isn't the true expectation of the QAT.
This whole ordeal made probation super super stressful for me, and it made it a bit of a mystery as to why some of the people didn't pass. Probation doesn't need to be stressful or mysterious. We need to write some sort of probation quidelines or something.
Scoring has always existed no?squirrelpascals wrote:
This sounds like it might be unpopular, but just knowing a number exists in the first place will put pressure on me, and I feel like not knowing what it is would only make that feeling worse. I would know that there is some sort of value that determines how i rank up against other bns, but assuming that we're not able to know what value that is (because it's kept secret), I won't know what exactly I'm able to do to perform better, forcing me to be perfect in every aespect (which isn't realistic). This sounds very similar to my experience that I explained on probation here.Kibbleru wrote:
While i do think a scoring system is useful for the evaluation of BNs, it's probably better to keep it private, so BNs wouldn't feel pressured by this number. The management team will simply use that scoring system for judgement of said BN.
I think a good middle ground for this would be making the viewing of the score sheet optional, because I imagine everyone will feel differently about being able to see their score :p So if a bn wants to see their score, they can request an invite to a google sheet that shows it or whatever, but if not that's ok too.
Kibbleru wrote:
Interesting point, I've actually never thought this was an issue since I assumed people knew that they should just not get any dq's and it'll be fine
This system will be similar to the old Beatmap Nominator Rankings back in 2014 and 2015. However, instead of focusing on subjective issues, this system will only account for unrankable and objective issues.
Hydria wrote:
Nominating two generic maps with 0 DQs in probation is a lot different than Nominating 6 maps, 5 of which are unique, with 1-2 DQs in probation. You punish the one that puts in the extra work.
Nao Tomori wrote:
or you could not rely on a quantitative system to measure qualitative issues? lmao
Kibbleru wrote:
We do take into account the number of nominations being made by said bn compared to the number of DQs they get.
Kibbleru wrote:
For the controversial factor, honestly it's a bit hard to get a good measure of. Perhaps it could be some factor of total drain time and star rating?
Kibbleru wrote:
I know star rating is not the best for judging controversy, but usually the harder the map, the more likely it would have problems.
Nao Tomori wrote:
second, and more importantly: unrankables and objective issues are not actually important or a good representation of a BN's skill. for example: BN A consistently pushes maps which have complex timing and snappings, which occasionally get DQ'd for some adjustments to timing or snapping. BN B exclusively nominates single bpm NHI 1/2 based anime maps which never get DQ'd.
BN A is at much, much, MUCH higher risk of getting kicked than BN B despite nominating more diverse or interesting content than the usual fare, and neither of the two maps getting DQ'd for being bad maps. Only counting objective DQ's does not mean that people can nominate risky maps - it just changes the definition of risky from "uses weird techniques" to "has an unquantized mp3" and doesn't address the core issue of people only wanting to nominate single bpm 1/2 based anime maps to avoid getting kicked for timing DQ's.
This is a fundamental problem with how QATs gave up on caring about map quality and only focusing on unrankables - unrankable issues are not major problems for the most part. A wrongly snapped slider takes a grand total of 2 seconds to fix. A 10ms red line adjustment is not a major problem that causes a good map to become terrible. Meanwhile people nominate all forms of complete and utter trash (Uta intro ver anyone?) which don't have unrankables and therefore suddenly they are fine.
I'm not saying to magically start giving a shit about mapping quality but this idea that a BN getting DQ's for timing is worse than a BN not getting DQ's because they nominate rankable trash is really stupid and shouldn't be in the score system. The score system should just be activity and SERIOUS unrankable issues (entire sections missing hitsoundings, large amounts of unsnaps due to fucked up green lines, 20+ms offset issues, etc.)
proposal wrote:
Disband the Quality Assurance Team
- (...) To fix this issue, we want to disband the QAT and replace it by a yet to be named “Management Team” that only focuses on promoting and evaluating Beatmap Nominators as well as making sure that the system runs smoothly.
proposal wrote:
Give the Disqualify button to the Beatmap Nominators
proposal wrote:
The Beatmap Nominator rules will be updated so that a map may only be disqualified if unrankable or objective issues are present, the mapper requests it, or in the event of a veto.
sounds good at first but has one major issue: a lot of BNs don't care about vetoes at all, they joined the BNG to push forward maps they like, not discuss vetoes on some random anime map. should only involve a part of the BNs (randomly selected group, volunteer group similar to QAH right now, idk). Point is, forcing ALL BNs do deal with every single veto seems pretty overdone and will also results in people either just not caring about their votes on a veto, or leaving BNG altogether because that's just not what they joined it for.proposal wrote:
Vetoes will be decided by the Beatmap Nominators - initiate a majority vote among all BNs (except for the one who placed the veto) of the respective mode.
proposal wrote:
Content related moderation will be handled by the Global Moderation Team
again? Didn't work before and probably won't work now unless it gets heavily adjusted (see first quote)proposal wrote:
A new score system for Beatmap Nominators will be introduced
proposal wrote:
Beatmap Nominator Applications will always be open
proposal wrote:
If you leave the Beatmap Nominators on your own merit though, you will be allowed to re-join within the next two weeks after your removal.
Vetoes will be decided by the Beatmap Nominators through majority vote
This, I agree.Lasse wrote:
sounds good at first but has one major issue: a lot of BNs don't care about vetoes at all, they joined the BNG to push forward maps they like, not discuss vetoes on some random anime map. should only involve a part of the BNs (randomly selected group, volunteer group similar to QAH right now, idk). Point is, forcing ALL BNs do deal with every single veto seems pretty overdone and will also results in people either just not caring about their votes on a veto, or leaving BNG altogether because that's just not what they joined it for.
I saw alot of my friends having good intention to nominate complex timing and snappings. But they all got punished by how many dqs they got, ofcourse there will be a lot of dqs, and I think that's inevitable. For starter, maybe reduce the punishment of these because imo many bns especially new ones are getting push to nominate stupid 1-2 jumps etc etc,Nao Tomori wrote:
second, and more importantly: unrankables and objective issues are not actually important or a good representation of a BN's skill. for example: BN A consistently pushes maps which have complex timing and snappings, which occasionally get DQ'd for some adjustments to timing or snapping. BN B exclusively nominates single bpm NHI 1/2 based anime maps which never get DQ'd.
BN A is at much, much, MUCH higher risk of getting kicked than BN B despite nominating more diverse or interesting content than the usual fare, and neither of the two maps getting DQ'd for being bad maps. Only counting objective DQ's does not mean that people can nominate risky maps - it just changes the definition of risky from "uses weird techniques" to "has an unquantized mp3" and doesn't address the core issue of people only wanting to nominate single bpm 1/2 based anime maps to avoid getting kicked for timing DQ's.
This is a fundamental problem with how QATs gave up on caring about map quality and only focusing on unrankables - unrankable issues are not major problems for the most part. A wrongly snapped slider takes a grand total of 2 seconds to fix. A 10ms red line adjustment is not a major problem that causes a good map to become terrible. Meanwhile people nominate all forms of complete and utter trash (Uta intro ver anyone?) which don't have unrankables and therefore suddenly they are fine.
I'm not saying to magically start giving a shit about mapping quality but this idea that a BN getting DQ's for timing is worse than a BN not getting DQ's because they nominate rankable trash is really stupid and shouldn't be in the score system. The score system should just be activity and SERIOUS unrankable issues (entire sections missing hitsoundings, large amounts of unsnaps due to fucked up green lines, 20+ms offset issues, etc.)
Beatmap Nominator Applications will always be open
The applications will still be composed of two parts, a ranking criteria proficiency test and a manual evaluation by members of the “Management Team”.
Nao Tomori wrote:
to be completely honest i dont think that even if the answers to the rc test are leaked it is a huge issue for 2 reasons:
1. the test itself is ridiculously easy
2. people who cheated to get in would presumably fail probation for bubbling a bunch of unrankables that they didnt know about
Nao Tomori wrote:
to be completely honest i dont think that even if the answers to the rc test are leaked it is a huge issue for 2 reasons:
1. the test itself is ridiculously easy
2. people who cheated to get in would presumably fail probation for bubbling a bunch of unrankables that they didnt know about
Uta wrote:
I think there is a reason for it to be easy. Well~ it's not really easy imo, more like tricky. especially for those whose just got into a process of being familiar with the RC test. Easy unrankables is the issue that BNs will mostly find throughtout their modding carrier. They can still learn while being a bn, myself have found some problems that I've never seen when I'm not a BN and I'm abit aware of that issues in the future. The process of knowing hard to find unrankables is still possible. Moreover, it happens rarely and only happens in complex maps. So if they are doing hard maps to nominate, they will know whats coming.
and yes, don't cheat please. if you cant pass an easy test like this. you pretty much deserve'nt becoming a BN.
Mao wrote:
Vetoes will be handled by a random jury of BNs. This random jury can be re-rolled by NAT if deemed necessary.BNs who leave the jury will be replaced as needed by another random member.