kowai
forum
QAT Restructure Follow-Up
posted
Total Posts
105
Sounds interesting, but i have a little problem with this part:
Okay so, sounds not bad but there is one aspect not explained. What if someone applied and got rejected. Will this person be set on a cooldown before they can re-apply or not?
It's a pretty important part. If there is no cooldown, people who got rejected and "Do not accept the decision" or pretty stubborn persons would spam their applications all the time and the "Management Team" would have to re-evaluate this person all the time new.
Applications for joining the Beatmap Nominators will mostly stay the same but instead of holding them every three months, they will be open at any point in time.
The applications will still be composed of two parts, a ranking criteria proficiency test and a manual evaluation by members of the “Management Team”.
If you are removed from the Beatmap Nominators, you will be put on a cooldown of three months before you can re-apply. If you leave the Beatmap Nominators on your own merit though, you will be allowed to re-join within the next two weeks after your removal.
Okay so, sounds not bad but there is one aspect not explained. What if someone applied and got rejected. Will this person be set on a cooldown before they can re-apply or not?
It's a pretty important part. If there is no cooldown, people who got rejected and "Do not accept the decision" or pretty stubborn persons would spam their applications all the time and the "Management Team" would have to re-evaluate this person all the time new.
Give the Disqualify button to the Beatmap Nominators
The Beatmap Nominators will take on the current QAT’s job of acting on reports in the Report a Qualified beatmap here! thread.
Vetoes will be decided by the Beatmap Nominators
Instead of a small group deciding about the fate of a map, the “Management Team” will initiate a majority vote among all BNs (except for the one who placed the veto) of the respective mode.
Top 10 plans to fail within weeks, guaranteed barely anyone will be trying to dq controversial or even just subjectively bad maps by april because it won't be worth the time and effort to do so.
You should clarify "If you are removed from the Beatmap Nominators, you will be put on a cooldown of three months before you can re-apply." to include "or your application is rejected" since that's presumably how the system would work.
I am concerned about 2 things. first: majority vote for all vetoes. I think this would be a lot of overhead and think that randomly selected pool of x amount of BNs would be better to avoid having to ping every BN and get their opinion on vetoed maps over and over, which would get tiring. Think of it like jury duty or something. If people are always asked "is this map which you really don't give a shit about rankable" then it will lose a sense of meaning resulting in people not looking at the maps or vetoes in detail and just saying yeah it's fine whatever.
second, and more importantly: unrankables and objective issues are not actually important or a good representation of a BN's skill. for example: BN A consistently pushes maps which have complex timing and snappings, which occasionally get DQ'd for some adjustments to timing or snapping. BN B exclusively nominates single bpm NHI 1/2 based anime maps which never get DQ'd.
BN A is at much, much, MUCH higher risk of getting kicked than BN B despite nominating more diverse or interesting content than the usual fare, and neither of the two maps getting DQ'd for being bad maps. Only counting objective DQ's does not mean that people can nominate risky maps - it just changes the definition of risky from "uses weird techniques" to "has an unquantized mp3" and doesn't address the core issue of people only wanting to nominate single bpm 1/2 based anime maps to avoid getting kicked for timing DQ's.
This is a fundamental problem with how QATs gave up on caring about map quality and only focusing on unrankables - unrankable issues are not major problems for the most part. A wrongly snapped slider takes a grand total of 2 seconds to fix. A 10ms red line adjustment is not a major problem that causes a good map to become terrible. Meanwhile people nominate all forms of complete and utter trash (Uta intro ver anyone?) which don't have unrankables and therefore suddenly they are fine.
I'm not saying to magically start giving a shit about mapping quality but this idea that a BN getting DQ's for timing is worse than a BN not getting DQ's because they nominate rankable trash is really stupid and shouldn't be in the score system. The score system should just be activity and SERIOUS unrankable issues (entire sections missing hitsoundings, large amounts of unsnaps due to fucked up green lines, 20+ms offset issues, etc.)
I am concerned about 2 things. first: majority vote for all vetoes. I think this would be a lot of overhead and think that randomly selected pool of x amount of BNs would be better to avoid having to ping every BN and get their opinion on vetoed maps over and over, which would get tiring. Think of it like jury duty or something. If people are always asked "is this map which you really don't give a shit about rankable" then it will lose a sense of meaning resulting in people not looking at the maps or vetoes in detail and just saying yeah it's fine whatever.
second, and more importantly: unrankables and objective issues are not actually important or a good representation of a BN's skill. for example: BN A consistently pushes maps which have complex timing and snappings, which occasionally get DQ'd for some adjustments to timing or snapping. BN B exclusively nominates single bpm NHI 1/2 based anime maps which never get DQ'd.
BN A is at much, much, MUCH higher risk of getting kicked than BN B despite nominating more diverse or interesting content than the usual fare, and neither of the two maps getting DQ'd for being bad maps. Only counting objective DQ's does not mean that people can nominate risky maps - it just changes the definition of risky from "uses weird techniques" to "has an unquantized mp3" and doesn't address the core issue of people only wanting to nominate single bpm 1/2 based anime maps to avoid getting kicked for timing DQ's.
This is a fundamental problem with how QATs gave up on caring about map quality and only focusing on unrankables - unrankable issues are not major problems for the most part. A wrongly snapped slider takes a grand total of 2 seconds to fix. A 10ms red line adjustment is not a major problem that causes a good map to become terrible. Meanwhile people nominate all forms of complete and utter trash (Uta intro ver anyone?) which don't have unrankables and therefore suddenly they are fine.
I'm not saying to magically start giving a shit about mapping quality but this idea that a BN getting DQ's for timing is worse than a BN not getting DQ's because they nominate rankable trash is really stupid and shouldn't be in the score system. The score system should just be activity and SERIOUS unrankable issues (entire sections missing hitsoundings, large amounts of unsnaps due to fucked up green lines, 20+ms offset issues, etc.)
"Disband the Quality Assurance Team"
All I needed to read.The “Quality Assurance Helpers” sub-group inside the BNs will continue checking every qualified map for unrankables and objective issues until an automated method replaces them. Moreover, the QAH will be renamed to avoid confusion with the former QAT.
The Beatmap Nominator rules will be updated so that a map may only be disqualified if unrankable or objective issues are present, the mapper requests it, or in the event of a veto.
Does this mean that probation BNs will also be able to DQ maps?
Beatmap Nominator Applications will always be open
Note that implementation of this system might take some time to prepare
Applications for joining the Beatmap Nominators will mostly stay the same but instead of holding them every three months, they will be open at any point in time.
Having the applications always open means that there would always be probation BNs, so would probation just disappear? Because I don't know how it could be done to be constantly looking over new people that don't come in a cycle or any set number, just random.
Just sounds a little confusing right now. ):
You guys should add a cooldown between how long people can apply for BN - perhaps two weeks or something. People could just spam requests every other day under this proposal since it's not listed...
edit: Hi Feerum! I didn't reload my page until after I posted this and you mentioned this already.
edit: Hi Feerum! I didn't reload my page until after I posted this and you mentioned this already.
Maybe make the name something more based towards mapping? "Management Team" seems to general and reminds me of some office job lmao.
Even "Mapping Management Team" would be better imo.
Even "Mapping Management Team" would be better imo.
Main thing I took away is that it’s just the current system but everything’s appropriately labeled instead
T_T
Re:vetos and votes
Will every BN be required to vote?
Are BNs allowed to abstain from the vote?
If yes, and a large majority of BNs opt to abstain from the vote (so that a very small portion of BNs actually voted on the map), what will happen?
Will probation BNs vote on vetos and if so, will their votes be as evenly weighted as full BNs?
Will every BN be required to vote?
Are BNs allowed to abstain from the vote?
If yes, and a large majority of BNs opt to abstain from the vote (so that a very small portion of BNs actually voted on the map), what will happen?
Will probation BNs vote on vetos and if so, will their votes be as evenly weighted as full BNs?
Main Issues:
Majority Vote regarding Vetoes
It isn't feasible to hold a vote across the entire BNG. Something like selecting x amount of readily available BN's would be better. I think 7 is the most feasible. Selection should be random, of course, and participation in vetoes should become a mandatory responsibility of BN's. Currently, it seems as if people who don't want to participate in votes can just not. There is no penalty for not participating, but as a result the vote may be skewed. Therefore, in order to ensure participation, I feel BN's who are chosen to participate in any veto mediation vote must also provide a short opinion on their choice (yes or no). I think it only has to be 2-3 sentences, nothing major, but this is to at least acknowledge that the BN has participated in the veto and isn't just going to blindly vote yes/no to everything and be non-toxic. (Voting no to a veto mediation is NOT being toxic, let's not set any bad precedents).
What happens after a Veto?
This one's rather straightforward. After a veto has been placed, and has been maintained through voting, the mapper is then forced to change the pattern or leave the map to be graved. Who will lift the veto? I'm assuming the original BN. But what if they continue to maintain that the mapper hasn't fully addressed the issue, or the mapper continues to call the veto'ing BN back to recheck without completely fixing the issue?
I want to add two additional rules to flesh out veto mediation scenarios:
- After a veto has been maintained, If the veto'ing BN does not recheck (and/or approve) the changes made to the map one month after the mapper has called them back for a recheck, the veto will be lifted. This will resolve both issues. It allows for mappers to "escape" a veto if the BN is not willing, or has grown tired of enforcing the veto and constantly rechecking the map. However, it also allows the BN time before having to recheck. If the mapper continues to call the BN back without addressing changes adequately, the BN is also welcome to take other requests, ask the mapper to rethink their concepts, and recheck a few weeks later. I think this is fair, as some vetoes for core issues are not fixable without major reworks anyways.
- If the veto'ing BN is removed, any of the BN's who voted "no" in the veto mediation can replace the BN and uphold the veto. If no one wishes to do so, then after 1 month, the veto will be lifted.
How to get into Management Team?
I'm sure a few people (hi Ascendance) will be interested primarily in this. Is there a process to get into Management Team? And what will happen to other aspects of the mapping/modding community? I'm talking about pushing forward new amendments and changes to the Ranking Criteria mainly. (Which is another subset issue). How will people be chosen for this specialized team?
BN Acceptance
Will there be any change to the way BN's are accepted going forward, now that reviews are no longer every 3 months but on a rolling basis? Will activity count be lowered etc...? I'm also interested in changes to selection criteria. Seeing as how this can potentially mean a lot more BN's entering more frequently, (as well as exiting).
Also if someone is rejected, what is the cooldown before they can reapply again?
Another note: I feel severity of a kick should be considered too. For example, BN's kicked for missing too many unrankable elements, or are inactive etc... should not be subject to a 3 month cooldown imo. I feel that for unrankable elements, this suggests the BN needs more training, or isn't thorough enough, but 1-2 months should be enough. For inactivity, well, can't a BN just resign if they know they will get kicked for inactivity? Then they would be able to reapply 2 weeks later. On the other end, I think behavior-related kicks could be 3 months. I think having a flat # of months is not necessary, and you guys could change the cooldown to suit the specific case of the BN being removed. Basically, make cooldowns more case-by-case and not necessarily do 3 months for everyone.
Majority Vote regarding Vetoes
It isn't feasible to hold a vote across the entire BNG. Something like selecting x amount of readily available BN's would be better. I think 7 is the most feasible. Selection should be random, of course, and participation in vetoes should become a mandatory responsibility of BN's. Currently, it seems as if people who don't want to participate in votes can just not. There is no penalty for not participating, but as a result the vote may be skewed. Therefore, in order to ensure participation, I feel BN's who are chosen to participate in any veto mediation vote must also provide a short opinion on their choice (yes or no). I think it only has to be 2-3 sentences, nothing major, but this is to at least acknowledge that the BN has participated in the veto and isn't just going to blindly vote yes/no to everything and be non-toxic. (Voting no to a veto mediation is NOT being toxic, let's not set any bad precedents).
What happens after a Veto?
This one's rather straightforward. After a veto has been placed, and has been maintained through voting, the mapper is then forced to change the pattern or leave the map to be graved. Who will lift the veto? I'm assuming the original BN. But what if they continue to maintain that the mapper hasn't fully addressed the issue, or the mapper continues to call the veto'ing BN back to recheck without completely fixing the issue?
I want to add two additional rules to flesh out veto mediation scenarios:
- After a veto has been maintained, If the veto'ing BN does not recheck (and/or approve) the changes made to the map one month after the mapper has called them back for a recheck, the veto will be lifted. This will resolve both issues. It allows for mappers to "escape" a veto if the BN is not willing, or has grown tired of enforcing the veto and constantly rechecking the map. However, it also allows the BN time before having to recheck. If the mapper continues to call the BN back without addressing changes adequately, the BN is also welcome to take other requests, ask the mapper to rethink their concepts, and recheck a few weeks later. I think this is fair, as some vetoes for core issues are not fixable without major reworks anyways.
- If the veto'ing BN is removed, any of the BN's who voted "no" in the veto mediation can replace the BN and uphold the veto. If no one wishes to do so, then after 1 month, the veto will be lifted.
How to get into Management Team?
I'm sure a few people (hi Ascendance) will be interested primarily in this. Is there a process to get into Management Team? And what will happen to other aspects of the mapping/modding community? I'm talking about pushing forward new amendments and changes to the Ranking Criteria mainly. (Which is another subset issue). How will people be chosen for this specialized team?
BN Acceptance
Will there be any change to the way BN's are accepted going forward, now that reviews are no longer every 3 months but on a rolling basis? Will activity count be lowered etc...? I'm also interested in changes to selection criteria. Seeing as how this can potentially mean a lot more BN's entering more frequently, (as well as exiting).
Also if someone is rejected, what is the cooldown before they can reapply again?
Another note: I feel severity of a kick should be considered too. For example, BN's kicked for missing too many unrankable elements, or are inactive etc... should not be subject to a 3 month cooldown imo. I feel that for unrankable elements, this suggests the BN needs more training, or isn't thorough enough, but 1-2 months should be enough. For inactivity, well, can't a BN just resign if they know they will get kicked for inactivity? Then they would be able to reapply 2 weeks later. On the other end, I think behavior-related kicks could be 3 months. I think having a flat # of months is not necessary, and you guys could change the cooldown to suit the specific case of the BN being removed. Basically, make cooldowns more case-by-case and not necessarily do 3 months for everyone.
Another change ugh..
I'll just follow along, dunno what to say..
I'll just follow along, dunno what to say..
why is BN score based on issues rather than # of maps ranked?
i don't like the idea of giving the DQ button to all nominators but everything else seems okay i guess.
I like where this is going at the moment.
However, I am having some concerns.
New Beatmap Nominators // Probation
How are new Beatmap Nominators and Probations going to be handled? Will they be able to participate in disqualifications, vetoes and majority votes?
Application cooldown
- Is there any cooldown for potential Beatmap Nominators to re-apply after they fail the test? If so, how long would it be?
- Just a clarifications, is there a cooldown for nominators leaving by their own? The follow sentence is a little confusing to me.
Does it means that a nominator leaving on their own will only have two weeks to come back to the team? If so, this short period of time seems unnecessary. In this case, it would only require the nominator to send in a "Leave of Absence" for said time. Instead, give the chance to the nominator to join back in before the next 30 days after their removal.
Or does it means they can join back 2 weeks after their removal?
However, I am having some concerns.
New Beatmap Nominators // Probation
How are new Beatmap Nominators and Probations going to be handled? Will they be able to participate in disqualifications, vetoes and majority votes?
Application cooldown
- Is there any cooldown for potential Beatmap Nominators to re-apply after they fail the test? If so, how long would it be?
- Just a clarifications, is there a cooldown for nominators leaving by their own? The follow sentence is a little confusing to me.
"If you leave the Beatmap Nominators on your own merit though, you will be allowed to re-join within the next two weeks after your removal."
Does it means that a nominator leaving on their own will only have two weeks to come back to the team? If so, this short period of time seems unnecessary. In this case, it would only require the nominator to send in a "Leave of Absence" for said time. Instead, give the chance to the nominator to join back in before the next 30 days after their removal.
Or does it means they can join back 2 weeks after their removal?
dudehacker wrote:
why is BN score based on issues rather than # of maps ranked?
beatmap nominators are in charge of, in essence, deciding what maps get ranked. therefore, under this, a bn doesn't actually need to have a ranked map, they just need to know what is high enough quality for ranked, and be able to spot issues within a map
tl;dr - a bn doesn't need to be a good mapper, but they do need to be a good modder
Proposal wrote:
The Beatmap Nominator rules will be updated so that a map may only be disqualified if unrankable or objective issues are present, the mapper requests it, or in the event of a veto.
So does this mean that "dqs for discussion" are essentially dead? This means that if the community is still discussing a controversial map that gets ranked in one day, nothing is stopping it from getting ranked. Unless a bn initiates a "veto for discussion?" Which sounds like a complete work around and totally separate from a veto's true purpose. Controversial maps are controversial for a reason, so I think that room for discussion amongst the community is important so that we can reference to it for similar future maps.
Proposal wrote:
A new score system for Beatmap Nominators will be introduced
However, instead of focusing on subjective issues, this system will only account for unrankable and objective issues.
It doesn't sound like it makes any sense to score bns only on unrankable issues they find, since that doesn't really consider the initial quality of the maps modded, which can give a false perception of a bn's activity.
Say, for example, bn #1 checks 20 maps that are all from fairly experienced mappers who know what they're doing, so he finds about 5 unrankable issues in total. bn #2 likes to help newer mappers out who are becoming more capable of ranking stuff, so he checks 4 maps in total and finds 10 unrankable issues. Assuming all the unrankable issues of all maps are found, bn #1 is pushing 5x more maps than bn #2 is through the ranking process. But under this scoring system bn #2's score is still 2x higher.
I've tracked all the unrankable issues I've found under my modding spreadsheet which needs to be updated, and using data from the past 5 months I found that the average amount of unrankable issues I find in each map comes to ~2.5. So I think a formula we can start from, measuring the amount of unrankable issues found per map, can look something like
So a score of 1 means you found an average amount of unrankables per map. Maps with no unrankable issues (from more experienced mappers) and maps that have a lot of unrankable issues (from newer mappers) will pretty much cancel eachother out.
I wrote a lot but I get this is still being developed :p Besides for those 2 things this proposal sounds super dope!
@dudehacker they never want to rank bns based on how active they are because then all the inactive people start yelling about how the active ones just yolo rank everything instead of thinking about quality =D
also they will never reward bns, so having a system that is positive (# of maps iconed) makes no sense because the only thing bns will ever get is punished; a negative system (# of maps dq'd/popped) fits that much better
also they will never reward bns, so having a system that is positive (# of maps iconed) makes no sense because the only thing bns will ever get is punished; a negative system (# of maps dq'd/popped) fits that much better
Kawawa
＊Give the Disqualify button to the Beatmap Nominators
＊Vetoes will be decided by the Beatmap Nominators
I think that even though BNs will have the power to veto maps, I am especially worried that mini-game BNs will not use this power as often as they should due to either disinterest or a fear of backlash. BNs will be in the same situation as QATs before them where they are afraid to do a veto or afraid to vote yes on veto votes for subjective issues. there is also a chance that circlejerk vetoes will happen. ＊Vetoes will be decided by the Beatmap Nominators
The QAH role is not as clear as it should be either since their main job was to assist the QAT. If the QAH has to directly veto instead of report less people will want to join due to having much more pressure on them than the average BN. There isn't really any reward for BNs doing a good job with this new system, instead it punishes them with more opportunities for community criticism and higher chance of kicking imo.
so not sure that QAT must be removed. I'm still think the current system is pretty solid, since it is just a inside problem.
Topic Starter
Monstrata wrote:
It isn't feasible to hold a vote across the entire BNG. Something like selecting x amount of readily available BN's would be better. I think 7 is the most feasible. Selection should be random, of course, and participation in vetoes should become a mandatory responsibility of BN's. Currently, it seems as if people who don't want to participate in votes can just not. There is no penalty for not participating, but as a result the vote may be skewed. Therefore, in order to ensure participation, I feel BN's who are chosen to participate in any veto mediation vote must also provide a short opinion on their choice (yes or no). I think it only has to be 2-3 sentences, nothing major, but this is to at least acknowledge that the BN has participated in the veto and isn't just going to blindly vote yes/no to everything and be non-toxic. (Voting no to a veto mediation is NOT being toxic, let's not set any bad precedents).
Randomly selecting BNs was my first thought too though but in that case, we'd need to anonymise it as a fewer number of people is probably more prone to bribery. Also in that case the mapper must have a chance to contest the decision made by the BNs because if you are unlucky and get 7 people selected that all are in favour of the veto despite maybe most of the other BNs being against the veto, the decision would not be representative of the whole group.
Monstrata wrote:
How to get into Management Team?
I'm sure a few people (hi Ascendance) will be interested primarily in this. Is there a process to get into Management Team? And what will happen to other aspects of the mapping/modding community? I'm talking about pushing forward new amendments and changes to the Ranking Criteria mainly. (Which is another subset issue). How will people be chosen for this specialized team?
This has yet to be decided but I think it will probably go into the same direction as QAT promotions in 2017 pre-upheaval where when somebody shows good management skills (e.g. managing different projects well) and good behaviour (as they will most likely have the same permissions as QAT have right now) they will be brought up in the "Management Team" and a majority vote/discussion will happen.
Monstrata wrote:
Also if someone is rejected, what is the cooldown before they can reapply again?
Yeah, this has been an oversight and totally needs to be added. I think 3 months would work well as that's the current length of a cycle but I personally wouldn't mind reducing it either.
Monstrata wrote:
Another note: I feel severity of a kick should be considered too. For example, BN's kicked for missing too many unrankable elements, or are inactive etc... should not be subject to a 3 month cooldown imo. I feel that for unrankable elements, this suggests the BN needs more training, or isn't thorough enough, but 1-2 months should be enough. For inactivity, well, can't a BN just resign if they know they will get kicked for inactivity? Then they would be able to reapply 2 weeks later. On the other end, I think behavior-related kicks could be 3 months. I think having a flat # of months is not necessary, and you guys could change the cooldown to suit the specific case of the BN being removed. Basically, make cooldowns more case-by-case and not necessarily do 3 months for everyone.
The numbers in the proposal are still quite arbitrary and will probably be adjusted when we polish it. I like this idea though!
pimp wrote:
i don't like the idea of giving the DQ button to all nominators
What's the problem with that? The BN rules will be adjusted to forbid any kind of abuse and I believe we can trust BNs with another button, especially since it can't really break anything.
______
We also still need to figure out how to handle probation exactly. I have seen very strong opinions on this from both sides - while some people think it's very important to see if new guys are fit for the role and to give people who messed up another chance, others think it should be replaced by a system like the old three strike system where whenever you have a major messup, you get stricken and once you accumulated three strikes you get removed.
I'm not entirely sure which one of these works best with the proposed systems so I'd like to get some more thoughts on that here too.
Moreover, I wouldn't get too hung up on the score system yet as its direction heavily depends on implementation. I feel like the best way to go about it would be a system where both the amount of dqs, their severity and the BNs activity are calculated into a score and then once a BN drops below a certain threshold, a bot pings the "Management Team" and they'll look closer into that BN's case.
We will also try to keep you updated with the score system when concrete planning has started, i.e. there will most likely be a separate proposal for that in the future.
FrenZ396 wrote:
You guys should add a cooldown between how long people can apply for BN - perhaps two weeks or something. People could just spam requests every other day under this proposal since it's not listed...
edit: Hi Feerum! I didn't reload my page until after I posted this and you mentioned this already.
I believe that was the plan tbh. Should be 3 month cd after getting rejected pretty much.
Anyways, liking this proposal better than the other one
Very much for this proposal. It's basically taking the current system and making it match the reality of things, which is pretty nice to do.
My only gripes are basically entirely covered by nao in this post, especially that first point. That second point would be nice definitely, but u get the same issues with that as you get with QAT enforcing quality standards onto maps, so I don't expect anything to be done about that
My only gripes are basically entirely covered by nao in this post, especially that first point. That second point would be nice definitely, but u get the same issues with that as you get with QAT enforcing quality standards onto maps, so I don't expect anything to be done about that
Second this.
Actually I would suggest that the Management Team should also take responsibilities or initiatives regarding other mapping/modding-wise events to revive them, like Spotlight, Contest Organization, RC Amendment, Education and Mentorship etc. I believe some of these are in a stasis and/or require more hands. It will make the main tasks of the team to be more higher-level, and genuinely "management and leadership-wise". These events would actually, in my eyes, make the team do greater for the entire community than dq'ing over mapsets xd.
Actually I would suggest that the Management Team should also take responsibilities or initiatives regarding other mapping/modding-wise events to revive them, like Spotlight, Contest Organization, RC Amendment, Education and Mentorship etc. I believe some of these are in a stasis and/or require more hands. It will make the main tasks of the team to be more higher-level, and genuinely "management and leadership-wise". These events would actually, in my eyes, make the team do greater for the entire community than dq'ing over mapsets xd.
Are we even back to BAT's era now?
Pog
Mao wrote:
We also still need to figure out how to handle probation exactly.
For starters, I'd like to suggest telling the bns on probation generally what Management will be looking for when they're evaluated. My readmission felt a bit like the QAT was saying "ok you're a bn now but you're also on probation, so yeah gl." There wasn't really much guidance as to how anyone on probation was being evaluated. Were the QAT looking for number of icons? bubble pops or dqs? activity? I got a lot of mixed responses and some rumors from people I asked (bns and like 2 qat i think). For myself I had the reasons from my previous probation failure to focus on improving but I didn't see any clarity as to how our performance was being measured for the other probation bns. This just put pressure on me to be perfect in every aspect of modding, which I know isn't the true expectation of the QAT.
This whole ordeal made probation super super stressful for me, and it made it a bit of a mystery as to why some of the people didn't pass. Probation doesn't need to be stressful or mysterious. We need to write some sort of probation quidelines or something.
A team should solely focus on doing something so that the team can keep maintained. I like the idea that MT(Management Team) "only" focuses on something team-wisely, as we don't want to see there are different branches across the MT like the branch QAT in 2017. Stuffs like Education or Mentorship shouldn't be the responsibilities of the MT. They can also be done by members of MT individually, and these stuffs are mostly community-wise, whoever is interested can participant, with that it also shows their organizational and managerial capabilities which help them easier to join the MT.
Also letting the QAH check objective and unrankable stuffs in qualified is a good point, and I think more weight of BN scores would be given to them based on the quantity they checked(encourage them). Meanwhile, if a map with unrankable issues went to ranked, the QAH who checked would get punished. The QAH is good to become a reward-more and punish-more role, they take the responsibility.
I believe with the new system change, the MT has many things to do at the beginning, like handle conflicts between different BNs and mappers after a DQ, moderation of the map discussion.
Also letting the QAH check objective and unrankable stuffs in qualified is a good point, and I think more weight of BN scores would be given to them based on the quantity they checked(encourage them). Meanwhile, if a map with unrankable issues went to ranked, the QAH who checked would get punished. The QAH is good to become a reward-more and punish-more role, they take the responsibility.
I believe with the new system change, the MT has many things to do at the beginning, like handle conflicts between different BNs and mappers after a DQ, moderation of the map discussion.
The only thing which came to my mind right after I finished reading the draft is the following question:
So, if the tests will stay, how are they going to look like? Or rather, how frequently are you going update them? Because there were cases where people got accepted with others' help (revealing the whole test and the answers and similar) with the test. The only solutions which come to mind is whether you come up with a dynamic system or you update the tests pretty much daily / or you make seperate tests for each individual applicants (or even more than one, given the chance they might fail). The first option would be far more ideal obviously but I can hardly imagine a possible way to do it, the second option is well....time consuming and redundant so it is probably not the way it should go as far as I can tell.
Now, obviously you mention that the "implementation of this system might take some time to prepare" but it is probably worth the time to play with the thoughts before you actually start making this system becoming real.
Also, please keep in mind that I'm fairly outdated with how BNs get promoted nowadays (thus I don't really know how a "proficiency test" really looks like today if those have been changed) so please ignore me if my question didn't make any sense in the current context.
Draft wrote:
Applications for joining the Beatmap Nominators will mostly stay the same but instead of holding them every three months, they will be open at any point in time.
The applications will still be composed of two parts, a ranking criteria proficiency test and a manual evaluation by members of the “Management Team”.
So, if the tests will stay, how are they going to look like? Or rather, how frequently are you going update them? Because there were cases where people got accepted with others' help (revealing the whole test and the answers and similar) with the test. The only solutions which come to mind is whether you come up with a dynamic system or you update the tests pretty much daily / or you make seperate tests for each individual applicants (or even more than one, given the chance they might fail). The first option would be far more ideal obviously but I can hardly imagine a possible way to do it, the second option is well....time consuming and redundant so it is probably not the way it should go as far as I can tell.
Now, obviously you mention that the "implementation of this system might take some time to prepare" but it is probably worth the time to play with the thoughts before you actually start making this system becoming real.
Also, please keep in mind that I'm fairly outdated with how BNs get promoted nowadays (thus I don't really know how a "proficiency test" really looks like today if those have been changed) so please ignore me if my question didn't make any sense in the current context.
UndeadCapulet wrote:
kowai
My biggest concern is ability to DQ as a Beatmap Nominator.
To be entrusted with an ability to disquality beatmaps requires a special kind of trust. Giving such ability to a big group of people sounds like a recipe for disaster without proper safeguard.
Either give BNs ability to mark beatmap for Disqualification but have the actual action done by the Management Team to protect against foul play.
Or in order to DQ a beatmap, it requires multiple BNs to hit the DQ button enough times to prevent a bad apple going berserk on the qualified section.
To be entrusted with an ability to disquality beatmaps requires a special kind of trust. Giving such ability to a big group of people sounds like a recipe for disaster without proper safeguard.
Either give BNs ability to mark beatmap for Disqualification but have the actual action done by the Management Team to protect against foul play.
Or in order to DQ a beatmap, it requires multiple BNs to hit the DQ button enough times to prevent a bad apple going berserk on the qualified section.
all BNs getting DQ ability is pretty risky. I think a different, more trusted group should handle DQs or requests for DQs. Maybe BNs above a score threshold, or the 10 best scoring BNs have the ability to DQ?
or maybe even have a DQ button for all BNs which has to have a reason attached to the request, and trusted members accept or decline the request?
or maybe even have a DQ button for all BNs which has to have a reason attached to the request, and trusted members accept or decline the request?
tbh I don't really see why we need to force a change when one is simply not needed, it seems like someone saw shiten and was like "oh qat sucks lets do something else". This new system is just gonna be the same thing except more annoying to deal with
Aside from that, how is a BN supposed to act on any reports if they aren't for strictly unrankable issues? The only way would be to just go ahead and veto the map if you think the concerns brought up by whoever are valid and just dq the map for that.
That being said I like the idea of BNs having a dq button for unrankables, having to wait for a qat to show up every time is a little stupid lol. But the issue with this is that the dq button would most likely replace the problem button. There's a huge difference between posting something as a suggestion and something as a problem, a lot of people tend to look at suggestions like "oh whatever it's just a suggestion" and only take problems seriously. If this gets implemented we would definitely require an actual dq button instead of just replacing one of the existing ones.
On top of that, having every single BN vote on every single veto is obviously not going to work and I believe I've seen Mao and others already suggest to just randomly pick a select number of BNs for this. While this is a lot better than having everyone vote it still kinda sucks, what if you only pick BNs that hate / love that particular map (guren lol)? In the end I believe the current system we have for this is the best, despite being kinda not so good.
This would also require a clarification as for what would actually give you minus points, what counts as a major issues (small metadata mistakes should never be relevant for example) and all that kind of stuff.
In the end I disagree with almost everything except changing the name of the qat and always open applications. I still don't believe we desperately need to force a change, the current system works as it is.
While I totally agree with changing the name (since lets be real, qat hasn't assured quality in forever), I don't think disbanding it entirely is necessary though.Proposal wrote:
Disband the Quality Assurance Team
A little concerned with this part, not about the potential of abuse or anything but more about the fact that "a map cannot be disqualified for anything but unrankable or objective issues". Might as well say only unrankables since that's the only truly objective issue, nearly everything else is mostly subjective. This also allowes the mapper to simply disregard any dq mods whatsoever without having to even try to explain why they don't want to change things, why explain when you can just say no.Proposal wrote:
Give the Disqualify button to the Beatmap Nominators
Aside from that, how is a BN supposed to act on any reports if they aren't for strictly unrankable issues? The only way would be to just go ahead and veto the map if you think the concerns brought up by whoever are valid and just dq the map for that.
That being said I like the idea of BNs having a dq button for unrankables, having to wait for a qat to show up every time is a little stupid lol. But the issue with this is that the dq button would most likely replace the problem button. There's a huge difference between posting something as a suggestion and something as a problem, a lot of people tend to look at suggestions like "oh whatever it's just a suggestion" and only take problems seriously. If this gets implemented we would definitely require an actual dq button instead of just replacing one of the existing ones.
Uhhh can we not? From what I've heard from a few qats it's already annoying as fuck to deal with, but the thing is if you become qat you kinda sign up for this stuff, forcing this on BNs who really don't give a shit about vetoed map #27 is not the best idea, most of them will probably just pass the map without looking at it because they don't feel evaluating a veto on a map they couldn't care less about.Proposal wrote:
Vetoes will be decided by the Beatmap Nominators
On top of that, having every single BN vote on every single veto is obviously not going to work and I believe I've seen Mao and others already suggest to just randomly pick a select number of BNs for this. While this is a lot better than having everyone vote it still kinda sucks, what if you only pick BNs that hate / love that particular map (guren lol)? In the end I believe the current system we have for this is the best, despite being kinda not so good.
I mean sure but like do we really need this? Not entirely sure if most of the gmt even cares about anything mapping related, I highly doubt most of them will want to moderate map threads, hell even the current qat barely does that (which is why this proposal is a thing btw xD). I'm fairly confident things wouldn't turn out nearly as bad every time if there was some actual moderation for controversial map threads (guren, shiten, etc.) instead of just saying "pls behave" after it's already way too late.Proposal wrote:
Content related moderation will be handled by the Global Moderation Team
I think nao already mentioned something about that, but what this is gonna do is promote nominating the most boring maps you could possibly find. Why nominate a map with 200 redlines / a set with 15 diffs with a lot of potential unrankables / similar if you can just nominate 5 diff anime TV sizes all day long? The risk of fucking up isn't worth it for most people already, and changing things will not help with it. On top of that what if people just nominate subjectively terrible maps all the time, but they don't have any unrankables, would that still make them a good BN? I believe being BN is about more than just checking a set for unrankables (even though that's basically what the job description is but you get the point).Proposal wrote:
A new score system for Beatmap Nominators will be introduced
This would also require a clarification as for what would actually give you minus points, what counts as a major issues (small metadata mistakes should never be relevant for example) and all that kind of stuff.
When will applicants be evaluated though? As soon as they send their application or will there be a set date each month? Will the management team really be able to handle this and not have delayed results? I definitely support this particular point though.Proposal wrote:
Beatmap Nominator Applications will always be open
In the end I disagree with almost everything except changing the name of the qat and always open applications. I still don't believe we desperately need to force a change, the current system works as it is.
The proposal is pretty good, most of the things brought up I really like.
Tho what about bn tests? If you get rejected, how much of time you gonna need to wait till next reapply? I still think we do need to change something in that regards too. Like, judging and so on.
So after you pass a test on RC, the next one is a test of "Whether a judge likes you or not"
Cmon lets get rid of that
And also, in regards of BN apps, can we kinda move from this Three Month stereotype? Here is whats happening with these mods: first 2 months people are just abusing kudosu with low quality mods, then in last month they do good stuff and then they apply.
Can we just move it to 6 weeks just like exactly 1 cycle of BN app?
Tho what about bn tests? If you get rejected, how much of time you gonna need to wait till next reapply? I still think we do need to change something in that regards too. Like, judging and so on.
So after you pass a test on RC, the next one is a test of "Whether a judge likes you or not"
Cmon lets get rid of that
And also, in regards of BN apps, can we kinda move from this Three Month stereotype? Here is whats happening with these mods: first 2 months people are just abusing kudosu with low quality mods, then in last month they do good stuff and then they apply.
Can we just move it to 6 weeks just like exactly 1 cycle of BN app?
Spkz wrote:
-
So, if the tests will stay, how are they going to look like? Or rather, how frequently are you going update them? Because there were cases where people got accepted with others' help (revealing the whole test and the answers and similar) with the test. The only solutions which come to mind is whether you come up with a dynamic system or you update the tests pretty much daily / or you make seperate tests for each individual applicants (or even more than one, given the chance they might fail). The first option would be far more ideal obviously but I can hardly imagine a possible way to do it, the second option is well....time consuming and redundant so it is probably not the way it should go as far as I can tell.
Now, obviously you mention that the "implementation of this system might take some time to prepare" but it is probably worth the time to play with the thoughts before you actually start making this system becoming real.
Also, please keep in mind that I'm fairly outdated with how BNs get promoted nowadays (thus I don't really know how a "proficiency test" really looks like today if those have been changed) so please ignore me if my question didn't make any sense in the current context.
The tests are actually pulled from a pile of a bunch of random questions. So i believe we will just delete / create new questions every so and then?
I feel like if BN's don't want to be involved in vetoes, they shouldn't have to. But then they hold a "non-opinion". Let the people who care about vetoes do the vetoing, and let the other BN"s who also care about vetoes handle the veto mediation.Mordred wrote:
Uhhh can we not? From what I've heard from a few qats it's already annoying as fuck to deal with, but the thing is if you become qat you kinda sign up for this stuff, forcing this on BNs who really don't give a shit about vetoed map #27 is not the best idea, most of them will probably just pass the map without looking at it because they don't feel evaluating a veto on a map they couldn't care less about.Proposal wrote:
Vetoes will be decided by the Beatmap Nominators
On top of that, having every single BN vote on every single veto is obviously not going to work and I believe I've seen Mao and others already suggest to just randomly pick a select number of BNs for this. While this is a lot better than having everyone vote it still kinda sucks, what if you only pick BNs that hate / love that particular map (guren lol)? In the end I believe the current system we have for this is the best, despite being kinda not so good.
If you don't care, it is no loss to you to ignore it imo. If the system becomes such that a lot of maps you think are fine are now being vetoed, then maybe it's time for you to step in and start countering them during mediations.
I don't really get why some of you guys are so concerned about BN's being able to disqualify. You've been able to qualify maps forever. How hard is it to just not hit disqualify unless you see objectively unrankable issues? Involving Management Team for every single DQ requires additional communication that imo is not necessary when you see an obviously offscreen slider, etc... (And these are really the only times you'd DQ without mapper's consent anyways).
The only time a BN can DQ without mapper's consent and without it being unrankable issues, is if the BN is placing a veto. And imo, I feel it should be mandatory for BN's who want to DQ a map for subjective issues to first give notice (24 hours) for mapper to respond. Therefore, I think it is appropriate to say maps can only be DQ'ed by a single BN if the BN plans to veto the map, and the mapper either hasn't responded in 24 hours, or has not given the BN an adequate response in 24 hours. For cases outside of this (past the 5th/6th day), I think the support of 2 BN's is necessary in order to DQ a map on like the 6th or 7th day for subjective issues. (This would basically be two BN's completely negating the nominations of the previous two).
The only time a BN can DQ without mapper's consent and without it being unrankable issues, is if the BN is placing a veto. And imo, I feel it should be mandatory for BN's who want to DQ a map for subjective issues to first give notice (24 hours) for mapper to respond. Therefore, I think it is appropriate to say maps can only be DQ'ed by a single BN if the BN plans to veto the map, and the mapper either hasn't responded in 24 hours, or has not given the BN an adequate response in 24 hours. For cases outside of this (past the 5th/6th day), I think the support of 2 BN's is necessary in order to DQ a map on like the 6th or 7th day for subjective issues. (This would basically be two BN's completely negating the nominations of the previous two).
Monstrata wrote:
I don't really get why some of you guys are so concerned about BN's being able to disqualify. You've been able to qualify maps forever. How hard is it to just not hit disqualify unless you see objectively unrankable issues? Involving Management Team for every single DQ requires additional communication that imo is not necessary when you see an obviously offscreen slider, etc... (And these are really the only times you'd DQ without mapper's consent anyways).
The only time a BN can DQ without mapper's consent and without it being unrankable issues, is if the BN is placing a veto. And imo, I feel it should be mandatory for BN's who want to DQ a map for subjective issues to first give notice (24 hours) for mapper to respond. Therefore, I think it is appropriate to say maps can only be DQ'ed by a single BN if the BN plans to veto the map, and the mapper either hasn't responded in 24 hours, or has not given the BN an adequate response in 24 hours. For cases outside of this (past the 5th/6th day), I think the support of 2 BN's is necessary in order to DQ a map on like the 6th or 7th day for subjective issues. (This would basically be two BN's completely negating the nominations of the previous two).
Additionally if it's clear that you are abusing DQ's then obviously the management team would do something about it?