forum

QAT Restructure Follow-Up

posted
Total Posts
105
show more
celerih
Very much for this proposal. It's basically taking the current system and making it match the reality of things, which is pretty nice to do.

My only gripes are basically entirely covered by nao in this post, especially that first point. That second point would be nice definitely, but u get the same issues with that as you get with QAT enforcing quality standards onto maps, so I don't expect anything to be done about that
Mafumafu
Second this.

Actually I would suggest that the Management Team should also take responsibilities or initiatives regarding other mapping/modding-wise events to revive them, like Spotlight, Contest Organization, RC Amendment, Education and Mentorship etc. I believe some of these are in a stasis and/or require more hands. It will make the main tasks of the team to be more higher-level, and genuinely "management and leadership-wise". These events would actually, in my eyes, make the team do greater for the entire community than dq'ing over mapsets xd.
lit120
Are we even back to BAT's era now?
Fu Xuan
Pog
squirrelpascals

Mao wrote:

We also still need to figure out how to handle probation exactly.


For starters, I'd like to suggest telling the bns on probation generally what Management will be looking for when they're evaluated. My readmission felt a bit like the QAT was saying "ok you're a bn now but you're also on probation, so yeah gl." There wasn't really much guidance as to how anyone on probation was being evaluated. Were the QAT looking for number of icons? bubble pops or dqs? activity? I got a lot of mixed responses and some rumors from people I asked (bns and like 2 qat i think). For myself I had the reasons from my previous probation failure to focus on improving but I didn't see any clarity as to how our performance was being measured for the other probation bns. This just put pressure on me to be perfect in every aspect of modding, which I know isn't the true expectation of the QAT.

This whole ordeal made probation super super stressful for me, and it made it a bit of a mystery as to why some of the people didn't pass. Probation doesn't need to be stressful or mysterious. We need to write some sort of probation quidelines or something.
Fycho
A team should solely focus on doing something so that the team can keep maintained. I like the idea that MT(Management Team) "only" focuses on something team-wisely, as we don't want to see there are different branches across the MT like the branch QAT in 2017. Stuffs like Education or Mentorship shouldn't be the responsibilities of the MT. They can also be done by members of MT individually, and these stuffs are mostly community-wise, whoever is interested can participant, with that it also shows their organizational and managerial capabilities which help them easier to join the MT.

Also letting the QAH check objective and unrankable stuffs in qualified is a good point, and I think more weight of BN scores would be given to them based on the quantity they checked(encourage them). Meanwhile, if a map with unrankable issues went to ranked, the QAH who checked would get punished. The QAH is good to become a reward-more and punish-more role, they take the responsibility.

I believe with the new system change, the MT has many things to do at the beginning, like handle conflicts between different BNs and mappers after a DQ, moderation of the map discussion.
Spkz
The only thing which came to my mind right after I finished reading the draft is the following question:

Draft wrote:

Applications for joining the Beatmap Nominators will mostly stay the same but instead of holding them every three months, they will be open at any point in time.

The applications will still be composed of two parts, a ranking criteria proficiency test and a manual evaluation by members of the “Management Team”.


So, if the tests will stay, how are they going to look like? Or rather, how frequently are you going update them? Because there were cases where people got accepted with others' help (revealing the whole test and the answers and similar) with the test. The only solutions which come to mind is whether you come up with a dynamic system or you update the tests pretty much daily / or you make seperate tests for each individual applicants (or even more than one, given the chance they might fail). The first option would be far more ideal obviously but I can hardly imagine a possible way to do it, the second option is well....time consuming and redundant so it is probably not the way it should go as far as I can tell.

Now, obviously you mention that the "implementation of this system might take some time to prepare" but it is probably worth the time to play with the thoughts before you actually start making this system becoming real.

Also, please keep in mind that I'm fairly outdated with how BNs get promoted nowadays (thus I don't really know how a "proficiency test" really looks like today if those have been changed) so please ignore me if my question didn't make any sense in the current context.
Yugu

UndeadCapulet wrote:

kowai
timemon
My biggest concern is ability to DQ as a Beatmap Nominator.
To be entrusted with an ability to disquality beatmaps requires a special kind of trust. Giving such ability to a big group of people sounds like a recipe for disaster without proper safeguard.

Either give BNs ability to mark beatmap for Disqualification but have the actual action done by the Management Team to protect against foul play.

Or in order to DQ a beatmap, it requires multiple BNs to hit the DQ button enough times to prevent a bad apple going berserk on the qualified section.
camellirite
all BNs getting DQ ability is pretty risky. I think a different, more trusted group should handle DQs or requests for DQs. Maybe BNs above a score threshold, or the 10 best scoring BNs have the ability to DQ?

or maybe even have a DQ button for all BNs which has to have a reason attached to the request, and trusted members accept or decline the request?
Mordred
tbh I don't really see why we need to force a change when one is simply not needed, it seems like someone saw shiten and was like "oh qat sucks lets do something else". This new system is just gonna be the same thing except more annoying to deal with

Proposal wrote:

Disband the Quality Assurance Team
While I totally agree with changing the name (since lets be real, qat hasn't assured quality in forever), I don't think disbanding it entirely is necessary though.

Proposal wrote:

Give the Disqualify button to the Beatmap Nominators
A little concerned with this part, not about the potential of abuse or anything but more about the fact that "a map cannot be disqualified for anything but unrankable or objective issues". Might as well say only unrankables since that's the only truly objective issue, nearly everything else is mostly subjective. This also allowes the mapper to simply disregard any dq mods whatsoever without having to even try to explain why they don't want to change things, why explain when you can just say no.
Aside from that, how is a BN supposed to act on any reports if they aren't for strictly unrankable issues? The only way would be to just go ahead and veto the map if you think the concerns brought up by whoever are valid and just dq the map for that.

That being said I like the idea of BNs having a dq button for unrankables, having to wait for a qat to show up every time is a little stupid lol. But the issue with this is that the dq button would most likely replace the problem button. There's a huge difference between posting something as a suggestion and something as a problem, a lot of people tend to look at suggestions like "oh whatever it's just a suggestion" and only take problems seriously. If this gets implemented we would definitely require an actual dq button instead of just replacing one of the existing ones.

Proposal wrote:

Vetoes will be decided by the Beatmap Nominators
Uhhh can we not? From what I've heard from a few qats it's already annoying as fuck to deal with, but the thing is if you become qat you kinda sign up for this stuff, forcing this on BNs who really don't give a shit about vetoed map #27 is not the best idea, most of them will probably just pass the map without looking at it because they don't feel evaluating a veto on a map they couldn't care less about.
On top of that, having every single BN vote on every single veto is obviously not going to work and I believe I've seen Mao and others already suggest to just randomly pick a select number of BNs for this. While this is a lot better than having everyone vote it still kinda sucks, what if you only pick BNs that hate / love that particular map (guren lol)? In the end I believe the current system we have for this is the best, despite being kinda not so good.

Proposal wrote:

Content related moderation will be handled by the Global Moderation Team
I mean sure but like do we really need this? Not entirely sure if most of the gmt even cares about anything mapping related, I highly doubt most of them will want to moderate map threads, hell even the current qat barely does that (which is why this proposal is a thing btw xD). I'm fairly confident things wouldn't turn out nearly as bad every time if there was some actual moderation for controversial map threads (guren, shiten, etc.) instead of just saying "pls behave" after it's already way too late.

Proposal wrote:

A new score system for Beatmap Nominators will be introduced
I think nao already mentioned something about that, but what this is gonna do is promote nominating the most boring maps you could possibly find. Why nominate a map with 200 redlines / a set with 15 diffs with a lot of potential unrankables / similar if you can just nominate 5 diff anime TV sizes all day long? The risk of fucking up isn't worth it for most people already, and changing things will not help with it. On top of that what if people just nominate subjectively terrible maps all the time, but they don't have any unrankables, would that still make them a good BN? I believe being BN is about more than just checking a set for unrankables (even though that's basically what the job description is but you get the point).
This would also require a clarification as for what would actually give you minus points, what counts as a major issues (small metadata mistakes should never be relevant for example) and all that kind of stuff.

Proposal wrote:

Beatmap Nominator Applications will always be open
When will applicants be evaluated though? As soon as they send their application or will there be a set date each month? Will the management team really be able to handle this and not have delayed results? I definitely support this particular point though.





In the end I disagree with almost everything except changing the name of the qat and always open applications. I still don't believe we desperately need to force a change, the current system works as it is.
hi-mei
The proposal is pretty good, most of the things brought up I really like.

Tho what about bn tests? If you get rejected, how much of time you gonna need to wait till next reapply? I still think we do need to change something in that regards too. Like, judging and so on.

So after you pass a test on RC, the next one is a test of "Whether a judge likes you or not"
Cmon lets get rid of that

And also, in regards of BN apps, can we kinda move from this Three Month stereotype? Here is whats happening with these mods: first 2 months people are just abusing kudosu with low quality mods, then in last month they do good stuff and then they apply.

Can we just move it to 6 weeks just like exactly 1 cycle of BN app?
Kibbleru

Spkz wrote:

-
So, if the tests will stay, how are they going to look like? Or rather, how frequently are you going update them? Because there were cases where people got accepted with others' help (revealing the whole test and the answers and similar) with the test. The only solutions which come to mind is whether you come up with a dynamic system or you update the tests pretty much daily / or you make seperate tests for each individual applicants (or even more than one, given the chance they might fail). The first option would be far more ideal obviously but I can hardly imagine a possible way to do it, the second option is well....time consuming and redundant so it is probably not the way it should go as far as I can tell.

Now, obviously you mention that the "implementation of this system might take some time to prepare" but it is probably worth the time to play with the thoughts before you actually start making this system becoming real.

Also, please keep in mind that I'm fairly outdated with how BNs get promoted nowadays (thus I don't really know how a "proficiency test" really looks like today if those have been changed) so please ignore me if my question didn't make any sense in the current context.



The tests are actually pulled from a pile of a bunch of random questions. So i believe we will just delete / create new questions every so and then?
Monstrata

Mordred wrote:

Proposal wrote:

Vetoes will be decided by the Beatmap Nominators
Uhhh can we not? From what I've heard from a few qats it's already annoying as fuck to deal with, but the thing is if you become qat you kinda sign up for this stuff, forcing this on BNs who really don't give a shit about vetoed map #27 is not the best idea, most of them will probably just pass the map without looking at it because they don't feel evaluating a veto on a map they couldn't care less about.
On top of that, having every single BN vote on every single veto is obviously not going to work and I believe I've seen Mao and others already suggest to just randomly pick a select number of BNs for this. While this is a lot better than having everyone vote it still kinda sucks, what if you only pick BNs that hate / love that particular map (guren lol)? In the end I believe the current system we have for this is the best, despite being kinda not so good.
I feel like if BN's don't want to be involved in vetoes, they shouldn't have to. But then they hold a "non-opinion". Let the people who care about vetoes do the vetoing, and let the other BN"s who also care about vetoes handle the veto mediation.

If you don't care, it is no loss to you to ignore it imo. If the system becomes such that a lot of maps you think are fine are now being vetoed, then maybe it's time for you to step in and start countering them during mediations.
Monstrata
I don't really get why some of you guys are so concerned about BN's being able to disqualify. You've been able to qualify maps forever. How hard is it to just not hit disqualify unless you see objectively unrankable issues? Involving Management Team for every single DQ requires additional communication that imo is not necessary when you see an obviously offscreen slider, etc... (And these are really the only times you'd DQ without mapper's consent anyways).

The only time a BN can DQ without mapper's consent and without it being unrankable issues, is if the BN is placing a veto. And imo, I feel it should be mandatory for BN's who want to DQ a map for subjective issues to first give notice (24 hours) for mapper to respond. Therefore, I think it is appropriate to say maps can only be DQ'ed by a single BN if the BN plans to veto the map, and the mapper either hasn't responded in 24 hours, or has not given the BN an adequate response in 24 hours. For cases outside of this (past the 5th/6th day), I think the support of 2 BN's is necessary in order to DQ a map on like the 6th or 7th day for subjective issues. (This would basically be two BN's completely negating the nominations of the previous two).
Kibbleru

Monstrata wrote:

I don't really get why some of you guys are so concerned about BN's being able to disqualify. You've been able to qualify maps forever. How hard is it to just not hit disqualify unless you see objectively unrankable issues? Involving Management Team for every single DQ requires additional communication that imo is not necessary when you see an obviously offscreen slider, etc... (And these are really the only times you'd DQ without mapper's consent anyways).

The only time a BN can DQ without mapper's consent and without it being unrankable issues, is if the BN is placing a veto. And imo, I feel it should be mandatory for BN's who want to DQ a map for subjective issues to first give notice (24 hours) for mapper to respond. Therefore, I think it is appropriate to say maps can only be DQ'ed by a single BN if the BN plans to veto the map, and the mapper either hasn't responded in 24 hours, or has not given the BN an adequate response in 24 hours. For cases outside of this (past the 5th/6th day), I think the support of 2 BN's is necessary in order to DQ a map on like the 6th or 7th day for subjective issues. (This would basically be two BN's completely negating the nominations of the previous two).


Additionally if it's clear that you are abusing DQ's then obviously the management team would do something about it?
Monstrata
@kibb Yep, exactly. I feel like it's not too hard to implement a 1-dq per day limit into BN functions too (similar to 3 qualifications per day etc...) so we don't have CDFA v3 lol.
Kibbleru
The one thing i want to avoid is a scoring system.

As someone who had first hand experience with the scoring system, having that number out there which determined your outcomes just put pressure on us.

While i do think a scoring system is useful for the evaluation of BNs, it's probably better to keep it private, so BNs wouldn't feel pressured by this number. The management team will simply use that scoring system for judgement of said BN.


Monstrata wrote:

@kibb Yep, exactly. I feel like it's not too hard to implement a 1-dq per day limit into BN functions too (similar to 3 qualifications per day etc...) so we don't have CDFA v3 lol.


I'm honestly not too concerned about cases like these, since they happen quite rarely, and are quite easily revertable, there isn't much damage one can do by chain dqing a bunch of maps anymore anyways since the timer doesn't reset.

And honestly, yolo qualifying a bunch of maps is something you can absolutely do right now, yet i don't see anyone doing it. So what's the problem?
Aurele

Kibbleru wrote:

The one thing i want to avoid is a scoring system.

As someone who had first hand experience with the scoring system, having that number out there which determined your outcomes just put pressure on us.

While i do think a scoring system is useful for the evaluation of BNs, it's probably better to keep it private, so BNs wouldn't feel pressured by this number. The management team will simply use that scoring system for judgement of said BN.


I am totally seconding this, as a victim of the scoring system back then, it is very pressuring for nothing.
Ephemeral
any sort of progression metric will be kept private and use for application analysis only, yes. people won't be outwardly judged by their numbers, that's just stupid.

also helps to address the elephant-in-the-room of people gaming the metric for easy BN ship if they have no idea how it works :D
squirrelpascals

Kibbleru wrote:

While i do think a scoring system is useful for the evaluation of BNs, it's probably better to keep it private, so BNs wouldn't feel pressured by this number. The management team will simply use that scoring system for judgement of said BN.


This sounds like it might be unpopular, but just knowing a number exists in the first place will put pressure on me, and I feel like not knowing what it is would only make that feeling worse. I would know that there is some sort of value that determines how i rank up against other bns, but assuming that we're not able to know what value that is (because it's kept secret), I won't know what exactly I'm able to do to perform better, forcing me to be perfect in every aespect (which isn't realistic). This sounds very similar to my experience that I explained on probation here.

I think a good middle ground for this would be making the viewing of the score sheet optional, because I imagine everyone will feel differently about being able to see their score :p So if a bn wants to see their score, they can request an invite to a google sheet that shows it or whatever, but if not that's ok too.
Kibbleru

squirrelpascals wrote:

Kibbleru wrote:

While i do think a scoring system is useful for the evaluation of BNs, it's probably better to keep it private, so BNs wouldn't feel pressured by this number. The management team will simply use that scoring system for judgement of said BN.


This sounds like it might be unpopular, but just knowing a number exists in the first place will put pressure on me, and I feel like not knowing what it is would only make that feeling worse. I would know that there is some sort of value that determines how i rank up against other bns, but assuming that we're not able to know what value that is (because it's kept secret), I won't know what exactly I'm able to do to perform better, forcing me to be perfect in every aespect (which isn't realistic). This sounds very similar to my experience that I explained on probation here.

I think a good middle ground for this would be making the viewing of the score sheet optional, because I imagine everyone will feel differently about being able to see their score :p So if a bn wants to see their score, they can request an invite to a google sheet that shows it or whatever, but if not that's ok too.



Well, believe it or not, a score exists already (sorta), it's just the number of nominations, dq's, etc. you get. almost everyone is judged on a case by case basis, we look carefully into the reasons and circumstances, that "score" is only used as a guideline.



squirrelpascals wrote:

Mao wrote:

We also still need to figure out how to handle probation exactly.


For starters, I'd like to suggest telling the bns on probation generally what Management will be looking for when they're evaluated. My readmission felt a bit like the QAT was saying "ok you're a bn now but you're also on probation, so yeah gl." There wasn't really much guidance as to how anyone on probation was being evaluated. Were the QAT looking for number of icons? bubble pops or dqs? activity? I got a lot of mixed responses and some rumors from people I asked (bns and like 2 qat i think). For myself I had the reasons from my previous probation failure to focus on improving but I didn't see any clarity as to how our performance was being measured for the other probation bns. This just put pressure on me to be perfect in every aspect of modding, which I know isn't the true expectation of the QAT.

This whole ordeal made probation super super stressful for me, and it made it a bit of a mystery as to why some of the people didn't pass. Probation doesn't need to be stressful or mysterious. We need to write some sort of probation quidelines or something.


Interesting point, I've actually never thought this was an issue since I assumed people knew that they should just not get any dq's and it'll be fine
Monstrata

squirrelpascals wrote:

Kibbleru wrote:

While i do think a scoring system is useful for the evaluation of BNs, it's probably better to keep it private, so BNs wouldn't feel pressured by this number. The management team will simply use that scoring system for judgement of said BN.
This sounds like it might be unpopular, but just knowing a number exists in the first place will put pressure on me, and I feel like not knowing what it is would only make that feeling worse. I would know that there is some sort of value that determines how i rank up against other bns, but assuming that we're not able to know what value that is (because it's kept secret), I won't know what exactly I'm able to do to perform better, forcing me to be perfect in every aespect (which isn't realistic). This sounds very similar to my experience that I explained on probation here.

I think a good middle ground for this would be making the viewing of the score sheet optional, because I imagine everyone will feel differently about being able to see their score :p So if a bn wants to see their score, they can request an invite to a google sheet that shows it or whatever, but if not that's ok too.
Scoring has always existed no?

I feel like it would be helpful for QAT management team to let BN's know if they are in danger of not meeting a certain internal scoring threshold, or are in danger of being put in probation/kicked. From what I'm seeing with the rework, Management Team will have far less work to do now, since anything DQ and veto related is no longer their responsibility, so having the extra task of letting BN's know they need to step up their work would be a good new task to give them I think.
Lumenite-
i've skimmed the thread, but i may have missed it-allow me to voice some concerns in a really, really nerdy way (and let me make clear i speak for taiko and taiko only)

i believe this proposal in giving BNs the DQ button and forcing all nominators to vote on a veto will result in a majoritarian democracy and tyranny of the majority in which if you find yourself in the minority over any subjective issue, you're tough out of luck. perhaps it may be a bit pessimisstic and paranoid, however i think this will create a war of pulling in your friends that agree with you into the BNG so they can agree with you and you can win vetoes.

i firmly believe that subjective more complicated issues that may or may not warrant a DQ should be left in the hands of those who are experienced in their knowledge. i can get behind all BNs allowed to DQ unrankable issues to keep things moving, however subjective things should be left to those who are experienced.

i think a good talking point to prevent the tyranny of the majority is to require a supermajority-or 2/3 vote-in order for a veto to be sustained or overruled. along with that, assuming probation would still exist, probationary BNs should only be allowed to DQ maps with unrankable issues, not subjective ones. those who have shown questionable judgment or are new to the BNG don't have the experience necessary to mediate in complicated issues, in my opinion.

if anyone has expressed these ideas or if there is a need for clarification, pls be nice to me c:
Hydria

Kibbleru wrote:

Interesting point, I've actually never thought this was an issue since I assumed people knew that they should just not get any dq's and it'll be fine


Nominating two generic maps with 0 DQs in probation is a lot different than Nominating 6 maps, 5 of which are unique, with 1-2 DQs in probation. You punish the one that puts in the extra work.

This system will be similar to the old Beatmap Nominator Rankings back in 2014 and 2015. However, instead of focusing on subjective issues, this system will only account for unrankable and objective issues.


I get that you're trying to avoid penalising mapsets that are controversial, but at the end of the day this change will still hurt those BNs, as you're objectively more likely to miss an objective mistake in something that's unique and controversial (or in a new mapper's first ranked set) than you are in someone that's profound in mapping pushing another 1-2 jump map.
Regardless of if BN removals are being judged on a case-by-case basis, this change will still deter BNs from going for controversial mapsets. People don't like being watched and judged with a number.
It's better to track someone's progress based on overall performance and not just objective dqs, even subjective dqs can need punishment if they're unwarranted and highly disagreed upon. Have people there that is qualified to track community progress and shifts to make the call on dq influence since they will always be more than a number.
Kibbleru

Hydria wrote:

Nominating two generic maps with 0 DQs in probation is a lot different than Nominating 6 maps, 5 of which are unique, with 1-2 DQs in probation. You punish the one that puts in the extra work.


We do take into account the number of nominations being made by said bn compared to the number of DQs they get.

For the controversial factor, honestly it's a bit hard to get a good measure of. Perhaps it could be some factor of total drain time and star rating?

I know star rating is not the best for judging controversy, but usually the harder the map, the more likely it would have problems.
Nao Tomori
or you could not rely on a quantitative system to measure qualitative issues? lmao
Kibbleru

Nao Tomori wrote:

or you could not rely on a quantitative system to measure qualitative issues? lmao


It's not like we completely rely on numbers? They're just helpful as a metric of judgement.
Aiseca

Kibbleru wrote:

We do take into account the number of nominations being made by said bn compared to the number of DQs they get.


- How about accounting more on severity rather than just numbers?

Kibbleru wrote:

For the controversial factor, honestly it's a bit hard to get a good measure of. Perhaps it could be some factor of total drain time and star rating?


-No, this cannot be used on several cases. On gameplay related cases, SR might be an indicator, but this working as a measure for "controversial"..... It just won't work..

-Try using that as metric when the issue isn't even influencing SR nor drain time but still made controversy, you will likely hit a brick wall on that explaining.


Kibbleru wrote:

I know star rating is not the best for judging controversy, but usually the harder the map, the more likely it would have problems.


-I think the controversy won't even happen in the first place if the controversy itself hasn't been dealt with an equally be controversial decision, ne?
Lasse

Nao Tomori wrote:

second, and more importantly: unrankables and objective issues are not actually important or a good representation of a BN's skill. for example: BN A consistently pushes maps which have complex timing and snappings, which occasionally get DQ'd for some adjustments to timing or snapping. BN B exclusively nominates single bpm NHI 1/2 based anime maps which never get DQ'd.
BN A is at much, much, MUCH higher risk of getting kicked than BN B despite nominating more diverse or interesting content than the usual fare, and neither of the two maps getting DQ'd for being bad maps. Only counting objective DQ's does not mean that people can nominate risky maps - it just changes the definition of risky from "uses weird techniques" to "has an unquantized mp3" and doesn't address the core issue of people only wanting to nominate single bpm 1/2 based anime maps to avoid getting kicked for timing DQ's.
This is a fundamental problem with how QATs gave up on caring about map quality and only focusing on unrankables - unrankable issues are not major problems for the most part. A wrongly snapped slider takes a grand total of 2 seconds to fix. A 10ms red line adjustment is not a major problem that causes a good map to become terrible. Meanwhile people nominate all forms of complete and utter trash (Uta intro ver anyone?) which don't have unrankables and therefore suddenly they are fine.
I'm not saying to magically start giving a shit about mapping quality but this idea that a BN getting DQ's for timing is worse than a BN not getting DQ's because they nominate rankable trash is really stupid and shouldn't be in the score system. The score system should just be activity and SERIOUS unrankable issues (entire sections missing hitsoundings, large amounts of unsnaps due to fucked up green lines, 20+ms offset issues, etc.)

agree with this so much, and it's one of the reasons why I tend to be a bit more lenient in bn reevals/probation evals when it comes to minor unrankables, especially stuff like wrong snaps on songs with weird snapping etc.



now to the proposal itself:

proposal wrote:

Disband the Quality Assurance Team
- (...) To fix this issue, we want to disband the QAT and replace it by a yet to be named “Management Team” that only focuses on promoting and evaluating Beatmap Nominators as well as making sure that the system runs smoothly.

Seems more like a rename and removal of the "qualified map checking", considering many of the QAT right now mainly do paperwork/bn management stuff and maybe some veto mediation, but don't even come close to checking the amount of qualified maps each month they are supposed to.

also how is this management team going to be decided regarding who gets added to it later etc. ?



proposal wrote:

Give the Disqualify button to the Beatmap Nominators

Not sure how to think about this considering there are people that can barely be trusted with the qualify button

proposal wrote:

The Beatmap Nominator rules will be updated so that a map may only be disqualified if unrankable or objective issues are present, the mapper requests it, or in the event of a veto.

veto part here really needs to be highlighted, since I've seen people see this as a "maps can only be DQd for objective issues now" when it's not the case, since BNs can now also veto qualified maps. I think this makes some sense overall, considering how most DQs happen right now



proposal wrote:

Vetoes will be decided by the Beatmap Nominators - initiate a majority vote among all BNs (except for the one who placed the veto) of the respective mode.
sounds good at first but has one major issue: a lot of BNs don't care about vetoes at all, they joined the BNG to push forward maps they like, not discuss vetoes on some random anime map. should only involve a part of the BNs (randomly selected group, volunteer group similar to QAH right now, idk). Point is, forcing ALL BNs do deal with every single veto seems pretty overdone and will also results in people either just not caring about their votes on a veto, or leaving BNG altogether because that's just not what they joined it for.



proposal wrote:

Content related moderation will be handled by the Global Moderation Team

yes, good change. This probably should have happened a long time ago



proposal wrote:

A new score system for Beatmap Nominators will be introduced
again? Didn't work before and probably won't work now unless it gets heavily adjusted (see first quote)



proposal wrote:

Beatmap Nominator Applications will always be open

good, just needs more details. I supposed the applications will be evaluated right away? daily or weekly? biweekly? ???



proposal wrote:

If you leave the Beatmap Nominators on your own merit though, you will be allowed to re-join within the next two weeks after your removal.

this should be a month imo, so people don't change their mind all the time and constantly leave and rejoin. also this point opens a loophole. consider for example a bn messing up a lot of nominations and expecting to get kicked for that. with this system they could instead just leave the bng and reapply two weeks later. Might make more sense to use a one or two month cooldown before you can reapply regardless of the reason you were removed. this timeframe would also make it pretty similar to current situation, since in both cases they usually just need to wait until the next round of bn apps, which happen every few months.

Furthermore, there also needs to be a cooldown after failing your application, so people don't just spam apply all the time (same timeframe as ^?)



think that's all I have to comment for now, but might have forgotten something
Venix
i think it might work better if you just keep bng as it is, make bigger and easier accesible qat (picked from active qahs for example) with nominate and dq button but without moderation privileges and management team with current qat permissions (but without responsibility to check all qualfiieds) which would manage it all

i feel dqing maps is more demanding responsibility than nominating them (i also think there's much people would like to join bng to just nominate maps) so would be better to keep more control on it along with team which would be able to discuss vetos etc. instead of suddenly giving like 50 people dq button

also, i don't really see any radical changes being made with this proposal in practice (exlcuding rolling bn apps and scoring system, but it's not related to qat itself), giving qat more of a free hand in terms of dq would result very similarly without all of this chaos happening i guess

also, while reading it consider that im not inside this system so i might see things working a bit differently than they really do
yaspo
Vetoes will be decided by the Beatmap Nominators through majority vote

Something I feel is missing here, is the step from the result of a majority vote to an actual post on the map thread. One good thing in the current system is that QAT can write a post that is tailored to the situation at hand, being able to pinpoint what a map's issues are and giving specific suggestions. With a majority vote I don't really see this happening.
It should be obvious that "this is the voting result, please comply with the modder" doesn't work, it's hardly "mediation" at that point and is too impersonal.
The new Management Team won't or shouldn't write a post, it seems like they are supposed to keep themselves uninvolved for vetoes. Having them write about a discussion and decision they didn't take part in seems kinda wrong?
That leaves the BNG, you could pick representatives of the winning side to deal with it. That however, is tricky when voting is anonymous since it might lead to the representatives losing their anonymity. Non-anonymous voting solves this, but honestly, I'm kinda scared of all the potential social dynamics that come with having statistically opposing parties within the same usergroup.
Other than that I agree with Lasse; voting works when everyone wants to give an input, not when everyone "should".

I'm also wary of the dq change. Generally permissions and responsibilities are distributed among people you trust with them, you trust that they don't have malicious intent -and- they are capable of making correct use of them. So, giving BNs the option to dq, as well as veto voting and a lack of QAT to deal with things for them is a huge expansion on those responsibilities, but they're still the same people. It seems very likely that mistakes will happen. It'll be very sad if this ends up having a reductionary effect on the BNG, or even another split into separate roles.

Also agree with Nao on the scoring system. You're not really encouraging anything here anyway, you're just not discouraging it. It's also the case that, whenever numbers get involved, people will always try to give a meaning to them, even if you only use them as a guideline.

The rest seems a bit up in the air. Disbanding QAT doesn't really remove the concept of quality, people might as well go yell at BNs for not vetoing maps they dislike. "Lack of Quality control in general" would be the easiest criticism against this.
Overall this proposal doesn't seem like it addresses the root of the problem and will likely be followed with another proposal in the near-distant future. The cycle repeats.
Ashton
May we also allow edit access to the proposal in google docs? It will allow for easier conversation as people will be able to directly comment on specific areas of the proposal as well as make it easier to have compiled information rather than a bunch of posts on a thread.
Uta

Lasse wrote:

sounds good at first but has one major issue: a lot of BNs don't care about vetoes at all, they joined the BNG to push forward maps they like, not discuss vetoes on some random anime map. should only involve a part of the BNs (randomly selected group, volunteer group similar to QAH right now, idk). Point is, forcing ALL BNs do deal with every single veto seems pretty overdone and will also results in people either just not caring about their votes on a veto, or leaving BNG altogether because that's just not what they joined it for.
This, I agree.

Nao Tomori wrote:

second, and more importantly: unrankables and objective issues are not actually important or a good representation of a BN's skill. for example: BN A consistently pushes maps which have complex timing and snappings, which occasionally get DQ'd for some adjustments to timing or snapping. BN B exclusively nominates single bpm NHI 1/2 based anime maps which never get DQ'd.
BN A is at much, much, MUCH higher risk of getting kicked than BN B despite nominating more diverse or interesting content than the usual fare, and neither of the two maps getting DQ'd for being bad maps. Only counting objective DQ's does not mean that people can nominate risky maps - it just changes the definition of risky from "uses weird techniques" to "has an unquantized mp3" and doesn't address the core issue of people only wanting to nominate single bpm 1/2 based anime maps to avoid getting kicked for timing DQ's.
This is a fundamental problem with how QATs gave up on caring about map quality and only focusing on unrankables - unrankable issues are not major problems for the most part. A wrongly snapped slider takes a grand total of 2 seconds to fix. A 10ms red line adjustment is not a major problem that causes a good map to become terrible. Meanwhile people nominate all forms of complete and utter trash (Uta intro ver anyone?) which don't have unrankables and therefore suddenly they are fine.
I'm not saying to magically start giving a shit about mapping quality but this idea that a BN getting DQ's for timing is worse than a BN not getting DQ's because they nominate rankable trash is really stupid and shouldn't be in the score system. The score system should just be activity and SERIOUS unrankable issues (entire sections missing hitsoundings, large amounts of unsnaps due to fucked up green lines, 20+ms offset issues, etc.)
I saw alot of my friends having good intention to nominate complex timing and snappings. But they all got punished by how many dqs they got, ofcourse there will be a lot of dqs, and I think that's inevitable. For starter, maybe reduce the punishment of these because imo many bns especially new ones are getting push to nominate stupid 1-2 jumps etc etc,
Hydria
Beatmap Nominator Applications will always be open


Ok lets handle this topic for a second.

The applications will still be composed of two parts, a ranking criteria proficiency test and a manual evaluation by members of the “Management Team”.


So, inevitably, the answers for the ranking criteria proficiency test will be leaked, secretly ofc, because the questions won't be unique (you haven't got that powerful of a team to do so) so they will be known to pretty much everyone applying, whilst the staff have been unaware for lets say 2 weeks - 2 months at the worst level of ignorance.

What happens then?

Ofc you can't allow any more people to take this test because it's no longer a test if they have the answers literally handed to them.

You really only have one choice, which is to close off the BN tests to re-write the questions in a way so that users can't just sail through in 10 min...but how long will this last until you're back in the same situation? Less than a month most likely.

There's a reason why no tests across the world are open 24/7, because it doesn't work. Sure, maybe having them open more frequently might be better, but always open isn't going to work. Also, that's putting a lot more stress on BN-checkers than they already have to deal with.
Nao Tomori
to be completely honest i dont think that even if the answers to the rc test are leaked it is a huge issue for 2 reasons:
1. the test itself is ridiculously easy
2. people who cheated to get in would presumably fail probation for bubbling a bunch of unrankables that they didnt know about
Hydria

Nao Tomori wrote:

to be completely honest i dont think that even if the answers to the rc test are leaked it is a huge issue for 2 reasons:
1. the test itself is ridiculously easy
2. people who cheated to get in would presumably fail probation for bubbling a bunch of unrankables that they didnt know about


then what's the point of the test in the first place
clayton
the "RC test" is basically an open-book test since you will be referring to the RC as you take it. I think the point is just to make sure that the applicant can interpret it correctly--- in this case, even if they know the answers before-hand, it doesn't really matter, since they'll still be made aware of the correct information as they answer the questions

cheating here just turns the test into a lecture. I don't think it's really a big deal if anyone cheats on it
Uta

Nao Tomori wrote:

to be completely honest i dont think that even if the answers to the rc test are leaked it is a huge issue for 2 reasons:
1. the test itself is ridiculously easy
2. people who cheated to get in would presumably fail probation for bubbling a bunch of unrankables that they didnt know about

I think there is a reason for it to be easy. Well~ it's not really easy imo, more like tricky. especially for those whose just got into a process of being familiar with the RC test. Easy unrankables is the issue that BNs will mostly find throughtout their modding carrier. They can still learn while being a bn, myself have found some problems that I've never seen when I'm not a BN and I'm abit aware of that issues in the future. The process of knowing hard to find unrankables is still possible. Moreover, it happens rarely and only happens in complex maps. So if they are doing hard maps to nominate, they will know whats coming.


and yes, don't cheat please. if you cant pass an easy test like this. you pretty much deserve'nt becoming a BN.
Aiseca

Uta wrote:

I think there is a reason for it to be easy. Well~ it's not really easy imo, more like tricky. especially for those whose just got into a process of being familiar with the RC test. Easy unrankables is the issue that BNs will mostly find throughtout their modding carrier. They can still learn while being a bn, myself have found some problems that I've never seen when I'm not a BN and I'm abit aware of that issues in the future. The process of knowing hard to find unrankables is still possible. Moreover, it happens rarely and only happens in complex maps. So if they are doing hard maps to nominate, they will know whats coming.

and yes, don't cheat please. if you cant pass an easy test like this. you pretty much deserve'nt becoming a BN.

Yup, it's more tricky than being easy.

Also, even if someone tried to cheat and got through, the way they'll perform thier duties will reflect on how they really are fit to be a BN or not.

Learning more about things is a natural process, the only thing that differs is the growth pattern of each individual. Cheating will take you somewhere; but not that far enough to cut through.

There's more to learn about, but not written; experience will tech you those instead along the way - and that cannot be cheated.
show more
Please sign in to reply.

New reply