While I somewhat agree with you, hi-mei, maybe you could have left names out.
abraker wrote:
People are speaking about controversial maps and the QAT's failure in handling them appropriately. Can we identify the maps and what was controversial about them so we have something to go off of? There can be controversy in context of abusing the star rating or pp, but there can also be controversy in context of mapping and design choices or some other context. Labeling them all under one term "controversial" does not paint the full picture.
Consider some issues will be unresolved indefinitely and perhaps it's the mapper's goal to keep the issues resolved indefinitely to at least retain some sort of status for the map. In other words, keeping a map qualified indefinitely will make the qualified section equivalent to loved. If consensus can't be reached within a reasonable amount of time, I suggest it go to loved instead of being kept in qualified.pimp wrote:
and if a map is considered controversial for whatever reason, it should stay on qualified section for indefinitive ammount of time instead of getting ranked by default (unless an unrankable issue was found, so the disqualify should be done imediately), this can also be used in case minor issues are found on any map, just don't let the map get ranked by default if the mapper is not around to respond to concerns, keep it qualified instead until the mapper is able to address issues.
Wait why do you say the result wouldn't be any different with any of the new systems being proposed? There are suggestions that give an opportunity for the community voice to be heard much better in the event of BN and QAT indecisiveness as mentioned. Quotes by pimp and me:Kibbleru wrote:
The thing is, i can't really see the result being different with any of the new systems being proposed
because inevitably, these controversial maps would still get qualified
i think this in the end boils down to mentality
most of the QAT (as well as the BNG) right now have more or less a neutral mindset on almost everything
shiten was a case that the outside community absolutely hated it
but within the BNG and QAT, most of us either stayed neutral or some even liked it a bit
and the ones who disliked the map refused to speak up about it, which inevitably caused it to get ranked pretty much without any resistance whatsoever
pimp wrote:
the staff wants proposals so i think something like this could work
QAT dealing with controversial maps:
what we know so far is that many qat members often are not willing to make final decisions on discussions because they don't want to be flamed by the people on the losing side. but the community blames the qat for allowing controversial maps to reach ranked. so it's one side blaming the other mostly.
so we should aim to give the community's opinion more relevance on controversial topics. for example deciding if certain maps belongs in ranked section based on the rating, or at least take the map rating more in consideration for the final decision
beatmaps cosidered controversial must to force to vote after playing the map, and replace the current rating to upvote and downvote.
QAT management:
deciding if applicants are capable of being part of the BN could be made a priority and the requirements for staying on the BN should be made as low as possible. there are many talented people out there that did well/could do well on the group but simply couldn't keep up with the activity requirement, made a few mistakes or other reasons. the community will keep growing so the group should keep as much of it's members as possible. talented people should not be discarded because they can't meet activity requirements, if you give them the freedom to work on their own pace they will usually perform as well as the more active members and they will not resign the role if they know they are still welcome with lower activity. ofc the less active members shoudn't be elegible for QAT promotion, the 2+ years contribution badge or anything like that. if someone has been inactive for a long time he could just have his nominatior privileges revoked, if he wishes to return to activity he can just contact the QAT to allow him to nominate again(also it seems like it's how it works for people with osu!dev role, they keep the role even after years inactive just don't have the dev privileges allowed...)
Qualified period:
qualified period should be shortened for maps that are easier to judge (for example a map with a short song with a simple spread should be qualified normally or faster if no issues are found, while a longer map with a lot of difficulties should take several days more than a normal qualification) and if a map is considered controversial for whatever reason, it should stay on qualified section for indefinitive ammount of time instead of getting ranked by default (unless an unrankable issue was found, so the disqualify should be done imediately), this can also be used in case minor issues are found on any map, just don't let the map get ranked by default if the mapper is not around to respond to concerns, keep it qualified instead until the mapper is able to address issues.
tldr:
*give community's opinion more relevance on controversial maps so they can no longer blame only the qat for what reaches ranked section.
*don't make qat rush to decide if a map should be dq'ed imediately or just reach ranked.
*reduce the management work by mostly focusing on who joins BN, mainly kick BN's with a bad attitude, give inactive bn's the possibility to return when they want.
abraker wrote:
If no individual can reliably judge a controversial map and the QAT are indecisive, then the only logical thing to do is gather feedback on a wider spectrum. I propose a last line of defense for the map, a decision to be invoked in agreement by both the QAT and the mapper, and can be invoked only once. Allow the map to sit in qualified for a longer period of time to gain player and only player feedback. This means prohibiting anyone with BN or QAT role from voting since they gave their opinions in modding discussion and that reached a standstill. Within the longer span of time the map sits in qualified, the map must reach a certain number of votes and certain number % in favor. The mapper can go around to spread the awareness of the map if the mapper feels like it will not get enough votes within the time span. To simplify the decision, players would be able to vote with only "yes" or "no". If the map doesn't satisfy the voting requirement to be ranked, then the mapper must adhere to what the modders/BN are saying or it will not be ranked.
If the BN/modders can't come up with an agreement with the mapper and the QAT are indecisive, then that is the only only option left. The proposed suggests it be part of the QAT's decision on whether to allow the map to be voted upon to determine its ranked status. So the decision would be in the hands of someone who full well knows the implications of doing such.Loctav wrote:
Anything that involves people voting on matters while not occupying a position of responsibility (BN/QAT for the matter of mapping and modding) will always backfire. (It's almost like you let people vote on silencing someone because they made a controversial statement)
Nao Tomori wrote:
The end result will be a drop in qualified maps, no actual change in their quality as QAT standards drop due to burn out from being forced to mod maps they aren't interested in, and even more pessimism and toxicity in the community about terrible maps being ranked because there will be literally no recourse to stop them once they get qualified.
Nao Tomori wrote:
First, let's look at what damage the tier system did: by reducing the amount of active BNs to approximately 4, it also had the amazing side effect of completely removing their willingness to uphold any quality standard. I don't need to explain myself here - just go look at pishi's mods from that time, they are basically the same thing as what is being suggested the QAT to do here when qualifying a map. That is because people want more than 5 maps in qualified - something that is sure to happen when you only check maps that are assigned to you rather than ones you are interested in, you will either not icon them or icon them if they are rankable without modding them because you don't want to mod them.
Nao Tomori wrote:
Second, removing the ability to mod a map after qualification completely removes any semblance of community input on the ranking process. Arguing that maps can be "dq modded" before qualify is not realistic, given the difference in exposure between bubbled and qualified status. If the goal behind this rework is to shut people who think bad maps are constantly being ranked, it will 100% have the exact opposite effect because now these so-called bad maps will get ranked and they cannot even post mods to try and improve the maps because they won't have unrankable issues.
Nao Tomori wrote:
Third, given that QAT will end up either not iconing anything or yolo iconing everything after a rankability check (this is literally what happened with the Tier system, so I have every reason to believe it will happen here), there will be the exact amount of these low-quality maps getting ranked. BNs are not going to stop bubbling things that other people think are garbage (Sotarks 1-2 spam, Hailie extra diffs, etc. etc.) unless QATs also start evaluating them based on subjective icon quality (which they are extremely opposed to for some godforsaken reason).
Nao Tomori wrote:
The end result will be a drop in qualified maps, no actual change in their quality as QAT standards drop due to burn out from being forced to mod maps they aren't interested in, and even more pessimism and toxicity in the community about terrible maps being ranked because there will be literally no recourse to stop them once they get qualified.
Maybe there could be some lasting thing that remains on your profile after you've completed a QAT term, much like the BN badges now but in shorter terms. That has always been one of the things that offput me about the BNG too, you do volunteer work but once you leave, there's basically nothing to show for it (unless 2y plus). Perhaps a small section of the profile could be dedicated to that, just small icons showing how much time someone has put into those areas. I honestly don't see why not, it's part of creating incentive.Kibbleru wrote:
I can see this happening. You may argue the 4 month term system will prevent people from burning out, but in the opposite way, it prevents people from being motivated at all, or you may even end up with people just yoloing shit because "i will get kicked after my term ends anyways".Nao Tomori wrote:
The end result will be a drop in qualified maps, no actual change in their quality as QAT standards drop due to burn out from being forced to mod maps they aren't interested in, and even more pessimism and toxicity in the community about terrible maps being ranked because there will be literally no recourse to stop them once they get qualified.
I can agree that it keeps the system from stagnating, but it also stops people from "caring".
I believe a 4 month term system will only never give the QAT enough time to get settled into their position and become more experienced.Kibbleru wrote:
I can see this happening. You may argue the 4 month term system will prevent people from burning out, but in the opposite way, it prevents people from being motivated at all, or you may even end up with people just yoloing shit because "i will get kicked after my term ends anyways".Nao Tomori wrote:
The end result will be a drop in qualified maps, no actual change in their quality as QAT standards drop due to burn out from being forced to mod maps they aren't interested in, and even more pessimism and toxicity in the community about terrible maps being ranked because there will be literally no recourse to stop them once they get qualified.
I can agree that it keeps the system from stagnating, but it also stops people from "caring".
on top of that I agree with everything nao saidProposal wrote:
Glossary
Core Team
Members of the Global Moderation Team (or osu!team in general), whose main responsibility is managing the system rather than necessarily being part of it. I really don't think GMT should be managing anything mapping related, most of them have no idea how anything here works, granted there are a few exceptions but overall I think this is a bad idea
Membership of the Quality Assurance Team
Membership in the QAT is limited to a four months term. Beatmap Nominators get elected/chosen to occupy the vacant seats within the QAT in order to receive the ability to proactively assure the quality of the currently nominated content. Whether or not you want do that for all game modes or just standard is in the open. removing very qualified people just because their term is over doesn't make sense to me, I don't want to replace people with less qualified ones just because of that
The QAT lose their moderation permissions and only retain their Beatmap Nominator privileges together with the disqualification button. I'm ok with this as long as the GMT starts actually moderating beatmap threads when necessary
The Core Team compiles a list of possible candidates at the end of every term based on (yet uncertain) criteria. (Probably candidates applying for the position simply)
Members of the Beatmap Nominators can vote for member(s) on the list.
Once a QAT drops out of the team for any reason, for example by not being re-elected or reaching the maximum amount of re-elections in a row, they are on cooldown for the position equal to the duration of their just expired membership in the QAT. again, I'm very against the idea of blocking qualified people from this position, though the idea of making the qat position a bit more accessible isn't something I'm entirely against
Changes to the Ranking System
The Beatmap Nominators lose their right to qualify beatmaps (second nomination). literally why
The QAT is the only team that can qualify beatmaps. Thus, they are forced to proactively process through all nominated content and only push the content forward that is deemed to have a sufficient quality level. forcing someone to do a voluntary job makes absolutely no sense, yes sure you can just remove them if they don't fullfil their quota but nobody should be pressured to check a lot of maps they don't want to check
Once hitting Qualified, a beatmap may only be disqualified when it directly violates the Ranking Criteria, rendering all subjective-level concerns moot, as the map has already passed quality revision by the QAT. this basically says improving the map is not allowed anymore, what if the mapper wants to change stuff from mods? not allowed because it doesn't violate the ranking criteria
Maps nominated by members of the Beatmap Nominators will end up in a queue checked by the QAT. Whether they are being assigned or can pick maps by themselves has yet to be decided. being assigned to maps is the worst thing you could do, as you said it's still tbd but this really shouldn't be pushed through. if anything everyone should be allowed to pick what the want
New members of the Beatmap Nominators will need to go through the same application process as now, their applications will be evaluated by the Core Team. makes sense
Frequently Asked Questions
Why do you want to limit the membership in the QAT to a specific time period?
We are trying to combat two issues with that. First, people that occupy a position are eventually finding a comfort zone that they refuse to leave, making it hard to keep new ideas, new opinions and new views flowing and spawning. Second, people that occupy a position for a while but lose motivation are rarely resigning by themselves and usually simply stop doing things but are clinging to their title. As soon as you start to questioning the legitimacy of their position based on their activity and motivation, they usually come up with reactions that reach from “I am busy with school” to “I am super motivated now! (for two days)” By limiting the membership to a specific time frame, the QAT will always be restocked with new faces that bring new opinions along. Therefore, instead of keeping the QAT in a dormant state of always the same people being in charge, a highly volatile QAT assures that new eras are initiated more frequently. I actually kind of agree with a few parts of this, mainly the "they don't resign and stop doing shit" part. But instead of forcing a term limit you could just enforce activity requirements a bit harder and actually remove people from the team if they fail to meet them
Why will the QAT lose their moderation privileges along with that?
Putting someone in charge with critical moderation tools that can affect every user (silences, deleting posts, access to logs and account histories) requires a very specific kind of trust and faith into each individual that receives these tools. This trust must come from the higher osu!team itself. Given how the QAT are supposed to be chosen, the trust does not come from the higher osu!team directly. The BNG will put people of their ranks in charge, which is fine in itself, but this doesn’t make them automatically inherit the trust that, in comparison, the GMT receives when being entrusted with critical and sensitive tools and private information (e.g. the account standings that concern nobody but the moderators/admins and the individual user) as I said, I'm totally fine with the qat losing moderation powers, but I really think it would require the GMT to become A LOT more active in terms of moderating beatmap threads to prevent them from getting out of control (see guren / shiten / etc). And I think most GMT actually care about any of this happening which is why I believe that it would be better for the qat to keep moderation powers. Also aren't new qat approved by mao who is an admin from what I know?
Wouldn’t the core team basically be what the QAT is now? How does that change anything?
First and foremost, the core team sure has its origin in the current QAT. Their task is also a task that needs to be continued. The reason why we don’t want the elected QAT to handle the incoming BN applications is simply because we want to avoid the creation of echo chambers. The core team - being in charge of conducting the BN applications, the application assessments, running the QAT elections and compiling the QAT candidate lists - is way smaller than the current QAT. Also their only task would be to conduct these management tasks and therefore forfeit their say in the matters that concern the elected QAT. They will stay separate and focus on running these tasks neutrally, so that the QAT and BNG can focus on their task while the organization and logistics is conducted with full transparency by the core team. They would be responsible for providing the BNG/QAT a proper work environment by running the applications and elections and by running the logistics and oversee the entire system in moderation fashion without injecting their mapping/modding opinion into any of these. what how is denying the "core team", who are supposedly higher than the qat, from anything else that isn't strictly what you listed a good idea? aren't they supposed to be more qualified?
How would you avoid that the core team isn’t just denying applying BNs or QAT candidates arbitrarily?
This is achieved by having the criteria for being a BN and QAT way more defined. While a behavioral and activity assessment is necessary to assure that the incoming people are not complete douches and actually do things in a satisfying quantity, we have to reconsider whether or not a proficiency assessment is worth it at this level. Given the QAT is supposedly returning to proactive quality assurance, incompetent people in the BNG would have their output gatekept and have their performance eventually be reported as insufficient by the QAT to the core team. pretty much agree with most of this, though you'll never be able to keep things completely neutral so why are you trying
Why do you make the Beatmap Nominators lose their right to nominate? Why is the QAT supposed to nominate now?
By making the QAT being the ones that put the final nomination onto a beatmap alone, we assure multiple things:
The beatmap is being approved by the QAT instead of disapproved. This is a positive reinforcement. A quality assessment that eventually results with approval will always cause more positivity than a group of people being solely dedicated to “disapprove or ignore” created content. The only approval a mapper received from the QAT so far is that QAT just ignored the map, completely uncertain whether or not is has been assessed or not. By having the QAT actually put an “ok” or “not ok” onto each map, every mapper at least received some sort of feedback from the QAT instead of just interpreting the radio silence as either the QAT actually being fine with it or the QAT being just lazy and ignorant to it.
Beatmaps being approved instead of disapproved solves a core issue with the Qualified system. The Qualified system is a system built around anticipation. Once you get your map qualified, you are anxiously watching the 7 days passing by. During that time, instead of being filled with positive anticipation, you are rather afraid that some asshole QAT comes around the corner and disqualifies your dear creation. It’s 7 days of horrible uncertainty. “Are they assessing it? Does anybody care? Does my map fly under the radar? Oh god, someone posted on my map, soon the QAT will spawn and nuke it! All the delays!” - You see, instead of creating an environment of willing cooperation, open discussions and finding a compromise by *agreeing*, the current system makes the QAT shoot down maps, which is by its very nature perceived negatively. In order to fix that, the QAT is supposed to deliver the final approval instead of being the Damocles sword that hangs down from the ceiling, impaling your creation by sheer chance. QATs delivering an approval brings certainty to mappers and community that this creation is good to go and actually has been proactively assessed. the idea of a more active approach instead of reactive is something I definitely support, but all this is going to accomplish is shifting the blame from BN x to QAT Y, people will just get even more mad at a different person because now it'd be "qat approved" and you can't do anything whatsoever against it
Isn’t that just another try to do something like the BN tiers we used to have?
Yes it is and it shows a lot of parallels. The BN tiers weren’t a bad idea in itself, however had several shortcomings: how were tiers not a bad idea? do we really want to go back to having what, 5 qualified maps at a time?
The assessment of who becomes a second tier BN was done via a test. Not only are tests itself always a very slippery slope in terms of validity, but also were all BNs forced to undergo this revision. Aside the rather disappointing outcome, the result led to people ending up in the second tier that never asked for it while keeping away those who actually wanted to be second tier but never have been given the chance to do so. By having the QATs elected, attaining such an equal position is done by choice, because the candidate wants to be a candidate and the BNs actually want this person to be in charge. valid point
The second tier BNs were by all means not organized. They were confronted with a wall of bubbled beatmaps they were supposed to check all alone. While this still holds true, the idea is to organize the QAT way better than the second tier BNs were organized (they weren’t organized at all). organization isn't gonna change much, people will still be overwhelmed due to the much larger number of BNs than QATs (even with the proposed system)
The second tier BNs were still facing a nomination limit which hindered them to nominate more than one map per day. This isn’t a thing anymore. that doesn't guarantee people will actually make use of it, many people never even hit the limit (speaking of current times at least)
The second tier BNs didn’t have the authority of a title and also did not receive additional prestige despite their higher rank. The QAT still existed above them, making the second tier BNs an unloved group of people set between the normal BNG and the QAT that would still shoot them down. In this suggestion however, the last nominators are also equal to the last gatekeepers, as Qualified will only account for Unrankables or deep deep disagreements from the very own team the new QAT will operate in. a "title" isn't neccessary)
It was absolutely unclear when new second tier BNs would be appointed and under what premise they are appointed. Having the QAT be assembled by recurring elections makes sure that there are always other people put in charge and that there are always enough active people around that are willingly taking the seat and people can actively pursue this position without being accounted to arbitrary choices of promotion from the former QAT. makes sense
This sounds like getting a beatmap ranked will be quite harder again, as you need to not only get yourself a BN but also a QAT now. What’s the point?
The point is that you can’t have it both ways. You can not ask for a dedicated team to proactively assess quality and at the same time have the same valencies available. Asking for better quality assurance will indefinitely raise the difficulty in getting something Ranked, whether it be simply taking longer or the quality requirement itself just rising. In order to combat the concern of being lost and forgotten among all nominated beatmaps and to not have the community play “fetch the QAT” after having fetched a BN, the general shared queue is organized by the core team, where all nominated maps are being collected and eventually distributed for revision among the residing QAT, so that eventually every beatmap nominated will receive some sort of feedback from the QAT. Whether or not the feedback will lead to a qualification or the QAT just leaving the workable feedback behind so the mapper can keep working on their creation is some other topic. The general queue assures that every nominated map will receive QAT feedback and combats uncertainty and the impossibility to “fetch a QAT”. However, in order to fulfill the gatekeeper and quality assurance task, the feedback can also be “negative” and will not always lead to a definite approval by the QAT either way. Sometimes, beatmaps are just in such a distant state from Ranked in their perception that beatmaps can be possibly bounced back to the BNG entirely (“popped without rebubble”), declaring that this needs way more polishing while pointing out the cornerstones of what it is lacking. The QAT will not be giving you a step-by-step introduction on what to fix exactly how but is only advised to word the shortcomings understandably and offer a direction, not a direct manual. this isn't going to assure quality at all, it only lowers the amount of people who might be "okay" with your map. do you actually every bn is in love with every map they nominate?
Why would you decline feedback being provided during Qualified stage? What about the community feedback?
Having the community leave feedback during Qualified stage is a nice idea on paper. You have experts or dedicated people push a map forward and present it on a silver platter to the community. However, as described in another question, in case of the map being *not accepted by the community*, it gets charged up with massive negativity. Getting your map disqualified is always a huge letdown. The Qualified stage is also a huge window of uncertainty. It puts mappers and supporters in anxiety, praying for nobody to show up and nag on something. Is this what we want? Hoping for people to slide through a critical revision phase without anybody bringing up suggestions? While suggestions are all appreciated and even dearly wanted by the mappers prior Qualified stage, as soon as the map hits Qualified, most mappers become massively reluctant to listen to the suggestions people provide then. The map already jumped through a lot of hoops, it has been assessed by modders, BNGs and QATs, people that are deeply involved into the scene of map creation. While we understand the desire to also offer the creation to the community, we also have to embrace the fact that you can never make it right for everybody. Qualified stage as it is now is also easily exploitable in order to gang up against specific mappers or maps, to keep a map in a limbo forever, especially since you can hold something back indefinitely for reasons that can be entirely subjective and are a simple matter of taste. By having the QAT proactively assess the quality of beatmaps BEFORE the Qualified stage is reached, we assure that every map hitting Qualified are “actually good to go”. Community feedback sure is valuable but feels a lot like a hindrance when only brought up during a phase where the mapper is the most reluctant to receive this feedback. By limiting Disqualifications to direct Ranking Criteria violations alone (as in seeing Qualified like a grace period for human mistakes to happen) and by limiting it to a stage where fellow QATs can deeply disagree with the assessment of their colleagues (avoid abuse, checks and balances), we make the Qualified stage be a stage that is way less filled with anxiety, because as long as your map sticks to the RC and isn’t over the top terrible, you have the certainty that your map will slide through because of the certainty of the map being good to go, as it is QAT approved. This change also assures that the community brings up their feedback at the point of the map review process where it is most desired by mappers: before it hits Qualified. I totally understand the "anxiety" part you mentioned, in fact I feel very nervous myself when I have something qualified and keep checking the thread very often, but while your "solution" removes this part, it will also add a part of "feeling disappointed / defeated" for anyone that disagrees with the map, as they are unable to do anything whatsoever against it