forum

Let's rework QAT stuff!

posted
Total Posts
99
Topic Starter
Noffy
Since Ephemeral didn't make a thread first here I am, going to repost what he put in the BN and QAT discord, so that we, as a community, can discuss it here! In the forums. Which has more permanence instead of any and all brainstorming and discussion being lost in discord chats.

Ephemeral wrote:

the current QAT system as we all know it today, is officially being declared as "end of life" as of this announcement.

what this means is that in the near future, ideally sometime during mid-January 2019, we'll be completely flipping the table on the entire QAT addition and promotion system, as well as introducing a clearly defined set of roles and expectations of all QAT members. this may or may not include a complete reselection of the current roster as well. all existing QAT will be allowed to migrate to the GMT if they wish to continue helping out on the moderation front in this particular instance as a one-off thing.

this will also include at the barest minimum, the institution of some kind of term limit and a goal of making the QAT much more immediately accessible to people who are currently engaged in the community.

until then, things will continue as they currently are. during this transition period, any and all feedback or proposals on possible new systems will be considered if properly fleshed out and presented for everyone to discuss. these proposals should seek to use as little development time as possible, since that's the kind of restriction the QAT has pretty much always operated under.


Ephemeral wrote:

btw we wont just "not have a QAT" if we hit some arbitrary point in mid-january/feb next year and have no solid system. this isn't some brexit shit.

the eol is just a kind of notice to say "hey, we should really seriously think about replacements/improvements to this system" and would like to have something ready to go by this date, etc.

a few core points to (maybe) focus on in your discussions:
* QAT engagement in "controversial" maps, current QAT are too placid and passive
* greater accessibility for people to both enter and exit the team
* the creation of a more focused role/responsibility for the QAT in the context of checking qualified maps


If you have any complete or semi-complete proposals concerning this, feel free to post them in a separate thread (in development) and then notifying a member of the QAT to add them to the list below. This way feedback can be given directly to the post rather than mixing everything here, thereby making things easier to follow.

List of Proposals



  1. Monstrata
  2. Loctav
  3. hi-mei
  4. UndeadCapulet
  5. Fycho


any offtopic posts will be removed
squirrelpascals
If we were to make separate threads for each issue to streamline discussion a little more I feel like that would be most productive.

e.g. Thread for QAT addition system, thread for QAT roles and responsibilities
JBHyperion
Moved this to Development so general community can weigh in also as this will likely have wide-reaching consequences that affect everyone.

Keep discussion on-topic and constructive please.
Voli
I've been one of the people rooting for a QAT rework for a long time, the current definition of what the QAT is just misleading since nobody actually checks for quality anymore, which leads to much of the community being confused on what they actually do, hence always the QAT definition memes on almost every controversial map.

The current roster also is way too one-sided as the OP mentioned, map DQ's are pretty much only ever for small unrankable issues anymore and most people there seem to be aboard the subjectivity train.

I also think the QAT shouldn't be an ''one-up'' from the BNG as it currently is, because the duties of the QAT should be different from what BNs do. Imo, they should be seen as separate teams with separate ways of joining them. This kind of hooks into the accessibility point the OP mentions.
Serizawa Haruki
It would be helpful to know if something concrete has been planned and if yes, what exactly will change. Otherwise it's hard to have a discussion on a topic nobody is informed about. What is the reason for this announcement and what goal are we trying to achieve? It seems like most people, including QAT members, don't know what this is all about.
celerih
QAT engagement in controversial maps. I've touched on this before, but basically no one wants this, except for a very select amount of people. The main point here to remember is that while yes a lot of people are crying that we need more nuking of "bad" maps, saying that QAT aren't assuring quality anymore, almost no one is willing to make the compromise that if more strict quality standards are enforced, not only the maps they don't like will get stopped, but also potentially maps they do like will also be stopped due to being low quality.

This is totally a doomed if u do, doomed if u don't situation, and the QAT are very much aware of this, given the amount of flame they deal with. And even when we get QAT engagement in controversial maps, like Shiten where multiple QAT engaged in discussion and gave their opinion, lo and behold ppl weren't satisfied either. People don't want the QAT to enforce more strict standards, they want the very specific maps they don't like to be stopped, nothing more.

There's a big distinction between the two, because one is actually taking care of the content of the game, while the other is a jerk reaction to seeing something you don't like.
Zexous

Voli wrote:

I've been one of the people rooting for a QAT rework for a long time, the current definition of what the QAT is just misleading since nobody actually checks for quality anymore, which leads to much of the community being confused on what they actually do, hence always the QAT definition memes on almost every controversial map.

The current roster also is way too one-sided as the OP mentioned, map DQ's are pretty much only ever for small unrankable issues anymore and most people there seem to be aboard the subjectivity train.

I also think the QAT shouldn't be an ''one-up'' from the BNG as it currently is, because the duties of the QAT should be different from what BNs do. Imo, they should be seen as separate teams with separate ways of joining them. This kind of hooks into the accessibility point the OP mentions.

This is difficult...I agree that one of the problems with the QAT is that they're not willing to concretely call things problems, but about the BNG having different duties...
The kinds of things QAT should be noticing and acting on, are also things the BNG should be speaking up on. But they don't, in the interests of a smoother nomination process. BNs get used to this tradeoff, and that's why the behavior is carried over into the QAT.
Irreversible
What bothered me ever since I "had to take my leave" from QAT was, as mentioned, how one sided it's become. We'll leave the reason to that open as I'm not going to assume whatsoever at this point. Obviously, when people have different opinions, sooner or later there will be a confrontation between two fronts. Good to see that this is being challenged now, as I personally believe the latest QAT additions haven't done much.

I hope that there will be a better solution towards everyone taking having a ranked map for granted. As time has passed, the meta shifted more and more to "you can say whatever you want, it somewhat makes your map reasonable" which is poison for an environment like here. If this means that QAT is not supposed to be existing like it is right now, then so be it - but at least there is no "fake" qualitypolice around anymore, which does everything, except assuring quality.

The future was meant to be more community driven, and per se that's not a bad thing. Thus I find it a step into the right direction and feel like either there should be actual quality polices, or none at all (apart from moderating misbehaviour, right Irre?). Seeing with how much odd stuff people get through, it's more of a confusion rather than anything else (also taking commissions by Sotarks etc. into account)

It's good that change will happen, but the OP Post doesn't quite say what change is going to happen exactly - so I will keep questioning whether the problem is actually being seen or not, because at times when I saw there's an upcoming change, it got fixed into something worse. (Sorry Eph, there were many instances where we were promised change.... Would be nice if there was more detailed information about what you've planned and what problems there are being tackled.
Mirash
In my opinion problem of QAT making a bad picture is in those controversial maps that are too different from what you usually see in a beatmap, so most of regular users who interact with it end up not liking it and blaming guys who mapped it/guys who let it through. Things should develop more smoothly, you don't just say community it is good and they will answer with yeah make more of this, unless they can get it with minimum of understanding mapping/modding/whatever it takes to enjoy the beatmap, like it happened with RLC maps or skystar stuff back then.

Maps that from a perspective of people that have no experience look like a 'randomly placed objects' or a 'first time editor map' should not get a ranked status out of nowhere just because they work and have logic if you are engaged in those sort of stuff, and people deciding whether a map is ok to be ranked are indeed engaged in those.

Solution is changing mindset of people that are involved with assurance of quality, you may say it will lead into how strict things in mapping were before but now it is too free for some people, so keeping a balance is what we need.

Or we can make community be more engaged into judging product they play, like having a star rating be not useless and encourage people to vote, instead of whats written under qualified right now ( https://delusional.s-ul.eu/BIFKPr05 ). Yeah it's not ideal but it'll help in special cases when beatmap is just unwanted by 95% with user rating of less than 5 and eventually will lead into hate on people who are responsible for this.
Kurokami
"QAT engagement in "controversial" maps, current QAT are too placid and passive "
It might be just me, but I do remember mentioning this while I was still a member and I always got passive answers from like 50% of them. I mean, sure this is a very sensitive topic and can cause a lot of negative feedbacks but in my opinion, if the team is not ready to face with them then they kinda shouldn't be in the team(?)

Before someone misunderstands this, "controversial" maps are not bad, and they surely have a place in the ranked section but in some cases, there are issues with these maps meanwhile those are being ignored too due being "controversial".
hi-mei
Listed my proposal down below. Please add it to the list.
JBHyperion
In all honestly, the OP is open-ended precisely because we are as unsure as to what the best way to proceed is as anyone else. The QAT has shifted in many directions over it's lifetime, sometimes for the better, and sometimes for the worse. A major problem now (at least from my perspective, although others have shared similar sentiments) is that the QAT is somewhat insular / stagnant, and we have had difficulties in agreeing to any significant, long-lasting change. I hope that by opening discussion up to the mapping community as a whole (since our roles / actions have an affect on everyone in it to some extent), we can gather some fresh ideas.

Many of the current QAT are in the position not only because they are experienced modders, but because they have (or had) the drive to realize the change they want to see in the game. I believe that osu! is a community where anyone can succeed as long as they have a good, well-thought idea and are prepared to work with the community to shape, improve, and eventually realize it. The paradox of this argument is that QAT is a very public-facing position with significant responsibility attached, and many feel paralyzed by this responsibility when it comes to making tough calls or proposing controversial ideas.

Many people are likely quite content to wait for change to happen and then debate whether it's good or not. However, I believe there is some merit and seeing what ideas and opinions may come of this, whether it be a complete overhaul, a satisfaction with the current way of working, or anything in between. This is not to say everyone in the team needs to be a "firecracker" personality - there should be plenty of space in the team for more reserved personalities too who wish to quietly contribute in their own way - but if rotating members in and out on a cyclical basis helps to bring new ideas, with less fear of backlash, then I don't think that's necessarily a bad thing.
hi-mei

Kurokami wrote:

"QAT engagement in "controversial" maps, current QAT are too placid and passive "
It might be just me, but I do remember mentioning this while I was still a member and I always got passive answers from like 50% of them. I mean, sure this is a very sensitive topic and can cause a lot of negative feedbacks but in my opinion, if the team is not ready to face with them then they kinda shouldn't be in the team(?)

Before someone misunderstands this, "controversial" maps are not bad, and they surely have a place in the ranked section but in some cases, there are issues with these maps meanwhile those are being ignored too due being "controversial".


Well yeah, another topic here is that most of the new QAT's taking their position wrongly. People are just got used to the fact that QAT is like an upgrade to BN status, but you are in a safer position and not obligated to be so active. Being a QAT now is only different from BN in a sense that they can Disqualify things for obvious reasons (if someone reports stuff, or kids on reddit is going reeeeeee mode on some map).

I really miss the old days where people were actually checking things for quality issues.

Now here we are, in the days where "quality" is a meme and everything is subjective, therefore you are "hater" if you raise any concerns to someone's map.
frukoyurdakul
I will go for some opinions by quoting the part of what Ephemeral said.

Ephemeral wrote:

a few core points to (maybe) focus on in your discussions:
* QAT engagement in "controversial" maps, current QAT are too placid and passive --> I definitely agree to this, QAT should leave the wiki and go for judging the maps mostly, or a seperate team should be created in order to create wiki stuff.
* greater accessibility for people to both enter and exit the team --> Define accessibility here. Is it more leniency to get accepted? Is it lowering the requirements or lowering the ability to judge and qualify stuff? If it could stay higher this proposition would not have done in the first place.
* the creation of a more focused role/responsibility for the QAT in the context of checking qualified maps --> I thought we already had QAH (Quality Assurance Helper) for this. To an extent, QAT (or whatever you might wanna call to wiki people) should do the wiki work mostly and QAH should actually judge and have the power of disqualification.


Above all this, I'd like to say stuff that has discussed before. The propositions that have been made to change the system (that I heard so far) are only 2, and I beg of you please do not even consider doing stuff like this. These would be:
  1. A bot (an artificial intelligence to be specific) to manage nominations, as in a bot that determines if a map should be nominated
  2. Judging by mappers' kudosu count they should have the ability of nominating, and even further count will give the power of disqualifying stuff.


Both of them will do well, if the main consideration is quantity over quality. However, in a community that seeks for quality over quantity these days, I would not recommend even discussing this or attempt to create stuff for it. As Irreversible said:

Irreversible wrote:

It's good that change will happen, but the OP Post doesn't quite say what change is going to happen exactly - so I will keep questioning whether the problem is actually being seen or not, because at times when I saw there's an upcoming change, it got fixed into something worse.


This is something we have to avoid if we want to improve stuff instead of building something worse that exists now.

Above all: This kind of change should be considered for osu! Standard only, as the smaller modes are going on their peak so far. Rotting the other modes down by that change shouldn't be an option, considering there are people who like to play mania, taiko and CTB as well. Having two different systems would not crush the structure, but in this case it may even improve it.

These are all my thoughts on this so far.
Aistre

hi-mei wrote:

3. Ranking criteria / QAT. The big amount of controversial maps are coming from the issue that these rules are EXTREMELY vague. Like, its so amorphous to the point where you can rank a minute of metronome sounds with 1-2-1-2 jumps on the same notes and literally nobody would do anything to this. I wanted to do this to make a precedent till I realized that there is a good way to solve this.

Set the rules already. There are lots bold concepts in mapping that were developed over a decade: flow, structure, spacing emphasis, visual spacing, volume adjustment, gimmicks, slider art etc. Make all this structured so people would learn from it.
The ranking criteria is supposed to be a set of objective terms that your map must comply to; these are the bare minimum of expectations for the possibility of your map being ranked. Controversial maps don't break any of these rules which means they can have the possibility of being ranked.

If you start adding rules that people find subjective it goes against the point of the ranking criteria. Take visuals for example as you're a fan of clean visuals, they don't break any of these rules objectively. Even overlaps that may be detrimental to readability is a guideline because in lower diffs deciding exactly what detracts from readability is subjective as people have their own definition of what they might consider unreadable. Telling whether two notes are stacked in the timeline and why it should not be allowed is pretty obvious, so that's why it's a rule.

These rules aren't vague, defining what is rankable and not rankable due to personal standards is a vague topic and that's why UC and Hailie maps pass through to ranked.

The mapping meta is always changing and evolving, it's only natural that it's going to change at some point sooner or later. Maybe this is the new meta, maybe something else in 5 months will be? I'm not saying I'm for this new style of mapping, but adding rules for flow, structure, spacing emphasis, visual spacing, volume adjustment, gimmicks, slider art etc. isn't the way to go here because the freedom of these and people coming up with their own ideas for them is what made mapping how it is today.
hi-mei

Alphabet wrote:

The ranking criteria is supposed to be a set of objective terms that your map must comply to; these are the bare minimum of expectations for the possibility of your map being ranked. Controversial maps don't break any of these rules which means they can have the possibility of being ranked.

If you start adding rules that people find subjective it goes against the point of the ranking criteria. Take visuals for example as you're a fan of clean visuals, they don't break any of these rules objectively. Even overlaps that may be detrimental to readability is a guideline because in lower diffs deciding exactly what detracts from readability is subjective as people have their own definition of what they might consider unreadable. Telling whether two notes are stacked in the timeline and why it should not be allowed is pretty obvious, so that's why it's a rule.

These rules aren't vague, defining what is rankable and not rankable due to personal standards is a vague topic and that's why UC and Hailie maps pass through to ranked.

The mapping meta is always changing and evolving, it's only natural that it's going to change at some point sooner or later. Maybe this is the new meta, maybe something else in 5 months will be? I'm not saying I'm for this new style of mapping, but adding rules for flow, structure, spacing emphasis, visual spacing, volume adjustment, gimmicks, slider art etc. isn't the way to go here because the freedom of these and people coming up with their own ideas for them is what made mapping how it is today.


The quality comes from combination of things. Thats the point here. Most of these controversial maps are taking the bar to the minimum of almost all the concepts making it unacceptable even to a regular player. Just read the goddamn feedback of regular players. If they are quastioning the quality (not playabilty) you know theres something not okay with the map. Not to mention the mappers are even making jokes from it.
Deca

hi-mei wrote:

2. BN / QAT promotions. Its now clear for everyone that the promotions are biased as fuck. Just admit it. Lots and lots of people are being wasted for personal controversies with the ones who "judge" them. We should definitely bring some transparency there. I definitely prefer a blind test over anything else. Why? because its fucking fair. Also, these "rounds" should basically go away already. What the point in working for literally months to get potentially denied? I have went thru this literally 3 times, modded over 100 maps over the 9 months and each time it was some bullshit like "mod more easy maps" "making kick sldiers from double is bad for maps" and so on. I spent literally 3 years on this mapping scene and wrote shit tons of useful stuff. Same for Billy Bill, Rizen, C00L etc. You should start recognizing people's effort.

In regards of transparency, once again:
The entire BN/QAT thing is buried in corruption to the point where you can see how some people are nominating maps in purpose of getting nominations for their maps. Its just not okay. (hello UC btw).

The rules should be set so nobody will complain. Lets just sort things out already.


As a BN, your job is not only to nominate and mod maps well, but also to act as a responsible and mature member of the community, hence why a blind test doesn't work.

Also, tests are the easiest thing to cheat on, far easier than the months of modding required and subjective analysis of mods.

Does the current system lend itself to nepotism? Undoubtedly. The latter system would, however, introduce many incompetent and/or toxic individuals into the BNG who are wholly unfit to be official representatives of the community.
hi-mei

Deca wrote:

As a BN, your job is not only to nominate and mod maps well, but also to act as a responsible and mature member of the community, hence why a blind test doesn't work.

Also, tests are the easiest thing to cheat on, far easier than the months of modding required and subjective analysis of mods.

Does the current system lend itself to nepotism? Undoubtedly. The latter system would, however, introduce many incompetent and/or toxic individuals into the BNG who are wholly unfit to be official representatives of the community.


Its been like 1.5 years since they removed tests. Look at what we are at now.

Also, in regards of
The latter system would, however, introduce many incompetent and/or toxic individuals into the BNG who are wholly unfit to be official representatives of the community.


Lets not forget about these things:
https://puu.sh/CiDgk/ae2097d231.mp4
https://puu.sh/CiDgM/cea09d7d3c.mp4
https://puu.sh/CiDs2/85a36cf510.png

This person is in QAT and he is an official representative of quality in this game. By the way he got into the BN and then into the QAT without any application or what so ever. They just took him because he is their friend. Nothing else. (talking about Yuii here)

Not to mention Sotarks who .. well. There are lots and lots of super bad things he did during this year (including that babymetal incident, insulting streamers, mappers, devs etc). But HEY he is STILL in BN group! What a miracle.

Tests are the only answer to this (in different form than it used to be). Otherwise we gonna struggle in the corruption for decade again.
Mordred

hi-mei wrote:

By the way he got into the BN and then into the QAT without any application or what so ever. They just took him because he is their friend. Nothing else.
lol ok then good to know I can just skip your post
Sotarks
hi-mei can you like not take stuff out of context and stuff that happenned in the past on an other platform that's not osu to use this against yuii- ? that's not nice move. and what's the point by doing that ?

They just took him because he is their friend. Nothing else.

yea man i'm pretty sure you know everything about yuii- and why he got qat, that's basicly it. but no he got in because he's good decision maker and it helps when you have lots of qat discussion and you need someone like that to make stuff going forward. other than that the guy is also a really great modder that also check qualified maps often. he also did qah work as a probation bn and also even before being bn, and made lots of really valid reports.

please actually think before throwing informations like that to make someone look bad. thank you.
show more
Please sign in to reply.

New reply