https://docs.google.com/document/d/1EbhpZviwcvAYM926IsKZ-Q8DKRH4N3_WdpYcD5JdD-U/edit?usp=sharing <-- post to read

https://twitter.com/Cheri_Osu/status/1369671204580306949 <-- Twitter post

On request by Ephemeral
-
community/forums/topics/1268711?n=14 <-- referring an QA extension/Adding a branch
community/forums/topics/1268711?n=15 ^ Me at the same time basically stated a similar thing in detail in a document

CONS ^ was similar to QAT and tried it before didn't work/don't want to repeat anything and lack of interest still

community/forums/topics/1268711?n=59 my adjustments on -White's post
Basically about making a third group w/no BN benefits for non-bns to do QAH to add incentive instead of relying on BNs who don't want to do it.
community/forums/topics/1268711?n=46 ^ the original

CONS ^ No point in doing this after being BN, Having to trust non-bns is risky even with considering they can't dq, not as reliable as a BN doing it, rewards can be considered iffy,etc.

community/forums/topics/1268711?n=85 <--- abolishing the system

community/forums/topics/1268711?n=126 ^ my tl;dr version why just abolishing it isn't the best idea

community/forums/topics/1268711?n=107 ^ the full version + just be me
explaining the same thing in my document a bit better

community/forums/topics/1268711?n=123 <-- refers to having QA as a group making RC proposals and such
community/forums/topics/1268711?n=130 ^ basically rejecting it as RC is fine as is

community/forums/topics/1268711?n=139 QoL changes could add

Since it's here too this is my response to the suggestion of splitting the BNG and QAT https://gist.github.com/VINXIS/25aeae000261f07ae1bc52f7f2c25043

tl;dr history suggests its a bad idea

A suggestion is a suggestion at the end + what happen in the past doesn't necessarily mean it shouldn't happen again especially since the game is much bigger now than it was back then and there was obvious a lot more mismanagement back that can be easily rectified if we look more into the core problems.

The suggestion at the end was nothing more my (and friend) 2cents on options to do, I imagine there is much more

I agree, we should be looking more into the core problems, I just felt it was necessary to respond to that and that the same thing will happen again because the environmental factors caused are still about the same because of those core problems

everything else in the post I think was good and how there has been so little action by the NAT either especially considering the amount of data that is pretty easily achievable that could be used to solve issues

VINXIS wrote:

everything else in the post I think was good and how there has been so little action by the NAT either especially considering the amount of data that is pretty easily achievable that could be used to solve issues

what sort of data do you mean here, exactly? is this in regards to punishing dq's or something else?

Hi, author of the first half of the google doc here

VINXIS wrote:

tl;dr history suggests splitting the BNG and QAT is a bad idea

I mostly came up with this because despite QAH being a volunteer position within the BNG, there is no real obligation attached to it, so there is no artificial motivation to perform QA as a BN (on top of the no/negative motivation to do so naturally). Splitting off the QAH/QAT/whatever you want to call it was what I envisioned as a rather extreme method of defining the roles of those who are actually willing to do this work (sure, there are probably none that are willing to do it to the extent that Cheri does, but I'm fairly sure there are plenty that would do QA at least to some extent if they knew it was to fix a system on the verge of collapse as things currently stand).

Theoretically, this would also be solvable with QA quotas for ALL BNs, but that would likely deincentivize BN applications at all, so not exactly an ideal solution. A large BNG is good with how big the game has gotten, but not good if the other critical systems in place (such as QA) can't even begin to keep up.

I recognize that the question of how to motivate QA activity is extremely difficult, but especially given the current additional load on the qualified section thanks to the mug event I think it's more important than ever that there is at least some serious discussion on what to do about it.

Some points of data I think would be beneficial off the top of my head

1. Nature of vetoes presented
2. Nature of the disqualifications in general
3. The states of qualified and ranked throughout this period of using such a system
4. Survey analysis

This is in regards to helping find out what may have indirectly/inadvertedly caused what when using the system

aidanbh wrote:

Splitting off the QAH/QAT/whatever you want to call it was what I envisioned as a rather extreme method of defining the roles of those who are actually willing to do this work (sure, there are probably none that are willing to do it to the extent that Cheri does, but I'm fairly sure there are plenty that would do QA at least to some extent if they knew it was to fix a system on the verge of collapse as things currently stand).

That's fair enough, though I do think the only time this type of change will be fruitful is when the treatment of the dq + discussion system is changed

VINXIS wrote:

That's fair enough, though I do think the only time this type of change will be fruitful is when the treatment of the dq + discussion system is changed

agree on this point, will discuss further at some point later today(?)

VINXIS wrote:

Since it's here too this is my response to the suggestion of splitting the BNG and QAT https://gist.github.com/VINXIS/25aeae000261f07ae1bc52f7f2c25043

tl;dr history suggests its a bad idea

You have a pretty fair point but it's important to note that the reasons the QAT was hated back then are rather easy to mitigate; qualified maps no longer give pp (which fueled a lot of player hatred towards disqualification) and often times QAT posts were very subjective and contentious (see: Asymmetry, first DQ is post 107 [did you know you can click the number and go to a specific post? I had no clue until now lol]). The pp debacle was settled a while ago, but I'd also argue that 'subjective DQ' debacle is effectively settled with the introduction of the veto (and mediation) system. As flawed as it is, it is a good replacement for the old "QAT says No thanks" system of before; now, a large portion of the BNG is required to give fairly detailed reasoning as to why a veto should or should not stand. A separate user group could just essentially be the 'janitors', clearing out obvious spread issues, straight-up missed unrankables etc.

I don't necessarily think that a new user group is the best decision, but it's definitely not as apples-to-apples as it might seem on the surface.

For the record, I entirely agree with the idea that the way we view disqualifications is the root cause, and I'd suspect that a lot of that has to do with the time aspect. You can wait days or weeks for a BN to find time for your map which you likely spent dozens of hours making, or you can search for days for BNs to wait for weeks, and by the time that the map is qualified, the absolute last thing you want to do is have a last-minute disqualification throw another wrench into the system. From a mapper's perspective (especially one without close ties to BNs), it isn't as much of mutually-beneficial quality assurance as it is just pure pain. Though honestly, I don't see that there is a fix for this issue without an extreme reform (like, no-BNG level of extreme) that, frankly, I'm not sure how it could possibly be done.

In my perspective at least as a mapper I think that if im given suggestions + stuff and im in a process where the discussions are not meant to be a benefit to the general ranking system, then the process of applying mods in qualified should be so that all i have to do is apply and update, no need for a DQ to apply and update, and Then have to wait for the bns to be free to check and nominate again after many days, and Then on top of that also have a punishment given to the bns just because I myself was receptive to feedback while in the qualified section that I thought would improve the map And were also not even unrankable issues Yo hailie can we get utsuroi qualified again yet, I think that that would at least to some nontrivial degree make it far less painful to be receptive to feedback in qualified for everyone, and also Wayyy back when the 'Fix Vetoes' Thread existed I was suggesting version history to make this far easier to do.

Also unrelated since this is 2 discussions at once, but I definitely think the 'subjective DQ' debacle is nowhere near effectively settled using the veto + mediation system because it is effectively the same process with the facade of a form of discussion occurring in the veto state and in the mediation state, which is why I was saying that the suggestion in the current environment of things would definitely not be good at all. I do agree that it is better than the shit QAT used to do for sure and removing pp and queue time reset has significantly helped, but I don't think that is enough for us to consider it environmentally different enough still because the root issues from then are still in effect today

nobody wants to do QAH work quite simply because its fucking boring and there's no incentive. you're not even doing an actual bn check you're just checking for unrankables and giving it a general pass of "yeah i guess its fine mapping wise"

speaking as someone who did a boatload of them in the past and will never do one again, I wish I could say something beyond "theyre boring" but it's just the facts. the fact that you still do them is honestly incredible to me (in a positive sense)

given that I don't see why anyone would actually want to be a QAT in your proposed system. its just being a BN but without any fun. and given that the compensation for the work will inevitably be nothing (as usual) nobody would want to do it lol

---

Also, given that mapping is a community sphere, I dislike how QAH work makes you look like an asshole if you dq any maps. people like to say "well the maps shoulda been good" and yeah you're literally right but we also live in a world with human beings that have emotions. nobody likes getting their map dq'd. being a QAHer puts you in a position where you're directly contributing to punishing your fellow BNs.

hard agree with how stupid and inconsistent the "severity" rating thing is

In response to hobbes:

giving some sort of incentive to do QA is an issue, and I do agree a lot of people would consider it boring, but I feel like that neglects the fact that some people enjoy things that are considered mundane or tedious to most. Since the community is growing so rapidly there are more people who would be willing to do said tedious tasks.

In regards to the comunity backlash aspect I feel reletivly the same way. Bigger comunity means more people who are able to not care about backlash.

----------------------------------------------------------------------------------------------

In general I feel like being less fixated on the singular proposed would be really beneficial for this entire ordeal since, at the end of the day it's just that. a suggestion from a singular person. It's sort of counterproductive to have such a strong unmovable stance in this sort of thread in my opinion since qualified is being held together by a single thread that's on the verge of snapping. New ideas, or ways to improve existing one's are the most important thing here.

----------------------------------------------------------------------------------------------

In regards to the proposed solution in the doc:

While I don't know if bringing back what is basically old QAT will be sustainable in the long term I feel like it's still worth discussing, even as a temporary fix while a more long term solution is worked on, especially since it doesn't seem like it would be overly hard dev side (may be wrong on this, so correct me if I'm wrong).

In my opinion old QAT would have worked if, one there was enough people who were willing to do the boring work and didn't care about community backlash, and two if there was some sort of way to moderate stupid decisions made by the QAT members.
I feel like the proposed solution of 3 teams (BN, NAT, QAT) could very well solve that issue. Some sort of objection function on the mapper's side that leads to discussion between NAT and the non offending members of the QAT, which could result in the offending party getting kicked if their DQ was too out of line. Having that out side pair of eye's (being the NAT) could potentially aliviate some to a lot of the bias in whether the DQ in question was out of line, and the appropriate punishment for the offending QAT member.

I also feel like the main reason why a lot the reason people who would otherwise be willing to check qualified don't is because being on a level playing-field with the people you disagree with is hard, so giving those people who would be willing to a role with similar power, or perceived power as NAT would make them more likely to check qualified.

Obviously this is just my two cents on the issue, but I strongly feel like something needs to be put in place to deal with qualified sooner than later, so going in circles on whether one idea will work really won't help.

well the alternatives are:
- slowing the entire ranking process WAY down in order to thoroughly check every map in qualified before letting it through. this would ensure quality, but would make a lot of people very angry as way fewer maps would be getting ranked and it would be tiring for those doing the checking. obviously not sustainable with the current scale of the game.
- ditch the QAH system entirely and just let stuff go through qualified (this is functionally what we have right now because as you said no one wants to do QA voluntarily). also not sustainable since as the game grows, so will the rate of major issues getting through qualified undetected.

so one of two things needs to happen:
- we find a group of people who, against all the classical theories of motivation, are willing and able to dedicate themselves almost exclusively to QAH work, and set them aside to do that specifically. I think Cheri proves that they exist, even if they are few and far between. it's also possible that a group of people exists who would not do it in a vacuum, but would do it with even a tiny bit of external motivation (such as the positional things mentioned in the document/by P1T)
- thoroughly rework some other part of the system to reduce the number of unrankables getting qualified in the first place. I know I said in the document that it was impossible to entirely eliminate human error in the BNG, but it may be possible, by raising the standards of BN performance, to get it down to an acceptable level such that QAH becomes unnecessary

honestly my take on this is -

issue is - right now qah isnt incentivised at all

also within the NAT we have quite a few people specialising in different things (think pishi & naxess compared to what noffy & seto do)

so what could happen to solve all this is have sub divisions to the NAT group (not seperate titles just offical different obligations within NAT) to include QAH work, so you have 3-4 NATs that their primary job is to do qah, 3-4 NATs that their primary job is to do new and current BN evals etc. (not delegating this among current NATs but promoting a few qah centric modders to fill these roles)

this would solve the issue of giving an incentive to do qah (getting nat leadership position) while also getting a more active and consistent userbase doing qah

i mean rn the qat probably dont want this change to be made - but its either this, make it mandatory for bns (which everyone will hate) or just not do qah at all (which seems counter-productive with the influx of new bns over the last two years or so)

i fully expect no changes to be made but with the current state the entirety of qah basically rides off of cheri doing the work - so what happens when she stops doing it?


also this would avoid trying to split up the bng which was tried before (think tier 1 & tier 2 bns etc)

https://docs.google.com/document/d/1BfOpVJt05aNobf0zoDTOFzgVyFbC7kzAPB4aj--QvGU/edit?usp=sharing Deviouspanda basically stated similar to what I was going for with this document but nonetheless i'm adding it here anyways since I was told to

All and all I simply just want to be able to not be the only one doing qualified and having solutions like above ^ would be a good thing even if temporarily to solved that problem until more reworks that will take much longer than the simple fix here.

Im not sure how it is for anyone else here but I think if I was in the position to, I would personally be more than down to do QA work or post mods on almost all maps in the qualified section myself if

1. it only focused on the aspect of improving the map and discussing with the mapper
2. it didn't have to affect anyone outside of the realm of the mapper and myself

which is why I wanted to suggest removing the dq punishment system and getting version history and everything instead of trying to "force" people to get into QA work, but idk how many more people feel similarly on working on QA so maybe making more people to QA work in the NAT or similar really would be the best option, idk tho

VINXIS wrote:

Im not sure how it is for anyone else here but I think if I was in the position to, I would personally be more than down to do QA work or post mods on almost all maps in the qualified section myself if

1. it only focused on the aspect of improving the map and discussing with the mapper
2. it didn't have to affect anyone outside of the realm of the mapper and myself

which is why I wanted to suggest removing the dq punishment system and getting version history and everything instead of trying to "force" people to get into QA work, but idk how many more people feel similarly on working on QA so maybe this really would be the best option, idk tho

I'm not denying that having such could be potentially helpful in the dear future as honestly I wouldn't even have to go through qualify the way I do if things was more laxed as such, but that is a system that takes work and still as well as still gotta considered that qa itself is a rather boring task on it's own.

My solution is basically to create management for the very core needed part of the game with the influx of mappers and nominators. So whilst yes a perfect world, going with solutions like this would be cool, but right now I think having something more QOL up until things are more stabled, should be prioritized.

fair enough, I just don't have much trust nor expectations for quick changes anymore because of how the past 14 years of osu have ONLY been just quick changes (and not just in the ranking system, in EVERYTHING related to osu) but it could be worth for now i guess


Edit: though at the very least the suggestion for raising QA activity value shuold really be done as well honestly

VINXIS wrote:

Im not sure how it is for anyone else here but I think if I was in the position to, I would personally be more than down to do QA work or post mods on almost all maps in the qualified section myself if

1. it only focused on the aspect of improving the map and discussing with the mapper
2. it didn't have to affect anyone outside of the realm of the mapper and myself

which is why I wanted to suggest removing the dq punishment system and getting version history and everything instead of trying to "force" people to get into QA work, but idk how many more people feel similarly on working on QA so maybe making more people to QA work in the NAT or similar really would be the best option, idk tho

Honestly I don't think getting rid of DQ punishment would be a bad idea, hell even just making it more lenient and consistent would help.

I feel like if there was quality control of some sort in place there would also be less need (excluding more extreme examples such as BNs who have a lot of generally agreed on questionable noms, but thats sort of case by case).

Some sort of QA being in place would likely encourage BNs to be a bit more careful in general so I feel like it very well might work.

Also having more people checking qualified would definitely make focusing more on improving the map and having discussion when it comes to subjective issues a lot more viable.

I feel like a lot of the issues (and you can correct me if im wrong here) people have with the way cheri DQs stem from the lack of time and support given to her to check qualified. Having more people doing the job would give more time to discuss, and provide more meaningful feedback/mods.

Checking qualified alone basically means to be able to get through everything and do more than check for unrankables DQing more immediately if the mapper doesn't want to change anything is more efficient, but given more time I'd assume she would prefer not to do that.

Yea i was talking about removing the DQ punishment/sev rating in the posts above since the focus for DQ punishments should really only be for bad actors, and i dont think someone modding a bunch of qualified maps improving the beatmaps and getting them changed shouldnt be considered as the BNs that nominated those beatmaps being bad actors for example, would even help with the current mods that cheri is doing in QA currently

VINXIS wrote:

Im not sure how it is for anyone else here but I think if I was in the position to, I would personally be more than down to do QA work or post mods on almost all maps in the qualified section myself if

1. it only focused on the aspect of improving the map and discussing with the mapper
2. it didn't have to affect anyone outside of the realm of the mapper and myself

which is why I wanted to suggest removing the dq punishment system and getting version history and everything instead of trying to "force" people to get into QA work, but idk how many more people feel similarly on working on QA so maybe making more people to QA work in the NAT or similar really would be the best option, idk tho

Figured I'll mention my experience as just a modder and never official staff: I personally enjoy looking at maps in Qualified but I'm almost always super reluctant to mention anything because the last thing I want to do is stir up shit where it isn't deserved (tho sometimes it is deserved NGL) and end up wasting the mapper's time, and as the system currently is the mapper is kinda incentivized to be as resistant to mods in Qualified as possible in order to not lose days of time to arguments, so it's kinda a lose-lose situation either way.

I'd be a lot more incentivized to mod (and look at) maps in Qualified if pointing out issues didn't have the chance of resulting in days-long wait periods, cuz that's just not fun for anyone at all

some thoughts on ongoing discussion

- remove DQ punishment and SEV rating (@VINXIS)
Very confused here, issues stated come across as exaggerated to me. Since we look at resets as a whole, BNs rarely (if ever) get punished for a single mistake but rather for having repeat issues or failing across the board. The other way around, a single DQ/reset in itself does not cause a BN to be punished. I'd hope at least BNs themselves understand this.
Similarly, as long as it doesn't fall under knowledge we judge on applications, feedback and improving maps is not harshly treated.

I don't know what "bad actors" are here, but regardless of being one BNs still have the objective task of making sure their nominations get ranked without issues. Removing "DQ punishment" and SEV rating essentially means to also remove accountability for this task being done well. Personally I struggle seeing how that works out okay in the long run, so I don't support it.

Improving the consistency of SEV rating we'll look into, nobody should be getting confused or afraid of these.

- QAT as a solution
Not sure but not a fan. Agree that repeating a failed history is something I don't want to pursue. I also don't see real suggestions that would improve on the old formula, lots of "maybe??"s here. The thought of surrounding context having changed enough to fix some of QAT's bad sides is appealing though.
Temporary QAT sounds like it'd become a permanent band-aid, knowing OSU. It'd need to be a transitional system but the other side of that hasn't been defined.

Otherwise I'm wondering what separates this potential QAT group from BNs in terms of ability and in terms of needed authority. What more is needed to check qualified maps and to offset the negatives that come with it.

- QAH
Boring, low interest, prone to backlash. Means it inherently needs an external motivation rather than relying on intrinsic efforts or inexplicable acts of altruism. Somehow something of value needs to be presented in return. Personally this is where I hit a dead end, we don't really have anything of value to offer (putting aside the QAT idea).

Maybe an improved star priority system could create such value? That's my best stab at introducing an idea of my own here.

Thinking about this topic has always put me between a rock and a hard place; it seems like there's never a good solution. So, I'm eager to see other people take a crack at it at the very least.

yaspo wrote:

some thoughts on ongoing discussion

- remove DQ punishment and SEV rating (@VINXIS)
Very confused here, issues stated come across as exaggerated to me. Since we look at resets as a whole, BNs rarely (if ever) get punished for a single mistake but rather for having repeat issues or failing across the board. The other way around, a single DQ/reset in itself does not cause a BN to be punished. I'd hope at least BNs themselves understand this.
Similarly, as long as it doesn't fall under knowledge we judge on applications, feedback and improving maps is not harshly treated.

I don't know what "bad actors" are here, but regardless of being one BNs still have the objective task of making sure their nominations get ranked without issues. Removing "DQ punishment" and SEV rating essentially means to also remove accountability for this task being done well. Personally I struggle seeing how that works out okay in the long run, so I don't support it.

Improving the consistency of SEV rating we'll look into, nobody should be getting confused or afraid of these.

- QAT as a solution
Not sure but not a fan. Agree that repeating a failed history is something I don't want to pursue. I also don't see real suggestions that would improve on the old formula, lots of "maybe??"s here. The thought of surrounding context having changed enough to fix some of QAT's bad sides is appealing though.
Temporary QAT sounds like it'd become a permanent band-aid, knowing OSU. It'd need to be a transitional system but the other side of that hasn't been defined.

Otherwise I'm wondering what separates this potential QAT group from BNs in terms of ability and in terms of needed authority. What more is needed to check qualified maps and to offset the negatives that come with it.

- QAH
Boring, low interest, prone to backlash. Means it inherently needs an external motivation rather than relying on intrinsic efforts or inexplicable acts of altruism. Somehow something of value needs to be presented in return. Personally this is where I hit a dead end, we don't really have anything of value to offer (putting aside the QAT idea).

Maybe an improved star priority system could create such value? That's my best stab at introducing an idea of my own here.

Thinking about this topic has always put me between a rock and a hard place; it seems like there's never a good solution. So, I'm eager to see other people take a crack at it at the very least.

Considering the whole entire ordeal of the problem is the fact we got nothing to work with and the idea of **QAT/Extension of NAT** would crack down on that more than what we have, a volunteer service with only one person doing. I truly disagree on the notion of denying this would be the best ideal situation just because of a claim "it happen before so we can't do it again" even after we explain in this thread why it is important being none of the current NAT care enough to do anything.


Completely dismissing the idea is such a bad move here when there is absolutely no incentive that the NAT has ever made since the time QAH was settle on the BN website and being that qualified is a mess, please understand this is close minded and rather only influence this issue to continue to happen.

As someone who checks qualified more than anyone else here, there is way more needed for a person who checks everything just as much as a regular NAT and I do not understand how does one not see that.

Moderation ability is always useful, supporter so we can download maps without having to pay for it, having a more of a shield from backlash (a title means more to people than you think), having a say in evals, etc.

Until you guys actually have a better solution to stop the mass backlog of 100 maps per week and relying on me to do so, I would like my idea of having this in the NAT to be in place (or something of similar extend)
or why am I even working this hard for people that don't seem to care here as you so willing to dismiss the idea so easily without any valuable idea yourself (that would work better here)?

yaspo wrote:

some thoughts on ongoing discussion

- remove DQ punishment and SEV rating (@VINXIS)
Very confused here, issues stated come across as exaggerated to me. Since we look at resets as a whole, BNs rarely (if ever) get punished for a single mistake but rather for having repeat issues or failing across the board. The other way around, a single DQ/reset in itself does not cause a BN to be punished. I'd hope at least BNs themselves understand this.
Similarly, as long as it doesn't fall under knowledge we judge on applications, feedback and improving maps is not harshly treated.

I don't know what "bad actors" are here, but regardless of being one BNs still have the objective task of making sure their nominations get ranked without issues. Removing "DQ punishment" and SEV rating essentially means to also remove accountability for this task being done well. Personally I struggle seeing how that works out okay in the long run, so I don't support it.

Improving the consistency of SEV rating we'll look into, nobody should be getting confused or afraid of these.

- QAT as a solution
Not sure but not a fan. Agree that repeating a failed history is something I don't want to pursue. I also don't see real suggestions that would improve on the old formula, lots of "maybe??"s here. The thought of surrounding context having changed enough to fix some of QAT's bad sides is appealing though.
Temporary QAT sounds like it'd become a permanent band-aid, knowing OSU. It'd need to be a transitional system but the other side of that hasn't been defined.

Otherwise I'm wondering what separates this potential QAT group from BNs in terms of ability and in terms of needed authority. What more is needed to check qualified maps and to offset the negatives that come with it.

- QAH
Boring, low interest, prone to backlash. Means it inherently needs an external motivation rather than relying on intrinsic efforts or inexplicable acts of altruism. Somehow something of value needs to be presented in return. Personally this is where I hit a dead end, we don't really have anything of value to offer (putting aside the QAT idea).

Maybe an improved star priority system could create such value? That's my best stab at introducing an idea of my own here.

Thinking about this topic has always put me between a rock and a hard place; it seems like there's never a good solution. So, I'm eager to see other people take a crack at it at the very least.

There really isn't any way to give external motivation for QAH.

There's the argument of "what if we just raise activity requirements, and make QAH checks worth a bit more", which sort of falls through because that will just lead to people doing more noms instead regardless of how much activity requirements are tweaked. The only real way you could make that work would be making QAH checks worth more than noms, and that basically leads back to essentially forcing people to do QAH, which people will get mad about.

Other than that the only other thing I could think of would be giving people who pass a certain check threshold a title as long as they keep up QA activity, but that likely won't work assuming the title doesn't come with any real perks, and if you're willing to give that title enough perks that it becomes worth going after you basically have a 3rd group at that point, which you disagree with.

There's no real clean solution to satisfy everyone, and I genuinly think having a 3rd group or a portion of NAT with the job of doing QA is the best option at this point.

yea

Regarding the "bad actors" statement I don't know what is defined as a bad actor outside of someone who like u said makes consistent mistakes, but I'd say that someone who is making consistent mistakes isn't the same as someone who doesn't complete the objective task of making sure their nominations get ranked without issues, but not sure about someone failing across the board

Exaggerated scenarios but

1. Someone who for example gets like 15+% of their nominations disqualified for only x issue will get punished since they are having repeat issues
2. Someone who had all of their nominations disqualified (let's say like 12, 4 each month in the 3 month period) for 12 completely different issues which were stated as suggestions given by people doing QAH work also will get punished because they are "failing across the board"

Is it intended to punish both or not? The response gives me mixed messages regarding that; if so, then what would be the reason to punish the second scenario if DQs similar to stuff like "changing soft sampleset to drum sampleset and raising vol by 10% for a section (lol deetz veto is 1/1)" or like "changing 2 sliders that unintentionally seemed out of design choice and caused a difficulty spike (this was 1/1 until I checked just now apparently it was rewritten and it's 0/0 now)" as examples occurred for all 12 nominations.

Is that really considered as failing across the board / not doing their objective task of making sure their nominations get ranked without issues? If not then nvm what I said this whole time loooool but I'd like to know what that actually does mean.

If so then this is my main issue because I don't think such a person should be considered as one that is failing their job as these are semi-unique cases and having that happen for all of them and be punished for that doesn't seem right to me at all. Everyone has their own accumulation of knowledge regarding mapping and some people are more knowledgeable about different stuff than others; sometimes the knowledge is conflicting and requires more discussion. I don't think that when stuff is treading these areas, there shouldn't be any punishment given for anyone participating

---


Oh edit, regarding the incentive of doing QAH work the easiest thing that can be done which was said above alrdy was just raising the value of QA work from 1/4 of a nomination tho i doubt thats gonna do much, could do something along the lines of some form of point -> reward system like a badge for example similar to mappers guild but then again its a bit aids to do stuff like this rn when responding to QA work on ur own map requires a lot of unnecessary work to

VINXIS wrote:

regarding the incentive of doing QAH work the easiest thing that can be done which was said above alrdy was just raising the value of QA work from 1/4 of a nomination tho i doubt thats gonna do much, could do something along the lines of some form of point -> reward system like a badge

admittedly I don't have much of a sample size here but the only active QAH said that badges and titles weren't really a motivating factor (and they have even turned down such bells and whistles in the past), and that the reason why a structured role/dedicated NAT position is not that is because such a position would come with a greater ability to actually exercise the job they are qualified to do - when looking at the kind of authority NATs command in beatmap threads as compared to regular BNs, one can see why such staying power would do well in the hands of QAH-type people.

I will admit that cheri is kind of a special case in this regard because the backbone she's built up from being one of the most hated figures in the entire game has made her fairly immune to people getting angry at DQs in thread (which has been mentioned multiple times as a major turn-off for QAHs) but I'm pretty sure there are a couple more people that would be willing to be brought on for this kind of role if it was in the form I just described

of course I could be entirely wrong about all of this but this is how I see the state of things from what I've observed/been told by multiple parties

Honestly, if the uncertainty of whether there are enough people in the current BNG who would even be interested, running a poll of some sort doesn't seem too out of the question.

Just make sure to explain some sort of (obviously vague) general expectation
(certain amount of checks need to be done per week, need to not care much about backlash, other stuff I cant think of right now)

If more people would like to do the qah job they would do it, just like some people mod outside the bng, the statistics show that some do but they're not even close to checking all daily qualified maps and that's normal since there are just too many, don't really think it can be solved tbh since the way people will see you won't change, you'll always be the bad guy that constantly checks maps looking for mistakes, also it's ultra boring and you gain absolutely nothing, volunteer jobs have their limits, i wouldn't expect people to perform consistently for free on this and tbh i don't see a way to reward them properly, most of what i've read here wouldn't change anything at all to make more people participate in qah and even less to make that participation consistent.

i'm chewing over a longer response to this once i'm confident i actually understand the issues at play here (presently 6 different people are talking about 6 different things). here's what i can suss out so far:

1. purportedly more bad maps are entering ranked because qualified isn't being checked
2. qualified isn't being checked because there's no real reason for anyone to check it
3. the only real reason to check qualified (QAH) is considered drudgework by 99% of the people who are in the position to do it, and thus don't do it (aka: QAH really fucking sucks and is a failed initiative)
4. mappers are reluctant to embrace feedback given during qualified phase because doing so imparts mandatory 7 day+ delays due to how the queue system works and other associated backlog with it, ergo they are punished for attempting to improve their map
5. the system as a whole (according to vinxis) pushes people away from actually making meaningful feedback because they get punished by current NAT practices for disqualifying maps with those issues
6. punishing people for leaving issues in maps provides incentive for people to hide/not talk about those issues, something that is becoming increasingly easier as the playerbase grows
7. all of this (and more) combines to paint disqualifications in an inherently negative light where nobody, not mappers, modders or players want to participate in the process
8. there is too much punishment and not enough incentive all around

is that about right?

@Eph
- i don't think "bad" maps getting ranked is due to qah inactivity as qah from conception is designed as a way to avoid unranks for cross screen 1/4 jump type things.***
- apathy towards checking qualified for unrankables can't be addressed because frankly bns have 0 incentive, reason to look at, or interest in maps other than ones they personally took a look at and wanted to push. fundamentally spending time sitting there looking at maps that don't interest you "for the greater good" of finding unrankables is a stupid process and one that peppy should have solved mechanically by now instead of dicking around with mania OD and stuff.
- mappers are reluctant to apply mods in qualified or even bubbled state because interacting with modders is a chore and relies too much on waiting for some arbitrary third party to sign off on work they aren't invested in compared to you and stalling a process which you put much more effort into than they did.
- the fact that 2 nats deciding to print a mug for $5 provoked the most activity out of bns in forever should tell you exactly what you need to know about carrots and sticks.

*** when people say bad, they are very rarely referring to maps with unrankables; people don't actually care about unrankables much. even things like spread quality aren't a huge factor in how people define quality offhand.
when people say bad, they mean either the top diff (or at least one of the top diffs) is subjectively low quality or the set as a whole is bad for meta reasons - pp farm diffs, low quality / low effort song, etc.
qah is not currently and was not ever designed to solve these issues, the whole bn system as a whole was. so any qah rework will not stop people from complaining about bad maps getting ranked. i would argue that stopping bad maps from getting ranked is not a good goal to begin with because aside from pp abuse which messes with competitive integrity of the game's leaderboard, the average "bad" map does not have a major impact on other maps or the game as a whole.

I mean the issue fits in to 1 line:

Noone is doing QAH

The rest of the discussion occurring here is just derivative of the issue and figuring out reasons why to figure out solutions, u can just do a root cause analysis flowchart/workflowy list to organize the shit too

Crissa wrote:

don't really think it can be solved tbh since the way people will see you won't change, you'll always be the bad guy that constantly checks maps looking for mistakes, also it's ultra boring and you gain absolutely nothing

yes thank you it is hardly a secret that QAH is currently a role with net negative returns, the goal here is to either come up with some form of motivating factor that is enough to see success OR, failing that, find some other system entirely to replace the current qualified system because it does not work if no one is there to enforce quality

just did QAH to get a better grasp of what it was like; in roughly 3 hours i managed to check 8 maps, a whole day's worth of ranked.

given i was also the only one in the qah list besides cheri, i'd have to spend 3 hours every single day doing essentially 2 nominations worth in activity, while also having people get mildly tilted at me for "dq modding" their map (if i find anything)

in my view QAH winds up being equivalent in time to just actually modding one of your BN reqs, except you're not modding anything other than severe issues (so it's not very interesting) and its' unlikely to be a set that you like (thus very boring to stare at for half an hour)

i dont really have any solutions to propose, just wanted to give some insight

peppy should pay BNs and the staffs to increase motivation, even by little.

their work don't get recognised at all. At least mappers can get their maps noticed in YT and mentioned by players.

Haachama chama~!!

paying people to do a dogshit boring/unfulfilling task they wouldn't ordinarily want to do is not the solution nor will it ever be the solution, stop suggesting it

In my opinion one part of the problem is that quality assurance is an almost 100% backloaded process at the moment.
In the worst case a map gets submitted, nominated and then qualified while only being checked by 2 people.
At that point the milk has already been spilled from a QA perspective.

A nomination is already some kind of a "seal of approval" greatly encouraging mappers to look for a second BN and at the same time deterring them from looking for more mods. At the same time it may deter modders from modding that map because they may be afraid of wasting their time on a bunch of defensive "this is already nominated so this must be fine" replies to any points they may rise. Ultimately for most mappers going for ranked the goal is to get the map ranked, not to make it as high quality as possible.

The current nomination process encourages looking for BNs instead of looking for mods if your map is even remotely good.
A process that encourages to get to the nomination on the fastest way possible is naturally not bound to produce quality but will instead result in maps getting nominated that barely meet a bottom line.

Endaris wrote:

In my opinion one part of the problem is that quality assurance is an almost 100% backloaded process at the moment.
In the worst case a map gets submitted, nominated and then qualified while only being checked by 2 people.
At that point the milk has already been spilled from a QA perspective.

A nomination is already some kind of a "seal of approval" greatly encouraging mappers to look for a second BN and at the same time deterring them from looking for more mods. At the same time it may deter modders from modding that map because they may be afraid of wasting their time on a bunch of defensive "this is already nominated so this must be fine" replies to any points they may rise. Ultimately for most mappers going for ranked the goal is to get the map ranked, not to make it as high quality as possible.

The current nomination process encourages looking for BNs instead of looking for mods if your map is even remotely good.
A process that encourages to get to the nomination on the fastest way possible is naturally not bound to produce quality but will instead result in maps getting nominated that barely meet a bottom line.

The thing is encouraging people to look for more mods doesn't solve the issue with QAH in the slightest. nothing guarantees those mods a mapper gets before qualified will point out anything useful. the difference between a map having 100 mods and almost none is so small at times that this wouldn't help. there also lies the issue of this not helping with unrankables very much since if the nominating BNs didn't catch them those non BN modders very well might not either.

Ur right Endaris in that people just aim for the lowest bar since that's the only standard that exists in the current timeframe of osu and just go straight to BNs and dont really care about much else, but getting people to look for more mods outside of the ranking process won't really be the solution since like wot P1 said "the difference between a map having 100 mods and almost none is so small at times", i dont think the incentivization of finding more mods will fix this issue

I'm also not even sure if this is related to QA itself. I guess it could be on the basis of higher standard -> less work needed when doing QA work; though I think focusing on raising the minimum standaard will help very little in alleviating the QA workload since the focus of QA work Right Now AFAIK is moreso on finding unrankable issues + issues that are Arguably unrankable and were missed such as spread issues and fucked lower diffs moreso than "raising quality" from the minimum standard; in other words, it could might as well just increase the workload moreso than anything else by raising a higher standard

quality assurance IS all about finding unrankables and obvious mistakes.
there's a veto system for subjective concerns.

I feel like we have historical proof why QAT didn't work. A small group of people deciding the ultimate fate of what gets ranked or not will just be a source of conflict between people. Also the job sucks and nobody wants to do it. Existing QAT members wud just burn out super fast.

Uberzolik wrote:

just did QAH to get a better grasp of what it was like; in roughly 3 hours i managed to check 8 maps, a whole day's worth of ranked.

given i was also the only one in the qah list besides cheri, i'd have to spend 3 hours every single day doing essentially 2 nominations worth in activity, while also having people get mildly tilted at me for "dq modding" their map (if i find anything)

in my view QAH winds up being equivalent in time to just actually modding one of your BN reqs, except you're not modding anything other than severe issues (so it's not very interesting) and its' unlikely to be a set that you like (thus very boring to stare at for half an hour)

my initial thought there is having QAH worth more from a BN activity perspective, but even that doesn't fix the fact that it is both probably going to be a set you don't like and screening purely for severe issues, based off your experience anyway

i'm not really seeing a tenable way to make QAH stuff more attractive to people without overweighting it in terms of offered reward/incentive, especially when the whole "dqs are bad" thing is going to turn anyone who does QAH with any regularity into a social pariah

doing QAH for activity is already overweighted as it is. you can easily farm enough activity to not need to nominate maps but nobody does it because there's no point.

Kibbleru wrote:

I feel like we have historical proof why QAT didn't work. A small group of people deciding the ultimate fate of what gets ranked or not will just be a source of conflict between people. Also, the job sucks and nobody wants to do it. Existing QAT members wud just burn out super fast.

I feel like this sort of ignores the proposed solutions. Both of the suggestions involving a QA group have involved keeping NAT in place. The issue with QAT was that they were, as you said, "A small group of people deciding the ultimate fate of what gets ranked", but the main issue with that was that they were essentially also the top of the chain of command. With either of the proposed solutions involving a QA group (either a separate QA group, or giving people in NAT a job of QA) in theory, NAT would also be on a level playing-field enough to speak out against QA doing stupid shit, and kick the offending members.

In regards to the issue of burnout, I feel like the game growing as much as it is, and just generally being a lot bigger makes it easier to just replace members who lose interest or can't keep up with the amount of work. This also sort of solves the issue of needing to replace people if someone in said group does something stupid.

I'll throw out an idea I thought of off hand, I don't particularly have any strong feelings either way but figure it might inspire an idea in someone else:

What if a QAH like group (could call it QAT, Assistant QAT, 'community modders' or some other title) was created that was relatively easy to get into, like you could have them take a written test similar to a BN and require X number of kudosu (just to weed out the extremely inexperienced ppl who know nothing). Then give them a little badge so they have some incentive and feel good about themselves and let them do QAT, but not have the ability to actually DQ. They could post mods on the thread and a BN or NAT could come along and DQ the map/resolve the mod if it was right/wrong. Or it could be where X number of these "QAH" need to agree on it being an issue before a BN/NAT gets involved. At the very least these "QAH" would be able to find obvious things like accidental cross-screen 1/4. Those who do an exceptional job could get a fast track to a BN promotion, as additional incentive, and those who do poorly could be weeded out.

Basically, BNs don't want to do QAT because it's boring and they could instead spend their time nominating. So what if we used the less experienced modders who can still identify issues to do it for the BNs? A lot of them might be able to identify issues, but maybe not at the same level/consistency that a full BN can. I'm sorta thinking along the lines of BN applicants who got reduced cooldowns or came close to reduced cooldowns, etc.

Anyways, dunno if that idea sucks, but hopefully it inspires some ideas in someone else

-White wrote:

I'll throw out an idea I thought of off hand, I don't particularly have any strong feelings either way but figure it might inspire an idea in someone else:

What if a QAH like group (could call it QAT, Assistant QAT, 'community modders' or some other title) was created that was relatively easy to get into, like you could have them take a written test similar to a BN and require X number of kudosu (just to weed out the extremely inexperienced ppl who know nothing). Then give them a little badge so they have some incentive and feel good about themselves and let them do QAT, but not have the ability to actually DQ. They could post mods on the thread and a BN or NAT could come along and DQ the map/resolve the mod if it was right/wrong. Or it could be where X number of these "QAH" need to agree on it being an issue before a BN/NAT gets involved. At the very least these "QAH" would be able to find obvious things like accidental cross-screen 1/4. Those who do an exceptional job could get a fast track to a BN promotion, as additional incentive, and those who do poorly could be weeded out.

Basically, BNs don't want to do QAT because it's boring and they could instead spend their time nominating. So what if we used the less experienced modders who can still identify issues to do it for the BNs? A lot of them might be able to identify issues, but maybe not at the same level/consistency that a full BN can. I'm sorta thinking along the lines of BN applicants who got reduced cooldowns or came close to reduced cooldowns, etc.

Anyways, dunno if that idea sucks, but hopefully it inspires some ideas in someone else

What BNs/mappers here are apparently believing - according to the replies to my previous post - is that normal modders are too incompetent to spot unrankable issues that slip past BNs which makes their contribution worthless for the purpose of quality assurance.

Regardless of whether you believe in the worth of these modders or not, if there's any chance they will actually spot something, the system should ideally have them check before a nomination/qualification to avoid the social and organisational minefield that the DQ process is right now.

Any osu! player already has the opportunity to leave such comments right now on qualified maps anyway, it's not a particularly inspiring task which is why none is doing it and I highly doubt some shiny badge with no power is going to change anything about it, not to mention the extra effort in supervising and "rating" these people which may as well result in a similar rating disaster as the current DQ point system.

White's idea seems somewhat sound on paper if it's for checking unrankables and not assuring "Quality". A lot of the social minefield in DQs results from them seeming too subjective or unnecessary. I believe DQs for strict unrankables are easier to tolerate despite still being annoying.

To me the main unknown factors in running something like this successfully would be along the lines of
1. does interest match the scope, even with potentially raising the ranked cap in mind
2. user retention or user inflow, it won't mean much if people growing bored leave faster than others join
otherwise everything else I can think of is more or less answerable (?), could be missing something though

1. can be polled
2. is hard to predict without actually trying it - it's hard to say how much these less experienced modders will value their badge

I'd also like to note that this doesn't necessarily guarantee fast tracking into BN, most applicants fail based on modding ability which this system doesn't train or test

-----

@Paul Blart - you're basically saying "users that are trustable with moderation powers are easy to recruit and replace" which unfortunately isn't true.
also fwiw I'm not against the QAT idea because it's "been done" but because the issues with it are known and rather harmful

yaspo wrote:

White's idea seems somewhat sound on paper if it's for checking unrankables and not assuring "Quality". A lot of the social minefield in DQs results from them seeming too subjective or unnecessary. I believe DQs for strict unrankables are easier to tolerate despite still being annoying.

To me the main unknown factors in running something like this successfully would be along the lines of
1. does interest match the scope, even with potentially raising the ranked cap in mind
2. user retention or user inflow, it won't mean much if people growing bored leave faster than others join
otherwise everything else I can think of is more or less answerable (?), could be missing something though

1. can be polled
2. is hard to predict without actually trying it - it's hard to say how much these less experienced modders will value their badge

I'd also like to note that this doesn't necessarily guarantee fast tracking into BN, most applicants fail based on modding ability which this system doesn't train or test

-----

@Paul Blart - you're basically saying "users that are trustable with moderation powers are easy to recruit and replace" which unfortunately isn't true.
also fwiw I'm not against the QAT idea because it's "been done" but because the issues with it are known and rather harmful

If the issues with the system are known wouldn't it be more beneficial to discuss them, and possible ways to prevent them?

Just seems kind of backwards to not even attempt to discuss why the old system failed, and what could be prevent that from happening again (especially since it feels like multiple people in this thread know why it failed). It takes less time to fix a system with a few problems (assuming you know what they are) than it does to create a new one from the ground up, so with a relatively urgent issue like this I don't see how brushing off discussion of fixing an old system is productive at all.

Anyways I don't feel like this will change anyones mind on even attempting to discuss why QAT didn't work and how to fix it, and I don't see any more than surface level discussion of other solutions, so I don't have much more to say with this thread.

You say fix a few problems - the problems were exactly what this thread is trying to discuss. The reason QAT burned out so fast and pushed for a change was basically because they were reduced to unrankable-checking bots (current QAH) and BN evaluators (still going alright overall). Similarly, the problem here ("problem" - I don't think it's actually a problem) is that nobody wants to check unrankables and do the bare minimum expected of two other BNs for them.

Then maybe take a step back, and consider the fact that QA doesn't have to be just checking for unrankables, and if QAT weren't reduced to doing that there wouldn't be as much burn out. If there's issue with QA not exclusively checking unrankables then that would be productive to discuss.

If people strongly believe QA should just be about unrankables, then maybe an actual bot with human supervision would be a better solution than trying to incentivize BNs to do the same job.

There is an actual bot with human supervision. It's called mapset verifier used by two BNs on a map before nominating. There is a position for checking non-unrankable issues in qualified. It's called being a BN and being able to mod qualified maps and DQ as necessary. The reason people don't want to do it isn't because of anything solvable - modding maps you find uninteresting on songs you may not like, as well as facing backlash from people for modding a map already considered finished (nominated by 2 BNs) is not something that is going to change. It just isn't. Applying an arbitrarily higher quality standard than the one already in place systematically (BN evaluations) is not a useful method of improving the ranked section, if that's the goal. More resources for improving at modding, as well as removing BNs who consistently nominate garbage maps (not just unrankables) are better methods than any QAH reforms that could be attempted.

i agree with nao and am of the opinion that this discussion won't rly end with anything productive unless the authors of the document and the people talking in the thread are using the same idea of quality assurance..

and whites idea isnt half bad

currently, quality assurance is only about unrankable issues, as stated here https://bn.mappersguild.com/qualityassurance

not sure who can see this page so

ofc, people doing qah will also try to spot "obvious mistakes" that don't break the ranking criteria since they are notifying everyone that they checked the map. they may also post minor stuff to the mapper to decide to fix or not... but disqualifying for anything beyond unrankables or obvious mistakes should not be considered QAH anymore.

anyone who's logged in to the bn site can see it

Nao Tomori wrote:

You say fix a few problems - the problems were exactly what this thread is trying to discuss. The reason QAT burned out so fast and pushed for a change was basically because they were reduced to unrankable-checking bots (current QAH) and BN evaluators (still going alright overall). Similarly, the problem here ("problem" - I don't think it's actually a problem) is that nobody wants to check unrankables and do the bare minimum expected of two other BNs for them.

yet judging from the responses in this thread, nobody seems to want to put things in place to make those BNs do the bare minimum either, so...

what do we do, exactly?

yaspo wrote:

White's idea seems somewhat sound on paper if it's for checking unrankables and not assuring "Quality". A lot of the social minefield in DQs results from them seeming too subjective or unnecessary. I believe DQs for strict unrankables are easier to tolerate despite still being annoying.

Yep, this is exactly what I meant. I do believe these "QAT" would be able to identify DQable issues like spread as well, but like I said, not to the point where they should get powers to DQ themselves. Suggesting an issue and letting other QAT weigh in and then a BN either agreeing or disagreeing would give these "QAT" valuable experience identifying issues and seeing what is valid/invalid for a DQ.

yaspo wrote:

To me the main unknown factors in running something like this successfully would be along the lines of
1. does interest match the scope, even with potentially raising the ranked cap in mind
2. user retention or user inflow, it won't mean much if people growing bored leave faster than others join
otherwise everything else I can think of is more or less answerable (?), could be missing something though

1. can be polled
2. is hard to predict without actually trying it - it's hard to say how much these less experienced modders will value their badge

The feeling I get is that there are plenty of people who are able to and willing to look at QFed maps if there was some sort of reward/recognition to it, myself included. I would say that the activity requirement would have to be very low or non-existent, like 1 QF check a month, and operate with the principle of sheer numbers. This would maybe not handle all the QF checks needed but would limit burnout by not forcing people to do QF checks. Yall could just put one NAT dedicated to managing this group and I think it would run fairly smoothly.

yaspo wrote:

I'd also like to note that this doesn't necessarily guarantee fast tracking into BN, most applicants fail based on modding ability which this system doesn't train or test

Fair. I was just thinking of ways to increase incentive. I think the experience of finding DQable issues and getting feedback on whether they are valid are is extremely valuable experience for a BN applicant anyways. Maybe a reduced cooldown for those who have exemplary performance in the QAT (i.e. if they applied and got a 3 month, but they have 10 QF checks a month and have been correct every time then maybe that could be reduced to 2 months instead). This would also give another information source for NAT doing BN application evals to make a more informed decision.

@Eph:
You (staff) are the ones responsible for incentivizing BNs to do the bare minimum. BNs are already "compelled" by way of evals to do that - with fairly good results, honestly, judging by the amount of recent unranks (0) - but we have seen that being more punitive does not improve BN work. Instead, as I mentioned earlier, basic incentives like *SUPPORTER FOR GOOD, ACTIVE BNS FOR EXAMPLE* would go a long way in further ensuring BNs put more care into their nominations and activity. You say nobody wants to put things in place - what more can volunteers put in place? Volunteers have already contributed the following to the mapping community:
- tools to check almost every unrankable
- a gigantic site to organize the official release of licensed content
- a similarly large site to support NAT administrative work like BN evals and QAH activity
- tools to hitsound maps
- tools to create exotic sliders
- tools to create amazing storyboards, and plenty more.
All of these things should have been provided or at least supported by you, the dev team, as part of the development of the infrastructure of the game. Instead, we got the amazing and renowned, very useful, and tremendously inventive HYPE SYSTEM!!!! woo hoo!, we got forced to use the new site which removed several aspects of modding (embedded images, formatting), made the process more tedious, didn't even remove irrelevant parts like kudosu or hypes, still doesn't have GDer functions or support despite being requested since day 1, and BNs got an amazing reward of 2!!!! entire months of supporter after being a BN for a year.
And then you ask what to do?
Support the fucking mapping scene instead of freezing everything since 2014 while lazer is deeeeefinitely getting released this year guys!!!!

@White: identifying unrankable issues (aside from spread concerns) is not modding. While objectively speaking fixing unrankables improves a map, pointing out unrankables is not perceived as adding significant value to the map as map quality has moved far beyond simply following the ranking criteria. Being able to find unrankables issues is not predictive of a modder's modding ability, which is why generally speaking that is not a criteria used in BN app evals beyond a bare-minimum to meet.
The idea does seem fine in conception and may solve the issue if people are actually motivated to do it. I just don't think they would long-term due to the "menial" nature of the work.

I decided to look over what was said and decided to do some adjustments of my own since I think the -White idea is a good compromise.

Consider this written this way to be easier to digested of what -White wrote + how I would go about such at the same time

Dedicated test + 200 kd requirement

- Whilst this still needs to be more lenient than getting into the bng itself, this still needs to be in place to prevent just anyone from getting in
(Optional - Also could add at least 2 mods to demonstrate their ability to check maps)

Incentives

- Whilst if you are a bn already, this may not sound appealing, **but** for anyone else, this can be the second best thing if they aren’t qualified for bn at this time

Things such:
- Quality Assurance Helper title and mini badge
- After 6 months of working on QA and after some evaluation, they can earn themselves a QA badge for their helpful doing


**Requirement**

- This shouldn’t be too discouraging but also shouldn’t be too lenient for anyone doing this either.

A good number would be at least 4 QA per month to maintain the title is enough to keep interest.

Leniency on BN app

Probably a good idea depending on how well the person is doing QA checks which should be evaluated by a NAT

I think the best this should do is reduce the cooldown by 1 month but not any more leniency than that.

^ Basically doing volunteer QA gives them more modding experience and it also helps demonstrate some of their modding ability (especially if they do things such as spread mods and whatnot.) like a resume for a job demonstrating your other abilities.


As long as these people above have no form of power such as dqing, I think having some dedicated group to checking issues as they work their way up to BN, where this may no longer be needed to them is a good way to have a balance cycle and since this still has an incentive for a badge and such, BNs themselves might want to join in (even if for a short period of time.)

I think the only thing that can be a problem with this is that it still needs management specifically for this (such as evaluating for badge or whether they should get a bn app cd i.g) as well as to needs to be tested if people actually be interested in it but I imagine that wouldn't be hard to do either.

hm i think we still miss the main issue which is motivation, a badge is just a badge.. they would just go for BN in that case. also i think we're on a different page about what QAH means since as described before it's about finding unrankables and nothing else, i'm assuming you also mean that they would be checking more important stuff like vetoable suggestions and bigger scope problems like spread and gameplay/design issues. in that case i believe bns would be more appropriate for the job since they have the experience to look up for those, which leaves us into the same place as before

Crissa wrote:

hm i think we still miss the main issue which is motivation, a badge is just a badge.. they would just go for BN in that case. also i think we're on a different page about what QAH means since as described before it's about finding unrankables and nothing else, i'm assuming you also mean that they would be checking more important stuff like vetoable suggestions and bigger scope problems like spread and gameplay/design issues. in that case i believe bns would be more appropriate for the job since they have the experience to look up for those, which leaves us into the same place as before

The thing is with BNs as it clearly has been shown, nobody is interested in it BN-wise and considering that vast majority of options (or what little of since there hasn't been any other solution) has been already turn down, it is time to think of something more likely to be kept up

and whilst a badge and title is still just a badge in title, it still something more than a person who doesn't already have the benefits of being a BN to come in place

Another thing, they can still go for BN while they are doing this considering there is plenty of people who can look for unrankables and other issues but still has something lacking to be a BN at the time and if they are all active and modding anyways, what harm is them to do this on their free time if they so choose?

Whilst we can't fully trust these people to do a BN's job, there is a reason why they won't have things such being able to dq a map and still need a BNs stamp of approval to dq their issues.

I still would like to wait for other solutions, but it is quite obvious this thread hasn't really gone too far on that end as it has been disagreement upon disagreement and as said, finding a solution to get BNs to do it is nearly impossible unless you create actual gain (such as the previous proposal I brought up here)

adding a whole system to a process that's not even needed to become a BN doesn't seem like a good idea, would just make a lot of participants frustrated and it's extra work to be handled by the NAT, do they want that?

your initial proposal makes more sense as the NAT would pretty much continue doing what they are doing while a new team perform qah (+nominations when they feel like).


also @Naoto Mori is right, doing qah won't really make normal users more qualified to become BN.
modding pending maps from mappers of all skill levels is how you gain experience.

pimp wrote:

adding a whole system to a process that's not even needed to become a BN doesn't seem like a good idea, would just make a lot of participants frustrated and it's extra work to be handled by the NAT, do they want that?

your initial proposal makes more sense as the NAT would pretty much continue doing what they are doing while a new team perform qah (+nominations when they feel like).

Fyi this isn't my proposal I was just adjusting the one that was brought up into this thread

and whilst I do think my other proposal would be better, it is quite clear NAT doesn't want to implement such and it doesn't feel like I can open room to discuss more on that if they continue to dismiss it rather than looking for ways to see how it can be better implemented

pimp wrote:

adding a whole system to a process that's not even needed to become a BN doesn't seem like a good idea, would just make a lot of participants frustrated and it's extra work to be handled by the NAT, do they want that?

Not really sure why this matters to the BN process? Nobody said it would become needed to become a BN, Cheri just suggested that it would give a nice badge and at MOST a 1 month cooldown reduction on a failed application. Of course, the specifics can be fine tuned. I'm also not really sure why the extra NAT work is an issue? I'm sure they'd be able to dedicate one member of the team to actually lead the team and distribute the DQ workload itself amongst the entire BNG at the very least. If they need to bring on another NAT to do that, sure, I don't think there is any downside to that at all. Hell, Cheri might even do it lol!

pimp wrote:

your initial proposal makes more sense as the NAT would pretty much continue doing what they are doing while a new team perform qah (+nominations when they feel like).

Isn't that just a new BNG? Might as well expand the BNG at that point, which would require lowering standards, which I don't think anyone suggested as a potential solution.

presumably this new team or new members of nat would be recruited from the current BN.

lowering standards doesn't seem like a solution to the problem that's being discussed here.

Two issues with the ongoing idea:

If QA would be a way to prepare for BN, adding dedicated and effective users who do QA to BN from QA could be a loss, as they'd become another BN who doesn't do QA. This would make promoting them a bonus for themselves but a loss to the system, so I don't think a hierarchy or progression like that would be beneficial in the long run.

____
Video version of following essay -> https://www.youtube.com/watch?v=sdxqz_AceW8

The following is a personal anecdote and issue

Back when metadata helpers were fully active in late 2017 and early 2018, what happened was BNs stopped trying to check metadata themselves at all whatsoever.

They instead started assuming the metadata helpers would check it for them, and became lazier.

This put more and more strain on the metadata helpers until most of them burned out and stopped checking qualified at all. Nearly every other map checked had metadata issues at one point. I was the only person actively doing metadata checks for most of early 2018, doing 369 metadata checks from December 2017 to June 2018, basically handling all of Qualified entirely by myself from March through June, so I know exactly how Cheri feels about doing QAH alone. I ended up working hours upon hours with others on efforts to write every unwritten rule into RC to point and force BNs to do themselves what they should have been doing to begin with. And then I quit doing active metadata checks on qualified myself. I was the last one standing for a long time and had totally burned out.

The attitude of "just qualify it to get meta checked lol" still persists to this day among multiple BNs, despite the entire RC being rewritten to enable their independence of doing metadata checks on their own, and despite the fact metadata helpers no longer even exist. That kind of attitude is one of very few things that will visibly piss me off, and that's when I make enough efforts to control my temper that most people regard me as nice or whatever.

This whole thread to me reads like Cheri is trying to do the same thing and save her sanity, but unlike metadata, there isn't a million unwritten rules we need to add to RC to help fix and address the problem by putting the power back in BN's hands.

It's been 3 years and people still primarily associate me with metadata and nothing else despite everything else I do, and I still get numerous messages a week from people asking me to check meta for them. I'll usually do it, but I really don't like the fact people still need to ask me.

-

My concern with QA is if it's too strong of a system, is BNs lowering their own standards yet again and using that as an excuse to be lazy in their own checks, assuming someone else will be more thorough than them later. Or whatever.

Some of this has also happened, to an extent, with QA, where BNs would argue it's not their fault if things get unranked because QA didn't check it, and try to push that responsibility onto others. However this has been less of an issue than before due to QA as an active system being... basically dead. My concern is with a fully fledged dedicated QA group as to where responsibilities lay and how to handle DQs versus unranks going forward.

____

Also yeah a shitton of ideas and things are limited by the only tools we have control over as NAT being third party ones. A lot of ideas discussed in the past end up dying off if it requires any actual on-site implementation. e.g. opening QA marks on maps to the general public won't work well if it can only be handled via the BN site. Most regular users would never even know it exists.

i don't really see the point in this

if the new QA can't dq, we're basically not changing anything. anyone can go on a qf map and post their concern as of now, it will be sent to alert-feed to most BNs and any that agrees with the post can dq

the only reason people might join it is if being a good QA makes joining BNG easier (as proposed), people will just join for 1-2 months until they can get BN easily and i can see so much going wrong because of that. QA will just be a group made to join BNG easily and having more BNs that don't always have the required modding ability doesn't sound good

also don't forget doing qah isn't only about spotting unrankables lol

ok but why do bns still not have any supporter

if youre coming up with a reward system then give it to everyone in the mean time. its like the least that can happen to actually make us feel somewhat responsible for anything to do with the game other than what we feel like doing.

if you want people to do qa (read: QUALITY assurance, not rankability assurance) you have to fix mapper's attitudes when it comes to dqing maps (or even just modding qualified maps) but gl ever doing that

even if there were actual rewards they won't be worth getting shit on by half of the community just because you dq'd a map for something that wasn't an unrankable (just look at cheri's most recent veto and how that was received)


oh also big agree with everything nao said but I don't expect to see any changes

Nao Tomori wrote:

@Eph:
You (staff) are the ones responsible for incentivizing BNs to do the bare minimum. BNs are already "compelled" by way of evals to do that - with fairly good results, honestly, judging by the amount of recent unranks (0) - but we have seen that being more punitive does not improve BN work. Instead, as I mentioned earlier, basic incentives like *SUPPORTER FOR GOOD, ACTIVE BNS FOR EXAMPLE* would go a long way in further ensuring BNs put more care into their nominations and activity. You say nobody wants to put things in place - what more can volunteers put in place? Volunteers have already contributed the following to the mapping community:
- tools to check almost every unrankable
- a gigantic site to organize the official release of licensed content
- a similarly large site to support NAT administrative work like BN evals and QAH activity
- tools to hitsound maps
- tools to create exotic sliders
- tools to create amazing storyboards, and plenty more.
All of these things should have been provided or at least supported by you, the dev team, as part of the development of the infrastructure of the game. Instead, we got the amazing and renowned, very useful, and tremendously inventive HYPE SYSTEM!!!! woo hoo!, we got forced to use the new site which removed several aspects of modding (embedded images, formatting), made the process more tedious, didn't even remove irrelevant parts like kudosu or hypes, still doesn't have GDer functions or support despite being requested since day 1, and BNs got an amazing reward of 2!!!! entire months of supporter after being a BN for a year.
And then you ask what to do?
Support the fucking mapping scene instead of freezing everything since 2014 while lazer is deeeeefinitely getting released this year guys!!!!

can't really argue with any of this, we're definitely not doing anywhere near as much as we should be from the dev side of things. i've raised this internally for further discussion (citing your post directly), will see what comes of it

QA is just something no one wants to do, its unincentivized and uninteresting. doing it is the equivalent of doing a unrankable check on some map you have no interest or investment in.
the issue is if theres no one doing it we would likely have more unranks.

So then just to make it simple you got a few options,
incentivze it by either forcing bns to have some sort of qa activity each month (which sucks but i dont think thats really avoidable in this situation)
Or just accept that is an unwanted postion and treat it like some sort of job where people doing it get constant month to month payment, badge updates, titles anything cause youre not gonna get people to do volunteer work like this out of the goodness of their hearts no matter how much they care about the community/game. (this also sucks since i doubt there will be any sort of offical support for this and if there is it will likely be less then desirable for staff or the people receiving)

what about a mug? 👍

True af unironically since its kinda interesting as to how receptive bns were to such an idea

Kudosu wrote:

what about a mug?

the event mugs are literally being funded by me and seto kousuke (funds from the bn roast stream are all going to that for starters) and maybe ephemeral (he offered more help after it got started) and would be really another thing on nao's list about things other community members are doing instead of actual from the top support.

i assume novelty would wear off as a permanent item hence different designs for each event to help counteract that (if future events are held)

half the reason i had that idea was because getting bns at least supporter is a thing that for some reason hasn't happened and can't seem to happen despite osu!direct enabling them to do their work more easily among other things

Noffy wrote:

Kudosu wrote:

what about a mug?

the event mugs are literally being funded by me and seto kousuke (funds from the bn roast stream are all going to that for starters) and maybe ephemeral (he offered more help after it got started) and would be really another thing on nao's list about things other community members are doing instead of actual from the top support.

tbh even then most ppl would probably just dq maps for unrankables like unsnaps or metadata. there really isn't many people that want to go through the hassle of dqing for quality concerns when you'll almost systematically face mappers backlash, even if rewarded. nobody feels like its their task

if only there were a way to enfore standards on bns (and without people getting mad at how elitist the ranked section becomes)

@Noffy

Whilst it is true that the point is to reduce the amount of work that a singular person to do Quality assurance, but amidst all of that the biggest thing people are failing to acknowledge within this thread is a singular person who is doing all of this checking within the system is a bn

Even you were acknowledge by QAT/NAT at the time with your ability to be promoted to a higher status, but yet the only thing I ever gotten was being criticized and pure hate even when the vetos/mods were justified whilst still not even considered at all fit for a higher position quite literal no reason at this point and many would agree on such.
(My memory may be fading as I have not remember full details on how you were fully promoted since it been a couple years but my point still stands regardless of such.)

Having things such as meme evaluations stating something similar along the lines “oh wow you did so much QA” basically in the amidst of all of this just felt belittling, and like the work I'm doing wasn't worth anything to you.

If it weren't for all of above ^, then I wouldn't even complain about being the only one (although I still make this thread to improve the system obviously because I do care about it).

Now leaving my personal issue aside, I am fully aware that this is a problem that needs to be discussed more internally above ^ but being that I have yet to even got that opportunity (dear lord I tried.) then placing in the thread is all I can do here.

EDITED: Arguably I do still wish to not have this conversation go any further on the thread since it's personal and I just simply wanted to speak out on it and also do not want to go off-topic even more when I am trying to find a solution (as it quite already is going off-topic too much).

so if you have anything against what I said please take it to dms.


Now as far as the proposal that was presented, I am aware it still has issues and such once they get BN, they won’t do QA anymore, but i’m trying to remain hopeful that it could succeed into having a continuous amount of people willing to come forward and once again, we won’t know if that would actually happen as stated unless tested (which obviously needs dev support,etc)

Ephemeral wrote:

Nao Tomori wrote:

@Eph:
You (staff) are the ones responsible for incentivizing BNs to do the bare minimum. BNs are already "compelled" by way of evals to do that - with fairly good results, honestly, judging by the amount of recent unranks (0) - but we have seen that being more punitive does not improve BN work. Instead, as I mentioned earlier, basic incentives like *SUPPORTER FOR GOOD, ACTIVE BNS FOR EXAMPLE* would go a long way in further ensuring BNs put more care into their nominations and activity. You say nobody wants to put things in place - what more can volunteers put in place? Volunteers have already contributed the following to the mapping community:
- tools to check almost every unrankable
- a gigantic site to organize the official release of licensed content
- a similarly large site to support NAT administrative work like BN evals and QAH activity
- tools to hitsound maps
- tools to create exotic sliders
- tools to create amazing storyboards, and plenty more.
All of these things should have been provided or at least supported by you, the dev team, as part of the development of the infrastructure of the game. Instead, we got the amazing and renowned, very useful, and tremendously inventive HYPE SYSTEM!!!! woo hoo!, we got forced to use the new site which removed several aspects of modding (embedded images, formatting), made the process more tedious, didn't even remove irrelevant parts like kudosu or hypes, still doesn't have GDer functions or support despite being requested since day 1, and BNs got an amazing reward of 2!!!! entire months of supporter after being a BN for a year.
And then you ask what to do?
Support the fucking mapping scene instead of freezing everything since 2014 while lazer is deeeeefinitely getting released this year guys!!!!

can't really argue with any of this, we're definitely not doing anywhere near as much as we should be from the dev side of things. i've raised this internally for further discussion (citing your post directly), will see what comes of it

i can argue with a lot of it. pretty entitled viewpoint on how the game should be operating. seems to be missing the part where the whole game's development, including decisions of what gets priority, is all in the open. if something isn't getting priority, it's because there is no push for it.

how about pushing forward your proposals of what *you* think should be done to fix things, word for word, rather than expecting some magic devs to fix something which if intrinsically hard to balance, implement, and provide a pleasing experience for everyone involved?

the fact you are splitting "the community" and "the team" in your head is an issue in my opinion. you may forget that a lot of what is done on osu-web is also planned by the same community, albeit a portion that is putting in that extra effort to make things happen on the main site or client.

your perspective is entitled and a touch saddening, and i'm thoroughly confused with how ephemeral can agree with this. i'm not even taking offence from it; it's just unfortunate that with all the work i've been doing to get every system transparent (and spending hundreds of hours replying to *every last* issue thread on github across 10+ projects, regularly reprioritising / reassessing feature requests as the community bumps them).

if you are looking for specific improvements, PLEASE go on osu-web, search for the thing you think needs attention and let us know with a reply. If it's not already there, open a new issue. you can meme that this is what i say in response to anything these days, but i can't stress enough that this is how meaningful action can be taken.

btw this is a terrible place for that conversation, let's stay on whatever the topic is

oh and let me add: the people that have created the systems you mention should be proud of themselves. also i hope they had fun doing so. because at the end of the day you should not forget that: this whole thing is here to enjoy.

that should include the roles of BN, QAT and DEV.

the primary goal should be to make things fun and avoid the situation of people shitting on the system for not providing the experience they want.

adding rewards is only a band aid IMHO. keep that in mind when thinking about the next steps forward. every new system or reward we add will be temporary adrenaline which the members of the team will become complacent with over time.

keeping things changing is about the best hope we have to keep people engaged, enjoying and interested in mapping. whether that be changing the mapping scene via ranking criteria changes, adding new systems which allow different ways to enjoy mapping or modding (stuff like the mappers' guild) etc.

Cheri wrote:

Even you were acknowledge by QAT/NAT at the time with your ability to be promoted to a higher status, but yet the only thing I ever gotten was being criticised and pure hate even when the vetos/mods were justified whilst still not even considered at all fit for a higher position quite literal no reason at this point and many would agree on such.

This is incorrect. I was looked over for promotion and two other metadata helpers were made QAT in early 2018 instead. Which left me sorely bitter especially when I became the most active member later on but was stuck having to ask other people to do DQs for me. Metadata QAT were disbanded entirely in early 2018 as well. My promotion in November 2018 was not because of my contribution in that area alone but general community presence. Nor did my efforts in metadata checks ever get acknowledged besides by Kurai personally and people taking advantage of my kindness to ask me to do more checks for them. So I really mean it when I say I can relate. I had already quit doing metadata checks for months by that point.

And the only time QAH itself was relevant to QAT promotions were 2018 and entirely done away with when we changed to NAT, because QAH no longer showed proficiency in primary NAT functions..

That NAT socially have more authority is a hold over from QAT days that I'm not sure how to fix. We always remind people of the fact that in modding discussions we are equal to BN, but it never really seem to set in for community point of view. And I know you mentioned before you feel like you could do more with that authority, and while that's probably true it doesn't make NAT the right fit for you either.

While I get that having a seperate QA group could help with that issue in particular, there's still a lot of other problems we would need to adjust for first. Such as how it'd make a group of people solely for putting other's maps down, and being subject to public contempt even moreso than BNs who do the work are already. This would be a demotivator that I'm not sure would balance out any pluses.

Cheri wrote:

Having things such as meme evaluations stating something similar along the lines “oh wow you did so much QA” basically in the amidst of all of this just felt belittling, and like the work I'm doing wasn't worth anything to you.

I am sincerely sorry that our evaluation messages, which are made to be positive and make people smile about, were done in a way which offended you. I hope I phrased that grammatically correct, I really mean it. It was never the case that the effort was not worth anything to us.

That said, there was a statement earlier in the thread along the lines of how badges or titles would not be worthwhile to consider as a base form for motivators because it isn't appreciated by current QAH, but when that is a sample size of 1 that's really not an accurate view to take. Especially when things other BNs do appreciate are actively disliked by you so we can already see that people's personal preferences don't really match up there.



_________


Apparently peppy posted another reply while i was writing this so i feel like this thread is starting to split into two ongoing conversations, oh no.

peppy:
I find it hilarious you declare me to be entitled when the only reason this game makes money for you is off the back of thousands and thousands of unrewarded and unsupported work being done by bored college students in the complete absence of useful development being done. I'm not interested in discussing specific improvements because you have no clue what any of the things I listed are and will not even consider the type of reforms I would consider useful paths forward to improving the ranked section, like licensing software made by community members, paying them to develop them full time, integrating them into lazer, and then creating actual infrastructure and rewards framework for BNs and NATs beyond the current comically insulting $4 a month in supporter that they get for free. None of these things would ever happen off an osu-web proposal. Don't even try to imply that useful progress is made and not just random one-off fluff things that do not contribute long term. It's insulting to everyone that cares about the game and stays informed about what the community wants.

peppy wrote:

how about pushing forward your proposals of what *you* think should be done to fix things, word for word, rather than expecting some magic devs to fix something which if intrinsically hard to balance, implement, and provide a pleasing experience for everyone involved?

he has, and they have. i've been translating as many reasonable things into bite-sized issues that i think are feasible for short-term implementation (with my limited knowledge of what is required of osu-web scope), something that you can see in action here across the 40~ ones i've made so far. naxess has also pushed a bunch - the running trend with these is that they're proposals which don't culminate in much discussion dev side, then get forgotten.

the reason why they don't get more fleshed out is that people don't fully know where the boundaries lie (github also complicates this for the layperson), how long-term a particular solution will be (as in, how long before it is replaced with something else?) and what the endgame is for the system. there's no real roadmap for any of this, no cohesive idea of where we really should be going.

some of them have been addressed, but the meatier ones have not and this is to be expected to some degree since they involve a fair bit more than just surface level changes, especially given the history of the modding/mapping scene and said changes (aka: bouncing around from one temporary solution to another over the course of a decade, with BAT/MAT/QAT/NAT).

peppy wrote:

your perspective is entitled and a touch saddening, and i'm thoroughly confused with how ephemeral can agree with this. i'm not even taking offence from it; it's just unfortunate that with all the work i've been doing to get every system transparent (and spending hundreds of hours replying to *every last* issue thread on github across 10+ projects, regularly reprioritising / reassessing feature requests as the community bumps them).

nobody is faulting you personally here for "not doing enough" despite what nao's post might sound like. or maybe he is, idk, but he doesn't see the full scope of things. it's just that there's been no clear, concrete direction to push for specific and actionable changes that would do something in the scene for a while now - perhaps i'm to blame for that.

peppy wrote:

if you are looking for specific improvements, PLEASE go on osu-web, search for the thing you think needs attention and let us know with a reply. If it's not already there, open a new issue. you can meme that this is what i say in response to anything these days, but i can't stress enough that this is how meaningful action can be taken.

if you take anything away from this thread, the stuff outlined in this issue are at the very least a basic stepping stone for getting the reward/incentive game BN-side back to where it should be. i've seen BNs regularly getting frustrated that they've run out of supporter across a variety of channels which is an absolutely horrifying thing to witness when these people are driving like 95% of the content inflow currently. if nothing else, we should at least give them the perma-supporter (while titled) that the old BAT had, since that's basically what they are now.

Noffy wrote:

Kudosu wrote:

what about a mug?

half the reason i had that idea was because getting bns at least supporter is a thing that for some reason hasn't happened and can't seem to happen despite osu!direct enabling them to do their work more easily among other things

will look into potentially making this happen, but let me touch on a couple of reasons as to why it hasn't happened yet (from my end).

firstly, the original aim of BNs was to have the group expand out into the hundreds, and be an automatic role in some kind of modding tier system. obviously we haven't really got to that point yet, but that was the reason for not wanting give-then-take a thing.

secondly, if the supporter requirement is just for osu!direct, then that has already been resolved on lazer. it would be amazing if we could start getting modders to move over to lazer, as that is one of the early goals i would like to hit. the editor is already in a pretty stable state but things like modding have their own subset of requirements, many of which may not yet be present. if a list was made i could most definitely give priority to the remaining requirements though.

maybe it's something which will get more people involved on the development side, aka. getting features in the client itself rather than third party systems.

peppy wrote:

Noffy wrote:

[quote="Kudosu"]what about a mug?

it would be amazing if we could start getting modders to move over to lazer, as that is one of the early goals i would like to hit. the editor is already in a pretty stable state but things like modding have their own subset of requirements, many of which may not yet be present. if a list was made i could most definitely give priority to the remaining requirements though.

i would like to help give feedback for this and try modding on lazer - is there somewhere would i be able to give feedback?

There's a simple solution if you don't want to incentivize doing QAH with some form of reward, whether tangible or in the form of preparation to become a BN.

Abolish QAH and don't look back.

QAH even in a working form (where there are enough people to actually check all qualified maps) does not improve the perceived quality of the ranked section in my view. Beyond unrankable issues, mp3 quality and metadata, the kind of issues QAH finds are not relevant for 95%+ of the playerbase. This is my opinion and what I have picked up from general sentiment, feel free to disagree.

Even if QAH is done on a map and issues are fixed, the mapper always tends to apply the bare minimum fixes needed for rank. The overall quality of the mapset is barely ever improved in a noticeable way.

Controversial maps a la Super Driver or Big Black receive attention on their own, without QAH. For other maps, if there's no veto mudslinging fest pretty much any QAH changes become a footnote or forgotten.

It is already the case that perceived ranked section quality is determined almost entirely by the good will/standards of the BNs choosing to nominate maps, as even low quality maps will eventually be ranked no matter what. Embrace this, evaluate BNs harsher on map quality beyond what someone decides to DQ for, and be more strict in BN applications in regards to what the applicants standards for their own work is as well as for others.

Edit since I read the discussion above just now
@peppy
- Supporter is useful not just for osu!direct but also for beatmap upload limits
- Search filters, friend list and favourite list size as well as download speed are also nice for BNs specifically

I did some BN modding over Christmas in lazer (because it actually works pretty well on mac, shout out for that). I can try to write up a more comprehensive list but for me the main issues were
- You can't see hitsound samples on hitobjects
- copying timestamps from in-game to the site doesn't work (though that might be a mac issue?)
- can't turn off hitlighting unless I'm blind and didn't see the setting which is pretty annoying while modding

DeviousPanda wrote:

peppy wrote:

Noffy wrote:

[quote="Kudosu"]what about a mug?

it would be amazing if we could start getting modders to move over to lazer, as that is one of the early goals i would like to hit. the editor is already in a pretty stable state but things like modding have their own subset of requirements, many of which may not yet be present. if a list was made i could most definitely give priority to the remaining requirements though.

i would like to help give feedback for this and try modding on lazer - is there somewhere would i be able to give feedback?

yep! https://github.com/ppy/osu/issues is where you can do this. please make sure to read the issue template and search before posting. generally we try to make one issue per... issue, but for cases like this i'm willing to make an exception if you find it easier to list off all the point in a single issue (myself or someone else will do the splitting out).

also apologies for barging into this thread. i haven't been following closely and don't even know what QAH is, so that should give you an idea of just how much is going on in this community. hopefully the things i've mentioned will give some perspective on how i see changes happening though.

the document in the initial post is a great start and touches on points that likely need to be addressed. we will continue to support any changes that you guys see beneficial (and roll them back if they turn out not to be).

i just ask that we don't start splitting the "community" and the "core team" (as was done in the post i responded to). i can't deal with that easily, because it goes against what i *still* see osu! as being. it's not a game run by some company which is profiting off mappers. if this is your opinion then i'd kindly ask you to take a step out and reassess.

if it helps, try and understand that i think daily about disabling the supporter system, but doing so would basically destroy the game as it is. we support a huge number of artists and run some pretty insane infrastructure, not to mention pay out most of the contributors that help with contests, development, etc.

i never want this game to be about money or profit, so it makes me sick when people take on these perspectives.

i likely won't be follow this thread closely, but will continue as always to watch for any activity that comes from this on github. carry on.

@noffy

"That NAT socially have more authority is a hold over from QAT days that I'm not sure how to fix. We always remind people of the fact that in modding discussions we are equal to BN, but it never really seem to set in for community point of view. And I know you mentioned before you feel like you could do more with that authority, and while that's probably true it doesn't make NAT the right fit for you either."

Regardless of if you think she deserves NAT, or if you think QA isn't a job of NAT, there's no denying the fact that abilities NAT have (moderation perms, supporter, perceived power through having a different title) would be beneficial for someone doing a shit ton of QA.

Regardless of if "BNs are supposed to have equal authority in threads" just saying that doesn't change the fact that it isn't perceived that way by the majority of BNs and mappers.

The title alone would do wonders, and with the amount of times DQ threads get out of hand and off topic I don't see why basically the only person actively making said threads shouldn't be able to moderate them herself.

Also saying that "QA isn't a primary function of NAT so you shouldn't be NAT" is a little funny to me since there are people like Naxess in NAT who are just there for dev things which yes aid with the primary function of NAT, but aren't the primary function.

You could also make the argument that QA also directly aids the primary goal of NAT, which is evaluating existing BNs, and looking over new applicants. BN evals are, from my understanding, heavily dependent on the amount/severity of DQs a given BN gets, so assuming cheri stopped DQing maps there would be next to no DQs in general, and NAT would have a lot less to work with when it comes time for evals.

In regards to creating a QA group:

You say that one of the major issues is that a QA group would essentially be made to just put mappers down and force their opinions on them. I'd like to ask you, how is that any different from the DQs BNs make on qualified maps? You said yourself that BNs and NAT have equal power on a modding thread, so why would it be any different with a QA group? I'd like to argue it being a separate, smaller group would even make it easier to prevent unreasonable DQs considering managing the actions of around 5 people is easier than managing the actions of close to 100.

The other issue you mentioned in regards to people not liking backlash from DQs and the likes is fairly easily solved by

1. picking people who aren't as bothered by it to begin with

and

2. getting a title with moderation powers, and higher perceived authority

obviously that would require seeing if there are enough people that fit the bill, but as shown by cheri, those people definitely exist and there would be no harm in looking.

(i have more to say on the topic of issues with proposed solutions and how to deal with them, but i am trying to think more on it, so i can post something more in depth)

I mean, if the Dev team is able to consider giving supporter to BNs: what if supporter was only given to BNs who met a minimum QA activity threshold? BN are already motivated to do nominations, it's the QA that needs the motivation.

peppy wrote:

oh and let me add: the people that have created the systems you mention should be proud of themselves. also i hope they had fun doing so. because at the end of the day you should not forget that: this whole thing is here to enjoy.
that should include the roles of BN, QAT and DEV.

peppy wrote:

i never want this game to be about money or profit, so it makes me sick when people take on these perspectives.

Except you're a company in legal terms and you're profiting with unpaid third party intellectual property, be it the music or the game levels themselves. Not sure what type of illusion you put on yourself but you're a company that makes profit, properlly registered with civil responsability (Law term which idk the American English translation for).

Even from a game dev perspective, we all know that the role of the game level designers is still considered part of the actual game dev team work. Mappers are not "game modders" with third party creations added into the code of the game, they're literally making the game content itself. I won't mention and go into details about BNs, GMTs, NATs, etc, but all I can say is that the only thing you're trully safe about is the people working through github, couse that license is useful and handy for this stuff.

Also, I just want to add that this is not an attack of any sorts or me pointing fingers at you. It's just a member of this community that happens to know one or two things about law and that is also concerned that this whole thing might one day blow up in your face and leaving you wondering "What I could have done to prevent this?". Needless to say, I don't want that to happen, but again, you need to reconsider your whole attitude towards the current content creation done in this game, the people making it work and how you even perceive osu! , couse the days this software could be considered a non-profitable "fan creation" based on osu! tatakae! ouendan are long gone.

@Net0 I have no words. Take my soul and rip it to pieces. All I can hope is that your viewpoint is not shared by the majority, else maybe we should just turn off mapping.

as an aside, the BN now have refreshing supporter for the duration that they hold their title, in line with other things like NAT/GMT. this is a good step forward, and long overdue!

grabs popcorn*

also thx for the supporter

To get back on topic, I'd be interested to know people's reason why they think QAH should exist in the first place and if those goals can even be realistically achieved in any way.

In my mind removing QAH won't be much different from the current state.

Sylvarus wrote:

To get back on topic, I'd be interested to know people's reason why they think QAH should exist in the first place and if those goals can even be achieved in any way.

even though the way I wrote things in the original doc may make it sound like I think QAH should exist, I'm certainly not insisting on that. the numerous suggestions of how to possibly motivate QA was operating under the assumption that the solution would be to fix the existing system, which I don't think is the only solution - replacement altogether is also viable and should be explored. I think I mentioned this actually in saying that one of the possible solutions is to tighten standards for the performance of BNs as a way to improve overall quality.

right now I'm just waiting to see which of these options those who actually have the ability to change things are more interested in adopting as things stand.