forum

"Quality check" a way to relieve Nominators

posted
Total Posts
27
This is a feature request. Feature requests can be voted up by supporters.
Current Priority: +13
show more
Topic Starter
Gohard
You are right but the idea was mainly to make nominators less crowded with requests. If you are a mapper like me trying to get ranked/quality feedback it's really hard to get to BN. Check BN's user pages - they are all closed due to a large number of requests. I spoke to a BN in a personal conversation and they said it would make their work a lot easier (filtering the maps I mean)
xenal
Feature request forum is for the client, not changes to the nomination system. I guess it should go into developement.

Duplicate of I can't find it.


Enforcing it is bad. As said, can't find where this was discussed, but a point was that experienced mapped forced to go through that is just waste of time. At that point, it simply becomes the MAT (a bunch of modders modding maps and can send it to BAT with higher priority because it's already checked).

dennischan wrote:

anyone having over 3 ranked maps and 300 kudosu being able to verify maps. (150-200 kudosu for minigames). The automatic process is to reduce management effort and fit within the framework of modding v2.
from this thread, which have one category similar.
abraker

Chlebozjadacz wrote:

At this point (or even earlier) people with low mapping knowledge start to flood Nominators with requests, often making it harder for actually good maps to reach the BN's. And because of that BN's have a lot of work explaining basic stuff which can often be pointless as many new mappers give up or deny the mods (the it's my style syndrome). I would like this problem to be fixed because as one of the mappers who try to get to a nominator, I can't reach them because they are overflowed with requests.
if the map is going to a nominator, then it should have gone through at least a couple modders who give feedback. Essentially, modders are current quality checkers. Why would a nominator accept maps that have not gone through proper checks by modders? If there more requests than nominators can handle, then we need more nominators

then again . . .

Suppose you have a couple modders that check parts of the map and still leave a ton of issues. That map is still not fit for nomination, and a waste of a nominators time. I think this can be easily solved even now if nominators require mappers to have playtesters comment in discussion that they approve the map for ranking.
Death

xenal wrote:

Feature request forum is for the client, not changes to the nomination system. I guess it should go into developement.
I've never seen this rule.
Your Good Self
edit

I just thought of this, what if instead of an official "Quality Checker" we can have each BN take it offline if they feel it's necessary. There's nothing stopping a BN from outsourcing their entire BN queue to a bunch of monkeys known acquaintances and boom, you have your quality checking system.


abraker wrote:

I think this can be easily solved even now if nominators require mappers to have playtesters comment in discussion that they approve the map for ranking.


Correct me if I'm wrong, but I believe this is supposedly what the Hype (formerly Star Priority) system is for. I say supposedly because I do admit that Hype is a poor indicator of map quality, and at times it feels more like a song popularity contest. The users hyping the map have zero modding experience (0 kudosu) and near-zero mapping experience.

I think we've also established that the problem we're trying to solve is the 'new mapper' low quality maps reaching BNs and wasting their time.
  1. These are very obvious issues that any half decent mapper/modder (i.e. the "Quality Checkers") can identify these issues before reaching the BN. Examples include:
    1. Wrong timing / mapping on 1/16 snap
    2. No sense of structure
    3. 100% hitsound volume
    4. Bad diff spread (lone Easy diff on a song that can have a bigger spread)
  2. The problem only occurs when the mapper has no ranked maps.
    1. After the first ranked map the mapper is expected to know better than to commit 'rookie mistakes'. There can still be other major quality problems, but they should not be as gross as the issues listed above. In such cases it can still go directly to the BN; our job is to filter out the issues listed above.


Tying all this together, I have but one proposal:
If a mapper has no ranked maps, only "Quality Checkers" may hype their maps.


This is to make Hype a more accurate indicator of map quality for new mappers. Experienced mappers can still have their popularity contest with no changes.

Additionally, I don't think managing "Quality Checkers" as an assigned role is a good idea. There seems to be a lot of work required in administering the role, which can further sap the BN's precious time. It should be an automatic criteria (like what xenal quoted earlier). New mapper issues are not that hard to identify so the barrier to entry does not need to be that high, but it must be sufficient to sieve out those who contribute to the mapping/modding community from those who want to hype maps just for the heck of it. The above quoted suggestion of 3 ranked maps and a handful of kudosu seems like a good idea, though it can be adjusted if needed.

Again, if any of my initial assumptions above are wrong, the proposal falls apart. Ignore me if that's the case.
Topic Starter
Gohard
The system that I have explained was a basic idea of how the new system should work, the main point is to make quality maps easier to reach BN's rather than bombarding them with a bunch of mixed stuff. The idea was meant to be discussed over to make it most suitable for managing ranking beatmaps. And I think this change would patrially include changing the osu client so this is in the right thread I guess
abraker

Your Good Self wrote:

Correct me if I'm wrong, but I believe this is supposedly what the Hype (formerly Star Priority) system is for. I say supposedly because I do admit that Hype is a poor indicator of map quality, and at times it feels more like a song popularity contest.
hmmm, tru

Your Good Self wrote:

The users hyping the map have zero modding experience (0 kudosu) and near-zero mapping experience.
That itself shouldn't be a problem. Users that have zero modding/mapping experience can still indicate whether the map is decent or utter shit. What is a problem is that those people can be mapper's friends that hype the map despite it being utter shit, and sadly even same can happen if it were a specialty role like "quality checker" hyping the map.
Topic Starter
Gohard

abraker wrote:

What is a problem is that those people can be mapper's friends that hype the map despite it being utter shit, and sadly even same can happen if it were a specialty role like "quality checker" hyping the map.

As I said ealier, if you were a quality checker that pushes shit friend maps to ranked you would lose your role quickly (nominator would be able to do it). They could be helping friends more to improve their map but thats ok
abraker
Ok... so another thing. Everyone will need to go through a quality checker like they go through a nominator now. If we are making another role because nominators cannot handle the amount of requests they get, wouldn't quality checkers eventually complain they need something to relieve the amount of request they are getting? I understand it's meant to filter, but you still have same number of people that want to make requests.
Sonnyc
This sounds like the old MAT when they can only give proto-bubbles which makes nothing happen but work as a quality checker. I first stepped inside the mapping ecosystem that time, and it was lot better than directly asking a BAT (BN nowadays). Actually there was an attempt, which was the BN-tier system. Tier 1 BNs checked and place a first nomination (bubble), and tier 2 BNs could focus their workloads more on the bubbled ones. But it failed because the amount of tier 2 BN were too less and no maps were actually being qualified.

Anyways I think some real quality indicator other than hype system will help the map being reach to BN and eventually the ranked status, but as it was a system that once exist and gone now, I'm not really sure repeating the same stuff will necessarily be the best. Afaik, the MAT (which is the quality checker in this case) who were able to give proto-bubbles were all promoted to have an ability to actually bubble (nominate) maps because people believed if they can give a pubble (quality indicator), then why should they not get the ability to give real nominations?
Your Good Self

abraker wrote:

Ok... so another thing. Everyone will need to go through a quality checker like they go through a nominator now. If we are making another role because nominators cannot handle the amount of requests they get, wouldn't quality checkers eventually complain they need something to relieve the amount of request they are getting? I understand it's meant to filter, but you still have same number of people that want to make requests.


The idea is that since the QC's job is only to do a rough filter, the entry requirement to be a QC is lower than that of a BN, and thus there will be a lot more QCs than there are BNs (original post suggests up to 10 QCs per BN). This much larger population of QCs is assumed to be able to handle the request volume.

For what it's worth, I still don't think this is needed as an official web feature. As per my edit above, right now a BN can still separately (i.e. managed through discord, forum pm, trello etc.) 'recruit' and manage any number of non-BN modder friends as 'quality checkers' to do the same filtering job. Additional development to make it a common and mandatory feature is not required.
Topic Starter
Gohard
Well I'm talking from a pre first ranked mapper perspective. And even I can see that there is a problem with spamming nominators, so the problem is probably even more serious than I say. If there were 100 times less requests (as it has been in 2012) this system would be redundant but at the current state it's really important. Even downloading and looking for 2 minutes at every map that was sent to you would take majority of your day. Not mentioning that most of people have work/school so BN's are being really picky about maps they check. I have myself asked a lot of BN's to look at my map but most of them before even showing the map say they are too busy - and I totally understand it. Because they are flooded by mixed quality maps. A lot of them daily (one BN said around 20 a day) imagine working on them all daily, you would go nuts after a week
abraker

Your Good Self wrote:

The idea is that since the QC's job is only to do a rough filter, the entry requirement to be a QC is lower than that of a BN, and thus there will be a lot more QCs than there are BNs (original post suggests up to 10 QCs per BN). This much larger population of QCs is assumed to be able to handle the request volume.
Ok, so what should be the requirements for being a quality checker?

Your Good Self wrote:

For what it's worth, I still don't think this is needed as an official web feature. As per my edit above, right now a BN can still separately (i.e. managed through discord, forum pm, trello etc.) 'recruit' and manage any number of non-BN modder friends as 'quality checkers' to do the same filtering job. Additional development to make it a common and mandatory feature is not required.
I think this is important. One fear I have is that if it does become an official role, it will repeat the same two-tier modding status hierarchy cycle that has been all throughout osu!'s history. Higher status modding role is made -> higher status modding role overwhelmed -> lower status modding role is made -> higher status modding role is found ineffective -> higher status modding role is disbanded and lower status modding role becomes the new higher status modding role -> repeat.
Your Good Self

abraker wrote:

Your Good Self wrote:

The idea is that since the QC's job is only to do a rough filter, the entry requirement to be a QC is lower than that of a BN, and thus there will be a lot more QCs than there are BNs (original post suggests up to 10 QCs per BN). This much larger population of QCs is assumed to be able to handle the request volume.
Ok, so what should be the requirements for being a quality checker?

Currently there are a few options being explored, outlined in the updated first post.

One is to have automatic qualification based on mapping/modding activity. I do however concede that without manual regulation of QCs it's still possible for shit maps to reach BN queues, abeit hopefully on a less regular basis. Someone with 3+ ranked maps should know better than to send a clearly unpolished map to BNs.

The other way is to have BNs manage and regulate the pool of QCs, which will be appointed one by one rather than automatically qualified. My concern is that it might turn out to be counterproductive, if there ends up being too much red tape involved in setting up and maintaining the system. If BNs have to manage their own QCs that means even less time to to actual BN checks.

Sonnyc wrote:

This sounds like the old MAT when they can only give proto-bubbles which makes nothing happen but work as a quality checker. I first stepped inside the mapping ecosystem that time, and it was lot better than directly asking a BAT (BN nowadays). Actually there was an attempt, which was the BN-tier system. Tier 1 BNs checked and place a first nomination (bubble), and tier 2 BNs could focus their workloads more on the bubbled ones. But it failed because the amount of tier 2 BN were too less and no maps were actually being qualified.

Anyways I think some real quality indicator other than hype system will help the map being reach to BN and eventually the ranked status, but as it was a system that once exist and gone now, I'm not really sure repeating the same stuff will necessarily be the best. Afaik, the MAT (which is the quality checker in this case) who were able to give proto-bubbles were all promoted to have an ability to actually bubble (nominate) maps because people believed if they can give a pubble (quality indicator), then why should they not get the ability to give real nominations?

Didn't see this post earlier, but I definitely agree. For the record, I wasn't around during the time of MAT/BAT. There was only BAT when I first started mapping.
If we want to bring back a system similar to the MAT/BAT, we'd have to address the concerns raised in the old systems to convince the older folks that history isn't gonna repeat itself.

Sonnyc wrote:

But it failed because the amount of tier 2 BN were too less and no maps were actually being qualified.

If we introduce QC on top of the current system, I wouldn't expect the existing BN population to be affected. In fact, the number of maps being qualified might actually go up as BNs will receive higher quality requests which have been approved by QC.

Sonnyc wrote:

Afaik, the MAT (which is the quality checker in this case) who were able to give proto-bubbles were all promoted to have an ability to actually bubble (nominate) maps because people believed if they can give a pubble (quality indicator), then why should they not get the ability to give real nominations?

Yeah, whoever is in MAT/QC long enough will eventually gain the skills and experience necessary to be a BAT/BN. But IMO that doesn't mean the MAT has to be shut down.
Experienced QCs should be able to promote to BN, while other semi-experienced modders can move into QC. QC must be maintained as a solid middle ground between the average joe and BN.

Like I said above I wasn't around to see MAT activity (or much of osu!'s mapping/modding history for that matter), so I could just be spouting nonsense here.

abraker wrote:

Your Good Self wrote:

For what it's worth, I still don't think this is needed as an official web feature. As per my edit above, right now a BN can still separately (i.e. managed through discord, forum pm, trello etc.) 'recruit' and manage any number of non-BN modder friends as 'quality checkers' to do the same filtering job. Additional development to make it a common and mandatory feature is not required.
I think this is important. One fear I have is that if it does become an official role, it will repeat the same two-tier modding status hierarchy cycle that has been all throughout osu!'s history. Higher status modding role is made -> higher status modding role overwhelmed -> lower status modding role is made -> higher status modding role is found ineffective -> higher status modding role is disbanded and lower status modding role becomes the new higher status modding role -> repeat.

I'm not against the idea of quality checking, just that I'm not entirely sure what the best implementation should be. Maybe a workaround solution already exists outside of the current osu! web features. Workarounds aren't good in the long term, but maybe if those options were explored and found to be good, we'd have a better case to go on here.
Topic Starter
Gohard

abraker wrote:

Your Good Self wrote:

For what it's worth, I still don't think this is needed as an official web feature. As per my edit above, right now a BN can still separately (i.e. managed through discord, forum pm, trello etc.) 'recruit' and manage any number of non-BN modder friends as 'quality checkers' to do the same filtering job. Additional development to make it a common and mandatory feature is not required.
I think this is important. One fear I have is that if it does become an official role, it will repeat the same two-tier modding status hierarchy cycle that has been all throughout osu!'s history. Higher status modding role is made -> higher status modding role overwhelmed -> lower status modding role is made -> higher status modding role is found ineffective -> higher status modding role is disbanded and lower status modding role becomes the new higher status modding role -> repeat.

I don't know why you think that. This system works in any place that manages any ammount of people; country,work etc. You hire people to do less important and requiring job to have more time to manage harder work yourself. It wouldn't make BN's work less important. To become a BN you must be really expirienced which QC wouldn't be capable of. Their work would be to be a rough filter for BN's, and this way they would actually have less work in the end.
anna apple
this idea is retarded


why don't you just mod a map like everyone else lol
Em0nky
hmm checking maps for problems before sending them to BNs. I wonder if there were a 7 letter word starting with moddin and ending with odding that results in maps with problems getting rid of problems.
Topic Starter
Gohard

Lego man wrote:

this idea is retarded


why don't you just mod a map like everyone else lol


I do mod other players maps and other players mod my maps but what about it? In the last month majority of mods I got is aesthetic/spacing mods. And saying this from your perspective is easy because you have 21 ranked maps and you have no problem to get attention from a BN lol

Not mentioning I got maps (NM requests) in my modding queue that had 0 hypes 3 aesthetic mods and were imo ready to get checked by a BN. Spacing emphasis, aesthetics, rhythm, placing was almost perfect, with no attention. Create a good map and you will be rewarded - only if you are already a ranked mapper or song you are mapping is popular (read anime)

I don't talk about myself. Don't get me wrong, I want an equal chance for everyone to get at least to know what they are doing wrong or why their map needs to be worked on if they put effort in making a map. I think it's fair. This way we would finally see some good songs in osu other than chika chika 20 times in a row
anna apple

Chlebozjadacz wrote:

. And saying this from your perspective is easy because you have 21 ranked maps and you have no problem to get attention from a BN lol
so instead of being an impotent modder and not being able to get your point across, this thread is ACTUALLY about you not getting enough BNs on your maps.

how in the world will this ever help you lol

hmmm making the process to nominate a map slightly harder because now u require people to "quality check" your map. hmm
and if you want in on a little secret, if your map is good it will speak for itself, you don't have to sit there and beg for a system to auto push your map to a BN because it was modded. Well good luck with that, BNs still won't like your map so whats the point in having someone "quality check" it lol.

anyways you didn't refute the idea that this system is terrible on its own

--- and for the record, ranking a map wasn't easy for me because i had ranked maps already. It was easy for me because my maps were GOOD, wonder why during the tier system I had a QAT and 2 tier 2 BNs nominate my marathon length map? hmmmm (first ranked map btw)
celerih
The bar for joining the BNG is already pretty low as is, so this 10:1 ratio for QC vs BN would force standards of who could be QC to be so incredibly low. Take for example the BN test, it's pretty easy and just requires you to be able to properly read the Ranking Criteria and even then a bunch of people manage to not even get 50% in that test. Now you're getting a bunch of people at a lower level than that to say oh yeah this is ready i think. This would just make it so a bunch of mappers who aren't actually ready to have their maps nominated be given false hope and in the end be even more disappointed and frustrated, since people with very low standards told them their maps were ready despite no BN actually being willing to nominate it.

So yeah, sounds like a bad idea overall. Yes it's not easy to have your maps be noticed by BNs, that's the whole point it's an incentive to strive for improvement. Ranking a map isn't really something that you deserve, it's something you need to work for.
Topic Starter
Gohard

Lego man wrote:

--- and for the record, ranking a map wasn't easy for me because i had ranked maps already. It was easy for me because my maps were GOOD, wonder why during the tier system I had a QAT and 2 tier 2 BNs nominate my marathon length map? hmmmm (first ranked map btw)

Well, we know already that your ego is big but I won't brag about you there. And mentioning that it wasn't that hard back in the day to be a Nominator/QAT, look at maps from back then for an example

Lego man wrote:

so instead of being an impotent modder and not being able to get your point across, this thread is ACTUALLY about you not getting enough BNs on your maps.

If you mean that this way, it would be about me, you, and any other person. And I do not get not enough BNs, I don't get them at all for the record
anna apple

Chlebozjadacz wrote:

If you mean that this way, it would be about me, you, and any other person. And I do not get not enough BNs, I don't get them at all for the record
could be bc BNs don't like people who whine about not getting their attention
Topic Starter
Gohard
Actually I don't even request to BN's because I can't. Look at BNs profile pages - Closed for request. Wonder why is that. The point of this thread is to help BNs mainly, remember that
celerih

Chlebozjadacz wrote:

Actually I don't even request to BN's because I can't. Look at BNs profile pages - Closed for request. Wonder why is that. The point of this thread is to help BNs mainly, remember that
And how would the main idea of this thread help with that in any way? Checking if a map is okay for rank takes an actual 15 seconds for a BN to do so that's not the part that's actually the issue. It's modding the set and going through the map like a glorified AiMod to make sure no unrankables are in there that is time consuming and the reason why BNs need to be closed after they have a certain amount of maps to check in front of them.

Giving them a list of hey these are fine we think wouldn't help because they'd have to make the same check they already do to see if the maps are fine in their eyes, which once again just takes 15 seconds anyways. So yeah, this system wouldn't really help with much since it's eliminating one of the already easiest steps for a bn to do themselves. If I was a BN while this system was in place, I'd just ignore it since I'd rather have a look and see if the map is ready by myself and my standards
dennischan
I mean celerih is actually a former BN so his opinion is kind of what other BNs also see during their daily work.

And honestly its not only BNs, players and other mappers alike can already get an idea whether they like the map immediately after playing, so making a filter here doesn't help as much as celerih said
Please sign in to reply.

New reply