Guest
home
beatmaps
rankings
community
store
help
forum
  1. Forums
  2. Beatmaps
  3. Mapping Discussion
  4. Beatmap Management

[Discussion/Proposal] Increasing Veto Uphold Threshold [Standard/General]

posted
Total Posts
51
wafer
790 posts
Joined December 2016
Topic Starter
wafer
Currently, the veto threshold is set at 50% (unofficially). After considering various factors, I believe that increasing this threshold to 66% (2/3rds of the BNG) would benefit the community.

Current Threshold: 50%

Proposed New Threshold: 66%

Current RC:
"Following mediation, the beatmap veto will either be dismissed if the proposed changes are deemed unnecessary or detrimental to the beatmap, or upheld if they are considered necessary or beneficial."

Proposed RC:
"Following mediation, the beatmap veto will either be dismissed if the proposed changes are deemed unnecessary or detrimental to the beatmap by less than 66% of the Beatmap Nominator Group, or upheld if they are considered necessary or beneficial by 66% or more of the Beatmap Nominator Group."

A higher threshold ensures that vetoes are only upheld when there is a stronger consensus among the BNG. This reduces the chance of vetoes being upheld by a narrow margin, which might not fully represent the community’s stance. This makes vetoes being upheld feeling more "fair" as a larger margin must agree.

This number also is close to the median of BNs and players who voted that they believed Pika Girl should be ranked (thanks AJT for this point)

% of all users voting pika girl should be ranked: ~85
% of BNs voting it should be ranked: ~55

This averages to around 70%, which is very close to 66% and helps further solidify the 66% as a better threshold.
Last edited by wafer , edited 1 time in total.
Smoke
Beatmap Nominator
377 posts
Joined August 2017
Smoke
agree
riot1133
Beatmap Nominator
552 posts
Joined March 2018
riot1133
i recommend using "two thirds" in the wording instead of 66% to avoid misinterpretations of the rc
Last edited by riot1133 , edited 1 time in total.
wafer
790 posts
Joined December 2016
Topic Starter
wafer
I agree, that makes sense to me.

The updated proposal would be as follows

Current Threshold: Half of the BNG

Proposed New Threshold: Two-thirds of the BNG

Current RC:
"Following mediation, the beatmap veto will either be dismissed if the proposed changes are deemed unnecessary or detrimental to the beatmap, or upheld if they are considered necessary or beneficial."

Proposed RC:
"Following mediation, the beatmap veto will either be dismissed if the proposed changes are deemed unnecessary or detrimental to the beatmap by less than two-thirds of the Beatmap Nominator Group, or upheld if they are considered necessary or beneficial by two-thirds or more of the Beatmap Nominator Group."
Malphs
Beatmap Nominator
276 posts
Joined November 2016
Malphs
I think peppy previously mentioned something around 75-80% would be better, and I agree.

It is a high valley, but it would get the point across much better at pointing out a map as unfitting for the ranked section compared to just 2/3 (66%).
Ryuusei Aika
Elite Mapper
1,116 posts
Joined January 2016
Ryuusei Aika
Agree, one detail is we may want to keep 2/3 instead of 66% to make it easier on many aspects (blah blah maths stuff). Edit: it has already been addressed, nice

Comparing with another proposal that has been raised in the BN server (75%), I think 2/3 -- which leans more to the BNG's favor -- can also give BNs more control on the ranking section without sacrificing too much the community perspective, and also shows more trust & respect to BNG as a team who keeps maintaining the infrastructure of the mapping ecosystem.

While this proposal came from discussion in standard mode I also believe the other modes will benefit from it as well.

Edit 2: Another suggestion came in in the BN server discussion as renaming "veto" as something that sounds less... scary? I am not a master of the nuisance of the English language, but I think something as "objection" may work.
Last edited by Ryuusei Aika , edited 2 times in total.
Basensorex
Beatmap Nominator
313 posts
Joined January 2018
Basensorex
dont think it should be weighted arbitrarily towards bn opinion when what gets ranked affects players far more than it does bns since they are obviously the vast majority of users playing ranked maps

a clean 70 or a 75-80% as peppy originally suggested probably makes more sense since the intention should be to keep vetoes as actual last resorts when quite literally a superhypermajority of bn members can agree something is unrankably bad

the higher the percentage the higher chance to avoid situations where bns get shot for 51-49 veto consensuses and then the next similar veto is 51-47 and the next 51-45 and so on until you reach a point where veto mediations arent actual representations of community opinion but rather representations of whos left alive (this might have already been happening for a while lol)
RandomeLoL
Nomination Assessment Team
222 posts
Joined September 2015
RandomeLoL
I believe this is a fair proposal. Requiring a supermajority rather than a simple majority better reflects the overall views of the BNG. Moreover, the impact that an upheld veto has versus a dismissed one is considerably larger. Requiring a bigger threshold therefore to ensure this is really something worth pursuing changes for is a fair change to be made.

Ryuusei Aika wrote:

Another suggestion came in in the BN server discussion as renaming "veto" as something that sounds less... scary? I am not a master of the nuisance of the English language, but I think something as "objection" may work.
Semantics. The moment people grow accustomed to the next term and fully associate it with the exact same process, it will be perceived the same way. Users with no idea on the term nor the process only really see the DQ and judge whether it's fair or not with their varying degrees of experience.
Noffy
Nomination Assessment Team
1,684 posts
Joined April 2012
Noffy
My only concern about making it a supermajority instead:

With the change to all mode, participation isn't currently required.

But if 30 people don't participate, their votes could have affected the super majority results significantly.

For example on the historia veto, 50 users agreed to one point of the 73 or so that voted. This is a clear supermajority of the people voting, but if somehow the remaining 30 that didn't vote would have been against the veto, the results would be different. With a simple majority it was close enough and a clear enough trend to not be an issue yet, but I think it's worth minding.


Should the veto stay open until 70 votes agree with the veto? Should it auto close after 7 days and anyone that didn't vote lost their chance to say something about it as it is now? Should it stay open until all 100+ people are forced to give input?
wafer
790 posts
Joined December 2016
Topic Starter
wafer
I think the 2nd option (what we have currently, 7 days or forever your hold peace) is the best?

maybe wording needs to be changed to "two-thirds of the voting Beatmap Nominators"? I think that'd clear up confusion and be consistent with what we currently have
Livermorium
295 posts
Joined March 2020
Livermorium
I agree with at least 66 percent to uphold the veto. especially because every bn has to vote requiring a higher threshold for upholding sounds fair. I always thought simple majority for upholding veto was a bit unfair for the mapper since 49percent of the bns can disagree but you are still forced to fix the "problem."
Greenshell
Beatmap Nominator
328 posts
Joined July 2016
Greenshell
the 2/3 majority was proposed a while ago i think and nothing came of it for some reason, personally think 75-80% is a bit much but 2/3 would be a great change
Shad0wStar
Global Moderator
177 posts
Joined April 2020
Shad0wStar
2/3 sounds good
Okoayu
Nomination Assessment Team / Elite Nominator
4,956 posts
Joined May 2012
Okoayu
In case the proposal to delete the system goes nowhere, i think taking the same thresholds as content review does would be better as it's more simple to remember
apollodw
325 posts
Joined January 2017
apollodw
this would be good -- much more preferable to me than outright removing the system. 2/3 or ~75%, either of them seem like fine options
Last edited by apollodw , edited 1 time in total.
AJT
551 posts
Joined August 2013
AJT
I’d take 70% (or higher) as that’s the actual average ((83.45+56.38)/2) and considering the small size of the BNG vs the prospective player base, the BNG are already at a decided advantage

66 is better than 50 though
Last edited by AJT , edited 1 time in total.
Monoseul
Elite Nominator
73 posts
Joined January 2020
Monoseul
+1

Don't really mind if it's 66% or 70% or such, but I definitely agree the current threshold should be higher and will help a lot for not only standard but all gamemodes.

Helps better represent the community to an extent and makes vetoes more "fair" instead of situations like 49% disagree 50% agree which I've always had a sour taste for.
niat0004
220 posts
Joined September 2017
niat0004
Agree. ½ sounds too low for preventing a map from being ranked for subjective reasons; 2/3 (or an even 70% to match the Content Voting Process for images) is better.
Bloxi
Beatmap Nominator
433 posts
Joined September 2016
Bloxi
I think an even 70% is good.
OnosakiHito
osu!taiko Paragon
6,842 posts
Joined March 2010
OnosakiHito

Okoratu wrote:

In case the proposal to delete the system goes nowhere, i think taking the same thresholds as content review does would be better as it's more simple to remember
Basically. So 70% is fine.
A r M i N
1,305 posts
Joined October 2014
A r M i N
Agree, but ppy suggesting 75-80, pika vote averaging at 70 makes 66 a odd number to settle on. Somewhere between 70 and 80 should be optimal
Drum-Hitnormal
Beatmap Nominator
1,727 posts
Joined March 2011
Drum-Hitnormal
can we make it the same threshold number as Content Review?

with NAT vote having same weight as GMT vs BN
wafer
790 posts
Joined December 2016
Topic Starter
wafer
I think seven-tenths is fine too for consistency for content review if that's the general sentiment
RandomeLoL
Nomination Assessment Team
222 posts
Joined September 2015
RandomeLoL
NAT votes shouldn't be valued more than BNs imho. Our nomination responsibilities are on par, and shouldn't necessarily be more valuable than other members in the BN team.
Lytian
5 posts
Joined September 2020
Lytian
agree, its kinda bizarre that half of BNs can disagree with a veto and it still be enforced, especially considering the massive impact a veto has by making whatever the map was vetoed over effectively unrankable unless the BN wants to gamble.
Noffy
Nomination Assessment Team
1,684 posts
Joined April 2012
Noffy
This thread seems to be having a general consensus and traction, so I created a BN vote on the topic as well to get an overview of everyone's opinions on it. BNs, please make sure to give your input and continue posting any additional thoughts on the proposal to this thread while the vote is ongoing.

https://bn.mappersguild.com/discussionvote?id=667aceebbd998423fdc57245
Maxim-Miau
Beatmap Nominator
106 posts
Joined July 2018
Maxim-Miau
makes sense to me so i also agree
SuzumeAyase
Beatmap Nominator
364 posts
Joined March 2019
SuzumeAyase
yes
Rocma
Former osu!catch Champion: 2022
527 posts
Joined November 2010
Rocma
totally agree but I think increasing the amount of member for mediation is first
Maxus
Nomination Assessment Team
1,282 posts
Joined May 2014
Maxus
Agree with this yeah
Furryswan
Elite Nominator
326 posts
Joined January 2017
Furryswan
At least this would prevent the conflicting opinions from the veto that still forces the mapper to change something that vetoers don't even know how.
Last edited by Furryswan , edited 2 times in total.
Gust
Beatmap Nominator
522 posts
Joined December 2011
Gust
Since the quality of a map is generally considered a subjective experience in the current environment, a higher ratio of agreed vetoes indicates a greater consensus that the map lacks quality from an objective standpoint.
A veto with a lower ratio is more likely to only delay the map's entry into the ranked section.
So I think 75~80% would be better, or even higher.
Last edited by Gust , edited 2 times in total.
Sotarks
3,524 posts
Joined May 2014
Sotarks
70% is good, I agree with that proposal.
honne
155 posts
Joined February 2014
honne
66
Tomiup
143 posts
Joined May 2019
Tomiup
yea around 75% should be good enough
MrRainbowDoge
286 posts
Joined July 2022
MrRainbowDoge
ye 70%~ seems fair
tatatat
922 posts
Joined March 2015
tatatat
seems good and I definitely have no bias or ulterior motives. lets rubber stamp it
Noffy
Nomination Assessment Team
1,684 posts
Joined April 2012
Noffy
Vote results are in

https://bn.mappersguild.com/discussionvote?id=667aceebbd998423fdc57245


Raise to 70%: 99/144 votes 68.75%

Other: 14/144 votes 9.72%

Don't change: 31/144 21.53%
Numbers are slightly different from the website since a few gmt voted when it was supposed to be current bn/nat only for now voting wise (sorry guys D: website bug). I edited to account for this and we'll fix the website listing for the vote soon.

I will note that several votes for yes indicated 2/3 would be suitable too, but otherwise the results are fairly clear for what direction we push for in the future.

Most "Other" votes agree with raising, but prefer 2/3

A few notes:
  1. The vote results do not mean a change is implemented yet. This requires a few other things, web updates, wiki updates, etc before it can be finalized.
  2. We need to figure out what to do about recent vetoes that fall in-between 50 and 70%
  3. The next NAT meeting will discuss getting this proposal implemented among other things. I can say it will likely be held as a stage event in osu! Dev, but the date is TBD. It'll be announced through several channels once it's scheduled.

If you have any additional thoughts on the in-between vetoes do feel free to add them here too
Last edited by Noffy , edited 4 times in total.
Basensorex
Beatmap Nominator
313 posts
Joined January 2018
Basensorex
will post what i said in bn server for visibility

assuming the threshold is updated to 70%:
-all past vetoes (or at least the ones from since vetoes became an all-bn thing rather than a small pool) that were upheld by less than 70% should be lifted
-vetoes that were upheld by 70% or more should be allowed to ask for remediation immediately, refreshing whatever remediation cooldown they were on previously

both of these are imo the most charitable but still fair options for mappers of vetoed sets, which should be the priority since vetoes are fundamentally a block on a mappers interpretation of the song and involve them the most
wafer
790 posts
Joined December 2016
Topic Starter
wafer
i agree with what basen said
MrRainbowDoge
286 posts
Joined July 2022
MrRainbowDoge
agree w/ basen
Tomiup
143 posts
Joined May 2019
Tomiup
+1
momoyo
Nomination Assessment Team
246 posts
Joined June 2018
momoyo

Basensorex wrote:

assuming the threshold is updated to 70%:
-all past vetoes (or at least the ones from since vetoes became an all-bn thing rather than a small pool) that were upheld by less than 70% should be lifted
-vetoes that were upheld by 70% or more should be allowed to ask for remediation immediately, refreshing whatever remediation cooldown they were on previously
I disagree with allowing vetoes that would've been upheld by the upcoming threshold to go under immediate mediation, no reason to. BNs are tired to having to deal with mediations all the time and it would've been upheld anyway under both thresholds anyway.

As for the under 70%; we are debating it in order to provide a shorter cooldown to said results, probably either 3 or 6 months but it is still under discussion.
Basensorex
Beatmap Nominator
313 posts
Joined January 2018
Basensorex

momoyo wrote:

As for the under 70%; we are debating it in order to provide a shorter cooldown to said results, probably either 3 or 6 months but it is still under discussion.
any reason why just lifting these vetoes doesnt work? is the assumption here that a different threshold will make people change their mediation consensus voting habits?

if so i dont really see how that would happen since a mediation comes down to whether a bn agrees with a veto to the point of the map needing changes or if they think the map is fine as is, there is no real motivation for people to have "bad faith" votes in which they vote against their actual interest since theres quite literally no strategic benefit to it, and so i dont think consensuses would change willy-nilly

it also seems kinda unfair for all recent vetoes that were upheld by less than 70% to have to wait an extended period of time to have a remediation instead of just lifting them considering all of them have been the catalysts for these changes in the first place. imagine being a mapper and knowing that a veto you fundamentally disagreed with a week ago wouldve not been upheld under new rules but now you're forced to wait another few months to simply have a remediation vote due to the timing being slightly against you

momoyo wrote:

I disagree with allowing vetoes that would've been upheld by the upcoming threshold to go under immediate mediation, no reason to. BNs are tired to having to deal with mediations all the time and it would've been upheld anyway under both thresholds anyway.
i can see where u guys are coming from from a workload and general annoyance perspective so im willing to not adhere to this viewpoint as much, i just think affording mappers of vetoed sets some charity instead of forcing them to continue waiting for their cooldowns when mapping/ranked is changing so much in recent times to allow for more freedom would probably be a good thing
Last edited by Basensorex , edited 6 times in total.
MrRainbowDoge
286 posts
Joined July 2022
MrRainbowDoge

momoyo wrote:

I disagree with allowing vetoes that would've been upheld by the upcoming threshold to go under immediate mediation, no reason to. BNs are tired to having to deal with mediations all the time and it would've been upheld anyway under both thresholds anyway.

As for the under 70%; we are debating it in order to provide a shorter cooldown to said results, probably either 3 or 6 months but it is still under discussion.
ok now thats not fair -1
wafer
790 posts
Joined December 2016
Topic Starter
wafer
I want to agree with what Basen said again,

It doesn't make sense for maps that underwent a remediation recently (going to say < 1 year, since that's the timeframe to propose a new remediation) to undergo ANOTHER remediation, when the results, should in theory, be almost identical as to the original mediation.

Edit:

someone said something in the BN server that I heavily agree with so I'm going to quote it here

Even if retroactively applying isn't possible..

"I personally vouch for immediate re-mediations if the mappers ask for it"

This seems like the best approach without feeling unfair to the mappers for not just waiting until the proposal went through. The option to remediate should also be communicated with the mappers TBH
Last edited by wafer , edited 2 times in total.
clayton
1,917 posts
Joined November 2013
clayton
kind of indifferent to the proposal I guess but I don't think the premise of "represent[ing] the community's stance" makes sense. having more BN required to swing the decision one way or another doesn't make it more representative in general, that only happens if the BNs on average favor upholding a veto more than "the community". this change would make veto outcomes less "representative" in cases where the averages of opinions about the veto are flipped. and pika girl vote's numbers aren't going to be relevant at all for vetoes that aren't concerning similar topics

if representing more people in veto outcomes is a priority then vetoes should try to involve more people...
Remyria
361 posts
Joined June 2012
Remyria
imo, regardless of new % chosen:
if the veto already has the proposed % or more => stay veto'd and no change to cooldown
if the veto has more than 50% but less than the proposed % => lower significantly/reset cooldown on remediation or lift it completely. I don't care either way, I don't have to go through mediation myself lol. I only have a strong opinion about the first case.

otherwise, that seems fair to me. thanks reddit(?)
Net0
Beatmap Nominator
1,272 posts
Joined October 2014
Net0
Small suggestion, but I think that it's important to establish a minimum quorum for the veto voting as well. A recent veto had less than 50% of the BNG involved, the samplesize of opinions feels limited and random that way.

I know it's a burden to deal with vetoes on maps a lot of BNs wouldn't be involved in the first place, but the current system leads to that now that QA enforcement is being done again on qualified and not in BN evaluations.
Serizawa Haruki
1,692 posts
Joined June 2015
Serizawa Haruki
I also think participation in veto mediations should either be mandatory or at least >50% (if not more) of eligible BNs must have voted for the result to be valid.
RandomeLoL
Nomination Assessment Team
222 posts
Joined September 2015
RandomeLoL
Regarding the two posts above, I've bumped a discussion internally regarding the viability of setting a minimum participation threshold. The hard thing is going to be how to properly implement it.

However I agree. A vote with a high abstention rate makes it lose its credibility. Plus given how strong the consequences of an upheld veto are for the mapper, ensuring they get a fair sampleset of votes is the least that we could do.
Please sign in to reply.

New reply

0
1 / 51