Hello, I strongly believe that the current mediation system is not good for the mapping scene as it currently favors the vetoing side through both a social, and mathematical edge, and by not offering the mapper any further recourse. In an inherently creative field such as mapping, this conservative stance toward subjective issues is not good as it inhibits the flow of new or unconventional ideas into the ranked section, thus providing a less varied experience for players. I will first explain the social edge that a vetoing BN has over a mapper who may be unknown or not have many connections in the community, and then I will explain how the current system is mathematically biased in favor of the vetoing BN.

The social edge that a vetoing BN may have is perhaps not as present if the mapper who’s map is being vetoed is a BN themselves or is a well known and connected mapper, however in the case of it being a relatively new mapper who does not have many ties to the community, the current system favors solidarity with your fellow BNs over standing up for the little guy.
The main issue here is that participation in the mediation is basically compulsory, and that the mediating BNs do not have any actual skin in the game. They do not need to put themselves at risk of backlash with their decision, as the BNs who nominated the map or the BN who vetoed the map did. This means that if they do not have particularly strong feelings toward the map or the veto, perhaps slightly agreeing with the vetoing BN but not believing that it is worth vetoing, it is easier for them to either vote agree or neutral to not upset their fellow BN rather than taking a stand and voting disagree.
While the extent of the advantage that these incentives offer may be debatable, there are no similar incentives that would skew the mediating BNs toward voting disagree at a general level, and the presence of this issue is one reason to make accommodations to protect the mapper and the variety of the ranked section.

The mathematical edge toward the vetoing BN is created by the fact that it takes 2 BNs to nominate a map, but only 1 vetoing BN to initiate a mediation. This removes 2 BNs who disagree with the veto (as they nominated the map) from the pool of possible mediators, but only 1 BN who agrees with the veto (because they vetoed it), thus skewing the probability that the majority of the randomly selected mediators votes to uphold the veto in comparison to the sentiment of the entire BNG.
You may ask whether this edge is really such a big deal, so I will elaborate on a hypothetical example of a randomly selected mediation where half of the total full BNs agree with the veto and half disagree.
For taiko, there are currently 23 full BNs, but I will adjust the number to 22 to make the 50/50 split easier to calculate, so there are 11 BNs who agree with the veto (including the vetoing BN), and 11 BNS who disagree with the veto (including the 2 nominating BNs). Since the nominating and vetoing BNs cannot participate in the mediation, this leaves 10 BNs who agree with the veto and 9 BNs who disagree with the veto as possible mediators, from which 7 people are randomly selected (The wiki says 5 for other game modes but for some reason they used 7 people in recent mediations). The probability that a random BN selected from the sample would agree with the veto in this scenario would be 52.63% (10/19) compared to the 50% (11/22) of the whole BNG that actually agree with the veto. However, this edge is even greater due to the fact that 7 people are selected. The calculations get a little bit complicated here so I will just tell you the numbers and leave references at the bottom of the post.
For the scenario in taiko that I just detailed, the probability of 4 or more BNs out of the 7 mediators to agree with the veto would be 57%, a probability 1.14x higher than the actual split of the whole BNG.
For standard we have 67 full BNs (I will simplify to 66 for the 50/50 scenario), so the probability of a majority of the 11 person mediators to uphold the veto would be 52.34%, not as big but still a significant edge as it is 1.05x the actual split of opinion.

In addition to a single mediation being skewed in favor of the vetoing BN, the way things are handled after the mediation concludes is also highly in favor of the side that would veto the map. If the veto is upheld, the mapper has basically no recourse, they are entirely at the mercy of the vetoing BN and the mediators, so if the actual BNG opinion on the issue is very split and the mapper has bad luck with which BNs are randomly assigned to the mediation, they have no recourse whatsoever beyond pleasing the vetoing and mediating BNs or letting the map grave.
However, if the vetoing BN has bad luck with the randomly assigned mediators and the veto is dismissed, they can still get one of their friends to veto the map again based on a similar but slightly different issue (maybe they can even omit an issue in their veto intentionally for this purpose) and they can try their luck again at getting a mediating jury more favorable toward upholding the veto. The rules don’t even state at the moment that the next vetoing BN must veto for a separate issue either, so the main solution to this problem would just be to solidify it in the rules regarding vetoes.

While vetoes may work well for very clear cut issues that a high majority of BNs agree on, the abovementioned issues make it quite unfair toward the mapper if the reason for the veto has a split opinion within the BNG, which calls for some sort of handicap to be given to the mapper to even the scales.
I also believe that the handicap should go beyond evening the scales and actually give a slight advantage to the mapper, since in a creative field such as mapping it makes more sense to be open and allow new or unconventional ideas, which will push the mapping scene further rather than being conservative.

My main suggestion would be requiring a 2/3 majority of BNs (excluding neutral votes) to agree with the veto in order to enforce it. For example in a scenario of a standard veto with 11 mediating BNs, 5 yes votes, 3 no votes and 3 neutral votes would result in the veto being dismissed (5/8 < 2/3), whereas 6 yes votes, 3 no votes and 2 neutral votes would result in the veto being upheld (6/9 >= 2/3).

This solution would allow vetoes to be upheld for clear cut scenarios where a strong majority of the BN agrees with the vetoing BN, ensuring that nothing overly outrageous will reach the ranked section, but would also allow contested issues to rule in favor of the mapper resulting in a ranked section that offers more variety for players.

Even if no favor is given to the mapper, the mathematical edge that I mentioned is an objective flaw in the system and needs fixing somehow to at least make the mediation fair.

Reference for the math part:
theory: https://en.wikipedia.org/wiki/Hypergeometric_distribution
calculator: https://rgbstudios.org/replacement-calc/?N=63&m=32&n=11&k=6

Hard agree

i agree, even without all the math stuff

even if there are only a select few bns who agree with something being valid in mapping, there should be no reason to hold the map back from ranked, assuming all mediators have valid reasoning to back up their vote. if this was somehow a problem, the problem would be with the bn selection process rather than the veto system

if there was a legitimate issue that should prevent a map from being ranked, a supermajority would agree on it

Agreed

strong agree with everything in this post, I had mentioned some of the issues in it in a recent veto mediation too

Agree with everything basically

big agree with everything here ^_^

Sounds fair. Agree

big agree

+1

I think that could be a good idea, maybe this is also an opportunity to consider some ideas proposed in this thread? It's a bit old but I think most issues discussed there are still relevant now and it's regrettable that all that discussion didn't bring much change. The problems with vetoes probably go beyond what was mentioned in this thread and maybe it's out of scope here but it might be worth considering.

agree

quality control via BN evaluations should (ideally) be sufficient to make sure the ranked section doesn't get too bad. usually horrible maps are near-unanimous in mediation anyway so the proposal would ensure that vetoes are only effective against severely low quality maps

i think if a mediation has half of its sample size disagreeing with it then there's probably no harm in allowing the map to be ranked as the general playerbase doesn't possess the amount of mapping knowledge that these bns do to even think of the map in such a way in most cases

sidenote: your post did not have to be that long lol

AJT wrote:

sidenote: your post did not have to be that long lol

Thought I would be up against more resistance so I wanted to be thorough in highlighting the objective flaws of the current system to ensure that at least some change is made...

good👍

You've got a point here, I agree.

Minimizing arbitrary thresholds to gather opinions on something is good, that much I agree on.

Deliberately fixing a threshold that ranges anywhere from 2% to 7% (as expressed by the OP) to a flat 16.6% (2/3 majority) is not an improvement on this - it is the exact opposite. Minimizing the bias towards mappers is good, as is minimizing the bias towards vetoers.

Improving vetoes should be done by involving as many people in them as possible (potentially even summoning the entire game mode cohort of BNs to participate) so that a proper spectrum of values and opinions are reflected in the assessment, and also by removing or minimizing neutral/partial responses to avoid people sitting on the fence and conflating the issue further. I think both of these will go a long way towards helping people feel less like they got dicked by a "bad roll" of mediators and so on.

I understand there's a lot of contention about vetoes, but it seems very unlikely that enforcing a supermajority will do anything but weaken the application of vetoes in favor of the mapper - remember that vetoes were created to address a specific problem, which is likely to return again if the solution to it is hamstrung out of usefulness.

The main problem is that vetoes are meant to address serious issues with a map that make it not deserving of ranked status. If at least a supermajority doesn’t agree with the veto it is more likely an issue of personal preference than a serious problem with the map. Stopping maps like these from getting ranked is hurtful to the ranked section as it ultimately restricts the type of content available as most people seem to agree.

The statistical bias is just an objective flaw with the current system that needs solving and would be mitigated with what I am suggesting.

agree ^ - ^

If more people agree than disagree on the ranked viability of a map, it should be addressed. Even 50% consensus on the topic belies that something significant is present and should be looked at. Increasing this to 66.66% only makes this harder to achieve on all grounds.

Moreover, the statistical bias would not be mitigated by your suggestion - it would be made considerably worse across the board, as expressed by my note mentioning that you'd be increasing a 2-7% bias to a guaranteed 16.66% instead. Unless I am misunderstanding your intention here, this is in no way any sort of measurable improvement and is quite literally the opposite.

I am confused as to how you can assert that a 50% consensus is "personal preference" and 66.66% is a "serious problem". Both are serious problems, and a simple majority is a fair call to assert that with. Half of a given sample thinking something requires change is grounds enough for change to be considered.

A whole bunch BNs agreeing something is so bad that it shouldn't be ranked shouldn't just be dismissed off hand. Additionally, framing "personal preference" as an invalid reason to stop something from ranked misses the point of ranked anyway - it's a conglomeration of widely agreed upon preferences that people evaluate maps on. Every ranking criteria, both explicit and hidden, is based on community-level "personal preferences" as it is. I would rather see a minimum threshold of Yes votes to be upheld as that represents a group consensus that the issue warrants a mandated change.

Addressing Ephemeral's statement: The 2-7% bias is towards the vetoing side, and the 16.66% bias you stated (which should be 10-14% from the calculation of op) is towards the mapper. Op have proved how potentially harmful to the mapping scene a bias towards the vetoing side is to creativity, and have also stated why they think that a bias towards the mapper is not a concern:

Chromaxx wrote:

I also believe that the handicap should go beyond evening the scales and actually give a slight advantage to the mapper, since in a creative field such as mapping it makes more sense to be open and allow new or unconventional ideas, which will push the mapping scene further rather than being conservative.

Also on the 50% rule: Is it currently "throw out all neutral votes and if 50% votes to uphold" or is it "if 50% of all 11/7 are for uphold"?

There is a difference between „looking at“ something and completely stopping a map from getting ranked. At the moment a simple 51% majority will put the mapper completely at the mercy of the vetoing BN with no recourse whatsoever.

The issues where a split of opinions occur are less about objective quality issues and more about ideological differences in what one thinks makes a good map. If 35% of people agree that something should be ranked, then denying the map from ranking is more harmful than letting it get ranked. The people who dislike the map are not forced to play it whereas the people who like it are denied from setting a score on a public leaderboard.

In addition, the current mediation system is based on whether you agree with the vetoing BN‘s point, not whether you strongly feel that the issue warrants blocking a map from being ranked. Having half of the BN feel that strongly about a map that they would go through all the effort of vetoing themselves is one thing, whereas simply agreeing on a mediation is another.

Imagine banning music genres like breakcore or dubstep from gaining exposure because half the population doesn‘t like it.

Beyond noting the potential for a slight statistical edge under current mediator number requirements, the OP has not proved anything other than that as a mapper themselves, they would preferentially prefer that their interests be considered more than the interests of everybody else participating in the veto system because to do so otherwise would be "stifling creativity". Of course they would - vetoes have been an extraordinarily unpopular tool from a mapper's POV since they were introduced, who wants to be told they have to "fix" something that is a product of their own creative expression?

Mappers being forced to take on feedback or criticism has been a contentious point since I joined the game in 2009. It even spawned the popular saying of "killing mapper spirit". Lashback against structures like this is not a new thing, though the systems available today are considerably fairer than they were in the past. The Ranked category has generally never been as free and permissive as the past two years.

No single metric I could pull up (numbers of maps ranked, numbers of vetoes upheld or dismissed) suggests that the stifling of mapper creativity exists as an actual and tangible problem at the present moment.

I contend that unfairly weighing (and therefore diminishing) one of the only ways to reasonably influence a map that a majority may feel is unsuitable for Ranked is considerably worse for the health of the mapping scene than the current 50% consensus percentage is - by several orders of imaginary magnitude. There has to be a fair and balanced way for the community to express their concerns about potentially problematic sets, and erring arguably the only one we have towards the mapper's opinion over that of the community is not fundamentally what osu! is about.

Maybe it's not so much about the numbers but rather how the veto system works as a whole so it might be more sensible to find other solutions that achieve similar results (see my post above). However, the point about the slight mathematical advantage for the vetoing side is an objective flaw that should at the very least be mitigated by requiring 2 BNs to place a veto.

Heavily agree with Ephemeral and Nao Tomori.

Ephemeral wrote:

Improving vetoes should be done by involving as many people in them as possible (potentially even summoning the entire game mode cohort of BNs to participate) so that a proper spectrum of values and opinions is reflected in the assessment.

This is by far the most effective solution in the current state of vetoes, the entirety of the game mode nominators should be selected as mediators excluding probationary BNs and those who may not want to participate in vetoes by other personal reasons (this option would be added in the BN site settings).

Having the entire BNG participate and including the nominating/vetoing BNs in the vote would at least be fair from a statistical standpoint, which is somewhat of an improvement.

The threshold for upholding the veto depends on how you see the ranked section.

If you want the ranked section to only consist of maps that most people agree on, a simple majority is fine.

However if you want a wider arrange of ideas represented and a ranked section that caters to many different tastes while avoiding serious quality issues, a higher threshold for upholding is more appropriate.

I would suggest putting this up for a vote within the team to see which direction people want the ranked section to take and to make changes accordingly.

increasing the number of mediators seems to be pretty universally agreed upon as a good idea so we should be seeing that soon. however, i do think the problem with this proposal is exactly what was just mentioned: the fact that your opinion regarding the ranked section heavily influences whether you'd want this change or not

i think eph's point about "self-interest" is rubbish as this pertains to the ranked section as a whole much more than it does to anything personal. the chance of any ones map getting vetoed in this day and age is very low, so i doubt many pushing for this change would do it in any form of self-interest. rather, they're spreading ideas that are consistent with their ideologies, favoring "freedom" or "creativity" in this case. this proposal is just a logical extension of that ideology, and unfortunately, due to the nature of the argument and how heavily opinionated it is, we're unlikely to see anything productive here. at least not while there's still a sizeable chunk of people that prefer to uphold the popular sense of "quality" for ranked

so instead, i think the best path forward is to improve the current system. emphasize the fact that upheld vetoes halt the ranking process during vetoes, lower the randomness factor of mediation (starting by increasing mediator count as mentioned earlier, possibly require 2 to veto to eliminate the mathematical edge), repurpose or remove the "partial agree" vote to minimize confusion—both in results and for mediators—and possibly more

i do plan to hold a vote on this specific topic regardless, but i wouldn't expect much to come out of it

I just don't see why this is really necessary at all. To me, it seems like you are upset some of your maps/nominations have been successfully vetod, as nobody else seems to be suffering from what you are putting forward as an unfair system (besides the people who I see in this thread who have also been impacted by the recent vetos). A BN's job is to decide what does and does not get ranked from a perspective of objective and subjective issues, from seeing what you've said in other places it really seems like you are upset that the majority of BNs disagree with what you want to see ranked and have shown discontent with the veto system as a whole. I just don't think that this proposal is being made in good faith and is instead a reaction to several successful vetos that would have not had succeeded under this proposal. All of these successful vetos weren't made in bad faith and were brought up by genuine concerns by other BNs; the only reason things are becoming vetod more is because mappers are pushing the boundaries past what most BNs think is acceptable, not because of a social or mathematical advantage.

The social dynamics of vetoing are not abused, it's not an issue that needs addressing, and trying to say that making it more difficult to successfully pass vetos will give newer mappers more protection just isn't true. If the social edge is as strong as you make it out to be, then going from 50% to 2/3 agree wouldn't be difficult for those determined. As somebody who has participated in multiple vetos and mediations, I don't think this point is important at all and personally have never felt a pressure for or against any mapper or vetoer or whatever; mediation is meant to be anonymous and impersonal, if there is a fault in that, making the margins larger is not a proper way of fixing it and is instead putting the vetoers at an unfair disadvantage, which would cause people to let more things they don't think should be ranked (and thus many others very well might not either) pass into ranked.

The mathematics don't add up, their foundation is flawed. You're treating it as if there is an exact 50/50 split of agreeing and disagreeing BNs on every map, which there obviously is not, and this invalidates the entire section. If anything, there being one vetoing BN and two disagreeing BNs off the bat puts the vetoing BN at a disadvantage in terms of numbers (argumentatively, not mediation-wise), which is why it's smart to do a joint veto. When the vetoer and nominating BNs are excluded from the mediation pool, it is a completely fair chance for both sides, even if there were 10 nominators and 1 vetoer, because the mediation pool has by default no opinion.

nifty, please keep the thread on-topic and avoid making assumptions about people and their "true intent" behind proposals, as that is entirely irrelevant to the proposal

your posts sums up to "i disagree" with barely any actual reasoning or substance. please provide proper counter-arguments and evidence to back up any claims

as for your last paragraph specifically, it does not matter whether there is a 50/50 split, what matters is that there is an uneven split between people who are excluded, which makes it naturally biased towards the vetoer -- something we want to avoid

anyway, like i've said before, this thread is unlikely to go anywhere because of how opinionated/philosophical it is, but people are free to bring in more arguments if there is any conclusive evidence to support the original thread. we're working on making changes to vetoes already to make them less confusing and less biased, stuff is just being held up cuz of coe and other internal matters. cheers

i can see this thread is 3 months old now, but it seems worth advocating any individual BNs who agree with this proposal vote against all vetos on principle. if a fraction of BNs did this, it could make up for the difference between 50 and 66.

i'm not good at math (correct me if this is wrong), but it seems it would only require 16.66% of BNs to participate in a boycott in order to up the percentage by that much to 66.66% in a given mode. maybe a bit more to counteract the mathematical bias mentioned in the original post. this comes out to 10.5 (so 10 or 11) in standard, about 6 in taiko, about 7 in mania, and 3 in ctb, before accounting for mathematical bias.

a number of BNs agreed with this proposal in this thread alone, with of course more who didn't reply. i ask such people to consider doing this, if they haven't already, while waiting for more systemic changes to the veto system.

archiving since opinions on this are pretty split and the thread didn't really go anywhere. also, the newly added veto challenge procedure helps with mitigating the destructive nature of vetoes: wiki/en/People/Beatmap_Nominators/Beatmap_Veto#challenging-a-beatmap-veto