=== CONTEXT ===

When a beatmap gets disqualified after a nomination, a nomination reset is issued to the BNs involved. Internally, each reset is marked by the NAT as either "minor" or "major", but only NAT members can see it. BNs are left guessing unless they ask directly (which can be awkward) or wait for their next evaluation.

This lack of transparency causes confusion, anxiety, and actually limits learning opportunities.

--------------------------------------------------

=== PROBLEMS ===

--> Problem #1) Nomination reset severity is hidden for no clear reason.

While resets still factor into evaluations, their severity is invisible. BNs have to guess if an issue was negligible or serious even though NAT already makes that judgment internally. Sometimes it's obvious, but context-heavy cases leave BNs uncertain. That ambiguity serves no real purpose and only does harm.

SEV was stressful, but at least it was transparent.
The SEV system caused anxiety, but you always knew how impactful a reset was and now BNs don’t know what’s being weighed against them until months later.

--------------------------------------------------

--> Problem #2) Hidden severities hurt growth and consistency.

BNs can’t learn from their mistakes if they don’t know early on which ones are seen as significant. A BN might dismiss multiple resets as minor, only to later find out some were major and too late to adjust.

Making severity public also improves NAT consistency. If severities are visible, they’re more likely to be applied in a clear and consistent way.

--------------------------------------------------

=== SOLUTION ===

--> Make nomination reset severity public (or at least visible to the BN, perhaps with toggle to set it to private would cool too.)

When a reset occurs, "minor" or "major" would be marked somewhere on the reset list on the BN site.
This replicates SEV’s clarity without its drawbacks, and sets a transparent standard for evaluations.

--------------------------------------------------

=== POSSIBLE COUNTER-ARGUMENTS ===

--> Counter-Argument #1)
“Making it public will cause stress or drama.”

I don't think it generates more stress than it already is when anyone can see the full extent of how many disqualifications you got on your BN site card.

People can mostly tell how serious a specific disqualification is, but adding the severity marker just makes it clear for everyone to serve as a future example.

--------------------------------------------------

--> Counter-Argument #2)
“BNs may challenge severity labels like they did with SEV.”

This is actually a benefit. If NAT members apply severity differently, transparency highlights it, encouraging internal alignment and clearer standards.

It also allows BNs to learn what counts as major or minor, fostering consistency, growth and also helping with aspiring NATs.

Questions or disagreements can become opportunities to clarify and document standards.

--------------------------------------------------

=== TL;DR ===

--> Make nomination reset severity (minor or major) public for better transparency.
--> Helps BNs self-improve, reduces anxiety, promotes NAT consistency.
--> No real downsides. Easy to implement. Should’ve happened already.

agree

another benefit is for Other BN to see what u are bad at, so they can avoid accepting map u take, or be extra careful in that area

just to be blunt, theres really no clear winning option here, as both directions cause some level of anxiety or whatever. anyway:

your counter-arguments are the exact reasons that it was hidden pretty much, and i believe that its to everyones benefit that it stays hidden like it is right now

people in the past used to tweak out and try to defend themselves any time they got a red sev, or a lot did anyway, which led to a lot of extra stress for the nat, as well as for the bn overly worrying over something they could easily ignore. the nat doesnt follow the color of the severity like its the bible, where a lot of bns tended to. all the gray/red indicates is that the reset should be looked into more during the evaluation process. we shouldnt have to explain ourselves over every gray/red sev, and you shouldnt have to fear every single gray/red sev either. sometimes people can have multiple of these on one card, and based on a myriad of other factors they may very well have nothing majorly marked on their eval

theres nothing stopping you from learning by investigating your own dqs with the current way things are presented, if thats the angle you're going for.

id much prefer it to stay hidden for everyones sake, but if enough bns agree it could maybe be put to a vote

@radar

"we shouldnt have to explain ourselves over every gray/red sev, and you shouldnt have to fear every single gray/red sev either. sometimes people can have multiple of these on one card, and based on a myriad of other factors they may very well have nothing majorly marked on their eval"

This is fair, but at the moment I don't even know what is a grey/red reset anymore, there is no specification for us and we can only assume what the severity is at the moment. Since the nomination stress will always be there no matter what, I'd just make it public so we don't have to make incorrect assumptions.

"theres nothing stopping you from learning by investigating your own dqs with the current way things are presented, if thats the angle you're going for."

That's exactly one of the points of this proposal. It is really hard to study and investigate our own disqualifications without asking NAT every time weather something was deemed minor, notable or severe.

In the previous SEV system where NAT will consider contexts like "it's something hard to spot", "hardly affects anything" or even "it's not that bad, it's just a technicality" we could study those much more clearly because it was there for us to see, but right now, we can't even see if any context or leniency was applied to it.

We can only study if something is assigned to the resets and we can it.

I like the idea of a vote though.

why can you only study if something is assigned to it? you should be learning from every disqualification regardless of the severity, as mistakes should be mitigated as much as possible. if you truly want to improve using your mistakes, aim to do so organically.

as i stated, we do not take severities as gospel, you shouldn't either. adapt and aim to avoid every disqualification you can and don't overthink a potential number or the blowback that you may receive for it

id like to point out that after a discussion in internal, every nat that came across this agreed with my post and would prefer to keep it the way it is.

would say, just dont even think about severities? i mean, theyre private to you, they really arent there. they are really just like bookmarks for us, ways to keep tabs on things without having to go through ourselves and dig through every dq.

on top of this, if you spend enough time in the bng you should probably be able to glean how bad something is regardless of if you have a number or bit of colored text telling you whether it is or isnt

like,
Bad: Significant impact on gameplay, rule break, egregiously incorrect timing (>+10 offset or something, new red lines), etc.
Not bad: Tag, dq to add a hitsound, something else stupid and not impactful

id say if you can rally up a very large number of bns in favor of a vote or a change here, then we can push a vote. otherwise id prefer to just leave it at this since i believe the entire nat would be in agreement with my stance & this is the first instance of discontent with the current system being pushed forward

I *can* study my disqualifications and I already try to do it, but it is much easier to analyze them when you have something to base yourself off, no?

This is not just about being worried about my own nomination evaluations, it's also about learning how to evaluate them as an aspiring NAT too. It is *because* NATs don't treat SEVs as gospel that I want to know and learn when and why someone would do fine on an evaluation period.

"on top of this, if you spend enough time in the bng you should probably be able to glean how bad something is regardless of if you have a number or bit of colored text telling you whether it is or isnt"

I understand there are certain resets that are objectively bad and others that are minor, like you mentioned.
However, nomination resets can also be placed in between those extremes and THOSE are the ones I would like learn more in detail, but that's not really that easy, becase I don't really know the thought process, logic and leniency applied by the NAT.

"id say if you can rally up a very large number of bns in favor of a vote or a change here, then we can push a vote. otherwise id prefer to just leave it at this since i believe the entire nat would be in agreement with my stance & this is the first instance of discontent with the current system being pushed forward"

I'm not saying the SEV rating would be such a game changer, but I don't get how keeping it hidden sounds so beneficial for anyone besides the NAT.

I'm pretty sure some other BNs are not content with it, but no one bothered to be vocal about something this "small".
People used to be really vocal when SEV rating was visible, but suddenly not anymore since it was hidden.

"eyes that don't see, heart that doesn't feel."

Anyways, since your sentiment is something so unanimous within the NAT, I doubt anything I suggest will change it, so I won't press further. If anyone else would like to express their opinion here, feel free.

Thanks for the discussion though.

Agree with radar pretty much on everything.

1. DQ = you missed something = you should learn from it regardless of severity.

2. If it's really very minor and hardly affects anything, you would know it yourself as a BN.

3. When in doubt, you can always reach out to a NAT. Idk if other modes NAT are as open as mania NAT but they shouldn't be hard to approach, it's literally one of their duties to help BNs improve. Having a conversation about your DQ with NAT will help you improve more than just staring at its SEV rating anyway.


Based on the above, I don't think we have much use for a public SEV rating or its replacement. And it's apparently a pain in the ass for NAT. So the benefit isn't worth taking the downside.

People used to be really vocal when SEV rating was visible, but suddenly not anymore since it was hidden.

This is exactly the point of why we have things as they are currently.

Many hours have been spent because people got way too hung up on the SEV of a DQ when it's really not the thing they need to care about.

Forest for the trees.

Agree, public would provide severity of each nomination more transparency so they can easily know which to focus on next time. NATs' feedback could work the same for sure but it’s likely unrealistic for them to write detailed feedback for every reset or there will be a BN who is willing to ask the NAT for every of their reset, tho they're welcome to do but it's just feel time-consuming for both parties

I'm not sure how public the SEV was in the past since I've only been here for a month. But if "public" means that every BN can see SEV of all other members then I agree it could lead to silent judgment or gossip among BNs, creating fears of having a bad record. In the end, I would suggest making SEV visible only to NAT and the individual BN invloved, not the whole world. Can also add a button for BNs to either private or public, same idea as BN apps and evals

I prefer it as it is now, SEV was very stressful, much more than current.

It was also more demotivating.
So yeah, prefer to keep as it is now with this system.

i agree with the post but we dont gotta type allat

making ur dq sevs public or at least visible to urself makes it a lot easier to know where u stand = less anxiety when compared to now where u can only rly guess what sev ur dqs were and will have no indication of such until ur eval

wiki/en/People/Nomination_Assessment_Team/SEV_rating

you can use this to estimate severity of a dq

I'm reading things like "know where you stand" and "I want to know how the NAT think" and just end up wondering why there is a need to pre-empt evaluations in performing these functions? You should be getting all the answers in your evaluation feedback.

Especially when taking "People can mostly tell how serious a specific disqualification is" in mind. I'm sure your own experience as a BN allows you to do this too.

Is it that you're specifically worried about grey zones?
Do you want immediate feedback since bad evals could lead to reduced chances at elite BN?
Is evaluation feedback not sufficient to help guide you?
Do you worry about getting too many bad evaluations due to unawareness?
Or anything else?

Im just poking around for a root cause here, since from experience the NAT do know that being able to see SEV ratings is less comfortable than them being hidden. For both the NAT and BNs themselves, fwiw.

i dont mean to be too autistic here, but being uncertain about how exactly severe your dqs were causes anticipatory anxiety, which tends to be worse than being certain of exactly how severe your dqs were.

the reason for wanting immediate knowledge of ur sevs, rather than waiting what could be months for an eval, is because it lets u be more accurate in assessing ur own performance and how to improve in the immediate term.

if anything at all, why not just let individual bns choose whether they can see their own sevs similar to how you can choose to receive subjective feedback in evals? this way, if ur someone who prefers to avoid anticipatory anxiety, you can tick it on. if you prefer not to know, you can tick it off. its also only your own sevs you can see so there isnt rly a social shaming aspect to it that i can see (at least not any more than there is now)

something like this isnt rly hard to implement, according to the linked wiki u guys have already gone back to using sev internally (which btw was that ever announced? first time seeing that we're back to using old sev) so whats stopping u from adding a switch that lets you view ur dq sevs like u used to be able to back in the day?

Well, here's the thing with anticipatory anxiety: what is it that you're anticipating will go badly and want to avoid?
I feel like we'd be better of starting to look there rather than just attacking a symptom and not solving the problem.

As for allowing bns to choose, I feel like that's the same as turning them on for everyone. Those who obsessed over these numbers will likely find themselves drawn to do that again.

yaspo wrote:

Well, here's the thing with anticipatory anxiety: what is it that you're anticipating will go badly and want to avoid?

if i get a bunch of dqs and only have a vague idea of how bad they were exactly, im going to be under more pressure, be more anxious, be more uncertain about my following evals results. this can all culminate in mental stress and performing worse and getting even more dqs than if u were 100% certain of how ur performance is and having a better idea of what to expect on ur next eval.

im not saying this is a universal experience as clearly a good portion of people feel more anxious with the opposite situation, but its definitely more than 1 person who feels worse with the current arrangement, and if that can be solved with a switch that can be turned on and off then why not allow it.

Basensorex wrote:

the reason for wanting immediate knowledge of ur sevs, rather than waiting what could be months for an eval, is because it lets u be more accurate in assessing ur own performance and how to improve in the immediate term.

you don't need to be able to see sev rating for this though. If you see one of your nominations getting disqualified for actual issues, you should always make a mental note for yourself and think about how you can avoid this mistake next time, regardless of its severity rating. If you really want to know what the nat thinks about this particular dq - you can use the sev handbook for estimation

NeKroMan4ik wrote:

If you really want to know what the nat thinks about this particular dq - you can use the sev handbook for estimation

im making the argument that u should just be able to see this instead of having to estimate and potentially being wrong as a lot of cases dont fall into strict generalizations and some peoples lines differ from others.

u should be making a counter argument with reasons as to why its not good to have the option to see ur dq sevs and not just saying "but u can get a potentially* accurate* estimation* of ur sev"

we (the collective nat, mostly me) have already made a couple counter arguments as to why its not good to see your sevs. i (as a sole individual) have also stated in this thread that there are downsides to both options. however, the current method of hiding sevs is the most beneficial one for all parties involved. there will be no further communications made by we (the collective nat, but mostly me) on this thread unless there is widespread desire for change here. which, based on my understanding thus far, is unlikely to happen.

as nekro posted in the thread, please just use wiki/en/People/Nomination_Assessment_Team/SEV_rating to determine how bad your dqs are. i believe that anyone who can acquire the title of Beatmap Nominator can do this

please refer to my other posts in this thread if you need a refresher on my opinions

i know in the past i have heard horror stories of the SEV ratings, and i feel like showing nomination "severity" would just bring back those same problems with anxiety and debating over the assigned rating.

everyone ever has linked the SEV rating page, and i'd agree to use that as a reference if you are clueless. if you especially need extra clarification i think you can always ask around to other BNs (and probably NAT) for more info

at least for me, i feel like it's best to treat every dq as a problem to learn from. all i know is that if a mapper requests changes, it's not major. if it's unrankable, it's something i need to remember. if it's for quality concerns, that's something to remember for your future modding.

i don't think it matters to know of a dq is major or minor. a dq for a problem is a dq for a problem, and that's something to learn from regardless of if NAT deem it "minor" or "major". i don't think bringing back the previously seen as problematic SEV rating is a benefit

at least in mania (can't speak for everyone, or the mania NAT themselves. just from my experience), i know NAT trust in the BNs to learn and improve when they can or make mistakes, and i don't think whether any problem major or minor is something to get hung up over. learn & improve anyways because it makes you better as a nominator, mapper, or basically anything

----

tl;dr: i don't think this is a good suggestion because of SEV's issues in the past. try to learn from your mistakes whenever you can, and remember for the future. if there's somehow nothing to learn, it's probably not a severe dq

You guys mentioned that we can estimate SEVs by ourselves and there's no need for transparecy.

In my estimations, in my last eval I nominated 16 maps. 4 got DQ'd. 2 were 0/0 SEV, one was 1/0 SEV and the other was 1/1 SEV. And yet I still got a NATs saying;

overall fine but should really be careful about these dqs

maps themselves seem fine more or less,

How does that even work? 16 maps where only 2 DQs mattered mean 88% where the 12% was not even something MAJOR. What is the target here? Does this mean that for every 10 nominations only 1 can get a DQ before NAT start saying "yo, you should be careful..."

I'm sorry but I don't get it.

I don't think this thread is for nitpicking your evaluations or how dqs are weighted; seems a different topic than the numbers being visible at all
there may be more surrounding context or it's just a simple misunderstanding, try asking a NAT about it

yaspo wrote:

I don't think this thread is for nitpicking your evaluations or how dqs are weighted; seems a different topic than the numbers being visible at all
there may be more surrounding context or it's just a simple misunderstanding, try asking a NAT about it

It's meant as an example (an inference argument if you will) of how the lack of transparency makes evaluations senseless from a perspective of someone who has no idea what's going on with their evaluators and the criterias being applied.

From a SEV standpoint, I passed all right. The actual eval is that, it was all right.

But from how the text and how it was worded, it felt like it was a much bigger deal than it actually was and there's no counter argument if they had agreed that I should get a warning.

I'm questioning that discrepancy. Numbers are objective, but words can imply narratives when it is convenient to do so. I'm not denying nuanced on evaluations, but having both a more objective guideline and rules of what you should strive for while keeping the human element of evaluation, for me, is the correct path.

Because right now, there's no example of what is a good activity or a good balance and what isn't.

As said earlier in the thread, SEV isn't taken as gospel by the NAT. They are not objective for them so neither should you try and treat them like that yourself.

For me personally: it's hard to think of a clear and concise way to convey what "good activity" might be. It can vary so much based on things like number of nominations, types of dqs, past history.
Which is why they are, as the linked SEV wiki page explains, treated on a case by case basis -- realistically there is no true middle ground without opening up to infinite technicalities and ensuing unnecessary arguments.

That's just me though.

A couple thoughts on Neto's example.

On one hand, NAT could consider being more elaborate in their feedback, at least when it contains something that looks like a warning. Having no public SEV makes evals more significant as the only source of guaranteed feedback for BNs.

But on the other hand, this also looks like overthinking tbh. From what I've seen in the public evaluations and just from common sense, NAT do not issue an official warning with evals like "basically fine but be careful with DQ". Warning evals are 50 page essays listing all the crimes you commited, lol.

There's no way to stop BNs from overthinking though, and being more elaborate would help a lot.

Sorry if I'm sounding dismissive of the entire thing, but wasn't the history of SEV:

- introduced as a private tracking to more easily categorize how severe a mistake was to make evaluating easier
- made public in an attempt to give people a better idea what was messed up in the eyes of NAT
=> this completely backfired and lead to people obsessing and arguing about their severity stuff
- the entire thing was then simplified to just be notable / not notable or something like that, internally
- a bit ago we went back to using SEV because the notable / not notable / etc. approach didn't capture enough nuance
=> here's a proposal to make the numbers public again (<- we are here)

the solution to make it public or at least visible to the bn in question with the counter argument of "i dont think it will stress people out" doesn't really compute to me, we'd just be mostly back where we were the last time this thing was tried and didn't really work out with no notable differences in approach? am I missing something?