Guest
home
beatmaps
rankings
community
store
help
forum
  1. Forums
  2. Beatmaps
  3. Mapping Discussion

[Proposal/Discussion] Re-instate the BN Evaluator system

posted
Total Posts
7
Hivie
osu!taiko Paragon
987 posts
Joined March 2019
Topic Starter
Hivie
Currently, eval delays are on the rise with more and more people complaining about them, with application/current BN evals possibly being 2 weeks overdue sometimes, especially considering the huge influx of standard applications when apps reopened.
Recently a new thing has been implemented, where well-performing BNs get evaluated every 6 months instead of 3, while it is a step in the right direction of reducing NAT workload, I still think there's room for improvement in order to split NAT workload in order to make evals happen on time.
That's why I'm proposing to re-use the BN Evaluator system that was trialed about a year ago, but with some changes to fix its previous flaws:

  1. Make it invite-only instead of it being RNG like last time.
    self-explanatory, that way NATs can choose good-performing BNs who can do the job well.
  2. Reduce the number of BN evaluators. This is so things don't devolve into a mess, and to reduce the chances of massively different standards from clashing in most evals.
    Could have 5-7 BN evaluators for standard, and 2-3 BN evaluators for other gamemodes.
  3. Enforce stricter activity requirements on them. That way, it's easier to cycle out inactive/barely active evaluators.
  4. If you really need to use a cycle system, you can have 3-month cycles and allow evaluators to stay for 4 consecutive cycles at max. That way, a good evaluator would get to eval for a year then have a 3-month break, and repeat.
    Could even include some kind of reward as an incentive, like a different profile title for current BN evaluators, and a badge for performing well for 4 consecutive cycles.
This system can also be used to spot out good candidates for the actual NAT role, and can even serve as a "Probationary NAT" position where a candidate can tip their toes into NAT work before jumping into the actual role.

Curious on what you guys think about this, all opinions are welcome!
Last edited by Hivie , edited 1 time in total.
enneya
Nomination Assessment Team
387 posts
Joined October 2017
enneya
yea

i'd personally only reuse this in function of adding new nat members in some sort of cycle, or whenever an nat leaves for whatever reason there would be a competent bn ready to fill in the potential gap in workforce created
though i feel like outside of enforcing stricter activity requirements you'd also need to enforce some stricter "quality requirements", in a sense of how well bns they let in perform generally, or how well they write evaluations during the trial (as in not missing glaring concerns, using proper reasoning without fallacy and their willingness to enforce adequate punishment on bns who don't perform well)

one concern with an invite-based system is there would be a good chance some bn who wants to contribute might get left behind somehow, but i guess you can find a way to mitigate that in the end, perhaps once you cycle through all the bns decently qualified for an evaluating position :3
Last edited by enneya , edited 1 time in total.
Nikakis
770 posts
Joined May 2014
Nikakis
or maybe just pay nat group since it's literally a work? there's no fun in ''evaluating'' 20 apps and wasting your time by managing 80+ bns, it's like you work for a corporation and you have to manage 100 employees for FREE. it's not like bns who just mod and nomimate maps for fun and that's it. I'm pretty sure if peppy and eph would give nats a payroll they wouldn't be a dead group.
Axer
434 posts
Joined October 2015
Axer
+1 for this idea, it could obviously do with some further collaboration, though I think that'll come along with this thread's continuity.

Nikakis wrote:

or maybe just pay nat group since it's literally a work?
As per usual, this has been brought up countless times before, and I guess the answer is always "easier said than done", as putting NATs into a contractual state is most likely not that simple of a deal.
radar
osu! Alumni
283 posts
Joined September 2015
radar
the system isnt even like, gone, mania used it very recently to scout new nat members (who were hand picked)

i think an issue is that we'd much rather just add someone to nat rather than have bns cycle through being bn evaluators, normally just the typical bn evaluations we send out are enough of a scouting process to know who we would benefit from adding. i have thought of a similar system before, but just adding people directly to nat is much easier for everyone involved in 99% of situations

Nikakis wrote:

or maybe just pay nat group since it's literally a work?
i would love to be paid but i also feel as though this would be counterproductive for everyone and change peoples perception of nat so i wouldnt love to be paid
Last edited by radar , edited 1 time in total.
Nikakis
770 posts
Joined May 2014
Nikakis

Axer wrote:

As per usual, this has been brought up countless times before, and I guess the answer is always "easier said than done", as putting NATs into a contractual state is most likely not that simple of a deal.
well maybe it's time to try it out? from what I know someone also needs to be 18+ of age in order to join nat. sticking to assumptions doesn't help either. my solution is more realistic than those overcomplicated stuff that you all bring. we could maybe ask a consultant or Monstrata to help out since they will probably know better.

radar wrote:

i would love to be paid but i also feel as though this would be counterproductive for everyone and change peoples perception of nat so i wouldnt love to be paid
I don't think this would affect the nat's concept because on it's own is a workload position since you only do evaluations, it's different from bn position which is more casual and ''fun''.
Last edited by Nikakis , edited 5 times in total.
Capu
osu! Alumni
351 posts
Joined March 2013
Capu
Standard closed applications for a while, which was probably a good thing to do and could probably be re-used imo. Taiko is in a pretty good state, with only a few delays recently because of unlucky timing circumstances between the members. We're also actively using BN evals from anyone who's interested which definitely helps sometimes, so it's not like the system is gone. Seeing how this already makes our situation easier shows that there doesn't need to be any changes, it should just be used more frequently for the gamemodes that seem to struggle.

There has also been the idea of cross-NAT feedbacks, where NATs from other gamemodes can help with writing feedbacks. Not evaluating modding for other modes. Just putting the individual evals into words, to reduce workload for mainly the standard mode. This is also something that we should probably put more seriousness into, as it can drastically reduce stress.

Making it invite-only is a terrible idea in my opinion, as this prevents people from actually improving their own evaluating skills and simultaneously prevents them from spotting things they could improve within their own modding. For taiko, there is no need to change anything at the moment, at least in my opinion.
Please sign in to reply.

New reply

0
1 / 7